2024-05-17T22:15:33.162 INFO:root:teuthology version: 0.0.1.dev276+g2127e87 2024-05-17T22:15:33.162 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/postgres}} --name pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi --archive /home/teuthworker/archive/pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7711773 --owner scheduled_pdonnell@teuthology -- /home/teuthworker/archive/pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7711773/orig.config.yaml 2024-05-17T22:15:33.218 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-17T22:15:33.307 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7711773 branch: wip-pdonnell-testing-20240517.012458-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/postgres}} email: pdonnell@redhat.com first_in_suite: false job_id: '7711773' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240517.012458-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 931809a9e5d295bd61984693cac44fe131ded4bc subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt install: ceph: flavor: default sha1: 931809a9e5d295bd61984693cac44fe131ded4bc kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs mntopts: - ms_mode=legacy - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240517.012458-debug sha1: 931809a9e5d295bd61984693cac44fe131ded4bc owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1967 sha1: 931809a9e5d295bd61984693cac44fe131ded4bc sleep_before_teardown: 0 subset: 59/512 suite: fs suite_branch: wip-pdonnell-testing-20240517.012458-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_931809a9e5d295bd61984693cac44fe131ded4bc/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 931809a9e5d295bd61984693cac44fe131ded4bc syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi012.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA+YERfcwhoXIzuXGqVKr/TxjgbdVXplWdIhcOhk4hEJ0tLTLNrwmht+1SMgMxP87kP80aD83MHslajFI7knvy4= smithi096.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFKme7VTXQybGKHf0WUQLWdrHEa7R2YTsXJ3gNR+T97i+1fRqP7smqWTtDldSuXCQTEmzLmOmwSDKY7ip8rYVUg= smithi151.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCY4r/zn1whKA5Muj+y5yttf/Duu8bDHmR8dWTEeofx9xBiXDI3QGZafvu+5GiMOBws1EBlkQo6P2dCS4YzVBUs= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: install-fix teuthology_sha1: 2127e87e3419439de550d2abbd7ed2aee2cdfb0e timestamp: 2024-05-17_18:30:44 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-05-17T22:15:33.308 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_931809a9e5d295bd61984693cac44fe131ded4bc/qa; will attempt to use it 2024-05-17T22:15:33.308 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_931809a9e5d295bd61984693cac44fe131ded4bc/qa/tasks 2024-05-17T22:15:33.308 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-17T22:15:33.310 INFO:teuthology.task.internal:Checking packages... 2024-05-17T22:15:33.329 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '931809a9e5d295bd61984693cac44fe131ded4bc' 2024-05-17T22:15:33.329 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-17T22:15:33.329 INFO:teuthology.packaging:ref: None 2024-05-17T22:15:33.329 INFO:teuthology.packaging:tag: None 2024-05-17T22:15:33.329 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240517.012458-debug 2024-05-17T22:15:33.329 INFO:teuthology.packaging:sha1: 931809a9e5d295bd61984693cac44fe131ded4bc 2024-05-17T22:15:33.329 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240517.012458-debug 2024-05-17T22:15:33.545 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3794.g931809a9 2024-05-17T22:15:33.547 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-17T22:15:33.553 INFO:teuthology.task.internal:no buildpackages task found 2024-05-17T22:15:33.553 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-17T22:15:33.675 INFO:teuthology.task.internal:Saving configuration 2024-05-17T22:15:33.700 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-17T22:15:33.705 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-17T22:15:33.732 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi012.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7711773', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-17 22:10:32.926815', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA+YERfcwhoXIzuXGqVKr/TxjgbdVXplWdIhcOhk4hEJ0tLTLNrwmht+1SMgMxP87kP80aD83MHslajFI7knvy4='} 2024-05-17T22:15:33.755 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi096.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7711773', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-17 22:10:32.925763', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFKme7VTXQybGKHf0WUQLWdrHEa7R2YTsXJ3gNR+T97i+1fRqP7smqWTtDldSuXCQTEmzLmOmwSDKY7ip8rYVUg='} 2024-05-17T22:15:33.775 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi151.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7711773', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-17 22:10:32.924509', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCY4r/zn1whKA5Muj+y5yttf/Duu8bDHmR8dWTEeofx9xBiXDI3QGZafvu+5GiMOBws1EBlkQo6P2dCS4YzVBUs='} 2024-05-17T22:15:33.776 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-17T22:15:33.781 INFO:teuthology.task.internal:roles: ubuntu@smithi012.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-17T22:15:33.781 INFO:teuthology.task.internal:roles: ubuntu@smithi096.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-17T22:15:33.781 INFO:teuthology.task.internal:roles: ubuntu@smithi151.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-17T22:15:33.781 INFO:teuthology.run_tasks:Running task console_log... 2024-05-17T22:15:33.914 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe051b77ac0>, signals=[15]) 2024-05-17T22:15:33.914 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-17T22:15:33.921 INFO:teuthology.task.internal:Opening connections... 2024-05-17T22:15:33.921 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi012.front.sepia.ceph.com 2024-05-17T22:15:33.922 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T22:15:33.997 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi096.front.sepia.ceph.com 2024-05-17T22:15:33.998 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T22:15:34.072 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi151.front.sepia.ceph.com 2024-05-17T22:15:34.073 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T22:15:34.148 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-17T22:15:34.155 DEBUG:teuthology.orchestra.run.smithi012:> uname -m 2024-05-17T22:15:34.173 INFO:teuthology.orchestra.run.smithi012.stdout:x86_64 2024-05-17T22:15:34.174 DEBUG:teuthology.orchestra.run.smithi012:> cat /etc/os-release 2024-05-17T22:15:34.231 INFO:teuthology.orchestra.run.smithi012.stdout:NAME="CentOS Stream" 2024-05-17T22:15:34.231 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION="9" 2024-05-17T22:15:34.231 INFO:teuthology.orchestra.run.smithi012.stdout:ID="centos" 2024-05-17T22:15:34.231 INFO:teuthology.orchestra.run.smithi012.stdout:ID_LIKE="rhel fedora" 2024-05-17T22:15:34.231 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION_ID="9" 2024-05-17T22:15:34.231 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM_ID="platform:el9" 2024-05-17T22:15:34.231 INFO:teuthology.orchestra.run.smithi012.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-17T22:15:34.232 INFO:teuthology.orchestra.run.smithi012.stdout:ANSI_COLOR="0;31" 2024-05-17T22:15:34.232 INFO:teuthology.orchestra.run.smithi012.stdout:LOGO="fedora-logo-icon" 2024-05-17T22:15:34.232 INFO:teuthology.orchestra.run.smithi012.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-17T22:15:34.232 INFO:teuthology.orchestra.run.smithi012.stdout:HOME_URL="https://centos.org/" 2024-05-17T22:15:34.232 INFO:teuthology.orchestra.run.smithi012.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-17T22:15:34.232 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-17T22:15:34.232 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-17T22:15:34.232 INFO:teuthology.lock.ops:Updating smithi012.front.sepia.ceph.com on lock server 2024-05-17T22:15:34.251 DEBUG:teuthology.orchestra.run.smithi096:> uname -m 2024-05-17T22:15:34.268 INFO:teuthology.orchestra.run.smithi096.stdout:x86_64 2024-05-17T22:15:34.269 DEBUG:teuthology.orchestra.run.smithi096:> cat /etc/os-release 2024-05-17T22:15:34.326 INFO:teuthology.orchestra.run.smithi096.stdout:NAME="CentOS Stream" 2024-05-17T22:15:34.326 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION="9" 2024-05-17T22:15:34.326 INFO:teuthology.orchestra.run.smithi096.stdout:ID="centos" 2024-05-17T22:15:34.326 INFO:teuthology.orchestra.run.smithi096.stdout:ID_LIKE="rhel fedora" 2024-05-17T22:15:34.326 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_ID="9" 2024-05-17T22:15:34.326 INFO:teuthology.orchestra.run.smithi096.stdout:PLATFORM_ID="platform:el9" 2024-05-17T22:15:34.326 INFO:teuthology.orchestra.run.smithi096.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-17T22:15:34.326 INFO:teuthology.orchestra.run.smithi096.stdout:ANSI_COLOR="0;31" 2024-05-17T22:15:34.326 INFO:teuthology.orchestra.run.smithi096.stdout:LOGO="fedora-logo-icon" 2024-05-17T22:15:34.326 INFO:teuthology.orchestra.run.smithi096.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-17T22:15:34.326 INFO:teuthology.orchestra.run.smithi096.stdout:HOME_URL="https://centos.org/" 2024-05-17T22:15:34.327 INFO:teuthology.orchestra.run.smithi096.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-17T22:15:34.327 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-17T22:15:34.327 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-17T22:15:34.327 INFO:teuthology.lock.ops:Updating smithi096.front.sepia.ceph.com on lock server 2024-05-17T22:15:34.351 DEBUG:teuthology.orchestra.run.smithi151:> uname -m 2024-05-17T22:15:34.369 INFO:teuthology.orchestra.run.smithi151.stdout:x86_64 2024-05-17T22:15:34.369 DEBUG:teuthology.orchestra.run.smithi151:> cat /etc/os-release 2024-05-17T22:15:34.427 INFO:teuthology.orchestra.run.smithi151.stdout:NAME="CentOS Stream" 2024-05-17T22:15:34.427 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION="9" 2024-05-17T22:15:34.427 INFO:teuthology.orchestra.run.smithi151.stdout:ID="centos" 2024-05-17T22:15:34.427 INFO:teuthology.orchestra.run.smithi151.stdout:ID_LIKE="rhel fedora" 2024-05-17T22:15:34.427 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION_ID="9" 2024-05-17T22:15:34.427 INFO:teuthology.orchestra.run.smithi151.stdout:PLATFORM_ID="platform:el9" 2024-05-17T22:15:34.427 INFO:teuthology.orchestra.run.smithi151.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-17T22:15:34.427 INFO:teuthology.orchestra.run.smithi151.stdout:ANSI_COLOR="0;31" 2024-05-17T22:15:34.427 INFO:teuthology.orchestra.run.smithi151.stdout:LOGO="fedora-logo-icon" 2024-05-17T22:15:34.428 INFO:teuthology.orchestra.run.smithi151.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-17T22:15:34.428 INFO:teuthology.orchestra.run.smithi151.stdout:HOME_URL="https://centos.org/" 2024-05-17T22:15:34.428 INFO:teuthology.orchestra.run.smithi151.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-17T22:15:34.428 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-17T22:15:34.428 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-17T22:15:34.428 INFO:teuthology.lock.ops:Updating smithi151.front.sepia.ceph.com on lock server 2024-05-17T22:15:34.442 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-17T22:15:34.452 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-17T22:15:34.515 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-17T22:15:34.516 DEBUG:teuthology.orchestra.run.smithi012:> test '!' -e /home/ubuntu/cephtest 2024-05-17T22:15:34.518 DEBUG:teuthology.orchestra.run.smithi096:> test '!' -e /home/ubuntu/cephtest 2024-05-17T22:15:34.521 DEBUG:teuthology.orchestra.run.smithi151:> test '!' -e /home/ubuntu/cephtest 2024-05-17T22:15:34.537 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-17T22:15:34.558 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-17T22:15:34.559 DEBUG:teuthology.orchestra.run.smithi012:> test -z $(ls -A /var/lib/ceph) 2024-05-17T22:15:34.576 DEBUG:teuthology.orchestra.run.smithi096:> test -z $(ls -A /var/lib/ceph) 2024-05-17T22:15:34.578 DEBUG:teuthology.orchestra.run.smithi151:> test -z $(ls -A /var/lib/ceph) 2024-05-17T22:15:34.690 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-17T22:15:34.756 INFO:teuthology.run_tasks:Running task kernel... 2024-05-17T22:15:34.771 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-05-17T22:15:34.771 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-05-17T22:15:34.771 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-05-17T22:15:34.771 DEBUG:teuthology.orchestra.run.smithi012:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-17T22:15:34.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T22:15:34.789 DEBUG:teuthology.orchestra.run.smithi012:> uname -r 2024-05-17T22:15:34.846 INFO:teuthology.orchestra.run.smithi012.stdout:5.14.0-437.el9.x86_64 2024-05-17T22:15:34.847 INFO:teuthology.task.kernel:Running kernel on smithi012: 5.14.0-437.el9.x86_64 2024-05-17T22:15:34.847 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum install -y kernel 2024-05-17T22:15:37.512 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - BaseOS 7.5 MB/s | 8.1 MB 00:01 2024-05-17T22:15:39.347 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - AppStream 28 MB/s | 19 MB 00:00 2024-05-17T22:15:43.606 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-05-17T22:15:45.139 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - Extras packages 69 kB/s | 16 kB 00:00 2024-05-17T22:15:47.799 INFO:teuthology.orchestra.run.smithi012.stdout:Extra Packages for Enterprise Linux 8.4 MB/s | 21 MB 00:02 2024-05-17T22:15:53.181 INFO:teuthology.orchestra.run.smithi012.stdout:lab-extras 37 kB/s | 1.7 kB 00:00 2024-05-17T22:15:55.246 INFO:teuthology.orchestra.run.smithi012.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-17T22:15:55.246 INFO:teuthology.orchestra.run.smithi012.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-17T22:15:55.340 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-05-17T22:15:55.346 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-05-17T22:15:55.346 INFO:teuthology.orchestra.run.smithi012.stdout: Package Architecture Version Repository Size 2024-05-17T22:15:55.346 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-05-17T22:15:55.346 INFO:teuthology.orchestra.run.smithi012.stdout:Installing: 2024-05-17T22:15:55.346 INFO:teuthology.orchestra.run.smithi012.stdout: kernel x86_64 5.14.0-446.el9 baseos 550 k 2024-05-17T22:15:55.347 INFO:teuthology.orchestra.run.smithi012.stdout:Installing dependencies: 2024-05-17T22:15:55.347 INFO:teuthology.orchestra.run.smithi012.stdout: kernel-core x86_64 5.14.0-446.el9 baseos 15 M 2024-05-17T22:15:55.347 INFO:teuthology.orchestra.run.smithi012.stdout: kernel-modules x86_64 5.14.0-446.el9 baseos 34 M 2024-05-17T22:15:55.347 INFO:teuthology.orchestra.run.smithi012.stdout: kernel-modules-core x86_64 5.14.0-446.el9 baseos 28 M 2024-05-17T22:15:55.347 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:15:55.347 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction Summary 2024-05-17T22:15:55.347 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-05-17T22:15:55.347 INFO:teuthology.orchestra.run.smithi012.stdout:Install 4 Packages 2024-05-17T22:15:55.347 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:15:55.348 INFO:teuthology.orchestra.run.smithi012.stdout:Total download size: 78 M 2024-05-17T22:15:55.348 INFO:teuthology.orchestra.run.smithi012.stdout:Installed size: 123 M 2024-05-17T22:15:55.348 INFO:teuthology.orchestra.run.smithi012.stdout:Downloading Packages: 2024-05-17T22:15:56.341 INFO:teuthology.orchestra.run.smithi012.stdout:(1/4): kernel-5.14.0-446.el9.x86_64.rpm 594 kB/s | 550 kB 00:00 2024-05-17T22:16:00.832 INFO:teuthology.orchestra.run.smithi012.stdout:(2/4): kernel-modules-core-5.14.0-446.el9.x86_6 6.2 MB/s | 28 MB 00:04 2024-05-17T22:16:02.524 INFO:teuthology.orchestra.run.smithi012.stdout:(3/4): kernel-core-5.14.0-446.el9.x86_64.rpm 2.2 MB/s | 15 MB 00:07 2024-05-17T22:16:04.757 INFO:teuthology.orchestra.run.smithi012.stdout:(4/4): kernel-modules-5.14.0-446.el9.x86_64.rpm 3.6 MB/s | 34 MB 00:09 2024-05-17T22:16:04.757 INFO:teuthology.orchestra.run.smithi012.stdout:-------------------------------------------------------------------------------- 2024-05-17T22:16:04.757 INFO:teuthology.orchestra.run.smithi012.stdout:Total 8.3 MB/s | 78 MB 00:09 2024-05-17T22:16:05.472 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction check 2024-05-17T22:16:05.884 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction check succeeded. 2024-05-17T22:16:05.884 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction test 2024-05-17T22:16:06.862 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction test succeeded. 2024-05-17T22:16:06.863 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction 2024-05-17T22:16:09.373 INFO:teuthology.orchestra.run.smithi012.stdout: Preparing : 1/1 2024-05-17T22:16:10.020 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : kernel-modules-core-5.14.0-446.el9.x86_64 1/4 2024-05-17T22:16:10.107 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-17T22:16:12.906 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-17T22:16:13.045 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-17T22:16:18.116 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-17T22:16:18.164 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : kernel-5.14.0-446.el9.x86_64 4/4 2024-05-17T22:16:23.258 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: kernel-modules-core-5.14.0-446.el9.x86_64 4/4 2024-05-17T22:16:54.325 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: kernel-core-5.14.0-446.el9.x86_64 4/4 2024-05-17T22:16:54.366 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: kernel-modules-5.14.0-446.el9.x86_64 4/4 2024-05-17T22:16:54.781 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: kernel-5.14.0-446.el9.x86_64 4/4 2024-05-17T22:16:54.781 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : kernel-5.14.0-446.el9.x86_64 1/4 2024-05-17T22:16:54.781 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-17T22:16:54.781 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-17T22:16:55.557 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : kernel-modules-core-5.14.0-446.el9.x86_64 4/4 2024-05-17T22:16:55.558 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:16:55.558 INFO:teuthology.orchestra.run.smithi012.stdout:Installed: 2024-05-17T22:16:55.558 INFO:teuthology.orchestra.run.smithi012.stdout: kernel-5.14.0-446.el9.x86_64 2024-05-17T22:16:55.558 INFO:teuthology.orchestra.run.smithi012.stdout: kernel-core-5.14.0-446.el9.x86_64 2024-05-17T22:16:55.558 INFO:teuthology.orchestra.run.smithi012.stdout: kernel-modules-5.14.0-446.el9.x86_64 2024-05-17T22:16:55.558 INFO:teuthology.orchestra.run.smithi012.stdout: kernel-modules-core-5.14.0-446.el9.x86_64 2024-05-17T22:16:55.558 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:16:55.558 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-05-17T22:16:55.923 DEBUG:teuthology.orchestra.run.smithi012:> rpm -q kernel | sort -rV | head -n 1 2024-05-17T22:16:55.991 INFO:teuthology.orchestra.run.smithi012.stdout:kernel-5.14.0-446.el9.x86_64 2024-05-17T22:16:55.991 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-446.el9.x86_64 2024-05-17T22:16:55.991 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-446.el9.x86_64 2024-05-17T22:16:55.991 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-17T22:16:55.992 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-17T22:16:55.992 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-05-17T22:16:55.992 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi012.front.sepia.ceph.com, path=None, version=distro) 2024-05-17T22:16:55.992 DEBUG:teuthology.orchestra.run.smithi012:> rpm -q kernel | sort -rV | head -n 1 2024-05-17T22:16:56.043 INFO:teuthology.orchestra.run.smithi012.stdout:kernel-5.14.0-446.el9.x86_64 2024-05-17T22:16:56.043 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-446.el9.x86_64 2024-05-17T22:16:56.044 DEBUG:teuthology.orchestra.run.smithi012:> sudo rpm -qi grub2-tools 2024-05-17T22:16:56.075 INFO:teuthology.orchestra.run.smithi012.stdout:Name : grub2-tools 2024-05-17T22:16:56.075 INFO:teuthology.orchestra.run.smithi012.stdout:Epoch : 1 2024-05-17T22:16:56.075 INFO:teuthology.orchestra.run.smithi012.stdout:Version : 2.06 2024-05-17T22:16:56.075 INFO:teuthology.orchestra.run.smithi012.stdout:Release : 68.el9 2024-05-17T22:16:56.075 INFO:teuthology.orchestra.run.smithi012.stdout:Architecture: x86_64 2024-05-17T22:16:56.075 INFO:teuthology.orchestra.run.smithi012.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-17T22:16:56.075 INFO:teuthology.orchestra.run.smithi012.stdout:Group : Unspecified 2024-05-17T22:16:56.075 INFO:teuthology.orchestra.run.smithi012.stdout:Size : 8264254 2024-05-17T22:16:56.075 INFO:teuthology.orchestra.run.smithi012.stdout:License : GPLv3+ 2024-05-17T22:16:56.076 INFO:teuthology.orchestra.run.smithi012.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-17T22:16:56.076 INFO:teuthology.orchestra.run.smithi012.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-17T22:16:56.076 INFO:teuthology.orchestra.run.smithi012.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-17T22:16:56.076 INFO:teuthology.orchestra.run.smithi012.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-17T22:16:56.076 INFO:teuthology.orchestra.run.smithi012.stdout:Packager : builder@centos.org 2024-05-17T22:16:56.076 INFO:teuthology.orchestra.run.smithi012.stdout:Vendor : CentOS 2024-05-17T22:16:56.076 INFO:teuthology.orchestra.run.smithi012.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-17T22:16:56.076 INFO:teuthology.orchestra.run.smithi012.stdout:Summary : Support tools for GRUB. 2024-05-17T22:16:56.076 INFO:teuthology.orchestra.run.smithi012.stdout:Description : 2024-05-17T22:16:56.076 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:16:56.076 INFO:teuthology.orchestra.run.smithi012.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-17T22:16:56.076 INFO:teuthology.orchestra.run.smithi012.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-17T22:16:56.076 INFO:teuthology.orchestra.run.smithi012.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-17T22:16:56.076 INFO:teuthology.orchestra.run.smithi012.stdout:hardware devices. 2024-05-17T22:16:56.077 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:16:56.077 INFO:teuthology.orchestra.run.smithi012.stdout:This subpackage provides tools for support of all platforms. 2024-05-17T22:16:56.077 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-17T22:16:56.077 INFO:teuthology.task.kernel:Updating grub on smithi012 to boot 5.14.0-446.el9.x86_64 2024-05-17T22:16:56.077 DEBUG:teuthology.orchestra.run.smithi012:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-17T22:16:56.675 INFO:teuthology.orchestra.run.smithi012.stderr:Generating grub configuration file ... 2024-05-17T22:16:58.063 INFO:teuthology.orchestra.run.smithi012.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-17T22:16:58.089 INFO:teuthology.orchestra.run.smithi012.stderr:done 2024-05-17T22:16:58.092 DEBUG:teuthology.orchestra.run.smithi012:> mktemp 2024-05-17T22:16:58.107 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/tmp.xko8nrTq1y 2024-05-17T22:16:58.107 DEBUG:teuthology.orchestra.run.smithi012:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.xko8nrTq1y 2024-05-17T22:16:58.173 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0666 /tmp/tmp.xko8nrTq1y 2024-05-17T22:16:58.321 DEBUG:teuthology.orchestra.remote:smithi012:/tmp/tmp.xko8nrTq1y is 6KB 2024-05-17T22:16:58.370 DEBUG:teuthology.orchestra.run.smithi012:> rm -fr /tmp/tmp.xko8nrTq1y 2024-05-17T22:16:58.385 DEBUG:teuthology.orchestra.run.smithi012:> sudo /bin/ls /boot/loader/entries || true 2024-05-17T22:16:58.451 INFO:teuthology.orchestra.run.smithi012.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-17T22:16:58.451 INFO:teuthology.orchestra.run.smithi012.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-17T22:16:58.451 INFO:teuthology.orchestra.run.smithi012.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-17T22:16:58.451 INFO:teuthology.orchestra.run.smithi012.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-446.el9.x86_64.conf 2024-05-17T22:16:58.452 DEBUG:teuthology.orchestra.run.smithi012:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-446.el9.x86_64 2024-05-17T22:16:58.607 DEBUG:teuthology.orchestra.run.smithi012:> sudo shutdown -r now 2024-05-17T22:17:28.610 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-17T22:17:28.610 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-05-17T22:17:28.611 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T22:17:47.039 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2024-05-17T22:17:56.040 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-05-17T22:17:56.041 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T22:18:14.430 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2024-05-17T22:18:26.432 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-05-17T22:18:26.433 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T22:18:26.791 DEBUG:teuthology.orchestra.run.smithi012:> true 2024-05-17T22:18:27.223 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-05-17T22:18:27.223 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-446.el9.x86_64"... 2024-05-17T22:18:27.223 DEBUG:teuthology.orchestra.run.smithi012:> uname -r 2024-05-17T22:18:27.239 INFO:teuthology.orchestra.run.smithi012.stdout:5.14.0-446.el9.x86_64 2024-05-17T22:18:27.240 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-446.el9.x86_64 vs 5.14.0-446.el9.x86_64 2024-05-17T22:18:27.240 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-17T22:18:27.240 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-17T22:18:28.242 DEBUG:teuthology.parallel:result is None 2024-05-17T22:18:28.242 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-17T22:18:28.250 INFO:teuthology.task.internal:Creating test directory... 2024-05-17T22:18:28.250 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-17T22:18:28.253 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-17T22:18:28.255 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-17T22:18:28.273 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-17T22:18:28.308 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-17T22:18:28.354 INFO:teuthology.task.internal:Creating archive directory... 2024-05-17T22:18:28.354 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-17T22:18:28.356 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-17T22:18:28.358 DEBUG:teuthology.orchestra.run.smithi151:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-17T22:18:28.472 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-17T22:18:28.479 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-17T22:18:28.479 DEBUG:teuthology.orchestra.run.smithi096:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-17T22:18:28.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T22:18:28.495 DEBUG:teuthology.orchestra.run.smithi151:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-17T22:18:28.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T22:18:28.512 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-17T22:18:28.514 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-17T22:18:28.539 DEBUG:teuthology.orchestra.run.smithi151:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-17T22:18:28.578 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-17T22:18:28.589 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-17T22:18:28.592 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-17T22:18:28.615 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-17T22:18:28.617 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-17T22:18:28.619 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-17T22:18:28.621 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-17T22:18:28.628 INFO:teuthology.task.internal:Configuring sudo... 2024-05-17T22:18:28.628 DEBUG:teuthology.orchestra.run.smithi012:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-17T22:18:28.630 DEBUG:teuthology.orchestra.run.smithi096:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-17T22:18:28.659 DEBUG:teuthology.orchestra.run.smithi151:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-17T22:18:28.688 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-17T22:18:28.697 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-17T22:18:28.697 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-17T22:18:28.699 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-17T22:18:28.725 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-17T22:18:28.744 DEBUG:teuthology.orchestra.run.smithi012:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-17T22:18:28.797 DEBUG:teuthology.orchestra.run.smithi012:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-17T22:18:28.849 DEBUG:teuthology.orchestra.run.smithi012:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-17T22:18:28.928 DEBUG:teuthology.orchestra.run.smithi012:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-17T22:18:28.972 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-17T22:18:28.972 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-17T22:18:29.057 DEBUG:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-17T22:18:29.099 DEBUG:teuthology.orchestra.run.smithi096:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-17T22:18:29.156 DEBUG:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-17T22:18:29.237 DEBUG:teuthology.orchestra.run.smithi096:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-17T22:18:29.281 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-05-17T22:18:29.281 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-17T22:18:29.346 DEBUG:teuthology.orchestra.run.smithi151:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-17T22:18:29.391 DEBUG:teuthology.orchestra.run.smithi151:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-17T22:18:29.448 DEBUG:teuthology.orchestra.run.smithi151:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-17T22:18:29.532 DEBUG:teuthology.orchestra.run.smithi151:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-17T22:18:29.572 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-05-17T22:18:29.572 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-17T22:18:29.638 DEBUG:teuthology.orchestra.run.smithi012:> sudo service rsyslog restart 2024-05-17T22:18:29.641 DEBUG:teuthology.orchestra.run.smithi096:> sudo service rsyslog restart 2024-05-17T22:18:29.643 DEBUG:teuthology.orchestra.run.smithi151:> sudo service rsyslog restart 2024-05-17T22:18:29.677 INFO:teuthology.orchestra.run.smithi096.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-17T22:18:29.690 INFO:teuthology.orchestra.run.smithi012.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-17T22:18:29.720 INFO:teuthology.orchestra.run.smithi151.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-17T22:18:30.084 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-17T22:18:30.092 INFO:teuthology.task.internal:Starting timer... 2024-05-17T22:18:30.092 INFO:teuthology.run_tasks:Running task pcp... 2024-05-17T22:18:30.141 INFO:teuthology.run_tasks:Running task selinux... 2024-05-17T22:18:30.214 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-17T22:18:30.214 DEBUG:teuthology.orchestra.run.smithi012:> sudo service auditd rotate 2024-05-17T22:18:30.299 INFO:teuthology.orchestra.run.smithi012.stdout:Rotating logs: 2024-05-17T22:18:30.301 DEBUG:teuthology.orchestra.run.smithi096:> sudo service auditd rotate 2024-05-17T22:18:30.357 INFO:teuthology.orchestra.run.smithi096.stdout:Rotating logs: 2024-05-17T22:18:30.359 DEBUG:teuthology.orchestra.run.smithi151:> sudo service auditd rotate 2024-05-17T22:18:30.402 INFO:teuthology.orchestra.run.smithi151.stdout:Rotating logs: 2024-05-17T22:18:30.403 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-17T22:18:30.404 DEBUG:teuthology.orchestra.run.smithi012:> /usr/sbin/getenforce 2024-05-17T22:18:30.425 INFO:teuthology.orchestra.run.smithi012.stdout:Permissive 2024-05-17T22:18:30.426 DEBUG:teuthology.orchestra.run.smithi096:> /usr/sbin/getenforce 2024-05-17T22:18:30.449 INFO:teuthology.orchestra.run.smithi096.stdout:Permissive 2024-05-17T22:18:30.449 DEBUG:teuthology.orchestra.run.smithi151:> /usr/sbin/getenforce 2024-05-17T22:18:30.474 INFO:teuthology.orchestra.run.smithi151.stdout:Permissive 2024-05-17T22:18:30.474 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi012.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi096.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi151.front.sepia.ceph.com': 'permissive'} 2024-05-17T22:18:30.474 DEBUG:teuthology.orchestra.run.smithi012:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-17T22:18:30.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T22:18:30.500 DEBUG:teuthology.orchestra.run.smithi096:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-17T22:18:30.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T22:18:30.526 DEBUG:teuthology.orchestra.run.smithi151:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-17T22:18:30.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T22:18:30.551 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-17T22:18:30.552 DEBUG:teuthology.orchestra.run.smithi012:> sudo /usr/sbin/setenforce permissive 2024-05-17T22:18:30.658 DEBUG:teuthology.orchestra.run.smithi096:> sudo /usr/sbin/setenforce permissive 2024-05-17T22:18:30.693 DEBUG:teuthology.orchestra.run.smithi151:> sudo /usr/sbin/setenforce permissive 2024-05-17T22:18:30.721 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-17T22:18:30.730 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-17T22:18:30.734 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-17T22:18:30.862 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-17T22:18:30.871 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-17T22:18:30.872 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi012.front.sepia.ceph.com,smithi096.front.sepia.ceph.com,smithi151.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-17T22:23:01.685 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi012.front.sepia.ceph.com'), Remote(name='ubuntu@smithi096.front.sepia.ceph.com'), Remote(name='ubuntu@smithi151.front.sepia.ceph.com')] 2024-05-17T22:23:01.687 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-05-17T22:23:01.688 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T22:23:01.773 DEBUG:teuthology.orchestra.run.smithi012:> true 2024-05-17T22:23:01.802 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-05-17T22:23:01.802 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-05-17T22:23:01.803 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T22:23:01.881 DEBUG:teuthology.orchestra.run.smithi096:> true 2024-05-17T22:23:01.958 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-05-17T22:23:01.959 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-05-17T22:23:01.959 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T22:23:02.036 DEBUG:teuthology.orchestra.run.smithi151:> true 2024-05-17T22:23:02.137 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-05-17T22:23:02.137 INFO:teuthology.run_tasks:Running task clock... 2024-05-17T22:23:02.147 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-17T22:23:02.147 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-17T22:23:02.148 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-17T22:23:02.150 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-17T22:23:02.151 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-17T22:23:02.153 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-17T22:23:02.153 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-17T22:23:02.180 INFO:teuthology.orchestra.run.smithi012.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-17T22:23:02.180 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-17T22:23:02.194 INFO:teuthology.orchestra.run.smithi012.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-17T22:23:02.195 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-17T22:23:02.205 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-17T22:23:02.218 INFO:teuthology.orchestra.run.smithi096.stderr:sudo: ntpd: command not found 2024-05-17T22:23:02.219 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-17T22:23:02.220 INFO:teuthology.orchestra.run.smithi012.stderr:sudo: ntpd: command not found 2024-05-17T22:23:02.229 INFO:teuthology.orchestra.run.smithi096.stdout:506 Cannot talk to daemon 2024-05-17T22:23:02.232 INFO:teuthology.orchestra.run.smithi012.stdout:506 Cannot talk to daemon 2024-05-17T22:23:02.241 INFO:teuthology.orchestra.run.smithi151.stderr:sudo: ntpd: command not found 2024-05-17T22:23:02.242 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-17T22:23:02.247 INFO:teuthology.orchestra.run.smithi012.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-17T22:23:02.251 INFO:teuthology.orchestra.run.smithi151.stdout:506 Cannot talk to daemon 2024-05-17T22:23:02.256 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-17T22:23:02.262 INFO:teuthology.orchestra.run.smithi012.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-17T22:23:02.264 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-17T22:23:02.277 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-17T22:23:02.318 INFO:teuthology.orchestra.run.smithi012.stderr:bash: line 1: ntpq: command not found 2024-05-17T22:23:02.322 INFO:teuthology.orchestra.run.smithi012.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-17T22:23:02.322 INFO:teuthology.orchestra.run.smithi012.stdout:=============================================================================== 2024-05-17T22:23:02.323 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T22:23:02.323 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T22:23:02.323 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T22:23:02.323 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T22:23:02.323 INFO:teuthology.orchestra.run.smithi096.stderr:bash: line 1: ntpq: command not found 2024-05-17T22:23:02.327 INFO:teuthology.orchestra.run.smithi096.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-17T22:23:02.327 INFO:teuthology.orchestra.run.smithi096.stdout:=============================================================================== 2024-05-17T22:23:02.327 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T22:23:02.327 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T22:23:02.327 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T22:23:02.327 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T22:23:02.331 INFO:teuthology.orchestra.run.smithi151.stderr:bash: line 1: ntpq: command not found 2024-05-17T22:23:02.338 INFO:teuthology.orchestra.run.smithi151.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-17T22:23:02.338 INFO:teuthology.orchestra.run.smithi151.stdout:=============================================================================== 2024-05-17T22:23:02.338 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T22:23:02.338 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T22:23:02.339 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T22:23:02.339 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T22:23:02.339 INFO:teuthology.run_tasks:Running task install... 2024-05-17T22:23:02.347 DEBUG:teuthology.task.install:project ceph 2024-05-17T22:23:02.347 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '931809a9e5d295bd61984693cac44fe131ded4bc'}} 2024-05-17T22:23:02.347 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '931809a9e5d295bd61984693cac44fe131ded4bc'} 2024-05-17T22:23:02.347 INFO:teuthology.task.install:Using flavor: default 2024-05-17T22:23:02.353 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-17T22:23:02.353 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-17T22:23:02.353 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '931809a9e5d295bd61984693cac44fe131ded4bc', 'tag': None, 'wait_for_package': False} 2024-05-17T22:23:02.354 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=931809a9e5d295bd61984693cac44fe131ded4bc 2024-05-17T22:23:02.355 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '931809a9e5d295bd61984693cac44fe131ded4bc', 'tag': None, 'wait_for_package': False} 2024-05-17T22:23:02.356 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=931809a9e5d295bd61984693cac44fe131ded4bc 2024-05-17T22:23:02.356 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '931809a9e5d295bd61984693cac44fe131ded4bc', 'tag': None, 'wait_for_package': False} 2024-05-17T22:23:02.357 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=931809a9e5d295bd61984693cac44fe131ded4bc 2024-05-17T22:23:02.502 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/centos/9/flavors/default/ 2024-05-17T22:23:02.502 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3794.g931809a9 2024-05-17T22:23:02.570 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/centos/9/flavors/default/ 2024-05-17T22:23:02.571 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3794.g931809a9 2024-05-17T22:23:02.586 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/centos/9/flavors/default/ 2024-05-17T22:23:02.586 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3794.g931809a9 2024-05-17T22:23:02.800 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-17T22:23:02.800 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-05-17T22:23:02.800 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-17T22:23:02.826 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-17T22:23:02.827 DEBUG:teuthology.orchestra.run.smithi096:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/931809a9e5d295bd61984693cac44fe131ded4bc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-17T22:23:02.892 DEBUG:teuthology.orchestra.run.smithi096:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-17T22:23:02.930 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-17T22:23:02.931 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-17T22:23:02.931 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-17T22:23:02.940 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-17T22:23:02.941 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-05-17T22:23:02.941 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-17T22:23:02.960 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-17T22:23:02.960 DEBUG:teuthology.orchestra.run.smithi012:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/931809a9e5d295bd61984693cac44fe131ded4bc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-17T22:23:02.964 DEBUG:teuthology.orchestra.run.smithi096:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-17T22:23:02.965 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-17T22:23:02.965 DEBUG:teuthology.orchestra.run.smithi151:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/931809a9e5d295bd61984693cac44fe131ded4bc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-17T22:23:03.027 DEBUG:teuthology.orchestra.run.smithi012:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-17T22:23:03.028 INFO:teuthology.orchestra.run.smithi096.stdout:check_obsoletes = 1 2024-05-17T22:23:03.029 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum clean all 2024-05-17T22:23:03.030 DEBUG:teuthology.orchestra.run.smithi151:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-17T22:23:03.103 DEBUG:teuthology.orchestra.run.smithi151:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-17T22:23:03.105 DEBUG:teuthology.orchestra.run.smithi012:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-17T22:23:03.166 INFO:teuthology.orchestra.run.smithi151.stdout:check_obsoletes = 1 2024-05-17T22:23:03.167 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum clean all 2024-05-17T22:23:03.170 INFO:teuthology.orchestra.run.smithi012.stdout:check_obsoletes = 1 2024-05-17T22:23:03.172 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum clean all 2024-05-17T22:23:03.256 INFO:teuthology.orchestra.run.smithi096.stdout:57 files removed 2024-05-17T22:23:03.284 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-17T22:23:03.399 INFO:teuthology.orchestra.run.smithi151.stdout:57 files removed 2024-05-17T22:23:03.418 INFO:teuthology.orchestra.run.smithi012.stdout:57 files removed 2024-05-17T22:23:03.427 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-17T22:23:03.445 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-17T22:23:04.680 INFO:teuthology.orchestra.run.smithi096.stdout:ceph packages for x86_64 73 kB/s | 83 kB 00:01 2024-05-17T22:23:04.698 INFO:teuthology.orchestra.run.smithi151.stdout:ceph packages for x86_64 82 kB/s | 83 kB 00:01 2024-05-17T22:23:04.827 INFO:teuthology.orchestra.run.smithi012.stdout:ceph packages for x86_64 76 kB/s | 83 kB 00:01 2024-05-17T22:23:05.315 INFO:teuthology.orchestra.run.smithi151.stdout:ceph noarch packages 20 kB/s | 12 kB 00:00 2024-05-17T22:23:05.388 INFO:teuthology.orchestra.run.smithi096.stdout:ceph noarch packages 18 kB/s | 12 kB 00:00 2024-05-17T22:23:05.428 INFO:teuthology.orchestra.run.smithi012.stdout:ceph noarch packages 22 kB/s | 12 kB 00:00 2024-05-17T22:23:05.931 INFO:teuthology.orchestra.run.smithi096.stdout:ceph source packages 3.8 kB/s | 1.9 kB 00:00 2024-05-17T22:23:05.932 INFO:teuthology.orchestra.run.smithi151.stdout:ceph source packages 3.3 kB/s | 1.9 kB 00:00 2024-05-17T22:23:06.042 INFO:teuthology.orchestra.run.smithi012.stdout:ceph source packages 3.4 kB/s | 1.9 kB 00:00 2024-05-17T22:23:06.441 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.1 MB 00:00 2024-05-17T22:23:06.564 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.1 MB 00:00 2024-05-17T22:23:06.654 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.1 MB 00:00 2024-05-17T22:23:08.201 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - AppStream 30 MB/s | 19 MB 00:00 2024-05-17T22:23:08.273 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - AppStream 31 MB/s | 19 MB 00:00 2024-05-17T22:23:12.326 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-05-17T22:23:12.482 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-05-17T22:23:13.744 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - Extras packages 62 kB/s | 16 kB 00:00 2024-05-17T22:23:13.891 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - Extras packages 67 kB/s | 16 kB 00:00 2024-05-17T22:23:15.109 INFO:teuthology.orchestra.run.smithi096.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-05-17T22:23:15.290 INFO:teuthology.orchestra.run.smithi151.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-05-17T22:23:20.268 INFO:teuthology.orchestra.run.smithi096.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-05-17T22:23:20.596 INFO:teuthology.orchestra.run.smithi151.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-05-17T22:23:22.527 INFO:teuthology.orchestra.run.smithi096.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-17T22:23:22.527 INFO:teuthology.orchestra.run.smithi096.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-17T22:23:22.538 INFO:teuthology.orchestra.run.smithi096.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-17T22:23:22.539 INFO:teuthology.orchestra.run.smithi096.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-17T22:23:22.540 INFO:teuthology.orchestra.run.smithi096.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-17T22:23:22.541 INFO:teuthology.orchestra.run.smithi096.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-17T22:23:22.542 INFO:teuthology.orchestra.run.smithi096.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-17T22:23:22.545 INFO:teuthology.orchestra.run.smithi096.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-17T22:23:22.545 INFO:teuthology.orchestra.run.smithi096.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-17T22:23:22.607 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-05-17T22:23:22.624 INFO:teuthology.orchestra.run.smithi096.stdout:======================================================================================= 2024-05-17T22:23:22.624 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-05-17T22:23:22.624 INFO:teuthology.orchestra.run.smithi096.stdout:======================================================================================= 2024-05-17T22:23:22.624 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-05-17T22:23:22.624 INFO:teuthology.orchestra.run.smithi096.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-17T22:23:22.624 INFO:teuthology.orchestra.run.smithi096.stdout: ceph x86_64 2:19.0.0-3794.g931809a9.el9 ceph 6.6 k 2024-05-17T22:23:22.624 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-base x86_64 2:19.0.0-3794.g931809a9.el9 ceph 5.4 M 2024-05-17T22:23:22.624 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-fuse x86_64 2:19.0.0-3794.g931809a9.el9 ceph 866 k 2024-05-17T22:23:22.624 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3794.g931809a9.el9 ceph 147 k 2024-05-17T22:23:22.624 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr x86_64 2:19.0.0-3794.g931809a9.el9 ceph 1.5 M 2024-05-17T22:23:22.624 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 143 k 2024-05-17T22:23:22.625 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 3.5 M 2024-05-17T22:23:22.625 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 7.4 M 2024-05-17T22:23:22.625 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-rook noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 49 k 2024-05-17T22:23:22.625 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-radosgw x86_64 2:19.0.0-3794.g931809a9.el9 ceph 10 M 2024-05-17T22:23:22.625 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-test x86_64 2:19.0.0-3794.g931809a9.el9 ceph 43 M 2024-05-17T22:23:22.625 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-volume noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 273 k 2024-05-17T22:23:22.625 INFO:teuthology.orchestra.run.smithi096.stdout: cephadm noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 763 k 2024-05-17T22:23:22.625 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-mirror x86_64 2:19.0.0-3794.g931809a9.el9 ceph 227 k 2024-05-17T22:23:22.625 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-shell x86_64 2:19.0.0-3794.g931809a9.el9 ceph 27 k 2024-05-17T22:23:22.626 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-top noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 18 k 2024-05-17T22:23:22.626 INFO:teuthology.orchestra.run.smithi096.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-17T22:23:22.626 INFO:teuthology.orchestra.run.smithi096.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-17T22:23:22.626 INFO:teuthology.orchestra.run.smithi096.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-17T22:23:22.626 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs-devel x86_64 2:19.0.0-3794.g931809a9.el9 ceph 32 k 2024-05-17T22:23:22.626 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs2 x86_64 2:19.0.0-3794.g931809a9.el9 ceph 759 k 2024-05-17T22:23:22.626 INFO:teuthology.orchestra.run.smithi096.stdout: librados-devel x86_64 2:19.0.0-3794.g931809a9.el9 ceph 127 k 2024-05-17T22:23:22.626 INFO:teuthology.orchestra.run.smithi096.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-17T22:23:22.626 INFO:teuthology.orchestra.run.smithi096.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-17T22:23:22.626 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-17T22:23:22.626 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-17T22:23:22.627 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-17T22:23:22.627 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-17T22:23:22.627 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cephfs x86_64 2:19.0.0-3794.g931809a9.el9 ceph 162 k 2024-05-17T22:23:22.627 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rados x86_64 2:19.0.0-3794.g931809a9.el9 ceph 321 k 2024-05-17T22:23:22.627 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rbd x86_64 2:19.0.0-3794.g931809a9.el9 ceph 300 k 2024-05-17T22:23:22.627 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rgw x86_64 2:19.0.0-3794.g931809a9.el9 ceph 99 k 2024-05-17T22:23:22.627 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-fuse x86_64 2:19.0.0-3794.g931809a9.el9 ceph 86 k 2024-05-17T22:23:22.627 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-mirror x86_64 2:19.0.0-3794.g931809a9.el9 ceph 3.0 M 2024-05-17T22:23:22.627 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-nbd x86_64 2:19.0.0-3794.g931809a9.el9 ceph 169 k 2024-05-17T22:23:22.627 INFO:teuthology.orchestra.run.smithi096.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-17T22:23:22.627 INFO:teuthology.orchestra.run.smithi096.stdout:Upgrading: 2024-05-17T22:23:22.627 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-17T22:23:22.627 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-17T22:23:22.628 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-17T22:23:22.628 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-17T22:23:22.628 INFO:teuthology.orchestra.run.smithi096.stdout: librados2 x86_64 2:19.0.0-3794.g931809a9.el9 ceph 3.3 M 2024-05-17T22:23:22.628 INFO:teuthology.orchestra.run.smithi096.stdout: librbd1 x86_64 2:19.0.0-3794.g931809a9.el9 ceph 3.1 M 2024-05-17T22:23:22.628 INFO:teuthology.orchestra.run.smithi096.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-17T22:23:22.628 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-17T22:23:22.628 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-05-17T22:23:22.628 INFO:teuthology.orchestra.run.smithi096.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-17T22:23:22.628 INFO:teuthology.orchestra.run.smithi096.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-17T22:23:22.628 INFO:teuthology.orchestra.run.smithi096.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-17T22:23:22.629 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-common x86_64 2:19.0.0-3794.g931809a9.el9 ceph 20 M 2024-05-17T22:23:22.629 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 34 k 2024-05-17T22:23:22.629 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mds x86_64 2:19.0.0-3794.g931809a9.el9 ceph 2.3 M 2024-05-17T22:23:22.629 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 274 k 2024-05-17T22:23:22.629 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mon x86_64 2:19.0.0-3794.g931809a9.el9 ceph 4.9 M 2024-05-17T22:23:22.629 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-osd x86_64 2:19.0.0-3794.g931809a9.el9 ceph 18 M 2024-05-17T22:23:22.629 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 16 k 2024-05-17T22:23:22.629 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-selinux x86_64 2:19.0.0-3794.g931809a9.el9 ceph 25 k 2024-05-17T22:23:22.629 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-17T22:23:22.629 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-17T22:23:22.630 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-17T22:23:22.630 INFO:teuthology.orchestra.run.smithi096.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-17T22:23:22.630 INFO:teuthology.orchestra.run.smithi096.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-17T22:23:22.630 INFO:teuthology.orchestra.run.smithi096.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-17T22:23:22.630 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-17T22:23:22.630 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-17T22:23:22.630 INFO:teuthology.orchestra.run.smithi096.stdout: libcephsqlite x86_64 2:19.0.0-3794.g931809a9.el9 ceph 158 k 2024-05-17T22:23:22.630 INFO:teuthology.orchestra.run.smithi096.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-17T22:23:22.630 INFO:teuthology.orchestra.run.smithi096.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-17T22:23:22.630 INFO:teuthology.orchestra.run.smithi096.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-17T22:23:22.631 INFO:teuthology.orchestra.run.smithi096.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-17T22:23:22.631 INFO:teuthology.orchestra.run.smithi096.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-17T22:23:22.631 INFO:teuthology.orchestra.run.smithi096.stdout: libradosstriper1 x86_64 2:19.0.0-3794.g931809a9.el9 ceph 501 k 2024-05-17T22:23:22.631 INFO:teuthology.orchestra.run.smithi096.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-17T22:23:22.631 INFO:teuthology.orchestra.run.smithi096.stdout: librgw2 x86_64 2:19.0.0-3794.g931809a9.el9 ceph 5.2 M 2024-05-17T22:23:22.631 INFO:teuthology.orchestra.run.smithi096.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-17T22:23:22.631 INFO:teuthology.orchestra.run.smithi096.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-17T22:23:22.631 INFO:teuthology.orchestra.run.smithi096.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-17T22:23:22.631 INFO:teuthology.orchestra.run.smithi096.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-17T22:23:22.631 INFO:teuthology.orchestra.run.smithi096.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-17T22:23:22.631 INFO:teuthology.orchestra.run.smithi096.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-17T22:23:22.631 INFO:teuthology.orchestra.run.smithi096.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-17T22:23:22.631 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-17T22:23:22.632 INFO:teuthology.orchestra.run.smithi096.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-17T22:23:22.632 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-17T22:23:22.632 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-17T22:23:22.632 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-17T22:23:22.632 INFO:teuthology.orchestra.run.smithi096.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-17T22:23:22.632 INFO:teuthology.orchestra.run.smithi096.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-17T22:23:22.632 INFO:teuthology.orchestra.run.smithi096.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-17T22:23:22.632 INFO:teuthology.orchestra.run.smithi096.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-17T22:23:22.632 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-17T22:23:22.632 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-argparse x86_64 2:19.0.0-3794.g931809a9.el9 ceph 45 k 2024-05-17T22:23:22.632 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-common x86_64 2:19.0.0-3794.g931809a9.el9 ceph 136 k 2024-05-17T22:23:22.632 INFO:teuthology.orchestra.run.smithi096.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-17T22:23:22.632 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-17T22:23:22.633 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-17T22:23:22.633 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-17T22:23:22.633 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-17T22:23:22.633 INFO:teuthology.orchestra.run.smithi096.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-17T22:23:22.633 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-17T22:23:22.633 INFO:teuthology.orchestra.run.smithi096.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-17T22:23:22.633 INFO:teuthology.orchestra.run.smithi096.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-17T22:23:22.633 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-17T22:23:22.633 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-17T22:23:22.633 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-17T22:23:22.633 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-17T22:23:22.633 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-17T22:23:22.633 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-17T22:23:22.633 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-17T22:23:22.634 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-17T22:23:22.634 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-17T22:23:22.634 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-17T22:23:22.634 INFO:teuthology.orchestra.run.smithi096.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-17T22:23:22.634 INFO:teuthology.orchestra.run.smithi096.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-17T22:23:22.634 INFO:teuthology.orchestra.run.smithi096.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-17T22:23:22.634 INFO:teuthology.orchestra.run.smithi096.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-17T22:23:22.634 INFO:teuthology.orchestra.run.smithi096.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-17T22:23:22.634 INFO:teuthology.orchestra.run.smithi096.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-17T22:23:22.634 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-17T22:23:22.634 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-17T22:23:22.634 INFO:teuthology.orchestra.run.smithi096.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-17T22:23:22.634 INFO:teuthology.orchestra.run.smithi096.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-17T22:23:22.635 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-17T22:23:22.635 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-17T22:23:22.635 INFO:teuthology.orchestra.run.smithi096.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-17T22:23:22.635 INFO:teuthology.orchestra.run.smithi096.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-17T22:23:22.635 INFO:teuthology.orchestra.run.smithi096.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-17T22:23:22.635 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-17T22:23:22.635 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-17T22:23:22.635 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-17T22:23:22.635 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-17T22:23:22.635 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-17T22:23:22.635 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-17T22:23:22.635 INFO:teuthology.orchestra.run.smithi096.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-17T22:23:22.635 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-17T22:23:22.635 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-17T22:23:22.636 INFO:teuthology.orchestra.run.smithi096.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-17T22:23:22.636 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-17T22:23:22.636 INFO:teuthology.orchestra.run.smithi096.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-17T22:23:22.636 INFO:teuthology.orchestra.run.smithi096.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-17T22:23:22.636 INFO:teuthology.orchestra.run.smithi096.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-17T22:23:22.636 INFO:teuthology.orchestra.run.smithi096.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-17T22:23:22.636 INFO:teuthology.orchestra.run.smithi096.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-17T22:23:22.636 INFO:teuthology.orchestra.run.smithi096.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-17T22:23:22.636 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-17T22:23:22.636 INFO:teuthology.orchestra.run.smithi096.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-17T22:23:22.636 INFO:teuthology.orchestra.run.smithi096.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-17T22:23:22.637 INFO:teuthology.orchestra.run.smithi096.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-17T22:23:22.637 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-17T22:23:22.637 INFO:teuthology.orchestra.run.smithi096.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-17T22:23:22.637 INFO:teuthology.orchestra.run.smithi096.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-17T22:23:22.637 INFO:teuthology.orchestra.run.smithi096.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-17T22:23:22.637 INFO:teuthology.orchestra.run.smithi096.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-17T22:23:22.637 INFO:teuthology.orchestra.run.smithi096.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-17T22:23:22.637 INFO:teuthology.orchestra.run.smithi096.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-17T22:23:22.637 INFO:teuthology.orchestra.run.smithi096.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-17T22:23:22.637 INFO:teuthology.orchestra.run.smithi096.stdout:Installing weak dependencies: 2024-05-17T22:23:22.637 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-17T22:23:22.638 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-17T22:23:22.638 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-05-17T22:23:22.638 INFO:teuthology.orchestra.run.smithi096.stdout:======================================================================================= 2024-05-17T22:23:22.638 INFO:teuthology.orchestra.run.smithi096.stdout:Install 148 Packages 2024-05-17T22:23:22.638 INFO:teuthology.orchestra.run.smithi096.stdout:Upgrade 8 Packages 2024-05-17T22:23:22.638 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-17T22:23:22.638 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 219 M 2024-05-17T22:23:22.639 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-05-17T22:23:22.947 INFO:teuthology.orchestra.run.smithi151.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-17T22:23:22.947 INFO:teuthology.orchestra.run.smithi151.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-17T22:23:22.958 INFO:teuthology.orchestra.run.smithi151.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-17T22:23:22.959 INFO:teuthology.orchestra.run.smithi151.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-17T22:23:22.960 INFO:teuthology.orchestra.run.smithi151.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-17T22:23:22.960 INFO:teuthology.orchestra.run.smithi151.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-17T22:23:22.962 INFO:teuthology.orchestra.run.smithi151.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-17T22:23:22.964 INFO:teuthology.orchestra.run.smithi151.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-17T22:23:22.965 INFO:teuthology.orchestra.run.smithi151.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-17T22:23:23.025 INFO:teuthology.orchestra.run.smithi151.stdout:Dependencies resolved. 2024-05-17T22:23:23.042 INFO:teuthology.orchestra.run.smithi151.stdout:======================================================================================= 2024-05-17T22:23:23.042 INFO:teuthology.orchestra.run.smithi151.stdout: Package Arch Version Repository Size 2024-05-17T22:23:23.043 INFO:teuthology.orchestra.run.smithi151.stdout:======================================================================================= 2024-05-17T22:23:23.043 INFO:teuthology.orchestra.run.smithi151.stdout:Installing: 2024-05-17T22:23:23.043 INFO:teuthology.orchestra.run.smithi151.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-17T22:23:23.043 INFO:teuthology.orchestra.run.smithi151.stdout: ceph x86_64 2:19.0.0-3794.g931809a9.el9 ceph 6.6 k 2024-05-17T22:23:23.043 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-base x86_64 2:19.0.0-3794.g931809a9.el9 ceph 5.4 M 2024-05-17T22:23:23.043 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-fuse x86_64 2:19.0.0-3794.g931809a9.el9 ceph 866 k 2024-05-17T22:23:23.043 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3794.g931809a9.el9 ceph 147 k 2024-05-17T22:23:23.043 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr x86_64 2:19.0.0-3794.g931809a9.el9 ceph 1.5 M 2024-05-17T22:23:23.043 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 143 k 2024-05-17T22:23:23.043 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 3.5 M 2024-05-17T22:23:23.043 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 7.4 M 2024-05-17T22:23:23.043 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-rook noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 49 k 2024-05-17T22:23:23.043 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-radosgw x86_64 2:19.0.0-3794.g931809a9.el9 ceph 10 M 2024-05-17T22:23:23.044 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-test x86_64 2:19.0.0-3794.g931809a9.el9 ceph 43 M 2024-05-17T22:23:23.044 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-volume noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 273 k 2024-05-17T22:23:23.044 INFO:teuthology.orchestra.run.smithi151.stdout: cephadm noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 763 k 2024-05-17T22:23:23.044 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-mirror x86_64 2:19.0.0-3794.g931809a9.el9 ceph 227 k 2024-05-17T22:23:23.044 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-shell x86_64 2:19.0.0-3794.g931809a9.el9 ceph 27 k 2024-05-17T22:23:23.044 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-top noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 18 k 2024-05-17T22:23:23.044 INFO:teuthology.orchestra.run.smithi151.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-17T22:23:23.044 INFO:teuthology.orchestra.run.smithi151.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-17T22:23:23.044 INFO:teuthology.orchestra.run.smithi151.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-17T22:23:23.044 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs-devel x86_64 2:19.0.0-3794.g931809a9.el9 ceph 32 k 2024-05-17T22:23:23.044 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs2 x86_64 2:19.0.0-3794.g931809a9.el9 ceph 759 k 2024-05-17T22:23:23.044 INFO:teuthology.orchestra.run.smithi151.stdout: librados-devel x86_64 2:19.0.0-3794.g931809a9.el9 ceph 127 k 2024-05-17T22:23:23.044 INFO:teuthology.orchestra.run.smithi151.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-17T22:23:23.044 INFO:teuthology.orchestra.run.smithi151.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-17T22:23:23.045 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-17T22:23:23.045 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-17T22:23:23.045 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-17T22:23:23.045 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-17T22:23:23.045 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cephfs x86_64 2:19.0.0-3794.g931809a9.el9 ceph 162 k 2024-05-17T22:23:23.045 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rados x86_64 2:19.0.0-3794.g931809a9.el9 ceph 321 k 2024-05-17T22:23:23.045 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rbd x86_64 2:19.0.0-3794.g931809a9.el9 ceph 300 k 2024-05-17T22:23:23.045 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rgw x86_64 2:19.0.0-3794.g931809a9.el9 ceph 99 k 2024-05-17T22:23:23.045 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-fuse x86_64 2:19.0.0-3794.g931809a9.el9 ceph 86 k 2024-05-17T22:23:23.045 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-mirror x86_64 2:19.0.0-3794.g931809a9.el9 ceph 3.0 M 2024-05-17T22:23:23.045 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-nbd x86_64 2:19.0.0-3794.g931809a9.el9 ceph 169 k 2024-05-17T22:23:23.045 INFO:teuthology.orchestra.run.smithi151.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-17T22:23:23.045 INFO:teuthology.orchestra.run.smithi151.stdout:Upgrading: 2024-05-17T22:23:23.046 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-17T22:23:23.046 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-17T22:23:23.046 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-17T22:23:23.046 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-17T22:23:23.046 INFO:teuthology.orchestra.run.smithi151.stdout: librados2 x86_64 2:19.0.0-3794.g931809a9.el9 ceph 3.3 M 2024-05-17T22:23:23.046 INFO:teuthology.orchestra.run.smithi151.stdout: librbd1 x86_64 2:19.0.0-3794.g931809a9.el9 ceph 3.1 M 2024-05-17T22:23:23.046 INFO:teuthology.orchestra.run.smithi151.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-17T22:23:23.046 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-17T22:23:23.046 INFO:teuthology.orchestra.run.smithi151.stdout:Installing dependencies: 2024-05-17T22:23:23.046 INFO:teuthology.orchestra.run.smithi151.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-17T22:23:23.046 INFO:teuthology.orchestra.run.smithi151.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-17T22:23:23.046 INFO:teuthology.orchestra.run.smithi151.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-17T22:23:23.046 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-common x86_64 2:19.0.0-3794.g931809a9.el9 ceph 20 M 2024-05-17T22:23:23.046 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 34 k 2024-05-17T22:23:23.047 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mds x86_64 2:19.0.0-3794.g931809a9.el9 ceph 2.3 M 2024-05-17T22:23:23.047 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 274 k 2024-05-17T22:23:23.047 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mon x86_64 2:19.0.0-3794.g931809a9.el9 ceph 4.9 M 2024-05-17T22:23:23.047 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-osd x86_64 2:19.0.0-3794.g931809a9.el9 ceph 18 M 2024-05-17T22:23:23.047 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 16 k 2024-05-17T22:23:23.047 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-selinux x86_64 2:19.0.0-3794.g931809a9.el9 ceph 25 k 2024-05-17T22:23:23.047 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-17T22:23:23.047 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-17T22:23:23.047 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-17T22:23:23.047 INFO:teuthology.orchestra.run.smithi151.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-17T22:23:23.047 INFO:teuthology.orchestra.run.smithi151.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-17T22:23:23.047 INFO:teuthology.orchestra.run.smithi151.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-17T22:23:23.047 INFO:teuthology.orchestra.run.smithi151.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-17T22:23:23.047 INFO:teuthology.orchestra.run.smithi151.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-17T22:23:23.048 INFO:teuthology.orchestra.run.smithi151.stdout: libcephsqlite x86_64 2:19.0.0-3794.g931809a9.el9 ceph 158 k 2024-05-17T22:23:23.048 INFO:teuthology.orchestra.run.smithi151.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-17T22:23:23.048 INFO:teuthology.orchestra.run.smithi151.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-17T22:23:23.048 INFO:teuthology.orchestra.run.smithi151.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-17T22:23:23.048 INFO:teuthology.orchestra.run.smithi151.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-17T22:23:23.048 INFO:teuthology.orchestra.run.smithi151.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-17T22:23:23.048 INFO:teuthology.orchestra.run.smithi151.stdout: libradosstriper1 x86_64 2:19.0.0-3794.g931809a9.el9 ceph 501 k 2024-05-17T22:23:23.048 INFO:teuthology.orchestra.run.smithi151.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-17T22:23:23.048 INFO:teuthology.orchestra.run.smithi151.stdout: librgw2 x86_64 2:19.0.0-3794.g931809a9.el9 ceph 5.2 M 2024-05-17T22:23:23.048 INFO:teuthology.orchestra.run.smithi151.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-17T22:23:23.048 INFO:teuthology.orchestra.run.smithi151.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-17T22:23:23.048 INFO:teuthology.orchestra.run.smithi151.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-17T22:23:23.048 INFO:teuthology.orchestra.run.smithi151.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-17T22:23:23.048 INFO:teuthology.orchestra.run.smithi151.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-17T22:23:23.048 INFO:teuthology.orchestra.run.smithi151.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-17T22:23:23.049 INFO:teuthology.orchestra.run.smithi151.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-17T22:23:23.049 INFO:teuthology.orchestra.run.smithi151.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-17T22:23:23.049 INFO:teuthology.orchestra.run.smithi151.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-17T22:23:23.049 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-17T22:23:23.049 INFO:teuthology.orchestra.run.smithi151.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-17T22:23:23.049 INFO:teuthology.orchestra.run.smithi151.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-17T22:23:23.049 INFO:teuthology.orchestra.run.smithi151.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-17T22:23:23.049 INFO:teuthology.orchestra.run.smithi151.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-17T22:23:23.049 INFO:teuthology.orchestra.run.smithi151.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-17T22:23:23.049 INFO:teuthology.orchestra.run.smithi151.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-17T22:23:23.049 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-17T22:23:23.049 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-argparse x86_64 2:19.0.0-3794.g931809a9.el9 ceph 45 k 2024-05-17T22:23:23.049 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-common x86_64 2:19.0.0-3794.g931809a9.el9 ceph 136 k 2024-05-17T22:23:23.049 INFO:teuthology.orchestra.run.smithi151.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-17T22:23:23.050 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-17T22:23:23.050 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-17T22:23:23.050 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-17T22:23:23.050 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-17T22:23:23.050 INFO:teuthology.orchestra.run.smithi151.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-17T22:23:23.050 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-17T22:23:23.050 INFO:teuthology.orchestra.run.smithi151.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-17T22:23:23.050 INFO:teuthology.orchestra.run.smithi151.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-17T22:23:23.050 INFO:teuthology.orchestra.run.smithi151.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-17T22:23:23.050 INFO:teuthology.orchestra.run.smithi151.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-17T22:23:23.050 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-17T22:23:23.050 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-17T22:23:23.050 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-17T22:23:23.050 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-17T22:23:23.051 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-17T22:23:23.051 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-17T22:23:23.051 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-17T22:23:23.051 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-17T22:23:23.051 INFO:teuthology.orchestra.run.smithi151.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-17T22:23:23.051 INFO:teuthology.orchestra.run.smithi151.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-17T22:23:23.051 INFO:teuthology.orchestra.run.smithi151.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-17T22:23:23.051 INFO:teuthology.orchestra.run.smithi151.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-17T22:23:23.051 INFO:teuthology.orchestra.run.smithi151.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-17T22:23:23.051 INFO:teuthology.orchestra.run.smithi151.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-17T22:23:23.051 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-17T22:23:23.051 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-17T22:23:23.051 INFO:teuthology.orchestra.run.smithi151.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-17T22:23:23.051 INFO:teuthology.orchestra.run.smithi151.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-17T22:23:23.051 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-17T22:23:23.052 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-17T22:23:23.052 INFO:teuthology.orchestra.run.smithi151.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-17T22:23:23.052 INFO:teuthology.orchestra.run.smithi151.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-17T22:23:23.052 INFO:teuthology.orchestra.run.smithi151.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-17T22:23:23.052 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-17T22:23:23.052 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-17T22:23:23.052 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-17T22:23:23.052 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-17T22:23:23.052 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-17T22:23:23.052 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-17T22:23:23.052 INFO:teuthology.orchestra.run.smithi151.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-17T22:23:23.052 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-17T22:23:23.052 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-17T22:23:23.052 INFO:teuthology.orchestra.run.smithi151.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-17T22:23:23.053 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-17T22:23:23.053 INFO:teuthology.orchestra.run.smithi151.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-17T22:23:23.053 INFO:teuthology.orchestra.run.smithi151.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-17T22:23:23.053 INFO:teuthology.orchestra.run.smithi151.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-17T22:23:23.053 INFO:teuthology.orchestra.run.smithi151.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-17T22:23:23.053 INFO:teuthology.orchestra.run.smithi151.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-17T22:23:23.053 INFO:teuthology.orchestra.run.smithi151.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-17T22:23:23.053 INFO:teuthology.orchestra.run.smithi151.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-17T22:23:23.053 INFO:teuthology.orchestra.run.smithi151.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-17T22:23:23.053 INFO:teuthology.orchestra.run.smithi151.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-17T22:23:23.053 INFO:teuthology.orchestra.run.smithi151.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-17T22:23:23.053 INFO:teuthology.orchestra.run.smithi151.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-17T22:23:23.053 INFO:teuthology.orchestra.run.smithi151.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-17T22:23:23.053 INFO:teuthology.orchestra.run.smithi151.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-17T22:23:23.053 INFO:teuthology.orchestra.run.smithi151.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-17T22:23:23.054 INFO:teuthology.orchestra.run.smithi151.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-17T22:23:23.054 INFO:teuthology.orchestra.run.smithi151.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-17T22:23:23.054 INFO:teuthology.orchestra.run.smithi151.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-17T22:23:23.054 INFO:teuthology.orchestra.run.smithi151.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-17T22:23:23.054 INFO:teuthology.orchestra.run.smithi151.stdout:Installing weak dependencies: 2024-05-17T22:23:23.054 INFO:teuthology.orchestra.run.smithi151.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-17T22:23:23.054 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-17T22:23:23.054 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction Summary 2024-05-17T22:23:23.054 INFO:teuthology.orchestra.run.smithi151.stdout:======================================================================================= 2024-05-17T22:23:23.054 INFO:teuthology.orchestra.run.smithi151.stdout:Install 148 Packages 2024-05-17T22:23:23.054 INFO:teuthology.orchestra.run.smithi151.stdout:Upgrade 8 Packages 2024-05-17T22:23:23.054 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-17T22:23:23.055 INFO:teuthology.orchestra.run.smithi151.stdout:Total download size: 219 M 2024-05-17T22:23:23.055 INFO:teuthology.orchestra.run.smithi151.stdout:Downloading Packages: 2024-05-17T22:23:23.427 INFO:teuthology.orchestra.run.smithi096.stdout:(1/156): ceph-19.0.0-3794.g931809a9.el9.x86_64. 40 kB/s | 6.6 kB 00:00 2024-05-17T22:23:23.689 INFO:teuthology.orchestra.run.smithi151.stdout:(1/156): ceph-19.0.0-3794.g931809a9.el9.x86_64. 26 kB/s | 6.6 kB 00:00 2024-05-17T22:23:23.951 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:16 2024-05-17T22:23:24.102 INFO:teuthology.orchestra.run.smithi096.stdout:(2/156): ceph-fuse-19.0.0-3794.g931809a9.el9.x8 1.3 MB/s | 866 kB 00:00 2024-05-17T22:23:24.186 INFO:teuthology.orchestra.run.smithi096.stdout:(3/156): ceph-immutable-object-cache-19.0.0-379 1.7 MB/s | 147 kB 00:00 2024-05-17T22:23:24.703 INFO:teuthology.orchestra.run.smithi096.stdout:(4/156): ceph-mds-19.0.0-3794.g931809a9.el9.x86 4.4 MB/s | 2.3 MB 00:00 2024-05-17T22:23:24.947 INFO:teuthology.orchestra.run.smithi151.stdout:(2/156): ceph-fuse-19.0.0-3794.g931809a9.el9.x8 688 kB/s | 866 kB 00:01 2024-05-17T22:23:25.011 INFO:teuthology.orchestra.run.smithi096.stdout:(5/156): ceph-mgr-19.0.0-3794.g931809a9.el9.x86 4.9 MB/s | 1.5 MB 00:00 2024-05-17T22:23:25.047 INFO:teuthology.orchestra.run.smithi151.stdout:(3/156): ceph-base-19.0.0-3794.g931809a9.el9.x8 3.3 MB/s | 5.4 MB 00:01 2024-05-17T22:23:25.181 INFO:teuthology.orchestra.run.smithi151.stdout:(4/156): ceph-immutable-object-cache-19.0.0-379 628 kB/s | 147 kB 00:00 2024-05-17T22:23:25.465 INFO:teuthology.orchestra.run.smithi151.stdout:(5/156): ceph-mds-19.0.0-3794.g931809a9.el9.x86 5.4 MB/s | 2.3 MB 00:00 2024-05-17T22:23:26.145 INFO:teuthology.orchestra.run.smithi096.stdout:(6/156): ceph-mon-19.0.0-3794.g931809a9.el9.x86 4.3 MB/s | 4.9 MB 00:01 2024-05-17T22:23:26.298 INFO:teuthology.orchestra.run.smithi151.stdout:(6/156): ceph-mon-19.0.0-3794.g931809a9.el9.x86 5.9 MB/s | 4.9 MB 00:00 2024-05-17T22:23:26.398 INFO:teuthology.orchestra.run.smithi151.stdout:(7/156): ceph-mgr-19.0.0-3794.g931809a9.el9.x86 1.2 MB/s | 1.5 MB 00:01 2024-05-17T22:23:27.137 INFO:teuthology.orchestra.run.smithi096.stdout:(7/156): ceph-common-19.0.0-3794.g931809a9.el9. 5.1 MB/s | 20 MB 00:03 2024-05-17T22:23:27.545 INFO:teuthology.orchestra.run.smithi096.stdout:(8/156): ceph-base-19.0.0-3794.g931809a9.el9.x8 1.3 MB/s | 5.4 MB 00:04 2024-05-17T22:23:27.678 INFO:teuthology.orchestra.run.smithi096.stdout:(9/156): ceph-selinux-19.0.0-3794.g931809a9.el9 189 kB/s | 25 kB 00:00 2024-05-17T22:23:27.890 INFO:teuthology.orchestra.run.smithi151.stdout:(8/156): ceph-common-19.0.0-3794.g931809a9.el9. 4.4 MB/s | 20 MB 00:04 2024-05-17T22:23:28.132 INFO:teuthology.orchestra.run.smithi151.stdout:(9/156): ceph-selinux-19.0.0-3794.g931809a9.el9 104 kB/s | 25 kB 00:00 2024-05-17T22:23:28.379 INFO:teuthology.orchestra.run.smithi096.stdout:(10/156): ceph-radosgw-19.0.0-3794.g931809a9.el 8.2 MB/s | 10 MB 00:01 2024-05-17T22:23:28.546 INFO:teuthology.orchestra.run.smithi096.stdout:(11/156): cephfs-mirror-19.0.0-3794.g931809a9.e 1.3 MB/s | 227 kB 00:00 2024-05-17T22:23:28.638 INFO:teuthology.orchestra.run.smithi096.stdout:(12/156): cephfs-shell-19.0.0-3794.g931809a9.el 295 kB/s | 27 kB 00:00 2024-05-17T22:23:28.888 INFO:teuthology.orchestra.run.smithi096.stdout:(13/156): libcephfs-devel-19.0.0-3794.g931809a9 127 kB/s | 32 kB 00:00 2024-05-17T22:23:29.093 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - CRB 4.4 MB/s | 6.2 MB 00:01 2024-05-17T22:23:29.246 INFO:teuthology.orchestra.run.smithi096.stdout:(14/156): libcephfs2-19.0.0-3794.g931809a9.el9. 2.1 MB/s | 759 kB 00:00 2024-05-17T22:23:29.346 INFO:teuthology.orchestra.run.smithi096.stdout:(15/156): libcephsqlite-19.0.0-3794.g931809a9.e 1.5 MB/s | 158 kB 00:00 2024-05-17T22:23:29.513 INFO:teuthology.orchestra.run.smithi096.stdout:(16/156): librados-devel-19.0.0-3794.g931809a9. 761 kB/s | 127 kB 00:00 2024-05-17T22:23:29.722 INFO:teuthology.orchestra.run.smithi096.stdout:(17/156): libradosstriper1-19.0.0-3794.g931809a 2.3 MB/s | 501 kB 00:00 2024-05-17T22:23:30.249 INFO:teuthology.orchestra.run.smithi151.stdout:(10/156): ceph-osd-19.0.0-3794.g931809a9.el9.x8 4.5 MB/s | 18 MB 00:03 2024-05-17T22:23:30.416 INFO:teuthology.orchestra.run.smithi151.stdout:(11/156): cephfs-mirror-19.0.0-3794.g931809a9.e 1.3 MB/s | 227 kB 00:00 2024-05-17T22:23:30.456 INFO:teuthology.orchestra.run.smithi096.stdout:(18/156): ceph-osd-19.0.0-3794.g931809a9.el9.x8 4.1 MB/s | 18 MB 00:04 2024-05-17T22:23:30.531 INFO:teuthology.orchestra.run.smithi096.stdout:(19/156): python3-ceph-argparse-19.0.0-3794.g93 605 kB/s | 45 kB 00:00 2024-05-17T22:23:30.532 INFO:teuthology.orchestra.run.smithi151.stdout:(12/156): ceph-radosgw-19.0.0-3794.g931809a9.el 2.5 MB/s | 10 MB 00:04 2024-05-17T22:23:30.558 INFO:teuthology.orchestra.run.smithi151.stdout:(13/156): cephfs-shell-19.0.0-3794.g931809a9.el 190 kB/s | 27 kB 00:00 2024-05-17T22:23:30.614 INFO:teuthology.orchestra.run.smithi096.stdout:(20/156): python3-ceph-common-19.0.0-3794.g9318 1.6 MB/s | 136 kB 00:00 2024-05-17T22:23:30.633 INFO:teuthology.orchestra.run.smithi151.stdout:(14/156): libcephfs-devel-19.0.0-3794.g931809a9 316 kB/s | 32 kB 00:00 2024-05-17T22:23:30.706 INFO:teuthology.orchestra.run.smithi096.stdout:(21/156): python3-cephfs-19.0.0-3794.g931809a9. 1.7 MB/s | 162 kB 00:00 2024-05-17T22:23:30.742 INFO:teuthology.orchestra.run.smithi151.stdout:(15/156): libcephsqlite-19.0.0-3794.g931809a9.e 1.4 MB/s | 158 kB 00:00 2024-05-17T22:23:30.815 INFO:teuthology.orchestra.run.smithi096.stdout:(22/156): python3-rados-19.0.0-3794.g931809a9.e 2.9 MB/s | 321 kB 00:00 2024-05-17T22:23:30.867 INFO:teuthology.orchestra.run.smithi151.stdout:(16/156): librados-devel-19.0.0-3794.g931809a9. 1.0 MB/s | 127 kB 00:00 2024-05-17T22:23:30.894 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - Extras packages 29 kB/s | 16 kB 00:00 2024-05-17T22:23:30.907 INFO:teuthology.orchestra.run.smithi096.stdout:(23/156): python3-rbd-19.0.0-3794.g931809a9.el9 3.2 MB/s | 300 kB 00:00 2024-05-17T22:23:30.984 INFO:teuthology.orchestra.run.smithi151.stdout:(17/156): libcephfs2-19.0.0-3794.g931809a9.el9. 1.7 MB/s | 759 kB 00:00 2024-05-17T22:23:31.016 INFO:teuthology.orchestra.run.smithi096.stdout:(24/156): librgw2-19.0.0-3794.g931809a9.el9.x86 4.0 MB/s | 5.2 MB 00:01 2024-05-17T22:23:31.041 INFO:teuthology.orchestra.run.smithi096.stdout:(25/156): python3-rgw-19.0.0-3794.g931809a9.el9 743 kB/s | 99 kB 00:00 2024-05-17T22:23:31.051 INFO:teuthology.orchestra.run.smithi151.stdout:(18/156): libradosstriper1-19.0.0-3794.g931809a 2.7 MB/s | 501 kB 00:00 2024-05-17T22:23:31.091 INFO:teuthology.orchestra.run.smithi096.stdout:(26/156): rbd-fuse-19.0.0-3794.g931809a9.el9.x8 1.1 MB/s | 86 kB 00:00 2024-05-17T22:23:31.160 INFO:teuthology.orchestra.run.smithi151.stdout:(19/156): python3-ceph-argparse-19.0.0-3794.g93 416 kB/s | 45 kB 00:00 2024-05-17T22:23:31.183 INFO:teuthology.orchestra.run.smithi096.stdout:(27/156): rbd-nbd-19.0.0-3794.g931809a9.el9.x86 1.8 MB/s | 169 kB 00:00 2024-05-17T22:23:31.260 INFO:teuthology.orchestra.run.smithi151.stdout:(20/156): python3-ceph-common-19.0.0-3794.g9318 1.3 MB/s | 136 kB 00:00 2024-05-17T22:23:31.267 INFO:teuthology.orchestra.run.smithi096.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3794.g 411 kB/s | 34 kB 00:00 2024-05-17T22:23:31.359 INFO:teuthology.orchestra.run.smithi096.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3794.g931809a 1.5 MB/s | 143 kB 00:00 2024-05-17T22:23:31.360 INFO:teuthology.orchestra.run.smithi151.stdout:(21/156): python3-cephfs-19.0.0-3794.g931809a9. 1.6 MB/s | 162 kB 00:00 2024-05-17T22:23:31.569 INFO:teuthology.orchestra.run.smithi151.stdout:(22/156): python3-rados-19.0.0-3794.g931809a9.e 1.5 MB/s | 321 kB 00:00 2024-05-17T22:23:31.667 INFO:teuthology.orchestra.run.smithi096.stdout:(30/156): rbd-mirror-19.0.0-3794.g931809a9.el9. 4.9 MB/s | 3.0 MB 00:00 2024-05-17T22:23:31.703 INFO:teuthology.orchestra.run.smithi151.stdout:(23/156): python3-rbd-19.0.0-3794.g931809a9.el9 2.2 MB/s | 300 kB 00:00 2024-05-17T22:23:31.828 INFO:teuthology.orchestra.run.smithi151.stdout:(24/156): python3-rgw-19.0.0-3794.g931809a9.el9 795 kB/s | 99 kB 00:00 2024-05-17T22:23:31.937 INFO:teuthology.orchestra.run.smithi151.stdout:(25/156): rbd-fuse-19.0.0-3794.g931809a9.el9.x8 791 kB/s | 86 kB 00:00 2024-05-17T22:23:32.645 INFO:teuthology.orchestra.run.smithi151.stdout:(26/156): librgw2-19.0.0-3794.g931809a9.el9.x86 3.1 MB/s | 5.2 MB 00:01 2024-05-17T22:23:32.704 INFO:teuthology.orchestra.run.smithi151.stdout:(27/156): rbd-nbd-19.0.0-3794.g931809a9.el9.x86 2.8 MB/s | 169 kB 00:00 2024-05-17T22:23:32.754 INFO:teuthology.orchestra.run.smithi151.stdout:(28/156): rbd-mirror-19.0.0-3794.g931809a9.el9. 3.7 MB/s | 3.0 MB 00:00 2024-05-17T22:23:32.779 INFO:teuthology.orchestra.run.smithi151.stdout:(29/156): ceph-grafana-dashboards-19.0.0-3794.g 455 kB/s | 34 kB 00:00 2024-05-17T22:23:32.793 INFO:teuthology.orchestra.run.smithi096.stdout:(31/156): ceph-test-19.0.0-3794.g931809a9.el9.x 8.5 MB/s | 43 MB 00:05 2024-05-17T22:23:32.821 INFO:teuthology.orchestra.run.smithi151.stdout:(30/156): ceph-mgr-cephadm-19.0.0-3794.g931809a 2.1 MB/s | 143 kB 00:00 2024-05-17T22:23:32.859 INFO:teuthology.orchestra.run.smithi096.stdout:(32/156): ceph-mgr-dashboard-19.0.0-3794.g93180 2.4 MB/s | 3.5 MB 00:01 2024-05-17T22:23:32.910 INFO:teuthology.orchestra.run.smithi096.stdout:(33/156): ceph-mgr-modules-core-19.0.0-3794.g93 2.3 MB/s | 274 kB 00:00 2024-05-17T22:23:32.935 INFO:teuthology.orchestra.run.smithi096.stdout:(34/156): ceph-mgr-rook-19.0.0-3794.g931809a9.e 656 kB/s | 49 kB 00:00 2024-05-17T22:23:32.994 INFO:teuthology.orchestra.run.smithi096.stdout:(35/156): ceph-prometheus-alerts-19.0.0-3794.g9 197 kB/s | 16 kB 00:00 2024-05-17T22:23:33.119 INFO:teuthology.orchestra.run.smithi096.stdout:(36/156): ceph-mgr-diskprediction-local-19.0.0- 5.1 MB/s | 7.4 MB 00:01 2024-05-17T22:23:33.144 INFO:teuthology.orchestra.run.smithi096.stdout:(37/156): ceph-volume-19.0.0-3794.g931809a9.el9 1.3 MB/s | 273 kB 00:00 2024-05-17T22:23:33.178 INFO:teuthology.orchestra.run.smithi096.stdout:(38/156): cephadm-19.0.0-3794.g931809a9.el9.noa 4.1 MB/s | 763 kB 00:00 2024-05-17T22:23:33.195 INFO:teuthology.orchestra.run.smithi096.stdout:(39/156): cephfs-top-19.0.0-3794.g931809a9.el9. 242 kB/s | 18 kB 00:00 2024-05-17T22:23:33.270 INFO:teuthology.orchestra.run.smithi096.stdout:(40/156): fuse-2.9.9-15.el9.x86_64.rpm 640 kB/s | 80 kB 00:00 2024-05-17T22:23:33.337 INFO:teuthology.orchestra.run.smithi096.stdout:(41/156): libgfortran-11.4.1-3.el9.x86_64.rpm 4.9 MB/s | 803 kB 00:00 2024-05-17T22:23:33.362 INFO:teuthology.orchestra.run.smithi096.stdout:(42/156): libquadmath-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 195 kB 00:00 2024-05-17T22:23:33.380 INFO:teuthology.orchestra.run.smithi151.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3794.g93180 5.9 MB/s | 3.5 MB 00:00 2024-05-17T22:23:33.387 INFO:teuthology.orchestra.run.smithi096.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-05-17T22:23:33.413 INFO:teuthology.orchestra.run.smithi096.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-05-17T22:23:33.438 INFO:teuthology.orchestra.run.smithi096.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-05-17T22:23:33.455 INFO:teuthology.orchestra.run.smithi151.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3794.g93 3.6 MB/s | 274 kB 00:00 2024-05-17T22:23:33.480 INFO:teuthology.orchestra.run.smithi096.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-05-17T22:23:33.505 INFO:teuthology.orchestra.run.smithi096.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 3.1 MB/s | 215 kB 00:00 2024-05-17T22:23:33.514 INFO:teuthology.orchestra.run.smithi151.stdout:(33/156): ceph-mgr-rook-19.0.0-3794.g931809a9.e 845 kB/s | 49 kB 00:00 2024-05-17T22:23:33.539 INFO:teuthology.orchestra.run.smithi096.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 6.1 MB/s | 1.2 MB 00:00 2024-05-17T22:23:33.564 INFO:teuthology.orchestra.run.smithi151.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3794.g9 329 kB/s | 16 kB 00:00 2024-05-17T22:23:33.572 INFO:teuthology.orchestra.run.smithi096.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 7.3 MB/s | 689 kB 00:00 2024-05-17T22:23:33.598 INFO:teuthology.orchestra.run.smithi096.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-05-17T22:23:33.623 INFO:teuthology.orchestra.run.smithi096.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 429 kB/s | 21 kB 00:00 2024-05-17T22:23:33.648 INFO:teuthology.orchestra.run.smithi096.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-05-17T22:23:33.682 INFO:teuthology.orchestra.run.smithi096.stdout:(53/156): bison-3.7.4-5.el9.x86_64.rpm 5.2 MB/s | 943 kB 00:00 2024-05-17T22:23:33.689 INFO:teuthology.orchestra.run.smithi151.stdout:(35/156): ceph-volume-19.0.0-3794.g931809a9.el9 2.1 MB/s | 273 kB 00:00 2024-05-17T22:23:33.699 INFO:teuthology.orchestra.run.smithi096.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 434 kB/s | 33 kB 00:00 2024-05-17T22:23:33.716 INFO:teuthology.orchestra.run.smithi096.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 541 kB/s | 18 kB 00:00 2024-05-17T22:23:33.733 INFO:teuthology.orchestra.run.smithi096.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 332 kB/s | 11 kB 00:00 2024-05-17T22:23:33.748 INFO:teuthology.orchestra.run.smithi012.stdout:Extra Packages for Enterprise Linux 7.8 MB/s | 21 MB 00:02 2024-05-17T22:23:33.758 INFO:teuthology.orchestra.run.smithi096.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 160 kB 00:00 2024-05-17T22:23:33.783 INFO:teuthology.orchestra.run.smithi096.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 904 kB/s | 45 kB 00:00 2024-05-17T22:23:33.817 INFO:teuthology.orchestra.run.smithi096.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-05-17T22:23:33.850 INFO:teuthology.orchestra.run.smithi096.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 582 kB 00:00 2024-05-17T22:23:33.876 INFO:teuthology.orchestra.run.smithi096.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 434 kB/s | 25 kB 00:00 2024-05-17T22:23:33.905 INFO:teuthology.orchestra.run.smithi096.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 875 kB/s | 47 kB 00:00 2024-05-17T22:23:33.923 INFO:teuthology.orchestra.run.smithi151.stdout:(36/156): cephadm-19.0.0-3794.g931809a9.el9.noa 3.2 MB/s | 763 kB 00:00 2024-05-17T22:23:33.935 INFO:teuthology.orchestra.run.smithi096.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-05-17T22:23:33.960 INFO:teuthology.orchestra.run.smithi096.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.4 MB/s | 188 kB 00:00 2024-05-17T22:23:33.986 INFO:teuthology.orchestra.run.smithi096.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 703 kB/s | 35 kB 00:00 2024-05-17T22:23:33.998 INFO:teuthology.orchestra.run.smithi151.stdout:(37/156): cephfs-top-19.0.0-3794.g931809a9.el9. 243 kB/s | 18 kB 00:00 2024-05-17T22:23:34.036 INFO:teuthology.orchestra.run.smithi096.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.7 MB/s | 3.0 MB 00:00 2024-05-17T22:23:34.107 INFO:teuthology.orchestra.run.smithi151.stdout:(38/156): fuse-2.9.9-15.el9.x86_64.rpm 742 kB/s | 80 kB 00:00 2024-05-17T22:23:34.153 INFO:teuthology.orchestra.run.smithi096.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-05-17T22:23:34.199 INFO:teuthology.orchestra.run.smithi151.stdout:(39/156): ceph-mgr-diskprediction-local-19.0.0- 5.4 MB/s | 7.4 MB 00:01 2024-05-17T22:23:34.211 INFO:teuthology.orchestra.run.smithi096.stdout:(68/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-05-17T22:23:34.270 INFO:teuthology.orchestra.run.smithi096.stdout:(69/156): openblas-openmp-0.3.21-2.el9.x86_64.r 15 MB/s | 4.7 MB 00:00 2024-05-17T22:23:34.282 INFO:teuthology.orchestra.run.smithi151.stdout:(40/156): libgfortran-11.4.1-3.el9.x86_64.rpm 4.5 MB/s | 803 kB 00:00 2024-05-17T22:23:34.295 INFO:teuthology.orchestra.run.smithi096.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 1.6 MB/s | 133 kB 00:00 2024-05-17T22:23:34.333 INFO:teuthology.orchestra.run.smithi151.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-05-17T22:23:34.337 INFO:teuthology.orchestra.run.smithi096.stdout:(71/156): postgresql-contrib-13.14-1.el9.x86_64 4.8 MB/s | 911 kB 00:00 2024-05-17T22:23:34.366 INFO:teuthology.orchestra.run.smithi151.stdout:(42/156): libquadmath-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 195 kB 00:00 2024-05-17T22:23:34.371 INFO:teuthology.orchestra.run.smithi096.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-05-17T22:23:34.396 INFO:teuthology.orchestra.run.smithi096.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-05-17T22:23:34.409 INFO:teuthology.orchestra.run.smithi151.stdout:(43/156): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-05-17T22:23:34.434 INFO:teuthology.orchestra.run.smithi151.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-05-17T22:23:34.446 INFO:teuthology.orchestra.run.smithi096.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-05-17T22:23:34.459 INFO:teuthology.orchestra.run.smithi151.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-05-17T22:23:34.493 INFO:teuthology.orchestra.run.smithi151.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-05-17T22:23:34.518 INFO:teuthology.orchestra.run.smithi151.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-05-17T22:23:34.522 INFO:teuthology.orchestra.run.smithi096.stdout:(75/156): postgresql-server-13.14-1.el9.x86_64. 23 MB/s | 5.8 MB 00:00 2024-05-17T22:23:34.547 INFO:teuthology.orchestra.run.smithi096.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-05-17T22:23:34.564 INFO:teuthology.orchestra.run.smithi096.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 612 kB/s | 26 kB 00:00 2024-05-17T22:23:34.581 INFO:teuthology.orchestra.run.smithi096.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 570 kB/s | 19 kB 00:00 2024-05-17T22:23:34.606 INFO:teuthology.orchestra.run.smithi096.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-05-17T22:23:34.623 INFO:teuthology.orchestra.run.smithi096.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 832 kB/s | 35 kB 00:00 2024-05-17T22:23:34.669 INFO:teuthology.orchestra.run.smithi151.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 3.8 MB/s | 689 kB 00:00 2024-05-17T22:23:34.706 INFO:teuthology.orchestra.run.smithi096.stdout:(81/156): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 6.0 MB 00:00 2024-05-17T22:23:34.718 INFO:teuthology.orchestra.run.smithi151.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 4.6 MB/s | 943 kB 00:00 2024-05-17T22:23:34.732 INFO:teuthology.orchestra.run.smithi096.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-05-17T22:23:34.744 INFO:teuthology.orchestra.run.smithi151.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 107 kB 00:00 2024-05-17T22:23:34.757 INFO:teuthology.orchestra.run.smithi096.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-05-17T22:23:34.769 INFO:teuthology.orchestra.run.smithi151.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 427 kB/s | 21 kB 00:00 2024-05-17T22:23:34.782 INFO:teuthology.orchestra.run.smithi096.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-17T22:23:34.794 INFO:teuthology.orchestra.run.smithi151.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-05-17T22:23:34.808 INFO:teuthology.orchestra.run.smithi096.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 837 kB/s | 42 kB 00:00 2024-05-17T22:23:34.820 INFO:teuthology.orchestra.run.smithi151.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 652 kB/s | 33 kB 00:00 2024-05-17T22:23:34.849 INFO:teuthology.orchestra.run.smithi096.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 3.9 MB/s | 269 kB 00:00 2024-05-17T22:23:34.862 INFO:teuthology.orchestra.run.smithi151.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 437 kB/s | 18 kB 00:00 2024-05-17T22:23:34.875 INFO:teuthology.orchestra.run.smithi096.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-05-17T22:23:34.900 INFO:teuthology.orchestra.run.smithi096.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-05-17T22:23:34.920 INFO:teuthology.orchestra.run.smithi151.stdout:(55/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-05-17T22:23:34.946 INFO:teuthology.orchestra.run.smithi151.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 133 kB/s | 11 kB 00:00 2024-05-17T22:23:34.971 INFO:teuthology.orchestra.run.smithi151.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-05-17T22:23:34.975 INFO:teuthology.orchestra.run.smithi096.stdout:(89/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 14 MB/s | 5.1 MB 00:00 2024-05-17T22:23:34.996 INFO:teuthology.orchestra.run.smithi151.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 902 kB/s | 45 kB 00:00 2024-05-17T22:23:35.001 INFO:teuthology.orchestra.run.smithi096.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 405 kB/s | 51 kB 00:00 2024-05-17T22:23:35.026 INFO:teuthology.orchestra.run.smithi096.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 427 kB/s | 54 kB 00:00 2024-05-17T22:23:35.038 INFO:teuthology.orchestra.run.smithi151.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-05-17T22:23:35.043 INFO:teuthology.orchestra.run.smithi096.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 993 kB/s | 42 kB 00:00 2024-05-17T22:23:35.068 INFO:teuthology.orchestra.run.smithi096.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-05-17T22:23:35.080 INFO:teuthology.orchestra.run.smithi151.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 6.8 MB/s | 582 kB 00:00 2024-05-17T22:23:35.102 INFO:teuthology.orchestra.run.smithi096.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-05-17T22:23:35.105 INFO:teuthology.orchestra.run.smithi151.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 380 kB/s | 25 kB 00:00 2024-05-17T22:23:35.127 INFO:teuthology.orchestra.run.smithi096.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 612 kB/s | 36 kB 00:00 2024-05-17T22:23:35.132 INFO:teuthology.orchestra.run.smithi151.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-05-17T22:23:35.152 INFO:teuthology.orchestra.run.smithi096.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-05-17T22:23:35.156 INFO:teuthology.orchestra.run.smithi151.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-05-17T22:23:35.177 INFO:teuthology.orchestra.run.smithi096.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-05-17T22:23:35.190 INFO:teuthology.orchestra.run.smithi151.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-05-17T22:23:35.203 INFO:teuthology.orchestra.run.smithi096.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-05-17T22:23:35.215 INFO:teuthology.orchestra.run.smithi151.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 602 kB/s | 35 kB 00:00 2024-05-17T22:23:35.228 INFO:teuthology.orchestra.run.smithi096.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-05-17T22:23:35.278 INFO:teuthology.orchestra.run.smithi096.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 844 kB/s | 64 kB 00:00 2024-05-17T22:23:35.304 INFO:teuthology.orchestra.run.smithi096.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 595 kB/s | 45 kB 00:00 2024-05-17T22:23:35.399 INFO:teuthology.orchestra.run.smithi151.stdout:(66/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 24 MB/s | 4.4 MB 00:00 2024-05-17T22:23:35.466 INFO:teuthology.orchestra.run.smithi151.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 24 MB/s | 1.6 MB 00:00 2024-05-17T22:23:35.533 INFO:teuthology.orchestra.run.smithi151.stdout:(68/156): postgresql-contrib-13.14-1.el9.x86_64 13 MB/s | 911 kB 00:00 2024-05-17T22:23:35.571 INFO:teuthology.orchestra.run.smithi096.stdout:(102/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-05-17T22:23:35.596 INFO:teuthology.orchestra.run.smithi096.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 70 kB/s | 22 kB 00:00 2024-05-17T22:23:35.600 INFO:teuthology.orchestra.run.smithi151.stdout:(69/156): openblas-openmp-0.3.21-2.el9.x86_64.r 12 MB/s | 4.7 MB 00:00 2024-05-17T22:23:35.625 INFO:teuthology.orchestra.run.smithi151.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-05-17T22:23:35.692 INFO:teuthology.orchestra.run.smithi151.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-05-17T22:23:35.767 INFO:teuthology.orchestra.run.smithi151.stdout:(72/156): postgresql-server-13.14-1.el9.x86_64. 35 MB/s | 5.8 MB 00:00 2024-05-17T22:23:35.793 INFO:teuthology.orchestra.run.smithi151.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 873 kB/s | 87 kB 00:00 2024-05-17T22:23:35.851 INFO:teuthology.orchestra.run.smithi151.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 4.1 MB/s | 244 kB 00:00 2024-05-17T22:23:35.938 INFO:teuthology.orchestra.run.smithi096.stdout:(104/156): gperftools-libs-2.9.1-2.el9.x86_64.r 904 kB/s | 309 kB 00:00 2024-05-17T22:23:35.952 INFO:teuthology.orchestra.run.smithi151.stdout:(75/156): python3-babel-2.9.1-2.el9.noarch.rpm 32 MB/s | 6.0 MB 00:00 2024-05-17T22:23:35.977 INFO:teuthology.orchestra.run.smithi151.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 1.9 MB/s | 248 kB 00:00 2024-05-17T22:23:35.980 INFO:teuthology.orchestra.run.smithi096.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 467 kB/s | 19 kB 00:00 2024-05-17T22:23:36.002 INFO:teuthology.orchestra.run.smithi151.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 510 kB/s | 26 kB 00:00 2024-05-17T22:23:36.028 INFO:teuthology.orchestra.run.smithi151.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 380 kB/s | 19 kB 00:00 2024-05-17T22:23:36.053 INFO:teuthology.orchestra.run.smithi151.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-05-17T22:23:36.078 INFO:teuthology.orchestra.run.smithi151.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 690 kB/s | 35 kB 00:00 2024-05-17T22:23:36.197 INFO:teuthology.orchestra.run.smithi096.stdout:(106/156): protobuf-compiler-3.14.0-13.el9.x86_ 967 kB/s | 863 kB 00:00 2024-05-17T22:23:36.289 INFO:teuthology.orchestra.run.smithi096.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 283 kB/s | 26 kB 00:00 2024-05-17T22:23:36.322 INFO:teuthology.orchestra.run.smithi096.stdout:(108/156): abseil-cpp-20211102.0-3.el9.x86_64.r 730 kB/s | 548 kB 00:00 2024-05-17T22:23:36.364 INFO:teuthology.orchestra.run.smithi096.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 646 kB/s | 49 kB 00:00 2024-05-17T22:23:36.390 INFO:teuthology.orchestra.run.smithi096.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-05-17T22:23:36.440 INFO:teuthology.orchestra.run.smithi096.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-05-17T22:23:36.537 INFO:teuthology.orchestra.run.smithi151.stdout:(81/156): ceph-test-19.0.0-3794.g931809a9.el9.x 5.1 MB/s | 43 MB 00:08 2024-05-17T22:23:36.562 INFO:teuthology.orchestra.run.smithi151.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 518 kB/s | 250 kB 00:00 2024-05-17T22:23:36.565 INFO:teuthology.orchestra.run.smithi096.stdout:(112/156): python3-asyncssh-2.13.2-5.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-05-17T22:23:36.604 INFO:teuthology.orchestra.run.smithi151.stdout:(83/156): python3-packaging-20.9-5.el9.noarch.r 1.8 MB/s | 77 kB 00:00 2024-05-17T22:23:36.607 INFO:teuthology.orchestra.run.smithi096.stdout:(113/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-05-17T22:23:36.649 INFO:teuthology.orchestra.run.smithi096.stdout:(114/156): python3-cachetools-4.2.4-1.el9.noarc 774 kB/s | 32 kB 00:00 2024-05-17T22:23:36.696 INFO:teuthology.orchestra.run.smithi151.stdout:(84/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.9 MB/s | 5.1 MB 00:00 2024-05-17T22:23:36.713 INFO:teuthology.orchestra.run.smithi151.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 389 kB/s | 42 kB 00:00 2024-05-17T22:23:36.738 INFO:teuthology.orchestra.run.smithi151.stdout:(86/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.1 MB/s | 222 kB 00:00 2024-05-17T22:23:36.749 INFO:teuthology.orchestra.run.smithi096.stdout:(115/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.3 MB/s | 839 kB 00:00 2024-05-17T22:23:36.763 INFO:teuthology.orchestra.run.smithi151.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-05-17T22:23:36.797 INFO:teuthology.orchestra.run.smithi151.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-05-17T22:23:36.822 INFO:teuthology.orchestra.run.smithi151.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 870 kB/s | 51 kB 00:00 2024-05-17T22:23:36.847 INFO:teuthology.orchestra.run.smithi151.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-05-17T22:23:36.849 INFO:teuthology.orchestra.run.smithi096.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 75 kB/s | 15 kB 00:00 2024-05-17T22:23:36.873 INFO:teuthology.orchestra.run.smithi151.stdout:(91/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.0 MB/s | 279 kB 00:00 2024-05-17T22:23:36.899 INFO:teuthology.orchestra.run.smithi151.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 828 kB/s | 42 kB 00:00 2024-05-17T22:23:36.908 INFO:teuthology.orchestra.run.smithi096.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.1 MB/s | 172 kB 00:00 2024-05-17T22:23:36.923 INFO:teuthology.orchestra.run.smithi151.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 878 kB/s | 44 kB 00:00 2024-05-17T22:23:36.957 INFO:teuthology.orchestra.run.smithi151.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-05-17T22:23:36.958 INFO:teuthology.orchestra.run.smithi096.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-05-17T22:23:36.983 INFO:teuthology.orchestra.run.smithi151.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 612 kB/s | 36 kB 00:00 2024-05-17T22:23:37.008 INFO:teuthology.orchestra.run.smithi151.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-05-17T22:23:37.026 INFO:teuthology.orchestra.run.smithi096.stdout:(119/156): python3-colorama-0.4.4-12.el9.noarch 527 kB/s | 35 kB 00:00 2024-05-17T22:23:37.033 INFO:teuthology.orchestra.run.smithi151.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-05-17T22:23:37.058 INFO:teuthology.orchestra.run.smithi151.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-05-17T22:23:37.084 INFO:teuthology.orchestra.run.smithi151.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-05-17T22:23:37.092 INFO:teuthology.orchestra.run.smithi096.stdout:(120/156): python3-google-auth-2.29.0-1.el9.noa 3.0 MB/s | 206 kB 00:00 2024-05-17T22:23:37.109 INFO:teuthology.orchestra.run.smithi151.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-17T22:23:37.134 INFO:teuthology.orchestra.run.smithi151.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 895 kB/s | 45 kB 00:00 2024-05-17T22:23:37.162 INFO:teuthology.orchestra.run.smithi151.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 425 kB/s | 22 kB 00:00 2024-05-17T22:23:37.176 INFO:teuthology.orchestra.run.smithi096.stdout:(121/156): libarrow-9.0.0-11.el9.x86_64.rpm 3.7 MB/s | 4.4 MB 00:01 2024-05-17T22:23:37.195 INFO:teuthology.orchestra.run.smithi151.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 14 MB/s | 863 kB 00:00 2024-05-17T22:23:37.202 INFO:teuthology.orchestra.run.smithi096.stdout:(122/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.0 MB/s | 303 kB 00:00 2024-05-17T22:23:37.252 INFO:teuthology.orchestra.run.smithi096.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 214 kB/s | 11 kB 00:00 2024-05-17T22:23:37.294 INFO:teuthology.orchestra.run.smithi096.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 423 kB/s | 18 kB 00:00 2024-05-17T22:23:37.327 INFO:teuthology.orchestra.run.smithi096.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 697 kB/s | 23 kB 00:00 2024-05-17T22:23:37.386 INFO:teuthology.orchestra.run.smithi096.stdout:(126/156): python3-grpcio-tools-1.46.7-10.el9.x 691 kB/s | 144 kB 00:00 2024-05-17T22:23:37.411 INFO:teuthology.orchestra.run.smithi096.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 233 kB/s | 19 kB 00:00 2024-05-17T22:23:37.437 INFO:teuthology.orchestra.run.smithi096.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-05-17T22:23:37.478 INFO:teuthology.orchestra.run.smithi096.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.1 MB/s | 46 kB 00:00 2024-05-17T22:23:37.512 INFO:teuthology.orchestra.run.smithi096.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-05-17T22:23:37.520 INFO:teuthology.orchestra.run.smithi151.stdout:(104/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-05-17T22:23:37.554 INFO:teuthology.orchestra.run.smithi096.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 1.4 MB/s | 58 kB 00:00 2024-05-17T22:23:37.638 INFO:teuthology.orchestra.run.smithi096.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-05-17T22:23:37.679 INFO:teuthology.orchestra.run.smithi096.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 396 kB/s | 16 kB 00:00 2024-05-17T22:23:37.722 INFO:teuthology.orchestra.run.smithi096.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.1 MB/s | 90 kB 00:00 2024-05-17T22:23:37.755 INFO:teuthology.orchestra.run.smithi096.stdout:(135/156): python3-kubernetes-26.1.0-2.el9.noar 3.0 MB/s | 1.0 MB 00:00 2024-05-17T22:23:37.780 INFO:teuthology.orchestra.run.smithi096.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 484 kB/s | 28 kB 00:00 2024-05-17T22:23:37.806 INFO:teuthology.orchestra.run.smithi096.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-05-17T22:23:37.864 INFO:teuthology.orchestra.run.smithi096.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-05-17T22:23:37.890 INFO:teuthology.orchestra.run.smithi096.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-05-17T22:23:37.931 INFO:teuthology.orchestra.run.smithi096.stdout:(140/156): python3-grpcio-1.46.7-10.el9.x86_64. 2.4 MB/s | 2.0 MB 00:00 2024-05-17T22:23:37.937 INFO:teuthology.orchestra.run.smithi151.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 47 kB/s | 19 kB 00:00 2024-05-17T22:23:37.957 INFO:teuthology.orchestra.run.smithi096.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 388 kB/s | 36 kB 00:00 2024-05-17T22:23:37.982 INFO:teuthology.orchestra.run.smithi096.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 553 kB/s | 51 kB 00:00 2024-05-17T22:23:38.021 INFO:teuthology.orchestra.run.smithi151.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 374 kB/s | 309 kB 00:00 2024-05-17T22:23:38.057 INFO:teuthology.orchestra.run.smithi096.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 896 kB/s | 90 kB 00:00 2024-05-17T22:23:38.080 INFO:teuthology.orchestra.run.smithi151.stdout:(107/156): abseil-cpp-20211102.0-3.el9.x86_64.r 597 kB/s | 548 kB 00:00 2024-05-17T22:23:38.083 INFO:teuthology.orchestra.run.smithi096.stdout:(144/156): python3-webob-1.8.7-6.el9.noarch.rpm 1.5 MB/s | 230 kB 00:00 2024-05-17T22:23:38.108 INFO:teuthology.orchestra.run.smithi096.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-05-17T22:23:38.138 INFO:teuthology.orchestra.run.smithi151.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 222 kB/s | 26 kB 00:00 2024-05-17T22:23:38.142 INFO:teuthology.orchestra.run.smithi096.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.6 MB/s | 427 kB 00:00 2024-05-17T22:23:38.167 INFO:teuthology.orchestra.run.smithi096.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-05-17T22:23:38.197 INFO:teuthology.orchestra.run.smithi151.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 416 kB/s | 49 kB 00:00 2024-05-17T22:23:38.264 INFO:teuthology.orchestra.run.smithi151.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 538 kB/s | 67 kB 00:00 2024-05-17T22:23:38.276 INFO:teuthology.orchestra.run.smithi096.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-05-17T22:23:38.339 INFO:teuthology.orchestra.run.smithi151.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-05-17T22:23:38.484 INFO:teuthology.orchestra.run.smithi096.stdout:(149/156): NetworkManager-1.47.90-1.el9.x86_64. 11 MB/s | 2.3 MB 00:00 2024-05-17T22:23:38.522 INFO:teuthology.orchestra.run.smithi151.stdout:(112/156): python3-asyncssh-2.13.2-5.el9.noarch 2.9 MB/s | 548 kB 00:00 2024-05-17T22:23:38.559 INFO:teuthology.orchestra.run.smithi096.stdout:(150/156): NetworkManager-libnm-1.47.90-1.el9.x 24 MB/s | 1.8 MB 00:00 2024-05-17T22:23:38.564 INFO:teuthology.orchestra.run.smithi151.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.7 MB/s | 839 kB 00:00 2024-05-17T22:23:38.601 INFO:teuthology.orchestra.run.smithi096.stdout:(151/156): NetworkManager-team-1.47.90-1.el9.x8 995 kB/s | 41 kB 00:00 2024-05-17T22:23:38.643 INFO:teuthology.orchestra.run.smithi096.stdout:(152/156): NetworkManager-tui-1.47.90-1.el9.x86 5.9 MB/s | 248 kB 00:00 2024-05-17T22:23:38.648 INFO:teuthology.orchestra.run.smithi151.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 346 kB/s | 43 kB 00:00 2024-05-17T22:23:38.682 INFO:teuthology.orchestra.run.smithi151.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 275 kB/s | 32 kB 00:00 2024-05-17T22:23:38.710 INFO:teuthology.orchestra.run.smithi096.stdout:(153/156): openssl-3.2.1-1.el9.x86_64.rpm 20 MB/s | 1.3 MB 00:00 2024-05-17T22:23:38.757 INFO:teuthology.orchestra.run.smithi151.stdout:(116/156): libarrow-9.0.0-11.el9.x86_64.rpm 5.4 MB/s | 4.4 MB 00:00 2024-05-17T22:23:38.782 INFO:teuthology.orchestra.run.smithi151.stdout:(117/156): python3-certifi-2021.10.8-2.el9.noar 112 kB/s | 15 kB 00:00 2024-05-17T22:23:38.794 INFO:teuthology.orchestra.run.smithi096.stdout:(154/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 29 MB/s | 2.4 MB 00:00 2024-05-17T22:23:38.807 INFO:teuthology.orchestra.run.smithi151.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-05-17T22:23:38.891 INFO:teuthology.orchestra.run.smithi151.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-05-17T22:23:39.017 INFO:teuthology.orchestra.run.smithi151.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 169 kB/s | 35 kB 00:00 2024-05-17T22:23:39.042 INFO:teuthology.orchestra.run.smithi151.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 1.3 MB/s | 206 kB 00:00 2024-05-17T22:23:39.067 INFO:teuthology.orchestra.run.smithi151.stdout:(122/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.0 MB/s | 303 kB 00:00 2024-05-17T22:23:39.079 INFO:teuthology.orchestra.run.smithi012.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-05-17T22:23:39.176 INFO:teuthology.orchestra.run.smithi151.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-05-17T22:23:39.186 INFO:teuthology.orchestra.run.smithi096.stdout:(155/156): librbd1-19.0.0-3794.g931809a9.el9.x8 3.0 MB/s | 3.1 MB 00:01 2024-05-17T22:23:39.201 INFO:teuthology.orchestra.run.smithi151.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 80 kB/s | 11 kB 00:00 2024-05-17T22:23:39.276 INFO:teuthology.orchestra.run.smithi151.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-05-17T22:23:39.327 INFO:teuthology.orchestra.run.smithi151.stdout:(126/156): python3-grpcio-1.46.7-10.el9.x86_64. 6.6 MB/s | 2.0 MB 00:00 2024-05-17T22:23:39.352 INFO:teuthology.orchestra.run.smithi151.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 154 kB/s | 23 kB 00:00 2024-05-17T22:23:39.385 INFO:teuthology.orchestra.run.smithi151.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 179 kB/s | 19 kB 00:00 2024-05-17T22:23:39.435 INFO:teuthology.orchestra.run.smithi151.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 181 kB/s | 20 kB 00:00 2024-05-17T22:23:39.486 INFO:teuthology.orchestra.run.smithi151.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 7.6 MB/s | 1.0 MB 00:00 2024-05-17T22:23:39.511 INFO:teuthology.orchestra.run.smithi151.stdout:(131/156): python3-logutils-0.3.5-21.el9.noarch 369 kB/s | 46 kB 00:00 2024-05-17T22:23:39.553 INFO:teuthology.orchestra.run.smithi151.stdout:(132/156): python3-more-itertools-8.12.0-2.el9. 671 kB/s | 79 kB 00:00 2024-05-17T22:23:39.595 INFO:teuthology.orchestra.run.smithi151.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 529 kB/s | 58 kB 00:00 2024-05-17T22:23:39.662 INFO:teuthology.orchestra.run.smithi151.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-05-17T22:23:39.687 INFO:teuthology.orchestra.run.smithi151.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 123 kB/s | 16 kB 00:00 2024-05-17T22:23:39.713 INFO:teuthology.orchestra.run.smithi151.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 768 kB/s | 90 kB 00:00 2024-05-17T22:23:39.780 INFO:teuthology.orchestra.run.smithi151.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 241 kB/s | 28 kB 00:00 2024-05-17T22:23:39.805 INFO:teuthology.orchestra.run.smithi151.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 263 kB/s | 31 kB 00:00 2024-05-17T22:23:39.830 INFO:teuthology.orchestra.run.smithi151.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-05-17T22:23:39.906 INFO:teuthology.orchestra.run.smithi151.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 472 kB/s | 59 kB 00:00 2024-05-17T22:23:39.931 INFO:teuthology.orchestra.run.smithi151.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 284 kB/s | 36 kB 00:00 2024-05-17T22:23:39.956 INFO:teuthology.orchestra.run.smithi151.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 406 kB/s | 51 kB 00:00 2024-05-17T22:23:40.040 INFO:teuthology.orchestra.run.smithi151.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-05-17T22:23:40.065 INFO:teuthology.orchestra.run.smithi151.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 672 kB/s | 90 kB 00:00 2024-05-17T22:23:40.090 INFO:teuthology.orchestra.run.smithi151.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-05-17T22:23:40.182 INFO:teuthology.orchestra.run.smithi151.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 140 kB/s | 20 kB 00:00 2024-05-17T22:23:40.224 INFO:teuthology.orchestra.run.smithi151.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 1.2 MB/s | 192 kB 00:00 2024-05-17T22:23:40.324 INFO:teuthology.orchestra.run.smithi151.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 6.8 MB/s | 1.6 MB 00:00 2024-05-17T22:23:40.666 INFO:teuthology.orchestra.run.smithi151.stdout:(149/156): NetworkManager-1.47.90-1.el9.x86_64. 6.8 MB/s | 2.3 MB 00:00 2024-05-17T22:23:40.816 INFO:teuthology.orchestra.run.smithi151.stdout:(150/156): NetworkManager-libnm-1.47.90-1.el9.x 12 MB/s | 1.8 MB 00:00 2024-05-17T22:23:40.883 INFO:teuthology.orchestra.run.smithi151.stdout:(151/156): NetworkManager-team-1.47.90-1.el9.x8 624 kB/s | 41 kB 00:00 2024-05-17T22:23:41.002 INFO:teuthology.orchestra.run.smithi096.stdout:(156/156): librados2-19.0.0-3794.g931809a9.el9. 1.2 MB/s | 3.3 MB 00:02 2024-05-17T22:23:41.005 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-05-17T22:23:41.006 INFO:teuthology.orchestra.run.smithi096.stdout:Total 12 MB/s | 219 MB 00:18 2024-05-17T22:23:41.108 INFO:teuthology.orchestra.run.smithi151.stdout:(152/156): NetworkManager-tui-1.47.90-1.el9.x86 1.1 MB/s | 248 kB 00:00 2024-05-17T22:23:41.417 INFO:teuthology.orchestra.run.smithi151.stdout:(153/156): librados2-19.0.0-3794.g931809a9.el9. 2.7 MB/s | 3.3 MB 00:01 2024-05-17T22:23:41.467 INFO:teuthology.orchestra.run.smithi151.stdout:(154/156): librbd1-19.0.0-3794.g931809a9.el9.x8 2.5 MB/s | 3.1 MB 00:01 2024-05-17T22:23:41.472 INFO:teuthology.orchestra.run.smithi012.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-17T22:23:41.473 INFO:teuthology.orchestra.run.smithi012.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-17T22:23:41.484 INFO:teuthology.orchestra.run.smithi012.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-17T22:23:41.485 INFO:teuthology.orchestra.run.smithi012.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-17T22:23:41.486 INFO:teuthology.orchestra.run.smithi012.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-17T22:23:41.487 INFO:teuthology.orchestra.run.smithi012.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-17T22:23:41.489 INFO:teuthology.orchestra.run.smithi012.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-17T22:23:41.491 INFO:teuthology.orchestra.run.smithi012.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-17T22:23:41.492 INFO:teuthology.orchestra.run.smithi012.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-17T22:23:41.509 INFO:teuthology.orchestra.run.smithi151.stdout:(155/156): openssl-3.2.1-1.el9.x86_64.rpm 3.4 MB/s | 1.3 MB 00:00 2024-05-17T22:23:41.556 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-05-17T22:23:41.571 INFO:teuthology.orchestra.run.smithi012.stdout:======================================================================================= 2024-05-17T22:23:41.571 INFO:teuthology.orchestra.run.smithi012.stdout: Package Arch Version Repository Size 2024-05-17T22:23:41.572 INFO:teuthology.orchestra.run.smithi012.stdout:======================================================================================= 2024-05-17T22:23:41.572 INFO:teuthology.orchestra.run.smithi012.stdout:Installing: 2024-05-17T22:23:41.572 INFO:teuthology.orchestra.run.smithi012.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-17T22:23:41.572 INFO:teuthology.orchestra.run.smithi012.stdout: ceph x86_64 2:19.0.0-3794.g931809a9.el9 ceph 6.6 k 2024-05-17T22:23:41.572 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-base x86_64 2:19.0.0-3794.g931809a9.el9 ceph 5.4 M 2024-05-17T22:23:41.572 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-fuse x86_64 2:19.0.0-3794.g931809a9.el9 ceph 866 k 2024-05-17T22:23:41.572 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3794.g931809a9.el9 ceph 147 k 2024-05-17T22:23:41.572 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr x86_64 2:19.0.0-3794.g931809a9.el9 ceph 1.5 M 2024-05-17T22:23:41.572 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 143 k 2024-05-17T22:23:41.572 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 3.5 M 2024-05-17T22:23:41.572 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 7.4 M 2024-05-17T22:23:41.572 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-rook noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 49 k 2024-05-17T22:23:41.572 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-radosgw x86_64 2:19.0.0-3794.g931809a9.el9 ceph 10 M 2024-05-17T22:23:41.572 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-test x86_64 2:19.0.0-3794.g931809a9.el9 ceph 43 M 2024-05-17T22:23:41.573 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-volume noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 273 k 2024-05-17T22:23:41.573 INFO:teuthology.orchestra.run.smithi012.stdout: cephadm noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 763 k 2024-05-17T22:23:41.573 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-mirror x86_64 2:19.0.0-3794.g931809a9.el9 ceph 227 k 2024-05-17T22:23:41.573 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-shell x86_64 2:19.0.0-3794.g931809a9.el9 ceph 27 k 2024-05-17T22:23:41.573 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-top noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 18 k 2024-05-17T22:23:41.573 INFO:teuthology.orchestra.run.smithi012.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-17T22:23:41.573 INFO:teuthology.orchestra.run.smithi012.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-17T22:23:41.573 INFO:teuthology.orchestra.run.smithi012.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-17T22:23:41.573 INFO:teuthology.orchestra.run.smithi012.stdout: libcephfs-devel x86_64 2:19.0.0-3794.g931809a9.el9 ceph 32 k 2024-05-17T22:23:41.573 INFO:teuthology.orchestra.run.smithi012.stdout: libcephfs2 x86_64 2:19.0.0-3794.g931809a9.el9 ceph 759 k 2024-05-17T22:23:41.573 INFO:teuthology.orchestra.run.smithi012.stdout: librados-devel x86_64 2:19.0.0-3794.g931809a9.el9 ceph 127 k 2024-05-17T22:23:41.573 INFO:teuthology.orchestra.run.smithi012.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-17T22:23:41.574 INFO:teuthology.orchestra.run.smithi012.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-17T22:23:41.574 INFO:teuthology.orchestra.run.smithi012.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-17T22:23:41.574 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-17T22:23:41.574 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-17T22:23:41.574 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-17T22:23:41.574 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cephfs x86_64 2:19.0.0-3794.g931809a9.el9 ceph 162 k 2024-05-17T22:23:41.574 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rados x86_64 2:19.0.0-3794.g931809a9.el9 ceph 321 k 2024-05-17T22:23:41.574 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rbd x86_64 2:19.0.0-3794.g931809a9.el9 ceph 300 k 2024-05-17T22:23:41.574 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rgw x86_64 2:19.0.0-3794.g931809a9.el9 ceph 99 k 2024-05-17T22:23:41.574 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-fuse x86_64 2:19.0.0-3794.g931809a9.el9 ceph 86 k 2024-05-17T22:23:41.574 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-mirror x86_64 2:19.0.0-3794.g931809a9.el9 ceph 3.0 M 2024-05-17T22:23:41.574 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-nbd x86_64 2:19.0.0-3794.g931809a9.el9 ceph 169 k 2024-05-17T22:23:41.575 INFO:teuthology.orchestra.run.smithi012.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-17T22:23:41.575 INFO:teuthology.orchestra.run.smithi012.stdout:Upgrading: 2024-05-17T22:23:41.575 INFO:teuthology.orchestra.run.smithi012.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-17T22:23:41.575 INFO:teuthology.orchestra.run.smithi012.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-17T22:23:41.575 INFO:teuthology.orchestra.run.smithi012.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-17T22:23:41.575 INFO:teuthology.orchestra.run.smithi012.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-17T22:23:41.575 INFO:teuthology.orchestra.run.smithi012.stdout: librados2 x86_64 2:19.0.0-3794.g931809a9.el9 ceph 3.3 M 2024-05-17T22:23:41.575 INFO:teuthology.orchestra.run.smithi012.stdout: librbd1 x86_64 2:19.0.0-3794.g931809a9.el9 ceph 3.1 M 2024-05-17T22:23:41.575 INFO:teuthology.orchestra.run.smithi012.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-17T22:23:41.575 INFO:teuthology.orchestra.run.smithi012.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-17T22:23:41.575 INFO:teuthology.orchestra.run.smithi012.stdout:Installing dependencies: 2024-05-17T22:23:41.575 INFO:teuthology.orchestra.run.smithi012.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-17T22:23:41.575 INFO:teuthology.orchestra.run.smithi012.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-17T22:23:41.575 INFO:teuthology.orchestra.run.smithi012.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-17T22:23:41.576 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-common x86_64 2:19.0.0-3794.g931809a9.el9 ceph 20 M 2024-05-17T22:23:41.576 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 34 k 2024-05-17T22:23:41.576 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mds x86_64 2:19.0.0-3794.g931809a9.el9 ceph 2.3 M 2024-05-17T22:23:41.576 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 274 k 2024-05-17T22:23:41.576 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mon x86_64 2:19.0.0-3794.g931809a9.el9 ceph 4.9 M 2024-05-17T22:23:41.576 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-osd x86_64 2:19.0.0-3794.g931809a9.el9 ceph 18 M 2024-05-17T22:23:41.576 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 16 k 2024-05-17T22:23:41.576 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-selinux x86_64 2:19.0.0-3794.g931809a9.el9 ceph 25 k 2024-05-17T22:23:41.576 INFO:teuthology.orchestra.run.smithi012.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-17T22:23:41.576 INFO:teuthology.orchestra.run.smithi012.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-17T22:23:41.576 INFO:teuthology.orchestra.run.smithi012.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-17T22:23:41.577 INFO:teuthology.orchestra.run.smithi012.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-17T22:23:41.577 INFO:teuthology.orchestra.run.smithi012.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-17T22:23:41.577 INFO:teuthology.orchestra.run.smithi012.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-17T22:23:41.577 INFO:teuthology.orchestra.run.smithi012.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-17T22:23:41.577 INFO:teuthology.orchestra.run.smithi012.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-17T22:23:41.577 INFO:teuthology.orchestra.run.smithi012.stdout: libcephsqlite x86_64 2:19.0.0-3794.g931809a9.el9 ceph 158 k 2024-05-17T22:23:41.577 INFO:teuthology.orchestra.run.smithi012.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-17T22:23:41.577 INFO:teuthology.orchestra.run.smithi012.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-17T22:23:41.577 INFO:teuthology.orchestra.run.smithi012.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-17T22:23:41.577 INFO:teuthology.orchestra.run.smithi012.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-17T22:23:41.577 INFO:teuthology.orchestra.run.smithi012.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-17T22:23:41.577 INFO:teuthology.orchestra.run.smithi012.stdout: libradosstriper1 x86_64 2:19.0.0-3794.g931809a9.el9 ceph 501 k 2024-05-17T22:23:41.577 INFO:teuthology.orchestra.run.smithi012.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-17T22:23:41.577 INFO:teuthology.orchestra.run.smithi012.stdout: librgw2 x86_64 2:19.0.0-3794.g931809a9.el9 ceph 5.2 M 2024-05-17T22:23:41.577 INFO:teuthology.orchestra.run.smithi012.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-17T22:23:41.578 INFO:teuthology.orchestra.run.smithi012.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-17T22:23:41.578 INFO:teuthology.orchestra.run.smithi012.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-17T22:23:41.578 INFO:teuthology.orchestra.run.smithi012.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-17T22:23:41.578 INFO:teuthology.orchestra.run.smithi012.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-17T22:23:41.578 INFO:teuthology.orchestra.run.smithi012.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-17T22:23:41.578 INFO:teuthology.orchestra.run.smithi012.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-17T22:23:41.578 INFO:teuthology.orchestra.run.smithi012.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-17T22:23:41.578 INFO:teuthology.orchestra.run.smithi012.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-17T22:23:41.578 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-17T22:23:41.578 INFO:teuthology.orchestra.run.smithi012.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-17T22:23:41.578 INFO:teuthology.orchestra.run.smithi012.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-17T22:23:41.578 INFO:teuthology.orchestra.run.smithi012.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-17T22:23:41.578 INFO:teuthology.orchestra.run.smithi012.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-17T22:23:41.578 INFO:teuthology.orchestra.run.smithi012.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-17T22:23:41.579 INFO:teuthology.orchestra.run.smithi012.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-17T22:23:41.579 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-17T22:23:41.579 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-argparse x86_64 2:19.0.0-3794.g931809a9.el9 ceph 45 k 2024-05-17T22:23:41.579 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-common x86_64 2:19.0.0-3794.g931809a9.el9 ceph 136 k 2024-05-17T22:23:41.579 INFO:teuthology.orchestra.run.smithi012.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-17T22:23:41.579 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-17T22:23:41.579 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-17T22:23:41.579 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-17T22:23:41.579 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-17T22:23:41.579 INFO:teuthology.orchestra.run.smithi012.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-17T22:23:41.579 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-17T22:23:41.579 INFO:teuthology.orchestra.run.smithi012.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-17T22:23:41.579 INFO:teuthology.orchestra.run.smithi012.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-17T22:23:41.579 INFO:teuthology.orchestra.run.smithi012.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-17T22:23:41.579 INFO:teuthology.orchestra.run.smithi012.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-17T22:23:41.580 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-17T22:23:41.580 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-17T22:23:41.580 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-17T22:23:41.580 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-17T22:23:41.580 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-17T22:23:41.580 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-17T22:23:41.580 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-17T22:23:41.580 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-17T22:23:41.580 INFO:teuthology.orchestra.run.smithi012.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-17T22:23:41.580 INFO:teuthology.orchestra.run.smithi012.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-17T22:23:41.580 INFO:teuthology.orchestra.run.smithi012.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-17T22:23:41.580 INFO:teuthology.orchestra.run.smithi012.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-17T22:23:41.580 INFO:teuthology.orchestra.run.smithi012.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-17T22:23:41.581 INFO:teuthology.orchestra.run.smithi012.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-17T22:23:41.581 INFO:teuthology.orchestra.run.smithi012.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-17T22:23:41.581 INFO:teuthology.orchestra.run.smithi012.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-17T22:23:41.581 INFO:teuthology.orchestra.run.smithi012.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-17T22:23:41.581 INFO:teuthology.orchestra.run.smithi012.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-17T22:23:41.581 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-17T22:23:41.581 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-17T22:23:41.581 INFO:teuthology.orchestra.run.smithi012.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-17T22:23:41.581 INFO:teuthology.orchestra.run.smithi012.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-17T22:23:41.581 INFO:teuthology.orchestra.run.smithi012.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-17T22:23:41.581 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-17T22:23:41.581 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-17T22:23:41.581 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-17T22:23:41.581 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-17T22:23:41.581 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-17T22:23:41.582 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-17T22:23:41.582 INFO:teuthology.orchestra.run.smithi012.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-17T22:23:41.582 INFO:teuthology.orchestra.run.smithi012.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-17T22:23:41.582 INFO:teuthology.orchestra.run.smithi012.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-17T22:23:41.582 INFO:teuthology.orchestra.run.smithi012.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-17T22:23:41.582 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-17T22:23:41.582 INFO:teuthology.orchestra.run.smithi012.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-17T22:23:41.582 INFO:teuthology.orchestra.run.smithi012.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-17T22:23:41.582 INFO:teuthology.orchestra.run.smithi012.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-17T22:23:41.582 INFO:teuthology.orchestra.run.smithi012.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-17T22:23:41.582 INFO:teuthology.orchestra.run.smithi012.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-17T22:23:41.582 INFO:teuthology.orchestra.run.smithi012.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-17T22:23:41.582 INFO:teuthology.orchestra.run.smithi012.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-17T22:23:41.582 INFO:teuthology.orchestra.run.smithi012.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-17T22:23:41.583 INFO:teuthology.orchestra.run.smithi012.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-17T22:23:41.583 INFO:teuthology.orchestra.run.smithi012.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-17T22:23:41.583 INFO:teuthology.orchestra.run.smithi012.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-17T22:23:41.583 INFO:teuthology.orchestra.run.smithi012.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-17T22:23:41.583 INFO:teuthology.orchestra.run.smithi012.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-17T22:23:41.583 INFO:teuthology.orchestra.run.smithi012.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-17T22:23:41.583 INFO:teuthology.orchestra.run.smithi012.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-17T22:23:41.583 INFO:teuthology.orchestra.run.smithi012.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-17T22:23:41.583 INFO:teuthology.orchestra.run.smithi012.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-17T22:23:41.583 INFO:teuthology.orchestra.run.smithi012.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-17T22:23:41.583 INFO:teuthology.orchestra.run.smithi012.stdout:Installing weak dependencies: 2024-05-17T22:23:41.583 INFO:teuthology.orchestra.run.smithi012.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-17T22:23:41.583 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:23:41.583 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction Summary 2024-05-17T22:23:41.583 INFO:teuthology.orchestra.run.smithi012.stdout:======================================================================================= 2024-05-17T22:23:41.583 INFO:teuthology.orchestra.run.smithi012.stdout:Install 148 Packages 2024-05-17T22:23:41.584 INFO:teuthology.orchestra.run.smithi012.stdout:Upgrade 8 Packages 2024-05-17T22:23:41.584 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:23:41.584 INFO:teuthology.orchestra.run.smithi012.stdout:Total download size: 219 M 2024-05-17T22:23:41.584 INFO:teuthology.orchestra.run.smithi012.stdout:Downloading Packages: 2024-05-17T22:23:41.643 INFO:teuthology.orchestra.run.smithi151.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 11 MB/s | 2.4 MB 00:00 2024-05-17T22:23:41.649 INFO:teuthology.orchestra.run.smithi151.stdout:-------------------------------------------------------------------------------- 2024-05-17T22:23:41.650 INFO:teuthology.orchestra.run.smithi151.stdout:Total 12 MB/s | 219 MB 00:18 2024-05-17T22:23:42.195 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-05-17T22:23:42.346 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-05-17T22:23:42.346 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-05-17T22:23:42.571 INFO:teuthology.orchestra.run.smithi012.stdout:(1/156): ceph-19.0.0-3794.g931809a9.el9.x86_64. 16 kB/s | 6.6 kB 00:00 2024-05-17T22:23:42.799 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction check 2024-05-17T22:23:42.953 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction check succeeded. 2024-05-17T22:23:42.953 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction test 2024-05-17T22:23:43.405 INFO:teuthology.orchestra.run.smithi012.stdout:(2/156): ceph-fuse-19.0.0-3794.g931809a9.el9.x8 1.0 MB/s | 866 kB 00:00 2024-05-17T22:23:43.571 INFO:teuthology.orchestra.run.smithi012.stdout:(3/156): ceph-immutable-object-cache-19.0.0-379 882 kB/s | 147 kB 00:00 2024-05-17T22:23:44.013 INFO:teuthology.orchestra.run.smithi012.stdout:(4/156): ceph-base-19.0.0-3794.g931809a9.el9.x8 2.9 MB/s | 5.4 MB 00:01 2024-05-17T22:23:44.110 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-05-17T22:23:44.111 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-05-17T22:23:44.297 INFO:teuthology.orchestra.run.smithi012.stdout:(5/156): ceph-mgr-19.0.0-3794.g931809a9.el9.x86 5.3 MB/s | 1.5 MB 00:00 2024-05-17T22:23:44.705 INFO:teuthology.orchestra.run.smithi012.stdout:(6/156): ceph-mds-19.0.0-3794.g931809a9.el9.x86 2.0 MB/s | 2.3 MB 00:01 2024-05-17T22:23:44.712 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction test succeeded. 2024-05-17T22:23:44.712 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction 2024-05-17T22:23:44.989 INFO:teuthology.orchestra.run.smithi012.stdout:(7/156): ceph-mon-19.0.0-3794.g931809a9.el9.x86 7.1 MB/s | 4.9 MB 00:00 2024-05-17T22:23:46.206 INFO:teuthology.orchestra.run.smithi012.stdout:(8/156): ceph-radosgw-19.0.0-3794.g931809a9.el9 8.3 MB/s | 10 MB 00:01 2024-05-17T22:23:46.306 INFO:teuthology.orchestra.run.smithi012.stdout:(9/156): ceph-selinux-19.0.0-3794.g931809a9.el9 253 kB/s | 25 kB 00:00 2024-05-17T22:23:47.240 INFO:teuthology.orchestra.run.smithi012.stdout:(10/156): ceph-common-19.0.0-3794.g931809a9.el9 3.9 MB/s | 20 MB 00:05 2024-05-17T22:23:47.348 INFO:teuthology.orchestra.run.smithi012.stdout:(11/156): cephfs-mirror-19.0.0-3794.g931809a9.e 2.1 MB/s | 227 kB 00:00 2024-05-17T22:23:47.440 INFO:teuthology.orchestra.run.smithi012.stdout:(12/156): cephfs-shell-19.0.0-3794.g931809a9.el 296 kB/s | 27 kB 00:00 2024-05-17T22:23:47.524 INFO:teuthology.orchestra.run.smithi012.stdout:(13/156): libcephfs-devel-19.0.0-3794.g931809a9 381 kB/s | 32 kB 00:00 2024-05-17T22:23:47.716 INFO:teuthology.orchestra.run.smithi012.stdout:(14/156): libcephfs2-19.0.0-3794.g931809a9.el9. 3.9 MB/s | 759 kB 00:00 2024-05-17T22:23:47.824 INFO:teuthology.orchestra.run.smithi012.stdout:(15/156): libcephsqlite-19.0.0-3794.g931809a9.e 1.4 MB/s | 158 kB 00:00 2024-05-17T22:23:47.933 INFO:teuthology.orchestra.run.smithi012.stdout:(16/156): librados-devel-19.0.0-3794.g931809a9. 1.1 MB/s | 127 kB 00:00 2024-05-17T22:23:48.300 INFO:teuthology.orchestra.run.smithi012.stdout:(17/156): libradosstriper1-19.0.0-3794.g931809a 1.3 MB/s | 501 kB 00:00 2024-05-17T22:23:50.166 INFO:teuthology.orchestra.run.smithi012.stdout:(18/156): librgw2-19.0.0-3794.g931809a9.el9.x86 2.8 MB/s | 5.2 MB 00:01 2024-05-17T22:23:50.250 INFO:teuthology.orchestra.run.smithi012.stdout:(19/156): python3-ceph-argparse-19.0.0-3794.g93 542 kB/s | 45 kB 00:00 2024-05-17T22:23:50.327 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-05-17T22:23:50.333 INFO:teuthology.orchestra.run.smithi012.stdout:(20/156): python3-ceph-common-19.0.0-3794.g9318 1.6 MB/s | 136 kB 00:00 2024-05-17T22:23:50.414 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-17T22:23:50.426 INFO:teuthology.orchestra.run.smithi012.stdout:(21/156): python3-cephfs-19.0.0-3794.g931809a9. 1.7 MB/s | 162 kB 00:00 2024-05-17T22:23:50.452 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-17T22:23:50.505 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-17T22:23:50.539 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-17T22:23:50.565 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-17T22:23:50.607 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-17T22:23:50.676 INFO:teuthology.orchestra.run.smithi012.stdout:(22/156): ceph-osd-19.0.0-3794.g931809a9.el9.x8 3.0 MB/s | 18 MB 00:05 2024-05-17T22:23:50.710 INFO:teuthology.orchestra.run.smithi012.stdout:(23/156): python3-rados-19.0.0-3794.g931809a9.e 1.1 MB/s | 321 kB 00:00 2024-05-17T22:23:50.741 INFO:teuthology.orchestra.run.smithi151.stdout: Preparing : 1/1 2024-05-17T22:23:50.802 INFO:teuthology.orchestra.run.smithi012.stdout:(24/156): python3-rbd-19.0.0-3794.g931809a9.el9 2.3 MB/s | 300 kB 00:00 2024-05-17T22:23:50.822 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-17T22:23:50.827 INFO:teuthology.orchestra.run.smithi012.stdout:(25/156): python3-rgw-19.0.0-3794.g931809a9.el9 849 kB/s | 99 kB 00:00 2024-05-17T22:23:50.860 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-17T22:23:50.869 INFO:teuthology.orchestra.run.smithi012.stdout:(26/156): rbd-fuse-19.0.0-3794.g931809a9.el9.x8 1.3 MB/s | 86 kB 00:00 2024-05-17T22:23:50.914 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-17T22:23:50.943 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-17T22:23:50.947 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-17T22:23:50.973 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-17T22:23:51.016 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-17T22:23:51.075 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : librados2-2:19.0.0-3794.g931809a9.el9.x86_64 8/164 2024-05-17T22:23:51.200 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librados2-2:19.0.0-3794.g931809a9.el9.x86_64 8/164 2024-05-17T22:23:51.234 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephfs2-2:19.0.0-3794.g931809a9.el9.x86_64 9/164 2024-05-17T22:23:51.269 INFO:teuthology.orchestra.run.smithi012.stdout:(27/156): ceph-test-19.0.0-3794.g931809a9.el9.x 8.7 MB/s | 43 MB 00:04 2024-05-17T22:23:51.294 INFO:teuthology.orchestra.run.smithi012.stdout:(28/156): rbd-nbd-19.0.0-3794.g931809a9.el9.x86 397 kB/s | 169 kB 00:00 2024-05-17T22:23:51.319 INFO:teuthology.orchestra.run.smithi012.stdout:(29/156): ceph-grafana-dashboards-19.0.0-3794.g 687 kB/s | 34 kB 00:00 2024-05-17T22:23:51.325 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libcephfs2-2:19.0.0-3794.g931809a9.el9.x86_64 9/164 2024-05-17T22:23:51.355 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rados-2:19.0.0-3794.g931809a9.el9.x86_64 10/164 2024-05-17T22:23:51.378 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-17T22:23:51.387 INFO:teuthology.orchestra.run.smithi012.stdout:(30/156): ceph-mgr-cephadm-19.0.0-3794.g931809a 1.5 MB/s | 143 kB 00:00 2024-05-17T22:23:51.490 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : librados2-2:19.0.0-3794.g931809a9.el9.x86_64 8/164 2024-05-17T22:23:51.505 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-17T22:23:51.561 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-17T22:23:51.589 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephsqlite-2:19.0.0-3794.g931809a9.el9.x86_64 13/164 2024-05-17T22:23:51.616 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librados2-2:19.0.0-3794.g931809a9.el9.x86_64 8/164 2024-05-17T22:23:51.658 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libcephfs2-2:19.0.0-3794.g931809a9.el9.x86_64 9/164 2024-05-17T22:23:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3794.g931809a9.el9.x86_64 13/164 2024-05-17T22:23:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libradosstriper1-2:19.0.0-3794.g931809a9.el9.x86 14/164 2024-05-17T22:23:51.750 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: libcephfs2-2:19.0.0-3794.g931809a9.el9.x86_64 9/164 2024-05-17T22:23:51.762 INFO:teuthology.orchestra.run.smithi012.stdout:(31/156): rbd-mirror-19.0.0-3794.g931809a9.el9. 3.3 MB/s | 3.0 MB 00:00 2024-05-17T22:23:51.779 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rados-2:19.0.0-3794.g931809a9.el9.x86_64 10/164 2024-05-17T22:23:51.785 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3794.g931809a9.el9.x86 14/164 2024-05-17T22:23:51.807 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-17T22:23:51.820 INFO:teuthology.orchestra.run.smithi012.stdout:(32/156): ceph-mgr-dashboard-19.0.0-3794.g93180 7.1 MB/s | 3.5 MB 00:00 2024-05-17T22:23:51.829 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-17T22:23:51.871 INFO:teuthology.orchestra.run.smithi012.stdout:(33/156): ceph-mgr-rook-19.0.0-3794.g931809a9.e 990 kB/s | 49 kB 00:00 2024-05-17T22:23:51.896 INFO:teuthology.orchestra.run.smithi012.stdout:(34/156): ceph-mgr-modules-core-19.0.0-3794.g93 2.0 MB/s | 274 kB 00:00 2024-05-17T22:23:51.921 INFO:teuthology.orchestra.run.smithi012.stdout:(35/156): ceph-prometheus-alerts-19.0.0-3794.g9 328 kB/s | 16 kB 00:00 2024-05-17T22:23:51.935 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-17T22:23:51.985 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-17T22:23:52.013 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libcephsqlite-2:19.0.0-3794.g931809a9.el9.x86_64 13/164 2024-05-17T22:23:52.015 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-17T22:23:52.030 INFO:teuthology.orchestra.run.smithi012.stdout:(36/156): ceph-volume-19.0.0-3794.g931809a9.el9 2.0 MB/s | 273 kB 00:00 2024-05-17T22:23:52.064 INFO:teuthology.orchestra.run.smithi012.stdout:(37/156): cephadm-19.0.0-3794.g931809a9.el9.noa 5.3 MB/s | 763 kB 00:00 2024-05-17T22:23:52.105 INFO:teuthology.orchestra.run.smithi012.stdout:(38/156): cephfs-top-19.0.0-3794.g931809a9.el9. 242 kB/s | 18 kB 00:00 2024-05-17T22:23:52.118 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3794.g931809a9.el9.x86_64 13/164 2024-05-17T22:23:52.147 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libradosstriper1-2:19.0.0-3794.g931809a9.el9.x86 14/164 2024-05-17T22:23:52.154 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-17T22:23:52.206 INFO:teuthology.orchestra.run.smithi012.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 566 kB/s | 80 kB 00:00 2024-05-17T22:23:52.209 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3794.g931809a9.el9.x86 14/164 2024-05-17T22:23:52.240 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-17T22:23:52.264 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-17T22:23:52.273 INFO:teuthology.orchestra.run.smithi012.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 195 kB 00:00 2024-05-17T22:23:52.298 INFO:teuthology.orchestra.run.smithi012.stdout:(41/156): libgfortran-11.4.1-3.el9.x86_64.rpm 4.1 MB/s | 803 kB 00:00 2024-05-17T22:23:52.304 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-17T22:23:52.323 INFO:teuthology.orchestra.run.smithi012.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-05-17T22:23:52.348 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-17T22:23:52.374 INFO:teuthology.orchestra.run.smithi012.stdout:(43/156): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-05-17T22:23:52.399 INFO:teuthology.orchestra.run.smithi012.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-05-17T22:23:52.424 INFO:teuthology.orchestra.run.smithi012.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-05-17T22:23:52.450 INFO:teuthology.orchestra.run.smithi012.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-05-17T22:23:52.456 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-17T22:23:52.456 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-17T22:23:52.475 INFO:teuthology.orchestra.run.smithi012.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-05-17T22:23:52.533 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-17T22:23:52.588 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-17T22:23:52.642 INFO:teuthology.orchestra.run.smithi012.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 3.5 MB/s | 689 kB 00:00 2024-05-17T22:23:52.701 INFO:teuthology.orchestra.run.smithi012.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 4.1 MB/s | 943 kB 00:00 2024-05-17T22:23:52.726 INFO:teuthology.orchestra.run.smithi012.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.3 MB/s | 107 kB 00:00 2024-05-17T22:23:52.743 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-17T22:23:52.751 INFO:teuthology.orchestra.run.smithi012.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 428 kB/s | 21 kB 00:00 2024-05-17T22:23:52.762 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-17T22:23:52.776 INFO:teuthology.orchestra.run.smithi012.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-05-17T22:23:52.802 INFO:teuthology.orchestra.run.smithi012.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 655 kB/s | 33 kB 00:00 2024-05-17T22:23:52.822 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-17T22:23:52.844 INFO:teuthology.orchestra.run.smithi012.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 437 kB/s | 18 kB 00:00 2024-05-17T22:23:52.918 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-17T22:23:52.922 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-17T22:23:52.936 INFO:teuthology.orchestra.run.smithi012.stdout:(55/156): ceph-mgr-diskprediction-local-19.0.0- 4.8 MB/s | 7.4 MB 00:01 2024-05-17T22:23:52.953 INFO:teuthology.orchestra.run.smithi012.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 102 kB/s | 11 kB 00:00 2024-05-17T22:23:52.955 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-17T22:23:52.986 INFO:teuthology.orchestra.run.smithi012.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-05-17T22:23:52.999 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-17T22:23:53.022 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-17T22:23:53.036 INFO:teuthology.orchestra.run.smithi012.stdout:(58/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-05-17T22:23:53.079 INFO:teuthology.orchestra.run.smithi012.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 7.0 MB/s | 662 kB 00:00 2024-05-17T22:23:53.096 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-17T22:23:53.120 INFO:teuthology.orchestra.run.smithi012.stdout:(60/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 270 kB/s | 45 kB 00:00 2024-05-17T22:23:53.138 INFO:teuthology.orchestra.run.smithi012.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 435 kB/s | 25 kB 00:00 2024-05-17T22:23:53.138 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-17T22:23:53.171 INFO:teuthology.orchestra.run.smithi012.stdout:(62/156): libtool-2.4.6-45.el9.x86_64.rpm 4.2 MB/s | 582 kB 00:00 2024-05-17T22:23:53.182 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-17T22:23:53.198 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-17T22:23:53.214 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-17T22:23:53.221 INFO:teuthology.orchestra.run.smithi012.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-05-17T22:23:53.242 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-17T22:23:53.247 INFO:teuthology.orchestra.run.smithi012.stdout:(64/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 377 kB/s | 47 kB 00:00 2024-05-17T22:23:53.272 INFO:teuthology.orchestra.run.smithi012.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-05-17T22:23:53.297 INFO:teuthology.orchestra.run.smithi012.stdout:(66/156): openblas-0.3.21-2.el9.x86_64.rpm 466 kB/s | 35 kB 00:00 2024-05-17T22:23:53.307 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-17T22:23:53.311 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-17T22:23:53.336 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-17T22:23:53.371 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-17T22:23:53.393 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-17T22:23:53.439 INFO:teuthology.orchestra.run.smithi012.stdout:(67/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 27 MB/s | 4.4 MB 00:00 2024-05-17T22:23:53.445 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-17T22:23:53.489 INFO:teuthology.orchestra.run.smithi012.stdout:(68/156): postgresql-13.14-1.el9.x86_64.rpm 8.3 MB/s | 1.6 MB 00:00 2024-05-17T22:23:53.495 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-17T22:23:53.531 INFO:teuthology.orchestra.run.smithi012.stdout:(69/156): postgresql-contrib-13.14-1.el9.x86_64 9.7 MB/s | 911 kB 00:00 2024-05-17T22:23:53.540 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-17T22:23:53.557 INFO:teuthology.orchestra.run.smithi012.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 1.9 MB/s | 133 kB 00:00 2024-05-17T22:23:53.572 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-17T22:23:53.576 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-17T22:23:53.599 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-17T22:23:53.669 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-17T22:23:53.690 INFO:teuthology.orchestra.run.smithi012.stdout:(71/156): postgresql-server-13.14-1.el9.x86_64. 37 MB/s | 5.8 MB 00:00 2024-05-17T22:23:53.707 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-17T22:23:53.724 INFO:teuthology.orchestra.run.smithi012.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 6.0 MB/s | 1.0 MB 00:00 2024-05-17T22:23:53.730 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-17T22:23:53.749 INFO:teuthology.orchestra.run.smithi012.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-05-17T22:23:53.779 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-17T22:23:53.783 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-17T22:23:53.799 INFO:teuthology.orchestra.run.smithi012.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-05-17T22:23:53.818 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ceph-common-2:19.0.0-3794.g931809a9.el9. 32/164 2024-05-17T22:23:53.850 INFO:teuthology.orchestra.run.smithi012.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-05-17T22:23:53.861 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ceph-argparse-2:19.0.0-3794.g931809a9.el 33/164 2024-05-17T22:23:53.890 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cephfs-2:19.0.0-3794.g931809a9.el9.x86_6 34/164 2024-05-17T22:23:53.900 INFO:teuthology.orchestra.run.smithi012.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 516 kB/s | 26 kB 00:00 2024-05-17T22:23:53.947 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-17T22:23:53.959 INFO:teuthology.orchestra.run.smithi012.stdout:(77/156): python3-jsonpointer-2.0-4.el9.noarch. 327 kB/s | 19 kB 00:00 2024-05-17T22:23:53.960 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-17T22:23:54.009 INFO:teuthology.orchestra.run.smithi012.stdout:(78/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-05-17T22:23:54.013 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : luarocks-3.9.2-5.el9.noarch 36/164 2024-05-17T22:23:54.059 INFO:teuthology.orchestra.run.smithi012.stdout:(79/156): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-05-17T22:23:54.088 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-17T22:23:54.127 INFO:teuthology.orchestra.run.smithi012.stdout:(80/156): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-05-17T22:23:54.164 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-17T22:23:54.177 INFO:teuthology.orchestra.run.smithi012.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-05-17T22:23:54.198 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-ceph-common-2:19.0.0-3794.g931809a9.el9. 32/164 2024-05-17T22:23:54.230 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-17T22:23:54.241 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-ceph-argparse-2:19.0.0-3794.g931809a9.el 33/164 2024-05-17T22:23:54.244 INFO:teuthology.orchestra.run.smithi012.stdout:(82/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 28 MB/s | 5.1 MB 00:00 2024-05-17T22:23:54.269 INFO:teuthology.orchestra.run.smithi012.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-05-17T22:23:54.270 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cephfs-2:19.0.0-3794.g931809a9.el9.x86_6 34/164 2024-05-17T22:23:54.282 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-17T22:23:54.294 INFO:teuthology.orchestra.run.smithi012.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-17T22:23:54.319 INFO:teuthology.orchestra.run.smithi012.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 840 kB/s | 42 kB 00:00 2024-05-17T22:23:54.342 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-17T22:23:54.385 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : luarocks-3.9.2-5.el9.noarch 36/164 2024-05-17T22:23:54.387 INFO:teuthology.orchestra.run.smithi012.stdout:(86/156): openblas-openmp-0.3.21-2.el9.x86_64.r 4.2 MB/s | 4.7 MB 00:01 2024-05-17T22:23:54.412 INFO:teuthology.orchestra.run.smithi012.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 2.2 MB/s | 269 kB 00:00 2024-05-17T22:23:54.437 INFO:teuthology.orchestra.run.smithi012.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-05-17T22:23:54.454 INFO:teuthology.orchestra.run.smithi012.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-05-17T22:23:54.469 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-17T22:23:54.479 INFO:teuthology.orchestra.run.smithi012.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 3.0 MB/s | 279 kB 00:00 2024-05-17T22:23:54.505 INFO:teuthology.orchestra.run.smithi012.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 800 kB/s | 54 kB 00:00 2024-05-17T22:23:54.538 INFO:teuthology.orchestra.run.smithi012.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 711 kB/s | 42 kB 00:00 2024-05-17T22:23:54.564 INFO:teuthology.orchestra.run.smithi012.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 752 kB/s | 44 kB 00:00 2024-05-17T22:23:54.589 INFO:teuthology.orchestra.run.smithi012.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-05-17T22:23:54.604 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-17T22:23:54.625 INFO:teuthology.orchestra.run.smithi012.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 589 kB/s | 36 kB 00:00 2024-05-17T22:23:54.648 INFO:teuthology.orchestra.run.smithi012.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 805 kB/s | 47 kB 00:00 2024-05-17T22:23:54.654 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-17T22:23:54.673 INFO:teuthology.orchestra.run.smithi012.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 6.2 MB/s | 305 kB 00:00 2024-05-17T22:23:54.699 INFO:teuthology.orchestra.run.smithi012.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-05-17T22:23:54.725 INFO:teuthology.orchestra.run.smithi012.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-05-17T22:23:54.749 INFO:teuthology.orchestra.run.smithi012.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-17T22:23:54.775 INFO:teuthology.orchestra.run.smithi012.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-05-17T22:23:54.810 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-17T22:23:54.834 INFO:teuthology.orchestra.run.smithi012.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 266 kB/s | 22 kB 00:00 2024-05-17T22:23:54.894 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-17T22:23:54.924 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-17T22:23:54.966 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-17T22:23:55.004 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librados-devel-2:19.0.0-3794.g931809a9.el9.x86_6 44/164 2024-05-17T22:23:55.045 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-17T22:23:55.059 INFO:teuthology.orchestra.run.smithi012.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-05-17T22:23:55.118 INFO:teuthology.orchestra.run.smithi012.stdout:(104/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.5 MB/s | 863 kB 00:00 2024-05-17T22:23:55.184 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-17T22:23:55.194 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-17T22:23:55.249 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-17T22:23:55.287 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-17T22:23:55.320 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-17T22:23:55.341 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-17T22:23:55.363 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-17T22:23:55.401 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librados-devel-2:19.0.0-3794.g931809a9.el9.x86_6 44/164 2024-05-17T22:23:55.405 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-17T22:23:55.443 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-17T22:23:55.476 INFO:teuthology.orchestra.run.smithi012.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 54 kB/s | 19 kB 00:00 2024-05-17T22:23:55.478 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-17T22:23:55.508 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-17T22:23:55.546 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-17T22:23:55.592 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-17T22:23:55.594 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-17T22:23:55.628 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-17T22:23:55.637 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-17T22:23:55.670 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-17T22:23:55.685 INFO:teuthology.orchestra.run.smithi012.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 515 kB/s | 309 kB 00:00 2024-05-17T22:23:55.713 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-17T22:23:55.720 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-17T22:23:55.735 INFO:teuthology.orchestra.run.smithi012.stdout:(107/156): abseil-cpp-20211102.0-3.el9.x86_64.r 608 kB/s | 548 kB 00:00 2024-05-17T22:23:55.785 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-17T22:23:55.849 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-17T22:23:55.858 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-17T22:23:55.868 INFO:teuthology.orchestra.run.smithi012.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 142 kB/s | 26 kB 00:00 2024-05-17T22:23:55.888 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-17T22:23:55.893 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-17T22:23:55.902 INFO:teuthology.orchestra.run.smithi012.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 291 kB/s | 49 kB 00:00 2024-05-17T22:23:55.923 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-17T22:23:55.925 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-17T22:23:55.965 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-17T22:23:55.972 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-17T22:23:56.008 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-17T22:23:56.019 INFO:teuthology.orchestra.run.smithi012.stdout:(110/156): libarrow-9.0.0-11.el9.x86_64.rpm 8.1 MB/s | 4.4 MB 00:00 2024-05-17T22:23:56.045 INFO:teuthology.orchestra.run.smithi012.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 386 kB/s | 67 kB 00:00 2024-05-17T22:23:56.051 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-17T22:23:56.059 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-17T22:23:56.070 INFO:teuthology.orchestra.run.smithi012.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 907 kB/s | 151 kB 00:00 2024-05-17T22:23:56.090 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-17T22:23:56.093 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-17T22:23:56.130 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-17T22:23:56.153 INFO:teuthology.orchestra.run.smithi012.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.1 MB/s | 839 kB 00:00 2024-05-17T22:23:56.156 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-17T22:23:56.180 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-17T22:23:56.229 INFO:teuthology.orchestra.run.smithi012.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 3.4 MB/s | 548 kB 00:00 2024-05-17T22:23:56.230 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-17T22:23:56.246 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-17T22:23:56.254 INFO:teuthology.orchestra.run.smithi012.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 236 kB/s | 43 kB 00:00 2024-05-17T22:23:56.273 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-17T22:23:56.279 INFO:teuthology.orchestra.run.smithi012.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 256 kB/s | 32 kB 00:00 2024-05-17T22:23:56.304 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-17T22:23:56.329 INFO:teuthology.orchestra.run.smithi012.stdout:(117/156): python3-certifi-2021.10.8-2.el9.noar 149 kB/s | 15 kB 00:00 2024-05-17T22:23:56.345 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-17T22:23:56.363 INFO:teuthology.orchestra.run.smithi012.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-05-17T22:23:56.373 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-17T22:23:56.397 INFO:teuthology.orchestra.run.smithi012.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-05-17T22:23:56.412 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-17T22:23:56.440 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-17T22:23:56.463 INFO:teuthology.orchestra.run.smithi012.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.2 MB/s | 303 kB 00:00 2024-05-17T22:23:56.470 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-17T22:23:56.489 INFO:teuthology.orchestra.run.smithi012.stdout:(121/156): python3-colorama-0.4.4-12.el9.noarch 281 kB/s | 35 kB 00:00 2024-05-17T22:23:56.510 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-17T22:23:56.531 INFO:teuthology.orchestra.run.smithi012.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 1.5 MB/s | 206 kB 00:00 2024-05-17T22:23:56.544 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-17T22:23:56.550 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-17T22:23:56.568 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-17T22:23:56.583 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-17T22:23:56.623 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-17T22:23:56.643 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-17T22:23:56.654 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-17T22:23:56.681 INFO:teuthology.orchestra.run.smithi012.stdout:(123/156): python3-grpcio-1.46.7-10.el9.x86_64. 9.4 MB/s | 2.0 MB 00:00 2024-05-17T22:23:56.684 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-17T22:23:56.706 INFO:teuthology.orchestra.run.smithi012.stdout:(124/156): python3-grpcio-tools-1.46.7-10.el9.x 665 kB/s | 144 kB 00:00 2024-05-17T22:23:56.718 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-17T22:23:56.723 INFO:teuthology.orchestra.run.smithi012.stdout:(125/156): python3-jaraco-8.2.1-3.el9.noarch.rp 56 kB/s | 11 kB 00:00 2024-05-17T22:23:56.752 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-17T22:23:56.770 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-17T22:23:56.793 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-17T22:23:56.798 INFO:teuthology.orchestra.run.smithi012.stdout:(126/156): python3-jaraco-classes-3.2.1-5.el9.n 151 kB/s | 18 kB 00:00 2024-05-17T22:23:56.809 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-17T22:23:56.824 INFO:teuthology.orchestra.run.smithi012.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 231 kB/s | 23 kB 00:00 2024-05-17T22:23:56.828 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-17T22:23:56.841 INFO:teuthology.orchestra.run.smithi012.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 166 kB/s | 19 kB 00:00 2024-05-17T22:23:56.899 INFO:teuthology.orchestra.run.smithi012.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 196 kB/s | 20 kB 00:00 2024-05-17T22:23:56.915 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-17T22:23:56.937 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-17T22:23:56.966 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-17T22:23:56.980 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-17T22:23:56.983 INFO:teuthology.orchestra.run.smithi012.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 6.4 MB/s | 1.0 MB 00:00 2024-05-17T22:23:56.997 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-17T22:23:57.008 INFO:teuthology.orchestra.run.smithi012.stdout:(131/156): python3-logutils-0.3.5-21.el9.noarch 277 kB/s | 46 kB 00:00 2024-05-17T22:23:57.019 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-17T22:23:57.033 INFO:teuthology.orchestra.run.smithi012.stdout:(132/156): python3-more-itertools-8.12.0-2.el9. 587 kB/s | 79 kB 00:00 2024-05-17T22:23:57.035 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-17T22:23:57.058 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-17T22:23:57.088 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-17T22:23:57.100 INFO:teuthology.orchestra.run.smithi012.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 494 kB/s | 58 kB 00:00 2024-05-17T22:23:57.122 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-17T22:23:57.133 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-17T22:23:57.156 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-17T22:23:57.159 INFO:teuthology.orchestra.run.smithi012.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-05-17T22:23:57.184 INFO:teuthology.orchestra.run.smithi012.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 109 kB/s | 16 kB 00:00 2024-05-17T22:23:57.191 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-17T22:23:57.198 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-17T22:23:57.210 INFO:teuthology.orchestra.run.smithi012.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 827 kB/s | 90 kB 00:00 2024-05-17T22:23:57.232 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-17T22:23:57.268 INFO:teuthology.orchestra.run.smithi012.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 259 kB/s | 28 kB 00:00 2024-05-17T22:23:57.294 INFO:teuthology.orchestra.run.smithi012.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 283 kB/s | 31 kB 00:00 2024-05-17T22:23:57.320 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-17T22:23:57.344 INFO:teuthology.orchestra.run.smithi012.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-05-17T22:23:57.362 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-17T22:23:57.377 INFO:teuthology.orchestra.run.smithi012.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 545 kB/s | 59 kB 00:00 2024-05-17T22:23:57.393 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-17T22:23:57.403 INFO:teuthology.orchestra.run.smithi012.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 329 kB/s | 36 kB 00:00 2024-05-17T22:23:57.432 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-17T22:23:57.461 INFO:teuthology.orchestra.run.smithi012.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 435 kB/s | 51 kB 00:00 2024-05-17T22:23:57.503 INFO:teuthology.orchestra.run.smithi012.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-05-17T22:23:57.528 INFO:teuthology.orchestra.run.smithi012.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 715 kB/s | 90 kB 00:00 2024-05-17T22:23:57.531 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-17T22:23:57.587 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-17T22:23:57.587 INFO:teuthology.orchestra.run.smithi012.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-05-17T22:23:57.612 INFO:teuthology.orchestra.run.smithi012.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 184 kB/s | 20 kB 00:00 2024-05-17T22:23:57.638 INFO:teuthology.orchestra.run.smithi012.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-05-17T22:23:57.738 INFO:teuthology.orchestra.run.smithi012.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-05-17T22:23:57.915 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-17T22:23:57.938 INFO:teuthology.orchestra.run.smithi012.stdout:(149/156): NetworkManager-1.47.90-1.el9.x86_64. 12 MB/s | 2.3 MB 00:00 2024-05-17T22:23:57.962 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-17T22:23:57.988 INFO:teuthology.orchestra.run.smithi012.stdout:(150/156): librbd1-19.0.0-3794.g931809a9.el9.x8 8.8 MB/s | 3.1 MB 00:00 2024-05-17T22:23:57.996 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-17T22:23:58.039 INFO:teuthology.orchestra.run.smithi012.stdout:(151/156): NetworkManager-libnm-1.47.90-1.el9.x 18 MB/s | 1.8 MB 00:00 2024-05-17T22:23:58.072 INFO:teuthology.orchestra.run.smithi012.stdout:(152/156): NetworkManager-team-1.47.90-1.el9.x8 497 kB/s | 41 kB 00:00 2024-05-17T22:23:58.097 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-17T22:23:58.098 INFO:teuthology.orchestra.run.smithi012.stdout:(153/156): NetworkManager-tui-1.47.90-1.el9.x86 4.1 MB/s | 248 kB 00:00 2024-05-17T22:23:58.162 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-17T22:23:58.173 INFO:teuthology.orchestra.run.smithi012.stdout:(154/156): librados2-19.0.0-3794.g931809a9.el9. 6.0 MB/s | 3.3 MB 00:00 2024-05-17T22:23:58.223 INFO:teuthology.orchestra.run.smithi012.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 19 MB/s | 2.4 MB 00:00 2024-05-17T22:23:58.251 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-17T22:23:58.256 INFO:teuthology.orchestra.run.smithi012.stdout:(156/156): openssl-3.2.1-1.el9.x86_64.rpm 7.3 MB/s | 1.3 MB 00:00 2024-05-17T22:23:58.263 INFO:teuthology.orchestra.run.smithi012.stdout:-------------------------------------------------------------------------------- 2024-05-17T22:23:58.263 INFO:teuthology.orchestra.run.smithi012.stdout:Total 13 MB/s | 219 MB 00:16 2024-05-17T22:23:58.290 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-17T22:23:58.324 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-17T22:23:58.367 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-17T22:23:58.400 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-17T22:23:58.467 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-17T22:23:58.503 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-17T22:23:58.513 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-17T22:23:58.575 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-17T22:23:58.591 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-17T22:23:58.665 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-17T22:23:58.694 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-17T22:23:58.865 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-17T22:23:58.900 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-17T22:23:58.978 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-17T22:23:59.236 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-17T22:23:59.345 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-17T22:23:59.508 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction check 2024-05-17T22:23:59.632 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-17T22:23:59.676 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction check succeeded. 2024-05-17T22:23:59.676 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction test 2024-05-17T22:23:59.749 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-17T22:24:00.574 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-17T22:24:00.650 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-17T22:24:00.677 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-17T22:24:00.710 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-17T22:24:01.009 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-17T22:24:01.027 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-17T22:24:01.065 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : librbd1-2:19.0.0-3794.g931809a9.el9.x86_64 101/164 2024-05-17T22:24:01.103 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-17T22:24:01.131 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-17T22:24:01.140 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librbd1-2:19.0.0-3794.g931809a9.el9.x86_64 101/164 2024-05-17T22:24:01.164 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-17T22:24:01.170 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rbd-2:19.0.0-3794.g931809a9.el9.x86_64 102/164 2024-05-17T22:24:01.206 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-17T22:24:01.484 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-17T22:24:01.518 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : librbd1-2:19.0.0-3794.g931809a9.el9.x86_64 101/164 2024-05-17T22:24:01.609 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librbd1-2:19.0.0-3794.g931809a9.el9.x86_64 101/164 2024-05-17T22:24:01.640 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rbd-2:19.0.0-3794.g931809a9.el9.x86_64 102/164 2024-05-17T22:24:01.676 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-17T22:24:01.694 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-17T22:24:01.718 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librgw2-2:19.0.0-3794.g931809a9.el9.x86_64 105/164 2024-05-17T22:24:01.789 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librgw2-2:19.0.0-3794.g931809a9.el9.x86_64 105/164 2024-05-17T22:24:01.811 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rgw-2:19.0.0-3794.g931809a9.el9.x86_64 106/164 2024-05-17T22:24:01.917 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction test succeeded. 2024-05-17T22:24:01.917 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction 2024-05-17T22:24:02.166 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-17T22:24:02.205 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librgw2-2:19.0.0-3794.g931809a9.el9.x86_64 105/164 2024-05-17T22:24:02.276 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librgw2-2:19.0.0-3794.g931809a9.el9.x86_64 105/164 2024-05-17T22:24:02.306 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rgw-2:19.0.0-3794.g931809a9.el9.x86_64 106/164 2024-05-17T22:24:03.968 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-common-2:19.0.0-3794.g931809a9.el9.x86_64 107/164 2024-05-17T22:24:03.999 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-common-2:19.0.0-3794.g931809a9.el9.x86_64 107/164 2024-05-17T22:24:04.554 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-common-2:19.0.0-3794.g931809a9.el9.x86_64 107/164 2024-05-17T22:24:04.581 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-common-2:19.0.0-3794.g931809a9.el9.x86_64 107/164 2024-05-17T22:24:04.594 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-base-2:19.0.0-3794.g931809a9.el9.x86_64 108/164 2024-05-17T22:24:04.619 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-common-2:19.0.0-3794.g931809a9.el9.x86_64 107/164 2024-05-17T22:24:04.668 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-base-2:19.0.0-3794.g931809a9.el9.x86_64 108/164 2024-05-17T22:24:04.668 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-17T22:24:04.668 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-17T22:24:04.668 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-17T22:24:04.712 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-selinux-2:19.0.0-3794.g931809a9.el9.x86_64 109/164 2024-05-17T22:24:05.175 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-common-2:19.0.0-3794.g931809a9.el9.x86_64 107/164 2024-05-17T22:24:05.214 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-base-2:19.0.0-3794.g931809a9.el9.x86_64 108/164 2024-05-17T22:24:05.305 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-base-2:19.0.0-3794.g931809a9.el9.x86_64 108/164 2024-05-17T22:24:05.306 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-17T22:24:05.306 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-17T22:24:05.306 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-17T22:24:05.340 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-selinux-2:19.0.0-3794.g931809a9.el9.x86_64 109/164 2024-05-17T22:24:08.215 INFO:teuthology.orchestra.run.smithi012.stdout: Preparing : 1/1 2024-05-17T22:24:08.303 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-17T22:24:08.341 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-17T22:24:08.424 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-17T22:24:08.462 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-17T22:24:08.487 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-17T22:24:08.546 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-17T22:24:08.901 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-17T22:24:09.013 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : librados2-2:19.0.0-3794.g931809a9.el9.x86_64 8/164 2024-05-17T22:24:09.142 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librados2-2:19.0.0-3794.g931809a9.el9.x86_64 8/164 2024-05-17T22:24:09.189 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libcephfs2-2:19.0.0-3794.g931809a9.el9.x86_64 9/164 2024-05-17T22:24:09.282 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libcephfs2-2:19.0.0-3794.g931809a9.el9.x86_64 9/164 2024-05-17T22:24:09.303 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-rados-2:19.0.0-3794.g931809a9.el9.x86_64 10/164 2024-05-17T22:24:09.497 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-17T22:24:09.558 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-17T22:24:09.585 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libcephsqlite-2:19.0.0-3794.g931809a9.el9.x86_64 13/164 2024-05-17T22:24:09.685 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3794.g931809a9.el9.x86_64 13/164 2024-05-17T22:24:09.711 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libradosstriper1-2:19.0.0-3794.g931809a9.el9.x86 14/164 2024-05-17T22:24:09.774 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3794.g931809a9.el9.x86 14/164 2024-05-17T22:24:09.796 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-17T22:24:09.822 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-17T22:24:10.015 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-17T22:24:10.184 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-17T22:24:10.343 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-17T22:24:10.404 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-17T22:24:10.504 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-17T22:24:10.580 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-17T22:24:10.787 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-17T22:24:10.896 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-17T22:24:10.926 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-17T22:24:10.961 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-17T22:24:11.036 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-17T22:24:11.078 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-17T22:24:11.121 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-17T22:24:11.155 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-17T22:24:11.181 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-17T22:24:11.252 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-17T22:24:11.312 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-17T22:24:11.342 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-17T22:24:11.377 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-17T22:24:11.456 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-17T22:24:11.488 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-ceph-common-2:19.0.0-3794.g931809a9.el9. 32/164 2024-05-17T22:24:11.529 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-ceph-argparse-2:19.0.0-3794.g931809a9.el 33/164 2024-05-17T22:24:11.560 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cephfs-2:19.0.0-3794.g931809a9.el9.x86_6 34/164 2024-05-17T22:24:11.635 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-17T22:24:11.684 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : luarocks-3.9.2-5.el9.noarch 36/164 2024-05-17T22:24:11.762 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-17T22:24:11.907 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-17T22:24:11.963 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-17T22:24:12.522 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-17T22:24:12.602 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-17T22:24:12.636 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-17T22:24:12.679 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-17T22:24:12.717 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : librados-devel-2:19.0.0-3794.g931809a9.el9.x86_6 44/164 2024-05-17T22:24:12.761 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-17T22:24:12.914 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-17T22:24:12.970 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-17T22:24:13.065 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-17T22:24:13.127 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-17T22:24:13.201 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-17T22:24:13.246 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-17T22:24:13.283 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-17T22:24:13.331 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-17T22:24:13.365 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-17T22:24:13.409 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-17T22:24:13.451 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-17T22:24:13.593 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-17T22:24:13.816 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-17T22:24:13.854 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-17T22:24:13.897 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-17T22:24:13.995 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-17T22:24:14.028 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-17T22:24:14.079 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-17T22:24:14.118 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-17T22:24:14.143 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-17T22:24:14.251 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-17T22:24:14.384 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-17T22:24:14.426 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-17T22:24:14.463 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-17T22:24:14.496 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-17T22:24:14.536 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-17T22:24:14.576 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-17T22:24:14.605 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-17T22:24:14.640 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-17T22:24:14.673 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-17T22:24:14.715 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-17T22:24:14.749 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-17T22:24:14.839 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-17T22:24:14.889 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-17T22:24:14.927 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-17T22:24:14.975 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-17T22:24:15.085 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-17T22:24:15.146 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-17T22:24:15.917 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-17T22:24:15.970 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-17T22:24:16.003 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-17T22:24:16.106 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-17T22:24:16.187 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-17T22:24:16.278 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-17T22:24:16.314 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-17T22:24:16.340 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3794.g931809a9.el9.x86_64 109/164 2024-05-17T22:24:16.340 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /sys 2024-05-17T22:24:16.340 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /proc 2024-05-17T22:24:16.340 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /mnt 2024-05-17T22:24:16.340 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /var/tmp 2024-05-17T22:24:16.340 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /home 2024-05-17T22:24:16.340 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /root 2024-05-17T22:24:16.340 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /tmp 2024-05-17T22:24:16.340 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-17T22:24:16.497 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-17T22:24:16.517 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-volume-2:19.0.0-3794.g931809a9.el9.noarch 110/164 2024-05-17T22:24:16.528 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-17T22:24:16.533 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-volume-2:19.0.0-3794.g931809a9.el9.noarch 110/164 2024-05-17T22:24:16.533 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T22:24:16.533 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-17T22:24:16.533 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-17T22:24:16.610 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-17T22:24:17.291 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-17T22:24:17.331 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3794.g931809a9.el9.x86_64 109/164 2024-05-17T22:24:17.331 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /sys 2024-05-17T22:24:17.331 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /proc 2024-05-17T22:24:17.332 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /mnt 2024-05-17T22:24:17.332 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /var/tmp 2024-05-17T22:24:17.332 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /home 2024-05-17T22:24:17.332 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /root 2024-05-17T22:24:17.332 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /tmp 2024-05-17T22:24:17.332 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-17T22:24:17.413 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-17T22:24:17.503 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-volume-2:19.0.0-3794.g931809a9.el9.noarch 110/164 2024-05-17T22:24:17.518 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-volume-2:19.0.0-3794.g931809a9.el9.noarch 110/164 2024-05-17T22:24:17.519 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T22:24:17.519 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-17T22:24:17.519 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-17T22:24:18.112 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-osd-2:19.0.0-3794.g931809a9.el9.x86_64 111/164 2024-05-17T22:24:18.138 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-osd-2:19.0.0-3794.g931809a9.el9.x86_64 111/164 2024-05-17T22:24:18.138 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T22:24:18.138 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-17T22:24:18.138 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-17T22:24:18.138 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-17T22:24:18.138 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-17T22:24:18.362 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mds-2:19.0.0-3794.g931809a9.el9.x86_64 112/164 2024-05-17T22:24:18.385 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mds-2:19.0.0-3794.g931809a9.el9.x86_64 112/164 2024-05-17T22:24:18.385 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T22:24:18.385 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-17T22:24:18.385 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-17T22:24:18.385 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-17T22:24:18.385 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-17T22:24:18.747 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-17T22:24:18.838 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mon-2:19.0.0-3794.g931809a9.el9.x86_64 113/164 2024-05-17T22:24:18.841 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-17T22:24:18.861 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mon-2:19.0.0-3794.g931809a9.el9.x86_64 113/164 2024-05-17T22:24:18.861 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T22:24:18.861 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-17T22:24:18.861 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-17T22:24:18.861 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-17T22:24:18.861 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-17T22:24:18.869 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-17T22:24:18.902 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-17T22:24:18.960 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-17T22:24:19.025 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-17T22:24:19.081 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-17T22:24:19.106 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-osd-2:19.0.0-3794.g931809a9.el9.x86_64 111/164 2024-05-17T22:24:19.131 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-osd-2:19.0.0-3794.g931809a9.el9.x86_64 111/164 2024-05-17T22:24:19.131 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T22:24:19.131 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-17T22:24:19.131 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-17T22:24:19.131 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-17T22:24:19.132 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-17T22:24:19.236 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-17T22:24:19.262 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-17T22:24:19.282 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : librbd1-2:19.0.0-3794.g931809a9.el9.x86_64 101/164 2024-05-17T22:24:19.310 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-17T22:24:19.357 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librbd1-2:19.0.0-3794.g931809a9.el9.x86_64 101/164 2024-05-17T22:24:19.364 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mds-2:19.0.0-3794.g931809a9.el9.x86_64 112/164 2024-05-17T22:24:19.387 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mds-2:19.0.0-3794.g931809a9.el9.x86_64 112/164 2024-05-17T22:24:19.387 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T22:24:19.387 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-17T22:24:19.387 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-17T22:24:19.387 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-17T22:24:19.387 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-17T22:24:19.387 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-rbd-2:19.0.0-3794.g931809a9.el9.x86_64 102/164 2024-05-17T22:24:19.397 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-17T22:24:19.427 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-17T22:24:19.486 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-17T22:24:19.626 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-17T22:24:19.660 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-17T22:24:19.701 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-17T22:24:19.737 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-17T22:24:19.848 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mon-2:19.0.0-3794.g931809a9.el9.x86_64 113/164 2024-05-17T22:24:19.871 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mon-2:19.0.0-3794.g931809a9.el9.x86_64 113/164 2024-05-17T22:24:19.871 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T22:24:19.871 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-17T22:24:19.871 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-17T22:24:19.872 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-17T22:24:19.872 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-17T22:24:19.940 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-17T22:24:19.968 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : librgw2-2:19.0.0-3794.g931809a9.el9.x86_64 105/164 2024-05-17T22:24:19.987 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-17T22:24:20.040 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librgw2-2:19.0.0-3794.g931809a9.el9.x86_64 105/164 2024-05-17T22:24:20.044 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-17T22:24:20.070 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-rgw-2:19.0.0-3794.g931809a9.el9.x86_64 106/164 2024-05-17T22:24:20.107 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-17T22:24:20.180 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: cephadm-2:19.0.0-3794.g931809a9.el9.noarch 125/164 2024-05-17T22:24:20.210 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephadm-2:19.0.0-3794.g931809a9.el9.noarch 125/164 2024-05-17T22:24:20.243 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3794.g931809a9.e 126/164 2024-05-17T22:24:20.295 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-17T22:24:20.305 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3794.g931809a9. 127/164 2024-05-17T22:24:20.334 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3794.g931809a9.el9.noa 128/164 2024-05-17T22:24:20.359 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-17T22:24:20.437 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-17T22:24:20.526 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-17T22:24:20.677 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-17T22:24:20.717 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-17T22:24:20.758 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-17T22:24:20.786 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-17T22:24:21.240 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: cephadm-2:19.0.0-3794.g931809a9.el9.noarch 125/164 2024-05-17T22:24:21.269 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : cephadm-2:19.0.0-3794.g931809a9.el9.noarch 125/164 2024-05-17T22:24:21.301 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3794.g931809a9.e 126/164 2024-05-17T22:24:21.363 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3794.g931809a9. 127/164 2024-05-17T22:24:21.376 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3794.g931809a9.el9.noa 128/164 2024-05-17T22:24:21.392 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3794.g931809a9.el9.noa 128/164 2024-05-17T22:24:21.410 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3794.g931809a9.el9.n 129/164 2024-05-17T22:24:22.373 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3794.g931809a9.el9.n 129/164 2024-05-17T22:24:22.403 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3794.g931 130/164 2024-05-17T22:24:22.426 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-common-2:19.0.0-3794.g931809a9.el9.x86_64 107/164 2024-05-17T22:24:22.440 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3794.g931809a9.el9.noa 128/164 2024-05-17T22:24:22.466 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-common-2:19.0.0-3794.g931809a9.el9.x86_64 107/164 2024-05-17T22:24:22.478 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3794.g931809a9.el9.n 129/164 2024-05-17T22:24:22.514 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3794.g931 130/164 2024-05-17T22:24:22.683 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3794.g931809a9.el 131/164 2024-05-17T22:24:22.713 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-2:19.0.0-3794.g931809a9.el9.x86_64 132/164 2024-05-17T22:24:22.737 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3794.g931809a9.el9.x86_64 132/164 2024-05-17T22:24:22.737 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T22:24:22.737 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-17T22:24:22.737 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-17T22:24:22.738 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-17T22:24:22.738 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-17T22:24:22.789 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-rook-2:19.0.0-3794.g931809a9.el9.noarch 133/164 2024-05-17T22:24:22.800 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3794.g931809a9.el9.noarch 133/164 2024-05-17T22:24:22.899 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-2:19.0.0-3794.g931809a9.el9.x86_64 134/164 2024-05-17T22:24:22.946 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-fuse-2:19.0.0-3794.g931809a9.el9.x86_64 135/164 2024-05-17T22:24:23.056 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-common-2:19.0.0-3794.g931809a9.el9.x86_64 107/164 2024-05-17T22:24:23.095 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-base-2:19.0.0-3794.g931809a9.el9.x86_64 108/164 2024-05-17T22:24:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-base-2:19.0.0-3794.g931809a9.el9.x86_64 108/164 2024-05-17T22:24:23.189 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-17T22:24:23.189 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-17T22:24:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:24:23.221 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-selinux-2:19.0.0-3794.g931809a9.el9.x86_64 109/164 2024-05-17T22:24:23.443 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3794.g931809a9.el9.n 129/164 2024-05-17T22:24:23.471 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3794.g931 130/164 2024-05-17T22:24:23.583 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3794.g931 130/164 2024-05-17T22:24:23.743 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3794.g931809a9.el 131/164 2024-05-17T22:24:23.782 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-2:19.0.0-3794.g931809a9.el9.x86_64 132/164 2024-05-17T22:24:23.805 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3794.g931809a9.el9.x86_64 132/164 2024-05-17T22:24:23.805 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T22:24:23.805 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-17T22:24:23.805 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-17T22:24:23.805 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-17T22:24:23.805 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-17T22:24:23.858 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-rook-2:19.0.0-3794.g931809a9.el9.noarch 133/164 2024-05-17T22:24:23.869 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3794.g931809a9.el9.noarch 133/164 2024-05-17T22:24:23.903 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-17T22:24:23.942 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-radosgw-2:19.0.0-3794.g931809a9.el9.x86_64 137/164 2024-05-17T22:24:23.965 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3794.g931809a9.el9.x86_64 137/164 2024-05-17T22:24:23.965 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T22:24:23.965 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-17T22:24:23.965 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-17T22:24:23.965 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-17T22:24:23.966 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-17T22:24:23.969 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-2:19.0.0-3794.g931809a9.el9.x86_64 134/164 2024-05-17T22:24:24.009 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3794.g93180 138/164 2024-05-17T22:24:24.014 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-fuse-2:19.0.0-3794.g931809a9.el9.x86_64 135/164 2024-05-17T22:24:24.032 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3794.g93180 138/164 2024-05-17T22:24:24.032 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T22:24:24.032 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-17T22:24:24.032 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-17T22:24:24.085 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephfs-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 139/164 2024-05-17T22:24:24.108 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 139/164 2024-05-17T22:24:24.108 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T22:24:24.108 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-17T22:24:24.108 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-17T22:24:24.108 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-17T22:24:24.108 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-17T22:24:24.419 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 140/164 2024-05-17T22:24:24.442 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 140/164 2024-05-17T22:24:24.442 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T22:24:24.442 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-17T22:24:24.442 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-17T22:24:24.442 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-17T22:24:24.442 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-17T22:24:24.954 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-17T22:24:24.994 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-radosgw-2:19.0.0-3794.g931809a9.el9.x86_64 137/164 2024-05-17T22:24:25.016 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3794.g931809a9.el9.x86_64 137/164 2024-05-17T22:24:25.016 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T22:24:25.017 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-17T22:24:25.017 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-17T22:24:25.017 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-17T22:24:25.017 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-17T22:24:25.062 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3794.g93180 138/164 2024-05-17T22:24:25.083 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3794.g93180 138/164 2024-05-17T22:24:25.084 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T22:24:25.084 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-17T22:24:25.084 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-17T22:24:25.129 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : cephfs-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 139/164 2024-05-17T22:24:25.151 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 139/164 2024-05-17T22:24:25.152 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T22:24:25.152 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-17T22:24:25.152 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-17T22:24:25.152 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-17T22:24:25.152 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-17T22:24:25.463 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : rbd-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 140/164 2024-05-17T22:24:25.485 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 140/164 2024-05-17T22:24:25.486 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T22:24:25.486 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-17T22:24:25.486 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-17T22:24:25.486 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-17T22:24:25.486 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-17T22:24:28.672 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-test-2:19.0.0-3794.g931809a9.el9.x86_64 141/164 2024-05-17T22:24:28.714 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-fuse-2:19.0.0-3794.g931809a9.el9.x86_64 142/164 2024-05-17T22:24:28.763 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-nbd-2:19.0.0-3794.g931809a9.el9.x86_64 143/164 2024-05-17T22:24:28.803 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-17T22:24:28.831 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-17T22:24:29.020 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephfs-shell-2:19.0.0-3794.g931809a9.el9.x86_64 146/164 2024-05-17T22:24:29.060 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-17T22:24:29.091 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephfs-devel-2:19.0.0-3794.g931809a9.el9.x86_ 148/164 2024-05-17T22:24:29.120 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 149/164 2024-05-17T22:24:29.167 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 150/164 2024-05-17T22:24:29.726 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-test-2:19.0.0-3794.g931809a9.el9.x86_64 141/164 2024-05-17T22:24:29.745 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-17T22:24:29.768 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : rbd-fuse-2:19.0.0-3794.g931809a9.el9.x86_64 142/164 2024-05-17T22:24:29.782 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-17T22:24:29.802 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-17T22:24:29.809 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : rbd-nbd-2:19.0.0-3794.g931809a9.el9.x86_64 143/164 2024-05-17T22:24:29.841 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-17T22:24:29.869 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-17T22:24:30.056 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : cephfs-shell-2:19.0.0-3794.g931809a9.el9.x86_64 146/164 2024-05-17T22:24:30.099 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-17T22:24:30.130 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libcephfs-devel-2:19.0.0-3794.g931809a9.el9.x86_ 148/164 2024-05-17T22:24:30.159 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 149/164 2024-05-17T22:24:30.198 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 150/164 2024-05-17T22:24:30.653 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephfs-top-2:19.0.0-3794.g931809a9.el9.noarch 152/164 2024-05-17T22:24:30.745 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-17T22:24:30.787 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-17T22:24:30.811 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-17T22:24:30.856 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-17T22:24:30.870 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-17T22:24:30.876 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-17T22:24:30.898 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-17T22:24:30.924 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-17T22:24:30.925 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-17T22:24:31.009 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-17T22:24:31.051 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-17T22:24:31.051 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-17T22:24:31.135 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-17T22:24:31.160 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-17T22:24:31.165 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-17T22:24:31.165 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-17T22:24:31.345 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-17T22:24:31.345 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-17T22:24:31.413 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-17T22:24:31.484 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-17T22:24:31.719 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : cephfs-top-2:19.0.0-3794.g931809a9.el9.noarch 152/164 2024-05-17T22:24:31.814 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-17T22:24:31.855 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-17T22:24:31.939 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-17T22:24:31.985 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-17T22:24:32.011 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-17T22:24:32.011 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-17T22:24:32.104 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-17T22:24:32.137 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-17T22:24:32.137 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-17T22:24:32.222 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-17T22:24:32.247 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-17T22:24:32.252 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-17T22:24:32.252 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-17T22:24:32.432 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-17T22:24:32.432 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-17T22:24:32.517 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-17T22:24:32.581 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-17T22:24:35.143 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3794.g931809a9.el9.x86_64 109/164 2024-05-17T22:24:35.143 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /sys 2024-05-17T22:24:35.143 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /proc 2024-05-17T22:24:35.143 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /mnt 2024-05-17T22:24:35.143 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /var/tmp 2024-05-17T22:24:35.143 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /home 2024-05-17T22:24:35.143 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /root 2024-05-17T22:24:35.143 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /tmp 2024-05-17T22:24:35.144 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:24:35.320 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-volume-2:19.0.0-3794.g931809a9.el9.noarch 110/164 2024-05-17T22:24:35.337 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-volume-2:19.0.0-3794.g931809a9.el9.noarch 110/164 2024-05-17T22:24:35.337 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T22:24:35.337 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-17T22:24:35.337 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:24:36.982 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-osd-2:19.0.0-3794.g931809a9.el9.x86_64 111/164 2024-05-17T22:24:37.011 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-osd-2:19.0.0-3794.g931809a9.el9.x86_64 111/164 2024-05-17T22:24:37.011 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T22:24:37.011 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-17T22:24:37.011 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-17T22:24:37.011 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-17T22:24:37.012 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:24:37.249 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mds-2:19.0.0-3794.g931809a9.el9.x86_64 112/164 2024-05-17T22:24:37.273 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-17T22:24:37.274 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-2:19.0.0-3794.g931809a9.el9.x86_64 1/164 2024-05-17T22:24:37.274 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-base-2:19.0.0-3794.g931809a9.el9.x86_64 2/164 2024-05-17T22:24:37.274 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-common-2:19.0.0-3794.g931809a9.el9.x86_64 3/164 2024-05-17T22:24:37.274 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-fuse-2:19.0.0-3794.g931809a9.el9.x86_64 4/164 2024-05-17T22:24:37.274 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3794.g93180 5/164 2024-05-17T22:24:37.274 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mds-2:19.0.0-3794.g931809a9.el9.x86_64 6/164 2024-05-17T22:24:37.274 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-2:19.0.0-3794.g931809a9.el9.x86_64 7/164 2024-05-17T22:24:37.274 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mon-2:19.0.0-3794.g931809a9.el9.x86_64 8/164 2024-05-17T22:24:37.274 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-osd-2:19.0.0-3794.g931809a9.el9.x86_64 9/164 2024-05-17T22:24:37.274 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-radosgw-2:19.0.0-3794.g931809a9.el9.x86_64 10/164 2024-05-17T22:24:37.274 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-selinux-2:19.0.0-3794.g931809a9.el9.x86_64 11/164 2024-05-17T22:24:37.274 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-test-2:19.0.0-3794.g931809a9.el9.x86_64 12/164 2024-05-17T22:24:37.274 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephfs-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 13/164 2024-05-17T22:24:37.275 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephfs-shell-2:19.0.0-3794.g931809a9.el9.x86_64 14/164 2024-05-17T22:24:37.275 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephfs-devel-2:19.0.0-3794.g931809a9.el9.x86_ 15/164 2024-05-17T22:24:37.275 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephfs2-2:19.0.0-3794.g931809a9.el9.x86_64 16/164 2024-05-17T22:24:37.275 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephsqlite-2:19.0.0-3794.g931809a9.el9.x86_64 17/164 2024-05-17T22:24:37.275 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados-devel-2:19.0.0-3794.g931809a9.el9.x86_6 18/164 2024-05-17T22:24:37.275 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libradosstriper1-2:19.0.0-3794.g931809a9.el9.x86 19/164 2024-05-17T22:24:37.275 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librgw2-2:19.0.0-3794.g931809a9.el9.x86_64 20/164 2024-05-17T22:24:37.275 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3794.g931809a9.el 21/164 2024-05-17T22:24:37.275 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ceph-common-2:19.0.0-3794.g931809a9.el9. 22/164 2024-05-17T22:24:37.275 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cephfs-2:19.0.0-3794.g931809a9.el9.x86_6 23/164 2024-05-17T22:24:37.275 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rados-2:19.0.0-3794.g931809a9.el9.x86_64 24/164 2024-05-17T22:24:37.275 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rbd-2:19.0.0-3794.g931809a9.el9.x86_64 25/164 2024-05-17T22:24:37.275 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rgw-2:19.0.0-3794.g931809a9.el9.x86_64 26/164 2024-05-17T22:24:37.275 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-fuse-2:19.0.0-3794.g931809a9.el9.x86_64 27/164 2024-05-17T22:24:37.275 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 28/164 2024-05-17T22:24:37.276 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-nbd-2:19.0.0-3794.g931809a9.el9.x86_64 29/164 2024-05-17T22:24:37.276 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3794.g931809a9. 30/164 2024-05-17T22:24:37.276 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3794.g931809a9.el9.noa 31/164 2024-05-17T22:24:37.276 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3794.g931809a9.el9.n 32/164 2024-05-17T22:24:37.276 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3794.g931 33/164 2024-05-17T22:24:37.276 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3794.g931809a9.el 34/164 2024-05-17T22:24:37.276 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3794.g931809a9.el9.noarch 35/164 2024-05-17T22:24:37.276 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3794.g931809a9.e 36/164 2024-05-17T22:24:37.276 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-volume-2:19.0.0-3794.g931809a9.el9.noarch 37/164 2024-05-17T22:24:37.276 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephadm-2:19.0.0-3794.g931809a9.el9.noarch 38/164 2024-05-17T22:24:37.276 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephfs-top-2:19.0.0-3794.g931809a9.el9.noarch 39/164 2024-05-17T22:24:37.276 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-17T22:24:37.277 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-17T22:24:37.277 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-17T22:24:37.277 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-17T22:24:37.277 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-17T22:24:37.277 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-17T22:24:37.277 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-17T22:24:37.277 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-17T22:24:37.277 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-17T22:24:37.277 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-17T22:24:37.277 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-17T22:24:37.277 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-17T22:24:37.277 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-17T22:24:37.277 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-17T22:24:37.277 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-17T22:24:37.277 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-17T22:24:37.278 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-17T22:24:37.278 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-17T22:24:37.278 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-17T22:24:37.278 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-17T22:24:37.278 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-17T22:24:37.278 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-17T22:24:37.278 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-17T22:24:37.278 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-17T22:24:37.278 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-17T22:24:37.278 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-17T22:24:37.278 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-17T22:24:37.278 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-17T22:24:37.278 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-17T22:24:37.279 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-17T22:24:37.279 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-17T22:24:37.279 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-17T22:24:37.279 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-17T22:24:37.279 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-17T22:24:37.279 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-17T22:24:37.279 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-17T22:24:37.279 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-17T22:24:37.279 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-17T22:24:37.279 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-17T22:24:37.279 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-17T22:24:37.279 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-17T22:24:37.279 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-17T22:24:37.279 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-17T22:24:37.280 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-17T22:24:37.280 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-17T22:24:37.280 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-17T22:24:37.280 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-17T22:24:37.280 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-17T22:24:37.280 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-17T22:24:37.280 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-17T22:24:37.280 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-17T22:24:37.280 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-17T22:24:37.280 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-17T22:24:37.280 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-17T22:24:37.280 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-17T22:24:37.280 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-17T22:24:37.280 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-17T22:24:37.281 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-17T22:24:37.281 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-17T22:24:37.281 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-17T22:24:37.281 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-17T22:24:37.281 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-17T22:24:37.281 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-17T22:24:37.281 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-17T22:24:37.281 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-17T22:24:37.281 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-17T22:24:37.281 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-17T22:24:37.281 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-17T22:24:37.281 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-17T22:24:37.281 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-17T22:24:37.281 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-17T22:24:37.281 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-17T22:24:37.282 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 112/164 2024-05-17T22:24:37.282 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-17T22:24:37.282 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-17T22:24:37.282 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-17T22:24:37.282 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-17T22:24:37.282 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-17T22:24:37.282 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-17T22:24:37.282 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-17T22:24:37.282 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-17T22:24:37.282 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-17T22:24:37.282 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-17T22:24:37.282 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-17T22:24:37.282 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-17T22:24:37.283 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-17T22:24:37.283 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-17T22:24:37.283 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-17T22:24:37.283 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-17T22:24:37.283 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-17T22:24:37.283 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-17T22:24:37.283 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-17T22:24:37.283 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-17T22:24:37.283 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-17T22:24:37.283 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-17T22:24:37.283 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-17T22:24:37.283 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-17T22:24:37.283 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-17T22:24:37.283 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-17T22:24:37.284 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-17T22:24:37.284 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-17T22:24:37.284 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-17T22:24:37.284 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-17T22:24:37.284 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-17T22:24:37.284 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-17T22:24:37.284 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-17T22:24:37.284 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-17T22:24:37.284 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-17T22:24:37.284 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-17T22:24:37.284 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados2-2:19.0.0-3794.g931809a9.el9.x86_64 149/164 2024-05-17T22:24:37.284 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-17T22:24:37.284 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librbd1-2:19.0.0-3794.g931809a9.el9.x86_64 151/164 2024-05-17T22:24:37.284 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-17T22:24:37.284 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/164 2024-05-17T22:24:37.285 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-17T22:24:37.285 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/164 2024-05-17T22:24:37.285 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-17T22:24:37.285 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/164 2024-05-17T22:24:37.285 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-17T22:24:37.285 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/164 2024-05-17T22:24:37.285 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-17T22:24:37.285 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-17T22:24:37.285 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-17T22:24:37.285 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-17T22:24:37.286 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mds-2:19.0.0-3794.g931809a9.el9.x86_64 112/164 2024-05-17T22:24:37.286 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T22:24:37.286 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-17T22:24:37.286 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-17T22:24:37.286 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-17T22:24:37.286 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:24:37.758 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mon-2:19.0.0-3794.g931809a9.el9.x86_64 113/164 2024-05-17T22:24:37.783 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mon-2:19.0.0-3794.g931809a9.el9.x86_64 113/164 2024-05-17T22:24:37.783 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T22:24:37.784 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-17T22:24:37.784 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-17T22:24:37.784 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-17T22:24:37.784 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:24:37.889 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-17T22:24:37.948 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-17T22:24:38.011 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-17T22:24:38.207 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-17T22:24:38.260 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-17T22:24:38.342 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-17T22:24:38.439 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-17T22:24:38.579 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-17T22:24:38.579 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-2:19.0.0-3794.g931809a9.el9.x86_64 1/164 2024-05-17T22:24:38.579 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-base-2:19.0.0-3794.g931809a9.el9.x86_64 2/164 2024-05-17T22:24:38.580 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-common-2:19.0.0-3794.g931809a9.el9.x86_64 3/164 2024-05-17T22:24:38.580 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-fuse-2:19.0.0-3794.g931809a9.el9.x86_64 4/164 2024-05-17T22:24:38.580 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3794.g93180 5/164 2024-05-17T22:24:38.580 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mds-2:19.0.0-3794.g931809a9.el9.x86_64 6/164 2024-05-17T22:24:38.580 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-2:19.0.0-3794.g931809a9.el9.x86_64 7/164 2024-05-17T22:24:38.580 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mon-2:19.0.0-3794.g931809a9.el9.x86_64 8/164 2024-05-17T22:24:38.580 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-osd-2:19.0.0-3794.g931809a9.el9.x86_64 9/164 2024-05-17T22:24:38.580 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-radosgw-2:19.0.0-3794.g931809a9.el9.x86_64 10/164 2024-05-17T22:24:38.580 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-selinux-2:19.0.0-3794.g931809a9.el9.x86_64 11/164 2024-05-17T22:24:38.580 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-test-2:19.0.0-3794.g931809a9.el9.x86_64 12/164 2024-05-17T22:24:38.580 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : cephfs-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 13/164 2024-05-17T22:24:38.580 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : cephfs-shell-2:19.0.0-3794.g931809a9.el9.x86_64 14/164 2024-05-17T22:24:38.580 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libcephfs-devel-2:19.0.0-3794.g931809a9.el9.x86_ 15/164 2024-05-17T22:24:38.580 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libcephfs2-2:19.0.0-3794.g931809a9.el9.x86_64 16/164 2024-05-17T22:24:38.580 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libcephsqlite-2:19.0.0-3794.g931809a9.el9.x86_64 17/164 2024-05-17T22:24:38.581 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librados-devel-2:19.0.0-3794.g931809a9.el9.x86_6 18/164 2024-05-17T22:24:38.581 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libradosstriper1-2:19.0.0-3794.g931809a9.el9.x86 19/164 2024-05-17T22:24:38.581 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librgw2-2:19.0.0-3794.g931809a9.el9.x86_64 20/164 2024-05-17T22:24:38.581 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3794.g931809a9.el 21/164 2024-05-17T22:24:38.581 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-ceph-common-2:19.0.0-3794.g931809a9.el9. 22/164 2024-05-17T22:24:38.581 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cephfs-2:19.0.0-3794.g931809a9.el9.x86_6 23/164 2024-05-17T22:24:38.581 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rados-2:19.0.0-3794.g931809a9.el9.x86_64 24/164 2024-05-17T22:24:38.581 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rbd-2:19.0.0-3794.g931809a9.el9.x86_64 25/164 2024-05-17T22:24:38.581 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rgw-2:19.0.0-3794.g931809a9.el9.x86_64 26/164 2024-05-17T22:24:38.581 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : rbd-fuse-2:19.0.0-3794.g931809a9.el9.x86_64 27/164 2024-05-17T22:24:38.581 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : rbd-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 28/164 2024-05-17T22:24:38.581 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : rbd-nbd-2:19.0.0-3794.g931809a9.el9.x86_64 29/164 2024-05-17T22:24:38.581 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3794.g931809a9. 30/164 2024-05-17T22:24:38.588 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-17T22:24:38.592 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3794.g931809a9.el9.noa 31/164 2024-05-17T22:24:38.592 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3794.g931809a9.el9.n 32/164 2024-05-17T22:24:38.592 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3794.g931 33/164 2024-05-17T22:24:38.592 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3794.g931809a9.el 34/164 2024-05-17T22:24:38.593 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3794.g931809a9.el9.noarch 35/164 2024-05-17T22:24:38.593 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3794.g931809a9.e 36/164 2024-05-17T22:24:38.593 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-volume-2:19.0.0-3794.g931809a9.el9.noarch 37/164 2024-05-17T22:24:38.593 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : cephadm-2:19.0.0-3794.g931809a9.el9.noarch 38/164 2024-05-17T22:24:38.593 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : cephfs-top-2:19.0.0-3794.g931809a9.el9.noarch 39/164 2024-05-17T22:24:38.593 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-17T22:24:38.593 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-17T22:24:38.593 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-17T22:24:38.593 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-17T22:24:38.593 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-17T22:24:38.593 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-17T22:24:38.593 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-17T22:24:38.593 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-17T22:24:38.593 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-17T22:24:38.593 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-17T22:24:38.594 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-17T22:24:38.594 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-17T22:24:38.594 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-17T22:24:38.594 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-17T22:24:38.594 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-17T22:24:38.594 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-17T22:24:38.594 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-17T22:24:38.594 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-17T22:24:38.594 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-17T22:24:38.594 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-17T22:24:38.594 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-17T22:24:38.594 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-17T22:24:38.594 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-17T22:24:38.594 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-17T22:24:38.595 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-17T22:24:38.595 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-17T22:24:38.595 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-17T22:24:38.595 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-17T22:24:38.595 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-17T22:24:38.595 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-17T22:24:38.595 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-17T22:24:38.595 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-17T22:24:38.595 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-17T22:24:38.595 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-17T22:24:38.595 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-17T22:24:38.595 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-17T22:24:38.595 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-17T22:24:38.596 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-17T22:24:38.596 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-17T22:24:38.596 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-17T22:24:38.596 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-17T22:24:38.596 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-17T22:24:38.596 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-17T22:24:38.596 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-17T22:24:38.596 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-17T22:24:38.596 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-17T22:24:38.596 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-17T22:24:38.596 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-17T22:24:38.596 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-17T22:24:38.596 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-17T22:24:38.596 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-17T22:24:38.597 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-17T22:24:38.597 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-17T22:24:38.597 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-17T22:24:38.597 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-17T22:24:38.597 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-17T22:24:38.597 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-17T22:24:38.597 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-17T22:24:38.597 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-17T22:24:38.597 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-17T22:24:38.597 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-17T22:24:38.597 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-17T22:24:38.597 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-17T22:24:38.597 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-17T22:24:38.597 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-17T22:24:38.597 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-17T22:24:38.597 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-17T22:24:38.598 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-17T22:24:38.598 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-17T22:24:38.598 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-17T22:24:38.598 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-17T22:24:38.598 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-17T22:24:38.598 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 112/164 2024-05-17T22:24:38.598 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-17T22:24:38.598 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-17T22:24:38.598 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-17T22:24:38.598 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-17T22:24:38.598 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-17T22:24:38.598 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-17T22:24:38.599 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-17T22:24:38.599 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-17T22:24:38.599 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-17T22:24:38.599 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-17T22:24:38.599 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-17T22:24:38.599 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-17T22:24:38.599 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-17T22:24:38.599 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-17T22:24:38.599 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-17T22:24:38.599 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-17T22:24:38.599 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-17T22:24:38.599 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-17T22:24:38.599 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-17T22:24:38.599 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-17T22:24:38.599 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-17T22:24:38.600 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-17T22:24:38.600 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-17T22:24:38.600 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-17T22:24:38.600 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-17T22:24:38.600 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-17T22:24:38.600 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-17T22:24:38.600 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-17T22:24:38.600 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-17T22:24:38.600 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-17T22:24:38.600 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-17T22:24:38.600 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-17T22:24:38.600 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-17T22:24:38.600 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-17T22:24:38.600 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-17T22:24:38.600 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-17T22:24:38.600 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librados2-2:19.0.0-3794.g931809a9.el9.x86_64 149/164 2024-05-17T22:24:38.601 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-17T22:24:38.601 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librbd1-2:19.0.0-3794.g931809a9.el9.x86_64 151/164 2024-05-17T22:24:38.601 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-17T22:24:38.601 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/164 2024-05-17T22:24:38.601 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-17T22:24:38.601 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/164 2024-05-17T22:24:38.601 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-17T22:24:38.601 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/164 2024-05-17T22:24:38.601 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-17T22:24:38.601 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/164 2024-05-17T22:24:38.601 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-17T22:24:38.601 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-17T22:24:38.601 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-17T22:24:38.601 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-17T22:24:38.627 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-17T22:24:38.659 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-17T22:24:38.686 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-17T22:24:39.053 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: cephadm-2:19.0.0-3794.g931809a9.el9.noarch 125/164 2024-05-17T22:24:39.076 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : cephadm-2:19.0.0-3794.g931809a9.el9.noarch 125/164 2024-05-17T22:24:39.109 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3794.g931809a9.e 126/164 2024-05-17T22:24:39.173 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3794.g931809a9. 127/164 2024-05-17T22:24:39.200 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3794.g931809a9.el9.noa 128/164 2024-05-17T22:24:39.986 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-17T22:24:39.987 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-17T22:24:39.987 INFO:teuthology.orchestra.run.smithi096.stdout:Upgraded: 2024-05-17T22:24:39.987 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-17T22:24:39.987 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-17T22:24:39.987 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-17T22:24:39.987 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-17T22:24:39.987 INFO:teuthology.orchestra.run.smithi096.stdout: librados2-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:39.987 INFO:teuthology.orchestra.run.smithi096.stdout: librbd1-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:39.987 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-17T22:24:39.987 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-17T22:24:39.987 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-05-17T22:24:39.987 INFO:teuthology.orchestra.run.smithi096.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-17T22:24:39.988 INFO:teuthology.orchestra.run.smithi096.stdout: automake-1.16.2-8.el9.noarch 2024-05-17T22:24:39.988 INFO:teuthology.orchestra.run.smithi096.stdout: bison-3.7.4-5.el9.x86_64 2024-05-17T22:24:39.988 INFO:teuthology.orchestra.run.smithi096.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-17T22:24:39.988 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:39.988 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-base-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:39.988 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-common-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:39.988 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-fuse-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:39.988 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-grafana-dashboards-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T22:24:39.988 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-immutable-object-cache-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:39.988 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mds-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:39.988 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:39.988 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-cephadm-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T22:24:39.988 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-dashboard-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T22:24:39.988 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T22:24:39.988 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-modules-core-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T22:24:39.989 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-rook-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T22:24:39.989 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mon-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:39.989 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-osd-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:39.989 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-prometheus-alerts-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T22:24:39.989 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-radosgw-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:39.989 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-selinux-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:39.989 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-test-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:39.989 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-volume-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T22:24:39.989 INFO:teuthology.orchestra.run.smithi096.stdout: cephadm-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T22:24:39.989 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:39.989 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-shell-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:39.989 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-top-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T22:24:39.989 INFO:teuthology.orchestra.run.smithi096.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-17T22:24:39.989 INFO:teuthology.orchestra.run.smithi096.stdout: flex-2.6.4-9.el9.x86_64 2024-05-17T22:24:39.989 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-17T22:24:39.990 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-17T22:24:39.990 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-17T22:24:39.990 INFO:teuthology.orchestra.run.smithi096.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-17T22:24:39.990 INFO:teuthology.orchestra.run.smithi096.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-17T22:24:39.990 INFO:teuthology.orchestra.run.smithi096.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-17T22:24:39.990 INFO:teuthology.orchestra.run.smithi096.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-17T22:24:39.990 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-17T22:24:39.990 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-17T22:24:39.990 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs-devel-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:39.990 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs2-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:39.990 INFO:teuthology.orchestra.run.smithi096.stdout: libcephsqlite-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:39.990 INFO:teuthology.orchestra.run.smithi096.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-17T22:24:39.990 INFO:teuthology.orchestra.run.smithi096.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-17T22:24:39.990 INFO:teuthology.orchestra.run.smithi096.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-17T22:24:39.991 INFO:teuthology.orchestra.run.smithi096.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-17T22:24:39.991 INFO:teuthology.orchestra.run.smithi096.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-17T22:24:39.991 INFO:teuthology.orchestra.run.smithi096.stdout: librados-devel-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:39.991 INFO:teuthology.orchestra.run.smithi096.stdout: libradosstriper1-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:39.991 INFO:teuthology.orchestra.run.smithi096.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-17T22:24:39.991 INFO:teuthology.orchestra.run.smithi096.stdout: librgw2-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:39.991 INFO:teuthology.orchestra.run.smithi096.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-17T22:24:39.991 INFO:teuthology.orchestra.run.smithi096.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-17T22:24:39.991 INFO:teuthology.orchestra.run.smithi096.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-17T22:24:39.991 INFO:teuthology.orchestra.run.smithi096.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-17T22:24:39.991 INFO:teuthology.orchestra.run.smithi096.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-17T22:24:39.991 INFO:teuthology.orchestra.run.smithi096.stdout: lua-5.4.4-4.el9.x86_64 2024-05-17T22:24:39.991 INFO:teuthology.orchestra.run.smithi096.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-17T22:24:39.991 INFO:teuthology.orchestra.run.smithi096.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-17T22:24:39.992 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-17T22:24:39.992 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-17T22:24:39.992 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-17T22:24:39.992 INFO:teuthology.orchestra.run.smithi096.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-17T22:24:39.992 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-13.14-1.el9.x86_64 2024-05-17T22:24:39.992 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-17T22:24:39.992 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-17T22:24:39.992 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-17T22:24:39.992 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-17T22:24:39.992 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-17T22:24:39.992 INFO:teuthology.orchestra.run.smithi096.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-17T22:24:39.992 INFO:teuthology.orchestra.run.smithi096.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-17T22:24:39.992 INFO:teuthology.orchestra.run.smithi096.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-17T22:24:39.993 INFO:teuthology.orchestra.run.smithi096.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-17T22:24:39.993 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-17T22:24:39.993 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-argparse-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:39.993 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-common-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:39.993 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cephfs-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:39.993 INFO:teuthology.orchestra.run.smithi096.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-17T22:24:39.993 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-17T22:24:39.993 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-17T22:24:39.993 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-17T22:24:39.993 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-17T22:24:39.993 INFO:teuthology.orchestra.run.smithi096.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-17T22:24:39.993 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-17T22:24:39.993 INFO:teuthology.orchestra.run.smithi096.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-17T22:24:39.993 INFO:teuthology.orchestra.run.smithi096.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-17T22:24:39.994 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-17T22:24:39.994 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-17T22:24:39.994 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-17T22:24:39.994 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-17T22:24:39.994 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-17T22:24:39.994 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-17T22:24:39.994 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-17T22:24:39.994 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-17T22:24:39.994 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-17T22:24:39.994 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-17T22:24:39.994 INFO:teuthology.orchestra.run.smithi096.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-17T22:24:39.994 INFO:teuthology.orchestra.run.smithi096.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-17T22:24:39.994 INFO:teuthology.orchestra.run.smithi096.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-17T22:24:39.995 INFO:teuthology.orchestra.run.smithi096.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-17T22:24:39.995 INFO:teuthology.orchestra.run.smithi096.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-17T22:24:39.995 INFO:teuthology.orchestra.run.smithi096.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-17T22:24:39.995 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-17T22:24:39.995 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-17T22:24:39.995 INFO:teuthology.orchestra.run.smithi096.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-17T22:24:39.995 INFO:teuthology.orchestra.run.smithi096.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-17T22:24:39.995 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-17T22:24:39.995 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ply-3.11-14.el9.noarch 2024-05-17T22:24:39.995 INFO:teuthology.orchestra.run.smithi096.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-17T22:24:39.995 INFO:teuthology.orchestra.run.smithi096.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-17T22:24:39.995 INFO:teuthology.orchestra.run.smithi096.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-17T22:24:39.995 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-17T22:24:39.996 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-17T22:24:39.996 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-17T22:24:39.996 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-17T22:24:39.996 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-17T22:24:39.996 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-17T22:24:39.996 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rados-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:39.996 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rbd-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:39.996 INFO:teuthology.orchestra.run.smithi096.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-17T22:24:39.996 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-17T22:24:39.996 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-17T22:24:39.996 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rgw-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:39.996 INFO:teuthology.orchestra.run.smithi096.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-17T22:24:39.996 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-17T22:24:39.997 INFO:teuthology.orchestra.run.smithi096.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-17T22:24:39.997 INFO:teuthology.orchestra.run.smithi096.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-17T22:24:39.997 INFO:teuthology.orchestra.run.smithi096.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-17T22:24:39.997 INFO:teuthology.orchestra.run.smithi096.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-17T22:24:39.997 INFO:teuthology.orchestra.run.smithi096.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-17T22:24:39.997 INFO:teuthology.orchestra.run.smithi096.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-17T22:24:39.997 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-17T22:24:39.997 INFO:teuthology.orchestra.run.smithi096.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-17T22:24:39.998 INFO:teuthology.orchestra.run.smithi096.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-17T22:24:39.998 INFO:teuthology.orchestra.run.smithi096.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-17T22:24:39.998 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-17T22:24:39.998 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-17T22:24:39.998 INFO:teuthology.orchestra.run.smithi096.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-17T22:24:39.998 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-fuse-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:39.998 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:39.998 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-nbd-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:39.998 INFO:teuthology.orchestra.run.smithi096.stdout: re2-1:20211101-3.el9.x86_64 2024-05-17T22:24:39.998 INFO:teuthology.orchestra.run.smithi096.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-17T22:24:39.998 INFO:teuthology.orchestra.run.smithi096.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-17T22:24:39.998 INFO:teuthology.orchestra.run.smithi096.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-17T22:24:39.998 INFO:teuthology.orchestra.run.smithi096.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-17T22:24:39.999 INFO:teuthology.orchestra.run.smithi096.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-17T22:24:39.999 INFO:teuthology.orchestra.run.smithi096.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-17T22:24:39.999 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-17T22:24:39.999 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-05-17T22:24:40.298 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3794.g931809a9.el9.noa 128/164 2024-05-17T22:24:40.335 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3794.g931809a9.el9.n 129/164 2024-05-17T22:24:40.424 DEBUG:teuthology.parallel:result is None 2024-05-17T22:24:41.348 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3794.g931809a9.el9.n 129/164 2024-05-17T22:24:41.376 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3794.g931 130/164 2024-05-17T22:24:41.404 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-17T22:24:41.404 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-17T22:24:41.404 INFO:teuthology.orchestra.run.smithi151.stdout:Upgraded: 2024-05-17T22:24:41.404 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-17T22:24:41.404 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-17T22:24:41.404 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-17T22:24:41.404 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-17T22:24:41.404 INFO:teuthology.orchestra.run.smithi151.stdout: librados2-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:41.404 INFO:teuthology.orchestra.run.smithi151.stdout: librbd1-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:41.404 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-17T22:24:41.404 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-17T22:24:41.405 INFO:teuthology.orchestra.run.smithi151.stdout:Installed: 2024-05-17T22:24:41.405 INFO:teuthology.orchestra.run.smithi151.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-17T22:24:41.405 INFO:teuthology.orchestra.run.smithi151.stdout: automake-1.16.2-8.el9.noarch 2024-05-17T22:24:41.405 INFO:teuthology.orchestra.run.smithi151.stdout: bison-3.7.4-5.el9.x86_64 2024-05-17T22:24:41.405 INFO:teuthology.orchestra.run.smithi151.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-17T22:24:41.405 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:41.405 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-base-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:41.405 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-common-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:41.405 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-fuse-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:41.405 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-grafana-dashboards-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T22:24:41.405 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-immutable-object-cache-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:41.405 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mds-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:41.405 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:41.406 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-cephadm-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T22:24:41.406 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-dashboard-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T22:24:41.406 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T22:24:41.406 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-modules-core-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T22:24:41.406 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-rook-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T22:24:41.406 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mon-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:41.406 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-osd-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:41.406 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-prometheus-alerts-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T22:24:41.406 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-radosgw-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:41.406 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-selinux-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:41.406 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-test-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:41.406 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-volume-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T22:24:41.406 INFO:teuthology.orchestra.run.smithi151.stdout: cephadm-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T22:24:41.406 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:41.407 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-shell-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:41.407 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-top-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T22:24:41.407 INFO:teuthology.orchestra.run.smithi151.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-17T22:24:41.407 INFO:teuthology.orchestra.run.smithi151.stdout: flex-2.6.4-9.el9.x86_64 2024-05-17T22:24:41.407 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-17T22:24:41.407 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-17T22:24:41.407 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-17T22:24:41.407 INFO:teuthology.orchestra.run.smithi151.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-17T22:24:41.407 INFO:teuthology.orchestra.run.smithi151.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-17T22:24:41.407 INFO:teuthology.orchestra.run.smithi151.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-17T22:24:41.407 INFO:teuthology.orchestra.run.smithi151.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-17T22:24:41.407 INFO:teuthology.orchestra.run.smithi151.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-17T22:24:41.407 INFO:teuthology.orchestra.run.smithi151.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-17T22:24:41.407 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs-devel-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:41.408 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs2-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:41.408 INFO:teuthology.orchestra.run.smithi151.stdout: libcephsqlite-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:41.408 INFO:teuthology.orchestra.run.smithi151.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-17T22:24:41.408 INFO:teuthology.orchestra.run.smithi151.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-17T22:24:41.408 INFO:teuthology.orchestra.run.smithi151.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-17T22:24:41.408 INFO:teuthology.orchestra.run.smithi151.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-17T22:24:41.408 INFO:teuthology.orchestra.run.smithi151.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-17T22:24:41.408 INFO:teuthology.orchestra.run.smithi151.stdout: librados-devel-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:41.408 INFO:teuthology.orchestra.run.smithi151.stdout: libradosstriper1-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:41.408 INFO:teuthology.orchestra.run.smithi151.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-17T22:24:41.408 INFO:teuthology.orchestra.run.smithi151.stdout: librgw2-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:41.409 INFO:teuthology.orchestra.run.smithi151.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-17T22:24:41.409 INFO:teuthology.orchestra.run.smithi151.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-17T22:24:41.409 INFO:teuthology.orchestra.run.smithi151.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-17T22:24:41.409 INFO:teuthology.orchestra.run.smithi151.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-17T22:24:41.409 INFO:teuthology.orchestra.run.smithi151.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-17T22:24:41.409 INFO:teuthology.orchestra.run.smithi151.stdout: lua-5.4.4-4.el9.x86_64 2024-05-17T22:24:41.409 INFO:teuthology.orchestra.run.smithi151.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-17T22:24:41.409 INFO:teuthology.orchestra.run.smithi151.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-17T22:24:41.409 INFO:teuthology.orchestra.run.smithi151.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-17T22:24:41.409 INFO:teuthology.orchestra.run.smithi151.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-17T22:24:41.409 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-17T22:24:41.409 INFO:teuthology.orchestra.run.smithi151.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-17T22:24:41.409 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-13.14-1.el9.x86_64 2024-05-17T22:24:41.410 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-17T22:24:41.410 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-17T22:24:41.410 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-17T22:24:41.410 INFO:teuthology.orchestra.run.smithi151.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-17T22:24:41.410 INFO:teuthology.orchestra.run.smithi151.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-17T22:24:41.410 INFO:teuthology.orchestra.run.smithi151.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-17T22:24:41.410 INFO:teuthology.orchestra.run.smithi151.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-17T22:24:41.410 INFO:teuthology.orchestra.run.smithi151.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-17T22:24:41.410 INFO:teuthology.orchestra.run.smithi151.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-17T22:24:41.410 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-17T22:24:41.410 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-argparse-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:41.410 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-common-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:41.411 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cephfs-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:41.411 INFO:teuthology.orchestra.run.smithi151.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-17T22:24:41.411 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-17T22:24:41.411 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-17T22:24:41.411 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-17T22:24:41.411 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-17T22:24:41.411 INFO:teuthology.orchestra.run.smithi151.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-17T22:24:41.411 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-17T22:24:41.411 INFO:teuthology.orchestra.run.smithi151.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-17T22:24:41.411 INFO:teuthology.orchestra.run.smithi151.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-17T22:24:41.411 INFO:teuthology.orchestra.run.smithi151.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-17T22:24:41.411 INFO:teuthology.orchestra.run.smithi151.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-17T22:24:41.411 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-17T22:24:41.411 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-17T22:24:41.411 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-17T22:24:41.412 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-17T22:24:41.412 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-17T22:24:41.412 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-17T22:24:41.412 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-17T22:24:41.412 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-17T22:24:41.412 INFO:teuthology.orchestra.run.smithi151.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-17T22:24:41.412 INFO:teuthology.orchestra.run.smithi151.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-17T22:24:41.412 INFO:teuthology.orchestra.run.smithi151.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-17T22:24:41.412 INFO:teuthology.orchestra.run.smithi151.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-17T22:24:41.412 INFO:teuthology.orchestra.run.smithi151.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-17T22:24:41.412 INFO:teuthology.orchestra.run.smithi151.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-17T22:24:41.412 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-17T22:24:41.412 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-17T22:24:41.412 INFO:teuthology.orchestra.run.smithi151.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-17T22:24:41.413 INFO:teuthology.orchestra.run.smithi151.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-17T22:24:41.413 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-17T22:24:41.413 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ply-3.11-14.el9.noarch 2024-05-17T22:24:41.413 INFO:teuthology.orchestra.run.smithi151.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-17T22:24:41.413 INFO:teuthology.orchestra.run.smithi151.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-17T22:24:41.413 INFO:teuthology.orchestra.run.smithi151.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-17T22:24:41.413 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-17T22:24:41.413 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-17T22:24:41.413 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-17T22:24:41.413 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-17T22:24:41.413 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-17T22:24:41.413 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-17T22:24:41.413 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rados-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:41.414 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rbd-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:41.414 INFO:teuthology.orchestra.run.smithi151.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-17T22:24:41.414 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-17T22:24:41.414 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-17T22:24:41.414 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rgw-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:41.414 INFO:teuthology.orchestra.run.smithi151.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-17T22:24:41.414 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-17T22:24:41.414 INFO:teuthology.orchestra.run.smithi151.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-17T22:24:41.414 INFO:teuthology.orchestra.run.smithi151.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-17T22:24:41.414 INFO:teuthology.orchestra.run.smithi151.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-17T22:24:41.414 INFO:teuthology.orchestra.run.smithi151.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-17T22:24:41.414 INFO:teuthology.orchestra.run.smithi151.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-17T22:24:41.414 INFO:teuthology.orchestra.run.smithi151.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-17T22:24:41.415 INFO:teuthology.orchestra.run.smithi151.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-17T22:24:41.415 INFO:teuthology.orchestra.run.smithi151.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-17T22:24:41.415 INFO:teuthology.orchestra.run.smithi151.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-17T22:24:41.415 INFO:teuthology.orchestra.run.smithi151.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-17T22:24:41.415 INFO:teuthology.orchestra.run.smithi151.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-17T22:24:41.415 INFO:teuthology.orchestra.run.smithi151.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-17T22:24:41.415 INFO:teuthology.orchestra.run.smithi151.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-17T22:24:41.415 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-fuse-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:41.415 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:41.415 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-nbd-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:41.415 INFO:teuthology.orchestra.run.smithi151.stdout: re2-1:20211101-3.el9.x86_64 2024-05-17T22:24:41.415 INFO:teuthology.orchestra.run.smithi151.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-17T22:24:41.415 INFO:teuthology.orchestra.run.smithi151.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-17T22:24:41.415 INFO:teuthology.orchestra.run.smithi151.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-17T22:24:41.415 INFO:teuthology.orchestra.run.smithi151.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-17T22:24:41.416 INFO:teuthology.orchestra.run.smithi151.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-17T22:24:41.416 INFO:teuthology.orchestra.run.smithi151.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-17T22:24:41.416 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-17T22:24:41.416 INFO:teuthology.orchestra.run.smithi151.stdout:Complete! 2024-05-17T22:24:41.496 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3794.g931 130/164 2024-05-17T22:24:41.664 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3794.g931809a9.el 131/164 2024-05-17T22:24:41.687 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-2:19.0.0-3794.g931809a9.el9.x86_64 132/164 2024-05-17T22:24:41.713 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3794.g931809a9.el9.x86_64 132/164 2024-05-17T22:24:41.713 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T22:24:41.713 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-17T22:24:41.713 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-17T22:24:41.713 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-17T22:24:41.714 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:24:41.763 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-rook-2:19.0.0-3794.g931809a9.el9.noarch 133/164 2024-05-17T22:24:41.775 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3794.g931809a9.el9.noarch 133/164 2024-05-17T22:24:41.796 DEBUG:teuthology.parallel:result is None 2024-05-17T22:24:41.868 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-2:19.0.0-3794.g931809a9.el9.x86_64 134/164 2024-05-17T22:24:41.928 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-fuse-2:19.0.0-3794.g931809a9.el9.x86_64 135/164 2024-05-17T22:24:42.925 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-17T22:24:42.957 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-radosgw-2:19.0.0-3794.g931809a9.el9.x86_64 137/164 2024-05-17T22:24:42.982 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3794.g931809a9.el9.x86_64 137/164 2024-05-17T22:24:42.982 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T22:24:42.982 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-17T22:24:42.982 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-17T22:24:42.982 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-17T22:24:42.982 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:24:43.024 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3794.g93180 138/164 2024-05-17T22:24:43.048 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3794.g93180 138/164 2024-05-17T22:24:43.049 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T22:24:43.049 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-17T22:24:43.049 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:24:43.100 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : cephfs-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 139/164 2024-05-17T22:24:43.125 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 139/164 2024-05-17T22:24:43.125 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T22:24:43.125 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-17T22:24:43.125 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-17T22:24:43.125 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-17T22:24:43.125 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:24:43.454 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : rbd-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 140/164 2024-05-17T22:24:43.476 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 140/164 2024-05-17T22:24:43.476 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T22:24:43.476 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-17T22:24:43.476 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-17T22:24:43.476 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-17T22:24:43.476 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:24:48.005 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-test-2:19.0.0-3794.g931809a9.el9.x86_64 141/164 2024-05-17T22:24:48.054 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : rbd-fuse-2:19.0.0-3794.g931809a9.el9.x86_64 142/164 2024-05-17T22:24:48.096 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : rbd-nbd-2:19.0.0-3794.g931809a9.el9.x86_64 143/164 2024-05-17T22:24:48.127 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-17T22:24:48.154 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-17T22:24:48.349 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : cephfs-shell-2:19.0.0-3794.g931809a9.el9.x86_64 146/164 2024-05-17T22:24:48.390 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-17T22:24:48.421 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libcephfs-devel-2:19.0.0-3794.g931809a9.el9.x86_ 148/164 2024-05-17T22:24:48.450 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 149/164 2024-05-17T22:24:48.496 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 150/164 2024-05-17T22:24:49.163 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-17T22:24:49.202 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-17T22:24:49.224 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-17T22:24:50.134 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : cephfs-top-2:19.0.0-3794.g931809a9.el9.noarch 152/164 2024-05-17T22:24:50.223 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-17T22:24:50.271 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-17T22:24:50.371 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-17T22:24:50.468 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-17T22:24:50.494 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-17T22:24:50.494 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-17T22:24:50.578 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-17T22:24:50.611 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-17T22:24:50.611 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-17T22:24:50.704 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-17T22:24:50.729 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-17T22:24:50.736 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-17T22:24:50.736 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-17T22:24:50.931 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-17T22:24:50.932 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-17T22:24:51.032 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-17T22:24:51.108 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-17T22:24:56.833 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-17T22:24:56.834 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-2:19.0.0-3794.g931809a9.el9.x86_64 1/164 2024-05-17T22:24:56.834 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-base-2:19.0.0-3794.g931809a9.el9.x86_64 2/164 2024-05-17T22:24:56.834 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-common-2:19.0.0-3794.g931809a9.el9.x86_64 3/164 2024-05-17T22:24:56.834 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-fuse-2:19.0.0-3794.g931809a9.el9.x86_64 4/164 2024-05-17T22:24:56.834 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3794.g93180 5/164 2024-05-17T22:24:56.834 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mds-2:19.0.0-3794.g931809a9.el9.x86_64 6/164 2024-05-17T22:24:56.834 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-2:19.0.0-3794.g931809a9.el9.x86_64 7/164 2024-05-17T22:24:56.834 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mon-2:19.0.0-3794.g931809a9.el9.x86_64 8/164 2024-05-17T22:24:56.834 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-osd-2:19.0.0-3794.g931809a9.el9.x86_64 9/164 2024-05-17T22:24:56.834 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-radosgw-2:19.0.0-3794.g931809a9.el9.x86_64 10/164 2024-05-17T22:24:56.834 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-selinux-2:19.0.0-3794.g931809a9.el9.x86_64 11/164 2024-05-17T22:24:56.834 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-test-2:19.0.0-3794.g931809a9.el9.x86_64 12/164 2024-05-17T22:24:56.835 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : cephfs-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 13/164 2024-05-17T22:24:56.835 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : cephfs-shell-2:19.0.0-3794.g931809a9.el9.x86_64 14/164 2024-05-17T22:24:56.835 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libcephfs-devel-2:19.0.0-3794.g931809a9.el9.x86_ 15/164 2024-05-17T22:24:56.835 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libcephfs2-2:19.0.0-3794.g931809a9.el9.x86_64 16/164 2024-05-17T22:24:56.835 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libcephsqlite-2:19.0.0-3794.g931809a9.el9.x86_64 17/164 2024-05-17T22:24:56.835 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librados-devel-2:19.0.0-3794.g931809a9.el9.x86_6 18/164 2024-05-17T22:24:56.835 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libradosstriper1-2:19.0.0-3794.g931809a9.el9.x86 19/164 2024-05-17T22:24:56.835 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librgw2-2:19.0.0-3794.g931809a9.el9.x86_64 20/164 2024-05-17T22:24:56.835 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3794.g931809a9.el 21/164 2024-05-17T22:24:56.835 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-ceph-common-2:19.0.0-3794.g931809a9.el9. 22/164 2024-05-17T22:24:56.835 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cephfs-2:19.0.0-3794.g931809a9.el9.x86_6 23/164 2024-05-17T22:24:56.835 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-rados-2:19.0.0-3794.g931809a9.el9.x86_64 24/164 2024-05-17T22:24:56.835 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-rbd-2:19.0.0-3794.g931809a9.el9.x86_64 25/164 2024-05-17T22:24:56.836 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-rgw-2:19.0.0-3794.g931809a9.el9.x86_64 26/164 2024-05-17T22:24:56.836 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : rbd-fuse-2:19.0.0-3794.g931809a9.el9.x86_64 27/164 2024-05-17T22:24:56.847 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : rbd-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 28/164 2024-05-17T22:24:56.847 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : rbd-nbd-2:19.0.0-3794.g931809a9.el9.x86_64 29/164 2024-05-17T22:24:56.847 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3794.g931809a9. 30/164 2024-05-17T22:24:56.847 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3794.g931809a9.el9.noa 31/164 2024-05-17T22:24:56.847 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3794.g931809a9.el9.n 32/164 2024-05-17T22:24:56.847 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3794.g931 33/164 2024-05-17T22:24:56.847 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3794.g931809a9.el 34/164 2024-05-17T22:24:56.847 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3794.g931809a9.el9.noarch 35/164 2024-05-17T22:24:56.847 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3794.g931809a9.e 36/164 2024-05-17T22:24:56.847 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-volume-2:19.0.0-3794.g931809a9.el9.noarch 37/164 2024-05-17T22:24:56.847 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : cephadm-2:19.0.0-3794.g931809a9.el9.noarch 38/164 2024-05-17T22:24:56.848 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : cephfs-top-2:19.0.0-3794.g931809a9.el9.noarch 39/164 2024-05-17T22:24:56.848 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-17T22:24:56.848 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-17T22:24:56.848 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-17T22:24:56.848 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-17T22:24:56.848 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-17T22:24:56.848 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-17T22:24:56.848 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-17T22:24:56.848 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-17T22:24:56.848 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-17T22:24:56.848 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-17T22:24:56.848 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-17T22:24:56.848 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-17T22:24:56.848 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-17T22:24:56.849 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-17T22:24:56.849 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-17T22:24:56.849 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-17T22:24:56.849 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-17T22:24:56.849 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-17T22:24:56.849 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-17T22:24:56.849 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-17T22:24:56.849 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-17T22:24:56.849 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-17T22:24:56.849 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-17T22:24:56.849 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-17T22:24:56.849 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-17T22:24:56.849 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-17T22:24:56.850 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-17T22:24:56.850 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-17T22:24:56.850 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-17T22:24:56.850 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-17T22:24:56.850 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-17T22:24:56.850 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-17T22:24:56.850 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-17T22:24:56.850 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-17T22:24:56.850 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-17T22:24:56.850 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-17T22:24:56.850 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-17T22:24:56.850 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-17T22:24:56.850 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-17T22:24:56.851 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-17T22:24:56.851 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-17T22:24:56.851 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-17T22:24:56.851 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-17T22:24:56.851 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-17T22:24:56.851 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-17T22:24:56.851 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-17T22:24:56.851 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-17T22:24:56.851 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-17T22:24:56.851 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-17T22:24:56.851 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-17T22:24:56.851 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-17T22:24:56.851 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-17T22:24:56.851 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-17T22:24:56.852 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-17T22:24:56.852 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-17T22:24:56.852 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-17T22:24:56.852 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-17T22:24:56.852 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-17T22:24:56.852 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-17T22:24:56.852 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-17T22:24:56.852 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-17T22:24:56.852 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-17T22:24:56.852 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-17T22:24:56.852 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-17T22:24:56.852 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-17T22:24:56.852 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-17T22:24:56.852 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-17T22:24:56.853 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-17T22:24:56.853 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-17T22:24:56.853 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-17T22:24:56.853 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-17T22:24:56.853 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-17T22:24:56.853 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 112/164 2024-05-17T22:24:56.853 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-17T22:24:56.853 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-17T22:24:56.853 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-17T22:24:56.853 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-17T22:24:56.853 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-17T22:24:56.853 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-17T22:24:56.853 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-17T22:24:56.853 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-17T22:24:56.854 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-17T22:24:56.854 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-17T22:24:56.854 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-17T22:24:56.854 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-17T22:24:56.854 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-17T22:24:56.854 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-17T22:24:56.854 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-17T22:24:56.854 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-17T22:24:56.854 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-17T22:24:56.854 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-17T22:24:56.854 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-17T22:24:56.854 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-17T22:24:56.854 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-17T22:24:56.855 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-17T22:24:56.855 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-17T22:24:56.855 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-17T22:24:56.855 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-17T22:24:56.855 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-17T22:24:56.855 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-17T22:24:56.855 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-17T22:24:56.855 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-17T22:24:56.855 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-17T22:24:56.855 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-17T22:24:56.855 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-17T22:24:56.855 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-17T22:24:56.855 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-17T22:24:56.855 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-17T22:24:56.856 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-17T22:24:56.856 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librados2-2:19.0.0-3794.g931809a9.el9.x86_64 149/164 2024-05-17T22:24:56.856 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-17T22:24:56.856 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librbd1-2:19.0.0-3794.g931809a9.el9.x86_64 151/164 2024-05-17T22:24:56.856 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-17T22:24:56.856 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/164 2024-05-17T22:24:56.856 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-17T22:24:56.856 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/164 2024-05-17T22:24:56.856 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-17T22:24:56.856 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/164 2024-05-17T22:24:56.856 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-17T22:24:56.856 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/164 2024-05-17T22:24:56.856 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-17T22:24:56.856 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-17T22:24:56.857 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-17T22:24:56.857 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-17T22:24:59.667 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-17T22:24:59.667 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:24:59.667 INFO:teuthology.orchestra.run.smithi012.stdout:Upgraded: 2024-05-17T22:24:59.667 INFO:teuthology.orchestra.run.smithi012.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-17T22:24:59.667 INFO:teuthology.orchestra.run.smithi012.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-17T22:24:59.667 INFO:teuthology.orchestra.run.smithi012.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-17T22:24:59.667 INFO:teuthology.orchestra.run.smithi012.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-17T22:24:59.667 INFO:teuthology.orchestra.run.smithi012.stdout: librados2-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:59.668 INFO:teuthology.orchestra.run.smithi012.stdout: librbd1-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:59.668 INFO:teuthology.orchestra.run.smithi012.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-17T22:24:59.668 INFO:teuthology.orchestra.run.smithi012.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-17T22:24:59.668 INFO:teuthology.orchestra.run.smithi012.stdout:Installed: 2024-05-17T22:24:59.668 INFO:teuthology.orchestra.run.smithi012.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-17T22:24:59.668 INFO:teuthology.orchestra.run.smithi012.stdout: automake-1.16.2-8.el9.noarch 2024-05-17T22:24:59.668 INFO:teuthology.orchestra.run.smithi012.stdout: bison-3.7.4-5.el9.x86_64 2024-05-17T22:24:59.668 INFO:teuthology.orchestra.run.smithi012.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-17T22:24:59.668 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:59.668 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-base-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:59.668 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-common-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:59.668 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-fuse-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:59.668 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-grafana-dashboards-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T22:24:59.669 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-immutable-object-cache-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:59.669 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mds-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:59.669 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:59.669 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-cephadm-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T22:24:59.669 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-dashboard-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T22:24:59.669 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T22:24:59.669 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-modules-core-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T22:24:59.669 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-rook-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T22:24:59.669 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mon-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:59.669 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-osd-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:59.669 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-prometheus-alerts-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T22:24:59.669 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-radosgw-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:59.669 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-selinux-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:59.670 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-test-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:59.670 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-volume-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T22:24:59.670 INFO:teuthology.orchestra.run.smithi012.stdout: cephadm-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T22:24:59.670 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:59.670 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-shell-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:59.670 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-top-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T22:24:59.670 INFO:teuthology.orchestra.run.smithi012.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-17T22:24:59.670 INFO:teuthology.orchestra.run.smithi012.stdout: flex-2.6.4-9.el9.x86_64 2024-05-17T22:24:59.670 INFO:teuthology.orchestra.run.smithi012.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-17T22:24:59.670 INFO:teuthology.orchestra.run.smithi012.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-17T22:24:59.670 INFO:teuthology.orchestra.run.smithi012.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-17T22:24:59.671 INFO:teuthology.orchestra.run.smithi012.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-17T22:24:59.671 INFO:teuthology.orchestra.run.smithi012.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-17T22:24:59.671 INFO:teuthology.orchestra.run.smithi012.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-17T22:24:59.671 INFO:teuthology.orchestra.run.smithi012.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-17T22:24:59.671 INFO:teuthology.orchestra.run.smithi012.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-17T22:24:59.671 INFO:teuthology.orchestra.run.smithi012.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-17T22:24:59.671 INFO:teuthology.orchestra.run.smithi012.stdout: libcephfs-devel-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:59.671 INFO:teuthology.orchestra.run.smithi012.stdout: libcephfs2-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:59.671 INFO:teuthology.orchestra.run.smithi012.stdout: libcephsqlite-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:59.671 INFO:teuthology.orchestra.run.smithi012.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-17T22:24:59.671 INFO:teuthology.orchestra.run.smithi012.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-17T22:24:59.671 INFO:teuthology.orchestra.run.smithi012.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-17T22:24:59.671 INFO:teuthology.orchestra.run.smithi012.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-17T22:24:59.672 INFO:teuthology.orchestra.run.smithi012.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-17T22:24:59.672 INFO:teuthology.orchestra.run.smithi012.stdout: librados-devel-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:59.672 INFO:teuthology.orchestra.run.smithi012.stdout: libradosstriper1-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:59.672 INFO:teuthology.orchestra.run.smithi012.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-17T22:24:59.672 INFO:teuthology.orchestra.run.smithi012.stdout: librgw2-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:59.672 INFO:teuthology.orchestra.run.smithi012.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-17T22:24:59.672 INFO:teuthology.orchestra.run.smithi012.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-17T22:24:59.672 INFO:teuthology.orchestra.run.smithi012.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-17T22:24:59.672 INFO:teuthology.orchestra.run.smithi012.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-17T22:24:59.672 INFO:teuthology.orchestra.run.smithi012.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-17T22:24:59.672 INFO:teuthology.orchestra.run.smithi012.stdout: lua-5.4.4-4.el9.x86_64 2024-05-17T22:24:59.673 INFO:teuthology.orchestra.run.smithi012.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-17T22:24:59.673 INFO:teuthology.orchestra.run.smithi012.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-17T22:24:59.673 INFO:teuthology.orchestra.run.smithi012.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-17T22:24:59.673 INFO:teuthology.orchestra.run.smithi012.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-17T22:24:59.673 INFO:teuthology.orchestra.run.smithi012.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-17T22:24:59.673 INFO:teuthology.orchestra.run.smithi012.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-17T22:24:59.673 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-13.14-1.el9.x86_64 2024-05-17T22:24:59.674 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-17T22:24:59.674 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-17T22:24:59.674 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-17T22:24:59.674 INFO:teuthology.orchestra.run.smithi012.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-17T22:24:59.674 INFO:teuthology.orchestra.run.smithi012.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-17T22:24:59.675 INFO:teuthology.orchestra.run.smithi012.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-17T22:24:59.675 INFO:teuthology.orchestra.run.smithi012.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-17T22:24:59.675 INFO:teuthology.orchestra.run.smithi012.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-17T22:24:59.675 INFO:teuthology.orchestra.run.smithi012.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-17T22:24:59.676 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-17T22:24:59.676 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-argparse-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:59.676 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-common-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:59.676 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cephfs-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:59.676 INFO:teuthology.orchestra.run.smithi012.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-17T22:24:59.676 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-17T22:24:59.677 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-17T22:24:59.677 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-17T22:24:59.677 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-17T22:24:59.677 INFO:teuthology.orchestra.run.smithi012.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-17T22:24:59.677 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-17T22:24:59.677 INFO:teuthology.orchestra.run.smithi012.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-17T22:24:59.678 INFO:teuthology.orchestra.run.smithi012.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-17T22:24:59.678 INFO:teuthology.orchestra.run.smithi012.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-17T22:24:59.678 INFO:teuthology.orchestra.run.smithi012.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-17T22:24:59.678 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-17T22:24:59.678 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-17T22:24:59.678 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-17T22:24:59.679 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-17T22:24:59.679 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-17T22:24:59.679 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-17T22:24:59.679 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-17T22:24:59.679 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-17T22:24:59.679 INFO:teuthology.orchestra.run.smithi012.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-17T22:24:59.680 INFO:teuthology.orchestra.run.smithi012.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-17T22:24:59.680 INFO:teuthology.orchestra.run.smithi012.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-17T22:24:59.680 INFO:teuthology.orchestra.run.smithi012.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-17T22:24:59.680 INFO:teuthology.orchestra.run.smithi012.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-17T22:24:59.680 INFO:teuthology.orchestra.run.smithi012.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-17T22:24:59.681 INFO:teuthology.orchestra.run.smithi012.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-17T22:24:59.681 INFO:teuthology.orchestra.run.smithi012.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-17T22:24:59.681 INFO:teuthology.orchestra.run.smithi012.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-17T22:24:59.681 INFO:teuthology.orchestra.run.smithi012.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-17T22:24:59.681 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-17T22:24:59.681 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ply-3.11-14.el9.noarch 2024-05-17T22:24:59.682 INFO:teuthology.orchestra.run.smithi012.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-17T22:24:59.682 INFO:teuthology.orchestra.run.smithi012.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-17T22:24:59.682 INFO:teuthology.orchestra.run.smithi012.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-17T22:24:59.682 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-17T22:24:59.682 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-17T22:24:59.682 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-17T22:24:59.683 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-17T22:24:59.683 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-17T22:24:59.683 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-17T22:24:59.683 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rados-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:59.683 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rbd-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:59.683 INFO:teuthology.orchestra.run.smithi012.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-17T22:24:59.683 INFO:teuthology.orchestra.run.smithi012.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-17T22:24:59.684 INFO:teuthology.orchestra.run.smithi012.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-17T22:24:59.684 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rgw-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:59.684 INFO:teuthology.orchestra.run.smithi012.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-17T22:24:59.684 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-17T22:24:59.685 INFO:teuthology.orchestra.run.smithi012.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-17T22:24:59.685 INFO:teuthology.orchestra.run.smithi012.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-17T22:24:59.685 INFO:teuthology.orchestra.run.smithi012.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-17T22:24:59.685 INFO:teuthology.orchestra.run.smithi012.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-17T22:24:59.685 INFO:teuthology.orchestra.run.smithi012.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-17T22:24:59.685 INFO:teuthology.orchestra.run.smithi012.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-17T22:24:59.686 INFO:teuthology.orchestra.run.smithi012.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-17T22:24:59.686 INFO:teuthology.orchestra.run.smithi012.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-17T22:24:59.686 INFO:teuthology.orchestra.run.smithi012.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-17T22:24:59.686 INFO:teuthology.orchestra.run.smithi012.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-17T22:24:59.686 INFO:teuthology.orchestra.run.smithi012.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-17T22:24:59.686 INFO:teuthology.orchestra.run.smithi012.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-17T22:24:59.687 INFO:teuthology.orchestra.run.smithi012.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-17T22:24:59.687 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-fuse-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:59.687 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:59.687 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-nbd-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T22:24:59.687 INFO:teuthology.orchestra.run.smithi012.stdout: re2-1:20211101-3.el9.x86_64 2024-05-17T22:24:59.687 INFO:teuthology.orchestra.run.smithi012.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-17T22:24:59.688 INFO:teuthology.orchestra.run.smithi012.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-17T22:24:59.688 INFO:teuthology.orchestra.run.smithi012.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-17T22:24:59.688 INFO:teuthology.orchestra.run.smithi012.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-17T22:24:59.688 INFO:teuthology.orchestra.run.smithi012.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-17T22:24:59.689 INFO:teuthology.orchestra.run.smithi012.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-17T22:24:59.689 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:24:59.689 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-05-17T22:25:00.127 DEBUG:teuthology.parallel:result is None 2024-05-17T22:25:00.128 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=931809a9e5d295bd61984693cac44fe131ded4bc 2024-05-17T22:25:00.279 DEBUG:teuthology.orchestra.run.smithi012:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-17T22:25:00.303 INFO:teuthology.orchestra.run.smithi012.stdout:19.0.0-3794.g931809a9.el9 2024-05-17T22:25:00.303 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3794.g931809a9.el9 2024-05-17T22:25:00.303 INFO:teuthology.task.install:The correct ceph version 19.0.0-3794.g931809a9 is installed. 2024-05-17T22:25:00.305 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=931809a9e5d295bd61984693cac44fe131ded4bc 2024-05-17T22:25:00.447 DEBUG:teuthology.orchestra.run.smithi096:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-17T22:25:00.473 INFO:teuthology.orchestra.run.smithi096.stdout:19.0.0-3794.g931809a9.el9 2024-05-17T22:25:00.473 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3794.g931809a9.el9 2024-05-17T22:25:00.473 INFO:teuthology.task.install:The correct ceph version 19.0.0-3794.g931809a9 is installed. 2024-05-17T22:25:00.475 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=931809a9e5d295bd61984693cac44fe131ded4bc 2024-05-17T22:25:00.615 DEBUG:teuthology.orchestra.run.smithi151:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-17T22:25:00.642 INFO:teuthology.orchestra.run.smithi151.stdout:19.0.0-3794.g931809a9.el9 2024-05-17T22:25:00.642 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3794.g931809a9.el9 2024-05-17T22:25:00.642 INFO:teuthology.task.install:The correct ceph version 19.0.0-3794.g931809a9 is installed. 2024-05-17T22:25:00.644 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-17T22:25:00.644 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-17T22:25:00.644 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-17T22:25:00.676 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-05-17T22:25:00.676 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-17T22:25:00.708 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-05-17T22:25:00.708 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-17T22:25:00.738 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-17T22:25:00.738 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-17T22:25:00.738 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/daemon-helper 2024-05-17T22:25:00.768 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-17T22:25:00.837 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-05-17T22:25:00.837 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/daemon-helper 2024-05-17T22:25:00.865 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-17T22:25:00.931 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-05-17T22:25:00.931 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/daemon-helper 2024-05-17T22:25:00.959 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-17T22:25:01.027 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-17T22:25:01.027 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-17T22:25:01.027 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-17T22:25:01.057 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-17T22:25:01.126 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-05-17T22:25:01.127 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-17T22:25:01.153 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-17T22:25:01.217 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-05-17T22:25:01.217 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-17T22:25:01.240 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-17T22:25:01.305 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-17T22:25:01.305 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-17T22:25:01.305 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/stdin-killer 2024-05-17T22:25:01.334 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-17T22:25:01.403 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-05-17T22:25:01.403 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/stdin-killer 2024-05-17T22:25:01.426 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-17T22:25:01.498 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-05-17T22:25:01.498 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/stdin-killer 2024-05-17T22:25:01.522 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-17T22:25:01.585 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-17T22:25:01.691 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '931809a9e5d295bd61984693cac44fe131ded4bc', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-05-17T22:25:01.691 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc 2024-05-17T22:25:01.692 INFO:tasks.cephadm:Cluster fsid is 4d593780-149c-11ef-bc9a-c7b262605968 2024-05-17T22:25:01.692 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-17T22:25:01.692 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.12', 'mon.b': '172.21.15.96', 'mon.c': '172.21.15.151'} 2024-05-17T22:25:01.692 INFO:tasks.cephadm:First mon is mon.a on smithi012 2024-05-17T22:25:01.692 INFO:tasks.cephadm:First mgr is x 2024-05-17T22:25:01.692 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-17T22:25:01.692 DEBUG:teuthology.orchestra.run.smithi012:> sudo hostname $(hostname -s) 2024-05-17T22:25:01.723 DEBUG:teuthology.orchestra.run.smithi096:> sudo hostname $(hostname -s) 2024-05-17T22:25:01.747 DEBUG:teuthology.orchestra.run.smithi151:> sudo hostname $(hostname -s) 2024-05-17T22:25:01.772 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-17T22:25:01.772 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=931809a9e5d295bd61984693cac44fe131ded4bc 2024-05-17T22:25:01.915 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '931809a9e5d295bd61984693cac44fe131ded4bc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79481/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3794-g931809a9', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3794.g931809a9'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-17 02:49:31.325180', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240517.012458-debug', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '931809a9e5d295bd61984693cac44fe131ded4bc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79479/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3794-g931809a9', 'node_name': '172.21.5.34+adami04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3794.g931809a9'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-17 02:47:11.778341', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240517.012458-debug', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-05-17T22:25:02.140 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-pdonnell-testing-20240517.012458-debug, sha1 931809a9e5d295bd61984693cac44fe131ded4bc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=931809a9e5d295bd61984693cac44fe131ded4bc 2024-05-17T22:25:02.141 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/centos/9/x86_64/flavors/default/cephadm 2024-05-17T22:25:02.141 INFO:tasks.cephadm:Downloading cephadm from url: https://2.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/centos/9/x86_64/flavors/default/cephadm 2024-05-17T22:25:02.142 DEBUG:teuthology.orchestra.run.smithi012:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-17T22:25:02.602 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r--. 1 ubuntu ubuntu 782889 May 17 22:25 /home/ubuntu/cephtest/cephadm 2024-05-17T22:25:02.602 DEBUG:teuthology.orchestra.run.smithi096:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-17T22:25:03.034 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-r--r--. 1 ubuntu ubuntu 782889 May 17 22:25 /home/ubuntu/cephtest/cephadm 2024-05-17T22:25:03.035 DEBUG:teuthology.orchestra.run.smithi151:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-17T22:25:03.513 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r--. 1 ubuntu ubuntu 782889 May 17 22:25 /home/ubuntu/cephtest/cephadm 2024-05-17T22:25:03.514 DEBUG:teuthology.orchestra.run.smithi012:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-17T22:25:03.534 DEBUG:teuthology.orchestra.run.smithi096:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-17T22:25:03.553 DEBUG:teuthology.orchestra.run.smithi151:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-17T22:25:03.585 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc on all hosts... 2024-05-17T22:25:03.585 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc pull 2024-05-17T22:25:03.588 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc pull 2024-05-17T22:25:03.595 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc pull 2024-05-17T22:25:04.413 INFO:teuthology.orchestra.run.smithi151.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc... 2024-05-17T22:25:04.432 INFO:teuthology.orchestra.run.smithi012.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc... 2024-05-17T22:25:04.526 INFO:teuthology.orchestra.run.smithi096.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc... 2024-05-17T22:25:46.087 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-05-17T22:25:46.087 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_version": "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)", 2024-05-17T22:25:46.087 INFO:teuthology.orchestra.run.smithi096.stdout: "image_id": "8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c", 2024-05-17T22:25:46.088 INFO:teuthology.orchestra.run.smithi096.stdout: "repo_digests": [ 2024-05-17T22:25:46.088 INFO:teuthology.orchestra.run.smithi096.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b" 2024-05-17T22:25:46.088 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-05-17T22:25:46.088 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-05-17T22:25:50.552 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-17T22:25:50.552 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_version": "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)", 2024-05-17T22:25:50.552 INFO:teuthology.orchestra.run.smithi151.stdout: "image_id": "8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c", 2024-05-17T22:25:50.552 INFO:teuthology.orchestra.run.smithi151.stdout: "repo_digests": [ 2024-05-17T22:25:50.552 INFO:teuthology.orchestra.run.smithi151.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b" 2024-05-17T22:25:50.552 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-05-17T22:25:50.552 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-17T22:25:58.431 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:25:58.431 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_version": "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)", 2024-05-17T22:25:58.431 INFO:teuthology.orchestra.run.smithi012.stdout: "image_id": "8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c", 2024-05-17T22:25:58.431 INFO:teuthology.orchestra.run.smithi012.stdout: "repo_digests": [ 2024-05-17T22:25:58.431 INFO:teuthology.orchestra.run.smithi012.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b" 2024-05-17T22:25:58.431 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-05-17T22:25:58.431 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:25:58.456 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /etc/ceph 2024-05-17T22:25:58.505 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /etc/ceph 2024-05-17T22:25:58.537 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /etc/ceph 2024-05-17T22:25:58.569 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 777 /etc/ceph 2024-05-17T22:25:58.595 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 777 /etc/ceph 2024-05-17T22:25:58.621 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 777 /etc/ceph 2024-05-17T22:25:58.649 INFO:tasks.cephadm:Writing seed config... 2024-05-17T22:25:58.650 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-17T22:25:58.650 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-17T22:25:58.650 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-17T22:25:58.650 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-17T22:25:58.650 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-17T22:25:58.650 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-05-17T22:25:58.650 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-17T22:25:58.650 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-17T22:25:58.651 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-17T22:25:58.651 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-17T22:25:58.651 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-17T22:25:58.651 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-17T22:25:58.651 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-17T22:25:58.651 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-17T22:25:58.651 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-17T22:25:58.651 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-17T22:25:58.651 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-17T22:25:58.651 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-05-17T22:25:58.651 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-17T22:25:58.651 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-17T22:25:58.651 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-17T22:25:58.651 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-17T22:25:58.651 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-17T22:25:58.652 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-17T22:25:58.652 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-17T22:25:58.652 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-17T22:25:58.652 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-17T22:25:58.652 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-17T22:25:58.652 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-17T22:25:58.652 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-17T22:25:58.652 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-17T22:25:58.652 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-17T22:25:58.652 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-05-17T22:25:58.652 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-05-17T22:25:58.652 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-17T22:25:58.652 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-17T22:25:58.653 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-05-17T22:25:58.653 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-17T22:25:58.653 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-17T22:25:58.653 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-17T22:25:58.653 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-17T22:25:58.653 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-17T22:25:58.654 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-17T22:25:58.654 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-17T22:25:58.654 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-05-17T22:25:58.654 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-17T22:25:58.655 DEBUG:teuthology.orchestra.run.smithi012:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-17T22:25:58.674 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 4d593780-149c-11ef-bc9a-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-17T22:25:58.675 DEBUG:teuthology.orchestra.run.smithi012:mon.a> sudo journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@mon.a.service 2024-05-17T22:25:58.717 DEBUG:teuthology.orchestra.run.smithi012:mgr.x> sudo journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@mgr.x.service 2024-05-17T22:25:58.759 INFO:tasks.cephadm:Bootstrapping... 2024-05-17T22:25:58.759 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc -v bootstrap --fsid 4d593780-149c-11ef-bc9a-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.12 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-17T22:25:58.958 INFO:teuthology.orchestra.run.smithi012.stdout:-------------------------------------------------------------------------------- 2024-05-17T22:25:58.959 INFO:teuthology.orchestra.run.smithi012.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc', '-v', 'bootstrap', '--fsid', '4d593780-149c-11ef-bc9a-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.12', '--skip-admin-label'] 2024-05-17T22:25:58.959 INFO:teuthology.orchestra.run.smithi012.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-17T22:25:58.959 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying podman|docker is present... 2024-05-17T22:25:58.986 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stdout 5.0.2 2024-05-17T22:25:58.986 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying lvm2 is present... 2024-05-17T22:25:58.986 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying time synchronization is in place... 2024-05-17T22:25:58.993 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-17T22:25:58.994 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-17T22:25:59.000 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-17T22:25:59.000 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-05-17T22:25:59.007 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout enabled 2024-05-17T22:25:59.013 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout active 2024-05-17T22:25:59.014 INFO:teuthology.orchestra.run.smithi012.stdout:Unit chronyd.service is enabled and running 2024-05-17T22:25:59.014 INFO:teuthology.orchestra.run.smithi012.stdout:Repeating the final host check... 2024-05-17T22:25:59.038 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stdout 5.0.2 2024-05-17T22:25:59.038 INFO:teuthology.orchestra.run.smithi012.stdout:podman (/bin/podman) version 5.0.2 is present 2024-05-17T22:25:59.038 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl is present 2024-05-17T22:25:59.038 INFO:teuthology.orchestra.run.smithi012.stdout:lvcreate is present 2024-05-17T22:25:59.045 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-17T22:25:59.045 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-17T22:25:59.051 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-17T22:25:59.051 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-05-17T22:25:59.058 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout enabled 2024-05-17T22:25:59.065 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout active 2024-05-17T22:25:59.065 INFO:teuthology.orchestra.run.smithi012.stdout:Unit chronyd.service is enabled and running 2024-05-17T22:25:59.065 INFO:teuthology.orchestra.run.smithi012.stdout:Host looks OK 2024-05-17T22:25:59.066 INFO:teuthology.orchestra.run.smithi012.stdout:Cluster fsid: 4d593780-149c-11ef-bc9a-c7b262605968 2024-05-17T22:25:59.066 INFO:teuthology.orchestra.run.smithi012.stdout:Acquiring lock 139793931628064 on /run/cephadm/4d593780-149c-11ef-bc9a-c7b262605968.lock 2024-05-17T22:25:59.066 INFO:teuthology.orchestra.run.smithi012.stdout:Lock 139793931628064 acquired on /run/cephadm/4d593780-149c-11ef-bc9a-c7b262605968.lock 2024-05-17T22:25:59.066 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying IP 172.21.15.12 port 3300 ... 2024-05-17T22:25:59.067 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying IP 172.21.15.12 port 6789 ... 2024-05-17T22:25:59.067 INFO:teuthology.orchestra.run.smithi012.stdout:Base mon IP(s) is [172.21.15.12:3300, 172.21.15.12:6789], mon addrv is [v2:172.21.15.12:3300,v1:172.21.15.12:6789] 2024-05-17T22:25:59.070 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.12 metric 100 2024-05-17T22:25:59.070 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.12 metric 100 2024-05-17T22:25:59.073 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-17T22:25:59.073 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-05-17T22:25:59.073 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-05-17T22:25:59.076 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-17T22:25:59.077 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-17T22:25:59.077 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-17T22:25:59.077 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout 3: ens1f0: mtu 1500 state UP qlen 1000 2024-05-17T22:25:59.077 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fef8:25ea/64 scope link noprefixroute 2024-05-17T22:25:59.077 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-17T22:25:59.078 INFO:teuthology.orchestra.run.smithi012.stdout:Mon IP `172.21.15.12` is in CIDR network `172.21.0.0/20` 2024-05-17T22:25:59.078 INFO:teuthology.orchestra.run.smithi012.stdout:Mon IP `172.21.15.12` is in CIDR network `172.21.0.0/20` 2024-05-17T22:25:59.078 INFO:teuthology.orchestra.run.smithi012.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-17T22:25:59.079 INFO:teuthology.orchestra.run.smithi012.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-17T22:25:59.079 INFO:teuthology.orchestra.run.smithi012.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc... 2024-05-17T22:25:59.575 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stdout 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c 2024-05-17T22:25:59.575 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc... 2024-05-17T22:25:59.575 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Getting image source signatures 2024-05-17T22:25:59.575 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Copying blob sha256:03bb380d3011cf6565f6cdcba1828bdc1bc4b91a44dc7ba4b31647b2a6675cd0 2024-05-17T22:25:59.575 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Copying blob sha256:d0e808cd455827ca3108db1b0125dcee9ffc148daa1d7dde963f550bc831d501 2024-05-17T22:25:59.575 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Copying config sha256:8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c 2024-05-17T22:25:59.575 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-17T22:26:00.471 INFO:teuthology.orchestra.run.smithi012.stdout:ceph: stdout ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev) 2024-05-17T22:26:00.471 INFO:teuthology.orchestra.run.smithi012.stdout:Ceph version: ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev) 2024-05-17T22:26:00.471 INFO:teuthology.orchestra.run.smithi012.stdout:Extracting ceph user uid/gid from container image... 2024-05-17T22:26:01.301 INFO:teuthology.orchestra.run.smithi012.stdout:stat: stdout 167 167 2024-05-17T22:26:01.301 INFO:teuthology.orchestra.run.smithi012.stdout:Creating initial keys... 2024-05-17T22:26:02.174 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph-authtool: stdout AQB52UdmIXpWJxAAVNqqFCM4LfNgaFe89FWw6Q== 2024-05-17T22:26:03.046 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph-authtool: stdout AQB62UdmLbqUHhAAmh2mdwaV15Dog4Ypp+Fo2g== 2024-05-17T22:26:04.017 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph-authtool: stdout AQB72UdmjL71GBAA5lzhi8nxA2XhMQm6vw5S+w== 2024-05-17T22:26:04.018 INFO:teuthology.orchestra.run.smithi012.stdout:Creating initial monmap... 2024-05-17T22:26:08.264 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-17T22:26:08.265 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-17T22:26:08.265 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 4d593780-149c-11ef-bc9a-c7b262605968 2024-05-17T22:26:08.265 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-17T22:26:08.265 INFO:teuthology.orchestra.run.smithi012.stdout:monmaptool for a [v2:172.21.15.12:3300,v1:172.21.15.12:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-17T22:26:08.265 INFO:teuthology.orchestra.run.smithi012.stdout:setting min_mon_release = quincy 2024-05-17T22:26:08.265 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: set fsid to 4d593780-149c-11ef-bc9a-c7b262605968 2024-05-17T22:26:08.265 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-17T22:26:08.265 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:26:08.265 INFO:teuthology.orchestra.run.smithi012.stdout:Creating mon... 2024-05-17T22:26:15.253 INFO:teuthology.orchestra.run.smithi012.stdout:create mon.a on 2024-05-17T22:26:15.489 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-17T22:26:15.684 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-17T22:26:16.045 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-4d593780-149c-11ef-bc9a-c7b262605968.target → /etc/systemd/system/ceph-4d593780-149c-11ef-bc9a-c7b262605968.target. 2024-05-17T22:26:16.045 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-4d593780-149c-11ef-bc9a-c7b262605968.target → /etc/systemd/system/ceph-4d593780-149c-11ef-bc9a-c7b262605968.target. 2024-05-17T22:26:16.287 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-4d593780-149c-11ef-bc9a-c7b262605968@mon.a 2024-05-17T22:26:16.287 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Failed to reset failed state of unit ceph-4d593780-149c-11ef-bc9a-c7b262605968@mon.a.service: Unit ceph-4d593780-149c-11ef-bc9a-c7b262605968@mon.a.service not loaded. 2024-05-17T22:26:16.457 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-4d593780-149c-11ef-bc9a-c7b262605968.target.wants/ceph-4d593780-149c-11ef-bc9a-c7b262605968@mon.a.service → /etc/systemd/system/ceph-4d593780-149c-11ef-bc9a-c7b262605968@.service. 2024-05-17T22:26:17.011 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-17T22:26:17.011 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout disabled 2024-05-17T22:26:17.019 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-17T22:26:17.019 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-05-17T22:26:17.019 INFO:teuthology.orchestra.run.smithi012.stdout:firewalld.service is not enabled 2024-05-17T22:26:17.019 INFO:teuthology.orchestra.run.smithi012.stdout:Not possible to enable service . firewalld.service is not available 2024-05-17T22:26:17.019 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mon to start... 2024-05-17T22:26:17.019 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mon... 2024-05-17T22:26:18.060 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:17 smithi012 ceph-mon[31260]: from='client.? 172.21.15.12:0/1826829290' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-17T22:26:18.068 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout cluster: 2024-05-17T22:26:18.068 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout id: 4d593780-149c-11ef-bc9a-c7b262605968 2024-05-17T22:26:18.068 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-17T22:26:18.068 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-05-17T22:26:18.068 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout services: 2024-05-17T22:26:18.068 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.595823s) 2024-05-17T22:26:18.068 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-17T22:26:18.068 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-17T22:26:18.068 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-05-17T22:26:18.068 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout data: 2024-05-17T22:26:18.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-17T22:26:18.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-17T22:26:18.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-17T22:26:18.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout pgs: 2024-05-17T22:26:18.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-05-17T22:26:18.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.466+0000 7f171c59e700 1 Processor -- start 2024-05-17T22:26:18.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.466+0000 7f171c59e700 1 -- start start 2024-05-17T22:26:18.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.467+0000 7f171c59e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f171407f690 0x7f171407dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:18.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.467+0000 7f171c59e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f171407e090 con 0x7f171407f690 2024-05-17T22:26:18.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.467+0000 7f171a33a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f171407f690 0x7f171407dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:26:18.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.467+0000 7f171a33a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f171407f690 0x7f171407dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35920/0 (socket says 172.21.15.12:35920) 2024-05-17T22:26:18.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.467+0000 7f171a33a700 1 -- 172.21.15.12:0/823184773 learned_addr learned my addr 172.21.15.12:0/823184773 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:26:18.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.467+0000 7f171a33a700 1 -- 172.21.15.12:0/823184773 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f171407e210 con 0x7f171407f690 2024-05-17T22:26:18.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.468+0000 7f171a33a700 1 --2- 172.21.15.12:0/823184773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f171407f690 0x7f171407dac0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f17100098a0 tx=0x7f1710009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29321cd00d3b71ae server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:26:18.070 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.468+0000 7f1719338700 1 -- 172.21.15.12:0/823184773 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1710004160 con 0x7f171407f690 2024-05-17T22:26:18.070 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.468+0000 7f1719338700 1 -- 172.21.15.12:0/823184773 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1710004300 con 0x7f171407f690 2024-05-17T22:26:18.070 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.468+0000 7f1719338700 1 -- 172.21.15.12:0/823184773 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17100036a0 con 0x7f171407f690 2024-05-17T22:26:18.070 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.468+0000 7f171c59e700 1 -- 172.21.15.12:0/823184773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f171407f690 msgr2=0x7f171407dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:26:18.070 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.468+0000 7f171c59e700 1 --2- 172.21.15.12:0/823184773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f171407f690 0x7f171407dac0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f17100098a0 tx=0x7f1710009bb0 comp rx=0 tx=0).stop 2024-05-17T22:26:18.070 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.469+0000 7f171c59e700 1 -- 172.21.15.12:0/823184773 shutdown_connections 2024-05-17T22:26:18.070 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.469+0000 7f171c59e700 1 --2- 172.21.15.12:0/823184773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f171407f690 0x7f171407dac0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:18.070 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.469+0000 7f171c59e700 1 -- 172.21.15.12:0/823184773 >> 172.21.15.12:0/823184773 conn(0x7f1714104c00 msgr2=0x7f17141070d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:26:18.070 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.469+0000 7f171c59e700 1 -- 172.21.15.12:0/823184773 shutdown_connections 2024-05-17T22:26:18.070 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.469+0000 7f171c59e700 1 -- 172.21.15.12:0/823184773 wait complete. 2024-05-17T22:26:18.070 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.469+0000 7f171c59e700 1 Processor -- start 2024-05-17T22:26:18.070 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.469+0000 7f171c59e700 1 -- start start 2024-05-17T22:26:18.070 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.470+0000 7f171c59e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f171407f690 0x7f171419f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:18.071 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.470+0000 7f171c59e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f171407e740 con 0x7f171407f690 2024-05-17T22:26:18.071 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.470+0000 7f171a33a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f171407f690 0x7f171419f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:26:18.071 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.470+0000 7f171a33a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f171407f690 0x7f171419f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35922/0 (socket says 172.21.15.12:35922) 2024-05-17T22:26:18.071 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.470+0000 7f171a33a700 1 -- 172.21.15.12:0/1826829290 learned_addr learned my addr 172.21.15.12:0/1826829290 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:26:18.071 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.470+0000 7f171a33a700 1 -- 172.21.15.12:0/1826829290 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f171419f600 con 0x7f171407f690 2024-05-17T22:26:18.071 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.470+0000 7f171a33a700 1 --2- 172.21.15.12:0/1826829290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f171407f690 0x7f171419f0c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1710000c00 tx=0x7f1710004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:26:18.071 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.470+0000 7f17077fe700 1 -- 172.21.15.12:0/1826829290 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1710003db0 con 0x7f171407f690 2024-05-17T22:26:18.071 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.470+0000 7f17077fe700 1 -- 172.21.15.12:0/1826829290 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1710011630 con 0x7f171407f690 2024-05-17T22:26:18.071 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.470+0000 7f171c59e700 1 -- 172.21.15.12:0/1826829290 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f171419f890 con 0x7f171407f690 2024-05-17T22:26:18.071 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.471+0000 7f17077fe700 1 -- 172.21.15.12:0/1826829290 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1710011950 con 0x7f171407f690 2024-05-17T22:26:18.071 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.471+0000 7f171c59e700 1 -- 172.21.15.12:0/1826829290 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17141a24a0 con 0x7f171407f690 2024-05-17T22:26:18.071 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.471+0000 7f17077fe700 1 -- 172.21.15.12:0/1826829290 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f171001a460 con 0x7f171407f690 2024-05-17T22:26:18.072 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.471+0000 7f17077fe700 1 -- 172.21.15.12:0/1826829290 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f171001b780 con 0x7f171407f690 2024-05-17T22:26:18.072 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.472+0000 7f171c59e700 1 -- 172.21.15.12:0/1826829290 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f171407db40 con 0x7f171407f690 2024-05-17T22:26:18.072 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.474+0000 7f17077fe700 1 -- 172.21.15.12:0/1826829290 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f1710031020 con 0x7f171407f690 2024-05-17T22:26:18.072 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.548+0000 7f171c59e700 1 -- 172.21.15.12:0/1826829290 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f17140644a0 con 0x7f171407f690 2024-05-17T22:26:18.072 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.548+0000 7f17077fe700 1 -- 172.21.15.12:0/1826829290 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f171001ba20 con 0x7f171407f690 2024-05-17T22:26:18.072 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.549+0000 7f171c59e700 1 -- 172.21.15.12:0/1826829290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f171407f690 msgr2=0x7f171419f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:26:18.072 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.549+0000 7f171c59e700 1 --2- 172.21.15.12:0/1826829290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f171407f690 0x7f171419f0c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1710000c00 tx=0x7f1710004750 comp rx=0 tx=0).stop 2024-05-17T22:26:18.072 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.550+0000 7f171c59e700 1 -- 172.21.15.12:0/1826829290 shutdown_connections 2024-05-17T22:26:18.072 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.550+0000 7f171c59e700 1 --2- 172.21.15.12:0/1826829290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f171407f690 0x7f171419f0c0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:18.072 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.550+0000 7f171c59e700 1 -- 172.21.15.12:0/1826829290 >> 172.21.15.12:0/1826829290 conn(0x7f1714104c00 msgr2=0x7f1714105860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:26:18.072 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.550+0000 7f171c59e700 1 -- 172.21.15.12:0/1826829290 shutdown_connections 2024-05-17T22:26:18.072 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:17.550+0000 7f171c59e700 1 -- 172.21.15.12:0/1826829290 wait complete. 2024-05-17T22:26:18.072 INFO:teuthology.orchestra.run.smithi012.stdout:mon is available 2024-05-17T22:26:18.073 INFO:teuthology.orchestra.run.smithi012.stdout:Assimilating anything we can from ceph.conf... 2024-05-17T22:26:19.111 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-05-17T22:26:19.117 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [global] 2024-05-17T22:26:19.117 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout fsid = 4d593780-149c-11ef-bc9a-c7b262605968 2024-05-17T22:26:19.117 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-17T22:26:19.117 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.12:3300,v1:172.21.15.12:6789] 2024-05-17T22:26:19.117 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-17T22:26:19.117 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-17T22:26:19.117 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-17T22:26:19.117 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-17T22:26:19.117 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-05-17T22:26:19.118 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-17T22:26:19.118 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-17T22:26:19.118 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-05-17T22:26:19.118 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [osd] 2024-05-17T22:26:19.118 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-17T22:26:19.118 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-17T22:26:19.118 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-17T22:26:19.118 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-17T22:26:19.118 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-17T22:26:19.118 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.520+0000 7f42d609d700 1 Processor -- start 2024-05-17T22:26:19.118 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.521+0000 7f42d609d700 1 -- start start 2024-05-17T22:26:19.118 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.521+0000 7f42d609d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42d0109ec0 0x7f42d010c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:19.118 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.521+0000 7f42d609d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f42d007d790 con 0x7f42d0109ec0 2024-05-17T22:26:19.119 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.521+0000 7f42cf7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42d0109ec0 0x7f42d010c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:26:19.119 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.521+0000 7f42cf7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42d0109ec0 0x7f42d010c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35932/0 (socket says 172.21.15.12:35932) 2024-05-17T22:26:19.119 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.521+0000 7f42cf7fe700 1 -- 172.21.15.12:0/3430976626 learned_addr learned my addr 172.21.15.12:0/3430976626 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:26:19.119 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.521+0000 7f42cf7fe700 1 -- 172.21.15.12:0/3430976626 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d010c820 con 0x7f42d0109ec0 2024-05-17T22:26:19.119 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.522+0000 7f42cf7fe700 1 --2- 172.21.15.12:0/3430976626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42d0109ec0 0x7f42d010c2e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f42c00098a0 tx=0x7f42c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28e15c8a23800b1f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:26:19.119 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.522+0000 7f42ce7fc700 1 -- 172.21.15.12:0/3430976626 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42c0004160 con 0x7f42d0109ec0 2024-05-17T22:26:19.119 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.522+0000 7f42ce7fc700 1 -- 172.21.15.12:0/3430976626 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42c0004300 con 0x7f42d0109ec0 2024-05-17T22:26:19.119 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.522+0000 7f42ce7fc700 1 -- 172.21.15.12:0/3430976626 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42c00036a0 con 0x7f42d0109ec0 2024-05-17T22:26:19.119 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.522+0000 7f42d609d700 1 -- 172.21.15.12:0/3430976626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42d0109ec0 msgr2=0x7f42d010c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:26:19.119 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.522+0000 7f42d609d700 1 --2- 172.21.15.12:0/3430976626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42d0109ec0 0x7f42d010c2e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f42c00098a0 tx=0x7f42c0009bb0 comp rx=0 tx=0).stop 2024-05-17T22:26:19.119 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.522+0000 7f42d609d700 1 -- 172.21.15.12:0/3430976626 shutdown_connections 2024-05-17T22:26:19.119 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.523+0000 7f42d609d700 1 --2- 172.21.15.12:0/3430976626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42d0109ec0 0x7f42d010c2e0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:19.119 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.523+0000 7f42d609d700 1 -- 172.21.15.12:0/3430976626 >> 172.21.15.12:0/3430976626 conn(0x7f42d0103ab0 msgr2=0x7f42d0105f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:26:19.119 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.523+0000 7f42d609d700 1 -- 172.21.15.12:0/3430976626 shutdown_connections 2024-05-17T22:26:19.120 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.523+0000 7f42d609d700 1 -- 172.21.15.12:0/3430976626 wait complete. 2024-05-17T22:26:19.120 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.523+0000 7f42d609d700 1 Processor -- start 2024-05-17T22:26:19.120 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.523+0000 7f42d609d700 1 -- start start 2024-05-17T22:26:19.120 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.524+0000 7f42d609d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42d0109ec0 0x7f42d019ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:19.120 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.524+0000 7f42d609d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f42d010cc30 con 0x7f42d0109ec0 2024-05-17T22:26:19.120 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.524+0000 7f42cf7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42d0109ec0 0x7f42d019ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:26:19.120 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.524+0000 7f42cf7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42d0109ec0 0x7f42d019ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35934/0 (socket says 172.21.15.12:35934) 2024-05-17T22:26:19.120 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.524+0000 7f42cf7fe700 1 -- 172.21.15.12:0/2907442701 learned_addr learned my addr 172.21.15.12:0/2907442701 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:26:19.120 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.524+0000 7f42cf7fe700 1 -- 172.21.15.12:0/2907442701 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d019d3c0 con 0x7f42d0109ec0 2024-05-17T22:26:19.120 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.524+0000 7f42cf7fe700 1 --2- 172.21.15.12:0/2907442701 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42d0109ec0 0x7f42d019ce80 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f42c0000c00 tx=0x7f42c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:26:19.120 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.524+0000 7f42ccff9700 1 -- 172.21.15.12:0/2907442701 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42c0003db0 con 0x7f42d0109ec0 2024-05-17T22:26:19.120 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.524+0000 7f42ccff9700 1 -- 172.21.15.12:0/2907442701 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42c0011630 con 0x7f42d0109ec0 2024-05-17T22:26:19.120 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.524+0000 7f42d609d700 1 -- 172.21.15.12:0/2907442701 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42d019d650 con 0x7f42d0109ec0 2024-05-17T22:26:19.121 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.524+0000 7f42ccff9700 1 -- 172.21.15.12:0/2907442701 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42c0011950 con 0x7f42d0109ec0 2024-05-17T22:26:19.121 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.524+0000 7f42d609d700 1 -- 172.21.15.12:0/2907442701 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42d01a0260 con 0x7f42d0109ec0 2024-05-17T22:26:19.121 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.525+0000 7f42ccff9700 1 -- 172.21.15.12:0/2907442701 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f42c0011af0 con 0x7f42d0109ec0 2024-05-17T22:26:19.121 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.525+0000 7f42ccff9700 1 -- 172.21.15.12:0/2907442701 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f42c001c790 con 0x7f42d0109ec0 2024-05-17T22:26:19.121 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.526+0000 7f42d609d700 1 -- 172.21.15.12:0/2907442701 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42d010a350 con 0x7f42d0109ec0 2024-05-17T22:26:19.121 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.528+0000 7f42ccff9700 1 -- 172.21.15.12:0/2907442701 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f42c0032020 con 0x7f42d0109ec0 2024-05-17T22:26:19.121 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.598+0000 7f42d609d700 1 -- 172.21.15.12:0/2907442701 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f42d0064510 con 0x7f42d0109ec0 2024-05-17T22:26:19.121 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.602+0000 7f42ccff9700 1 -- 172.21.15.12:0/2907442701 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+465 (secure 0 0 0) 0x7f42c0004260 con 0x7f42d0109ec0 2024-05-17T22:26:19.121 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.603+0000 7f42ccff9700 1 -- 172.21.15.12:0/2907442701 <== mon.0 v2:172.21.15.12:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f42c001ce20 con 0x7f42d0109ec0 2024-05-17T22:26:19.121 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.603+0000 7f42d609d700 1 -- 172.21.15.12:0/2907442701 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42d0109ec0 msgr2=0x7f42d019ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:26:19.121 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.603+0000 7f42d609d700 1 --2- 172.21.15.12:0/2907442701 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42d0109ec0 0x7f42d019ce80 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f42c0000c00 tx=0x7f42c0004750 comp rx=0 tx=0).stop 2024-05-17T22:26:19.121 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.604+0000 7f42d609d700 1 -- 172.21.15.12:0/2907442701 shutdown_connections 2024-05-17T22:26:19.121 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.604+0000 7f42d609d700 1 --2- 172.21.15.12:0/2907442701 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42d0109ec0 0x7f42d019ce80 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:19.121 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.604+0000 7f42d609d700 1 -- 172.21.15.12:0/2907442701 >> 172.21.15.12:0/2907442701 conn(0x7f42d0103ab0 msgr2=0x7f42d0104860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:26:19.121 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.604+0000 7f42d609d700 1 -- 172.21.15.12:0/2907442701 shutdown_connections 2024-05-17T22:26:19.122 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:18.604+0000 7f42d609d700 1 -- 172.21.15.12:0/2907442701 wait complete. 2024-05-17T22:26:19.122 INFO:teuthology.orchestra.run.smithi012.stdout:Generating new minimal ceph.conf... 2024-05-17T22:26:19.225 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:18 smithi012 ceph-mon[31260]: from='client.? 172.21.15.12:0/2907442701' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-17T22:26:19.225 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:18 smithi012 ceph-mon[31260]: from='client.? 172.21.15.12:0/2907442701' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-17T22:26:20.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:19 smithi012 ceph-mon[31260]: from='client.? 172.21.15.12:0/2209998639' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:26:21.309 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.540+0000 7f58078ac700 1 Processor -- start 2024-05-17T22:26:21.309 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.540+0000 7f58078ac700 1 -- start start 2024-05-17T22:26:21.309 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.540+0000 7f58078ac700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5800109e90 0x7f580010c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:21.309 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.540+0000 7f58078ac700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f580007d6a0 con 0x7f5800109e90 2024-05-17T22:26:21.309 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.541+0000 7f5805648700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5800109e90 0x7f580010c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:26:21.309 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.541+0000 7f5805648700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5800109e90 0x7f580010c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35946/0 (socket says 172.21.15.12:35946) 2024-05-17T22:26:21.309 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.541+0000 7f5805648700 1 -- 172.21.15.12:0/2714145401 learned_addr learned my addr 172.21.15.12:0/2714145401 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:26:21.309 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.541+0000 7f5805648700 1 -- 172.21.15.12:0/2714145401 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f580010c7f0 con 0x7f5800109e90 2024-05-17T22:26:21.309 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.541+0000 7f5805648700 1 --2- 172.21.15.12:0/2714145401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5800109e90 0x7f580010c2b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f57f0009e80 tx=0x7f57f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8676a9889710306d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:26:21.309 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.541+0000 7f57fffff700 1 -- 172.21.15.12:0/2714145401 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57f0017070 con 0x7f5800109e90 2024-05-17T22:26:21.309 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.541+0000 7f57fffff700 1 -- 172.21.15.12:0/2714145401 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f57f000b8c0 con 0x7f5800109e90 2024-05-17T22:26:21.310 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.541+0000 7f57fffff700 1 -- 172.21.15.12:0/2714145401 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57f0004490 con 0x7f5800109e90 2024-05-17T22:26:21.310 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.542+0000 7f58078ac700 1 -- 172.21.15.12:0/2714145401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5800109e90 msgr2=0x7f580010c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:26:21.310 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.542+0000 7f58078ac700 1 --2- 172.21.15.12:0/2714145401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5800109e90 0x7f580010c2b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f57f0009e80 tx=0x7f57f000b330 comp rx=0 tx=0).stop 2024-05-17T22:26:21.310 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.542+0000 7f58078ac700 1 -- 172.21.15.12:0/2714145401 shutdown_connections 2024-05-17T22:26:21.310 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.542+0000 7f58078ac700 1 --2- 172.21.15.12:0/2714145401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5800109e90 0x7f580010c2b0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:21.310 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.542+0000 7f58078ac700 1 -- 172.21.15.12:0/2714145401 >> 172.21.15.12:0/2714145401 conn(0x7f5800103ac0 msgr2=0x7f5800105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:26:21.310 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.542+0000 7f58078ac700 1 -- 172.21.15.12:0/2714145401 shutdown_connections 2024-05-17T22:26:21.310 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.542+0000 7f58078ac700 1 -- 172.21.15.12:0/2714145401 wait complete. 2024-05-17T22:26:21.310 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.543+0000 7f58078ac700 1 Processor -- start 2024-05-17T22:26:21.310 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.543+0000 7f58078ac700 1 -- start start 2024-05-17T22:26:21.310 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.543+0000 7f58078ac700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5800109e90 0x7f580019d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:21.310 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.543+0000 7f58078ac700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f580010d7b0 con 0x7f5800109e90 2024-05-17T22:26:21.310 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.543+0000 7f5805648700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5800109e90 0x7f580019d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:26:21.311 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.543+0000 7f5805648700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5800109e90 0x7f580019d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35958/0 (socket says 172.21.15.12:35958) 2024-05-17T22:26:21.311 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.543+0000 7f5805648700 1 -- 172.21.15.12:0/2209998639 learned_addr learned my addr 172.21.15.12:0/2209998639 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:26:21.311 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.544+0000 7f5805648700 1 -- 172.21.15.12:0/2209998639 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f580019dc40 con 0x7f5800109e90 2024-05-17T22:26:21.311 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.544+0000 7f5805648700 1 --2- 172.21.15.12:0/2209998639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5800109e90 0x7f580019d700 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f57f000b7b0 tx=0x7f57f000bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:26:21.311 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.544+0000 7f57fe7fc700 1 -- 172.21.15.12:0/2209998639 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57f0020020 con 0x7f5800109e90 2024-05-17T22:26:21.311 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.544+0000 7f57fe7fc700 1 -- 172.21.15.12:0/2209998639 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f57f0012430 con 0x7f5800109e90 2024-05-17T22:26:21.311 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.544+0000 7f58078ac700 1 -- 172.21.15.12:0/2209998639 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f580019ded0 con 0x7f5800109e90 2024-05-17T22:26:21.311 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.544+0000 7f57fe7fc700 1 -- 172.21.15.12:0/2209998639 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57f0017040 con 0x7f5800109e90 2024-05-17T22:26:21.311 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.544+0000 7f58078ac700 1 -- 172.21.15.12:0/2209998639 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58001a0ae0 con 0x7f5800109e90 2024-05-17T22:26:21.311 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.544+0000 7f57fe7fc700 1 -- 172.21.15.12:0/2209998639 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f57f0023020 con 0x7f5800109e90 2024-05-17T22:26:21.311 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.544+0000 7f57fe7fc700 1 -- 172.21.15.12:0/2209998639 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f57f000f050 con 0x7f5800109e90 2024-05-17T22:26:21.311 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.545+0000 7f58078ac700 1 -- 172.21.15.12:0/2209998639 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f580010a320 con 0x7f5800109e90 2024-05-17T22:26:21.311 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.547+0000 7f57fe7fc700 1 -- 172.21.15.12:0/2209998639 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f57f002a020 con 0x7f5800109e90 2024-05-17T22:26:21.311 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.618+0000 7f58078ac700 1 -- 172.21.15.12:0/2209998639 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f58001a10d0 con 0x7f5800109e90 2024-05-17T22:26:21.311 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.618+0000 7f57fe7fc700 1 -- 172.21.15.12:0/2209998639 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f57f000b860 con 0x7f5800109e90 2024-05-17T22:26:21.312 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.619+0000 7f58078ac700 1 -- 172.21.15.12:0/2209998639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5800109e90 msgr2=0x7f580019d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:26:21.312 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.619+0000 7f58078ac700 1 --2- 172.21.15.12:0/2209998639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5800109e90 0x7f580019d700 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f57f000b7b0 tx=0x7f57f000bd50 comp rx=0 tx=0).stop 2024-05-17T22:26:21.312 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.619+0000 7f58078ac700 1 -- 172.21.15.12:0/2209998639 shutdown_connections 2024-05-17T22:26:21.312 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.619+0000 7f58078ac700 1 --2- 172.21.15.12:0/2209998639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5800109e90 0x7f580019d700 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:21.312 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.619+0000 7f58078ac700 1 -- 172.21.15.12:0/2209998639 >> 172.21.15.12:0/2209998639 conn(0x7f5800103ac0 msgr2=0x7f58001043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:26:21.312 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.619+0000 7f58078ac700 1 -- 172.21.15.12:0/2209998639 shutdown_connections 2024-05-17T22:26:21.312 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:19.619+0000 7f58078ac700 1 -- 172.21.15.12:0/2209998639 wait complete. 2024-05-17T22:26:21.312 INFO:teuthology.orchestra.run.smithi012.stdout:Restarting the monitor... 2024-05-17T22:26:21.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:21 smithi012 systemd[1]: Stopping Ceph mon.a for 4d593780-149c-11ef-bc9a-c7b262605968... 2024-05-17T22:26:21.880 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:21 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mon-a[31256]: 2024-05-17T22:26:21.594+0000 7fa433837700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T22:26:21.880 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:21 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mon-a[31256]: 2024-05-17T22:26:21.594+0000 7fa433837700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-17T22:26:21.880 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:21 smithi012 podman[31506]: 2024-05-17 22:26:21.667423127 +0000 UTC m=+0.160847086 container died 0d590a4089c0ac7cb517a0098ccfc8fb2f9401afb54146bc037c321f2ace0aa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mon-a, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-17T22:26:22.188 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:21 smithi012 podman[31506]: 2024-05-17 22:26:21.978208596 +0000 UTC m=+0.471632557 container cleanup 0d590a4089c0ac7cb517a0098ccfc8fb2f9401afb54146bc037c321f2ace0aa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-17T22:26:22.189 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:21 smithi012 bash[31506]: ceph-4d593780-149c-11ef-bc9a-c7b262605968-mon-a 2024-05-17T22:26:22.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 podman[31519]: 2024-05-17 22:26:22.187487772 +0000 UTC m=+0.510786314 container remove 0d590a4089c0ac7cb517a0098ccfc8fb2f9401afb54146bc037c321f2ace0aa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514) 2024-05-17T22:26:22.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 systemd[1]: ceph-4d593780-149c-11ef-bc9a-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-17T22:26:22.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 systemd[1]: Stopped Ceph mon.a for 4d593780-149c-11ef-bc9a-c7b262605968. 2024-05-17T22:26:22.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 systemd[1]: Starting Ceph mon.a for 4d593780-149c-11ef-bc9a-c7b262605968... 2024-05-17T22:26:22.904 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 podman[31613]: 2024-05-17 22:26:22.535325747 +0000 UTC m=+0.017487430 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc 2024-05-17T22:26:22.904 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 podman[31613]: 2024-05-17 22:26:22.638376346 +0000 UTC m=+0.120538021 container create 094715d271e66aa5d1c22c288d8f4145e04d35b0727765f704ae06c0acb67537 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mon-a, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514) 2024-05-17T22:26:22.904 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 podman[31613]: 2024-05-17 22:26:22.865558532 +0000 UTC m=+0.347720196 container init 094715d271e66aa5d1c22c288d8f4145e04d35b0727765f704ae06c0acb67537 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-17T22:26:22.904 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 podman[31613]: 2024-05-17 22:26:22.868772199 +0000 UTC m=+0.350933864 container start 094715d271e66aa5d1c22c288d8f4145e04d35b0727765f704ae06c0acb67537 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-17T22:26:22.963 INFO:teuthology.orchestra.run.smithi012.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-17T22:26:23.224 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: set uid:gid to 167:167 (ceph:ceph) 2024-05-17T22:26:23.224 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev), process ceph-mon, pid 2 2024-05-17T22:26:23.225 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: pidfile_write: ignore empty --pid-file 2024-05-17T22:26:23.225 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: load: jerasure load: lrc 2024-05-17T22:26:23.225 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: RocksDB version: 7.9.2 2024-05-17T22:26:23.225 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Git sha 0 2024-05-17T22:26:23.225 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Compile date 2024-05-17 01:53:57 2024-05-17T22:26:23.225 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: DB SUMMARY 2024-05-17T22:26:23.225 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: DB Session ID: ZLXQOI6J1GLODDJ4RE9G 2024-05-17T22:26:23.225 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: CURRENT file: CURRENT 2024-05-17T22:26:23.225 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: IDENTITY file: IDENTITY 2024-05-17T22:26:23.225 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-17T22:26:23.225 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-17T22:26:23.225 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104056 ; 2024-05-17T22:26:23.225 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.error_if_exists: 0 2024-05-17T22:26:23.225 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.create_if_missing: 0 2024-05-17T22:26:23.226 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.paranoid_checks: 1 2024-05-17T22:26:23.226 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-17T22:26:23.226 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-17T22:26:23.226 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-17T22:26:23.226 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.env: 0x56145132e500 2024-05-17T22:26:23.226 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.fs: PosixFileSystem 2024-05-17T22:26:23.226 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.info_log: 0x561453914080 2024-05-17T22:26:23.226 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.max_file_opening_threads: 16 2024-05-17T22:26:23.226 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.statistics: (nil) 2024-05-17T22:26:23.226 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.use_fsync: 0 2024-05-17T22:26:23.226 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.max_log_file_size: 0 2024-05-17T22:26:23.226 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-17T22:26:23.226 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-17T22:26:23.227 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.keep_log_file_num: 1000 2024-05-17T22:26:23.227 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.recycle_log_file_num: 0 2024-05-17T22:26:23.227 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.allow_fallocate: 1 2024-05-17T22:26:23.227 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.allow_mmap_reads: 0 2024-05-17T22:26:23.227 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.allow_mmap_writes: 0 2024-05-17T22:26:23.227 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.use_direct_reads: 0 2024-05-17T22:26:23.227 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-17T22:26:23.227 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.create_missing_column_families: 0 2024-05-17T22:26:23.227 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.db_log_dir: 2024-05-17T22:26:23.227 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.wal_dir: 2024-05-17T22:26:23.227 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-17T22:26:23.227 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-17T22:26:23.227 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-17T22:26:23.227 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-17T22:26:23.227 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-17T22:26:23.228 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-17T22:26:23.228 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.advise_random_on_open: 1 2024-05-17T22:26:23.228 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.db_write_buffer_size: 0 2024-05-17T22:26:23.228 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.write_buffer_manager: 0x5614537c8b40 2024-05-17T22:26:23.228 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-17T22:26:23.228 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-17T22:26:23.228 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-17T22:26:23.228 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.rate_limiter: (nil) 2024-05-17T22:26:23.228 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-17T22:26:23.228 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.wal_recovery_mode: 2 2024-05-17T22:26:23.228 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.enable_thread_tracking: 0 2024-05-17T22:26:23.228 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.enable_pipelined_write: 0 2024-05-17T22:26:23.228 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.unordered_write: 0 2024-05-17T22:26:23.228 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-17T22:26:23.229 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-17T22:26:23.229 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-17T22:26:23.229 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-17T22:26:23.229 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.row_cache: None 2024-05-17T22:26:23.229 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.wal_filter: None 2024-05-17T22:26:23.229 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-17T22:26:23.229 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.allow_ingest_behind: 0 2024-05-17T22:26:23.229 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.two_write_queues: 0 2024-05-17T22:26:23.229 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.manual_wal_flush: 0 2024-05-17T22:26:23.229 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.wal_compression: 0 2024-05-17T22:26:23.229 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.atomic_flush: 0 2024-05-17T22:26:23.229 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-17T22:26:23.229 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-17T22:26:23.229 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-17T22:26:23.229 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.log_readahead_size: 0 2024-05-17T22:26:23.230 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-17T22:26:23.230 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.best_efforts_recovery: 0 2024-05-17T22:26:23.230 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-17T22:26:23.230 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-17T22:26:23.230 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.allow_data_in_errors: 0 2024-05-17T22:26:23.230 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.db_host_id: __hostname__ 2024-05-17T22:26:23.230 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-17T22:26:23.231 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.max_background_jobs: 2 2024-05-17T22:26:23.231 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.max_background_compactions: -1 2024-05-17T22:26:23.231 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.max_subcompactions: 1 2024-05-17T22:26:23.232 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-17T22:26:23.232 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-17T22:26:23.232 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-17T22:26:23.232 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.max_total_wal_size: 0 2024-05-17T22:26:23.232 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-17T22:26:23.232 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-17T22:26:23.232 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-17T22:26:23.232 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-17T22:26:23.232 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.max_open_files: -1 2024-05-17T22:26:23.232 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.bytes_per_sync: 0 2024-05-17T22:26:23.232 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-17T22:26:23.232 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-17T22:26:23.232 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.compaction_readahead_size: 0 2024-05-17T22:26:23.232 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.max_background_flushes: -1 2024-05-17T22:26:23.233 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Compression algorithms supported: 2024-05-17T22:26:23.233 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-17T22:26:23.233 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: kZSTD supported: 0 2024-05-17T22:26:23.233 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: kXpressCompression supported: 0 2024-05-17T22:26:23.233 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: kLZ4HCCompression supported: 1 2024-05-17T22:26:23.233 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: kZlibCompression supported: 1 2024-05-17T22:26:23.233 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: kSnappyCompression supported: 1 2024-05-17T22:26:23.233 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: kLZ4Compression supported: 1 2024-05-17T22:26:23.233 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: kBZip2Compression supported: 0 2024-05-17T22:26:23.233 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-17T22:26:23.233 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-17T22:26:23.233 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-17T22:26:23.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-17T22:26:23.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-17T22:26:23.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.merge_operator: 2024-05-17T22:26:23.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.compaction_filter: None 2024-05-17T22:26:23.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.compaction_filter_factory: None 2024-05-17T22:26:23.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.sst_partitioner_factory: None 2024-05-17T22:26:23.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-17T22:26:23.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-17T22:26:23.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561453914240) 2024-05-17T22:26:23.234 INFO:journalctl@ceph.mon.a.smithi012.stdout: cache_index_and_filter_blocks: 1 2024-05-17T22:26:23.234 INFO:journalctl@ceph.mon.a.smithi012.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-17T22:26:23.234 INFO:journalctl@ceph.mon.a.smithi012.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-17T22:26:23.234 INFO:journalctl@ceph.mon.a.smithi012.stdout: pin_top_level_index_and_filter: 1 2024-05-17T22:26:23.234 INFO:journalctl@ceph.mon.a.smithi012.stdout: index_type: 0 2024-05-17T22:26:23.235 INFO:journalctl@ceph.mon.a.smithi012.stdout: data_block_index_type: 0 2024-05-17T22:26:23.235 INFO:journalctl@ceph.mon.a.smithi012.stdout: index_shortening: 1 2024-05-17T22:26:23.235 INFO:journalctl@ceph.mon.a.smithi012.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-17T22:26:23.235 INFO:journalctl@ceph.mon.a.smithi012.stdout: checksum: 4 2024-05-17T22:26:23.235 INFO:journalctl@ceph.mon.a.smithi012.stdout: no_block_cache: 0 2024-05-17T22:26:23.235 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_cache: 0x5614538111f0 2024-05-17T22:26:23.235 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_cache_name: BinnedLRUCache 2024-05-17T22:26:23.235 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_cache_options: 2024-05-17T22:26:23.235 INFO:journalctl@ceph.mon.a.smithi012.stdout: capacity : 536870912 2024-05-17T22:26:23.235 INFO:journalctl@ceph.mon.a.smithi012.stdout: num_shard_bits : 4 2024-05-17T22:26:23.235 INFO:journalctl@ceph.mon.a.smithi012.stdout: strict_capacity_limit : 0 2024-05-17T22:26:23.235 INFO:journalctl@ceph.mon.a.smithi012.stdout: high_pri_pool_ratio: 0.000 2024-05-17T22:26:23.235 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_cache_compressed: (nil) 2024-05-17T22:26:23.236 INFO:journalctl@ceph.mon.a.smithi012.stdout: persistent_cache: (nil) 2024-05-17T22:26:23.236 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_size: 4096 2024-05-17T22:26:23.236 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_size_deviation: 10 2024-05-17T22:26:23.236 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_restart_interval: 16 2024-05-17T22:26:23.236 INFO:journalctl@ceph.mon.a.smithi012.stdout: index_block_restart_interval: 1 2024-05-17T22:26:23.236 INFO:journalctl@ceph.mon.a.smithi012.stdout: metadata_block_size: 4096 2024-05-17T22:26:23.236 INFO:journalctl@ceph.mon.a.smithi012.stdout: partition_filters: 0 2024-05-17T22:26:23.236 INFO:journalctl@ceph.mon.a.smithi012.stdout: use_delta_encoding: 1 2024-05-17T22:26:23.236 INFO:journalctl@ceph.mon.a.smithi012.stdout: filter_policy: bloomfilter 2024-05-17T22:26:23.236 INFO:journalctl@ceph.mon.a.smithi012.stdout: whole_key_filtering: 1 2024-05-17T22:26:23.236 INFO:journalctl@ceph.mon.a.smithi012.stdout: verify_compression: 0 2024-05-17T22:26:23.236 INFO:journalctl@ceph.mon.a.smithi012.stdout: read_amp_bytes_per_bit: 0 2024-05-17T22:26:23.236 INFO:journalctl@ceph.mon.a.smithi012.stdout: format_version: 5 2024-05-17T22:26:23.236 INFO:journalctl@ceph.mon.a.smithi012.stdout: enable_index_compression: 1 2024-05-17T22:26:23.236 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_align: 0 2024-05-17T22:26:23.237 INFO:journalctl@ceph.mon.a.smithi012.stdout: max_auto_readahead_size: 262144 2024-05-17T22:26:23.237 INFO:journalctl@ceph.mon.a.smithi012.stdout: prepopulate_block_cache: 0 2024-05-17T22:26:23.237 INFO:journalctl@ceph.mon.a.smithi012.stdout: initial_auto_readahead_size: 8192 2024-05-17T22:26:23.237 INFO:journalctl@ceph.mon.a.smithi012.stdout: num_file_reads_for_auto_readahead: 2 2024-05-17T22:26:23.237 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.write_buffer_size: 33554432 2024-05-17T22:26:23.237 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.max_write_buffer_number: 2 2024-05-17T22:26:23.237 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.compression: NoCompression 2024-05-17T22:26:23.237 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.bottommost_compression: Disabled 2024-05-17T22:26:23.237 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.prefix_extractor: nullptr 2024-05-17T22:26:23.237 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-17T22:26:23.237 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.num_levels: 7 2024-05-17T22:26:23.237 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-17T22:26:23.237 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-17T22:26:23.237 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-17T22:26:23.238 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-17T22:26:23.238 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-17T22:26:23.238 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-17T22:26:23.238 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-17T22:26:23.238 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-17T22:26:23.238 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-17T22:26:23.238 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-17T22:26:23.238 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-17T22:26:23.238 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-17T22:26:23.238 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-17T22:26:23.238 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.compression_opts.level: 32767 2024-05-17T22:26:23.238 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.compression_opts.strategy: 0 2024-05-17T22:26:23.238 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-17T22:26:23.238 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-17T22:26:23.239 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-17T22:26:23.239 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-17T22:26:23.239 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.compression_opts.enabled: false 2024-05-17T22:26:23.239 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-17T22:26:23.239 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-17T22:26:23.239 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-17T22:26:23.239 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-17T22:26:23.239 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.target_file_size_base: 67108864 2024-05-17T22:26:23.239 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-17T22:26:23.239 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-17T22:26:23.239 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-17T22:26:23.239 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-17T22:26:23.239 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-17T22:26:23.239 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-17T22:26:23.239 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-17T22:26:23.240 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-17T22:26:23.240 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-17T22:26:23.240 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-17T22:26:23.240 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-17T22:26:23.240 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-17T22:26:23.240 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-17T22:26:23.240 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-17T22:26:23.240 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.arena_block_size: 1048576 2024-05-17T22:26:23.240 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-17T22:26:23.240 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-17T22:26:23.240 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.disable_auto_compactions: 0 2024-05-17T22:26:23.240 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-17T22:26:23.240 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-17T22:26:23.240 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-17T22:26:23.240 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-17T22:26:23.241 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-17T22:26:23.241 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-17T22:26:23.241 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-17T22:26:23.241 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-17T22:26:23.241 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-17T22:26:23.241 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-17T22:26:23.241 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-17T22:26:23.241 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.inplace_update_support: 0 2024-05-17T22:26:23.241 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-17T22:26:23.241 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-17T22:26:23.241 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-17T22:26:23.241 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-17T22:26:23.241 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.bloom_locality: 0 2024-05-17T22:26:23.241 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.max_successive_merges: 0 2024-05-17T22:26:23.242 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-17T22:26:23.242 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.paranoid_file_checks: 0 2024-05-17T22:26:23.242 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.force_consistency_checks: 1 2024-05-17T22:26:23.242 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.report_bg_io_stats: 0 2024-05-17T22:26:23.242 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.ttl: 2592000 2024-05-17T22:26:23.242 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-17T22:26:23.242 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-17T22:26:23.242 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-17T22:26:23.242 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.enable_blob_files: false 2024-05-17T22:26:23.242 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.min_blob_size: 0 2024-05-17T22:26:23.242 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.blob_file_size: 268435456 2024-05-17T22:26:23.242 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-17T22:26:23.242 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-17T22:26:23.242 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-17T22:26:23.243 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-17T22:26:23.243 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-17T22:26:23.243 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.blob_file_starting_level: 0 2024-05-17T22:26:23.243 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-17T22:26:23.243 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-17T22:26:23.243 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-17T22:26:23.243 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 026be285-4177-43f3-a4a1-fac799425838 2024-05-17T22:26:23.243 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715984782910295, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-17T22:26:23.243 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-17T22:26:23.243 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715984782911555, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99283, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 322, "table_properties": {"data_size": 97229, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13919, "raw_average_key_size": 49, "raw_value_size": 89622, "raw_average_value_size": 318, "num_data_blocks": 12, "num_entries": 281, "num_filter_entries": 281, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715984782, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "026be285-4177-43f3-a4a1-fac799425838", "db_session_id": "ZLXQOI6J1GLODDJ4RE9G", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-17T22:26:23.243 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715984782911641, "job": 1, "event": "recovery_finished"} 2024-05-17T22:26:23.243 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-17T22:26:23.243 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-17T22:26:23.243 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5614537f3c00 2024-05-17T22:26:23.244 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: DB pointer 0x5614585e8000 2024-05-17T22:26:23.244 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-17T22:26:23.244 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-17T22:26:23.244 INFO:journalctl@ceph.mon.a.smithi012.stdout: ** DB Stats ** 2024-05-17T22:26:23.244 INFO:journalctl@ceph.mon.a.smithi012.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-17T22:26:23.244 INFO:journalctl@ceph.mon.a.smithi012.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-17T22:26:23.244 INFO:journalctl@ceph.mon.a.smithi012.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-17T22:26:23.244 INFO:journalctl@ceph.mon.a.smithi012.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-17T22:26:23.244 INFO:journalctl@ceph.mon.a.smithi012.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-17T22:26:23.244 INFO:journalctl@ceph.mon.a.smithi012.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-17T22:26:23.244 INFO:journalctl@ceph.mon.a.smithi012.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-17T22:26:23.244 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-05-17T22:26:23.244 INFO:journalctl@ceph.mon.a.smithi012.stdout: ** Compaction Stats [default] ** 2024-05-17T22:26:23.244 INFO:journalctl@ceph.mon.a.smithi012.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-17T22:26:23.245 INFO:journalctl@ceph.mon.a.smithi012.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-17T22:26:23.245 INFO:journalctl@ceph.mon.a.smithi012.stdout: L0 2/0 98.82 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 138.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-17T22:26:23.245 INFO:journalctl@ceph.mon.a.smithi012.stdout: Sum 2/0 98.82 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 138.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-17T22:26:23.245 INFO:journalctl@ceph.mon.a.smithi012.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 138.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-17T22:26:23.245 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-05-17T22:26:23.245 INFO:journalctl@ceph.mon.a.smithi012.stdout: ** Compaction Stats [default] ** 2024-05-17T22:26:23.245 INFO:journalctl@ceph.mon.a.smithi012.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-17T22:26:23.245 INFO:journalctl@ceph.mon.a.smithi012.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-17T22:26:23.245 INFO:journalctl@ceph.mon.a.smithi012.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 138.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-17T22:26:23.245 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-05-17T22:26:23.245 INFO:journalctl@ceph.mon.a.smithi012.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-17T22:26:23.245 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-05-17T22:26:23.245 INFO:journalctl@ceph.mon.a.smithi012.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-17T22:26:23.245 INFO:journalctl@ceph.mon.a.smithi012.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-17T22:26:23.246 INFO:journalctl@ceph.mon.a.smithi012.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-17T22:26:23.246 INFO:journalctl@ceph.mon.a.smithi012.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-17T22:26:23.246 INFO:journalctl@ceph.mon.a.smithi012.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-17T22:26:23.246 INFO:journalctl@ceph.mon.a.smithi012.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-17T22:26:23.246 INFO:journalctl@ceph.mon.a.smithi012.stdout: Cumulative compaction: 0.00 GB write, 27.16 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-17T22:26:23.246 INFO:journalctl@ceph.mon.a.smithi012.stdout: Interval compaction: 0.00 GB write, 27.16 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-17T22:26:23.246 INFO:journalctl@ceph.mon.a.smithi012.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-17T22:26:23.246 INFO:journalctl@ceph.mon.a.smithi012.stdout: Block cache BinnedLRUCache@0x5614538111f0#2 capacity: 512.00 MB usage: 1.41 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-17T22:26:23.246 INFO:journalctl@ceph.mon.a.smithi012.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.91 KB,0.000172853%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-17T22:26:23.246 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-05-17T22:26:23.246 INFO:journalctl@ceph.mon.a.smithi012.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-17T22:26:23.246 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: starting mon.a rank 0 at public addrs [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] at bind addrs [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 4d593780-149c-11ef-bc9a-c7b262605968 2024-05-17T22:26:23.246 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: mon.a@-1(???) e1 preinit fsid 4d593780-149c-11ef-bc9a-c7b262605968 2024-05-17T22:26:23.246 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: mon.a@-1(???).mds e1 new map 2024-05-17T22:26:23.246 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: mon.a@-1(???).mds e1 print_map 2024-05-17T22:26:23.247 INFO:journalctl@ceph.mon.a.smithi012.stdout: e1 2024-05-17T22:26:23.247 INFO:journalctl@ceph.mon.a.smithi012.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-17T22:26:23.247 INFO:journalctl@ceph.mon.a.smithi012.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-17T22:26:23.247 INFO:journalctl@ceph.mon.a.smithi012.stdout: legacy client fscid: -1 2024-05-17T22:26:23.247 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-05-17T22:26:23.247 INFO:journalctl@ceph.mon.a.smithi012.stdout: No filesystems configured 2024-05-17T22:26:23.247 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-17T22:26:23.247 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-17T22:26:23.247 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-17T22:26:23.247 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-17T22:26:23.247 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-17T22:26:23.247 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 bash[31613]: 094715d271e66aa5d1c22c288d8f4145e04d35b0727765f704ae06c0acb67537 2024-05-17T22:26:23.247 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 systemd[1]: Started Ceph mon.a for 4d593780-149c-11ef-bc9a-c7b262605968. 2024-05-17T22:26:23.247 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-17T22:26:23.248 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: monmap epoch 1 2024-05-17T22:26:23.248 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: fsid 4d593780-149c-11ef-bc9a-c7b262605968 2024-05-17T22:26:23.248 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: last_changed 2024-05-17T22:26:04.404494+0000 2024-05-17T22:26:23.248 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: created 2024-05-17T22:26:04.404494+0000 2024-05-17T22:26:23.248 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: min_mon_release 19 (squid) 2024-05-17T22:26:23.248 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: election_strategy: 1 2024-05-17T22:26:23.248 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-05-17T22:26:23.248 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: fsmap 2024-05-17T22:26:23.248 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: osdmap e1: 0 total, 0 up, 0 in 2024-05-17T22:26:23.248 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:22 smithi012 ceph-mon[31632]: mgrmap e1: no daemons active 2024-05-17T22:26:24.067 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.435+0000 7f74b1c1c700 1 Processor -- start 2024-05-17T22:26:24.071 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.435+0000 7f74b1c1c700 1 -- start start 2024-05-17T22:26:24.071 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.435+0000 7f74b1c1c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74ac108270 0x7f74ac108680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:24.072 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.435+0000 7f74b1c1c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f74ac108c50 con 0x7f74ac108270 2024-05-17T22:26:24.072 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.436+0000 7f74ab7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74ac108270 0x7f74ac108680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:26:24.072 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.436+0000 7f74ab7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74ac108270 0x7f74ac108680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48638/0 (socket says 172.21.15.12:48638) 2024-05-17T22:26:24.072 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.436+0000 7f74ab7fe700 1 -- 172.21.15.12:0/2891124335 learned_addr learned my addr 172.21.15.12:0/2891124335 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:26:24.072 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.436+0000 7f74ab7fe700 1 -- 172.21.15.12:0/2891124335 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74ac109440 con 0x7f74ac108270 2024-05-17T22:26:24.072 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.436+0000 7f74ab7fe700 1 --2- 172.21.15.12:0/2891124335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74ac108270 0x7f74ac108680 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f749c00bb40 tx=0x7f749c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e27e792b3093c1e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:26:24.072 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.436+0000 7f74aa7fc700 1 -- 172.21.15.12:0/2891124335 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f749c00dbe0 con 0x7f74ac108270 2024-05-17T22:26:24.072 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.437+0000 7f74aa7fc700 1 -- 172.21.15.12:0/2891124335 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f749c0044c0 con 0x7f74ac108270 2024-05-17T22:26:24.072 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.437+0000 7f74aa7fc700 1 -- 172.21.15.12:0/2891124335 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f749c010470 con 0x7f74ac108270 2024-05-17T22:26:24.072 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.437+0000 7f74b1c1c700 1 -- 172.21.15.12:0/2891124335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74ac108270 msgr2=0x7f74ac108680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:26:24.072 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.437+0000 7f74b1c1c700 1 --2- 172.21.15.12:0/2891124335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74ac108270 0x7f74ac108680 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f749c00bb40 tx=0x7f749c00be50 comp rx=0 tx=0).stop 2024-05-17T22:26:24.072 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.437+0000 7f74b1c1c700 1 -- 172.21.15.12:0/2891124335 shutdown_connections 2024-05-17T22:26:24.072 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.437+0000 7f74b1c1c700 1 --2- 172.21.15.12:0/2891124335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74ac108270 0x7f74ac108680 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:24.072 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.437+0000 7f74b1c1c700 1 -- 172.21.15.12:0/2891124335 >> 172.21.15.12:0/2891124335 conn(0x7f74ac103a80 msgr2=0x7f74ac105f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:26:24.073 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.437+0000 7f74b1c1c700 1 -- 172.21.15.12:0/2891124335 shutdown_connections 2024-05-17T22:26:24.073 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.437+0000 7f74b1c1c700 1 -- 172.21.15.12:0/2891124335 wait complete. 2024-05-17T22:26:24.073 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.438+0000 7f74b1c1c700 1 Processor -- start 2024-05-17T22:26:24.073 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.438+0000 7f74b1c1c700 1 -- start start 2024-05-17T22:26:24.073 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.438+0000 7f74b1c1c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74ac108270 0x7f74ac19b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:24.073 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.438+0000 7f74b1c1c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f74ac07c100 con 0x7f74ac108270 2024-05-17T22:26:24.073 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.439+0000 7f74ab7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74ac108270 0x7f74ac19b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:26:24.073 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.439+0000 7f74ab7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74ac108270 0x7f74ac19b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48650/0 (socket says 172.21.15.12:48650) 2024-05-17T22:26:24.073 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.439+0000 7f74ab7fe700 1 -- 172.21.15.12:0/2308763258 learned_addr learned my addr 172.21.15.12:0/2308763258 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:26:24.073 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.439+0000 7f74ab7fe700 1 -- 172.21.15.12:0/2308763258 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74ac19b9e0 con 0x7f74ac108270 2024-05-17T22:26:24.073 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.439+0000 7f74ab7fe700 1 --2- 172.21.15.12:0/2308763258 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74ac108270 0x7f74ac19b4a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f749c00bfd0 tx=0x7f749c003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:26:24.073 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.439+0000 7f74a8ff9700 1 -- 172.21.15.12:0/2308763258 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f749c027070 con 0x7f74ac108270 2024-05-17T22:26:24.073 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.439+0000 7f74a8ff9700 1 -- 172.21.15.12:0/2308763258 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f749c010c00 con 0x7f74ac108270 2024-05-17T22:26:24.073 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.439+0000 7f74b1c1c700 1 -- 172.21.15.12:0/2308763258 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74ac19bc70 con 0x7f74ac108270 2024-05-17T22:26:24.074 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.439+0000 7f74a8ff9700 1 -- 172.21.15.12:0/2308763258 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f749c022b70 con 0x7f74ac108270 2024-05-17T22:26:24.074 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.439+0000 7f74b1c1c700 1 -- 172.21.15.12:0/2308763258 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74ac19e880 con 0x7f74ac108270 2024-05-17T22:26:24.074 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.440+0000 7f74a8ff9700 1 -- 172.21.15.12:0/2308763258 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f749c02a070 con 0x7f74ac108270 2024-05-17T22:26:24.074 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.440+0000 7f74a8ff9700 1 -- 172.21.15.12:0/2308763258 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f749c01f070 con 0x7f74ac108270 2024-05-17T22:26:24.074 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.440+0000 7f74b1c1c700 1 -- 172.21.15.12:0/2308763258 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74ac108680 con 0x7f74ac108270 2024-05-17T22:26:24.074 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.443+0000 7f74a8ff9700 1 -- 172.21.15.12:0/2308763258 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f749c031030 con 0x7f74ac108270 2024-05-17T22:26:24.074 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.514+0000 7f74b1c1c700 1 -- 172.21.15.12:0/2308763258 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f74ac194640 con 0x7f74ac108270 2024-05-17T22:26:24.074 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.516+0000 7f74a8ff9700 1 -- 172.21.15.12:0/2308763258 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f74ac194640 con 0x7f74ac108270 2024-05-17T22:26:24.074 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.517+0000 7f74b1c1c700 1 -- 172.21.15.12:0/2308763258 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74ac108270 msgr2=0x7f74ac19b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:26:24.074 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.517+0000 7f74b1c1c700 1 --2- 172.21.15.12:0/2308763258 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74ac108270 0x7f74ac19b4a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f749c00bfd0 tx=0x7f749c003dc0 comp rx=0 tx=0).stop 2024-05-17T22:26:24.074 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.517+0000 7f74b1c1c700 1 -- 172.21.15.12:0/2308763258 shutdown_connections 2024-05-17T22:26:24.074 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.517+0000 7f74b1c1c700 1 --2- 172.21.15.12:0/2308763258 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74ac108270 0x7f74ac19b4a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:24.074 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.517+0000 7f74b1c1c700 1 -- 172.21.15.12:0/2308763258 >> 172.21.15.12:0/2308763258 conn(0x7f74ac103a80 msgr2=0x7f74ac191700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:26:24.074 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.517+0000 7f74b1c1c700 1 -- 172.21.15.12:0/2308763258 shutdown_connections 2024-05-17T22:26:24.075 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:23.517+0000 7f74b1c1c700 1 -- 172.21.15.12:0/2308763258 wait complete. 2024-05-17T22:26:24.075 INFO:teuthology.orchestra.run.smithi012.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-17T22:26:24.098 INFO:teuthology.orchestra.run.smithi012.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-17T22:26:24.098 INFO:teuthology.orchestra.run.smithi012.stdout:Creating mgr... 2024-05-17T22:26:24.098 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-17T22:26:24.098 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-17T22:26:24.361 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-4d593780-149c-11ef-bc9a-c7b262605968@mgr.x 2024-05-17T22:26:24.362 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Failed to reset failed state of unit ceph-4d593780-149c-11ef-bc9a-c7b262605968@mgr.x.service: Unit ceph-4d593780-149c-11ef-bc9a-c7b262605968@mgr.x.service not loaded. 2024-05-17T22:26:24.513 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-4d593780-149c-11ef-bc9a-c7b262605968.target.wants/ceph-4d593780-149c-11ef-bc9a-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-4d593780-149c-11ef-bc9a-c7b262605968@.service. 2024-05-17T22:26:24.742 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:24 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/2308763258' entity='client.admin' 2024-05-17T22:26:24.742 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:24 smithi012 systemd[1]: Starting Ceph mgr.x for 4d593780-149c-11ef-bc9a-c7b262605968... 2024-05-17T22:26:25.018 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:24 smithi012 podman[31847]: 2024-05-17 22:26:24.740573214 +0000 UTC m=+0.101874336 container create 16e3da619b4f49bf7fe5b2376cc0de31ff82e408d9f88026cf9d42d2bf577e87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-17T22:26:25.019 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:24 smithi012 podman[31847]: 2024-05-17 22:26:24.656750474 +0000 UTC m=+0.018051607 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc 2024-05-17T22:26:25.019 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:24 smithi012 podman[31847]: 2024-05-17 22:26:24.942735228 +0000 UTC m=+0.304036359 container init 16e3da619b4f49bf7fe5b2376cc0de31ff82e408d9f88026cf9d42d2bf577e87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-17T22:26:25.019 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:24 smithi012 podman[31847]: 2024-05-17 22:26:24.94556885 +0000 UTC m=+0.306869988 container start 16e3da619b4f49bf7fe5b2376cc0de31ff82e408d9f88026cf9d42d2bf577e87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-17T22:26:25.019 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:25 smithi012 bash[31847]: 16e3da619b4f49bf7fe5b2376cc0de31ff82e408d9f88026cf9d42d2bf577e87 2024-05-17T22:26:25.035 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-17T22:26:25.035 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout disabled 2024-05-17T22:26:25.044 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-17T22:26:25.044 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-05-17T22:26:25.044 INFO:teuthology.orchestra.run.smithi012.stdout:firewalld.service is not enabled 2024-05-17T22:26:25.044 INFO:teuthology.orchestra.run.smithi012.stdout:Not possible to enable service . firewalld.service is not available 2024-05-17T22:26:25.055 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-17T22:26:25.055 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout disabled 2024-05-17T22:26:25.065 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-17T22:26:25.065 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-05-17T22:26:25.065 INFO:teuthology.orchestra.run.smithi012.stdout:firewalld.service is not enabled 2024-05-17T22:26:25.065 INFO:teuthology.orchestra.run.smithi012.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-17T22:26:25.065 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mgr to start... 2024-05-17T22:26:25.065 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mgr... 2024-05-17T22:26:25.320 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:25 smithi012 systemd[1]: Started Ceph mgr.x for 4d593780-149c-11ef-bc9a-c7b262605968. 2024-05-17T22:26:25.320 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:25 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:25.130+0000 7f6191fdf200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-17T22:26:25.590 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:25 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:25.328+0000 7f6191fdf200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-17T22:26:25.591 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:25 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:25.435+0000 7f6191fdf200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-17T22:26:26.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:25 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/4066680150' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-17T22:26:26.091 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:25 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:25.772+0000 7f6191fdf200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-17T22:26:26.091 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:25 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:25.864+0000 7f6191fdf200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-17T22:26:26.163 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-05-17T22:26:26.166 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-05-17T22:26:26.166 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsid": "4d593780-149c-11ef-bc9a-c7b262605968", 2024-05-17T22:26:26.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "health": { 2024-05-17T22:26:26.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-17T22:26:26.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-17T22:26:26.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-17T22:26:26.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-17T22:26:26.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-17T22:26:26.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-17T22:26:26.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 0 2024-05-17T22:26:26.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-05-17T22:26:26.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-17T22:26:26.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "a" 2024-05-17T22:26:26.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-05-17T22:26:26.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-17T22:26:26.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-17T22:26:26.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T22:26:26.169 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-17T22:26:26.169 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-17T22:26:26.169 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-17T22:26:26.169 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-17T22:26:26.169 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T22:26:26.169 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-17T22:26:26.178 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-17T22:26:26.178 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-17T22:26:26.178 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-17T22:26:26.178 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-17T22:26:26.178 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-17T22:26:26.178 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-17T22:26:26.178 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-17T22:26:26.179 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-17T22:26:26.179 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-17T22:26:26.179 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-17T22:26:26.179 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-17T22:26:26.179 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-17T22:26:26.179 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-17T22:26:26.179 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-17T22:26:26.179 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-17T22:26:26.179 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-17T22:26:26.179 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-17T22:26:26.179 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T22:26:26.179 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-17T22:26:26.179 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-17T22:26:26.180 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-17T22:26:26.180 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-17T22:26:26.180 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-17T22:26:26.180 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-17T22:26:26.180 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-17T22:26:26.180 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-17T22:26:26.180 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-17T22:26:26.180 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "restful" 2024-05-17T22:26:26.180 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-05-17T22:26:26.180 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-17T22:26:26.180 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-17T22:26:26.180 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-17T22:26:26.180 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T22:26:26.180 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modified": "2024-05-17T22:26:16.956295+0000", 2024-05-17T22:26:26.181 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-17T22:26:26.181 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-17T22:26:26.181 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-17T22:26:26.181 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-05-17T22:26:26.181 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.550+0000 7f138fc4c700 1 Processor -- start 2024-05-17T22:26:26.181 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.551+0000 7f138fc4c700 1 -- start start 2024-05-17T22:26:26.181 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.551+0000 7f138fc4c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1388108270 0x7f1388108680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:26.181 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.551+0000 7f138fc4c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1388108c50 con 0x7f1388108270 2024-05-17T22:26:26.181 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.551+0000 7f138d9e8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1388108270 0x7f1388108680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:26:26.181 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.551+0000 7f138d9e8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1388108270 0x7f1388108680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48672/0 (socket says 172.21.15.12:48672) 2024-05-17T22:26:26.181 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.551+0000 7f138d9e8700 1 -- 172.21.15.12:0/316390485 learned_addr learned my addr 172.21.15.12:0/316390485 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:26:26.181 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.552+0000 7f138d9e8700 1 -- 172.21.15.12:0/316390485 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1388109440 con 0x7f1388108270 2024-05-17T22:26:26.181 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.552+0000 7f138d9e8700 1 --2- 172.21.15.12:0/316390485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1388108270 0x7f1388108680 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f13840098a0 tx=0x7f1384009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afe8b410ed6b818b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:26:26.182 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.552+0000 7f138c9e6700 1 -- 172.21.15.12:0/316390485 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1384004160 con 0x7f1388108270 2024-05-17T22:26:26.182 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.552+0000 7f138c9e6700 1 -- 172.21.15.12:0/316390485 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1384004300 con 0x7f1388108270 2024-05-17T22:26:26.182 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.552+0000 7f138c9e6700 1 -- 172.21.15.12:0/316390485 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1384011af0 con 0x7f1388108270 2024-05-17T22:26:26.182 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.553+0000 7f138fc4c700 1 -- 172.21.15.12:0/316390485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1388108270 msgr2=0x7f1388108680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:26:26.182 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.553+0000 7f138fc4c700 1 --2- 172.21.15.12:0/316390485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1388108270 0x7f1388108680 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f13840098a0 tx=0x7f1384009bb0 comp rx=0 tx=0).stop 2024-05-17T22:26:26.182 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.553+0000 7f138fc4c700 1 -- 172.21.15.12:0/316390485 shutdown_connections 2024-05-17T22:26:26.182 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.553+0000 7f138fc4c700 1 --2- 172.21.15.12:0/316390485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1388108270 0x7f1388108680 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:26.182 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.553+0000 7f138fc4c700 1 -- 172.21.15.12:0/316390485 >> 172.21.15.12:0/316390485 conn(0x7f1388103a80 msgr2=0x7f1388105f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:26:26.182 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.553+0000 7f138fc4c700 1 -- 172.21.15.12:0/316390485 shutdown_connections 2024-05-17T22:26:26.182 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.554+0000 7f138fc4c700 1 -- 172.21.15.12:0/316390485 wait complete. 2024-05-17T22:26:26.183 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.554+0000 7f138fc4c700 1 Processor -- start 2024-05-17T22:26:26.183 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.554+0000 7f138fc4c700 1 -- start start 2024-05-17T22:26:26.183 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.554+0000 7f138fc4c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1388108270 0x7f1388105be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:26.183 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.554+0000 7f138fc4c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f138810c7f0 con 0x7f1388108270 2024-05-17T22:26:26.183 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.555+0000 7f138d9e8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1388108270 0x7f1388105be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:26:26.183 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.555+0000 7f138d9e8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1388108270 0x7f1388105be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48682/0 (socket says 172.21.15.12:48682) 2024-05-17T22:26:26.183 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.555+0000 7f138d9e8700 1 -- 172.21.15.12:0/4066680150 learned_addr learned my addr 172.21.15.12:0/4066680150 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:26:26.183 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.555+0000 7f138d9e8700 1 -- 172.21.15.12:0/4066680150 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1388106120 con 0x7f1388108270 2024-05-17T22:26:26.183 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.555+0000 7f138d9e8700 1 --2- 172.21.15.12:0/4066680150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1388108270 0x7f1388105be0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1384009fb0 tx=0x7f1384011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:26:26.184 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.555+0000 7f137affd700 1 -- 172.21.15.12:0/4066680150 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13840036a0 con 0x7f1388108270 2024-05-17T22:26:26.184 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.555+0000 7f137affd700 1 -- 172.21.15.12:0/4066680150 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f138401a430 con 0x7f1388108270 2024-05-17T22:26:26.184 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.555+0000 7f138fc4c700 1 -- 172.21.15.12:0/4066680150 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1388104370 con 0x7f1388108270 2024-05-17T22:26:26.184 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.555+0000 7f137affd700 1 -- 172.21.15.12:0/4066680150 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1384019550 con 0x7f1388108270 2024-05-17T22:26:26.184 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.556+0000 7f138fc4c700 1 -- 172.21.15.12:0/4066680150 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1388104850 con 0x7f1388108270 2024-05-17T22:26:26.184 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.556+0000 7f137affd700 1 -- 172.21.15.12:0/4066680150 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1384019b90 con 0x7f1388108270 2024-05-17T22:26:26.184 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.556+0000 7f137affd700 1 -- 172.21.15.12:0/4066680150 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1384024730 con 0x7f1388108270 2024-05-17T22:26:26.184 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.557+0000 7f138fc4c700 1 -- 172.21.15.12:0/4066680150 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1388108680 con 0x7f1388108270 2024-05-17T22:26:26.184 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.559+0000 7f137affd700 1 -- 172.21.15.12:0/4066680150 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f1384043ec0 con 0x7f1388108270 2024-05-17T22:26:26.184 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.634+0000 7f138fc4c700 1 -- 172.21.15.12:0/4066680150 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f13881a37e0 con 0x7f1388108270 2024-05-17T22:26:26.184 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.635+0000 7f137affd700 1 -- 172.21.15.12:0/4066680150 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f1384032020 con 0x7f1388108270 2024-05-17T22:26:26.184 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.636+0000 7f138fc4c700 1 -- 172.21.15.12:0/4066680150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1388108270 msgr2=0x7f1388105be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:26:26.184 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.636+0000 7f138fc4c700 1 --2- 172.21.15.12:0/4066680150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1388108270 0x7f1388105be0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1384009fb0 tx=0x7f1384011940 comp rx=0 tx=0).stop 2024-05-17T22:26:26.184 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.636+0000 7f138fc4c700 1 -- 172.21.15.12:0/4066680150 shutdown_connections 2024-05-17T22:26:26.185 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.636+0000 7f138fc4c700 1 --2- 172.21.15.12:0/4066680150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1388108270 0x7f1388105be0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:26.185 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.636+0000 7f138fc4c700 1 -- 172.21.15.12:0/4066680150 >> 172.21.15.12:0/4066680150 conn(0x7f1388103a80 msgr2=0x7f1388193d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:26:26.185 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.636+0000 7f138fc4c700 1 -- 172.21.15.12:0/4066680150 shutdown_connections 2024-05-17T22:26:26.185 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:25.636+0000 7f138fc4c700 1 -- 172.21.15.12:0/4066680150 wait complete. 2024-05-17T22:26:26.185 INFO:teuthology.orchestra.run.smithi012.stdout:mgr not available, waiting (1/15)... 2024-05-17T22:26:27.491 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:27 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:27.136+0000 7f6191fdf200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-17T22:26:27.492 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:27 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:27.312+0000 7f6191fdf200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-17T22:26:27.492 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:27 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:27.490+0000 7f6191fdf200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-17T22:26:28.090 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:27 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:27.790+0000 7f6191fdf200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-17T22:26:28.090 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:27 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:27.888+0000 7f6191fdf200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-17T22:26:28.428 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:28 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:28.099+0000 7f6191fdf200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-17T22:26:28.428 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:28 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:28.309+0000 7f6191fdf200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-17T22:26:28.974 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:28 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/140011528' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-17T22:26:29.332 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:28 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:28.971+0000 7f6191fdf200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-17T22:26:29.332 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:29 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:29.207+0000 7f6191fdf200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-17T22:26:29.336 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-05-17T22:26:29.336 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-05-17T22:26:29.336 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsid": "4d593780-149c-11ef-bc9a-c7b262605968", 2024-05-17T22:26:29.336 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "health": { 2024-05-17T22:26:29.336 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-17T22:26:29.336 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-17T22:26:29.337 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-17T22:26:29.337 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-17T22:26:29.337 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-17T22:26:29.337 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-17T22:26:29.337 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 0 2024-05-17T22:26:29.337 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-05-17T22:26:29.337 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-17T22:26:29.337 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "a" 2024-05-17T22:26:29.337 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-05-17T22:26:29.337 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-05-17T22:26:29.337 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-17T22:26:29.337 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T22:26:29.337 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-17T22:26:29.338 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-17T22:26:29.338 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-17T22:26:29.338 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-17T22:26:29.338 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T22:26:29.338 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-17T22:26:29.338 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-17T22:26:29.345 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-17T22:26:29.346 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-17T22:26:29.346 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-17T22:26:29.346 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-17T22:26:29.346 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-17T22:26:29.346 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-17T22:26:29.346 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-17T22:26:29.346 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-17T22:26:29.346 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-17T22:26:29.346 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-17T22:26:29.346 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-17T22:26:29.346 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-17T22:26:29.346 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-17T22:26:29.347 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-17T22:26:29.347 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-17T22:26:29.347 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-17T22:26:29.347 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T22:26:29.347 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-17T22:26:29.347 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-17T22:26:29.347 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-17T22:26:29.347 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-17T22:26:29.347 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-17T22:26:29.347 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-17T22:26:29.348 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-17T22:26:29.348 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-17T22:26:29.348 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-17T22:26:29.348 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "restful" 2024-05-17T22:26:29.348 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-05-17T22:26:29.348 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-17T22:26:29.348 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-17T22:26:29.348 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-17T22:26:29.348 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T22:26:29.348 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modified": "2024-05-17T22:26:16.956295+0000", 2024-05-17T22:26:29.349 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-17T22:26:29.349 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-17T22:26:29.349 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-17T22:26:29.349 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-05-17T22:26:29.349 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.643+0000 7f0eebaa0700 1 Processor -- start 2024-05-17T22:26:29.349 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.643+0000 7f0eebaa0700 1 -- start start 2024-05-17T22:26:29.349 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.643+0000 7f0eebaa0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ee407d640 0x7f0ee410c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:29.349 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.643+0000 7f0eebaa0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee410c5f0 con 0x7f0ee407d640 2024-05-17T22:26:29.349 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.643+0000 7f0ee983c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ee407d640 0x7f0ee410c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:26:29.349 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.644+0000 7f0ee983c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ee407d640 0x7f0ee410c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48684/0 (socket says 172.21.15.12:48684) 2024-05-17T22:26:29.349 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.644+0000 7f0ee983c700 1 -- 172.21.15.12:0/3953646697 learned_addr learned my addr 172.21.15.12:0/3953646697 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:26:29.350 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.644+0000 7f0ee983c700 1 -- 172.21.15.12:0/3953646697 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ee410c770 con 0x7f0ee407d640 2024-05-17T22:26:29.350 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.644+0000 7f0ee983c700 1 --2- 172.21.15.12:0/3953646697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ee407d640 0x7f0ee410c000 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0ee00098a0 tx=0x7f0ee0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b62e9a20827d9c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:26:29.350 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.644+0000 7f0ee883a700 1 -- 172.21.15.12:0/3953646697 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ee0004160 con 0x7f0ee407d640 2024-05-17T22:26:29.350 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.644+0000 7f0ee883a700 1 -- 172.21.15.12:0/3953646697 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0ee0004300 con 0x7f0ee407d640 2024-05-17T22:26:29.350 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.644+0000 7f0ee883a700 1 -- 172.21.15.12:0/3953646697 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ee0011af0 con 0x7f0ee407d640 2024-05-17T22:26:29.350 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.645+0000 7f0eebaa0700 1 -- 172.21.15.12:0/3953646697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ee407d640 msgr2=0x7f0ee410c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:26:29.352 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.645+0000 7f0eebaa0700 1 --2- 172.21.15.12:0/3953646697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ee407d640 0x7f0ee410c000 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0ee00098a0 tx=0x7f0ee0009bb0 comp rx=0 tx=0).stop 2024-05-17T22:26:29.352 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.645+0000 7f0eebaa0700 1 -- 172.21.15.12:0/3953646697 shutdown_connections 2024-05-17T22:26:29.352 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.645+0000 7f0eebaa0700 1 --2- 172.21.15.12:0/3953646697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ee407d640 0x7f0ee410c000 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:29.352 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.645+0000 7f0eebaa0700 1 -- 172.21.15.12:0/3953646697 >> 172.21.15.12:0/3953646697 conn(0x7f0ee4103ba0 msgr2=0x7f0ee4106010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:26:29.352 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.645+0000 7f0eebaa0700 1 -- 172.21.15.12:0/3953646697 shutdown_connections 2024-05-17T22:26:29.352 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.646+0000 7f0eebaa0700 1 -- 172.21.15.12:0/3953646697 wait complete. 2024-05-17T22:26:29.352 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.646+0000 7f0eebaa0700 1 Processor -- start 2024-05-17T22:26:29.352 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.647+0000 7f0eebaa0700 1 -- start start 2024-05-17T22:26:29.352 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.647+0000 7f0eebaa0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ee407d640 0x7f0ee419d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:29.352 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.647+0000 7f0eebaa0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee410d780 con 0x7f0ee407d640 2024-05-17T22:26:29.353 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.647+0000 7f0ee983c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ee407d640 0x7f0ee419d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:26:29.353 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.647+0000 7f0ee983c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ee407d640 0x7f0ee419d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48700/0 (socket says 172.21.15.12:48700) 2024-05-17T22:26:29.353 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.647+0000 7f0ee983c700 1 -- 172.21.15.12:0/140011528 learned_addr learned my addr 172.21.15.12:0/140011528 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:26:29.353 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.647+0000 7f0ee983c700 1 -- 172.21.15.12:0/140011528 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ee419dc90 con 0x7f0ee407d640 2024-05-17T22:26:29.353 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.647+0000 7f0ee983c700 1 --2- 172.21.15.12:0/140011528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ee407d640 0x7f0ee419d750 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0ee00117b0 tx=0x7f0ee0011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:26:29.353 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.648+0000 7f0edaffd700 1 -- 172.21.15.12:0/140011528 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ee0003af0 con 0x7f0ee407d640 2024-05-17T22:26:29.353 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.648+0000 7f0edaffd700 1 -- 172.21.15.12:0/140011528 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0ee0020070 con 0x7f0ee407d640 2024-05-17T22:26:29.353 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.648+0000 7f0eebaa0700 1 -- 172.21.15.12:0/140011528 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ee419df20 con 0x7f0ee407d640 2024-05-17T22:26:29.353 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.648+0000 7f0edaffd700 1 -- 172.21.15.12:0/140011528 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ee0019630 con 0x7f0ee407d640 2024-05-17T22:26:29.353 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.648+0000 7f0eebaa0700 1 -- 172.21.15.12:0/140011528 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ee41a0b30 con 0x7f0ee407d640 2024-05-17T22:26:29.353 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.648+0000 7f0edaffd700 1 -- 172.21.15.12:0/140011528 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0ee0019c30 con 0x7f0ee407d640 2024-05-17T22:26:29.353 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.648+0000 7f0edaffd700 1 -- 172.21.15.12:0/140011528 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0ee000b720 con 0x7f0ee407d640 2024-05-17T22:26:29.353 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.649+0000 7f0eebaa0700 1 -- 172.21.15.12:0/140011528 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ee410a070 con 0x7f0ee407d640 2024-05-17T22:26:29.354 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.653+0000 7f0edaffd700 1 -- 172.21.15.12:0/140011528 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f0ee001a9c0 con 0x7f0ee407d640 2024-05-17T22:26:29.354 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.729+0000 7f0eebaa0700 1 -- 172.21.15.12:0/140011528 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f0ee41a1120 con 0x7f0ee407d640 2024-05-17T22:26:29.354 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.730+0000 7f0edaffd700 1 -- 172.21.15.12:0/140011528 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f0ee00042c0 con 0x7f0ee407d640 2024-05-17T22:26:29.354 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.731+0000 7f0eebaa0700 1 -- 172.21.15.12:0/140011528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ee407d640 msgr2=0x7f0ee419d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:26:29.354 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.731+0000 7f0eebaa0700 1 --2- 172.21.15.12:0/140011528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ee407d640 0x7f0ee419d750 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0ee00117b0 tx=0x7f0ee0011b70 comp rx=0 tx=0).stop 2024-05-17T22:26:29.354 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.731+0000 7f0eebaa0700 1 -- 172.21.15.12:0/140011528 shutdown_connections 2024-05-17T22:26:29.354 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.731+0000 7f0eebaa0700 1 --2- 172.21.15.12:0/140011528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ee407d640 0x7f0ee419d750 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:29.354 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.731+0000 7f0eebaa0700 1 -- 172.21.15.12:0/140011528 >> 172.21.15.12:0/140011528 conn(0x7f0ee4103ba0 msgr2=0x7f0ee4104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:26:29.354 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.731+0000 7f0eebaa0700 1 -- 172.21.15.12:0/140011528 shutdown_connections 2024-05-17T22:26:29.354 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:28.731+0000 7f0eebaa0700 1 -- 172.21.15.12:0/140011528 wait complete. 2024-05-17T22:26:29.354 INFO:teuthology.orchestra.run.smithi012.stdout:mgr not available, waiting (2/15)... 2024-05-17T22:26:30.090 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:29 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:29.675+0000 7f6191fdf200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-17T22:26:30.091 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:29 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:29.777+0000 7f6191fdf200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-17T22:26:30.537 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:30 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:30.162+0000 7f6191fdf200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-17T22:26:30.538 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:30 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:30.535+0000 7f6191fdf200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-17T22:26:30.842 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:30 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:30.633+0000 7f6191fdf200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-17T22:26:31.340 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:30 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:30.888+0000 7f6191fdf200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-17T22:26:31.341 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:31 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:31.027+0000 7f6191fdf200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-17T22:26:31.341 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:31 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:31.128+0000 7f6191fdf200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-17T22:26:31.770 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:31 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:31.476+0000 7f6191fdf200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-17T22:26:31.770 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:31 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:31.573+0000 7f6191fdf200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-17T22:26:32.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:31 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/2841042172' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-17T22:26:32.426 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-05-17T22:26:32.435 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-05-17T22:26:32.435 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsid": "4d593780-149c-11ef-bc9a-c7b262605968", 2024-05-17T22:26:32.435 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "health": { 2024-05-17T22:26:32.435 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-17T22:26:32.436 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-17T22:26:32.436 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-17T22:26:32.436 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-17T22:26:32.436 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-17T22:26:32.436 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-17T22:26:32.436 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 0 2024-05-17T22:26:32.436 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-05-17T22:26:32.436 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-17T22:26:32.436 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "a" 2024-05-17T22:26:32.436 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-05-17T22:26:32.436 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-05-17T22:26:32.436 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-17T22:26:32.436 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T22:26:32.436 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-17T22:26:32.437 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-17T22:26:32.437 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-17T22:26:32.437 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-17T22:26:32.437 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T22:26:32.437 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-17T22:26:32.437 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-17T22:26:32.437 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-17T22:26:32.437 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-17T22:26:32.437 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-17T22:26:32.437 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-17T22:26:32.437 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-17T22:26:32.437 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-17T22:26:32.437 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-17T22:26:32.437 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-17T22:26:32.437 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-17T22:26:32.438 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-17T22:26:32.438 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-17T22:26:32.438 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-17T22:26:32.438 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-17T22:26:32.438 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-17T22:26:32.438 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-17T22:26:32.438 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-17T22:26:32.438 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T22:26:32.438 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-17T22:26:32.438 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-17T22:26:32.438 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-17T22:26:32.438 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-17T22:26:32.438 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-17T22:26:32.438 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-17T22:26:32.439 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-17T22:26:32.439 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-17T22:26:32.439 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-17T22:26:32.439 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "restful" 2024-05-17T22:26:32.439 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-05-17T22:26:32.439 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-17T22:26:32.439 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-17T22:26:32.439 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-17T22:26:32.439 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T22:26:32.439 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modified": "2024-05-17T22:26:16.956295+0000", 2024-05-17T22:26:32.439 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-17T22:26:32.439 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-17T22:26:32.439 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-17T22:26:32.439 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-05-17T22:26:32.440 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.799+0000 7f066d2fb700 1 Processor -- start 2024-05-17T22:26:32.440 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.799+0000 7f066d2fb700 1 -- start start 2024-05-17T22:26:32.440 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.799+0000 7f066d2fb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0668109ee0 0x7f066810c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:32.440 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.799+0000 7f066d2fb700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f066807d770 con 0x7f0668109ee0 2024-05-17T22:26:32.440 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.799+0000 7f0666ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0668109ee0 0x7f066810c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:26:32.440 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.799+0000 7f0666ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0668109ee0 0x7f066810c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35436/0 (socket says 172.21.15.12:35436) 2024-05-17T22:26:32.440 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.799+0000 7f0666ffd700 1 -- 172.21.15.12:0/1785328744 learned_addr learned my addr 172.21.15.12:0/1785328744 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:26:32.440 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.800+0000 7f0666ffd700 1 -- 172.21.15.12:0/1785328744 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f066810c840 con 0x7f0668109ee0 2024-05-17T22:26:32.440 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.800+0000 7f0666ffd700 1 --2- 172.21.15.12:0/1785328744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0668109ee0 0x7f066810c300 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f06580098a0 tx=0x7f0658009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5116a84b88aca219 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:26:32.440 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.800+0000 7f0665ffb700 1 -- 172.21.15.12:0/1785328744 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0658004160 con 0x7f0668109ee0 2024-05-17T22:26:32.440 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.800+0000 7f0665ffb700 1 -- 172.21.15.12:0/1785328744 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0658004300 con 0x7f0668109ee0 2024-05-17T22:26:32.440 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.800+0000 7f0665ffb700 1 -- 172.21.15.12:0/1785328744 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0658011af0 con 0x7f0668109ee0 2024-05-17T22:26:32.440 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.801+0000 7f066d2fb700 1 -- 172.21.15.12:0/1785328744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0668109ee0 msgr2=0x7f066810c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:26:32.441 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.801+0000 7f066d2fb700 1 --2- 172.21.15.12:0/1785328744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0668109ee0 0x7f066810c300 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f06580098a0 tx=0x7f0658009bb0 comp rx=0 tx=0).stop 2024-05-17T22:26:32.441 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.801+0000 7f066d2fb700 1 -- 172.21.15.12:0/1785328744 shutdown_connections 2024-05-17T22:26:32.441 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.801+0000 7f066d2fb700 1 --2- 172.21.15.12:0/1785328744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0668109ee0 0x7f066810c300 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:32.441 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.801+0000 7f066d2fb700 1 -- 172.21.15.12:0/1785328744 >> 172.21.15.12:0/1785328744 conn(0x7f0668103ab0 msgr2=0x7f0668105f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:26:32.441 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.801+0000 7f066d2fb700 1 -- 172.21.15.12:0/1785328744 shutdown_connections 2024-05-17T22:26:32.441 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.801+0000 7f066d2fb700 1 -- 172.21.15.12:0/1785328744 wait complete. 2024-05-17T22:26:32.441 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.802+0000 7f066d2fb700 1 Processor -- start 2024-05-17T22:26:32.441 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.802+0000 7f066d2fb700 1 -- start start 2024-05-17T22:26:32.441 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.802+0000 7f066d2fb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0668109ee0 0x7f066819d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:32.441 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.802+0000 7f066d2fb700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f066810d7a0 con 0x7f0668109ee0 2024-05-17T22:26:32.441 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.802+0000 7f0666ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0668109ee0 0x7f066819d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:26:32.441 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.802+0000 7f0666ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0668109ee0 0x7f066819d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35450/0 (socket says 172.21.15.12:35450) 2024-05-17T22:26:32.441 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.802+0000 7f0666ffd700 1 -- 172.21.15.12:0/2841042172 learned_addr learned my addr 172.21.15.12:0/2841042172 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:26:32.441 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.802+0000 7f0666ffd700 1 -- 172.21.15.12:0/2841042172 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f066819dc80 con 0x7f0668109ee0 2024-05-17T22:26:32.442 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.803+0000 7f0666ffd700 1 --2- 172.21.15.12:0/2841042172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0668109ee0 0x7f066819d740 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f06580117b0 tx=0x7f0658011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:26:32.442 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.803+0000 7f064ffff700 1 -- 172.21.15.12:0/2841042172 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0658003af0 con 0x7f0668109ee0 2024-05-17T22:26:32.442 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.803+0000 7f064ffff700 1 -- 172.21.15.12:0/2841042172 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0658020070 con 0x7f0668109ee0 2024-05-17T22:26:32.442 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.803+0000 7f066d2fb700 1 -- 172.21.15.12:0/2841042172 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f066819df10 con 0x7f0668109ee0 2024-05-17T22:26:32.442 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.803+0000 7f064ffff700 1 -- 172.21.15.12:0/2841042172 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0658019630 con 0x7f0668109ee0 2024-05-17T22:26:32.442 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.803+0000 7f066d2fb700 1 -- 172.21.15.12:0/2841042172 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06681a0b20 con 0x7f0668109ee0 2024-05-17T22:26:32.442 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.803+0000 7f064ffff700 1 -- 172.21.15.12:0/2841042172 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0658019c80 con 0x7f0668109ee0 2024-05-17T22:26:32.442 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.803+0000 7f064ffff700 1 -- 172.21.15.12:0/2841042172 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f065800b720 con 0x7f0668109ee0 2024-05-17T22:26:32.442 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.804+0000 7f066d2fb700 1 -- 172.21.15.12:0/2841042172 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f066810a370 con 0x7f0668109ee0 2024-05-17T22:26:32.442 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.806+0000 7f064ffff700 1 -- 172.21.15.12:0/2841042172 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f06580042a0 con 0x7f0668109ee0 2024-05-17T22:26:32.442 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.879+0000 7f066d2fb700 1 -- 172.21.15.12:0/2841042172 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f06681a15d0 con 0x7f0668109ee0 2024-05-17T22:26:32.442 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.880+0000 7f064ffff700 1 -- 172.21.15.12:0/2841042172 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f065801a9c0 con 0x7f0668109ee0 2024-05-17T22:26:32.442 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.881+0000 7f066d2fb700 1 -- 172.21.15.12:0/2841042172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0668109ee0 msgr2=0x7f066819d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:26:32.442 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.881+0000 7f066d2fb700 1 --2- 172.21.15.12:0/2841042172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0668109ee0 0x7f066819d740 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f06580117b0 tx=0x7f0658011b70 comp rx=0 tx=0).stop 2024-05-17T22:26:32.443 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.881+0000 7f066d2fb700 1 -- 172.21.15.12:0/2841042172 shutdown_connections 2024-05-17T22:26:32.443 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.881+0000 7f066d2fb700 1 --2- 172.21.15.12:0/2841042172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0668109ee0 0x7f066819d740 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:32.443 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.881+0000 7f066d2fb700 1 -- 172.21.15.12:0/2841042172 >> 172.21.15.12:0/2841042172 conn(0x7f0668103ab0 msgr2=0x7f06681043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:26:32.443 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.881+0000 7f066d2fb700 1 -- 172.21.15.12:0/2841042172 shutdown_connections 2024-05-17T22:26:32.443 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:31.881+0000 7f066d2fb700 1 -- 172.21.15.12:0/2841042172 wait complete. 2024-05-17T22:26:32.443 INFO:teuthology.orchestra.run.smithi012.stdout:mgr not available, waiting (3/15)... 2024-05-17T22:26:32.590 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:32 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:32.230+0000 7f6191fdf200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-17T22:26:32.591 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:32 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:32.458+0000 7f6191fdf200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-17T22:26:33.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:32 smithi012 ceph-mon[31632]: Activating manager daemon x 2024-05-17T22:26:33.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:32 smithi012 ceph-mon[31632]: mgrmap e2: x(active, starting, since 0.00373871s) 2024-05-17T22:26:33.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:32 smithi012 ceph-mon[31632]: from='mgr.14100 172.21.15.12:0/3497407937' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-17T22:26:33.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:32 smithi012 ceph-mon[31632]: from='mgr.14100 172.21.15.12:0/3497407937' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-17T22:26:33.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:32 smithi012 ceph-mon[31632]: from='mgr.14100 172.21.15.12:0/3497407937' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-17T22:26:33.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:32 smithi012 ceph-mon[31632]: from='mgr.14100 172.21.15.12:0/3497407937' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T22:26:33.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:32 smithi012 ceph-mon[31632]: from='mgr.14100 172.21.15.12:0/3497407937' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-17T22:26:33.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:32 smithi012 ceph-mon[31632]: Manager daemon x is now available 2024-05-17T22:26:33.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:32 smithi012 ceph-mon[31632]: from='mgr.14100 172.21.15.12:0/3497407937' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-17T22:26:33.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:32 smithi012 ceph-mon[31632]: from='mgr.14100 172.21.15.12:0/3497407937' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-17T22:26:33.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:32 smithi012 ceph-mon[31632]: from='mgr.14100 172.21.15.12:0/3497407937' entity='mgr.x' 2024-05-17T22:26:33.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:32 smithi012 ceph-mon[31632]: from='mgr.14100 172.21.15.12:0/3497407937' entity='mgr.x' 2024-05-17T22:26:33.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:32 smithi012 ceph-mon[31632]: from='mgr.14100 172.21.15.12:0/3497407937' entity='mgr.x' 2024-05-17T22:26:34.785 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:34 smithi012 ceph-mon[31632]: mgrmap e3: x(active, since 1.0084s) 2024-05-17T22:26:35.531 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:35 smithi012 ceph-mon[31632]: mgrmap e4: x(active, since 2s) 2024-05-17T22:26:35.531 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:35 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/2515140688' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-17T22:26:35.834 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-05-17T22:26:35.834 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-05-17T22:26:35.834 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsid": "4d593780-149c-11ef-bc9a-c7b262605968", 2024-05-17T22:26:35.834 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "health": { 2024-05-17T22:26:35.834 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-17T22:26:35.834 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-17T22:26:35.835 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-17T22:26:35.835 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-17T22:26:35.835 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-17T22:26:35.835 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-17T22:26:35.835 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 0 2024-05-17T22:26:35.835 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-05-17T22:26:35.835 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-17T22:26:35.835 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "a" 2024-05-17T22:26:35.835 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-05-17T22:26:35.835 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-05-17T22:26:35.835 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-17T22:26:35.835 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T22:26:35.835 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-17T22:26:35.836 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-17T22:26:35.836 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-17T22:26:35.836 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-17T22:26:35.836 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T22:26:35.836 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-17T22:26:35.836 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-17T22:26:35.836 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-17T22:26:35.836 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-17T22:26:35.836 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-17T22:26:35.836 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-17T22:26:35.836 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-17T22:26:35.836 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-17T22:26:35.836 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-17T22:26:35.836 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-17T22:26:35.836 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-17T22:26:35.837 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-17T22:26:35.837 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-17T22:26:35.837 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-17T22:26:35.837 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-17T22:26:35.837 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-17T22:26:35.837 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-17T22:26:35.837 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-17T22:26:35.837 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T22:26:35.837 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-17T22:26:35.837 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-17T22:26:35.837 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-17T22:26:35.837 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-17T22:26:35.837 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-17T22:26:35.837 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-17T22:26:35.837 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-17T22:26:35.838 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-17T22:26:35.838 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-17T22:26:35.838 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "restful" 2024-05-17T22:26:35.838 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-05-17T22:26:35.838 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-17T22:26:35.838 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-17T22:26:35.838 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-17T22:26:35.838 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T22:26:35.838 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modified": "2024-05-17T22:26:16.956295+0000", 2024-05-17T22:26:35.838 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-17T22:26:35.838 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-05-17T22:26:35.838 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-17T22:26:35.838 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-05-17T22:26:35.839 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:34.909+0000 7fc07923a700 1 Processor -- start 2024-05-17T22:26:35.839 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:34.909+0000 7fc07923a700 1 -- start start 2024-05-17T22:26:35.839 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:34.909+0000 7fc07923a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc074107d00 0x7fc074108110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:35.839 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:34.909+0000 7fc07923a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc07407d620 con 0x7fc074107d00 2024-05-17T22:26:35.839 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:34.910+0000 7fc072d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc074107d00 0x7fc074108110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:26:35.839 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:34.910+0000 7fc072d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc074107d00 0x7fc074108110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35518/0 (socket says 172.21.15.12:35518) 2024-05-17T22:26:35.839 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:34.910+0000 7fc072d9d700 1 -- 172.21.15.12:0/1397841222 learned_addr learned my addr 172.21.15.12:0/1397841222 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:26:35.839 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:34.910+0000 7fc072d9d700 1 -- 172.21.15.12:0/1397841222 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc074108650 con 0x7fc074107d00 2024-05-17T22:26:35.839 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:34.910+0000 7fc072d9d700 1 --2- 172.21.15.12:0/1397841222 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc074107d00 0x7fc074108110 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc064009e80 tx=0x7fc06400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a2fa442d2d5dc02 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:26:35.839 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:34.910+0000 7fc071d9b700 1 -- 172.21.15.12:0/1397841222 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc064017070 con 0x7fc074107d00 2024-05-17T22:26:35.839 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:34.910+0000 7fc071d9b700 1 -- 172.21.15.12:0/1397841222 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc06400b8c0 con 0x7fc074107d00 2024-05-17T22:26:35.839 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:34.910+0000 7fc071d9b700 1 -- 172.21.15.12:0/1397841222 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc064004490 con 0x7fc074107d00 2024-05-17T22:26:35.839 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:34.911+0000 7fc07923a700 1 -- 172.21.15.12:0/1397841222 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc074107d00 msgr2=0x7fc074108110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:26:35.839 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:34.911+0000 7fc07923a700 1 --2- 172.21.15.12:0/1397841222 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc074107d00 0x7fc074108110 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc064009e80 tx=0x7fc06400b330 comp rx=0 tx=0).stop 2024-05-17T22:26:35.840 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:34.911+0000 7fc07923a700 1 -- 172.21.15.12:0/1397841222 shutdown_connections 2024-05-17T22:26:35.840 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:34.911+0000 7fc07923a700 1 --2- 172.21.15.12:0/1397841222 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc074107d00 0x7fc074108110 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:35.840 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:34.911+0000 7fc07923a700 1 -- 172.21.15.12:0/1397841222 >> 172.21.15.12:0/1397841222 conn(0x7fc074103ac0 msgr2=0x7fc074105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:26:35.840 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:34.911+0000 7fc07923a700 1 -- 172.21.15.12:0/1397841222 shutdown_connections 2024-05-17T22:26:35.840 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:34.911+0000 7fc07923a700 1 -- 172.21.15.12:0/1397841222 wait complete. 2024-05-17T22:26:35.840 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:34.912+0000 7fc07923a700 1 Processor -- start 2024-05-17T22:26:35.840 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:34.912+0000 7fc07923a700 1 -- start start 2024-05-17T22:26:35.840 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:34.913+0000 7fc07923a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc074107d00 0x7fc07419f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:35.840 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:34.913+0000 7fc07923a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc074109550 con 0x7fc074107d00 2024-05-17T22:26:35.840 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:34.913+0000 7fc072d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc074107d00 0x7fc07419f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:26:35.840 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:34.913+0000 7fc072d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc074107d00 0x7fc07419f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35526/0 (socket says 172.21.15.12:35526) 2024-05-17T22:26:35.840 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:34.913+0000 7fc072d9d700 1 -- 172.21.15.12:0/2515140688 learned_addr learned my addr 172.21.15.12:0/2515140688 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:26:35.840 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:34.913+0000 7fc072d9d700 1 -- 172.21.15.12:0/2515140688 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc07419fde0 con 0x7fc074107d00 2024-05-17T22:26:35.841 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:34.913+0000 7fc072d9d700 1 --2- 172.21.15.12:0/2515140688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc074107d00 0x7fc07419f8a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc064004220 tx=0x7fc06400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:26:35.841 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:34.913+0000 7fc06bfff700 1 -- 172.21.15.12:0/2515140688 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc064020020 con 0x7fc074107d00 2024-05-17T22:26:35.841 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:34.913+0000 7fc06bfff700 1 -- 172.21.15.12:0/2515140688 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc064012430 con 0x7fc074107d00 2024-05-17T22:26:35.841 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:34.913+0000 7fc07923a700 1 -- 172.21.15.12:0/2515140688 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0741a0070 con 0x7fc074107d00 2024-05-17T22:26:35.841 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:34.913+0000 7fc06bfff700 1 -- 172.21.15.12:0/2515140688 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc064017040 con 0x7fc074107d00 2024-05-17T22:26:35.841 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:34.913+0000 7fc07923a700 1 -- 172.21.15.12:0/2515140688 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0741a2c80 con 0x7fc074107d00 2024-05-17T22:26:35.841 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:34.914+0000 7fc06bfff700 1 -- 172.21.15.12:0/2515140688 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7fc064023020 con 0x7fc074107d00 2024-05-17T22:26:35.841 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:34.914+0000 7fc07923a700 1 -- 172.21.15.12:0/2515140688 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc074109c20 con 0x7fc074107d00 2024-05-17T22:26:35.841 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:34.915+0000 7fc06bfff700 1 --2- 172.21.15.12:0/2515140688 >> [v2:172.21.15.12:6800/1039789215,v1:172.21.15.12:6801/1039789215] conn(0x7fc06003e490 0x7fc0600409a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:35.841 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:34.915+0000 7fc06bfff700 1 -- 172.21.15.12:0/2515140688 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc06400f050 con 0x7fc074107d00 2024-05-17T22:26:35.841 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:34.915+0000 7fc07259c700 1 --2- 172.21.15.12:0/2515140688 >> [v2:172.21.15.12:6800/1039789215,v1:172.21.15.12:6801/1039789215] conn(0x7fc06003e490 0x7fc0600409a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:26:35.841 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:34.915+0000 7fc07259c700 1 --2- 172.21.15.12:0/2515140688 >> [v2:172.21.15.12:6800/1039789215,v1:172.21.15.12:6801/1039789215] conn(0x7fc06003e490 0x7fc0600409a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc05c006fd0 tx=0x7fc05c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:26:35.841 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:34.920+0000 7fc06bfff700 1 -- 172.21.15.12:0/2515140688 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc06402a020 con 0x7fc074107d00 2024-05-17T22:26:35.841 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:35.228+0000 7fc07923a700 1 -- 172.21.15.12:0/2515140688 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc074064540 con 0x7fc074107d00 2024-05-17T22:26:35.842 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:35.228+0000 7fc06bfff700 1 -- 172.21.15.12:0/2515140688 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fc06402a200 con 0x7fc074107d00 2024-05-17T22:26:35.842 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:35.231+0000 7fc07923a700 1 -- 172.21.15.12:0/2515140688 >> [v2:172.21.15.12:6800/1039789215,v1:172.21.15.12:6801/1039789215] conn(0x7fc06003e490 msgr2=0x7fc0600409a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:26:35.842 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:35.231+0000 7fc07923a700 1 --2- 172.21.15.12:0/2515140688 >> [v2:172.21.15.12:6800/1039789215,v1:172.21.15.12:6801/1039789215] conn(0x7fc06003e490 0x7fc0600409a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc05c006fd0 tx=0x7fc05c006e40 comp rx=0 tx=0).stop 2024-05-17T22:26:35.842 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:35.231+0000 7fc07923a700 1 -- 172.21.15.12:0/2515140688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc074107d00 msgr2=0x7fc07419f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:26:35.842 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:35.231+0000 7fc07923a700 1 --2- 172.21.15.12:0/2515140688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc074107d00 0x7fc07419f8a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc064004220 tx=0x7fc06400bd50 comp rx=0 tx=0).stop 2024-05-17T22:26:35.842 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:35.231+0000 7fc07923a700 1 -- 172.21.15.12:0/2515140688 shutdown_connections 2024-05-17T22:26:35.842 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:35.231+0000 7fc07923a700 1 --2- 172.21.15.12:0/2515140688 >> [v2:172.21.15.12:6800/1039789215,v1:172.21.15.12:6801/1039789215] conn(0x7fc06003e490 0x7fc0600409a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:35.842 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:35.231+0000 7fc07923a700 1 --2- 172.21.15.12:0/2515140688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc074107d00 0x7fc07419f8a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:35.842 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:35.231+0000 7fc07923a700 1 -- 172.21.15.12:0/2515140688 >> 172.21.15.12:0/2515140688 conn(0x7fc074103ac0 msgr2=0x7fc0741939d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:26:35.842 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:35.231+0000 7fc07923a700 1 -- 172.21.15.12:0/2515140688 shutdown_connections 2024-05-17T22:26:35.842 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:35.231+0000 7fc07923a700 1 -- 172.21.15.12:0/2515140688 wait complete. 2024-05-17T22:26:35.842 INFO:teuthology.orchestra.run.smithi012.stdout:mgr is available 2024-05-17T22:26:36.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:36 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/3203655371' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-17T22:26:37.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-05-17T22:26:37.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [global] 2024-05-17T22:26:37.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout fsid = 4d593780-149c-11ef-bc9a-c7b262605968 2024-05-17T22:26:37.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-17T22:26:37.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-17T22:26:37.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-17T22:26:37.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-17T22:26:37.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-17T22:26:37.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-05-17T22:26:37.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-17T22:26:37.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-17T22:26:37.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-05-17T22:26:37.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [osd] 2024-05-17T22:26:37.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-17T22:26:37.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-17T22:26:37.131 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-17T22:26:37.131 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-17T22:26:37.131 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-17T22:26:37.131 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.322+0000 7f8d216a5700 1 Processor -- start 2024-05-17T22:26:37.131 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.322+0000 7f8d216a5700 1 -- start start 2024-05-17T22:26:37.131 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.322+0000 7f8d216a5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d1c1082a0 0x7f8d1c1086b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:37.131 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.322+0000 7f8d216a5700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1c108c80 con 0x7f8d1c1082a0 2024-05-17T22:26:37.131 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.322+0000 7f8d1affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d1c1082a0 0x7f8d1c1086b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:26:37.131 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.322+0000 7f8d1affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d1c1082a0 0x7f8d1c1086b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35542/0 (socket says 172.21.15.12:35542) 2024-05-17T22:26:37.131 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.322+0000 7f8d1affd700 1 -- 172.21.15.12:0/963827784 learned_addr learned my addr 172.21.15.12:0/963827784 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:26:37.131 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.322+0000 7f8d1affd700 1 -- 172.21.15.12:0/963827784 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d1c109470 con 0x7f8d1c1082a0 2024-05-17T22:26:37.131 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.323+0000 7f8d1affd700 1 --2- 172.21.15.12:0/963827784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d1c1082a0 0x7f8d1c1086b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c0098a0 tx=0x7f8d0c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9945f6791b5bd463 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:26:37.131 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.323+0000 7f8d19ffb700 1 -- 172.21.15.12:0/963827784 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d0c004160 con 0x7f8d1c1082a0 2024-05-17T22:26:37.131 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.323+0000 7f8d19ffb700 1 -- 172.21.15.12:0/963827784 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8d0c004300 con 0x7f8d1c1082a0 2024-05-17T22:26:37.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.323+0000 7f8d19ffb700 1 -- 172.21.15.12:0/963827784 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d0c011af0 con 0x7f8d1c1082a0 2024-05-17T22:26:37.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.323+0000 7f8d216a5700 1 -- 172.21.15.12:0/963827784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d1c1082a0 msgr2=0x7f8d1c1086b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:26:37.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.323+0000 7f8d216a5700 1 --2- 172.21.15.12:0/963827784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d1c1082a0 0x7f8d1c1086b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c0098a0 tx=0x7f8d0c009bb0 comp rx=0 tx=0).stop 2024-05-17T22:26:37.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.324+0000 7f8d216a5700 1 -- 172.21.15.12:0/963827784 shutdown_connections 2024-05-17T22:26:37.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.324+0000 7f8d216a5700 1 --2- 172.21.15.12:0/963827784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d1c1082a0 0x7f8d1c1086b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:37.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.324+0000 7f8d216a5700 1 -- 172.21.15.12:0/963827784 >> 172.21.15.12:0/963827784 conn(0x7f8d1c103ab0 msgr2=0x7f8d1c105f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:26:37.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.324+0000 7f8d216a5700 1 -- 172.21.15.12:0/963827784 shutdown_connections 2024-05-17T22:26:37.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.324+0000 7f8d216a5700 1 -- 172.21.15.12:0/963827784 wait complete. 2024-05-17T22:26:37.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.324+0000 7f8d216a5700 1 Processor -- start 2024-05-17T22:26:37.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.324+0000 7f8d216a5700 1 -- start start 2024-05-17T22:26:37.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.325+0000 7f8d216a5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d1c1082a0 0x7f8d1c105c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:37.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.325+0000 7f8d216a5700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1c10c820 con 0x7f8d1c1082a0 2024-05-17T22:26:37.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.325+0000 7f8d1affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d1c1082a0 0x7f8d1c105c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:26:37.133 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.325+0000 7f8d1affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d1c1082a0 0x7f8d1c105c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35552/0 (socket says 172.21.15.12:35552) 2024-05-17T22:26:37.133 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.325+0000 7f8d1affd700 1 -- 172.21.15.12:0/3203655371 learned_addr learned my addr 172.21.15.12:0/3203655371 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:26:37.133 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.325+0000 7f8d1affd700 1 -- 172.21.15.12:0/3203655371 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d1c106150 con 0x7f8d1c1082a0 2024-05-17T22:26:37.133 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.325+0000 7f8d1affd700 1 --2- 172.21.15.12:0/3203655371 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d1c1082a0 0x7f8d1c105c10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c009fb0 tx=0x7f8d0c011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:26:37.133 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.325+0000 7f8d03fff700 1 -- 172.21.15.12:0/3203655371 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d0c0036a0 con 0x7f8d1c1082a0 2024-05-17T22:26:37.133 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.325+0000 7f8d03fff700 1 -- 172.21.15.12:0/3203655371 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8d0c01a430 con 0x7f8d1c1082a0 2024-05-17T22:26:37.133 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.326+0000 7f8d216a5700 1 -- 172.21.15.12:0/3203655371 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d1c1043a0 con 0x7f8d1c1082a0 2024-05-17T22:26:37.133 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.326+0000 7f8d03fff700 1 -- 172.21.15.12:0/3203655371 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d0c019550 con 0x7f8d1c1082a0 2024-05-17T22:26:37.133 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.326+0000 7f8d216a5700 1 -- 172.21.15.12:0/3203655371 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d1c104880 con 0x7f8d1c1082a0 2024-05-17T22:26:37.133 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.326+0000 7f8d216a5700 1 -- 172.21.15.12:0/3203655371 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d1c1086b0 con 0x7f8d1c1082a0 2024-05-17T22:26:37.133 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.327+0000 7f8d03fff700 1 -- 172.21.15.12:0/3203655371 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f8d0c019800 con 0x7f8d1c1082a0 2024-05-17T22:26:37.133 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.327+0000 7f8d03fff700 1 --2- 172.21.15.12:0/3203655371 >> [v2:172.21.15.12:6800/1039789215,v1:172.21.15.12:6801/1039789215] conn(0x7f8d0403e4e0 0x7f8d040409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:37.133 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.327+0000 7f8d03fff700 1 -- 172.21.15.12:0/3203655371 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8d0c0537f0 con 0x7f8d1c1082a0 2024-05-17T22:26:37.133 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.327+0000 7f8d1a7fc700 1 --2- 172.21.15.12:0/3203655371 >> [v2:172.21.15.12:6800/1039789215,v1:172.21.15.12:6801/1039789215] conn(0x7f8d0403e4e0 0x7f8d040409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:26:37.134 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.327+0000 7f8d1a7fc700 1 --2- 172.21.15.12:0/3203655371 >> [v2:172.21.15.12:6800/1039789215,v1:172.21.15.12:6801/1039789215] conn(0x7f8d0403e4e0 0x7f8d040409f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8d10006fd0 tx=0x7f8d10006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:26:37.134 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.332+0000 7f8d03fff700 1 -- 172.21.15.12:0/3203655371 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8d0c017e90 con 0x7f8d1c1082a0 2024-05-17T22:26:37.134 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.548+0000 7f8d216a5700 1 -- 172.21.15.12:0/3203655371 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f8d1c1a4950 con 0x7f8d1c1082a0 2024-05-17T22:26:37.134 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.550+0000 7f8d03fff700 1 -- 172.21.15.12:0/3203655371 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7f8d0c023390 con 0x7f8d1c1082a0 2024-05-17T22:26:37.134 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.552+0000 7f8d216a5700 1 -- 172.21.15.12:0/3203655371 >> [v2:172.21.15.12:6800/1039789215,v1:172.21.15.12:6801/1039789215] conn(0x7f8d0403e4e0 msgr2=0x7f8d040409f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:26:37.134 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.552+0000 7f8d216a5700 1 --2- 172.21.15.12:0/3203655371 >> [v2:172.21.15.12:6800/1039789215,v1:172.21.15.12:6801/1039789215] conn(0x7f8d0403e4e0 0x7f8d040409f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8d10006fd0 tx=0x7f8d10006e40 comp rx=0 tx=0).stop 2024-05-17T22:26:37.134 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.552+0000 7f8d216a5700 1 -- 172.21.15.12:0/3203655371 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d1c1082a0 msgr2=0x7f8d1c105c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:26:37.134 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.552+0000 7f8d216a5700 1 --2- 172.21.15.12:0/3203655371 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d1c1082a0 0x7f8d1c105c10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c009fb0 tx=0x7f8d0c011940 comp rx=0 tx=0).stop 2024-05-17T22:26:37.134 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.552+0000 7f8d216a5700 1 -- 172.21.15.12:0/3203655371 shutdown_connections 2024-05-17T22:26:37.135 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.552+0000 7f8d216a5700 1 --2- 172.21.15.12:0/3203655371 >> [v2:172.21.15.12:6800/1039789215,v1:172.21.15.12:6801/1039789215] conn(0x7f8d0403e4e0 0x7f8d040409f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:37.135 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.552+0000 7f8d216a5700 1 --2- 172.21.15.12:0/3203655371 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d1c1082a0 0x7f8d1c105c10 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:37.135 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.552+0000 7f8d216a5700 1 -- 172.21.15.12:0/3203655371 >> 172.21.15.12:0/3203655371 conn(0x7f8d1c103ab0 msgr2=0x7f8d1c193d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:26:37.135 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.553+0000 7f8d216a5700 1 -- 172.21.15.12:0/3203655371 shutdown_connections 2024-05-17T22:26:37.135 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:36.553+0000 7f8d216a5700 1 -- 172.21.15.12:0/3203655371 wait complete. 2024-05-17T22:26:37.135 INFO:teuthology.orchestra.run.smithi012.stdout:Enabling cephadm module... 2024-05-17T22:26:38.146 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:37 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/1348185937' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-17T22:26:38.146 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:38 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: ignoring --setuser ceph since I am not root 2024-05-17T22:26:38.146 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:38 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: ignoring --setgroup ceph since I am not root 2024-05-17T22:26:38.147 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:38 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:38.144+0000 7f4ca64ff200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-17T22:26:38.450 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:38 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:38.340+0000 7f4ca64ff200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-17T22:26:38.576 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.594+0000 7fec95a90700 1 Processor -- start 2024-05-17T22:26:38.576 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.594+0000 7fec95a90700 1 -- start start 2024-05-17T22:26:38.577 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.594+0000 7fec95a90700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec901093f0 0x7fec90109800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:38.577 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.594+0000 7fec95a90700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fec90109dd0 con 0x7fec901093f0 2024-05-17T22:26:38.577 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.594+0000 7fec8f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec901093f0 0x7fec90109800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:26:38.577 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.594+0000 7fec8f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec901093f0 0x7fec90109800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35556/0 (socket says 172.21.15.12:35556) 2024-05-17T22:26:38.577 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.594+0000 7fec8f7fe700 1 -- 172.21.15.12:0/2849357778 learned_addr learned my addr 172.21.15.12:0/2849357778 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:26:38.577 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.595+0000 7fec8f7fe700 1 -- 172.21.15.12:0/2849357778 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec9010a5c0 con 0x7fec901093f0 2024-05-17T22:26:38.577 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.595+0000 7fec8f7fe700 1 --2- 172.21.15.12:0/2849357778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec901093f0 0x7fec90109800 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fec800098a0 tx=0x7fec80009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbe6bda6a3c4c69 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:26:38.577 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.595+0000 7fec8e7fc700 1 -- 172.21.15.12:0/2849357778 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec80004160 con 0x7fec901093f0 2024-05-17T22:26:38.577 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.595+0000 7fec8e7fc700 1 -- 172.21.15.12:0/2849357778 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fec80004300 con 0x7fec901093f0 2024-05-17T22:26:38.577 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.595+0000 7fec8e7fc700 1 -- 172.21.15.12:0/2849357778 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec80011af0 con 0x7fec901093f0 2024-05-17T22:26:38.577 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.596+0000 7fec95a90700 1 -- 172.21.15.12:0/2849357778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec901093f0 msgr2=0x7fec90109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:26:38.577 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.596+0000 7fec95a90700 1 --2- 172.21.15.12:0/2849357778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec901093f0 0x7fec90109800 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fec800098a0 tx=0x7fec80009bb0 comp rx=0 tx=0).stop 2024-05-17T22:26:38.577 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.596+0000 7fec95a90700 1 -- 172.21.15.12:0/2849357778 shutdown_connections 2024-05-17T22:26:38.578 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.596+0000 7fec95a90700 1 --2- 172.21.15.12:0/2849357778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec901093f0 0x7fec90109800 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:38.578 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.596+0000 7fec95a90700 1 -- 172.21.15.12:0/2849357778 >> 172.21.15.12:0/2849357778 conn(0x7fec90104c00 msgr2=0x7fec901070d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:26:38.578 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.596+0000 7fec95a90700 1 -- 172.21.15.12:0/2849357778 shutdown_connections 2024-05-17T22:26:38.578 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.596+0000 7fec95a90700 1 -- 172.21.15.12:0/2849357778 wait complete. 2024-05-17T22:26:38.578 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.596+0000 7fec95a90700 1 Processor -- start 2024-05-17T22:26:38.578 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.597+0000 7fec95a90700 1 -- start start 2024-05-17T22:26:38.578 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.597+0000 7fec95a90700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec901093f0 0x7fec9019d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:38.578 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.597+0000 7fec95a90700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fec9010d970 con 0x7fec901093f0 2024-05-17T22:26:38.578 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.597+0000 7fec8f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec901093f0 0x7fec9019d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:26:38.578 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.597+0000 7fec8f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec901093f0 0x7fec9019d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35560/0 (socket says 172.21.15.12:35560) 2024-05-17T22:26:38.578 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.597+0000 7fec8f7fe700 1 -- 172.21.15.12:0/1348185937 learned_addr learned my addr 172.21.15.12:0/1348185937 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:26:38.578 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.597+0000 7fec8f7fe700 1 -- 172.21.15.12:0/1348185937 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec9019dcd0 con 0x7fec901093f0 2024-05-17T22:26:38.578 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.597+0000 7fec8f7fe700 1 --2- 172.21.15.12:0/1348185937 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec901093f0 0x7fec9019d790 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fec80009fb0 tx=0x7fec80011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:26:38.579 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.598+0000 7fec8cff9700 1 -- 172.21.15.12:0/1348185937 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec80011bc0 con 0x7fec901093f0 2024-05-17T22:26:38.579 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.598+0000 7fec8cff9700 1 -- 172.21.15.12:0/1348185937 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fec80024440 con 0x7fec901093f0 2024-05-17T22:26:38.579 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.598+0000 7fec95a90700 1 -- 172.21.15.12:0/1348185937 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec9019df60 con 0x7fec901093f0 2024-05-17T22:26:38.579 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.598+0000 7fec8cff9700 1 -- 172.21.15.12:0/1348185937 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec8001a5c0 con 0x7fec901093f0 2024-05-17T22:26:38.579 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.598+0000 7fec95a90700 1 -- 172.21.15.12:0/1348185937 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec901a0b70 con 0x7fec901093f0 2024-05-17T22:26:38.579 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.599+0000 7fec8cff9700 1 -- 172.21.15.12:0/1348185937 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7fec8001a830 con 0x7fec901093f0 2024-05-17T22:26:38.579 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.599+0000 7fec95a90700 1 -- 172.21.15.12:0/1348185937 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec90109800 con 0x7fec901093f0 2024-05-17T22:26:38.579 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.599+0000 7fec8cff9700 1 --2- 172.21.15.12:0/1348185937 >> [v2:172.21.15.12:6800/1039789215,v1:172.21.15.12:6801/1039789215] conn(0x7fec7803e4e0 0x7fec780409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:38.579 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.599+0000 7fec8cff9700 1 -- 172.21.15.12:0/1348185937 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fec800537b0 con 0x7fec901093f0 2024-05-17T22:26:38.579 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.599+0000 7fec8effd700 1 --2- 172.21.15.12:0/1348185937 >> [v2:172.21.15.12:6800/1039789215,v1:172.21.15.12:6801/1039789215] conn(0x7fec7803e4e0 0x7fec780409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:26:38.579 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.599+0000 7fec8effd700 1 --2- 172.21.15.12:0/1348185937 >> [v2:172.21.15.12:6800/1039789215,v1:172.21.15.12:6801/1039789215] conn(0x7fec7803e4e0 0x7fec780409f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fec84006fd0 tx=0x7fec84006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:26:38.579 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.604+0000 7fec8cff9700 1 -- 172.21.15.12:0/1348185937 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fec8000b3e0 con 0x7fec901093f0 2024-05-17T22:26:38.579 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.881+0000 7fec95a90700 1 -- 172.21.15.12:0/1348185937 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fec901a1640 con 0x7fec901093f0 2024-05-17T22:26:38.580 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.933+0000 7fec8cff9700 1 -- 172.21.15.12:0/1348185937 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fec80029df0 con 0x7fec901093f0 2024-05-17T22:26:38.580 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.935+0000 7fec8cff9700 1 -- 172.21.15.12:0/1348185937 <== mon.0 v2:172.21.15.12:3300/0 8 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7fec800245f0 con 0x7fec901093f0 2024-05-17T22:26:38.580 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.936+0000 7fec95a90700 1 -- 172.21.15.12:0/1348185937 >> [v2:172.21.15.12:6800/1039789215,v1:172.21.15.12:6801/1039789215] conn(0x7fec7803e4e0 msgr2=0x7fec780409f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:26:38.580 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.936+0000 7fec95a90700 1 --2- 172.21.15.12:0/1348185937 >> [v2:172.21.15.12:6800/1039789215,v1:172.21.15.12:6801/1039789215] conn(0x7fec7803e4e0 0x7fec780409f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fec84006fd0 tx=0x7fec84006e40 comp rx=0 tx=0).stop 2024-05-17T22:26:38.580 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.936+0000 7fec95a90700 1 -- 172.21.15.12:0/1348185937 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec901093f0 msgr2=0x7fec9019d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:26:38.580 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.936+0000 7fec95a90700 1 --2- 172.21.15.12:0/1348185937 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec901093f0 0x7fec9019d790 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fec80009fb0 tx=0x7fec80011890 comp rx=0 tx=0).stop 2024-05-17T22:26:38.580 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.936+0000 7fec95a90700 1 -- 172.21.15.12:0/1348185937 shutdown_connections 2024-05-17T22:26:38.580 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.936+0000 7fec95a90700 1 --2- 172.21.15.12:0/1348185937 >> [v2:172.21.15.12:6800/1039789215,v1:172.21.15.12:6801/1039789215] conn(0x7fec7803e4e0 0x7fec780409f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:38.580 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.936+0000 7fec95a90700 1 --2- 172.21.15.12:0/1348185937 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec901093f0 0x7fec9019d790 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:38.580 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.936+0000 7fec95a90700 1 -- 172.21.15.12:0/1348185937 >> 172.21.15.12:0/1348185937 conn(0x7fec90104c00 msgr2=0x7fec901054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:26:38.580 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.936+0000 7fec95a90700 1 -- 172.21.15.12:0/1348185937 shutdown_connections 2024-05-17T22:26:38.580 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:37.936+0000 7fec95a90700 1 -- 172.21.15.12:0/1348185937 wait complete. 2024-05-17T22:26:38.703 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:38 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:38.448+0000 7f4ca64ff200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-17T22:26:38.972 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:38 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/1348185937' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-17T22:26:38.972 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:38 smithi012 ceph-mon[31632]: mgrmap e5: x(active, since 5s) 2024-05-17T22:26:38.972 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:38 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:38.779+0000 7f4ca64ff200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-17T22:26:38.972 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:38 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:38.871+0000 7f4ca64ff200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-17T22:26:39.934 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-05-17T22:26:39.934 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-17T22:26:39.934 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-17T22:26:39.935 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-17T22:26:39.935 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-17T22:26:39.935 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-05-17T22:26:39.935 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:38.992+0000 7f4b4d57a700 1 Processor -- start 2024-05-17T22:26:39.935 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:38.993+0000 7f4b4d57a700 1 -- start start 2024-05-17T22:26:39.935 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:38.993+0000 7f4b4d57a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b48109ee0 0x7f4b4810c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:39.935 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:38.993+0000 7f4b4d57a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4807d770 con 0x7f4b48109ee0 2024-05-17T22:26:39.935 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:38.993+0000 7f4b46ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b48109ee0 0x7f4b4810c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:26:39.935 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:38.993+0000 7f4b46ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b48109ee0 0x7f4b4810c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35576/0 (socket says 172.21.15.12:35576) 2024-05-17T22:26:39.935 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:38.993+0000 7f4b46ffd700 1 -- 172.21.15.12:0/491767990 learned_addr learned my addr 172.21.15.12:0/491767990 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:26:39.935 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:38.993+0000 7f4b46ffd700 1 -- 172.21.15.12:0/491767990 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b4810c840 con 0x7f4b48109ee0 2024-05-17T22:26:39.935 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:38.994+0000 7f4b46ffd700 1 --2- 172.21.15.12:0/491767990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b48109ee0 0x7f4b4810c300 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4b380098a0 tx=0x7f4b38009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba026b6c01ae1d35 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:26:39.935 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:38.994+0000 7f4b45ffb700 1 -- 172.21.15.12:0/491767990 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b38004160 con 0x7f4b48109ee0 2024-05-17T22:26:39.936 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:38.994+0000 7f4b45ffb700 1 -- 172.21.15.12:0/491767990 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4b38004300 con 0x7f4b48109ee0 2024-05-17T22:26:39.936 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:38.994+0000 7f4b45ffb700 1 -- 172.21.15.12:0/491767990 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b38011af0 con 0x7f4b48109ee0 2024-05-17T22:26:39.936 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:38.994+0000 7f4b4d57a700 1 -- 172.21.15.12:0/491767990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b48109ee0 msgr2=0x7f4b4810c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:26:39.936 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:38.994+0000 7f4b4d57a700 1 --2- 172.21.15.12:0/491767990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b48109ee0 0x7f4b4810c300 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4b380098a0 tx=0x7f4b38009bb0 comp rx=0 tx=0).stop 2024-05-17T22:26:39.936 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:38.995+0000 7f4b4d57a700 1 -- 172.21.15.12:0/491767990 shutdown_connections 2024-05-17T22:26:39.936 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:38.995+0000 7f4b4d57a700 1 --2- 172.21.15.12:0/491767990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b48109ee0 0x7f4b4810c300 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:39.936 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:38.995+0000 7f4b4d57a700 1 -- 172.21.15.12:0/491767990 >> 172.21.15.12:0/491767990 conn(0x7f4b48103ab0 msgr2=0x7f4b48105f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:26:39.936 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:38.995+0000 7f4b4d57a700 1 -- 172.21.15.12:0/491767990 shutdown_connections 2024-05-17T22:26:39.936 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:38.995+0000 7f4b4d57a700 1 -- 172.21.15.12:0/491767990 wait complete. 2024-05-17T22:26:39.936 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:38.995+0000 7f4b4d57a700 1 Processor -- start 2024-05-17T22:26:39.936 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:38.996+0000 7f4b4d57a700 1 -- start start 2024-05-17T22:26:39.936 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:38.996+0000 7f4b4d57a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b48109ee0 0x7f4b4819d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:39.936 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:38.996+0000 7f4b4d57a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4810d7a0 con 0x7f4b48109ee0 2024-05-17T22:26:39.937 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:38.996+0000 7f4b46ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b48109ee0 0x7f4b4819d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:26:39.937 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:38.996+0000 7f4b46ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b48109ee0 0x7f4b4819d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35588/0 (socket says 172.21.15.12:35588) 2024-05-17T22:26:39.937 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:38.996+0000 7f4b46ffd700 1 -- 172.21.15.12:0/4237201404 learned_addr learned my addr 172.21.15.12:0/4237201404 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:26:39.937 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:38.996+0000 7f4b46ffd700 1 -- 172.21.15.12:0/4237201404 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b4819dc80 con 0x7f4b48109ee0 2024-05-17T22:26:39.937 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:38.996+0000 7f4b46ffd700 1 --2- 172.21.15.12:0/4237201404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b48109ee0 0x7f4b4819d740 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4b380117b0 tx=0x7f4b38011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:26:39.937 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:38.997+0000 7f4b3ffff700 1 -- 172.21.15.12:0/4237201404 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b38003af0 con 0x7f4b48109ee0 2024-05-17T22:26:39.937 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:38.997+0000 7f4b3ffff700 1 -- 172.21.15.12:0/4237201404 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4b38020070 con 0x7f4b48109ee0 2024-05-17T22:26:39.937 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:38.997+0000 7f4b3ffff700 1 -- 172.21.15.12:0/4237201404 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b38019630 con 0x7f4b48109ee0 2024-05-17T22:26:39.937 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:38.997+0000 7f4b4d57a700 1 -- 172.21.15.12:0/4237201404 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b4819df10 con 0x7f4b48109ee0 2024-05-17T22:26:39.937 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:38.997+0000 7f4b4d57a700 1 -- 172.21.15.12:0/4237201404 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b481a0b20 con 0x7f4b48109ee0 2024-05-17T22:26:39.937 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:38.998+0000 7f4b3ffff700 1 -- 172.21.15.12:0/4237201404 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f4b380198a0 con 0x7f4b48109ee0 2024-05-17T22:26:39.937 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:38.998+0000 7f4b4d57a700 1 -- 172.21.15.12:0/4237201404 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b4810a370 con 0x7f4b48109ee0 2024-05-17T22:26:39.937 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:38.998+0000 7f4b3ffff700 1 --2- 172.21.15.12:0/4237201404 >> [v2:172.21.15.12:6800/1039789215,v1:172.21.15.12:6801/1039789215] conn(0x7f4b3403e580 0x7f4b34040a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:39.938 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:38.998+0000 7f4b3ffff700 1 -- 172.21.15.12:0/4237201404 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4b38053a80 con 0x7f4b48109ee0 2024-05-17T22:26:39.938 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:38.998+0000 7f4b467fc700 1 -- 172.21.15.12:0/4237201404 >> [v2:172.21.15.12:6800/1039789215,v1:172.21.15.12:6801/1039789215] conn(0x7f4b3403e580 msgr2=0x7f4b34040a90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/1039789215 2024-05-17T22:26:39.938 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:38.998+0000 7f4b467fc700 1 --2- 172.21.15.12:0/4237201404 >> [v2:172.21.15.12:6800/1039789215,v1:172.21.15.12:6801/1039789215] conn(0x7f4b3403e580 0x7f4b34040a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-17T22:26:39.938 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:39.003+0000 7f4b3ffff700 1 -- 172.21.15.12:0/4237201404 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4b38016260 con 0x7f4b48109ee0 2024-05-17T22:26:39.938 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:39.199+0000 7f4b467fc700 1 -- 172.21.15.12:0/4237201404 >> [v2:172.21.15.12:6800/1039789215,v1:172.21.15.12:6801/1039789215] conn(0x7f4b3403e580 msgr2=0x7f4b34040a90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/1039789215 2024-05-17T22:26:39.938 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:39.199+0000 7f4b467fc700 1 --2- 172.21.15.12:0/4237201404 >> [v2:172.21.15.12:6800/1039789215,v1:172.21.15.12:6801/1039789215] conn(0x7f4b3403e580 0x7f4b34040a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-17T22:26:39.938 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:39.307+0000 7f4b4d57a700 1 -- 172.21.15.12:0/4237201404 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f4b481a15c0 con 0x7f4b48109ee0 2024-05-17T22:26:39.938 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:39.308+0000 7f4b3ffff700 1 -- 172.21.15.12:0/4237201404 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f4b380534a0 con 0x7f4b48109ee0 2024-05-17T22:26:39.938 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:39.315+0000 7f4b4d57a700 1 -- 172.21.15.12:0/4237201404 >> [v2:172.21.15.12:6800/1039789215,v1:172.21.15.12:6801/1039789215] conn(0x7f4b3403e580 msgr2=0x7f4b34040a90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-17T22:26:39.938 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:39.315+0000 7f4b4d57a700 1 --2- 172.21.15.12:0/4237201404 >> [v2:172.21.15.12:6800/1039789215,v1:172.21.15.12:6801/1039789215] conn(0x7f4b3403e580 0x7f4b34040a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:39.938 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:39.315+0000 7f4b4d57a700 1 -- 172.21.15.12:0/4237201404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b48109ee0 msgr2=0x7f4b4819d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:26:39.938 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:39.315+0000 7f4b4d57a700 1 --2- 172.21.15.12:0/4237201404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b48109ee0 0x7f4b4819d740 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4b380117b0 tx=0x7f4b38011b70 comp rx=0 tx=0).stop 2024-05-17T22:26:39.938 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:39.316+0000 7f4b4d57a700 1 -- 172.21.15.12:0/4237201404 shutdown_connections 2024-05-17T22:26:39.939 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:39.316+0000 7f4b4d57a700 1 --2- 172.21.15.12:0/4237201404 >> [v2:172.21.15.12:6800/1039789215,v1:172.21.15.12:6801/1039789215] conn(0x7f4b3403e580 0x7f4b34040a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:39.939 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:39.316+0000 7f4b4d57a700 1 --2- 172.21.15.12:0/4237201404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b48109ee0 0x7f4b4819d740 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:39.939 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:39.316+0000 7f4b4d57a700 1 -- 172.21.15.12:0/4237201404 >> 172.21.15.12:0/4237201404 conn(0x7f4b48103ab0 msgr2=0x7f4b481043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:26:39.939 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:39.317+0000 7f4b4d57a700 1 -- 172.21.15.12:0/4237201404 shutdown_connections 2024-05-17T22:26:39.939 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:39.317+0000 7f4b4d57a700 1 -- 172.21.15.12:0/4237201404 wait complete. 2024-05-17T22:26:39.939 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for the mgr to restart... 2024-05-17T22:26:39.939 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mgr epoch 5... 2024-05-17T22:26:40.177 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:39 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/4237201404' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-17T22:26:40.177 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:40 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:40.173+0000 7f4ca64ff200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-17T22:26:40.543 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:40 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:40.362+0000 7f4ca64ff200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-17T22:26:40.544 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:40 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:40.542+0000 7f4ca64ff200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-17T22:26:41.161 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:40 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:40.848+0000 7f4ca64ff200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-17T22:26:41.161 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:40 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:40.946+0000 7f4ca64ff200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-17T22:26:41.161 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:41 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:41.159+0000 7f4ca64ff200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-17T22:26:41.590 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:41 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:41.363+0000 7f4ca64ff200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-17T22:26:42.090 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:41 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:41.997+0000 7f4ca64ff200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-17T22:26:42.590 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:42 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:42.231+0000 7f4ca64ff200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-17T22:26:43.090 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:42 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:42.681+0000 7f4ca64ff200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-17T22:26:43.091 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:42 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:42.784+0000 7f4ca64ff200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-17T22:26:43.553 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:43 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:43.174+0000 7f4ca64ff200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-17T22:26:43.553 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:43 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:43.551+0000 7f4ca64ff200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-17T22:26:43.840 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:43 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:43.650+0000 7f4ca64ff200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-17T22:26:44.340 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:43 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:43.908+0000 7f4ca64ff200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-17T22:26:44.341 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:44 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:44.051+0000 7f4ca64ff200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-17T22:26:44.341 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:44 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:44.153+0000 7f4ca64ff200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-17T22:26:44.841 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:44 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:44.494+0000 7f4ca64ff200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-17T22:26:44.841 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:44 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:44.593+0000 7f4ca64ff200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-17T22:26:45.519 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:45 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:45.229+0000 7f4ca64ff200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-17T22:26:45.519 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:45 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:26:45.464+0000 7f4ca64ff200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-17T22:26:45.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:45 smithi012 ceph-mon[31632]: Active manager daemon x restarted 2024-05-17T22:26:45.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:45 smithi012 ceph-mon[31632]: Activating manager daemon x 2024-05-17T22:26:45.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:45 smithi012 ceph-mon[31632]: osdmap e2: 0 total, 0 up, 0 in 2024-05-17T22:26:45.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:45 smithi012 ceph-mon[31632]: mgrmap e6: x(active, starting, since 0.00407926s) 2024-05-17T22:26:45.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:45 smithi012 ceph-mon[31632]: from='mgr.14120 172.21.15.12:0/3665817882' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T22:26:45.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:45 smithi012 ceph-mon[31632]: from='mgr.14120 172.21.15.12:0/3665817882' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-17T22:26:45.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:45 smithi012 ceph-mon[31632]: from='mgr.14120 172.21.15.12:0/3665817882' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-17T22:26:45.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:45 smithi012 ceph-mon[31632]: from='mgr.14120 172.21.15.12:0/3665817882' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-17T22:26:45.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:45 smithi012 ceph-mon[31632]: from='mgr.14120 172.21.15.12:0/3665817882' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-17T22:26:45.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:45 smithi012 ceph-mon[31632]: Manager daemon x is now available 2024-05-17T22:26:45.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:45 smithi012 ceph-mon[31632]: from='mgr.14120 172.21.15.12:0/3665817882' entity='mgr.x' 2024-05-17T22:26:45.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:45 smithi012 ceph-mon[31632]: from='mgr.14120 172.21.15.12:0/3665817882' entity='mgr.x' 2024-05-17T22:26:45.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:45 smithi012 ceph-mon[31632]: from='mgr.14120 172.21.15.12:0/3665817882' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:26:45.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:45 smithi012 ceph-mon[31632]: from='mgr.14120 172.21.15.12:0/3665817882' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:26:46.840 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:46 smithi012 ceph-mon[31632]: Found migration_current of "None". Setting to last migration. 2024-05-17T22:26:46.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:46 smithi012 ceph-mon[31632]: from='mgr.14120 172.21.15.12:0/3665817882' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-17T22:26:46.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:46 smithi012 ceph-mon[31632]: from='mgr.14120 172.21.15.12:0/3665817882' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-17T22:26:46.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:46 smithi012 ceph-mon[31632]: from='mgr.14120 172.21.15.12:0/3665817882' entity='mgr.x' 2024-05-17T22:26:46.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:46 smithi012 ceph-mon[31632]: from='mgr.14120 172.21.15.12:0/3665817882' entity='mgr.x' 2024-05-17T22:26:46.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:46 smithi012 ceph-mon[31632]: mgrmap e7: x(active, since 1.0067s) 2024-05-17T22:26:47.242 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-05-17T22:26:47.242 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-17T22:26:47.242 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-17T22:26:47.242 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-05-17T22:26:47.242 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:40.384+0000 7fa5b60c1700 1 Processor -- start 2024-05-17T22:26:47.242 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:40.384+0000 7fa5b60c1700 1 -- start start 2024-05-17T22:26:47.242 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:40.384+0000 7fa5b60c1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5b0109470 0x7fa5b0109880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:47.243 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:40.384+0000 7fa5b60c1700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b0109e50 con 0x7fa5b0109470 2024-05-17T22:26:47.243 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:40.384+0000 7fa5af7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5b0109470 0x7fa5b0109880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:26:47.243 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:40.385+0000 7fa5af7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5b0109470 0x7fa5b0109880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33448/0 (socket says 172.21.15.12:33448) 2024-05-17T22:26:47.243 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:40.385+0000 7fa5af7fe700 1 -- 172.21.15.12:0/1660337436 learned_addr learned my addr 172.21.15.12:0/1660337436 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:26:47.243 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:40.385+0000 7fa5af7fe700 1 -- 172.21.15.12:0/1660337436 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5b010a5f0 con 0x7fa5b0109470 2024-05-17T22:26:47.243 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:40.385+0000 7fa5af7fe700 1 --2- 172.21.15.12:0/1660337436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5b0109470 0x7fa5b0109880 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa5a40098a0 tx=0x7fa5a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e06d21acb7cab9b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:26:47.243 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:40.385+0000 7fa5ae7fc700 1 -- 172.21.15.12:0/1660337436 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5a4004160 con 0x7fa5b0109470 2024-05-17T22:26:47.243 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:40.385+0000 7fa5ae7fc700 1 -- 172.21.15.12:0/1660337436 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa5a4004300 con 0x7fa5b0109470 2024-05-17T22:26:47.243 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:40.385+0000 7fa5ae7fc700 1 -- 172.21.15.12:0/1660337436 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5a4011af0 con 0x7fa5b0109470 2024-05-17T22:26:47.243 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:40.386+0000 7fa5b60c1700 1 -- 172.21.15.12:0/1660337436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5b0109470 msgr2=0x7fa5b0109880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:26:47.243 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:40.386+0000 7fa5b60c1700 1 --2- 172.21.15.12:0/1660337436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5b0109470 0x7fa5b0109880 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa5a40098a0 tx=0x7fa5a4009bb0 comp rx=0 tx=0).stop 2024-05-17T22:26:47.243 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:40.386+0000 7fa5b60c1700 1 -- 172.21.15.12:0/1660337436 shutdown_connections 2024-05-17T22:26:47.244 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:40.386+0000 7fa5b60c1700 1 --2- 172.21.15.12:0/1660337436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5b0109470 0x7fa5b0109880 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:47.244 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:40.386+0000 7fa5b60c1700 1 -- 172.21.15.12:0/1660337436 >> 172.21.15.12:0/1660337436 conn(0x7fa5b0104ce0 msgr2=0x7fa5b0107150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:26:47.244 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:40.386+0000 7fa5b60c1700 1 -- 172.21.15.12:0/1660337436 shutdown_connections 2024-05-17T22:26:47.244 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:40.386+0000 7fa5b60c1700 1 -- 172.21.15.12:0/1660337436 wait complete. 2024-05-17T22:26:47.244 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:40.387+0000 7fa5b60c1700 1 Processor -- start 2024-05-17T22:26:47.244 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:40.387+0000 7fa5b60c1700 1 -- start start 2024-05-17T22:26:47.244 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:40.387+0000 7fa5b60c1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5b0109470 0x7fa5b019d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:47.244 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:40.387+0000 7fa5b60c1700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b010d820 con 0x7fa5b0109470 2024-05-17T22:26:47.244 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:40.387+0000 7fa5af7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5b0109470 0x7fa5b019d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:26:47.244 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:40.387+0000 7fa5af7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5b0109470 0x7fa5b019d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33458/0 (socket says 172.21.15.12:33458) 2024-05-17T22:26:47.244 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:40.387+0000 7fa5af7fe700 1 -- 172.21.15.12:0/1653658441 learned_addr learned my addr 172.21.15.12:0/1653658441 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:26:47.244 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:40.388+0000 7fa5af7fe700 1 -- 172.21.15.12:0/1653658441 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5b019dc90 con 0x7fa5b0109470 2024-05-17T22:26:47.244 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:40.388+0000 7fa5af7fe700 1 --2- 172.21.15.12:0/1653658441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5b0109470 0x7fa5b019d750 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa5a40117b0 tx=0x7fa5a4011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:26:47.244 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:40.388+0000 7fa5acff9700 1 -- 172.21.15.12:0/1653658441 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5a4003af0 con 0x7fa5b0109470 2024-05-17T22:26:47.244 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:40.388+0000 7fa5acff9700 1 -- 172.21.15.12:0/1653658441 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa5a4020070 con 0x7fa5b0109470 2024-05-17T22:26:47.245 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:40.388+0000 7fa5b60c1700 1 -- 172.21.15.12:0/1653658441 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5b019df20 con 0x7fa5b0109470 2024-05-17T22:26:47.245 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:40.388+0000 7fa5acff9700 1 -- 172.21.15.12:0/1653658441 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5a4019630 con 0x7fa5b0109470 2024-05-17T22:26:47.245 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:40.388+0000 7fa5b60c1700 1 -- 172.21.15.12:0/1653658441 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5b01a0b30 con 0x7fa5b0109470 2024-05-17T22:26:47.245 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:40.389+0000 7fa5acff9700 1 -- 172.21.15.12:0/1653658441 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7fa5a40198f0 con 0x7fa5b0109470 2024-05-17T22:26:47.245 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:40.389+0000 7fa5acff9700 1 --2- 172.21.15.12:0/1653658441 >> [v2:172.21.15.12:6800/1039789215,v1:172.21.15.12:6801/1039789215] conn(0x7fa59803e530 0x7fa598040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:47.245 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:40.389+0000 7fa5acff9700 1 -- 172.21.15.12:0/1653658441 --> [v2:172.21.15.12:6800/1039789215,v1:172.21.15.12:6801/1039789215] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa598041150 con 0x7fa59803e530 2024-05-17T22:26:47.245 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:40.389+0000 7fa5aeffd700 1 -- 172.21.15.12:0/1653658441 >> [v2:172.21.15.12:6800/1039789215,v1:172.21.15.12:6801/1039789215] conn(0x7fa59803e530 msgr2=0x7fa598040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/1039789215 2024-05-17T22:26:47.245 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:40.389+0000 7fa5acff9700 1 -- 172.21.15.12:0/1653658441 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa5a4054a00 con 0x7fa5b0109470 2024-05-17T22:26:47.245 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:40.389+0000 7fa5aeffd700 1 --2- 172.21.15.12:0/1653658441 >> [v2:172.21.15.12:6800/1039789215,v1:172.21.15.12:6801/1039789215] conn(0x7fa59803e530 0x7fa598040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-17T22:26:47.245 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:40.590+0000 7fa5aeffd700 1 -- 172.21.15.12:0/1653658441 >> [v2:172.21.15.12:6800/1039789215,v1:172.21.15.12:6801/1039789215] conn(0x7fa59803e530 msgr2=0x7fa598040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/1039789215 2024-05-17T22:26:47.245 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:40.590+0000 7fa5aeffd700 1 --2- 172.21.15.12:0/1653658441 >> [v2:172.21.15.12:6800/1039789215,v1:172.21.15.12:6801/1039789215] conn(0x7fa59803e530 0x7fa598040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-17T22:26:47.245 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:40.991+0000 7fa5aeffd700 1 -- 172.21.15.12:0/1653658441 >> [v2:172.21.15.12:6800/1039789215,v1:172.21.15.12:6801/1039789215] conn(0x7fa59803e530 msgr2=0x7fa598040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/1039789215 2024-05-17T22:26:47.245 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:40.991+0000 7fa5aeffd700 1 --2- 172.21.15.12:0/1653658441 >> [v2:172.21.15.12:6800/1039789215,v1:172.21.15.12:6801/1039789215] conn(0x7fa59803e530 0x7fa598040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-17T22:26:47.245 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:41.792+0000 7fa5aeffd700 1 -- 172.21.15.12:0/1653658441 >> [v2:172.21.15.12:6800/1039789215,v1:172.21.15.12:6801/1039789215] conn(0x7fa59803e530 msgr2=0x7fa598040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/1039789215 2024-05-17T22:26:47.246 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:41.792+0000 7fa5aeffd700 1 --2- 172.21.15.12:0/1653658441 >> [v2:172.21.15.12:6800/1039789215,v1:172.21.15.12:6801/1039789215] conn(0x7fa59803e530 0x7fa598040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-17T22:26:47.246 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:43.393+0000 7fa5aeffd700 1 -- 172.21.15.12:0/1653658441 >> [v2:172.21.15.12:6800/1039789215,v1:172.21.15.12:6801/1039789215] conn(0x7fa59803e530 msgr2=0x7fa598040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/1039789215 2024-05-17T22:26:47.246 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:43.393+0000 7fa5aeffd700 1 --2- 172.21.15.12:0/1653658441 >> [v2:172.21.15.12:6800/1039789215,v1:172.21.15.12:6801/1039789215] conn(0x7fa59803e530 0x7fa598040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-17T22:26:47.246 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:45.468+0000 7fa5acff9700 1 -- 172.21.15.12:0/1653658441 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mgrmap(e 6) ==== 49989+0+0 (secure 0 0 0) 0x7fa5a400b4d0 con 0x7fa5b0109470 2024-05-17T22:26:47.246 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:45.468+0000 7fa5acff9700 1 -- 172.21.15.12:0/1653658441 >> [v2:172.21.15.12:6800/1039789215,v1:172.21.15.12:6801/1039789215] conn(0x7fa59803e530 msgr2=0x7fa598040a40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-17T22:26:47.246 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:45.468+0000 7fa5acff9700 1 --2- 172.21.15.12:0/1653658441 >> [v2:172.21.15.12:6800/1039789215,v1:172.21.15.12:6801/1039789215] conn(0x7fa59803e530 0x7fa598040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:47.246 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:46.469+0000 7fa5acff9700 1 -- 172.21.15.12:0/1653658441 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7fa5a401a9c0 con 0x7fa5b0109470 2024-05-17T22:26:47.249 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:46.470+0000 7fa5acff9700 1 --2- 172.21.15.12:0/1653658441 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7fa598041e70 0x7fa598044290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:47.249 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:46.470+0000 7fa5acff9700 1 -- 172.21.15.12:0/1653658441 --> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa598041150 con 0x7fa598041e70 2024-05-17T22:26:47.249 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:46.470+0000 7fa5aeffd700 1 --2- 172.21.15.12:0/1653658441 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7fa598041e70 0x7fa598044290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:26:47.249 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:46.471+0000 7fa5aeffd700 1 --2- 172.21.15.12:0/1653658441 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7fa598041e70 0x7fa598044290 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0003a10 tx=0x7fa5a00092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:26:47.249 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:46.471+0000 7fa5acff9700 1 -- 172.21.15.12:0/1653658441 <== mgr.14120 v2:172.21.15.12:6800/2250046521 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fa598041150 con 0x7fa598041e70 2024-05-17T22:26:47.249 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:46.479+0000 7fa5b60c1700 1 -- 172.21.15.12:0/1653658441 --> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fa5b0109900 con 0x7fa598041e70 2024-05-17T22:26:47.250 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:46.480+0000 7fa5acff9700 1 -- 172.21.15.12:0/1653658441 <== mgr.14120 v2:172.21.15.12:6800/2250046521 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fa5b0109900 con 0x7fa598041e70 2024-05-17T22:26:47.250 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:46.480+0000 7fa5b60c1700 1 -- 172.21.15.12:0/1653658441 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7fa598041e70 msgr2=0x7fa598044290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:26:47.250 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:46.480+0000 7fa5b60c1700 1 --2- 172.21.15.12:0/1653658441 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7fa598041e70 0x7fa598044290 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0003a10 tx=0x7fa5a00092b0 comp rx=0 tx=0).stop 2024-05-17T22:26:47.250 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:46.480+0000 7fa5b60c1700 1 -- 172.21.15.12:0/1653658441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5b0109470 msgr2=0x7fa5b019d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:26:47.250 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:46.480+0000 7fa5b60c1700 1 --2- 172.21.15.12:0/1653658441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5b0109470 0x7fa5b019d750 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa5a40117b0 tx=0x7fa5a4011b70 comp rx=0 tx=0).stop 2024-05-17T22:26:47.250 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:46.480+0000 7fa5b60c1700 1 -- 172.21.15.12:0/1653658441 shutdown_connections 2024-05-17T22:26:47.250 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:46.480+0000 7fa5b60c1700 1 --2- 172.21.15.12:0/1653658441 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7fa598041e70 0x7fa598044290 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:47.250 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:46.480+0000 7fa5b60c1700 1 --2- 172.21.15.12:0/1653658441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5b0109470 0x7fa5b019d750 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:47.250 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:46.480+0000 7fa5b60c1700 1 --2- 172.21.15.12:0/1653658441 >> [v2:172.21.15.12:6800/1039789215,v1:172.21.15.12:6801/1039789215] conn(0x7fa59803e530 0x7fa598040a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:47.250 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:46.480+0000 7fa5b60c1700 1 -- 172.21.15.12:0/1653658441 >> 172.21.15.12:0/1653658441 conn(0x7fa5b0104ce0 msgr2=0x7fa5b0105590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:26:47.250 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:46.480+0000 7fa5b60c1700 1 -- 172.21.15.12:0/1653658441 shutdown_connections 2024-05-17T22:26:47.250 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:46.480+0000 7fa5b60c1700 1 -- 172.21.15.12:0/1653658441 wait complete. 2024-05-17T22:26:47.250 INFO:teuthology.orchestra.run.smithi012.stdout:mgr epoch 5 is available 2024-05-17T22:26:47.251 INFO:teuthology.orchestra.run.smithi012.stdout:Setting orchestrator backend to cephadm... 2024-05-17T22:26:47.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:47 smithi012 ceph-mon[31632]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-17T22:26:47.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:47 smithi012 ceph-mon[31632]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-17T22:26:47.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:47 smithi012 ceph-mon[31632]: [17/May/2024:22:26:46] ENGINE Bus STARTING 2024-05-17T22:26:47.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:47 smithi012 ceph-mon[31632]: [17/May/2024:22:26:46] ENGINE Serving on http://172.21.15.12:8765 2024-05-17T22:26:47.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:47 smithi012 ceph-mon[31632]: from='mgr.14120 172.21.15.12:0/3665817882' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:26:48.523 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.693+0000 7effd77fc700 1 Processor -- start 2024-05-17T22:26:48.523 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.693+0000 7effd77fc700 1 -- start start 2024-05-17T22:26:48.523 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.694+0000 7effd77fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effd0109e50 0x7effd010c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:48.523 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.694+0000 7effd77fc700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7effd007ce10 con 0x7effd0109e50 2024-05-17T22:26:48.523 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.694+0000 7effd5598700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effd0109e50 0x7effd010c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:26:48.523 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.694+0000 7effd5598700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effd0109e50 0x7effd010c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33526/0 (socket says 172.21.15.12:33526) 2024-05-17T22:26:48.523 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.694+0000 7effd5598700 1 -- 172.21.15.12:0/4280521728 learned_addr learned my addr 172.21.15.12:0/4280521728 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:26:48.523 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.694+0000 7effd5598700 1 -- 172.21.15.12:0/4280521728 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd010c7b0 con 0x7effd0109e50 2024-05-17T22:26:48.523 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.694+0000 7effd5598700 1 --2- 172.21.15.12:0/4280521728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effd0109e50 0x7effd010c270 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7effc0009e80 tx=0x7effc000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca89e0a299d176b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:26:48.523 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.694+0000 7effcffff700 1 -- 172.21.15.12:0/4280521728 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effc0017070 con 0x7effd0109e50 2024-05-17T22:26:48.524 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.695+0000 7effcffff700 1 -- 172.21.15.12:0/4280521728 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7effc000b8c0 con 0x7effd0109e50 2024-05-17T22:26:48.524 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.695+0000 7effcffff700 1 -- 172.21.15.12:0/4280521728 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effc0004490 con 0x7effd0109e50 2024-05-17T22:26:48.524 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.695+0000 7effd77fc700 1 -- 172.21.15.12:0/4280521728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effd0109e50 msgr2=0x7effd010c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:26:48.524 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.695+0000 7effd77fc700 1 --2- 172.21.15.12:0/4280521728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effd0109e50 0x7effd010c270 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7effc0009e80 tx=0x7effc000b330 comp rx=0 tx=0).stop 2024-05-17T22:26:48.524 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.695+0000 7effd77fc700 1 -- 172.21.15.12:0/4280521728 shutdown_connections 2024-05-17T22:26:48.524 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.695+0000 7effd77fc700 1 --2- 172.21.15.12:0/4280521728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effd0109e50 0x7effd010c270 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:48.524 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.695+0000 7effd77fc700 1 -- 172.21.15.12:0/4280521728 >> 172.21.15.12:0/4280521728 conn(0x7effd0103a80 msgr2=0x7effd0105f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:26:48.524 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.695+0000 7effd77fc700 1 -- 172.21.15.12:0/4280521728 shutdown_connections 2024-05-17T22:26:48.524 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.695+0000 7effd77fc700 1 -- 172.21.15.12:0/4280521728 wait complete. 2024-05-17T22:26:48.524 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.696+0000 7effd77fc700 1 Processor -- start 2024-05-17T22:26:48.524 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.696+0000 7effd77fc700 1 -- start start 2024-05-17T22:26:48.524 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.696+0000 7effd77fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effd0109e50 0x7effd019d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:48.524 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.696+0000 7effd77fc700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7effd010d770 con 0x7effd0109e50 2024-05-17T22:26:48.524 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.697+0000 7effd5598700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effd0109e50 0x7effd019d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:26:48.525 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.697+0000 7effd5598700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effd0109e50 0x7effd019d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33542/0 (socket says 172.21.15.12:33542) 2024-05-17T22:26:48.525 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.697+0000 7effd5598700 1 -- 172.21.15.12:0/770077389 learned_addr learned my addr 172.21.15.12:0/770077389 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:26:48.525 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.697+0000 7effd5598700 1 -- 172.21.15.12:0/770077389 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd019dc40 con 0x7effd0109e50 2024-05-17T22:26:48.525 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.697+0000 7effd5598700 1 --2- 172.21.15.12:0/770077389 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effd0109e50 0x7effd019d700 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7effc0004220 tx=0x7effc000bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:26:48.525 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.697+0000 7effce7fc700 1 -- 172.21.15.12:0/770077389 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effc0020020 con 0x7effd0109e50 2024-05-17T22:26:48.525 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.697+0000 7effce7fc700 1 -- 172.21.15.12:0/770077389 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7effc0012430 con 0x7effd0109e50 2024-05-17T22:26:48.525 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.697+0000 7effd77fc700 1 -- 172.21.15.12:0/770077389 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effd019ded0 con 0x7effd0109e50 2024-05-17T22:26:48.525 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.697+0000 7effce7fc700 1 -- 172.21.15.12:0/770077389 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effc0017040 con 0x7effd0109e50 2024-05-17T22:26:48.525 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.697+0000 7effd77fc700 1 -- 172.21.15.12:0/770077389 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effd01a0ae0 con 0x7effd0109e50 2024-05-17T22:26:48.525 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.698+0000 7effd77fc700 1 -- 172.21.15.12:0/770077389 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effd010a2e0 con 0x7effd0109e50 2024-05-17T22:26:48.525 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.698+0000 7effce7fc700 1 -- 172.21.15.12:0/770077389 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7effc0023020 con 0x7effd0109e50 2024-05-17T22:26:48.525 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.699+0000 7effce7fc700 1 --2- 172.21.15.12:0/770077389 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7effb803e4e0 0x7effb80409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:48.525 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.699+0000 7effce7fc700 1 -- 172.21.15.12:0/770077389 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7effc000f050 con 0x7effd0109e50 2024-05-17T22:26:48.525 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.699+0000 7effd4d97700 1 --2- 172.21.15.12:0/770077389 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7effb803e4e0 0x7effb80409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:26:48.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.699+0000 7effd4d97700 1 --2- 172.21.15.12:0/770077389 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7effb803e4e0 0x7effb80409f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7effc4006fd0 tx=0x7effc4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:26:48.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.704+0000 7effce7fc700 1 -- 172.21.15.12:0/770077389 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7effc002a020 con 0x7effd0109e50 2024-05-17T22:26:48.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.938+0000 7effd77fc700 1 -- 172.21.15.12:0/770077389 --> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7effd0104ab0 con 0x7effb803e4e0 2024-05-17T22:26:48.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.943+0000 7effce7fc700 1 -- 172.21.15.12:0/770077389 <== mgr.14120 v2:172.21.15.12:6800/2250046521 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7effd0104ab0 con 0x7effb803e4e0 2024-05-17T22:26:48.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.946+0000 7effd77fc700 1 -- 172.21.15.12:0/770077389 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7effb803e4e0 msgr2=0x7effb80409f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:26:48.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.946+0000 7effd77fc700 1 --2- 172.21.15.12:0/770077389 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7effb803e4e0 0x7effb80409f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7effc4006fd0 tx=0x7effc4006e40 comp rx=0 tx=0).stop 2024-05-17T22:26:48.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.946+0000 7effd77fc700 1 -- 172.21.15.12:0/770077389 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effd0109e50 msgr2=0x7effd019d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:26:48.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.946+0000 7effd77fc700 1 --2- 172.21.15.12:0/770077389 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effd0109e50 0x7effd019d700 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7effc0004220 tx=0x7effc000bd50 comp rx=0 tx=0).stop 2024-05-17T22:26:48.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.946+0000 7effd77fc700 1 -- 172.21.15.12:0/770077389 shutdown_connections 2024-05-17T22:26:48.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.946+0000 7effd77fc700 1 --2- 172.21.15.12:0/770077389 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7effb803e4e0 0x7effb80409f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:48.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.946+0000 7effd77fc700 1 --2- 172.21.15.12:0/770077389 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effd0109e50 0x7effd019d700 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:48.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.946+0000 7effd77fc700 1 -- 172.21.15.12:0/770077389 >> 172.21.15.12:0/770077389 conn(0x7effd0103a80 msgr2=0x7effd01043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:26:48.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.946+0000 7effd77fc700 1 -- 172.21.15.12:0/770077389 shutdown_connections 2024-05-17T22:26:48.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:47.946+0000 7effd77fc700 1 -- 172.21.15.12:0/770077389 wait complete. 2024-05-17T22:26:48.801 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:48 smithi012 ceph-mon[31632]: [17/May/2024:22:26:47] ENGINE Serving on https://172.21.15.12:7150 2024-05-17T22:26:48.801 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:48 smithi012 ceph-mon[31632]: [17/May/2024:22:26:47] ENGINE Bus STARTED 2024-05-17T22:26:48.801 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:48 smithi012 ceph-mon[31632]: mgrmap e8: x(active, since 2s) 2024-05-17T22:26:48.801 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:48 smithi012 ceph-mon[31632]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:26:48.801 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:48 smithi012 ceph-mon[31632]: from='mgr.14120 172.21.15.12:0/3665817882' entity='mgr.x' 2024-05-17T22:26:48.801 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:48 smithi012 ceph-mon[31632]: from='mgr.14120 172.21.15.12:0/3665817882' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:26:49.707 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-17T22:26:49.707 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:48.925+0000 7f1ebf5c8700 1 Processor -- start 2024-05-17T22:26:49.707 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:48.925+0000 7f1ebf5c8700 1 -- start start 2024-05-17T22:26:49.708 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:48.925+0000 7f1ebf5c8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1eb8109e80 0x7f1eb810c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:49.708 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:48.925+0000 7f1ebf5c8700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb807d7d0 con 0x7f1eb8109e80 2024-05-17T22:26:49.708 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:48.925+0000 7f1ebd364700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1eb8109e80 0x7f1eb810c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:26:49.708 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:48.925+0000 7f1ebd364700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1eb8109e80 0x7f1eb810c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33556/0 (socket says 172.21.15.12:33556) 2024-05-17T22:26:49.708 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:48.925+0000 7f1ebd364700 1 -- 172.21.15.12:0/2718781103 learned_addr learned my addr 172.21.15.12:0/2718781103 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:26:49.708 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:48.926+0000 7f1ebd364700 1 -- 172.21.15.12:0/2718781103 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb810c7e0 con 0x7f1eb8109e80 2024-05-17T22:26:49.708 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:48.926+0000 7f1ebd364700 1 --2- 172.21.15.12:0/2718781103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1eb8109e80 0x7f1eb810c2a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1eb40098a0 tx=0x7f1eb4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5640c2c6129b5bd server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:26:49.708 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:48.926+0000 7f1eaffff700 1 -- 172.21.15.12:0/2718781103 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1eb4004160 con 0x7f1eb8109e80 2024-05-17T22:26:49.708 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:48.926+0000 7f1eaffff700 1 -- 172.21.15.12:0/2718781103 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1eb4004300 con 0x7f1eb8109e80 2024-05-17T22:26:49.709 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:48.926+0000 7f1eaffff700 1 -- 172.21.15.12:0/2718781103 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1eb4011af0 con 0x7f1eb8109e80 2024-05-17T22:26:49.709 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:48.927+0000 7f1ebf5c8700 1 -- 172.21.15.12:0/2718781103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1eb8109e80 msgr2=0x7f1eb810c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:26:49.709 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:48.927+0000 7f1ebf5c8700 1 --2- 172.21.15.12:0/2718781103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1eb8109e80 0x7f1eb810c2a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1eb40098a0 tx=0x7f1eb4009bb0 comp rx=0 tx=0).stop 2024-05-17T22:26:49.709 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:48.927+0000 7f1ebf5c8700 1 -- 172.21.15.12:0/2718781103 shutdown_connections 2024-05-17T22:26:49.709 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:48.927+0000 7f1ebf5c8700 1 --2- 172.21.15.12:0/2718781103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1eb8109e80 0x7f1eb810c2a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:49.709 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:48.927+0000 7f1ebf5c8700 1 -- 172.21.15.12:0/2718781103 >> 172.21.15.12:0/2718781103 conn(0x7f1eb8103ab0 msgr2=0x7f1eb8105f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:26:49.709 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:48.927+0000 7f1ebf5c8700 1 -- 172.21.15.12:0/2718781103 shutdown_connections 2024-05-17T22:26:49.709 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:48.927+0000 7f1ebf5c8700 1 -- 172.21.15.12:0/2718781103 wait complete. 2024-05-17T22:26:49.709 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:48.928+0000 7f1ebf5c8700 1 Processor -- start 2024-05-17T22:26:49.709 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:48.928+0000 7f1ebf5c8700 1 -- start start 2024-05-17T22:26:49.709 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:48.928+0000 7f1ebf5c8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1eb8109e80 0x7f1eb819d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:49.709 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:48.928+0000 7f1ebf5c8700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb810d7a0 con 0x7f1eb8109e80 2024-05-17T22:26:49.710 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:48.928+0000 7f1ebd364700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1eb8109e80 0x7f1eb819d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:26:49.710 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:48.928+0000 7f1ebd364700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1eb8109e80 0x7f1eb819d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33560/0 (socket says 172.21.15.12:33560) 2024-05-17T22:26:49.710 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:48.928+0000 7f1ebd364700 1 -- 172.21.15.12:0/44765519 learned_addr learned my addr 172.21.15.12:0/44765519 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:26:49.710 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:48.928+0000 7f1ebd364700 1 -- 172.21.15.12:0/44765519 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb819dcb0 con 0x7f1eb8109e80 2024-05-17T22:26:49.710 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:48.929+0000 7f1ebd364700 1 --2- 172.21.15.12:0/44765519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1eb8109e80 0x7f1eb819d770 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4009fb0 tx=0x7f1eb4011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:26:49.710 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:48.929+0000 7f1eae7fc700 1 -- 172.21.15.12:0/44765519 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1eb40036a0 con 0x7f1eb8109e80 2024-05-17T22:26:49.710 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:48.929+0000 7f1eae7fc700 1 -- 172.21.15.12:0/44765519 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1eb401a430 con 0x7f1eb8109e80 2024-05-17T22:26:49.710 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:48.929+0000 7f1eae7fc700 1 -- 172.21.15.12:0/44765519 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1eb4019550 con 0x7f1eb8109e80 2024-05-17T22:26:49.710 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:48.929+0000 7f1ebf5c8700 1 -- 172.21.15.12:0/44765519 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eb819df40 con 0x7f1eb8109e80 2024-05-17T22:26:49.710 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:48.929+0000 7f1ebf5c8700 1 -- 172.21.15.12:0/44765519 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eb81a0b50 con 0x7f1eb8109e80 2024-05-17T22:26:49.710 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:48.930+0000 7f1eae7fc700 1 -- 172.21.15.12:0/44765519 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f1eb4019850 con 0x7f1eb8109e80 2024-05-17T22:26:49.710 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:48.930+0000 7f1ebf5c8700 1 -- 172.21.15.12:0/44765519 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1eb810a310 con 0x7f1eb8109e80 2024-05-17T22:26:49.711 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:48.930+0000 7f1eae7fc700 1 --2- 172.21.15.12:0/44765519 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f1ea403e4e0 0x7f1ea40409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:49.711 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:48.930+0000 7f1eae7fc700 1 -- 172.21.15.12:0/44765519 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1eb4053860 con 0x7f1eb8109e80 2024-05-17T22:26:49.711 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:48.930+0000 7f1ebcb63700 1 --2- 172.21.15.12:0/44765519 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f1ea403e4e0 0x7f1ea40409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:26:49.711 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:48.931+0000 7f1ebcb63700 1 --2- 172.21.15.12:0/44765519 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f1ea403e4e0 0x7f1ea40409f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8006fd0 tx=0x7f1ea8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:26:49.711 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:48.935+0000 7f1eae7fc700 1 -- 172.21.15.12:0/44765519 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1eb4017e90 con 0x7f1eb8109e80 2024-05-17T22:26:49.711 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:49.153+0000 7f1ebf5c8700 1 -- 172.21.15.12:0/44765519 --> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f1eb8104b00 con 0x7f1ea403e4e0 2024-05-17T22:26:49.711 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:49.153+0000 7f1eae7fc700 1 -- 172.21.15.12:0/44765519 <== mgr.14120 v2:172.21.15.12:6800/2250046521 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f1eb8104b00 con 0x7f1ea403e4e0 2024-05-17T22:26:49.711 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:49.155+0000 7f1ebf5c8700 1 -- 172.21.15.12:0/44765519 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f1ea403e4e0 msgr2=0x7f1ea40409f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:26:49.711 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:49.155+0000 7f1ebf5c8700 1 --2- 172.21.15.12:0/44765519 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f1ea403e4e0 0x7f1ea40409f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8006fd0 tx=0x7f1ea8006e40 comp rx=0 tx=0).stop 2024-05-17T22:26:49.711 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:49.155+0000 7f1ebf5c8700 1 -- 172.21.15.12:0/44765519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1eb8109e80 msgr2=0x7f1eb819d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:26:49.711 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:49.155+0000 7f1ebf5c8700 1 --2- 172.21.15.12:0/44765519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1eb8109e80 0x7f1eb819d770 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4009fb0 tx=0x7f1eb4011940 comp rx=0 tx=0).stop 2024-05-17T22:26:49.711 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:49.155+0000 7f1ebf5c8700 1 -- 172.21.15.12:0/44765519 shutdown_connections 2024-05-17T22:26:49.712 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:49.156+0000 7f1ebf5c8700 1 --2- 172.21.15.12:0/44765519 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f1ea403e4e0 0x7f1ea40409f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:49.712 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:49.156+0000 7f1ebf5c8700 1 --2- 172.21.15.12:0/44765519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1eb8109e80 0x7f1eb819d770 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:49.712 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:49.156+0000 7f1ebf5c8700 1 -- 172.21.15.12:0/44765519 >> 172.21.15.12:0/44765519 conn(0x7f1eb8103ab0 msgr2=0x7f1eb81043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:26:49.712 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:49.156+0000 7f1ebf5c8700 1 -- 172.21.15.12:0/44765519 shutdown_connections 2024-05-17T22:26:49.712 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:49.156+0000 7f1ebf5c8700 1 -- 172.21.15.12:0/44765519 wait complete. 2024-05-17T22:26:49.712 INFO:teuthology.orchestra.run.smithi012.stdout:Generating ssh key... 2024-05-17T22:26:50.340 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:50 smithi012 ceph-mon[31632]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:26:51.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:51 smithi012 ceph-mon[31632]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:26:51.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:51 smithi012 ceph-mon[31632]: Generating ssh key... 2024-05-17T22:26:51.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:51 smithi012 ceph-mon[31632]: from='mgr.14120 172.21.15.12:0/3665817882' entity='mgr.x' 2024-05-17T22:26:51.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:51 smithi012 ceph-mon[31632]: from='mgr.14120 172.21.15.12:0/3665817882' entity='mgr.x' 2024-05-17T22:26:51.342 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:51 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: Generating public/private rsa key pair. 2024-05-17T22:26:51.342 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:51 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: Your identification has been saved in /tmp/tmpmgy6gmhz/key. 2024-05-17T22:26:51.342 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:51 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: Your public key has been saved in /tmp/tmpmgy6gmhz/key.pub. 2024-05-17T22:26:51.342 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:51 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: The key fingerprint is: 2024-05-17T22:26:51.342 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:51 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: SHA256:gyIjt2ojzz7AH4D7PXDAFTxtBvkubJHWVM+QNAbfJWI ceph-4d593780-149c-11ef-bc9a-c7b262605968 2024-05-17T22:26:51.342 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:51 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: The key's randomart image is: 2024-05-17T22:26:51.343 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:51 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: +---[RSA 3072]----+ 2024-05-17T22:26:51.343 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:51 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: | .o=.+E.. . | 2024-05-17T22:26:51.343 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:51 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: | = =+.B o | 2024-05-17T22:26:51.343 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:51 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: |.. . O . + | 2024-05-17T22:26:51.343 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:51 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: |..o + o. | 2024-05-17T22:26:51.343 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:51 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: |o.=+.o. S | 2024-05-17T22:26:51.343 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:51 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: |o+.==.. . | 2024-05-17T22:26:51.343 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:51 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: | oo=.. | 2024-05-17T22:26:51.344 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:51 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: |.=o.o | 2024-05-17T22:26:51.344 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:51 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: |+o=. . | 2024-05-17T22:26:51.344 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:26:51 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: +----[SHA256]-----+ 2024-05-17T22:26:51.651 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:50.140+0000 7f794d131700 1 Processor -- start 2024-05-17T22:26:51.651 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:50.140+0000 7f794d131700 1 -- start start 2024-05-17T22:26:51.652 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:50.140+0000 7f794d131700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7948109430 0x7f7948109840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:51.652 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:50.140+0000 7f794d131700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7948109e10 con 0x7f7948109430 2024-05-17T22:26:51.652 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:50.141+0000 7f7946d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7948109430 0x7f7948109840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:26:51.652 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:50.141+0000 7f7946d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7948109430 0x7f7948109840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40092/0 (socket says 172.21.15.12:40092) 2024-05-17T22:26:51.652 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:50.141+0000 7f7946d9d700 1 -- 172.21.15.12:0/554452462 learned_addr learned my addr 172.21.15.12:0/554452462 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:26:51.652 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:50.141+0000 7f7946d9d700 1 -- 172.21.15.12:0/554452462 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f794810a560 con 0x7f7948109430 2024-05-17T22:26:51.652 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:50.141+0000 7f7946d9d700 1 --2- 172.21.15.12:0/554452462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7948109430 0x7f7948109840 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f79380098a0 tx=0x7f7938009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2fa16d9f5ff91ea server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:26:51.652 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:50.141+0000 7f7945d9b700 1 -- 172.21.15.12:0/554452462 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7938004160 con 0x7f7948109430 2024-05-17T22:26:51.652 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:50.141+0000 7f7945d9b700 1 -- 172.21.15.12:0/554452462 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7938004300 con 0x7f7948109430 2024-05-17T22:26:51.652 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:50.141+0000 7f7945d9b700 1 -- 172.21.15.12:0/554452462 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7938011af0 con 0x7f7948109430 2024-05-17T22:26:51.652 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:50.142+0000 7f794d131700 1 -- 172.21.15.12:0/554452462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7948109430 msgr2=0x7f7948109840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:26:51.652 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:50.142+0000 7f794d131700 1 --2- 172.21.15.12:0/554452462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7948109430 0x7f7948109840 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f79380098a0 tx=0x7f7938009bb0 comp rx=0 tx=0).stop 2024-05-17T22:26:51.652 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:50.142+0000 7f794d131700 1 -- 172.21.15.12:0/554452462 shutdown_connections 2024-05-17T22:26:51.652 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:50.142+0000 7f794d131700 1 --2- 172.21.15.12:0/554452462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7948109430 0x7f7948109840 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:51.652 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:50.142+0000 7f794d131700 1 -- 172.21.15.12:0/554452462 >> 172.21.15.12:0/554452462 conn(0x7f7948104c80 msgr2=0x7f7948107110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:26:51.653 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:50.142+0000 7f794d131700 1 -- 172.21.15.12:0/554452462 shutdown_connections 2024-05-17T22:26:51.653 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:50.142+0000 7f794d131700 1 -- 172.21.15.12:0/554452462 wait complete. 2024-05-17T22:26:51.653 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:50.143+0000 7f794d131700 1 Processor -- start 2024-05-17T22:26:51.653 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:50.143+0000 7f794d131700 1 -- start start 2024-05-17T22:26:51.653 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:50.143+0000 7f794d131700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7948109430 0x7f794819d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:51.653 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:50.143+0000 7f794d131700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f794810d910 con 0x7f7948109430 2024-05-17T22:26:51.653 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:50.143+0000 7f7946d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7948109430 0x7f794819d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:26:51.653 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:50.143+0000 7f7946d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7948109430 0x7f794819d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40104/0 (socket says 172.21.15.12:40104) 2024-05-17T22:26:51.653 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:50.143+0000 7f7946d9d700 1 -- 172.21.15.12:0/3463601018 learned_addr learned my addr 172.21.15.12:0/3463601018 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:26:51.653 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:50.144+0000 7f7946d9d700 1 -- 172.21.15.12:0/3463601018 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f794819dcf0 con 0x7f7948109430 2024-05-17T22:26:51.653 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:50.144+0000 7f7946d9d700 1 --2- 172.21.15.12:0/3463601018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7948109430 0x7f794819d7b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7938009fb0 tx=0x7f7938011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:26:51.653 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:50.144+0000 7f792ffff700 1 -- 172.21.15.12:0/3463601018 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7938011bc0 con 0x7f7948109430 2024-05-17T22:26:51.653 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:50.144+0000 7f792ffff700 1 -- 172.21.15.12:0/3463601018 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7938024440 con 0x7f7948109430 2024-05-17T22:26:51.653 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:50.144+0000 7f794d131700 1 -- 172.21.15.12:0/3463601018 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f794819df80 con 0x7f7948109430 2024-05-17T22:26:51.653 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:50.144+0000 7f792ffff700 1 -- 172.21.15.12:0/3463601018 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f793801a5c0 con 0x7f7948109430 2024-05-17T22:26:51.654 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:50.144+0000 7f794d131700 1 -- 172.21.15.12:0/3463601018 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79481a0b90 con 0x7f7948109430 2024-05-17T22:26:51.654 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:50.145+0000 7f794d131700 1 -- 172.21.15.12:0/3463601018 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7948109840 con 0x7f7948109430 2024-05-17T22:26:51.654 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:50.145+0000 7f792ffff700 1 -- 172.21.15.12:0/3463601018 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f793801a880 con 0x7f7948109430 2024-05-17T22:26:51.654 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:50.145+0000 7f792ffff700 1 --2- 172.21.15.12:0/3463601018 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f793003e4e0 0x7f79300409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:51.654 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:50.146+0000 7f792ffff700 1 -- 172.21.15.12:0/3463601018 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f79380537f0 con 0x7f7948109430 2024-05-17T22:26:51.654 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:50.146+0000 7f794659c700 1 --2- 172.21.15.12:0/3463601018 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f793003e4e0 0x7f79300409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:26:51.654 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:50.146+0000 7f794659c700 1 --2- 172.21.15.12:0/3463601018 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f793003e4e0 0x7f79300409f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f793c006fd0 tx=0x7f793c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:26:51.654 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:50.150+0000 7f792ffff700 1 -- 172.21.15.12:0/3463601018 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f793800b3e0 con 0x7f7948109430 2024-05-17T22:26:51.654 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:50.366+0000 7f794d131700 1 -- 172.21.15.12:0/3463601018 --> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f7948105c90 con 0x7f793003e4e0 2024-05-17T22:26:51.654 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:51.071+0000 7f792ffff700 1 -- 172.21.15.12:0/3463601018 <== mgr.14120 v2:172.21.15.12:6800/2250046521 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7948105c90 con 0x7f793003e4e0 2024-05-17T22:26:51.654 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:51.073+0000 7f794d131700 1 -- 172.21.15.12:0/3463601018 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f793003e4e0 msgr2=0x7f79300409f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:26:51.654 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:51.073+0000 7f794d131700 1 --2- 172.21.15.12:0/3463601018 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f793003e4e0 0x7f79300409f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f793c006fd0 tx=0x7f793c006e40 comp rx=0 tx=0).stop 2024-05-17T22:26:51.654 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:51.073+0000 7f794d131700 1 -- 172.21.15.12:0/3463601018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7948109430 msgr2=0x7f794819d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:26:51.654 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:51.073+0000 7f794d131700 1 --2- 172.21.15.12:0/3463601018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7948109430 0x7f794819d7b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7938009fb0 tx=0x7f7938011890 comp rx=0 tx=0).stop 2024-05-17T22:26:51.654 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:51.073+0000 7f794d131700 1 -- 172.21.15.12:0/3463601018 shutdown_connections 2024-05-17T22:26:51.655 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:51.073+0000 7f794d131700 1 --2- 172.21.15.12:0/3463601018 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f793003e4e0 0x7f79300409f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:51.655 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:51.073+0000 7f794d131700 1 --2- 172.21.15.12:0/3463601018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7948109430 0x7f794819d7b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:51.655 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:51.073+0000 7f794d131700 1 -- 172.21.15.12:0/3463601018 >> 172.21.15.12:0/3463601018 conn(0x7f7948104c80 msgr2=0x7f7948105580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:26:51.655 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:51.073+0000 7f794d131700 1 -- 172.21.15.12:0/3463601018 shutdown_connections 2024-05-17T22:26:51.655 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:51.073+0000 7f794d131700 1 -- 172.21.15.12:0/3463601018 wait complete. 2024-05-17T22:26:52.893 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNYHWTfx4yK8LyjN8rZ/nFK/ER8PwFD9w7EYfCCmK6f1w3vx3Cs7pRTxtsrZzrfcwvVOcAm4+C/1n2IfXF1bBKfrNksNhkKdP+TOXImvHrxOyMC7tF1hsT0ULp+K8FNHv577OtIyKsye/PMeMtuLArBUi7eMl5D8JtFHxrxQYMWozNLIZHe+tXEXtdnYkoJ1FNY15L7GIjoIDdcrO5HSApzyf2mFVsOeECJE/nHZpiNQ3AJBAFi4kA4wH3QBtFfcNeycC+3+lt3QuNX37aBBpRTxqupL4qUlEEABn6lLUniH7t9gTVtyXyGWVBrm9iYcLchwQt9vaEUGkTrnTEZIYpTYc5e1UYBmJmSI76LNaxcb4/9Sna+ji2lM61v92w07RwH7TpROFjb9kDfQSLH8QiXyo77ja7q6ieAts0HgXrXRTxKYsdA2J+2Xw1so3EjrOPz7hhIcOIAEYGWDqcy6ZOkqrR4ME0np3pDG76CKPl90WDwtBbiXg4H421P9SZiQ0= ceph-4d593780-149c-11ef-bc9a-c7b262605968 2024-05-17T22:26:52.893 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.083+0000 7fba6c65a700 1 Processor -- start 2024-05-17T22:26:52.893 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.083+0000 7fba6c65a700 1 -- start start 2024-05-17T22:26:52.893 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.084+0000 7fba6c65a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba64109330 0x7fba64109740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:52.893 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.084+0000 7fba6c65a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fba64109d50 con 0x7fba64109330 2024-05-17T22:26:52.893 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.084+0000 7fba6a3f6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba64109330 0x7fba64109740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:26:52.893 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.084+0000 7fba6a3f6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba64109330 0x7fba64109740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40120/0 (socket says 172.21.15.12:40120) 2024-05-17T22:26:52.894 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.084+0000 7fba6a3f6700 1 -- 172.21.15.12:0/4005416697 learned_addr learned my addr 172.21.15.12:0/4005416697 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:26:52.894 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.084+0000 7fba6a3f6700 1 -- 172.21.15.12:0/4005416697 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba6410a570 con 0x7fba64109330 2024-05-17T22:26:52.894 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.084+0000 7fba6a3f6700 1 --2- 172.21.15.12:0/4005416697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba64109330 0x7fba64109740 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fba600098a0 tx=0x7fba60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e18c26abec865f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:26:52.894 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.084+0000 7fba693f4700 1 -- 172.21.15.12:0/4005416697 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba60004160 con 0x7fba64109330 2024-05-17T22:26:52.894 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.084+0000 7fba693f4700 1 -- 172.21.15.12:0/4005416697 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fba60004300 con 0x7fba64109330 2024-05-17T22:26:52.894 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.085+0000 7fba693f4700 1 -- 172.21.15.12:0/4005416697 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba60011af0 con 0x7fba64109330 2024-05-17T22:26:52.894 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.085+0000 7fba6c65a700 1 -- 172.21.15.12:0/4005416697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba64109330 msgr2=0x7fba64109740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:26:52.894 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.085+0000 7fba6c65a700 1 --2- 172.21.15.12:0/4005416697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba64109330 0x7fba64109740 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fba600098a0 tx=0x7fba60009bb0 comp rx=0 tx=0).stop 2024-05-17T22:26:52.894 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.085+0000 7fba6c65a700 1 -- 172.21.15.12:0/4005416697 shutdown_connections 2024-05-17T22:26:52.894 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.085+0000 7fba6c65a700 1 --2- 172.21.15.12:0/4005416697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba64109330 0x7fba64109740 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:52.894 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.085+0000 7fba6c65a700 1 -- 172.21.15.12:0/4005416697 >> 172.21.15.12:0/4005416697 conn(0x7fba64104ca0 msgr2=0x7fba64107170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:26:52.894 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.085+0000 7fba6c65a700 1 -- 172.21.15.12:0/4005416697 shutdown_connections 2024-05-17T22:26:52.894 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.085+0000 7fba6c65a700 1 -- 172.21.15.12:0/4005416697 wait complete. 2024-05-17T22:26:52.895 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.086+0000 7fba6c65a700 1 Processor -- start 2024-05-17T22:26:52.895 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.086+0000 7fba6c65a700 1 -- start start 2024-05-17T22:26:52.895 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.086+0000 7fba6c65a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba64109330 0x7fba6419d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:52.895 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.086+0000 7fba6c65a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fba6410d830 con 0x7fba64109330 2024-05-17T22:26:52.895 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.086+0000 7fba6a3f6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba64109330 0x7fba6419d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:26:52.895 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.086+0000 7fba6a3f6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba64109330 0x7fba6419d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40132/0 (socket says 172.21.15.12:40132) 2024-05-17T22:26:52.895 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.086+0000 7fba6a3f6700 1 -- 172.21.15.12:0/1754941262 learned_addr learned my addr 172.21.15.12:0/1754941262 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:26:52.895 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.087+0000 7fba6a3f6700 1 -- 172.21.15.12:0/1754941262 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba6419dbf0 con 0x7fba64109330 2024-05-17T22:26:52.895 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.087+0000 7fba6a3f6700 1 --2- 172.21.15.12:0/1754941262 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba64109330 0x7fba6419d6b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fba60009fb0 tx=0x7fba60011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:26:52.895 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.087+0000 7fba5b7fe700 1 -- 172.21.15.12:0/1754941262 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba600036a0 con 0x7fba64109330 2024-05-17T22:26:52.895 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.087+0000 7fba5b7fe700 1 -- 172.21.15.12:0/1754941262 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fba60024440 con 0x7fba64109330 2024-05-17T22:26:52.895 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.087+0000 7fba6c65a700 1 -- 172.21.15.12:0/1754941262 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba6419de80 con 0x7fba64109330 2024-05-17T22:26:52.895 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.087+0000 7fba5b7fe700 1 -- 172.21.15.12:0/1754941262 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba6001a5c0 con 0x7fba64109330 2024-05-17T22:26:52.895 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.087+0000 7fba6c65a700 1 -- 172.21.15.12:0/1754941262 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba641a0a90 con 0x7fba64109330 2024-05-17T22:26:52.896 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.088+0000 7fba5b7fe700 1 -- 172.21.15.12:0/1754941262 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fba6001a830 con 0x7fba64109330 2024-05-17T22:26:52.896 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.088+0000 7fba6c65a700 1 -- 172.21.15.12:0/1754941262 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba641097c0 con 0x7fba64109330 2024-05-17T22:26:52.896 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.088+0000 7fba5b7fe700 1 --2- 172.21.15.12:0/1754941262 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7fba5003e530 0x7fba50040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:52.896 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.088+0000 7fba5b7fe700 1 -- 172.21.15.12:0/1754941262 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fba60053a70 con 0x7fba64109330 2024-05-17T22:26:52.896 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.089+0000 7fba69bf5700 1 --2- 172.21.15.12:0/1754941262 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7fba5003e530 0x7fba50040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:26:52.896 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.089+0000 7fba69bf5700 1 --2- 172.21.15.12:0/1754941262 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7fba5003e530 0x7fba50040a40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fba54006fd0 tx=0x7fba54006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:26:52.896 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.093+0000 7fba5b7fe700 1 -- 172.21.15.12:0/1754941262 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fba600533c0 con 0x7fba64109330 2024-05-17T22:26:52.896 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.311+0000 7fba6c65a700 1 -- 172.21.15.12:0/1754941262 --> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fba64105cf0 con 0x7fba5003e530 2024-05-17T22:26:52.896 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.312+0000 7fba5b7fe700 1 -- 172.21.15.12:0/1754941262 <== mgr.14120 v2:172.21.15.12:6800/2250046521 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fba64105cf0 con 0x7fba5003e530 2024-05-17T22:26:52.896 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.314+0000 7fba6c65a700 1 -- 172.21.15.12:0/1754941262 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7fba5003e530 msgr2=0x7fba50040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:26:52.896 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.314+0000 7fba6c65a700 1 --2- 172.21.15.12:0/1754941262 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7fba5003e530 0x7fba50040a40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fba54006fd0 tx=0x7fba54006e40 comp rx=0 tx=0).stop 2024-05-17T22:26:52.896 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.314+0000 7fba6c65a700 1 -- 172.21.15.12:0/1754941262 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba64109330 msgr2=0x7fba6419d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:26:52.896 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.314+0000 7fba6c65a700 1 --2- 172.21.15.12:0/1754941262 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba64109330 0x7fba6419d6b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fba60009fb0 tx=0x7fba60011890 comp rx=0 tx=0).stop 2024-05-17T22:26:52.896 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.314+0000 7fba6c65a700 1 -- 172.21.15.12:0/1754941262 shutdown_connections 2024-05-17T22:26:52.897 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.314+0000 7fba6c65a700 1 --2- 172.21.15.12:0/1754941262 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7fba5003e530 0x7fba50040a40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:52.897 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.314+0000 7fba6c65a700 1 --2- 172.21.15.12:0/1754941262 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba64109330 0x7fba6419d6b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:52.897 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.314+0000 7fba6c65a700 1 -- 172.21.15.12:0/1754941262 >> 172.21.15.12:0/1754941262 conn(0x7fba64104ca0 msgr2=0x7fba641055e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:26:52.897 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.314+0000 7fba6c65a700 1 -- 172.21.15.12:0/1754941262 shutdown_connections 2024-05-17T22:26:52.897 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:52.314+0000 7fba6c65a700 1 -- 172.21.15.12:0/1754941262 wait complete. 2024-05-17T22:26:52.897 INFO:teuthology.orchestra.run.smithi012.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-17T22:26:52.897 INFO:teuthology.orchestra.run.smithi012.stdout:Adding key to root@localhost authorized_keys... 2024-05-17T22:26:52.897 INFO:teuthology.orchestra.run.smithi012.stdout:Adding host smithi012... 2024-05-17T22:26:53.163 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:53 smithi012 ceph-mon[31632]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:26:54.161 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:54 smithi012 ceph-mon[31632]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi012", "addr": "172.21.15.12", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:26:55.340 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:55 smithi012 ceph-mon[31632]: Deploying cephadm binary to smithi012 2024-05-17T22:26:58.468 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout Added host 'smithi012' with addr '172.21.15.12' 2024-05-17T22:26:58.468 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:53.284+0000 7f19dcedd700 1 Processor -- start 2024-05-17T22:26:58.468 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:53.284+0000 7f19dcedd700 1 -- start start 2024-05-17T22:26:58.468 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:53.285+0000 7f19dcedd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19d8109f00 0x7f19d810c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:58.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:53.285+0000 7f19dcedd700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f19d807d750 con 0x7f19d8109f00 2024-05-17T22:26:58.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:53.285+0000 7f19d659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19d8109f00 0x7f19d810c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:26:58.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:53.285+0000 7f19d659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19d8109f00 0x7f19d810c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40140/0 (socket says 172.21.15.12:40140) 2024-05-17T22:26:58.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:53.285+0000 7f19d659c700 1 -- 172.21.15.12:0/1071798777 learned_addr learned my addr 172.21.15.12:0/1071798777 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:26:58.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:53.285+0000 7f19d659c700 1 -- 172.21.15.12:0/1071798777 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19d810c860 con 0x7f19d8109f00 2024-05-17T22:26:58.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:53.285+0000 7f19d659c700 1 --2- 172.21.15.12:0/1071798777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19d8109f00 0x7f19d810c320 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f19c80098a0 tx=0x7f19c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b61f31029b2226c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:26:58.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:53.285+0000 7f19d559a700 1 -- 172.21.15.12:0/1071798777 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19c8004160 con 0x7f19d8109f00 2024-05-17T22:26:58.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:53.285+0000 7f19d559a700 1 -- 172.21.15.12:0/1071798777 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f19c8004300 con 0x7f19d8109f00 2024-05-17T22:26:58.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:53.285+0000 7f19d559a700 1 -- 172.21.15.12:0/1071798777 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19c8011af0 con 0x7f19d8109f00 2024-05-17T22:26:58.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:53.286+0000 7f19dcedd700 1 -- 172.21.15.12:0/1071798777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19d8109f00 msgr2=0x7f19d810c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:26:58.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:53.286+0000 7f19dcedd700 1 --2- 172.21.15.12:0/1071798777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19d8109f00 0x7f19d810c320 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f19c80098a0 tx=0x7f19c8009bb0 comp rx=0 tx=0).stop 2024-05-17T22:26:58.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:53.286+0000 7f19dcedd700 1 -- 172.21.15.12:0/1071798777 shutdown_connections 2024-05-17T22:26:58.469 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:53.286+0000 7f19dcedd700 1 --2- 172.21.15.12:0/1071798777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19d8109f00 0x7f19d810c320 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:58.470 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:53.286+0000 7f19dcedd700 1 -- 172.21.15.12:0/1071798777 >> 172.21.15.12:0/1071798777 conn(0x7f19d8103ab0 msgr2=0x7f19d8105f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:26:58.470 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:53.286+0000 7f19dcedd700 1 -- 172.21.15.12:0/1071798777 shutdown_connections 2024-05-17T22:26:58.470 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:53.286+0000 7f19dcedd700 1 -- 172.21.15.12:0/1071798777 wait complete. 2024-05-17T22:26:58.470 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:53.287+0000 7f19dcedd700 1 Processor -- start 2024-05-17T22:26:58.470 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:53.287+0000 7f19dcedd700 1 -- start start 2024-05-17T22:26:58.470 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:53.287+0000 7f19dcedd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19d8109f00 0x7f19d8109680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:58.470 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:53.287+0000 7f19dcedd700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f19d810d760 con 0x7f19d8109f00 2024-05-17T22:26:58.470 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:53.288+0000 7f19d659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19d8109f00 0x7f19d8109680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:26:58.470 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:53.288+0000 7f19d659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19d8109f00 0x7f19d8109680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40142/0 (socket says 172.21.15.12:40142) 2024-05-17T22:26:58.470 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:53.288+0000 7f19d659c700 1 -- 172.21.15.12:0/1529259456 learned_addr learned my addr 172.21.15.12:0/1529259456 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:26:58.471 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:53.288+0000 7f19d659c700 1 -- 172.21.15.12:0/1529259456 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19d8109bc0 con 0x7f19d8109f00 2024-05-17T22:26:58.471 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:53.288+0000 7f19d659c700 1 --2- 172.21.15.12:0/1529259456 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19d8109f00 0x7f19d8109680 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f19c8009fb0 tx=0x7f19c8011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:26:58.471 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:53.288+0000 7f19c77fe700 1 -- 172.21.15.12:0/1529259456 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19c80036a0 con 0x7f19d8109f00 2024-05-17T22:26:58.471 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:53.288+0000 7f19c77fe700 1 -- 172.21.15.12:0/1529259456 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f19c801a430 con 0x7f19d8109f00 2024-05-17T22:26:58.471 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:53.288+0000 7f19dcedd700 1 -- 172.21.15.12:0/1529259456 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19d8107e10 con 0x7f19d8109f00 2024-05-17T22:26:58.471 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:53.288+0000 7f19c77fe700 1 -- 172.21.15.12:0/1529259456 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19c8019550 con 0x7f19d8109f00 2024-05-17T22:26:58.471 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:53.288+0000 7f19dcedd700 1 -- 172.21.15.12:0/1529259456 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19d81082f0 con 0x7f19d8109f00 2024-05-17T22:26:58.471 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:53.289+0000 7f19c77fe700 1 -- 172.21.15.12:0/1529259456 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f19c8019800 con 0x7f19d8109f00 2024-05-17T22:26:58.471 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:53.290+0000 7f19c77fe700 1 --2- 172.21.15.12:0/1529259456 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f19c003e530 0x7f19c0040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:26:58.471 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:53.290+0000 7f19dcedd700 1 -- 172.21.15.12:0/1529259456 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19d810a390 con 0x7f19d8109f00 2024-05-17T22:26:58.472 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:53.290+0000 7f19c77fe700 1 -- 172.21.15.12:0/1529259456 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f19c8054840 con 0x7f19d8109f00 2024-05-17T22:26:58.472 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:53.290+0000 7f19d5d9b700 1 --2- 172.21.15.12:0/1529259456 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f19c003e530 0x7f19c0040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:26:58.472 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:53.290+0000 7f19d5d9b700 1 --2- 172.21.15.12:0/1529259456 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f19c003e530 0x7f19c0040a40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f19cc006fd0 tx=0x7f19cc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:26:58.472 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:53.295+0000 7f19c77fe700 1 -- 172.21.15.12:0/1529259456 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f19c8017e90 con 0x7f19d8109f00 2024-05-17T22:26:58.472 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:53.513+0000 7f19dcedd700 1 -- 172.21.15.12:0/1529259456 --> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi012", "addr": "172.21.15.12", "target": ["mon-mgr", ""]}) -- 0x7f19d8104b20 con 0x7f19c003e530 2024-05-17T22:26:58.472 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:57.867+0000 7f19c77fe700 1 -- 172.21.15.12:0/1529259456 <== mgr.14120 v2:172.21.15.12:6800/2250046521 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f19d8104b20 con 0x7f19c003e530 2024-05-17T22:26:58.472 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:57.870+0000 7f19dcedd700 1 -- 172.21.15.12:0/1529259456 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f19c003e530 msgr2=0x7f19c0040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:26:58.472 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:57.870+0000 7f19dcedd700 1 --2- 172.21.15.12:0/1529259456 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f19c003e530 0x7f19c0040a40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f19cc006fd0 tx=0x7f19cc006e40 comp rx=0 tx=0).stop 2024-05-17T22:26:58.472 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:57.870+0000 7f19dcedd700 1 -- 172.21.15.12:0/1529259456 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19d8109f00 msgr2=0x7f19d8109680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:26:58.472 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:57.870+0000 7f19dcedd700 1 --2- 172.21.15.12:0/1529259456 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19d8109f00 0x7f19d8109680 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f19c8009fb0 tx=0x7f19c8011940 comp rx=0 tx=0).stop 2024-05-17T22:26:58.472 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:57.870+0000 7f19dcedd700 1 -- 172.21.15.12:0/1529259456 shutdown_connections 2024-05-17T22:26:58.473 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:57.870+0000 7f19dcedd700 1 --2- 172.21.15.12:0/1529259456 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f19c003e530 0x7f19c0040a40 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:58.473 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:57.870+0000 7f19dcedd700 1 --2- 172.21.15.12:0/1529259456 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19d8109f00 0x7f19d8109680 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:26:58.473 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:57.870+0000 7f19dcedd700 1 -- 172.21.15.12:0/1529259456 >> 172.21.15.12:0/1529259456 conn(0x7f19d8103ab0 msgr2=0x7f19d8104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:26:58.473 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:57.870+0000 7f19dcedd700 1 -- 172.21.15.12:0/1529259456 shutdown_connections 2024-05-17T22:26:58.473 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:57.870+0000 7f19dcedd700 1 -- 172.21.15.12:0/1529259456 wait complete. 2024-05-17T22:26:58.473 INFO:teuthology.orchestra.run.smithi012.stdout:Deploying unmanaged mon service... 2024-05-17T22:26:59.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:58 smithi012 ceph-mon[31632]: from='mgr.14120 172.21.15.12:0/3665817882' entity='mgr.x' 2024-05-17T22:26:59.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:58 smithi012 ceph-mon[31632]: Added host smithi012 2024-05-17T22:26:59.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:26:58 smithi012 ceph-mon[31632]: from='mgr.14120 172.21.15.12:0/3665817882' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:27:00.518 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:00 smithi012 ceph-mon[31632]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:27:00.518 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:00 smithi012 ceph-mon[31632]: Saving service mon spec with placement count:5 2024-05-17T22:27:00.518 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:00 smithi012 ceph-mon[31632]: from='mgr.14120 172.21.15.12:0/3665817882' entity='mgr.x' 2024-05-17T22:27:00.784 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-17T22:27:00.784 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:58.956+0000 7f89e30c3700 1 Processor -- start 2024-05-17T22:27:00.784 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:58.956+0000 7f89e30c3700 1 -- start start 2024-05-17T22:27:00.784 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:58.956+0000 7f89e30c3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89dc109ed0 0x7f89dc10c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:00.784 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:58.956+0000 7f89e30c3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f89dc07d660 con 0x7f89dc109ed0 2024-05-17T22:27:00.784 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:58.956+0000 7f89e0e5f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89dc109ed0 0x7f89dc10c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:00.785 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:58.956+0000 7f89e0e5f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89dc109ed0 0x7f89dc10c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40156/0 (socket says 172.21.15.12:40156) 2024-05-17T22:27:00.785 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:58.956+0000 7f89e0e5f700 1 -- 172.21.15.12:0/2095574718 learned_addr learned my addr 172.21.15.12:0/2095574718 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:27:00.785 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:58.957+0000 7f89e0e5f700 1 -- 172.21.15.12:0/2095574718 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89dc10c830 con 0x7f89dc109ed0 2024-05-17T22:27:00.785 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:58.957+0000 7f89e0e5f700 1 --2- 172.21.15.12:0/2095574718 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89dc109ed0 0x7f89dc10c2f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f89d00098a0 tx=0x7f89d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f999ca8c591e4447 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:00.785 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:58.957+0000 7f89db7fe700 1 -- 172.21.15.12:0/2095574718 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89d0004160 con 0x7f89dc109ed0 2024-05-17T22:27:00.785 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:58.957+0000 7f89db7fe700 1 -- 172.21.15.12:0/2095574718 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f89d0004300 con 0x7f89dc109ed0 2024-05-17T22:27:00.785 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:58.957+0000 7f89db7fe700 1 -- 172.21.15.12:0/2095574718 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89d0011af0 con 0x7f89dc109ed0 2024-05-17T22:27:00.785 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:58.958+0000 7f89e30c3700 1 -- 172.21.15.12:0/2095574718 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89dc109ed0 msgr2=0x7f89dc10c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:00.785 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:58.958+0000 7f89e30c3700 1 --2- 172.21.15.12:0/2095574718 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89dc109ed0 0x7f89dc10c2f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f89d00098a0 tx=0x7f89d0009bb0 comp rx=0 tx=0).stop 2024-05-17T22:27:00.785 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:58.958+0000 7f89e30c3700 1 -- 172.21.15.12:0/2095574718 shutdown_connections 2024-05-17T22:27:00.785 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:58.958+0000 7f89e30c3700 1 --2- 172.21.15.12:0/2095574718 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89dc109ed0 0x7f89dc10c2f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:00.785 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:58.958+0000 7f89e30c3700 1 -- 172.21.15.12:0/2095574718 >> 172.21.15.12:0/2095574718 conn(0x7f89dc103ac0 msgr2=0x7f89dc105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:27:00.785 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:58.958+0000 7f89e30c3700 1 -- 172.21.15.12:0/2095574718 shutdown_connections 2024-05-17T22:27:00.785 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:58.958+0000 7f89e30c3700 1 -- 172.21.15.12:0/2095574718 wait complete. 2024-05-17T22:27:00.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:58.959+0000 7f89e30c3700 1 Processor -- start 2024-05-17T22:27:00.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:58.959+0000 7f89e30c3700 1 -- start start 2024-05-17T22:27:00.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:58.959+0000 7f89e30c3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89dc109ed0 0x7f89dc19d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:00.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:58.959+0000 7f89e30c3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f89dc10d790 con 0x7f89dc109ed0 2024-05-17T22:27:00.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:58.959+0000 7f89e0e5f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89dc109ed0 0x7f89dc19d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:00.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:58.959+0000 7f89e0e5f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89dc109ed0 0x7f89dc19d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40166/0 (socket says 172.21.15.12:40166) 2024-05-17T22:27:00.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:58.959+0000 7f89e0e5f700 1 -- 172.21.15.12:0/3675074525 learned_addr learned my addr 172.21.15.12:0/3675074525 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:27:00.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:58.959+0000 7f89e0e5f700 1 -- 172.21.15.12:0/3675074525 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89dc19dc80 con 0x7f89dc109ed0 2024-05-17T22:27:00.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:58.959+0000 7f89e0e5f700 1 --2- 172.21.15.12:0/3675074525 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89dc109ed0 0x7f89dc19d740 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f89d00117b0 tx=0x7f89d0011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:00.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:58.960+0000 7f89d9ffb700 1 -- 172.21.15.12:0/3675074525 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89d0003af0 con 0x7f89dc109ed0 2024-05-17T22:27:00.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:58.960+0000 7f89d9ffb700 1 -- 172.21.15.12:0/3675074525 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f89d0020070 con 0x7f89dc109ed0 2024-05-17T22:27:00.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:58.960+0000 7f89e30c3700 1 -- 172.21.15.12:0/3675074525 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89dc19df10 con 0x7f89dc109ed0 2024-05-17T22:27:00.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:58.960+0000 7f89d9ffb700 1 -- 172.21.15.12:0/3675074525 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89d0019630 con 0x7f89dc109ed0 2024-05-17T22:27:00.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:58.960+0000 7f89e30c3700 1 -- 172.21.15.12:0/3675074525 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89dc1a0b20 con 0x7f89dc109ed0 2024-05-17T22:27:00.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:58.961+0000 7f89e30c3700 1 -- 172.21.15.12:0/3675074525 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89dc10a360 con 0x7f89dc109ed0 2024-05-17T22:27:00.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:58.961+0000 7f89d9ffb700 1 -- 172.21.15.12:0/3675074525 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f89d00198f0 con 0x7f89dc109ed0 2024-05-17T22:27:00.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:58.961+0000 7f89d9ffb700 1 --2- 172.21.15.12:0/3675074525 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f89cc03e530 0x7f89cc040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:00.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:58.961+0000 7f89d9ffb700 1 -- 172.21.15.12:0/3675074525 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f89d0053c60 con 0x7f89dc109ed0 2024-05-17T22:27:00.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:58.961+0000 7f89dbfff700 1 --2- 172.21.15.12:0/3675074525 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f89cc03e530 0x7f89cc040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:00.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:58.962+0000 7f89dbfff700 1 --2- 172.21.15.12:0/3675074525 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f89cc03e530 0x7f89cc040a40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f89c8006fd0 tx=0x7f89c8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:00.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:58.966+0000 7f89d9ffb700 1 -- 172.21.15.12:0/3675074525 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f89d00042a0 con 0x7f89dc109ed0 2024-05-17T22:27:00.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:59.198+0000 7f89e30c3700 1 -- 172.21.15.12:0/3675074525 --> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f89dc1a1110 con 0x7f89cc03e530 2024-05-17T22:27:00.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:59.204+0000 7f89d9ffb700 1 -- 172.21.15.12:0/3675074525 <== mgr.14120 v2:172.21.15.12:6800/2250046521 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f89dc1a1110 con 0x7f89cc03e530 2024-05-17T22:27:00.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:59.206+0000 7f89e30c3700 1 -- 172.21.15.12:0/3675074525 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f89cc03e530 msgr2=0x7f89cc040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:00.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:59.206+0000 7f89e30c3700 1 --2- 172.21.15.12:0/3675074525 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f89cc03e530 0x7f89cc040a40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f89c8006fd0 tx=0x7f89c8006e40 comp rx=0 tx=0).stop 2024-05-17T22:27:00.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:59.206+0000 7f89e30c3700 1 -- 172.21.15.12:0/3675074525 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89dc109ed0 msgr2=0x7f89dc19d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:00.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:59.206+0000 7f89e30c3700 1 --2- 172.21.15.12:0/3675074525 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89dc109ed0 0x7f89dc19d740 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f89d00117b0 tx=0x7f89d0011b70 comp rx=0 tx=0).stop 2024-05-17T22:27:00.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:59.206+0000 7f89e30c3700 1 -- 172.21.15.12:0/3675074525 shutdown_connections 2024-05-17T22:27:00.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:59.206+0000 7f89e30c3700 1 --2- 172.21.15.12:0/3675074525 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f89cc03e530 0x7f89cc040a40 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:00.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:59.206+0000 7f89e30c3700 1 --2- 172.21.15.12:0/3675074525 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89dc109ed0 0x7f89dc19d740 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:00.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:59.206+0000 7f89e30c3700 1 -- 172.21.15.12:0/3675074525 >> 172.21.15.12:0/3675074525 conn(0x7f89dc103ac0 msgr2=0x7f89dc104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:27:00.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:59.206+0000 7f89e30c3700 1 -- 172.21.15.12:0/3675074525 shutdown_connections 2024-05-17T22:27:00.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:26:59.206+0000 7f89e30c3700 1 -- 172.21.15.12:0/3675074525 wait complete. 2024-05-17T22:27:00.788 INFO:teuthology.orchestra.run.smithi012.stdout:Deploying unmanaged mgr service... 2024-05-17T22:27:01.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:01 smithi012 ceph-mon[31632]: from='mgr.14120 172.21.15.12:0/3665817882' entity='mgr.x' 2024-05-17T22:27:01.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:01 smithi012 ceph-mon[31632]: from='mgr.14120 172.21.15.12:0/3665817882' entity='mgr.x' 2024-05-17T22:27:02.828 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:02 smithi012 ceph-mon[31632]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:27:02.828 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:02 smithi012 ceph-mon[31632]: Saving service mgr spec with placement count:2 2024-05-17T22:27:02.828 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:02 smithi012 ceph-mon[31632]: from='mgr.14120 172.21.15.12:0/3665817882' entity='mgr.x' 2024-05-17T22:27:02.835 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-17T22:27:02.835 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.202+0000 7f8a44bcf700 1 Processor -- start 2024-05-17T22:27:02.835 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.203+0000 7f8a44bcf700 1 -- start start 2024-05-17T22:27:02.835 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.203+0000 7f8a44bcf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a401093d0 0x7f8a401097e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:02.835 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.203+0000 7f8a44bcf700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8a40109db0 con 0x7f8a401093d0 2024-05-17T22:27:02.835 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.203+0000 7f8a3e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a401093d0 0x7f8a401097e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:02.835 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.203+0000 7f8a3e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a401093d0 0x7f8a401097e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44822/0 (socket says 172.21.15.12:44822) 2024-05-17T22:27:02.836 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.203+0000 7f8a3e59c700 1 -- 172.21.15.12:0/2752399460 learned_addr learned my addr 172.21.15.12:0/2752399460 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:27:02.836 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.204+0000 7f8a3e59c700 1 -- 172.21.15.12:0/2752399460 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a4010a5a0 con 0x7f8a401093d0 2024-05-17T22:27:02.836 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.204+0000 7f8a3e59c700 1 --2- 172.21.15.12:0/2752399460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a401093d0 0x7f8a401097e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8a300098a0 tx=0x7f8a30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=562d861cb4cafa61 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:02.836 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.204+0000 7f8a3d59a700 1 -- 172.21.15.12:0/2752399460 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a30004160 con 0x7f8a401093d0 2024-05-17T22:27:02.836 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.204+0000 7f8a3d59a700 1 -- 172.21.15.12:0/2752399460 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a30004300 con 0x7f8a401093d0 2024-05-17T22:27:02.836 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.204+0000 7f8a3d59a700 1 -- 172.21.15.12:0/2752399460 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a30011a20 con 0x7f8a401093d0 2024-05-17T22:27:02.836 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.205+0000 7f8a44bcf700 1 -- 172.21.15.12:0/2752399460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a401093d0 msgr2=0x7f8a401097e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:02.836 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.205+0000 7f8a44bcf700 1 --2- 172.21.15.12:0/2752399460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a401093d0 0x7f8a401097e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8a300098a0 tx=0x7f8a30009bb0 comp rx=0 tx=0).stop 2024-05-17T22:27:02.836 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.205+0000 7f8a44bcf700 1 -- 172.21.15.12:0/2752399460 shutdown_connections 2024-05-17T22:27:02.836 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.205+0000 7f8a44bcf700 1 --2- 172.21.15.12:0/2752399460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a401093d0 0x7f8a401097e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:02.836 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.205+0000 7f8a44bcf700 1 -- 172.21.15.12:0/2752399460 >> 172.21.15.12:0/2752399460 conn(0x7f8a40104c80 msgr2=0x7f8a401070d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:27:02.836 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.205+0000 7f8a44bcf700 1 -- 172.21.15.12:0/2752399460 shutdown_connections 2024-05-17T22:27:02.836 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.205+0000 7f8a44bcf700 1 -- 172.21.15.12:0/2752399460 wait complete. 2024-05-17T22:27:02.836 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.206+0000 7f8a44bcf700 1 Processor -- start 2024-05-17T22:27:02.837 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.206+0000 7f8a44bcf700 1 -- start start 2024-05-17T22:27:02.837 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.206+0000 7f8a44bcf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a401093d0 0x7f8a4007ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:02.837 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.206+0000 7f8a44bcf700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4010d950 con 0x7f8a401093d0 2024-05-17T22:27:02.837 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.206+0000 7f8a3e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a401093d0 0x7f8a4007ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:02.837 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.206+0000 7f8a3e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a401093d0 0x7f8a4007ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44838/0 (socket says 172.21.15.12:44838) 2024-05-17T22:27:02.837 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.206+0000 7f8a3e59c700 1 -- 172.21.15.12:0/1059779358 learned_addr learned my addr 172.21.15.12:0/1059779358 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:27:02.837 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.207+0000 7f8a3e59c700 1 -- 172.21.15.12:0/1059779358 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a4007ef60 con 0x7f8a401093d0 2024-05-17T22:27:02.837 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.207+0000 7f8a3e59c700 1 --2- 172.21.15.12:0/1059779358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a401093d0 0x7f8a4007ea20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8a30009520 tx=0x7f8a30011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:02.837 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.207+0000 7f8a2f7fe700 1 -- 172.21.15.12:0/1059779358 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a300039e0 con 0x7f8a401093d0 2024-05-17T22:27:02.837 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.207+0000 7f8a44bcf700 1 -- 172.21.15.12:0/1059779358 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a4007d1b0 con 0x7f8a401093d0 2024-05-17T22:27:02.837 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.207+0000 7f8a2f7fe700 1 -- 172.21.15.12:0/1059779358 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a3001b8f0 con 0x7f8a401093d0 2024-05-17T22:27:02.837 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.207+0000 7f8a2f7fe700 1 -- 172.21.15.12:0/1059779358 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a300197d0 con 0x7f8a401093d0 2024-05-17T22:27:02.837 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.207+0000 7f8a44bcf700 1 -- 172.21.15.12:0/1059779358 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a4007d690 con 0x7f8a401093d0 2024-05-17T22:27:02.837 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.208+0000 7f8a44bcf700 1 -- 172.21.15.12:0/1059779358 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a401097e0 con 0x7f8a401093d0 2024-05-17T22:27:02.837 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.208+0000 7f8a2f7fe700 1 -- 172.21.15.12:0/1059779358 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f8a30019a60 con 0x7f8a401093d0 2024-05-17T22:27:02.838 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.209+0000 7f8a2f7fe700 1 --2- 172.21.15.12:0/1059779358 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f8a2803e580 0x7f8a28040a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:02.838 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.209+0000 7f8a2f7fe700 1 -- 172.21.15.12:0/1059779358 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8a30053f20 con 0x7f8a401093d0 2024-05-17T22:27:02.838 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.209+0000 7f8a3dd9b700 1 --2- 172.21.15.12:0/1059779358 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f8a2803e580 0x7f8a28040a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:02.838 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.209+0000 7f8a3dd9b700 1 --2- 172.21.15.12:0/1059779358 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f8a2803e580 0x7f8a28040a90 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8a34006fd0 tx=0x7f8a34006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:02.838 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.218+0000 7f8a2f7fe700 1 -- 172.21.15.12:0/1059779358 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8a3001f030 con 0x7f8a401093d0 2024-05-17T22:27:02.838 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.465+0000 7f8a44bcf700 1 -- 172.21.15.12:0/1059779358 --> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f8a40106c30 con 0x7f8a2803e580 2024-05-17T22:27:02.838 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.474+0000 7f8a2f7fe700 1 -- 172.21.15.12:0/1059779358 <== mgr.14120 v2:172.21.15.12:6800/2250046521 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f8a40106c30 con 0x7f8a2803e580 2024-05-17T22:27:02.838 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.477+0000 7f8a44bcf700 1 -- 172.21.15.12:0/1059779358 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f8a2803e580 msgr2=0x7f8a28040a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:02.838 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.477+0000 7f8a44bcf700 1 --2- 172.21.15.12:0/1059779358 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f8a2803e580 0x7f8a28040a90 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8a34006fd0 tx=0x7f8a34006e40 comp rx=0 tx=0).stop 2024-05-17T22:27:02.838 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.477+0000 7f8a44bcf700 1 -- 172.21.15.12:0/1059779358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a401093d0 msgr2=0x7f8a4007ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:02.838 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.477+0000 7f8a44bcf700 1 --2- 172.21.15.12:0/1059779358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a401093d0 0x7f8a4007ea20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8a30009520 tx=0x7f8a30011aa0 comp rx=0 tx=0).stop 2024-05-17T22:27:02.838 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.478+0000 7f8a44bcf700 1 -- 172.21.15.12:0/1059779358 shutdown_connections 2024-05-17T22:27:02.838 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.478+0000 7f8a44bcf700 1 --2- 172.21.15.12:0/1059779358 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f8a2803e580 0x7f8a28040a90 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:02.839 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.478+0000 7f8a44bcf700 1 --2- 172.21.15.12:0/1059779358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a401093d0 0x7f8a4007ea20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:02.839 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.478+0000 7f8a44bcf700 1 -- 172.21.15.12:0/1059779358 >> 172.21.15.12:0/1059779358 conn(0x7f8a40104c80 msgr2=0x7f8a40106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:27:02.839 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.478+0000 7f8a44bcf700 1 -- 172.21.15.12:0/1059779358 shutdown_connections 2024-05-17T22:27:02.839 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:01.478+0000 7f8a44bcf700 1 -- 172.21.15.12:0/1059779358 wait complete. 2024-05-17T22:27:03.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:03 smithi012 ceph-mon[31632]: from='mgr.14120 172.21.15.12:0/3665817882' entity='mgr.x' 2024-05-17T22:27:04.143 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.319+0000 7fc241ff8700 1 Processor -- start 2024-05-17T22:27:04.143 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.320+0000 7fc241ff8700 1 -- start start 2024-05-17T22:27:04.143 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.320+0000 7fc241ff8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc23c109e80 0x7fc23c10c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:04.143 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.320+0000 7fc241ff8700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc23c07d7d0 con 0x7fc23c109e80 2024-05-17T22:27:04.143 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.320+0000 7fc23b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc23c109e80 0x7fc23c10c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:04.143 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.320+0000 7fc23b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc23c109e80 0x7fc23c10c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44848/0 (socket says 172.21.15.12:44848) 2024-05-17T22:27:04.144 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.320+0000 7fc23b7fe700 1 -- 172.21.15.12:0/332258039 learned_addr learned my addr 172.21.15.12:0/332258039 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:27:04.144 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.320+0000 7fc23b7fe700 1 -- 172.21.15.12:0/332258039 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc23c10c7e0 con 0x7fc23c109e80 2024-05-17T22:27:04.144 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.321+0000 7fc23b7fe700 1 --2- 172.21.15.12:0/332258039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc23c109e80 0x7fc23c10c2a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc22c0098a0 tx=0x7fc22c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b0d4542fc48a1f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:04.144 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.321+0000 7fc23a7fc700 1 -- 172.21.15.12:0/332258039 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc22c004160 con 0x7fc23c109e80 2024-05-17T22:27:04.144 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.321+0000 7fc23a7fc700 1 -- 172.21.15.12:0/332258039 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc22c004300 con 0x7fc23c109e80 2024-05-17T22:27:04.144 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.321+0000 7fc23a7fc700 1 -- 172.21.15.12:0/332258039 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc22c011a20 con 0x7fc23c109e80 2024-05-17T22:27:04.144 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.322+0000 7fc241ff8700 1 -- 172.21.15.12:0/332258039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc23c109e80 msgr2=0x7fc23c10c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:04.144 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.322+0000 7fc241ff8700 1 --2- 172.21.15.12:0/332258039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc23c109e80 0x7fc23c10c2a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc22c0098a0 tx=0x7fc22c009bb0 comp rx=0 tx=0).stop 2024-05-17T22:27:04.144 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.322+0000 7fc241ff8700 1 -- 172.21.15.12:0/332258039 shutdown_connections 2024-05-17T22:27:04.144 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.322+0000 7fc241ff8700 1 --2- 172.21.15.12:0/332258039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc23c109e80 0x7fc23c10c2a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:04.144 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.322+0000 7fc241ff8700 1 -- 172.21.15.12:0/332258039 >> 172.21.15.12:0/332258039 conn(0x7fc23c103ab0 msgr2=0x7fc23c105f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:27:04.144 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.322+0000 7fc241ff8700 1 -- 172.21.15.12:0/332258039 shutdown_connections 2024-05-17T22:27:04.144 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.322+0000 7fc241ff8700 1 -- 172.21.15.12:0/332258039 wait complete. 2024-05-17T22:27:04.145 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.323+0000 7fc241ff8700 1 Processor -- start 2024-05-17T22:27:04.145 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.323+0000 7fc241ff8700 1 -- start start 2024-05-17T22:27:04.145 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.323+0000 7fc241ff8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc23c109e80 0x7fc23c109600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:04.145 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.323+0000 7fc241ff8700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc23c10d7e0 con 0x7fc23c109e80 2024-05-17T22:27:04.145 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.323+0000 7fc23b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc23c109e80 0x7fc23c109600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:04.145 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.323+0000 7fc23b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc23c109e80 0x7fc23c109600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44856/0 (socket says 172.21.15.12:44856) 2024-05-17T22:27:04.145 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.323+0000 7fc23b7fe700 1 -- 172.21.15.12:0/3935174250 learned_addr learned my addr 172.21.15.12:0/3935174250 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:27:04.145 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.324+0000 7fc23b7fe700 1 -- 172.21.15.12:0/3935174250 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc23c109b40 con 0x7fc23c109e80 2024-05-17T22:27:04.145 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.324+0000 7fc23b7fe700 1 --2- 172.21.15.12:0/3935174250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc23c109e80 0x7fc23c109600 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc22c000c00 tx=0x7fc22c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:04.145 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.324+0000 7fc238ff9700 1 -- 172.21.15.12:0/3935174250 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc22c003aa0 con 0x7fc23c109e80 2024-05-17T22:27:04.145 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.324+0000 7fc238ff9700 1 -- 172.21.15.12:0/3935174250 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc22c01b8f0 con 0x7fc23c109e80 2024-05-17T22:27:04.145 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.324+0000 7fc241ff8700 1 -- 172.21.15.12:0/3935174250 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc23c107d90 con 0x7fc23c109e80 2024-05-17T22:27:04.145 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.324+0000 7fc238ff9700 1 -- 172.21.15.12:0/3935174250 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc22c023890 con 0x7fc23c109e80 2024-05-17T22:27:04.146 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.324+0000 7fc241ff8700 1 -- 172.21.15.12:0/3935174250 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc23c108270 con 0x7fc23c109e80 2024-05-17T22:27:04.146 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.325+0000 7fc241ff8700 1 -- 172.21.15.12:0/3935174250 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc23c10a290 con 0x7fc23c109e80 2024-05-17T22:27:04.146 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.325+0000 7fc238ff9700 1 -- 172.21.15.12:0/3935174250 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fc22c02c3f0 con 0x7fc23c109e80 2024-05-17T22:27:04.146 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.326+0000 7fc238ff9700 1 --2- 172.21.15.12:0/3935174250 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7fc22403e530 0x7fc224040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:04.146 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.326+0000 7fc238ff9700 1 -- 172.21.15.12:0/3935174250 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc22c053e40 con 0x7fc23c109e80 2024-05-17T22:27:04.146 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.326+0000 7fc23affd700 1 --2- 172.21.15.12:0/3935174250 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7fc22403e530 0x7fc224040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:04.146 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.326+0000 7fc23affd700 1 --2- 172.21.15.12:0/3935174250 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7fc22403e530 0x7fc224040a40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc230006fd0 tx=0x7fc230006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:04.146 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.331+0000 7fc238ff9700 1 -- 172.21.15.12:0/3935174250 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc22c01f030 con 0x7fc23c109e80 2024-05-17T22:27:04.146 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.552+0000 7fc241ff8700 1 -- 172.21.15.12:0/3935174250 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fc23c1a37a0 con 0x7fc23c109e80 2024-05-17T22:27:04.146 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.556+0000 7fc238ff9700 1 -- 172.21.15.12:0/3935174250 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fc22c00f2e0 con 0x7fc23c109e80 2024-05-17T22:27:04.146 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.558+0000 7fc241ff8700 1 -- 172.21.15.12:0/3935174250 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7fc22403e530 msgr2=0x7fc224040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:04.146 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.558+0000 7fc241ff8700 1 --2- 172.21.15.12:0/3935174250 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7fc22403e530 0x7fc224040a40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc230006fd0 tx=0x7fc230006e40 comp rx=0 tx=0).stop 2024-05-17T22:27:04.146 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.558+0000 7fc241ff8700 1 -- 172.21.15.12:0/3935174250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc23c109e80 msgr2=0x7fc23c109600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:04.147 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.558+0000 7fc241ff8700 1 --2- 172.21.15.12:0/3935174250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc23c109e80 0x7fc23c109600 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc22c000c00 tx=0x7fc22c011aa0 comp rx=0 tx=0).stop 2024-05-17T22:27:04.147 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.559+0000 7fc241ff8700 1 -- 172.21.15.12:0/3935174250 shutdown_connections 2024-05-17T22:27:04.147 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.559+0000 7fc241ff8700 1 --2- 172.21.15.12:0/3935174250 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7fc22403e530 0x7fc224040a40 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:04.147 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.559+0000 7fc241ff8700 1 --2- 172.21.15.12:0/3935174250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc23c109e80 0x7fc23c109600 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:04.147 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.559+0000 7fc241ff8700 1 -- 172.21.15.12:0/3935174250 >> 172.21.15.12:0/3935174250 conn(0x7fc23c103ab0 msgr2=0x7fc23c104370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:27:04.147 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.559+0000 7fc241ff8700 1 -- 172.21.15.12:0/3935174250 shutdown_connections 2024-05-17T22:27:04.147 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:03.559+0000 7fc241ff8700 1 -- 172.21.15.12:0/3935174250 wait complete. 2024-05-17T22:27:04.712 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:04 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/3935174250' entity='client.admin' 2024-05-17T22:27:04.712 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:04 smithi012 ceph-mon[31632]: from='mgr.14120 172.21.15.12:0/3665817882' entity='mgr.x' 2024-05-17T22:27:05.418 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.564+0000 7fc1cf20d700 1 Processor -- start 2024-05-17T22:27:05.418 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.564+0000 7fc1cf20d700 1 -- start start 2024-05-17T22:27:05.419 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.564+0000 7fc1cf20d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1c8109ea0 0x7fc1c810c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:05.419 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.564+0000 7fc1cf20d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c807d6a0 con 0x7fc1c8109ea0 2024-05-17T22:27:05.419 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.565+0000 7fc1ccfa9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1c8109ea0 0x7fc1c810c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:05.419 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.565+0000 7fc1ccfa9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1c8109ea0 0x7fc1c810c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44868/0 (socket says 172.21.15.12:44868) 2024-05-17T22:27:05.419 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.565+0000 7fc1ccfa9700 1 -- 172.21.15.12:0/1154477413 learned_addr learned my addr 172.21.15.12:0/1154477413 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:27:05.419 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.565+0000 7fc1ccfa9700 1 -- 172.21.15.12:0/1154477413 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c810c800 con 0x7fc1c8109ea0 2024-05-17T22:27:05.419 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.565+0000 7fc1ccfa9700 1 --2- 172.21.15.12:0/1154477413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1c8109ea0 0x7fc1c810c2c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc0098a0 tx=0x7fc1bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20287308fe8f3e3d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:05.419 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.565+0000 7fc1c77fe700 1 -- 172.21.15.12:0/1154477413 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc1bc004160 con 0x7fc1c8109ea0 2024-05-17T22:27:05.419 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.565+0000 7fc1c77fe700 1 -- 172.21.15.12:0/1154477413 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1bc004300 con 0x7fc1c8109ea0 2024-05-17T22:27:05.419 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.565+0000 7fc1c77fe700 1 -- 172.21.15.12:0/1154477413 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc1bc011a20 con 0x7fc1c8109ea0 2024-05-17T22:27:05.419 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.566+0000 7fc1cf20d700 1 -- 172.21.15.12:0/1154477413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1c8109ea0 msgr2=0x7fc1c810c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:05.419 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.566+0000 7fc1cf20d700 1 --2- 172.21.15.12:0/1154477413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1c8109ea0 0x7fc1c810c2c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc0098a0 tx=0x7fc1bc009bb0 comp rx=0 tx=0).stop 2024-05-17T22:27:05.419 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.566+0000 7fc1cf20d700 1 -- 172.21.15.12:0/1154477413 shutdown_connections 2024-05-17T22:27:05.419 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.566+0000 7fc1cf20d700 1 --2- 172.21.15.12:0/1154477413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1c8109ea0 0x7fc1c810c2c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:05.420 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.566+0000 7fc1cf20d700 1 -- 172.21.15.12:0/1154477413 >> 172.21.15.12:0/1154477413 conn(0x7fc1c8103af0 msgr2=0x7fc1c8105fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:27:05.420 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.566+0000 7fc1cf20d700 1 -- 172.21.15.12:0/1154477413 shutdown_connections 2024-05-17T22:27:05.420 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.566+0000 7fc1cf20d700 1 -- 172.21.15.12:0/1154477413 wait complete. 2024-05-17T22:27:05.420 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.567+0000 7fc1cf20d700 1 Processor -- start 2024-05-17T22:27:05.420 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.567+0000 7fc1cf20d700 1 -- start start 2024-05-17T22:27:05.420 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.567+0000 7fc1cf20d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1c8109ea0 0x7fc1c819d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:05.420 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.567+0000 7fc1cf20d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c810d800 con 0x7fc1c8109ea0 2024-05-17T22:27:05.420 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.567+0000 7fc1ccfa9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1c8109ea0 0x7fc1c819d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:05.420 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.567+0000 7fc1ccfa9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1c8109ea0 0x7fc1c819d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44874/0 (socket says 172.21.15.12:44874) 2024-05-17T22:27:05.420 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.567+0000 7fc1ccfa9700 1 -- 172.21.15.12:0/655186309 learned_addr learned my addr 172.21.15.12:0/655186309 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:27:05.420 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.568+0000 7fc1ccfa9700 1 -- 172.21.15.12:0/655186309 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c819dc00 con 0x7fc1c8109ea0 2024-05-17T22:27:05.420 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.568+0000 7fc1ccfa9700 1 --2- 172.21.15.12:0/655186309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1c8109ea0 0x7fc1c819d6c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc0119f0 tx=0x7fc1bc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:05.420 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.568+0000 7fc1c5ffb700 1 -- 172.21.15.12:0/655186309 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc1bc0039e0 con 0x7fc1c8109ea0 2024-05-17T22:27:05.420 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.568+0000 7fc1c5ffb700 1 -- 172.21.15.12:0/655186309 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1bc01b8f0 con 0x7fc1c8109ea0 2024-05-17T22:27:05.420 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.568+0000 7fc1cf20d700 1 -- 172.21.15.12:0/655186309 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1c819de90 con 0x7fc1c8109ea0 2024-05-17T22:27:05.421 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.568+0000 7fc1c5ffb700 1 -- 172.21.15.12:0/655186309 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc1bc0237a0 con 0x7fc1c8109ea0 2024-05-17T22:27:05.421 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.568+0000 7fc1cf20d700 1 -- 172.21.15.12:0/655186309 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1c81a0aa0 con 0x7fc1c8109ea0 2024-05-17T22:27:05.421 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.569+0000 7fc1cf20d700 1 -- 172.21.15.12:0/655186309 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1c810a2b0 con 0x7fc1c8109ea0 2024-05-17T22:27:05.421 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.569+0000 7fc1c5ffb700 1 -- 172.21.15.12:0/655186309 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fc1bc023aa0 con 0x7fc1c8109ea0 2024-05-17T22:27:05.421 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.570+0000 7fc1c5ffb700 1 --2- 172.21.15.12:0/655186309 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7fc1b803e4e0 0x7fc1b80409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:05.421 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.570+0000 7fc1c5ffb700 1 -- 172.21.15.12:0/655186309 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc1bc053e40 con 0x7fc1c8109ea0 2024-05-17T22:27:05.421 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.570+0000 7fc1c7fff700 1 --2- 172.21.15.12:0/655186309 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7fc1b803e4e0 0x7fc1b80409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:05.421 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.570+0000 7fc1c7fff700 1 --2- 172.21.15.12:0/655186309 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7fc1b803e4e0 0x7fc1b80409f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4006fd0 tx=0x7fc1b4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:05.421 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.575+0000 7fc1c5ffb700 1 -- 172.21.15.12:0/655186309 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc1bc01f030 con 0x7fc1c8109ea0 2024-05-17T22:27:05.421 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.792+0000 7fc1cf20d700 1 -- 172.21.15.12:0/655186309 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fc1c81a1090 con 0x7fc1c8109ea0 2024-05-17T22:27:05.421 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.795+0000 7fc1c5ffb700 1 -- 172.21.15.12:0/655186309 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fc1c81a1090 con 0x7fc1c8109ea0 2024-05-17T22:27:05.421 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.798+0000 7fc1cf20d700 1 -- 172.21.15.12:0/655186309 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7fc1b803e4e0 msgr2=0x7fc1b80409f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:05.421 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.798+0000 7fc1cf20d700 1 --2- 172.21.15.12:0/655186309 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7fc1b803e4e0 0x7fc1b80409f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4006fd0 tx=0x7fc1b4006e40 comp rx=0 tx=0).stop 2024-05-17T22:27:05.421 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.798+0000 7fc1cf20d700 1 -- 172.21.15.12:0/655186309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1c8109ea0 msgr2=0x7fc1c819d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:05.421 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.798+0000 7fc1cf20d700 1 --2- 172.21.15.12:0/655186309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1c8109ea0 0x7fc1c819d6c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc0119f0 tx=0x7fc1bc011ad0 comp rx=0 tx=0).stop 2024-05-17T22:27:05.422 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.798+0000 7fc1cf20d700 1 -- 172.21.15.12:0/655186309 shutdown_connections 2024-05-17T22:27:05.422 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.798+0000 7fc1cf20d700 1 --2- 172.21.15.12:0/655186309 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7fc1b803e4e0 0x7fc1b80409f0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:05.422 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.798+0000 7fc1cf20d700 1 --2- 172.21.15.12:0/655186309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1c8109ea0 0x7fc1c819d6c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:05.422 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.798+0000 7fc1cf20d700 1 -- 172.21.15.12:0/655186309 >> 172.21.15.12:0/655186309 conn(0x7fc1c8103af0 msgr2=0x7fc1c8104340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:27:05.422 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.798+0000 7fc1cf20d700 1 -- 172.21.15.12:0/655186309 shutdown_connections 2024-05-17T22:27:05.422 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:04.798+0000 7fc1cf20d700 1 -- 172.21.15.12:0/655186309 wait complete. 2024-05-17T22:27:05.422 INFO:teuthology.orchestra.run.smithi012.stdout:Enabling the dashboard module... 2024-05-17T22:27:06.065 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:05 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/655186309' entity='client.admin' 2024-05-17T22:27:07.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:06 smithi012 ceph-mon[31632]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:27:07.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:06 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/955038351' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-17T22:27:07.091 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:27:06 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: ignoring --setuser ceph since I am not root 2024-05-17T22:27:07.091 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:27:06 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: ignoring --setgroup ceph since I am not root 2024-05-17T22:27:07.091 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:27:07 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:27:07.022+0000 7f3c03ffc200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-17T22:27:07.422 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:27:07 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:27:07.218+0000 7f3c03ffc200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-17T22:27:07.423 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:27:07 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:27:07.326+0000 7f3c03ffc200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-17T22:27:07.430 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:05.845+0000 7f536e79f700 1 Processor -- start 2024-05-17T22:27:07.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:05.845+0000 7f536e79f700 1 -- start start 2024-05-17T22:27:07.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:05.846+0000 7f536e79f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5368109e50 0x7f536810c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:07.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:05.846+0000 7f536e79f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f536807ce10 con 0x7f5368109e50 2024-05-17T22:27:07.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:05.846+0000 7f5367fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5368109e50 0x7f536810c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:07.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:05.846+0000 7f5367fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5368109e50 0x7f536810c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44878/0 (socket says 172.21.15.12:44878) 2024-05-17T22:27:07.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:05.846+0000 7f5367fff700 1 -- 172.21.15.12:0/3298034415 learned_addr learned my addr 172.21.15.12:0/3298034415 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:27:07.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:05.846+0000 7f5367fff700 1 -- 172.21.15.12:0/3298034415 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f536810c7b0 con 0x7f5368109e50 2024-05-17T22:27:07.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:05.846+0000 7f5367fff700 1 --2- 172.21.15.12:0/3298034415 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5368109e50 0x7f536810c270 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f53580098a0 tx=0x7f5358009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb70f07039ec368 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:07.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:05.846+0000 7f53677fe700 1 -- 172.21.15.12:0/3298034415 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5358004160 con 0x7f5368109e50 2024-05-17T22:27:07.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:05.847+0000 7f53677fe700 1 -- 172.21.15.12:0/3298034415 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5358004300 con 0x7f5368109e50 2024-05-17T22:27:07.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:05.847+0000 7f53677fe700 1 -- 172.21.15.12:0/3298034415 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5358011a20 con 0x7f5368109e50 2024-05-17T22:27:07.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:05.847+0000 7f536e79f700 1 -- 172.21.15.12:0/3298034415 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5368109e50 msgr2=0x7f536810c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:07.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:05.847+0000 7f536e79f700 1 --2- 172.21.15.12:0/3298034415 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5368109e50 0x7f536810c270 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f53580098a0 tx=0x7f5358009bb0 comp rx=0 tx=0).stop 2024-05-17T22:27:07.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:05.847+0000 7f536e79f700 1 -- 172.21.15.12:0/3298034415 shutdown_connections 2024-05-17T22:27:07.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:05.847+0000 7f536e79f700 1 --2- 172.21.15.12:0/3298034415 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5368109e50 0x7f536810c270 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:07.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:05.847+0000 7f536e79f700 1 -- 172.21.15.12:0/3298034415 >> 172.21.15.12:0/3298034415 conn(0x7f5368103a80 msgr2=0x7f5368105f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:27:07.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:05.847+0000 7f536e79f700 1 -- 172.21.15.12:0/3298034415 shutdown_connections 2024-05-17T22:27:07.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:05.847+0000 7f536e79f700 1 -- 172.21.15.12:0/3298034415 wait complete. 2024-05-17T22:27:07.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:05.848+0000 7f536e79f700 1 Processor -- start 2024-05-17T22:27:07.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:05.848+0000 7f536e79f700 1 -- start start 2024-05-17T22:27:07.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:05.848+0000 7f536e79f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5368109e50 0x7f536819d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:07.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:05.848+0000 7f536e79f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f536810d7b0 con 0x7f5368109e50 2024-05-17T22:27:07.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:05.848+0000 7f5367fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5368109e50 0x7f536819d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:07.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:05.848+0000 7f5367fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5368109e50 0x7f536819d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44894/0 (socket says 172.21.15.12:44894) 2024-05-17T22:27:07.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:05.849+0000 7f5367fff700 1 -- 172.21.15.12:0/955038351 learned_addr learned my addr 172.21.15.12:0/955038351 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:27:07.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:05.849+0000 7f5367fff700 1 -- 172.21.15.12:0/955038351 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f536819dc00 con 0x7f5368109e50 2024-05-17T22:27:07.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:05.849+0000 7f5367fff700 1 --2- 172.21.15.12:0/955038351 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5368109e50 0x7f536819d6c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f53580119f0 tx=0x7f5358011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:07.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:05.849+0000 7f5365ffb700 1 -- 172.21.15.12:0/955038351 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53580039e0 con 0x7f5368109e50 2024-05-17T22:27:07.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:05.849+0000 7f5365ffb700 1 -- 172.21.15.12:0/955038351 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f535801b8f0 con 0x7f5368109e50 2024-05-17T22:27:07.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:05.849+0000 7f536e79f700 1 -- 172.21.15.12:0/955038351 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f536819de90 con 0x7f5368109e50 2024-05-17T22:27:07.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:05.849+0000 7f5365ffb700 1 -- 172.21.15.12:0/955038351 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53580237a0 con 0x7f5368109e50 2024-05-17T22:27:07.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:05.849+0000 7f536e79f700 1 -- 172.21.15.12:0/955038351 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53681a0aa0 con 0x7f5368109e50 2024-05-17T22:27:07.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:05.850+0000 7f536e79f700 1 -- 172.21.15.12:0/955038351 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f536810a260 con 0x7f5368109e50 2024-05-17T22:27:07.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:05.850+0000 7f5365ffb700 1 -- 172.21.15.12:0/955038351 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f5358023a50 con 0x7f5368109e50 2024-05-17T22:27:07.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:05.850+0000 7f5365ffb700 1 --2- 172.21.15.12:0/955038351 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f535403e530 0x7f5354040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:07.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:05.851+0000 7f5365ffb700 1 -- 172.21.15.12:0/955038351 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5358053d90 con 0x7f5368109e50 2024-05-17T22:27:07.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:05.851+0000 7f535ffff700 1 --2- 172.21.15.12:0/955038351 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f535403e530 0x7f5354040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:07.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:05.851+0000 7f535ffff700 1 --2- 172.21.15.12:0/955038351 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f535403e530 0x7f5354040a40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5350006fd0 tx=0x7f5350006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:07.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:05.855+0000 7f5365ffb700 1 -- 172.21.15.12:0/955038351 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f535801f030 con 0x7f5368109e50 2024-05-17T22:27:07.434 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:06.133+0000 7f536e79f700 1 -- 172.21.15.12:0/955038351 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f53681a1090 con 0x7f5368109e50 2024-05-17T22:27:07.434 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:06.798+0000 7f5365ffb700 1 -- 172.21.15.12:0/955038351 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f535800b3e0 con 0x7f5368109e50 2024-05-17T22:27:07.434 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:06.800+0000 7f5365ffb700 1 -- 172.21.15.12:0/955038351 <== mon.0 v2:172.21.15.12:3300/0 8 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f535805a080 con 0x7f5368109e50 2024-05-17T22:27:07.434 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:06.801+0000 7f536e79f700 1 -- 172.21.15.12:0/955038351 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f535403e530 msgr2=0x7f5354040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:07.434 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:06.801+0000 7f536e79f700 1 --2- 172.21.15.12:0/955038351 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f535403e530 0x7f5354040a40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5350006fd0 tx=0x7f5350006e40 comp rx=0 tx=0).stop 2024-05-17T22:27:07.434 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:06.801+0000 7f536e79f700 1 -- 172.21.15.12:0/955038351 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5368109e50 msgr2=0x7f536819d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:07.434 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:06.801+0000 7f536e79f700 1 --2- 172.21.15.12:0/955038351 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5368109e50 0x7f536819d6c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f53580119f0 tx=0x7f5358011ad0 comp rx=0 tx=0).stop 2024-05-17T22:27:07.434 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:06.801+0000 7f536e79f700 1 -- 172.21.15.12:0/955038351 shutdown_connections 2024-05-17T22:27:07.434 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:06.801+0000 7f536e79f700 1 --2- 172.21.15.12:0/955038351 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f535403e530 0x7f5354040a40 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:07.434 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:06.801+0000 7f536e79f700 1 --2- 172.21.15.12:0/955038351 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5368109e50 0x7f536819d6c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:07.434 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:06.801+0000 7f536e79f700 1 -- 172.21.15.12:0/955038351 >> 172.21.15.12:0/955038351 conn(0x7f5368103a80 msgr2=0x7f5368104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:27:07.434 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:06.802+0000 7f536e79f700 1 -- 172.21.15.12:0/955038351 shutdown_connections 2024-05-17T22:27:07.434 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:06.802+0000 7f536e79f700 1 -- 172.21.15.12:0/955038351 wait complete. 2024-05-17T22:27:07.685 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:27:07 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:27:07.657+0000 7f3c03ffc200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-17T22:27:08.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:07 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/955038351' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-17T22:27:08.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:07 smithi012 ceph-mon[31632]: mgrmap e9: x(active, since 21s) 2024-05-17T22:27:08.091 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:27:07 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:27:07.751+0000 7f3c03ffc200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-17T22:27:08.762 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-05-17T22:27:08.762 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-17T22:27:08.762 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-17T22:27:08.762 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-17T22:27:08.762 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-17T22:27:08.762 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-05-17T22:27:08.762 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:07.909+0000 7f5fd1b67700 1 Processor -- start 2024-05-17T22:27:08.762 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:07.909+0000 7f5fd1b67700 1 -- start start 2024-05-17T22:27:08.762 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:07.910+0000 7f5fd1b67700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fcc107cd0 0x7f5fcc1080e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:08.762 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:07.910+0000 7f5fd1b67700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5fcc07d660 con 0x7f5fcc107cd0 2024-05-17T22:27:08.762 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:07.910+0000 7f5fcb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fcc107cd0 0x7f5fcc1080e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:08.763 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:07.910+0000 7f5fcb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fcc107cd0 0x7f5fcc1080e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44934/0 (socket says 172.21.15.12:44934) 2024-05-17T22:27:08.763 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:07.910+0000 7f5fcb7fe700 1 -- 172.21.15.12:0/2871749805 learned_addr learned my addr 172.21.15.12:0/2871749805 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:27:08.763 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:07.910+0000 7f5fcb7fe700 1 -- 172.21.15.12:0/2871749805 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fcc108620 con 0x7f5fcc107cd0 2024-05-17T22:27:08.763 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:07.910+0000 7f5fcb7fe700 1 --2- 172.21.15.12:0/2871749805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fcc107cd0 0x7f5fcc1080e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc0098a0 tx=0x7f5fbc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6de897684ab7f6b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:08.763 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:07.911+0000 7f5fca7fc700 1 -- 172.21.15.12:0/2871749805 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fbc004160 con 0x7f5fcc107cd0 2024-05-17T22:27:08.763 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:07.911+0000 7f5fca7fc700 1 -- 172.21.15.12:0/2871749805 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fbc004300 con 0x7f5fcc107cd0 2024-05-17T22:27:08.763 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:07.911+0000 7f5fca7fc700 1 -- 172.21.15.12:0/2871749805 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fbc011a20 con 0x7f5fcc107cd0 2024-05-17T22:27:08.763 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:07.911+0000 7f5fd1b67700 1 -- 172.21.15.12:0/2871749805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fcc107cd0 msgr2=0x7f5fcc1080e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:08.763 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:07.911+0000 7f5fd1b67700 1 --2- 172.21.15.12:0/2871749805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fcc107cd0 0x7f5fcc1080e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc0098a0 tx=0x7f5fbc009bb0 comp rx=0 tx=0).stop 2024-05-17T22:27:08.763 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:07.911+0000 7f5fd1b67700 1 -- 172.21.15.12:0/2871749805 shutdown_connections 2024-05-17T22:27:08.763 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:07.911+0000 7f5fd1b67700 1 --2- 172.21.15.12:0/2871749805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fcc107cd0 0x7f5fcc1080e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:08.763 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:07.912+0000 7f5fd1b67700 1 -- 172.21.15.12:0/2871749805 >> 172.21.15.12:0/2871749805 conn(0x7f5fcc103af0 msgr2=0x7f5fcc105fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:27:08.763 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:07.912+0000 7f5fd1b67700 1 -- 172.21.15.12:0/2871749805 shutdown_connections 2024-05-17T22:27:08.764 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:07.912+0000 7f5fd1b67700 1 -- 172.21.15.12:0/2871749805 wait complete. 2024-05-17T22:27:08.764 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:07.913+0000 7f5fd1b67700 1 Processor -- start 2024-05-17T22:27:08.764 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:07.913+0000 7f5fd1b67700 1 -- start start 2024-05-17T22:27:08.764 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:07.913+0000 7f5fd1b67700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fcc107cd0 0x7f5fcc19f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:08.764 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:07.913+0000 7f5fd1b67700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5fcc1096e0 con 0x7f5fcc107cd0 2024-05-17T22:27:08.764 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:07.913+0000 7f5fcb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fcc107cd0 0x7f5fcc19f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:08.764 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:07.913+0000 7f5fcb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fcc107cd0 0x7f5fcc19f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44938/0 (socket says 172.21.15.12:44938) 2024-05-17T22:27:08.764 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:07.913+0000 7f5fcb7fe700 1 -- 172.21.15.12:0/2870045345 learned_addr learned my addr 172.21.15.12:0/2870045345 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:27:08.764 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:07.914+0000 7f5fcb7fe700 1 -- 172.21.15.12:0/2870045345 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fcc19fe30 con 0x7f5fcc107cd0 2024-05-17T22:27:08.764 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:07.914+0000 7f5fcb7fe700 1 --2- 172.21.15.12:0/2870045345 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fcc107cd0 0x7f5fcc19f8f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc000c00 tx=0x7f5fbc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:08.764 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:07.914+0000 7f5fc8ff9700 1 -- 172.21.15.12:0/2870045345 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fbc003aa0 con 0x7f5fcc107cd0 2024-05-17T22:27:08.764 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:07.914+0000 7f5fc8ff9700 1 -- 172.21.15.12:0/2870045345 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fbc01b8f0 con 0x7f5fcc107cd0 2024-05-17T22:27:08.764 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:07.914+0000 7f5fd1b67700 1 -- 172.21.15.12:0/2870045345 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fcc1a00c0 con 0x7f5fcc107cd0 2024-05-17T22:27:08.764 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:07.914+0000 7f5fc8ff9700 1 -- 172.21.15.12:0/2870045345 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fbc023890 con 0x7f5fcc107cd0 2024-05-17T22:27:08.765 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:07.914+0000 7f5fd1b67700 1 -- 172.21.15.12:0/2870045345 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fcc1a2cd0 con 0x7f5fcc107cd0 2024-05-17T22:27:08.765 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:07.915+0000 7f5fc8ff9700 1 -- 172.21.15.12:0/2870045345 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f5fbc02c3f0 con 0x7f5fcc107cd0 2024-05-17T22:27:08.765 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:07.915+0000 7f5fd1b67700 1 -- 172.21.15.12:0/2870045345 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fcc1080e0 con 0x7f5fcc107cd0 2024-05-17T22:27:08.765 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:07.915+0000 7f5fc8ff9700 1 --2- 172.21.15.12:0/2870045345 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f5fb803e5e0 0x7f5fb8040af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:08.765 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:07.916+0000 7f5fc8ff9700 1 -- 172.21.15.12:0/2870045345 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5fbc053e90 con 0x7f5fcc107cd0 2024-05-17T22:27:08.765 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:07.916+0000 7f5fcaffd700 1 -- 172.21.15.12:0/2870045345 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f5fb803e5e0 msgr2=0x7f5fb8040af0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/2250046521 2024-05-17T22:27:08.765 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:07.916+0000 7f5fcaffd700 1 --2- 172.21.15.12:0/2870045345 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f5fb803e5e0 0x7f5fb8040af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-17T22:27:08.765 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:07.921+0000 7f5fc8ff9700 1 -- 172.21.15.12:0/2870045345 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5fbc01f030 con 0x7f5fcc107cd0 2024-05-17T22:27:08.765 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:08.117+0000 7f5fcaffd700 1 -- 172.21.15.12:0/2870045345 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f5fb803e5e0 msgr2=0x7f5fb8040af0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/2250046521 2024-05-17T22:27:08.765 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:08.117+0000 7f5fcaffd700 1 --2- 172.21.15.12:0/2870045345 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f5fb803e5e0 0x7f5fb8040af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-17T22:27:08.765 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:08.243+0000 7f5fd1b67700 1 -- 172.21.15.12:0/2870045345 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f5fcc064510 con 0x7f5fcc107cd0 2024-05-17T22:27:08.765 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:08.243+0000 7f5fc8ff9700 1 -- 172.21.15.12:0/2870045345 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f5fbc00f2e0 con 0x7f5fcc107cd0 2024-05-17T22:27:08.765 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:08.246+0000 7f5fd1b67700 1 -- 172.21.15.12:0/2870045345 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f5fb803e5e0 msgr2=0x7f5fb8040af0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-17T22:27:08.766 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:08.246+0000 7f5fd1b67700 1 --2- 172.21.15.12:0/2870045345 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f5fb803e5e0 0x7f5fb8040af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:08.766 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:08.246+0000 7f5fd1b67700 1 -- 172.21.15.12:0/2870045345 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fcc107cd0 msgr2=0x7f5fcc19f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:08.766 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:08.246+0000 7f5fd1b67700 1 --2- 172.21.15.12:0/2870045345 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fcc107cd0 0x7f5fcc19f8f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc000c00 tx=0x7f5fbc011aa0 comp rx=0 tx=0).stop 2024-05-17T22:27:08.766 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:08.246+0000 7f5fd1b67700 1 -- 172.21.15.12:0/2870045345 shutdown_connections 2024-05-17T22:27:08.766 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:08.246+0000 7f5fd1b67700 1 --2- 172.21.15.12:0/2870045345 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f5fb803e5e0 0x7f5fb8040af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:08.766 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:08.246+0000 7f5fd1b67700 1 --2- 172.21.15.12:0/2870045345 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fcc107cd0 0x7f5fcc19f8f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:08.766 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:08.246+0000 7f5fd1b67700 1 -- 172.21.15.12:0/2870045345 >> 172.21.15.12:0/2870045345 conn(0x7f5fcc103af0 msgr2=0x7f5fcc193900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:27:08.766 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:08.246+0000 7f5fd1b67700 1 -- 172.21.15.12:0/2870045345 shutdown_connections 2024-05-17T22:27:08.766 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:08.246+0000 7f5fd1b67700 1 -- 172.21.15.12:0/2870045345 wait complete. 2024-05-17T22:27:08.766 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for the mgr to restart... 2024-05-17T22:27:08.766 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mgr epoch 9... 2024-05-17T22:27:09.006 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:08 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/2870045345' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-17T22:27:09.341 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:27:09 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:27:09.043+0000 7f3c03ffc200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-17T22:27:09.341 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:27:09 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:27:09.231+0000 7f3c03ffc200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-17T22:27:09.723 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:27:09 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:27:09.413+0000 7f3c03ffc200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-17T22:27:09.723 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:27:09 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:27:09.721+0000 7f3c03ffc200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-17T22:27:10.036 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:27:09 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:27:09.820+0000 7f3c03ffc200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-17T22:27:10.036 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:27:10 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:27:10.034+0000 7f3c03ffc200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-17T22:27:10.340 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:27:10 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:27:10.240+0000 7f3c03ffc200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-17T22:27:11.090 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:27:10 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:27:10.879+0000 7f3c03ffc200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-17T22:27:11.568 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:27:11 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:27:11.116+0000 7f3c03ffc200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-17T22:27:11.841 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:27:11 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:27:11.566+0000 7f3c03ffc200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-17T22:27:11.841 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:27:11 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:27:11.670+0000 7f3c03ffc200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-17T22:27:12.341 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:27:12 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:27:12.061+0000 7f3c03ffc200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-17T22:27:12.805 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:27:12 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:27:12.440+0000 7f3c03ffc200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-17T22:27:12.805 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:27:12 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:27:12.541+0000 7f3c03ffc200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-17T22:27:13.090 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:27:12 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:27:12.803+0000 7f3c03ffc200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-17T22:27:13.091 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:27:12 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:27:12.946+0000 7f3c03ffc200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-17T22:27:13.091 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:27:13 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:27:13.050+0000 7f3c03ffc200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-17T22:27:13.841 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:27:13 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:27:13.392+0000 7f3c03ffc200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-17T22:27:13.841 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:27:13 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:27:13.492+0000 7f3c03ffc200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-17T22:27:14.426 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:27:14 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:27:14.136+0000 7f3c03ffc200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-17T22:27:14.426 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:27:14 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:27:14.370+0000 7f3c03ffc200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-17T22:27:14.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:14 smithi012 ceph-mon[31632]: Active manager daemon x restarted 2024-05-17T22:27:14.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:14 smithi012 ceph-mon[31632]: Activating manager daemon x 2024-05-17T22:27:14.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:14 smithi012 ceph-mon[31632]: osdmap e3: 0 total, 0 up, 0 in 2024-05-17T22:27:14.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:14 smithi012 ceph-mon[31632]: mgrmap e10: x(active, starting, since 0.00413213s) 2024-05-17T22:27:14.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:14 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T22:27:14.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:14 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-17T22:27:14.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:14 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-17T22:27:14.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:14 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-17T22:27:14.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:14 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-17T22:27:14.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:14 smithi012 ceph-mon[31632]: Manager daemon x is now available 2024-05-17T22:27:14.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:14 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:27:15.730 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:15 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-17T22:27:15.730 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:15 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-17T22:27:15.730 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:15 smithi012 ceph-mon[31632]: [17/May/2024:22:27:14] ENGINE Bus STARTING 2024-05-17T22:27:15.730 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:15 smithi012 ceph-mon[31632]: mgrmap e11: x(active, since 1.00728s) 2024-05-17T22:27:16.298 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-05-17T22:27:16.298 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-17T22:27:16.298 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-17T22:27:16.298 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-05-17T22:27:16.298 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:09.221+0000 7f41712bb700 1 Processor -- start 2024-05-17T22:27:16.298 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:09.221+0000 7f41712bb700 1 -- start start 2024-05-17T22:27:16.298 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:09.221+0000 7f41712bb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f416c1093d0 0x7f416c1097e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:16.298 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:09.221+0000 7f41712bb700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f416c109db0 con 0x7f416c1093d0 2024-05-17T22:27:16.299 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:09.221+0000 7f416affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f416c1093d0 0x7f416c1097e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:16.299 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:09.222+0000 7f416affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f416c1093d0 0x7f416c1097e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44952/0 (socket says 172.21.15.12:44952) 2024-05-17T22:27:16.299 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:09.222+0000 7f416affd700 1 -- 172.21.15.12:0/1657747819 learned_addr learned my addr 172.21.15.12:0/1657747819 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:27:16.299 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:09.222+0000 7f416affd700 1 -- 172.21.15.12:0/1657747819 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f416c10a5a0 con 0x7f416c1093d0 2024-05-17T22:27:16.299 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:09.222+0000 7f416affd700 1 --2- 172.21.15.12:0/1657747819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f416c1093d0 0x7f416c1097e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f41540098a0 tx=0x7f4154009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e53f67f7b6a492e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:16.299 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:09.222+0000 7f4169ffb700 1 -- 172.21.15.12:0/1657747819 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4154004160 con 0x7f416c1093d0 2024-05-17T22:27:16.299 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:09.222+0000 7f4169ffb700 1 -- 172.21.15.12:0/1657747819 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4154004300 con 0x7f416c1093d0 2024-05-17T22:27:16.299 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:09.222+0000 7f4169ffb700 1 -- 172.21.15.12:0/1657747819 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4154011a20 con 0x7f416c1093d0 2024-05-17T22:27:16.299 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:09.223+0000 7f41712bb700 1 -- 172.21.15.12:0/1657747819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f416c1093d0 msgr2=0x7f416c1097e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:16.299 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:09.223+0000 7f41712bb700 1 --2- 172.21.15.12:0/1657747819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f416c1093d0 0x7f416c1097e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f41540098a0 tx=0x7f4154009bb0 comp rx=0 tx=0).stop 2024-05-17T22:27:16.299 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:09.223+0000 7f41712bb700 1 -- 172.21.15.12:0/1657747819 shutdown_connections 2024-05-17T22:27:16.299 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:09.223+0000 7f41712bb700 1 --2- 172.21.15.12:0/1657747819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f416c1093d0 0x7f416c1097e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:16.299 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:09.223+0000 7f41712bb700 1 -- 172.21.15.12:0/1657747819 >> 172.21.15.12:0/1657747819 conn(0x7f416c104c80 msgr2=0x7f416c1070d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:27:16.299 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:09.223+0000 7f41712bb700 1 -- 172.21.15.12:0/1657747819 shutdown_connections 2024-05-17T22:27:16.300 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:09.223+0000 7f41712bb700 1 -- 172.21.15.12:0/1657747819 wait complete. 2024-05-17T22:27:16.300 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:09.224+0000 7f41712bb700 1 Processor -- start 2024-05-17T22:27:16.300 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:09.224+0000 7f41712bb700 1 -- start start 2024-05-17T22:27:16.300 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:09.225+0000 7f41712bb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f416c1093d0 0x7f416c19d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:16.300 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:09.225+0000 7f41712bb700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f416c10d950 con 0x7f416c1093d0 2024-05-17T22:27:16.300 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:09.225+0000 7f416affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f416c1093d0 0x7f416c19d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:16.300 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:09.225+0000 7f416affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f416c1093d0 0x7f416c19d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44968/0 (socket says 172.21.15.12:44968) 2024-05-17T22:27:16.300 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:09.225+0000 7f416affd700 1 -- 172.21.15.12:0/821193419 learned_addr learned my addr 172.21.15.12:0/821193419 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:27:16.300 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:09.225+0000 7f416affd700 1 -- 172.21.15.12:0/821193419 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f416c19dc40 con 0x7f416c1093d0 2024-05-17T22:27:16.300 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:09.225+0000 7f416affd700 1 --2- 172.21.15.12:0/821193419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f416c1093d0 0x7f416c19d700 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4154009520 tx=0x7f4154011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:16.300 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:09.225+0000 7f4163fff700 1 -- 172.21.15.12:0/821193419 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41540039e0 con 0x7f416c1093d0 2024-05-17T22:27:16.300 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:09.225+0000 7f41712bb700 1 -- 172.21.15.12:0/821193419 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f416c19ded0 con 0x7f416c1093d0 2024-05-17T22:27:16.300 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:09.225+0000 7f4163fff700 1 -- 172.21.15.12:0/821193419 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f415401b8f0 con 0x7f416c1093d0 2024-05-17T22:27:16.301 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:09.225+0000 7f41712bb700 1 -- 172.21.15.12:0/821193419 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f416c1a0ae0 con 0x7f416c1093d0 2024-05-17T22:27:16.301 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:09.225+0000 7f4163fff700 1 -- 172.21.15.12:0/821193419 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41540197d0 con 0x7f416c1093d0 2024-05-17T22:27:16.301 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:09.227+0000 7f4163fff700 1 -- 172.21.15.12:0/821193419 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f4154019ac0 con 0x7f416c1093d0 2024-05-17T22:27:16.301 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:09.227+0000 7f4163fff700 1 --2- 172.21.15.12:0/821193419 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f415803e5e0 0x7f4158040af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:16.301 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:09.227+0000 7f4163fff700 1 -- 172.21.15.12:0/821193419 --> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4158041200 con 0x7f415803e5e0 2024-05-17T22:27:16.301 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:09.227+0000 7f4163fff700 1 -- 172.21.15.12:0/821193419 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4154053fc0 con 0x7f416c1093d0 2024-05-17T22:27:16.301 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:09.227+0000 7f416a7fc700 1 -- 172.21.15.12:0/821193419 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f415803e5e0 msgr2=0x7f4158040af0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/2250046521 2024-05-17T22:27:16.301 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:09.227+0000 7f416a7fc700 1 --2- 172.21.15.12:0/821193419 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f415803e5e0 0x7f4158040af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-17T22:27:16.301 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:09.428+0000 7f416a7fc700 1 -- 172.21.15.12:0/821193419 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f415803e5e0 msgr2=0x7f4158040af0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/2250046521 2024-05-17T22:27:16.301 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:09.428+0000 7f416a7fc700 1 --2- 172.21.15.12:0/821193419 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f415803e5e0 0x7f4158040af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-17T22:27:16.301 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:09.829+0000 7f416a7fc700 1 -- 172.21.15.12:0/821193419 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f415803e5e0 msgr2=0x7f4158040af0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/2250046521 2024-05-17T22:27:16.301 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:09.829+0000 7f416a7fc700 1 --2- 172.21.15.12:0/821193419 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f415803e5e0 0x7f4158040af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-17T22:27:16.302 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:10.630+0000 7f416a7fc700 1 -- 172.21.15.12:0/821193419 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f415803e5e0 msgr2=0x7f4158040af0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/2250046521 2024-05-17T22:27:16.302 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:10.630+0000 7f416a7fc700 1 --2- 172.21.15.12:0/821193419 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f415803e5e0 0x7f4158040af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-17T22:27:16.302 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:12.231+0000 7f416a7fc700 1 -- 172.21.15.12:0/821193419 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f415803e5e0 msgr2=0x7f4158040af0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/2250046521 2024-05-17T22:27:16.302 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:12.231+0000 7f416a7fc700 1 --2- 172.21.15.12:0/821193419 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f415803e5e0 0x7f4158040af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-17T22:27:16.302 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:14.374+0000 7f4163fff700 1 -- 172.21.15.12:0/821193419 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mgrmap(e 10) ==== 50002+0+0 (secure 0 0 0) 0x7f415402a350 con 0x7f416c1093d0 2024-05-17T22:27:16.302 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:14.375+0000 7f4163fff700 1 -- 172.21.15.12:0/821193419 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f415803e5e0 msgr2=0x7f4158040af0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-17T22:27:16.302 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:14.375+0000 7f4163fff700 1 --2- 172.21.15.12:0/821193419 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f415803e5e0 0x7f4158040af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:16.302 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:15.376+0000 7f4163fff700 1 -- 172.21.15.12:0/821193419 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7f4154007cf0 con 0x7f416c1093d0 2024-05-17T22:27:16.302 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:15.377+0000 7f4163fff700 1 --2- 172.21.15.12:0/821193419 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f4154053b50 0x7f4158043fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:16.302 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:15.377+0000 7f4163fff700 1 -- 172.21.15.12:0/821193419 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4154008e50 con 0x7f4154053b50 2024-05-17T22:27:16.302 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:15.377+0000 7f416a7fc700 1 --2- 172.21.15.12:0/821193419 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f4154053b50 0x7f4158043fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:16.302 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:15.378+0000 7f416a7fc700 1 --2- 172.21.15.12:0/821193419 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f4154053b50 0x7f4158043fd0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f415c003a10 tx=0x7f415c009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:16.302 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:15.379+0000 7f4163fff700 1 -- 172.21.15.12:0/821193419 <== mgr.14152 v2:172.21.15.12:6800/1388514047 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f4154008e50 con 0x7f4154053b50 2024-05-17T22:27:16.303 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:15.387+0000 7f41712bb700 1 -- 172.21.15.12:0/821193419 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f416c1097e0 con 0x7f4154053b50 2024-05-17T22:27:16.303 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:15.387+0000 7f4163fff700 1 -- 172.21.15.12:0/821193419 <== mgr.14152 v2:172.21.15.12:6800/1388514047 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f416c1097e0 con 0x7f4154053b50 2024-05-17T22:27:16.303 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:15.387+0000 7f41712bb700 1 -- 172.21.15.12:0/821193419 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f4154053b50 msgr2=0x7f4158043fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:16.303 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:15.387+0000 7f41712bb700 1 --2- 172.21.15.12:0/821193419 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f4154053b50 0x7f4158043fd0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f415c003a10 tx=0x7f415c009270 comp rx=0 tx=0).stop 2024-05-17T22:27:16.303 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:15.387+0000 7f41712bb700 1 -- 172.21.15.12:0/821193419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f416c1093d0 msgr2=0x7f416c19d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:16.303 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:15.387+0000 7f41712bb700 1 --2- 172.21.15.12:0/821193419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f416c1093d0 0x7f416c19d700 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4154009520 tx=0x7f4154011aa0 comp rx=0 tx=0).stop 2024-05-17T22:27:16.303 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:15.387+0000 7f41712bb700 1 -- 172.21.15.12:0/821193419 shutdown_connections 2024-05-17T22:27:16.303 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:15.387+0000 7f41712bb700 1 --2- 172.21.15.12:0/821193419 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f4154053b50 0x7f4158043fd0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:16.303 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:15.387+0000 7f41712bb700 1 --2- 172.21.15.12:0/821193419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f416c1093d0 0x7f416c19d700 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:16.303 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:15.387+0000 7f41712bb700 1 --2- 172.21.15.12:0/821193419 >> [v2:172.21.15.12:6800/2250046521,v1:172.21.15.12:6801/2250046521] conn(0x7f415803e5e0 0x7f4158040af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:16.303 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:15.387+0000 7f41712bb700 1 -- 172.21.15.12:0/821193419 >> 172.21.15.12:0/821193419 conn(0x7f416c104c80 msgr2=0x7f416c106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:27:16.303 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:15.388+0000 7f41712bb700 1 -- 172.21.15.12:0/821193419 shutdown_connections 2024-05-17T22:27:16.303 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:15.388+0000 7f41712bb700 1 -- 172.21.15.12:0/821193419 wait complete. 2024-05-17T22:27:16.304 INFO:teuthology.orchestra.run.smithi012.stdout:mgr epoch 9 is available 2024-05-17T22:27:16.304 INFO:teuthology.orchestra.run.smithi012.stdout:Generating a dashboard self-signed certificate... 2024-05-17T22:27:16.793 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:16 smithi012 ceph-mon[31632]: [17/May/2024:22:27:14] ENGINE Serving on https://172.21.15.12:7150 2024-05-17T22:27:16.793 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:16 smithi012 ceph-mon[31632]: [17/May/2024:22:27:14] ENGINE Serving on http://172.21.15.12:8765 2024-05-17T22:27:16.793 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:16 smithi012 ceph-mon[31632]: [17/May/2024:22:27:14] ENGINE Bus STARTED 2024-05-17T22:27:16.793 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:16 smithi012 ceph-mon[31632]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-17T22:27:16.793 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:16 smithi012 ceph-mon[31632]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-17T22:27:16.794 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:16 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:27:17.745 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-17T22:27:17.745 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:16.748+0000 7f11203c2700 1 Processor -- start 2024-05-17T22:27:17.745 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:16.748+0000 7f11203c2700 1 -- start start 2024-05-17T22:27:17.745 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:16.749+0000 7f11203c2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1118109e50 0x7f111810c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:17.745 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:16.749+0000 7f11203c2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f111807ce10 con 0x7f1118109e50 2024-05-17T22:27:17.745 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:16.749+0000 7f111e15e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1118109e50 0x7f111810c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:17.745 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:16.749+0000 7f111e15e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1118109e50 0x7f111810c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32978/0 (socket says 172.21.15.12:32978) 2024-05-17T22:27:17.745 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:16.749+0000 7f111e15e700 1 -- 172.21.15.12:0/2161393042 learned_addr learned my addr 172.21.15.12:0/2161393042 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:27:17.745 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:16.749+0000 7f111e15e700 1 -- 172.21.15.12:0/2161393042 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f111810c7b0 con 0x7f1118109e50 2024-05-17T22:27:17.745 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:16.750+0000 7f111e15e700 1 --2- 172.21.15.12:0/2161393042 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1118109e50 0x7f111810c270 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f11140098a0 tx=0x7f1114009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24767fba2d9dc277 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:17.746 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:16.750+0000 7f111d15c700 1 -- 172.21.15.12:0/2161393042 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1114004160 con 0x7f1118109e50 2024-05-17T22:27:17.746 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:16.750+0000 7f111d15c700 1 -- 172.21.15.12:0/2161393042 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1114004300 con 0x7f1118109e50 2024-05-17T22:27:17.746 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:16.750+0000 7f111d15c700 1 -- 172.21.15.12:0/2161393042 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1114011a20 con 0x7f1118109e50 2024-05-17T22:27:17.746 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:16.751+0000 7f11203c2700 1 -- 172.21.15.12:0/2161393042 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1118109e50 msgr2=0x7f111810c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:17.746 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:16.751+0000 7f11203c2700 1 --2- 172.21.15.12:0/2161393042 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1118109e50 0x7f111810c270 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f11140098a0 tx=0x7f1114009bb0 comp rx=0 tx=0).stop 2024-05-17T22:27:17.746 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:16.751+0000 7f11203c2700 1 -- 172.21.15.12:0/2161393042 shutdown_connections 2024-05-17T22:27:17.746 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:16.751+0000 7f11203c2700 1 --2- 172.21.15.12:0/2161393042 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1118109e50 0x7f111810c270 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:17.746 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:16.751+0000 7f11203c2700 1 -- 172.21.15.12:0/2161393042 >> 172.21.15.12:0/2161393042 conn(0x7f1118103a80 msgr2=0x7f1118105f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:27:17.746 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:16.751+0000 7f11203c2700 1 -- 172.21.15.12:0/2161393042 shutdown_connections 2024-05-17T22:27:17.746 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:16.751+0000 7f11203c2700 1 -- 172.21.15.12:0/2161393042 wait complete. 2024-05-17T22:27:17.746 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:16.752+0000 7f11203c2700 1 Processor -- start 2024-05-17T22:27:17.746 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:16.752+0000 7f11203c2700 1 -- start start 2024-05-17T22:27:17.746 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:16.752+0000 7f11203c2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1118109e50 0x7f111819d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:17.746 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:16.752+0000 7f11203c2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f111810d7b0 con 0x7f1118109e50 2024-05-17T22:27:17.747 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:16.752+0000 7f111e15e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1118109e50 0x7f111819d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:17.747 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:16.752+0000 7f111e15e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1118109e50 0x7f111819d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32990/0 (socket says 172.21.15.12:32990) 2024-05-17T22:27:17.747 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:16.752+0000 7f111e15e700 1 -- 172.21.15.12:0/1291229804 learned_addr learned my addr 172.21.15.12:0/1291229804 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:27:17.747 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:16.752+0000 7f111e15e700 1 -- 172.21.15.12:0/1291229804 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f111819dc00 con 0x7f1118109e50 2024-05-17T22:27:17.747 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:16.753+0000 7f111e15e700 1 --2- 172.21.15.12:0/1291229804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1118109e50 0x7f111819d6c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f11140119f0 tx=0x7f1114011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:17.747 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:16.753+0000 7f110b7fe700 1 -- 172.21.15.12:0/1291229804 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11140039e0 con 0x7f1118109e50 2024-05-17T22:27:17.747 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:16.753+0000 7f11203c2700 1 -- 172.21.15.12:0/1291229804 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f111819de90 con 0x7f1118109e50 2024-05-17T22:27:17.747 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:16.753+0000 7f110b7fe700 1 -- 172.21.15.12:0/1291229804 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f111401b8f0 con 0x7f1118109e50 2024-05-17T22:27:17.747 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:16.753+0000 7f110b7fe700 1 -- 172.21.15.12:0/1291229804 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11140237a0 con 0x7f1118109e50 2024-05-17T22:27:17.747 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:16.753+0000 7f11203c2700 1 -- 172.21.15.12:0/1291229804 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11181a0aa0 con 0x7f1118109e50 2024-05-17T22:27:17.747 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:16.754+0000 7f11203c2700 1 -- 172.21.15.12:0/1291229804 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f111810a260 con 0x7f1118109e50 2024-05-17T22:27:17.747 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:16.754+0000 7f110b7fe700 1 -- 172.21.15.12:0/1291229804 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7f111402c3f0 con 0x7f1118109e50 2024-05-17T22:27:17.747 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:16.755+0000 7f110b7fe700 1 --2- 172.21.15.12:0/1291229804 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f110403e420 0x7f1104040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:17.747 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:16.755+0000 7f110b7fe700 1 -- 172.21.15.12:0/1291229804 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1114053de0 con 0x7f1118109e50 2024-05-17T22:27:17.748 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:16.755+0000 7f111d95d700 1 --2- 172.21.15.12:0/1291229804 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f110403e420 0x7f1104040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:17.748 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:16.755+0000 7f111d95d700 1 --2- 172.21.15.12:0/1291229804 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f110403e420 0x7f1104040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f110c006fd0 tx=0x7f110c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:17.748 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:16.760+0000 7f110b7fe700 1 -- 172.21.15.12:0/1291229804 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f111401f030 con 0x7f1118109e50 2024-05-17T22:27:17.748 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:16.791+0000 7f110b7fe700 1 -- 172.21.15.12:0/1291229804 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f111402cca0 con 0x7f1118109e50 2024-05-17T22:27:17.748 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:16.990+0000 7f11203c2700 1 -- 172.21.15.12:0/1291229804 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f1118104a90 con 0x7f110403e420 2024-05-17T22:27:17.748 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:17.101+0000 7f110b7fe700 1 -- 172.21.15.12:0/1291229804 <== mgr.14152 v2:172.21.15.12:6800/1388514047 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f1118104a90 con 0x7f110403e420 2024-05-17T22:27:17.748 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:17.103+0000 7f11203c2700 1 -- 172.21.15.12:0/1291229804 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f110403e420 msgr2=0x7f1104040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:17.748 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:17.103+0000 7f11203c2700 1 --2- 172.21.15.12:0/1291229804 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f110403e420 0x7f1104040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f110c006fd0 tx=0x7f110c006e40 comp rx=0 tx=0).stop 2024-05-17T22:27:17.748 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:17.103+0000 7f11203c2700 1 -- 172.21.15.12:0/1291229804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1118109e50 msgr2=0x7f111819d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:17.748 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:17.103+0000 7f11203c2700 1 --2- 172.21.15.12:0/1291229804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1118109e50 0x7f111819d6c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f11140119f0 tx=0x7f1114011ad0 comp rx=0 tx=0).stop 2024-05-17T22:27:17.748 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:17.103+0000 7f11203c2700 1 -- 172.21.15.12:0/1291229804 shutdown_connections 2024-05-17T22:27:17.748 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:17.103+0000 7f11203c2700 1 --2- 172.21.15.12:0/1291229804 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f110403e420 0x7f1104040930 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:17.749 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:17.103+0000 7f11203c2700 1 --2- 172.21.15.12:0/1291229804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1118109e50 0x7f111819d6c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:17.749 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:17.103+0000 7f11203c2700 1 -- 172.21.15.12:0/1291229804 >> 172.21.15.12:0/1291229804 conn(0x7f1118103a80 msgr2=0x7f1118104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:27:17.749 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:17.104+0000 7f11203c2700 1 -- 172.21.15.12:0/1291229804 shutdown_connections 2024-05-17T22:27:17.749 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:17.104+0000 7f11203c2700 1 -- 172.21.15.12:0/1291229804 wait complete. 2024-05-17T22:27:17.749 INFO:teuthology.orchestra.run.smithi012.stdout:Creating initial admin user... 2024-05-17T22:27:18.010 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:17 smithi012 ceph-mon[31632]: mgrmap e12: x(active, since 2s) 2024-05-17T22:27:18.010 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:17 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:27:18.010 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:17 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:27:18.010 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:17 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:27:19.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:18 smithi012 ceph-mon[31632]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:27:19.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:18 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:27:19.982 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$Hz3Q7CQQgUl4jqNIgeI/Ru2hjM8MPE79J4dc6JpgkXvlV3C4EELqy", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715984838, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-17T22:27:19.983 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.245+0000 7f163eb9b700 1 Processor -- start 2024-05-17T22:27:19.983 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.245+0000 7f163eb9b700 1 -- start start 2024-05-17T22:27:19.983 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.246+0000 7f163eb9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1638109870 0x7f1638109c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:19.983 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.246+0000 7f163eb9b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f163810a250 con 0x7f1638109870 2024-05-17T22:27:19.983 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.246+0000 7f163c937700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1638109870 0x7f1638109c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:19.983 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.246+0000 7f163c937700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1638109870 0x7f1638109c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32994/0 (socket says 172.21.15.12:32994) 2024-05-17T22:27:19.983 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.246+0000 7f163c937700 1 -- 172.21.15.12:0/3552735644 learned_addr learned my addr 172.21.15.12:0/3552735644 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:27:19.983 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.246+0000 7f163c937700 1 -- 172.21.15.12:0/3552735644 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16381084a0 con 0x7f1638109870 2024-05-17T22:27:19.983 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.246+0000 7f163c937700 1 --2- 172.21.15.12:0/3552735644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1638109870 0x7f1638109c80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f162c0098a0 tx=0x7f162c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d54ad3ccf20aa5f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:19.983 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.247+0000 7f16377fe700 1 -- 172.21.15.12:0/3552735644 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f162c004160 con 0x7f1638109870 2024-05-17T22:27:19.983 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.247+0000 7f16377fe700 1 -- 172.21.15.12:0/3552735644 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f162c004300 con 0x7f1638109870 2024-05-17T22:27:19.983 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.247+0000 7f16377fe700 1 -- 172.21.15.12:0/3552735644 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f162c011a20 con 0x7f1638109870 2024-05-17T22:27:19.983 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.247+0000 7f163eb9b700 1 -- 172.21.15.12:0/3552735644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1638109870 msgr2=0x7f1638109c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:19.984 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.247+0000 7f163eb9b700 1 --2- 172.21.15.12:0/3552735644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1638109870 0x7f1638109c80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f162c0098a0 tx=0x7f162c009bb0 comp rx=0 tx=0).stop 2024-05-17T22:27:19.984 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.247+0000 7f163eb9b700 1 -- 172.21.15.12:0/3552735644 shutdown_connections 2024-05-17T22:27:19.984 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.247+0000 7f163eb9b700 1 --2- 172.21.15.12:0/3552735644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1638109870 0x7f1638109c80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:19.984 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.247+0000 7f163eb9b700 1 -- 172.21.15.12:0/3552735644 >> 172.21.15.12:0/3552735644 conn(0x7f1638103230 msgr2=0x7f1638105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:27:19.984 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.248+0000 7f163eb9b700 1 -- 172.21.15.12:0/3552735644 shutdown_connections 2024-05-17T22:27:19.984 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.248+0000 7f163eb9b700 1 -- 172.21.15.12:0/3552735644 wait complete. 2024-05-17T22:27:19.984 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.248+0000 7f163eb9b700 1 Processor -- start 2024-05-17T22:27:19.984 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.248+0000 7f163eb9b700 1 -- start start 2024-05-17T22:27:19.984 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.248+0000 7f163eb9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1638109870 0x7f163819c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:19.984 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.248+0000 7f163eb9b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f163810c860 con 0x7f1638109870 2024-05-17T22:27:19.984 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.248+0000 7f163c937700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1638109870 0x7f163819c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:19.984 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.249+0000 7f163c937700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1638109870 0x7f163819c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33004/0 (socket says 172.21.15.12:33004) 2024-05-17T22:27:19.984 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.249+0000 7f163c937700 1 -- 172.21.15.12:0/1913697853 learned_addr learned my addr 172.21.15.12:0/1913697853 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:27:19.985 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.249+0000 7f163c937700 1 -- 172.21.15.12:0/1913697853 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f163819cb90 con 0x7f1638109870 2024-05-17T22:27:19.985 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.249+0000 7f163c937700 1 --2- 172.21.15.12:0/1913697853 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1638109870 0x7f163819c650 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f162c011a20 tx=0x7f162c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:19.985 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.249+0000 7f1635ffb700 1 -- 172.21.15.12:0/1913697853 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f162c003af0 con 0x7f1638109870 2024-05-17T22:27:19.985 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.249+0000 7f163eb9b700 1 -- 172.21.15.12:0/1913697853 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f163819ce20 con 0x7f1638109870 2024-05-17T22:27:19.985 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.249+0000 7f1635ffb700 1 -- 172.21.15.12:0/1913697853 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f162c01b8f0 con 0x7f1638109870 2024-05-17T22:27:19.985 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.249+0000 7f1635ffb700 1 -- 172.21.15.12:0/1913697853 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f162c023890 con 0x7f1638109870 2024-05-17T22:27:19.985 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.249+0000 7f163eb9b700 1 -- 172.21.15.12:0/1913697853 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f163819fa30 con 0x7f1638109870 2024-05-17T22:27:19.985 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.250+0000 7f1635ffb700 1 -- 172.21.15.12:0/1913697853 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f162c02c3f0 con 0x7f1638109870 2024-05-17T22:27:19.985 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.250+0000 7f163eb9b700 1 -- 172.21.15.12:0/1913697853 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f163819d280 con 0x7f1638109870 2024-05-17T22:27:19.985 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.251+0000 7f1635ffb700 1 --2- 172.21.15.12:0/1913697853 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f162803e530 0x7f1628040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:19.985 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.251+0000 7f1635ffb700 1 -- 172.21.15.12:0/1913697853 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f162c054070 con 0x7f1638109870 2024-05-17T22:27:19.985 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.251+0000 7f1637fff700 1 --2- 172.21.15.12:0/1913697853 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f162803e530 0x7f1628040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:19.986 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.251+0000 7f1637fff700 1 --2- 172.21.15.12:0/1913697853 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f162803e530 0x7f1628040a40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1624006fd0 tx=0x7f1624006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:19.986 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.256+0000 7f1635ffb700 1 -- 172.21.15.12:0/1913697853 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f162c01f030 con 0x7f1638109870 2024-05-17T22:27:19.986 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.515+0000 7f163eb9b700 1 -- 172.21.15.12:0/1913697853 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f1638104220 con 0x7f162803e530 2024-05-17T22:27:19.986 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.762+0000 7f1635ffb700 1 -- 172.21.15.12:0/1913697853 <== mgr.14152 v2:172.21.15.12:6800/1388514047 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f1638104220 con 0x7f162803e530 2024-05-17T22:27:19.986 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.765+0000 7f163eb9b700 1 -- 172.21.15.12:0/1913697853 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f162803e530 msgr2=0x7f1628040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:19.986 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.765+0000 7f163eb9b700 1 --2- 172.21.15.12:0/1913697853 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f162803e530 0x7f1628040a40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1624006fd0 tx=0x7f1624006e40 comp rx=0 tx=0).stop 2024-05-17T22:27:19.986 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.765+0000 7f163eb9b700 1 -- 172.21.15.12:0/1913697853 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1638109870 msgr2=0x7f163819c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:19.986 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.765+0000 7f163eb9b700 1 --2- 172.21.15.12:0/1913697853 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1638109870 0x7f163819c650 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f162c011a20 tx=0x7f162c011b00 comp rx=0 tx=0).stop 2024-05-17T22:27:19.986 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.765+0000 7f163eb9b700 1 -- 172.21.15.12:0/1913697853 shutdown_connections 2024-05-17T22:27:19.986 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.765+0000 7f163eb9b700 1 --2- 172.21.15.12:0/1913697853 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f162803e530 0x7f1628040a40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:19.986 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.765+0000 7f163eb9b700 1 --2- 172.21.15.12:0/1913697853 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1638109870 0x7f163819c650 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:19.986 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.765+0000 7f163eb9b700 1 -- 172.21.15.12:0/1913697853 >> 172.21.15.12:0/1913697853 conn(0x7f1638103230 msgr2=0x7f1638103b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:27:19.986 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.765+0000 7f163eb9b700 1 -- 172.21.15.12:0/1913697853 shutdown_connections 2024-05-17T22:27:19.987 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:18.765+0000 7f163eb9b700 1 -- 172.21.15.12:0/1913697853 wait complete. 2024-05-17T22:27:19.987 INFO:teuthology.orchestra.run.smithi012.stdout:Fetching dashboard port number... 2024-05-17T22:27:20.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:19 smithi012 ceph-mon[31632]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:27:20.932 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:20 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/2041745539' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-17T22:27:21.166 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 8443 2024-05-17T22:27:21.166 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.403+0000 7fbfef80c700 1 Processor -- start 2024-05-17T22:27:21.166 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.403+0000 7fbfef80c700 1 -- start start 2024-05-17T22:27:21.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.403+0000 7fbfef80c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfe8107cc0 0x7fbfe81080d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:21.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.403+0000 7fbfef80c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe807cd90 con 0x7fbfe8107cc0 2024-05-17T22:27:21.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.403+0000 7fbfed5a8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfe8107cc0 0x7fbfe81080d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:21.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.403+0000 7fbfed5a8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfe8107cc0 0x7fbfe81080d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40236/0 (socket says 172.21.15.12:40236) 2024-05-17T22:27:21.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.403+0000 7fbfed5a8700 1 -- 172.21.15.12:0/2977944391 learned_addr learned my addr 172.21.15.12:0/2977944391 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:27:21.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.404+0000 7fbfed5a8700 1 -- 172.21.15.12:0/2977944391 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfe8108610 con 0x7fbfe8107cc0 2024-05-17T22:27:21.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.404+0000 7fbfed5a8700 1 --2- 172.21.15.12:0/2977944391 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfe8107cc0 0x7fbfe81080d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbfe40098a0 tx=0x7fbfe4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c5ab2a19ba2f4c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:21.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.404+0000 7fbfdffff700 1 -- 172.21.15.12:0/2977944391 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfe4004160 con 0x7fbfe8107cc0 2024-05-17T22:27:21.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.404+0000 7fbfdffff700 1 -- 172.21.15.12:0/2977944391 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfe4004300 con 0x7fbfe8107cc0 2024-05-17T22:27:21.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.404+0000 7fbfdffff700 1 -- 172.21.15.12:0/2977944391 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfe4011a20 con 0x7fbfe8107cc0 2024-05-17T22:27:21.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.405+0000 7fbfef80c700 1 -- 172.21.15.12:0/2977944391 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfe8107cc0 msgr2=0x7fbfe81080d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:21.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.405+0000 7fbfef80c700 1 --2- 172.21.15.12:0/2977944391 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfe8107cc0 0x7fbfe81080d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbfe40098a0 tx=0x7fbfe4009bb0 comp rx=0 tx=0).stop 2024-05-17T22:27:21.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.405+0000 7fbfef80c700 1 -- 172.21.15.12:0/2977944391 shutdown_connections 2024-05-17T22:27:21.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.405+0000 7fbfef80c700 1 --2- 172.21.15.12:0/2977944391 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfe8107cc0 0x7fbfe81080d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:21.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.405+0000 7fbfef80c700 1 -- 172.21.15.12:0/2977944391 >> 172.21.15.12:0/2977944391 conn(0x7fbfe8103a80 msgr2=0x7fbfe8105f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:27:21.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.405+0000 7fbfef80c700 1 -- 172.21.15.12:0/2977944391 shutdown_connections 2024-05-17T22:27:21.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.405+0000 7fbfef80c700 1 -- 172.21.15.12:0/2977944391 wait complete. 2024-05-17T22:27:21.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.406+0000 7fbfef80c700 1 Processor -- start 2024-05-17T22:27:21.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.406+0000 7fbfef80c700 1 -- start start 2024-05-17T22:27:21.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.406+0000 7fbfef80c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfe8107cc0 0x7fbfe819f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:21.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.406+0000 7fbfef80c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe8109550 con 0x7fbfe8107cc0 2024-05-17T22:27:21.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.406+0000 7fbfed5a8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfe8107cc0 0x7fbfe819f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:21.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.406+0000 7fbfed5a8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfe8107cc0 0x7fbfe819f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40238/0 (socket says 172.21.15.12:40238) 2024-05-17T22:27:21.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.406+0000 7fbfed5a8700 1 -- 172.21.15.12:0/2041745539 learned_addr learned my addr 172.21.15.12:0/2041745539 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:27:21.169 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.406+0000 7fbfed5a8700 1 -- 172.21.15.12:0/2041745539 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfe819fe40 con 0x7fbfe8107cc0 2024-05-17T22:27:21.169 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.406+0000 7fbfed5a8700 1 --2- 172.21.15.12:0/2041745539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfe8107cc0 0x7fbfe819f900 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4000c00 tx=0x7fbfe4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:21.169 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.407+0000 7fbfde7fc700 1 -- 172.21.15.12:0/2041745539 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfe4003aa0 con 0x7fbfe8107cc0 2024-05-17T22:27:21.169 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.407+0000 7fbfef80c700 1 -- 172.21.15.12:0/2041745539 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfe81a00d0 con 0x7fbfe8107cc0 2024-05-17T22:27:21.169 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.407+0000 7fbfde7fc700 1 -- 172.21.15.12:0/2041745539 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfe401b8f0 con 0x7fbfe8107cc0 2024-05-17T22:27:21.169 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.407+0000 7fbfde7fc700 1 -- 172.21.15.12:0/2041745539 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfe4023890 con 0x7fbfe8107cc0 2024-05-17T22:27:21.169 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.407+0000 7fbfef80c700 1 -- 172.21.15.12:0/2041745539 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfe81a2ce0 con 0x7fbfe8107cc0 2024-05-17T22:27:21.169 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.408+0000 7fbfef80c700 1 -- 172.21.15.12:0/2041745539 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfe8109b80 con 0x7fbfe8107cc0 2024-05-17T22:27:21.169 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.408+0000 7fbfde7fc700 1 -- 172.21.15.12:0/2041745539 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7fbfe402c3f0 con 0x7fbfe8107cc0 2024-05-17T22:27:21.169 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.408+0000 7fbfde7fc700 1 --2- 172.21.15.12:0/2041745539 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fbfd403e580 0x7fbfd4040a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:21.169 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.408+0000 7fbfde7fc700 1 -- 172.21.15.12:0/2041745539 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbfe40540f0 con 0x7fbfe8107cc0 2024-05-17T22:27:21.169 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.408+0000 7fbfecda7700 1 --2- 172.21.15.12:0/2041745539 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fbfd403e580 0x7fbfd4040a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:21.169 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.408+0000 7fbfecda7700 1 --2- 172.21.15.12:0/2041745539 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fbfd403e580 0x7fbfd4040a90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8006fd0 tx=0x7fbfd8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:21.169 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.413+0000 7fbfde7fc700 1 -- 172.21.15.12:0/2041745539 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbfe401f030 con 0x7fbfe8107cc0 2024-05-17T22:27:21.170 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.628+0000 7fbfef80c700 1 -- 172.21.15.12:0/2041745539 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fbfe8064540 con 0x7fbfe8107cc0 2024-05-17T22:27:21.170 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.629+0000 7fbfde7fc700 1 -- 172.21.15.12:0/2041745539 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fbfe401bda0 con 0x7fbfe8107cc0 2024-05-17T22:27:21.170 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.631+0000 7fbfef80c700 1 -- 172.21.15.12:0/2041745539 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fbfd403e580 msgr2=0x7fbfd4040a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:21.170 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.631+0000 7fbfef80c700 1 --2- 172.21.15.12:0/2041745539 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fbfd403e580 0x7fbfd4040a90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8006fd0 tx=0x7fbfd8006e40 comp rx=0 tx=0).stop 2024-05-17T22:27:21.170 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.631+0000 7fbfef80c700 1 -- 172.21.15.12:0/2041745539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfe8107cc0 msgr2=0x7fbfe819f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:21.170 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.631+0000 7fbfef80c700 1 --2- 172.21.15.12:0/2041745539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfe8107cc0 0x7fbfe819f900 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4000c00 tx=0x7fbfe4011aa0 comp rx=0 tx=0).stop 2024-05-17T22:27:21.170 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.631+0000 7fbfef80c700 1 -- 172.21.15.12:0/2041745539 shutdown_connections 2024-05-17T22:27:21.170 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.631+0000 7fbfef80c700 1 --2- 172.21.15.12:0/2041745539 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fbfd403e580 0x7fbfd4040a90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:21.170 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.631+0000 7fbfef80c700 1 --2- 172.21.15.12:0/2041745539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfe8107cc0 0x7fbfe819f900 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:21.170 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.631+0000 7fbfef80c700 1 -- 172.21.15.12:0/2041745539 >> 172.21.15.12:0/2041745539 conn(0x7fbfe8103a80 msgr2=0x7fbfe8193910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:27:21.170 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.632+0000 7fbfef80c700 1 -- 172.21.15.12:0/2041745539 shutdown_connections 2024-05-17T22:27:21.170 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:20.632+0000 7fbfef80c700 1 -- 172.21.15.12:0/2041745539 wait complete. 2024-05-17T22:27:21.174 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-17T22:27:21.174 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout disabled 2024-05-17T22:27:21.182 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-17T22:27:21.182 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-05-17T22:27:21.183 INFO:teuthology.orchestra.run.smithi012.stdout:firewalld.service is not enabled 2024-05-17T22:27:21.183 INFO:teuthology.orchestra.run.smithi012.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-17T22:27:21.185 INFO:teuthology.orchestra.run.smithi012.stdout:Ceph Dashboard is now available at: 2024-05-17T22:27:21.185 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:27:21.185 INFO:teuthology.orchestra.run.smithi012.stdout: URL: https://smithi012.front.sepia.ceph.com:8443/ 2024-05-17T22:27:21.185 INFO:teuthology.orchestra.run.smithi012.stdout: User: admin 2024-05-17T22:27:21.185 INFO:teuthology.orchestra.run.smithi012.stdout: Password: pfpjmayqs2 2024-05-17T22:27:21.185 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:27:21.185 INFO:teuthology.orchestra.run.smithi012.stdout:Saving cluster configuration to /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/config directory 2024-05-17T22:27:21.186 INFO:teuthology.orchestra.run.smithi012.stdout:Enabling autotune for osd_memory_target 2024-05-17T22:27:21.840 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:21 smithi012 ceph-mon[31632]: mgrmap e13: x(active, since 6s) 2024-05-17T22:27:22.405 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.587+0000 7ff40782f700 1 Processor -- start 2024-05-17T22:27:22.405 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.588+0000 7ff40782f700 1 -- start start 2024-05-17T22:27:22.405 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.588+0000 7ff40782f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff40007f670 0x7ff40007daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:22.405 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.588+0000 7ff40782f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff40007dfe0 con 0x7ff40007f670 2024-05-17T22:27:22.406 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.588+0000 7ff4055cb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff40007f670 0x7ff40007daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:22.406 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.588+0000 7ff4055cb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff40007f670 0x7ff40007daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40248/0 (socket says 172.21.15.12:40248) 2024-05-17T22:27:22.406 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.588+0000 7ff4055cb700 1 -- 172.21.15.12:0/843042482 learned_addr learned my addr 172.21.15.12:0/843042482 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:27:22.406 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.588+0000 7ff4055cb700 1 -- 172.21.15.12:0/843042482 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff40007e160 con 0x7ff40007f670 2024-05-17T22:27:22.406 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.589+0000 7ff4055cb700 1 --2- 172.21.15.12:0/843042482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff40007f670 0x7ff40007daa0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc0098a0 tx=0x7ff3fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d04299c3c7f324c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:22.406 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.589+0000 7ff3f7fff700 1 -- 172.21.15.12:0/843042482 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3fc004160 con 0x7ff40007f670 2024-05-17T22:27:22.406 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.589+0000 7ff3f7fff700 1 -- 172.21.15.12:0/843042482 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3fc004300 con 0x7ff40007f670 2024-05-17T22:27:22.406 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.589+0000 7ff3f7fff700 1 -- 172.21.15.12:0/843042482 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3fc011a20 con 0x7ff40007f670 2024-05-17T22:27:22.406 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.590+0000 7ff40782f700 1 -- 172.21.15.12:0/843042482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff40007f670 msgr2=0x7ff40007daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:22.406 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.590+0000 7ff40782f700 1 --2- 172.21.15.12:0/843042482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff40007f670 0x7ff40007daa0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc0098a0 tx=0x7ff3fc009bb0 comp rx=0 tx=0).stop 2024-05-17T22:27:22.406 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.590+0000 7ff40782f700 1 -- 172.21.15.12:0/843042482 shutdown_connections 2024-05-17T22:27:22.406 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.590+0000 7ff40782f700 1 --2- 172.21.15.12:0/843042482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff40007f670 0x7ff40007daa0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:22.406 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.590+0000 7ff40782f700 1 -- 172.21.15.12:0/843042482 >> 172.21.15.12:0/843042482 conn(0x7ff400104c00 msgr2=0x7ff4001070d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:27:22.407 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.590+0000 7ff40782f700 1 -- 172.21.15.12:0/843042482 shutdown_connections 2024-05-17T22:27:22.407 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.590+0000 7ff40782f700 1 -- 172.21.15.12:0/843042482 wait complete. 2024-05-17T22:27:22.407 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.590+0000 7ff40782f700 1 Processor -- start 2024-05-17T22:27:22.407 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.590+0000 7ff40782f700 1 -- start start 2024-05-17T22:27:22.407 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.591+0000 7ff40782f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff40007f670 0x7ff40019f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:22.407 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.591+0000 7ff40782f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff40010f9f0 con 0x7ff40007f670 2024-05-17T22:27:22.407 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.591+0000 7ff4055cb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff40007f670 0x7ff40019f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:22.407 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.591+0000 7ff4055cb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff40007f670 0x7ff40019f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40256/0 (socket says 172.21.15.12:40256) 2024-05-17T22:27:22.407 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.591+0000 7ff4055cb700 1 -- 172.21.15.12:0/1555851606 learned_addr learned my addr 172.21.15.12:0/1555851606 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:27:22.407 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.591+0000 7ff4055cb700 1 -- 172.21.15.12:0/1555851606 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff40019fe60 con 0x7ff40007f670 2024-05-17T22:27:22.407 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.591+0000 7ff4055cb700 1 --2- 172.21.15.12:0/1555851606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff40007f670 0x7ff40019f920 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc009520 tx=0x7ff3fc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:22.407 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.591+0000 7ff3f67fc700 1 -- 172.21.15.12:0/1555851606 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3fc0039e0 con 0x7ff40007f670 2024-05-17T22:27:22.407 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.591+0000 7ff3f67fc700 1 -- 172.21.15.12:0/1555851606 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3fc01b8f0 con 0x7ff40007f670 2024-05-17T22:27:22.407 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.591+0000 7ff3f67fc700 1 -- 172.21.15.12:0/1555851606 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3fc0197d0 con 0x7ff40007f670 2024-05-17T22:27:22.408 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.591+0000 7ff40782f700 1 -- 172.21.15.12:0/1555851606 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4001a00f0 con 0x7ff40007f670 2024-05-17T22:27:22.408 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.592+0000 7ff40782f700 1 -- 172.21.15.12:0/1555851606 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4001a2d00 con 0x7ff40007f670 2024-05-17T22:27:22.408 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.592+0000 7ff40782f700 1 -- 172.21.15.12:0/1555851606 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff40007daa0 con 0x7ff40007f670 2024-05-17T22:27:22.408 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.592+0000 7ff3f67fc700 1 -- 172.21.15.12:0/1555851606 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7ff3fc019a40 con 0x7ff40007f670 2024-05-17T22:27:22.408 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.593+0000 7ff3f67fc700 1 --2- 172.21.15.12:0/1555851606 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff3ec03e680 0x7ff3ec040b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:22.408 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.593+0000 7ff3f67fc700 1 -- 172.21.15.12:0/1555851606 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff3fc054130 con 0x7ff40007f670 2024-05-17T22:27:22.408 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.593+0000 7ff404dca700 1 --2- 172.21.15.12:0/1555851606 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff3ec03e680 0x7ff3ec040b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:22.408 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.593+0000 7ff404dca700 1 --2- 172.21.15.12:0/1555851606 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff3ec03e680 0x7ff3ec040b90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0006fd0 tx=0x7ff3f0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:22.408 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.598+0000 7ff3f67fc700 1 -- 172.21.15.12:0/1555851606 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff3fc01f030 con 0x7ff40007f670 2024-05-17T22:27:22.408 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.812+0000 7ff40782f700 1 -- 172.21.15.12:0/1555851606 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7ff4001a3410 con 0x7ff40007f670 2024-05-17T22:27:22.408 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.812+0000 7ff3f67fc700 1 -- 172.21.15.12:0/1555851606 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7ff3fc0105e0 con 0x7ff40007f670 2024-05-17T22:27:22.408 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.815+0000 7ff40782f700 1 -- 172.21.15.12:0/1555851606 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff3ec03e680 msgr2=0x7ff3ec040b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:22.408 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.815+0000 7ff40782f700 1 --2- 172.21.15.12:0/1555851606 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff3ec03e680 0x7ff3ec040b90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0006fd0 tx=0x7ff3f0006e40 comp rx=0 tx=0).stop 2024-05-17T22:27:22.409 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.815+0000 7ff40782f700 1 -- 172.21.15.12:0/1555851606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff40007f670 msgr2=0x7ff40019f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:22.409 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.815+0000 7ff40782f700 1 --2- 172.21.15.12:0/1555851606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff40007f670 0x7ff40019f920 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc009520 tx=0x7ff3fc011aa0 comp rx=0 tx=0).stop 2024-05-17T22:27:22.409 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.815+0000 7ff40782f700 1 -- 172.21.15.12:0/1555851606 shutdown_connections 2024-05-17T22:27:22.409 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.815+0000 7ff40782f700 1 --2- 172.21.15.12:0/1555851606 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff3ec03e680 0x7ff3ec040b90 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:22.409 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.815+0000 7ff40782f700 1 --2- 172.21.15.12:0/1555851606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff40007f670 0x7ff40019f920 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:22.409 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.815+0000 7ff40782f700 1 -- 172.21.15.12:0/1555851606 >> 172.21.15.12:0/1555851606 conn(0x7ff400104c00 msgr2=0x7ff4001054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:27:22.409 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.815+0000 7ff40782f700 1 -- 172.21.15.12:0/1555851606 shutdown_connections 2024-05-17T22:27:22.409 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:21.815+0000 7ff40782f700 1 -- 172.21.15.12:0/1555851606 wait complete. 2024-05-17T22:27:23.770 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:22.845+0000 7f5aadc22700 1 Processor -- start 2024-05-17T22:27:23.771 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:22.845+0000 7f5aadc22700 1 -- start start 2024-05-17T22:27:23.771 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:22.845+0000 7f5aadc22700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5aa810aa00 0x7f5aa810ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:23.771 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:22.845+0000 7f5aadc22700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa810b3e0 con 0x7f5aa810aa00 2024-05-17T22:27:23.771 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:22.845+0000 7f5aa77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5aa810aa00 0x7f5aa810ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:23.771 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:22.845+0000 7f5aa77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5aa810aa00 0x7f5aa810ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40266/0 (socket says 172.21.15.12:40266) 2024-05-17T22:27:23.771 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:22.845+0000 7f5aa77fe700 1 -- 172.21.15.12:0/1636922291 learned_addr learned my addr 172.21.15.12:0/1636922291 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:27:23.771 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:22.846+0000 7f5aa77fe700 1 -- 172.21.15.12:0/1636922291 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aa810b560 con 0x7f5aa810aa00 2024-05-17T22:27:23.771 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:22.846+0000 7f5aa77fe700 1 --2- 172.21.15.12:0/1636922291 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5aa810aa00 0x7f5aa810ae10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5a980098a0 tx=0x7f5a98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff4369e148e0978 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:23.771 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:22.846+0000 7f5aa6ffd700 1 -- 172.21.15.12:0/1636922291 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a98004160 con 0x7f5aa810aa00 2024-05-17T22:27:23.771 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:22.846+0000 7f5aa6ffd700 1 -- 172.21.15.12:0/1636922291 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a98004300 con 0x7f5aa810aa00 2024-05-17T22:27:23.771 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:22.846+0000 7f5aa6ffd700 1 -- 172.21.15.12:0/1636922291 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a98011a20 con 0x7f5aa810aa00 2024-05-17T22:27:23.771 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:22.847+0000 7f5aadc22700 1 -- 172.21.15.12:0/1636922291 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5aa810aa00 msgr2=0x7f5aa810ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:23.771 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:22.847+0000 7f5aadc22700 1 --2- 172.21.15.12:0/1636922291 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5aa810aa00 0x7f5aa810ae10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5a980098a0 tx=0x7f5a98009bb0 comp rx=0 tx=0).stop 2024-05-17T22:27:23.771 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:22.847+0000 7f5aadc22700 1 -- 172.21.15.12:0/1636922291 shutdown_connections 2024-05-17T22:27:23.772 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:22.847+0000 7f5aadc22700 1 --2- 172.21.15.12:0/1636922291 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5aa810aa00 0x7f5aa810ae10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:23.772 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:22.847+0000 7f5aadc22700 1 -- 172.21.15.12:0/1636922291 >> 172.21.15.12:0/1636922291 conn(0x7f5aa8103230 msgr2=0x7f5aa8105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:27:23.772 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:22.847+0000 7f5aadc22700 1 -- 172.21.15.12:0/1636922291 shutdown_connections 2024-05-17T22:27:23.772 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:22.847+0000 7f5aadc22700 1 -- 172.21.15.12:0/1636922291 wait complete. 2024-05-17T22:27:23.772 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:22.848+0000 7f5aadc22700 1 Processor -- start 2024-05-17T22:27:23.772 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:22.848+0000 7f5aadc22700 1 -- start start 2024-05-17T22:27:23.772 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:22.848+0000 7f5aadc22700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5aa810aa00 0x7f5aa8109ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:23.772 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:22.848+0000 7f5aadc22700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa8108290 con 0x7f5aa810aa00 2024-05-17T22:27:23.772 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:22.848+0000 7f5aa77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5aa810aa00 0x7f5aa8109ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:23.772 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:22.848+0000 7f5aa77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5aa810aa00 0x7f5aa8109ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40272/0 (socket says 172.21.15.12:40272) 2024-05-17T22:27:23.772 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:22.848+0000 7f5aa77fe700 1 -- 172.21.15.12:0/2232446304 learned_addr learned my addr 172.21.15.12:0/2232446304 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:27:23.772 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:22.848+0000 7f5aa77fe700 1 -- 172.21.15.12:0/2232446304 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aa810a430 con 0x7f5aa810aa00 2024-05-17T22:27:23.772 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:22.849+0000 7f5aa77fe700 1 --2- 172.21.15.12:0/2232446304 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5aa810aa00 0x7f5aa8109ef0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5a98000c00 tx=0x7f5a98011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:23.772 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:22.849+0000 7f5aa57fa700 1 -- 172.21.15.12:0/2232446304 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a98003aa0 con 0x7f5aa810aa00 2024-05-17T22:27:23.773 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:22.849+0000 7f5aa57fa700 1 -- 172.21.15.12:0/2232446304 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a9801b8f0 con 0x7f5aa810aa00 2024-05-17T22:27:23.773 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:22.849+0000 7f5aadc22700 1 -- 172.21.15.12:0/2232446304 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aa8108680 con 0x7f5aa810aa00 2024-05-17T22:27:23.773 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:22.849+0000 7f5aa57fa700 1 -- 172.21.15.12:0/2232446304 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a98023890 con 0x7f5aa810aa00 2024-05-17T22:27:23.773 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:22.849+0000 7f5aadc22700 1 -- 172.21.15.12:0/2232446304 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aa8108b60 con 0x7f5aa810aa00 2024-05-17T22:27:23.773 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:22.850+0000 7f5aa57fa700 1 -- 172.21.15.12:0/2232446304 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f5a9802c3f0 con 0x7f5aa810aa00 2024-05-17T22:27:23.773 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:22.850+0000 7f5aadc22700 1 -- 172.21.15.12:0/2232446304 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5aa810ae10 con 0x7f5aa810aa00 2024-05-17T22:27:23.773 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:22.850+0000 7f5aa57fa700 1 --2- 172.21.15.12:0/2232446304 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5a9403e620 0x7f5a94040b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:23.773 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:22.850+0000 7f5aa57fa700 1 -- 172.21.15.12:0/2232446304 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5a98054010 con 0x7f5aa810aa00 2024-05-17T22:27:23.773 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:22.850+0000 7f5a9effd700 1 --2- 172.21.15.12:0/2232446304 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5a9403e620 0x7f5a94040b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:23.773 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:22.851+0000 7f5a9effd700 1 --2- 172.21.15.12:0/2232446304 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5a9403e620 0x7f5a94040b30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5a90006fd0 tx=0x7f5a90006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:23.773 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:22.855+0000 7f5aa57fa700 1 -- 172.21.15.12:0/2232446304 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5a9801f030 con 0x7f5aa810aa00 2024-05-17T22:27:23.773 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:23.187+0000 7f5aadc22700 1 -- 172.21.15.12:0/2232446304 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f5aa81a3790 con 0x7f5aa810aa00 2024-05-17T22:27:23.773 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:23.190+0000 7f5aa57fa700 1 -- 172.21.15.12:0/2232446304 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f5a98010380 con 0x7f5aa810aa00 2024-05-17T22:27:23.773 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:23.192+0000 7f5aadc22700 1 -- 172.21.15.12:0/2232446304 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5a9403e620 msgr2=0x7f5a94040b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:23.774 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:23.192+0000 7f5aadc22700 1 --2- 172.21.15.12:0/2232446304 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5a9403e620 0x7f5a94040b30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5a90006fd0 tx=0x7f5a90006e40 comp rx=0 tx=0).stop 2024-05-17T22:27:23.774 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:23.192+0000 7f5aadc22700 1 -- 172.21.15.12:0/2232446304 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5aa810aa00 msgr2=0x7f5aa8109ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:23.774 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:23.192+0000 7f5aadc22700 1 --2- 172.21.15.12:0/2232446304 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5aa810aa00 0x7f5aa8109ef0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5a98000c00 tx=0x7f5a98011aa0 comp rx=0 tx=0).stop 2024-05-17T22:27:23.774 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:23.192+0000 7f5aadc22700 1 -- 172.21.15.12:0/2232446304 shutdown_connections 2024-05-17T22:27:23.774 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:23.192+0000 7f5aadc22700 1 --2- 172.21.15.12:0/2232446304 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5a9403e620 0x7f5a94040b30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:23.774 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:23.192+0000 7f5aadc22700 1 --2- 172.21.15.12:0/2232446304 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5aa810aa00 0x7f5aa8109ef0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:23.774 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:23.192+0000 7f5aadc22700 1 -- 172.21.15.12:0/2232446304 >> 172.21.15.12:0/2232446304 conn(0x7f5aa8103230 msgr2=0x7f5aa8103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:27:23.774 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:23.192+0000 7f5aadc22700 1 -- 172.21.15.12:0/2232446304 shutdown_connections 2024-05-17T22:27:23.774 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-05-17T22:27:23.192+0000 7f5aadc22700 1 -- 172.21.15.12:0/2232446304 wait complete. 2024-05-17T22:27:23.774 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-17T22:27:23.774 INFO:teuthology.orchestra.run.smithi012.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-17T22:27:23.774 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:27:23.774 INFO:teuthology.orchestra.run.smithi012.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-17T22:27:23.774 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:27:23.775 INFO:teuthology.orchestra.run.smithi012.stdout:Or, if you are only running a single cluster on this host: 2024-05-17T22:27:23.775 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:27:23.775 INFO:teuthology.orchestra.run.smithi012.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-17T22:27:23.775 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:27:23.775 INFO:teuthology.orchestra.run.smithi012.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-17T22:27:23.775 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:27:23.775 INFO:teuthology.orchestra.run.smithi012.stdout: ceph telemetry on 2024-05-17T22:27:23.775 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:27:23.775 INFO:teuthology.orchestra.run.smithi012.stdout:For more information see: 2024-05-17T22:27:23.775 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:27:23.775 INFO:teuthology.orchestra.run.smithi012.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-17T22:27:23.775 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:27:23.775 INFO:teuthology.orchestra.run.smithi012.stdout:Bootstrap complete. 2024-05-17T22:27:23.812 INFO:tasks.cephadm:Fetching config... 2024-05-17T22:27:23.813 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-17T22:27:23.813 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-17T22:27:23.831 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-17T22:27:23.831 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-17T22:27:23.831 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-17T22:27:23.888 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-17T22:27:23.888 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-17T22:27:23.888 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-17T22:27:23.955 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-17T22:27:23.956 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-17T22:27:23.956 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-17T22:27:24.011 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-17T22:27:24.012 DEBUG:teuthology.orchestra.run.smithi012:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNYHWTfx4yK8LyjN8rZ/nFK/ER8PwFD9w7EYfCCmK6f1w3vx3Cs7pRTxtsrZzrfcwvVOcAm4+C/1n2IfXF1bBKfrNksNhkKdP+TOXImvHrxOyMC7tF1hsT0ULp+K8FNHv577OtIyKsye/PMeMtuLArBUi7eMl5D8JtFHxrxQYMWozNLIZHe+tXEXtdnYkoJ1FNY15L7GIjoIDdcrO5HSApzyf2mFVsOeECJE/nHZpiNQ3AJBAFi4kA4wH3QBtFfcNeycC+3+lt3QuNX37aBBpRTxqupL4qUlEEABn6lLUniH7t9gTVtyXyGWVBrm9iYcLchwQt9vaEUGkTrnTEZIYpTYc5e1UYBmJmSI76LNaxcb4/9Sna+ji2lM61v92w07RwH7TpROFjb9kDfQSLH8QiXyo77ja7q6ieAts0HgXrXRTxKYsdA2J+2Xw1so3EjrOPz7hhIcOIAEYGWDqcy6ZOkqrR4ME0np3pDG76CKPl90WDwtBbiXg4H421P9SZiQ0= ceph-4d593780-149c-11ef-bc9a-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-17T22:27:24.088 INFO:teuthology.orchestra.run.smithi012.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNYHWTfx4yK8LyjN8rZ/nFK/ER8PwFD9w7EYfCCmK6f1w3vx3Cs7pRTxtsrZzrfcwvVOcAm4+C/1n2IfXF1bBKfrNksNhkKdP+TOXImvHrxOyMC7tF1hsT0ULp+K8FNHv577OtIyKsye/PMeMtuLArBUi7eMl5D8JtFHxrxQYMWozNLIZHe+tXEXtdnYkoJ1FNY15L7GIjoIDdcrO5HSApzyf2mFVsOeECJE/nHZpiNQ3AJBAFi4kA4wH3QBtFfcNeycC+3+lt3QuNX37aBBpRTxqupL4qUlEEABn6lLUniH7t9gTVtyXyGWVBrm9iYcLchwQt9vaEUGkTrnTEZIYpTYc5e1UYBmJmSI76LNaxcb4/9Sna+ji2lM61v92w07RwH7TpROFjb9kDfQSLH8QiXyo77ja7q6ieAts0HgXrXRTxKYsdA2J+2Xw1so3EjrOPz7hhIcOIAEYGWDqcy6ZOkqrR4ME0np3pDG76CKPl90WDwtBbiXg4H421P9SZiQ0= ceph-4d593780-149c-11ef-bc9a-c7b262605968 2024-05-17T22:27:24.100 DEBUG:teuthology.orchestra.run.smithi096:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNYHWTfx4yK8LyjN8rZ/nFK/ER8PwFD9w7EYfCCmK6f1w3vx3Cs7pRTxtsrZzrfcwvVOcAm4+C/1n2IfXF1bBKfrNksNhkKdP+TOXImvHrxOyMC7tF1hsT0ULp+K8FNHv577OtIyKsye/PMeMtuLArBUi7eMl5D8JtFHxrxQYMWozNLIZHe+tXEXtdnYkoJ1FNY15L7GIjoIDdcrO5HSApzyf2mFVsOeECJE/nHZpiNQ3AJBAFi4kA4wH3QBtFfcNeycC+3+lt3QuNX37aBBpRTxqupL4qUlEEABn6lLUniH7t9gTVtyXyGWVBrm9iYcLchwQt9vaEUGkTrnTEZIYpTYc5e1UYBmJmSI76LNaxcb4/9Sna+ji2lM61v92w07RwH7TpROFjb9kDfQSLH8QiXyo77ja7q6ieAts0HgXrXRTxKYsdA2J+2Xw1so3EjrOPz7hhIcOIAEYGWDqcy6ZOkqrR4ME0np3pDG76CKPl90WDwtBbiXg4H421P9SZiQ0= ceph-4d593780-149c-11ef-bc9a-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-17T22:27:24.138 INFO:teuthology.orchestra.run.smithi096.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNYHWTfx4yK8LyjN8rZ/nFK/ER8PwFD9w7EYfCCmK6f1w3vx3Cs7pRTxtsrZzrfcwvVOcAm4+C/1n2IfXF1bBKfrNksNhkKdP+TOXImvHrxOyMC7tF1hsT0ULp+K8FNHv577OtIyKsye/PMeMtuLArBUi7eMl5D8JtFHxrxQYMWozNLIZHe+tXEXtdnYkoJ1FNY15L7GIjoIDdcrO5HSApzyf2mFVsOeECJE/nHZpiNQ3AJBAFi4kA4wH3QBtFfcNeycC+3+lt3QuNX37aBBpRTxqupL4qUlEEABn6lLUniH7t9gTVtyXyGWVBrm9iYcLchwQt9vaEUGkTrnTEZIYpTYc5e1UYBmJmSI76LNaxcb4/9Sna+ji2lM61v92w07RwH7TpROFjb9kDfQSLH8QiXyo77ja7q6ieAts0HgXrXRTxKYsdA2J+2Xw1so3EjrOPz7hhIcOIAEYGWDqcy6ZOkqrR4ME0np3pDG76CKPl90WDwtBbiXg4H421P9SZiQ0= ceph-4d593780-149c-11ef-bc9a-c7b262605968 2024-05-17T22:27:24.150 DEBUG:teuthology.orchestra.run.smithi151:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNYHWTfx4yK8LyjN8rZ/nFK/ER8PwFD9w7EYfCCmK6f1w3vx3Cs7pRTxtsrZzrfcwvVOcAm4+C/1n2IfXF1bBKfrNksNhkKdP+TOXImvHrxOyMC7tF1hsT0ULp+K8FNHv577OtIyKsye/PMeMtuLArBUi7eMl5D8JtFHxrxQYMWozNLIZHe+tXEXtdnYkoJ1FNY15L7GIjoIDdcrO5HSApzyf2mFVsOeECJE/nHZpiNQ3AJBAFi4kA4wH3QBtFfcNeycC+3+lt3QuNX37aBBpRTxqupL4qUlEEABn6lLUniH7t9gTVtyXyGWVBrm9iYcLchwQt9vaEUGkTrnTEZIYpTYc5e1UYBmJmSI76LNaxcb4/9Sna+ji2lM61v92w07RwH7TpROFjb9kDfQSLH8QiXyo77ja7q6ieAts0HgXrXRTxKYsdA2J+2Xw1so3EjrOPz7hhIcOIAEYGWDqcy6ZOkqrR4ME0np3pDG76CKPl90WDwtBbiXg4H421P9SZiQ0= ceph-4d593780-149c-11ef-bc9a-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-17T22:27:24.189 INFO:teuthology.orchestra.run.smithi151.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNYHWTfx4yK8LyjN8rZ/nFK/ER8PwFD9w7EYfCCmK6f1w3vx3Cs7pRTxtsrZzrfcwvVOcAm4+C/1n2IfXF1bBKfrNksNhkKdP+TOXImvHrxOyMC7tF1hsT0ULp+K8FNHv577OtIyKsye/PMeMtuLArBUi7eMl5D8JtFHxrxQYMWozNLIZHe+tXEXtdnYkoJ1FNY15L7GIjoIDdcrO5HSApzyf2mFVsOeECJE/nHZpiNQ3AJBAFi4kA4wH3QBtFfcNeycC+3+lt3QuNX37aBBpRTxqupL4qUlEEABn6lLUniH7t9gTVtyXyGWVBrm9iYcLchwQt9vaEUGkTrnTEZIYpTYc5e1UYBmJmSI76LNaxcb4/9Sna+ji2lM61v92w07RwH7TpROFjb9kDfQSLH8QiXyo77ja7q6ieAts0HgXrXRTxKYsdA2J+2Xw1so3EjrOPz7hhIcOIAEYGWDqcy6ZOkqrR4ME0np3pDG76CKPl90WDwtBbiXg4H421P9SZiQ0= ceph-4d593780-149c-11ef-bc9a-c7b262605968 2024-05-17T22:27:24.201 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-17T22:27:24.340 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:24 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/2232446304' entity='client.admin' 2024-05-17T22:27:24.448 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:27:26.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:26.305+0000 7f8dcb777700 1 -- 172.21.15.12:0/399317682 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8dc4104050 msgr2=0x7f8dc4106470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:26.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:26.305+0000 7f8dcb777700 1 --2- 172.21.15.12:0/399317682 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8dc4104050 0x7f8dc4106470 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8db400b0a0 tx=0x7f8db400b3b0 comp rx=0 tx=0).stop 2024-05-17T22:27:26.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:26.305+0000 7f8dcb777700 1 -- 172.21.15.12:0/399317682 shutdown_connections 2024-05-17T22:27:26.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:26.305+0000 7f8dcb777700 1 --2- 172.21.15.12:0/399317682 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8dc4104050 0x7f8dc4106470 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:26.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:26.305+0000 7f8dcb777700 1 -- 172.21.15.12:0/399317682 >> 172.21.15.12:0/399317682 conn(0x7f8dc40fdc80 msgr2=0x7f8dc4100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:27:26.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:26.306+0000 7f8dcb777700 1 -- 172.21.15.12:0/399317682 shutdown_connections 2024-05-17T22:27:26.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:26.306+0000 7f8dcb777700 1 -- 172.21.15.12:0/399317682 wait complete. 2024-05-17T22:27:26.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:26.306+0000 7f8dcb777700 1 Processor -- start 2024-05-17T22:27:26.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:26.306+0000 7f8dcb777700 1 -- start start 2024-05-17T22:27:26.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:26.306+0000 7f8dcb777700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8dc4104050 0x7f8dc4199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:26.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:26.306+0000 7f8dcb777700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc4109510 con 0x7f8dc4104050 2024-05-17T22:27:26.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:26.307+0000 7f8dc9513700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8dc4104050 0x7f8dc4199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:26.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:26.307+0000 7f8dc9513700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8dc4104050 0x7f8dc4199030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40288/0 (socket says 172.21.15.12:40288) 2024-05-17T22:27:26.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:26.307+0000 7f8dc9513700 1 -- 172.21.15.12:0/2169341808 learned_addr learned my addr 172.21.15.12:0/2169341808 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:27:26.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:26.307+0000 7f8dc9513700 1 -- 172.21.15.12:0/2169341808 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dc4199570 con 0x7f8dc4104050 2024-05-17T22:27:26.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:26.307+0000 7f8dc9513700 1 --2- 172.21.15.12:0/2169341808 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8dc4104050 0x7f8dc4199030 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8db4015040 tx=0x7f8db4003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:26.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:26.307+0000 7f8dc27fc700 1 -- 172.21.15.12:0/2169341808 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8db4021020 con 0x7f8dc4104050 2024-05-17T22:27:26.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:26.307+0000 7f8dc27fc700 1 -- 172.21.15.12:0/2169341808 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8db40044b0 con 0x7f8dc4104050 2024-05-17T22:27:26.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:26.307+0000 7f8dcb777700 1 -- 172.21.15.12:0/2169341808 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dc4199800 con 0x7f8dc4104050 2024-05-17T22:27:26.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:26.307+0000 7f8dc27fc700 1 -- 172.21.15.12:0/2169341808 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8db4017040 con 0x7f8dc4104050 2024-05-17T22:27:26.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:26.307+0000 7f8dcb777700 1 -- 172.21.15.12:0/2169341808 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dc419c410 con 0x7f8dc4104050 2024-05-17T22:27:26.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:26.308+0000 7f8dcb777700 1 -- 172.21.15.12:0/2169341808 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dc4109c90 con 0x7f8dc4104050 2024-05-17T22:27:26.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:26.308+0000 7f8dc27fc700 1 -- 172.21.15.12:0/2169341808 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f8db4024020 con 0x7f8dc4104050 2024-05-17T22:27:26.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:26.309+0000 7f8dc27fc700 1 --2- 172.21.15.12:0/2169341808 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8dac03e510 0x7f8dac040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:26.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:26.309+0000 7f8dc27fc700 1 -- 172.21.15.12:0/2169341808 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8db400f050 con 0x7f8dc4104050 2024-05-17T22:27:26.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:26.309+0000 7f8dc8d12700 1 --2- 172.21.15.12:0/2169341808 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8dac03e510 0x7f8dac040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:26.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:26.309+0000 7f8dc8d12700 1 --2- 172.21.15.12:0/2169341808 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8dac03e510 0x7f8dac040a20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8db8006fd0 tx=0x7f8db8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:26.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:26.314+0000 7f8dc27fc700 1 -- 172.21.15.12:0/2169341808 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8db402b020 con 0x7f8dc4104050 2024-05-17T22:27:26.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:26.536+0000 7f8dcb777700 1 -- 172.21.15.12:0/2169341808 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f8dc4064440 con 0x7f8dc4104050 2024-05-17T22:27:26.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:26.539+0000 7f8dc27fc700 1 -- 172.21.15.12:0/2169341808 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f8db402e9e0 con 0x7f8dc4104050 2024-05-17T22:27:26.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:26.541+0000 7f8dcb777700 1 -- 172.21.15.12:0/2169341808 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8dac03e510 msgr2=0x7f8dac040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:26.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:26.541+0000 7f8dcb777700 1 --2- 172.21.15.12:0/2169341808 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8dac03e510 0x7f8dac040a20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8db8006fd0 tx=0x7f8db8006e40 comp rx=0 tx=0).stop 2024-05-17T22:27:26.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:26.541+0000 7f8dcb777700 1 -- 172.21.15.12:0/2169341808 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8dc4104050 msgr2=0x7f8dc4199030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:26.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:26.541+0000 7f8dcb777700 1 --2- 172.21.15.12:0/2169341808 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8dc4104050 0x7f8dc4199030 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8db4015040 tx=0x7f8db4003ce0 comp rx=0 tx=0).stop 2024-05-17T22:27:26.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:26.542+0000 7f8dcb777700 1 -- 172.21.15.12:0/2169341808 shutdown_connections 2024-05-17T22:27:26.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:26.542+0000 7f8dcb777700 1 --2- 172.21.15.12:0/2169341808 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8dac03e510 0x7f8dac040a20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:26.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:26.542+0000 7f8dcb777700 1 --2- 172.21.15.12:0/2169341808 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8dc4104050 0x7f8dc4199030 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:26.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:26.542+0000 7f8dcb777700 1 -- 172.21.15.12:0/2169341808 >> 172.21.15.12:0/2169341808 conn(0x7f8dc40fdc80 msgr2=0x7f8dc40fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:27:26.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:26.542+0000 7f8dcb777700 1 -- 172.21.15.12:0/2169341808 shutdown_connections 2024-05-17T22:27:26.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:26.542+0000 7f8dcb777700 1 -- 172.21.15.12:0/2169341808 wait complete. 2024-05-17T22:27:27.143 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-17T22:27:27.144 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-17T22:27:27.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:27 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/2169341808' entity='client.admin' 2024-05-17T22:27:27.680 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:27:29.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:29.878+0000 7f954992b700 1 -- 172.21.15.12:0/2982797130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9544102490 msgr2=0x7f95441028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:29.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:29.878+0000 7f954992b700 1 --2- 172.21.15.12:0/2982797130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9544102490 0x7f95441028a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f953400b0a0 tx=0x7f953400b3b0 comp rx=0 tx=0).stop 2024-05-17T22:27:29.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:29.878+0000 7f954992b700 1 -- 172.21.15.12:0/2982797130 shutdown_connections 2024-05-17T22:27:29.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:29.878+0000 7f954992b700 1 --2- 172.21.15.12:0/2982797130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9544102490 0x7f95441028a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:29.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:29.878+0000 7f954992b700 1 -- 172.21.15.12:0/2982797130 >> 172.21.15.12:0/2982797130 conn(0x7f95440fdce0 msgr2=0x7f9544100170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:27:29.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:29.878+0000 7f954992b700 1 -- 172.21.15.12:0/2982797130 shutdown_connections 2024-05-17T22:27:29.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:29.878+0000 7f954992b700 1 -- 172.21.15.12:0/2982797130 wait complete. 2024-05-17T22:27:29.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:29.879+0000 7f954992b700 1 Processor -- start 2024-05-17T22:27:29.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:29.879+0000 7f954992b700 1 -- start start 2024-05-17T22:27:29.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:29.879+0000 7f954992b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9544102490 0x7f9544197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:29.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:29.879+0000 7f954992b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9544108460 con 0x7f9544102490 2024-05-17T22:27:29.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:29.879+0000 7f9542ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9544102490 0x7f9544197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:29.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:29.879+0000 7f9542ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9544102490 0x7f9544197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56034/0 (socket says 172.21.15.12:56034) 2024-05-17T22:27:29.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:29.879+0000 7f9542ffd700 1 -- 172.21.15.12:0/2505755126 learned_addr learned my addr 172.21.15.12:0/2505755126 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:27:29.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:29.880+0000 7f9542ffd700 1 -- 172.21.15.12:0/2505755126 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9544198500 con 0x7f9544102490 2024-05-17T22:27:29.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:29.880+0000 7f9542ffd700 1 --2- 172.21.15.12:0/2505755126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9544102490 0x7f9544197fc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9534007730 tx=0x7f9534003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:29.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:29.880+0000 7f9548929700 1 -- 172.21.15.12:0/2505755126 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9534021020 con 0x7f9544102490 2024-05-17T22:27:29.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:29.880+0000 7f9548929700 1 -- 172.21.15.12:0/2505755126 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f953400bdd0 con 0x7f9544102490 2024-05-17T22:27:29.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:29.880+0000 7f954992b700 1 -- 172.21.15.12:0/2505755126 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9544198790 con 0x7f9544102490 2024-05-17T22:27:29.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:29.880+0000 7f9548929700 1 -- 172.21.15.12:0/2505755126 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9534017040 con 0x7f9544102490 2024-05-17T22:27:29.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:29.880+0000 7f954992b700 1 -- 172.21.15.12:0/2505755126 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f954419b3a0 con 0x7f9544102490 2024-05-17T22:27:29.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:29.881+0000 7f9548929700 1 -- 172.21.15.12:0/2505755126 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f9534024020 con 0x7f9544102490 2024-05-17T22:27:29.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:29.881+0000 7f9548929700 1 --2- 172.21.15.12:0/2505755126 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f952c03e570 0x7f952c040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:29.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:29.881+0000 7f9548929700 1 -- 172.21.15.12:0/2505755126 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f953400f050 con 0x7f9544102490 2024-05-17T22:27:29.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:29.881+0000 7f954992b700 1 -- 172.21.15.12:0/2505755126 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9544108be0 con 0x7f9544102490 2024-05-17T22:27:29.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:29.881+0000 7f95427fc700 1 --2- 172.21.15.12:0/2505755126 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f952c03e570 0x7f952c040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:29.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:29.882+0000 7f95427fc700 1 --2- 172.21.15.12:0/2505755126 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f952c03e570 0x7f952c040a80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9538006fd0 tx=0x7f9538006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:29.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:29.887+0000 7f9548929700 1 -- 172.21.15.12:0/2505755126 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f953402b020 con 0x7f9544102490 2024-05-17T22:27:30.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:30.104+0000 7f954992b700 1 -- 172.21.15.12:0/2505755126 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f9544063c70 con 0x7f952c03e570 2024-05-17T22:27:30.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:30.108+0000 7f9548929700 1 -- 172.21.15.12:0/2505755126 <== mgr.14152 v2:172.21.15.12:6800/1388514047 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9544063c70 con 0x7f952c03e570 2024-05-17T22:27:30.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:30.110+0000 7f954992b700 1 -- 172.21.15.12:0/2505755126 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f952c03e570 msgr2=0x7f952c040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:30.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:30.110+0000 7f954992b700 1 --2- 172.21.15.12:0/2505755126 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f952c03e570 0x7f952c040a80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9538006fd0 tx=0x7f9538006e40 comp rx=0 tx=0).stop 2024-05-17T22:27:30.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:30.110+0000 7f954992b700 1 -- 172.21.15.12:0/2505755126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9544102490 msgr2=0x7f9544197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:30.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:30.110+0000 7f954992b700 1 --2- 172.21.15.12:0/2505755126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9544102490 0x7f9544197fc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9534007730 tx=0x7f9534003ce0 comp rx=0 tx=0).stop 2024-05-17T22:27:30.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:30.110+0000 7f954992b700 1 -- 172.21.15.12:0/2505755126 shutdown_connections 2024-05-17T22:27:30.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:30.110+0000 7f954992b700 1 --2- 172.21.15.12:0/2505755126 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f952c03e570 0x7f952c040a80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:30.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:30.110+0000 7f954992b700 1 --2- 172.21.15.12:0/2505755126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9544102490 0x7f9544197fc0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:30.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:30.110+0000 7f954992b700 1 -- 172.21.15.12:0/2505755126 >> 172.21.15.12:0/2505755126 conn(0x7f95440fdce0 msgr2=0x7f95440fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:27:30.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:30.110+0000 7f954992b700 1 -- 172.21.15.12:0/2505755126 shutdown_connections 2024-05-17T22:27:30.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:30.110+0000 7f954992b700 1 -- 172.21.15.12:0/2505755126 wait complete. 2024-05-17T22:27:30.657 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:30 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:27:30.657 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:30 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:27:30.657 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:30 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi012", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:27:30.657 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:30 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:27:30.657 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:30 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:27:30.657 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:30 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:27:30.657 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:30 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:27:30.657 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:30 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:27:30.658 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:30 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:27:30.658 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:30 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:27:30.746 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi096 2024-05-17T22:27:30.746 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-05-17T22:27:30.746 DEBUG:teuthology.orchestra.run.smithi096:> dd of=/etc/ceph/ceph.conf 2024-05-17T22:27:30.765 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-05-17T22:27:30.765 DEBUG:teuthology.orchestra.run.smithi096:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-17T22:27:30.823 INFO:tasks.cephadm:Adding host smithi096 to orchestrator... 2024-05-17T22:27:30.823 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph orch host add smithi096 2024-05-17T22:27:31.073 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:27:31.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:31 smithi012 ceph-mon[31632]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:27:31.487 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:31 smithi012 ceph-mon[31632]: Updating smithi012:/etc/ceph/ceph.conf 2024-05-17T22:27:31.487 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:31 smithi012 ceph-mon[31632]: Updating smithi012:/var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T22:27:31.487 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:31 smithi012 ceph-mon[31632]: Updating smithi012:/etc/ceph/ceph.client.admin.keyring 2024-05-17T22:27:31.487 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:31 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:27:31.487 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:31 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:27:31.487 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:31 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:27:32.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:32 smithi012 ceph-mon[31632]: Updating smithi012:/var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-17T22:27:32.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:32.678+0000 7f4246870700 1 -- 172.21.15.12:0/2031618489 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4240104610 msgr2=0x7f4240104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:32.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:32.678+0000 7f4246870700 1 --2- 172.21.15.12:0/2031618489 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4240104610 0x7f4240104a20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4230009910 tx=0x7f4230009c20 comp rx=0 tx=0).stop 2024-05-17T22:27:32.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:32.679+0000 7f4246870700 1 -- 172.21.15.12:0/2031618489 shutdown_connections 2024-05-17T22:27:32.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:32.679+0000 7f4246870700 1 --2- 172.21.15.12:0/2031618489 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4240104610 0x7f4240104a20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:32.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:32.679+0000 7f4246870700 1 -- 172.21.15.12:0/2031618489 >> 172.21.15.12:0/2031618489 conn(0x7f42400fe0a0 msgr2=0x7f42401004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:27:32.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:32.679+0000 7f4246870700 1 -- 172.21.15.12:0/2031618489 shutdown_connections 2024-05-17T22:27:32.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:32.679+0000 7f4246870700 1 -- 172.21.15.12:0/2031618489 wait complete. 2024-05-17T22:27:32.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:32.679+0000 7f4246870700 1 Processor -- start 2024-05-17T22:27:32.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:32.679+0000 7f4246870700 1 -- start start 2024-05-17T22:27:32.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:32.680+0000 7f4246870700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4240104610 0x7f42401992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:32.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:32.680+0000 7f4246870700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f424010da70 con 0x7f4240104610 2024-05-17T22:27:32.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:32.680+0000 7f423ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4240104610 0x7f42401992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:32.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:32.680+0000 7f423ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4240104610 0x7f42401992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56054/0 (socket says 172.21.15.12:56054) 2024-05-17T22:27:32.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:32.680+0000 7f423ffff700 1 -- 172.21.15.12:0/2428041316 learned_addr learned my addr 172.21.15.12:0/2428041316 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:27:32.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:32.680+0000 7f423ffff700 1 -- 172.21.15.12:0/2428041316 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4240199830 con 0x7f4240104610 2024-05-17T22:27:32.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:32.680+0000 7f423ffff700 1 --2- 172.21.15.12:0/2428041316 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4240104610 0x7f42401992f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4230006010 tx=0x7f4230005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:32.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:32.680+0000 7f423dffb700 1 -- 172.21.15.12:0/2428041316 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4230026070 con 0x7f4240104610 2024-05-17T22:27:32.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:32.680+0000 7f423dffb700 1 -- 172.21.15.12:0/2428041316 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f423000bc00 con 0x7f4240104610 2024-05-17T22:27:32.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:32.680+0000 7f4246870700 1 -- 172.21.15.12:0/2428041316 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4240199ac0 con 0x7f4240104610 2024-05-17T22:27:32.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:32.680+0000 7f423dffb700 1 -- 172.21.15.12:0/2428041316 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4230003910 con 0x7f4240104610 2024-05-17T22:27:32.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:32.681+0000 7f4246870700 1 -- 172.21.15.12:0/2428041316 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4240199f20 con 0x7f4240104610 2024-05-17T22:27:32.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:32.681+0000 7f423dffb700 1 -- 172.21.15.12:0/2428041316 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f4230029070 con 0x7f4240104610 2024-05-17T22:27:32.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:32.682+0000 7f4246870700 1 -- 172.21.15.12:0/2428041316 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f424006b6e0 con 0x7f4240104610 2024-05-17T22:27:32.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:32.682+0000 7f423dffb700 1 --2- 172.21.15.12:0/2428041316 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f422c03e570 0x7f422c040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:32.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:32.682+0000 7f423dffb700 1 -- 172.21.15.12:0/2428041316 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f423001e070 con 0x7f4240104610 2024-05-17T22:27:32.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:32.682+0000 7f4237fff700 1 --2- 172.21.15.12:0/2428041316 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f422c03e570 0x7f422c040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:32.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:32.682+0000 7f4237fff700 1 --2- 172.21.15.12:0/2428041316 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f422c03e570 0x7f422c040a80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4228006fd0 tx=0x7f4228006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:32.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:32.687+0000 7f423dffb700 1 -- 172.21.15.12:0/2428041316 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4230013070 con 0x7f4240104610 2024-05-17T22:27:32.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:32.906+0000 7f4246870700 1 -- 172.21.15.12:0/2428041316 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi096", "target": ["mon-mgr", ""]}) -- 0x7f424019a6c0 con 0x7f422c03e570 2024-05-17T22:27:34.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:33 smithi012 ceph-mon[31632]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi096", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:27:34.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:33 smithi012 ceph-mon[31632]: Deploying cephadm binary to smithi096 2024-05-17T22:27:35.340 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:34 smithi012 ceph-mon[31632]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:27:37.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:37.134+0000 7f423dffb700 1 -- 172.21.15.12:0/2428041316 <== mgr.14152 v2:172.21.15.12:6800/1388514047 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f424019a6c0 con 0x7f422c03e570 2024-05-17T22:27:37.136 INFO:teuthology.orchestra.run.smithi012.stdout:Added host 'smithi096' with addr '172.21.15.96' 2024-05-17T22:27:37.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:37.137+0000 7f4246870700 1 -- 172.21.15.12:0/2428041316 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f422c03e570 msgr2=0x7f422c040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:37.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:37.137+0000 7f4246870700 1 --2- 172.21.15.12:0/2428041316 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f422c03e570 0x7f422c040a80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4228006fd0 tx=0x7f4228006e40 comp rx=0 tx=0).stop 2024-05-17T22:27:37.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:37.137+0000 7f4246870700 1 -- 172.21.15.12:0/2428041316 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4240104610 msgr2=0x7f42401992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:37.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:37.137+0000 7f4246870700 1 --2- 172.21.15.12:0/2428041316 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4240104610 0x7f42401992f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4230006010 tx=0x7f4230005780 comp rx=0 tx=0).stop 2024-05-17T22:27:37.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:37.137+0000 7f4246870700 1 -- 172.21.15.12:0/2428041316 shutdown_connections 2024-05-17T22:27:37.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:37.137+0000 7f4246870700 1 --2- 172.21.15.12:0/2428041316 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f422c03e570 0x7f422c040a80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:37.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:37.137+0000 7f4246870700 1 --2- 172.21.15.12:0/2428041316 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4240104610 0x7f42401992f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:37.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:37.137+0000 7f4246870700 1 -- 172.21.15.12:0/2428041316 >> 172.21.15.12:0/2428041316 conn(0x7f42400fe0a0 msgr2=0x7f4240105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:27:37.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:37.137+0000 7f4246870700 1 -- 172.21.15.12:0/2428041316 shutdown_connections 2024-05-17T22:27:37.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:37.138+0000 7f4246870700 1 -- 172.21.15.12:0/2428041316 wait complete. 2024-05-17T22:27:37.716 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:37 smithi012 ceph-mon[31632]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:27:37.717 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:37 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:27:37.717 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:37 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:27:37.718 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph orch host ls --format=json 2024-05-17T22:27:37.973 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:27:38.786 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:38 smithi012 ceph-mon[31632]: Added host smithi096 2024-05-17T22:27:38.786 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:38 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:27:38.786 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:38 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:27:39.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:39.572+0000 7f504ade6700 1 -- 172.21.15.12:0/953437064 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5044102610 msgr2=0x7f5044102a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:39.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:39.572+0000 7f504ade6700 1 --2- 172.21.15.12:0/953437064 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5044102610 0x7f5044102a20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f502c009910 tx=0x7f502c009c20 comp rx=0 tx=0).stop 2024-05-17T22:27:39.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:39.572+0000 7f504ade6700 1 -- 172.21.15.12:0/953437064 shutdown_connections 2024-05-17T22:27:39.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:39.572+0000 7f504ade6700 1 --2- 172.21.15.12:0/953437064 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5044102610 0x7f5044102a20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:39.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:39.572+0000 7f504ade6700 1 -- 172.21.15.12:0/953437064 >> 172.21.15.12:0/953437064 conn(0x7f50440fe0a0 msgr2=0x7f50441004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:27:39.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:39.572+0000 7f504ade6700 1 -- 172.21.15.12:0/953437064 shutdown_connections 2024-05-17T22:27:39.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:39.573+0000 7f504ade6700 1 -- 172.21.15.12:0/953437064 wait complete. 2024-05-17T22:27:39.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:39.573+0000 7f504ade6700 1 Processor -- start 2024-05-17T22:27:39.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:39.574+0000 7f504ade6700 1 -- start start 2024-05-17T22:27:39.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:39.574+0000 7f504ade6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5044102610 0x7f50441992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:39.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:39.574+0000 7f504ade6700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f504410c900 con 0x7f5044102610 2024-05-17T22:27:39.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:39.574+0000 7f5048b82700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5044102610 0x7f50441992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:39.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:39.574+0000 7f5048b82700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5044102610 0x7f50441992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56066/0 (socket says 172.21.15.12:56066) 2024-05-17T22:27:39.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:39.574+0000 7f5048b82700 1 -- 172.21.15.12:0/2964286135 learned_addr learned my addr 172.21.15.12:0/2964286135 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:27:39.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:39.574+0000 7f5048b82700 1 -- 172.21.15.12:0/2964286135 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5044199800 con 0x7f5044102610 2024-05-17T22:27:39.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:39.574+0000 7f5048b82700 1 --2- 172.21.15.12:0/2964286135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5044102610 0x7f50441992c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f502c0056d0 tx=0x7f502c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:39.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:39.575+0000 7f5041ffb700 1 -- 172.21.15.12:0/2964286135 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f502c026070 con 0x7f5044102610 2024-05-17T22:27:39.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:39.575+0000 7f504ade6700 1 -- 172.21.15.12:0/2964286135 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5044199af0 con 0x7f5044102610 2024-05-17T22:27:39.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:39.575+0000 7f5041ffb700 1 -- 172.21.15.12:0/2964286135 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f502c00bba0 con 0x7f5044102610 2024-05-17T22:27:39.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:39.575+0000 7f5041ffb700 1 -- 172.21.15.12:0/2964286135 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f502c003920 con 0x7f5044102610 2024-05-17T22:27:39.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:39.575+0000 7f504ade6700 1 -- 172.21.15.12:0/2964286135 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f504419daf0 con 0x7f5044102610 2024-05-17T22:27:39.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:39.576+0000 7f504ade6700 1 -- 172.21.15.12:0/2964286135 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5044103ac0 con 0x7f5044102610 2024-05-17T22:27:39.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:39.576+0000 7f5041ffb700 1 -- 172.21.15.12:0/2964286135 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f502c029070 con 0x7f5044102610 2024-05-17T22:27:39.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:39.576+0000 7f5041ffb700 1 --2- 172.21.15.12:0/2964286135 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f503003e510 0x7f5030040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:39.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:39.576+0000 7f5041ffb700 1 -- 172.21.15.12:0/2964286135 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f502c01e070 con 0x7f5044102610 2024-05-17T22:27:39.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:39.576+0000 7f5043fff700 1 --2- 172.21.15.12:0/2964286135 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f503003e510 0x7f5030040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:39.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:39.577+0000 7f5043fff700 1 --2- 172.21.15.12:0/2964286135 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f503003e510 0x7f5030040a20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f503400ad30 tx=0x7f50340093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:39.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:39.581+0000 7f5041ffb700 1 -- 172.21.15.12:0/2964286135 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f502c013070 con 0x7f5044102610 2024-05-17T22:27:39.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:39.808+0000 7f504ade6700 1 -- 172.21.15.12:0/2964286135 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f504419a620 con 0x7f503003e510 2024-05-17T22:27:39.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:39.809+0000 7f5041ffb700 1 -- 172.21.15.12:0/2964286135 <== mgr.14152 v2:172.21.15.12:6800/1388514047 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f504419a620 con 0x7f503003e510 2024-05-17T22:27:39.811 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:27:39.811 INFO:teuthology.orchestra.run.smithi012.stdout:[{"addr": "172.21.15.12", "hostname": "smithi012", "labels": [], "status": ""}, {"addr": "172.21.15.96", "hostname": "smithi096", "labels": [], "status": ""}] 2024-05-17T22:27:39.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:39.811+0000 7f504ade6700 1 -- 172.21.15.12:0/2964286135 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f503003e510 msgr2=0x7f5030040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:39.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:39.811+0000 7f504ade6700 1 --2- 172.21.15.12:0/2964286135 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f503003e510 0x7f5030040a20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f503400ad30 tx=0x7f50340093f0 comp rx=0 tx=0).stop 2024-05-17T22:27:39.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:39.811+0000 7f504ade6700 1 -- 172.21.15.12:0/2964286135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5044102610 msgr2=0x7f50441992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:39.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:39.811+0000 7f504ade6700 1 --2- 172.21.15.12:0/2964286135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5044102610 0x7f50441992c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f502c0056d0 tx=0x7f502c0057b0 comp rx=0 tx=0).stop 2024-05-17T22:27:39.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:39.811+0000 7f504ade6700 1 -- 172.21.15.12:0/2964286135 shutdown_connections 2024-05-17T22:27:39.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:39.811+0000 7f504ade6700 1 --2- 172.21.15.12:0/2964286135 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f503003e510 0x7f5030040a20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:39.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:39.811+0000 7f504ade6700 1 --2- 172.21.15.12:0/2964286135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5044102610 0x7f50441992c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:39.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:39.811+0000 7f504ade6700 1 -- 172.21.15.12:0/2964286135 >> 172.21.15.12:0/2964286135 conn(0x7f50440fe0a0 msgr2=0x7f504406ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:27:39.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:39.812+0000 7f504ade6700 1 -- 172.21.15.12:0/2964286135 shutdown_connections 2024-05-17T22:27:39.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:39.812+0000 7f504ade6700 1 -- 172.21.15.12:0/2964286135 wait complete. 2024-05-17T22:27:39.825 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:39 smithi012 ceph-mon[31632]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:27:40.403 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi151 2024-05-17T22:27:40.403 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-05-17T22:27:40.403 DEBUG:teuthology.orchestra.run.smithi151:> dd of=/etc/ceph/ceph.conf 2024-05-17T22:27:40.422 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-05-17T22:27:40.425 DEBUG:teuthology.orchestra.run.smithi151:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-17T22:27:40.480 INFO:tasks.cephadm:Adding host smithi151 to orchestrator... 2024-05-17T22:27:40.480 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph orch host add smithi151 2024-05-17T22:27:40.738 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:27:41.031 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:40 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:27:41.031 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:40 smithi012 ceph-mon[31632]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T22:27:41.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:41 smithi012 ceph-mon[31632]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:27:42.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:42.157+0000 7f69c4416700 1 -- 172.21.15.12:0/1755088465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69bc108490 msgr2=0x7f69bc1088a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:42.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:42.157+0000 7f69c4416700 1 --2- 172.21.15.12:0/1755088465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69bc108490 0x7f69bc1088a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f69b8009910 tx=0x7f69b8009c20 comp rx=0 tx=0).stop 2024-05-17T22:27:42.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:42.157+0000 7f69c4416700 1 -- 172.21.15.12:0/1755088465 shutdown_connections 2024-05-17T22:27:42.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:42.157+0000 7f69c4416700 1 --2- 172.21.15.12:0/1755088465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69bc108490 0x7f69bc1088a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:42.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:42.157+0000 7f69c4416700 1 -- 172.21.15.12:0/1755088465 >> 172.21.15.12:0/1755088465 conn(0x7f69bc0fe160 msgr2=0x7f69bc1005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:27:42.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:42.157+0000 7f69c4416700 1 -- 172.21.15.12:0/1755088465 shutdown_connections 2024-05-17T22:27:42.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:42.157+0000 7f69c4416700 1 -- 172.21.15.12:0/1755088465 wait complete. 2024-05-17T22:27:42.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:42.158+0000 7f69c4416700 1 Processor -- start 2024-05-17T22:27:42.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:42.158+0000 7f69c4416700 1 -- start start 2024-05-17T22:27:42.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:42.158+0000 7f69c4416700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69bc108490 0x7f69bc19a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:42.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:42.158+0000 7f69c4416700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f69bc10c960 con 0x7f69bc108490 2024-05-17T22:27:42.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:42.158+0000 7f69c21b2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69bc108490 0x7f69bc19a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:42.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:42.158+0000 7f69c21b2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69bc108490 0x7f69bc19a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49408/0 (socket says 172.21.15.12:49408) 2024-05-17T22:27:42.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:42.158+0000 7f69c21b2700 1 -- 172.21.15.12:0/1167113058 learned_addr learned my addr 172.21.15.12:0/1167113058 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:27:42.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:42.159+0000 7f69c21b2700 1 -- 172.21.15.12:0/1167113058 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69bc06c730 con 0x7f69bc108490 2024-05-17T22:27:42.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:42.159+0000 7f69c21b2700 1 --2- 172.21.15.12:0/1167113058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69bc108490 0x7f69bc19a160 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f69b80092e0 tx=0x7f69b8005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:42.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:42.159+0000 7f69b37fe700 1 -- 172.21.15.12:0/1167113058 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69b8026070 con 0x7f69bc108490 2024-05-17T22:27:42.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:42.159+0000 7f69c4416700 1 -- 172.21.15.12:0/1167113058 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69bc19a760 con 0x7f69bc108490 2024-05-17T22:27:42.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:42.159+0000 7f69b37fe700 1 -- 172.21.15.12:0/1167113058 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69b800b750 con 0x7f69bc108490 2024-05-17T22:27:42.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:42.159+0000 7f69b37fe700 1 -- 172.21.15.12:0/1167113058 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69b80038a0 con 0x7f69bc108490 2024-05-17T22:27:42.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:42.159+0000 7f69c4416700 1 -- 172.21.15.12:0/1167113058 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69bc19ac20 con 0x7f69bc108490 2024-05-17T22:27:42.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:42.160+0000 7f69b37fe700 1 -- 172.21.15.12:0/1167113058 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f69b8029070 con 0x7f69bc108490 2024-05-17T22:27:42.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:42.160+0000 7f69b37fe700 1 --2- 172.21.15.12:0/1167113058 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f69a803e510 0x7f69a8040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:42.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:42.160+0000 7f69c4416700 1 -- 172.21.15.12:0/1167113058 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69bc103b10 con 0x7f69bc108490 2024-05-17T22:27:42.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:42.160+0000 7f69c19b1700 1 --2- 172.21.15.12:0/1167113058 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f69a803e510 0x7f69a8040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:42.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:42.160+0000 7f69b37fe700 1 -- 172.21.15.12:0/1167113058 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f69b801e070 con 0x7f69bc108490 2024-05-17T22:27:42.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:42.161+0000 7f69c19b1700 1 --2- 172.21.15.12:0/1167113058 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f69a803e510 0x7f69a8040a20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f69ac006fd0 tx=0x7f69ac006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:42.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:42.166+0000 7f69b37fe700 1 -- 172.21.15.12:0/1167113058 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f69b8013070 con 0x7f69bc108490 2024-05-17T22:27:42.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:42.384+0000 7f69c4416700 1 -- 172.21.15.12:0/1167113058 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi151", "target": ["mon-mgr", ""]}) -- 0x7f69bc19af00 con 0x7f69a803e510 2024-05-17T22:27:44.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:43 smithi012 ceph-mon[31632]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:27:44.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:43 smithi012 ceph-mon[31632]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi151", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:27:44.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:43 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:27:44.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:43 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:27:44.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:43 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:27:44.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:43 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:27:44.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:43 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi096", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:27:44.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:43 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:27:44.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:43 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:27:44.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:43 smithi012 ceph-mon[31632]: Updating smithi096:/etc/ceph/ceph.conf 2024-05-17T22:27:45.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:44 smithi012 ceph-mon[31632]: Deploying cephadm binary to smithi151 2024-05-17T22:27:45.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:44 smithi012 ceph-mon[31632]: Updating smithi096:/var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T22:27:45.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:44 smithi012 ceph-mon[31632]: Updating smithi096:/etc/ceph/ceph.client.admin.keyring 2024-05-17T22:27:46.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:45 smithi012 ceph-mon[31632]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:27:46.671 INFO:teuthology.orchestra.run.smithi012.stdout:Added host 'smithi151' with addr '172.21.15.151' 2024-05-17T22:27:46.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:46.669+0000 7f69b37fe700 1 -- 172.21.15.12:0/1167113058 <== mgr.14152 v2:172.21.15.12:6800/1388514047 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f69bc19af00 con 0x7f69a803e510 2024-05-17T22:27:46.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:46.672+0000 7f69c4416700 1 -- 172.21.15.12:0/1167113058 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f69a803e510 msgr2=0x7f69a8040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:46.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:46.672+0000 7f69c4416700 1 --2- 172.21.15.12:0/1167113058 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f69a803e510 0x7f69a8040a20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f69ac006fd0 tx=0x7f69ac006e40 comp rx=0 tx=0).stop 2024-05-17T22:27:46.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:46.672+0000 7f69c4416700 1 -- 172.21.15.12:0/1167113058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69bc108490 msgr2=0x7f69bc19a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:46.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:46.672+0000 7f69c4416700 1 --2- 172.21.15.12:0/1167113058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69bc108490 0x7f69bc19a160 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f69b80092e0 tx=0x7f69b8005780 comp rx=0 tx=0).stop 2024-05-17T22:27:46.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:46.672+0000 7f69c4416700 1 -- 172.21.15.12:0/1167113058 shutdown_connections 2024-05-17T22:27:46.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:46.672+0000 7f69c4416700 1 --2- 172.21.15.12:0/1167113058 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f69a803e510 0x7f69a8040a20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:46.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:46.672+0000 7f69c4416700 1 --2- 172.21.15.12:0/1167113058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69bc108490 0x7f69bc19a160 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:46.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:46.672+0000 7f69c4416700 1 -- 172.21.15.12:0/1167113058 >> 172.21.15.12:0/1167113058 conn(0x7f69bc0fe160 msgr2=0x7f69bc100a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:27:46.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:46.673+0000 7f69c4416700 1 -- 172.21.15.12:0/1167113058 shutdown_connections 2024-05-17T22:27:46.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:46.673+0000 7f69c4416700 1 -- 172.21.15.12:0/1167113058 wait complete. 2024-05-17T22:27:47.284 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph orch host ls --format=json 2024-05-17T22:27:47.530 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:27:47.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:47 smithi012 ceph-mon[31632]: Updating smithi096:/var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-17T22:27:47.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:47 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:27:47.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:47 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:27:47.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:47 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:27:47.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:47 smithi012 ceph-mon[31632]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:27:47.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:47 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:27:47.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:47 smithi012 ceph-mon[31632]: Added host smithi151 2024-05-17T22:27:47.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:47 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:27:47.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:47 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:27:48.930 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:48 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:27:49.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:49.136+0000 7f7a3e9e1700 1 -- 172.21.15.12:0/439323823 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a381023c0 msgr2=0x7f7a381027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:49.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:49.136+0000 7f7a3e9e1700 1 --2- 172.21.15.12:0/439323823 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a381023c0 0x7f7a381027d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7a28009910 tx=0x7f7a28009c20 comp rx=0 tx=0).stop 2024-05-17T22:27:49.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:49.136+0000 7f7a3e9e1700 1 -- 172.21.15.12:0/439323823 shutdown_connections 2024-05-17T22:27:49.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:49.136+0000 7f7a3e9e1700 1 --2- 172.21.15.12:0/439323823 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a381023c0 0x7f7a381027d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:49.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:49.136+0000 7f7a3e9e1700 1 -- 172.21.15.12:0/439323823 >> 172.21.15.12:0/439323823 conn(0x7f7a380fe0b0 msgr2=0x7f7a38100500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:27:49.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:49.136+0000 7f7a3e9e1700 1 -- 172.21.15.12:0/439323823 shutdown_connections 2024-05-17T22:27:49.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:49.136+0000 7f7a3e9e1700 1 -- 172.21.15.12:0/439323823 wait complete. 2024-05-17T22:27:49.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:49.137+0000 7f7a3e9e1700 1 Processor -- start 2024-05-17T22:27:49.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:49.137+0000 7f7a3e9e1700 1 -- start start 2024-05-17T22:27:49.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:49.137+0000 7f7a3e9e1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a381023c0 0x7f7a3819b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:49.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:49.137+0000 7f7a3e9e1700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3810fc90 con 0x7f7a381023c0 2024-05-17T22:27:49.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:49.137+0000 7f7a37fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a381023c0 0x7f7a3819b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:49.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:49.137+0000 7f7a37fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a381023c0 0x7f7a3819b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49422/0 (socket says 172.21.15.12:49422) 2024-05-17T22:27:49.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:49.137+0000 7f7a37fff700 1 -- 172.21.15.12:0/531275088 learned_addr learned my addr 172.21.15.12:0/531275088 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:27:49.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:49.138+0000 7f7a37fff700 1 -- 172.21.15.12:0/531275088 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a3819ba50 con 0x7f7a381023c0 2024-05-17T22:27:49.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:49.138+0000 7f7a37fff700 1 --2- 172.21.15.12:0/531275088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a381023c0 0x7f7a3819b510 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7a280056d0 tx=0x7f7a280057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:49.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:49.138+0000 7f7a357fa700 1 -- 172.21.15.12:0/531275088 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a28026070 con 0x7f7a381023c0 2024-05-17T22:27:49.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:49.138+0000 7f7a357fa700 1 -- 172.21.15.12:0/531275088 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a2800bc00 con 0x7f7a381023c0 2024-05-17T22:27:49.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:49.138+0000 7f7a3e9e1700 1 -- 172.21.15.12:0/531275088 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a3819bce0 con 0x7f7a381023c0 2024-05-17T22:27:49.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:49.138+0000 7f7a357fa700 1 -- 172.21.15.12:0/531275088 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a28003920 con 0x7f7a381023c0 2024-05-17T22:27:49.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:49.138+0000 7f7a3e9e1700 1 -- 172.21.15.12:0/531275088 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a3819c140 con 0x7f7a381023c0 2024-05-17T22:27:49.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:49.139+0000 7f7a357fa700 1 -- 172.21.15.12:0/531275088 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f7a28029070 con 0x7f7a381023c0 2024-05-17T22:27:49.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:49.139+0000 7f7a3e9e1700 1 -- 172.21.15.12:0/531275088 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a3806c100 con 0x7f7a381023c0 2024-05-17T22:27:49.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:49.140+0000 7f7a357fa700 1 --2- 172.21.15.12:0/531275088 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f7a2003e570 0x7f7a20040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:49.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:49.140+0000 7f7a357fa700 1 -- 172.21.15.12:0/531275088 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7a2801e070 con 0x7f7a381023c0 2024-05-17T22:27:49.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:49.140+0000 7f7a377fe700 1 --2- 172.21.15.12:0/531275088 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f7a2003e570 0x7f7a20040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:49.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:49.140+0000 7f7a377fe700 1 --2- 172.21.15.12:0/531275088 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f7a2003e570 0x7f7a20040a80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c006fd0 tx=0x7f7a2c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:49.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:49.145+0000 7f7a357fa700 1 -- 172.21.15.12:0/531275088 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7a28013070 con 0x7f7a381023c0 2024-05-17T22:27:49.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:49.367+0000 7f7a3e9e1700 1 -- 172.21.15.12:0/531275088 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7a3819c8e0 con 0x7f7a2003e570 2024-05-17T22:27:49.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:49.368+0000 7f7a357fa700 1 -- 172.21.15.12:0/531275088 <== mgr.14152 v2:172.21.15.12:6800/1388514047 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f7a3819c8e0 con 0x7f7a2003e570 2024-05-17T22:27:49.370 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:27:49.370 INFO:teuthology.orchestra.run.smithi012.stdout:[{"addr": "172.21.15.12", "hostname": "smithi012", "labels": [], "status": ""}, {"addr": "172.21.15.96", "hostname": "smithi096", "labels": [], "status": ""}, {"addr": "172.21.15.151", "hostname": "smithi151", "labels": [], "status": ""}] 2024-05-17T22:27:49.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:49.370+0000 7f7a3e9e1700 1 -- 172.21.15.12:0/531275088 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f7a2003e570 msgr2=0x7f7a20040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:49.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:49.370+0000 7f7a3e9e1700 1 --2- 172.21.15.12:0/531275088 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f7a2003e570 0x7f7a20040a80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c006fd0 tx=0x7f7a2c006e40 comp rx=0 tx=0).stop 2024-05-17T22:27:49.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:49.370+0000 7f7a3e9e1700 1 -- 172.21.15.12:0/531275088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a381023c0 msgr2=0x7f7a3819b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:49.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:49.370+0000 7f7a3e9e1700 1 --2- 172.21.15.12:0/531275088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a381023c0 0x7f7a3819b510 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7a280056d0 tx=0x7f7a280057b0 comp rx=0 tx=0).stop 2024-05-17T22:27:49.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:49.370+0000 7f7a3e9e1700 1 -- 172.21.15.12:0/531275088 shutdown_connections 2024-05-17T22:27:49.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:49.370+0000 7f7a3e9e1700 1 --2- 172.21.15.12:0/531275088 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f7a2003e570 0x7f7a20040a80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:49.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:49.370+0000 7f7a3e9e1700 1 --2- 172.21.15.12:0/531275088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a381023c0 0x7f7a3819b510 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:49.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:49.370+0000 7f7a3e9e1700 1 -- 172.21.15.12:0/531275088 >> 172.21.15.12:0/531275088 conn(0x7f7a380fe0b0 msgr2=0x7f7a38100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:27:49.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:49.371+0000 7f7a3e9e1700 1 -- 172.21.15.12:0/531275088 shutdown_connections 2024-05-17T22:27:49.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:49.371+0000 7f7a3e9e1700 1 -- 172.21.15.12:0/531275088 wait complete. 2024-05-17T22:27:49.958 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:49 smithi012 ceph-mon[31632]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:27:49.958 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:49 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:27:49.988 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-17T22:27:49.988 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph osd crush tunables default 2024-05-17T22:27:50.234 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:27:50.997 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:50 smithi012 ceph-mon[31632]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T22:27:51.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:51.755+0000 7f6da845d700 1 -- 172.21.15.12:0/2184890037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6da01095d0 msgr2=0x7f6da01099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:51.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:51.755+0000 7f6da845d700 1 --2- 172.21.15.12:0/2184890037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6da01095d0 0x7f6da01099e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c009910 tx=0x7f6d9c009c20 comp rx=0 tx=0).stop 2024-05-17T22:27:51.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:51.755+0000 7f6da845d700 1 -- 172.21.15.12:0/2184890037 shutdown_connections 2024-05-17T22:27:51.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:51.755+0000 7f6da845d700 1 --2- 172.21.15.12:0/2184890037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6da01095d0 0x7f6da01099e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:51.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:51.755+0000 7f6da845d700 1 -- 172.21.15.12:0/2184890037 >> 172.21.15.12:0/2184890037 conn(0x7f6da00ff220 msgr2=0x7f6da0101670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:27:51.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:51.755+0000 7f6da845d700 1 -- 172.21.15.12:0/2184890037 shutdown_connections 2024-05-17T22:27:51.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:51.755+0000 7f6da845d700 1 -- 172.21.15.12:0/2184890037 wait complete. 2024-05-17T22:27:51.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:51.756+0000 7f6da845d700 1 Processor -- start 2024-05-17T22:27:51.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:51.756+0000 7f6da845d700 1 -- start start 2024-05-17T22:27:51.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:51.757+0000 7f6da845d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6da01095d0 0x7f6da01992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:51.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:51.757+0000 7f6da845d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6da010dac0 con 0x7f6da01095d0 2024-05-17T22:27:51.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:51.757+0000 7f6da61f9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6da01095d0 0x7f6da01992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:51.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:51.757+0000 7f6da61f9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6da01095d0 0x7f6da01992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53740/0 (socket says 172.21.15.12:53740) 2024-05-17T22:27:51.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:51.757+0000 7f6da61f9700 1 -- 172.21.15.12:0/2220548000 learned_addr learned my addr 172.21.15.12:0/2220548000 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:27:51.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:51.757+0000 7f6da61f9700 1 -- 172.21.15.12:0/2220548000 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6da01997f0 con 0x7f6da01095d0 2024-05-17T22:27:51.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:51.757+0000 7f6da61f9700 1 --2- 172.21.15.12:0/2220548000 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6da01095d0 0x7f6da01992b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c0056d0 tx=0x7f6d9c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:51.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:51.757+0000 7f6d937fe700 1 -- 172.21.15.12:0/2220548000 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d9c027070 con 0x7f6da01095d0 2024-05-17T22:27:51.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:51.757+0000 7f6d937fe700 1 -- 172.21.15.12:0/2220548000 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d9c00be10 con 0x7f6da01095d0 2024-05-17T22:27:51.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:51.757+0000 7f6da845d700 1 -- 172.21.15.12:0/2220548000 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6da0199a80 con 0x7f6da01095d0 2024-05-17T22:27:51.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:51.757+0000 7f6d937fe700 1 -- 172.21.15.12:0/2220548000 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d9c022d20 con 0x7f6da01095d0 2024-05-17T22:27:51.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:51.757+0000 7f6da845d700 1 -- 172.21.15.12:0/2220548000 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6da019dae0 con 0x7f6da01095d0 2024-05-17T22:27:51.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:51.758+0000 7f6da845d700 1 -- 172.21.15.12:0/2220548000 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6da0104c10 con 0x7f6da01095d0 2024-05-17T22:27:51.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:51.758+0000 7f6d937fe700 1 -- 172.21.15.12:0/2220548000 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f6d9c02a070 con 0x7f6da01095d0 2024-05-17T22:27:51.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:51.759+0000 7f6d937fe700 1 --2- 172.21.15.12:0/2220548000 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f6d8c03e510 0x7f6d8c040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:51.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:51.759+0000 7f6d937fe700 1 -- 172.21.15.12:0/2220548000 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6d9c01e030 con 0x7f6da01095d0 2024-05-17T22:27:51.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:51.759+0000 7f6da59f8700 1 --2- 172.21.15.12:0/2220548000 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f6d8c03e510 0x7f6d8c040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:51.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:51.759+0000 7f6da59f8700 1 --2- 172.21.15.12:0/2220548000 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f6d8c03e510 0x7f6d8c040a20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6d94006fd0 tx=0x7f6d94006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:51.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:51.764+0000 7f6d937fe700 1 -- 172.21.15.12:0/2220548000 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6d9c013070 con 0x7f6da01095d0 2024-05-17T22:27:51.840 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:51 smithi012 ceph-mon[31632]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:27:51.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:51.985+0000 7f6da845d700 1 -- 172.21.15.12:0/2220548000 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f6da019a610 con 0x7f6da01095d0 2024-05-17T22:27:52.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:52.718+0000 7f6d937fe700 1 -- 172.21.15.12:0/2220548000 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f6d9c00d340 con 0x7f6da01095d0 2024-05-17T22:27:52.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:52.721+0000 7f6da845d700 1 -- 172.21.15.12:0/2220548000 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f6d8c03e510 msgr2=0x7f6d8c040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:52.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:52.721+0000 7f6da845d700 1 --2- 172.21.15.12:0/2220548000 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f6d8c03e510 0x7f6d8c040a20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6d94006fd0 tx=0x7f6d94006e40 comp rx=0 tx=0).stop 2024-05-17T22:27:52.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:52.721+0000 7f6da845d700 1 -- 172.21.15.12:0/2220548000 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6da01095d0 msgr2=0x7f6da01992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:52.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:52.721+0000 7f6da845d700 1 --2- 172.21.15.12:0/2220548000 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6da01095d0 0x7f6da01992b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c0056d0 tx=0x7f6d9c0057b0 comp rx=0 tx=0).stop 2024-05-17T22:27:52.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:52.721+0000 7f6da845d700 1 -- 172.21.15.12:0/2220548000 shutdown_connections 2024-05-17T22:27:52.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:52.721+0000 7f6da845d700 1 --2- 172.21.15.12:0/2220548000 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f6d8c03e510 0x7f6d8c040a20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:52.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:52.721+0000 7f6da845d700 1 --2- 172.21.15.12:0/2220548000 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6da01095d0 0x7f6da01992b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:52.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:52.721+0000 7f6da845d700 1 -- 172.21.15.12:0/2220548000 >> 172.21.15.12:0/2220548000 conn(0x7f6da00ff220 msgr2=0x7f6da0101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:27:52.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:52.721+0000 7f6da845d700 1 -- 172.21.15.12:0/2220548000 shutdown_connections 2024-05-17T22:27:52.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:27:52.721+0000 7f6da845d700 1 -- 172.21.15.12:0/2220548000 wait complete. 2024-05-17T22:27:52.724 INFO:teuthology.orchestra.run.smithi012.stderr:adjusted tunables profile to default 2024-05-17T22:27:53.079 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:52 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/2220548000' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-17T22:27:53.079 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:52 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:27:53.079 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:52 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:27:53.079 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:52 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:27:53.079 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:52 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:27:53.079 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:52 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi151", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:27:53.079 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:52 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:27:53.079 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:52 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:27:53.364 INFO:tasks.cephadm:Adding mon.a on smithi012 2024-05-17T22:27:53.364 INFO:tasks.cephadm:Adding mon.b on smithi096 2024-05-17T22:27:53.365 INFO:tasks.cephadm:Adding mon.c on smithi151 2024-05-17T22:27:53.365 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph orch apply mon '3;smithi012:172.21.15.12=a;smithi096:172.21.15.96=b;smithi151:172.21.15.151=c' 2024-05-17T22:27:53.590 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T22:27:54.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:53 smithi012 ceph-mon[31632]: Updating smithi151:/etc/ceph/ceph.conf 2024-05-17T22:27:54.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:53 smithi012 ceph-mon[31632]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:27:54.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:53 smithi012 ceph-mon[31632]: Updating smithi151:/var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T22:27:54.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:53 smithi012 ceph-mon[31632]: Updating smithi151:/etc/ceph/ceph.client.admin.keyring 2024-05-17T22:27:54.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:53 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/2220548000' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-17T22:27:54.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:53 smithi012 ceph-mon[31632]: osdmap e4: 0 total, 0 up, 0 in 2024-05-17T22:27:54.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:53 smithi012 ceph-mon[31632]: Updating smithi151:/var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-17T22:27:54.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:53 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:27:54.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:53 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:27:54.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:53 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:27:55.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:55.096+0000 7f0b412d0700 1 -- 172.21.15.151:0/1990404455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b3c1095b0 msgr2=0x7f0b3c1099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:55.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:55.096+0000 7f0b412d0700 1 --2- 172.21.15.151:0/1990404455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b3c1095b0 0x7f0b3c1099c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0b30009910 tx=0x7f0b30009c20 comp rx=0 tx=0).stop 2024-05-17T22:27:55.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:55.096+0000 7f0b412d0700 1 -- 172.21.15.151:0/1990404455 shutdown_connections 2024-05-17T22:27:55.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:55.096+0000 7f0b412d0700 1 --2- 172.21.15.151:0/1990404455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b3c1095b0 0x7f0b3c1099c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:55.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:55.096+0000 7f0b412d0700 1 -- 172.21.15.151:0/1990404455 >> 172.21.15.151:0/1990404455 conn(0x7f0b3c0ff1e0 msgr2=0x7f0b3c101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:27:55.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:55.096+0000 7f0b412d0700 1 -- 172.21.15.151:0/1990404455 shutdown_connections 2024-05-17T22:27:55.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:55.096+0000 7f0b412d0700 1 -- 172.21.15.151:0/1990404455 wait complete. 2024-05-17T22:27:55.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:55.097+0000 7f0b412d0700 1 Processor -- start 2024-05-17T22:27:55.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:55.097+0000 7f0b412d0700 1 -- start start 2024-05-17T22:27:55.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:55.097+0000 7f0b412d0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b3c1095b0 0x7f0b3c075b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:55.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:55.097+0000 7f0b412d0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0b3c10daa0 con 0x7f0b3c1095b0 2024-05-17T22:27:55.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:55.098+0000 7f0b3affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b3c1095b0 0x7f0b3c075b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:55.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:55.098+0000 7f0b3affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b3c1095b0 0x7f0b3c075b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.151:39550/0 (socket says 172.21.15.151:39550) 2024-05-17T22:27:55.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:55.098+0000 7f0b3affd700 1 -- 172.21.15.151:0/4166612211 learned_addr learned my addr 172.21.15.151:0/4166612211 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-17T22:27:55.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:55.098+0000 7f0b3affd700 1 -- 172.21.15.151:0/4166612211 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b3c077980 con 0x7f0b3c1095b0 2024-05-17T22:27:55.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:55.098+0000 7f0b3affd700 1 --2- 172.21.15.151:0/4166612211 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b3c1095b0 0x7f0b3c075b30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0b300092e0 tx=0x7f0b30005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:55.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:55.098+0000 7f0b23fff700 1 -- 172.21.15.151:0/4166612211 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b30026070 con 0x7f0b3c1095b0 2024-05-17T22:27:55.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:55.098+0000 7f0b412d0700 1 -- 172.21.15.151:0/4166612211 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b3c076070 con 0x7f0b3c1095b0 2024-05-17T22:27:55.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:55.098+0000 7f0b23fff700 1 -- 172.21.15.151:0/4166612211 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b3000b750 con 0x7f0b3c1095b0 2024-05-17T22:27:55.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:55.098+0000 7f0b23fff700 1 -- 172.21.15.151:0/4166612211 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b300038a0 con 0x7f0b3c1095b0 2024-05-17T22:27:55.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:55.098+0000 7f0b412d0700 1 -- 172.21.15.151:0/4166612211 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b3c076470 con 0x7f0b3c1095b0 2024-05-17T22:27:55.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:55.099+0000 7f0b412d0700 1 -- 172.21.15.151:0/4166612211 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b3c104bf0 con 0x7f0b3c1095b0 2024-05-17T22:27:55.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:55.100+0000 7f0b23fff700 1 -- 172.21.15.151:0/4166612211 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f0b30029070 con 0x7f0b3c1095b0 2024-05-17T22:27:55.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:55.100+0000 7f0b23fff700 1 --2- 172.21.15.151:0/4166612211 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f0b2403e510 0x7f0b24040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:55.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:55.100+0000 7f0b23fff700 1 -- 172.21.15.151:0/4166612211 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0b3001e070 con 0x7f0b3c1095b0 2024-05-17T22:27:55.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:55.100+0000 7f0b3a7fc700 1 --2- 172.21.15.151:0/4166612211 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f0b2403e510 0x7f0b24040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:55.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:55.101+0000 7f0b3a7fc700 1 --2- 172.21.15.151:0/4166612211 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f0b2403e510 0x7f0b24040a20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c006fd0 tx=0x7f0b2c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:55.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:55.105+0000 7f0b23fff700 1 -- 172.21.15.151:0/4166612211 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0b30013070 con 0x7f0b3c1095b0 2024-05-17T22:27:55.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:55.320+0000 7f0b412d0700 1 -- 172.21.15.151:0/4166612211 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi012:172.21.15.12=a;smithi096:172.21.15.96=b;smithi151:172.21.15.151=c", "target": ["mon-mgr", ""]}) -- 0x7f0b3c077100 con 0x7f0b2403e510 2024-05-17T22:27:55.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:55.328+0000 7f0b23fff700 1 -- 172.21.15.151:0/4166612211 <== mgr.14152 v2:172.21.15.12:6800/1388514047 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0b3c077100 con 0x7f0b2403e510 2024-05-17T22:27:55.330 INFO:teuthology.orchestra.run.smithi151.stdout:Scheduled mon update... 2024-05-17T22:27:55.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:55.329+0000 7f0b412d0700 1 -- 172.21.15.151:0/4166612211 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f0b2403e510 msgr2=0x7f0b24040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:55.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:55.329+0000 7f0b412d0700 1 --2- 172.21.15.151:0/4166612211 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f0b2403e510 0x7f0b24040a20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c006fd0 tx=0x7f0b2c006e40 comp rx=0 tx=0).stop 2024-05-17T22:27:55.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:55.329+0000 7f0b412d0700 1 -- 172.21.15.151:0/4166612211 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b3c1095b0 msgr2=0x7f0b3c075b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:55.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:55.329+0000 7f0b412d0700 1 --2- 172.21.15.151:0/4166612211 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b3c1095b0 0x7f0b3c075b30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0b300092e0 tx=0x7f0b30005780 comp rx=0 tx=0).stop 2024-05-17T22:27:55.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:55.329+0000 7f0b412d0700 1 -- 172.21.15.151:0/4166612211 shutdown_connections 2024-05-17T22:27:55.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:55.329+0000 7f0b412d0700 1 --2- 172.21.15.151:0/4166612211 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f0b2403e510 0x7f0b24040a20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:55.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:55.329+0000 7f0b412d0700 1 --2- 172.21.15.151:0/4166612211 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b3c1095b0 0x7f0b3c075b30 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:55.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:55.329+0000 7f0b412d0700 1 -- 172.21.15.151:0/4166612211 >> 172.21.15.151:0/4166612211 conn(0x7f0b3c0ff1e0 msgr2=0x7f0b3c100170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:27:55.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:55.329+0000 7f0b412d0700 1 -- 172.21.15.151:0/4166612211 shutdown_connections 2024-05-17T22:27:55.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:55.329+0000 7f0b412d0700 1 -- 172.21.15.151:0/4166612211 wait complete. 2024-05-17T22:27:55.942 DEBUG:teuthology.orchestra.run.smithi096:mon.b> sudo journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@mon.b.service 2024-05-17T22:27:55.945 DEBUG:teuthology.orchestra.run.smithi151:mon.c> sudo journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@mon.c.service 2024-05-17T22:27:55.947 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-17T22:27:55.948 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-17T22:27:56.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:55 smithi012 ceph-mon[31632]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:27:56.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:55 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:27:56.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:55 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:27:56.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:55 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:27:56.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:55 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:27:56.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:55 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:27:56.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:55 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T22:27:56.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:55 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:27:56.669 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T22:27:57.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:56 smithi012 ceph-mon[31632]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi012:172.21.15.12=a;smithi096:172.21.15.96=b;smithi151:172.21.15.151=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:27:57.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:56 smithi012 ceph-mon[31632]: Saving service mon spec with placement smithi012:172.21.15.12=a;smithi096:172.21.15.96=b;smithi151:172.21.15.151=c;count:3 2024-05-17T22:27:57.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:56 smithi012 ceph-mon[31632]: Deploying daemon mon.c on smithi151 2024-05-17T22:27:57.992 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:57 smithi012 ceph-mon[31632]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:27:59.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:59.978+0000 7f3912aaf700 1 -- 172.21.15.151:0/672891521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f390c1095d0 msgr2=0x7f390c1099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:27:59.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:59.978+0000 7f3912aaf700 1 --2- 172.21.15.151:0/672891521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f390c1095d0 0x7f390c1099e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3908009910 tx=0x7f3908009c20 comp rx=0 tx=0).stop 2024-05-17T22:27:59.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:59.978+0000 7f3912aaf700 1 -- 172.21.15.151:0/672891521 shutdown_connections 2024-05-17T22:27:59.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:59.978+0000 7f3912aaf700 1 --2- 172.21.15.151:0/672891521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f390c1095d0 0x7f390c1099e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:27:59.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:59.978+0000 7f3912aaf700 1 -- 172.21.15.151:0/672891521 >> 172.21.15.151:0/672891521 conn(0x7f390c0ff200 msgr2=0x7f390c101650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:27:59.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:59.978+0000 7f3912aaf700 1 -- 172.21.15.151:0/672891521 shutdown_connections 2024-05-17T22:27:59.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:59.978+0000 7f3912aaf700 1 -- 172.21.15.151:0/672891521 wait complete. 2024-05-17T22:27:59.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:59.979+0000 7f3912aaf700 1 Processor -- start 2024-05-17T22:27:59.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:59.979+0000 7f3912aaf700 1 -- start start 2024-05-17T22:27:59.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:59.979+0000 7f3912aaf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f390c1095d0 0x7f390c075b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:59.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:59.979+0000 7f3912aaf700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f390c10dac0 con 0x7f390c1095d0 2024-05-17T22:27:59.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:59.979+0000 7f391084b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f390c1095d0 0x7f390c075b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:59.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:59.979+0000 7f391084b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f390c1095d0 0x7f390c075b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.151:41744/0 (socket says 172.21.15.151:41744) 2024-05-17T22:27:59.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:59.979+0000 7f391084b700 1 -- 172.21.15.151:0/2038040273 learned_addr learned my addr 172.21.15.151:0/2038040273 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-17T22:27:59.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:59.980+0000 7f391084b700 1 -- 172.21.15.151:0/2038040273 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f390c077980 con 0x7f390c1095d0 2024-05-17T22:27:59.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:59.980+0000 7f391084b700 1 --2- 172.21.15.151:0/2038040273 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f390c1095d0 0x7f390c075b30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f39080056d0 tx=0x7f39080057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:59.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:59.980+0000 7f3901ffb700 1 -- 172.21.15.151:0/2038040273 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3908027070 con 0x7f390c1095d0 2024-05-17T22:27:59.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:59.980+0000 7f3901ffb700 1 -- 172.21.15.151:0/2038040273 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f390800be10 con 0x7f390c1095d0 2024-05-17T22:27:59.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:59.980+0000 7f3912aaf700 1 -- 172.21.15.151:0/2038040273 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f390c076070 con 0x7f390c1095d0 2024-05-17T22:27:59.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:59.980+0000 7f3901ffb700 1 -- 172.21.15.151:0/2038040273 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3908022d20 con 0x7f390c1095d0 2024-05-17T22:27:59.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:59.980+0000 7f3912aaf700 1 -- 172.21.15.151:0/2038040273 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f390c076470 con 0x7f390c1095d0 2024-05-17T22:27:59.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:59.981+0000 7f3912aaf700 1 -- 172.21.15.151:0/2038040273 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f390c104c10 con 0x7f390c1095d0 2024-05-17T22:27:59.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:59.981+0000 7f3901ffb700 1 -- 172.21.15.151:0/2038040273 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f390802a070 con 0x7f390c1095d0 2024-05-17T22:27:59.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:59.982+0000 7f3901ffb700 1 --2- 172.21.15.151:0/2038040273 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f38f403e510 0x7f38f4040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:27:59.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:59.982+0000 7f3901ffb700 1 -- 172.21.15.151:0/2038040273 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f390801e030 con 0x7f390c1095d0 2024-05-17T22:27:59.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:59.982+0000 7f3903fff700 1 --2- 172.21.15.151:0/2038040273 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f38f403e510 0x7f38f4040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:27:59.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:59.982+0000 7f3903fff700 1 --2- 172.21.15.151:0/2038040273 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f38f403e510 0x7f38f4040a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f38fc006fd0 tx=0x7f38fc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:27:59.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:27:59.991+0000 7f3901ffb700 1 -- 172.21.15.151:0/2038040273 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3908013070 con 0x7f390c1095d0 2024-05-17T22:28:00.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:27:59 smithi012 ceph-mon[31632]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:00.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:00.292+0000 7f3912aaf700 1 -- 172.21.15.151:0/2038040273 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f390c1a3480 con 0x7f390c1095d0 2024-05-17T22:28:00.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:00.293+0000 7f3901ffb700 1 -- 172.21.15.151:0/2038040273 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f3908031090 con 0x7f390c1095d0 2024-05-17T22:28:00.296 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-17T22:28:00.297 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":1,"fsid":"4d593780-149c-11ef-bc9a-c7b262605968","modified":"2024-05-17T22:26:04.404494Z","created":"2024-05-17T22:26:04.404494Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-17T22:28:00.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:00.296+0000 7f3912aaf700 1 -- 172.21.15.151:0/2038040273 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f38f403e510 msgr2=0x7f38f4040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:28:00.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:00.296+0000 7f3912aaf700 1 --2- 172.21.15.151:0/2038040273 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f38f403e510 0x7f38f4040a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f38fc006fd0 tx=0x7f38fc006e40 comp rx=0 tx=0).stop 2024-05-17T22:28:00.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:00.296+0000 7f3912aaf700 1 -- 172.21.15.151:0/2038040273 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f390c1095d0 msgr2=0x7f390c075b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:28:00.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:00.296+0000 7f3912aaf700 1 --2- 172.21.15.151:0/2038040273 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f390c1095d0 0x7f390c075b30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f39080056d0 tx=0x7f39080057b0 comp rx=0 tx=0).stop 2024-05-17T22:28:00.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:00.296+0000 7f3912aaf700 1 -- 172.21.15.151:0/2038040273 shutdown_connections 2024-05-17T22:28:00.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:00.296+0000 7f3912aaf700 1 --2- 172.21.15.151:0/2038040273 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f38f403e510 0x7f38f4040a20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:28:00.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:00.296+0000 7f3912aaf700 1 --2- 172.21.15.151:0/2038040273 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f390c1095d0 0x7f390c075b30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:28:00.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:00.296+0000 7f3912aaf700 1 -- 172.21.15.151:0/2038040273 >> 172.21.15.151:0/2038040273 conn(0x7f390c0ff200 msgr2=0x7f390c101b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:28:00.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:00.296+0000 7f3912aaf700 1 -- 172.21.15.151:0/2038040273 shutdown_connections 2024-05-17T22:28:00.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:00.296+0000 7f3912aaf700 1 -- 172.21.15.151:0/2038040273 wait complete. 2024-05-17T22:28:00.299 INFO:teuthology.orchestra.run.smithi151.stderr:dumped monmap epoch 1 2024-05-17T22:28:00.423 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 podman[36070]: 2024-05-17 22:28:00.421130524 +0000 UTC m=+0.362566264 container init 4e7aba1032ccfa00d24703300e98d418673be9849b41ccf2981f57a0ac8e13ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mon-c, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T22:28:00.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 podman[36070]: 2024-05-17 22:28:00.425204316 +0000 UTC m=+0.366640057 container start 4e7aba1032ccfa00d24703300e98d418673be9849b41ccf2981f57a0ac8e13ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mon-c, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-17T22:28:00.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: set uid:gid to 167:167 (ceph:ceph) 2024-05-17T22:28:00.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev), process ceph-mon, pid 2 2024-05-17T22:28:00.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: pidfile_write: ignore empty --pid-file 2024-05-17T22:28:00.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: load: jerasure load: lrc 2024-05-17T22:28:00.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: RocksDB version: 7.9.2 2024-05-17T22:28:00.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Git sha 0 2024-05-17T22:28:00.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Compile date 2024-05-17 01:53:57 2024-05-17T22:28:00.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: DB SUMMARY 2024-05-17T22:28:00.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: DB Session ID: 2DG32HLUYCX96JDKJJRL 2024-05-17T22:28:00.742 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: CURRENT file: CURRENT 2024-05-17T22:28:00.742 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: IDENTITY file: IDENTITY 2024-05-17T22:28:00.742 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-17T22:28:00.742 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-05-17T22:28:00.742 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-05-17T22:28:00.742 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.error_if_exists: 0 2024-05-17T22:28:00.742 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.create_if_missing: 0 2024-05-17T22:28:00.742 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.paranoid_checks: 1 2024-05-17T22:28:00.742 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-17T22:28:00.742 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-17T22:28:00.742 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-17T22:28:00.743 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.env: 0x5604a9c30500 2024-05-17T22:28:00.743 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.fs: PosixFileSystem 2024-05-17T22:28:00.743 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.info_log: 0x5604ab69a080 2024-05-17T22:28:00.743 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.max_file_opening_threads: 16 2024-05-17T22:28:00.743 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.statistics: (nil) 2024-05-17T22:28:00.743 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.use_fsync: 0 2024-05-17T22:28:00.743 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.max_log_file_size: 0 2024-05-17T22:28:00.743 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-17T22:28:00.743 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-17T22:28:00.743 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.keep_log_file_num: 1000 2024-05-17T22:28:00.743 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.recycle_log_file_num: 0 2024-05-17T22:28:00.744 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.allow_fallocate: 1 2024-05-17T22:28:00.744 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.allow_mmap_reads: 0 2024-05-17T22:28:00.744 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.allow_mmap_writes: 0 2024-05-17T22:28:00.744 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.use_direct_reads: 0 2024-05-17T22:28:00.744 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-17T22:28:00.744 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.create_missing_column_families: 0 2024-05-17T22:28:00.744 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.db_log_dir: 2024-05-17T22:28:00.744 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.wal_dir: 2024-05-17T22:28:00.744 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-17T22:28:00.744 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-17T22:28:00.744 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-17T22:28:00.744 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-17T22:28:00.744 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-17T22:28:00.744 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-17T22:28:00.744 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.advise_random_on_open: 1 2024-05-17T22:28:00.745 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.db_write_buffer_size: 0 2024-05-17T22:28:00.745 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.write_buffer_manager: 0x5604ab54eb40 2024-05-17T22:28:00.745 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-17T22:28:00.745 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-17T22:28:00.745 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-17T22:28:00.745 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.rate_limiter: (nil) 2024-05-17T22:28:00.745 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-17T22:28:00.745 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.wal_recovery_mode: 2 2024-05-17T22:28:00.745 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.enable_thread_tracking: 0 2024-05-17T22:28:00.745 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.enable_pipelined_write: 0 2024-05-17T22:28:00.745 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.unordered_write: 0 2024-05-17T22:28:00.745 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-17T22:28:00.745 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-17T22:28:00.745 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-17T22:28:00.745 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-17T22:28:00.746 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.row_cache: None 2024-05-17T22:28:00.746 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.wal_filter: None 2024-05-17T22:28:00.746 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-17T22:28:00.746 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.allow_ingest_behind: 0 2024-05-17T22:28:00.746 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.two_write_queues: 0 2024-05-17T22:28:00.746 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.manual_wal_flush: 0 2024-05-17T22:28:00.746 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.wal_compression: 0 2024-05-17T22:28:00.746 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.atomic_flush: 0 2024-05-17T22:28:00.746 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-17T22:28:00.746 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-17T22:28:00.747 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-17T22:28:00.748 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.log_readahead_size: 0 2024-05-17T22:28:00.748 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-17T22:28:00.748 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.best_efforts_recovery: 0 2024-05-17T22:28:00.748 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-17T22:28:00.748 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-17T22:28:00.748 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.allow_data_in_errors: 0 2024-05-17T22:28:00.748 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.db_host_id: __hostname__ 2024-05-17T22:28:00.748 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-17T22:28:00.748 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.max_background_jobs: 2 2024-05-17T22:28:00.748 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.max_background_compactions: -1 2024-05-17T22:28:00.748 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.max_subcompactions: 1 2024-05-17T22:28:00.748 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-17T22:28:00.748 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-17T22:28:00.748 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-17T22:28:00.749 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.max_total_wal_size: 0 2024-05-17T22:28:00.749 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-17T22:28:00.749 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-17T22:28:00.749 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-17T22:28:00.749 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-17T22:28:00.749 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.max_open_files: -1 2024-05-17T22:28:00.749 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.bytes_per_sync: 0 2024-05-17T22:28:00.749 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-17T22:28:00.749 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-17T22:28:00.749 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.compaction_readahead_size: 0 2024-05-17T22:28:00.749 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.max_background_flushes: -1 2024-05-17T22:28:00.749 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Compression algorithms supported: 2024-05-17T22:28:00.749 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-17T22:28:00.749 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: kZSTD supported: 0 2024-05-17T22:28:00.750 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: kXpressCompression supported: 0 2024-05-17T22:28:00.750 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: kLZ4HCCompression supported: 1 2024-05-17T22:28:00.750 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: kZlibCompression supported: 1 2024-05-17T22:28:00.750 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: kSnappyCompression supported: 1 2024-05-17T22:28:00.750 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: kLZ4Compression supported: 1 2024-05-17T22:28:00.750 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: kBZip2Compression supported: 0 2024-05-17T22:28:00.750 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-17T22:28:00.750 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-17T22:28:00.750 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-05-17T22:28:00.750 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-17T22:28:00.750 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-17T22:28:00.750 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.merge_operator: 2024-05-17T22:28:00.751 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.compaction_filter: None 2024-05-17T22:28:00.751 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.compaction_filter_factory: None 2024-05-17T22:28:00.751 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.sst_partitioner_factory: None 2024-05-17T22:28:00.751 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-17T22:28:00.751 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-17T22:28:00.751 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5604ab69a240) 2024-05-17T22:28:00.751 INFO:journalctl@ceph.mon.c.smithi151.stdout: cache_index_and_filter_blocks: 1 2024-05-17T22:28:00.751 INFO:journalctl@ceph.mon.c.smithi151.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-17T22:28:00.751 INFO:journalctl@ceph.mon.c.smithi151.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-17T22:28:00.751 INFO:journalctl@ceph.mon.c.smithi151.stdout: pin_top_level_index_and_filter: 1 2024-05-17T22:28:00.751 INFO:journalctl@ceph.mon.c.smithi151.stdout: index_type: 0 2024-05-17T22:28:00.751 INFO:journalctl@ceph.mon.c.smithi151.stdout: data_block_index_type: 0 2024-05-17T22:28:00.751 INFO:journalctl@ceph.mon.c.smithi151.stdout: index_shortening: 1 2024-05-17T22:28:00.751 INFO:journalctl@ceph.mon.c.smithi151.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-17T22:28:00.751 INFO:journalctl@ceph.mon.c.smithi151.stdout: checksum: 4 2024-05-17T22:28:00.752 INFO:journalctl@ceph.mon.c.smithi151.stdout: no_block_cache: 0 2024-05-17T22:28:00.752 INFO:journalctl@ceph.mon.c.smithi151.stdout: block_cache: 0x5604ab5971f0 2024-05-17T22:28:00.752 INFO:journalctl@ceph.mon.c.smithi151.stdout: block_cache_name: BinnedLRUCache 2024-05-17T22:28:00.752 INFO:journalctl@ceph.mon.c.smithi151.stdout: block_cache_options: 2024-05-17T22:28:00.752 INFO:journalctl@ceph.mon.c.smithi151.stdout: capacity : 536870912 2024-05-17T22:28:00.752 INFO:journalctl@ceph.mon.c.smithi151.stdout: num_shard_bits : 4 2024-05-17T22:28:00.752 INFO:journalctl@ceph.mon.c.smithi151.stdout: strict_capacity_limit : 0 2024-05-17T22:28:00.752 INFO:journalctl@ceph.mon.c.smithi151.stdout: high_pri_pool_ratio: 0.000 2024-05-17T22:28:00.752 INFO:journalctl@ceph.mon.c.smithi151.stdout: block_cache_compressed: (nil) 2024-05-17T22:28:00.752 INFO:journalctl@ceph.mon.c.smithi151.stdout: persistent_cache: (nil) 2024-05-17T22:28:00.752 INFO:journalctl@ceph.mon.c.smithi151.stdout: block_size: 4096 2024-05-17T22:28:00.752 INFO:journalctl@ceph.mon.c.smithi151.stdout: block_size_deviation: 10 2024-05-17T22:28:00.753 INFO:journalctl@ceph.mon.c.smithi151.stdout: block_restart_interval: 16 2024-05-17T22:28:00.753 INFO:journalctl@ceph.mon.c.smithi151.stdout: index_block_restart_interval: 1 2024-05-17T22:28:00.753 INFO:journalctl@ceph.mon.c.smithi151.stdout: metadata_block_size: 4096 2024-05-17T22:28:00.753 INFO:journalctl@ceph.mon.c.smithi151.stdout: partition_filters: 0 2024-05-17T22:28:00.753 INFO:journalctl@ceph.mon.c.smithi151.stdout: use_delta_encoding: 1 2024-05-17T22:28:00.753 INFO:journalctl@ceph.mon.c.smithi151.stdout: filter_policy: bloomfilter 2024-05-17T22:28:00.753 INFO:journalctl@ceph.mon.c.smithi151.stdout: whole_key_filtering: 1 2024-05-17T22:28:00.753 INFO:journalctl@ceph.mon.c.smithi151.stdout: verify_compression: 0 2024-05-17T22:28:00.753 INFO:journalctl@ceph.mon.c.smithi151.stdout: read_amp_bytes_per_bit: 0 2024-05-17T22:28:00.753 INFO:journalctl@ceph.mon.c.smithi151.stdout: format_version: 5 2024-05-17T22:28:00.753 INFO:journalctl@ceph.mon.c.smithi151.stdout: enable_index_compression: 1 2024-05-17T22:28:00.753 INFO:journalctl@ceph.mon.c.smithi151.stdout: block_align: 0 2024-05-17T22:28:00.753 INFO:journalctl@ceph.mon.c.smithi151.stdout: max_auto_readahead_size: 262144 2024-05-17T22:28:00.753 INFO:journalctl@ceph.mon.c.smithi151.stdout: prepopulate_block_cache: 0 2024-05-17T22:28:00.754 INFO:journalctl@ceph.mon.c.smithi151.stdout: initial_auto_readahead_size: 8192 2024-05-17T22:28:00.754 INFO:journalctl@ceph.mon.c.smithi151.stdout: num_file_reads_for_auto_readahead: 2 2024-05-17T22:28:00.754 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.write_buffer_size: 33554432 2024-05-17T22:28:00.754 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.max_write_buffer_number: 2 2024-05-17T22:28:00.754 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.compression: NoCompression 2024-05-17T22:28:00.754 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.bottommost_compression: Disabled 2024-05-17T22:28:00.754 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.prefix_extractor: nullptr 2024-05-17T22:28:00.755 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-17T22:28:00.755 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.num_levels: 7 2024-05-17T22:28:00.755 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-17T22:28:00.755 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-17T22:28:00.755 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-17T22:28:00.755 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-17T22:28:00.756 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-17T22:28:00.756 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-17T22:28:00.756 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-17T22:28:00.756 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-17T22:28:00.756 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-17T22:28:00.756 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-17T22:28:00.756 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-17T22:28:00.756 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-17T22:28:00.756 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-17T22:28:00.756 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.compression_opts.level: 32767 2024-05-17T22:28:00.757 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.compression_opts.strategy: 0 2024-05-17T22:28:00.757 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-17T22:28:00.757 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-17T22:28:00.757 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-17T22:28:00.757 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-17T22:28:00.757 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.compression_opts.enabled: false 2024-05-17T22:28:00.757 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-17T22:28:00.757 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-17T22:28:00.757 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-17T22:28:00.757 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-17T22:28:00.757 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.target_file_size_base: 67108864 2024-05-17T22:28:00.757 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-17T22:28:00.757 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-17T22:28:00.757 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-17T22:28:00.758 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-17T22:28:00.758 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-17T22:28:00.758 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-17T22:28:00.758 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-17T22:28:00.758 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-17T22:28:00.758 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-17T22:28:00.758 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-17T22:28:00.758 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-17T22:28:00.758 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-17T22:28:00.758 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-17T22:28:00.759 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-17T22:28:00.759 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.arena_block_size: 1048576 2024-05-17T22:28:00.759 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-17T22:28:00.759 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-17T22:28:00.759 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.disable_auto_compactions: 0 2024-05-17T22:28:00.759 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-17T22:28:00.759 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-17T22:28:00.759 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-17T22:28:00.759 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-17T22:28:00.759 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-17T22:28:00.759 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-17T22:28:00.759 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-17T22:28:00.760 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-17T22:28:00.760 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-17T22:28:00.760 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-17T22:28:00.760 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-17T22:28:00.760 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.inplace_update_support: 0 2024-05-17T22:28:00.760 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-17T22:28:00.760 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-17T22:28:00.760 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-17T22:28:00.760 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-17T22:28:00.760 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.bloom_locality: 0 2024-05-17T22:28:00.760 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.max_successive_merges: 0 2024-05-17T22:28:00.760 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-17T22:28:00.760 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.paranoid_file_checks: 0 2024-05-17T22:28:00.760 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.force_consistency_checks: 1 2024-05-17T22:28:00.761 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.report_bg_io_stats: 0 2024-05-17T22:28:00.761 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.ttl: 2592000 2024-05-17T22:28:00.761 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-17T22:28:00.761 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-17T22:28:00.761 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-17T22:28:00.761 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.enable_blob_files: false 2024-05-17T22:28:00.761 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.min_blob_size: 0 2024-05-17T22:28:00.761 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.blob_file_size: 268435456 2024-05-17T22:28:00.761 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-17T22:28:00.761 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-17T22:28:00.761 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-17T22:28:00.761 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-17T22:28:00.761 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-17T22:28:00.762 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.blob_file_starting_level: 0 2024-05-17T22:28:00.762 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-17T22:28:00.762 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-17T22:28:00.762 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-17T22:28:00.762 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: bda196c8-588e-4018-874a-a04e7b7166d3 2024-05-17T22:28:00.762 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715984880465066, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-17T22:28:00.762 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-17T22:28:00.762 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715984880465488, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715984880, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "bda196c8-588e-4018-874a-a04e7b7166d3", "db_session_id": "2DG32HLUYCX96JDKJJRL", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-17T22:28:00.762 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715984880465573, "job": 1, "event": "recovery_finished"} 2024-05-17T22:28:00.763 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-17T22:28:00.763 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-17T22:28:00.763 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5604ab579c00 2024-05-17T22:28:00.763 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: DB pointer 0x5604b036e000 2024-05-17T22:28:00.763 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-17T22:28:00.763 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-17T22:28:00.763 INFO:journalctl@ceph.mon.c.smithi151.stdout: ** DB Stats ** 2024-05-17T22:28:00.763 INFO:journalctl@ceph.mon.c.smithi151.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-17T22:28:00.764 INFO:journalctl@ceph.mon.c.smithi151.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-17T22:28:00.764 INFO:journalctl@ceph.mon.c.smithi151.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-17T22:28:00.764 INFO:journalctl@ceph.mon.c.smithi151.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-17T22:28:00.764 INFO:journalctl@ceph.mon.c.smithi151.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-17T22:28:00.764 INFO:journalctl@ceph.mon.c.smithi151.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-17T22:28:00.764 INFO:journalctl@ceph.mon.c.smithi151.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-17T22:28:00.764 INFO:journalctl@ceph.mon.c.smithi151.stdout: 2024-05-17T22:28:00.765 INFO:journalctl@ceph.mon.c.smithi151.stdout: ** Compaction Stats [default] ** 2024-05-17T22:28:00.765 INFO:journalctl@ceph.mon.c.smithi151.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-17T22:28:00.765 INFO:journalctl@ceph.mon.c.smithi151.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-17T22:28:00.765 INFO:journalctl@ceph.mon.c.smithi151.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-17T22:28:00.765 INFO:journalctl@ceph.mon.c.smithi151.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-17T22:28:00.765 INFO:journalctl@ceph.mon.c.smithi151.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-17T22:28:00.765 INFO:journalctl@ceph.mon.c.smithi151.stdout: 2024-05-17T22:28:00.765 INFO:journalctl@ceph.mon.c.smithi151.stdout: ** Compaction Stats [default] ** 2024-05-17T22:28:00.765 INFO:journalctl@ceph.mon.c.smithi151.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-17T22:28:00.765 INFO:journalctl@ceph.mon.c.smithi151.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-17T22:28:00.765 INFO:journalctl@ceph.mon.c.smithi151.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-17T22:28:00.765 INFO:journalctl@ceph.mon.c.smithi151.stdout: 2024-05-17T22:28:00.765 INFO:journalctl@ceph.mon.c.smithi151.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-17T22:28:00.765 INFO:journalctl@ceph.mon.c.smithi151.stdout: 2024-05-17T22:28:00.766 INFO:journalctl@ceph.mon.c.smithi151.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-17T22:28:00.766 INFO:journalctl@ceph.mon.c.smithi151.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-17T22:28:00.766 INFO:journalctl@ceph.mon.c.smithi151.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-17T22:28:00.766 INFO:journalctl@ceph.mon.c.smithi151.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-17T22:28:00.766 INFO:journalctl@ceph.mon.c.smithi151.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-17T22:28:00.766 INFO:journalctl@ceph.mon.c.smithi151.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-17T22:28:00.766 INFO:journalctl@ceph.mon.c.smithi151.stdout: Cumulative compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-17T22:28:00.766 INFO:journalctl@ceph.mon.c.smithi151.stdout: Interval compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-17T22:28:00.766 INFO:journalctl@ceph.mon.c.smithi151.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-17T22:28:00.766 INFO:journalctl@ceph.mon.c.smithi151.stdout: Block cache BinnedLRUCache@0x5604ab5971f0#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-05-17T22:28:00.766 INFO:journalctl@ceph.mon.c.smithi151.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-17T22:28:00.766 INFO:journalctl@ceph.mon.c.smithi151.stdout: 2024-05-17T22:28:00.766 INFO:journalctl@ceph.mon.c.smithi151.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-17T22:28:00.767 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-05-17T22:28:00.767 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: using public_addr v2:172.21.15.151:0/0 -> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] 2024-05-17T22:28:00.767 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: starting mon.c rank -1 at public addrs [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] at bind addrs [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 4d593780-149c-11ef-bc9a-c7b262605968 2024-05-17T22:28:00.767 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: mon.c@-1(???) e0 preinit fsid 4d593780-149c-11ef-bc9a-c7b262605968 2024-05-17T22:28:00.767 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: mon.c@-1(synchronizing).mds e1 new map 2024-05-17T22:28:00.767 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: mon.c@-1(synchronizing).mds e1 print_map 2024-05-17T22:28:00.767 INFO:journalctl@ceph.mon.c.smithi151.stdout: e1 2024-05-17T22:28:00.767 INFO:journalctl@ceph.mon.c.smithi151.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-17T22:28:00.767 INFO:journalctl@ceph.mon.c.smithi151.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-17T22:28:00.767 INFO:journalctl@ceph.mon.c.smithi151.stdout: legacy client fscid: -1 2024-05-17T22:28:00.767 INFO:journalctl@ceph.mon.c.smithi151.stdout: 2024-05-17T22:28:00.767 INFO:journalctl@ceph.mon.c.smithi151.stdout: No filesystems configured 2024-05-17T22:28:00.767 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-17T22:28:00.767 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-17T22:28:00.768 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-17T22:28:00.768 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-17T22:28:00.768 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-17T22:28:00.768 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-17T22:28:00.768 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-17T22:28:00.768 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-17T22:28:00.768 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-17T22:28:00.768 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-17T22:28:00.768 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:00.768 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:00.768 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:28:00.768 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: Added host smithi096 2024-05-17T22:28:00.768 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:00.768 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:00.769 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:00.769 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:00.769 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T22:28:00.769 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:00.769 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:00.769 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi151", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:28:00.769 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:00.769 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:00.769 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:00.769 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:00.769 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi096", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:28:00.769 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:00.769 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:28:00.769 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: Updating smithi096:/etc/ceph/ceph.conf 2024-05-17T22:28:00.770 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: Deploying cephadm binary to smithi151 2024-05-17T22:28:00.770 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: Updating smithi096:/var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T22:28:00.770 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: Updating smithi096:/etc/ceph/ceph.client.admin.keyring 2024-05-17T22:28:00.770 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:00.770 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: Updating smithi096:/var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-17T22:28:00.770 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:00.770 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:00.770 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:00.770 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:00.770 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:00.770 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: Added host smithi151 2024-05-17T22:28:00.770 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:28:00.771 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:00.771 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:00.771 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:00.771 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:00.771 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T22:28:00.771 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:00.771 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/2220548000' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-17T22:28:00.771 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:00.771 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:00.771 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:00.771 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:00.771 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi151", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:28:00.771 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:00.771 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:28:00.772 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: Updating smithi151:/etc/ceph/ceph.conf 2024-05-17T22:28:00.772 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:00.772 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: Updating smithi151:/var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T22:28:00.772 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: Updating smithi151:/etc/ceph/ceph.client.admin.keyring 2024-05-17T22:28:00.772 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/2220548000' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-17T22:28:00.772 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: osdmap e4: 0 total, 0 up, 0 in 2024-05-17T22:28:00.772 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: Updating smithi151:/var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-17T22:28:00.772 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:00.772 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:00.772 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:00.772 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:00.772 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:00.772 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:28:00.772 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:00.773 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:28:00.773 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:00.773 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T22:28:00.773 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:00.773 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi012:172.21.15.12=a;smithi096:172.21.15.96=b;smithi151:172.21.15.151=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:28:00.773 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: Saving service mon spec with placement smithi012:172.21.15.12=a;smithi096:172.21.15.96=b;smithi151:172.21.15.151=c;count:3 2024-05-17T22:28:00.773 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: Deploying daemon mon.c on smithi151 2024-05-17T22:28:00.773 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:00.773 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:00.773 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 ceph-mon[36101]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-17T22:28:00.773 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 bash[36070]: 4e7aba1032ccfa00d24703300e98d418673be9849b41ccf2981f57a0ac8e13ea 2024-05-17T22:28:00.773 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:00 smithi151 systemd[1]: Started Ceph mon.c for 4d593780-149c-11ef-bc9a-c7b262605968. 2024-05-17T22:28:02.077 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-17T22:28:02.078 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-17T22:28:02.320 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T22:28:05.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:05 smithi012 ceph-mon[31632]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:05.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:05 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T22:28:05.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:05 smithi012 ceph-mon[31632]: mon.a calling monitor election 2024-05-17T22:28:05.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:05 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T22:28:05.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:05 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T22:28:05.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:05 smithi012 ceph-mon[31632]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:05.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:05 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T22:28:05.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:05 smithi012 ceph-mon[31632]: mon.c calling monitor election 2024-05-17T22:28:05.843 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:05 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T22:28:05.843 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:05 smithi012 ceph-mon[31632]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:05.843 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:05 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T22:28:05.843 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:05 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T22:28:05.843 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:05 smithi012 ceph-mon[31632]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-17T22:28:05.843 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:05 smithi012 ceph-mon[31632]: monmap epoch 2 2024-05-17T22:28:05.843 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:05 smithi012 ceph-mon[31632]: fsid 4d593780-149c-11ef-bc9a-c7b262605968 2024-05-17T22:28:05.843 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:05 smithi012 ceph-mon[31632]: last_changed 2024-05-17T22:28:00.490187+0000 2024-05-17T22:28:05.843 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:05 smithi012 ceph-mon[31632]: created 2024-05-17T22:26:04.404494+0000 2024-05-17T22:28:05.843 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:05 smithi012 ceph-mon[31632]: min_mon_release 19 (squid) 2024-05-17T22:28:05.843 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:05 smithi012 ceph-mon[31632]: election_strategy: 1 2024-05-17T22:28:05.843 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:05 smithi012 ceph-mon[31632]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-05-17T22:28:05.843 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:05 smithi012 ceph-mon[31632]: 1: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.c 2024-05-17T22:28:05.843 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:05 smithi012 ceph-mon[31632]: fsmap 2024-05-17T22:28:05.844 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:05 smithi012 ceph-mon[31632]: osdmap e4: 0 total, 0 up, 0 in 2024-05-17T22:28:05.844 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:05 smithi012 ceph-mon[31632]: mgrmap e13: x(active, since 51s) 2024-05-17T22:28:05.844 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:05 smithi012 ceph-mon[31632]: overall HEALTH_OK 2024-05-17T22:28:05.844 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:05 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:05.844 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:05 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:05.844 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:05 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:05.844 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:05 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T22:28:05.844 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:05 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:05.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:05 smithi151 ceph-mon[36101]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:05.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:05 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T22:28:05.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:05 smithi151 ceph-mon[36101]: mon.a calling monitor election 2024-05-17T22:28:05.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:05 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T22:28:05.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:05 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T22:28:05.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:05 smithi151 ceph-mon[36101]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:05.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:05 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T22:28:05.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:05 smithi151 ceph-mon[36101]: mon.c calling monitor election 2024-05-17T22:28:05.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:05 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T22:28:05.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:05 smithi151 ceph-mon[36101]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:05.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:05 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T22:28:05.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:05 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T22:28:05.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:05 smithi151 ceph-mon[36101]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-17T22:28:05.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:05 smithi151 ceph-mon[36101]: monmap epoch 2 2024-05-17T22:28:05.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:05 smithi151 ceph-mon[36101]: fsid 4d593780-149c-11ef-bc9a-c7b262605968 2024-05-17T22:28:05.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:05 smithi151 ceph-mon[36101]: last_changed 2024-05-17T22:28:00.490187+0000 2024-05-17T22:28:05.992 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:05 smithi151 ceph-mon[36101]: created 2024-05-17T22:26:04.404494+0000 2024-05-17T22:28:05.992 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:05 smithi151 ceph-mon[36101]: min_mon_release 19 (squid) 2024-05-17T22:28:05.992 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:05 smithi151 ceph-mon[36101]: election_strategy: 1 2024-05-17T22:28:05.992 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:05 smithi151 ceph-mon[36101]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-05-17T22:28:05.992 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:05 smithi151 ceph-mon[36101]: 1: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.c 2024-05-17T22:28:05.992 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:05 smithi151 ceph-mon[36101]: fsmap 2024-05-17T22:28:05.992 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:05 smithi151 ceph-mon[36101]: osdmap e4: 0 total, 0 up, 0 in 2024-05-17T22:28:05.992 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:05 smithi151 ceph-mon[36101]: mgrmap e13: x(active, since 51s) 2024-05-17T22:28:05.992 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:05 smithi151 ceph-mon[36101]: overall HEALTH_OK 2024-05-17T22:28:05.992 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:05 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:05.992 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:05 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:05.992 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:05 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:05.992 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:05 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T22:28:05.992 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:05 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:06.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:06.810+0000 7f60b049b700 1 -- 172.21.15.151:0/3708605315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60900035c0 msgr2=0x7f6090005aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:28:06.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:06.810+0000 7f60b049b700 1 --2- 172.21.15.151:0/3708605315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60900035c0 0x7f6090005aa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f60a806b5e0 tx=0x7f6098009e80 comp rx=0 tx=0).stop 2024-05-17T22:28:06.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:06.811+0000 7f60b049b700 1 -- 172.21.15.151:0/3708605315 shutdown_connections 2024-05-17T22:28:06.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:06.811+0000 7f60b049b700 1 --2- 172.21.15.151:0/3708605315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60900035c0 0x7f6090005aa0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:28:06.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:06.811+0000 7f60b049b700 1 -- 172.21.15.151:0/3708605315 >> 172.21.15.151:0/3708605315 conn(0x7f60a8078c00 msgr2=0x7f60a8079010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:28:06.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:06.811+0000 7f60b049b700 1 -- 172.21.15.151:0/3708605315 shutdown_connections 2024-05-17T22:28:06.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:06.811+0000 7f60b049b700 1 -- 172.21.15.151:0/3708605315 wait complete. 2024-05-17T22:28:06.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:06.811+0000 7f60b049b700 1 Processor -- start 2024-05-17T22:28:06.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:06.811+0000 7f60b049b700 1 -- start start 2024-05-17T22:28:06.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:06.812+0000 7f60b049b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60900035c0 0x7f60a8199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:28:06.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:06.812+0000 7f60b049b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60a8199980 0x7f60a819dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:28:06.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:06.812+0000 7f60b049b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f60a810a520 con 0x7f60a8199980 2024-05-17T22:28:06.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:06.812+0000 7f60b049b700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f60a810a3a0 con 0x7f60900035c0 2024-05-17T22:28:06.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:06.812+0000 7f60ae237700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60900035c0 0x7f60a8199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:28:06.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:06.812+0000 7f60ae237700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60900035c0 0x7f60a8199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56748/0 (socket says 172.21.15.151:56748) 2024-05-17T22:28:06.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:06.812+0000 7f60ae237700 1 -- 172.21.15.151:0/3533792671 learned_addr learned my addr 172.21.15.151:0/3533792671 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-17T22:28:06.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:06.812+0000 7f60ae237700 1 -- 172.21.15.151:0/3533792671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60900035c0 msgr2=0x7f60a8199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-17T22:28:06.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:06.812+0000 7f60ae237700 1 -- 172.21.15.151:0/3533792671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60900035c0 msgr2=0x7f60a8199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-17T22:28:06.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:06.812+0000 7f60ae237700 1 --2- 172.21.15.151:0/3533792671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60900035c0 0x7f60a8199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-17T22:28:06.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:06.812+0000 7f60ada36700 1 --2- 172.21.15.151:0/3533792671 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60a8199980 0x7f60a819dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:28:06.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:06.812+0000 7f60ae237700 1 --2- 172.21.15.151:0/3533792671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60900035c0 0x7f60a8199440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-17T22:28:06.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:06.812+0000 7f60ada36700 1 -- 172.21.15.151:0/3533792671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60900035c0 msgr2=0x7f60a8199440 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-17T22:28:06.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:06.812+0000 7f60ada36700 1 --2- 172.21.15.151:0/3533792671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60900035c0 0x7f60a8199440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:28:06.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:06.812+0000 7f60ada36700 1 -- 172.21.15.151:0/3533792671 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a8199f80 con 0x7f60a8199980 2024-05-17T22:28:06.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:06.813+0000 7f60ada36700 1 --2- 172.21.15.151:0/3533792671 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60a8199980 0x7f60a819dc70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f609c00b4d0 tx=0x7f609c00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:28:06.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:06.813+0000 7f60a77fe700 1 -- 172.21.15.151:0/3533792671 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f609c011630 con 0x7f60a8199980 2024-05-17T22:28:06.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:06.813+0000 7f60a77fe700 1 -- 172.21.15.151:0/3533792671 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f609c011c80 con 0x7f60a8199980 2024-05-17T22:28:06.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:06.813+0000 7f60a77fe700 1 -- 172.21.15.151:0/3533792671 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f609c010da0 con 0x7f60a8199980 2024-05-17T22:28:06.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:06.813+0000 7f60b049b700 1 -- 172.21.15.151:0/3533792671 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60a819e1b0 con 0x7f60a8199980 2024-05-17T22:28:06.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:06.813+0000 7f60b049b700 1 -- 172.21.15.151:0/3533792671 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60a819e7c0 con 0x7f60a8199980 2024-05-17T22:28:06.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:06.814+0000 7f60b049b700 1 -- 172.21.15.151:0/3533792671 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60a8103cf0 con 0x7f60a8199980 2024-05-17T22:28:06.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:06.814+0000 7f60a77fe700 1 -- 172.21.15.151:0/3533792671 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f609c018020 con 0x7f60a8199980 2024-05-17T22:28:06.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:06.814+0000 7f60a77fe700 1 --2- 172.21.15.151:0/3533792671 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f609403e320 0x7f6094040830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:28:06.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:06.814+0000 7f60a77fe700 1 -- 172.21.15.151:0/3533792671 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f609c0520a0 con 0x7f60a8199980 2024-05-17T22:28:06.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:06.814+0000 7f60ae237700 1 --2- 172.21.15.151:0/3533792671 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f609403e320 0x7f6094040830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:28:06.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:06.815+0000 7f60ae237700 1 --2- 172.21.15.151:0/3533792671 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f609403e320 0x7f6094040830 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f609801c040 tx=0x7f60980091d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:28:06.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:06.820+0000 7f60a77fe700 1 -- 172.21.15.151:0/3533792671 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f609c01f020 con 0x7f60a8199980 2024-05-17T22:28:06.822 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:06 smithi151 ceph-mon[36101]: Deploying daemon mon.b on smithi096 2024-05-17T22:28:06.822 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:06 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T22:28:06.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:06 smithi012 ceph-mon[31632]: Deploying daemon mon.b on smithi096 2024-05-17T22:28:06.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:06 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T22:28:07.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:07.114+0000 7f60b049b700 1 -- 172.21.15.151:0/3533792671 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f60a819a830 con 0x7f60a8199980 2024-05-17T22:28:07.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:07.115+0000 7f60a77fe700 1 -- 172.21.15.151:0/3533792671 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f60a819a830 con 0x7f60a8199980 2024-05-17T22:28:07.117 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-17T22:28:07.117 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":2,"fsid":"4d593780-149c-11ef-bc9a-c7b262605968","modified":"2024-05-17T22:28:00.490187Z","created":"2024-05-17T22:26:04.404494Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:3300","nonce":0},{"type":"v1","addr":"172.21.15.151:6789","nonce":0}]},"addr":"172.21.15.151:6789/0","public_addr":"172.21.15.151:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-17T22:28:07.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:07.117+0000 7f60b049b700 1 -- 172.21.15.151:0/3533792671 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f609403e320 msgr2=0x7f6094040830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:28:07.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:07.117+0000 7f60b049b700 1 --2- 172.21.15.151:0/3533792671 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f609403e320 0x7f6094040830 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f609801c040 tx=0x7f60980091d0 comp rx=0 tx=0).stop 2024-05-17T22:28:07.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:07.117+0000 7f60b049b700 1 -- 172.21.15.151:0/3533792671 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60a8199980 msgr2=0x7f60a819dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:28:07.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:07.117+0000 7f60b049b700 1 --2- 172.21.15.151:0/3533792671 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60a8199980 0x7f60a819dc70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f609c00b4d0 tx=0x7f609c00b890 comp rx=0 tx=0).stop 2024-05-17T22:28:07.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:07.117+0000 7f60b049b700 1 -- 172.21.15.151:0/3533792671 shutdown_connections 2024-05-17T22:28:07.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:07.117+0000 7f60b049b700 1 --2- 172.21.15.151:0/3533792671 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f609403e320 0x7f6094040830 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:28:07.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:07.117+0000 7f60b049b700 1 --2- 172.21.15.151:0/3533792671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60900035c0 0x7f60a8199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:28:07.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:07.117+0000 7f60b049b700 1 --2- 172.21.15.151:0/3533792671 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60a8199980 0x7f60a819dc70 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:28:07.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:07.117+0000 7f60b049b700 1 -- 172.21.15.151:0/3533792671 >> 172.21.15.151:0/3533792671 conn(0x7f60a8078c00 msgr2=0x7f60a81020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:28:07.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:07.117+0000 7f60b049b700 1 -- 172.21.15.151:0/3533792671 shutdown_connections 2024-05-17T22:28:07.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:07.117+0000 7f60b049b700 1 -- 172.21.15.151:0/3533792671 wait complete. 2024-05-17T22:28:07.120 INFO:teuthology.orchestra.run.smithi151.stderr:dumped monmap epoch 2 2024-05-17T22:28:07.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:07 smithi012 ceph-mon[31632]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:07.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:07 smithi012 ceph-mon[31632]: from='client.? 172.21.15.151:0/3533792671' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-17T22:28:07.841 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:28:07 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:28:07.490+0000 7f3bc9a70700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-17T22:28:07.844 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:07 smithi151 ceph-mon[36101]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:07.844 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:07 smithi151 ceph-mon[36101]: from='client.? 172.21.15.151:0/3533792671' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-17T22:28:08.877 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-17T22:28:08.878 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-17T22:28:09.122 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T22:28:09.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 podman[36045]: 2024-05-17 22:28:09.134012453 +0000 UTC m=+0.016987800 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T22:28:09.612 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 podman[36045]: 2024-05-17 22:28:09.477885243 +0000 UTC m=+0.360860584 container init 10fbdf09832dc9524878610c7c1ad0e86c74c45cfdada026bef7ff40d8c8f8c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mon-b, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-17T22:28:09.612 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 podman[36045]: 2024-05-17 22:28:09.481436056 +0000 UTC m=+0.364411388 container start 10fbdf09832dc9524878610c7c1ad0e86c74c45cfdada026bef7ff40d8c8f8c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0) 2024-05-17T22:28:09.613 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: set uid:gid to 167:167 (ceph:ceph) 2024-05-17T22:28:09.613 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev), process ceph-mon, pid 2 2024-05-17T22:28:09.613 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: pidfile_write: ignore empty --pid-file 2024-05-17T22:28:09.613 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: load: jerasure load: lrc 2024-05-17T22:28:09.613 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: RocksDB version: 7.9.2 2024-05-17T22:28:09.613 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Git sha 0 2024-05-17T22:28:09.613 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Compile date 2024-05-17 01:53:57 2024-05-17T22:28:09.613 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: DB SUMMARY 2024-05-17T22:28:09.613 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: DB Session ID: 4AVQCC6TTB46Q7GB3PEY 2024-05-17T22:28:09.613 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: CURRENT file: CURRENT 2024-05-17T22:28:09.613 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: IDENTITY file: IDENTITY 2024-05-17T22:28:09.613 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-17T22:28:09.613 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-05-17T22:28:09.614 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-05-17T22:28:09.614 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.error_if_exists: 0 2024-05-17T22:28:09.614 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.create_if_missing: 0 2024-05-17T22:28:09.614 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.paranoid_checks: 1 2024-05-17T22:28:09.614 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-17T22:28:09.614 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-17T22:28:09.614 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-17T22:28:09.614 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.env: 0x55a3fb98d500 2024-05-17T22:28:09.614 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.fs: PosixFileSystem 2024-05-17T22:28:09.614 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.info_log: 0x55a3fc784080 2024-05-17T22:28:09.614 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.max_file_opening_threads: 16 2024-05-17T22:28:09.614 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.statistics: (nil) 2024-05-17T22:28:09.615 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.use_fsync: 0 2024-05-17T22:28:09.615 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.max_log_file_size: 0 2024-05-17T22:28:09.615 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-17T22:28:09.615 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-17T22:28:09.615 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.keep_log_file_num: 1000 2024-05-17T22:28:09.615 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.recycle_log_file_num: 0 2024-05-17T22:28:09.615 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.allow_fallocate: 1 2024-05-17T22:28:09.615 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.allow_mmap_reads: 0 2024-05-17T22:28:09.615 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.allow_mmap_writes: 0 2024-05-17T22:28:09.615 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.use_direct_reads: 0 2024-05-17T22:28:09.615 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-17T22:28:09.616 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.create_missing_column_families: 0 2024-05-17T22:28:09.616 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.db_log_dir: 2024-05-17T22:28:09.616 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.wal_dir: 2024-05-17T22:28:09.616 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-17T22:28:09.616 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-17T22:28:09.616 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-17T22:28:09.616 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-17T22:28:09.616 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-17T22:28:09.616 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-17T22:28:09.616 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.advise_random_on_open: 1 2024-05-17T22:28:09.616 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.db_write_buffer_size: 0 2024-05-17T22:28:09.616 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.write_buffer_manager: 0x55a3fc638b40 2024-05-17T22:28:09.616 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-17T22:28:09.616 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-17T22:28:09.617 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-17T22:28:09.617 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.rate_limiter: (nil) 2024-05-17T22:28:09.617 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-17T22:28:09.617 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.wal_recovery_mode: 2 2024-05-17T22:28:09.617 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.enable_thread_tracking: 0 2024-05-17T22:28:09.617 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.enable_pipelined_write: 0 2024-05-17T22:28:09.617 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.unordered_write: 0 2024-05-17T22:28:09.617 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-17T22:28:09.617 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-17T22:28:09.617 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-17T22:28:09.617 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-17T22:28:09.617 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.row_cache: None 2024-05-17T22:28:09.617 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.wal_filter: None 2024-05-17T22:28:09.619 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-17T22:28:09.619 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.allow_ingest_behind: 0 2024-05-17T22:28:09.619 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.two_write_queues: 0 2024-05-17T22:28:09.619 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.manual_wal_flush: 0 2024-05-17T22:28:09.619 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.wal_compression: 0 2024-05-17T22:28:09.619 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.atomic_flush: 0 2024-05-17T22:28:09.619 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-17T22:28:09.619 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-17T22:28:09.619 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-17T22:28:09.619 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.log_readahead_size: 0 2024-05-17T22:28:09.620 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-17T22:28:09.620 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.best_efforts_recovery: 0 2024-05-17T22:28:09.620 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-17T22:28:09.620 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-17T22:28:09.620 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.allow_data_in_errors: 0 2024-05-17T22:28:09.620 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.db_host_id: __hostname__ 2024-05-17T22:28:09.620 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-17T22:28:09.620 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.max_background_jobs: 2 2024-05-17T22:28:09.620 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.max_background_compactions: -1 2024-05-17T22:28:09.620 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.max_subcompactions: 1 2024-05-17T22:28:09.620 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-17T22:28:09.620 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-17T22:28:09.620 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-17T22:28:09.620 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.max_total_wal_size: 0 2024-05-17T22:28:09.621 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-17T22:28:09.621 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-17T22:28:09.621 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-17T22:28:09.621 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-17T22:28:09.621 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.max_open_files: -1 2024-05-17T22:28:09.621 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.bytes_per_sync: 0 2024-05-17T22:28:09.621 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-17T22:28:09.621 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-17T22:28:09.621 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.compaction_readahead_size: 0 2024-05-17T22:28:09.621 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.max_background_flushes: -1 2024-05-17T22:28:09.621 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Compression algorithms supported: 2024-05-17T22:28:09.621 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-17T22:28:09.621 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: kZSTD supported: 0 2024-05-17T22:28:09.621 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: kXpressCompression supported: 0 2024-05-17T22:28:09.621 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: kLZ4HCCompression supported: 1 2024-05-17T22:28:09.622 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: kZlibCompression supported: 1 2024-05-17T22:28:09.622 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: kSnappyCompression supported: 1 2024-05-17T22:28:09.622 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: kLZ4Compression supported: 1 2024-05-17T22:28:09.622 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: kBZip2Compression supported: 0 2024-05-17T22:28:09.622 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-17T22:28:09.622 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-17T22:28:09.622 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-05-17T22:28:09.622 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-17T22:28:09.622 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-17T22:28:09.622 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.merge_operator: 2024-05-17T22:28:09.622 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.compaction_filter: None 2024-05-17T22:28:09.622 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.compaction_filter_factory: None 2024-05-17T22:28:09.622 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.sst_partitioner_factory: None 2024-05-17T22:28:09.622 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-17T22:28:09.623 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-17T22:28:09.623 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55a3fc784240) 2024-05-17T22:28:09.623 INFO:journalctl@ceph.mon.b.smithi096.stdout: cache_index_and_filter_blocks: 1 2024-05-17T22:28:09.623 INFO:journalctl@ceph.mon.b.smithi096.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-17T22:28:09.623 INFO:journalctl@ceph.mon.b.smithi096.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-17T22:28:09.623 INFO:journalctl@ceph.mon.b.smithi096.stdout: pin_top_level_index_and_filter: 1 2024-05-17T22:28:09.623 INFO:journalctl@ceph.mon.b.smithi096.stdout: index_type: 0 2024-05-17T22:28:09.623 INFO:journalctl@ceph.mon.b.smithi096.stdout: data_block_index_type: 0 2024-05-17T22:28:09.623 INFO:journalctl@ceph.mon.b.smithi096.stdout: index_shortening: 1 2024-05-17T22:28:09.623 INFO:journalctl@ceph.mon.b.smithi096.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-17T22:28:09.623 INFO:journalctl@ceph.mon.b.smithi096.stdout: checksum: 4 2024-05-17T22:28:09.623 INFO:journalctl@ceph.mon.b.smithi096.stdout: no_block_cache: 0 2024-05-17T22:28:09.623 INFO:journalctl@ceph.mon.b.smithi096.stdout: block_cache: 0x55a3fc6811f0 2024-05-17T22:28:09.623 INFO:journalctl@ceph.mon.b.smithi096.stdout: block_cache_name: BinnedLRUCache 2024-05-17T22:28:09.624 INFO:journalctl@ceph.mon.b.smithi096.stdout: block_cache_options: 2024-05-17T22:28:09.624 INFO:journalctl@ceph.mon.b.smithi096.stdout: capacity : 536870912 2024-05-17T22:28:09.624 INFO:journalctl@ceph.mon.b.smithi096.stdout: num_shard_bits : 4 2024-05-17T22:28:09.624 INFO:journalctl@ceph.mon.b.smithi096.stdout: strict_capacity_limit : 0 2024-05-17T22:28:09.624 INFO:journalctl@ceph.mon.b.smithi096.stdout: high_pri_pool_ratio: 0.000 2024-05-17T22:28:09.624 INFO:journalctl@ceph.mon.b.smithi096.stdout: block_cache_compressed: (nil) 2024-05-17T22:28:09.624 INFO:journalctl@ceph.mon.b.smithi096.stdout: persistent_cache: (nil) 2024-05-17T22:28:09.624 INFO:journalctl@ceph.mon.b.smithi096.stdout: block_size: 4096 2024-05-17T22:28:09.624 INFO:journalctl@ceph.mon.b.smithi096.stdout: block_size_deviation: 10 2024-05-17T22:28:09.624 INFO:journalctl@ceph.mon.b.smithi096.stdout: block_restart_interval: 16 2024-05-17T22:28:09.624 INFO:journalctl@ceph.mon.b.smithi096.stdout: index_block_restart_interval: 1 2024-05-17T22:28:09.624 INFO:journalctl@ceph.mon.b.smithi096.stdout: metadata_block_size: 4096 2024-05-17T22:28:09.624 INFO:journalctl@ceph.mon.b.smithi096.stdout: partition_filters: 0 2024-05-17T22:28:09.625 INFO:journalctl@ceph.mon.b.smithi096.stdout: use_delta_encoding: 1 2024-05-17T22:28:09.625 INFO:journalctl@ceph.mon.b.smithi096.stdout: filter_policy: bloomfilter 2024-05-17T22:28:09.625 INFO:journalctl@ceph.mon.b.smithi096.stdout: whole_key_filtering: 1 2024-05-17T22:28:09.625 INFO:journalctl@ceph.mon.b.smithi096.stdout: verify_compression: 0 2024-05-17T22:28:09.625 INFO:journalctl@ceph.mon.b.smithi096.stdout: read_amp_bytes_per_bit: 0 2024-05-17T22:28:09.625 INFO:journalctl@ceph.mon.b.smithi096.stdout: format_version: 5 2024-05-17T22:28:09.625 INFO:journalctl@ceph.mon.b.smithi096.stdout: enable_index_compression: 1 2024-05-17T22:28:09.625 INFO:journalctl@ceph.mon.b.smithi096.stdout: block_align: 0 2024-05-17T22:28:09.625 INFO:journalctl@ceph.mon.b.smithi096.stdout: max_auto_readahead_size: 262144 2024-05-17T22:28:09.625 INFO:journalctl@ceph.mon.b.smithi096.stdout: prepopulate_block_cache: 0 2024-05-17T22:28:09.625 INFO:journalctl@ceph.mon.b.smithi096.stdout: initial_auto_readahead_size: 8192 2024-05-17T22:28:09.625 INFO:journalctl@ceph.mon.b.smithi096.stdout: num_file_reads_for_auto_readahead: 2 2024-05-17T22:28:09.625 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.write_buffer_size: 33554432 2024-05-17T22:28:09.625 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.max_write_buffer_number: 2 2024-05-17T22:28:09.625 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.compression: NoCompression 2024-05-17T22:28:09.626 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.bottommost_compression: Disabled 2024-05-17T22:28:09.626 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.prefix_extractor: nullptr 2024-05-17T22:28:09.626 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-17T22:28:09.626 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.num_levels: 7 2024-05-17T22:28:09.626 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-17T22:28:09.626 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-17T22:28:09.626 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-17T22:28:09.626 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-17T22:28:09.626 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-17T22:28:09.626 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-17T22:28:09.626 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-17T22:28:09.626 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-17T22:28:09.627 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-17T22:28:09.627 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-17T22:28:09.627 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-17T22:28:09.627 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-17T22:28:09.627 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-17T22:28:09.627 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.compression_opts.level: 32767 2024-05-17T22:28:09.627 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.compression_opts.strategy: 0 2024-05-17T22:28:09.627 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-17T22:28:09.627 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-17T22:28:09.627 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-17T22:28:09.627 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-17T22:28:09.627 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.compression_opts.enabled: false 2024-05-17T22:28:09.627 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-17T22:28:09.627 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-17T22:28:09.627 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-17T22:28:09.628 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-17T22:28:09.628 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.target_file_size_base: 67108864 2024-05-17T22:28:09.628 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-17T22:28:09.628 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-17T22:28:09.628 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-17T22:28:09.628 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-17T22:28:09.628 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-17T22:28:09.628 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-17T22:28:09.628 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-17T22:28:09.628 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-17T22:28:09.628 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-17T22:28:09.628 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-17T22:28:09.629 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-17T22:28:09.629 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-17T22:28:09.629 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-17T22:28:09.629 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-17T22:28:09.629 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.arena_block_size: 1048576 2024-05-17T22:28:09.629 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-17T22:28:09.629 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-17T22:28:09.629 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.disable_auto_compactions: 0 2024-05-17T22:28:09.629 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-17T22:28:09.629 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-17T22:28:09.629 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-17T22:28:09.629 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-17T22:28:09.629 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-17T22:28:09.629 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-17T22:28:09.629 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-17T22:28:09.630 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-17T22:28:09.630 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-17T22:28:09.630 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-17T22:28:09.630 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-17T22:28:09.630 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.inplace_update_support: 0 2024-05-17T22:28:09.630 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-17T22:28:09.630 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-17T22:28:09.630 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-17T22:28:09.630 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-17T22:28:09.630 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.bloom_locality: 0 2024-05-17T22:28:09.630 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.max_successive_merges: 0 2024-05-17T22:28:09.630 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-17T22:28:09.630 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.paranoid_file_checks: 0 2024-05-17T22:28:09.630 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.force_consistency_checks: 1 2024-05-17T22:28:09.631 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.report_bg_io_stats: 0 2024-05-17T22:28:09.631 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.ttl: 2592000 2024-05-17T22:28:09.631 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-17T22:28:09.631 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-17T22:28:09.631 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 bash[36045]: 10fbdf09832dc9524878610c7c1ad0e86c74c45cfdada026bef7ff40d8c8f8c8 2024-05-17T22:28:09.631 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-17T22:28:09.631 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.enable_blob_files: false 2024-05-17T22:28:09.631 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.min_blob_size: 0 2024-05-17T22:28:09.631 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.blob_file_size: 268435456 2024-05-17T22:28:09.631 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-17T22:28:09.631 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-17T22:28:09.631 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-17T22:28:09.631 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-17T22:28:09.631 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-17T22:28:09.631 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.blob_file_starting_level: 0 2024-05-17T22:28:09.631 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-17T22:28:09.632 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-17T22:28:09.632 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-17T22:28:09.632 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 436200ef-6daf-4bb6-a778-d92db870cc83 2024-05-17T22:28:09.632 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715984889523211, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-17T22:28:09.632 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-17T22:28:09.632 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715984889523798, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715984889, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "436200ef-6daf-4bb6-a778-d92db870cc83", "db_session_id": "4AVQCC6TTB46Q7GB3PEY", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-17T22:28:09.632 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715984889523909, "job": 1, "event": "recovery_finished"} 2024-05-17T22:28:09.632 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-17T22:28:09.632 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-17T22:28:09.632 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55a3fc663c00 2024-05-17T22:28:09.632 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: DB pointer 0x55a401458000 2024-05-17T22:28:09.632 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-17T22:28:09.632 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-17T22:28:09.632 INFO:journalctl@ceph.mon.b.smithi096.stdout: ** DB Stats ** 2024-05-17T22:28:09.632 INFO:journalctl@ceph.mon.b.smithi096.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-17T22:28:09.633 INFO:journalctl@ceph.mon.b.smithi096.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-17T22:28:09.633 INFO:journalctl@ceph.mon.b.smithi096.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-17T22:28:09.633 INFO:journalctl@ceph.mon.b.smithi096.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-17T22:28:09.633 INFO:journalctl@ceph.mon.b.smithi096.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-17T22:28:09.633 INFO:journalctl@ceph.mon.b.smithi096.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-17T22:28:09.633 INFO:journalctl@ceph.mon.b.smithi096.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-17T22:28:09.633 INFO:journalctl@ceph.mon.b.smithi096.stdout: 2024-05-17T22:28:09.633 INFO:journalctl@ceph.mon.b.smithi096.stdout: ** Compaction Stats [default] ** 2024-05-17T22:28:09.633 INFO:journalctl@ceph.mon.b.smithi096.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-17T22:28:09.633 INFO:journalctl@ceph.mon.b.smithi096.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-17T22:28:09.633 INFO:journalctl@ceph.mon.b.smithi096.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-17T22:28:09.633 INFO:journalctl@ceph.mon.b.smithi096.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-17T22:28:09.633 INFO:journalctl@ceph.mon.b.smithi096.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-17T22:28:09.633 INFO:journalctl@ceph.mon.b.smithi096.stdout: 2024-05-17T22:28:09.633 INFO:journalctl@ceph.mon.b.smithi096.stdout: ** Compaction Stats [default] ** 2024-05-17T22:28:09.633 INFO:journalctl@ceph.mon.b.smithi096.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-17T22:28:09.634 INFO:journalctl@ceph.mon.b.smithi096.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-17T22:28:09.634 INFO:journalctl@ceph.mon.b.smithi096.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-17T22:28:09.634 INFO:journalctl@ceph.mon.b.smithi096.stdout: 2024-05-17T22:28:09.634 INFO:journalctl@ceph.mon.b.smithi096.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-17T22:28:09.634 INFO:journalctl@ceph.mon.b.smithi096.stdout: 2024-05-17T22:28:09.634 INFO:journalctl@ceph.mon.b.smithi096.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-17T22:28:09.634 INFO:journalctl@ceph.mon.b.smithi096.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-17T22:28:09.634 INFO:journalctl@ceph.mon.b.smithi096.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-17T22:28:09.634 INFO:journalctl@ceph.mon.b.smithi096.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-17T22:28:09.634 INFO:journalctl@ceph.mon.b.smithi096.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-17T22:28:09.634 INFO:journalctl@ceph.mon.b.smithi096.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-17T22:28:09.634 INFO:journalctl@ceph.mon.b.smithi096.stdout: Cumulative compaction: 0.00 GB write, 0.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-17T22:28:09.634 INFO:journalctl@ceph.mon.b.smithi096.stdout: Interval compaction: 0.00 GB write, 0.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-17T22:28:09.634 INFO:journalctl@ceph.mon.b.smithi096.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-17T22:28:09.634 INFO:journalctl@ceph.mon.b.smithi096.stdout: Block cache BinnedLRUCache@0x55a3fc6811f0#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-05-17T22:28:09.635 INFO:journalctl@ceph.mon.b.smithi096.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-17T22:28:09.635 INFO:journalctl@ceph.mon.b.smithi096.stdout: 2024-05-17T22:28:09.635 INFO:journalctl@ceph.mon.b.smithi096.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-17T22:28:09.635 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-05-17T22:28:09.635 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: using public_addr v2:172.21.15.96:0/0 -> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] 2024-05-17T22:28:09.635 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: starting mon.b rank -1 at public addrs [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] at bind addrs [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 4d593780-149c-11ef-bc9a-c7b262605968 2024-05-17T22:28:09.635 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: mon.b@-1(???) e0 preinit fsid 4d593780-149c-11ef-bc9a-c7b262605968 2024-05-17T22:28:09.635 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 systemd[1]: Started Ceph mon.b for 4d593780-149c-11ef-bc9a-c7b262605968. 2024-05-17T22:28:09.635 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: mon.b@-1(synchronizing).mds e1 new map 2024-05-17T22:28:09.635 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: mon.b@-1(synchronizing).mds e1 print_map 2024-05-17T22:28:09.635 INFO:journalctl@ceph.mon.b.smithi096.stdout: e1 2024-05-17T22:28:09.635 INFO:journalctl@ceph.mon.b.smithi096.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-17T22:28:09.635 INFO:journalctl@ceph.mon.b.smithi096.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-17T22:28:09.635 INFO:journalctl@ceph.mon.b.smithi096.stdout: legacy client fscid: -1 2024-05-17T22:28:09.635 INFO:journalctl@ceph.mon.b.smithi096.stdout: 2024-05-17T22:28:09.636 INFO:journalctl@ceph.mon.b.smithi096.stdout: No filesystems configured 2024-05-17T22:28:09.636 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-17T22:28:09.636 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-17T22:28:09.636 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-17T22:28:09.636 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-17T22:28:09.636 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-17T22:28:09.636 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-17T22:28:09.636 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-17T22:28:09.636 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-17T22:28:09.636 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-17T22:28:09.636 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-17T22:28:09.636 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:09.636 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:09.636 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:28:09.636 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: Added host smithi096 2024-05-17T22:28:09.636 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:09.637 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:09.637 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:09.637 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:09.637 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T22:28:09.637 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:09.637 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:09.637 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi151", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:28:09.637 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:09.637 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:09.637 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:09.637 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:09.637 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi096", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:28:09.637 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:09.637 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:28:09.637 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: Updating smithi096:/etc/ceph/ceph.conf 2024-05-17T22:28:09.637 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: Deploying cephadm binary to smithi151 2024-05-17T22:28:09.638 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: Updating smithi096:/var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T22:28:09.638 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: Updating smithi096:/etc/ceph/ceph.client.admin.keyring 2024-05-17T22:28:09.638 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:09.638 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: Updating smithi096:/var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-17T22:28:09.638 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:09.638 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:09.638 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:09.638 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:09.638 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:09.638 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: Added host smithi151 2024-05-17T22:28:09.638 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:28:09.638 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:09.638 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:09.638 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:09.638 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:09.639 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T22:28:09.639 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:09.639 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/2220548000' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-17T22:28:09.639 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:09.639 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:09.639 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:09.639 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:09.639 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi151", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:28:09.639 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:09.639 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:28:09.639 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: Updating smithi151:/etc/ceph/ceph.conf 2024-05-17T22:28:09.639 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:09.639 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: Updating smithi151:/var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T22:28:09.639 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: Updating smithi151:/etc/ceph/ceph.client.admin.keyring 2024-05-17T22:28:09.639 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/2220548000' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-17T22:28:09.640 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: osdmap e4: 0 total, 0 up, 0 in 2024-05-17T22:28:09.640 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: Updating smithi151:/var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-17T22:28:09.640 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:09.640 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:09.640 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:09.640 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:09.640 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:09.640 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:28:09.640 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:09.640 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:28:09.640 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:09.640 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T22:28:09.640 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:09.640 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi012:172.21.15.12=a;smithi096:172.21.15.96=b;smithi151:172.21.15.151=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:28:09.640 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: Saving service mon spec with placement smithi012:172.21.15.12=a;smithi096:172.21.15.96=b;smithi151:172.21.15.151=c;count:3 2024-05-17T22:28:09.640 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: Deploying daemon mon.c on smithi151 2024-05-17T22:28:09.641 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:09.641 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:09.641 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:09.641 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T22:28:09.641 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: mon.a calling monitor election 2024-05-17T22:28:09.641 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T22:28:09.641 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T22:28:09.641 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:09.641 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T22:28:09.641 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: mon.c calling monitor election 2024-05-17T22:28:09.641 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T22:28:09.641 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:09.641 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T22:28:09.641 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T22:28:09.642 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-17T22:28:09.642 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: monmap epoch 2 2024-05-17T22:28:09.642 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: fsid 4d593780-149c-11ef-bc9a-c7b262605968 2024-05-17T22:28:09.642 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: last_changed 2024-05-17T22:28:00.490187+0000 2024-05-17T22:28:09.642 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: created 2024-05-17T22:26:04.404494+0000 2024-05-17T22:28:09.642 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: min_mon_release 19 (squid) 2024-05-17T22:28:09.642 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: election_strategy: 1 2024-05-17T22:28:09.642 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-05-17T22:28:09.642 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: 1: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.c 2024-05-17T22:28:09.642 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: fsmap 2024-05-17T22:28:09.642 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: osdmap e4: 0 total, 0 up, 0 in 2024-05-17T22:28:09.642 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: mgrmap e13: x(active, since 51s) 2024-05-17T22:28:09.642 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: overall HEALTH_OK 2024-05-17T22:28:09.642 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:09.643 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:09.643 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:09.643 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T22:28:09.643 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:09.643 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: Deploying daemon mon.b on smithi096 2024-05-17T22:28:09.643 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T22:28:09.643 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:09.643 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: from='client.? 172.21.15.151:0/3533792671' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-17T22:28:09.643 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:09 smithi096 ceph-mon[36065]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-17T22:28:14.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.983+0000 7f8f0ab1f700 1 -- 172.21.15.151:0/650909514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ef40035c0 msgr2=0x7f8ef4005aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:28:14.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.983+0000 7f8f0ab1f700 1 --2- 172.21.15.151:0/650909514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ef40035c0 0x7f8ef4005aa0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8f041046e0 tx=0x7f8ef800fe10 comp rx=0 tx=0).stop 2024-05-17T22:28:14.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.983+0000 7f8f0ab1f700 1 -- 172.21.15.151:0/650909514 shutdown_connections 2024-05-17T22:28:14.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.983+0000 7f8f0ab1f700 1 --2- 172.21.15.151:0/650909514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ef40035c0 0x7f8ef4005aa0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:28:14.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.983+0000 7f8f0ab1f700 1 -- 172.21.15.151:0/650909514 >> 172.21.15.151:0/650909514 conn(0x7f8f040ff2f0 msgr2=0x7f8f04101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:28:14.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.983+0000 7f8f0ab1f700 1 -- 172.21.15.151:0/650909514 shutdown_connections 2024-05-17T22:28:14.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.983+0000 7f8f0ab1f700 1 -- 172.21.15.151:0/650909514 wait complete. 2024-05-17T22:28:14.986 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:14 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T22:28:14.986 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:14 smithi151 ceph-mon[36101]: mon.a calling monitor election 2024-05-17T22:28:14.986 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:14 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T22:28:14.987 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:14 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T22:28:14.987 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:14 smithi151 ceph-mon[36101]: mon.c calling monitor election 2024-05-17T22:28:14.987 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:14 smithi151 ceph-mon[36101]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:14.987 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:14 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T22:28:14.987 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:14 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T22:28:14.987 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:14 smithi151 ceph-mon[36101]: mon.b calling monitor election 2024-05-17T22:28:14.987 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:14 smithi151 ceph-mon[36101]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:14.987 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:14 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T22:28:14.987 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:14 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T22:28:14.987 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:14 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T22:28:14.987 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:14 smithi151 ceph-mon[36101]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-17T22:28:14.987 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:14 smithi151 ceph-mon[36101]: monmap epoch 3 2024-05-17T22:28:14.987 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:14 smithi151 ceph-mon[36101]: fsid 4d593780-149c-11ef-bc9a-c7b262605968 2024-05-17T22:28:14.987 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:14 smithi151 ceph-mon[36101]: last_changed 2024-05-17T22:28:09.554969+0000 2024-05-17T22:28:14.988 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:14 smithi151 ceph-mon[36101]: created 2024-05-17T22:26:04.404494+0000 2024-05-17T22:28:14.988 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:14 smithi151 ceph-mon[36101]: min_mon_release 19 (squid) 2024-05-17T22:28:14.988 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:14 smithi151 ceph-mon[36101]: election_strategy: 1 2024-05-17T22:28:14.988 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:14 smithi151 ceph-mon[36101]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-05-17T22:28:14.988 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:14 smithi151 ceph-mon[36101]: 1: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.c 2024-05-17T22:28:14.988 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:14 smithi151 ceph-mon[36101]: 2: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.b 2024-05-17T22:28:14.988 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:14 smithi151 ceph-mon[36101]: fsmap 2024-05-17T22:28:14.988 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:14 smithi151 ceph-mon[36101]: osdmap e4: 0 total, 0 up, 0 in 2024-05-17T22:28:14.988 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:14 smithi151 ceph-mon[36101]: mgrmap e13: x(active, since 60s) 2024-05-17T22:28:14.988 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:14 smithi151 ceph-mon[36101]: overall HEALTH_OK 2024-05-17T22:28:14.988 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:14 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:14.988 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:14 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:14.988 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:14 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:14.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.984+0000 7f8f0ab1f700 1 Processor -- start 2024-05-17T22:28:14.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.984+0000 7f8f0ab1f700 1 -- start start 2024-05-17T22:28:14.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.985+0000 7f8f0ab1f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ef40035c0 0x7f8f041995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:28:14.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.985+0000 7f8f0ab1f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f04199ae0 0x7f8f0419ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:28:14.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.985+0000 7f8f0ab1f700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f0419e310 0x7f8f0419e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:28:14.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.985+0000 7f8f0ab1f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0410b580 con 0x7f8f04199ae0 2024-05-17T22:28:14.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.985+0000 7f8f0ab1f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0410b400 con 0x7f8ef40035c0 2024-05-17T22:28:14.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.985+0000 7f8f0ab1f700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0410b700 con 0x7f8f0419e310 2024-05-17T22:28:14.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.985+0000 7f8f090bc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f0419e310 0x7f8f0419e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:28:14.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.985+0000 7f8f090bc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f0419e310 0x7f8f0419e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56750/0 (socket says 172.21.15.151:56750) 2024-05-17T22:28:14.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.985+0000 7f8f088bb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ef40035c0 0x7f8f041995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:28:14.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.985+0000 7f8f090bc700 1 -- 172.21.15.151:0/3772152181 learned_addr learned my addr 172.21.15.151:0/3772152181 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-17T22:28:14.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.985+0000 7f8f03fff700 1 --2- 172.21.15.151:0/3772152181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f04199ae0 0x7f8f0419ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:28:14.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.985+0000 7f8f090bc700 1 -- 172.21.15.151:0/3772152181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f0419e310 msgr2=0x7f8f0419e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-05-17T22:28:14.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.985+0000 7f8f090bc700 1 -- 172.21.15.151:0/3772152181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f0419e310 msgr2=0x7f8f0419e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-17T22:28:14.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.985+0000 7f8f090bc700 1 --2- 172.21.15.151:0/3772152181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f0419e310 0x7f8f0419e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-17T22:28:14.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.985+0000 7f8f090bc700 1 --2- 172.21.15.151:0/3772152181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f0419e310 0x7f8f0419e720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-17T22:28:14.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.985+0000 7f8f088bb700 1 -- 172.21.15.151:0/3772152181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ef40035c0 msgr2=0x7f8f041995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-17T22:28:14.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.985+0000 7f8f088bb700 1 -- 172.21.15.151:0/3772152181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ef40035c0 msgr2=0x7f8f041995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-17T22:28:14.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.985+0000 7f8f088bb700 1 --2- 172.21.15.151:0/3772152181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ef40035c0 0x7f8f041995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-17T22:28:14.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.985+0000 7f8f088bb700 1 --2- 172.21.15.151:0/3772152181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ef40035c0 0x7f8f041995a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-17T22:28:14.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.986+0000 7f8f03fff700 1 -- 172.21.15.151:0/3772152181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f0419e310 msgr2=0x7f8f0419e720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-17T22:28:14.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.986+0000 7f8f03fff700 1 --2- 172.21.15.151:0/3772152181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f0419e310 0x7f8f0419e720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:28:14.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.986+0000 7f8f03fff700 1 -- 172.21.15.151:0/3772152181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ef40035c0 msgr2=0x7f8f041995a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-17T22:28:14.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.986+0000 7f8f03fff700 1 --2- 172.21.15.151:0/3772152181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ef40035c0 0x7f8f041995a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:28:14.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.986+0000 7f8f03fff700 1 -- 172.21.15.151:0/3772152181 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f041a6080 con 0x7f8f04199ae0 2024-05-17T22:28:14.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.986+0000 7f8f03fff700 1 --2- 172.21.15.151:0/3772152181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f04199ae0 0x7f8f0419ddd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8ef000b550 tx=0x7f8ef000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:28:14.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.986+0000 7f8f01ffb700 1 -- 172.21.15.151:0/3772152181 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef0004230 con 0x7f8f04199ae0 2024-05-17T22:28:14.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.986+0000 7f8f01ffb700 1 -- 172.21.15.151:0/3772152181 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ef00043d0 con 0x7f8f04199ae0 2024-05-17T22:28:14.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.986+0000 7f8f0ab1f700 1 -- 172.21.15.151:0/3772152181 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f04075ac0 con 0x7f8f04199ae0 2024-05-17T22:28:14.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.986+0000 7f8f01ffb700 1 -- 172.21.15.151:0/3772152181 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef0011b90 con 0x7f8f04199ae0 2024-05-17T22:28:14.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.987+0000 7f8f0ab1f700 1 -- 172.21.15.151:0/3772152181 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f04076080 con 0x7f8f04199ae0 2024-05-17T22:28:14.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.988+0000 7f8f01ffb700 1 -- 172.21.15.151:0/3772152181 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f8ef001a400 con 0x7f8f04199ae0 2024-05-17T22:28:14.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.988+0000 7f8f0ab1f700 1 -- 172.21.15.151:0/3772152181 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f0410c670 con 0x7f8f04199ae0 2024-05-17T22:28:14.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.988+0000 7f8f01ffb700 1 --2- 172.21.15.151:0/3772152181 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8ee803e9a0 0x7f8ee8040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:28:14.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.988+0000 7f8f01ffb700 1 -- 172.21.15.151:0/3772152181 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8ef0052340 con 0x7f8f04199ae0 2024-05-17T22:28:14.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.988+0000 7f8f088bb700 1 --2- 172.21.15.151:0/3772152181 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8ee803e9a0 0x7f8ee8040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:28:14.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.989+0000 7f8f088bb700 1 --2- 172.21.15.151:0/3772152181 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8ee803e9a0 0x7f8ee8040eb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800f9c0 tx=0x7f8ef8001e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:28:14.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:14.993+0000 7f8f01ffb700 1 -- 172.21.15.151:0/3772152181 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8ef001a890 con 0x7f8f04199ae0 2024-05-17T22:28:15.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:14 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T22:28:15.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:14 smithi096 ceph-mon[36065]: mon.a calling monitor election 2024-05-17T22:28:15.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:14 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T22:28:15.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:14 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T22:28:15.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:14 smithi096 ceph-mon[36065]: mon.c calling monitor election 2024-05-17T22:28:15.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:14 smithi096 ceph-mon[36065]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:15.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:14 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T22:28:15.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:14 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T22:28:15.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:14 smithi096 ceph-mon[36065]: mon.b calling monitor election 2024-05-17T22:28:15.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:14 smithi096 ceph-mon[36065]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:15.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:14 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T22:28:15.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:14 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T22:28:15.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:14 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T22:28:15.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:14 smithi096 ceph-mon[36065]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-17T22:28:15.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:14 smithi096 ceph-mon[36065]: monmap epoch 3 2024-05-17T22:28:15.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:14 smithi096 ceph-mon[36065]: fsid 4d593780-149c-11ef-bc9a-c7b262605968 2024-05-17T22:28:15.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:14 smithi096 ceph-mon[36065]: last_changed 2024-05-17T22:28:09.554969+0000 2024-05-17T22:28:15.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:14 smithi096 ceph-mon[36065]: created 2024-05-17T22:26:04.404494+0000 2024-05-17T22:28:15.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:14 smithi096 ceph-mon[36065]: min_mon_release 19 (squid) 2024-05-17T22:28:15.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:14 smithi096 ceph-mon[36065]: election_strategy: 1 2024-05-17T22:28:15.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:14 smithi096 ceph-mon[36065]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-05-17T22:28:15.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:14 smithi096 ceph-mon[36065]: 1: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.c 2024-05-17T22:28:15.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:14 smithi096 ceph-mon[36065]: 2: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.b 2024-05-17T22:28:15.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:14 smithi096 ceph-mon[36065]: fsmap 2024-05-17T22:28:15.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:14 smithi096 ceph-mon[36065]: osdmap e4: 0 total, 0 up, 0 in 2024-05-17T22:28:15.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:14 smithi096 ceph-mon[36065]: mgrmap e13: x(active, since 60s) 2024-05-17T22:28:15.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:14 smithi096 ceph-mon[36065]: overall HEALTH_OK 2024-05-17T22:28:15.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:14 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:15.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:14 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:15.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:14 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:15.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:14 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T22:28:15.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:14 smithi012 ceph-mon[31632]: mon.a calling monitor election 2024-05-17T22:28:15.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:14 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T22:28:15.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:14 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T22:28:15.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:14 smithi012 ceph-mon[31632]: mon.c calling monitor election 2024-05-17T22:28:15.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:14 smithi012 ceph-mon[31632]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:15.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:14 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T22:28:15.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:14 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T22:28:15.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:14 smithi012 ceph-mon[31632]: mon.b calling monitor election 2024-05-17T22:28:15.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:14 smithi012 ceph-mon[31632]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:15.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:14 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T22:28:15.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:14 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T22:28:15.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:14 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T22:28:15.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:14 smithi012 ceph-mon[31632]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-17T22:28:15.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:14 smithi012 ceph-mon[31632]: monmap epoch 3 2024-05-17T22:28:15.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:14 smithi012 ceph-mon[31632]: fsid 4d593780-149c-11ef-bc9a-c7b262605968 2024-05-17T22:28:15.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:14 smithi012 ceph-mon[31632]: last_changed 2024-05-17T22:28:09.554969+0000 2024-05-17T22:28:15.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:14 smithi012 ceph-mon[31632]: created 2024-05-17T22:26:04.404494+0000 2024-05-17T22:28:15.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:14 smithi012 ceph-mon[31632]: min_mon_release 19 (squid) 2024-05-17T22:28:15.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:14 smithi012 ceph-mon[31632]: election_strategy: 1 2024-05-17T22:28:15.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:14 smithi012 ceph-mon[31632]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-05-17T22:28:15.093 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:14 smithi012 ceph-mon[31632]: 1: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.c 2024-05-17T22:28:15.093 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:14 smithi012 ceph-mon[31632]: 2: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.b 2024-05-17T22:28:15.093 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:14 smithi012 ceph-mon[31632]: fsmap 2024-05-17T22:28:15.093 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:14 smithi012 ceph-mon[31632]: osdmap e4: 0 total, 0 up, 0 in 2024-05-17T22:28:15.093 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:14 smithi012 ceph-mon[31632]: mgrmap e13: x(active, since 60s) 2024-05-17T22:28:15.093 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:14 smithi012 ceph-mon[31632]: overall HEALTH_OK 2024-05-17T22:28:15.093 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:14 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:15.093 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:14 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:15.093 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:14 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:15.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:15.316+0000 7f8f0ab1f700 1 -- 172.21.15.151:0/3772152181 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f8f0419a900 con 0x7f8f04199ae0 2024-05-17T22:28:15.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:15.316+0000 7f8f01ffb700 1 -- 172.21.15.151:0/3772152181 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1281 (secure 0 0 0) 0x7f8ef0003be0 con 0x7f8f04199ae0 2024-05-17T22:28:15.318 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-17T22:28:15.318 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":3,"fsid":"4d593780-149c-11ef-bc9a-c7b262605968","modified":"2024-05-17T22:28:09.554969Z","created":"2024-05-17T22:26:04.404494Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:3300","nonce":0},{"type":"v1","addr":"172.21.15.151:6789","nonce":0}]},"addr":"172.21.15.151:6789/0","public_addr":"172.21.15.151:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:3300","nonce":0},{"type":"v1","addr":"172.21.15.96:6789","nonce":0}]},"addr":"172.21.15.96:6789/0","public_addr":"172.21.15.96:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-05-17T22:28:15.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:15.319+0000 7f8f0ab1f700 1 -- 172.21.15.151:0/3772152181 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8ee803e9a0 msgr2=0x7f8ee8040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:28:15.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:15.319+0000 7f8f0ab1f700 1 --2- 172.21.15.151:0/3772152181 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8ee803e9a0 0x7f8ee8040eb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800f9c0 tx=0x7f8ef8001e50 comp rx=0 tx=0).stop 2024-05-17T22:28:15.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:15.319+0000 7f8f0ab1f700 1 -- 172.21.15.151:0/3772152181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f04199ae0 msgr2=0x7f8f0419ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:28:15.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:15.319+0000 7f8f0ab1f700 1 --2- 172.21.15.151:0/3772152181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f04199ae0 0x7f8f0419ddd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8ef000b550 tx=0x7f8ef000b910 comp rx=0 tx=0).stop 2024-05-17T22:28:15.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:15.319+0000 7f8f0ab1f700 1 -- 172.21.15.151:0/3772152181 shutdown_connections 2024-05-17T22:28:15.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:15.319+0000 7f8f0ab1f700 1 --2- 172.21.15.151:0/3772152181 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8ee803e9a0 0x7f8ee8040eb0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:28:15.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:15.319+0000 7f8f0ab1f700 1 --2- 172.21.15.151:0/3772152181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f0419e310 0x7f8f0419e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:28:15.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:15.319+0000 7f8f0ab1f700 1 --2- 172.21.15.151:0/3772152181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ef40035c0 0x7f8f041995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:28:15.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:15.319+0000 7f8f0ab1f700 1 --2- 172.21.15.151:0/3772152181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f04199ae0 0x7f8f0419ddd0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:28:15.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:15.319+0000 7f8f0ab1f700 1 -- 172.21.15.151:0/3772152181 >> 172.21.15.151:0/3772152181 conn(0x7f8f040ff2f0 msgr2=0x7f8f04102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:28:15.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:15.319+0000 7f8f0ab1f700 1 -- 172.21.15.151:0/3772152181 shutdown_connections 2024-05-17T22:28:15.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:15.319+0000 7f8f0ab1f700 1 -- 172.21.15.151:0/3772152181 wait complete. 2024-05-17T22:28:15.322 INFO:teuthology.orchestra.run.smithi151.stderr:dumped monmap epoch 3 2024-05-17T22:28:15.874 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-17T22:28:15.874 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph config generate-minimal-conf 2024-05-17T22:28:15.904 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:15 smithi012 ceph-mon[31632]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:15.904 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:15 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:15.904 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:15 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:28:15.904 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:15 smithi012 ceph-mon[31632]: from='client.? 172.21.15.151:0/3772152181' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-17T22:28:15.904 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:15 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T22:28:15.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:15 smithi151 ceph-mon[36101]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:15.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:15 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:15.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:15 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:28:15.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:15 smithi151 ceph-mon[36101]: from='client.? 172.21.15.151:0/3772152181' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-17T22:28:15.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:15 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T22:28:16.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:15 smithi096 ceph-mon[36065]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:16.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:15 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:16.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:15 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:28:16.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:15 smithi096 ceph-mon[36065]: from='client.? 172.21.15.151:0/3772152181' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-17T22:28:16.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:15 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T22:28:16.136 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:28:16.841 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:28:16 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:28:16.554+0000 7f3bc9a70700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-17T22:28:17.252 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:16 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:17.252 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:16 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:17.252 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:16 smithi012 ceph-mon[31632]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:17.252 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:16 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:17.253 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:16 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:17.253 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:16 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:17.253 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:16 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:28:17.253 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:16 smithi012 ceph-mon[31632]: Updating smithi012:/etc/ceph/ceph.conf 2024-05-17T22:28:17.253 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:16 smithi012 ceph-mon[31632]: Updating smithi096:/etc/ceph/ceph.conf 2024-05-17T22:28:17.253 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:16 smithi012 ceph-mon[31632]: Updating smithi151:/etc/ceph/ceph.conf 2024-05-17T22:28:17.253 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:16 smithi012 ceph-mon[31632]: Updating smithi096:/var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T22:28:17.253 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:16 smithi012 ceph-mon[31632]: Updating smithi012:/var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T22:28:17.253 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:16 smithi012 ceph-mon[31632]: Updating smithi151:/var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T22:28:17.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:16 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:17.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:16 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:17.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:16 smithi096 ceph-mon[36065]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:17.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:16 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:17.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:16 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:17.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:16 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:17.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:16 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:28:17.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:16 smithi096 ceph-mon[36065]: Updating smithi012:/etc/ceph/ceph.conf 2024-05-17T22:28:17.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:16 smithi096 ceph-mon[36065]: Updating smithi096:/etc/ceph/ceph.conf 2024-05-17T22:28:17.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:16 smithi096 ceph-mon[36065]: Updating smithi151:/etc/ceph/ceph.conf 2024-05-17T22:28:17.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:16 smithi096 ceph-mon[36065]: Updating smithi096:/var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T22:28:17.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:16 smithi096 ceph-mon[36065]: Updating smithi012:/var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T22:28:17.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:16 smithi096 ceph-mon[36065]: Updating smithi151:/var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T22:28:17.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:16 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:17.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:16 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:17.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:16 smithi151 ceph-mon[36101]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:17.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:16 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:17.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:16 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:17.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:16 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:17.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:16 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:28:17.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:16 smithi151 ceph-mon[36101]: Updating smithi012:/etc/ceph/ceph.conf 2024-05-17T22:28:17.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:16 smithi151 ceph-mon[36101]: Updating smithi096:/etc/ceph/ceph.conf 2024-05-17T22:28:17.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:16 smithi151 ceph-mon[36101]: Updating smithi151:/etc/ceph/ceph.conf 2024-05-17T22:28:17.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:16 smithi151 ceph-mon[36101]: Updating smithi096:/var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T22:28:17.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:16 smithi151 ceph-mon[36101]: Updating smithi012:/var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T22:28:17.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:16 smithi151 ceph-mon[36101]: Updating smithi151:/var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T22:28:17.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:17.789+0000 7f76def40700 1 -- 172.21.15.12:0/2521954087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76d8109590 msgr2=0x7f76d81099a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:28:17.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:17.789+0000 7f76def40700 1 --2- 172.21.15.12:0/2521954087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76d8109590 0x7f76d81099a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f76cc009910 tx=0x7f76cc009c20 comp rx=0 tx=0).stop 2024-05-17T22:28:17.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:17.789+0000 7f76def40700 1 -- 172.21.15.12:0/2521954087 shutdown_connections 2024-05-17T22:28:17.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:17.789+0000 7f76def40700 1 --2- 172.21.15.12:0/2521954087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76d8109590 0x7f76d81099a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:28:17.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:17.789+0000 7f76def40700 1 -- 172.21.15.12:0/2521954087 >> 172.21.15.12:0/2521954087 conn(0x7f76d80fd870 msgr2=0x7f76d80ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:28:17.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:17.789+0000 7f76def40700 1 -- 172.21.15.12:0/2521954087 shutdown_connections 2024-05-17T22:28:17.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:17.789+0000 7f76def40700 1 -- 172.21.15.12:0/2521954087 wait complete. 2024-05-17T22:28:17.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:17.790+0000 7f76def40700 1 Processor -- start 2024-05-17T22:28:17.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:17.790+0000 7f76def40700 1 -- start start 2024-05-17T22:28:17.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:17.791+0000 7f76def40700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76d8109590 0x7f76d81995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:28:17.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:17.791+0000 7f76def40700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76d8199b30 0x7f76d819de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:28:17.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:17.791+0000 7f76def40700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76d819e360 0x7f76d819e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:28:17.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:17.791+0000 7f76def40700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f76d810dac0 con 0x7f76d8199b30 2024-05-17T22:28:17.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:17.791+0000 7f76d7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76d8199b30 0x7f76d819de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:28:17.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:17.791+0000 7f76def40700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f76d810d940 con 0x7f76d8109590 2024-05-17T22:28:17.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:17.791+0000 7f76def40700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f76d810dc40 con 0x7f76d819e360 2024-05-17T22:28:17.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:17.791+0000 7f76d7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76d8199b30 0x7f76d819de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42234/0 (socket says 172.21.15.12:42234) 2024-05-17T22:28:17.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:17.791+0000 7f76d7fff700 1 -- 172.21.15.12:0/4203710446 learned_addr learned my addr 172.21.15.12:0/4203710446 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:28:17.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:17.791+0000 7f76dccdc700 1 --2- 172.21.15.12:0/4203710446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76d8109590 0x7f76d81995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:28:17.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:17.791+0000 7f76dd4dd700 1 --2- 172.21.15.12:0/4203710446 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76d819e360 0x7f76d819e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:28:17.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:17.791+0000 7f76d7fff700 1 -- 172.21.15.12:0/4203710446 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76d819e360 msgr2=0x7f76d819e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:28:17.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:17.791+0000 7f76d7fff700 1 --2- 172.21.15.12:0/4203710446 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76d819e360 0x7f76d819e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:28:17.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:17.791+0000 7f76d7fff700 1 -- 172.21.15.12:0/4203710446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76d8109590 msgr2=0x7f76d81995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:28:17.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:17.791+0000 7f76d7fff700 1 --2- 172.21.15.12:0/4203710446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76d8109590 0x7f76d81995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:28:17.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:17.791+0000 7f76d7fff700 1 -- 172.21.15.12:0/4203710446 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d81a60d0 con 0x7f76d8199b30 2024-05-17T22:28:17.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:17.791+0000 7f76d7fff700 1 --2- 172.21.15.12:0/4203710446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76d8199b30 0x7f76d819de20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f76c400b780 tx=0x7f76c400ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:28:17.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:17.792+0000 7f76d5ffb700 1 -- 172.21.15.12:0/4203710446 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c4004230 con 0x7f76d8199b30 2024-05-17T22:28:17.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:17.792+0000 7f76d5ffb700 1 -- 172.21.15.12:0/4203710446 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76c40043d0 con 0x7f76d8199b30 2024-05-17T22:28:17.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:17.792+0000 7f76d5ffb700 1 -- 172.21.15.12:0/4203710446 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c4011b20 con 0x7f76d8199b30 2024-05-17T22:28:17.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:17.792+0000 7f76def40700 1 -- 172.21.15.12:0/4203710446 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76d81a63c0 con 0x7f76d8199b30 2024-05-17T22:28:17.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:17.792+0000 7f76def40700 1 -- 172.21.15.12:0/4203710446 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76d81a6900 con 0x7f76d8199b30 2024-05-17T22:28:17.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:17.793+0000 7f76d5ffb700 1 -- 172.21.15.12:0/4203710446 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f76c4011cc0 con 0x7f76d8199b30 2024-05-17T22:28:17.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:17.793+0000 7f76def40700 1 -- 172.21.15.12:0/4203710446 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76d8104d10 con 0x7f76d8199b30 2024-05-17T22:28:17.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:17.793+0000 7f76d5ffb700 1 --2- 172.21.15.12:0/4203710446 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f76c803e9a0 0x7f76c8040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:28:17.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:17.793+0000 7f76d5ffb700 1 -- 172.21.15.12:0/4203710446 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f76c4052390 con 0x7f76d8199b30 2024-05-17T22:28:17.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:17.794+0000 7f76dccdc700 1 --2- 172.21.15.12:0/4203710446 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f76c803e9a0 0x7f76c8040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:28:17.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:17.794+0000 7f76dccdc700 1 --2- 172.21.15.12:0/4203710446 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f76c803e9a0 0x7f76c8040eb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f76cc005e00 tx=0x7f76cc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:28:17.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:17.799+0000 7f76d5ffb700 1 -- 172.21.15.12:0/4203710446 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f76c4020740 con 0x7f76d8199b30 2024-05-17T22:28:18.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:18.015+0000 7f76def40700 1 -- 172.21.15.12:0/4203710446 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f76d81a6fe0 con 0x7f76d8199b30 2024-05-17T22:28:18.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:18.016+0000 7f76d5ffb700 1 -- 172.21.15.12:0/4203710446 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f76c40151c0 con 0x7f76d8199b30 2024-05-17T22:28:18.018 INFO:teuthology.orchestra.run.smithi012.stdout:# minimal ceph.conf for 4d593780-149c-11ef-bc9a-c7b262605968 2024-05-17T22:28:18.018 INFO:teuthology.orchestra.run.smithi012.stdout:[global] 2024-05-17T22:28:18.018 INFO:teuthology.orchestra.run.smithi012.stdout: fsid = 4d593780-149c-11ef-bc9a-c7b262605968 2024-05-17T22:28:18.018 INFO:teuthology.orchestra.run.smithi012.stdout: mon_host = [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] 2024-05-17T22:28:18.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:18.018+0000 7f76def40700 1 -- 172.21.15.12:0/4203710446 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f76c803e9a0 msgr2=0x7f76c8040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:28:18.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:18.018+0000 7f76def40700 1 --2- 172.21.15.12:0/4203710446 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f76c803e9a0 0x7f76c8040eb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f76cc005e00 tx=0x7f76cc01a040 comp rx=0 tx=0).stop 2024-05-17T22:28:18.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:18.018+0000 7f76def40700 1 -- 172.21.15.12:0/4203710446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76d8199b30 msgr2=0x7f76d819de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:28:18.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:18.018+0000 7f76def40700 1 --2- 172.21.15.12:0/4203710446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76d8199b30 0x7f76d819de20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f76c400b780 tx=0x7f76c400ba90 comp rx=0 tx=0).stop 2024-05-17T22:28:18.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:18.018+0000 7f76def40700 1 -- 172.21.15.12:0/4203710446 shutdown_connections 2024-05-17T22:28:18.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:18.018+0000 7f76def40700 1 --2- 172.21.15.12:0/4203710446 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f76c803e9a0 0x7f76c8040eb0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:28:18.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:18.018+0000 7f76def40700 1 --2- 172.21.15.12:0/4203710446 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76d819e360 0x7f76d819e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:28:18.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:18.018+0000 7f76def40700 1 --2- 172.21.15.12:0/4203710446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76d8109590 0x7f76d81995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:28:18.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:18.018+0000 7f76def40700 1 --2- 172.21.15.12:0/4203710446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76d8199b30 0x7f76d819de20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:28:18.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:18.018+0000 7f76def40700 1 -- 172.21.15.12:0/4203710446 >> 172.21.15.12:0/4203710446 conn(0x7f76d80fd870 msgr2=0x7f76d80fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:28:18.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:18.018+0000 7f76def40700 1 -- 172.21.15.12:0/4203710446 shutdown_connections 2024-05-17T22:28:18.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:18.018+0000 7f76def40700 1 -- 172.21.15.12:0/4203710446 wait complete. 2024-05-17T22:28:18.316 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:18 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:18.316 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:18 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:18.316 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:18 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:18.316 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:18 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:18.316 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:18 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:18.316 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:18 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:18.317 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:18 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:18.317 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:18 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:18.317 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:18 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:18.317 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:18 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:18.317 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:18 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:18.317 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:18 smithi012 ceph-mon[31632]: Reconfiguring mon.a (unknown last config time)... 2024-05-17T22:28:18.317 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:18 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T22:28:18.317 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:18 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-17T22:28:18.317 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:18 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:18.317 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:18 smithi012 ceph-mon[31632]: Reconfiguring daemon mon.a on smithi012 2024-05-17T22:28:18.317 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:18 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/4203710446' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:18.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:18 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:18.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:18 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:18.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:18 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:18.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:18 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:18.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:18 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:18.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:18 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:18.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:18 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:18.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:18 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:18.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:18 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:18.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:18 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:18.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:18 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:18.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:18 smithi151 ceph-mon[36101]: Reconfiguring mon.a (unknown last config time)... 2024-05-17T22:28:18.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:18 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T22:28:18.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:18 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-17T22:28:18.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:18 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:18.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:18 smithi151 ceph-mon[36101]: Reconfiguring daemon mon.a on smithi012 2024-05-17T22:28:18.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:18 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/4203710446' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:18.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:18 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:18.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:18 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:18.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:18 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:18.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:18 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:18.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:18 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:18.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:18 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:18.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:18 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:18.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:18 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:18.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:18 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:18.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:18 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:18.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:18 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:18.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:18 smithi096 ceph-mon[36065]: Reconfiguring mon.a (unknown last config time)... 2024-05-17T22:28:18.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:18 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T22:28:18.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:18 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-17T22:28:18.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:18 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:18.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:18 smithi096 ceph-mon[36065]: Reconfiguring daemon mon.a on smithi012 2024-05-17T22:28:18.550 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:18 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/4203710446' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:19.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:19 smithi012 ceph-mon[31632]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:19.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:19 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:19.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:19 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:19.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:19 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T22:28:19.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:19 smithi151 ceph-mon[36101]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:19.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:19 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:19.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:19 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:19.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:19 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T22:28:19.520 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-17T22:28:19.521 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-17T22:28:19.521 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/ceph/ceph.conf 2024-05-17T22:28:19.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:19 smithi096 ceph-mon[36065]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:19.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:19 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:19.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:19 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:19.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:19 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T22:28:19.589 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-17T22:28:19.589 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-17T22:28:19.656 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-05-17T22:28:19.656 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/ceph/ceph.conf 2024-05-17T22:28:19.682 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-05-17T22:28:19.682 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-17T22:28:19.746 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-05-17T22:28:19.747 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/ceph/ceph.conf 2024-05-17T22:28:19.778 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-05-17T22:28:19.778 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-17T22:28:19.846 INFO:tasks.cephadm:Adding mgr.x on smithi012 2024-05-17T22:28:19.846 INFO:tasks.cephadm:Adding mgr.y on smithi096 2024-05-17T22:28:19.846 INFO:tasks.cephadm:Adding mgr.z on smithi151 2024-05-17T22:28:19.847 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph orch apply mgr '3;smithi012=x;smithi096=y;smithi151=z' 2024-05-17T22:28:20.120 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T22:28:20.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:20 smithi096 ceph-mon[36065]: Reconfiguring mon.b (monmap changed)... 2024-05-17T22:28:20.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:20 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-17T22:28:20.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:20 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:20.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:20 smithi096 ceph-mon[36065]: Reconfiguring daemon mon.b on smithi096 2024-05-17T22:28:20.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:20 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:20.412 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:20 smithi151 ceph-mon[36101]: Reconfiguring mon.b (monmap changed)... 2024-05-17T22:28:20.412 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:20 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-17T22:28:20.412 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:20 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:20.412 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:20 smithi151 ceph-mon[36101]: Reconfiguring daemon mon.b on smithi096 2024-05-17T22:28:20.412 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:20 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:20.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:20 smithi012 ceph-mon[31632]: Reconfiguring mon.b (monmap changed)... 2024-05-17T22:28:20.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:20 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-17T22:28:20.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:20 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:20.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:20 smithi012 ceph-mon[31632]: Reconfiguring daemon mon.b on smithi096 2024-05-17T22:28:20.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:20 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:21.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.472+0000 7f3cbd494700 1 -- 172.21.15.151:0/2336881722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cb8101d00 msgr2=0x7f3cb8102110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:28:21.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.472+0000 7f3cbd494700 1 --2- 172.21.15.151:0/2336881722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cb8101d00 0x7f3cb8102110 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8009910 tx=0x7f3ca8009c20 comp rx=0 tx=0).stop 2024-05-17T22:28:21.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.472+0000 7f3cbd494700 1 -- 172.21.15.151:0/2336881722 shutdown_connections 2024-05-17T22:28:21.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.472+0000 7f3cbd494700 1 --2- 172.21.15.151:0/2336881722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cb8101d00 0x7f3cb8102110 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:28:21.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.472+0000 7f3cbd494700 1 -- 172.21.15.151:0/2336881722 >> 172.21.15.151:0/2336881722 conn(0x7f3cb80fd570 msgr2=0x7f3cb80ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:28:21.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.472+0000 7f3cbd494700 1 -- 172.21.15.151:0/2336881722 shutdown_connections 2024-05-17T22:28:21.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.472+0000 7f3cbd494700 1 -- 172.21.15.151:0/2336881722 wait complete. 2024-05-17T22:28:21.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.473+0000 7f3cbd494700 1 Processor -- start 2024-05-17T22:28:21.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.473+0000 7f3cbd494700 1 -- start start 2024-05-17T22:28:21.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.473+0000 7f3cbd494700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cb8101d00 0x7f3cb80fe470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:28:21.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.473+0000 7f3cbd494700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3cb819c060 0x7f3cb819e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:28:21.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.473+0000 7f3cbd494700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3cb819e9c0 0x7f3cb81a0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:28:21.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.473+0000 7f3cbd494700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb8107cd0 con 0x7f3cb8101d00 2024-05-17T22:28:21.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.473+0000 7f3cbd494700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb8107b50 con 0x7f3cb819c060 2024-05-17T22:28:21.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.473+0000 7f3cbd494700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb8107e50 con 0x7f3cb819e9c0 2024-05-17T22:28:21.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.474+0000 7f3cb77fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3cb819e9c0 0x7f3cb81a0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:28:21.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.474+0000 7f3cb77fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3cb819e9c0 0x7f3cb81a0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46744/0 (socket says 172.21.15.151:46744) 2024-05-17T22:28:21.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.474+0000 7f3cb6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cb8101d00 0x7f3cb80fe470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:28:21.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.474+0000 7f3cb77fe700 1 -- 172.21.15.151:0/986902256 learned_addr learned my addr 172.21.15.151:0/986902256 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-17T22:28:21.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.474+0000 7f3cb67fc700 1 --2- 172.21.15.151:0/986902256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3cb819c060 0x7f3cb819e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:28:21.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.474+0000 7f3cb6ffd700 1 -- 172.21.15.151:0/986902256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3cb819e9c0 msgr2=0x7f3cb81a0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:28:21.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.474+0000 7f3cb6ffd700 1 --2- 172.21.15.151:0/986902256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3cb819e9c0 0x7f3cb81a0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:28:21.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.474+0000 7f3cb6ffd700 1 -- 172.21.15.151:0/986902256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3cb819c060 msgr2=0x7f3cb819e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:28:21.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.474+0000 7f3cb6ffd700 1 --2- 172.21.15.151:0/986902256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3cb819c060 0x7f3cb819e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:28:21.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.474+0000 7f3cb6ffd700 1 -- 172.21.15.151:0/986902256 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cb81a1670 con 0x7f3cb8101d00 2024-05-17T22:28:21.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.474+0000 7f3cb77fe700 1 --2- 172.21.15.151:0/986902256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3cb819e9c0 0x7f3cb81a0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:28:21.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.474+0000 7f3cb67fc700 1 --2- 172.21.15.151:0/986902256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3cb819c060 0x7f3cb819e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:28:21.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.474+0000 7f3cb6ffd700 1 --2- 172.21.15.151:0/986902256 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cb8101d00 0x7f3cb80fe470 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8102d30 tx=0x7f3ca8004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:28:21.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.474+0000 7f3c9ffff700 1 -- 172.21.15.151:0/986902256 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca8026070 con 0x7f3cb8101d00 2024-05-17T22:28:21.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.474+0000 7f3cbd494700 1 -- 172.21.15.151:0/986902256 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cb81a1900 con 0x7f3cb8101d00 2024-05-17T22:28:21.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.474+0000 7f3c9ffff700 1 -- 172.21.15.151:0/986902256 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ca800b750 con 0x7f3cb8101d00 2024-05-17T22:28:21.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.475+0000 7f3c9ffff700 1 -- 172.21.15.151:0/986902256 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca80038e0 con 0x7f3cb8101d00 2024-05-17T22:28:21.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.475+0000 7f3cbd494700 1 -- 172.21.15.151:0/986902256 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cb81a1e60 con 0x7f3cb8101d00 2024-05-17T22:28:21.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.476+0000 7f3cbd494700 1 -- 172.21.15.151:0/986902256 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cb8102190 con 0x7f3cb8101d00 2024-05-17T22:28:21.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.476+0000 7f3c9ffff700 1 -- 172.21.15.151:0/986902256 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f3ca8029070 con 0x7f3cb8101d00 2024-05-17T22:28:21.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.476+0000 7f3c9ffff700 1 --2- 172.21.15.151:0/986902256 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f3ca003e9a0 0x7f3ca0040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:28:21.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.476+0000 7f3c9ffff700 1 -- 172.21.15.151:0/986902256 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3ca80535a0 con 0x7f3cb8101d00 2024-05-17T22:28:21.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.476+0000 7f3cb67fc700 1 --2- 172.21.15.151:0/986902256 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f3ca003e9a0 0x7f3ca0040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:28:21.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.477+0000 7f3cb67fc700 1 --2- 172.21.15.151:0/986902256 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f3ca003e9a0 0x7f3ca0040eb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3cb80fde20 tx=0x7f3cac008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:28:21.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.482+0000 7f3c9ffff700 1 -- 172.21.15.151:0/986902256 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3ca8013070 con 0x7f3cb8101d00 2024-05-17T22:28:21.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.707+0000 7f3cbd494700 1 -- 172.21.15.151:0/986902256 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi012=x;smithi096=y;smithi151=z", "target": ["mon-mgr", ""]}) -- 0x7f3cb80648d0 con 0x7f3ca003e9a0 2024-05-17T22:28:21.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.714+0000 7f3c9ffff700 1 -- 172.21.15.151:0/986902256 <== mgr.14152 v2:172.21.15.12:6800/1388514047 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f3cb80648d0 con 0x7f3ca003e9a0 2024-05-17T22:28:21.716 INFO:teuthology.orchestra.run.smithi151.stdout:Scheduled mgr update... 2024-05-17T22:28:21.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.716+0000 7f3cbd494700 1 -- 172.21.15.151:0/986902256 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f3ca003e9a0 msgr2=0x7f3ca0040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:28:21.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.716+0000 7f3cbd494700 1 --2- 172.21.15.151:0/986902256 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f3ca003e9a0 0x7f3ca0040eb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3cb80fde20 tx=0x7f3cac008040 comp rx=0 tx=0).stop 2024-05-17T22:28:21.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.716+0000 7f3cbd494700 1 -- 172.21.15.151:0/986902256 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cb8101d00 msgr2=0x7f3cb80fe470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:28:21.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.716+0000 7f3cbd494700 1 --2- 172.21.15.151:0/986902256 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cb8101d00 0x7f3cb80fe470 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8102d30 tx=0x7f3ca8004ed0 comp rx=0 tx=0).stop 2024-05-17T22:28:21.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.716+0000 7f3cbd494700 1 -- 172.21.15.151:0/986902256 shutdown_connections 2024-05-17T22:28:21.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.716+0000 7f3cbd494700 1 --2- 172.21.15.151:0/986902256 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f3ca003e9a0 0x7f3ca0040eb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:28:21.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.716+0000 7f3cbd494700 1 --2- 172.21.15.151:0/986902256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3cb819e9c0 0x7f3cb81a0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:28:21.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.716+0000 7f3cbd494700 1 --2- 172.21.15.151:0/986902256 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cb8101d00 0x7f3cb80fe470 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:28:21.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.716+0000 7f3cbd494700 1 --2- 172.21.15.151:0/986902256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3cb819c060 0x7f3cb819e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:28:21.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.716+0000 7f3cbd494700 1 -- 172.21.15.151:0/986902256 >> 172.21.15.151:0/986902256 conn(0x7f3cb80fd570 msgr2=0x7f3cb80feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:28:21.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.716+0000 7f3cbd494700 1 -- 172.21.15.151:0/986902256 shutdown_connections 2024-05-17T22:28:21.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:28:21.716+0000 7f3cbd494700 1 -- 172.21.15.151:0/986902256 wait complete. 2024-05-17T22:28:21.888 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:21 smithi151 ceph-mon[36101]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:21.888 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:21 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:21.888 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:21 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:21.888 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:21 smithi151 ceph-mon[36101]: Reconfiguring mon.c (monmap changed)... 2024-05-17T22:28:21.889 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:21 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T22:28:21.889 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:21 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-17T22:28:21.889 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:21 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:21.889 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:21 smithi151 ceph-mon[36101]: Reconfiguring daemon mon.c on smithi151 2024-05-17T22:28:22.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:21 smithi096 ceph-mon[36065]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:22.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:21 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:22.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:21 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:22.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:21 smithi096 ceph-mon[36065]: Reconfiguring mon.c (monmap changed)... 2024-05-17T22:28:22.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:21 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T22:28:22.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:21 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-17T22:28:22.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:21 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:22.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:21 smithi096 ceph-mon[36065]: Reconfiguring daemon mon.c on smithi151 2024-05-17T22:28:22.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:21 smithi012 ceph-mon[31632]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:22.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:21 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:22.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:21 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:22.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:21 smithi012 ceph-mon[31632]: Reconfiguring mon.c (monmap changed)... 2024-05-17T22:28:22.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:21 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T22:28:22.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:21 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-17T22:28:22.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:21 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:22.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:21 smithi012 ceph-mon[31632]: Reconfiguring daemon mon.c on smithi151 2024-05-17T22:28:22.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:22 smithi151 ceph-mon[36101]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi012=x;smithi096=y;smithi151=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:28:22.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:22 smithi151 ceph-mon[36101]: Saving service mgr spec with placement smithi012=x;smithi096=y;smithi151=z;count:3 2024-05-17T22:28:22.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:22 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:22.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:22 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:22.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:22 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:22.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:22 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:28:23.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:22 smithi096 ceph-mon[36065]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi012=x;smithi096=y;smithi151=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:28:23.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:22 smithi096 ceph-mon[36065]: Saving service mgr spec with placement smithi012=x;smithi096=y;smithi151=z;count:3 2024-05-17T22:28:23.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:22 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:23.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:22 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:23.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:22 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:23.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:22 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:28:23.062 DEBUG:teuthology.orchestra.run.smithi096:mgr.y> sudo journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@mgr.y.service 2024-05-17T22:28:23.065 DEBUG:teuthology.orchestra.run.smithi151:mgr.z> sudo journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@mgr.z.service 2024-05-17T22:28:23.067 INFO:tasks.cephadm:Deploying OSDs... 2024-05-17T22:28:23.068 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-17T22:28:23.068 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/scratch_devs of=/dev/stdout 2024-05-17T22:28:23.084 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-17T22:28:23.084 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_1 2024-05-17T22:28:23.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:22 smithi012 ceph-mon[31632]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi012=x;smithi096=y;smithi151=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:28:23.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:22 smithi012 ceph-mon[31632]: Saving service mgr spec with placement smithi012=x;smithi096=y;smithi151=z;count:3 2024-05-17T22:28:23.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:22 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:23.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:22 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:23.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:22 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:23.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:22 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:28:23.106 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-17T22:28:23.106 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T22:28:23.106 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-17T22:28:23.106 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T22:28:23.106 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T22:28:23.106 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-05-17 22:27:28.045409955 +0000 2024-05-17T22:28:23.106 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-05-17 22:22:16.442863442 +0000 2024-05-17T22:28:23.106 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-05-17 22:22:16.442863442 +0000 2024-05-17T22:28:23.106 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2024-05-17 22:22:16.442863442 +0000 2024-05-17T22:28:23.107 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-17T22:28:23.174 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-05-17T22:28:23.174 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-05-17T22:28:23.174 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000212653 s, 2.4 MB/s 2024-05-17T22:28:23.176 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-17T22:28:23.233 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_2 2024-05-17T22:28:23.288 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-17T22:28:23.289 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T22:28:23.289 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-17T22:28:23.289 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T22:28:23.289 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T22:28:23.289 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-05-17 22:27:28.076409701 +0000 2024-05-17T22:28:23.289 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-05-17 22:22:16.797860602 +0000 2024-05-17T22:28:23.289 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-05-17 22:22:16.797860602 +0000 2024-05-17T22:28:23.289 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2024-05-17 22:22:16.797860602 +0000 2024-05-17T22:28:23.289 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-17T22:28:23.353 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-05-17T22:28:23.353 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-05-17T22:28:23.353 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000196282 s, 2.6 MB/s 2024-05-17T22:28:23.355 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-17T22:28:23.411 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_3 2024-05-17T22:28:23.468 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-17T22:28:23.468 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T22:28:23.468 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-17T22:28:23.468 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T22:28:23.468 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T22:28:23.468 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-05-17 22:27:28.103409480 +0000 2024-05-17T22:28:23.468 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-05-17 22:22:17.143857834 +0000 2024-05-17T22:28:23.468 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-05-17 22:22:17.143857834 +0000 2024-05-17T22:28:23.468 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2024-05-17 22:22:17.143857834 +0000 2024-05-17T22:28:23.469 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-17T22:28:23.532 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-05-17T22:28:23.532 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-05-17T22:28:23.532 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000215875 s, 2.4 MB/s 2024-05-17T22:28:23.533 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-17T22:28:23.590 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_4 2024-05-17T22:28:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-17T22:28:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T22:28:23.647 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-05-17T22:28:23.647 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T22:28:23.647 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T22:28:23.647 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-05-17 22:27:28.130409258 +0000 2024-05-17T22:28:23.647 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-05-17 22:22:17.487855081 +0000 2024-05-17T22:28:23.647 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-05-17 22:22:17.487855081 +0000 2024-05-17T22:28:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2024-05-17 22:22:17.487855081 +0000 2024-05-17T22:28:23.648 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-17T22:28:23.711 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-05-17T22:28:23.711 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-05-17T22:28:23.711 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.00021631 s, 2.4 MB/s 2024-05-17T22:28:23.712 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-17T22:28:23.770 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-05-17T22:28:23.770 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/scratch_devs of=/dev/stdout 2024-05-17T22:28:23.790 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-17T22:28:23.790 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_1 2024-05-17T22:28:23.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:23 smithi012 ceph-mon[31632]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:23.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:23 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:23.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:23 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:28:23.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:23 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:23.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:23 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T22:28:23.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:23 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-17T22:28:23.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:23 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-17T22:28:23.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:23 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:23.849 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-17T22:28:23.849 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T22:28:23.849 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-17T22:28:23.849 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T22:28:23.849 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T22:28:23.849 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-05-17 22:28:08.940002554 +0000 2024-05-17T22:28:23.850 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-05-17 22:22:18.884571785 +0000 2024-05-17T22:28:23.850 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-05-17 22:22:18.884571785 +0000 2024-05-17T22:28:23.850 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-05-17 22:22:18.884571785 +0000 2024-05-17T22:28:23.850 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-17T22:28:23.914 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-05-17T22:28:23.914 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-05-17T22:28:23.915 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000287062 s, 1.8 MB/s 2024-05-17T22:28:23.916 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-17T22:28:23.973 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_2 2024-05-17T22:28:23.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:23 smithi151 ceph-mon[36101]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:23.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:23 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:23.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:23 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:28:23.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:23 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:23.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:23 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T22:28:23.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:23 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-17T22:28:23.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:23 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-17T22:28:23.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:23 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:24.028 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-17T22:28:24.029 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T22:28:24.029 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-05-17T22:28:24.029 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T22:28:24.029 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T22:28:24.029 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-05-17 22:28:08.941002531 +0000 2024-05-17T22:28:24.029 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-05-17 22:22:19.212564172 +0000 2024-05-17T22:28:24.029 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-05-17 22:22:19.212564172 +0000 2024-05-17T22:28:24.029 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-05-17 22:22:19.212564172 +0000 2024-05-17T22:28:24.029 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-17T22:28:24.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:23 smithi096 ceph-mon[36065]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:24.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:23 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:24.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:23 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:28:24.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:23 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:24.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:23 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T22:28:24.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:23 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-17T22:28:24.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:23 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-17T22:28:24.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:23 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:24.071 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-05-17T22:28:24.072 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-05-17T22:28:24.072 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000180601 s, 2.8 MB/s 2024-05-17T22:28:24.073 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-17T22:28:24.129 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_3 2024-05-17T22:28:24.185 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-17T22:28:24.185 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T22:28:24.185 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-05-17T22:28:24.185 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T22:28:24.185 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T22:28:24.185 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-05-17 22:28:08.941002531 +0000 2024-05-17T22:28:24.185 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-05-17 22:22:19.545556442 +0000 2024-05-17T22:28:24.185 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-05-17 22:22:19.545556442 +0000 2024-05-17T22:28:24.185 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-05-17 22:22:19.545556442 +0000 2024-05-17T22:28:24.185 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-17T22:28:24.247 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-05-17T22:28:24.247 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-05-17T22:28:24.248 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000181516 s, 2.8 MB/s 2024-05-17T22:28:24.249 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-17T22:28:24.304 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_4 2024-05-17T22:28:24.360 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-17T22:28:24.360 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T22:28:24.360 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-05-17T22:28:24.360 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T22:28:24.360 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T22:28:24.360 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-05-17 22:28:08.941002531 +0000 2024-05-17T22:28:24.360 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-05-17 22:22:19.879548690 +0000 2024-05-17T22:28:24.360 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-05-17 22:22:19.879548690 +0000 2024-05-17T22:28:24.361 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-05-17 22:22:19.879548690 +0000 2024-05-17T22:28:24.361 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-17T22:28:24.423 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-05-17T22:28:24.423 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-05-17T22:28:24.423 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000214938 s, 2.4 MB/s 2024-05-17T22:28:24.424 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-17T22:28:24.479 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-05-17T22:28:24.480 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/scratch_devs of=/dev/stdout 2024-05-17T22:28:24.495 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-17T22:28:24.495 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_1 2024-05-17T22:28:24.550 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-17T22:28:24.550 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T22:28:24.551 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-05-17T22:28:24.551 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T22:28:24.551 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T22:28:24.551 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-05-17 22:27:59.700821398 +0000 2024-05-17T22:28:24.551 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-05-17 22:22:20.519571612 +0000 2024-05-17T22:28:24.551 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-05-17 22:22:20.519571612 +0000 2024-05-17T22:28:24.551 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-05-17 22:22:20.519571612 +0000 2024-05-17T22:28:24.551 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-17T22:28:24.613 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-05-17T22:28:24.613 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-05-17T22:28:24.613 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000210517 s, 2.4 MB/s 2024-05-17T22:28:24.614 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-17T22:28:24.670 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_2 2024-05-17T22:28:24.725 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-17T22:28:24.725 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T22:28:24.725 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-05-17T22:28:24.726 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T22:28:24.726 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T22:28:24.726 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-05-17 22:27:59.700821398 +0000 2024-05-17T22:28:24.726 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-05-17 22:22:20.859567781 +0000 2024-05-17T22:28:24.726 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-05-17 22:22:20.859567781 +0000 2024-05-17T22:28:24.726 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-05-17 22:22:20.859567781 +0000 2024-05-17T22:28:24.726 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-17T22:28:24.787 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:24 smithi151 ceph-mon[36101]: Deploying daemon mgr.z on smithi151 2024-05-17T22:28:24.789 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-05-17T22:28:24.789 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-05-17T22:28:24.789 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000167708 s, 3.1 MB/s 2024-05-17T22:28:24.790 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-17T22:28:24.847 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_3 2024-05-17T22:28:24.903 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-17T22:28:24.903 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T22:28:24.903 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-17T22:28:24.903 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T22:28:24.903 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T22:28:24.903 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-05-17 22:28:24.890539053 +0000 2024-05-17T22:28:24.903 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-05-17 22:22:21.198563960 +0000 2024-05-17T22:28:24.903 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-05-17 22:22:21.198563960 +0000 2024-05-17T22:28:24.903 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-05-17 22:22:21.198563960 +0000 2024-05-17T22:28:24.903 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-17T22:28:24.966 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-05-17T22:28:24.966 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-05-17T22:28:24.966 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000240833 s, 2.1 MB/s 2024-05-17T22:28:24.967 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-17T22:28:25.024 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_4 2024-05-17T22:28:25.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:24 smithi096 ceph-mon[36065]: Deploying daemon mgr.z on smithi151 2024-05-17T22:28:25.079 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-17T22:28:25.079 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T22:28:25.079 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-05-17T22:28:25.079 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T22:28:25.079 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T22:28:25.079 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-05-17 22:28:24.890539053 +0000 2024-05-17T22:28:25.079 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-05-17 22:22:21.553559960 +0000 2024-05-17T22:28:25.079 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-05-17 22:22:21.553559960 +0000 2024-05-17T22:28:25.080 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-05-17 22:22:21.553559960 +0000 2024-05-17T22:28:25.080 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-17T22:28:25.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:24 smithi012 ceph-mon[31632]: Deploying daemon mgr.z on smithi151 2024-05-17T22:28:25.142 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-05-17T22:28:25.142 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-05-17T22:28:25.142 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000181433 s, 2.8 MB/s 2024-05-17T22:28:25.143 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-17T22:28:25.200 INFO:tasks.cephadm:Deploying osd.0 on smithi012 with /dev/vg_nvme/lv_4... 2024-05-17T22:28:25.200 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-17T22:28:25.456 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:28:25.733 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 17 22:28:25 smithi151 systemd[1]: Started Ceph mgr.z for 4d593780-149c-11ef-bc9a-c7b262605968. 2024-05-17T22:28:25.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:25 smithi012 ceph-mon[31632]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:25.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:25 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:25.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:25 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:25.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:25 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:25.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:25 smithi151 ceph-mon[36101]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:25.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:25 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:25.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:25 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:25.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:25 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:25.990 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 17 22:28:25 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-z[37933]: 2024-05-17T22:28:25.736+0000 7fef208d0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-17T22:28:25.991 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 17 22:28:25 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-z[37933]: 2024-05-17T22:28:25.920+0000 7fef208d0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-17T22:28:26.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:25 smithi096 ceph-mon[36065]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:26.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:25 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:26.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:25 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:26.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:25 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:26.330 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 17 22:28:26 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-z[37933]: 2024-05-17T22:28:26.021+0000 7fef208d0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-17T22:28:26.330 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 17 22:28:26 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-z[37933]: 2024-05-17T22:28:26.328+0000 7fef208d0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-17T22:28:26.721 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 17 22:28:26 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-z[37933]: 2024-05-17T22:28:26.414+0000 7fef208d0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-17T22:28:26.784 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:26 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T22:28:26.784 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:26 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-17T22:28:26.784 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:26 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-17T22:28:26.784 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:26 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:26.784 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:26 smithi012 ceph-mon[31632]: Deploying daemon mgr.y on smithi096 2024-05-17T22:28:26.870 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:26 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T22:28:26.870 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:26 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-17T22:28:26.870 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:26 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-17T22:28:26.870 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:26 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:26.871 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:26 smithi096 ceph-mon[36065]: Deploying daemon mgr.y on smithi096 2024-05-17T22:28:26.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:26 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T22:28:26.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:26 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-17T22:28:26.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:26 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-17T22:28:26.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:26 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:26.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:26 smithi151 ceph-mon[36101]: Deploying daemon mgr.y on smithi096 2024-05-17T22:28:27.374 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-17T22:28:27.374 INFO:teuthology.orchestra.run.smithi012.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-17T22:28:27.374 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10+0 records in 2024-05-17T22:28:27.374 INFO:teuthology.orchestra.run.smithi012.stderr:10+0 records out 2024-05-17T22:28:27.374 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112317 s, 934 MB/s 2024-05-17T22:28:27.374 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping successful for: 2024-05-17T22:28:27.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:27 smithi096 ceph-mon[36065]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:27.799 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 17 22:28:27 smithi096 podman[37436]: 2024-05-17 22:28:27.652122929 +0000 UTC m=+0.017144576 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T22:28:27.959 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:27 smithi012 ceph-mon[31632]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:27.960 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi012:vg_nvme/lv_4 2024-05-17T22:28:27.978 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:27 smithi151 ceph-mon[36101]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:27.979 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 17 22:28:27 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-z[37933]: 2024-05-17T22:28:27.655+0000 7fef208d0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-17T22:28:27.979 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 17 22:28:27 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-z[37933]: 2024-05-17T22:28:27.828+0000 7fef208d0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-17T22:28:28.127 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 17 22:28:27 smithi096 podman[37436]: 2024-05-17 22:28:27.950661251 +0000 UTC m=+0.315682885 container init b2c251c0b1058e3ac7181754684a563a14688fccbd6c1e95cbee8ed1f66db4f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_CLEAN=True, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-17T22:28:28.127 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 17 22:28:27 smithi096 podman[37436]: 2024-05-17 22:28:27.953270928 +0000 UTC m=+0.318292562 container start b2c251c0b1058e3ac7181754684a563a14688fccbd6c1e95cbee8ed1f66db4f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-y, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.build-date=20240514) 2024-05-17T22:28:28.127 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 17 22:28:27 smithi096 ceph-mgr[37455]: -- 172.21.15.96:0/716625400 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56433497c5a0 con 0x564339706480 2024-05-17T22:28:28.127 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 17 22:28:28 smithi096 bash[37436]: b2c251c0b1058e3ac7181754684a563a14688fccbd6c1e95cbee8ed1f66db4f3 2024-05-17T22:28:28.127 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 17 22:28:28 smithi096 systemd[1]: Started Ceph mgr.y for 4d593780-149c-11ef-bc9a-c7b262605968. 2024-05-17T22:28:28.218 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:28:28.240 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 17 22:28:28 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-z[37933]: 2024-05-17T22:28:28.005+0000 7fef208d0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-17T22:28:28.432 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 17 22:28:28 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-y[37451]: 2024-05-17T22:28:28.125+0000 7f97b560a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-17T22:28:28.433 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 17 22:28:28 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-y[37451]: 2024-05-17T22:28:28.318+0000 7f97b560a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-17T22:28:28.433 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 17 22:28:28 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-y[37451]: 2024-05-17T22:28:28.430+0000 7f97b560a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-17T22:28:28.604 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 17 22:28:28 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-z[37933]: 2024-05-17T22:28:28.297+0000 7fef208d0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-17T22:28:28.604 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 17 22:28:28 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-z[37933]: 2024-05-17T22:28:28.393+0000 7fef208d0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-17T22:28:28.604 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 17 22:28:28 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-z[37933]: 2024-05-17T22:28:28.602+0000 7fef208d0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-17T22:28:28.990 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 17 22:28:28 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-z[37933]: 2024-05-17T22:28:28.804+0000 7fef208d0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-17T22:28:29.048 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 17 22:28:28 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-y[37451]: 2024-05-17T22:28:28.791+0000 7f97b560a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-17T22:28:29.048 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 17 22:28:28 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-y[37451]: 2024-05-17T22:28:28.915+0000 7f97b560a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-17T22:28:29.311 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:29 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:29.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:29 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:29.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:29 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:29.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:29 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:29.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:29 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:28:29.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:29 smithi012 ceph-mon[31632]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:29.354 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:29 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:29.354 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:29 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:29.355 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:29 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:29.355 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:29 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:29.355 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:29 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:28:29.355 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:29 smithi096 ceph-mon[36065]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:29.443 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:29 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:29.443 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:29 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:29.443 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:29 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:29.443 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:29 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:29.443 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:29 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:28:29.443 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:29 smithi151 ceph-mon[36101]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:29.740 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 17 22:28:29 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-z[37933]: 2024-05-17T22:28:29.440+0000 7fef208d0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-17T22:28:29.740 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 17 22:28:29 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-z[37933]: 2024-05-17T22:28:29.672+0000 7fef208d0200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-17T22:28:29.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:29.826+0000 7fb3ecc92700 1 -- 172.21.15.12:0/1885973274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3e8104dc0 msgr2=0x7fb3e81051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:28:29.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:29.826+0000 7fb3ecc92700 1 --2- 172.21.15.12:0/1885973274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3e8104dc0 0x7fb3e81051d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb3d00099c0 tx=0x7fb3d0009cd0 comp rx=0 tx=0).stop 2024-05-17T22:28:29.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:29.826+0000 7fb3ecc92700 1 -- 172.21.15.12:0/1885973274 shutdown_connections 2024-05-17T22:28:29.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:29.826+0000 7fb3ecc92700 1 --2- 172.21.15.12:0/1885973274 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3e8105760 0x7fb3e810c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:28:29.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:29.826+0000 7fb3ecc92700 1 --2- 172.21.15.12:0/1885973274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3e8103c90 0x7fb3e81040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:28:29.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:29.826+0000 7fb3ecc92700 1 --2- 172.21.15.12:0/1885973274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3e8104dc0 0x7fb3e81051d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:28:29.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:29.826+0000 7fb3ecc92700 1 -- 172.21.15.12:0/1885973274 >> 172.21.15.12:0/1885973274 conn(0x7fb3e80ff4c0 msgr2=0x7fb3e8101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:28:29.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:29.826+0000 7fb3ecc92700 1 -- 172.21.15.12:0/1885973274 shutdown_connections 2024-05-17T22:28:29.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:29.826+0000 7fb3ecc92700 1 -- 172.21.15.12:0/1885973274 wait complete. 2024-05-17T22:28:29.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:29.827+0000 7fb3ecc92700 1 Processor -- start 2024-05-17T22:28:29.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:29.827+0000 7fb3ecc92700 1 -- start start 2024-05-17T22:28:29.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:29.827+0000 7fb3ecc92700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3e8103c90 0x7fb3e819a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:28:29.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:29.827+0000 7fb3ecc92700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3e8104dc0 0x7fb3e819a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:28:29.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:29.827+0000 7fb3ecc92700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3e8105760 0x7fb3e81a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:28:29.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:29.827+0000 7fb3ecc92700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e810f700 con 0x7fb3e8104dc0 2024-05-17T22:28:29.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:29.827+0000 7fb3ecc92700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e810f580 con 0x7fb3e8105760 2024-05-17T22:28:29.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:29.827+0000 7fb3ecc92700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e810f880 con 0x7fb3e8103c90 2024-05-17T22:28:29.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:29.828+0000 7fb3ddd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3e8104dc0 0x7fb3e819a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:28:29.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:29.828+0000 7fb3e659c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3e8103c90 0x7fb3e819a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:28:29.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:29.828+0000 7fb3e659c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3e8103c90 0x7fb3e819a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:41008/0 (socket says 172.21.15.12:41008) 2024-05-17T22:28:29.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:29.828+0000 7fb3ddd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3e8104dc0 0x7fb3e819a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36220/0 (socket says 172.21.15.12:36220) 2024-05-17T22:28:29.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:29.828+0000 7fb3e659c700 1 -- 172.21.15.12:0/3904897833 learned_addr learned my addr 172.21.15.12:0/3904897833 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:28:29.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:29.828+0000 7fb3e6d9d700 1 --2- 172.21.15.12:0/3904897833 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3e8105760 0x7fb3e81a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:28:29.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:29.828+0000 7fb3ddd9b700 1 -- 172.21.15.12:0/3904897833 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3e8103c90 msgr2=0x7fb3e819a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:28:29.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:29.828+0000 7fb3ddd9b700 1 --2- 172.21.15.12:0/3904897833 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3e8103c90 0x7fb3e819a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:28:29.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:29.828+0000 7fb3ddd9b700 1 -- 172.21.15.12:0/3904897833 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3e8105760 msgr2=0x7fb3e81a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:28:29.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:29.828+0000 7fb3ddd9b700 1 --2- 172.21.15.12:0/3904897833 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3e8105760 0x7fb3e81a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:28:29.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:29.828+0000 7fb3ddd9b700 1 -- 172.21.15.12:0/3904897833 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3e81a1dc0 con 0x7fb3e8104dc0 2024-05-17T22:28:29.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:29.828+0000 7fb3e6d9d700 1 --2- 172.21.15.12:0/3904897833 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3e8105760 0x7fb3e81a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:28:29.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:29.828+0000 7fb3ddd9b700 1 --2- 172.21.15.12:0/3904897833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3e8104dc0 0x7fb3e819a7a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb3d00098e0 tx=0x7fb3d0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:28:29.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:29.828+0000 7fb3dffff700 1 -- 172.21.15.12:0/3904897833 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0027070 con 0x7fb3e8104dc0 2024-05-17T22:28:29.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:29.828+0000 7fb3dffff700 1 -- 172.21.15.12:0/3904897833 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3d0022470 con 0x7fb3e8104dc0 2024-05-17T22:28:29.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:29.828+0000 7fb3ecc92700 1 -- 172.21.15.12:0/3904897833 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3e81a2050 con 0x7fb3e8104dc0 2024-05-17T22:28:29.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:29.829+0000 7fb3dffff700 1 -- 172.21.15.12:0/3904897833 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0009e60 con 0x7fb3e8104dc0 2024-05-17T22:28:29.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:29.829+0000 7fb3ecc92700 1 -- 172.21.15.12:0/3904897833 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3e81a24e0 con 0x7fb3e8104dc0 2024-05-17T22:28:29.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:29.830+0000 7fb3ecc92700 1 -- 172.21.15.12:0/3904897833 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3e81040a0 con 0x7fb3e8104dc0 2024-05-17T22:28:29.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:29.830+0000 7fb3dffff700 1 -- 172.21.15.12:0/3904897833 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fb3d002a070 con 0x7fb3e8104dc0 2024-05-17T22:28:29.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:29.830+0000 7fb3dffff700 1 --2- 172.21.15.12:0/3904897833 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb3d403e750 0x7fb3d4040c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:28:29.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:29.830+0000 7fb3dffff700 1 -- 172.21.15.12:0/3904897833 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb3d0053320 con 0x7fb3e8104dc0 2024-05-17T22:28:29.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:29.830+0000 7fb3e659c700 1 --2- 172.21.15.12:0/3904897833 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb3d403e750 0x7fb3d4040c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:28:29.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:29.831+0000 7fb3e659c700 1 --2- 172.21.15.12:0/3904897833 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb3d403e750 0x7fb3d4040c60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb3d800f5e0 tx=0x7fb3d800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:28:29.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:29.835+0000 7fb3dffff700 1 -- 172.21.15.12:0/3904897833 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb3d0013070 con 0x7fb3e8104dc0 2024-05-17T22:28:30.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:30.066+0000 7fb3ecc92700 1 -- 172.21.15.12:0/3904897833 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fb3e8064740 con 0x7fb3d403e750 2024-05-17T22:28:30.453 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 17 22:28:30 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-z[37933]: 2024-05-17T22:28:30.101+0000 7fef208d0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-17T22:28:30.453 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 17 22:28:30 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-z[37933]: 2024-05-17T22:28:30.197+0000 7fef208d0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-17T22:28:30.454 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 17 22:28:30 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-y[37451]: 2024-05-17T22:28:30.156+0000 7f97b560a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-17T22:28:30.454 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 17 22:28:30 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-y[37451]: 2024-05-17T22:28:30.323+0000 7f97b560a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-17T22:28:30.693 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:30 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:30.693 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:30 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:30.693 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:30 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:30.694 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:30 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:30.694 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:30 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:30.694 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:30 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:30.694 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:30 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:28:30.694 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:30 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:30.694 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:30 smithi012 ceph-mon[31632]: Reconfiguring mgr.x (unknown last config time)... 2024-05-17T22:28:30.694 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:30 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T22:28:30.694 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:30 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-17T22:28:30.694 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:30 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:30.694 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:30 smithi012 ceph-mon[31632]: Reconfiguring daemon mgr.x on smithi012 2024-05-17T22:28:30.694 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:30 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T22:28:30.694 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:30 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T22:28:30.694 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:30 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:30.740 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 17 22:28:30 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-z[37933]: 2024-05-17T22:28:30.569+0000 7fef208d0200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-17T22:28:30.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:30 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:30.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:30 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:30.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:30 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:30.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:30 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:30.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:30 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:30.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:30 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:30.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:30 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:28:30.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:30 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:30.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:30 smithi151 ceph-mon[36101]: Reconfiguring mgr.x (unknown last config time)... 2024-05-17T22:28:30.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:30 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T22:28:30.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:30 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-17T22:28:30.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:30 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:30.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:30 smithi151 ceph-mon[36101]: Reconfiguring daemon mgr.x on smithi012 2024-05-17T22:28:30.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:30 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T22:28:30.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:30 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T22:28:30.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:30 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:30.782 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 17 22:28:30 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-y[37451]: 2024-05-17T22:28:30.494+0000 7f97b560a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-17T22:28:30.782 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:30 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:30.782 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:30 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:30.783 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:30 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:30.783 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:30 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:30.783 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:30 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:30.783 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:30 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:30.783 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:30 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:28:30.783 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:30 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:30.783 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:30 smithi096 ceph-mon[36065]: Reconfiguring mgr.x (unknown last config time)... 2024-05-17T22:28:30.783 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:30 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T22:28:30.783 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:30 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-17T22:28:30.783 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:30 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:30.783 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:30 smithi096 ceph-mon[36065]: Reconfiguring daemon mgr.x on smithi012 2024-05-17T22:28:30.783 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:30 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T22:28:30.783 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:30 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T22:28:30.784 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:30 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:31.048 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 17 22:28:30 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-y[37451]: 2024-05-17T22:28:30.780+0000 7f97b560a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-17T22:28:31.048 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 17 22:28:30 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-y[37451]: 2024-05-17T22:28:30.872+0000 7f97b560a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-17T22:28:31.240 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 17 22:28:30 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-z[37933]: 2024-05-17T22:28:30.922+0000 7fef208d0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-17T22:28:31.240 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 17 22:28:31 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-z[37933]: 2024-05-17T22:28:31.014+0000 7fef208d0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-17T22:28:31.548 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 17 22:28:31 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-y[37451]: 2024-05-17T22:28:31.072+0000 7f97b560a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-17T22:28:31.548 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 17 22:28:31 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-y[37451]: 2024-05-17T22:28:31.261+0000 7f97b560a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-17T22:28:31.652 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 17 22:28:31 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-z[37933]: 2024-05-17T22:28:31.257+0000 7fef208d0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-17T22:28:31.652 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 17 22:28:31 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-z[37933]: 2024-05-17T22:28:31.388+0000 7fef208d0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-17T22:28:31.652 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 17 22:28:31 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-z[37933]: 2024-05-17T22:28:31.484+0000 7fef208d0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-17T22:28:31.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:31 smithi012 ceph-mon[31632]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:28:31.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:31 smithi012 ceph-mon[31632]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:31.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:31 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:31.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:31 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:31.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:31 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:28:31.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:31 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:31.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:31 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:28:31.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:31 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:31.911 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 17 22:28:31 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-z[37933]: 2024-05-17T22:28:31.817+0000 7fef208d0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-17T22:28:31.911 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 17 22:28:31 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-z[37933]: 2024-05-17T22:28:31.909+0000 7fef208d0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-17T22:28:31.912 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:31 smithi151 ceph-mon[36101]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:28:31.912 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:31 smithi151 ceph-mon[36101]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:31.912 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:31 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:31.912 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:31 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:31.912 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:31 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:28:31.912 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:31 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:31.912 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:31 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:28:31.912 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:31 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:32.048 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 17 22:28:31 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-y[37451]: 2024-05-17T22:28:31.864+0000 7f97b560a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-17T22:28:32.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:31 smithi096 ceph-mon[36065]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:28:32.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:31 smithi096 ceph-mon[36065]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:32.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:31 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:32.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:31 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:32.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:31 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:28:32.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:31 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:32.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:31 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:28:32.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:31 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:32.500 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 17 22:28:32 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-y[37451]: 2024-05-17T22:28:32.083+0000 7f97b560a200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-17T22:28:32.798 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 17 22:28:32 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-y[37451]: 2024-05-17T22:28:32.498+0000 7f97b560a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-17T22:28:32.798 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 17 22:28:32 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-y[37451]: 2024-05-17T22:28:32.594+0000 7f97b560a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-17T22:28:32.924 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 17 22:28:32 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-z[37933]: 2024-05-17T22:28:32.514+0000 7fef208d0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-17T22:28:32.924 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 17 22:28:32 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-z[37933]: 2024-05-17T22:28:32.731+0000 7fef208d0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-17T22:28:33.298 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 17 22:28:32 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-y[37451]: 2024-05-17T22:28:32.960+0000 7f97b560a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-17T22:28:33.642 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 17 22:28:33 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-y[37451]: 2024-05-17T22:28:33.309+0000 7f97b560a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-17T22:28:33.642 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 17 22:28:33 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-y[37451]: 2024-05-17T22:28:33.400+0000 7f97b560a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-17T22:28:33.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:33.663+0000 7fb3dffff700 1 -- 172.21.15.12:0/3904897833 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mgrmap(e 14) ==== 99937+0+0 (secure 0 0 0) 0x7fb3d0017b90 con 0x7fb3e8104dc0 2024-05-17T22:28:33.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:33 smithi151 ceph-mon[36101]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:33.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:33 smithi151 ceph-mon[36101]: Standby manager daemon z started 2024-05-17T22:28:33.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:33 smithi151 ceph-mon[36101]: from='mgr.? 172.21.15.151:0/847934657' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-17T22:28:33.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:33 smithi151 ceph-mon[36101]: from='mgr.? 172.21.15.151:0/847934657' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-17T22:28:33.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:33 smithi151 ceph-mon[36101]: from='mgr.? 172.21.15.151:0/847934657' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-17T22:28:33.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:33 smithi151 ceph-mon[36101]: from='mgr.? 172.21.15.151:0/847934657' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-17T22:28:34.042 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:33 smithi012 ceph-mon[31632]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:34.042 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:33 smithi012 ceph-mon[31632]: Standby manager daemon z started 2024-05-17T22:28:34.042 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:33 smithi012 ceph-mon[31632]: from='mgr.? 172.21.15.151:0/847934657' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-17T22:28:34.043 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:33 smithi012 ceph-mon[31632]: from='mgr.? 172.21.15.151:0/847934657' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-17T22:28:34.043 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:33 smithi012 ceph-mon[31632]: from='mgr.? 172.21.15.151:0/847934657' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-17T22:28:34.043 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:33 smithi012 ceph-mon[31632]: from='mgr.? 172.21.15.151:0/847934657' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-17T22:28:34.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:33 smithi096 ceph-mon[36065]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:34.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:33 smithi096 ceph-mon[36065]: Standby manager daemon z started 2024-05-17T22:28:34.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:33 smithi096 ceph-mon[36065]: from='mgr.? 172.21.15.151:0/847934657' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-17T22:28:34.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:33 smithi096 ceph-mon[36065]: from='mgr.? 172.21.15.151:0/847934657' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-17T22:28:34.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:33 smithi096 ceph-mon[36065]: from='mgr.? 172.21.15.151:0/847934657' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-17T22:28:34.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:33 smithi096 ceph-mon[36065]: from='mgr.? 172.21.15.151:0/847934657' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-17T22:28:34.048 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 17 22:28:33 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-y[37451]: 2024-05-17T22:28:33.640+0000 7f97b560a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-17T22:28:34.048 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 17 22:28:33 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-y[37451]: 2024-05-17T22:28:33.777+0000 7f97b560a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-17T22:28:34.049 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 17 22:28:33 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-y[37451]: 2024-05-17T22:28:33.872+0000 7f97b560a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-17T22:28:34.548 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 17 22:28:34 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-y[37451]: 2024-05-17T22:28:34.191+0000 7f97b560a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-17T22:28:34.548 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 17 22:28:34 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-y[37451]: 2024-05-17T22:28:34.283+0000 7f97b560a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-17T22:28:34.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:34 smithi151 ceph-mon[36101]: mgrmap e14: x(active, since 79s), standbys: z 2024-05-17T22:28:34.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:34 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-17T22:28:34.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:34 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/1453704319' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d42798f-dc5f-4e66-bce2-ec33dc789e4d"}]: dispatch 2024-05-17T22:28:34.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:34 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/1453704319' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d42798f-dc5f-4e66-bce2-ec33dc789e4d"}]': finished 2024-05-17T22:28:34.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:34 smithi151 ceph-mon[36101]: osdmap e5: 1 total, 0 up, 1 in 2024-05-17T22:28:34.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:34 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T22:28:34.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:34 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/857314910' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T22:28:35.049 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 17 22:28:34 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-y[37451]: 2024-05-17T22:28:34.885+0000 7f97b560a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-17T22:28:35.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:34 smithi096 ceph-mon[36065]: mgrmap e14: x(active, since 79s), standbys: z 2024-05-17T22:28:35.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:34 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-17T22:28:35.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:34 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/1453704319' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d42798f-dc5f-4e66-bce2-ec33dc789e4d"}]: dispatch 2024-05-17T22:28:35.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:34 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/1453704319' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d42798f-dc5f-4e66-bce2-ec33dc789e4d"}]': finished 2024-05-17T22:28:35.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:34 smithi096 ceph-mon[36065]: osdmap e5: 1 total, 0 up, 1 in 2024-05-17T22:28:35.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:34 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T22:28:35.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:34 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/857314910' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T22:28:35.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:34 smithi012 ceph-mon[31632]: mgrmap e14: x(active, since 79s), standbys: z 2024-05-17T22:28:35.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:34 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-17T22:28:35.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:34 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/1453704319' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d42798f-dc5f-4e66-bce2-ec33dc789e4d"}]: dispatch 2024-05-17T22:28:35.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:34 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/1453704319' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d42798f-dc5f-4e66-bce2-ec33dc789e4d"}]': finished 2024-05-17T22:28:35.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:34 smithi012 ceph-mon[31632]: osdmap e5: 1 total, 0 up, 1 in 2024-05-17T22:28:35.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:34 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T22:28:35.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:34 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/857314910' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T22:28:35.548 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 17 22:28:35 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-y[37451]: 2024-05-17T22:28:35.099+0000 7f97b560a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-17T22:28:35.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:35.716+0000 7fb3dffff700 1 -- 172.21.15.12:0/3904897833 <== mon.0 v2:172.21.15.12:3300/0 8 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7fb3d0017670 con 0x7fb3e8104dc0 2024-05-17T22:28:35.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:35 smithi151 ceph-mon[36101]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:35.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:35 smithi151 ceph-mon[36101]: Standby manager daemon y started 2024-05-17T22:28:35.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:35 smithi151 ceph-mon[36101]: from='mgr.? 172.21.15.96:0/949416364' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-17T22:28:35.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:35 smithi151 ceph-mon[36101]: from='mgr.? 172.21.15.96:0/949416364' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-17T22:28:35.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:35 smithi151 ceph-mon[36101]: from='mgr.? 172.21.15.96:0/949416364' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-17T22:28:35.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:35 smithi151 ceph-mon[36101]: from='mgr.? 172.21.15.96:0/949416364' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-17T22:28:36.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:35 smithi096 ceph-mon[36065]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:36.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:35 smithi096 ceph-mon[36065]: Standby manager daemon y started 2024-05-17T22:28:36.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:35 smithi096 ceph-mon[36065]: from='mgr.? 172.21.15.96:0/949416364' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-17T22:28:36.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:35 smithi096 ceph-mon[36065]: from='mgr.? 172.21.15.96:0/949416364' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-17T22:28:36.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:35 smithi096 ceph-mon[36065]: from='mgr.? 172.21.15.96:0/949416364' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-17T22:28:36.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:35 smithi096 ceph-mon[36065]: from='mgr.? 172.21.15.96:0/949416364' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-17T22:28:36.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:35 smithi012 ceph-mon[31632]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:36.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:35 smithi012 ceph-mon[31632]: Standby manager daemon y started 2024-05-17T22:28:36.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:35 smithi012 ceph-mon[31632]: from='mgr.? 172.21.15.96:0/949416364' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-17T22:28:36.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:35 smithi012 ceph-mon[31632]: from='mgr.? 172.21.15.96:0/949416364' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-17T22:28:36.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:35 smithi012 ceph-mon[31632]: from='mgr.? 172.21.15.96:0/949416364' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-17T22:28:36.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:35 smithi012 ceph-mon[31632]: from='mgr.? 172.21.15.96:0/949416364' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-17T22:28:36.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:36 smithi151 ceph-mon[36101]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-05-17T22:28:36.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:36 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-17T22:28:37.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:36 smithi096 ceph-mon[36065]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-05-17T22:28:37.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:36 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-17T22:28:37.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:36 smithi012 ceph-mon[31632]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-05-17T22:28:37.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:36 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-17T22:28:37.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:37 smithi151 ceph-mon[36101]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:38.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:37 smithi096 ceph-mon[36065]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:38.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:37 smithi012 ceph-mon[31632]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:39.808 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:39 smithi012 ceph-mon[31632]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:39.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:39 smithi151 ceph-mon[36101]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:40.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:39 smithi096 ceph-mon[36065]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:41.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:41 smithi151 ceph-mon[36101]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:41.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:41 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-17T22:28:41.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:41 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:42.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:41 smithi096 ceph-mon[36065]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:42.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:41 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-17T22:28:42.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:41 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:42.067 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:41 smithi012 ceph-mon[31632]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:42.067 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:41 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-17T22:28:42.067 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:41 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:42.788 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:42 smithi012 ceph-mon[31632]: Deploying daemon osd.0 on smithi012 2024-05-17T22:28:42.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:42 smithi151 ceph-mon[36101]: Deploying daemon osd.0 on smithi012 2024-05-17T22:28:43.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:42 smithi096 ceph-mon[36065]: Deploying daemon osd.0 on smithi012 2024-05-17T22:28:44.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:43 smithi096 ceph-mon[36065]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:44.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:43 smithi012 ceph-mon[31632]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:44.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:43 smithi151 ceph-mon[36101]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:46.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:45 smithi096 ceph-mon[36065]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:46.093 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:45 smithi012 ceph-mon[31632]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:46.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:45 smithi151 ceph-mon[36101]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:47.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:47 smithi012 ceph-mon[31632]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:47.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:47 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:47.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:47 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:47.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:47 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:28:47.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:47 smithi151 ceph-mon[36101]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:47.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:47 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:47.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:47 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:47.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:47 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:28:48.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:47 smithi096 ceph-mon[36065]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:48.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:47 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:48.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:47 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:48.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:47 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:28:49.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:49 smithi012 ceph-mon[31632]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:49.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:49 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:49.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:49 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:49.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:49 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:49.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:49 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:28:49.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:49 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:49.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:49 smithi151 ceph-mon[36101]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:49.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:49 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:49.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:49 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:49.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:49 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:49.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:49 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:28:49.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:49 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:50.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:49 smithi096 ceph-mon[36065]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:50.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:49 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:50.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:49 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:50.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:49 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:50.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:49 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:28:50.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:49 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:50.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:50.075+0000 7fb3dffff700 1 -- 172.21.15.12:0/3904897833 <== mgr.14152 v2:172.21.15.12:6800/1388514047 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb3e8064740 con 0x7fb3d403e750 2024-05-17T22:28:50.079 INFO:teuthology.orchestra.run.smithi012.stdout:Created osd(s) 0 on host 'smithi012' 2024-05-17T22:28:50.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:50.078+0000 7fb3ecc92700 1 -- 172.21.15.12:0/3904897833 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb3d403e750 msgr2=0x7fb3d4040c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:28:50.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:50.078+0000 7fb3ecc92700 1 --2- 172.21.15.12:0/3904897833 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb3d403e750 0x7fb3d4040c60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb3d800f5e0 tx=0x7fb3d800b040 comp rx=0 tx=0).stop 2024-05-17T22:28:50.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:50.078+0000 7fb3ecc92700 1 -- 172.21.15.12:0/3904897833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3e8104dc0 msgr2=0x7fb3e819a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:28:50.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:50.078+0000 7fb3ecc92700 1 --2- 172.21.15.12:0/3904897833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3e8104dc0 0x7fb3e819a7a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb3d00098e0 tx=0x7fb3d0009580 comp rx=0 tx=0).stop 2024-05-17T22:28:50.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:50.078+0000 7fb3ecc92700 1 -- 172.21.15.12:0/3904897833 shutdown_connections 2024-05-17T22:28:50.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:50.078+0000 7fb3ecc92700 1 --2- 172.21.15.12:0/3904897833 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb3d403e750 0x7fb3d4040c60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:28:50.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:50.078+0000 7fb3ecc92700 1 --2- 172.21.15.12:0/3904897833 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3e8105760 0x7fb3e81a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:28:50.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:50.078+0000 7fb3ecc92700 1 --2- 172.21.15.12:0/3904897833 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3e8103c90 0x7fb3e819a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:28:50.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:50.078+0000 7fb3ecc92700 1 --2- 172.21.15.12:0/3904897833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3e8104dc0 0x7fb3e819a7a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:28:50.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:50.078+0000 7fb3ecc92700 1 -- 172.21.15.12:0/3904897833 >> 172.21.15.12:0/3904897833 conn(0x7fb3e80ff4c0 msgr2=0x7fb3e8107ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:28:50.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:50.078+0000 7fb3ecc92700 1 -- 172.21.15.12:0/3904897833 shutdown_connections 2024-05-17T22:28:50.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:28:50.078+0000 7fb3ecc92700 1 -- 172.21.15.12:0/3904897833 wait complete. 2024-05-17T22:28:50.716 DEBUG:teuthology.orchestra.run.smithi012:osd.0> sudo journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@osd.0.service 2024-05-17T22:28:50.718 INFO:tasks.cephadm:Deploying osd.1 on smithi096 with /dev/vg_nvme/lv_4... 2024-05-17T22:28:50.719 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-17T22:28:50.957 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.b/config 2024-05-17T22:28:51.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:51 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:51.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:51 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:51.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:51 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:28:51.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:51 smithi012 ceph-mon[31632]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:51.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:51 smithi012 ceph-mon[31632]: from='osd.0 [v2:172.21.15.12:6802/2015605593,v1:172.21.15.12:6803/2015605593]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-17T22:28:51.372 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:51 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:51.372 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:51 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:51.372 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:51 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:28:51.372 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:51 smithi096 ceph-mon[36065]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:51.372 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:51 smithi096 ceph-mon[36065]: from='osd.0 [v2:172.21.15.12:6802/2015605593,v1:172.21.15.12:6803/2015605593]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-17T22:28:51.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:51 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:51.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:51 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:51.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:51 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:28:51.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:51 smithi151 ceph-mon[36101]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:51.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:51 smithi151 ceph-mon[36101]: from='osd.0 [v2:172.21.15.12:6802/2015605593,v1:172.21.15.12:6803/2015605593]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-17T22:28:52.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:52 smithi012 ceph-mon[31632]: from='osd.0 [v2:172.21.15.12:6802/2015605593,v1:172.21.15.12:6803/2015605593]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-17T22:28:52.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:52 smithi012 ceph-mon[31632]: osdmap e6: 1 total, 0 up, 1 in 2024-05-17T22:28:52.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:52 smithi012 ceph-mon[31632]: from='osd.0 [v2:172.21.15.12:6802/2015605593,v1:172.21.15.12:6803/2015605593]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-05-17T22:28:52.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:52 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T22:28:52.092 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 17 22:28:52 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-0[45860]: 2024-05-17T22:28:52.074+0000 7fb4d44e5700 -1 osd.0 0 waiting for initial osdmap 2024-05-17T22:28:52.092 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 17 22:28:52 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-0[45860]: 2024-05-17T22:28:52.077+0000 7fb4d029a700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T22:28:52.283 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:52 smithi096 ceph-mon[36065]: from='osd.0 [v2:172.21.15.12:6802/2015605593,v1:172.21.15.12:6803/2015605593]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-17T22:28:52.283 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:52 smithi096 ceph-mon[36065]: osdmap e6: 1 total, 0 up, 1 in 2024-05-17T22:28:52.283 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:52 smithi096 ceph-mon[36065]: from='osd.0 [v2:172.21.15.12:6802/2015605593,v1:172.21.15.12:6803/2015605593]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-05-17T22:28:52.283 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:52 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T22:28:52.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:52 smithi151 ceph-mon[36101]: from='osd.0 [v2:172.21.15.12:6802/2015605593,v1:172.21.15.12:6803/2015605593]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-17T22:28:52.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:52 smithi151 ceph-mon[36101]: osdmap e6: 1 total, 0 up, 1 in 2024-05-17T22:28:52.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:52 smithi151 ceph-mon[36101]: from='osd.0 [v2:172.21.15.12:6802/2015605593,v1:172.21.15.12:6803/2015605593]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-05-17T22:28:52.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:52 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T22:28:52.830 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-17T22:28:52.831 INFO:teuthology.orchestra.run.smithi096.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-17T22:28:52.831 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10+0 records in 2024-05-17T22:28:52.831 INFO:teuthology.orchestra.run.smithi096.stderr:10+0 records out 2024-05-17T22:28:52.831 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0123937 s, 846 MB/s 2024-05-17T22:28:52.831 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping successful for: 2024-05-17T22:28:53.165 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:53 smithi096 ceph-mon[36065]: from='osd.0 [v2:172.21.15.12:6802/2015605593,v1:172.21.15.12:6803/2015605593]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-05-17T22:28:53.165 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:53 smithi096 ceph-mon[36065]: osdmap e7: 1 total, 0 up, 1 in 2024-05-17T22:28:53.165 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:53 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T22:28:53.165 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:53 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T22:28:53.165 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:53 smithi096 ceph-mon[36065]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:53.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:53 smithi012 ceph-mon[31632]: from='osd.0 [v2:172.21.15.12:6802/2015605593,v1:172.21.15.12:6803/2015605593]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-05-17T22:28:53.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:53 smithi012 ceph-mon[31632]: osdmap e7: 1 total, 0 up, 1 in 2024-05-17T22:28:53.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:53 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T22:28:53.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:53 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T22:28:53.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:53 smithi012 ceph-mon[31632]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:53.441 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi096:vg_nvme/lv_4 2024-05-17T22:28:53.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:53 smithi151 ceph-mon[36101]: from='osd.0 [v2:172.21.15.12:6802/2015605593,v1:172.21.15.12:6803/2015605593]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-05-17T22:28:53.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:53 smithi151 ceph-mon[36101]: osdmap e7: 1 total, 0 up, 1 in 2024-05-17T22:28:53.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:53 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T22:28:53.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:53 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T22:28:53.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:53 smithi151 ceph-mon[36101]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T22:28:53.678 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.b/config 2024-05-17T22:28:54.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:54 smithi096 ceph-mon[36065]: purged_snaps scrub starts 2024-05-17T22:28:54.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:54 smithi096 ceph-mon[36065]: purged_snaps scrub ok 2024-05-17T22:28:54.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:54 smithi096 ceph-mon[36065]: osd.0 [v2:172.21.15.12:6802/2015605593,v1:172.21.15.12:6803/2015605593] boot 2024-05-17T22:28:54.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:54 smithi096 ceph-mon[36065]: osdmap e8: 1 total, 1 up, 1 in 2024-05-17T22:28:54.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:54 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T22:28:54.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:54 smithi096 ceph-mon[36065]: Detected new or changed devices on smithi012 2024-05-17T22:28:54.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:54 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:54.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:54 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:54.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:54 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:28:54.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:54 smithi096 ceph-mon[36065]: Adjusting osd_memory_target on smithi012 to 17093M 2024-05-17T22:28:54.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:54 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:54.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:54 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:54.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:54 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:28:54.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:54 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:54.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:54 smithi012 ceph-mon[31632]: purged_snaps scrub starts 2024-05-17T22:28:54.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:54 smithi012 ceph-mon[31632]: purged_snaps scrub ok 2024-05-17T22:28:54.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:54 smithi012 ceph-mon[31632]: osd.0 [v2:172.21.15.12:6802/2015605593,v1:172.21.15.12:6803/2015605593] boot 2024-05-17T22:28:54.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:54 smithi012 ceph-mon[31632]: osdmap e8: 1 total, 1 up, 1 in 2024-05-17T22:28:54.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:54 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T22:28:54.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:54 smithi012 ceph-mon[31632]: Detected new or changed devices on smithi012 2024-05-17T22:28:54.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:54 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:54.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:54 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:54.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:54 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:28:54.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:54 smithi012 ceph-mon[31632]: Adjusting osd_memory_target on smithi012 to 17093M 2024-05-17T22:28:54.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:54 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:54.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:54 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:54.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:54 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:28:54.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:54 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:54.392 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:54 smithi151 ceph-mon[36101]: purged_snaps scrub starts 2024-05-17T22:28:54.392 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:54 smithi151 ceph-mon[36101]: purged_snaps scrub ok 2024-05-17T22:28:54.392 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:54 smithi151 ceph-mon[36101]: osd.0 [v2:172.21.15.12:6802/2015605593,v1:172.21.15.12:6803/2015605593] boot 2024-05-17T22:28:54.392 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:54 smithi151 ceph-mon[36101]: osdmap e8: 1 total, 1 up, 1 in 2024-05-17T22:28:54.392 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:54 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T22:28:54.392 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:54 smithi151 ceph-mon[36101]: Detected new or changed devices on smithi012 2024-05-17T22:28:54.392 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:54 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:54.392 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:54 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:54.392 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:54 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:28:54.392 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:54 smithi151 ceph-mon[36101]: Adjusting osd_memory_target on smithi012 to 17093M 2024-05-17T22:28:54.392 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:54 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:54.392 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:54 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:54.392 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:54 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:28:54.392 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:54 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:28:55.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:28:55.251+0000 7f2040880700 1 -- 172.21.15.96:0/3255438718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f203c103dd0 msgr2=0x7f203c10a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:28:55.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:28:55.251+0000 7f2040880700 1 --2- 172.21.15.96:0/3255438718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f203c103dd0 0x7f203c10a680 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f203400b0a0 tx=0x7f203400b3b0 comp rx=0 tx=0).stop 2024-05-17T22:28:55.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:28:55.252+0000 7f2040880700 1 -- 172.21.15.96:0/3255438718 shutdown_connections 2024-05-17T22:28:55.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:28:55.252+0000 7f2040880700 1 --2- 172.21.15.96:0/3255438718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f203c103dd0 0x7f203c10a680 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:28:55.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:28:55.252+0000 7f2040880700 1 --2- 172.21.15.96:0/3255438718 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f203c102300 0x7f203c102710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:28:55.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:28:55.252+0000 7f2040880700 1 --2- 172.21.15.96:0/3255438718 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f203c103430 0x7f203c103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:28:55.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:28:55.252+0000 7f2040880700 1 -- 172.21.15.96:0/3255438718 >> 172.21.15.96:0/3255438718 conn(0x7f203c0fdb50 msgr2=0x7f203c0fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:28:55.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:28:55.252+0000 7f2040880700 1 -- 172.21.15.96:0/3255438718 shutdown_connections 2024-05-17T22:28:55.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:28:55.252+0000 7f2040880700 1 -- 172.21.15.96:0/3255438718 wait complete. 2024-05-17T22:28:55.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:28:55.252+0000 7f2040880700 1 Processor -- start 2024-05-17T22:28:55.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:28:55.253+0000 7f2040880700 1 -- start start 2024-05-17T22:28:55.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:28:55.253+0000 7f2040880700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f203c102300 0x7f203c19a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:28:55.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:28:55.253+0000 7f2040880700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f203c103430 0x7f203c19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:28:55.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:28:55.253+0000 7f2040880700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f203c103dd0 0x7f203c1a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:28:55.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:28:55.253+0000 7f2040880700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f203c10f750 con 0x7f203c103430 2024-05-17T22:28:55.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:28:55.253+0000 7f2040880700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f203c10f5d0 con 0x7f203c102300 2024-05-17T22:28:55.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:28:55.253+0000 7f2040880700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f203c10f8d0 con 0x7f203c103dd0 2024-05-17T22:28:55.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:28:55.253+0000 7f203a59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f203c102300 0x7f203c19a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:28:55.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:28:55.253+0000 7f203a59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f203c102300 0x7f203c19a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60746/0 (socket says 172.21.15.96:60746) 2024-05-17T22:28:55.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:28:55.253+0000 7f203a59c700 1 -- 172.21.15.96:0/4057381030 learned_addr learned my addr 172.21.15.96:0/4057381030 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-05-17T22:28:55.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:28:55.253+0000 7f203ad9d700 1 --2- 172.21.15.96:0/4057381030 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f203c103dd0 0x7f203c1a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:28:55.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:28:55.253+0000 7f2031d9b700 1 --2- 172.21.15.96:0/4057381030 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f203c103430 0x7f203c19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:28:55.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:28:55.253+0000 7f203a59c700 1 -- 172.21.15.96:0/4057381030 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f203c103dd0 msgr2=0x7f203c1a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:28:55.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:28:55.253+0000 7f203a59c700 1 --2- 172.21.15.96:0/4057381030 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f203c103dd0 0x7f203c1a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:28:55.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:28:55.253+0000 7f203a59c700 1 -- 172.21.15.96:0/4057381030 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f203c103430 msgr2=0x7f203c19a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:28:55.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:28:55.253+0000 7f203a59c700 1 --2- 172.21.15.96:0/4057381030 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f203c103430 0x7f203c19a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:28:55.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:28:55.253+0000 7f203a59c700 1 -- 172.21.15.96:0/4057381030 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f203c1a1d70 con 0x7f203c102300 2024-05-17T22:28:55.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:28:55.253+0000 7f203ad9d700 1 --2- 172.21.15.96:0/4057381030 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f203c103dd0 0x7f203c1a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:28:55.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:28:55.253+0000 7f203a59c700 1 --2- 172.21.15.96:0/4057381030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f203c102300 0x7f203c19a210 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f202c00e970 tx=0x7f202c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:28:55.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:28:55.254+0000 7f2033fff700 1 -- 172.21.15.96:0/4057381030 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202c00cbd0 con 0x7f203c102300 2024-05-17T22:28:55.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:28:55.254+0000 7f2033fff700 1 -- 172.21.15.96:0/4057381030 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f202c00cd70 con 0x7f203c102300 2024-05-17T22:28:55.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:28:55.254+0000 7f2040880700 1 -- 172.21.15.96:0/4057381030 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f203c1a2060 con 0x7f203c102300 2024-05-17T22:28:55.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:28:55.254+0000 7f2033fff700 1 -- 172.21.15.96:0/4057381030 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202c005680 con 0x7f203c102300 2024-05-17T22:28:55.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:28:55.254+0000 7f2040880700 1 -- 172.21.15.96:0/4057381030 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f203c1a25a0 con 0x7f203c102300 2024-05-17T22:28:55.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:28:55.255+0000 7f2040880700 1 -- 172.21.15.96:0/4057381030 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f203c102710 con 0x7f203c102300 2024-05-17T22:28:55.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:28:55.257+0000 7f2033fff700 1 -- 172.21.15.96:0/4057381030 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f202c004750 con 0x7f203c102300 2024-05-17T22:28:55.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:28:55.257+0000 7f2033fff700 1 --2- 172.21.15.96:0/4057381030 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f20280b27f0 0x7f20280b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:28:55.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:28:55.258+0000 7f2033fff700 1 -- 172.21.15.96:0/4057381030 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f202c0e0c00 con 0x7f203c102300 2024-05-17T22:28:55.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:28:55.258+0000 7f2031d9b700 1 --2- 172.21.15.96:0/4057381030 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f20280b27f0 0x7f20280b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:28:55.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:28:55.258+0000 7f2031d9b700 1 --2- 172.21.15.96:0/4057381030 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f20280b27f0 0x7f20280b4d00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2024005950 tx=0x7f20240058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:28:55.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:28:55.261+0000 7f2033fff700 1 -- 172.21.15.96:0/4057381030 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f202c054df0 con 0x7f203c102300 2024-05-17T22:28:55.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:55 smithi096 ceph-mon[36065]: osdmap e9: 1 total, 1 up, 1 in 2024-05-17T22:28:55.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:55 smithi096 ceph-mon[36065]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T22:28:55.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:28:55.473+0000 7f2040880700 1 -- 172.21.15.96:0/4057381030 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f203c064740 con 0x7f20280b27f0 2024-05-17T22:28:55.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:55 smithi151 ceph-mon[36101]: osdmap e9: 1 total, 1 up, 1 in 2024-05-17T22:28:55.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:55 smithi151 ceph-mon[36101]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T22:28:55.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:55 smithi012 ceph-mon[31632]: osdmap e9: 1 total, 1 up, 1 in 2024-05-17T22:28:55.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:55 smithi012 ceph-mon[31632]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T22:28:56.377 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:56 smithi096 ceph-mon[36065]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:28:56.377 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:56 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T22:28:56.377 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:56 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T22:28:56.377 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:56 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:56.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:56 smithi151 ceph-mon[36101]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:28:56.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:56 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T22:28:56.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:56 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T22:28:56.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:56 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:56.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:56 smithi012 ceph-mon[31632]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:28:56.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:56 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T22:28:56.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:56 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T22:28:56.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:56 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:28:57.225 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:57 smithi096 ceph-mon[36065]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T22:28:57.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:57 smithi151 ceph-mon[36101]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T22:28:57.492 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:57 smithi012 ceph-mon[31632]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T22:28:58.440 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:58 smithi096 ceph-mon[36065]: from='client.? 172.21.15.96:0/3627650180' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58afecc7-119b-4c16-a401-640fb77d4714"}]: dispatch 2024-05-17T22:28:58.440 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:58 smithi096 ceph-mon[36065]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58afecc7-119b-4c16-a401-640fb77d4714"}]: dispatch 2024-05-17T22:28:58.440 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:58 smithi096 ceph-mon[36065]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "58afecc7-119b-4c16-a401-640fb77d4714"}]': finished 2024-05-17T22:28:58.440 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:58 smithi096 ceph-mon[36065]: osdmap e10: 2 total, 1 up, 2 in 2024-05-17T22:28:58.440 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:58 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T22:28:58.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:58 smithi151 ceph-mon[36101]: from='client.? 172.21.15.96:0/3627650180' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58afecc7-119b-4c16-a401-640fb77d4714"}]: dispatch 2024-05-17T22:28:58.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:58 smithi151 ceph-mon[36101]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58afecc7-119b-4c16-a401-640fb77d4714"}]: dispatch 2024-05-17T22:28:58.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:58 smithi151 ceph-mon[36101]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "58afecc7-119b-4c16-a401-640fb77d4714"}]': finished 2024-05-17T22:28:58.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:58 smithi151 ceph-mon[36101]: osdmap e10: 2 total, 1 up, 2 in 2024-05-17T22:28:58.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:58 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T22:28:58.505 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:58 smithi012 ceph-mon[31632]: from='client.? 172.21.15.96:0/3627650180' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58afecc7-119b-4c16-a401-640fb77d4714"}]: dispatch 2024-05-17T22:28:58.505 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:58 smithi012 ceph-mon[31632]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58afecc7-119b-4c16-a401-640fb77d4714"}]: dispatch 2024-05-17T22:28:58.505 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:58 smithi012 ceph-mon[31632]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "58afecc7-119b-4c16-a401-640fb77d4714"}]': finished 2024-05-17T22:28:58.505 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:58 smithi012 ceph-mon[31632]: osdmap e10: 2 total, 1 up, 2 in 2024-05-17T22:28:58.505 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:58 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T22:28:59.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:59 smithi151 ceph-mon[36101]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T22:28:59.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:28:59 smithi151 ceph-mon[36101]: from='client.? 172.21.15.96:0/2860975536' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T22:28:59.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:59 smithi096 ceph-mon[36065]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T22:28:59.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:28:59 smithi096 ceph-mon[36065]: from='client.? 172.21.15.96:0/2860975536' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T22:28:59.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:59 smithi012 ceph-mon[31632]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T22:28:59.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:28:59 smithi012 ceph-mon[31632]: from='client.? 172.21.15.96:0/2860975536' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T22:29:01.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:01 smithi151 ceph-mon[36101]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T22:29:01.787 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:01 smithi096 ceph-mon[36065]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T22:29:01.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:01 smithi012 ceph-mon[31632]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T22:29:03.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:03 smithi096 ceph-mon[36065]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T22:29:03.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:03 smithi151 ceph-mon[36101]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T22:29:03.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:03 smithi012 ceph-mon[31632]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T22:29:05.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:05 smithi151 ceph-mon[36101]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T22:29:05.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:05 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-17T22:29:05.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:05 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:29:05.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:05 smithi096 ceph-mon[36065]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T22:29:05.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:05 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-17T22:29:05.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:05 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:29:05.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:05 smithi012 ceph-mon[31632]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T22:29:05.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:05 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-17T22:29:05.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:05 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:29:06.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:06 smithi151 ceph-mon[36101]: Deploying daemon osd.1 on smithi096 2024-05-17T22:29:06.788 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:06 smithi096 ceph-mon[36065]: Deploying daemon osd.1 on smithi096 2024-05-17T22:29:06.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:06 smithi012 ceph-mon[31632]: Deploying daemon osd.1 on smithi096 2024-05-17T22:29:07.721 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:07 smithi096 ceph-mon[36065]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T22:29:07.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:07 smithi151 ceph-mon[36101]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T22:29:07.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:07 smithi012 ceph-mon[31632]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T22:29:09.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:09 smithi151 ceph-mon[36101]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T22:29:09.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:09 smithi096 ceph-mon[36065]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T22:29:09.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:09 smithi012 ceph-mon[31632]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T22:29:11.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:11 smithi151 ceph-mon[36101]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T22:29:11.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:11 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:11.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:11 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:11.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:11 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:29:12.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:11 smithi096 ceph-mon[36065]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T22:29:12.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:11 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:12.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:11 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:12.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:11 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:29:12.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:11 smithi012 ceph-mon[31632]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T22:29:12.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:11 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:12.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:11 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:12.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:11 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:29:13.680 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:13 smithi096 ceph-mon[36065]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T22:29:13.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:29:13.741+0000 7f2033fff700 1 -- 172.21.15.96:0/4057381030 <== mgr.14152 v2:172.21.15.12:6800/1388514047 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f203c064740 con 0x7f20280b27f0 2024-05-17T22:29:13.743 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 1 on host 'smithi096' 2024-05-17T22:29:13.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:29:13.744+0000 7f2040880700 1 -- 172.21.15.96:0/4057381030 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f20280b27f0 msgr2=0x7f20280b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:29:13.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:29:13.744+0000 7f2040880700 1 --2- 172.21.15.96:0/4057381030 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f20280b27f0 0x7f20280b4d00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2024005950 tx=0x7f20240058e0 comp rx=0 tx=0).stop 2024-05-17T22:29:13.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:29:13.744+0000 7f2040880700 1 -- 172.21.15.96:0/4057381030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f203c102300 msgr2=0x7f203c19a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:29:13.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:29:13.744+0000 7f2040880700 1 --2- 172.21.15.96:0/4057381030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f203c102300 0x7f203c19a210 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f202c00e970 tx=0x7f202c00ec80 comp rx=0 tx=0).stop 2024-05-17T22:29:13.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:29:13.744+0000 7f2040880700 1 -- 172.21.15.96:0/4057381030 shutdown_connections 2024-05-17T22:29:13.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:29:13.744+0000 7f2040880700 1 --2- 172.21.15.96:0/4057381030 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f20280b27f0 0x7f20280b4d00 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:29:13.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:29:13.744+0000 7f2040880700 1 --2- 172.21.15.96:0/4057381030 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f203c103dd0 0x7f203c1a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:29:13.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:29:13.744+0000 7f2040880700 1 --2- 172.21.15.96:0/4057381030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f203c102300 0x7f203c19a210 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:29:13.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:29:13.744+0000 7f2040880700 1 --2- 172.21.15.96:0/4057381030 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f203c103430 0x7f203c19a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:29:13.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:29:13.744+0000 7f2040880700 1 -- 172.21.15.96:0/4057381030 >> 172.21.15.96:0/4057381030 conn(0x7f203c0fdb50 msgr2=0x7f203c106660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:29:13.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:29:13.744+0000 7f2040880700 1 -- 172.21.15.96:0/4057381030 shutdown_connections 2024-05-17T22:29:13.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:29:13.744+0000 7f2040880700 1 -- 172.21.15.96:0/4057381030 wait complete. 2024-05-17T22:29:13.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:13 smithi151 ceph-mon[36101]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T22:29:14.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:13 smithi012 ceph-mon[31632]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T22:29:14.460 DEBUG:teuthology.orchestra.run.smithi096:osd.1> sudo journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@osd.1.service 2024-05-17T22:29:14.462 INFO:tasks.cephadm:Deploying osd.2 on smithi151 with /dev/vg_nvme/lv_4... 2024-05-17T22:29:14.463 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-17T22:29:14.708 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T22:29:14.712 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 17 22:29:14 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-1[40656]: 2024-05-17T22:29:14.533+0000 7f9377cd07c0 -1 osd.1 0 log_to_monitors true 2024-05-17T22:29:14.830 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:14 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:14.830 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:14 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:14.830 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:14 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:14.830 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:14 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:14.830 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:14 smithi151 ceph-mon[36101]: from='osd.1 [v2:172.21.15.96:6800/1238316358,v1:172.21.15.96:6801/1238316358]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-17T22:29:14.830 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:14 smithi151 ceph-mon[36101]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-17T22:29:15.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:14 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:15.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:14 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:15.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:14 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:15.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:14 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:15.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:14 smithi096 ceph-mon[36065]: from='osd.1 [v2:172.21.15.96:6800/1238316358,v1:172.21.15.96:6801/1238316358]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-17T22:29:15.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:14 smithi096 ceph-mon[36065]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-17T22:29:15.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:14 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:15.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:14 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:15.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:14 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:15.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:14 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:15.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:14 smithi012 ceph-mon[31632]: from='osd.1 [v2:172.21.15.96:6800/1238316358,v1:172.21.15.96:6801/1238316358]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-17T22:29:15.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:14 smithi012 ceph-mon[31632]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-17T22:29:16.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:15 smithi096 ceph-mon[36065]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T22:29:16.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:15 smithi096 ceph-mon[36065]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-17T22:29:16.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:15 smithi096 ceph-mon[36065]: from='osd.1 [v2:172.21.15.96:6800/1238316358,v1:172.21.15.96:6801/1238316358]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-17T22:29:16.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:15 smithi096 ceph-mon[36065]: osdmap e11: 2 total, 1 up, 2 in 2024-05-17T22:29:16.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:15 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T22:29:16.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:15 smithi096 ceph-mon[36065]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-17T22:29:16.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:15 smithi012 ceph-mon[31632]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T22:29:16.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:15 smithi012 ceph-mon[31632]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-17T22:29:16.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:15 smithi012 ceph-mon[31632]: from='osd.1 [v2:172.21.15.96:6800/1238316358,v1:172.21.15.96:6801/1238316358]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-17T22:29:16.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:15 smithi012 ceph-mon[31632]: osdmap e11: 2 total, 1 up, 2 in 2024-05-17T22:29:16.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:15 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T22:29:16.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:15 smithi012 ceph-mon[31632]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-17T22:29:16.109 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:15 smithi151 ceph-mon[36101]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T22:29:16.109 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:15 smithi151 ceph-mon[36101]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-17T22:29:16.109 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:15 smithi151 ceph-mon[36101]: from='osd.1 [v2:172.21.15.96:6800/1238316358,v1:172.21.15.96:6801/1238316358]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-17T22:29:16.109 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:15 smithi151 ceph-mon[36101]: osdmap e11: 2 total, 1 up, 2 in 2024-05-17T22:29:16.109 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:15 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T22:29:16.109 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:15 smithi151 ceph-mon[36101]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-17T22:29:16.665 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-17T22:29:16.665 INFO:teuthology.orchestra.run.smithi151.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-17T22:29:16.666 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10+0 records in 2024-05-17T22:29:16.666 INFO:teuthology.orchestra.run.smithi151.stderr:10+0 records out 2024-05-17T22:29:16.666 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126125 s, 831 MB/s 2024-05-17T22:29:16.666 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping successful for: 2024-05-17T22:29:16.798 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 17 22:29:16 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-1[40656]: 2024-05-17T22:29:16.543+0000 7f936db35700 -1 osd.1 0 waiting for initial osdmap 2024-05-17T22:29:16.798 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 17 22:29:16 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-1[40656]: 2024-05-17T22:29:16.548+0000 7f93650e1700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T22:29:17.058 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:16 smithi151 ceph-mon[36101]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-05-17T22:29:17.059 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:16 smithi151 ceph-mon[36101]: osdmap e12: 2 total, 1 up, 2 in 2024-05-17T22:29:17.059 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:16 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T22:29:17.059 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:16 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T22:29:17.059 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:16 smithi151 ceph-mon[36101]: from='osd.1 ' entity='osd.1' 2024-05-17T22:29:17.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:16 smithi012 ceph-mon[31632]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-05-17T22:29:17.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:16 smithi012 ceph-mon[31632]: osdmap e12: 2 total, 1 up, 2 in 2024-05-17T22:29:17.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:16 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T22:29:17.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:16 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T22:29:17.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:16 smithi012 ceph-mon[31632]: from='osd.1 ' entity='osd.1' 2024-05-17T22:29:17.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:16 smithi096 ceph-mon[36065]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-05-17T22:29:17.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:16 smithi096 ceph-mon[36065]: osdmap e12: 2 total, 1 up, 2 in 2024-05-17T22:29:17.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:16 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T22:29:17.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:16 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T22:29:17.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:16 smithi096 ceph-mon[36065]: from='osd.1 ' entity='osd.1' 2024-05-17T22:29:17.361 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi151:vg_nvme/lv_4 2024-05-17T22:29:17.610 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T22:29:17.945 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:17 smithi151 ceph-mon[36101]: purged_snaps scrub starts 2024-05-17T22:29:17.945 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:17 smithi151 ceph-mon[36101]: purged_snaps scrub ok 2024-05-17T22:29:17.945 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:17 smithi151 ceph-mon[36101]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T22:29:17.945 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:17 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T22:29:17.945 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:17 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:17.945 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:17 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:17.946 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:17 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:29:17.946 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:17 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:17.946 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:17 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:29:17.946 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:17 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:29:17.946 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:17 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:17.946 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:17 smithi151 ceph-mon[36101]: osd.1 [v2:172.21.15.96:6800/1238316358,v1:172.21.15.96:6801/1238316358] boot 2024-05-17T22:29:17.946 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:17 smithi151 ceph-mon[36101]: osdmap e13: 2 total, 2 up, 2 in 2024-05-17T22:29:17.946 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:17 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T22:29:17.946 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:17 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:29:17.946 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:17 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:29:17.946 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:17 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:29:17.946 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:17 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:18.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:17 smithi012 ceph-mon[31632]: purged_snaps scrub starts 2024-05-17T22:29:18.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:17 smithi012 ceph-mon[31632]: purged_snaps scrub ok 2024-05-17T22:29:18.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:17 smithi012 ceph-mon[31632]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T22:29:18.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:17 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T22:29:18.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:17 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:18.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:17 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:18.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:17 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:29:18.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:17 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:18.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:17 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:29:18.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:17 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:29:18.093 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:17 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:18.093 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:17 smithi012 ceph-mon[31632]: osd.1 [v2:172.21.15.96:6800/1238316358,v1:172.21.15.96:6801/1238316358] boot 2024-05-17T22:29:18.093 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:17 smithi012 ceph-mon[31632]: osdmap e13: 2 total, 2 up, 2 in 2024-05-17T22:29:18.093 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:17 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T22:29:18.093 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:17 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:29:18.093 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:17 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:29:18.093 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:17 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:29:18.093 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:17 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:18.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:17 smithi096 ceph-mon[36065]: purged_snaps scrub starts 2024-05-17T22:29:18.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:17 smithi096 ceph-mon[36065]: purged_snaps scrub ok 2024-05-17T22:29:18.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:17 smithi096 ceph-mon[36065]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T22:29:18.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:17 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T22:29:18.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:17 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:18.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:17 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:18.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:17 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:29:18.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:17 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:18.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:17 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:29:18.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:17 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:29:18.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:17 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:18.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:17 smithi096 ceph-mon[36065]: osd.1 [v2:172.21.15.96:6800/1238316358,v1:172.21.15.96:6801/1238316358] boot 2024-05-17T22:29:18.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:17 smithi096 ceph-mon[36065]: osdmap e13: 2 total, 2 up, 2 in 2024-05-17T22:29:18.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:17 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T22:29:18.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:17 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:29:18.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:17 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:29:18.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:17 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:29:18.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:17 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:19.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:19.087+0000 7fd27d964700 1 -- 172.21.15.151:0/1990665479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd278103ab0 msgr2=0x7fd278103f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:29:19.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:19.087+0000 7fd27d964700 1 --2- 172.21.15.151:0/1990665479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd278103ab0 0x7fd278103f60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd260009990 tx=0x7fd260009ca0 comp rx=0 tx=0).stop 2024-05-17T22:29:19.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:19.087+0000 7fd27d964700 1 -- 172.21.15.151:0/1990665479 shutdown_connections 2024-05-17T22:29:19.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:19.087+0000 7fd27d964700 1 --2- 172.21.15.151:0/1990665479 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd278104620 0x7fd27810b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:29:19.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:19.087+0000 7fd27d964700 1 --2- 172.21.15.151:0/1990665479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2781030d0 0x7fd2781034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:29:19.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:19.087+0000 7fd27d964700 1 --2- 172.21.15.151:0/1990665479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd278103ab0 0x7fd278103f60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:29:19.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:19.087+0000 7fd27d964700 1 -- 172.21.15.151:0/1990665479 >> 172.21.15.151:0/1990665479 conn(0x7fd2780fdb50 msgr2=0x7fd2780fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:29:19.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:19.087+0000 7fd27d964700 1 -- 172.21.15.151:0/1990665479 shutdown_connections 2024-05-17T22:29:19.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:19.087+0000 7fd27d964700 1 -- 172.21.15.151:0/1990665479 wait complete. 2024-05-17T22:29:19.092 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:18 smithi151 ceph-mon[36101]: Detected new or changed devices on smithi096 2024-05-17T22:29:19.092 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:18 smithi151 ceph-mon[36101]: Adjusting osd_memory_target on smithi096 to 17088M 2024-05-17T22:29:19.092 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:18 smithi151 ceph-mon[36101]: osdmap e14: 2 total, 2 up, 2 in 2024-05-17T22:29:19.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:19.088+0000 7fd27d964700 1 Processor -- start 2024-05-17T22:29:19.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:19.088+0000 7fd27d964700 1 -- start start 2024-05-17T22:29:19.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:19.089+0000 7fd27d964700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2781030d0 0x7fd27819c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:29:19.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:19.089+0000 7fd27d964700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd278103ab0 0x7fd27819ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:29:19.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:19.089+0000 7fd27d964700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd278104620 0x7fd2781a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:29:19.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:19.089+0000 7fd27d964700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd27810e610 con 0x7fd278104620 2024-05-17T22:29:19.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:19.089+0000 7fd27d964700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd27810e490 con 0x7fd2781030d0 2024-05-17T22:29:19.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:19.089+0000 7fd27d964700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd27810e790 con 0x7fd278103ab0 2024-05-17T22:29:19.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:19.089+0000 7fd26e7fc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd278103ab0 0x7fd27819ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:29:19.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:19.089+0000 7fd26e7fc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd278103ab0 0x7fd27819ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:38022/0 (socket says 172.21.15.151:38022) 2024-05-17T22:29:19.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:19.089+0000 7fd26e7fc700 1 -- 172.21.15.151:0/806399838 learned_addr learned my addr 172.21.15.151:0/806399838 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-17T22:29:19.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:19.089+0000 7fd276ffd700 1 --2- 172.21.15.151:0/806399838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2781030d0 0x7fd27819c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:29:19.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:19.089+0000 7fd2777fe700 1 --2- 172.21.15.151:0/806399838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd278104620 0x7fd2781a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:29:19.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:19.089+0000 7fd26e7fc700 1 -- 172.21.15.151:0/806399838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2781030d0 msgr2=0x7fd27819c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:29:19.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:19.089+0000 7fd26e7fc700 1 --2- 172.21.15.151:0/806399838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2781030d0 0x7fd27819c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:29:19.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:19.089+0000 7fd26e7fc700 1 -- 172.21.15.151:0/806399838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd278104620 msgr2=0x7fd2781a39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:29:19.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:19.089+0000 7fd26e7fc700 1 --2- 172.21.15.151:0/806399838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd278104620 0x7fd2781a39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:29:19.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:19.089+0000 7fd26e7fc700 1 -- 172.21.15.151:0/806399838 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2781a4050 con 0x7fd278103ab0 2024-05-17T22:29:19.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:19.089+0000 7fd2777fe700 1 --2- 172.21.15.151:0/806399838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd278104620 0x7fd2781a39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:29:19.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:19.089+0000 7fd26e7fc700 1 --2- 172.21.15.151:0/806399838 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd278103ab0 0x7fd27819ca30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd260006010 tx=0x7fd2600047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:29:19.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:19.089+0000 7fd27c962700 1 -- 172.21.15.151:0/806399838 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd260027070 con 0x7fd278103ab0 2024-05-17T22:29:19.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:19.090+0000 7fd27d964700 1 -- 172.21.15.151:0/806399838 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2781a42e0 con 0x7fd278103ab0 2024-05-17T22:29:19.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:19.090+0000 7fd27c962700 1 -- 172.21.15.151:0/806399838 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2600040c0 con 0x7fd278103ab0 2024-05-17T22:29:19.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:19.090+0000 7fd27c962700 1 -- 172.21.15.151:0/806399838 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd260022d90 con 0x7fd278103ab0 2024-05-17T22:29:19.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:19.090+0000 7fd27d964700 1 -- 172.21.15.151:0/806399838 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2781a4730 con 0x7fd278103ab0 2024-05-17T22:29:19.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:19.091+0000 7fd27d964700 1 -- 172.21.15.151:0/806399838 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2781034e0 con 0x7fd278103ab0 2024-05-17T22:29:19.096 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:18 smithi012 ceph-mon[31632]: Detected new or changed devices on smithi096 2024-05-17T22:29:19.096 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:18 smithi012 ceph-mon[31632]: Adjusting osd_memory_target on smithi096 to 17088M 2024-05-17T22:29:19.096 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:18 smithi012 ceph-mon[31632]: osdmap e14: 2 total, 2 up, 2 in 2024-05-17T22:29:19.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:19.092+0000 7fd27c962700 1 -- 172.21.15.151:0/806399838 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7fd26002a070 con 0x7fd278103ab0 2024-05-17T22:29:19.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:19.093+0000 7fd27c962700 1 --2- 172.21.15.151:0/806399838 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd2640b27f0 0x7fd2640b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:29:19.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:19.093+0000 7fd27c962700 1 -- 172.21.15.151:0/806399838 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7fd2600e1d30 con 0x7fd278103ab0 2024-05-17T22:29:19.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:19.093+0000 7fd276ffd700 1 --2- 172.21.15.151:0/806399838 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd2640b27f0 0x7fd2640b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:29:19.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:19.094+0000 7fd276ffd700 1 --2- 172.21.15.151:0/806399838 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd2640b27f0 0x7fd2640b4d00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd268005950 tx=0x7fd26800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:29:19.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:19.096+0000 7fd27c962700 1 -- 172.21.15.151:0/806399838 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd260031080 con 0x7fd278103ab0 2024-05-17T22:29:19.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:18 smithi096 ceph-mon[36065]: Detected new or changed devices on smithi096 2024-05-17T22:29:19.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:18 smithi096 ceph-mon[36065]: Adjusting osd_memory_target on smithi096 to 17088M 2024-05-17T22:29:19.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:18 smithi096 ceph-mon[36065]: osdmap e14: 2 total, 2 up, 2 in 2024-05-17T22:29:19.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:19.311+0000 7fd27d964700 1 -- 172.21.15.151:0/806399838 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fd2780ffc50 con 0x7fd2640b27f0 2024-05-17T22:29:20.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:19 smithi012 ceph-mon[31632]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-17T22:29:20.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:19 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T22:29:20.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:19 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T22:29:20.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:19 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:29:20.119 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:19 smithi151 ceph-mon[36101]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-17T22:29:20.119 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:19 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T22:29:20.120 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:19 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T22:29:20.120 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:19 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:29:20.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:19 smithi096 ceph-mon[36065]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-17T22:29:20.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:19 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T22:29:20.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:19 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T22:29:20.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:19 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:29:21.036 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:20 smithi151 ceph-mon[36101]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:29:21.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:20 smithi012 ceph-mon[31632]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:29:21.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:20 smithi096 ceph-mon[36065]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:29:22.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:21 smithi012 ceph-mon[31632]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-17T22:29:22.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:21 smithi151 ceph-mon[36101]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-17T22:29:22.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:21 smithi096 ceph-mon[36065]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-17T22:29:23.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:22 smithi012 ceph-mon[31632]: from='client.? 172.21.15.151:0/3608565928' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4bef4049-482f-49fb-82fb-38052400eb2a"}]: dispatch 2024-05-17T22:29:23.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:22 smithi012 ceph-mon[31632]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4bef4049-482f-49fb-82fb-38052400eb2a"}]: dispatch 2024-05-17T22:29:23.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:22 smithi012 ceph-mon[31632]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4bef4049-482f-49fb-82fb-38052400eb2a"}]': finished 2024-05-17T22:29:23.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:22 smithi012 ceph-mon[31632]: osdmap e15: 3 total, 2 up, 3 in 2024-05-17T22:29:23.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:22 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T22:29:23.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:22 smithi012 ceph-mon[31632]: from='client.? 172.21.15.151:0/3459980986' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T22:29:23.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:22 smithi151 ceph-mon[36101]: from='client.? 172.21.15.151:0/3608565928' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4bef4049-482f-49fb-82fb-38052400eb2a"}]: dispatch 2024-05-17T22:29:23.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:22 smithi151 ceph-mon[36101]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4bef4049-482f-49fb-82fb-38052400eb2a"}]: dispatch 2024-05-17T22:29:23.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:22 smithi151 ceph-mon[36101]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4bef4049-482f-49fb-82fb-38052400eb2a"}]': finished 2024-05-17T22:29:23.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:22 smithi151 ceph-mon[36101]: osdmap e15: 3 total, 2 up, 3 in 2024-05-17T22:29:23.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:22 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T22:29:23.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:22 smithi151 ceph-mon[36101]: from='client.? 172.21.15.151:0/3459980986' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T22:29:23.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:22 smithi096 ceph-mon[36065]: from='client.? 172.21.15.151:0/3608565928' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4bef4049-482f-49fb-82fb-38052400eb2a"}]: dispatch 2024-05-17T22:29:23.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:22 smithi096 ceph-mon[36065]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4bef4049-482f-49fb-82fb-38052400eb2a"}]: dispatch 2024-05-17T22:29:23.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:22 smithi096 ceph-mon[36065]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4bef4049-482f-49fb-82fb-38052400eb2a"}]': finished 2024-05-17T22:29:23.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:22 smithi096 ceph-mon[36065]: osdmap e15: 3 total, 2 up, 3 in 2024-05-17T22:29:23.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:22 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T22:29:23.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:22 smithi096 ceph-mon[36065]: from='client.? 172.21.15.151:0/3459980986' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T22:29:24.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:23 smithi012 ceph-mon[31632]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-17T22:29:24.142 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:23 smithi151 ceph-mon[36101]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-17T22:29:24.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:23 smithi096 ceph-mon[36065]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-17T22:29:25.930 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:25 smithi151 ceph-mon[36101]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-17T22:29:26.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:25 smithi012 ceph-mon[31632]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-17T22:29:26.100 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:25 smithi096 ceph-mon[36065]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-17T22:29:28.065 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:27 smithi151 ceph-mon[36101]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-17T22:29:28.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:27 smithi012 ceph-mon[31632]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-17T22:29:28.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:27 smithi096 ceph-mon[36065]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-17T22:29:29.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:29 smithi151 ceph-mon[36101]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-17T22:29:29.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:29 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-17T22:29:29.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:29 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:29:30.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:29 smithi012 ceph-mon[31632]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-17T22:29:30.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:29 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-17T22:29:30.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:29 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:29:30.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:29 smithi096 ceph-mon[36065]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-17T22:29:30.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:29 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-17T22:29:30.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:29 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:29:30.942 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:30 smithi151 ceph-mon[36101]: Deploying daemon osd.2 on smithi151 2024-05-17T22:29:31.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:30 smithi012 ceph-mon[31632]: Deploying daemon osd.2 on smithi151 2024-05-17T22:29:31.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:30 smithi096 ceph-mon[36065]: Deploying daemon osd.2 on smithi151 2024-05-17T22:29:32.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:31 smithi012 ceph-mon[31632]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-17T22:29:32.213 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:31 smithi151 ceph-mon[36101]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-17T22:29:32.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:31 smithi096 ceph-mon[36065]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-17T22:29:33.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:33 smithi151 ceph-mon[36101]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-17T22:29:34.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:33 smithi096 ceph-mon[36065]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-17T22:29:34.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:33 smithi012 ceph-mon[31632]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-17T22:29:35.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:35 smithi151 ceph-mon[36101]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-17T22:29:35.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:35 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:35.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:35 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:35.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:35 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:29:35.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:35 smithi012 ceph-mon[31632]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-17T22:29:35.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:35 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:35.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:35 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:35.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:35 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:29:36.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:35 smithi096 ceph-mon[36065]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-17T22:29:36.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:35 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:36.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:35 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:36.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:35 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:29:37.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:37 smithi096 ceph-mon[36065]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-17T22:29:37.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:37 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:37.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:37 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:37.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:37 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:29:37.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:37 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:29:37.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:37 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:37.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:37 smithi012 ceph-mon[31632]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-17T22:29:37.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:37 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:37.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:37 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:37.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:37 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:29:37.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:37 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:29:37.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:37 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:37.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:37 smithi151 ceph-mon[36101]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-17T22:29:37.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:37 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:37.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:37 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:37.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:37 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:29:37.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:37 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:29:37.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:37 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:38.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:38.330+0000 7fd27c962700 1 -- 172.21.15.151:0/806399838 <== mgr.14152 v2:172.21.15.12:6800/1388514047 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd2780ffc50 con 0x7fd2640b27f0 2024-05-17T22:29:38.333 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 2 on host 'smithi151' 2024-05-17T22:29:38.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:38.333+0000 7fd27d964700 1 -- 172.21.15.151:0/806399838 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd2640b27f0 msgr2=0x7fd2640b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:29:38.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:38.333+0000 7fd27d964700 1 --2- 172.21.15.151:0/806399838 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd2640b27f0 0x7fd2640b4d00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd268005950 tx=0x7fd26800a300 comp rx=0 tx=0).stop 2024-05-17T22:29:38.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:38.333+0000 7fd27d964700 1 -- 172.21.15.151:0/806399838 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd278103ab0 msgr2=0x7fd27819ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:29:38.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:38.333+0000 7fd27d964700 1 --2- 172.21.15.151:0/806399838 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd278103ab0 0x7fd27819ca30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd260006010 tx=0x7fd2600047b0 comp rx=0 tx=0).stop 2024-05-17T22:29:38.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:38.333+0000 7fd27d964700 1 -- 172.21.15.151:0/806399838 shutdown_connections 2024-05-17T22:29:38.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:38.333+0000 7fd27d964700 1 --2- 172.21.15.151:0/806399838 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd2640b27f0 0x7fd2640b4d00 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:29:38.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:38.333+0000 7fd27d964700 1 --2- 172.21.15.151:0/806399838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd278104620 0x7fd2781a39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:29:38.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:38.333+0000 7fd27d964700 1 --2- 172.21.15.151:0/806399838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2781030d0 0x7fd27819c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:29:38.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:38.333+0000 7fd27d964700 1 --2- 172.21.15.151:0/806399838 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd278103ab0 0x7fd27819ca30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:29:38.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:38.333+0000 7fd27d964700 1 -- 172.21.15.151:0/806399838 >> 172.21.15.151:0/806399838 conn(0x7fd2780fdb50 msgr2=0x7fd2780ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:29:38.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:38.333+0000 7fd27d964700 1 -- 172.21.15.151:0/806399838 shutdown_connections 2024-05-17T22:29:38.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:29:38.333+0000 7fd27d964700 1 -- 172.21.15.151:0/806399838 wait complete. 2024-05-17T22:29:38.963 DEBUG:teuthology.orchestra.run.smithi151:osd.2> sudo journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@osd.2.service 2024-05-17T22:29:38.965 INFO:tasks.cephadm:Deploying osd.3 on smithi012 with /dev/vg_nvme/lv_3... 2024-05-17T22:29:38.965 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-17T22:29:39.226 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:29:39.575 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:39 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:39.575 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:39 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:39.575 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:39 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:29:39.575 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:39 smithi012 ceph-mon[31632]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-17T22:29:39.575 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:39 smithi012 ceph-mon[31632]: from='osd.2 [v2:172.21.15.151:6800/148729815,v1:172.21.15.151:6801/148729815]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-17T22:29:39.575 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:39 smithi012 ceph-mon[31632]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-17T22:29:39.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:39 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:39.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:39 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:39.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:39 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:29:39.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:39 smithi151 ceph-mon[36101]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-17T22:29:39.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:39 smithi151 ceph-mon[36101]: from='osd.2 [v2:172.21.15.151:6800/148729815,v1:172.21.15.151:6801/148729815]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-17T22:29:39.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:39 smithi151 ceph-mon[36101]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-17T22:29:39.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:39 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:39.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:39 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:39.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:39 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:29:39.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:39 smithi096 ceph-mon[36065]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-17T22:29:39.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:39 smithi096 ceph-mon[36065]: from='osd.2 [v2:172.21.15.151:6800/148729815,v1:172.21.15.151:6801/148729815]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-17T22:29:39.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:39 smithi096 ceph-mon[36065]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-17T22:29:40.458 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:40 smithi151 ceph-mon[36101]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-17T22:29:40.458 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:40 smithi151 ceph-mon[36101]: osdmap e16: 3 total, 2 up, 3 in 2024-05-17T22:29:40.458 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:40 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T22:29:40.458 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:40 smithi151 ceph-mon[36101]: from='osd.2 [v2:172.21.15.151:6800/148729815,v1:172.21.15.151:6801/148729815]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-17T22:29:40.458 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:40 smithi151 ceph-mon[36101]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-17T22:29:40.580 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:40 smithi012 ceph-mon[31632]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-17T22:29:40.580 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:40 smithi012 ceph-mon[31632]: osdmap e16: 3 total, 2 up, 3 in 2024-05-17T22:29:40.581 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:40 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T22:29:40.581 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:40 smithi012 ceph-mon[31632]: from='osd.2 [v2:172.21.15.151:6800/148729815,v1:172.21.15.151:6801/148729815]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-17T22:29:40.581 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:40 smithi012 ceph-mon[31632]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-17T22:29:40.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:40 smithi096 ceph-mon[36065]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-17T22:29:40.828 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:40 smithi096 ceph-mon[36065]: osdmap e16: 3 total, 2 up, 3 in 2024-05-17T22:29:40.828 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:40 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T22:29:40.828 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:40 smithi096 ceph-mon[36065]: from='osd.2 [v2:172.21.15.151:6800/148729815,v1:172.21.15.151:6801/148729815]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-17T22:29:40.828 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:40 smithi096 ceph-mon[36065]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-17T22:29:41.166 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-17T22:29:41.166 INFO:teuthology.orchestra.run.smithi012.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-17T22:29:41.166 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10+0 records in 2024-05-17T22:29:41.166 INFO:teuthology.orchestra.run.smithi012.stderr:10+0 records out 2024-05-17T22:29:41.166 INFO:teuthology.orchestra.run.smithi012.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.012603 s, 832 MB/s 2024-05-17T22:29:41.167 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping successful for: 2024-05-17T22:29:41.342 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 17 22:29:41 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-2[41218]: 2024-05-17T22:29:41.064+0000 7fa940ce2700 -1 osd.2 0 waiting for initial osdmap 2024-05-17T22:29:41.342 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 17 22:29:41 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-2[41218]: 2024-05-17T22:29:41.069+0000 7fa938a8f700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T22:29:41.480 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:41 smithi012 ceph-mon[31632]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-05-17T22:29:41.480 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:41 smithi012 ceph-mon[31632]: osdmap e17: 3 total, 2 up, 3 in 2024-05-17T22:29:41.480 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:41 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T22:29:41.480 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:41 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T22:29:41.480 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:41 smithi012 ceph-mon[31632]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-17T22:29:41.481 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:41 smithi012 ceph-mon[31632]: from='osd.2 ' entity='osd.2' 2024-05-17T22:29:41.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:41 smithi151 ceph-mon[36101]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-05-17T22:29:41.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:41 smithi151 ceph-mon[36101]: osdmap e17: 3 total, 2 up, 3 in 2024-05-17T22:29:41.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:41 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T22:29:41.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:41 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T22:29:41.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:41 smithi151 ceph-mon[36101]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-17T22:29:41.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:41 smithi151 ceph-mon[36101]: from='osd.2 ' entity='osd.2' 2024-05-17T22:29:41.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:41 smithi096 ceph-mon[36065]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-05-17T22:29:41.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:41 smithi096 ceph-mon[36065]: osdmap e17: 3 total, 2 up, 3 in 2024-05-17T22:29:41.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:41 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T22:29:41.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:41 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T22:29:41.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:41 smithi096 ceph-mon[36065]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-17T22:29:41.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:41 smithi096 ceph-mon[36065]: from='osd.2 ' entity='osd.2' 2024-05-17T22:29:41.868 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi012:vg_nvme/lv_3 2024-05-17T22:29:42.125 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:29:42.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:42 smithi151 ceph-mon[36101]: purged_snaps scrub starts 2024-05-17T22:29:42.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:42 smithi151 ceph-mon[36101]: purged_snaps scrub ok 2024-05-17T22:29:42.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:42 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T22:29:42.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:42 smithi151 ceph-mon[36101]: osd.2 [v2:172.21.15.151:6800/148729815,v1:172.21.15.151:6801/148729815] boot 2024-05-17T22:29:42.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:42 smithi151 ceph-mon[36101]: osdmap e18: 3 total, 3 up, 3 in 2024-05-17T22:29:42.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:42 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T22:29:42.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:42 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:42.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:42 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:42.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:42 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:29:42.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:42 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:42.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:42 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:29:42.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:42 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:29:42.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:42 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:42.499 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:42 smithi012 ceph-mon[31632]: purged_snaps scrub starts 2024-05-17T22:29:42.500 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:42 smithi012 ceph-mon[31632]: purged_snaps scrub ok 2024-05-17T22:29:42.500 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:42 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T22:29:42.500 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:42 smithi012 ceph-mon[31632]: osd.2 [v2:172.21.15.151:6800/148729815,v1:172.21.15.151:6801/148729815] boot 2024-05-17T22:29:42.500 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:42 smithi012 ceph-mon[31632]: osdmap e18: 3 total, 3 up, 3 in 2024-05-17T22:29:42.500 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:42 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T22:29:42.500 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:42 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:42.500 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:42 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:42.500 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:42 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:29:42.500 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:42 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:42.500 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:42 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:29:42.500 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:42 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:29:42.500 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:42 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:42.761 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:42 smithi096 ceph-mon[36065]: purged_snaps scrub starts 2024-05-17T22:29:42.761 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:42 smithi096 ceph-mon[36065]: purged_snaps scrub ok 2024-05-17T22:29:42.761 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:42 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T22:29:42.761 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:42 smithi096 ceph-mon[36065]: osd.2 [v2:172.21.15.151:6800/148729815,v1:172.21.15.151:6801/148729815] boot 2024-05-17T22:29:42.761 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:42 smithi096 ceph-mon[36065]: osdmap e18: 3 total, 3 up, 3 in 2024-05-17T22:29:42.761 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:42 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T22:29:42.761 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:42 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:42.761 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:42 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:42.762 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:42 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:29:42.762 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:42 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:42.762 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:42 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:29:42.762 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:42 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:29:42.762 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:42 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:43.679 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:43 smithi012 ceph-mon[31632]: Detected new or changed devices on smithi151 2024-05-17T22:29:43.679 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:43 smithi012 ceph-mon[31632]: Adjusting osd_memory_target on smithi151 to 17088M 2024-05-17T22:29:43.679 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:43 smithi012 ceph-mon[31632]: pgmap v82: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-17T22:29:43.679 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:43 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-17T22:29:43.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:43 smithi151 ceph-mon[36101]: Detected new or changed devices on smithi151 2024-05-17T22:29:43.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:43 smithi151 ceph-mon[36101]: Adjusting osd_memory_target on smithi151 to 17088M 2024-05-17T22:29:43.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:43 smithi151 ceph-mon[36101]: pgmap v82: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-17T22:29:43.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:43 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-17T22:29:43.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:43 smithi096 ceph-mon[36065]: Detected new or changed devices on smithi151 2024-05-17T22:29:43.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:43 smithi096 ceph-mon[36065]: Adjusting osd_memory_target on smithi151 to 17088M 2024-05-17T22:29:43.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:43 smithi096 ceph-mon[36065]: pgmap v82: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-17T22:29:43.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:43 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-17T22:29:43.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:29:43.904+0000 7f34e9038700 1 -- 172.21.15.12:0/2958257387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34e4104dc0 msgr2=0x7f34e41051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:29:43.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:29:43.904+0000 7f34e9038700 1 --2- 172.21.15.12:0/2958257387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34e4104dc0 0x7f34e41051d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f34d80099c0 tx=0x7f34d8009cd0 comp rx=0 tx=0).stop 2024-05-17T22:29:43.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:29:43.904+0000 7f34e9038700 1 -- 172.21.15.12:0/2958257387 shutdown_connections 2024-05-17T22:29:43.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:29:43.904+0000 7f34e9038700 1 --2- 172.21.15.12:0/2958257387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f34e4105760 0x7f34e410c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:29:43.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:29:43.904+0000 7f34e9038700 1 --2- 172.21.15.12:0/2958257387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34e4103c90 0x7f34e41040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:29:43.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:29:43.904+0000 7f34e9038700 1 --2- 172.21.15.12:0/2958257387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34e4104dc0 0x7f34e41051d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:29:43.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:29:43.904+0000 7f34e9038700 1 -- 172.21.15.12:0/2958257387 >> 172.21.15.12:0/2958257387 conn(0x7f34e40ff4c0 msgr2=0x7f34e4101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:29:43.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:29:43.904+0000 7f34e9038700 1 -- 172.21.15.12:0/2958257387 shutdown_connections 2024-05-17T22:29:43.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:29:43.904+0000 7f34e9038700 1 -- 172.21.15.12:0/2958257387 wait complete. 2024-05-17T22:29:43.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:29:43.905+0000 7f34e9038700 1 Processor -- start 2024-05-17T22:29:43.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:29:43.905+0000 7f34e9038700 1 -- start start 2024-05-17T22:29:43.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:29:43.905+0000 7f34e9038700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34e4103c90 0x7f34e419a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:29:43.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:29:43.905+0000 7f34e9038700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f34e4104dc0 0x7f34e419a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:29:43.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:29:43.905+0000 7f34e9038700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34e4105760 0x7f34e41a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:29:43.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:29:43.905+0000 7f34e9038700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f34e410f700 con 0x7f34e4103c90 2024-05-17T22:29:43.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:29:43.905+0000 7f34e9038700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f34e410f580 con 0x7f34e4105760 2024-05-17T22:29:43.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:29:43.905+0000 7f34e9038700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f34e410f880 con 0x7f34e4104dc0 2024-05-17T22:29:43.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:29:43.906+0000 7f34e2d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34e4103c90 0x7f34e419a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:29:43.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:29:43.906+0000 7f34e259c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f34e4104dc0 0x7f34e419a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:29:43.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:29:43.906+0000 7f34e2d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34e4103c90 0x7f34e419a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52542/0 (socket says 172.21.15.12:52542) 2024-05-17T22:29:43.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:29:43.906+0000 7f34e2d9d700 1 -- 172.21.15.12:0/2137024384 learned_addr learned my addr 172.21.15.12:0/2137024384 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:29:43.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:29:43.906+0000 7f34e359e700 1 --2- 172.21.15.12:0/2137024384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34e4105760 0x7f34e41a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:29:43.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:29:43.906+0000 7f34e359e700 1 -- 172.21.15.12:0/2137024384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f34e4104dc0 msgr2=0x7f34e419a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:29:43.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:29:43.906+0000 7f34e359e700 1 --2- 172.21.15.12:0/2137024384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f34e4104dc0 0x7f34e419a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:29:43.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:29:43.906+0000 7f34e359e700 1 -- 172.21.15.12:0/2137024384 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34e4103c90 msgr2=0x7f34e419a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:29:43.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:29:43.906+0000 7f34e359e700 1 --2- 172.21.15.12:0/2137024384 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34e4103c90 0x7f34e419a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:29:43.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:29:43.906+0000 7f34e359e700 1 -- 172.21.15.12:0/2137024384 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34e41a1de0 con 0x7f34e4105760 2024-05-17T22:29:43.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:29:43.906+0000 7f34e2d9d700 1 --2- 172.21.15.12:0/2137024384 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34e4103c90 0x7f34e419a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:29:43.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:29:43.906+0000 7f34e359e700 1 --2- 172.21.15.12:0/2137024384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34e4105760 0x7f34e41a1760 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f34dc00efc0 tx=0x7f34dc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:29:43.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:29:43.906+0000 7f34cbfff700 1 -- 172.21.15.12:0/2137024384 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34dc0092e0 con 0x7f34e4105760 2024-05-17T22:29:43.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:29:43.906+0000 7f34cbfff700 1 -- 172.21.15.12:0/2137024384 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34dc004bb0 con 0x7f34e4105760 2024-05-17T22:29:43.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:29:43.906+0000 7f34e9038700 1 -- 172.21.15.12:0/2137024384 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34e4075a60 con 0x7f34e4105760 2024-05-17T22:29:43.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:29:43.907+0000 7f34cbfff700 1 -- 172.21.15.12:0/2137024384 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34dc004020 con 0x7f34e4105760 2024-05-17T22:29:43.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:29:43.907+0000 7f34e9038700 1 -- 172.21.15.12:0/2137024384 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34e4075f50 con 0x7f34e4105760 2024-05-17T22:29:43.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:29:43.908+0000 7f34e9038700 1 -- 172.21.15.12:0/2137024384 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34e41040a0 con 0x7f34e4105760 2024-05-17T22:29:43.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:29:43.909+0000 7f34cbfff700 1 -- 172.21.15.12:0/2137024384 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f34dc01b030 con 0x7f34e4105760 2024-05-17T22:29:43.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:29:43.910+0000 7f34cbfff700 1 --2- 172.21.15.12:0/2137024384 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f34cc0b27f0 0x7f34cc0b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:29:43.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:29:43.910+0000 7f34e2d9d700 1 --2- 172.21.15.12:0/2137024384 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f34cc0b27f0 0x7f34cc0b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:29:43.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:29:43.910+0000 7f34cbfff700 1 -- 172.21.15.12:0/2137024384 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3226+0+0 (secure 0 0 0) 0x7f34dc0e0940 con 0x7f34e4105760 2024-05-17T22:29:43.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:29:43.911+0000 7f34e2d9d700 1 --2- 172.21.15.12:0/2137024384 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f34cc0b27f0 0x7f34cc0b4d00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f34d4005950 tx=0x7f34d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:29:43.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:29:43.914+0000 7f34cbfff700 1 -- 172.21.15.12:0/2137024384 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f34dc022020 con 0x7f34e4105760 2024-05-17T22:29:44.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:29:44.139+0000 7f34e9038700 1 -- 172.21.15.12:0/2137024384 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f34e40776d0 con 0x7f34cc0b27f0 2024-05-17T22:29:44.413 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:44 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-17T22:29:44.413 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:44 smithi012 ceph-mon[31632]: osdmap e19: 3 total, 3 up, 3 in 2024-05-17T22:29:44.413 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:44 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-17T22:29:44.413 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:44 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T22:29:44.413 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:44 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T22:29:44.413 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:44 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:29:44.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:44 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-17T22:29:44.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:44 smithi151 ceph-mon[36101]: osdmap e19: 3 total, 3 up, 3 in 2024-05-17T22:29:44.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:44 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-17T22:29:44.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:44 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T22:29:44.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:44 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T22:29:44.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:44 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:29:44.657 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 17 22:29:44 smithi151 sudo[45250]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-17T22:29:44.657 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 17 22:29:44 smithi151 sudo[45250]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T22:29:44.657 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 17 22:29:44 smithi151 sudo[45250]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T22:29:44.657 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 17 22:29:44 smithi151 sudo[45250]: pam_unix(sudo:session): session closed for user root 2024-05-17T22:29:44.657 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 17 22:29:44 smithi151 sudo[45253]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-17T22:29:44.657 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 17 22:29:44 smithi151 sudo[45253]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T22:29:44.657 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 17 22:29:44 smithi151 sudo[45253]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T22:29:44.657 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 17 22:29:44 smithi151 sudo[45253]: pam_unix(sudo:session): session closed for user root 2024-05-17T22:29:44.694 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:44 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-17T22:29:44.694 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:44 smithi096 ceph-mon[36065]: osdmap e19: 3 total, 3 up, 3 in 2024-05-17T22:29:44.694 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:44 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-17T22:29:44.694 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:44 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T22:29:44.694 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:44 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T22:29:44.694 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:44 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:29:44.694 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:44 smithi096 sudo[44630]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-17T22:29:44.695 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 17 22:29:44 smithi096 sudo[44624]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-17T22:29:44.695 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 17 22:29:44 smithi096 sudo[44624]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T22:29:44.695 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 17 22:29:44 smithi096 sudo[44624]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T22:29:44.695 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 17 22:29:44 smithi096 sudo[44624]: pam_unix(sudo:session): session closed for user root 2024-05-17T22:29:44.695 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 17 22:29:44 smithi096 sudo[44627]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-17T22:29:44.695 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 17 22:29:44 smithi096 sudo[44627]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T22:29:44.695 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 17 22:29:44 smithi096 sudo[44627]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T22:29:44.695 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 17 22:29:44 smithi096 sudo[44627]: pam_unix(sudo:session): session closed for user root 2024-05-17T22:29:44.800 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:44 smithi012 sudo[50592]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-17T22:29:44.801 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:44 smithi012 sudo[50592]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T22:29:44.801 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:44 smithi012 sudo[50592]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T22:29:44.801 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:44 smithi012 sudo[50592]: pam_unix(sudo:session): session closed for user root 2024-05-17T22:29:44.801 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:44 smithi012 sudo[50608]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-17T22:29:44.801 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:44 smithi012 sudo[50608]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T22:29:44.801 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:44 smithi012 sudo[50608]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T22:29:44.801 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:44 smithi012 sudo[50608]: pam_unix(sudo:session): session closed for user root 2024-05-17T22:29:44.801 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 17 22:29:44 smithi012 sudo[50574]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-17T22:29:44.801 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 17 22:29:44 smithi012 sudo[50574]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T22:29:44.801 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 17 22:29:44 smithi012 sudo[50574]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T22:29:44.801 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 17 22:29:44 smithi012 sudo[50574]: pam_unix(sudo:session): session closed for user root 2024-05-17T22:29:44.801 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 17 22:29:44 smithi012 sudo[50577]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-17T22:29:44.802 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 17 22:29:44 smithi012 sudo[50577]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T22:29:44.802 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 17 22:29:44 smithi012 sudo[50577]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T22:29:44.802 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 17 22:29:44 smithi012 sudo[50577]: pam_unix(sudo:session): session closed for user root 2024-05-17T22:29:44.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:44 smithi151 sudo[45256]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-17T22:29:44.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:44 smithi151 sudo[45256]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T22:29:44.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:44 smithi151 sudo[45256]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T22:29:44.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:44 smithi151 sudo[45256]: pam_unix(sudo:session): session closed for user root 2024-05-17T22:29:44.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:44 smithi151 sudo[45259]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-17T22:29:44.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:44 smithi151 sudo[45259]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T22:29:44.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:44 smithi151 sudo[45259]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T22:29:44.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:44 smithi151 sudo[45259]: pam_unix(sudo:session): session closed for user root 2024-05-17T22:29:45.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:44 smithi096 sudo[44630]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T22:29:45.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:44 smithi096 sudo[44630]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T22:29:45.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:44 smithi096 sudo[44630]: pam_unix(sudo:session): session closed for user root 2024-05-17T22:29:45.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:44 smithi096 sudo[44633]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-17T22:29:45.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:44 smithi096 sudo[44633]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T22:29:45.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:44 smithi096 sudo[44633]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T22:29:45.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:44 smithi096 sudo[44633]: pam_unix(sudo:session): session closed for user root 2024-05-17T22:29:45.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:29:45.371+0000 7f34cbfff700 1 -- 172.21.15.12:0/2137024384 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f34dc012050 con 0x7f34e4105760 2024-05-17T22:29:45.499 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:45 smithi012 ceph-mon[31632]: from='client.24185 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:29:45.500 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:45 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-17T22:29:45.500 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:45 smithi012 ceph-mon[31632]: osdmap e20: 3 total, 3 up, 3 in 2024-05-17T22:29:45.500 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:45 smithi012 ceph-mon[31632]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-17T22:29:45.500 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:45 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:45.500 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:45 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:45.500 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:45 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:45.500 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:45 smithi012 ceph-mon[31632]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-17T22:29:45.500 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:45 smithi012 ceph-mon[31632]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-17T22:29:45.500 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:45 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T22:29:45.500 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:45 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T22:29:45.500 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:45 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T22:29:45.500 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:45 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T22:29:45.500 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:45 smithi012 ceph-mon[31632]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-17T22:29:45.501 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:45 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T22:29:45.501 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:45 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T22:29:45.501 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:45 smithi012 ceph-mon[31632]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-17T22:29:45.501 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:45 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T22:29:45.501 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:45 smithi012 ceph-mon[31632]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-17T22:29:45.501 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:45 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T22:29:45.501 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:45 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T22:29:45.501 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:45 smithi012 ceph-mon[31632]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-17T22:29:45.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:45 smithi151 ceph-mon[36101]: from='client.24185 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:29:45.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:45 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-17T22:29:45.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:45 smithi151 ceph-mon[36101]: osdmap e20: 3 total, 3 up, 3 in 2024-05-17T22:29:45.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:45 smithi151 ceph-mon[36101]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-17T22:29:45.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:45 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:45.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:45 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:45.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:45 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:45.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:45 smithi151 ceph-mon[36101]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-17T22:29:45.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:45 smithi151 ceph-mon[36101]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-17T22:29:45.742 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:45 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T22:29:45.742 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:45 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T22:29:45.742 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:45 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T22:29:45.742 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:45 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T22:29:45.742 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:45 smithi151 ceph-mon[36101]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-17T22:29:45.742 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:45 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T22:29:45.742 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:45 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T22:29:45.742 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:45 smithi151 ceph-mon[36101]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-17T22:29:45.742 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:45 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T22:29:45.742 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:45 smithi151 ceph-mon[36101]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-17T22:29:45.742 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:45 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T22:29:45.742 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:45 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T22:29:45.742 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:45 smithi151 ceph-mon[36101]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-17T22:29:45.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:45 smithi096 ceph-mon[36065]: from='client.24185 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:29:45.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:45 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-17T22:29:45.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:45 smithi096 ceph-mon[36065]: osdmap e20: 3 total, 3 up, 3 in 2024-05-17T22:29:45.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:45 smithi096 ceph-mon[36065]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-17T22:29:45.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:45 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:45.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:45 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:45.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:45 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:29:45.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:45 smithi096 ceph-mon[36065]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-17T22:29:45.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:45 smithi096 ceph-mon[36065]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-17T22:29:45.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:45 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T22:29:45.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:45 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T22:29:45.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:45 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T22:29:45.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:45 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T22:29:45.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:45 smithi096 ceph-mon[36065]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-17T22:29:45.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:45 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T22:29:45.800 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:45 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T22:29:45.800 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:45 smithi096 ceph-mon[36065]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-17T22:29:45.800 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:45 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T22:29:45.800 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:45 smithi096 ceph-mon[36065]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-17T22:29:45.800 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:45 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T22:29:45.800 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:45 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T22:29:45.800 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:45 smithi096 ceph-mon[36065]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-17T22:29:46.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:46 smithi151 ceph-mon[36101]: osdmap e21: 3 total, 3 up, 3 in 2024-05-17T22:29:46.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:46 smithi151 ceph-mon[36101]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-17T22:29:46.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:46 smithi096 ceph-mon[36065]: osdmap e21: 3 total, 3 up, 3 in 2024-05-17T22:29:46.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:46 smithi096 ceph-mon[36065]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-17T22:29:46.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:46 smithi012 ceph-mon[31632]: osdmap e21: 3 total, 3 up, 3 in 2024-05-17T22:29:46.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:46 smithi012 ceph-mon[31632]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-17T22:29:47.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:47 smithi151 ceph-mon[36101]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-17T22:29:47.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:47 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/3382722979' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fed8e36-82a2-49e4-82a3-16e6d9de87f5"}]: dispatch 2024-05-17T22:29:47.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:47 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/3382722979' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6fed8e36-82a2-49e4-82a3-16e6d9de87f5"}]': finished 2024-05-17T22:29:47.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:47 smithi151 ceph-mon[36101]: osdmap e22: 4 total, 3 up, 4 in 2024-05-17T22:29:47.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:47 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T22:29:47.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:47 smithi096 ceph-mon[36065]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-17T22:29:47.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:47 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/3382722979' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fed8e36-82a2-49e4-82a3-16e6d9de87f5"}]: dispatch 2024-05-17T22:29:47.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:47 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/3382722979' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6fed8e36-82a2-49e4-82a3-16e6d9de87f5"}]': finished 2024-05-17T22:29:47.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:47 smithi096 ceph-mon[36065]: osdmap e22: 4 total, 3 up, 4 in 2024-05-17T22:29:47.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:47 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T22:29:47.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:47 smithi012 ceph-mon[31632]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-17T22:29:47.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:47 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/3382722979' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fed8e36-82a2-49e4-82a3-16e6d9de87f5"}]: dispatch 2024-05-17T22:29:47.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:47 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/3382722979' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6fed8e36-82a2-49e4-82a3-16e6d9de87f5"}]': finished 2024-05-17T22:29:47.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:47 smithi012 ceph-mon[31632]: osdmap e22: 4 total, 3 up, 4 in 2024-05-17T22:29:47.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:47 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T22:29:48.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:48 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/1268017209' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T22:29:48.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:48 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/1268017209' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T22:29:48.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:48 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/1268017209' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T22:29:49.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:49 smithi151 ceph-mon[36101]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-05-17T22:29:49.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:49 smithi096 ceph-mon[36065]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-05-17T22:29:49.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:49 smithi012 ceph-mon[31632]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-05-17T22:29:51.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:51 smithi151 ceph-mon[36101]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-17T22:29:51.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:51 smithi096 ceph-mon[36065]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-17T22:29:51.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:51 smithi012 ceph-mon[31632]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-17T22:29:53.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:53 smithi151 ceph-mon[36101]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-17T22:29:53.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:53 smithi096 ceph-mon[36065]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-17T22:29:53.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:53 smithi012 ceph-mon[31632]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-17T22:29:54.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:54 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-17T22:29:54.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:54 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:29:54.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:54 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-17T22:29:54.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:54 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:29:54.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:54 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-17T22:29:54.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:54 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:29:55.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:55 smithi151 ceph-mon[36101]: Deploying daemon osd.3 on smithi012 2024-05-17T22:29:55.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:55 smithi151 ceph-mon[36101]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-17T22:29:55.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:55 smithi096 ceph-mon[36065]: Deploying daemon osd.3 on smithi012 2024-05-17T22:29:55.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:55 smithi096 ceph-mon[36065]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-17T22:29:55.801 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:55 smithi012 ceph-mon[31632]: Deploying daemon osd.3 on smithi012 2024-05-17T22:29:55.801 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:55 smithi012 ceph-mon[31632]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-17T22:29:57.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:57 smithi151 ceph-mon[36101]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-17T22:29:57.797 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:57 smithi012 ceph-mon[31632]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-17T22:29:57.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:57 smithi096 ceph-mon[36065]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-17T22:29:59.535 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:29:59 smithi012 ceph-mon[31632]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-17T22:29:59.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:29:59 smithi151 ceph-mon[36101]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-17T22:29:59.797 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:29:59 smithi096 ceph-mon[36065]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-17T22:30:01.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:01.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:01.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:30:01.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:00 smithi151 ceph-mon[36101]: overall HEALTH_OK 2024-05-17T22:30:01.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:00 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:01.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:00 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:01.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:00 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:30:01.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:00 smithi096 ceph-mon[36065]: overall HEALTH_OK 2024-05-17T22:30:01.329 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:00 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:01.329 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:00 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:01.329 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:00 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:30:01.329 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:00 smithi012 ceph-mon[31632]: overall HEALTH_OK 2024-05-17T22:30:02.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:01 smithi151 ceph-mon[36101]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-17T22:30:02.265 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:01 smithi012 ceph-mon[31632]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-17T22:30:02.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:01 smithi096 ceph-mon[36065]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-17T22:30:03.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:30:03.703+0000 7f34cbfff700 1 -- 172.21.15.12:0/2137024384 <== mgr.14152 v2:172.21.15.12:6800/1388514047 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f34e40776d0 con 0x7f34cc0b27f0 2024-05-17T22:30:03.706 INFO:teuthology.orchestra.run.smithi012.stdout:Created osd(s) 3 on host 'smithi012' 2024-05-17T22:30:03.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:30:03.705+0000 7f34e9038700 1 -- 172.21.15.12:0/2137024384 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f34cc0b27f0 msgr2=0x7f34cc0b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:30:03.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:30:03.705+0000 7f34e9038700 1 --2- 172.21.15.12:0/2137024384 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f34cc0b27f0 0x7f34cc0b4d00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f34d4005950 tx=0x7f34d4009260 comp rx=0 tx=0).stop 2024-05-17T22:30:03.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:30:03.705+0000 7f34e9038700 1 -- 172.21.15.12:0/2137024384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34e4105760 msgr2=0x7f34e41a1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:30:03.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:30:03.705+0000 7f34e9038700 1 --2- 172.21.15.12:0/2137024384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34e4105760 0x7f34e41a1760 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f34dc00efc0 tx=0x7f34dc00c3a0 comp rx=0 tx=0).stop 2024-05-17T22:30:03.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:30:03.705+0000 7f34e9038700 1 -- 172.21.15.12:0/2137024384 shutdown_connections 2024-05-17T22:30:03.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:30:03.705+0000 7f34e9038700 1 --2- 172.21.15.12:0/2137024384 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f34cc0b27f0 0x7f34cc0b4d00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:30:03.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:30:03.705+0000 7f34e9038700 1 --2- 172.21.15.12:0/2137024384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34e4105760 0x7f34e41a1760 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:30:03.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:30:03.705+0000 7f34e9038700 1 --2- 172.21.15.12:0/2137024384 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34e4103c90 0x7f34e419a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:30:03.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:30:03.705+0000 7f34e9038700 1 --2- 172.21.15.12:0/2137024384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f34e4104dc0 0x7f34e419a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:30:03.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:30:03.705+0000 7f34e9038700 1 -- 172.21.15.12:0/2137024384 >> 172.21.15.12:0/2137024384 conn(0x7f34e40ff4c0 msgr2=0x7f34e4107ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:30:03.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:30:03.705+0000 7f34e9038700 1 -- 172.21.15.12:0/2137024384 shutdown_connections 2024-05-17T22:30:03.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:30:03.705+0000 7f34e9038700 1 -- 172.21.15.12:0/2137024384 wait complete. 2024-05-17T22:30:03.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:03 smithi151 ceph-mon[36101]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-17T22:30:03.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:03 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:03.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:03 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:03.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:03 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:30:03.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:03 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:30:03.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:03 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:03.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:03 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:03.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:03 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:04.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:03 smithi096 ceph-mon[36065]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-17T22:30:04.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:03 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:04.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:03 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:04.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:03 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:30:04.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:03 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:30:04.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:03 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:04.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:03 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:04.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:03 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:04.084 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:03 smithi012 ceph-mon[31632]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-17T22:30:04.084 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:03 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:04.084 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:03 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:04.084 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:03 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:30:04.085 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:03 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:30:04.085 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:03 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:04.085 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:03 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:04.085 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:03 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:04.454 DEBUG:teuthology.orchestra.run.smithi012:osd.3> sudo journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@osd.3.service 2024-05-17T22:30:04.456 INFO:tasks.cephadm:Deploying osd.4 on smithi096 with /dev/vg_nvme/lv_3... 2024-05-17T22:30:04.457 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-17T22:30:04.699 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.b/config 2024-05-17T22:30:04.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:04 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:30:04.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:04 smithi096 ceph-mon[36065]: from='osd.3 [v2:172.21.15.12:6810/3563309320,v1:172.21.15.12:6811/3563309320]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-17T22:30:04.953 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:04 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:30:04.953 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:04 smithi012 ceph-mon[31632]: from='osd.3 [v2:172.21.15.12:6810/3563309320,v1:172.21.15.12:6811/3563309320]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-17T22:30:04.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:04 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:30:04.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:04 smithi151 ceph-mon[36101]: from='osd.3 [v2:172.21.15.12:6810/3563309320,v1:172.21.15.12:6811/3563309320]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-17T22:30:05.967 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:05 smithi096 ceph-mon[36065]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-17T22:30:05.967 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:05 smithi096 ceph-mon[36065]: from='osd.3 [v2:172.21.15.12:6810/3563309320,v1:172.21.15.12:6811/3563309320]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-17T22:30:05.967 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:05 smithi096 ceph-mon[36065]: osdmap e23: 4 total, 3 up, 4 in 2024-05-17T22:30:05.967 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:05 smithi096 ceph-mon[36065]: from='osd.3 [v2:172.21.15.12:6810/3563309320,v1:172.21.15.12:6811/3563309320]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-05-17T22:30:05.967 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:05 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T22:30:05.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:05 smithi151 ceph-mon[36101]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-17T22:30:05.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:05 smithi151 ceph-mon[36101]: from='osd.3 [v2:172.21.15.12:6810/3563309320,v1:172.21.15.12:6811/3563309320]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-17T22:30:05.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:05 smithi151 ceph-mon[36101]: osdmap e23: 4 total, 3 up, 4 in 2024-05-17T22:30:05.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:05 smithi151 ceph-mon[36101]: from='osd.3 [v2:172.21.15.12:6810/3563309320,v1:172.21.15.12:6811/3563309320]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-05-17T22:30:05.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:05 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T22:30:06.050 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:05 smithi012 ceph-mon[31632]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-17T22:30:06.050 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:05 smithi012 ceph-mon[31632]: from='osd.3 [v2:172.21.15.12:6810/3563309320,v1:172.21.15.12:6811/3563309320]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-17T22:30:06.050 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:05 smithi012 ceph-mon[31632]: osdmap e23: 4 total, 3 up, 4 in 2024-05-17T22:30:06.051 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:05 smithi012 ceph-mon[31632]: from='osd.3 [v2:172.21.15.12:6810/3563309320,v1:172.21.15.12:6811/3563309320]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-05-17T22:30:06.051 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:05 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T22:30:06.511 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-17T22:30:06.511 INFO:teuthology.orchestra.run.smithi096.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-17T22:30:06.511 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10+0 records in 2024-05-17T22:30:06.511 INFO:teuthology.orchestra.run.smithi096.stderr:10+0 records out 2024-05-17T22:30:06.511 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0122858 s, 853 MB/s 2024-05-17T22:30:06.511 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping successful for: 2024-05-17T22:30:06.740 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 17 22:30:06 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-3[52771]: 2024-05-17T22:30:06.480+0000 7f71e9084700 -1 osd.3 0 waiting for initial osdmap 2024-05-17T22:30:06.740 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 17 22:30:06 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-3[52771]: 2024-05-17T22:30:06.487+0000 7f71e0630700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T22:30:07.034 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:06 smithi012 ceph-mon[31632]: from='osd.3 [v2:172.21.15.12:6810/3563309320,v1:172.21.15.12:6811/3563309320]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-05-17T22:30:07.034 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:06 smithi012 ceph-mon[31632]: osdmap e24: 4 total, 3 up, 4 in 2024-05-17T22:30:07.034 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:06 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T22:30:07.034 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:06 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T22:30:07.034 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:06 smithi012 ceph-mon[31632]: from='osd.3 [v2:172.21.15.12:6810/3563309320,v1:172.21.15.12:6811/3563309320]' entity='osd.3' 2024-05-17T22:30:07.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:06 smithi096 ceph-mon[36065]: from='osd.3 [v2:172.21.15.12:6810/3563309320,v1:172.21.15.12:6811/3563309320]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-05-17T22:30:07.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:06 smithi096 ceph-mon[36065]: osdmap e24: 4 total, 3 up, 4 in 2024-05-17T22:30:07.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:06 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T22:30:07.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:06 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T22:30:07.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:06 smithi096 ceph-mon[36065]: from='osd.3 [v2:172.21.15.12:6810/3563309320,v1:172.21.15.12:6811/3563309320]' entity='osd.3' 2024-05-17T22:30:07.199 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi096:vg_nvme/lv_3 2024-05-17T22:30:07.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:06 smithi151 ceph-mon[36101]: from='osd.3 [v2:172.21.15.12:6810/3563309320,v1:172.21.15.12:6811/3563309320]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-05-17T22:30:07.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:06 smithi151 ceph-mon[36101]: osdmap e24: 4 total, 3 up, 4 in 2024-05-17T22:30:07.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:06 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T22:30:07.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:06 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T22:30:07.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:06 smithi151 ceph-mon[36101]: from='osd.3 [v2:172.21.15.12:6810/3563309320,v1:172.21.15.12:6811/3563309320]' entity='osd.3' 2024-05-17T22:30:07.429 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.b/config 2024-05-17T22:30:07.841 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:07 smithi096 ceph-mon[36065]: purged_snaps scrub starts 2024-05-17T22:30:07.841 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:07 smithi096 ceph-mon[36065]: purged_snaps scrub ok 2024-05-17T22:30:07.841 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:07 smithi096 ceph-mon[36065]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-17T22:30:07.841 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:07 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T22:30:07.841 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:07 smithi096 ceph-mon[36065]: osd.3 [v2:172.21.15.12:6810/3563309320,v1:172.21.15.12:6811/3563309320] boot 2024-05-17T22:30:07.841 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:07 smithi096 ceph-mon[36065]: osdmap e25: 4 total, 4 up, 4 in 2024-05-17T22:30:07.841 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:07 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T22:30:08.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:07 smithi012 ceph-mon[31632]: purged_snaps scrub starts 2024-05-17T22:30:08.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:07 smithi012 ceph-mon[31632]: purged_snaps scrub ok 2024-05-17T22:30:08.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:07 smithi012 ceph-mon[31632]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-17T22:30:08.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:07 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T22:30:08.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:07 smithi012 ceph-mon[31632]: osd.3 [v2:172.21.15.12:6810/3563309320,v1:172.21.15.12:6811/3563309320] boot 2024-05-17T22:30:08.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:07 smithi012 ceph-mon[31632]: osdmap e25: 4 total, 4 up, 4 in 2024-05-17T22:30:08.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:07 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T22:30:08.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:07 smithi151 ceph-mon[36101]: purged_snaps scrub starts 2024-05-17T22:30:08.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:07 smithi151 ceph-mon[36101]: purged_snaps scrub ok 2024-05-17T22:30:08.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:07 smithi151 ceph-mon[36101]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-17T22:30:08.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:07 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T22:30:08.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:07 smithi151 ceph-mon[36101]: osd.3 [v2:172.21.15.12:6810/3563309320,v1:172.21.15.12:6811/3563309320] boot 2024-05-17T22:30:08.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:07 smithi151 ceph-mon[36101]: osdmap e25: 4 total, 4 up, 4 in 2024-05-17T22:30:08.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:07 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T22:30:08.866 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:08 smithi096 ceph-mon[36065]: osdmap e26: 4 total, 4 up, 4 in 2024-05-17T22:30:08.867 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:08 smithi096 ceph-mon[36065]: Detected new or changed devices on smithi012 2024-05-17T22:30:08.867 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:08 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:08.867 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:08 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:08.867 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:08 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:30:08.867 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:08 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:30:08.867 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:08 smithi096 ceph-mon[36065]: Adjusting osd_memory_target on smithi012 to 8546M 2024-05-17T22:30:08.867 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:08 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:08.867 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:08 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:30:08.867 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:08 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:30:08.867 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:08 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:08.867 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:08 smithi096 ceph-mon[36065]: osdmap e27: 4 total, 4 up, 4 in 2024-05-17T22:30:09.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:08 smithi012 ceph-mon[31632]: osdmap e26: 4 total, 4 up, 4 in 2024-05-17T22:30:09.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:08 smithi012 ceph-mon[31632]: Detected new or changed devices on smithi012 2024-05-17T22:30:09.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:08 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:09.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:08 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:09.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:08 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:30:09.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:08 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:30:09.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:08 smithi012 ceph-mon[31632]: Adjusting osd_memory_target on smithi012 to 8546M 2024-05-17T22:30:09.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:08 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:09.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:08 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:30:09.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:08 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:30:09.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:08 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:09.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:08 smithi012 ceph-mon[31632]: osdmap e27: 4 total, 4 up, 4 in 2024-05-17T22:30:09.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:09.124+0000 7f238428b700 1 -- 172.21.15.96:0/3424498972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f237c1049a0 msgr2=0x7f237c10b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:30:09.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:09.124+0000 7f238428b700 1 --2- 172.21.15.96:0/3424498972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f237c1049a0 0x7f237c10b2a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f237400b0a0 tx=0x7f237400b3b0 comp rx=0 tx=0).stop 2024-05-17T22:30:09.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:09.124+0000 7f238428b700 1 -- 172.21.15.96:0/3424498972 shutdown_connections 2024-05-17T22:30:09.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:09.124+0000 7f238428b700 1 --2- 172.21.15.96:0/3424498972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f237c1049a0 0x7f237c10b2a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:30:09.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:09.124+0000 7f238428b700 1 --2- 172.21.15.96:0/3424498972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f237c103150 0x7f237c103560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:30:09.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:09.124+0000 7f238428b700 1 --2- 172.21.15.96:0/3424498972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f237c104050 0x7f237c104460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:30:09.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:09.124+0000 7f238428b700 1 -- 172.21.15.96:0/3424498972 >> 172.21.15.96:0/3424498972 conn(0x7f237c0fdb50 msgr2=0x7f237c0fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:30:09.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:09.124+0000 7f238428b700 1 -- 172.21.15.96:0/3424498972 shutdown_connections 2024-05-17T22:30:09.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:09.125+0000 7f238428b700 1 -- 172.21.15.96:0/3424498972 wait complete. 2024-05-17T22:30:09.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:09.125+0000 7f238428b700 1 Processor -- start 2024-05-17T22:30:09.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:09.125+0000 7f238428b700 1 -- start start 2024-05-17T22:30:09.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:09.125+0000 7f238428b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f237c103150 0x7f237c199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:30:09.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:09.125+0000 7f238428b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f237c104050 0x7f237c1998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:30:09.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:09.125+0000 7f238428b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f237c1049a0 0x7f237c1a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:30:09.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:09.125+0000 7f238428b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f237c10e8a0 con 0x7f237c1049a0 2024-05-17T22:30:09.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:09.125+0000 7f238428b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f237c10e720 con 0x7f237c104050 2024-05-17T22:30:09.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:09.125+0000 7f238428b700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f237c10ea20 con 0x7f237c103150 2024-05-17T22:30:09.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:09.125+0000 7f2381826700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f237c104050 0x7f237c1998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:30:09.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:09.126+0000 7f2381826700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f237c104050 0x7f237c1998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50468/0 (socket says 172.21.15.96:50468) 2024-05-17T22:30:09.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:09.126+0000 7f2381826700 1 -- 172.21.15.96:0/2370570208 learned_addr learned my addr 172.21.15.96:0/2370570208 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-05-17T22:30:09.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:09.126+0000 7f2382828700 1 --2- 172.21.15.96:0/2370570208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f237c1049a0 0x7f237c1a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:30:09.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:09.126+0000 7f2382027700 1 --2- 172.21.15.96:0/2370570208 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f237c103150 0x7f237c199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:30:09.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:09.126+0000 7f2381826700 1 -- 172.21.15.96:0/2370570208 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f237c103150 msgr2=0x7f237c199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:30:09.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:09.126+0000 7f2381826700 1 --2- 172.21.15.96:0/2370570208 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f237c103150 0x7f237c199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:30:09.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:09.126+0000 7f2381826700 1 -- 172.21.15.96:0/2370570208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f237c1049a0 msgr2=0x7f237c1a0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:30:09.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:09.126+0000 7f2381826700 1 --2- 172.21.15.96:0/2370570208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f237c1049a0 0x7f237c1a0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:30:09.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:09.126+0000 7f2381826700 1 -- 172.21.15.96:0/2370570208 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f237c101db0 con 0x7f237c104050 2024-05-17T22:30:09.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:09.126+0000 7f2382828700 1 --2- 172.21.15.96:0/2370570208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f237c1049a0 0x7f237c1a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:30:09.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:09.126+0000 7f2381826700 1 --2- 172.21.15.96:0/2370570208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f237c104050 0x7f237c1998b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f236c00e9a0 tx=0x7f236c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:30:09.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:09.126+0000 7f23737fe700 1 -- 172.21.15.96:0/2370570208 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236c00ca10 con 0x7f237c104050 2024-05-17T22:30:09.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:09.126+0000 7f23737fe700 1 -- 172.21.15.96:0/2370570208 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f236c00cbb0 con 0x7f237c104050 2024-05-17T22:30:09.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:09.126+0000 7f238428b700 1 -- 172.21.15.96:0/2370570208 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f237c102040 con 0x7f237c104050 2024-05-17T22:30:09.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:09.126+0000 7f23737fe700 1 -- 172.21.15.96:0/2370570208 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236c018680 con 0x7f237c104050 2024-05-17T22:30:09.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:09.126+0000 7f238428b700 1 -- 172.21.15.96:0/2370570208 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f237c102650 con 0x7f237c104050 2024-05-17T22:30:09.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:09.127+0000 7f238428b700 1 -- 172.21.15.96:0/2370570208 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f237c103560 con 0x7f237c104050 2024-05-17T22:30:09.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:09.129+0000 7f23737fe700 1 -- 172.21.15.96:0/2370570208 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f236c010430 con 0x7f237c104050 2024-05-17T22:30:09.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:09.130+0000 7f23737fe700 1 --2- 172.21.15.96:0/2370570208 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f23680b2900 0x7f23680b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:30:09.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:09.130+0000 7f23737fe700 1 -- 172.21.15.96:0/2370570208 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3669+0+0 (secure 0 0 0) 0x7f236c0e1420 con 0x7f237c104050 2024-05-17T22:30:09.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:09.130+0000 7f2382027700 1 --2- 172.21.15.96:0/2370570208 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f23680b2900 0x7f23680b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:30:09.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:09.131+0000 7f2382027700 1 --2- 172.21.15.96:0/2370570208 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f23680b2900 0x7f23680b4e10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f237c103fe0 tx=0x7f2378009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:30:09.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:09.133+0000 7f23737fe700 1 -- 172.21.15.96:0/2370570208 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f236c0aa360 con 0x7f237c104050 2024-05-17T22:30:09.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:08 smithi151 ceph-mon[36101]: osdmap e26: 4 total, 4 up, 4 in 2024-05-17T22:30:09.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:08 smithi151 ceph-mon[36101]: Detected new or changed devices on smithi012 2024-05-17T22:30:09.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:08 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:09.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:08 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:09.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:08 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:30:09.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:08 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:30:09.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:08 smithi151 ceph-mon[36101]: Adjusting osd_memory_target on smithi012 to 8546M 2024-05-17T22:30:09.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:08 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:09.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:08 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:30:09.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:08 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:30:09.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:08 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:09.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:08 smithi151 ceph-mon[36101]: osdmap e27: 4 total, 4 up, 4 in 2024-05-17T22:30:09.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:09.343+0000 7f238428b700 1 -- 172.21.15.96:0/2370570208 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f237c02e770 con 0x7f23680b2900 2024-05-17T22:30:10.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:09 smithi012 ceph-mon[31632]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-17T22:30:10.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:09 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T22:30:10.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:09 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T22:30:10.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:09 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:30:10.167 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:09 smithi096 ceph-mon[36065]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-17T22:30:10.167 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T22:30:10.167 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T22:30:10.167 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:30:10.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:09 smithi151 ceph-mon[36101]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-17T22:30:10.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:09 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T22:30:10.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:09 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T22:30:10.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:09 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:30:11.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:10 smithi012 ceph-mon[31632]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:30:11.100 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:10 smithi096 ceph-mon[36065]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:30:11.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:10 smithi151 ceph-mon[36101]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:30:12.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:11 smithi151 ceph-mon[36101]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-17T22:30:12.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:11 smithi096 ceph-mon[36065]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-17T22:30:12.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:11 smithi012 ceph-mon[31632]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-17T22:30:13.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:12 smithi151 ceph-mon[36101]: from='client.? 172.21.15.96:0/3115563536' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0e8689a-2816-4089-ade3-931960bae4f3"}]: dispatch 2024-05-17T22:30:13.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:12 smithi151 ceph-mon[36101]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0e8689a-2816-4089-ade3-931960bae4f3"}]: dispatch 2024-05-17T22:30:13.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:12 smithi151 ceph-mon[36101]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0e8689a-2816-4089-ade3-931960bae4f3"}]': finished 2024-05-17T22:30:13.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:12 smithi151 ceph-mon[36101]: osdmap e28: 5 total, 4 up, 5 in 2024-05-17T22:30:13.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:12 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T22:30:13.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:12 smithi151 ceph-mon[36101]: from='client.? 172.21.15.96:0/265861257' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T22:30:13.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:12 smithi096 ceph-mon[36065]: from='client.? 172.21.15.96:0/3115563536' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0e8689a-2816-4089-ade3-931960bae4f3"}]: dispatch 2024-05-17T22:30:13.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:12 smithi096 ceph-mon[36065]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0e8689a-2816-4089-ade3-931960bae4f3"}]: dispatch 2024-05-17T22:30:13.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:12 smithi096 ceph-mon[36065]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0e8689a-2816-4089-ade3-931960bae4f3"}]': finished 2024-05-17T22:30:13.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:12 smithi096 ceph-mon[36065]: osdmap e28: 5 total, 4 up, 5 in 2024-05-17T22:30:13.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:12 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T22:30:13.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:12 smithi096 ceph-mon[36065]: from='client.? 172.21.15.96:0/265861257' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T22:30:13.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:12 smithi012 ceph-mon[31632]: from='client.? 172.21.15.96:0/3115563536' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0e8689a-2816-4089-ade3-931960bae4f3"}]: dispatch 2024-05-17T22:30:13.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:12 smithi012 ceph-mon[31632]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0e8689a-2816-4089-ade3-931960bae4f3"}]: dispatch 2024-05-17T22:30:13.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:12 smithi012 ceph-mon[31632]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0e8689a-2816-4089-ade3-931960bae4f3"}]': finished 2024-05-17T22:30:13.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:12 smithi012 ceph-mon[31632]: osdmap e28: 5 total, 4 up, 5 in 2024-05-17T22:30:13.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:12 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T22:30:13.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:12 smithi012 ceph-mon[31632]: from='client.? 172.21.15.96:0/265861257' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T22:30:14.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:13 smithi151 ceph-mon[36101]: pgmap v107: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-17T22:30:14.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:13 smithi096 ceph-mon[36065]: pgmap v107: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-17T22:30:14.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:13 smithi012 ceph-mon[31632]: pgmap v107: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-17T22:30:15.996 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:15 smithi096 ceph-mon[36065]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-17T22:30:16.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:15 smithi151 ceph-mon[36101]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-17T22:30:16.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:15 smithi012 ceph-mon[31632]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-17T22:30:18.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:17 smithi096 ceph-mon[36065]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-17T22:30:18.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:17 smithi151 ceph-mon[36101]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-17T22:30:18.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:17 smithi012 ceph-mon[31632]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-17T22:30:19.979 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:19 smithi096 ceph-mon[36065]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-17T22:30:19.979 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:19 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-17T22:30:19.979 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:19 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:30:20.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:19 smithi151 ceph-mon[36101]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-17T22:30:20.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:19 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-17T22:30:20.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:19 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:30:20.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:19 smithi012 ceph-mon[31632]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-17T22:30:20.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:19 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-17T22:30:20.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:19 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:30:20.976 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:20 smithi096 ceph-mon[36065]: Deploying daemon osd.4 on smithi096 2024-05-17T22:30:21.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:20 smithi151 ceph-mon[36101]: Deploying daemon osd.4 on smithi096 2024-05-17T22:30:21.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:20 smithi012 ceph-mon[31632]: Deploying daemon osd.4 on smithi096 2024-05-17T22:30:21.972 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:21 smithi096 ceph-mon[36065]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T22:30:22.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:21 smithi151 ceph-mon[36101]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T22:30:22.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:21 smithi012 ceph-mon[31632]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T22:30:24.164 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:23 smithi096 ceph-mon[36065]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-05-17T22:30:24.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:23 smithi151 ceph-mon[36101]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-05-17T22:30:24.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:23 smithi012 ceph-mon[31632]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-05-17T22:30:26.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:25 smithi096 ceph-mon[36065]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-17T22:30:26.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:25 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:26.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:25 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:26.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:25 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:30:26.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:25 smithi012 ceph-mon[31632]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-17T22:30:26.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:25 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:26.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:25 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:26.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:25 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:30:26.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:25 smithi151 ceph-mon[36101]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-17T22:30:26.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:25 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:26.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:25 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:26.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:25 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:30:27.948 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:27 smithi096 ceph-mon[36065]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-17T22:30:27.949 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:27 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:27.949 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:27 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:27.949 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:27 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:30:27.949 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:27 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:30:27.949 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:27 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:28.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:27 smithi012 ceph-mon[31632]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-17T22:30:28.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:27 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:28.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:27 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:28.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:27 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:30:28.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:27 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:30:28.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:27 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:28.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:27 smithi151 ceph-mon[36101]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-17T22:30:28.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:27 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:28.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:27 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:28.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:27 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:30:28.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:27 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:30:28.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:27 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:28.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:28.714+0000 7f23737fe700 1 -- 172.21.15.96:0/2370570208 <== mgr.14152 v2:172.21.15.12:6800/1388514047 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f237c02e770 con 0x7f23680b2900 2024-05-17T22:30:28.717 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 4 on host 'smithi096' 2024-05-17T22:30:28.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:28.716+0000 7f238428b700 1 -- 172.21.15.96:0/2370570208 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f23680b2900 msgr2=0x7f23680b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:30:28.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:28.716+0000 7f238428b700 1 --2- 172.21.15.96:0/2370570208 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f23680b2900 0x7f23680b4e10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f237c103fe0 tx=0x7f2378009190 comp rx=0 tx=0).stop 2024-05-17T22:30:28.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:28.716+0000 7f238428b700 1 -- 172.21.15.96:0/2370570208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f237c104050 msgr2=0x7f237c1998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:30:28.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:28.716+0000 7f238428b700 1 --2- 172.21.15.96:0/2370570208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f237c104050 0x7f237c1998b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f236c00e9a0 tx=0x7f236c00ecb0 comp rx=0 tx=0).stop 2024-05-17T22:30:28.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:28.716+0000 7f238428b700 1 -- 172.21.15.96:0/2370570208 shutdown_connections 2024-05-17T22:30:28.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:28.716+0000 7f238428b700 1 --2- 172.21.15.96:0/2370570208 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f23680b2900 0x7f23680b4e10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:30:28.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:28.716+0000 7f238428b700 1 --2- 172.21.15.96:0/2370570208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f237c1049a0 0x7f237c1a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:30:28.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:28.716+0000 7f238428b700 1 --2- 172.21.15.96:0/2370570208 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f237c103150 0x7f237c199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:30:28.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:28.716+0000 7f238428b700 1 --2- 172.21.15.96:0/2370570208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f237c104050 0x7f237c1998b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:30:28.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:28.716+0000 7f238428b700 1 -- 172.21.15.96:0/2370570208 >> 172.21.15.96:0/2370570208 conn(0x7f237c0fdb50 msgr2=0x7f237c107280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:30:28.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:28.717+0000 7f238428b700 1 -- 172.21.15.96:0/2370570208 shutdown_connections 2024-05-17T22:30:28.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:30:28.717+0000 7f238428b700 1 -- 172.21.15.96:0/2370570208 wait complete. 2024-05-17T22:30:29.356 DEBUG:teuthology.orchestra.run.smithi096:osd.4> sudo journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@osd.4.service 2024-05-17T22:30:29.359 INFO:tasks.cephadm:Deploying osd.5 on smithi151 with /dev/vg_nvme/lv_3... 2024-05-17T22:30:29.359 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-17T22:30:29.607 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T22:30:29.976 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:29 smithi151 ceph-mon[36101]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-17T22:30:29.977 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:29 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:29.977 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:29 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:29.977 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:29 smithi151 ceph-mon[36101]: from='osd.4 [v2:172.21.15.96:6808/2219103166,v1:172.21.15.96:6809/2219103166]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-17T22:30:29.977 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:29 smithi151 ceph-mon[36101]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-17T22:30:29.977 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:29 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:30:30.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:29 smithi096 ceph-mon[36065]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-17T22:30:30.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:29 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:30.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:29 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:30.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:29 smithi096 ceph-mon[36065]: from='osd.4 [v2:172.21.15.96:6808/2219103166,v1:172.21.15.96:6809/2219103166]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-17T22:30:30.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:29 smithi096 ceph-mon[36065]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-17T22:30:30.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:29 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:30:30.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:29 smithi012 ceph-mon[31632]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-17T22:30:30.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:29 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:30.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:29 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:30.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:29 smithi012 ceph-mon[31632]: from='osd.4 [v2:172.21.15.96:6808/2219103166,v1:172.21.15.96:6809/2219103166]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-17T22:30:30.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:29 smithi012 ceph-mon[31632]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-17T22:30:30.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:29 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:30:30.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:30 smithi096 ceph-mon[36065]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-17T22:30:30.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:30 smithi096 ceph-mon[36065]: from='osd.4 [v2:172.21.15.96:6808/2219103166,v1:172.21.15.96:6809/2219103166]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-17T22:30:30.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:30 smithi096 ceph-mon[36065]: osdmap e29: 5 total, 4 up, 5 in 2024-05-17T22:30:30.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:30 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T22:30:30.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:30 smithi096 ceph-mon[36065]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-17T22:30:30.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:30 smithi151 ceph-mon[36101]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-17T22:30:30.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:30 smithi151 ceph-mon[36101]: from='osd.4 [v2:172.21.15.96:6808/2219103166,v1:172.21.15.96:6809/2219103166]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-17T22:30:30.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:30 smithi151 ceph-mon[36101]: osdmap e29: 5 total, 4 up, 5 in 2024-05-17T22:30:30.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:30 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T22:30:30.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:30 smithi151 ceph-mon[36101]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-17T22:30:31.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:30 smithi012 ceph-mon[31632]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-17T22:30:31.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:30 smithi012 ceph-mon[31632]: from='osd.4 [v2:172.21.15.96:6808/2219103166,v1:172.21.15.96:6809/2219103166]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-17T22:30:31.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:30 smithi012 ceph-mon[31632]: osdmap e29: 5 total, 4 up, 5 in 2024-05-17T22:30:31.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:30 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T22:30:31.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:30 smithi012 ceph-mon[31632]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-17T22:30:31.626 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-17T22:30:31.626 INFO:teuthology.orchestra.run.smithi151.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-17T22:30:31.627 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10+0 records in 2024-05-17T22:30:31.627 INFO:teuthology.orchestra.run.smithi151.stderr:10+0 records out 2024-05-17T22:30:31.627 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0138553 s, 757 MB/s 2024-05-17T22:30:31.627 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping successful for: 2024-05-17T22:30:31.693 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 17 22:30:31 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-4[47442]: 2024-05-17T22:30:31.371+0000 7ff3c41cf700 -1 osd.4 0 waiting for initial osdmap 2024-05-17T22:30:31.693 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 17 22:30:31 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-4[47442]: 2024-05-17T22:30:31.378+0000 7ff3bb77b700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T22:30:31.780 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:31 smithi151 ceph-mon[36101]: purged_snaps scrub starts 2024-05-17T22:30:31.780 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:31 smithi151 ceph-mon[36101]: purged_snaps scrub ok 2024-05-17T22:30:31.780 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:31 smithi151 ceph-mon[36101]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-17T22:30:31.780 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:31 smithi151 ceph-mon[36101]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-05-17T22:30:31.780 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:31 smithi151 ceph-mon[36101]: osdmap e30: 5 total, 4 up, 5 in 2024-05-17T22:30:31.780 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:31 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T22:30:31.780 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:31 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T22:30:31.780 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:31 smithi151 ceph-mon[36101]: from='osd.4 ' entity='osd.4' 2024-05-17T22:30:31.970 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:31 smithi096 ceph-mon[36065]: purged_snaps scrub starts 2024-05-17T22:30:31.970 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:31 smithi096 ceph-mon[36065]: purged_snaps scrub ok 2024-05-17T22:30:31.970 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:31 smithi096 ceph-mon[36065]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-17T22:30:31.970 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:31 smithi096 ceph-mon[36065]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-05-17T22:30:31.970 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:31 smithi096 ceph-mon[36065]: osdmap e30: 5 total, 4 up, 5 in 2024-05-17T22:30:31.970 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:31 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T22:30:31.970 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:31 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T22:30:31.970 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:31 smithi096 ceph-mon[36065]: from='osd.4 ' entity='osd.4' 2024-05-17T22:30:32.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:31 smithi012 ceph-mon[31632]: purged_snaps scrub starts 2024-05-17T22:30:32.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:31 smithi012 ceph-mon[31632]: purged_snaps scrub ok 2024-05-17T22:30:32.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:31 smithi012 ceph-mon[31632]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-17T22:30:32.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:31 smithi012 ceph-mon[31632]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-05-17T22:30:32.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:31 smithi012 ceph-mon[31632]: osdmap e30: 5 total, 4 up, 5 in 2024-05-17T22:30:32.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:31 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T22:30:32.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:31 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T22:30:32.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:31 smithi012 ceph-mon[31632]: from='osd.4 ' entity='osd.4' 2024-05-17T22:30:32.306 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi151:vg_nvme/lv_3 2024-05-17T22:30:32.545 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T22:30:32.921 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:32 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T22:30:32.921 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:32 smithi151 ceph-mon[36101]: osd.4 [v2:172.21.15.96:6808/2219103166,v1:172.21.15.96:6809/2219103166] boot 2024-05-17T22:30:32.921 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:32 smithi151 ceph-mon[36101]: osdmap e31: 5 total, 5 up, 5 in 2024-05-17T22:30:32.921 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:32 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T22:30:32.921 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:32 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:32.921 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:32 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:32.921 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:32 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:30:32.921 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:32 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:30:32.921 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:32 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:32.921 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:32 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:30:32.921 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:32 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:30:32.921 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:32 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:33.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:32 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T22:30:33.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:32 smithi096 ceph-mon[36065]: osd.4 [v2:172.21.15.96:6808/2219103166,v1:172.21.15.96:6809/2219103166] boot 2024-05-17T22:30:33.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:32 smithi096 ceph-mon[36065]: osdmap e31: 5 total, 5 up, 5 in 2024-05-17T22:30:33.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:32 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T22:30:33.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:32 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:33.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:32 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:33.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:32 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:30:33.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:32 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:30:33.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:32 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:33.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:32 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:30:33.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:32 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:30:33.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:32 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:33.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:32 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T22:30:33.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:32 smithi012 ceph-mon[31632]: osd.4 [v2:172.21.15.96:6808/2219103166,v1:172.21.15.96:6809/2219103166] boot 2024-05-17T22:30:33.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:32 smithi012 ceph-mon[31632]: osdmap e31: 5 total, 5 up, 5 in 2024-05-17T22:30:33.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:32 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T22:30:33.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:32 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:33.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:32 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:33.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:32 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:30:33.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:32 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:30:33.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:32 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:33.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:32 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:30:33.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:32 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:30:33.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:32 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:33.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:33 smithi151 ceph-mon[36101]: Detected new or changed devices on smithi096 2024-05-17T22:30:33.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:33 smithi151 ceph-mon[36101]: Adjusting osd_memory_target on smithi096 to 8544M 2024-05-17T22:30:33.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:33 smithi151 ceph-mon[36101]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-17T22:30:34.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:33 smithi096 ceph-mon[36065]: Detected new or changed devices on smithi096 2024-05-17T22:30:34.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:33 smithi096 ceph-mon[36065]: Adjusting osd_memory_target on smithi096 to 8544M 2024-05-17T22:30:34.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:33 smithi096 ceph-mon[36065]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-17T22:30:34.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:33 smithi012 ceph-mon[31632]: Detected new or changed devices on smithi096 2024-05-17T22:30:34.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:33 smithi012 ceph-mon[31632]: Adjusting osd_memory_target on smithi096 to 8544M 2024-05-17T22:30:34.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:33 smithi012 ceph-mon[31632]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-17T22:30:34.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:34.253+0000 7f53b4c0c700 1 -- 172.21.15.151:0/1242842897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53b01070f0 msgr2=0x7f53b0109510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:30:34.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:34.253+0000 7f53b4c0c700 1 --2- 172.21.15.151:0/1242842897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53b01070f0 0x7f53b0109510 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5398009990 tx=0x7f5398009ca0 comp rx=0 tx=0).stop 2024-05-17T22:30:34.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:34.253+0000 7f53b4c0c700 1 -- 172.21.15.151:0/1242842897 shutdown_connections 2024-05-17T22:30:34.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:34.253+0000 7f53b4c0c700 1 --2- 172.21.15.151:0/1242842897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53b0109a50 0x7f53b010be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:30:34.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:34.253+0000 7f53b4c0c700 1 --2- 172.21.15.151:0/1242842897 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53b0104790 0x7f53b0106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:30:34.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:34.253+0000 7f53b4c0c700 1 --2- 172.21.15.151:0/1242842897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53b01070f0 0x7f53b0109510 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:30:34.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:34.253+0000 7f53b4c0c700 1 -- 172.21.15.151:0/1242842897 >> 172.21.15.151:0/1242842897 conn(0x7f53b00fe3c0 msgr2=0x7f53b0100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:30:34.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:34.253+0000 7f53b4c0c700 1 -- 172.21.15.151:0/1242842897 shutdown_connections 2024-05-17T22:30:34.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:34.253+0000 7f53b4c0c700 1 -- 172.21.15.151:0/1242842897 wait complete. 2024-05-17T22:30:34.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:34.254+0000 7f53b4c0c700 1 Processor -- start 2024-05-17T22:30:34.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:34.254+0000 7f53b4c0c700 1 -- start start 2024-05-17T22:30:34.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:34.254+0000 7f53b4c0c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53b0104790 0x7f53b019a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:30:34.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:34.254+0000 7f53b4c0c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53b01070f0 0x7f53b019a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:30:34.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:34.254+0000 7f53b4c0c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53b0109a50 0x7f53b01a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:30:34.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:34.254+0000 7f53b4c0c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f53b010f710 con 0x7f53b0104790 2024-05-17T22:30:34.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:34.254+0000 7f53b4c0c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f53b010f590 con 0x7f53b01070f0 2024-05-17T22:30:34.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:34.254+0000 7f53b4c0c700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f53b010f890 con 0x7f53b0109a50 2024-05-17T22:30:34.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:34.254+0000 7f53aed9d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53b0109a50 0x7f53b01a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:30:34.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:34.254+0000 7f53aed9d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53b0109a50 0x7f53b01a17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57672/0 (socket says 172.21.15.151:57672) 2024-05-17T22:30:34.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:34.254+0000 7f53aed9d700 1 -- 172.21.15.151:0/373072609 learned_addr learned my addr 172.21.15.151:0/373072609 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-17T22:30:34.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:34.254+0000 7f53ae59c700 1 --2- 172.21.15.151:0/373072609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53b0104790 0x7f53b019a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:30:34.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:34.255+0000 7f53a7fff700 1 --2- 172.21.15.151:0/373072609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53b01070f0 0x7f53b019a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:30:34.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:34.255+0000 7f53aed9d700 1 -- 172.21.15.151:0/373072609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53b01070f0 msgr2=0x7f53b019a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:30:34.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:34.255+0000 7f53aed9d700 1 --2- 172.21.15.151:0/373072609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53b01070f0 0x7f53b019a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:30:34.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:34.255+0000 7f53aed9d700 1 -- 172.21.15.151:0/373072609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53b0104790 msgr2=0x7f53b019a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:30:34.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:34.255+0000 7f53aed9d700 1 --2- 172.21.15.151:0/373072609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53b0104790 0x7f53b019a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:30:34.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:34.255+0000 7f53aed9d700 1 -- 172.21.15.151:0/373072609 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53b01a1dc0 con 0x7f53b0109a50 2024-05-17T22:30:34.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:34.255+0000 7f53aed9d700 1 --2- 172.21.15.151:0/373072609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53b0109a50 0x7f53b01a17d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f53a8007d40 tx=0x7f53a800a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:30:34.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:34.255+0000 7f53a77fe700 1 -- 172.21.15.151:0/373072609 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a8016020 con 0x7f53b0109a50 2024-05-17T22:30:34.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:34.255+0000 7f53b4c0c700 1 -- 172.21.15.151:0/373072609 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53b01a20b0 con 0x7f53b0109a50 2024-05-17T22:30:34.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:34.255+0000 7f53a77fe700 1 -- 172.21.15.151:0/373072609 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53a800ac90 con 0x7f53b0109a50 2024-05-17T22:30:34.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:34.255+0000 7f53a77fe700 1 -- 172.21.15.151:0/373072609 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a80115e0 con 0x7f53b0109a50 2024-05-17T22:30:34.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:34.255+0000 7f53b4c0c700 1 -- 172.21.15.151:0/373072609 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53b01a25f0 con 0x7f53b0109a50 2024-05-17T22:30:34.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:34.256+0000 7f53b4c0c700 1 -- 172.21.15.151:0/373072609 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53b0107500 con 0x7f53b0109a50 2024-05-17T22:30:34.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:34.258+0000 7f53a77fe700 1 -- 172.21.15.151:0/373072609 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f53a8019020 con 0x7f53b0109a50 2024-05-17T22:30:34.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:34.258+0000 7f53a77fe700 1 --2- 172.21.15.151:0/373072609 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f539c0b2880 0x7f539c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:30:34.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:34.258+0000 7f53a77fe700 1 -- 172.21.15.151:0/373072609 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4101+0+0 (secure 0 0 0) 0x7f53a80e15e0 con 0x7f53b0109a50 2024-05-17T22:30:34.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:34.258+0000 7f53ae59c700 1 --2- 172.21.15.151:0/373072609 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f539c0b2880 0x7f539c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:30:34.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:34.259+0000 7f53ae59c700 1 --2- 172.21.15.151:0/373072609 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f539c0b2880 0x7f539c0b4d90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f53b006c0a0 tx=0x7f53a000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:30:34.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:34.262+0000 7f53a77fe700 1 -- 172.21.15.151:0/373072609 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f53a8020020 con 0x7f53b0109a50 2024-05-17T22:30:34.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:34.473+0000 7f53b4c0c700 1 -- 172.21.15.151:0/373072609 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f53b00646d0 con 0x7f539c0b2880 2024-05-17T22:30:34.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:34 smithi151 ceph-mon[36101]: osdmap e32: 5 total, 5 up, 5 in 2024-05-17T22:30:34.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:34 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T22:30:34.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:34 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T22:30:34.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:34 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:30:35.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:34 smithi096 ceph-mon[36065]: osdmap e32: 5 total, 5 up, 5 in 2024-05-17T22:30:35.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:34 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T22:30:35.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:34 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T22:30:35.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:34 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:30:35.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:34 smithi012 ceph-mon[31632]: osdmap e32: 5 total, 5 up, 5 in 2024-05-17T22:30:35.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:34 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T22:30:35.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:34 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T22:30:35.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:34 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:30:35.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:35 smithi151 ceph-mon[36101]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-17T22:30:35.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:35 smithi151 ceph-mon[36101]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:30:36.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:35 smithi096 ceph-mon[36065]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-17T22:30:36.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:35 smithi096 ceph-mon[36065]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:30:36.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:35 smithi012 ceph-mon[31632]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-17T22:30:36.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:35 smithi012 ceph-mon[31632]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:30:37.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:37 smithi151 ceph-mon[36101]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-17T22:30:37.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:37 smithi151 ceph-mon[36101]: from='client.? 172.21.15.151:0/4092384521' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e76ba432-94ff-4998-aaba-2645d9e6055d"}]: dispatch 2024-05-17T22:30:37.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:37 smithi151 ceph-mon[36101]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e76ba432-94ff-4998-aaba-2645d9e6055d"}]: dispatch 2024-05-17T22:30:37.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:37 smithi151 ceph-mon[36101]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e76ba432-94ff-4998-aaba-2645d9e6055d"}]': finished 2024-05-17T22:30:37.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:37 smithi151 ceph-mon[36101]: osdmap e33: 6 total, 5 up, 6 in 2024-05-17T22:30:37.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:37 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T22:30:38.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:37 smithi096 ceph-mon[36065]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-17T22:30:38.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:37 smithi096 ceph-mon[36065]: from='client.? 172.21.15.151:0/4092384521' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e76ba432-94ff-4998-aaba-2645d9e6055d"}]: dispatch 2024-05-17T22:30:38.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:37 smithi096 ceph-mon[36065]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e76ba432-94ff-4998-aaba-2645d9e6055d"}]: dispatch 2024-05-17T22:30:38.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:37 smithi096 ceph-mon[36065]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e76ba432-94ff-4998-aaba-2645d9e6055d"}]': finished 2024-05-17T22:30:38.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:37 smithi096 ceph-mon[36065]: osdmap e33: 6 total, 5 up, 6 in 2024-05-17T22:30:38.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:37 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T22:30:38.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:37 smithi012 ceph-mon[31632]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-17T22:30:38.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:37 smithi012 ceph-mon[31632]: from='client.? 172.21.15.151:0/4092384521' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e76ba432-94ff-4998-aaba-2645d9e6055d"}]: dispatch 2024-05-17T22:30:38.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:37 smithi012 ceph-mon[31632]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e76ba432-94ff-4998-aaba-2645d9e6055d"}]: dispatch 2024-05-17T22:30:38.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:37 smithi012 ceph-mon[31632]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e76ba432-94ff-4998-aaba-2645d9e6055d"}]': finished 2024-05-17T22:30:38.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:37 smithi012 ceph-mon[31632]: osdmap e33: 6 total, 5 up, 6 in 2024-05-17T22:30:38.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:37 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T22:30:38.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:38 smithi151 ceph-mon[36101]: from='client.? 172.21.15.151:0/4115749632' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T22:30:39.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:38 smithi096 ceph-mon[36065]: from='client.? 172.21.15.151:0/4115749632' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T22:30:39.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:38 smithi012 ceph-mon[31632]: from='client.? 172.21.15.151:0/4115749632' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T22:30:39.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:39 smithi151 ceph-mon[36101]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-17T22:30:40.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:39 smithi096 ceph-mon[36065]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-17T22:30:40.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:39 smithi012 ceph-mon[31632]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-17T22:30:41.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:41 smithi151 ceph-mon[36101]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-17T22:30:42.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:41 smithi096 ceph-mon[36065]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-17T22:30:42.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:41 smithi012 ceph-mon[31632]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-17T22:30:43.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:43 smithi151 ceph-mon[36101]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-17T22:30:44.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:43 smithi096 ceph-mon[36065]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-17T22:30:44.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:43 smithi012 ceph-mon[31632]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-17T22:30:44.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:44 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-17T22:30:44.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:44 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:30:45.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:44 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-17T22:30:45.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:44 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:30:45.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:44 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-17T22:30:45.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:44 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:30:45.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:45 smithi151 ceph-mon[36101]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-17T22:30:45.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:45 smithi151 ceph-mon[36101]: Deploying daemon osd.5 on smithi151 2024-05-17T22:30:46.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:45 smithi096 ceph-mon[36065]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-17T22:30:46.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:45 smithi096 ceph-mon[36065]: Deploying daemon osd.5 on smithi151 2024-05-17T22:30:46.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:45 smithi012 ceph-mon[31632]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-17T22:30:46.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:45 smithi012 ceph-mon[31632]: Deploying daemon osd.5 on smithi151 2024-05-17T22:30:47.897 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:47 smithi151 ceph-mon[36101]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-17T22:30:48.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:47 smithi096 ceph-mon[36065]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-17T22:30:48.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:47 smithi012 ceph-mon[31632]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-17T22:30:50.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:49 smithi096 ceph-mon[36065]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-17T22:30:50.089 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:49 smithi151 ceph-mon[36101]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-17T22:30:50.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:49 smithi012 ceph-mon[31632]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-17T22:30:51.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:51 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:51.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:51 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:51.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:51 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:30:51.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:51 smithi151 ceph-mon[36101]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-17T22:30:51.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:51 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:51.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:51 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:51.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:51 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:30:51.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:51 smithi012 ceph-mon[31632]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-17T22:30:51.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:51 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:51.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:51 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:51.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:51 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:30:51.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:51 smithi096 ceph-mon[36065]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-17T22:30:53.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:53 smithi151 ceph-mon[36101]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-17T22:30:53.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:53 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:53.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:53 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:53.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:53 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:30:53.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:53 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:30:53.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:53 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:53.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:53 smithi096 ceph-mon[36065]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-17T22:30:53.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:53 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:53.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:53 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:53.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:53 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:30:53.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:53 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:30:53.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:53 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:53.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:53 smithi012 ceph-mon[31632]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-17T22:30:53.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:53 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:53.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:53 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:53.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:53 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:30:53.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:53 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:30:53.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:53 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:54.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:54.346+0000 7f53a77fe700 1 -- 172.21.15.151:0/373072609 <== mgr.14152 v2:172.21.15.12:6800/1388514047 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f53b00646d0 con 0x7f539c0b2880 2024-05-17T22:30:54.349 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 5 on host 'smithi151' 2024-05-17T22:30:54.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:54.348+0000 7f53b4c0c700 1 -- 172.21.15.151:0/373072609 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f539c0b2880 msgr2=0x7f539c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:30:54.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:54.348+0000 7f53b4c0c700 1 --2- 172.21.15.151:0/373072609 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f539c0b2880 0x7f539c0b4d90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f53b006c0a0 tx=0x7f53a000a380 comp rx=0 tx=0).stop 2024-05-17T22:30:54.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:54.348+0000 7f53b4c0c700 1 -- 172.21.15.151:0/373072609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53b0109a50 msgr2=0x7f53b01a17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:30:54.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:54.348+0000 7f53b4c0c700 1 --2- 172.21.15.151:0/373072609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53b0109a50 0x7f53b01a17d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f53a8007d40 tx=0x7f53a800a340 comp rx=0 tx=0).stop 2024-05-17T22:30:54.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:54.348+0000 7f53b4c0c700 1 -- 172.21.15.151:0/373072609 shutdown_connections 2024-05-17T22:30:54.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:54.348+0000 7f53b4c0c700 1 --2- 172.21.15.151:0/373072609 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f539c0b2880 0x7f539c0b4d90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:30:54.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:54.348+0000 7f53b4c0c700 1 --2- 172.21.15.151:0/373072609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53b0109a50 0x7f53b01a17d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:30:54.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:54.348+0000 7f53b4c0c700 1 --2- 172.21.15.151:0/373072609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53b0104790 0x7f53b019a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:30:54.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:54.348+0000 7f53b4c0c700 1 --2- 172.21.15.151:0/373072609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53b01070f0 0x7f53b019a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:30:54.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:54.348+0000 7f53b4c0c700 1 -- 172.21.15.151:0/373072609 >> 172.21.15.151:0/373072609 conn(0x7f53b00fe3c0 msgr2=0x7f53b01053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:30:54.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:54.348+0000 7f53b4c0c700 1 -- 172.21.15.151:0/373072609 shutdown_connections 2024-05-17T22:30:54.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:30:54.348+0000 7f53b4c0c700 1 -- 172.21.15.151:0/373072609 wait complete. 2024-05-17T22:30:54.675 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:54 smithi151 ceph-mon[36101]: from='osd.5 [v2:172.21.15.151:6808/2589835078,v1:172.21.15.151:6809/2589835078]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-17T22:30:54.676 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:54 smithi151 ceph-mon[36101]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-17T22:30:54.676 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:54 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:54.676 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:54 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:54.676 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:54 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:30:54.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:54 smithi096 ceph-mon[36065]: from='osd.5 [v2:172.21.15.151:6808/2589835078,v1:172.21.15.151:6809/2589835078]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-17T22:30:54.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:54 smithi096 ceph-mon[36065]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-17T22:30:54.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:54 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:54.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:54 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:54.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:54 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:30:54.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:54 smithi012 ceph-mon[31632]: from='osd.5 [v2:172.21.15.151:6808/2589835078,v1:172.21.15.151:6809/2589835078]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-17T22:30:54.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:54 smithi012 ceph-mon[31632]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-17T22:30:54.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:54 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:54.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:54 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:54.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:54 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:30:55.038 DEBUG:teuthology.orchestra.run.smithi151:osd.5> sudo journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@osd.5.service 2024-05-17T22:30:55.040 INFO:tasks.cephadm:Deploying osd.6 on smithi012 with /dev/vg_nvme/lv_2... 2024-05-17T22:30:55.040 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-17T22:30:55.301 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:30:55.682 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:55 smithi012 ceph-mon[31632]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-17T22:30:55.683 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:55 smithi012 ceph-mon[31632]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-17T22:30:55.683 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:55 smithi012 ceph-mon[31632]: osdmap e34: 6 total, 5 up, 6 in 2024-05-17T22:30:55.683 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:55 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T22:30:55.683 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:55 smithi012 ceph-mon[31632]: from='osd.5 [v2:172.21.15.151:6808/2589835078,v1:172.21.15.151:6809/2589835078]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-17T22:30:55.683 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:55 smithi012 ceph-mon[31632]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-17T22:30:55.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:55 smithi151 ceph-mon[36101]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-17T22:30:55.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:55 smithi151 ceph-mon[36101]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-17T22:30:55.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:55 smithi151 ceph-mon[36101]: osdmap e34: 6 total, 5 up, 6 in 2024-05-17T22:30:55.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:55 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T22:30:55.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:55 smithi151 ceph-mon[36101]: from='osd.5 [v2:172.21.15.151:6808/2589835078,v1:172.21.15.151:6809/2589835078]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-17T22:30:55.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:55 smithi151 ceph-mon[36101]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-17T22:30:55.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:55 smithi096 ceph-mon[36065]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-17T22:30:55.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:55 smithi096 ceph-mon[36065]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-17T22:30:55.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:55 smithi096 ceph-mon[36065]: osdmap e34: 6 total, 5 up, 6 in 2024-05-17T22:30:55.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:55 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T22:30:55.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:55 smithi096 ceph-mon[36065]: from='osd.5 [v2:172.21.15.151:6808/2589835078,v1:172.21.15.151:6809/2589835078]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-17T22:30:55.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:55 smithi096 ceph-mon[36065]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-17T22:30:56.490 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:30:56 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5[48151]: 2024-05-17T22:30:56.248+0000 7f96d7739700 -1 osd.5 0 waiting for initial osdmap 2024-05-17T22:30:56.491 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:30:56 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5[48151]: 2024-05-17T22:30:56.256+0000 7f96cece5700 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T22:30:56.787 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:56 smithi012 ceph-mon[31632]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-05-17T22:30:56.787 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:56 smithi012 ceph-mon[31632]: osdmap e35: 6 total, 5 up, 6 in 2024-05-17T22:30:56.787 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:56 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T22:30:56.787 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:56 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T22:30:56.787 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:56 smithi012 ceph-mon[31632]: from='osd.5 ' entity='osd.5' 2024-05-17T22:30:56.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:56 smithi096 ceph-mon[36065]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-05-17T22:30:56.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:56 smithi096 ceph-mon[36065]: osdmap e35: 6 total, 5 up, 6 in 2024-05-17T22:30:56.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:56 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T22:30:56.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:56 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T22:30:56.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:56 smithi096 ceph-mon[36065]: from='osd.5 ' entity='osd.5' 2024-05-17T22:30:56.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:56 smithi151 ceph-mon[36101]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-05-17T22:30:56.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:56 smithi151 ceph-mon[36101]: osdmap e35: 6 total, 5 up, 6 in 2024-05-17T22:30:56.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:56 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T22:30:56.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:56 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T22:30:56.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:56 smithi151 ceph-mon[36101]: from='osd.5 ' entity='osd.5' 2024-05-17T22:30:57.391 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-17T22:30:57.391 INFO:teuthology.orchestra.run.smithi012.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-17T22:30:57.392 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10+0 records in 2024-05-17T22:30:57.392 INFO:teuthology.orchestra.run.smithi012.stderr:10+0 records out 2024-05-17T22:30:57.392 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111601 s, 940 MB/s 2024-05-17T22:30:57.392 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping successful for: 2024-05-17T22:30:57.663 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:57 smithi151 ceph-mon[36101]: purged_snaps scrub starts 2024-05-17T22:30:57.663 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:57 smithi151 ceph-mon[36101]: purged_snaps scrub ok 2024-05-17T22:30:57.664 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:57 smithi151 ceph-mon[36101]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-17T22:30:57.664 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:57 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T22:30:57.664 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:57 smithi151 ceph-mon[36101]: osd.5 [v2:172.21.15.151:6808/2589835078,v1:172.21.15.151:6809/2589835078] boot 2024-05-17T22:30:57.664 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:57 smithi151 ceph-mon[36101]: osdmap e36: 6 total, 6 up, 6 in 2024-05-17T22:30:57.664 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:57 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T22:30:57.704 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:57 smithi012 ceph-mon[31632]: purged_snaps scrub starts 2024-05-17T22:30:57.704 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:57 smithi012 ceph-mon[31632]: purged_snaps scrub ok 2024-05-17T22:30:57.704 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:57 smithi012 ceph-mon[31632]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-17T22:30:57.704 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:57 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T22:30:57.704 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:57 smithi012 ceph-mon[31632]: osd.5 [v2:172.21.15.151:6808/2589835078,v1:172.21.15.151:6809/2589835078] boot 2024-05-17T22:30:57.704 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:57 smithi012 ceph-mon[31632]: osdmap e36: 6 total, 6 up, 6 in 2024-05-17T22:30:57.704 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:57 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T22:30:57.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:57 smithi096 ceph-mon[36065]: purged_snaps scrub starts 2024-05-17T22:30:57.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:57 smithi096 ceph-mon[36065]: purged_snaps scrub ok 2024-05-17T22:30:57.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:57 smithi096 ceph-mon[36065]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-17T22:30:57.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:57 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T22:30:57.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:57 smithi096 ceph-mon[36065]: osd.5 [v2:172.21.15.151:6808/2589835078,v1:172.21.15.151:6809/2589835078] boot 2024-05-17T22:30:57.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:57 smithi096 ceph-mon[36065]: osdmap e36: 6 total, 6 up, 6 in 2024-05-17T22:30:57.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:57 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T22:30:58.169 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi012:vg_nvme/lv_2 2024-05-17T22:30:58.424 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:30:58.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:58 smithi151 ceph-mon[36101]: osdmap e37: 6 total, 6 up, 6 in 2024-05-17T22:30:58.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:58 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:58.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:58 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:58.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:58 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:30:58.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:58 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:30:58.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:58 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:58.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:58 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:30:58.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:58 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:30:58.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:58 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:58.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:58 smithi096 ceph-mon[36065]: osdmap e37: 6 total, 6 up, 6 in 2024-05-17T22:30:58.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:58 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:58.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:58 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:58.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:58 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:30:58.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:58 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:30:58.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:58 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:58.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:58 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:30:58.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:58 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:30:58.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:58 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:58.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:58 smithi012 ceph-mon[31632]: osdmap e37: 6 total, 6 up, 6 in 2024-05-17T22:30:58.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:58 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:58.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:58 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:58.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:58 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:30:58.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:58 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:30:58.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:58 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:58.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:58 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:30:58.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:58 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:30:58.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:58 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:30:59.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:59 smithi012 ceph-mon[31632]: Detected new or changed devices on smithi151 2024-05-17T22:30:59.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:59 smithi012 ceph-mon[31632]: Adjusting osd_memory_target on smithi151 to 8544M 2024-05-17T22:30:59.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:59 smithi012 ceph-mon[31632]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-17T22:30:59.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:30:59 smithi012 ceph-mon[31632]: osdmap e38: 6 total, 6 up, 6 in 2024-05-17T22:30:59.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:59 smithi096 ceph-mon[36065]: Detected new or changed devices on smithi151 2024-05-17T22:30:59.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:59 smithi096 ceph-mon[36065]: Adjusting osd_memory_target on smithi151 to 8544M 2024-05-17T22:30:59.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:59 smithi096 ceph-mon[36065]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-17T22:30:59.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:30:59 smithi096 ceph-mon[36065]: osdmap e38: 6 total, 6 up, 6 in 2024-05-17T22:30:59.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:59 smithi151 ceph-mon[36101]: Detected new or changed devices on smithi151 2024-05-17T22:30:59.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:59 smithi151 ceph-mon[36101]: Adjusting osd_memory_target on smithi151 to 8544M 2024-05-17T22:30:59.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:59 smithi151 ceph-mon[36101]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-17T22:30:59.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:30:59 smithi151 ceph-mon[36101]: osdmap e38: 6 total, 6 up, 6 in 2024-05-17T22:31:00.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:00.067+0000 7fdd0dad8700 1 -- 172.21.15.12:0/2364852640 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd08105810 msgr2=0x7fdd0810c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:31:00.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:00.067+0000 7fdd0dad8700 1 --2- 172.21.15.12:0/2364852640 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd08105810 0x7fdd0810c0c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdd000099c0 tx=0x7fdd00009cd0 comp rx=0 tx=0).stop 2024-05-17T22:31:00.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:00.067+0000 7fdd0dad8700 1 -- 172.21.15.12:0/2364852640 shutdown_connections 2024-05-17T22:31:00.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:00.067+0000 7fdd0dad8700 1 --2- 172.21.15.12:0/2364852640 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd08105810 0x7fdd0810c0c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:31:00.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:00.067+0000 7fdd0dad8700 1 --2- 172.21.15.12:0/2364852640 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd08103d40 0x7fdd08104150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:31:00.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:00.067+0000 7fdd0dad8700 1 --2- 172.21.15.12:0/2364852640 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd08104e70 0x7fdd08105280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:31:00.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:00.067+0000 7fdd0dad8700 1 -- 172.21.15.12:0/2364852640 >> 172.21.15.12:0/2364852640 conn(0x7fdd080ff550 msgr2=0x7fdd08101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:31:00.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:00.067+0000 7fdd0dad8700 1 -- 172.21.15.12:0/2364852640 shutdown_connections 2024-05-17T22:31:00.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:00.067+0000 7fdd0dad8700 1 -- 172.21.15.12:0/2364852640 wait complete. 2024-05-17T22:31:00.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:00.068+0000 7fdd0dad8700 1 Processor -- start 2024-05-17T22:31:00.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:00.068+0000 7fdd0dad8700 1 -- start start 2024-05-17T22:31:00.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:00.068+0000 7fdd0dad8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd08103d40 0x7fdd0819a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:31:00.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:00.068+0000 7fdd0dad8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd08104e70 0x7fdd0819a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:31:00.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:00.068+0000 7fdd0dad8700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd08105810 0x7fdd081a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:31:00.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:00.069+0000 7fdd0dad8700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0810f830 con 0x7fdd08103d40 2024-05-17T22:31:00.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:00.069+0000 7fdd0dad8700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0810f6b0 con 0x7fdd08104e70 2024-05-17T22:31:00.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:00.069+0000 7fdd0dad8700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0810f9b0 con 0x7fdd08105810 2024-05-17T22:31:00.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:00.069+0000 7fdd077fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd08103d40 0x7fdd0819a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:31:00.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:00.069+0000 7fdd077fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd08103d40 0x7fdd0819a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58734/0 (socket says 172.21.15.12:58734) 2024-05-17T22:31:00.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:00.069+0000 7fdd077fe700 1 -- 172.21.15.12:0/3129620241 learned_addr learned my addr 172.21.15.12:0/3129620241 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:31:00.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:00.069+0000 7fdd06ffd700 1 --2- 172.21.15.12:0/3129620241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd08104e70 0x7fdd0819a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:31:00.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:00.069+0000 7fdd07fff700 1 --2- 172.21.15.12:0/3129620241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd08105810 0x7fdd081a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:31:00.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:00.069+0000 7fdd077fe700 1 -- 172.21.15.12:0/3129620241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd08105810 msgr2=0x7fdd081a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:31:00.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:00.069+0000 7fdd077fe700 1 --2- 172.21.15.12:0/3129620241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd08105810 0x7fdd081a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:31:00.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:00.069+0000 7fdd077fe700 1 -- 172.21.15.12:0/3129620241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd08104e70 msgr2=0x7fdd0819a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:31:00.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:00.069+0000 7fdd077fe700 1 --2- 172.21.15.12:0/3129620241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd08104e70 0x7fdd0819a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:31:00.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:00.069+0000 7fdd077fe700 1 -- 172.21.15.12:0/3129620241 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd081a1dd0 con 0x7fdd08103d40 2024-05-17T22:31:00.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:00.069+0000 7fdd077fe700 1 --2- 172.21.15.12:0/3129620241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd08103d40 0x7fdd0819a300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8007df0 tx=0x7fdcf800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:31:00.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:00.069+0000 7fdd04ff9700 1 -- 172.21.15.12:0/3129620241 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8016020 con 0x7fdd08103d40 2024-05-17T22:31:00.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:00.069+0000 7fdd04ff9700 1 -- 172.21.15.12:0/3129620241 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcf800ad40 con 0x7fdd08103d40 2024-05-17T22:31:00.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:00.070+0000 7fdd0dad8700 1 -- 172.21.15.12:0/3129620241 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd081a20c0 con 0x7fdd08103d40 2024-05-17T22:31:00.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:00.070+0000 7fdd04ff9700 1 -- 172.21.15.12:0/3129620241 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8011700 con 0x7fdd08103d40 2024-05-17T22:31:00.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:00.070+0000 7fdd0dad8700 1 -- 172.21.15.12:0/3129620241 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd081a2650 con 0x7fdd08103d40 2024-05-17T22:31:00.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:00.071+0000 7fdd0dad8700 1 -- 172.21.15.12:0/3129620241 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd08104150 con 0x7fdd08103d40 2024-05-17T22:31:00.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:00.072+0000 7fdd04ff9700 1 -- 172.21.15.12:0/3129620241 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fdcf8019020 con 0x7fdd08103d40 2024-05-17T22:31:00.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:00.073+0000 7fdd04ff9700 1 --2- 172.21.15.12:0/3129620241 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fdcf40b2900 0x7fdcf40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:31:00.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:00.074+0000 7fdd06ffd700 1 --2- 172.21.15.12:0/3129620241 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fdcf40b2900 0x7fdcf40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:31:00.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:00.074+0000 7fdd04ff9700 1 -- 172.21.15.12:0/3129620241 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4533+0+0 (secure 0 0 0) 0x7fdcf80e1500 con 0x7fdd08103d40 2024-05-17T22:31:00.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:00.074+0000 7fdd06ffd700 1 --2- 172.21.15.12:0/3129620241 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fdcf40b2900 0x7fdcf40b4e10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdd0819b3b0 tx=0x7fdcf0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:31:00.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:00.077+0000 7fdd04ff9700 1 -- 172.21.15.12:0/3129620241 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdcf8020070 con 0x7fdd08103d40 2024-05-17T22:31:00.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:00.301+0000 7fdd0dad8700 1 -- 172.21.15.12:0/3129620241 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fdd08064720 con 0x7fdcf40b2900 2024-05-17T22:31:00.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:00 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T22:31:00.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:00 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T22:31:00.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:00 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:31:00.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:00 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T22:31:00.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:00 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T22:31:00.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:00 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:31:00.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T22:31:00.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T22:31:00.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:31:01.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:01 smithi096 ceph-mon[36065]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:31:01.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:01 smithi096 ceph-mon[36065]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-17T22:31:01.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:01 smithi012 ceph-mon[31632]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:31:01.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:01 smithi012 ceph-mon[31632]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-17T22:31:01.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:01 smithi151 ceph-mon[36101]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:31:01.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:01 smithi151 ceph-mon[36101]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-17T22:31:03.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:03 smithi096 ceph-mon[36065]: pgmap v142: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-17T22:31:03.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:03 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/3252459936' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7ece35c-efd1-40df-95da-05a0cc147372"}]: dispatch 2024-05-17T22:31:03.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:03 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/3252459936' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b7ece35c-efd1-40df-95da-05a0cc147372"}]': finished 2024-05-17T22:31:03.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:03 smithi096 ceph-mon[36065]: osdmap e39: 7 total, 6 up, 7 in 2024-05-17T22:31:03.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:03 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T22:31:03.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:03 smithi012 ceph-mon[31632]: pgmap v142: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-17T22:31:03.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:03 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/3252459936' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7ece35c-efd1-40df-95da-05a0cc147372"}]: dispatch 2024-05-17T22:31:03.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:03 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/3252459936' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b7ece35c-efd1-40df-95da-05a0cc147372"}]': finished 2024-05-17T22:31:03.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:03 smithi012 ceph-mon[31632]: osdmap e39: 7 total, 6 up, 7 in 2024-05-17T22:31:03.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:03 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T22:31:03.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:03 smithi151 ceph-mon[36101]: pgmap v142: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-17T22:31:03.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:03 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/3252459936' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7ece35c-efd1-40df-95da-05a0cc147372"}]: dispatch 2024-05-17T22:31:03.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:03 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/3252459936' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b7ece35c-efd1-40df-95da-05a0cc147372"}]': finished 2024-05-17T22:31:03.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:03 smithi151 ceph-mon[36101]: osdmap e39: 7 total, 6 up, 7 in 2024-05-17T22:31:03.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:03 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T22:31:04.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:04 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/2558639462' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T22:31:04.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:04 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/2558639462' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T22:31:04.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:04 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/2558639462' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T22:31:05.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:05 smithi096 ceph-mon[36065]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-05-17T22:31:05.815 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:05 smithi151 ceph-mon[36101]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-05-17T22:31:05.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:05 smithi012 ceph-mon[31632]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-05-17T22:31:07.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:07 smithi096 ceph-mon[36065]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-17T22:31:07.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:07 smithi012 ceph-mon[31632]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-17T22:31:07.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:07 smithi151 ceph-mon[36101]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-17T22:31:09.639 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:09 smithi012 ceph-mon[31632]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T22:31:09.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:09 smithi096 ceph-mon[36065]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T22:31:09.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:09 smithi151 ceph-mon[36101]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T22:31:11.661 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:11 smithi012 ceph-mon[31632]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T22:31:11.661 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:11 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-17T22:31:11.662 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:11 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:31:11.662 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:11 smithi012 ceph-mon[31632]: Deploying daemon osd.6 on smithi012 2024-05-17T22:31:11.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:11 smithi096 ceph-mon[36065]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T22:31:11.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:11 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-17T22:31:11.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:11 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:31:11.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:11 smithi096 ceph-mon[36065]: Deploying daemon osd.6 on smithi012 2024-05-17T22:31:11.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:11 smithi151 ceph-mon[36101]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T22:31:11.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:11 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-17T22:31:11.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:11 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:31:11.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:11 smithi151 ceph-mon[36101]: Deploying daemon osd.6 on smithi012 2024-05-17T22:31:13.790 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:13 smithi012 ceph-mon[31632]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T22:31:13.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:13 smithi096 ceph-mon[36065]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T22:31:13.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:13 smithi151 ceph-mon[36101]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T22:31:15.746 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:15 smithi012 ceph-mon[31632]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-17T22:31:15.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:15 smithi096 ceph-mon[36065]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-17T22:31:15.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:15 smithi151 ceph-mon[36101]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-17T22:31:17.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:17 smithi151 ceph-mon[36101]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-17T22:31:17.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:17 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:17.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:17 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:17.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:17 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:31:18.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:17 smithi096 ceph-mon[36065]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-17T22:31:18.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:17 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:18.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:17 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:18.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:17 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:31:18.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:17 smithi012 ceph-mon[31632]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-17T22:31:18.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:17 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:18.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:17 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:18.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:17 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:31:19.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:19 smithi012 ceph-mon[31632]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-17T22:31:19.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:19 smithi151 ceph-mon[36101]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-17T22:31:20.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:19 smithi096 ceph-mon[36065]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-17T22:31:20.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:20.822+0000 7fdd04ff9700 1 -- 172.21.15.12:0/3129620241 <== mgr.14152 v2:172.21.15.12:6800/1388514047 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdd08064720 con 0x7fdcf40b2900 2024-05-17T22:31:20.825 INFO:teuthology.orchestra.run.smithi012.stdout:Created osd(s) 6 on host 'smithi012' 2024-05-17T22:31:20.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:20.825+0000 7fdd0dad8700 1 -- 172.21.15.12:0/3129620241 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fdcf40b2900 msgr2=0x7fdcf40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:31:20.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:20.825+0000 7fdd0dad8700 1 --2- 172.21.15.12:0/3129620241 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fdcf40b2900 0x7fdcf40b4e10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdd0819b3b0 tx=0x7fdcf0009190 comp rx=0 tx=0).stop 2024-05-17T22:31:20.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:20.825+0000 7fdd0dad8700 1 -- 172.21.15.12:0/3129620241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd08103d40 msgr2=0x7fdd0819a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:31:20.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:20.825+0000 7fdd0dad8700 1 --2- 172.21.15.12:0/3129620241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd08103d40 0x7fdd0819a300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8007df0 tx=0x7fdcf800a340 comp rx=0 tx=0).stop 2024-05-17T22:31:20.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:20.825+0000 7fdd0dad8700 1 -- 172.21.15.12:0/3129620241 shutdown_connections 2024-05-17T22:31:20.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:20.825+0000 7fdd0dad8700 1 --2- 172.21.15.12:0/3129620241 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fdcf40b2900 0x7fdcf40b4e10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:31:20.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:20.825+0000 7fdd0dad8700 1 --2- 172.21.15.12:0/3129620241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd08105810 0x7fdd081a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:31:20.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:20.825+0000 7fdd0dad8700 1 --2- 172.21.15.12:0/3129620241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd08103d40 0x7fdd0819a300 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:31:20.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:20.825+0000 7fdd0dad8700 1 --2- 172.21.15.12:0/3129620241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd08104e70 0x7fdd0819a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:31:20.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:20.825+0000 7fdd0dad8700 1 -- 172.21.15.12:0/3129620241 >> 172.21.15.12:0/3129620241 conn(0x7fdd080ff550 msgr2=0x7fdd081080a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:31:20.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:20.825+0000 7fdd0dad8700 1 -- 172.21.15.12:0/3129620241 shutdown_connections 2024-05-17T22:31:20.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:31:20.825+0000 7fdd0dad8700 1 -- 172.21.15.12:0/3129620241 wait complete. 2024-05-17T22:31:20.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:20 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:20.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:20 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:20.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:20 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:31:20.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:20 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:31:20.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:20 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:20.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:20 smithi151 ceph-mon[36101]: from='osd.6 [v2:172.21.15.12:6818/3112661724,v1:172.21.15.12:6819/3112661724]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-17T22:31:21.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:20 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:21.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:20 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:21.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:20 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:31:21.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:20 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:31:21.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:20 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:21.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:20 smithi096 ceph-mon[36065]: from='osd.6 [v2:172.21.15.12:6818/3112661724,v1:172.21.15.12:6819/3112661724]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-17T22:31:21.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:20 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:21.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:20 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:21.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:20 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:31:21.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:20 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:31:21.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:20 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:21.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:20 smithi012 ceph-mon[31632]: from='osd.6 [v2:172.21.15.12:6818/3112661724,v1:172.21.15.12:6819/3112661724]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-17T22:31:21.640 DEBUG:teuthology.orchestra.run.smithi012:osd.6> sudo journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@osd.6.service 2024-05-17T22:31:21.643 INFO:tasks.cephadm:Deploying osd.7 on smithi096 with /dev/vg_nvme/lv_2... 2024-05-17T22:31:21.643 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-17T22:31:21.874 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:21 smithi012 ceph-mon[31632]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-17T22:31:21.874 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:21 smithi012 ceph-mon[31632]: from='osd.6 [v2:172.21.15.12:6818/3112661724,v1:172.21.15.12:6819/3112661724]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-17T22:31:21.874 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:21 smithi012 ceph-mon[31632]: osdmap e40: 7 total, 6 up, 7 in 2024-05-17T22:31:21.874 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:21 smithi012 ceph-mon[31632]: from='osd.6 [v2:172.21.15.12:6818/3112661724,v1:172.21.15.12:6819/3112661724]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-05-17T22:31:21.874 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:21 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T22:31:21.874 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:21 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:21.874 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:21 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:21.874 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:21 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:31:21.874 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:31:21 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:31:21.742+0000 7f460ca5e700 -1 osd.6 0 waiting for initial osdmap 2024-05-17T22:31:21.875 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:31:21 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:31:21.754+0000 7f4608813700 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T22:31:21.888 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.b/config 2024-05-17T22:31:22.009 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:21 smithi096 ceph-mon[36065]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-17T22:31:22.009 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:21 smithi096 ceph-mon[36065]: from='osd.6 [v2:172.21.15.12:6818/3112661724,v1:172.21.15.12:6819/3112661724]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-17T22:31:22.009 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:21 smithi096 ceph-mon[36065]: osdmap e40: 7 total, 6 up, 7 in 2024-05-17T22:31:22.009 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:21 smithi096 ceph-mon[36065]: from='osd.6 [v2:172.21.15.12:6818/3112661724,v1:172.21.15.12:6819/3112661724]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-05-17T22:31:22.009 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:21 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T22:31:22.009 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:21 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:22.009 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:21 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:22.009 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:21 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:31:22.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:21 smithi151 ceph-mon[36101]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-17T22:31:22.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:21 smithi151 ceph-mon[36101]: from='osd.6 [v2:172.21.15.12:6818/3112661724,v1:172.21.15.12:6819/3112661724]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-17T22:31:22.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:21 smithi151 ceph-mon[36101]: osdmap e40: 7 total, 6 up, 7 in 2024-05-17T22:31:22.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:21 smithi151 ceph-mon[36101]: from='osd.6 [v2:172.21.15.12:6818/3112661724,v1:172.21.15.12:6819/3112661724]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-05-17T22:31:22.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:21 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T22:31:22.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:21 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:22.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:21 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:22.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:21 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:31:23.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:22 smithi096 ceph-mon[36065]: from='osd.6 [v2:172.21.15.12:6818/3112661724,v1:172.21.15.12:6819/3112661724]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-05-17T22:31:23.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:22 smithi096 ceph-mon[36065]: osdmap e41: 7 total, 6 up, 7 in 2024-05-17T22:31:23.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:22 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T22:31:23.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:22 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T22:31:23.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:22 smithi012 ceph-mon[31632]: from='osd.6 [v2:172.21.15.12:6818/3112661724,v1:172.21.15.12:6819/3112661724]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-05-17T22:31:23.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:22 smithi012 ceph-mon[31632]: osdmap e41: 7 total, 6 up, 7 in 2024-05-17T22:31:23.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:22 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T22:31:23.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:22 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T22:31:23.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:22 smithi151 ceph-mon[36101]: from='osd.6 [v2:172.21.15.12:6818/3112661724,v1:172.21.15.12:6819/3112661724]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-05-17T22:31:23.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:22 smithi151 ceph-mon[36101]: osdmap e41: 7 total, 6 up, 7 in 2024-05-17T22:31:23.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:22 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T22:31:23.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:22 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T22:31:24.010 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:23 smithi012 ceph-mon[31632]: purged_snaps scrub starts 2024-05-17T22:31:24.010 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:23 smithi012 ceph-mon[31632]: purged_snaps scrub ok 2024-05-17T22:31:24.010 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:23 smithi012 ceph-mon[31632]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-17T22:31:24.010 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:23 smithi012 ceph-mon[31632]: osd.6 [v2:172.21.15.12:6818/3112661724,v1:172.21.15.12:6819/3112661724] boot 2024-05-17T22:31:24.010 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:23 smithi012 ceph-mon[31632]: osdmap e42: 7 total, 7 up, 7 in 2024-05-17T22:31:24.010 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:23 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T22:31:24.021 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-17T22:31:24.021 INFO:teuthology.orchestra.run.smithi096.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-17T22:31:24.021 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10+0 records in 2024-05-17T22:31:24.021 INFO:teuthology.orchestra.run.smithi096.stderr:10+0 records out 2024-05-17T22:31:24.021 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126774 s, 827 MB/s 2024-05-17T22:31:24.021 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping successful for: 2024-05-17T22:31:24.021 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:23 smithi096 ceph-mon[36065]: purged_snaps scrub starts 2024-05-17T22:31:24.021 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:23 smithi096 ceph-mon[36065]: purged_snaps scrub ok 2024-05-17T22:31:24.021 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:23 smithi096 ceph-mon[36065]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-17T22:31:24.021 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:23 smithi096 ceph-mon[36065]: osd.6 [v2:172.21.15.12:6818/3112661724,v1:172.21.15.12:6819/3112661724] boot 2024-05-17T22:31:24.021 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:23 smithi096 ceph-mon[36065]: osdmap e42: 7 total, 7 up, 7 in 2024-05-17T22:31:24.021 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:23 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T22:31:24.141 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:23 smithi151 ceph-mon[36101]: purged_snaps scrub starts 2024-05-17T22:31:24.141 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:23 smithi151 ceph-mon[36101]: purged_snaps scrub ok 2024-05-17T22:31:24.141 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:23 smithi151 ceph-mon[36101]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-17T22:31:24.141 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:23 smithi151 ceph-mon[36101]: osd.6 [v2:172.21.15.12:6818/3112661724,v1:172.21.15.12:6819/3112661724] boot 2024-05-17T22:31:24.142 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:23 smithi151 ceph-mon[36101]: osdmap e42: 7 total, 7 up, 7 in 2024-05-17T22:31:24.142 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:23 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T22:31:24.678 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi096:vg_nvme/lv_2 2024-05-17T22:31:24.934 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.b/config 2024-05-17T22:31:25.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:24 smithi096 ceph-mon[36065]: osdmap e43: 7 total, 7 up, 7 in 2024-05-17T22:31:25.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:24 smithi012 ceph-mon[31632]: osdmap e43: 7 total, 7 up, 7 in 2024-05-17T22:31:25.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:24 smithi151 ceph-mon[36101]: osdmap e43: 7 total, 7 up, 7 in 2024-05-17T22:31:26.031 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:25 smithi096 ceph-mon[36065]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-17T22:31:26.031 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:25 smithi096 ceph-mon[36065]: Detected new or changed devices on smithi012 2024-05-17T22:31:26.031 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:25 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:26.031 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:25 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:26.032 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:25 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:31:26.032 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:25 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:31:26.032 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:25 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:31:26.032 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:25 smithi096 ceph-mon[36065]: Adjusting osd_memory_target on smithi012 to 5697M 2024-05-17T22:31:26.032 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:25 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:26.032 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:25 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:31:26.032 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:25 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:31:26.032 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:25 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:26.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:25 smithi012 ceph-mon[31632]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-17T22:31:26.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:25 smithi012 ceph-mon[31632]: Detected new or changed devices on smithi012 2024-05-17T22:31:26.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:25 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:26.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:25 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:26.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:25 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:31:26.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:25 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:31:26.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:25 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:31:26.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:25 smithi012 ceph-mon[31632]: Adjusting osd_memory_target on smithi012 to 5697M 2024-05-17T22:31:26.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:25 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:26.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:25 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:31:26.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:25 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:31:26.093 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:25 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:26.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:25 smithi151 ceph-mon[36101]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-17T22:31:26.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:25 smithi151 ceph-mon[36101]: Detected new or changed devices on smithi012 2024-05-17T22:31:26.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:25 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:26.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:25 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:26.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:25 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:31:26.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:25 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:31:26.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:25 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:31:26.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:25 smithi151 ceph-mon[36101]: Adjusting osd_memory_target on smithi012 to 5697M 2024-05-17T22:31:26.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:25 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:26.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:25 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:31:26.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:25 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:31:26.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:25 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:26.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:26.548+0000 7f86fe9ba700 1 -- 172.21.15.96:0/2218872168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86f8103cc0 msgr2=0x7f86f81040d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:31:26.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:26.548+0000 7f86fe9ba700 1 --2- 172.21.15.96:0/2218872168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86f8103cc0 0x7f86f81040d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f86ec009990 tx=0x7f86ec009ca0 comp rx=0 tx=0).stop 2024-05-17T22:31:26.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:26.548+0000 7f86fe9ba700 1 -- 172.21.15.96:0/2218872168 shutdown_connections 2024-05-17T22:31:26.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:26.548+0000 7f86fe9ba700 1 --2- 172.21.15.96:0/2218872168 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86f8104660 0x7f86f810af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:31:26.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:26.548+0000 7f86fe9ba700 1 --2- 172.21.15.96:0/2218872168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86f8102b90 0x7f86f8102fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:31:26.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:26.548+0000 7f86fe9ba700 1 --2- 172.21.15.96:0/2218872168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86f8103cc0 0x7f86f81040d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:31:26.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:26.548+0000 7f86fe9ba700 1 -- 172.21.15.96:0/2218872168 >> 172.21.15.96:0/2218872168 conn(0x7f86f80fe3c0 msgr2=0x7f86f8100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:31:26.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:26.548+0000 7f86fe9ba700 1 -- 172.21.15.96:0/2218872168 shutdown_connections 2024-05-17T22:31:26.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:26.548+0000 7f86fe9ba700 1 -- 172.21.15.96:0/2218872168 wait complete. 2024-05-17T22:31:26.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:26.549+0000 7f86fe9ba700 1 Processor -- start 2024-05-17T22:31:26.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:26.549+0000 7f86fe9ba700 1 -- start start 2024-05-17T22:31:26.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:26.549+0000 7f86fe9ba700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86f8102b90 0x7f86f806caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:31:26.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:26.549+0000 7f86fe9ba700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86f8103cc0 0x7f86f819f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:31:26.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:26.549+0000 7f86fe9ba700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86f8104660 0x7f86f81a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:31:26.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:26.549+0000 7f86fe9ba700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f86f810e600 con 0x7f86f8103cc0 2024-05-17T22:31:26.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:26.549+0000 7f86fe9ba700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f86f810e480 con 0x7f86f8104660 2024-05-17T22:31:26.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:26.549+0000 7f86fe9ba700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f86f810e780 con 0x7f86f8102b90 2024-05-17T22:31:26.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:26.549+0000 7f86fcf57700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86f8104660 0x7f86f81a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:31:26.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:26.549+0000 7f86fcf57700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86f8104660 0x7f86f81a15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50250/0 (socket says 172.21.15.96:50250) 2024-05-17T22:31:26.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:26.549+0000 7f86fcf57700 1 -- 172.21.15.96:0/2415064554 learned_addr learned my addr 172.21.15.96:0/2415064554 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-05-17T22:31:26.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:26.549+0000 7f86f77fe700 1 --2- 172.21.15.96:0/2415064554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86f8103cc0 0x7f86f819f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:31:26.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:26.549+0000 7f86f7fff700 1 --2- 172.21.15.96:0/2415064554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86f8102b90 0x7f86f806caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:31:26.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:26.550+0000 7f86fcf57700 1 -- 172.21.15.96:0/2415064554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86f8102b90 msgr2=0x7f86f806caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:31:26.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:26.550+0000 7f86fcf57700 1 --2- 172.21.15.96:0/2415064554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86f8102b90 0x7f86f806caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:31:26.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:26.550+0000 7f86fcf57700 1 -- 172.21.15.96:0/2415064554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86f8103cc0 msgr2=0x7f86f819f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:31:26.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:26.550+0000 7f86fcf57700 1 --2- 172.21.15.96:0/2415064554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86f8103cc0 0x7f86f819f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:31:26.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:26.550+0000 7f86fcf57700 1 -- 172.21.15.96:0/2415064554 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f81a1de0 con 0x7f86f8104660 2024-05-17T22:31:26.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:26.550+0000 7f86f7fff700 1 --2- 172.21.15.96:0/2415064554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86f8102b90 0x7f86f806caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:31:26.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:26.550+0000 7f86fcf57700 1 --2- 172.21.15.96:0/2415064554 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86f8104660 0x7f86f81a15b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f86f0007ee0 tx=0x7f86f000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:31:26.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:26.550+0000 7f86f57fa700 1 -- 172.21.15.96:0/2415064554 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f000dbd0 con 0x7f86f8104660 2024-05-17T22:31:26.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:26.550+0000 7f86fe9ba700 1 -- 172.21.15.96:0/2415064554 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86f81a20d0 con 0x7f86f8104660 2024-05-17T22:31:26.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:26.550+0000 7f86f57fa700 1 -- 172.21.15.96:0/2415064554 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86f000dd70 con 0x7f86f8104660 2024-05-17T22:31:26.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:26.550+0000 7f86f57fa700 1 -- 172.21.15.96:0/2415064554 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f0005990 con 0x7f86f8104660 2024-05-17T22:31:26.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:26.550+0000 7f86fe9ba700 1 -- 172.21.15.96:0/2415064554 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86f81a25c0 con 0x7f86f8104660 2024-05-17T22:31:26.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:26.551+0000 7f86fe9ba700 1 -- 172.21.15.96:0/2415064554 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86f8102fa0 con 0x7f86f8104660 2024-05-17T22:31:26.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:26.553+0000 7f86f57fa700 1 -- 172.21.15.96:0/2415064554 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f86f001a020 con 0x7f86f8104660 2024-05-17T22:31:26.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:26.554+0000 7f86f57fa700 1 --2- 172.21.15.96:0/2415064554 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f86e00b2880 0x7f86e00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:31:26.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:26.554+0000 7f86f57fa700 1 -- 172.21.15.96:0/2415064554 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4965+0+0 (secure 0 0 0) 0x7f86f00e1390 con 0x7f86f8104660 2024-05-17T22:31:26.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:26.554+0000 7f86f7fff700 1 --2- 172.21.15.96:0/2415064554 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f86e00b2880 0x7f86e00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:31:26.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:26.555+0000 7f86f7fff700 1 --2- 172.21.15.96:0/2415064554 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f86e00b2880 0x7f86e00b4d90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f86e8005950 tx=0x7f86e80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:31:26.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:26.557+0000 7f86f57fa700 1 -- 172.21.15.96:0/2415064554 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f86f0010070 con 0x7f86f8104660 2024-05-17T22:31:26.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:26.768+0000 7f86fe9ba700 1 -- 172.21.15.96:0/2415064554 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f86f8064720 con 0x7f86e00b2880 2024-05-17T22:31:28.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:27 smithi096 ceph-mon[36065]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-17T22:31:28.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:27 smithi096 ceph-mon[36065]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:31:28.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:27 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T22:31:28.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:27 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T22:31:28.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:27 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:31:28.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:27 smithi012 ceph-mon[31632]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-17T22:31:28.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:27 smithi012 ceph-mon[31632]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:31:28.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:27 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T22:31:28.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:27 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T22:31:28.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:27 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:31:28.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:27 smithi151 ceph-mon[36101]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-17T22:31:28.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:27 smithi151 ceph-mon[36101]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:31:28.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:27 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T22:31:28.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:27 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T22:31:28.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:27 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:31:30.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:29 smithi096 ceph-mon[36065]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-17T22:31:30.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:29 smithi096 ceph-mon[36065]: from='client.? 172.21.15.96:0/184948009' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a65bf30-daf5-4fba-b67e-8e513d88a971"}]: dispatch 2024-05-17T22:31:30.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:29 smithi096 ceph-mon[36065]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a65bf30-daf5-4fba-b67e-8e513d88a971"}]: dispatch 2024-05-17T22:31:30.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:29 smithi096 ceph-mon[36065]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6a65bf30-daf5-4fba-b67e-8e513d88a971"}]': finished 2024-05-17T22:31:30.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:29 smithi096 ceph-mon[36065]: osdmap e44: 8 total, 7 up, 8 in 2024-05-17T22:31:30.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:29 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T22:31:30.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:29 smithi012 ceph-mon[31632]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-17T22:31:30.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:29 smithi012 ceph-mon[31632]: from='client.? 172.21.15.96:0/184948009' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a65bf30-daf5-4fba-b67e-8e513d88a971"}]: dispatch 2024-05-17T22:31:30.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:29 smithi012 ceph-mon[31632]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a65bf30-daf5-4fba-b67e-8e513d88a971"}]: dispatch 2024-05-17T22:31:30.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:29 smithi012 ceph-mon[31632]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6a65bf30-daf5-4fba-b67e-8e513d88a971"}]': finished 2024-05-17T22:31:30.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:29 smithi012 ceph-mon[31632]: osdmap e44: 8 total, 7 up, 8 in 2024-05-17T22:31:30.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:29 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T22:31:30.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:29 smithi151 ceph-mon[36101]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-17T22:31:30.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:29 smithi151 ceph-mon[36101]: from='client.? 172.21.15.96:0/184948009' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a65bf30-daf5-4fba-b67e-8e513d88a971"}]: dispatch 2024-05-17T22:31:30.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:29 smithi151 ceph-mon[36101]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a65bf30-daf5-4fba-b67e-8e513d88a971"}]: dispatch 2024-05-17T22:31:30.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:29 smithi151 ceph-mon[36101]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6a65bf30-daf5-4fba-b67e-8e513d88a971"}]': finished 2024-05-17T22:31:30.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:29 smithi151 ceph-mon[36101]: osdmap e44: 8 total, 7 up, 8 in 2024-05-17T22:31:30.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:29 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T22:31:31.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:30 smithi096 ceph-mon[36065]: from='client.? 172.21.15.96:0/1827734144' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T22:31:31.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:30 smithi012 ceph-mon[31632]: from='client.? 172.21.15.96:0/1827734144' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T22:31:31.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:30 smithi151 ceph-mon[36101]: from='client.? 172.21.15.96:0/1827734144' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T22:31:32.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:31 smithi096 ceph-mon[36065]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-17T22:31:32.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:31 smithi012 ceph-mon[31632]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-17T22:31:32.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:31 smithi151 ceph-mon[36101]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-17T22:31:33.900 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:33 smithi096 ceph-mon[36065]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-17T22:31:34.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:33 smithi012 ceph-mon[31632]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-17T22:31:34.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:33 smithi151 ceph-mon[36101]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-17T22:31:36.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:35 smithi096 ceph-mon[36065]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-17T22:31:36.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:35 smithi012 ceph-mon[31632]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-17T22:31:36.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:35 smithi151 ceph-mon[36101]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-17T22:31:38.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:37 smithi096 ceph-mon[36065]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-17T22:31:38.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:37 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-17T22:31:38.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:37 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:31:38.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:37 smithi096 ceph-mon[36065]: Deploying daemon osd.7 on smithi096 2024-05-17T22:31:38.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:37 smithi012 ceph-mon[31632]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-17T22:31:38.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:37 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-17T22:31:38.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:37 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:31:38.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:37 smithi012 ceph-mon[31632]: Deploying daemon osd.7 on smithi096 2024-05-17T22:31:38.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:37 smithi151 ceph-mon[36101]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-17T22:31:38.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:37 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-17T22:31:38.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:37 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:31:38.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:37 smithi151 ceph-mon[36101]: Deploying daemon osd.7 on smithi096 2024-05-17T22:31:39.936 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:39 smithi096 ceph-mon[36065]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-17T22:31:40.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:39 smithi012 ceph-mon[31632]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-17T22:31:40.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:39 smithi151 ceph-mon[36101]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-17T22:31:41.993 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:41 smithi096 ceph-mon[36065]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-17T22:31:42.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:41 smithi012 ceph-mon[31632]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-17T22:31:42.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:41 smithi151 ceph-mon[36101]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-17T22:31:43.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:43 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:43.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:43 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:43.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:43 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:31:43.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:43 smithi151 ceph-mon[36101]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-17T22:31:43.520 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:43 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:43.520 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:43 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:43.520 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:43 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:31:43.520 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:43 smithi096 ceph-mon[36065]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-17T22:31:43.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:43 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:43.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:43 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:43.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:43 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:31:43.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:43 smithi012 ceph-mon[31632]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-17T22:31:45.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:45 smithi096 ceph-mon[36065]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-17T22:31:45.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:45 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:45.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:45 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:45.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:45 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:31:45.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:45 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:31:45.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:45 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:45.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:45 smithi012 ceph-mon[31632]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-17T22:31:45.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:45 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:45.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:45 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:45.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:45 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:31:45.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:45 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:31:45.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:45 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:45.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:45.934+0000 7f86f57fa700 1 -- 172.21.15.96:0/2415064554 <== mgr.14152 v2:172.21.15.12:6800/1388514047 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f86f8064720 con 0x7f86e00b2880 2024-05-17T22:31:45.937 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 7 on host 'smithi096' 2024-05-17T22:31:45.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:45.937+0000 7f86fe9ba700 1 -- 172.21.15.96:0/2415064554 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f86e00b2880 msgr2=0x7f86e00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:31:45.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:45.937+0000 7f86fe9ba700 1 --2- 172.21.15.96:0/2415064554 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f86e00b2880 0x7f86e00b4d90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f86e8005950 tx=0x7f86e80058e0 comp rx=0 tx=0).stop 2024-05-17T22:31:45.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:45.937+0000 7f86fe9ba700 1 -- 172.21.15.96:0/2415064554 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86f8104660 msgr2=0x7f86f81a15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:31:45.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:45.937+0000 7f86fe9ba700 1 --2- 172.21.15.96:0/2415064554 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86f8104660 0x7f86f81a15b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f86f0007ee0 tx=0x7f86f000d3b0 comp rx=0 tx=0).stop 2024-05-17T22:31:45.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:45.937+0000 7f86fe9ba700 1 -- 172.21.15.96:0/2415064554 shutdown_connections 2024-05-17T22:31:45.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:45.937+0000 7f86fe9ba700 1 --2- 172.21.15.96:0/2415064554 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f86e00b2880 0x7f86e00b4d90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:31:45.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:45.937+0000 7f86fe9ba700 1 --2- 172.21.15.96:0/2415064554 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86f8104660 0x7f86f81a15b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:31:45.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:45.937+0000 7f86fe9ba700 1 --2- 172.21.15.96:0/2415064554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86f8102b90 0x7f86f806caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:31:45.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:45.937+0000 7f86fe9ba700 1 --2- 172.21.15.96:0/2415064554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86f8103cc0 0x7f86f819f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:31:45.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:45.937+0000 7f86fe9ba700 1 -- 172.21.15.96:0/2415064554 >> 172.21.15.96:0/2415064554 conn(0x7f86f80fe3c0 msgr2=0x7f86f8106ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:31:45.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:45.937+0000 7f86fe9ba700 1 -- 172.21.15.96:0/2415064554 shutdown_connections 2024-05-17T22:31:45.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:31:45.937+0000 7f86fe9ba700 1 -- 172.21.15.96:0/2415064554 wait complete. 2024-05-17T22:31:45.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:45 smithi151 ceph-mon[36101]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-17T22:31:45.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:45 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:45.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:45 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:45.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:45 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:31:45.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:45 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:31:45.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:45 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:46.645 DEBUG:teuthology.orchestra.run.smithi096:osd.7> sudo journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@osd.7.service 2024-05-17T22:31:46.647 INFO:tasks.cephadm:Deploying osd.8 on smithi151 with /dev/vg_nvme/lv_2... 2024-05-17T22:31:46.648 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-17T22:31:46.895 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T22:31:46.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:46 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:46.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:46 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:46.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:46 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:31:46.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:46 smithi151 ceph-mon[36101]: from='osd.7 [v2:172.21.15.96:6816/2449989110,v1:172.21.15.96:6817/2449989110]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-17T22:31:46.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:46 smithi151 ceph-mon[36101]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-17T22:31:47.140 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:46 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:47.140 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:46 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:47.140 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:46 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:31:47.140 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:46 smithi096 ceph-mon[36065]: from='osd.7 [v2:172.21.15.96:6816/2449989110,v1:172.21.15.96:6817/2449989110]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-17T22:31:47.141 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:46 smithi096 ceph-mon[36065]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-17T22:31:47.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:46 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:47.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:46 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:47.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:46 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:31:47.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:46 smithi012 ceph-mon[31632]: from='osd.7 [v2:172.21.15.96:6816/2449989110,v1:172.21.15.96:6817/2449989110]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-17T22:31:47.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:46 smithi012 ceph-mon[31632]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-17T22:31:47.962 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:47 smithi096 ceph-mon[36065]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-17T22:31:47.962 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:47 smithi096 ceph-mon[36065]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-17T22:31:47.962 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:47 smithi096 ceph-mon[36065]: osdmap e45: 8 total, 7 up, 8 in 2024-05-17T22:31:47.962 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:47 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T22:31:47.962 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:47 smithi096 ceph-mon[36065]: from='osd.7 [v2:172.21.15.96:6816/2449989110,v1:172.21.15.96:6817/2449989110]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-17T22:31:47.962 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:47 smithi096 ceph-mon[36065]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-17T22:31:47.962 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:31:47 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:31:47.907+0000 7fb6ee19d700 -1 osd.7 0 waiting for initial osdmap 2024-05-17T22:31:47.963 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:31:47 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:31:47.915+0000 7fb6e9f52700 -1 osd.7 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T22:31:48.183 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:47 smithi151 ceph-mon[36101]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-17T22:31:48.183 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:47 smithi151 ceph-mon[36101]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-17T22:31:48.183 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:47 smithi151 ceph-mon[36101]: osdmap e45: 8 total, 7 up, 8 in 2024-05-17T22:31:48.183 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:47 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T22:31:48.183 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:47 smithi151 ceph-mon[36101]: from='osd.7 [v2:172.21.15.96:6816/2449989110,v1:172.21.15.96:6817/2449989110]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-17T22:31:48.183 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:47 smithi151 ceph-mon[36101]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-17T22:31:48.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:47 smithi012 ceph-mon[31632]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-17T22:31:48.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:47 smithi012 ceph-mon[31632]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-17T22:31:48.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:47 smithi012 ceph-mon[31632]: osdmap e45: 8 total, 7 up, 8 in 2024-05-17T22:31:48.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:47 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T22:31:48.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:47 smithi012 ceph-mon[31632]: from='osd.7 [v2:172.21.15.96:6816/2449989110,v1:172.21.15.96:6817/2449989110]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-17T22:31:48.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:47 smithi012 ceph-mon[31632]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-17T22:31:48.902 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-17T22:31:48.902 INFO:teuthology.orchestra.run.smithi151.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-17T22:31:48.903 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10+0 records in 2024-05-17T22:31:48.903 INFO:teuthology.orchestra.run.smithi151.stderr:10+0 records out 2024-05-17T22:31:48.903 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0122844 s, 854 MB/s 2024-05-17T22:31:48.903 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping successful for: 2024-05-17T22:31:49.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:48 smithi096 ceph-mon[36065]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-05-17T22:31:49.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:48 smithi096 ceph-mon[36065]: osdmap e46: 8 total, 7 up, 8 in 2024-05-17T22:31:49.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:48 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T22:31:49.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:48 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T22:31:49.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:48 smithi096 ceph-mon[36065]: osd.7 [v2:172.21.15.96:6816/2449989110,v1:172.21.15.96:6817/2449989110] boot 2024-05-17T22:31:49.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:48 smithi096 ceph-mon[36065]: osdmap e47: 8 total, 8 up, 8 in 2024-05-17T22:31:49.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:48 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T22:31:49.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:48 smithi151 ceph-mon[36101]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-05-17T22:31:49.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:48 smithi151 ceph-mon[36101]: osdmap e46: 8 total, 7 up, 8 in 2024-05-17T22:31:49.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:48 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T22:31:49.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:48 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T22:31:49.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:48 smithi151 ceph-mon[36101]: osd.7 [v2:172.21.15.96:6816/2449989110,v1:172.21.15.96:6817/2449989110] boot 2024-05-17T22:31:49.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:48 smithi151 ceph-mon[36101]: osdmap e47: 8 total, 8 up, 8 in 2024-05-17T22:31:49.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:48 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T22:31:49.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:48 smithi012 ceph-mon[31632]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-05-17T22:31:49.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:48 smithi012 ceph-mon[31632]: osdmap e46: 8 total, 7 up, 8 in 2024-05-17T22:31:49.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:48 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T22:31:49.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:48 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T22:31:49.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:48 smithi012 ceph-mon[31632]: osd.7 [v2:172.21.15.96:6816/2449989110,v1:172.21.15.96:6817/2449989110] boot 2024-05-17T22:31:49.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:48 smithi012 ceph-mon[31632]: osdmap e47: 8 total, 8 up, 8 in 2024-05-17T22:31:49.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:48 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T22:31:49.608 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi151:vg_nvme/lv_2 2024-05-17T22:31:49.854 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T22:31:50.121 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:49 smithi151 ceph-mon[36101]: purged_snaps scrub starts 2024-05-17T22:31:50.122 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:49 smithi151 ceph-mon[36101]: purged_snaps scrub ok 2024-05-17T22:31:50.122 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:49 smithi151 ceph-mon[36101]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-17T22:31:50.122 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:49 smithi151 ceph-mon[36101]: osdmap e48: 8 total, 8 up, 8 in 2024-05-17T22:31:50.122 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:49 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:50.122 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:49 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:50.122 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:49 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:31:50.122 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:49 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:31:50.122 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:49 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:31:50.122 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:49 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:50.122 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:49 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:31:50.122 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:49 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:31:50.122 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:49 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:50.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:49 smithi096 ceph-mon[36065]: purged_snaps scrub starts 2024-05-17T22:31:50.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:49 smithi096 ceph-mon[36065]: purged_snaps scrub ok 2024-05-17T22:31:50.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:49 smithi096 ceph-mon[36065]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-17T22:31:50.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:49 smithi096 ceph-mon[36065]: osdmap e48: 8 total, 8 up, 8 in 2024-05-17T22:31:50.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:49 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:50.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:49 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:50.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:49 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:31:50.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:49 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:31:50.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:49 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:31:50.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:49 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:50.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:49 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:31:50.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:49 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:31:50.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:49 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:50.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:49 smithi012 ceph-mon[31632]: purged_snaps scrub starts 2024-05-17T22:31:50.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:49 smithi012 ceph-mon[31632]: purged_snaps scrub ok 2024-05-17T22:31:50.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:49 smithi012 ceph-mon[31632]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-17T22:31:50.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:49 smithi012 ceph-mon[31632]: osdmap e48: 8 total, 8 up, 8 in 2024-05-17T22:31:50.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:49 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:50.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:49 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:50.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:49 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:31:50.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:49 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:31:50.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:49 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:31:50.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:49 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:50.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:49 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:31:50.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:49 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:31:50.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:49 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:31:50.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:50 smithi151 ceph-mon[36101]: Detected new or changed devices on smithi096 2024-05-17T22:31:50.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:50 smithi151 ceph-mon[36101]: Adjusting osd_memory_target on smithi096 to 5696M 2024-05-17T22:31:50.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:50 smithi151 ceph-mon[36101]: osdmap e49: 8 total, 8 up, 8 in 2024-05-17T22:31:51.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:50 smithi096 ceph-mon[36065]: Detected new or changed devices on smithi096 2024-05-17T22:31:51.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:50 smithi096 ceph-mon[36065]: Adjusting osd_memory_target on smithi096 to 5696M 2024-05-17T22:31:51.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:50 smithi096 ceph-mon[36065]: osdmap e49: 8 total, 8 up, 8 in 2024-05-17T22:31:51.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:50 smithi012 ceph-mon[31632]: Detected new or changed devices on smithi096 2024-05-17T22:31:51.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:50 smithi012 ceph-mon[31632]: Adjusting osd_memory_target on smithi096 to 5696M 2024-05-17T22:31:51.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:50 smithi012 ceph-mon[31632]: osdmap e49: 8 total, 8 up, 8 in 2024-05-17T22:31:51.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:31:51.612+0000 7f98445af700 1 -- 172.21.15.151:0/3327114235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f983c103cc0 msgr2=0x7f983c1040d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:31:51.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:31:51.612+0000 7f98445af700 1 --2- 172.21.15.151:0/3327114235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f983c103cc0 0x7f983c1040d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f98380099c0 tx=0x7f9838009cd0 comp rx=0 tx=0).stop 2024-05-17T22:31:51.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:31:51.612+0000 7f98445af700 1 -- 172.21.15.151:0/3327114235 shutdown_connections 2024-05-17T22:31:51.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:31:51.612+0000 7f98445af700 1 --2- 172.21.15.151:0/3327114235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f983c104660 0x7f983c10af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:31:51.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:31:51.612+0000 7f98445af700 1 --2- 172.21.15.151:0/3327114235 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f983c102b90 0x7f983c102fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:31:51.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:31:51.612+0000 7f98445af700 1 --2- 172.21.15.151:0/3327114235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f983c103cc0 0x7f983c1040d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:31:51.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:31:51.612+0000 7f98445af700 1 -- 172.21.15.151:0/3327114235 >> 172.21.15.151:0/3327114235 conn(0x7f983c0fe3c0 msgr2=0x7f983c100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:31:51.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:31:51.612+0000 7f98445af700 1 -- 172.21.15.151:0/3327114235 shutdown_connections 2024-05-17T22:31:51.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:31:51.612+0000 7f98445af700 1 -- 172.21.15.151:0/3327114235 wait complete. 2024-05-17T22:31:51.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:31:51.612+0000 7f98445af700 1 Processor -- start 2024-05-17T22:31:51.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:31:51.612+0000 7f98445af700 1 -- start start 2024-05-17T22:31:51.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:31:51.613+0000 7f98445af700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f983c102b90 0x7f983c10d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:31:51.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:31:51.613+0000 7f98445af700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f983c103cc0 0x7f983c10bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:31:51.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:31:51.613+0000 7f98445af700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f983c104660 0x7f983c10c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:31:51.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:31:51.613+0000 7f98445af700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f983c10f740 con 0x7f983c104660 2024-05-17T22:31:51.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:31:51.613+0000 7f98445af700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f983c10f5c0 con 0x7f983c102b90 2024-05-17T22:31:51.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:31:51.613+0000 7f98445af700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f983c10f8c0 con 0x7f983c103cc0 2024-05-17T22:31:51.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:31:51.613+0000 7f9841b4a700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f983c103cc0 0x7f983c10bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:31:51.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:31:51.613+0000 7f9841b4a700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f983c103cc0 0x7f983c10bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42404/0 (socket says 172.21.15.151:42404) 2024-05-17T22:31:51.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:31:51.613+0000 7f9841b4a700 1 -- 172.21.15.151:0/392509910 learned_addr learned my addr 172.21.15.151:0/392509910 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-17T22:31:51.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:31:51.613+0000 7f984234b700 1 --2- 172.21.15.151:0/392509910 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f983c102b90 0x7f983c10d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:31:51.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:31:51.613+0000 7f9842b4c700 1 --2- 172.21.15.151:0/392509910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f983c104660 0x7f983c10c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:31:51.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:31:51.613+0000 7f9841b4a700 1 -- 172.21.15.151:0/392509910 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f983c102b90 msgr2=0x7f983c10d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:31:51.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:31:51.613+0000 7f9841b4a700 1 --2- 172.21.15.151:0/392509910 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f983c102b90 0x7f983c10d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:31:51.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:31:51.613+0000 7f9841b4a700 1 -- 172.21.15.151:0/392509910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f983c104660 msgr2=0x7f983c10c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:31:51.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:31:51.613+0000 7f9841b4a700 1 --2- 172.21.15.151:0/392509910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f983c104660 0x7f983c10c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:31:51.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:31:51.613+0000 7f9841b4a700 1 -- 172.21.15.151:0/392509910 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f983c10c760 con 0x7f983c103cc0 2024-05-17T22:31:51.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:31:51.613+0000 7f984234b700 1 --2- 172.21.15.151:0/392509910 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f983c102b90 0x7f983c10d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:31:51.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:31:51.613+0000 7f9842b4c700 1 --2- 172.21.15.151:0/392509910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f983c104660 0x7f983c10c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:31:51.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:31:51.613+0000 7f9841b4a700 1 --2- 172.21.15.151:0/392509910 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f983c103cc0 0x7f983c10bb50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9838000c00 tx=0x7f9838009f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:31:51.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:31:51.613+0000 7f98337fe700 1 -- 172.21.15.151:0/392509910 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838027070 con 0x7f983c103cc0 2024-05-17T22:31:51.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:31:51.613+0000 7f98337fe700 1 -- 172.21.15.151:0/392509910 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9838004120 con 0x7f983c103cc0 2024-05-17T22:31:51.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:31:51.613+0000 7f98337fe700 1 -- 172.21.15.151:0/392509910 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838022e40 con 0x7f983c103cc0 2024-05-17T22:31:51.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:31:51.614+0000 7f98445af700 1 -- 172.21.15.151:0/392509910 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f983c1a4350 con 0x7f983c103cc0 2024-05-17T22:31:51.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:31:51.614+0000 7f98445af700 1 -- 172.21.15.151:0/392509910 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f983c1a46f0 con 0x7f983c103cc0 2024-05-17T22:31:51.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:31:51.615+0000 7f98445af700 1 -- 172.21.15.151:0/392509910 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f983c102fa0 con 0x7f983c103cc0 2024-05-17T22:31:51.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:31:51.616+0000 7f98337fe700 1 -- 172.21.15.151:0/392509910 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f983802a070 con 0x7f983c103cc0 2024-05-17T22:31:51.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:31:51.616+0000 7f98337fe700 1 --2- 172.21.15.151:0/392509910 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f98280b2900 0x7f98280b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:31:51.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:31:51.617+0000 7f98337fe700 1 -- 172.21.15.151:0/392509910 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5397+0+0 (secure 0 0 0) 0x7f98380e24a0 con 0x7f983c103cc0 2024-05-17T22:31:51.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:31:51.617+0000 7f984234b700 1 --2- 172.21.15.151:0/392509910 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f98280b2900 0x7f98280b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:31:51.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:31:51.617+0000 7f984234b700 1 --2- 172.21.15.151:0/392509910 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f98280b2900 0x7f98280b4e10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f982c005950 tx=0x7f982c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:31:51.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:31:51.621+0000 7f98337fe700 1 -- 172.21.15.151:0/392509910 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9838013070 con 0x7f983c103cc0 2024-05-17T22:31:51.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:31:51.833+0000 7f98445af700 1 -- 172.21.15.151:0/392509910 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f983c064720 con 0x7f98280b2900 2024-05-17T22:31:52.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:51 smithi151 ceph-mon[36101]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-05-17T22:31:52.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:51 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T22:31:52.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:51 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T22:31:52.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:51 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:31:52.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:51 smithi096 ceph-mon[36065]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-05-17T22:31:52.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:51 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T22:31:52.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:51 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T22:31:52.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:51 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:31:52.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:51 smithi012 ceph-mon[31632]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-05-17T22:31:52.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:51 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T22:31:52.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:51 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T22:31:52.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:51 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:31:53.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:52 smithi151 ceph-mon[36101]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:31:53.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:52 smithi096 ceph-mon[36065]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:31:53.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:52 smithi012 ceph-mon[31632]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:31:54.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:53 smithi151 ceph-mon[36101]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-05-17T22:31:54.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:53 smithi096 ceph-mon[36065]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-05-17T22:31:54.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:53 smithi012 ceph-mon[31632]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-05-17T22:31:55.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:54 smithi151 ceph-mon[36101]: from='client.? 172.21.15.151:0/1250191537' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad62ea44-e416-4a3e-bbbe-93f2a710dbc1"}]: dispatch 2024-05-17T22:31:55.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:54 smithi151 ceph-mon[36101]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad62ea44-e416-4a3e-bbbe-93f2a710dbc1"}]: dispatch 2024-05-17T22:31:55.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:54 smithi151 ceph-mon[36101]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad62ea44-e416-4a3e-bbbe-93f2a710dbc1"}]': finished 2024-05-17T22:31:55.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:54 smithi151 ceph-mon[36101]: osdmap e50: 9 total, 8 up, 9 in 2024-05-17T22:31:55.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:54 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T22:31:55.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:54 smithi096 ceph-mon[36065]: from='client.? 172.21.15.151:0/1250191537' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad62ea44-e416-4a3e-bbbe-93f2a710dbc1"}]: dispatch 2024-05-17T22:31:55.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:54 smithi096 ceph-mon[36065]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad62ea44-e416-4a3e-bbbe-93f2a710dbc1"}]: dispatch 2024-05-17T22:31:55.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:54 smithi096 ceph-mon[36065]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad62ea44-e416-4a3e-bbbe-93f2a710dbc1"}]': finished 2024-05-17T22:31:55.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:54 smithi096 ceph-mon[36065]: osdmap e50: 9 total, 8 up, 9 in 2024-05-17T22:31:55.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:54 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T22:31:55.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:54 smithi012 ceph-mon[31632]: from='client.? 172.21.15.151:0/1250191537' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad62ea44-e416-4a3e-bbbe-93f2a710dbc1"}]: dispatch 2024-05-17T22:31:55.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:54 smithi012 ceph-mon[31632]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad62ea44-e416-4a3e-bbbe-93f2a710dbc1"}]: dispatch 2024-05-17T22:31:55.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:54 smithi012 ceph-mon[31632]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad62ea44-e416-4a3e-bbbe-93f2a710dbc1"}]': finished 2024-05-17T22:31:55.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:54 smithi012 ceph-mon[31632]: osdmap e50: 9 total, 8 up, 9 in 2024-05-17T22:31:55.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:54 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T22:31:56.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:55 smithi151 ceph-mon[36101]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-05-17T22:31:56.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:55 smithi151 ceph-mon[36101]: from='client.? 172.21.15.151:0/377591597' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T22:31:56.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:55 smithi096 ceph-mon[36065]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-05-17T22:31:56.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:55 smithi096 ceph-mon[36065]: from='client.? 172.21.15.151:0/377591597' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T22:31:56.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:55 smithi012 ceph-mon[31632]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-05-17T22:31:56.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:55 smithi012 ceph-mon[31632]: from='client.? 172.21.15.151:0/377591597' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T22:31:58.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:57 smithi151 ceph-mon[36101]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-17T22:31:58.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:57 smithi096 ceph-mon[36065]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-17T22:31:58.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:57 smithi012 ceph-mon[31632]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-17T22:31:59.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:31:59 smithi151 ceph-mon[36101]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-17T22:32:00.297 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:31:59 smithi096 ceph-mon[36065]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-17T22:32:00.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:31:59 smithi012 ceph-mon[31632]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-17T22:32:02.108 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:01 smithi151 ceph-mon[36101]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T22:32:02.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:01 smithi096 ceph-mon[36065]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T22:32:02.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:01 smithi012 ceph-mon[31632]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T22:32:03.062 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:02 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-17T22:32:03.062 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:02 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:32:03.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:02 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-17T22:32:03.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:02 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:32:03.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:02 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-17T22:32:03.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:02 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:32:04.041 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:03 smithi151 ceph-mon[36101]: Deploying daemon osd.8 on smithi151 2024-05-17T22:32:04.042 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:03 smithi151 ceph-mon[36101]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T22:32:04.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:03 smithi096 ceph-mon[36065]: Deploying daemon osd.8 on smithi151 2024-05-17T22:32:04.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:03 smithi096 ceph-mon[36065]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T22:32:04.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:03 smithi012 ceph-mon[31632]: Deploying daemon osd.8 on smithi151 2024-05-17T22:32:04.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:03 smithi012 ceph-mon[31632]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T22:32:05.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:04 smithi151 ceph-mon[36101]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-17T22:32:05.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:04 smithi096 ceph-mon[36065]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-17T22:32:05.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:04 smithi012 ceph-mon[31632]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-17T22:32:07.642 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:07 smithi151 ceph-mon[36101]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-17T22:32:07.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:07 smithi096 ceph-mon[36065]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-17T22:32:07.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:07 smithi012 ceph-mon[31632]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-17T22:32:08.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:08 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:32:08.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:08 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:08.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:08 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:08.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:08 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:32:08.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:08 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:08.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:08 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:08.897 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:08 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:32:08.898 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:08 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:08.898 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:08 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:09.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:09 smithi151 ceph-mon[36101]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-17T22:32:09.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:09 smithi096 ceph-mon[36065]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-17T22:32:09.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:09 smithi012 ceph-mon[31632]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-17T22:32:11.760 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:11 smithi151 ceph-mon[36101]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-17T22:32:11.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:11 smithi096 ceph-mon[36065]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-17T22:32:11.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:11 smithi012 ceph-mon[31632]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-17T22:32:12.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:32:12.552+0000 7f98337fe700 1 -- 172.21.15.151:0/392509910 <== mgr.14152 v2:172.21.15.12:6800/1388514047 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f983c064720 con 0x7f98280b2900 2024-05-17T22:32:12.556 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 8 on host 'smithi151' 2024-05-17T22:32:12.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:32:12.554+0000 7f98445af700 1 -- 172.21.15.151:0/392509910 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f98280b2900 msgr2=0x7f98280b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:32:12.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:32:12.554+0000 7f98445af700 1 --2- 172.21.15.151:0/392509910 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f98280b2900 0x7f98280b4e10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f982c005950 tx=0x7f982c0058e0 comp rx=0 tx=0).stop 2024-05-17T22:32:12.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:32:12.554+0000 7f98445af700 1 -- 172.21.15.151:0/392509910 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f983c103cc0 msgr2=0x7f983c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:32:12.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:32:12.554+0000 7f98445af700 1 --2- 172.21.15.151:0/392509910 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f983c103cc0 0x7f983c10bb50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9838000c00 tx=0x7f9838009f80 comp rx=0 tx=0).stop 2024-05-17T22:32:12.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:32:12.554+0000 7f98445af700 1 -- 172.21.15.151:0/392509910 shutdown_connections 2024-05-17T22:32:12.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:32:12.554+0000 7f98445af700 1 --2- 172.21.15.151:0/392509910 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f98280b2900 0x7f98280b4e10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:32:12.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:32:12.554+0000 7f98445af700 1 --2- 172.21.15.151:0/392509910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f983c104660 0x7f983c10c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:32:12.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:32:12.554+0000 7f98445af700 1 --2- 172.21.15.151:0/392509910 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f983c102b90 0x7f983c10d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:32:12.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:32:12.554+0000 7f98445af700 1 --2- 172.21.15.151:0/392509910 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f983c103cc0 0x7f983c10bb50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:32:12.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:32:12.554+0000 7f98445af700 1 -- 172.21.15.151:0/392509910 >> 172.21.15.151:0/392509910 conn(0x7f983c0fe3c0 msgr2=0x7f983c106ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:32:12.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:32:12.554+0000 7f98445af700 1 -- 172.21.15.151:0/392509910 shutdown_connections 2024-05-17T22:32:12.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:32:12.555+0000 7f98445af700 1 -- 172.21.15.151:0/392509910 wait complete. 2024-05-17T22:32:12.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:12 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:12.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:12 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:12.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:12 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:32:12.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:12 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:32:12.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:12 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:12.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:12 smithi151 ceph-mon[36101]: from='osd.8 [v2:172.21.15.151:6816/2545224628,v1:172.21.15.151:6817/2545224628]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-17T22:32:12.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:12 smithi151 ceph-mon[36101]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-17T22:32:12.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:12 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:12.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:12 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:12.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:12 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:12.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:12 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:12.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:12 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:32:12.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:12 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:32:12.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:12 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:12.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:12 smithi096 ceph-mon[36065]: from='osd.8 [v2:172.21.15.151:6816/2545224628,v1:172.21.15.151:6817/2545224628]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-17T22:32:12.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:12 smithi096 ceph-mon[36065]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-17T22:32:12.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:12 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:12.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:12 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:12.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:12 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:12.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:12 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:12.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:12 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:32:12.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:12 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:32:12.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:12 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:12.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:12 smithi012 ceph-mon[31632]: from='osd.8 [v2:172.21.15.151:6816/2545224628,v1:172.21.15.151:6817/2545224628]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-17T22:32:12.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:12 smithi012 ceph-mon[31632]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-17T22:32:12.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:12 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:12.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:12 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:13.266 DEBUG:teuthology.orchestra.run.smithi151:osd.8> sudo journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@osd.8.service 2024-05-17T22:32:13.268 INFO:tasks.cephadm:Deploying osd.9 on smithi012 with /dev/vg_nvme/lv_1... 2024-05-17T22:32:13.268 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-17T22:32:13.529 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:32:13.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:13 smithi012 ceph-mon[31632]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-17T22:32:13.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:13 smithi012 ceph-mon[31632]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-17T22:32:13.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:13 smithi012 ceph-mon[31632]: osdmap e51: 9 total, 8 up, 9 in 2024-05-17T22:32:13.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:13 smithi012 ceph-mon[31632]: from='osd.8 [v2:172.21.15.151:6816/2545224628,v1:172.21.15.151:6817/2545224628]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-17T22:32:13.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:13 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T22:32:13.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:13 smithi012 ceph-mon[31632]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-17T22:32:13.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:13 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:32:13.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:13 smithi096 ceph-mon[36065]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-17T22:32:13.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:13 smithi096 ceph-mon[36065]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-17T22:32:13.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:13 smithi096 ceph-mon[36065]: osdmap e51: 9 total, 8 up, 9 in 2024-05-17T22:32:13.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:13 smithi096 ceph-mon[36065]: from='osd.8 [v2:172.21.15.151:6816/2545224628,v1:172.21.15.151:6817/2545224628]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-17T22:32:13.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:13 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T22:32:13.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:13 smithi096 ceph-mon[36065]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-17T22:32:13.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:13 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:32:13.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:13 smithi151 ceph-mon[36101]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-17T22:32:13.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:13 smithi151 ceph-mon[36101]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-17T22:32:13.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:13 smithi151 ceph-mon[36101]: osdmap e51: 9 total, 8 up, 9 in 2024-05-17T22:32:13.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:13 smithi151 ceph-mon[36101]: from='osd.8 [v2:172.21.15.151:6816/2545224628,v1:172.21.15.151:6817/2545224628]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-17T22:32:13.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:13 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T22:32:13.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:13 smithi151 ceph-mon[36101]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-17T22:32:13.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:13 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:32:14.550 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:32:14 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:32:14.254+0000 7f5b77329700 -1 osd.8 0 waiting for initial osdmap 2024-05-17T22:32:14.550 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:32:14 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:32:14.265+0000 7f5b6f0d6700 -1 osd.8 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T22:32:14.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:14 smithi012 ceph-mon[31632]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-05-17T22:32:14.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:14 smithi012 ceph-mon[31632]: osdmap e52: 9 total, 8 up, 9 in 2024-05-17T22:32:14.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:14 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T22:32:14.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:14 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T22:32:14.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:14 smithi012 ceph-mon[31632]: from='osd.8 ' entity='osd.8' 2024-05-17T22:32:14.857 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:14 smithi151 ceph-mon[36101]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-05-17T22:32:14.857 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:14 smithi151 ceph-mon[36101]: osdmap e52: 9 total, 8 up, 9 in 2024-05-17T22:32:14.858 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:14 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T22:32:14.858 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:14 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T22:32:14.858 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:14 smithi151 ceph-mon[36101]: from='osd.8 ' entity='osd.8' 2024-05-17T22:32:15.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:14 smithi096 ceph-mon[36065]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-05-17T22:32:15.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:14 smithi096 ceph-mon[36065]: osdmap e52: 9 total, 8 up, 9 in 2024-05-17T22:32:15.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:14 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T22:32:15.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:14 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T22:32:15.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:14 smithi096 ceph-mon[36065]: from='osd.8 ' entity='osd.8' 2024-05-17T22:32:15.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:15 smithi012 ceph-mon[31632]: purged_snaps scrub starts 2024-05-17T22:32:15.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:15 smithi012 ceph-mon[31632]: purged_snaps scrub ok 2024-05-17T22:32:15.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:15 smithi012 ceph-mon[31632]: pgmap v192: 1 pgs: 1 peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-17T22:32:15.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:15 smithi012 ceph-mon[31632]: osd.8 [v2:172.21.15.151:6816/2545224628,v1:172.21.15.151:6817/2545224628] boot 2024-05-17T22:32:15.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:15 smithi012 ceph-mon[31632]: osdmap e53: 9 total, 9 up, 9 in 2024-05-17T22:32:15.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:15 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T22:32:15.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:15 smithi012 ceph-mon[31632]: osdmap e54: 9 total, 9 up, 9 in 2024-05-17T22:32:15.844 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-17T22:32:15.844 INFO:teuthology.orchestra.run.smithi012.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-17T22:32:15.845 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10+0 records in 2024-05-17T22:32:15.845 INFO:teuthology.orchestra.run.smithi012.stderr:10+0 records out 2024-05-17T22:32:15.845 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113249 s, 926 MB/s 2024-05-17T22:32:15.845 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping successful for: 2024-05-17T22:32:15.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:15 smithi151 ceph-mon[36101]: purged_snaps scrub starts 2024-05-17T22:32:15.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:15 smithi151 ceph-mon[36101]: purged_snaps scrub ok 2024-05-17T22:32:15.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:15 smithi151 ceph-mon[36101]: pgmap v192: 1 pgs: 1 peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-17T22:32:15.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:15 smithi151 ceph-mon[36101]: osd.8 [v2:172.21.15.151:6816/2545224628,v1:172.21.15.151:6817/2545224628] boot 2024-05-17T22:32:15.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:15 smithi151 ceph-mon[36101]: osdmap e53: 9 total, 9 up, 9 in 2024-05-17T22:32:15.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:15 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T22:32:15.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:15 smithi151 ceph-mon[36101]: osdmap e54: 9 total, 9 up, 9 in 2024-05-17T22:32:16.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:15 smithi096 ceph-mon[36065]: purged_snaps scrub starts 2024-05-17T22:32:16.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:15 smithi096 ceph-mon[36065]: purged_snaps scrub ok 2024-05-17T22:32:16.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:15 smithi096 ceph-mon[36065]: pgmap v192: 1 pgs: 1 peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-17T22:32:16.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:15 smithi096 ceph-mon[36065]: osd.8 [v2:172.21.15.151:6816/2545224628,v1:172.21.15.151:6817/2545224628] boot 2024-05-17T22:32:16.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:15 smithi096 ceph-mon[36065]: osdmap e53: 9 total, 9 up, 9 in 2024-05-17T22:32:16.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:15 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T22:32:16.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:15 smithi096 ceph-mon[36065]: osdmap e54: 9 total, 9 up, 9 in 2024-05-17T22:32:16.590 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi012:vg_nvme/lv_1 2024-05-17T22:32:16.854 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:32:17.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:17 smithi012 ceph-mon[31632]: pgmap v195: 1 pgs: 1 peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-17T22:32:17.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:17 smithi012 ceph-mon[31632]: osdmap e55: 9 total, 9 up, 9 in 2024-05-17T22:32:17.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:17 smithi012 ceph-mon[31632]: Detected new or changed devices on smithi151 2024-05-17T22:32:17.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:17 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:17.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:17 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:17.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:17 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:32:17.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:17 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:32:17.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:17 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:32:17.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:17 smithi012 ceph-mon[31632]: Adjusting osd_memory_target on smithi151 to 5696M 2024-05-17T22:32:17.843 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:17 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:17.843 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:17 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:32:17.843 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:17 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:32:17.843 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:17 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:17.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:17 smithi151 ceph-mon[36101]: pgmap v195: 1 pgs: 1 peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-17T22:32:17.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:17 smithi151 ceph-mon[36101]: osdmap e55: 9 total, 9 up, 9 in 2024-05-17T22:32:17.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:17 smithi151 ceph-mon[36101]: Detected new or changed devices on smithi151 2024-05-17T22:32:17.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:17 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:17.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:17 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:17.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:17 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:32:17.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:17 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:32:17.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:17 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:32:17.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:17 smithi151 ceph-mon[36101]: Adjusting osd_memory_target on smithi151 to 5696M 2024-05-17T22:32:17.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:17 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:17.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:17 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:32:17.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:17 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:32:17.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:17 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:18.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:17 smithi096 ceph-mon[36065]: pgmap v195: 1 pgs: 1 peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-17T22:32:18.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:17 smithi096 ceph-mon[36065]: osdmap e55: 9 total, 9 up, 9 in 2024-05-17T22:32:18.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:17 smithi096 ceph-mon[36065]: Detected new or changed devices on smithi151 2024-05-17T22:32:18.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:17 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:18.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:17 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:18.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:17 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:32:18.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:17 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:32:18.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:17 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:32:18.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:17 smithi096 ceph-mon[36065]: Adjusting osd_memory_target on smithi151 to 5696M 2024-05-17T22:32:18.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:17 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:18.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:17 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:32:18.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:17 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:32:18.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:17 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:18.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:18.540+0000 7fe3e7186700 1 -- 172.21.15.12:0/2705679335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3e0104dc0 msgr2=0x7fe3e01051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:32:18.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:18.540+0000 7fe3e7186700 1 --2- 172.21.15.12:0/2705679335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3e0104dc0 0x7fe3e01051d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc009920 tx=0x7fe3cc009c30 comp rx=0 tx=0).stop 2024-05-17T22:32:18.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:18.540+0000 7fe3e7186700 1 -- 172.21.15.12:0/2705679335 shutdown_connections 2024-05-17T22:32:18.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:18.540+0000 7fe3e7186700 1 --2- 172.21.15.12:0/2705679335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3e0105760 0x7fe3e010c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:32:18.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:18.540+0000 7fe3e7186700 1 --2- 172.21.15.12:0/2705679335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3e0103c90 0x7fe3e01040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:32:18.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:18.540+0000 7fe3e7186700 1 --2- 172.21.15.12:0/2705679335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3e0104dc0 0x7fe3e01051d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:32:18.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:18.540+0000 7fe3e7186700 1 -- 172.21.15.12:0/2705679335 >> 172.21.15.12:0/2705679335 conn(0x7fe3e00ff4c0 msgr2=0x7fe3e0101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:32:18.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:18.540+0000 7fe3e7186700 1 -- 172.21.15.12:0/2705679335 shutdown_connections 2024-05-17T22:32:18.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:18.540+0000 7fe3e7186700 1 -- 172.21.15.12:0/2705679335 wait complete. 2024-05-17T22:32:18.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:18.540+0000 7fe3e7186700 1 Processor -- start 2024-05-17T22:32:18.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:18.541+0000 7fe3e7186700 1 -- start start 2024-05-17T22:32:18.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:18.541+0000 7fe3e7186700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3e0103c90 0x7fe3e00774d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:32:18.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:18.541+0000 7fe3e7186700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3e0104dc0 0x7fe3e0075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:32:18.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:18.541+0000 7fe3e7186700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3e0105760 0x7fe3e00760c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:32:18.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:18.541+0000 7fe3e7186700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e010f730 con 0x7fe3e0104dc0 2024-05-17T22:32:18.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:18.541+0000 7fe3e7186700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e010f5b0 con 0x7fe3e0103c90 2024-05-17T22:32:18.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:18.541+0000 7fe3e7186700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e010f8b0 con 0x7fe3e0105760 2024-05-17T22:32:18.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:18.541+0000 7fe3dffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3e0104dc0 0x7fe3e0075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:32:18.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:18.541+0000 7fe3dffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3e0104dc0 0x7fe3e0075b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55994/0 (socket says 172.21.15.12:55994) 2024-05-17T22:32:18.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:18.541+0000 7fe3dffff700 1 -- 172.21.15.12:0/360578834 learned_addr learned my addr 172.21.15.12:0/360578834 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:32:18.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:18.541+0000 7fe3e5723700 1 --2- 172.21.15.12:0/360578834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3e0105760 0x7fe3e00760c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:32:18.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:18.541+0000 7fe3e4f22700 1 --2- 172.21.15.12:0/360578834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3e0103c90 0x7fe3e00774d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:32:18.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:18.541+0000 7fe3dffff700 1 -- 172.21.15.12:0/360578834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3e0105760 msgr2=0x7fe3e00760c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:32:18.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:18.541+0000 7fe3dffff700 1 --2- 172.21.15.12:0/360578834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3e0105760 0x7fe3e00760c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:32:18.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:18.541+0000 7fe3dffff700 1 -- 172.21.15.12:0/360578834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3e0103c90 msgr2=0x7fe3e00774d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:32:18.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:18.541+0000 7fe3dffff700 1 --2- 172.21.15.12:0/360578834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3e0103c90 0x7fe3e00774d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:32:18.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:18.541+0000 7fe3dffff700 1 -- 172.21.15.12:0/360578834 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3e0076820 con 0x7fe3e0104dc0 2024-05-17T22:32:18.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:18.542+0000 7fe3dffff700 1 --2- 172.21.15.12:0/360578834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3e0104dc0 0x7fe3e0075b80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc00fc20 tx=0x7fe3cc005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:32:18.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:18.542+0000 7fe3ddffb700 1 -- 172.21.15.12:0/360578834 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc005490 con 0x7fe3e0104dc0 2024-05-17T22:32:18.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:18.542+0000 7fe3e7186700 1 -- 172.21.15.12:0/360578834 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3e01a43f0 con 0x7fe3e0104dc0 2024-05-17T22:32:18.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:18.542+0000 7fe3ddffb700 1 -- 172.21.15.12:0/360578834 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3cc005630 con 0x7fe3e0104dc0 2024-05-17T22:32:18.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:18.542+0000 7fe3ddffb700 1 -- 172.21.15.12:0/360578834 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc003a70 con 0x7fe3e0104dc0 2024-05-17T22:32:18.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:18.542+0000 7fe3e7186700 1 -- 172.21.15.12:0/360578834 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3e01a4790 con 0x7fe3e0104dc0 2024-05-17T22:32:18.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:18.543+0000 7fe3e7186700 1 -- 172.21.15.12:0/360578834 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3e01040a0 con 0x7fe3e0104dc0 2024-05-17T22:32:18.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:18.545+0000 7fe3ddffb700 1 -- 172.21.15.12:0/360578834 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fe3cc003cf0 con 0x7fe3e0104dc0 2024-05-17T22:32:18.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:18.546+0000 7fe3ddffb700 1 --2- 172.21.15.12:0/360578834 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe3d00b2880 0x7fe3d00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:32:18.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:18.546+0000 7fe3e4f22700 1 --2- 172.21.15.12:0/360578834 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe3d00b2880 0x7fe3d00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:32:18.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:18.546+0000 7fe3ddffb700 1 -- 172.21.15.12:0/360578834 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5829+0+0 (secure 0 0 0) 0x7fe3cc0e2b40 con 0x7fe3e0104dc0 2024-05-17T22:32:18.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:18.546+0000 7fe3e4f22700 1 --2- 172.21.15.12:0/360578834 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe3d00b2880 0x7fe3d00b4d90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0104cc0 tx=0x7fe3d4009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:32:18.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:18.549+0000 7fe3ddffb700 1 -- 172.21.15.12:0/360578834 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe3cc013030 con 0x7fe3e0104dc0 2024-05-17T22:32:18.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:18.776+0000 7fe3e7186700 1 -- 172.21.15.12:0/360578834 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fe3e0064820 con 0x7fe3d00b2880 2024-05-17T22:32:19.693 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:19 smithi012 ceph-mon[31632]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-17T22:32:19.693 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:19 smithi012 ceph-mon[31632]: from='client.14439 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:32:19.694 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:19 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T22:32:19.694 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:19 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T22:32:19.694 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:19 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:32:19.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:19 smithi151 ceph-mon[36101]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-17T22:32:19.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:19 smithi151 ceph-mon[36101]: from='client.14439 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:32:19.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:19 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T22:32:19.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:19 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T22:32:19.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:19 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:32:20.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:19 smithi096 ceph-mon[36065]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-17T22:32:20.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:19 smithi096 ceph-mon[36065]: from='client.14439 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:32:20.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:19 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T22:32:20.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:19 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T22:32:20.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:19 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:32:21.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:21 smithi012 ceph-mon[31632]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-17T22:32:21.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:21 smithi151 ceph-mon[36101]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-17T22:32:22.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:21 smithi096 ceph-mon[36065]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-17T22:32:22.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:22 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/61292525' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1275de08-2a3c-48f3-920b-dd68b4024d42"}]: dispatch 2024-05-17T22:32:22.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:22 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/61292525' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1275de08-2a3c-48f3-920b-dd68b4024d42"}]': finished 2024-05-17T22:32:22.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:22 smithi012 ceph-mon[31632]: osdmap e56: 10 total, 9 up, 10 in 2024-05-17T22:32:22.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:22 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T22:32:22.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:22 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/494836326' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T22:32:22.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:22 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/61292525' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1275de08-2a3c-48f3-920b-dd68b4024d42"}]: dispatch 2024-05-17T22:32:22.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:22 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/61292525' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1275de08-2a3c-48f3-920b-dd68b4024d42"}]': finished 2024-05-17T22:32:22.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:22 smithi151 ceph-mon[36101]: osdmap e56: 10 total, 9 up, 10 in 2024-05-17T22:32:22.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:22 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T22:32:22.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:22 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/494836326' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T22:32:23.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:22 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/61292525' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1275de08-2a3c-48f3-920b-dd68b4024d42"}]: dispatch 2024-05-17T22:32:23.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:22 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/61292525' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1275de08-2a3c-48f3-920b-dd68b4024d42"}]': finished 2024-05-17T22:32:23.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:22 smithi096 ceph-mon[36065]: osdmap e56: 10 total, 9 up, 10 in 2024-05-17T22:32:23.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:22 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T22:32:23.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:22 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/494836326' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T22:32:24.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:23 smithi151 ceph-mon[36101]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-05-17T22:32:24.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:23 smithi096 ceph-mon[36065]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-05-17T22:32:24.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:23 smithi012 ceph-mon[31632]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-05-17T22:32:26.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:25 smithi151 ceph-mon[36101]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-17T22:32:26.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:25 smithi096 ceph-mon[36065]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-17T22:32:26.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:25 smithi012 ceph-mon[31632]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-17T22:32:28.235 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:27 smithi012 ceph-mon[31632]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T22:32:28.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:27 smithi151 ceph-mon[36101]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T22:32:28.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:27 smithi096 ceph-mon[36065]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T22:32:30.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:29 smithi151 ceph-mon[36101]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T22:32:30.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:29 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-17T22:32:30.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:29 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:32:30.298 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:29 smithi012 ceph-mon[31632]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T22:32:30.298 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:29 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-17T22:32:30.298 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:29 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:32:30.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:29 smithi096 ceph-mon[36065]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T22:32:30.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:29 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-17T22:32:30.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:29 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:32:31.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:30 smithi151 ceph-mon[36101]: Deploying daemon osd.9 on smithi012 2024-05-17T22:32:31.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:30 smithi096 ceph-mon[36065]: Deploying daemon osd.9 on smithi012 2024-05-17T22:32:31.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:30 smithi012 ceph-mon[31632]: Deploying daemon osd.9 on smithi012 2024-05-17T22:32:32.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:31 smithi012 ceph-mon[31632]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-17T22:32:32.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:31 smithi151 ceph-mon[36101]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-17T22:32:32.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:31 smithi096 ceph-mon[36065]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-17T22:32:34.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:33 smithi012 ceph-mon[31632]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-17T22:32:34.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:33 smithi151 ceph-mon[36101]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-17T22:32:34.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:33 smithi096 ceph-mon[36065]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-17T22:32:36.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:35 smithi151 ceph-mon[36101]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-17T22:32:36.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:35 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:36.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:35 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:36.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:35 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:32:36.274 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:35 smithi012 ceph-mon[31632]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-17T22:32:36.274 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:35 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:36.274 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:35 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:36.274 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:35 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:32:36.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:35 smithi096 ceph-mon[36065]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-17T22:32:36.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:35 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:36.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:35 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:36.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:35 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:32:38.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:37 smithi012 ceph-mon[31632]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-17T22:32:38.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:37 smithi151 ceph-mon[36101]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-17T22:32:38.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:37 smithi096 ceph-mon[36065]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-17T22:32:40.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:39 smithi012 ceph-mon[31632]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-17T22:32:40.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:39 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:40.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:39 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:40.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:39 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:32:40.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:39 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:32:40.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:39 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:40.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:39 smithi012 ceph-mon[31632]: from='osd.9 [v2:172.21.15.12:6826/2100661273,v1:172.21.15.12:6827/2100661273]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-17T22:32:40.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:40.198+0000 7fe3ddffb700 1 -- 172.21.15.12:0/360578834 <== mgr.14152 v2:172.21.15.12:6800/1388514047 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe3e0064820 con 0x7fe3d00b2880 2024-05-17T22:32:40.201 INFO:teuthology.orchestra.run.smithi012.stdout:Created osd(s) 9 on host 'smithi012' 2024-05-17T22:32:40.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:40.201+0000 7fe3e7186700 1 -- 172.21.15.12:0/360578834 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe3d00b2880 msgr2=0x7fe3d00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:32:40.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:40.201+0000 7fe3e7186700 1 --2- 172.21.15.12:0/360578834 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe3d00b2880 0x7fe3d00b4d90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0104cc0 tx=0x7fe3d4009310 comp rx=0 tx=0).stop 2024-05-17T22:32:40.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:40.201+0000 7fe3e7186700 1 -- 172.21.15.12:0/360578834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3e0104dc0 msgr2=0x7fe3e0075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:32:40.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:40.201+0000 7fe3e7186700 1 --2- 172.21.15.12:0/360578834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3e0104dc0 0x7fe3e0075b80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc00fc20 tx=0x7fe3cc005270 comp rx=0 tx=0).stop 2024-05-17T22:32:40.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:40.201+0000 7fe3e7186700 1 -- 172.21.15.12:0/360578834 shutdown_connections 2024-05-17T22:32:40.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:40.201+0000 7fe3e7186700 1 --2- 172.21.15.12:0/360578834 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe3d00b2880 0x7fe3d00b4d90 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:32:40.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:40.201+0000 7fe3e7186700 1 --2- 172.21.15.12:0/360578834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3e0105760 0x7fe3e00760c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:32:40.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:40.201+0000 7fe3e7186700 1 --2- 172.21.15.12:0/360578834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3e0103c90 0x7fe3e00774d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:32:40.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:40.201+0000 7fe3e7186700 1 --2- 172.21.15.12:0/360578834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3e0104dc0 0x7fe3e0075b80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:32:40.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:40.201+0000 7fe3e7186700 1 -- 172.21.15.12:0/360578834 >> 172.21.15.12:0/360578834 conn(0x7fe3e00ff4c0 msgr2=0x7fe3e01080c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:32:40.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:40.201+0000 7fe3e7186700 1 -- 172.21.15.12:0/360578834 shutdown_connections 2024-05-17T22:32:40.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:32:40.201+0000 7fe3e7186700 1 -- 172.21.15.12:0/360578834 wait complete. 2024-05-17T22:32:40.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:39 smithi151 ceph-mon[36101]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-17T22:32:40.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:39 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:40.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:39 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:40.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:39 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:32:40.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:39 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:32:40.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:39 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:40.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:39 smithi151 ceph-mon[36101]: from='osd.9 [v2:172.21.15.12:6826/2100661273,v1:172.21.15.12:6827/2100661273]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-17T22:32:40.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:39 smithi096 ceph-mon[36065]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-17T22:32:40.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:39 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:40.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:39 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:40.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:39 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:32:40.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:39 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:32:40.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:39 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:40.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:39 smithi096 ceph-mon[36065]: from='osd.9 [v2:172.21.15.12:6826/2100661273,v1:172.21.15.12:6827/2100661273]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-17T22:32:40.839 DEBUG:teuthology.orchestra.run.smithi012:osd.9> sudo journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@osd.9.service 2024-05-17T22:32:40.842 INFO:tasks.cephadm:Deploying osd.10 on smithi096 with /dev/vg_nvme/lv_1... 2024-05-17T22:32:40.842 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-17T22:32:41.085 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.b/config 2024-05-17T22:32:41.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:41 smithi012 ceph-mon[31632]: from='osd.9 [v2:172.21.15.12:6826/2100661273,v1:172.21.15.12:6827/2100661273]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-17T22:32:41.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:41 smithi012 ceph-mon[31632]: osdmap e57: 10 total, 9 up, 10 in 2024-05-17T22:32:41.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:41 smithi012 ceph-mon[31632]: from='osd.9 [v2:172.21.15.12:6826/2100661273,v1:172.21.15.12:6827/2100661273]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-05-17T22:32:41.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:41 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T22:32:41.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:41 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:41.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:41 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:32:41.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:41 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:41.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:41 smithi012 ceph-mon[31632]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-17T22:32:41.092 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:32:41 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:32:41.043+0000 7f11033ba700 -1 osd.9 0 waiting for initial osdmap 2024-05-17T22:32:41.092 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:32:41 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:32:41.058+0000 7f10ff16f700 -1 osd.9 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T22:32:41.188 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:41 smithi096 ceph-mon[36065]: from='osd.9 [v2:172.21.15.12:6826/2100661273,v1:172.21.15.12:6827/2100661273]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-17T22:32:41.188 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:41 smithi096 ceph-mon[36065]: osdmap e57: 10 total, 9 up, 10 in 2024-05-17T22:32:41.188 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:41 smithi096 ceph-mon[36065]: from='osd.9 [v2:172.21.15.12:6826/2100661273,v1:172.21.15.12:6827/2100661273]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-05-17T22:32:41.188 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:41 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T22:32:41.188 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:41 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:41.188 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:41 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:32:41.188 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:41 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:41.188 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:41 smithi096 ceph-mon[36065]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-17T22:32:41.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:41 smithi151 ceph-mon[36101]: from='osd.9 [v2:172.21.15.12:6826/2100661273,v1:172.21.15.12:6827/2100661273]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-17T22:32:41.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:41 smithi151 ceph-mon[36101]: osdmap e57: 10 total, 9 up, 10 in 2024-05-17T22:32:41.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:41 smithi151 ceph-mon[36101]: from='osd.9 [v2:172.21.15.12:6826/2100661273,v1:172.21.15.12:6827/2100661273]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-05-17T22:32:41.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:41 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T22:32:41.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:41 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:41.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:41 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:32:41.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:41 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:41.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:41 smithi151 ceph-mon[36101]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-17T22:32:42.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:42 smithi012 ceph-mon[31632]: from='osd.9 [v2:172.21.15.12:6826/2100661273,v1:172.21.15.12:6827/2100661273]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-05-17T22:32:42.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:42 smithi012 ceph-mon[31632]: osdmap e58: 10 total, 9 up, 10 in 2024-05-17T22:32:42.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:42 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T22:32:42.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:42 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T22:32:42.253 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:42 smithi096 ceph-mon[36065]: from='osd.9 [v2:172.21.15.12:6826/2100661273,v1:172.21.15.12:6827/2100661273]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-05-17T22:32:42.253 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:42 smithi096 ceph-mon[36065]: osdmap e58: 10 total, 9 up, 10 in 2024-05-17T22:32:42.253 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:42 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T22:32:42.253 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:42 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T22:32:42.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:42 smithi151 ceph-mon[36101]: from='osd.9 [v2:172.21.15.12:6826/2100661273,v1:172.21.15.12:6827/2100661273]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-05-17T22:32:42.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:42 smithi151 ceph-mon[36101]: osdmap e58: 10 total, 9 up, 10 in 2024-05-17T22:32:42.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:42 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T22:32:42.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:42 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T22:32:43.134 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-17T22:32:43.134 INFO:teuthology.orchestra.run.smithi096.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-17T22:32:43.134 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10+0 records in 2024-05-17T22:32:43.134 INFO:teuthology.orchestra.run.smithi096.stderr:10+0 records out 2024-05-17T22:32:43.135 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0131669 s, 796 MB/s 2024-05-17T22:32:43.135 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping successful for: 2024-05-17T22:32:43.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:43 smithi012 ceph-mon[31632]: purged_snaps scrub starts 2024-05-17T22:32:43.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:43 smithi012 ceph-mon[31632]: purged_snaps scrub ok 2024-05-17T22:32:43.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:43 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T22:32:43.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:43 smithi012 ceph-mon[31632]: osd.9 [v2:172.21.15.12:6826/2100661273,v1:172.21.15.12:6827/2100661273] boot 2024-05-17T22:32:43.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:43 smithi012 ceph-mon[31632]: osdmap e59: 10 total, 10 up, 10 in 2024-05-17T22:32:43.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:43 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T22:32:43.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:43 smithi012 ceph-mon[31632]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-17T22:32:43.432 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:43 smithi096 ceph-mon[36065]: purged_snaps scrub starts 2024-05-17T22:32:43.432 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:43 smithi096 ceph-mon[36065]: purged_snaps scrub ok 2024-05-17T22:32:43.432 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:43 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T22:32:43.432 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:43 smithi096 ceph-mon[36065]: osd.9 [v2:172.21.15.12:6826/2100661273,v1:172.21.15.12:6827/2100661273] boot 2024-05-17T22:32:43.433 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:43 smithi096 ceph-mon[36065]: osdmap e59: 10 total, 10 up, 10 in 2024-05-17T22:32:43.433 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:43 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T22:32:43.433 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:43 smithi096 ceph-mon[36065]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-17T22:32:43.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:43 smithi151 ceph-mon[36101]: purged_snaps scrub starts 2024-05-17T22:32:43.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:43 smithi151 ceph-mon[36101]: purged_snaps scrub ok 2024-05-17T22:32:43.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:43 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T22:32:43.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:43 smithi151 ceph-mon[36101]: osd.9 [v2:172.21.15.12:6826/2100661273,v1:172.21.15.12:6827/2100661273] boot 2024-05-17T22:32:43.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:43 smithi151 ceph-mon[36101]: osdmap e59: 10 total, 10 up, 10 in 2024-05-17T22:32:43.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:43 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T22:32:43.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:43 smithi151 ceph-mon[36101]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-17T22:32:43.852 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi096:vg_nvme/lv_1 2024-05-17T22:32:44.089 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.b/config 2024-05-17T22:32:44.210 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:44 smithi096 ceph-mon[36065]: osdmap e60: 10 total, 10 up, 10 in 2024-05-17T22:32:44.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:44 smithi012 ceph-mon[31632]: osdmap e60: 10 total, 10 up, 10 in 2024-05-17T22:32:44.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:44 smithi151 ceph-mon[36101]: osdmap e60: 10 total, 10 up, 10 in 2024-05-17T22:32:45.453 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:45 smithi096 ceph-mon[36065]: Detected new or changed devices on smithi012 2024-05-17T22:32:45.453 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:45 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:45.453 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:45 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:45.453 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:45 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:32:45.453 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:45 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:32:45.453 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:45 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:32:45.453 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:45 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:32:45.454 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:45 smithi096 ceph-mon[36065]: Adjusting osd_memory_target on smithi012 to 4273M 2024-05-17T22:32:45.454 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:45 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:45.454 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:45 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:32:45.454 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:45 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:32:45.454 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:45 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:45.454 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:45 smithi096 ceph-mon[36065]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-17T22:32:45.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:45 smithi151 ceph-mon[36101]: Detected new or changed devices on smithi012 2024-05-17T22:32:45.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:45 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:45.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:45 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:45.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:45 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:32:45.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:45 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:32:45.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:45 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:32:45.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:45 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:32:45.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:45 smithi151 ceph-mon[36101]: Adjusting osd_memory_target on smithi012 to 4273M 2024-05-17T22:32:45.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:45 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:45.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:45 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:32:45.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:45 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:32:45.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:45 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:45.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:45 smithi151 ceph-mon[36101]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-17T22:32:45.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:45 smithi012 ceph-mon[31632]: Detected new or changed devices on smithi012 2024-05-17T22:32:45.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:45 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:45.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:45 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:45.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:45 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:32:45.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:45 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:32:45.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:45 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:32:45.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:45 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:32:45.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:45 smithi012 ceph-mon[31632]: Adjusting osd_memory_target on smithi012 to 4273M 2024-05-17T22:32:45.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:45 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:45.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:45 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:32:45.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:45 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:32:45.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:45 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:32:45.593 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:45 smithi012 ceph-mon[31632]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-17T22:32:45.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:32:45.668+0000 7f50baee3700 1 -- 172.21.15.96:0/3136866443 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50b4104660 msgr2=0x7f50b410af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:32:45.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:32:45.668+0000 7f50baee3700 1 --2- 172.21.15.96:0/3136866443 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50b4104660 0x7f50b410af10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f50ac00b0a0 tx=0x7f50ac00b3b0 comp rx=0 tx=0).stop 2024-05-17T22:32:45.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:32:45.668+0000 7f50baee3700 1 -- 172.21.15.96:0/3136866443 shutdown_connections 2024-05-17T22:32:45.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:32:45.668+0000 7f50baee3700 1 --2- 172.21.15.96:0/3136866443 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50b4104660 0x7f50b410af10 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:32:45.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:32:45.668+0000 7f50baee3700 1 --2- 172.21.15.96:0/3136866443 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50b4102b90 0x7f50b4102fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:32:45.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:32:45.668+0000 7f50baee3700 1 --2- 172.21.15.96:0/3136866443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f50b4103cc0 0x7f50b41040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:32:45.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:32:45.668+0000 7f50baee3700 1 -- 172.21.15.96:0/3136866443 >> 172.21.15.96:0/3136866443 conn(0x7f50b40fe3c0 msgr2=0x7f50b4100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:32:45.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:32:45.668+0000 7f50baee3700 1 -- 172.21.15.96:0/3136866443 shutdown_connections 2024-05-17T22:32:45.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:32:45.668+0000 7f50baee3700 1 -- 172.21.15.96:0/3136866443 wait complete. 2024-05-17T22:32:45.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:32:45.669+0000 7f50baee3700 1 Processor -- start 2024-05-17T22:32:45.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:32:45.669+0000 7f50baee3700 1 -- start start 2024-05-17T22:32:45.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:32:45.669+0000 7f50baee3700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50b4102b90 0x7f50b406caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:32:45.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:32:45.669+0000 7f50baee3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f50b4103cc0 0x7f50b419eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:32:45.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:32:45.669+0000 7f50baee3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50b4104660 0x7f50b41a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:32:45.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:32:45.669+0000 7f50baee3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f50b410e610 con 0x7f50b4103cc0 2024-05-17T22:32:45.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:32:45.669+0000 7f50baee3700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f50b410e490 con 0x7f50b4104660 2024-05-17T22:32:45.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:32:45.669+0000 7f50baee3700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f50b410e790 con 0x7f50b4102b90 2024-05-17T22:32:45.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:32:45.669+0000 7f50b9480700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50b4104660 0x7f50b41a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:32:45.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:32:45.669+0000 7f50b9480700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50b4104660 0x7f50b41a1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56252/0 (socket says 172.21.15.96:56252) 2024-05-17T22:32:45.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:32:45.669+0000 7f50b9480700 1 -- 172.21.15.96:0/3891607372 learned_addr learned my addr 172.21.15.96:0/3891607372 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-05-17T22:32:45.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:32:45.669+0000 7f50b3fff700 1 --2- 172.21.15.96:0/3891607372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f50b4103cc0 0x7f50b419eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:32:45.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:32:45.669+0000 7f50b8c7f700 1 --2- 172.21.15.96:0/3891607372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50b4102b90 0x7f50b406caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:32:45.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:32:45.669+0000 7f50b9480700 1 -- 172.21.15.96:0/3891607372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50b4102b90 msgr2=0x7f50b406caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:32:45.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:32:45.669+0000 7f50b9480700 1 --2- 172.21.15.96:0/3891607372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50b4102b90 0x7f50b406caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:32:45.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:32:45.669+0000 7f50b9480700 1 -- 172.21.15.96:0/3891607372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f50b4103cc0 msgr2=0x7f50b419eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:32:45.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:32:45.669+0000 7f50b9480700 1 --2- 172.21.15.96:0/3891607372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f50b4103cc0 0x7f50b419eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:32:45.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:32:45.669+0000 7f50b9480700 1 -- 172.21.15.96:0/3891607372 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50b41a1d90 con 0x7f50b4104660 2024-05-17T22:32:45.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:32:45.669+0000 7f50b8c7f700 1 --2- 172.21.15.96:0/3891607372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50b4102b90 0x7f50b406caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:32:45.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:32:45.670+0000 7f50b9480700 1 --2- 172.21.15.96:0/3891607372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50b4104660 0x7f50b41a1560 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f50ac0062a0 tx=0x7f50ac004dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:32:45.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:32:45.670+0000 7f50b1ffb700 1 -- 172.21.15.96:0/3891607372 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac021070 con 0x7f50b4104660 2024-05-17T22:32:45.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:32:45.670+0000 7f50baee3700 1 -- 172.21.15.96:0/3891607372 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50b41a2020 con 0x7f50b4104660 2024-05-17T22:32:45.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:32:45.670+0000 7f50b1ffb700 1 -- 172.21.15.96:0/3891607372 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50ac007b80 con 0x7f50b4104660 2024-05-17T22:32:45.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:32:45.670+0000 7f50b1ffb700 1 -- 172.21.15.96:0/3891607372 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac0047b0 con 0x7f50b4104660 2024-05-17T22:32:45.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:32:45.670+0000 7f50baee3700 1 -- 172.21.15.96:0/3891607372 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50b41a2520 con 0x7f50b4104660 2024-05-17T22:32:45.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:32:45.671+0000 7f50baee3700 1 -- 172.21.15.96:0/3891607372 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50b4102fa0 con 0x7f50b4104660 2024-05-17T22:32:45.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:32:45.672+0000 7f50b1ffb700 1 -- 172.21.15.96:0/3891607372 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f50ac003a00 con 0x7f50b4104660 2024-05-17T22:32:45.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:32:45.673+0000 7f50b1ffb700 1 --2- 172.21.15.96:0/3891607372 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f50a40b2900 0x7f50a40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:32:45.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:32:45.673+0000 7f50b1ffb700 1 -- 172.21.15.96:0/3891607372 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6261+0+0 (secure 0 0 0) 0x7f50ac00f020 con 0x7f50b4104660 2024-05-17T22:32:45.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:32:45.673+0000 7f50b8c7f700 1 --2- 172.21.15.96:0/3891607372 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f50a40b2900 0x7f50a40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:32:45.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:32:45.674+0000 7f50b8c7f700 1 --2- 172.21.15.96:0/3891607372 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f50a40b2900 0x7f50a40b4e10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f50a8005950 tx=0x7f50a8005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:32:45.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:32:45.676+0000 7f50b1ffb700 1 -- 172.21.15.96:0/3891607372 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f50ac02a020 con 0x7f50b4104660 2024-05-17T22:32:45.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:32:45.888+0000 7f50baee3700 1 -- 172.21.15.96:0/3891607372 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f50b4064720 con 0x7f50a40b2900 2024-05-17T22:32:46.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:46 smithi151 ceph-mon[36101]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:32:46.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:46 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T22:32:46.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:46 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T22:32:46.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:46 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:32:46.499 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:46 smithi096 ceph-mon[36065]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:32:46.499 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:46 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T22:32:46.499 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:46 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T22:32:46.499 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:46 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:32:46.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:46 smithi012 ceph-mon[31632]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:32:46.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:46 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T22:32:46.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:46 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T22:32:46.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:46 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:32:47.471 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:47 smithi096 ceph-mon[36065]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-17T22:32:47.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:47 smithi151 ceph-mon[36101]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-17T22:32:47.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:47 smithi012 ceph-mon[31632]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-17T22:32:49.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:49 smithi096 ceph-mon[36065]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-17T22:32:49.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:49 smithi096 ceph-mon[36065]: from='client.? 172.21.15.96:0/1461617525' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "347e4e40-9a9d-43fe-8f52-fdd3199a2a5c"}]: dispatch 2024-05-17T22:32:49.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:49 smithi096 ceph-mon[36065]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "347e4e40-9a9d-43fe-8f52-fdd3199a2a5c"}]: dispatch 2024-05-17T22:32:49.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:49 smithi096 ceph-mon[36065]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "347e4e40-9a9d-43fe-8f52-fdd3199a2a5c"}]': finished 2024-05-17T22:32:49.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:49 smithi096 ceph-mon[36065]: osdmap e61: 11 total, 10 up, 11 in 2024-05-17T22:32:49.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:49 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T22:32:49.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:49 smithi096 ceph-mon[36065]: from='client.? 172.21.15.96:0/3902384566' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T22:32:49.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:49 smithi012 ceph-mon[31632]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-17T22:32:49.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:49 smithi012 ceph-mon[31632]: from='client.? 172.21.15.96:0/1461617525' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "347e4e40-9a9d-43fe-8f52-fdd3199a2a5c"}]: dispatch 2024-05-17T22:32:49.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:49 smithi012 ceph-mon[31632]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "347e4e40-9a9d-43fe-8f52-fdd3199a2a5c"}]: dispatch 2024-05-17T22:32:49.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:49 smithi012 ceph-mon[31632]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "347e4e40-9a9d-43fe-8f52-fdd3199a2a5c"}]': finished 2024-05-17T22:32:49.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:49 smithi012 ceph-mon[31632]: osdmap e61: 11 total, 10 up, 11 in 2024-05-17T22:32:49.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:49 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T22:32:49.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:49 smithi012 ceph-mon[31632]: from='client.? 172.21.15.96:0/3902384566' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T22:32:49.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:49 smithi151 ceph-mon[36101]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-17T22:32:49.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:49 smithi151 ceph-mon[36101]: from='client.? 172.21.15.96:0/1461617525' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "347e4e40-9a9d-43fe-8f52-fdd3199a2a5c"}]: dispatch 2024-05-17T22:32:49.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:49 smithi151 ceph-mon[36101]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "347e4e40-9a9d-43fe-8f52-fdd3199a2a5c"}]: dispatch 2024-05-17T22:32:49.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:49 smithi151 ceph-mon[36101]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "347e4e40-9a9d-43fe-8f52-fdd3199a2a5c"}]': finished 2024-05-17T22:32:49.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:49 smithi151 ceph-mon[36101]: osdmap e61: 11 total, 10 up, 11 in 2024-05-17T22:32:49.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:49 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T22:32:49.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:49 smithi151 ceph-mon[36101]: from='client.? 172.21.15.96:0/3902384566' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T22:32:51.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:51 smithi096 ceph-mon[36065]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-17T22:32:51.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:51 smithi012 ceph-mon[31632]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-17T22:32:51.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:51 smithi151 ceph-mon[36101]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-17T22:32:53.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:53 smithi096 ceph-mon[36065]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-17T22:32:53.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:53 smithi012 ceph-mon[31632]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-17T22:32:53.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:53 smithi151 ceph-mon[36101]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-17T22:32:55.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:55 smithi096 ceph-mon[36065]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-17T22:32:55.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:55 smithi012 ceph-mon[31632]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-17T22:32:55.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:55 smithi151 ceph-mon[36101]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-17T22:32:56.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:56 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-17T22:32:56.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:56 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:32:56.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:56 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-17T22:32:56.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:56 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:32:56.891 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:56 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-17T22:32:56.891 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:56 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:32:57.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:57 smithi096 ceph-mon[36065]: Deploying daemon osd.10 on smithi096 2024-05-17T22:32:57.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:57 smithi096 ceph-mon[36065]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-17T22:32:57.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:57 smithi012 ceph-mon[31632]: Deploying daemon osd.10 on smithi096 2024-05-17T22:32:57.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:57 smithi012 ceph-mon[31632]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-17T22:32:57.903 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:57 smithi151 ceph-mon[36101]: Deploying daemon osd.10 on smithi096 2024-05-17T22:32:57.903 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:57 smithi151 ceph-mon[36101]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-17T22:32:59.702 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:32:59 smithi096 ceph-mon[36065]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-17T22:32:59.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:32:59 smithi012 ceph-mon[31632]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-17T22:32:59.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:32:59 smithi151 ceph-mon[36101]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-17T22:33:01.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:01 smithi096 ceph-mon[36065]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-17T22:33:01.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:01 smithi012 ceph-mon[31632]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-17T22:33:01.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:01 smithi151 ceph-mon[36101]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-17T22:33:02.743 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:02 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:33:02.743 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:02 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:02.743 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:02 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:02.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:02 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:33:02.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:02 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:02.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:02 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:02.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:02 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:33:02.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:02 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:02.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:02 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:03.738 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:03 smithi096 ceph-mon[36065]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-17T22:33:03.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:03 smithi012 ceph-mon[31632]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-17T22:33:03.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:03 smithi151 ceph-mon[36101]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-17T22:33:05.640 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:05 smithi096 ceph-mon[36065]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-17T22:33:05.640 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:05 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:05.641 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:05 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:05.641 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:05 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:33:05.641 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:05 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:33:05.641 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:05 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:05.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:05 smithi012 ceph-mon[31632]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-17T22:33:05.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:05 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:05.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:05 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:05.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:05 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:33:05.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:05 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:33:05.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:05 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:05.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:05 smithi151 ceph-mon[36101]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-17T22:33:05.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:05 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:05.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:05 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:05.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:05 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:33:05.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:05 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:33:05.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:05 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:06.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:33:06.006+0000 7f50b1ffb700 1 -- 172.21.15.96:0/3891607372 <== mgr.14152 v2:172.21.15.12:6800/1388514047 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f50b4064720 con 0x7f50a40b2900 2024-05-17T22:33:06.009 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 10 on host 'smithi096' 2024-05-17T22:33:06.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:33:06.009+0000 7f50baee3700 1 -- 172.21.15.96:0/3891607372 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f50a40b2900 msgr2=0x7f50a40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:33:06.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:33:06.009+0000 7f50baee3700 1 --2- 172.21.15.96:0/3891607372 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f50a40b2900 0x7f50a40b4e10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f50a8005950 tx=0x7f50a8005f50 comp rx=0 tx=0).stop 2024-05-17T22:33:06.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:33:06.009+0000 7f50baee3700 1 -- 172.21.15.96:0/3891607372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50b4104660 msgr2=0x7f50b41a1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:33:06.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:33:06.009+0000 7f50baee3700 1 --2- 172.21.15.96:0/3891607372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50b4104660 0x7f50b41a1560 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f50ac0062a0 tx=0x7f50ac004dc0 comp rx=0 tx=0).stop 2024-05-17T22:33:06.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:33:06.009+0000 7f50baee3700 1 -- 172.21.15.96:0/3891607372 shutdown_connections 2024-05-17T22:33:06.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:33:06.009+0000 7f50baee3700 1 --2- 172.21.15.96:0/3891607372 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f50a40b2900 0x7f50a40b4e10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:06.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:33:06.009+0000 7f50baee3700 1 --2- 172.21.15.96:0/3891607372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50b4104660 0x7f50b41a1560 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:06.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:33:06.009+0000 7f50baee3700 1 --2- 172.21.15.96:0/3891607372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50b4102b90 0x7f50b406caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:06.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:33:06.009+0000 7f50baee3700 1 --2- 172.21.15.96:0/3891607372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f50b4103cc0 0x7f50b419eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:06.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:33:06.009+0000 7f50baee3700 1 -- 172.21.15.96:0/3891607372 >> 172.21.15.96:0/3891607372 conn(0x7f50b40fe3c0 msgr2=0x7f50b4106ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:33:06.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:33:06.009+0000 7f50baee3700 1 -- 172.21.15.96:0/3891607372 shutdown_connections 2024-05-17T22:33:06.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-17T22:33:06.009+0000 7f50baee3700 1 -- 172.21.15.96:0/3891607372 wait complete. 2024-05-17T22:33:06.713 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:06 smithi096 ceph-mon[36065]: from='osd.10 [v2:172.21.15.96:6824/304016309,v1:172.21.15.96:6825/304016309]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-17T22:33:06.713 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:06 smithi096 ceph-mon[36065]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-17T22:33:06.713 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:06 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:06.713 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:06 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:06.714 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:06 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:33:06.740 DEBUG:teuthology.orchestra.run.smithi096:osd.10> sudo journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@osd.10.service 2024-05-17T22:33:06.743 INFO:tasks.cephadm:Deploying osd.11 on smithi151 with /dev/vg_nvme/lv_1... 2024-05-17T22:33:06.743 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-17T22:33:06.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:06 smithi012 ceph-mon[31632]: from='osd.10 [v2:172.21.15.96:6824/304016309,v1:172.21.15.96:6825/304016309]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-17T22:33:06.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:06 smithi012 ceph-mon[31632]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-17T22:33:06.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:06 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:06.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:06 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:06.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:06 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:33:06.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:06 smithi151 ceph-mon[36101]: from='osd.10 [v2:172.21.15.96:6824/304016309,v1:172.21.15.96:6825/304016309]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-17T22:33:06.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:06 smithi151 ceph-mon[36101]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-17T22:33:06.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:06 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:06.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:06 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:06.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:06 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:33:06.993 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T22:33:07.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:07 smithi151 ceph-mon[36101]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-17T22:33:07.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:07 smithi151 ceph-mon[36101]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-17T22:33:07.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:07 smithi151 ceph-mon[36101]: osdmap e62: 11 total, 10 up, 11 in 2024-05-17T22:33:07.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:07 smithi151 ceph-mon[36101]: from='osd.10 [v2:172.21.15.96:6824/304016309,v1:172.21.15.96:6825/304016309]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-17T22:33:07.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:07 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T22:33:07.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:07 smithi151 ceph-mon[36101]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-17T22:33:07.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:07 smithi012 ceph-mon[31632]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-17T22:33:07.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:07 smithi012 ceph-mon[31632]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-17T22:33:07.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:07 smithi012 ceph-mon[31632]: osdmap e62: 11 total, 10 up, 11 in 2024-05-17T22:33:07.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:07 smithi012 ceph-mon[31632]: from='osd.10 [v2:172.21.15.96:6824/304016309,v1:172.21.15.96:6825/304016309]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-17T22:33:07.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:07 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T22:33:07.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:07 smithi012 ceph-mon[31632]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-17T22:33:07.967 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:07 smithi096 ceph-mon[36065]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-17T22:33:07.967 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:07 smithi096 ceph-mon[36065]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-17T22:33:07.967 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:07 smithi096 ceph-mon[36065]: osdmap e62: 11 total, 10 up, 11 in 2024-05-17T22:33:07.968 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:07 smithi096 ceph-mon[36065]: from='osd.10 [v2:172.21.15.96:6824/304016309,v1:172.21.15.96:6825/304016309]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-17T22:33:07.968 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:07 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T22:33:07.968 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:07 smithi096 ceph-mon[36065]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-17T22:33:07.968 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:33:07 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:33:07.556+0000 7f16d2111700 -1 osd.10 0 waiting for initial osdmap 2024-05-17T22:33:07.968 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:33:07 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:33:07.567+0000 7f16cdec6700 -1 osd.10 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T22:33:08.712 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:08 smithi151 ceph-mon[36101]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-05-17T22:33:08.712 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:08 smithi151 ceph-mon[36101]: osdmap e63: 11 total, 10 up, 11 in 2024-05-17T22:33:08.712 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:08 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T22:33:08.712 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:08 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T22:33:08.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:08 smithi096 ceph-mon[36065]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-05-17T22:33:08.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:08 smithi096 ceph-mon[36065]: osdmap e63: 11 total, 10 up, 11 in 2024-05-17T22:33:08.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:08 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T22:33:08.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:08 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T22:33:08.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:08 smithi012 ceph-mon[31632]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-05-17T22:33:08.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:08 smithi012 ceph-mon[31632]: osdmap e63: 11 total, 10 up, 11 in 2024-05-17T22:33:08.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:08 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T22:33:08.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:08 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T22:33:09.171 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-17T22:33:09.171 INFO:teuthology.orchestra.run.smithi151.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-17T22:33:09.171 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10+0 records in 2024-05-17T22:33:09.171 INFO:teuthology.orchestra.run.smithi151.stderr:10+0 records out 2024-05-17T22:33:09.171 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114664 s, 914 MB/s 2024-05-17T22:33:09.171 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping successful for: 2024-05-17T22:33:09.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:09 smithi151 ceph-mon[36101]: purged_snaps scrub starts 2024-05-17T22:33:09.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:09 smithi151 ceph-mon[36101]: purged_snaps scrub ok 2024-05-17T22:33:09.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:09 smithi151 ceph-mon[36101]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-17T22:33:09.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:09 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T22:33:09.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:09 smithi151 ceph-mon[36101]: osd.10 [v2:172.21.15.96:6824/304016309,v1:172.21.15.96:6825/304016309] boot 2024-05-17T22:33:09.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:09 smithi151 ceph-mon[36101]: osdmap e64: 11 total, 11 up, 11 in 2024-05-17T22:33:09.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:09 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T22:33:09.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:09 smithi012 ceph-mon[31632]: purged_snaps scrub starts 2024-05-17T22:33:09.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:09 smithi012 ceph-mon[31632]: purged_snaps scrub ok 2024-05-17T22:33:09.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:09 smithi012 ceph-mon[31632]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-17T22:33:09.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:09 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T22:33:09.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:09 smithi012 ceph-mon[31632]: osd.10 [v2:172.21.15.96:6824/304016309,v1:172.21.15.96:6825/304016309] boot 2024-05-17T22:33:09.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:09 smithi012 ceph-mon[31632]: osdmap e64: 11 total, 11 up, 11 in 2024-05-17T22:33:09.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:09 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T22:33:09.876 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:09 smithi096 ceph-mon[36065]: purged_snaps scrub starts 2024-05-17T22:33:09.876 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:09 smithi096 ceph-mon[36065]: purged_snaps scrub ok 2024-05-17T22:33:09.876 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:09 smithi096 ceph-mon[36065]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-17T22:33:09.877 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T22:33:09.877 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:09 smithi096 ceph-mon[36065]: osd.10 [v2:172.21.15.96:6824/304016309,v1:172.21.15.96:6825/304016309] boot 2024-05-17T22:33:09.877 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:09 smithi096 ceph-mon[36065]: osdmap e64: 11 total, 11 up, 11 in 2024-05-17T22:33:09.877 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T22:33:09.993 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi151:vg_nvme/lv_1 2024-05-17T22:33:10.239 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T22:33:10.618 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:10 smithi151 ceph-mon[36101]: osdmap e65: 11 total, 11 up, 11 in 2024-05-17T22:33:10.619 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:10 smithi151 ceph-mon[36101]: Detected new or changed devices on smithi096 2024-05-17T22:33:10.619 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:10 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:10.619 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:10 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:10.619 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:10 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:33:10.619 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:10 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:33:10.619 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:10 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:33:10.619 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:10 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:33:10.619 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:10 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:10.619 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:10 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:33:10.619 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:10 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:33:10.619 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:10 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:10.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:10 smithi012 ceph-mon[31632]: osdmap e65: 11 total, 11 up, 11 in 2024-05-17T22:33:10.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:10 smithi012 ceph-mon[31632]: Detected new or changed devices on smithi096 2024-05-17T22:33:10.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:10 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:10.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:10 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:10.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:10 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:33:10.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:10 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:33:10.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:10 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:33:10.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:10 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:33:10.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:10 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:10.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:10 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:33:10.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:10 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:33:10.843 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:10 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:11.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:10 smithi096 ceph-mon[36065]: osdmap e65: 11 total, 11 up, 11 in 2024-05-17T22:33:11.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:10 smithi096 ceph-mon[36065]: Detected new or changed devices on smithi096 2024-05-17T22:33:11.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:10 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:11.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:10 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:11.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:10 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:33:11.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:10 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:33:11.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:10 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:33:11.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:10 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:33:11.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:10 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:11.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:10 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:33:11.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:10 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:33:11.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:10 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:11.681 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:11 smithi151 ceph-mon[36101]: Adjusting osd_memory_target on smithi096 to 4272M 2024-05-17T22:33:11.681 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:11 smithi151 ceph-mon[36101]: pgmap v233: 1 pgs: 1 remapped+peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-05-17T22:33:11.681 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:11 smithi151 ceph-mon[36101]: osdmap e66: 11 total, 11 up, 11 in 2024-05-17T22:33:11.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:11 smithi012 ceph-mon[31632]: Adjusting osd_memory_target on smithi096 to 4272M 2024-05-17T22:33:11.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:11 smithi012 ceph-mon[31632]: pgmap v233: 1 pgs: 1 remapped+peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-05-17T22:33:11.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:11 smithi012 ceph-mon[31632]: osdmap e66: 11 total, 11 up, 11 in 2024-05-17T22:33:12.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:11.941+0000 7f9bd5bb3700 1 -- 172.21.15.151:0/476422289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bd01070c0 msgr2=0x7f9bd01094e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:33:12.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:11.941+0000 7f9bd5bb3700 1 --2- 172.21.15.151:0/476422289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bd01070c0 0x7f9bd01094e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4009990 tx=0x7f9bc4009ca0 comp rx=0 tx=0).stop 2024-05-17T22:33:12.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:11.941+0000 7f9bd5bb3700 1 -- 172.21.15.151:0/476422289 shutdown_connections 2024-05-17T22:33:12.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:11.941+0000 7f9bd5bb3700 1 --2- 172.21.15.151:0/476422289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bd0109a20 0x7f9bd010be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:12.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:11.941+0000 7f9bd5bb3700 1 --2- 172.21.15.151:0/476422289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bd0101fd0 0x7f9bd0102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:12.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:11.941+0000 7f9bd5bb3700 1 --2- 172.21.15.151:0/476422289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bd01070c0 0x7f9bd01094e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:12.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:11.941+0000 7f9bd5bb3700 1 -- 172.21.15.151:0/476422289 >> 172.21.15.151:0/476422289 conn(0x7f9bd00fdb40 msgr2=0x7f9bd00fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:33:12.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:11.941+0000 7f9bd5bb3700 1 -- 172.21.15.151:0/476422289 shutdown_connections 2024-05-17T22:33:12.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:11.941+0000 7f9bd5bb3700 1 -- 172.21.15.151:0/476422289 wait complete. 2024-05-17T22:33:12.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:11.942+0000 7f9bd5bb3700 1 Processor -- start 2024-05-17T22:33:12.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:11.942+0000 7f9bd5bb3700 1 -- start start 2024-05-17T22:33:12.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:11.942+0000 7f9bd5bb3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bd0101fd0 0x7f9bd019a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:33:12.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:11.942+0000 7f9bd5bb3700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bd01070c0 0x7f9bd019a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:33:12.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:11.942+0000 7f9bd5bb3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bd0109a20 0x7f9bd01a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:33:12.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:11.942+0000 7f9bd5bb3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd010f6e0 con 0x7f9bd0109a20 2024-05-17T22:33:12.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:11.942+0000 7f9bd5bb3700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd010f560 con 0x7f9bd0101fd0 2024-05-17T22:33:12.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:11.942+0000 7f9bd5bb3700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd010f860 con 0x7f9bd01070c0 2024-05-17T22:33:12.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:11.942+0000 7f9bceffd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bd01070c0 0x7f9bd019a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:33:12.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:11.942+0000 7f9bceffd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bd01070c0 0x7f9bd019a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43862/0 (socket says 172.21.15.151:43862) 2024-05-17T22:33:12.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:11.942+0000 7f9bceffd700 1 -- 172.21.15.151:0/3488988793 learned_addr learned my addr 172.21.15.151:0/3488988793 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-17T22:33:12.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:11.943+0000 7f9bcffff700 1 --2- 172.21.15.151:0/3488988793 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bd0109a20 0x7f9bd01a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:33:12.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:11.943+0000 7f9bcf7fe700 1 --2- 172.21.15.151:0/3488988793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bd0101fd0 0x7f9bd019a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:33:12.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:11.943+0000 7f9bceffd700 1 -- 172.21.15.151:0/3488988793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bd0101fd0 msgr2=0x7f9bd019a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:33:12.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:11.943+0000 7f9bceffd700 1 --2- 172.21.15.151:0/3488988793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bd0101fd0 0x7f9bd019a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:12.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:11.943+0000 7f9bceffd700 1 -- 172.21.15.151:0/3488988793 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bd0109a20 msgr2=0x7f9bd01a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:33:12.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:11.943+0000 7f9bceffd700 1 --2- 172.21.15.151:0/3488988793 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bd0109a20 0x7f9bd01a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:12.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:11.943+0000 7f9bceffd700 1 -- 172.21.15.151:0/3488988793 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bd01a1dc0 con 0x7f9bd01070c0 2024-05-17T22:33:12.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:11.943+0000 7f9bceffd700 1 --2- 172.21.15.151:0/3488988793 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bd01070c0 0x7f9bd019a7a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9bc40098e0 tx=0x7f9bc40047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:33:12.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:11.943+0000 7f9bccff9700 1 -- 172.21.15.151:0/3488988793 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc4026070 con 0x7f9bd01070c0 2024-05-17T22:33:12.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:11.943+0000 7f9bd5bb3700 1 -- 172.21.15.151:0/3488988793 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bd01a2050 con 0x7f9bd01070c0 2024-05-17T22:33:12.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:11.943+0000 7f9bccff9700 1 -- 172.21.15.151:0/3488988793 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bc4021410 con 0x7f9bd01070c0 2024-05-17T22:33:12.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:11.943+0000 7f9bccff9700 1 -- 172.21.15.151:0/3488988793 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc4017650 con 0x7f9bd01070c0 2024-05-17T22:33:12.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:11.943+0000 7f9bd5bb3700 1 -- 172.21.15.151:0/3488988793 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bd01a2500 con 0x7f9bd01070c0 2024-05-17T22:33:12.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:11.944+0000 7f9bd5bb3700 1 -- 172.21.15.151:0/3488988793 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bd0102400 con 0x7f9bd01070c0 2024-05-17T22:33:12.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:11.946+0000 7f9bccff9700 1 -- 172.21.15.151:0/3488988793 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f9bc4029070 con 0x7f9bd01070c0 2024-05-17T22:33:12.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:11.946+0000 7f9bccff9700 1 --2- 172.21.15.151:0/3488988793 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9bb80b2880 0x7f9bb80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:33:12.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:11.946+0000 7f9bccff9700 1 -- 172.21.15.151:0/3488988793 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6694+0+0 (secure 0 0 0) 0x7f9bc40e2cb0 con 0x7f9bd01070c0 2024-05-17T22:33:12.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:11.946+0000 7f9bcf7fe700 1 --2- 172.21.15.151:0/3488988793 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9bb80b2880 0x7f9bb80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:33:12.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:11.947+0000 7f9bcf7fe700 1 --2- 172.21.15.151:0/3488988793 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9bb80b2880 0x7f9bb80b4d90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9bc000f540 tx=0x7f9bc000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:33:12.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:11.950+0000 7f9bccff9700 1 -- 172.21.15.151:0/3488988793 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9bc4013070 con 0x7f9bd01070c0 2024-05-17T22:33:12.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:11 smithi096 ceph-mon[36065]: Adjusting osd_memory_target on smithi096 to 4272M 2024-05-17T22:33:12.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:11 smithi096 ceph-mon[36065]: pgmap v233: 1 pgs: 1 remapped+peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-05-17T22:33:12.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:11 smithi096 ceph-mon[36065]: osdmap e66: 11 total, 11 up, 11 in 2024-05-17T22:33:12.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:12.162+0000 7f9bd5bb3700 1 -- 172.21.15.151:0/3488988793 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f9bd00646c0 con 0x7f9bb80b2880 2024-05-17T22:33:12.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:12 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T22:33:12.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:12 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T22:33:12.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:12 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:33:12.889 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:12 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T22:33:12.889 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:12 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T22:33:12.889 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:12 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:33:13.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:12 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T22:33:13.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:12 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T22:33:13.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:12 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:33:13.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:13 smithi012 ceph-mon[31632]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:33:13.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:13 smithi012 ceph-mon[31632]: pgmap v235: 1 pgs: 1 remapped+peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-05-17T22:33:13.936 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:13 smithi151 ceph-mon[36101]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:33:13.936 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:13 smithi151 ceph-mon[36101]: pgmap v235: 1 pgs: 1 remapped+peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-05-17T22:33:14.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:13 smithi096 ceph-mon[36065]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:33:14.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:13 smithi096 ceph-mon[36065]: pgmap v235: 1 pgs: 1 remapped+peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-05-17T22:33:15.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:15 smithi151 ceph-mon[36101]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-17T22:33:15.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:15 smithi151 ceph-mon[36101]: from='client.? 172.21.15.151:0/812147237' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee8d3502-8b3c-4e6f-824f-3a97e19111ae"}]: dispatch 2024-05-17T22:33:15.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:15 smithi151 ceph-mon[36101]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee8d3502-8b3c-4e6f-824f-3a97e19111ae"}]: dispatch 2024-05-17T22:33:15.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:15 smithi151 ceph-mon[36101]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ee8d3502-8b3c-4e6f-824f-3a97e19111ae"}]': finished 2024-05-17T22:33:15.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:15 smithi151 ceph-mon[36101]: osdmap e67: 12 total, 11 up, 12 in 2024-05-17T22:33:15.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:15 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T22:33:16.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:15 smithi096 ceph-mon[36065]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-17T22:33:16.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:15 smithi096 ceph-mon[36065]: from='client.? 172.21.15.151:0/812147237' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee8d3502-8b3c-4e6f-824f-3a97e19111ae"}]: dispatch 2024-05-17T22:33:16.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:15 smithi096 ceph-mon[36065]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee8d3502-8b3c-4e6f-824f-3a97e19111ae"}]: dispatch 2024-05-17T22:33:16.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:15 smithi096 ceph-mon[36065]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ee8d3502-8b3c-4e6f-824f-3a97e19111ae"}]': finished 2024-05-17T22:33:16.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:15 smithi096 ceph-mon[36065]: osdmap e67: 12 total, 11 up, 12 in 2024-05-17T22:33:16.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:15 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T22:33:16.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:15 smithi012 ceph-mon[31632]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-17T22:33:16.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:15 smithi012 ceph-mon[31632]: from='client.? 172.21.15.151:0/812147237' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee8d3502-8b3c-4e6f-824f-3a97e19111ae"}]: dispatch 2024-05-17T22:33:16.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:15 smithi012 ceph-mon[31632]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee8d3502-8b3c-4e6f-824f-3a97e19111ae"}]: dispatch 2024-05-17T22:33:16.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:15 smithi012 ceph-mon[31632]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ee8d3502-8b3c-4e6f-824f-3a97e19111ae"}]': finished 2024-05-17T22:33:16.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:15 smithi012 ceph-mon[31632]: osdmap e67: 12 total, 11 up, 12 in 2024-05-17T22:33:16.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:15 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T22:33:16.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:16 smithi151 ceph-mon[36101]: from='client.? 172.21.15.151:0/2957893104' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T22:33:17.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:16 smithi096 ceph-mon[36065]: from='client.? 172.21.15.151:0/2957893104' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T22:33:17.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:16 smithi012 ceph-mon[31632]: from='client.? 172.21.15.151:0/2957893104' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T22:33:17.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:17 smithi151 ceph-mon[36101]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-05-17T22:33:18.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:17 smithi096 ceph-mon[36065]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-05-17T22:33:18.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:17 smithi012 ceph-mon[31632]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-05-17T22:33:19.846 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:19 smithi151 ceph-mon[36101]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-17T22:33:20.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:19 smithi096 ceph-mon[36065]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-17T22:33:20.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:19 smithi012 ceph-mon[31632]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-17T22:33:21.783 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:21 smithi151 ceph-mon[36101]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T22:33:22.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:21 smithi096 ceph-mon[36065]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T22:33:22.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:21 smithi012 ceph-mon[31632]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T22:33:23.827 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:23 smithi151 ceph-mon[36101]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T22:33:23.827 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:23 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-17T22:33:23.828 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:23 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:33:23.828 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:23 smithi151 ceph-mon[36101]: Deploying daemon osd.11 on smithi151 2024-05-17T22:33:24.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:23 smithi096 ceph-mon[36065]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T22:33:24.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:23 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-17T22:33:24.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:23 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:33:24.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:23 smithi096 ceph-mon[36065]: Deploying daemon osd.11 on smithi151 2024-05-17T22:33:24.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:23 smithi012 ceph-mon[31632]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T22:33:24.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:23 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-17T22:33:24.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:23 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:33:24.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:23 smithi012 ceph-mon[31632]: Deploying daemon osd.11 on smithi151 2024-05-17T22:33:25.916 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:25 smithi151 ceph-mon[36101]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-17T22:33:26.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:25 smithi096 ceph-mon[36065]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-17T22:33:26.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:25 smithi012 ceph-mon[31632]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-17T22:33:27.864 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:27 smithi151 ceph-mon[36101]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-17T22:33:28.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:27 smithi096 ceph-mon[36065]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-17T22:33:28.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:27 smithi012 ceph-mon[31632]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-17T22:33:29.831 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:29 smithi151 ceph-mon[36101]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-17T22:33:29.832 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:29 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:29.832 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:29 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:29.832 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:29 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:33:29.893 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:29 smithi012 ceph-mon[31632]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-17T22:33:29.894 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:29 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:29.894 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:29 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:29.894 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:29 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:33:30.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:29 smithi096 ceph-mon[36065]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-17T22:33:30.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:29 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:30.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:29 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:30.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:29 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:33:31.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:31 smithi151 ceph-mon[36101]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-17T22:33:32.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:31 smithi096 ceph-mon[36065]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-17T22:33:32.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:31 smithi012 ceph-mon[31632]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-17T22:33:32.835 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 11 on host 'smithi151' 2024-05-17T22:33:32.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:32.833+0000 7f9bccff9700 1 -- 172.21.15.151:0/3488988793 <== mgr.14152 v2:172.21.15.12:6800/1388514047 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f9bd00646c0 con 0x7f9bb80b2880 2024-05-17T22:33:32.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:32.835+0000 7f9bd5bb3700 1 -- 172.21.15.151:0/3488988793 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9bb80b2880 msgr2=0x7f9bb80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:33:32.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:32.835+0000 7f9bd5bb3700 1 --2- 172.21.15.151:0/3488988793 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9bb80b2880 0x7f9bb80b4d90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9bc000f540 tx=0x7f9bc000b040 comp rx=0 tx=0).stop 2024-05-17T22:33:32.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:32.835+0000 7f9bd5bb3700 1 -- 172.21.15.151:0/3488988793 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bd01070c0 msgr2=0x7f9bd019a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:33:32.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:32.835+0000 7f9bd5bb3700 1 --2- 172.21.15.151:0/3488988793 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bd01070c0 0x7f9bd019a7a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9bc40098e0 tx=0x7f9bc40047b0 comp rx=0 tx=0).stop 2024-05-17T22:33:32.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:32.835+0000 7f9bd5bb3700 1 -- 172.21.15.151:0/3488988793 shutdown_connections 2024-05-17T22:33:32.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:32.835+0000 7f9bd5bb3700 1 --2- 172.21.15.151:0/3488988793 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9bb80b2880 0x7f9bb80b4d90 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:32.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:32.835+0000 7f9bd5bb3700 1 --2- 172.21.15.151:0/3488988793 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bd0109a20 0x7f9bd01a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:32.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:32.835+0000 7f9bd5bb3700 1 --2- 172.21.15.151:0/3488988793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bd0101fd0 0x7f9bd019a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:32.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:32.835+0000 7f9bd5bb3700 1 --2- 172.21.15.151:0/3488988793 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bd01070c0 0x7f9bd019a7a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:32.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:32.835+0000 7f9bd5bb3700 1 -- 172.21.15.151:0/3488988793 >> 172.21.15.151:0/3488988793 conn(0x7f9bd00fdb40 msgr2=0x7f9bd0107f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:33:32.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:32.836+0000 7f9bd5bb3700 1 -- 172.21.15.151:0/3488988793 shutdown_connections 2024-05-17T22:33:32.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:32.836+0000 7f9bd5bb3700 1 -- 172.21.15.151:0/3488988793 wait complete. 2024-05-17T22:33:33.532 DEBUG:teuthology.orchestra.run.smithi151:osd.11> sudo journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@osd.11.service 2024-05-17T22:33:33.534 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-17T22:33:33.535 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph osd stat -f json 2024-05-17T22:33:33.802 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:33:33.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:33 smithi012 ceph-mon[31632]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-17T22:33:33.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:33 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:33.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:33 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:33.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:33 smithi012 ceph-mon[31632]: from='osd.11 [v2:172.21.15.151:6824/3822307314,v1:172.21.15.151:6825/3822307314]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-17T22:33:33.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:33 smithi012 ceph-mon[31632]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-17T22:33:33.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:33 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:33.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:33 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:34.237 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:33 smithi151 ceph-mon[36101]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-17T22:33:34.238 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:33 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:34.238 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:33 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:34.238 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:33 smithi151 ceph-mon[36101]: from='osd.11 [v2:172.21.15.151:6824/3822307314,v1:172.21.15.151:6825/3822307314]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-17T22:33:34.238 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:33 smithi151 ceph-mon[36101]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-17T22:33:34.238 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:33 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:34.238 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:33 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:34.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:33 smithi096 ceph-mon[36065]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-17T22:33:34.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:33 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:34.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:33 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:34.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:33 smithi096 ceph-mon[36065]: from='osd.11 [v2:172.21.15.151:6824/3822307314,v1:172.21.15.151:6825/3822307314]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-17T22:33:34.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:33 smithi096 ceph-mon[36065]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-17T22:33:34.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:33 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:34.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:33 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:35.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:34 smithi012 ceph-mon[31632]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-17T22:33:35.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:34 smithi012 ceph-mon[31632]: osdmap e68: 12 total, 11 up, 12 in 2024-05-17T22:33:35.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:34 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T22:33:35.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:34 smithi012 ceph-mon[31632]: from='osd.11 [v2:172.21.15.151:6824/3822307314,v1:172.21.15.151:6825/3822307314]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-17T22:33:35.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:34 smithi012 ceph-mon[31632]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-17T22:33:35.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:34 smithi151 ceph-mon[36101]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-17T22:33:35.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:34 smithi151 ceph-mon[36101]: osdmap e68: 12 total, 11 up, 12 in 2024-05-17T22:33:35.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:34 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T22:33:35.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:34 smithi151 ceph-mon[36101]: from='osd.11 [v2:172.21.15.151:6824/3822307314,v1:172.21.15.151:6825/3822307314]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-17T22:33:35.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:34 smithi151 ceph-mon[36101]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-17T22:33:35.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:34 smithi096 ceph-mon[36065]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-17T22:33:35.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:34 smithi096 ceph-mon[36065]: osdmap e68: 12 total, 11 up, 12 in 2024-05-17T22:33:35.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:34 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T22:33:35.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:34 smithi096 ceph-mon[36065]: from='osd.11 [v2:172.21.15.151:6824/3822307314,v1:172.21.15.151:6825/3822307314]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-17T22:33:35.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:34 smithi096 ceph-mon[36065]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-17T22:33:35.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.672+0000 7fba943d8700 1 -- 172.21.15.12:0/2369567085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba8c101c70 msgr2=0x7fba8c102080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:33:35.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.672+0000 7fba943d8700 1 --2- 172.21.15.12:0/2369567085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba8c101c70 0x7fba8c102080 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fba7c009990 tx=0x7fba7c009ca0 comp rx=0 tx=0).stop 2024-05-17T22:33:35.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.672+0000 7fba943d8700 1 -- 172.21.15.12:0/2369567085 shutdown_connections 2024-05-17T22:33:35.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.672+0000 7fba943d8700 1 --2- 172.21.15.12:0/2369567085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba8c1025c0 0x7fba8c10c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:35.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.672+0000 7fba943d8700 1 --2- 172.21.15.12:0/2369567085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba8c105220 0x7fba8c105630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:35.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.672+0000 7fba943d8700 1 --2- 172.21.15.12:0/2369567085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba8c101c70 0x7fba8c102080 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:35.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.672+0000 7fba943d8700 1 -- 172.21.15.12:0/2369567085 >> 172.21.15.12:0/2369567085 conn(0x7fba8c0fdb50 msgr2=0x7fba8c0fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:33:35.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.672+0000 7fba943d8700 1 -- 172.21.15.12:0/2369567085 shutdown_connections 2024-05-17T22:33:35.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.672+0000 7fba943d8700 1 -- 172.21.15.12:0/2369567085 wait complete. 2024-05-17T22:33:35.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.673+0000 7fba943d8700 1 Processor -- start 2024-05-17T22:33:35.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.673+0000 7fba943d8700 1 -- start start 2024-05-17T22:33:35.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.673+0000 7fba943d8700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba8c101c70 0x7fba8c1047f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:33:35.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.673+0000 7fba943d8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba8c1025c0 0x7fba8c102ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:33:35.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.673+0000 7fba943d8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba8c105220 0x7fba8c1033e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:33:35.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.673+0000 7fba943d8700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fba8c10f700 con 0x7fba8c105220 2024-05-17T22:33:35.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.673+0000 7fba943d8700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fba8c10f580 con 0x7fba8c1025c0 2024-05-17T22:33:35.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.673+0000 7fba943d8700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fba8c10f880 con 0x7fba8c101c70 2024-05-17T22:33:35.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.673+0000 7fba92975700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba8c105220 0x7fba8c1033e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:33:35.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.673+0000 7fba92975700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba8c105220 0x7fba8c1033e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56730/0 (socket says 172.21.15.12:56730) 2024-05-17T22:33:35.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.673+0000 7fba92975700 1 -- 172.21.15.12:0/3129645434 learned_addr learned my addr 172.21.15.12:0/3129645434 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:33:35.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.673+0000 7fba92174700 1 --2- 172.21.15.12:0/3129645434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba8c101c70 0x7fba8c1047f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:33:35.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.673+0000 7fba91973700 1 --2- 172.21.15.12:0/3129645434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba8c1025c0 0x7fba8c102ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:33:35.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.673+0000 7fba92975700 1 -- 172.21.15.12:0/3129645434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba8c101c70 msgr2=0x7fba8c1047f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:33:35.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.673+0000 7fba92975700 1 --2- 172.21.15.12:0/3129645434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba8c101c70 0x7fba8c1047f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:35.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.673+0000 7fba92975700 1 -- 172.21.15.12:0/3129645434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba8c1025c0 msgr2=0x7fba8c102ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:33:35.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.673+0000 7fba92975700 1 --2- 172.21.15.12:0/3129645434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba8c1025c0 0x7fba8c102ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:35.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.673+0000 7fba92975700 1 -- 172.21.15.12:0/3129645434 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba8c103b40 con 0x7fba8c105220 2024-05-17T22:33:35.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.673+0000 7fba91973700 1 --2- 172.21.15.12:0/3129645434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba8c1025c0 0x7fba8c102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:33:35.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.673+0000 7fba92975700 1 --2- 172.21.15.12:0/3129645434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba8c105220 0x7fba8c1033e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fba8400ef30 tx=0x7fba8400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:33:35.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.674+0000 7fba837fe700 1 -- 172.21.15.12:0/3129645434 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba840092e0 con 0x7fba8c105220 2024-05-17T22:33:35.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.674+0000 7fba943d8700 1 -- 172.21.15.12:0/3129645434 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba8c1a4330 con 0x7fba8c105220 2024-05-17T22:33:35.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.674+0000 7fba837fe700 1 -- 172.21.15.12:0/3129645434 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba84004bb0 con 0x7fba8c105220 2024-05-17T22:33:35.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.674+0000 7fba837fe700 1 -- 172.21.15.12:0/3129645434 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba84004280 con 0x7fba8c105220 2024-05-17T22:33:35.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.674+0000 7fba943d8700 1 -- 172.21.15.12:0/3129645434 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba8c1a4790 con 0x7fba8c105220 2024-05-17T22:33:35.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.675+0000 7fba943d8700 1 -- 172.21.15.12:0/3129645434 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba8c102080 con 0x7fba8c105220 2024-05-17T22:33:35.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.677+0000 7fba837fe700 1 -- 172.21.15.12:0/3129645434 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fba8401b030 con 0x7fba8c105220 2024-05-17T22:33:35.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.678+0000 7fba837fe700 1 --2- 172.21.15.12:0/3129645434 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fba780b2880 0x7fba780b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:33:35.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.678+0000 7fba92174700 1 --2- 172.21.15.12:0/3129645434 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fba780b2880 0x7fba780b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:33:35.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.678+0000 7fba837fe700 1 -- 172.21.15.12:0/3129645434 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6847+0+0 (secure 0 0 0) 0x7fba840e2210 con 0x7fba8c105220 2024-05-17T22:33:35.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.679+0000 7fba92174700 1 --2- 172.21.15.12:0/3129645434 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fba780b2880 0x7fba780b4d90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fba88005950 tx=0x7fba8800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:33:35.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.681+0000 7fba837fe700 1 -- 172.21.15.12:0/3129645434 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fba84022020 con 0x7fba8c105220 2024-05-17T22:33:35.804 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:33:35 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:33:35.548+0000 7fab24ea9700 -1 osd.11 0 waiting for initial osdmap 2024-05-17T22:33:35.804 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:33:35 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:33:35.564+0000 7fab1cc56700 -1 osd.11 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T22:33:35.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.898+0000 7fba943d8700 1 -- 172.21.15.12:0/3129645434 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fba8c064580 con 0x7fba8c105220 2024-05-17T22:33:35.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.898+0000 7fba837fe700 1 -- 172.21.15.12:0/3129645434 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v69) ==== 74+0+133 (secure 0 0 0) 0x7fba84010040 con 0x7fba8c105220 2024-05-17T22:33:35.901 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:33:35.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.900+0000 7fba943d8700 1 -- 172.21.15.12:0/3129645434 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fba780b2880 msgr2=0x7fba780b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:33:35.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.900+0000 7fba943d8700 1 --2- 172.21.15.12:0/3129645434 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fba780b2880 0x7fba780b4d90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fba88005950 tx=0x7fba8800b410 comp rx=0 tx=0).stop 2024-05-17T22:33:35.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.900+0000 7fba943d8700 1 -- 172.21.15.12:0/3129645434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba8c105220 msgr2=0x7fba8c1033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:33:35.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.900+0000 7fba943d8700 1 --2- 172.21.15.12:0/3129645434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba8c105220 0x7fba8c1033e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fba8400ef30 tx=0x7fba8400c3a0 comp rx=0 tx=0).stop 2024-05-17T22:33:35.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.901+0000 7fba943d8700 1 -- 172.21.15.12:0/3129645434 shutdown_connections 2024-05-17T22:33:35.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.901+0000 7fba943d8700 1 --2- 172.21.15.12:0/3129645434 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fba780b2880 0x7fba780b4d90 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:35.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.901+0000 7fba943d8700 1 --2- 172.21.15.12:0/3129645434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba8c105220 0x7fba8c1033e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:35.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.901+0000 7fba943d8700 1 --2- 172.21.15.12:0/3129645434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba8c101c70 0x7fba8c1047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:35.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.901+0000 7fba943d8700 1 --2- 172.21.15.12:0/3129645434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba8c1025c0 0x7fba8c102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:35.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.901+0000 7fba943d8700 1 -- 172.21.15.12:0/3129645434 >> 172.21.15.12:0/3129645434 conn(0x7fba8c0fdb50 msgr2=0x7fba8c1081c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:33:35.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.901+0000 7fba943d8700 1 -- 172.21.15.12:0/3129645434 shutdown_connections 2024-05-17T22:33:35.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:35.901+0000 7fba943d8700 1 -- 172.21.15.12:0/3129645434 wait complete. 2024-05-17T22:33:36.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:35 smithi012 ceph-mon[31632]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-17T22:33:36.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:35 smithi012 ceph-mon[31632]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-05-17T22:33:36.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:35 smithi012 ceph-mon[31632]: osdmap e69: 12 total, 11 up, 12 in 2024-05-17T22:33:36.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:35 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T22:33:36.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:35 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T22:33:36.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:35 smithi012 ceph-mon[31632]: from='osd.11 ' entity='osd.11' 2024-05-17T22:33:36.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:35 smithi151 ceph-mon[36101]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-17T22:33:36.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:35 smithi151 ceph-mon[36101]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-05-17T22:33:36.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:35 smithi151 ceph-mon[36101]: osdmap e69: 12 total, 11 up, 12 in 2024-05-17T22:33:36.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:35 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T22:33:36.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:35 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T22:33:36.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:35 smithi151 ceph-mon[36101]: from='osd.11 ' entity='osd.11' 2024-05-17T22:33:36.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:35 smithi096 ceph-mon[36065]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-17T22:33:36.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:35 smithi096 ceph-mon[36065]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-05-17T22:33:36.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:35 smithi096 ceph-mon[36065]: osdmap e69: 12 total, 11 up, 12 in 2024-05-17T22:33:36.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:35 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T22:33:36.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:35 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T22:33:36.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:35 smithi096 ceph-mon[36065]: from='osd.11 ' entity='osd.11' 2024-05-17T22:33:36.495 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":69,"num_osds":12,"num_up_osds":11,"osd_up_since":1715985188,"num_in_osds":12,"osd_in_since":1715985195,"num_remapped_pgs":0} 2024-05-17T22:33:37.122 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:36 smithi151 ceph-mon[36101]: purged_snaps scrub starts 2024-05-17T22:33:37.122 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:36 smithi151 ceph-mon[36101]: purged_snaps scrub ok 2024-05-17T22:33:37.122 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:36 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T22:33:37.122 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:36 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/3129645434' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-17T22:33:37.122 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:36 smithi151 ceph-mon[36101]: osd.11 [v2:172.21.15.151:6824/3822307314,v1:172.21.15.151:6825/3822307314] boot 2024-05-17T22:33:37.122 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:36 smithi151 ceph-mon[36101]: osdmap e70: 12 total, 12 up, 12 in 2024-05-17T22:33:37.122 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:36 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T22:33:37.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:36 smithi096 ceph-mon[36065]: purged_snaps scrub starts 2024-05-17T22:33:37.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:36 smithi096 ceph-mon[36065]: purged_snaps scrub ok 2024-05-17T22:33:37.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:36 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T22:33:37.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:36 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/3129645434' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-17T22:33:37.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:36 smithi096 ceph-mon[36065]: osd.11 [v2:172.21.15.151:6824/3822307314,v1:172.21.15.151:6825/3822307314] boot 2024-05-17T22:33:37.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:36 smithi096 ceph-mon[36065]: osdmap e70: 12 total, 12 up, 12 in 2024-05-17T22:33:37.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:36 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T22:33:37.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:36 smithi012 ceph-mon[31632]: purged_snaps scrub starts 2024-05-17T22:33:37.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:36 smithi012 ceph-mon[31632]: purged_snaps scrub ok 2024-05-17T22:33:37.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:36 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T22:33:37.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:36 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/3129645434' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-17T22:33:37.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:36 smithi012 ceph-mon[31632]: osd.11 [v2:172.21.15.151:6824/3822307314,v1:172.21.15.151:6825/3822307314] boot 2024-05-17T22:33:37.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:36 smithi012 ceph-mon[31632]: osdmap e70: 12 total, 12 up, 12 in 2024-05-17T22:33:37.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:36 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T22:33:37.496 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph osd stat -f json 2024-05-17T22:33:37.757 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:33:38.143 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:37 smithi012 ceph-mon[31632]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-17T22:33:38.143 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:37 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:38.143 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:37 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:38.143 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:37 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:33:38.143 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:37 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:33:38.143 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:37 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:33:38.143 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:37 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:33:38.143 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:37 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:38.143 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:37 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:33:38.143 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:37 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:33:38.143 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:37 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:38.144 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:37 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:33:38.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:37 smithi151 ceph-mon[36101]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-17T22:33:38.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:37 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:38.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:37 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:38.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:37 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:33:38.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:37 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:33:38.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:37 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:33:38.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:37 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:33:38.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:37 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:38.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:37 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:33:38.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:37 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:33:38.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:37 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:38.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:37 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:33:38.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:37 smithi096 ceph-mon[36065]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-17T22:33:38.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:37 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:38.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:37 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:38.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:37 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:33:38.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:37 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:33:38.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:37 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:33:38.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:37 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:33:38.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:37 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:38.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:37 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:33:38.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:37 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:33:38.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:37 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:38.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:37 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:33:39.126 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:38 smithi012 ceph-mon[31632]: Detected new or changed devices on smithi151 2024-05-17T22:33:39.126 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:38 smithi012 ceph-mon[31632]: Adjusting osd_memory_target on smithi151 to 4272M 2024-05-17T22:33:39.126 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:38 smithi012 ceph-mon[31632]: osdmap e71: 12 total, 12 up, 12 in 2024-05-17T22:33:39.126 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:38 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:33:39.126 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:38 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:33:39.126 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:38 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:39.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:38 smithi151 ceph-mon[36101]: Detected new or changed devices on smithi151 2024-05-17T22:33:39.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:38 smithi151 ceph-mon[36101]: Adjusting osd_memory_target on smithi151 to 4272M 2024-05-17T22:33:39.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:38 smithi151 ceph-mon[36101]: osdmap e71: 12 total, 12 up, 12 in 2024-05-17T22:33:39.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:38 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:33:39.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:38 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:33:39.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:38 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:39.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:38 smithi096 ceph-mon[36065]: Detected new or changed devices on smithi151 2024-05-17T22:33:39.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:38 smithi096 ceph-mon[36065]: Adjusting osd_memory_target on smithi151 to 4272M 2024-05-17T22:33:39.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:38 smithi096 ceph-mon[36065]: osdmap e71: 12 total, 12 up, 12 in 2024-05-17T22:33:39.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:38 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:33:39.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:38 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:33:39.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:38 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:39.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.682+0000 7fb5c9c58700 1 -- 172.21.15.12:0/2065213405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5c4102fe0 msgr2=0x7fb5c41033f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:33:39.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.682+0000 7fb5c9c58700 1 --2- 172.21.15.12:0/2065213405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5c4102fe0 0x7fb5c41033f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac007510 tx=0x7fb5ac00af90 comp rx=0 tx=0).stop 2024-05-17T22:33:39.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.682+0000 7fb5c9c58700 1 -- 172.21.15.12:0/2065213405 shutdown_connections 2024-05-17T22:33:39.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.682+0000 7fb5c9c58700 1 --2- 172.21.15.12:0/2065213405 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5c4103980 0x7fb5c410a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:39.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.682+0000 7fb5c9c58700 1 --2- 172.21.15.12:0/2065213405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5c4101eb0 0x7fb5c41022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:39.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.682+0000 7fb5c9c58700 1 --2- 172.21.15.12:0/2065213405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5c4102fe0 0x7fb5c41033f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:39.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.682+0000 7fb5c9c58700 1 -- 172.21.15.12:0/2065213405 >> 172.21.15.12:0/2065213405 conn(0x7fb5c40fd6e0 msgr2=0x7fb5c40ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:33:39.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.682+0000 7fb5c9c58700 1 -- 172.21.15.12:0/2065213405 shutdown_connections 2024-05-17T22:33:39.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.682+0000 7fb5c9c58700 1 -- 172.21.15.12:0/2065213405 wait complete. 2024-05-17T22:33:39.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.682+0000 7fb5c9c58700 1 Processor -- start 2024-05-17T22:33:39.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.683+0000 7fb5c9c58700 1 -- start start 2024-05-17T22:33:39.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.683+0000 7fb5c9c58700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5c4101eb0 0x7fb5c419a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:33:39.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.683+0000 7fb5c9c58700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5c4102fe0 0x7fb5c419a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:33:39.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.683+0000 7fb5c9c58700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5c4103980 0x7fb5c41a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:33:39.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.683+0000 7fb5c9c58700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c410d4e0 con 0x7fb5c4103980 2024-05-17T22:33:39.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.683+0000 7fb5c9c58700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c410d360 con 0x7fb5c4102fe0 2024-05-17T22:33:39.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.683+0000 7fb5c9c58700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c410d660 con 0x7fb5c4101eb0 2024-05-17T22:33:39.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.683+0000 7fb5c3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5c4103980 0x7fb5c41a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:33:39.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.683+0000 7fb5c3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5c4103980 0x7fb5c41a17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56758/0 (socket says 172.21.15.12:56758) 2024-05-17T22:33:39.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.683+0000 7fb5c3fff700 1 -- 172.21.15.12:0/4272822581 learned_addr learned my addr 172.21.15.12:0/4272822581 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:33:39.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.683+0000 7fb5c37fe700 1 --2- 172.21.15.12:0/4272822581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5c4101eb0 0x7fb5c419a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:33:39.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.683+0000 7fb5c2ffd700 1 --2- 172.21.15.12:0/4272822581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5c4102fe0 0x7fb5c419a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:33:39.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.683+0000 7fb5c3fff700 1 -- 172.21.15.12:0/4272822581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5c4101eb0 msgr2=0x7fb5c419a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:33:39.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.683+0000 7fb5c3fff700 1 --2- 172.21.15.12:0/4272822581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5c4101eb0 0x7fb5c419a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:39.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.683+0000 7fb5c3fff700 1 -- 172.21.15.12:0/4272822581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5c4102fe0 msgr2=0x7fb5c419a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:33:39.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.684+0000 7fb5c3fff700 1 --2- 172.21.15.12:0/4272822581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5c4102fe0 0x7fb5c419a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:39.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.684+0000 7fb5c3fff700 1 -- 172.21.15.12:0/4272822581 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c41a1e60 con 0x7fb5c4103980 2024-05-17T22:33:39.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.684+0000 7fb5c37fe700 1 --2- 172.21.15.12:0/4272822581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5c4101eb0 0x7fb5c419a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:33:39.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.684+0000 7fb5c3fff700 1 --2- 172.21.15.12:0/4272822581 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5c4103980 0x7fb5c41a17e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc00e940 tx=0x7fb5bc00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:33:39.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.684+0000 7fb5c0ff9700 1 -- 172.21.15.12:0/4272822581 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc00cbd0 con 0x7fb5c4103980 2024-05-17T22:33:39.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.684+0000 7fb5c9c58700 1 -- 172.21.15.12:0/4272822581 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5c41a2150 con 0x7fb5c4103980 2024-05-17T22:33:39.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.684+0000 7fb5c0ff9700 1 -- 172.21.15.12:0/4272822581 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5bc00ee90 con 0x7fb5c4103980 2024-05-17T22:33:39.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.684+0000 7fb5c0ff9700 1 -- 172.21.15.12:0/4272822581 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc005590 con 0x7fb5c4103980 2024-05-17T22:33:39.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.684+0000 7fb5c9c58700 1 -- 172.21.15.12:0/4272822581 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5c41a2690 con 0x7fb5c4103980 2024-05-17T22:33:39.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.685+0000 7fb5c9c58700 1 -- 172.21.15.12:0/4272822581 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5c41022c0 con 0x7fb5c4103980 2024-05-17T22:33:39.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.687+0000 7fb5c0ff9700 1 -- 172.21.15.12:0/4272822581 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fb5bc004750 con 0x7fb5c4103980 2024-05-17T22:33:39.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.688+0000 7fb5c0ff9700 1 --2- 172.21.15.12:0/4272822581 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb5b00b2880 0x7fb5b00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:33:39.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.688+0000 7fb5c37fe700 1 --2- 172.21.15.12:0/4272822581 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb5b00b2880 0x7fb5b00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:33:39.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.688+0000 7fb5c0ff9700 1 -- 172.21.15.12:0/4272822581 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fb5bc0e2220 con 0x7fb5c4103980 2024-05-17T22:33:39.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.688+0000 7fb5c37fe700 1 --2- 172.21.15.12:0/4272822581 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb5b00b2880 0x7fb5b00b4d90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4102ee0 tx=0x7fb5b40091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:33:39.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.691+0000 7fb5c0ff9700 1 -- 172.21.15.12:0/4272822581 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb5bc054df0 con 0x7fb5c4103980 2024-05-17T22:33:39.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.909+0000 7fb5c9c58700 1 -- 172.21.15.12:0/4272822581 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fb5c4064650 con 0x7fb5c4103980 2024-05-17T22:33:39.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.910+0000 7fb5c0ff9700 1 -- 172.21.15.12:0/4272822581 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v71) ==== 74+0+133 (secure 0 0 0) 0x7fb5bc0aa3e0 con 0x7fb5c4103980 2024-05-17T22:33:39.913 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:33:39.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.912+0000 7fb5c9c58700 1 -- 172.21.15.12:0/4272822581 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb5b00b2880 msgr2=0x7fb5b00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:33:39.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.912+0000 7fb5c9c58700 1 --2- 172.21.15.12:0/4272822581 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb5b00b2880 0x7fb5b00b4d90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4102ee0 tx=0x7fb5b40091b0 comp rx=0 tx=0).stop 2024-05-17T22:33:39.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.912+0000 7fb5c9c58700 1 -- 172.21.15.12:0/4272822581 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5c4103980 msgr2=0x7fb5c41a17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:33:39.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.912+0000 7fb5c9c58700 1 --2- 172.21.15.12:0/4272822581 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5c4103980 0x7fb5c41a17e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc00e940 tx=0x7fb5bc00ec50 comp rx=0 tx=0).stop 2024-05-17T22:33:39.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.912+0000 7fb5c9c58700 1 -- 172.21.15.12:0/4272822581 shutdown_connections 2024-05-17T22:33:39.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.912+0000 7fb5c9c58700 1 --2- 172.21.15.12:0/4272822581 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb5b00b2880 0x7fb5b00b4d90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:39.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.912+0000 7fb5c9c58700 1 --2- 172.21.15.12:0/4272822581 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5c4103980 0x7fb5c41a17e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:39.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.912+0000 7fb5c9c58700 1 --2- 172.21.15.12:0/4272822581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5c4101eb0 0x7fb5c419a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:39.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.912+0000 7fb5c9c58700 1 --2- 172.21.15.12:0/4272822581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5c4102fe0 0x7fb5c419a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:39.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.912+0000 7fb5c9c58700 1 -- 172.21.15.12:0/4272822581 >> 172.21.15.12:0/4272822581 conn(0x7fb5c40fd6e0 msgr2=0x7fb5c4106280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:33:39.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.912+0000 7fb5c9c58700 1 -- 172.21.15.12:0/4272822581 shutdown_connections 2024-05-17T22:33:39.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:39.912+0000 7fb5c9c58700 1 -- 172.21.15.12:0/4272822581 wait complete. 2024-05-17T22:33:40.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:39 smithi012 ceph-mon[31632]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:33:40.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:39 smithi151 ceph-mon[36101]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:33:40.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:39 smithi096 ceph-mon[36065]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:33:40.631 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":71,"num_osds":12,"num_up_osds":12,"osd_up_since":1715985216,"num_in_osds":12,"osd_in_since":1715985195,"num_remapped_pgs":0} 2024-05-17T22:33:40.632 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-17T22:33:40.883 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:33:41.114 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:40 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/4272822581' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-17T22:33:41.142 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:40 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/4272822581' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-17T22:33:41.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:40 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/4272822581' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-17T22:33:42.037 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:41 smithi012 ceph-mon[31632]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:33:42.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:41 smithi151 ceph-mon[36101]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:33:42.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:41 smithi096 ceph-mon[36065]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:33:42.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.668+0000 7fad590de700 1 -- 172.21.15.12:0/1993097378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad54109190 msgr2=0x7fad54110af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:33:42.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.668+0000 7fad590de700 1 --2- 172.21.15.12:0/1993097378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad54109190 0x7fad54110af0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fad4c00b0a0 tx=0x7fad4c00b3b0 comp rx=0 tx=0).stop 2024-05-17T22:33:42.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.668+0000 7fad590de700 1 -- 172.21.15.12:0/1993097378 shutdown_connections 2024-05-17T22:33:42.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.668+0000 7fad590de700 1 --2- 172.21.15.12:0/1993097378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad54109190 0x7fad54110af0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:42.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.668+0000 7fad590de700 1 --2- 172.21.15.12:0/1993097378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad54103f10 0x7fad54104320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:42.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.668+0000 7fad590de700 1 --2- 172.21.15.12:0/1993097378 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fad54108840 0x7fad54108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:42.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.668+0000 7fad590de700 1 -- 172.21.15.12:0/1993097378 >> 172.21.15.12:0/1993097378 conn(0x7fad540ff9b0 msgr2=0x7fad54101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:33:42.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.668+0000 7fad590de700 1 -- 172.21.15.12:0/1993097378 shutdown_connections 2024-05-17T22:33:42.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.668+0000 7fad590de700 1 -- 172.21.15.12:0/1993097378 wait complete. 2024-05-17T22:33:42.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.669+0000 7fad590de700 1 Processor -- start 2024-05-17T22:33:42.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.669+0000 7fad590de700 1 -- start start 2024-05-17T22:33:42.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.669+0000 7fad590de700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad54103f10 0x7fad54075b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:33:42.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.669+0000 7fad590de700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad54108840 0x7fad54076070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:33:42.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.670+0000 7fad590de700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fad54109190 0x7fad540765b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:33:42.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.670+0000 7fad590de700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fad54113e40 con 0x7fad54108840 2024-05-17T22:33:42.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.670+0000 7fad590de700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fad54113cc0 con 0x7fad54103f10 2024-05-17T22:33:42.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.670+0000 7fad590de700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fad54113fc0 con 0x7fad54109190 2024-05-17T22:33:42.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.670+0000 7fad5259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad54108840 0x7fad54076070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:33:42.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.670+0000 7fad5259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad54108840 0x7fad54076070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46398/0 (socket says 172.21.15.12:46398) 2024-05-17T22:33:42.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.670+0000 7fad5259c700 1 -- 172.21.15.12:0/283509410 learned_addr learned my addr 172.21.15.12:0/283509410 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:33:42.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.670+0000 7fad52d9d700 1 --2- 172.21.15.12:0/283509410 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad54103f10 0x7fad54075b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:33:42.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.670+0000 7fad5359e700 1 --2- 172.21.15.12:0/283509410 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fad54109190 0x7fad540765b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:33:42.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.670+0000 7fad5259c700 1 -- 172.21.15.12:0/283509410 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fad54109190 msgr2=0x7fad540765b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:33:42.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.670+0000 7fad5259c700 1 --2- 172.21.15.12:0/283509410 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fad54109190 0x7fad540765b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:42.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.670+0000 7fad5259c700 1 -- 172.21.15.12:0/283509410 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad54103f10 msgr2=0x7fad54075b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:33:42.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.670+0000 7fad5259c700 1 --2- 172.21.15.12:0/283509410 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad54103f10 0x7fad54075b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:42.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.670+0000 7fad5259c700 1 -- 172.21.15.12:0/283509410 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad541a8b00 con 0x7fad54108840 2024-05-17T22:33:42.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.670+0000 7fad5259c700 1 --2- 172.21.15.12:0/283509410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad54108840 0x7fad54076070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fad4800b500 tx=0x7fad4800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:33:42.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.670+0000 7fad3bfff700 1 -- 172.21.15.12:0/283509410 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad48015020 con 0x7fad54108840 2024-05-17T22:33:42.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.670+0000 7fad590de700 1 -- 172.21.15.12:0/283509410 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad541a8df0 con 0x7fad54108840 2024-05-17T22:33:42.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.670+0000 7fad3bfff700 1 -- 172.21.15.12:0/283509410 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad48011ce0 con 0x7fad54108840 2024-05-17T22:33:42.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.670+0000 7fad3bfff700 1 -- 172.21.15.12:0/283509410 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad48010d40 con 0x7fad54108840 2024-05-17T22:33:42.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.671+0000 7fad590de700 1 -- 172.21.15.12:0/283509410 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad541a92a0 con 0x7fad54108840 2024-05-17T22:33:42.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.672+0000 7fad590de700 1 -- 172.21.15.12:0/283509410 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad54104320 con 0x7fad54108840 2024-05-17T22:33:42.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.673+0000 7fad3bfff700 1 -- 172.21.15.12:0/283509410 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fad48018020 con 0x7fad54108840 2024-05-17T22:33:42.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.674+0000 7fad3bfff700 1 --2- 172.21.15.12:0/283509410 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fad3c0b2880 0x7fad3c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:33:42.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.674+0000 7fad52d9d700 1 --2- 172.21.15.12:0/283509410 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fad3c0b2880 0x7fad3c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:33:42.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.674+0000 7fad3bfff700 1 -- 172.21.15.12:0/283509410 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fad480e1540 con 0x7fad54108840 2024-05-17T22:33:42.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.674+0000 7fad52d9d700 1 --2- 172.21.15.12:0/283509410 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fad3c0b2880 0x7fad3c0b4d90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fad44005950 tx=0x7fad4400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:33:42.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.678+0000 7fad3bfff700 1 -- 172.21.15.12:0/283509410 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fad4801f020 con 0x7fad54108840 2024-05-17T22:33:42.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.892+0000 7fad590de700 1 -- 172.21.15.12:0/283509410 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad54077060 con 0x7fad54108840 2024-05-17T22:33:42.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.893+0000 7fad3bfff700 1 -- 172.21.15.12:0/283509410 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+18422 (secure 0 0 0) 0x7fad480a9660 con 0x7fad54108840 2024-05-17T22:33:42.896 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:33:42.896 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":71,"fsid":"4d593780-149c-11ef-bc9a-c7b262605968","created":"2024-05-17T22:26:16.955626+0000","modified":"2024-05-17T22:33:37.840929+0000","last_up_change":"2024-05-17T22:33:36.551093+0000","last_in_change":"2024-05-17T22:33:15.032677+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T22:29:42.475513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d42798f-dc5f-4e66-bce2-ec33dc789e4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6803","nonce":2015605593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6805","nonce":2015605593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6809","nonce":2015605593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6807","nonce":2015605593}]},"public_addr":"172.21.15.12:6803/2015605593","cluster_addr":"172.21.15.12:6805/2015605593","heartbeat_back_addr":"172.21.15.12:6809/2015605593","heartbeat_front_addr":"172.21.15.12:6807/2015605593","state":["exists","up"]},{"osd":1,"uuid":"58afecc7-119b-4c16-a401-640fb77d4714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6801","nonce":1238316358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6803","nonce":1238316358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6807","nonce":1238316358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6805","nonce":1238316358}]},"public_addr":"172.21.15.96:6801/1238316358","cluster_addr":"172.21.15.96:6803/1238316358","heartbeat_back_addr":"172.21.15.96:6807/1238316358","heartbeat_front_addr":"172.21.15.96:6805/1238316358","state":["exists","up"]},{"osd":2,"uuid":"4bef4049-482f-49fb-82fb-38052400eb2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6801","nonce":148729815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6803","nonce":148729815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6807","nonce":148729815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6805","nonce":148729815}]},"public_addr":"172.21.15.151:6801/148729815","cluster_addr":"172.21.15.151:6803/148729815","heartbeat_back_addr":"172.21.15.151:6807/148729815","heartbeat_front_addr":"172.21.15.151:6805/148729815","state":["exists","up"]},{"osd":3,"uuid":"6fed8e36-82a2-49e4-82a3-16e6d9de87f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6811","nonce":3563309320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6813","nonce":3563309320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6817","nonce":3563309320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6815","nonce":3563309320}]},"public_addr":"172.21.15.12:6811/3563309320","cluster_addr":"172.21.15.12:6813/3563309320","heartbeat_back_addr":"172.21.15.12:6817/3563309320","heartbeat_front_addr":"172.21.15.12:6815/3563309320","state":["exists","up"]},{"osd":4,"uuid":"c0e8689a-2816-4089-ade3-931960bae4f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6809","nonce":2219103166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6811","nonce":2219103166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6815","nonce":2219103166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6813","nonce":2219103166}]},"public_addr":"172.21.15.96:6809/2219103166","cluster_addr":"172.21.15.96:6811/2219103166","heartbeat_back_addr":"172.21.15.96:6815/2219103166","heartbeat_front_addr":"172.21.15.96:6813/2219103166","state":["exists","up"]},{"osd":5,"uuid":"e76ba432-94ff-4998-aaba-2645d9e6055d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6809","nonce":2589835078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6811","nonce":2589835078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6815","nonce":2589835078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6813","nonce":2589835078}]},"public_addr":"172.21.15.151:6809/2589835078","cluster_addr":"172.21.15.151:6811/2589835078","heartbeat_back_addr":"172.21.15.151:6815/2589835078","heartbeat_front_addr":"172.21.15.151:6813/2589835078","state":["exists","up"]},{"osd":6,"uuid":"b7ece35c-efd1-40df-95da-05a0cc147372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6819","nonce":3112661724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6821","nonce":3112661724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6825","nonce":3112661724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6823","nonce":3112661724}]},"public_addr":"172.21.15.12:6819/3112661724","cluster_addr":"172.21.15.12:6821/3112661724","heartbeat_back_addr":"172.21.15.12:6825/3112661724","heartbeat_front_addr":"172.21.15.12:6823/3112661724","state":["exists","up"]},{"osd":7,"uuid":"6a65bf30-daf5-4fba-b67e-8e513d88a971","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6817","nonce":2449989110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6819","nonce":2449989110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6823","nonce":2449989110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6821","nonce":2449989110}]},"public_addr":"172.21.15.96:6817/2449989110","cluster_addr":"172.21.15.96:6819/2449989110","heartbeat_back_addr":"172.21.15.96:6823/2449989110","heartbeat_front_addr":"172.21.15.96:6821/2449989110","state":["exists","up"]},{"osd":8,"uuid":"ad62ea44-e416-4a3e-bbbe-93f2a710dbc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6817","nonce":2545224628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6819","nonce":2545224628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6823","nonce":2545224628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6821","nonce":2545224628}]},"public_addr":"172.21.15.151:6817/2545224628","cluster_addr":"172.21.15.151:6819/2545224628","heartbeat_back_addr":"172.21.15.151:6823/2545224628","heartbeat_front_addr":"172.21.15.151:6821/2545224628","state":["exists","up"]},{"osd":9,"uuid":"1275de08-2a3c-48f3-920b-dd68b4024d42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6827","nonce":2100661273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6829","nonce":2100661273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6833","nonce":2100661273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6831","nonce":2100661273}]},"public_addr":"172.21.15.12:6827/2100661273","cluster_addr":"172.21.15.12:6829/2100661273","heartbeat_back_addr":"172.21.15.12:6833/2100661273","heartbeat_front_addr":"172.21.15.12:6831/2100661273","state":["exists","up"]},{"osd":10,"uuid":"347e4e40-9a9d-43fe-8f52-fdd3199a2a5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6825","nonce":304016309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6827","nonce":304016309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6831","nonce":304016309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6829","nonce":304016309}]},"public_addr":"172.21.15.96:6825/304016309","cluster_addr":"172.21.15.96:6827/304016309","heartbeat_back_addr":"172.21.15.96:6831/304016309","heartbeat_front_addr":"172.21.15.96:6829/304016309","state":["exists","up"]},{"osd":11,"uuid":"ee8d3502-8b3c-4e6f-824f-3a97e19111ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6825","nonce":3822307314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6827","nonce":3822307314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6831","nonce":3822307314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6829","nonce":3822307314}]},"public_addr":"172.21.15.151:6825/3822307314","cluster_addr":"172.21.15.151:6827/3822307314","heartbeat_back_addr":"172.21.15.151:6831/3822307314","heartbeat_front_addr":"172.21.15.151:6829/3822307314","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:28:51.542055+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:29:15.566424+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:29:39.500981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:04.797907+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:29.655650+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:55.211880+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:31:21.559462+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:31:47.154852+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:32:13.114892+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:32:40.529708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:33:06.930942+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:33:34.061783+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/2682700253":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6801/2250046521":"2024-05-18T22:27:14.372972+0000","172.21.15.12:0/4241094239":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/4172568611":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/3512529242":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6800/1039789215":"2024-05-18T22:26:45.466719+0000","172.21.15.12:6800/2250046521":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6801/1039789215":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/2043758453":"2024-05-18T22:27:14.372972+0000","172.21.15.12:0/2694593732":"2024-05-18T22:26:45.466719+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T22:33:42.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.895+0000 7fad590de700 1 -- 172.21.15.12:0/283509410 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fad3c0b2880 msgr2=0x7fad3c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:33:42.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.895+0000 7fad590de700 1 --2- 172.21.15.12:0/283509410 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fad3c0b2880 0x7fad3c0b4d90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fad44005950 tx=0x7fad4400a300 comp rx=0 tx=0).stop 2024-05-17T22:33:42.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.895+0000 7fad590de700 1 -- 172.21.15.12:0/283509410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad54108840 msgr2=0x7fad54076070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:33:42.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.895+0000 7fad590de700 1 --2- 172.21.15.12:0/283509410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad54108840 0x7fad54076070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fad4800b500 tx=0x7fad4800b810 comp rx=0 tx=0).stop 2024-05-17T22:33:42.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.895+0000 7fad590de700 1 -- 172.21.15.12:0/283509410 shutdown_connections 2024-05-17T22:33:42.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.895+0000 7fad590de700 1 --2- 172.21.15.12:0/283509410 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fad3c0b2880 0x7fad3c0b4d90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:42.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.895+0000 7fad590de700 1 --2- 172.21.15.12:0/283509410 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fad54109190 0x7fad540765b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:42.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.895+0000 7fad590de700 1 --2- 172.21.15.12:0/283509410 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad54103f10 0x7fad54075b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:42.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.895+0000 7fad590de700 1 --2- 172.21.15.12:0/283509410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad54108840 0x7fad54076070 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:42.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.895+0000 7fad590de700 1 -- 172.21.15.12:0/283509410 >> 172.21.15.12:0/283509410 conn(0x7fad540ff9b0 msgr2=0x7fad54101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:33:42.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.895+0000 7fad590de700 1 -- 172.21.15.12:0/283509410 shutdown_connections 2024-05-17T22:33:42.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:42.895+0000 7fad590de700 1 -- 172.21.15.12:0/283509410 wait complete. 2024-05-17T22:33:43.705 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-17T22:29:42.475513+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-17T22:33:43.705 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-17T22:33:43.962 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:33:44.084 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:43 smithi012 ceph-mon[31632]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:33:44.084 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:43 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/283509410' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:33:44.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:43 smithi151 ceph-mon[36101]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:33:44.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:43 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/283509410' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:33:44.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:43 smithi096 ceph-mon[36065]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:33:44.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:43 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/283509410' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:33:45.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.627+0000 7f9c9ee57700 1 -- 172.21.15.12:0/2401815222 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c98108840 msgr2=0x7f9c98108c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:33:45.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.627+0000 7f9c9ee57700 1 --2- 172.21.15.12:0/2401815222 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c98108840 0x7f9c98108c50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9c84009990 tx=0x7f9c84009ca0 comp rx=0 tx=0).stop 2024-05-17T22:33:45.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.627+0000 7f9c9ee57700 1 -- 172.21.15.12:0/2401815222 shutdown_connections 2024-05-17T22:33:45.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.627+0000 7f9c9ee57700 1 --2- 172.21.15.12:0/2401815222 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c98109190 0x7f9c98110af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:45.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.627+0000 7f9c9ee57700 1 --2- 172.21.15.12:0/2401815222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c98103f10 0x7f9c98104320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:45.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.627+0000 7f9c9ee57700 1 --2- 172.21.15.12:0/2401815222 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c98108840 0x7f9c98108c50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:45.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.627+0000 7f9c9ee57700 1 -- 172.21.15.12:0/2401815222 >> 172.21.15.12:0/2401815222 conn(0x7f9c980ff9b0 msgr2=0x7f9c98101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:33:45.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.627+0000 7f9c9ee57700 1 -- 172.21.15.12:0/2401815222 shutdown_connections 2024-05-17T22:33:45.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.627+0000 7f9c9ee57700 1 -- 172.21.15.12:0/2401815222 wait complete. 2024-05-17T22:33:45.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.628+0000 7f9c9ee57700 1 Processor -- start 2024-05-17T22:33:45.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.628+0000 7f9c9ee57700 1 -- start start 2024-05-17T22:33:45.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.629+0000 7f9c9ee57700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c98103f10 0x7f9c9819a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:33:45.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.629+0000 7f9c9ee57700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c98108840 0x7f9c9819ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:33:45.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.629+0000 7f9c9ee57700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c98109190 0x7f9c9819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:33:45.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.629+0000 7f9c9ee57700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9c98113dc0 con 0x7f9c98109190 2024-05-17T22:33:45.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.629+0000 7f9c9ee57700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9c98113c40 con 0x7f9c98108840 2024-05-17T22:33:45.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.629+0000 7f9c9ee57700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9c98113f40 con 0x7f9c98103f10 2024-05-17T22:33:45.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.629+0000 7f9c9d3f4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c98109190 0x7f9c9819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:33:45.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.629+0000 7f9c9d3f4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c98109190 0x7f9c9819f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46414/0 (socket says 172.21.15.12:46414) 2024-05-17T22:33:45.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.629+0000 7f9c9d3f4700 1 -- 172.21.15.12:0/870652778 learned_addr learned my addr 172.21.15.12:0/870652778 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:33:45.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.629+0000 7f9c9cbf3700 1 --2- 172.21.15.12:0/870652778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c98103f10 0x7f9c9819a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:33:45.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.629+0000 7f9c97fff700 1 --2- 172.21.15.12:0/870652778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c98108840 0x7f9c9819ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:33:45.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.629+0000 7f9c9d3f4700 1 -- 172.21.15.12:0/870652778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c98103f10 msgr2=0x7f9c9819a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:33:45.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.629+0000 7f9c9d3f4700 1 --2- 172.21.15.12:0/870652778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c98103f10 0x7f9c9819a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:45.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.629+0000 7f9c9d3f4700 1 -- 172.21.15.12:0/870652778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c98108840 msgr2=0x7f9c9819ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:33:45.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.629+0000 7f9c9d3f4700 1 --2- 172.21.15.12:0/870652778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c98108840 0x7f9c9819ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:45.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.629+0000 7f9c9d3f4700 1 -- 172.21.15.12:0/870652778 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c9819f6b0 con 0x7f9c98109190 2024-05-17T22:33:45.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.629+0000 7f9c97fff700 1 --2- 172.21.15.12:0/870652778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c98108840 0x7f9c9819ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:33:45.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.629+0000 7f9c9d3f4700 1 --2- 172.21.15.12:0/870652778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c98109190 0x7f9c9819f010 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9c9000ada0 tx=0x7f9c9000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:33:45.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.629+0000 7f9c95ffb700 1 -- 172.21.15.12:0/870652778 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c90017020 con 0x7f9c98109190 2024-05-17T22:33:45.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.630+0000 7f9c9ee57700 1 -- 172.21.15.12:0/870652778 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c9819f9a0 con 0x7f9c98109190 2024-05-17T22:33:45.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.630+0000 7f9c95ffb700 1 -- 172.21.15.12:0/870652778 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c9000ce30 con 0x7f9c98109190 2024-05-17T22:33:45.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.630+0000 7f9c95ffb700 1 -- 172.21.15.12:0/870652778 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c90005350 con 0x7f9c98109190 2024-05-17T22:33:45.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.630+0000 7f9c9ee57700 1 -- 172.21.15.12:0/870652778 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c981a72c0 con 0x7f9c98109190 2024-05-17T22:33:45.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.631+0000 7f9c9ee57700 1 -- 172.21.15.12:0/870652778 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c98104320 con 0x7f9c98109190 2024-05-17T22:33:45.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.633+0000 7f9c95ffb700 1 -- 172.21.15.12:0/870652778 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f9c9000c980 con 0x7f9c98109190 2024-05-17T22:33:45.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.633+0000 7f9c95ffb700 1 --2- 172.21.15.12:0/870652778 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9c880b2880 0x7f9c880b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:33:45.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.634+0000 7f9c95ffb700 1 -- 172.21.15.12:0/870652778 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f9c900e19a0 con 0x7f9c98109190 2024-05-17T22:33:45.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.634+0000 7f9c9cbf3700 1 --2- 172.21.15.12:0/870652778 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9c880b2880 0x7f9c880b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:33:45.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.634+0000 7f9c9cbf3700 1 --2- 172.21.15.12:0/870652778 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9c880b2880 0x7f9c880b4d90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c009c60 tx=0x7f9c8c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:33:45.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.637+0000 7f9c95ffb700 1 -- 172.21.15.12:0/870652778 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9c90021020 con 0x7f9c98109190 2024-05-17T22:33:45.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.847+0000 7f9c9ee57700 1 -- 172.21.15.12:0/870652778 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9c9802ea80 con 0x7f9c98109190 2024-05-17T22:33:45.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.848+0000 7f9c95ffb700 1 -- 172.21.15.12:0/870652778 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7f9c9000a1f0 con 0x7f9c98109190 2024-05-17T22:33:45.851 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-05-17T22:33:45.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.850+0000 7f9c9ee57700 1 -- 172.21.15.12:0/870652778 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9c880b2880 msgr2=0x7f9c880b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:33:45.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.850+0000 7f9c9ee57700 1 --2- 172.21.15.12:0/870652778 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9c880b2880 0x7f9c880b4d90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c009c60 tx=0x7f9c8c009260 comp rx=0 tx=0).stop 2024-05-17T22:33:45.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.850+0000 7f9c9ee57700 1 -- 172.21.15.12:0/870652778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c98109190 msgr2=0x7f9c9819f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:33:45.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.850+0000 7f9c9ee57700 1 --2- 172.21.15.12:0/870652778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c98109190 0x7f9c9819f010 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9c9000ada0 tx=0x7f9c9000c330 comp rx=0 tx=0).stop 2024-05-17T22:33:45.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.850+0000 7f9c9ee57700 1 -- 172.21.15.12:0/870652778 shutdown_connections 2024-05-17T22:33:45.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.850+0000 7f9c9ee57700 1 --2- 172.21.15.12:0/870652778 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9c880b2880 0x7f9c880b4d90 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:45.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.850+0000 7f9c9ee57700 1 --2- 172.21.15.12:0/870652778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c98109190 0x7f9c9819f010 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:45.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.850+0000 7f9c9ee57700 1 --2- 172.21.15.12:0/870652778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c98103f10 0x7f9c9819a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:45.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.850+0000 7f9c9ee57700 1 --2- 172.21.15.12:0/870652778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c98108840 0x7f9c9819ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:45.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.850+0000 7f9c9ee57700 1 -- 172.21.15.12:0/870652778 >> 172.21.15.12:0/870652778 conn(0x7f9c980ff9b0 msgr2=0x7f9c98101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:33:45.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.850+0000 7f9c9ee57700 1 -- 172.21.15.12:0/870652778 shutdown_connections 2024-05-17T22:33:45.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:45.850+0000 7f9c9ee57700 1 -- 172.21.15.12:0/870652778 wait complete. 2024-05-17T22:33:46.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:45 smithi012 ceph-mon[31632]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:33:46.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:45 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/870652778' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-17T22:33:46.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:45 smithi151 ceph-mon[36101]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:33:46.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:45 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/870652778' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-17T22:33:46.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:45 smithi096 ceph-mon[36065]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:33:46.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:45 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/870652778' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-17T22:33:46.643 INFO:tasks.cephadm:Adding mds.a on smithi012 2024-05-17T22:33:46.643 INFO:tasks.cephadm:Adding mds.d on smithi012 2024-05-17T22:33:46.643 INFO:tasks.cephadm:Adding mds.g on smithi012 2024-05-17T22:33:46.643 INFO:tasks.cephadm:Adding mds.j on smithi012 2024-05-17T22:33:46.643 INFO:tasks.cephadm:Adding mds.b on smithi096 2024-05-17T22:33:46.643 INFO:tasks.cephadm:Adding mds.e on smithi096 2024-05-17T22:33:46.643 INFO:tasks.cephadm:Adding mds.h on smithi096 2024-05-17T22:33:46.643 INFO:tasks.cephadm:Adding mds.k on smithi096 2024-05-17T22:33:46.643 INFO:tasks.cephadm:Adding mds.c on smithi151 2024-05-17T22:33:46.644 INFO:tasks.cephadm:Adding mds.f on smithi151 2024-05-17T22:33:46.644 INFO:tasks.cephadm:Adding mds.i on smithi151 2024-05-17T22:33:46.644 INFO:tasks.cephadm:Adding mds.l on smithi151 2024-05-17T22:33:46.644 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph orch apply mds all '12;smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l' 2024-05-17T22:33:46.894 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T22:33:48.128 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:47 smithi151 ceph-mon[36101]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:33:48.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:47 smithi096 ceph-mon[36065]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:33:48.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:47 smithi012 ceph-mon[31632]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:33:48.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.669+0000 7f7a6553b700 1 -- 172.21.15.151:0/2710062162 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a60105760 msgr2=0x7f7a6010c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:33:48.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.669+0000 7f7a6553b700 1 --2- 172.21.15.151:0/2710062162 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a60105760 0x7f7a6010c010 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7a5800b0d0 tx=0x7f7a5800b3e0 comp rx=0 tx=0).stop 2024-05-17T22:33:48.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.670+0000 7f7a6553b700 1 -- 172.21.15.151:0/2710062162 shutdown_connections 2024-05-17T22:33:48.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.670+0000 7f7a6553b700 1 --2- 172.21.15.151:0/2710062162 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a60105760 0x7f7a6010c010 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:48.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.670+0000 7f7a6553b700 1 --2- 172.21.15.151:0/2710062162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a60103c90 0x7f7a601040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:48.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.670+0000 7f7a6553b700 1 --2- 172.21.15.151:0/2710062162 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a60104dc0 0x7f7a601051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:48.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.670+0000 7f7a6553b700 1 -- 172.21.15.151:0/2710062162 >> 172.21.15.151:0/2710062162 conn(0x7f7a600ff4c0 msgr2=0x7f7a60101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:33:48.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.670+0000 7f7a6553b700 1 -- 172.21.15.151:0/2710062162 shutdown_connections 2024-05-17T22:33:48.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.670+0000 7f7a6553b700 1 -- 172.21.15.151:0/2710062162 wait complete. 2024-05-17T22:33:48.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.670+0000 7f7a6553b700 1 Processor -- start 2024-05-17T22:33:48.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.670+0000 7f7a6553b700 1 -- start start 2024-05-17T22:33:48.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.671+0000 7f7a6553b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a60103c90 0x7f7a600774d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:33:48.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.671+0000 7f7a6553b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a60104dc0 0x7f7a60075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:33:48.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.671+0000 7f7a6553b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a60105760 0x7f7a600760c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:33:48.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.671+0000 7f7a6553b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6010f710 con 0x7f7a60104dc0 2024-05-17T22:33:48.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.671+0000 7f7a6553b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6010f590 con 0x7f7a60105760 2024-05-17T22:33:48.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.671+0000 7f7a6553b700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6010f890 con 0x7f7a60103c90 2024-05-17T22:33:48.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.671+0000 7f7a5effd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a60103c90 0x7f7a600774d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:33:48.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.671+0000 7f7a5f7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a60105760 0x7f7a600760c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:33:48.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.671+0000 7f7a5f7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a60105760 0x7f7a600760c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.151:57140/0 (socket says 172.21.15.151:57140) 2024-05-17T22:33:48.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.671+0000 7f7a5effd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a60103c90 0x7f7a600774d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47804/0 (socket says 172.21.15.151:47804) 2024-05-17T22:33:48.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.671+0000 7f7a5f7fe700 1 -- 172.21.15.151:0/3859760584 learned_addr learned my addr 172.21.15.151:0/3859760584 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-17T22:33:48.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.671+0000 7f7a5e7fc700 1 --2- 172.21.15.151:0/3859760584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a60104dc0 0x7f7a60075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:33:48.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.671+0000 7f7a5effd700 1 -- 172.21.15.151:0/3859760584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a60105760 msgr2=0x7f7a600760c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:33:48.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.671+0000 7f7a5effd700 1 --2- 172.21.15.151:0/3859760584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a60105760 0x7f7a600760c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:48.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.671+0000 7f7a5effd700 1 -- 172.21.15.151:0/3859760584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a60104dc0 msgr2=0x7f7a60075b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:33:48.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.671+0000 7f7a5effd700 1 --2- 172.21.15.151:0/3859760584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a60104dc0 0x7f7a60075b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:48.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.671+0000 7f7a5effd700 1 -- 172.21.15.151:0/3859760584 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a60076820 con 0x7f7a60103c90 2024-05-17T22:33:48.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.671+0000 7f7a5f7fe700 1 --2- 172.21.15.151:0/3859760584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a60105760 0x7f7a600760c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:33:48.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.672+0000 7f7a5effd700 1 --2- 172.21.15.151:0/3859760584 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a60103c90 0x7f7a600774d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7a60104cc0 tx=0x7f7a5400dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:33:48.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.672+0000 7f7a47fff700 1 -- 172.21.15.151:0/3859760584 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a54016070 con 0x7f7a60103c90 2024-05-17T22:33:48.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.672+0000 7f7a6553b700 1 -- 172.21.15.151:0/3859760584 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a601a4300 con 0x7f7a60103c90 2024-05-17T22:33:48.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.672+0000 7f7a47fff700 1 -- 172.21.15.151:0/3859760584 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a5400fd40 con 0x7f7a60103c90 2024-05-17T22:33:48.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.672+0000 7f7a47fff700 1 -- 172.21.15.151:0/3859760584 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5400e3b0 con 0x7f7a60103c90 2024-05-17T22:33:48.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.672+0000 7f7a6553b700 1 -- 172.21.15.151:0/3859760584 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a601a4830 con 0x7f7a60103c90 2024-05-17T22:33:48.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.673+0000 7f7a6553b700 1 -- 172.21.15.151:0/3859760584 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a601040a0 con 0x7f7a60103c90 2024-05-17T22:33:48.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.675+0000 7f7a47fff700 1 -- 172.21.15.151:0/3859760584 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7a54019070 con 0x7f7a60103c90 2024-05-17T22:33:48.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.675+0000 7f7a47fff700 1 --2- 172.21.15.151:0/3859760584 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f7a480b2900 0x7f7a480b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:33:48.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.676+0000 7f7a47fff700 1 -- 172.21.15.151:0/3859760584 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f7a540e1690 con 0x7f7a60103c90 2024-05-17T22:33:48.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.676+0000 7f7a5e7fc700 1 --2- 172.21.15.151:0/3859760584 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f7a480b2900 0x7f7a480b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:33:48.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.676+0000 7f7a5e7fc700 1 --2- 172.21.15.151:0/3859760584 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f7a480b2900 0x7f7a480b4e10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7a60076e80 tx=0x7f7a5000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:33:48.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.680+0000 7f7a47fff700 1 -- 172.21.15.151:0/3859760584 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7a540a9850 con 0x7f7a60103c90 2024-05-17T22:33:48.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.896+0000 7f7a6553b700 1 -- 172.21.15.151:0/3859760584 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l", "target": ["mon-mgr", ""]}) -- 0x7f7a60063e70 con 0x7f7a480b2900 2024-05-17T22:33:48.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.905+0000 7f7a47fff700 1 -- 172.21.15.151:0/3859760584 <== mgr.14152 v2:172.21.15.12:6800/1388514047 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f7a60063e70 con 0x7f7a480b2900 2024-05-17T22:33:48.908 INFO:teuthology.orchestra.run.smithi151.stdout:Scheduled mds.all update... 2024-05-17T22:33:48.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.907+0000 7f7a6553b700 1 -- 172.21.15.151:0/3859760584 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f7a480b2900 msgr2=0x7f7a480b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:33:48.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.907+0000 7f7a6553b700 1 --2- 172.21.15.151:0/3859760584 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f7a480b2900 0x7f7a480b4e10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7a60076e80 tx=0x7f7a5000b410 comp rx=0 tx=0).stop 2024-05-17T22:33:48.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.907+0000 7f7a6553b700 1 -- 172.21.15.151:0/3859760584 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a60103c90 msgr2=0x7f7a600774d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:33:48.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.907+0000 7f7a6553b700 1 --2- 172.21.15.151:0/3859760584 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a60103c90 0x7f7a600774d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7a60104cc0 tx=0x7f7a5400dc30 comp rx=0 tx=0).stop 2024-05-17T22:33:48.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.907+0000 7f7a6553b700 1 -- 172.21.15.151:0/3859760584 shutdown_connections 2024-05-17T22:33:48.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.907+0000 7f7a6553b700 1 --2- 172.21.15.151:0/3859760584 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f7a480b2900 0x7f7a480b4e10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:48.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.907+0000 7f7a6553b700 1 --2- 172.21.15.151:0/3859760584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a60105760 0x7f7a600760c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:48.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.907+0000 7f7a6553b700 1 --2- 172.21.15.151:0/3859760584 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a60103c90 0x7f7a600774d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:48.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.907+0000 7f7a6553b700 1 --2- 172.21.15.151:0/3859760584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a60104dc0 0x7f7a60075b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:48.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.907+0000 7f7a6553b700 1 -- 172.21.15.151:0/3859760584 >> 172.21.15.151:0/3859760584 conn(0x7f7a600ff4c0 msgr2=0x7f7a60107ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:33:48.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.908+0000 7f7a6553b700 1 -- 172.21.15.151:0/3859760584 shutdown_connections 2024-05-17T22:33:48.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-17T22:33:48.908+0000 7f7a6553b700 1 -- 172.21.15.151:0/3859760584 wait complete. 2024-05-17T22:33:49.565 DEBUG:teuthology.orchestra.run.smithi012:mds.a> sudo journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@mds.a.service 2024-05-17T22:33:49.569 DEBUG:teuthology.orchestra.run.smithi012:mds.d> sudo journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@mds.d.service 2024-05-17T22:33:49.613 DEBUG:teuthology.orchestra.run.smithi012:mds.g> sudo journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@mds.g.service 2024-05-17T22:33:49.656 DEBUG:teuthology.orchestra.run.smithi012:mds.j> sudo journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@mds.j.service 2024-05-17T22:33:49.698 DEBUG:teuthology.orchestra.run.smithi096:mds.b> sudo journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@mds.b.service 2024-05-17T22:33:49.701 DEBUG:teuthology.orchestra.run.smithi096:mds.e> sudo journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@mds.e.service 2024-05-17T22:33:49.744 DEBUG:teuthology.orchestra.run.smithi096:mds.h> sudo journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@mds.h.service 2024-05-17T22:33:49.787 DEBUG:teuthology.orchestra.run.smithi096:mds.k> sudo journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@mds.k.service 2024-05-17T22:33:49.830 DEBUG:teuthology.orchestra.run.smithi151:mds.c> sudo journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@mds.c.service 2024-05-17T22:33:49.832 DEBUG:teuthology.orchestra.run.smithi151:mds.f> sudo journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@mds.f.service 2024-05-17T22:33:49.875 DEBUG:teuthology.orchestra.run.smithi151:mds.i> sudo journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@mds.i.service 2024-05-17T22:33:49.918 DEBUG:teuthology.orchestra.run.smithi151:mds.l> sudo journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@mds.l.service 2024-05-17T22:33:49.960 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-17T22:33:49.960 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T22:33:50.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:49 smithi096 ceph-mon[36065]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:33:50.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:49 smithi096 ceph-mon[36065]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:33:50.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:49 smithi096 ceph-mon[36065]: Saving service mds.all spec with placement smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l;count:12 2024-05-17T22:33:50.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:49 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:50.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:49 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:33:50.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:49 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:33:50.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:49 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:33:50.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:49 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:50.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:49 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T22:33:50.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:49 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T22:33:50.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:49 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:33:50.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:49 smithi096 ceph-mon[36065]: Deploying daemon mds.l on smithi151 2024-05-17T22:33:50.214 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:33:50.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:49 smithi151 ceph-mon[36101]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:33:50.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:49 smithi151 ceph-mon[36101]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:33:50.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:49 smithi151 ceph-mon[36101]: Saving service mds.all spec with placement smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l;count:12 2024-05-17T22:33:50.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:49 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:50.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:49 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:33:50.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:49 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:33:50.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:49 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:33:50.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:49 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:50.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:49 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T22:33:50.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:49 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T22:33:50.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:49 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:33:50.242 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:49 smithi151 ceph-mon[36101]: Deploying daemon mds.l on smithi151 2024-05-17T22:33:50.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:49 smithi012 ceph-mon[31632]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:33:50.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:49 smithi012 ceph-mon[31632]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:33:50.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:49 smithi012 ceph-mon[31632]: Saving service mds.all spec with placement smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l;count:12 2024-05-17T22:33:50.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:49 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:50.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:49 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:33:50.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:49 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:33:50.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:49 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:33:50.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:49 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:50.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:49 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T22:33:50.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:49 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T22:33:50.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:49 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:33:50.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:49 smithi012 ceph-mon[31632]: Deploying daemon mds.l on smithi151 2024-05-17T22:33:51.491 INFO:journalctl@ceph.mds.l.smithi151.stdout:May 17 22:33:51 smithi151 systemd[1]: Starting Ceph mds.l for 4d593780-149c-11ef-bc9a-c7b262605968... 2024-05-17T22:33:51.749 INFO:journalctl@ceph.mds.l.smithi151.stdout:May 17 22:33:51 smithi151 podman[66381]: 2024-05-17 22:33:51.392934646 +0000 UTC m=+0.017119888 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T22:33:51.749 INFO:journalctl@ceph.mds.l.smithi151.stdout:May 17 22:33:51 smithi151 podman[66381]: 2024-05-17 22:33:51.501803104 +0000 UTC m=+0.125988327 container create c7816c3d6909cba88c1b0032a98345f37bc66e4b23614eeba179a693322fb3a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-l, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-17T22:33:51.749 INFO:journalctl@ceph.mds.l.smithi151.stdout:May 17 22:33:51 smithi151 podman[66381]: 2024-05-17 22:33:51.746607603 +0000 UTC m=+0.370792822 container init c7816c3d6909cba88c1b0032a98345f37bc66e4b23614eeba179a693322fb3a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-l, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T22:33:51.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:51.980+0000 7f8cf747f700 1 -- 172.21.15.12:0/782229200 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cf0102e00 msgr2=0x7f8cf0103210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:33:51.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:51.980+0000 7f8cf747f700 1 --2- 172.21.15.12:0/782229200 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cf0102e00 0x7f8cf0103210 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8cec009990 tx=0x7f8cec009ca0 comp rx=0 tx=0).stop 2024-05-17T22:33:51.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:51.980+0000 7f8cf747f700 1 -- 172.21.15.12:0/782229200 shutdown_connections 2024-05-17T22:33:51.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:51.980+0000 7f8cf747f700 1 --2- 172.21.15.12:0/782229200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cf0108080 0x7f8cf010f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:51.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:51.980+0000 7f8cf747f700 1 --2- 172.21.15.12:0/782229200 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cf0102e00 0x7f8cf0103210 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:51.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:51.980+0000 7f8cf747f700 1 --2- 172.21.15.12:0/782229200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8cf0107730 0x7f8cf0107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:51.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:51.980+0000 7f8cf747f700 1 -- 172.21.15.12:0/782229200 >> 172.21.15.12:0/782229200 conn(0x7f8cf0079bd0 msgr2=0x7f8cf0079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:33:51.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:51.980+0000 7f8cf747f700 1 -- 172.21.15.12:0/782229200 shutdown_connections 2024-05-17T22:33:51.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:51.980+0000 7f8cf747f700 1 -- 172.21.15.12:0/782229200 wait complete. 2024-05-17T22:33:51.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:51 smithi012 ceph-mon[31632]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:33:51.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:51 smithi012 ceph-mon[31632]: mds.? [v2:172.21.15.151:6832/3554053059,v1:172.21.15.151:6833/3554053059] up:boot 2024-05-17T22:33:51.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:51 smithi012 ceph-mon[31632]: fsmap 1 up:standby 2024-05-17T22:33:51.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:51 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-17T22:33:51.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:51.981+0000 7f8cf747f700 1 Processor -- start 2024-05-17T22:33:51.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:51.981+0000 7f8cf747f700 1 -- start start 2024-05-17T22:33:51.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:51.982+0000 7f8cf747f700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cf0102e00 0x7f8cf019b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:33:51.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:51.982+0000 7f8cf747f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cf0107730 0x7f8cf019bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:33:51.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:51.982+0000 7f8cf747f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8cf0108080 0x7f8cf019c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:33:51.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:51.982+0000 7f8cf747f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf0112cb0 con 0x7f8cf0107730 2024-05-17T22:33:51.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:51.982+0000 7f8cf747f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf0112b30 con 0x7f8cf0108080 2024-05-17T22:33:51.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:51.982+0000 7f8cf747f700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf0112e30 con 0x7f8cf0102e00 2024-05-17T22:33:51.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:51.982+0000 7f8cf4a1a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cf0107730 0x7f8cf019bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:33:51.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:51.982+0000 7f8cf4a1a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cf0107730 0x7f8cf019bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42452/0 (socket says 172.21.15.12:42452) 2024-05-17T22:33:51.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:51.982+0000 7f8cf4a1a700 1 -- 172.21.15.12:0/2448796790 learned_addr learned my addr 172.21.15.12:0/2448796790 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:33:51.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:51.982+0000 7f8cf5a1c700 1 --2- 172.21.15.12:0/2448796790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8cf0108080 0x7f8cf019c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:33:51.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:51.982+0000 7f8cf521b700 1 --2- 172.21.15.12:0/2448796790 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cf0102e00 0x7f8cf019b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:33:51.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:51.982+0000 7f8cf4a1a700 1 -- 172.21.15.12:0/2448796790 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cf0102e00 msgr2=0x7f8cf019b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:33:51.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:51.982+0000 7f8cf4a1a700 1 --2- 172.21.15.12:0/2448796790 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cf0102e00 0x7f8cf019b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:51.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:51.982+0000 7f8cf4a1a700 1 -- 172.21.15.12:0/2448796790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8cf0108080 msgr2=0x7f8cf019c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:33:51.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:51.982+0000 7f8cf4a1a700 1 --2- 172.21.15.12:0/2448796790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8cf0108080 0x7f8cf019c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:51.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:51.982+0000 7f8cf4a1a700 1 -- 172.21.15.12:0/2448796790 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf0199490 con 0x7f8cf0107730 2024-05-17T22:33:51.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:51.982+0000 7f8cf4a1a700 1 --2- 172.21.15.12:0/2448796790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cf0107730 0x7f8cf019bb60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8ce000e970 tx=0x7f8ce000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:33:51.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:51.983+0000 7f8ce67fc700 1 -- 172.21.15.12:0/2448796790 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce0004a30 con 0x7f8cf0107730 2024-05-17T22:33:51.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:51.983+0000 7f8ce67fc700 1 -- 172.21.15.12:0/2448796790 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ce000cc90 con 0x7f8cf0107730 2024-05-17T22:33:51.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:51.983+0000 7f8ce67fc700 1 -- 172.21.15.12:0/2448796790 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce0005610 con 0x7f8cf0107730 2024-05-17T22:33:51.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:51.983+0000 7f8cf747f700 1 -- 172.21.15.12:0/2448796790 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cf0199780 con 0x7f8cf0107730 2024-05-17T22:33:51.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:51.983+0000 7f8cf747f700 1 -- 172.21.15.12:0/2448796790 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cf0199d10 con 0x7f8cf0107730 2024-05-17T22:33:51.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:51.984+0000 7f8cf747f700 1 -- 172.21.15.12:0/2448796790 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cf0103210 con 0x7f8cf0107730 2024-05-17T22:33:51.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:51.985+0000 7f8ce67fc700 1 -- 172.21.15.12:0/2448796790 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f8ce000c7e0 con 0x7f8cf0107730 2024-05-17T22:33:51.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:51.986+0000 7f8ce67fc700 1 --2- 172.21.15.12:0/2448796790 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8cdc0b2900 0x7f8cdc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:33:51.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:51.986+0000 7f8cf521b700 1 --2- 172.21.15.12:0/2448796790 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8cdc0b2900 0x7f8cdc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:33:51.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:51.986+0000 7f8ce67fc700 1 -- 172.21.15.12:0/2448796790 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f8ce00e20b0 con 0x7f8cf0107730 2024-05-17T22:33:51.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:51.987+0000 7f8cf521b700 1 --2- 172.21.15.12:0/2448796790 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8cdc0b2900 0x7f8cdc0b4e10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8cec00b2e0 tx=0x7f8cec005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:33:51.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:51.989+0000 7f8ce67fc700 1 -- 172.21.15.12:0/2448796790 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8ce0054df0 con 0x7f8cf0107730 2024-05-17T22:33:52.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:51 smithi151 ceph-mon[36101]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:33:52.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:51 smithi151 ceph-mon[36101]: mds.? [v2:172.21.15.151:6832/3554053059,v1:172.21.15.151:6833/3554053059] up:boot 2024-05-17T22:33:52.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:51 smithi151 ceph-mon[36101]: fsmap 1 up:standby 2024-05-17T22:33:52.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:51 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-17T22:33:52.241 INFO:journalctl@ceph.mds.l.smithi151.stdout:May 17 22:33:51 smithi151 podman[66381]: 2024-05-17 22:33:51.749182361 +0000 UTC m=+0.373367591 container start c7816c3d6909cba88c1b0032a98345f37bc66e4b23614eeba179a693322fb3a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, ceph=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-17T22:33:52.241 INFO:journalctl@ceph.mds.l.smithi151.stdout:May 17 22:33:51 smithi151 ceph-mds[66401]: -- 172.21.15.151:0/741715741 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560605059a40 con 0x56060509f180 2024-05-17T22:33:52.241 INFO:journalctl@ceph.mds.l.smithi151.stdout:May 17 22:33:51 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-l[66397]: starting mds.l at 2024-05-17T22:33:52.241 INFO:journalctl@ceph.mds.l.smithi151.stdout:May 17 22:33:51 smithi151 bash[66381]: c7816c3d6909cba88c1b0032a98345f37bc66e4b23614eeba179a693322fb3a1 2024-05-17T22:33:52.242 INFO:journalctl@ceph.mds.l.smithi151.stdout:May 17 22:33:51 smithi151 systemd[1]: Started Ceph mds.l for 4d593780-149c-11ef-bc9a-c7b262605968. 2024-05-17T22:33:52.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:52.253+0000 7f8cf747f700 1 -- 172.21.15.12:0/2448796790 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8cf006bc70 con 0x7f8cf0107730 2024-05-17T22:33:52.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:52.254+0000 7f8ce67fc700 1 -- 172.21.15.12:0/2448796790 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f8cf006bc70 con 0x7f8cf0107730 2024-05-17T22:33:52.257 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:33:52.257 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3554053059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3554053059},{"type":"v1","addr":"172.21.15.151:6833","nonce":3554053059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-17T22:33:52.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:52.256+0000 7f8cf747f700 1 -- 172.21.15.12:0/2448796790 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8cdc0b2900 msgr2=0x7f8cdc0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:33:52.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:52.256+0000 7f8cf747f700 1 --2- 172.21.15.12:0/2448796790 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8cdc0b2900 0x7f8cdc0b4e10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8cec00b2e0 tx=0x7f8cec005960 comp rx=0 tx=0).stop 2024-05-17T22:33:52.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:52.256+0000 7f8cf747f700 1 -- 172.21.15.12:0/2448796790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cf0107730 msgr2=0x7f8cf019bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:33:52.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:52.256+0000 7f8cf747f700 1 --2- 172.21.15.12:0/2448796790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cf0107730 0x7f8cf019bb60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8ce000e970 tx=0x7f8ce000ec80 comp rx=0 tx=0).stop 2024-05-17T22:33:52.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:52.256+0000 7f8cf747f700 1 -- 172.21.15.12:0/2448796790 shutdown_connections 2024-05-17T22:33:52.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:52.256+0000 7f8cf747f700 1 --2- 172.21.15.12:0/2448796790 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8cdc0b2900 0x7f8cdc0b4e10 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:52.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:52.256+0000 7f8cf747f700 1 --2- 172.21.15.12:0/2448796790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8cf0108080 0x7f8cf019c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:52.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:52.256+0000 7f8cf747f700 1 --2- 172.21.15.12:0/2448796790 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cf0102e00 0x7f8cf019b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:52.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:52.257+0000 7f8cf747f700 1 --2- 172.21.15.12:0/2448796790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cf0107730 0x7f8cf019bb60 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:52.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:52.257+0000 7f8cf747f700 1 -- 172.21.15.12:0/2448796790 >> 172.21.15.12:0/2448796790 conn(0x7f8cf0079bd0 msgr2=0x7f8cf0105870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:33:52.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:52.257+0000 7f8cf747f700 1 -- 172.21.15.12:0/2448796790 shutdown_connections 2024-05-17T22:33:52.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:52.257+0000 7f8cf747f700 1 -- 172.21.15.12:0/2448796790 wait complete. 2024-05-17T22:33:52.260 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 2 2024-05-17T22:33:52.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:51 smithi096 ceph-mon[36065]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:33:52.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:51 smithi096 ceph-mon[36065]: mds.? [v2:172.21.15.151:6832/3554053059,v1:172.21.15.151:6833/3554053059] up:boot 2024-05-17T22:33:52.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:51 smithi096 ceph-mon[36065]: fsmap 1 up:standby 2024-05-17T22:33:52.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:51 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-17T22:33:53.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:52 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:53.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:52 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:53.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:52 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:53.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:52 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T22:33:53.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:52 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T22:33:53.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:52 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:33:53.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:52 smithi151 ceph-mon[36101]: Deploying daemon mds.k on smithi096 2024-05-17T22:33:53.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:52 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/2448796790' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:33:53.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:52 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:53.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:52 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:53.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:52 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:53.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:52 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T22:33:53.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:52 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T22:33:53.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:52 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:33:53.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:52 smithi096 ceph-mon[36065]: Deploying daemon mds.k on smithi096 2024-05-17T22:33:53.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:52 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/2448796790' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:33:53.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:52 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:53.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:52 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:53.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:52 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:53.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:52 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T22:33:53.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:52 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T22:33:53.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:52 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:33:53.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:52 smithi012 ceph-mon[31632]: Deploying daemon mds.k on smithi096 2024-05-17T22:33:53.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:52 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/2448796790' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:33:54.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:53 smithi151 ceph-mon[36101]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:33:54.285 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:53 smithi096 ceph-mon[36065]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:33:54.285 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 17 22:33:54 smithi096 systemd[1]: Starting Ceph mds.k for 4d593780-149c-11ef-bc9a-c7b262605968... 2024-05-17T22:33:54.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:53 smithi012 ceph-mon[31632]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:33:54.549 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 17 22:33:54 smithi096 podman[65549]: 2024-05-17 22:33:54.282357033 +0000 UTC m=+0.112811846 container create 74fca3e3537123d1ec0aaea202e9f8613086527d41ea35542a52cb7a1eae735a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-k, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-17T22:33:54.549 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 17 22:33:54 smithi096 podman[65549]: 2024-05-17 22:33:54.18676927 +0000 UTC m=+0.017224090 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T22:33:54.876 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T22:33:55.049 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 17 22:33:54 smithi096 podman[65549]: 2024-05-17 22:33:54.551617994 +0000 UTC m=+0.382072816 container init 74fca3e3537123d1ec0aaea202e9f8613086527d41ea35542a52cb7a1eae735a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514) 2024-05-17T22:33:55.049 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 17 22:33:54 smithi096 podman[65549]: 2024-05-17 22:33:54.554310889 +0000 UTC m=+0.384765712 container start 74fca3e3537123d1ec0aaea202e9f8613086527d41ea35542a52cb7a1eae735a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-k, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.build-date=20240514, ceph=True, GIT_CLEAN=True) 2024-05-17T22:33:55.049 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 17 22:33:54 smithi096 ceph-mds[65568]: -- 172.21.15.96:0/846557182 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ea8ea8fa40 con 0x55ea8ead5600 2024-05-17T22:33:55.049 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 17 22:33:54 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-k[65564]: starting mds.k at 2024-05-17T22:33:55.049 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 17 22:33:54 smithi096 bash[65549]: 74fca3e3537123d1ec0aaea202e9f8613086527d41ea35542a52cb7a1eae735a 2024-05-17T22:33:55.049 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 17 22:33:54 smithi096 systemd[1]: Started Ceph mds.k for 4d593780-149c-11ef-bc9a-c7b262605968. 2024-05-17T22:33:55.133 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:33:55.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:55 smithi012 ceph-mon[31632]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:33:55.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:55 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:55.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:55 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:55.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:55 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:55.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:55 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T22:33:55.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:55 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T22:33:55.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:55 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:33:55.843 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:55 smithi012 ceph-mon[31632]: Deploying daemon mds.j on smithi012 2024-05-17T22:33:55.843 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:55 smithi012 ceph-mon[31632]: mds.? [v2:172.21.15.96:6832/4228325569,v1:172.21.15.96:6833/4228325569] up:boot 2024-05-17T22:33:55.843 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:55 smithi012 ceph-mon[31632]: fsmap 2 up:standby 2024-05-17T22:33:55.843 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:55 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-17T22:33:55.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:55 smithi151 ceph-mon[36101]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:33:55.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:55 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:55.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:55 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:55.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:55 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:55.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:55 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T22:33:55.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:55 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T22:33:55.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:55 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:33:55.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:55 smithi151 ceph-mon[36101]: Deploying daemon mds.j on smithi012 2024-05-17T22:33:55.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:55 smithi151 ceph-mon[36101]: mds.? [v2:172.21.15.96:6832/4228325569,v1:172.21.15.96:6833/4228325569] up:boot 2024-05-17T22:33:55.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:55 smithi151 ceph-mon[36101]: fsmap 2 up:standby 2024-05-17T22:33:55.992 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:55 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-17T22:33:56.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:55 smithi096 ceph-mon[36065]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:33:56.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:55 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:56.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:55 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:56.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:55 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:56.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:55 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T22:33:56.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:55 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T22:33:56.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:55 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:33:56.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:55 smithi096 ceph-mon[36065]: Deploying daemon mds.j on smithi012 2024-05-17T22:33:56.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:55 smithi096 ceph-mon[36065]: mds.? [v2:172.21.15.96:6832/4228325569,v1:172.21.15.96:6833/4228325569] up:boot 2024-05-17T22:33:56.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:55 smithi096 ceph-mon[36065]: fsmap 2 up:standby 2024-05-17T22:33:56.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:55 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-17T22:33:57.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.614+0000 7f2044180700 1 -- 172.21.15.12:0/356399103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f203c102e00 msgr2=0x7f203c103210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:33:57.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.614+0000 7f2044180700 1 --2- 172.21.15.12:0/356399103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f203c102e00 0x7f203c103210 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f203800b0a0 tx=0x7f203800b3b0 comp rx=0 tx=0).stop 2024-05-17T22:33:57.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.615+0000 7f2044180700 1 -- 172.21.15.12:0/356399103 shutdown_connections 2024-05-17T22:33:57.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.615+0000 7f2044180700 1 --2- 172.21.15.12:0/356399103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f203c108080 0x7f203c10f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:57.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.615+0000 7f2044180700 1 --2- 172.21.15.12:0/356399103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f203c102e00 0x7f203c103210 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:57.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.615+0000 7f2044180700 1 --2- 172.21.15.12:0/356399103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f203c107730 0x7f203c107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:57.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.615+0000 7f2044180700 1 -- 172.21.15.12:0/356399103 >> 172.21.15.12:0/356399103 conn(0x7f203c079bd0 msgr2=0x7f203c079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:33:57.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.615+0000 7f2044180700 1 -- 172.21.15.12:0/356399103 shutdown_connections 2024-05-17T22:33:57.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.615+0000 7f2044180700 1 -- 172.21.15.12:0/356399103 wait complete. 2024-05-17T22:33:57.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.616+0000 7f2044180700 1 Processor -- start 2024-05-17T22:33:57.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.616+0000 7f2044180700 1 -- start start 2024-05-17T22:33:57.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.616+0000 7f2044180700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f203c102e00 0x7f203c199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:33:57.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.616+0000 7f2044180700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f203c107730 0x7f203c199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:33:57.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.616+0000 7f2044180700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f203c108080 0x7f203c19dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:33:57.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.616+0000 7f2044180700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f203c112cb0 con 0x7f203c102e00 2024-05-17T22:33:57.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.616+0000 7f2044180700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f203c112b30 con 0x7f203c107730 2024-05-17T22:33:57.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.616+0000 7f2044180700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f203c112e30 con 0x7f203c108080 2024-05-17T22:33:57.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.616+0000 7f2041f1c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f203c102e00 0x7f203c199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:33:57.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.616+0000 7f204171b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f203c107730 0x7f203c199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:33:57.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.616+0000 7f204271d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f203c108080 0x7f203c19dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:33:57.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.616+0000 7f204171b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f203c107730 0x7f203c199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:46698/0 (socket says 172.21.15.12:46698) 2024-05-17T22:33:57.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.616+0000 7f204171b700 1 -- 172.21.15.12:0/2736558223 learned_addr learned my addr 172.21.15.12:0/2736558223 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:33:57.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.616+0000 7f2041f1c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f203c102e00 0x7f203c199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42478/0 (socket says 172.21.15.12:42478) 2024-05-17T22:33:57.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.617+0000 7f204171b700 1 -- 172.21.15.12:0/2736558223 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f203c108080 msgr2=0x7f203c19dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:33:57.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.617+0000 7f204171b700 1 --2- 172.21.15.12:0/2736558223 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f203c108080 0x7f203c19dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:57.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.617+0000 7f204171b700 1 -- 172.21.15.12:0/2736558223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f203c102e00 msgr2=0x7f203c199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:33:57.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.617+0000 7f204171b700 1 --2- 172.21.15.12:0/2736558223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f203c102e00 0x7f203c199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:57.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.617+0000 7f204171b700 1 -- 172.21.15.12:0/2736558223 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f203c19e560 con 0x7f203c107730 2024-05-17T22:33:57.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.617+0000 7f2041f1c700 1 --2- 172.21.15.12:0/2736558223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f203c102e00 0x7f203c199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:33:57.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.617+0000 7f204171b700 1 --2- 172.21.15.12:0/2736558223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f203c107730 0x7f203c199c30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f202c00d980 tx=0x7f202c00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:33:57.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.617+0000 7f2032ffd700 1 -- 172.21.15.12:0/2736558223 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202c017070 con 0x7f203c107730 2024-05-17T22:33:57.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.617+0000 7f2032ffd700 1 -- 172.21.15.12:0/2736558223 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f202c00fd40 con 0x7f203c107730 2024-05-17T22:33:57.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.617+0000 7f2044180700 1 -- 172.21.15.12:0/2736558223 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f203c19e850 con 0x7f203c107730 2024-05-17T22:33:57.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.617+0000 7f2032ffd700 1 -- 172.21.15.12:0/2736558223 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202c00e3b0 con 0x7f203c107730 2024-05-17T22:33:57.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.617+0000 7f2044180700 1 -- 172.21.15.12:0/2736558223 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f203c06b680 con 0x7f203c107730 2024-05-17T22:33:57.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.618+0000 7f2044180700 1 -- 172.21.15.12:0/2736558223 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f203c1042b0 con 0x7f203c107730 2024-05-17T22:33:57.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.620+0000 7f2032ffd700 1 -- 172.21.15.12:0/2736558223 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f202c010610 con 0x7f203c107730 2024-05-17T22:33:57.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.621+0000 7f2032ffd700 1 --2- 172.21.15.12:0/2736558223 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f20280b2880 0x7f20280b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:33:57.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.621+0000 7f2041f1c700 1 --2- 172.21.15.12:0/2736558223 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f20280b2880 0x7f20280b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:33:57.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.621+0000 7f2032ffd700 1 -- 172.21.15.12:0/2736558223 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f202c0e17d0 con 0x7f203c107730 2024-05-17T22:33:57.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.621+0000 7f2041f1c700 1 --2- 172.21.15.12:0/2736558223 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f20280b2880 0x7f20280b4d90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2038009900 tx=0x7f203801c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:33:57.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.628+0000 7f2032ffd700 1 -- 172.21.15.12:0/2736558223 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f202c0a9990 con 0x7f203c107730 2024-05-17T22:33:57.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.894+0000 7f2044180700 1 -- 172.21.15.12:0/2736558223 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f203c103210 con 0x7f203c107730 2024-05-17T22:33:57.897 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:57 smithi012 ceph-mon[31632]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:33:57.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.895+0000 7f2032ffd700 1 -- 172.21.15.12:0/2736558223 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f202c0a9290 con 0x7f203c107730 2024-05-17T22:33:57.898 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:33:57.898 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3554053059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3554053059},{"type":"v1","addr":"172.21.15.151:6833","nonce":3554053059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/4228325569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":4228325569},{"type":"v1","addr":"172.21.15.96:6833","nonce":4228325569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-17T22:33:57.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.897+0000 7f2044180700 1 -- 172.21.15.12:0/2736558223 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f20280b2880 msgr2=0x7f20280b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:33:57.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.897+0000 7f2044180700 1 --2- 172.21.15.12:0/2736558223 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f20280b2880 0x7f20280b4d90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2038009900 tx=0x7f203801c040 comp rx=0 tx=0).stop 2024-05-17T22:33:57.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.897+0000 7f2044180700 1 -- 172.21.15.12:0/2736558223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f203c107730 msgr2=0x7f203c199c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:33:57.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.897+0000 7f2044180700 1 --2- 172.21.15.12:0/2736558223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f203c107730 0x7f203c199c30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f202c00d980 tx=0x7f202c00dc90 comp rx=0 tx=0).stop 2024-05-17T22:33:57.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.897+0000 7f2044180700 1 -- 172.21.15.12:0/2736558223 shutdown_connections 2024-05-17T22:33:57.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.897+0000 7f2044180700 1 --2- 172.21.15.12:0/2736558223 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f20280b2880 0x7f20280b4d90 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:57.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.897+0000 7f2044180700 1 --2- 172.21.15.12:0/2736558223 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f203c108080 0x7f203c19dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:57.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.897+0000 7f2044180700 1 --2- 172.21.15.12:0/2736558223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f203c102e00 0x7f203c199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:57.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.897+0000 7f2044180700 1 --2- 172.21.15.12:0/2736558223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f203c107730 0x7f203c199c30 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:33:57.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.897+0000 7f2044180700 1 -- 172.21.15.12:0/2736558223 >> 172.21.15.12:0/2736558223 conn(0x7f203c079bd0 msgr2=0x7f203c1064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:33:57.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.897+0000 7f2044180700 1 -- 172.21.15.12:0/2736558223 shutdown_connections 2024-05-17T22:33:57.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:33:57.897+0000 7f2044180700 1 -- 172.21.15.12:0/2736558223 wait complete. 2024-05-17T22:33:57.901 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 3 2024-05-17T22:33:57.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:57 smithi151 ceph-mon[36101]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:33:58.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:57 smithi096 ceph-mon[36065]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:33:58.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:58 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/2736558223' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:33:58.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:58 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/2736558223' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:33:59.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:58 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/2736558223' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:33:59.594 INFO:journalctl@ceph.mds.j.smithi012.stdout:May 17 22:33:59 smithi012 systemd[1]: Started Ceph mds.j for 4d593780-149c-11ef-bc9a-c7b262605968. 2024-05-17T22:33:59.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:59 smithi151 ceph-mon[36101]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:33:59.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:59 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:59.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:59 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:59.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:59 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:33:59.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:59 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T22:33:59.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:59 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T22:33:59.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:33:59 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:34:00.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:59 smithi096 ceph-mon[36065]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:00.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:59 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:00.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:59 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:00.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:59 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:00.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:59 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T22:34:00.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:59 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T22:34:00.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:33:59 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:34:00.061 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:59 smithi012 ceph-mon[31632]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:00.061 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:59 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:00.061 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:59 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:00.061 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:59 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:00.061 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:59 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T22:34:00.061 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:59 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T22:34:00.061 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:33:59 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:34:00.785 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T22:34:00.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:00 smithi151 ceph-mon[36101]: Deploying daemon mds.d on smithi012 2024-05-17T22:34:00.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:00 smithi151 ceph-mon[36101]: mds.? [v2:172.21.15.12:6834/1407219319,v1:172.21.15.12:6835/1407219319] up:boot 2024-05-17T22:34:00.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:00 smithi151 ceph-mon[36101]: fsmap 3 up:standby 2024-05-17T22:34:00.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-17T22:34:01.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:00 smithi096 ceph-mon[36065]: Deploying daemon mds.d on smithi012 2024-05-17T22:34:01.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:00 smithi096 ceph-mon[36065]: mds.? [v2:172.21.15.12:6834/1407219319,v1:172.21.15.12:6835/1407219319] up:boot 2024-05-17T22:34:01.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:00 smithi096 ceph-mon[36065]: fsmap 3 up:standby 2024-05-17T22:34:01.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:00 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-17T22:34:01.066 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:00 smithi012 ceph-mon[31632]: Deploying daemon mds.d on smithi012 2024-05-17T22:34:01.066 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:00 smithi012 ceph-mon[31632]: mds.? [v2:172.21.15.12:6834/1407219319,v1:172.21.15.12:6835/1407219319] up:boot 2024-05-17T22:34:01.066 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:00 smithi012 ceph-mon[31632]: fsmap 3 up:standby 2024-05-17T22:34:01.066 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:00 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-17T22:34:01.154 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:34:01.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:01 smithi151 ceph-mon[36101]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:02.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:01 smithi096 ceph-mon[36065]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:02.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:01 smithi012 ceph-mon[31632]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:03.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.242+0000 7f2c82d29700 1 -- 172.21.15.12:0/3113187468 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c7c102e00 msgr2=0x7f2c7c103210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:03.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.242+0000 7f2c82d29700 1 --2- 172.21.15.12:0/3113187468 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c7c102e00 0x7f2c7c103210 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2c70009990 tx=0x7f2c70009ca0 comp rx=0 tx=0).stop 2024-05-17T22:34:03.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.242+0000 7f2c82d29700 1 -- 172.21.15.12:0/3113187468 shutdown_connections 2024-05-17T22:34:03.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.242+0000 7f2c82d29700 1 --2- 172.21.15.12:0/3113187468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c7c108080 0x7f2c7c10f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:03.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.242+0000 7f2c82d29700 1 --2- 172.21.15.12:0/3113187468 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c7c102e00 0x7f2c7c103210 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:03.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.242+0000 7f2c82d29700 1 --2- 172.21.15.12:0/3113187468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c7c107730 0x7f2c7c107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:03.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.242+0000 7f2c82d29700 1 -- 172.21.15.12:0/3113187468 >> 172.21.15.12:0/3113187468 conn(0x7f2c7c079bd0 msgr2=0x7f2c7c079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:34:03.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.242+0000 7f2c82d29700 1 -- 172.21.15.12:0/3113187468 shutdown_connections 2024-05-17T22:34:03.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.242+0000 7f2c82d29700 1 -- 172.21.15.12:0/3113187468 wait complete. 2024-05-17T22:34:03.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.243+0000 7f2c82d29700 1 Processor -- start 2024-05-17T22:34:03.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.243+0000 7f2c82d29700 1 -- start start 2024-05-17T22:34:03.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.243+0000 7f2c82d29700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c7c102e00 0x7f2c7c1a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:03.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.243+0000 7f2c82d29700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c7c107730 0x7f2c7c1a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:03.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.243+0000 7f2c82d29700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c7c108080 0x7f2c7c19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:03.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.243+0000 7f2c82d29700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7c112db0 con 0x7f2c7c107730 2024-05-17T22:34:03.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.243+0000 7f2c82d29700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7c112c30 con 0x7f2c7c108080 2024-05-17T22:34:03.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.243+0000 7f2c82d29700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7c112f30 con 0x7f2c7c102e00 2024-05-17T22:34:03.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.243+0000 7f2c7bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c7c107730 0x7f2c7c1a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:03.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.243+0000 7f2c7bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c7c107730 0x7f2c7c1a0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41308/0 (socket says 172.21.15.12:41308) 2024-05-17T22:34:03.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.243+0000 7f2c7bfff700 1 -- 172.21.15.12:0/3842403174 learned_addr learned my addr 172.21.15.12:0/3842403174 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:34:03.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.243+0000 7f2c80ac5700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c7c102e00 0x7f2c7c1a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:03.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.243+0000 7f2c812c6700 1 --2- 172.21.15.12:0/3842403174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c7c108080 0x7f2c7c19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:03.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.244+0000 7f2c7bfff700 1 -- 172.21.15.12:0/3842403174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c7c102e00 msgr2=0x7f2c7c1a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:03.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.244+0000 7f2c7bfff700 1 --2- 172.21.15.12:0/3842403174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c7c102e00 0x7f2c7c1a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:03.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.244+0000 7f2c7bfff700 1 -- 172.21.15.12:0/3842403174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c7c108080 msgr2=0x7f2c7c19a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:03.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.244+0000 7f2c7bfff700 1 --2- 172.21.15.12:0/3842403174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c7c108080 0x7f2c7c19a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:03.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.244+0000 7f2c7bfff700 1 -- 172.21.15.12:0/3842403174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c7c19ae80 con 0x7f2c7c107730 2024-05-17T22:34:03.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.244+0000 7f2c812c6700 1 --2- 172.21.15.12:0/3842403174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c7c108080 0x7f2c7c19a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:34:03.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.244+0000 7f2c7bfff700 1 --2- 172.21.15.12:0/3842403174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c7c107730 0x7f2c7c1a0b00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2c6800b500 tx=0x7f2c6800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:34:03.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.244+0000 7f2c79ffb700 1 -- 172.21.15.12:0/3842403174 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c68015020 con 0x7f2c7c107730 2024-05-17T22:34:03.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.244+0000 7f2c82d29700 1 -- 172.21.15.12:0/3842403174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c7c19b170 con 0x7f2c7c107730 2024-05-17T22:34:03.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.244+0000 7f2c79ffb700 1 -- 172.21.15.12:0/3842403174 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c68011ce0 con 0x7f2c7c107730 2024-05-17T22:34:03.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.244+0000 7f2c79ffb700 1 -- 172.21.15.12:0/3842403174 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c68010d40 con 0x7f2c7c107730 2024-05-17T22:34:03.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.244+0000 7f2c82d29700 1 -- 172.21.15.12:0/3842403174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c7c1a7370 con 0x7f2c7c107730 2024-05-17T22:34:03.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.245+0000 7f2c82d29700 1 -- 172.21.15.12:0/3842403174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c7c1042b0 con 0x7f2c7c107730 2024-05-17T22:34:03.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.247+0000 7f2c79ffb700 1 -- 172.21.15.12:0/3842403174 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f2c68018020 con 0x7f2c7c107730 2024-05-17T22:34:03.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.247+0000 7f2c79ffb700 1 --2- 172.21.15.12:0/3842403174 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f2c6c0b2880 0x7f2c6c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:03.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.247+0000 7f2c80ac5700 1 --2- 172.21.15.12:0/3842403174 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f2c6c0b2880 0x7f2c6c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:03.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.248+0000 7f2c79ffb700 1 -- 172.21.15.12:0/3842403174 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f2c680e1a40 con 0x7f2c7c107730 2024-05-17T22:34:03.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.248+0000 7f2c80ac5700 1 --2- 172.21.15.12:0/3842403174 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f2c6c0b2880 0x7f2c6c0b4d90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2c7000b5c0 tx=0x7f2c70005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:34:03.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.251+0000 7f2c79ffb700 1 -- 172.21.15.12:0/3842403174 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2c6801f020 con 0x7f2c7c107730 2024-05-17T22:34:03.385 INFO:journalctl@ceph.mds.d.smithi012.stdout:May 17 22:34:03 smithi012 podman[71912]: 2024-05-17 22:34:03.348561297 +0000 UTC m=+0.794918554 container start 16550dd7efce5dc440eb13283a78176d8498b7ea81fba66b668142a2242c5ff3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-d, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514) 2024-05-17T22:34:03.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.539+0000 7f2c82d29700 1 -- 172.21.15.12:0/3842403174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c7c19bc60 con 0x7f2c7c107730 2024-05-17T22:34:03.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.539+0000 7f2c79ffb700 1 -- 172.21.15.12:0/3842403174 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7f2c680a9c00 con 0x7f2c7c107730 2024-05-17T22:34:03.544 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:34:03.544 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1407219319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1407219319},{"type":"v1","addr":"172.21.15.12:6835","nonce":1407219319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3554053059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3554053059},{"type":"v1","addr":"172.21.15.151:6833","nonce":3554053059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/4228325569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":4228325569},{"type":"v1","addr":"172.21.15.96:6833","nonce":4228325569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-17T22:34:03.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.542+0000 7f2c82d29700 1 -- 172.21.15.12:0/3842403174 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f2c6c0b2880 msgr2=0x7f2c6c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:03.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.542+0000 7f2c82d29700 1 --2- 172.21.15.12:0/3842403174 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f2c6c0b2880 0x7f2c6c0b4d90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2c7000b5c0 tx=0x7f2c70005960 comp rx=0 tx=0).stop 2024-05-17T22:34:03.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.542+0000 7f2c82d29700 1 -- 172.21.15.12:0/3842403174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c7c107730 msgr2=0x7f2c7c1a0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:03.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.542+0000 7f2c82d29700 1 --2- 172.21.15.12:0/3842403174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c7c107730 0x7f2c7c1a0b00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2c6800b500 tx=0x7f2c6800b810 comp rx=0 tx=0).stop 2024-05-17T22:34:03.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.542+0000 7f2c82d29700 1 -- 172.21.15.12:0/3842403174 shutdown_connections 2024-05-17T22:34:03.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.542+0000 7f2c82d29700 1 --2- 172.21.15.12:0/3842403174 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f2c6c0b2880 0x7f2c6c0b4d90 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:03.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.542+0000 7f2c82d29700 1 --2- 172.21.15.12:0/3842403174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c7c108080 0x7f2c7c19a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:03.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.542+0000 7f2c82d29700 1 --2- 172.21.15.12:0/3842403174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c7c102e00 0x7f2c7c1a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:03.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.542+0000 7f2c82d29700 1 --2- 172.21.15.12:0/3842403174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c7c107730 0x7f2c7c1a0b00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:03.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.542+0000 7f2c82d29700 1 -- 172.21.15.12:0/3842403174 >> 172.21.15.12:0/3842403174 conn(0x7f2c7c079bd0 msgr2=0x7f2c7c1023f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:34:03.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.542+0000 7f2c82d29700 1 -- 172.21.15.12:0/3842403174 shutdown_connections 2024-05-17T22:34:03.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:03.542+0000 7f2c82d29700 1 -- 172.21.15.12:0/3842403174 wait complete. 2024-05-17T22:34:03.548 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 4 2024-05-17T22:34:03.736 INFO:journalctl@ceph.mds.d.smithi012.stdout:May 17 22:34:03 smithi012 ceph-mds[71970]: -- 172.21.15.12:0/3709665009 <== mon.0 v2:172.21.15.12:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f5619a5a40 con 0x55f5619eb600 2024-05-17T22:34:03.736 INFO:journalctl@ceph.mds.d.smithi012.stdout:May 17 22:34:03 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-d[71944]: starting mds.d at 2024-05-17T22:34:03.736 INFO:journalctl@ceph.mds.d.smithi012.stdout:May 17 22:34:03 smithi012 bash[71912]: 16550dd7efce5dc440eb13283a78176d8498b7ea81fba66b668142a2242c5ff3 2024-05-17T22:34:03.737 INFO:journalctl@ceph.mds.d.smithi012.stdout:May 17 22:34:03 smithi012 systemd[1]: Started Ceph mds.d for 4d593780-149c-11ef-bc9a-c7b262605968. 2024-05-17T22:34:03.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:03 smithi151 ceph-mon[36101]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:03.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:03 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:03.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:03 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:03.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:03 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:03.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:03 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T22:34:03.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:03 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/3842403174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:34:03.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:03 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T22:34:03.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:03 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:34:04.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:03 smithi096 ceph-mon[36065]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:04.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:03 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:04.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:03 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:04.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:03 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:04.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:03 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T22:34:04.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:03 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/3842403174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:34:04.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:03 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T22:34:04.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:03 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:34:04.068 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:03 smithi012 ceph-mon[31632]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:04.068 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:03 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:04.068 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:03 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:04.068 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:03 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:04.068 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:03 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T22:34:04.068 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:03 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/3842403174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:34:04.069 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:03 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T22:34:04.069 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:03 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:34:05.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:04 smithi096 ceph-mon[36065]: Deploying daemon mds.e on smithi096 2024-05-17T22:34:05.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:04 smithi096 ceph-mon[36065]: mds.? [v2:172.21.15.12:6836/630977886,v1:172.21.15.12:6837/630977886] up:boot 2024-05-17T22:34:05.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:04 smithi096 ceph-mon[36065]: fsmap 4 up:standby 2024-05-17T22:34:05.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:04 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-17T22:34:05.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:04 smithi012 ceph-mon[31632]: Deploying daemon mds.e on smithi096 2024-05-17T22:34:05.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:04 smithi012 ceph-mon[31632]: mds.? [v2:172.21.15.12:6836/630977886,v1:172.21.15.12:6837/630977886] up:boot 2024-05-17T22:34:05.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:04 smithi012 ceph-mon[31632]: fsmap 4 up:standby 2024-05-17T22:34:05.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:04 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-17T22:34:05.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:04 smithi151 ceph-mon[36101]: Deploying daemon mds.e on smithi096 2024-05-17T22:34:05.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:04 smithi151 ceph-mon[36101]: mds.? [v2:172.21.15.12:6836/630977886,v1:172.21.15.12:6837/630977886] up:boot 2024-05-17T22:34:05.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:04 smithi151 ceph-mon[36101]: fsmap 4 up:standby 2024-05-17T22:34:05.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:04 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-17T22:34:05.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:05 smithi096 ceph-mon[36065]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:06.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:05 smithi012 ceph-mon[31632]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:06.200 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T22:34:06.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:05 smithi151 ceph-mon[36101]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:06.470 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:34:06.548 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 17 22:34:06 smithi096 systemd[1]: Started Ceph mds.e for 4d593780-149c-11ef-bc9a-c7b262605968. 2024-05-17T22:34:07.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:07 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:07.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:07 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:07.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:07 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:07.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:07 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T22:34:07.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:07 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T22:34:07.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:07 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:34:07.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:07 smithi012 ceph-mon[31632]: Deploying daemon mds.f on smithi151 2024-05-17T22:34:07.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:07 smithi012 ceph-mon[31632]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:07.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:07 smithi012 ceph-mon[31632]: mds.? [v2:172.21.15.96:6834/1758442232,v1:172.21.15.96:6835/1758442232] up:boot 2024-05-17T22:34:07.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:07 smithi012 ceph-mon[31632]: fsmap 5 up:standby 2024-05-17T22:34:07.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:07 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-17T22:34:07.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:07 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:07.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:07 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:07.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:07 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:07.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:07 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T22:34:07.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:07 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T22:34:07.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:07 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:34:07.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:07 smithi151 ceph-mon[36101]: Deploying daemon mds.f on smithi151 2024-05-17T22:34:07.742 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:07 smithi151 ceph-mon[36101]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:07.742 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:07 smithi151 ceph-mon[36101]: mds.? [v2:172.21.15.96:6834/1758442232,v1:172.21.15.96:6835/1758442232] up:boot 2024-05-17T22:34:07.742 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:07 smithi151 ceph-mon[36101]: fsmap 5 up:standby 2024-05-17T22:34:07.742 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:07 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-17T22:34:07.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:07 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:07.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:07 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:07.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:07 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:07.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:07 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T22:34:07.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:07 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T22:34:07.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:07 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:34:07.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:07 smithi096 ceph-mon[36065]: Deploying daemon mds.f on smithi151 2024-05-17T22:34:07.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:07 smithi096 ceph-mon[36065]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:07.800 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:07 smithi096 ceph-mon[36065]: mds.? [v2:172.21.15.96:6834/1758442232,v1:172.21.15.96:6835/1758442232] up:boot 2024-05-17T22:34:07.800 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:07 smithi096 ceph-mon[36065]: fsmap 5 up:standby 2024-05-17T22:34:07.800 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:07 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-17T22:34:08.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.217+0000 7f9643f5f700 1 -- 172.21.15.12:0/3090939192 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f963c108080 msgr2=0x7f963c10f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:08.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.217+0000 7f9643f5f700 1 --2- 172.21.15.12:0/3090939192 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f963c108080 0x7f963c10f9e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9634009990 tx=0x7f9634009ca0 comp rx=0 tx=0).stop 2024-05-17T22:34:08.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.217+0000 7f9643f5f700 1 -- 172.21.15.12:0/3090939192 shutdown_connections 2024-05-17T22:34:08.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.217+0000 7f9643f5f700 1 --2- 172.21.15.12:0/3090939192 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f963c108080 0x7f963c10f9e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:08.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.217+0000 7f9643f5f700 1 --2- 172.21.15.12:0/3090939192 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f963c102e00 0x7f963c103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:08.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.217+0000 7f9643f5f700 1 --2- 172.21.15.12:0/3090939192 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f963c107730 0x7f963c107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:08.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.217+0000 7f9643f5f700 1 -- 172.21.15.12:0/3090939192 >> 172.21.15.12:0/3090939192 conn(0x7f963c079bd0 msgr2=0x7f963c079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:34:08.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.217+0000 7f9643f5f700 1 -- 172.21.15.12:0/3090939192 shutdown_connections 2024-05-17T22:34:08.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.217+0000 7f9643f5f700 1 -- 172.21.15.12:0/3090939192 wait complete. 2024-05-17T22:34:08.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.218+0000 7f9643f5f700 1 Processor -- start 2024-05-17T22:34:08.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.218+0000 7f9643f5f700 1 -- start start 2024-05-17T22:34:08.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.218+0000 7f9643f5f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f963c102e00 0x7f963c19b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:08.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.218+0000 7f9643f5f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f963c107730 0x7f963c19bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:08.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.218+0000 7f9643f5f700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f963c108080 0x7f963c19c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:08.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.218+0000 7f9643f5f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f963c112d30 con 0x7f963c102e00 2024-05-17T22:34:08.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.218+0000 7f9643f5f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f963c112bb0 con 0x7f963c107730 2024-05-17T22:34:08.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.218+0000 7f9643f5f700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f963c112eb0 con 0x7f963c108080 2024-05-17T22:34:08.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.218+0000 7f9641cfb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f963c102e00 0x7f963c19b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:08.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.218+0000 7f9641cfb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f963c102e00 0x7f963c19b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41334/0 (socket says 172.21.15.12:41334) 2024-05-17T22:34:08.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.218+0000 7f9641cfb700 1 -- 172.21.15.12:0/1291650954 learned_addr learned my addr 172.21.15.12:0/1291650954 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:34:08.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.218+0000 7f96424fc700 1 --2- 172.21.15.12:0/1291650954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f963c108080 0x7f963c19c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:08.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.218+0000 7f96414fa700 1 --2- 172.21.15.12:0/1291650954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f963c107730 0x7f963c19bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:08.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.219+0000 7f9641cfb700 1 -- 172.21.15.12:0/1291650954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f963c108080 msgr2=0x7f963c19c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:08.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.219+0000 7f9641cfb700 1 --2- 172.21.15.12:0/1291650954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f963c108080 0x7f963c19c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:08.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.219+0000 7f9641cfb700 1 -- 172.21.15.12:0/1291650954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f963c107730 msgr2=0x7f963c19bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:08.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.219+0000 7f9641cfb700 1 --2- 172.21.15.12:0/1291650954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f963c107730 0x7f963c19bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:08.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.219+0000 7f9641cfb700 1 -- 172.21.15.12:0/1291650954 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f963c1a6a70 con 0x7f963c102e00 2024-05-17T22:34:08.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.219+0000 7f96424fc700 1 --2- 172.21.15.12:0/1291650954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f963c108080 0x7f963c19c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:34:08.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.219+0000 7f9641cfb700 1 --2- 172.21.15.12:0/1291650954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f963c102e00 0x7f963c19b720 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f962c00ba50 tx=0x7f962c00bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:34:08.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.219+0000 7f963affd700 1 -- 172.21.15.12:0/1291650954 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c019070 con 0x7f963c102e00 2024-05-17T22:34:08.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.219+0000 7f9643f5f700 1 -- 172.21.15.12:0/1291650954 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f963c1a6d60 con 0x7f963c102e00 2024-05-17T22:34:08.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.219+0000 7f963affd700 1 -- 172.21.15.12:0/1291650954 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f962c00cde0 con 0x7f963c102e00 2024-05-17T22:34:08.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.219+0000 7f963affd700 1 -- 172.21.15.12:0/1291650954 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c007370 con 0x7f963c102e00 2024-05-17T22:34:08.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.220+0000 7f9643f5f700 1 -- 172.21.15.12:0/1291650954 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f963c1a7210 con 0x7f963c102e00 2024-05-17T22:34:08.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.221+0000 7f9643f5f700 1 -- 172.21.15.12:0/1291650954 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f963c103210 con 0x7f963c102e00 2024-05-17T22:34:08.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.223+0000 7f963affd700 1 -- 172.21.15.12:0/1291650954 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f962c01c070 con 0x7f963c102e00 2024-05-17T22:34:08.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.224+0000 7f963affd700 1 --2- 172.21.15.12:0/1291650954 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f96240b2900 0x7f96240b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:08.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.224+0000 7f96414fa700 1 --2- 172.21.15.12:0/1291650954 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f96240b2900 0x7f96240b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:08.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.224+0000 7f963affd700 1 -- 172.21.15.12:0/1291650954 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f962c0e1a40 con 0x7f963c102e00 2024-05-17T22:34:08.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.224+0000 7f96414fa700 1 --2- 172.21.15.12:0/1291650954 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f96240b2900 0x7f96240b4e10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f963c07a120 tx=0x7f963000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:34:08.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.227+0000 7f963affd700 1 -- 172.21.15.12:0/1291650954 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f962c023070 con 0x7f963c102e00 2024-05-17T22:34:08.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.487+0000 7f9643f5f700 1 -- 172.21.15.12:0/1291650954 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f963c02ea80 con 0x7f963c102e00 2024-05-17T22:34:08.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.488+0000 7f963affd700 1 -- 172.21.15.12:0/1291650954 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4693 (secure 0 0 0) 0x7f962c0a9b60 con 0x7f963c102e00 2024-05-17T22:34:08.492 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:34:08.492 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1407219319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1407219319},{"type":"v1","addr":"172.21.15.12:6835","nonce":1407219319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/630977886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":630977886},{"type":"v1","addr":"172.21.15.12:6837","nonce":630977886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3554053059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3554053059},{"type":"v1","addr":"172.21.15.151:6833","nonce":3554053059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/4228325569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":4228325569},{"type":"v1","addr":"172.21.15.96:6833","nonce":4228325569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1758442232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1758442232},{"type":"v1","addr":"172.21.15.96:6835","nonce":1758442232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-17T22:34:08.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.490+0000 7f9643f5f700 1 -- 172.21.15.12:0/1291650954 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f96240b2900 msgr2=0x7f96240b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:08.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.490+0000 7f9643f5f700 1 --2- 172.21.15.12:0/1291650954 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f96240b2900 0x7f96240b4e10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f963c07a120 tx=0x7f963000a300 comp rx=0 tx=0).stop 2024-05-17T22:34:08.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.490+0000 7f9643f5f700 1 -- 172.21.15.12:0/1291650954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f963c102e00 msgr2=0x7f963c19b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:08.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.490+0000 7f9643f5f700 1 --2- 172.21.15.12:0/1291650954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f963c102e00 0x7f963c19b720 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f962c00ba50 tx=0x7f962c00bd60 comp rx=0 tx=0).stop 2024-05-17T22:34:08.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.490+0000 7f9643f5f700 1 -- 172.21.15.12:0/1291650954 shutdown_connections 2024-05-17T22:34:08.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.490+0000 7f9643f5f700 1 --2- 172.21.15.12:0/1291650954 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f96240b2900 0x7f96240b4e10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:08.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.490+0000 7f9643f5f700 1 --2- 172.21.15.12:0/1291650954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f963c108080 0x7f963c19c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:08.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.490+0000 7f9643f5f700 1 --2- 172.21.15.12:0/1291650954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f963c102e00 0x7f963c19b720 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:08.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.490+0000 7f9643f5f700 1 --2- 172.21.15.12:0/1291650954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f963c107730 0x7f963c19bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:08.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.490+0000 7f9643f5f700 1 -- 172.21.15.12:0/1291650954 >> 172.21.15.12:0/1291650954 conn(0x7f963c079bd0 msgr2=0x7f963c105810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:34:08.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.491+0000 7f9643f5f700 1 -- 172.21.15.12:0/1291650954 shutdown_connections 2024-05-17T22:34:08.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:08.491+0000 7f9643f5f700 1 -- 172.21.15.12:0/1291650954 wait complete. 2024-05-17T22:34:08.495 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 6 2024-05-17T22:34:08.783 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:08 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/1291650954' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:34:08.783 INFO:journalctl@ceph.mds.f.smithi151.stdout:May 17 22:34:08 smithi151 podman[66688]: 2024-05-17 22:34:08.648293287 +0000 UTC m=+0.016888238 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T22:34:08.784 INFO:journalctl@ceph.mds.f.smithi151.stdout:May 17 22:34:08 smithi151 podman[66688]: 2024-05-17 22:34:08.781433084 +0000 UTC m=+0.150028044 container create 3033908f42192f9b0ec7e43f52f203545201f0c9a294c57a50bb6dec92d0450f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-f, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-17T22:34:08.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:08 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/1291650954' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:34:08.800 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:08 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/1291650954' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:34:09.055 INFO:journalctl@ceph.mds.f.smithi151.stdout:May 17 22:34:09 smithi151 podman[66688]: 2024-05-17 22:34:09.018015737 +0000 UTC m=+0.386610665 container init 3033908f42192f9b0ec7e43f52f203545201f0c9a294c57a50bb6dec92d0450f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-f, ceph=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a) 2024-05-17T22:34:09.055 INFO:journalctl@ceph.mds.f.smithi151.stdout:May 17 22:34:09 smithi151 podman[66688]: 2024-05-17 22:34:09.020567813 +0000 UTC m=+0.389162758 container start 3033908f42192f9b0ec7e43f52f203545201f0c9a294c57a50bb6dec92d0450f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-f, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-17T22:34:09.490 INFO:journalctl@ceph.mds.f.smithi151.stdout:May 17 22:34:09 smithi151 ceph-mds[66707]: -- 172.21.15.151:0/3320856329 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559bf2d21a40 con 0x559bf2d67180 2024-05-17T22:34:09.491 INFO:journalctl@ceph.mds.f.smithi151.stdout:May 17 22:34:09 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-f[66703]: starting mds.f at 2024-05-17T22:34:09.491 INFO:journalctl@ceph.mds.f.smithi151.stdout:May 17 22:34:09 smithi151 bash[66688]: 3033908f42192f9b0ec7e43f52f203545201f0c9a294c57a50bb6dec92d0450f 2024-05-17T22:34:09.491 INFO:journalctl@ceph.mds.f.smithi151.stdout:May 17 22:34:09 smithi151 systemd[1]: Started Ceph mds.f for 4d593780-149c-11ef-bc9a-c7b262605968. 2024-05-17T22:34:09.826 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:09 smithi151 ceph-mon[36101]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:09.826 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:09 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:09.826 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:09 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:09.826 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:09 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:09.826 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:09 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T22:34:09.827 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:09 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T22:34:09.827 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:09 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:34:09.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:09 smithi012 ceph-mon[31632]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:09.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:09 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:09.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:09 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:09.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:09 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:09.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:09 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T22:34:09.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:09 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T22:34:09.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:09 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:34:10.001 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:09 smithi096 ceph-mon[36065]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:10.001 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:10.001 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:10.002 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:10.002 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T22:34:10.002 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T22:34:10.002 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:34:10.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:10 smithi012 ceph-mon[31632]: Deploying daemon mds.c on smithi151 2024-05-17T22:34:10.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:10 smithi012 ceph-mon[31632]: mds.? [v2:172.21.15.151:6834/542323754,v1:172.21.15.151:6835/542323754] up:boot 2024-05-17T22:34:10.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:10 smithi012 ceph-mon[31632]: fsmap 6 up:standby 2024-05-17T22:34:10.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:10 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-17T22:34:10.849 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:10 smithi151 ceph-mon[36101]: Deploying daemon mds.c on smithi151 2024-05-17T22:34:10.849 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:10 smithi151 ceph-mon[36101]: mds.? [v2:172.21.15.151:6834/542323754,v1:172.21.15.151:6835/542323754] up:boot 2024-05-17T22:34:10.849 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:10 smithi151 ceph-mon[36101]: fsmap 6 up:standby 2024-05-17T22:34:10.849 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:10 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-17T22:34:11.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:10 smithi096 ceph-mon[36065]: Deploying daemon mds.c on smithi151 2024-05-17T22:34:11.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:10 smithi096 ceph-mon[36065]: mds.? [v2:172.21.15.151:6834/542323754,v1:172.21.15.151:6835/542323754] up:boot 2024-05-17T22:34:11.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:10 smithi096 ceph-mon[36065]: fsmap 6 up:standby 2024-05-17T22:34:11.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:10 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-17T22:34:11.166 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T22:34:11.432 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:34:11.491 INFO:journalctl@ceph.mds.c.smithi151.stdout:May 17 22:34:11 smithi151 systemd[1]: Starting Ceph mds.c for 4d593780-149c-11ef-bc9a-c7b262605968... 2024-05-17T22:34:11.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:11 smithi012 ceph-mon[31632]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:11.932 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:11 smithi151 ceph-mon[36101]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:11.932 INFO:journalctl@ceph.mds.c.smithi151.stdout:May 17 22:34:11 smithi151 podman[66993]: 2024-05-17 22:34:11.452254917 +0000 UTC m=+0.017391424 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T22:34:11.932 INFO:journalctl@ceph.mds.c.smithi151.stdout:May 17 22:34:11 smithi151 podman[66993]: 2024-05-17 22:34:11.576212857 +0000 UTC m=+0.141349353 container create 25c9781f4db4dbebe6e976afa70f11bd2d279ca79220528b484570a41dfc5d33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-17T22:34:11.933 INFO:journalctl@ceph.mds.c.smithi151.stdout:May 17 22:34:11 smithi151 podman[66993]: 2024-05-17 22:34:11.930569138 +0000 UTC m=+0.495705639 container init 25c9781f4db4dbebe6e976afa70f11bd2d279ca79220528b484570a41dfc5d33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-c, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-17T22:34:12.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:11 smithi096 ceph-mon[36065]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:12.241 INFO:journalctl@ceph.mds.c.smithi151.stdout:May 17 22:34:11 smithi151 podman[66993]: 2024-05-17 22:34:11.933192383 +0000 UTC m=+0.498328884 container start 25c9781f4db4dbebe6e976afa70f11bd2d279ca79220528b484570a41dfc5d33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, io.buildah.version=1.33.5) 2024-05-17T22:34:12.241 INFO:journalctl@ceph.mds.c.smithi151.stdout:May 17 22:34:11 smithi151 ceph-mds[67012]: -- 172.21.15.151:0/2056733585 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b263cb1a40 con 0x55b263cf7180 2024-05-17T22:34:12.241 INFO:journalctl@ceph.mds.c.smithi151.stdout:May 17 22:34:11 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-c[67008]: starting mds.c at 2024-05-17T22:34:12.241 INFO:journalctl@ceph.mds.c.smithi151.stdout:May 17 22:34:12 smithi151 bash[66993]: 25c9781f4db4dbebe6e976afa70f11bd2d279ca79220528b484570a41dfc5d33 2024-05-17T22:34:12.241 INFO:journalctl@ceph.mds.c.smithi151.stdout:May 17 22:34:12 smithi151 systemd[1]: Started Ceph mds.c for 4d593780-149c-11ef-bc9a-c7b262605968. 2024-05-17T22:34:13.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.053+0000 7f979fc83700 1 -- 172.21.15.12:0/3821463855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9798102ab0 msgr2=0x7f9798102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:13.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.053+0000 7f979fc83700 1 --2- 172.21.15.12:0/3821463855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9798102ab0 0x7f9798102ec0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9794009990 tx=0x7f9794009ca0 comp rx=0 tx=0).stop 2024-05-17T22:34:13.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.053+0000 7f979fc83700 1 -- 172.21.15.12:0/3821463855 shutdown_connections 2024-05-17T22:34:13.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.053+0000 7f979fc83700 1 --2- 172.21.15.12:0/3821463855 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9798110660 0x7f9798112a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:13.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.053+0000 7f979fc83700 1 --2- 172.21.15.12:0/3821463855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9798102ab0 0x7f9798102ec0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:13.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.053+0000 7f979fc83700 1 --2- 172.21.15.12:0/3821463855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9798103400 0x7f9798110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:13.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.053+0000 7f979fc83700 1 -- 172.21.15.12:0/3821463855 >> 172.21.15.12:0/3821463855 conn(0x7f9798079ae0 msgr2=0x7f9798079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:34:13.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.053+0000 7f979fc83700 1 -- 172.21.15.12:0/3821463855 shutdown_connections 2024-05-17T22:34:13.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.053+0000 7f979fc83700 1 -- 172.21.15.12:0/3821463855 wait complete. 2024-05-17T22:34:13.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.054+0000 7f979fc83700 1 Processor -- start 2024-05-17T22:34:13.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.054+0000 7f979fc83700 1 -- start start 2024-05-17T22:34:13.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.054+0000 7f979fc83700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9798102ab0 0x7f97981a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:13.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.054+0000 7f979fc83700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9798103400 0x7f97981a2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:13.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.054+0000 7f979fc83700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9798110660 0x7f979819c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:13.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.054+0000 7f979fc83700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9798115ff0 con 0x7f9798110660 2024-05-17T22:34:13.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.054+0000 7f979fc83700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9798115e70 con 0x7f9798102ab0 2024-05-17T22:34:13.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.054+0000 7f979fc83700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9798116170 con 0x7f9798103400 2024-05-17T22:34:13.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.054+0000 7f979e220700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9798110660 0x7f979819c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:13.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.054+0000 7f979e220700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9798110660 0x7f979819c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49160/0 (socket says 172.21.15.12:49160) 2024-05-17T22:34:13.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.054+0000 7f979e220700 1 -- 172.21.15.12:0/2940133102 learned_addr learned my addr 172.21.15.12:0/2940133102 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:34:13.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.054+0000 7f979da1f700 1 --2- 172.21.15.12:0/2940133102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9798102ab0 0x7f97981a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:13.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.054+0000 7f979d21e700 1 --2- 172.21.15.12:0/2940133102 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9798103400 0x7f97981a2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:13.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.055+0000 7f979e220700 1 -- 172.21.15.12:0/2940133102 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9798103400 msgr2=0x7f97981a2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:13.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.055+0000 7f979e220700 1 --2- 172.21.15.12:0/2940133102 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9798103400 0x7f97981a2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:13.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.055+0000 7f979e220700 1 -- 172.21.15.12:0/2940133102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9798102ab0 msgr2=0x7f97981a2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:13.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.055+0000 7f979e220700 1 --2- 172.21.15.12:0/2940133102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9798102ab0 0x7f97981a2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:13.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.055+0000 7f979e220700 1 -- 172.21.15.12:0/2940133102 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f979819ce60 con 0x7f9798110660 2024-05-17T22:34:13.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.055+0000 7f979d21e700 1 --2- 172.21.15.12:0/2940133102 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9798103400 0x7f97981a2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:34:13.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.055+0000 7f979e220700 1 --2- 172.21.15.12:0/2940133102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9798110660 0x7f979819c7c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9790007ee0 tx=0x7f979000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:34:13.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.055+0000 7f978effd700 1 -- 172.21.15.12:0/2940133102 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979000dbd0 con 0x7f9798110660 2024-05-17T22:34:13.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.055+0000 7f978effd700 1 -- 172.21.15.12:0/2940133102 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f979000dd70 con 0x7f9798110660 2024-05-17T22:34:13.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.055+0000 7f979fc83700 1 -- 172.21.15.12:0/2940133102 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f979819d150 con 0x7f9798110660 2024-05-17T22:34:13.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.056+0000 7f978effd700 1 -- 172.21.15.12:0/2940133102 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9790005990 con 0x7f9798110660 2024-05-17T22:34:13.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.056+0000 7f979fc83700 1 -- 172.21.15.12:0/2940133102 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97981a93e0 con 0x7f9798110660 2024-05-17T22:34:13.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.057+0000 7f979fc83700 1 -- 172.21.15.12:0/2940133102 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f979806c850 con 0x7f9798110660 2024-05-17T22:34:13.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.058+0000 7f978effd700 1 -- 172.21.15.12:0/2940133102 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f979001a020 con 0x7f9798110660 2024-05-17T22:34:13.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.059+0000 7f978effd700 1 --2- 172.21.15.12:0/2940133102 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f97840b2900 0x7f97840b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:13.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.059+0000 7f978effd700 1 -- 172.21.15.12:0/2940133102 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f97900e1aa0 con 0x7f9798110660 2024-05-17T22:34:13.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.059+0000 7f979da1f700 1 --2- 172.21.15.12:0/2940133102 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f97840b2900 0x7f97840b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:13.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.060+0000 7f979da1f700 1 --2- 172.21.15.12:0/2940133102 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f97840b2900 0x7f97840b4e10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f979400b5c0 tx=0x7f979401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:34:13.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.063+0000 7f978effd700 1 -- 172.21.15.12:0/2940133102 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9790010070 con 0x7f9798110660 2024-05-17T22:34:13.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:13 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:13.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:13 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:13.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:13 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:13.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:13 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T22:34:13.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:13 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T22:34:13.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:13 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:34:13.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:13 smithi096 ceph-mon[36065]: Deploying daemon mds.h on smithi096 2024-05-17T22:34:13.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:13 smithi096 ceph-mon[36065]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:13.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:13 smithi096 ceph-mon[36065]: mds.? [v2:172.21.15.151:6836/3403324038,v1:172.21.15.151:6837/3403324038] up:boot 2024-05-17T22:34:13.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:13 smithi096 ceph-mon[36065]: fsmap 7 up:standby 2024-05-17T22:34:13.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:13 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-17T22:34:13.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.328+0000 7f979fc83700 1 -- 172.21.15.12:0/2940133102 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9798102ec0 con 0x7f9798110660 2024-05-17T22:34:13.331 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:13 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:13.331 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:13 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:13.331 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:13 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:13.332 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:13 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T22:34:13.332 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:13 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T22:34:13.332 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:13 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:34:13.332 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:13 smithi012 ceph-mon[31632]: Deploying daemon mds.h on smithi096 2024-05-17T22:34:13.332 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:13 smithi012 ceph-mon[31632]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:13.332 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:13 smithi012 ceph-mon[31632]: mds.? [v2:172.21.15.151:6836/3403324038,v1:172.21.15.151:6837/3403324038] up:boot 2024-05-17T22:34:13.332 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:13 smithi012 ceph-mon[31632]: fsmap 7 up:standby 2024-05-17T22:34:13.332 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:13 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-17T22:34:13.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.329+0000 7f978effd700 1 -- 172.21.15.12:0/2940133102 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6346 (secure 0 0 0) 0x7f97900a9c60 con 0x7f9798110660 2024-05-17T22:34:13.333 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:34:13.333 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1407219319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1407219319},{"type":"v1","addr":"172.21.15.12:6835","nonce":1407219319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/630977886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":630977886},{"type":"v1","addr":"172.21.15.12:6837","nonce":630977886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3554053059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3554053059},{"type":"v1","addr":"172.21.15.151:6833","nonce":3554053059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/4228325569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":4228325569},{"type":"v1","addr":"172.21.15.96:6833","nonce":4228325569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/542323754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":542323754},{"type":"v1","addr":"172.21.15.151:6835","nonce":542323754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1758442232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1758442232},{"type":"v1","addr":"172.21.15.96:6835","nonce":1758442232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/3403324038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3403324038},{"type":"v1","addr":"172.21.15.151:6837","nonce":3403324038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-05-17T22:34:13.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.331+0000 7f979fc83700 1 -- 172.21.15.12:0/2940133102 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f97840b2900 msgr2=0x7f97840b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:13.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.331+0000 7f979fc83700 1 --2- 172.21.15.12:0/2940133102 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f97840b2900 0x7f97840b4e10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f979400b5c0 tx=0x7f979401a040 comp rx=0 tx=0).stop 2024-05-17T22:34:13.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.331+0000 7f979fc83700 1 -- 172.21.15.12:0/2940133102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9798110660 msgr2=0x7f979819c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:13.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.331+0000 7f979fc83700 1 --2- 172.21.15.12:0/2940133102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9798110660 0x7f979819c7c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9790007ee0 tx=0x7f979000d3b0 comp rx=0 tx=0).stop 2024-05-17T22:34:13.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.331+0000 7f979fc83700 1 -- 172.21.15.12:0/2940133102 shutdown_connections 2024-05-17T22:34:13.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.331+0000 7f979fc83700 1 --2- 172.21.15.12:0/2940133102 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f97840b2900 0x7f97840b4e10 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:13.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.331+0000 7f979fc83700 1 --2- 172.21.15.12:0/2940133102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9798110660 0x7f979819c7c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:13.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.331+0000 7f979fc83700 1 --2- 172.21.15.12:0/2940133102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9798102ab0 0x7f97981a2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:13.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.331+0000 7f979fc83700 1 --2- 172.21.15.12:0/2940133102 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9798103400 0x7f97981a2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:13.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.331+0000 7f979fc83700 1 -- 172.21.15.12:0/2940133102 >> 172.21.15.12:0/2940133102 conn(0x7f9798079ae0 msgr2=0x7f9798111aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:34:13.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.331+0000 7f979fc83700 1 -- 172.21.15.12:0/2940133102 shutdown_connections 2024-05-17T22:34:13.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:13.331+0000 7f979fc83700 1 -- 172.21.15.12:0/2940133102 wait complete. 2024-05-17T22:34:13.335 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 8 2024-05-17T22:34:13.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:13 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:13.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:13 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:13.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:13 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:13.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:13 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T22:34:13.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:13 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T22:34:13.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:13 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:34:13.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:13 smithi151 ceph-mon[36101]: Deploying daemon mds.h on smithi096 2024-05-17T22:34:13.492 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:13 smithi151 ceph-mon[36101]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:13.492 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:13 smithi151 ceph-mon[36101]: mds.? [v2:172.21.15.151:6836/3403324038,v1:172.21.15.151:6837/3403324038] up:boot 2024-05-17T22:34:13.492 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:13 smithi151 ceph-mon[36101]: fsmap 7 up:standby 2024-05-17T22:34:13.492 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:13 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-17T22:34:14.200 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:14 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/2940133102' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:34:14.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:14 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/2940133102' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:34:14.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:14 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/2940133102' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:34:14.549 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 17 22:34:14 smithi096 systemd[1]: Starting Ceph mds.h for 4d593780-149c-11ef-bc9a-c7b262605968... 2024-05-17T22:34:14.880 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 17 22:34:14 smithi096 podman[66168]: 2024-05-17 22:34:14.498947252 +0000 UTC m=+0.017201757 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T22:34:14.880 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 17 22:34:14 smithi096 podman[66168]: 2024-05-17 22:34:14.616962854 +0000 UTC m=+0.135217335 container create fe0a8da3710df63b0f880e8a03ec06759fe61b95326e63757e8aa5dda6681b81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-h, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True) 2024-05-17T22:34:14.880 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 17 22:34:14 smithi096 podman[66168]: 2024-05-17 22:34:14.878489761 +0000 UTC m=+0.396744248 container init fe0a8da3710df63b0f880e8a03ec06759fe61b95326e63757e8aa5dda6681b81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-h, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS) 2024-05-17T22:34:15.152 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 17 22:34:14 smithi096 podman[66168]: 2024-05-17 22:34:14.881186689 +0000 UTC m=+0.399441177 container start fe0a8da3710df63b0f880e8a03ec06759fe61b95326e63757e8aa5dda6681b81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-h, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-17T22:34:15.153 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 17 22:34:14 smithi096 ceph-mds[66187]: -- 172.21.15.96:0/3950305136 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557df568fa40 con 0x557df56d5180 2024-05-17T22:34:15.153 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 17 22:34:14 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-h[66183]: starting mds.h at 2024-05-17T22:34:15.153 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 17 22:34:14 smithi096 bash[66168]: fe0a8da3710df63b0f880e8a03ec06759fe61b95326e63757e8aa5dda6681b81 2024-05-17T22:34:15.153 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 17 22:34:14 smithi096 systemd[1]: Started Ceph mds.h for 4d593780-149c-11ef-bc9a-c7b262605968. 2024-05-17T22:34:15.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:15 smithi151 ceph-mon[36101]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:15.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:15 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:15.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:15 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:15.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:15 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:15.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:15 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T22:34:15.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:15 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T22:34:15.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:15 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:34:15.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:15 smithi096 ceph-mon[36065]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:15.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:15 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:15.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:15 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:15.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:15 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:15.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:15 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T22:34:15.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:15 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T22:34:15.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:15 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:34:15.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:15 smithi012 ceph-mon[31632]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:15.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:15 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:15.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:15 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:15.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:15 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:15.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:15 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T22:34:15.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:15 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T22:34:15.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:15 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:34:15.966 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T22:34:16.232 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:34:16.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:16 smithi151 ceph-mon[36101]: Deploying daemon mds.i on smithi151 2024-05-17T22:34:16.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:16 smithi151 ceph-mon[36101]: mds.? [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] up:boot 2024-05-17T22:34:16.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:16 smithi151 ceph-mon[36101]: fsmap 8 up:standby 2024-05-17T22:34:16.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:16 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-17T22:34:16.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:16 smithi012 ceph-mon[31632]: Deploying daemon mds.i on smithi151 2024-05-17T22:34:16.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:16 smithi012 ceph-mon[31632]: mds.? [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] up:boot 2024-05-17T22:34:16.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:16 smithi012 ceph-mon[31632]: fsmap 8 up:standby 2024-05-17T22:34:16.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:16 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-17T22:34:16.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:16 smithi096 ceph-mon[36065]: Deploying daemon mds.i on smithi151 2024-05-17T22:34:16.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:16 smithi096 ceph-mon[36065]: mds.? [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] up:boot 2024-05-17T22:34:16.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:16 smithi096 ceph-mon[36065]: fsmap 8 up:standby 2024-05-17T22:34:16.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:16 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-17T22:34:17.325 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:17 smithi012 ceph-mon[31632]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:17.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:17 smithi151 ceph-mon[36101]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:17.491 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 17 22:34:17 smithi151 systemd[1]: Starting Ceph mds.i for 4d593780-149c-11ef-bc9a-c7b262605968... 2024-05-17T22:34:17.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:17 smithi096 ceph-mon[36065]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:17.878 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 17 22:34:17 smithi151 podman[67294]: 2024-05-17 22:34:17.42122627 +0000 UTC m=+0.016912678 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T22:34:17.878 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 17 22:34:17 smithi151 podman[67294]: 2024-05-17 22:34:17.598010984 +0000 UTC m=+0.193697388 container create b5eb2658cde1e034703198d8fbb5ec0cb6aff966ca3010b30eebbde55f689108 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, ceph=True, GIT_CLEAN=True) 2024-05-17T22:34:17.878 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 17 22:34:17 smithi151 podman[67294]: 2024-05-17 22:34:17.876392736 +0000 UTC m=+0.472079139 container init b5eb2658cde1e034703198d8fbb5ec0cb6aff966ca3010b30eebbde55f689108 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-i, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T22:34:17.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:17.912+0000 7f027e70f700 1 -- 172.21.15.12:0/1509022657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0278103f10 msgr2=0x7f0278104320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:17.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:17.912+0000 7f027e70f700 1 --2- 172.21.15.12:0/1509022657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0278103f10 0x7f0278104320 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0268009990 tx=0x7f0268009ca0 comp rx=0 tx=0).stop 2024-05-17T22:34:17.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:17.912+0000 7f027e70f700 1 -- 172.21.15.12:0/1509022657 shutdown_connections 2024-05-17T22:34:17.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:17.912+0000 7f027e70f700 1 --2- 172.21.15.12:0/1509022657 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0278109190 0x7f0278110af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:17.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:17.912+0000 7f027e70f700 1 --2- 172.21.15.12:0/1509022657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0278103f10 0x7f0278104320 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:17.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:17.912+0000 7f027e70f700 1 --2- 172.21.15.12:0/1509022657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0278108840 0x7f0278108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:17.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:17.912+0000 7f027e70f700 1 -- 172.21.15.12:0/1509022657 >> 172.21.15.12:0/1509022657 conn(0x7f02780ff9b0 msgr2=0x7f0278101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:34:17.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:17.912+0000 7f027e70f700 1 -- 172.21.15.12:0/1509022657 shutdown_connections 2024-05-17T22:34:17.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:17.912+0000 7f027e70f700 1 -- 172.21.15.12:0/1509022657 wait complete. 2024-05-17T22:34:17.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:17.913+0000 7f027e70f700 1 Processor -- start 2024-05-17T22:34:17.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:17.913+0000 7f027e70f700 1 -- start start 2024-05-17T22:34:17.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:17.913+0000 7f027e70f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0278103f10 0x7f027819a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:17.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:17.913+0000 7f027e70f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0278108840 0x7f027819ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:17.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:17.913+0000 7f027e70f700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0278109190 0x7f027819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:17.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:17.913+0000 7f027e70f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0278113dc0 con 0x7f0278103f10 2024-05-17T22:34:17.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:17.913+0000 7f027e70f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0278113c40 con 0x7f0278108840 2024-05-17T22:34:17.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:17.913+0000 7f027e70f700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0278113f40 con 0x7f0278109190 2024-05-17T22:34:17.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:17.913+0000 7f0277fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0278103f10 0x7f027819a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:17.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:17.913+0000 7f0277fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0278103f10 0x7f027819a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49176/0 (socket says 172.21.15.12:49176) 2024-05-17T22:34:17.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:17.914+0000 7f0277fff700 1 -- 172.21.15.12:0/1353841498 learned_addr learned my addr 172.21.15.12:0/1353841498 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:34:17.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:17.914+0000 7f027ccac700 1 --2- 172.21.15.12:0/1353841498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0278109190 0x7f027819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:17.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:17.914+0000 7f02777fe700 1 --2- 172.21.15.12:0/1353841498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0278108840 0x7f027819ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:17.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:17.914+0000 7f0277fff700 1 -- 172.21.15.12:0/1353841498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0278109190 msgr2=0x7f027819f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:17.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:17.914+0000 7f0277fff700 1 --2- 172.21.15.12:0/1353841498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0278109190 0x7f027819f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:17.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:17.914+0000 7f0277fff700 1 -- 172.21.15.12:0/1353841498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0278108840 msgr2=0x7f027819ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:17.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:17.914+0000 7f0277fff700 1 --2- 172.21.15.12:0/1353841498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0278108840 0x7f027819ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:17.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:17.914+0000 7f0277fff700 1 -- 172.21.15.12:0/1353841498 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f027819f6b0 con 0x7f0278103f10 2024-05-17T22:34:17.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:17.914+0000 7f0277fff700 1 --2- 172.21.15.12:0/1353841498 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0278103f10 0x7f027819a7e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0268000c00 tx=0x7f0268009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:34:17.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:17.914+0000 7f02757fa700 1 -- 172.21.15.12:0/1353841498 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0268027070 con 0x7f0278103f10 2024-05-17T22:34:17.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:17.914+0000 7f027e70f700 1 -- 172.21.15.12:0/1353841498 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f027819f940 con 0x7f0278103f10 2024-05-17T22:34:17.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:17.914+0000 7f02757fa700 1 -- 172.21.15.12:0/1353841498 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0268022470 con 0x7f0278103f10 2024-05-17T22:34:17.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:17.914+0000 7f02757fa700 1 -- 172.21.15.12:0/1353841498 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0268017650 con 0x7f0278103f10 2024-05-17T22:34:17.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:17.914+0000 7f027e70f700 1 -- 172.21.15.12:0/1353841498 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02781a72c0 con 0x7f0278103f10 2024-05-17T22:34:17.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:17.916+0000 7f027e70f700 1 -- 172.21.15.12:0/1353841498 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02781053c0 con 0x7f0278103f10 2024-05-17T22:34:17.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:17.917+0000 7f02757fa700 1 -- 172.21.15.12:0/1353841498 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f026802a070 con 0x7f0278103f10 2024-05-17T22:34:17.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:17.918+0000 7f02757fa700 1 --2- 172.21.15.12:0/1353841498 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f02600b2880 0x7f02600b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:17.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:17.918+0000 7f02777fe700 1 --2- 172.21.15.12:0/1353841498 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f02600b2880 0x7f02600b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:17.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:17.918+0000 7f02757fa700 1 -- 172.21.15.12:0/1353841498 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f02680e2d60 con 0x7f0278103f10 2024-05-17T22:34:17.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:17.919+0000 7f02777fe700 1 --2- 172.21.15.12:0/1353841498 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f02600b2880 0x7f02600b4d90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0278107440 tx=0x7f026c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:34:17.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:17.921+0000 7f02757fa700 1 -- 172.21.15.12:0/1353841498 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0268013070 con 0x7f0278103f10 2024-05-17T22:34:18.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:18.193+0000 7f027e70f700 1 -- 172.21.15.12:0/1353841498 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0278104320 con 0x7f0278103f10 2024-05-17T22:34:18.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:18.194+0000 7f02757fa700 1 -- 172.21.15.12:0/1353841498 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8001 (secure 0 0 0) 0x7f02680aafb0 con 0x7f0278103f10 2024-05-17T22:34:18.197 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:34:18.197 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1407219319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1407219319},{"type":"v1","addr":"172.21.15.12:6835","nonce":1407219319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/630977886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":630977886},{"type":"v1","addr":"172.21.15.12:6837","nonce":630977886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3554053059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3554053059},{"type":"v1","addr":"172.21.15.151:6833","nonce":3554053059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/4228325569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":4228325569},{"type":"v1","addr":"172.21.15.96:6833","nonce":4228325569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/542323754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":542323754},{"type":"v1","addr":"172.21.15.151:6835","nonce":542323754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1758442232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1758442232},{"type":"v1","addr":"172.21.15.96:6835","nonce":1758442232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/3403324038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3403324038},{"type":"v1","addr":"172.21.15.151:6837","nonce":3403324038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/2338353849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2338353849},{"type":"v1","addr":"172.21.15.151:6839","nonce":2338353849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/4222574338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":4222574338},{"type":"v1","addr":"172.21.15.96:6837","nonce":4222574338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-17T22:34:18.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:18.197+0000 7f027e70f700 1 -- 172.21.15.12:0/1353841498 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f02600b2880 msgr2=0x7f02600b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:18.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:18.197+0000 7f027e70f700 1 --2- 172.21.15.12:0/1353841498 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f02600b2880 0x7f02600b4d90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0278107440 tx=0x7f026c0073d0 comp rx=0 tx=0).stop 2024-05-17T22:34:18.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:18.197+0000 7f027e70f700 1 -- 172.21.15.12:0/1353841498 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0278103f10 msgr2=0x7f027819a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:18.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:18.197+0000 7f027e70f700 1 --2- 172.21.15.12:0/1353841498 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0278103f10 0x7f027819a7e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0268000c00 tx=0x7f0268009580 comp rx=0 tx=0).stop 2024-05-17T22:34:18.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:18.197+0000 7f027e70f700 1 -- 172.21.15.12:0/1353841498 shutdown_connections 2024-05-17T22:34:18.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:18.197+0000 7f027e70f700 1 --2- 172.21.15.12:0/1353841498 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f02600b2880 0x7f02600b4d90 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:18.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:18.197+0000 7f027e70f700 1 --2- 172.21.15.12:0/1353841498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0278109190 0x7f027819f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:18.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:18.197+0000 7f027e70f700 1 --2- 172.21.15.12:0/1353841498 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0278103f10 0x7f027819a7e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:18.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:18.197+0000 7f027e70f700 1 --2- 172.21.15.12:0/1353841498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0278108840 0x7f027819ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:18.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:18.197+0000 7f027e70f700 1 -- 172.21.15.12:0/1353841498 >> 172.21.15.12:0/1353841498 conn(0x7f02780ff9b0 msgr2=0x7f0278101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:34:18.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:18.197+0000 7f027e70f700 1 -- 172.21.15.12:0/1353841498 shutdown_connections 2024-05-17T22:34:18.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:18.197+0000 7f027e70f700 1 -- 172.21.15.12:0/1353841498 wait complete. 2024-05-17T22:34:18.201 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 10 2024-05-17T22:34:18.241 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 17 22:34:17 smithi151 podman[67294]: 2024-05-17 22:34:17.879082992 +0000 UTC m=+0.474769398 container start b5eb2658cde1e034703198d8fbb5ec0cb6aff966ca3010b30eebbde55f689108 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-i, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.build-date=20240514, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-17T22:34:18.241 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 17 22:34:17 smithi151 ceph-mds[67313]: -- 172.21.15.151:0/2580066170 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561a4b999a40 con 0x561a4b9dfa80 2024-05-17T22:34:18.241 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 17 22:34:17 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-i[67309]: starting mds.i at 2024-05-17T22:34:18.241 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 17 22:34:17 smithi151 bash[67294]: b5eb2658cde1e034703198d8fbb5ec0cb6aff966ca3010b30eebbde55f689108 2024-05-17T22:34:18.241 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 17 22:34:17 smithi151 systemd[1]: Started Ceph mds.i for 4d593780-149c-11ef-bc9a-c7b262605968. 2024-05-17T22:34:19.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:19 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:19.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:19 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:19.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:19 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:19.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:19 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T22:34:19.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:19 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T22:34:19.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:19 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:34:19.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:19 smithi012 ceph-mon[31632]: Deploying daemon mds.g on smithi012 2024-05-17T22:34:19.093 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:19 smithi012 ceph-mon[31632]: mds.? [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] up:boot 2024-05-17T22:34:19.093 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:19 smithi012 ceph-mon[31632]: fsmap 9 up:standby 2024-05-17T22:34:19.093 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:19 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-17T22:34:19.093 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:19 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/1353841498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:34:19.093 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:19 smithi012 ceph-mon[31632]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:19.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:19 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:19.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:19 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:19.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:19 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:19.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:19 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T22:34:19.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:19 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T22:34:19.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:19 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:34:19.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:19 smithi096 ceph-mon[36065]: Deploying daemon mds.g on smithi012 2024-05-17T22:34:19.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:19 smithi096 ceph-mon[36065]: mds.? [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] up:boot 2024-05-17T22:34:19.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:19 smithi096 ceph-mon[36065]: fsmap 9 up:standby 2024-05-17T22:34:19.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:19 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-17T22:34:19.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:19 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/1353841498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:34:19.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:19 smithi096 ceph-mon[36065]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:19.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:19 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:19.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:19 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:19.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:19 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:19.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:19 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T22:34:19.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:19 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T22:34:19.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:19 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:34:19.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:19 smithi151 ceph-mon[36101]: Deploying daemon mds.g on smithi012 2024-05-17T22:34:19.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:19 smithi151 ceph-mon[36101]: mds.? [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] up:boot 2024-05-17T22:34:19.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:19 smithi151 ceph-mon[36101]: fsmap 9 up:standby 2024-05-17T22:34:19.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:19 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-17T22:34:19.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:19 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/1353841498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:34:19.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:19 smithi151 ceph-mon[36101]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:20.810 INFO:journalctl@ceph.mds.g.smithi012.stdout:May 17 22:34:20 smithi012 systemd[1]: Starting Ceph mds.g for 4d593780-149c-11ef-bc9a-c7b262605968... 2024-05-17T22:34:20.864 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T22:34:21.073 INFO:journalctl@ceph.mds.g.smithi012.stdout:May 17 22:34:20 smithi012 podman[72741]: 2024-05-17 22:34:20.710727478 +0000 UTC m=+0.018363823 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T22:34:21.073 INFO:journalctl@ceph.mds.g.smithi012.stdout:May 17 22:34:20 smithi012 podman[72741]: 2024-05-17 22:34:20.842589493 +0000 UTC m=+0.150225825 container create 54adbc3cf4cf59a67f5dffac4cf3083d805a69b47922568aa1d0fd59d3ed7455 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-g, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514) 2024-05-17T22:34:21.073 INFO:journalctl@ceph.mds.g.smithi012.stdout:May 17 22:34:21 smithi012 podman[72741]: 2024-05-17 22:34:21.070744438 +0000 UTC m=+0.378380778 container init 54adbc3cf4cf59a67f5dffac4cf3083d805a69b47922568aa1d0fd59d3ed7455 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-g, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-17T22:34:21.242 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:34:21.342 INFO:journalctl@ceph.mds.g.smithi012.stdout:May 17 22:34:21 smithi012 podman[72741]: 2024-05-17 22:34:21.073992029 +0000 UTC m=+0.381628370 container start 54adbc3cf4cf59a67f5dffac4cf3083d805a69b47922568aa1d0fd59d3ed7455 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-g, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-17T22:34:21.342 INFO:journalctl@ceph.mds.g.smithi012.stdout:May 17 22:34:21 smithi012 ceph-mds[72790]: -- 172.21.15.12:0/4165768392 <== mon.0 v2:172.21.15.12:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5620b1b79a40 con 0x5620b1bbfa80 2024-05-17T22:34:21.342 INFO:journalctl@ceph.mds.g.smithi012.stdout:May 17 22:34:21 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-g[72774]: starting mds.g at 2024-05-17T22:34:21.342 INFO:journalctl@ceph.mds.g.smithi012.stdout:May 17 22:34:21 smithi012 bash[72741]: 54adbc3cf4cf59a67f5dffac4cf3083d805a69b47922568aa1d0fd59d3ed7455 2024-05-17T22:34:21.342 INFO:journalctl@ceph.mds.g.smithi012.stdout:May 17 22:34:21 smithi012 systemd[1]: Started Ceph mds.g for 4d593780-149c-11ef-bc9a-c7b262605968. 2024-05-17T22:34:21.618 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:21 smithi012 ceph-mon[31632]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:21.618 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:21 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:21.619 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:21 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:21.619 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:21 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:21.619 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:21 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T22:34:21.619 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:21 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T22:34:21.619 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:21 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:34:21.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:21 smithi096 ceph-mon[36065]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:21.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:21 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:21.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:21 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:21.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:21 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:21.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:21 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T22:34:21.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:21 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T22:34:21.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:21 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:34:21.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:21 smithi151 ceph-mon[36101]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:21.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:21 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:21.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:21 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:21.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:21 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:21.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:21 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T22:34:21.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:21 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T22:34:21.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:21 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:34:22.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:22 smithi012 ceph-mon[31632]: Deploying daemon mds.a on smithi012 2024-05-17T22:34:22.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:22 smithi012 ceph-mon[31632]: mds.? [v2:172.21.15.12:6838/3731914742,v1:172.21.15.12:6839/3731914742] up:boot 2024-05-17T22:34:22.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:22 smithi012 ceph-mon[31632]: fsmap 10 up:standby 2024-05-17T22:34:22.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:22 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-17T22:34:22.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:22 smithi096 ceph-mon[36065]: Deploying daemon mds.a on smithi012 2024-05-17T22:34:22.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:22 smithi096 ceph-mon[36065]: mds.? [v2:172.21.15.12:6838/3731914742,v1:172.21.15.12:6839/3731914742] up:boot 2024-05-17T22:34:22.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:22 smithi096 ceph-mon[36065]: fsmap 10 up:standby 2024-05-17T22:34:22.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:22 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-17T22:34:22.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:22.923+0000 7f6085af0700 1 -- 172.21.15.12:0/3965491644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6080105770 msgr2=0x7f608010dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:22.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:22.923+0000 7f6085af0700 1 --2- 172.21.15.12:0/3965491644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6080105770 0x7f608010dcc0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6068009990 tx=0x7f6068009ca0 comp rx=0 tx=0).stop 2024-05-17T22:34:22.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:22.923+0000 7f6085af0700 1 -- 172.21.15.12:0/3965491644 shutdown_connections 2024-05-17T22:34:22.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:22.923+0000 7f6085af0700 1 --2- 172.21.15.12:0/3965491644 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f608010e250 0x7f6080110760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:22.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:22.923+0000 7f6085af0700 1 --2- 172.21.15.12:0/3965491644 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6080104e20 0x7f6080105230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:22.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:22.923+0000 7f6085af0700 1 --2- 172.21.15.12:0/3965491644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6080105770 0x7f608010dcc0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:22.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:22.923+0000 7f6085af0700 1 -- 172.21.15.12:0/3965491644 >> 172.21.15.12:0/3965491644 conn(0x7f6080079af0 msgr2=0x7f6080079f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:34:22.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:22.923+0000 7f6085af0700 1 -- 172.21.15.12:0/3965491644 shutdown_connections 2024-05-17T22:34:22.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:22.923+0000 7f6085af0700 1 -- 172.21.15.12:0/3965491644 wait complete. 2024-05-17T22:34:22.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:22.924+0000 7f6085af0700 1 Processor -- start 2024-05-17T22:34:22.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:22.924+0000 7f6085af0700 1 -- start start 2024-05-17T22:34:22.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:22.925+0000 7f6085af0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6080104e20 0x7f6080102b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:22.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:22.925+0000 7f6085af0700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6080105770 0x7f6080103050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:22.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:22.925+0000 7f6085af0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f608010e250 0x7f6080103590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:22.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:22.925+0000 7f6085af0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6080113ed0 con 0x7f608010e250 2024-05-17T22:34:22.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:22.925+0000 7f6085af0700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6080113d50 con 0x7f6080104e20 2024-05-17T22:34:22.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:22.925+0000 7f6085af0700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6080114050 con 0x7f6080105770 2024-05-17T22:34:22.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:22.925+0000 7f607ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f608010e250 0x7f6080103590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:22.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:22.925+0000 7f607ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f608010e250 0x7f6080103590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45840/0 (socket says 172.21.15.12:45840) 2024-05-17T22:34:22.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:22.925+0000 7f607ffff700 1 -- 172.21.15.12:0/2655678894 learned_addr learned my addr 172.21.15.12:0/2655678894 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:34:22.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:22.925+0000 7f607effd700 1 --2- 172.21.15.12:0/2655678894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6080105770 0x7f6080103050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:22.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:22.925+0000 7f607f7fe700 1 --2- 172.21.15.12:0/2655678894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6080104e20 0x7f6080102b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:22.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:22.925+0000 7f607ffff700 1 -- 172.21.15.12:0/2655678894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6080105770 msgr2=0x7f6080103050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:22.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:22.925+0000 7f607ffff700 1 --2- 172.21.15.12:0/2655678894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6080105770 0x7f6080103050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:22.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:22.925+0000 7f607ffff700 1 -- 172.21.15.12:0/2655678894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6080104e20 msgr2=0x7f6080102b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:22.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:22.925+0000 7f607ffff700 1 --2- 172.21.15.12:0/2655678894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6080104e20 0x7f6080102b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:22.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:22.925+0000 7f607ffff700 1 -- 172.21.15.12:0/2655678894 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60801a8cb0 con 0x7f608010e250 2024-05-17T22:34:22.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:22.926+0000 7f607ffff700 1 --2- 172.21.15.12:0/2655678894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f608010e250 0x7f6080103590 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f607800b550 tx=0x7f607800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:34:22.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:22.926+0000 7f607cff9700 1 -- 172.21.15.12:0/2655678894 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6078015020 con 0x7f608010e250 2024-05-17T22:34:22.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:22.926+0000 7f607cff9700 1 -- 172.21.15.12:0/2655678894 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6078011ce0 con 0x7f608010e250 2024-05-17T22:34:22.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:22.926+0000 7f6085af0700 1 -- 172.21.15.12:0/2655678894 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60801a8fa0 con 0x7f608010e250 2024-05-17T22:34:22.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:22.926+0000 7f607cff9700 1 -- 172.21.15.12:0/2655678894 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6078010c90 con 0x7f608010e250 2024-05-17T22:34:22.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:22.926+0000 7f6085af0700 1 -- 172.21.15.12:0/2655678894 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60801a93f0 con 0x7f608010e250 2024-05-17T22:34:22.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:22.927+0000 7f6085af0700 1 -- 172.21.15.12:0/2655678894 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60801052b0 con 0x7f608010e250 2024-05-17T22:34:22.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:22.929+0000 7f607cff9700 1 -- 172.21.15.12:0/2655678894 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f6078018020 con 0x7f608010e250 2024-05-17T22:34:22.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:22.930+0000 7f607cff9700 1 --2- 172.21.15.12:0/2655678894 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f606c0b2880 0x7f606c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:22.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:22.930+0000 7f607f7fe700 1 --2- 172.21.15.12:0/2655678894 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f606c0b2880 0x7f606c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:22.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:22.930+0000 7f607cff9700 1 -- 172.21.15.12:0/2655678894 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f60780e1950 con 0x7f608010e250 2024-05-17T22:34:22.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:22.930+0000 7f607f7fe700 1 --2- 172.21.15.12:0/2655678894 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f606c0b2880 0x7f606c0b4d90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f607000a400 tx=0x7f607000d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:34:22.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:22.933+0000 7f607cff9700 1 -- 172.21.15.12:0/2655678894 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f607801f020 con 0x7f608010e250 2024-05-17T22:34:22.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:22 smithi151 ceph-mon[36101]: Deploying daemon mds.a on smithi012 2024-05-17T22:34:22.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:22 smithi151 ceph-mon[36101]: mds.? [v2:172.21.15.12:6838/3731914742,v1:172.21.15.12:6839/3731914742] up:boot 2024-05-17T22:34:22.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:22 smithi151 ceph-mon[36101]: fsmap 10 up:standby 2024-05-17T22:34:22.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:22 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-17T22:34:23.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:23.200+0000 7f6085af0700 1 -- 172.21.15.12:0/2655678894 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6080064440 con 0x7f608010e250 2024-05-17T22:34:23.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:23.201+0000 7f607cff9700 1 -- 172.21.15.12:0/2655678894 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8827 (secure 0 0 0) 0x7f60780a9b10 con 0x7f608010e250 2024-05-17T22:34:23.203 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:34:23.204 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1407219319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1407219319},{"type":"v1","addr":"172.21.15.12:6835","nonce":1407219319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/630977886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":630977886},{"type":"v1","addr":"172.21.15.12:6837","nonce":630977886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3731914742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3731914742},{"type":"v1","addr":"172.21.15.12:6839","nonce":3731914742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3554053059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3554053059},{"type":"v1","addr":"172.21.15.151:6833","nonce":3554053059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/4228325569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":4228325569},{"type":"v1","addr":"172.21.15.96:6833","nonce":4228325569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/542323754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":542323754},{"type":"v1","addr":"172.21.15.151:6835","nonce":542323754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1758442232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1758442232},{"type":"v1","addr":"172.21.15.96:6835","nonce":1758442232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/3403324038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3403324038},{"type":"v1","addr":"172.21.15.151:6837","nonce":3403324038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/2338353849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2338353849},{"type":"v1","addr":"172.21.15.151:6839","nonce":2338353849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/4222574338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":4222574338},{"type":"v1","addr":"172.21.15.96:6837","nonce":4222574338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-17T22:34:23.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:23.203+0000 7f6085af0700 1 -- 172.21.15.12:0/2655678894 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f606c0b2880 msgr2=0x7f606c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:23.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:23.203+0000 7f6085af0700 1 --2- 172.21.15.12:0/2655678894 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f606c0b2880 0x7f606c0b4d90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f607000a400 tx=0x7f607000d040 comp rx=0 tx=0).stop 2024-05-17T22:34:23.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:23.203+0000 7f6085af0700 1 -- 172.21.15.12:0/2655678894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f608010e250 msgr2=0x7f6080103590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:23.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:23.203+0000 7f6085af0700 1 --2- 172.21.15.12:0/2655678894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f608010e250 0x7f6080103590 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f607800b550 tx=0x7f607800b860 comp rx=0 tx=0).stop 2024-05-17T22:34:23.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:23.203+0000 7f6085af0700 1 -- 172.21.15.12:0/2655678894 shutdown_connections 2024-05-17T22:34:23.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:23.203+0000 7f6085af0700 1 --2- 172.21.15.12:0/2655678894 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f606c0b2880 0x7f606c0b4d90 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:23.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:23.203+0000 7f6085af0700 1 --2- 172.21.15.12:0/2655678894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f608010e250 0x7f6080103590 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:23.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:23.203+0000 7f6085af0700 1 --2- 172.21.15.12:0/2655678894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6080104e20 0x7f6080102b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:23.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:23.203+0000 7f6085af0700 1 --2- 172.21.15.12:0/2655678894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6080105770 0x7f6080103050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:23.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:23.203+0000 7f6085af0700 1 -- 172.21.15.12:0/2655678894 >> 172.21.15.12:0/2655678894 conn(0x7f6080079af0 msgr2=0x7f60800ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:34:23.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:23.203+0000 7f6085af0700 1 -- 172.21.15.12:0/2655678894 shutdown_connections 2024-05-17T22:34:23.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:23.203+0000 7f6085af0700 1 -- 172.21.15.12:0/2655678894 wait complete. 2024-05-17T22:34:23.207 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 11 2024-05-17T22:34:23.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:23 smithi096 ceph-mon[36065]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:23.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:23 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/2655678894' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:34:23.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:23 smithi012 ceph-mon[31632]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:23.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:23 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/2655678894' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:34:23.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:23 smithi151 ceph-mon[36101]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:23.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:23 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/2655678894' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:34:25.546 INFO:journalctl@ceph.mds.a.smithi012.stdout:May 17 22:34:25 smithi012 systemd[1]: Started Ceph mds.a for 4d593780-149c-11ef-bc9a-c7b262605968. 2024-05-17T22:34:25.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:25 smithi096 ceph-mon[36065]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:25.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:25 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:25.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:25 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:25.800 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:25 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:25.800 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:25 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T22:34:25.800 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:25 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T22:34:25.800 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:25 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:34:25.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:25 smithi012 ceph-mon[31632]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:25.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:25 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:25.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:25 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:25.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:25 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:25.843 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:25 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T22:34:25.843 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:25 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T22:34:25.843 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:25 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:34:25.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:25 smithi151 ceph-mon[36101]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:25.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:25 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:25.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:25 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:25.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:25 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:25.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:25 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T22:34:25.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:25 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T22:34:25.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:25 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:34:26.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:26 smithi096 ceph-mon[36065]: Deploying daemon mds.b on smithi096 2024-05-17T22:34:26.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:26 smithi096 ceph-mon[36065]: mds.? [v2:172.21.15.12:6840/3642142501,v1:172.21.15.12:6841/3642142501] up:boot 2024-05-17T22:34:26.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:26 smithi096 ceph-mon[36065]: fsmap 11 up:standby 2024-05-17T22:34:26.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:26 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-17T22:34:26.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:26 smithi012 ceph-mon[31632]: Deploying daemon mds.b on smithi096 2024-05-17T22:34:26.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:26 smithi012 ceph-mon[31632]: mds.? [v2:172.21.15.12:6840/3642142501,v1:172.21.15.12:6841/3642142501] up:boot 2024-05-17T22:34:26.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:26 smithi012 ceph-mon[31632]: fsmap 11 up:standby 2024-05-17T22:34:26.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:26 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-17T22:34:26.851 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T22:34:26.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:26 smithi151 ceph-mon[36101]: Deploying daemon mds.b on smithi096 2024-05-17T22:34:26.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:26 smithi151 ceph-mon[36101]: mds.? [v2:172.21.15.12:6840/3642142501,v1:172.21.15.12:6841/3642142501] up:boot 2024-05-17T22:34:26.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:26 smithi151 ceph-mon[36101]: fsmap 11 up:standby 2024-05-17T22:34:26.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:26 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-17T22:34:27.120 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:34:27.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:27 smithi096 ceph-mon[36065]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:27.799 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 17 22:34:27 smithi096 systemd[1]: Starting Ceph mds.b for 4d593780-149c-11ef-bc9a-c7b262605968... 2024-05-17T22:34:27.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:27 smithi012 ceph-mon[31632]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:27.892 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:27 smithi151 ceph-mon[36101]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:28.094 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 17 22:34:27 smithi096 podman[66475]: 2024-05-17 22:34:27.733652384 +0000 UTC m=+0.017015931 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T22:34:28.094 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 17 22:34:27 smithi096 podman[66475]: 2024-05-17 22:34:27.847085257 +0000 UTC m=+0.130448826 container create 78c4d4ccf405f61b52db9c52150f79308ae0ba7eeedda418197bf292292f2190 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-b, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-17T22:34:28.094 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 17 22:34:28 smithi096 podman[66475]: 2024-05-17 22:34:28.091755087 +0000 UTC m=+0.375118634 container init 78c4d4ccf405f61b52db9c52150f79308ae0ba7eeedda418197bf292292f2190 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-17T22:34:28.549 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 17 22:34:28 smithi096 podman[66475]: 2024-05-17 22:34:28.094386223 +0000 UTC m=+0.377749770 container start 78c4d4ccf405f61b52db9c52150f79308ae0ba7eeedda418197bf292292f2190 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-17T22:34:28.549 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 17 22:34:28 smithi096 ceph-mds[66494]: -- 172.21.15.96:0/418226890 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e97f64fa40 con 0x55e97f695a80 2024-05-17T22:34:28.549 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 17 22:34:28 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-b[66490]: starting mds.b at 2024-05-17T22:34:28.549 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 17 22:34:28 smithi096 bash[66475]: 78c4d4ccf405f61b52db9c52150f79308ae0ba7eeedda418197bf292292f2190 2024-05-17T22:34:28.549 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 17 22:34:28 smithi096 systemd[1]: Started Ceph mds.b for 4d593780-149c-11ef-bc9a-c7b262605968. 2024-05-17T22:34:28.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:28.850+0000 7fb96bdaa700 1 -- 172.21.15.12:0/2588151772 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb964107750 msgr2=0x7fb964107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:28.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:28.850+0000 7fb96bdaa700 1 --2- 172.21.15.12:0/2588151772 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb964107750 0x7fb964107b60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb958009990 tx=0x7fb958009ca0 comp rx=0 tx=0).stop 2024-05-17T22:34:28.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:28.850+0000 7fb96bdaa700 1 -- 172.21.15.12:0/2588151772 shutdown_connections 2024-05-17T22:34:28.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:28.850+0000 7fb96bdaa700 1 --2- 172.21.15.12:0/2588151772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9641080a0 0x7fb96410fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:28.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:28.850+0000 7fb96bdaa700 1 --2- 172.21.15.12:0/2588151772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb964102e70 0x7fb964103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:28.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:28.850+0000 7fb96bdaa700 1 --2- 172.21.15.12:0/2588151772 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb964107750 0x7fb964107b60 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:28.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:28.850+0000 7fb96bdaa700 1 -- 172.21.15.12:0/2588151772 >> 172.21.15.12:0/2588151772 conn(0x7fb964079ae0 msgr2=0x7fb964079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:34:28.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:28.850+0000 7fb96bdaa700 1 -- 172.21.15.12:0/2588151772 shutdown_connections 2024-05-17T22:34:28.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:28.850+0000 7fb96bdaa700 1 -- 172.21.15.12:0/2588151772 wait complete. 2024-05-17T22:34:28.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:28.851+0000 7fb96bdaa700 1 Processor -- start 2024-05-17T22:34:28.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:28.851+0000 7fb96bdaa700 1 -- start start 2024-05-17T22:34:28.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:28.852+0000 7fb96bdaa700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb964102e70 0x7fb964105210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:28.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:28.852+0000 7fb96bdaa700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb964107750 0x7fb964105750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:28.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:28.852+0000 7fb96bdaa700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9641080a0 0x7fb964105c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:28.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:28.852+0000 7fb96bdaa700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb964112cb0 con 0x7fb964107750 2024-05-17T22:34:28.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:28.852+0000 7fb96bdaa700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb964112b30 con 0x7fb964102e70 2024-05-17T22:34:28.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:28.852+0000 7fb96bdaa700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb964112e30 con 0x7fb9641080a0 2024-05-17T22:34:28.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:28.852+0000 7fb969345700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb964107750 0x7fb964105750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:28.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:28.852+0000 7fb969345700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb964107750 0x7fb964105750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45878/0 (socket says 172.21.15.12:45878) 2024-05-17T22:34:28.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:28.852+0000 7fb969345700 1 -- 172.21.15.12:0/2802230402 learned_addr learned my addr 172.21.15.12:0/2802230402 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:34:28.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:28.852+0000 7fb969b46700 1 --2- 172.21.15.12:0/2802230402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb964102e70 0x7fb964105210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:28.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:28.852+0000 7fb96a347700 1 --2- 172.21.15.12:0/2802230402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9641080a0 0x7fb964105c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:28.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:28.852+0000 7fb969345700 1 -- 172.21.15.12:0/2802230402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9641080a0 msgr2=0x7fb964105c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:28.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:28.852+0000 7fb969345700 1 --2- 172.21.15.12:0/2802230402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9641080a0 0x7fb964105c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:28.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:28.852+0000 7fb969345700 1 -- 172.21.15.12:0/2802230402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb964102e70 msgr2=0x7fb964105210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:28.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:28.852+0000 7fb969345700 1 --2- 172.21.15.12:0/2802230402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb964102e70 0x7fb964105210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:28.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:28.852+0000 7fb969345700 1 -- 172.21.15.12:0/2802230402 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9641a8c00 con 0x7fb964107750 2024-05-17T22:34:28.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:28.852+0000 7fb969345700 1 --2- 172.21.15.12:0/2802230402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb964107750 0x7fb964105750 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb958000c00 tx=0x7fb95800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:34:28.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:28.853+0000 7fb956ffd700 1 -- 172.21.15.12:0/2802230402 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb958027070 con 0x7fb964107750 2024-05-17T22:34:28.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:28.853+0000 7fb96bdaa700 1 -- 172.21.15.12:0/2802230402 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9641a8e90 con 0x7fb964107750 2024-05-17T22:34:28.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:28.853+0000 7fb96bdaa700 1 -- 172.21.15.12:0/2802230402 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9641a9290 con 0x7fb964107750 2024-05-17T22:34:28.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:28.853+0000 7fb956ffd700 1 -- 172.21.15.12:0/2802230402 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb958022470 con 0x7fb964107750 2024-05-17T22:34:28.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:28.853+0000 7fb956ffd700 1 -- 172.21.15.12:0/2802230402 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb958017650 con 0x7fb964107750 2024-05-17T22:34:28.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:28.854+0000 7fb96bdaa700 1 -- 172.21.15.12:0/2802230402 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb948005330 con 0x7fb964107750 2024-05-17T22:34:28.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:28.857+0000 7fb956ffd700 1 -- 172.21.15.12:0/2802230402 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fb95802a070 con 0x7fb964107750 2024-05-17T22:34:28.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:28.857+0000 7fb956ffd700 1 --2- 172.21.15.12:0/2802230402 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb9500b2900 0x7fb9500b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:28.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:28.857+0000 7fb969b46700 1 --2- 172.21.15.12:0/2802230402 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb9500b2900 0x7fb9500b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:28.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:28.857+0000 7fb956ffd700 1 -- 172.21.15.12:0/2802230402 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fb9580e3060 con 0x7fb964107750 2024-05-17T22:34:28.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:28.858+0000 7fb969b46700 1 --2- 172.21.15.12:0/2802230402 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb9500b2900 0x7fb9500b4e10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb960009c60 tx=0x7fb960009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:34:28.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:28.861+0000 7fb956ffd700 1 -- 172.21.15.12:0/2802230402 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb958031070 con 0x7fb964107750 2024-05-17T22:34:29.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:29.142+0000 7fb96bdaa700 1 -- 172.21.15.12:0/2802230402 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb948005a40 con 0x7fb964107750 2024-05-17T22:34:29.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:29.143+0000 7fb956ffd700 1 -- 172.21.15.12:0/2802230402 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10479 (secure 0 0 0) 0x7fb958013070 con 0x7fb964107750 2024-05-17T22:34:29.146 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:34:29.146 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1407219319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1407219319},{"type":"v1","addr":"172.21.15.12:6835","nonce":1407219319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/630977886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":630977886},{"type":"v1","addr":"172.21.15.12:6837","nonce":630977886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3731914742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3731914742},{"type":"v1","addr":"172.21.15.12:6839","nonce":3731914742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/3642142501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3642142501},{"type":"v1","addr":"172.21.15.12:6841","nonce":3642142501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3554053059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3554053059},{"type":"v1","addr":"172.21.15.151:6833","nonce":3554053059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/4228325569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":4228325569},{"type":"v1","addr":"172.21.15.96:6833","nonce":4228325569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/542323754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":542323754},{"type":"v1","addr":"172.21.15.151:6835","nonce":542323754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1758442232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1758442232},{"type":"v1","addr":"172.21.15.96:6835","nonce":1758442232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/3403324038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3403324038},{"type":"v1","addr":"172.21.15.151:6837","nonce":3403324038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/2338353849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2338353849},{"type":"v1","addr":"172.21.15.151:6839","nonce":2338353849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/4222574338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":4222574338},{"type":"v1","addr":"172.21.15.96:6837","nonce":4222574338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24482,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/2986295519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2986295519},{"type":"v1","addr":"172.21.15.96:6839","nonce":2986295519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-05-17T22:34:29.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:29.145+0000 7fb96bdaa700 1 -- 172.21.15.12:0/2802230402 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb9500b2900 msgr2=0x7fb9500b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:29.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:29.145+0000 7fb96bdaa700 1 --2- 172.21.15.12:0/2802230402 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb9500b2900 0x7fb9500b4e10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb960009c60 tx=0x7fb960009260 comp rx=0 tx=0).stop 2024-05-17T22:34:29.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:29.145+0000 7fb96bdaa700 1 -- 172.21.15.12:0/2802230402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb964107750 msgr2=0x7fb964105750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:29.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:29.145+0000 7fb96bdaa700 1 --2- 172.21.15.12:0/2802230402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb964107750 0x7fb964105750 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb958000c00 tx=0x7fb95800bfd0 comp rx=0 tx=0).stop 2024-05-17T22:34:29.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:29.145+0000 7fb96bdaa700 1 -- 172.21.15.12:0/2802230402 shutdown_connections 2024-05-17T22:34:29.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:29.145+0000 7fb96bdaa700 1 --2- 172.21.15.12:0/2802230402 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb9500b2900 0x7fb9500b4e10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:29.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:29.145+0000 7fb96bdaa700 1 --2- 172.21.15.12:0/2802230402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9641080a0 0x7fb964105c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:29.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:29.145+0000 7fb96bdaa700 1 --2- 172.21.15.12:0/2802230402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb964102e70 0x7fb964105210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:29.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:29.145+0000 7fb96bdaa700 1 --2- 172.21.15.12:0/2802230402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb964107750 0x7fb964105750 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:29.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:29.145+0000 7fb96bdaa700 1 -- 172.21.15.12:0/2802230402 >> 172.21.15.12:0/2802230402 conn(0x7fb964079ae0 msgr2=0x7fb96406c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:34:29.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:29.146+0000 7fb96bdaa700 1 -- 172.21.15.12:0/2802230402 shutdown_connections 2024-05-17T22:34:29.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:29.146+0000 7fb96bdaa700 1 -- 172.21.15.12:0/2802230402 wait complete. 2024-05-17T22:34:29.150 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 13 2024-05-17T22:34:29.447 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:29 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:29.447 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:29 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:29.447 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:29 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:29.447 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:29 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:29.447 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:29 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:29.447 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:29 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:34:29.447 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:29 smithi151 ceph-mon[36101]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:29.447 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:29 smithi151 ceph-mon[36101]: mds.? [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] up:boot 2024-05-17T22:34:29.447 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:29 smithi151 ceph-mon[36101]: fsmap 12 up:standby 2024-05-17T22:34:29.447 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:29 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-17T22:34:29.447 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:29 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/2802230402' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:34:29.459 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:29 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:29.459 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:29 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:29.459 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:29 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:29.459 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:29 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:29.459 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:29 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:29.459 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:29 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:34:29.459 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:29 smithi012 ceph-mon[31632]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:29.459 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:29 smithi012 ceph-mon[31632]: mds.? [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] up:boot 2024-05-17T22:34:29.459 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:29 smithi012 ceph-mon[31632]: fsmap 12 up:standby 2024-05-17T22:34:29.459 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:29 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-17T22:34:29.459 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:29 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/2802230402' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:34:29.480 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:29 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:29.480 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:29 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:29.480 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:29 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:29.481 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:29 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:29.481 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:29 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:29.481 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:29 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:34:29.481 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:29 smithi096 ceph-mon[36065]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:29.481 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:29 smithi096 ceph-mon[36065]: mds.? [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] up:boot 2024-05-17T22:34:29.481 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:29 smithi096 ceph-mon[36065]: fsmap 12 up:standby 2024-05-17T22:34:29.481 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:29 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-17T22:34:29.481 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:29 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/2802230402' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:34:30.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:30 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:30.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:30 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:30.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:30 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:30.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:30 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:30.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:30 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:30.592 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-05-17T22:34:30.725 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:30 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:30.725 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:30 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:30.725 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:30 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:30.726 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:30 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:30.726 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:30 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:30.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:30 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:30.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:30 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:30.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:30 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:30.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:30 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:30.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:30 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:30.864 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:34:31.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:31 smithi012 ceph-mon[31632]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:31.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:31 smithi151 ceph-mon[36101]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:31.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:31 smithi096 ceph-mon[36065]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:32.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.549+0000 7f657b905700 1 -- 172.21.15.12:0/1834880220 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6574108080 msgr2=0x7f657410f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:32.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.549+0000 7f657b905700 1 --2- 172.21.15.12:0/1834880220 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6574108080 0x7f657410f9e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f656c00b0a0 tx=0x7f656c00b3b0 comp rx=0 tx=0).stop 2024-05-17T22:34:32.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.549+0000 7f657b905700 1 -- 172.21.15.12:0/1834880220 shutdown_connections 2024-05-17T22:34:32.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.549+0000 7f657b905700 1 --2- 172.21.15.12:0/1834880220 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6574108080 0x7f657410f9e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:32.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.549+0000 7f657b905700 1 --2- 172.21.15.12:0/1834880220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6574102e00 0x7f6574103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:32.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.549+0000 7f657b905700 1 --2- 172.21.15.12:0/1834880220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6574107730 0x7f6574107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:32.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.549+0000 7f657b905700 1 -- 172.21.15.12:0/1834880220 >> 172.21.15.12:0/1834880220 conn(0x7f6574079bd0 msgr2=0x7f6574079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:34:32.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.550+0000 7f657b905700 1 -- 172.21.15.12:0/1834880220 shutdown_connections 2024-05-17T22:34:32.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.550+0000 7f657b905700 1 -- 172.21.15.12:0/1834880220 wait complete. 2024-05-17T22:34:32.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.550+0000 7f657b905700 1 Processor -- start 2024-05-17T22:34:32.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.550+0000 7f657b905700 1 -- start start 2024-05-17T22:34:32.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.551+0000 7f657b905700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6574102e00 0x7f657419b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:32.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.551+0000 7f657b905700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6574107730 0x7f657419bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:32.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.551+0000 7f657b905700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6574108080 0x7f657419c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:32.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.551+0000 7f657b905700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6574112db0 con 0x7f6574107730 2024-05-17T22:34:32.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.551+0000 7f657b905700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6574112c30 con 0x7f6574102e00 2024-05-17T22:34:32.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.551+0000 7f657b905700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6574112f30 con 0x7f6574108080 2024-05-17T22:34:32.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.551+0000 7f6578ea0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6574107730 0x7f657419bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:32.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.551+0000 7f6578ea0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6574107730 0x7f657419bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44244/0 (socket says 172.21.15.12:44244) 2024-05-17T22:34:32.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.551+0000 7f6578ea0700 1 -- 172.21.15.12:0/1559285341 learned_addr learned my addr 172.21.15.12:0/1559285341 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:34:32.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.551+0000 7f65796a1700 1 --2- 172.21.15.12:0/1559285341 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6574102e00 0x7f657419b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:32.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.551+0000 7f6579ea2700 1 --2- 172.21.15.12:0/1559285341 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6574108080 0x7f657419c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:32.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.551+0000 7f6578ea0700 1 -- 172.21.15.12:0/1559285341 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6574108080 msgr2=0x7f657419c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:32.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.551+0000 7f6578ea0700 1 --2- 172.21.15.12:0/1559285341 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6574108080 0x7f657419c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:32.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.551+0000 7f6578ea0700 1 -- 172.21.15.12:0/1559285341 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6574102e00 msgr2=0x7f657419b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:32.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.551+0000 7f6578ea0700 1 --2- 172.21.15.12:0/1559285341 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6574102e00 0x7f657419b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:32.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.551+0000 7f6578ea0700 1 -- 172.21.15.12:0/1559285341 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65741a6ad0 con 0x7f6574107730 2024-05-17T22:34:32.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.551+0000 7f6579ea2700 1 --2- 172.21.15.12:0/1559285341 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6574108080 0x7f657419c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:34:32.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.551+0000 7f6578ea0700 1 --2- 172.21.15.12:0/1559285341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6574107730 0x7f657419bcc0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f656400b500 tx=0x7f656400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:34:32.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.552+0000 7f656a7fc700 1 -- 172.21.15.12:0/1559285341 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6564015020 con 0x7f6574107730 2024-05-17T22:34:32.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.552+0000 7f656a7fc700 1 -- 172.21.15.12:0/1559285341 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6564011ce0 con 0x7f6574107730 2024-05-17T22:34:32.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.552+0000 7f657b905700 1 -- 172.21.15.12:0/1559285341 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65741a6dc0 con 0x7f6574107730 2024-05-17T22:34:32.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.552+0000 7f656a7fc700 1 -- 172.21.15.12:0/1559285341 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6564010d40 con 0x7f6574107730 2024-05-17T22:34:32.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.552+0000 7f657b905700 1 -- 172.21.15.12:0/1559285341 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65741a7270 con 0x7f6574107730 2024-05-17T22:34:32.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.553+0000 7f657b905700 1 -- 172.21.15.12:0/1559285341 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f657410b110 con 0x7f6574107730 2024-05-17T22:34:32.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.554+0000 7f656a7fc700 1 -- 172.21.15.12:0/1559285341 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f6564018020 con 0x7f6574107730 2024-05-17T22:34:32.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.555+0000 7f656a7fc700 1 --2- 172.21.15.12:0/1559285341 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f65600b2880 0x7f65600b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:32.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.555+0000 7f65796a1700 1 --2- 172.21.15.12:0/1559285341 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f65600b2880 0x7f65600b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:32.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.555+0000 7f656a7fc700 1 -- 172.21.15.12:0/1559285341 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f65640e14b0 con 0x7f6574107730 2024-05-17T22:34:32.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.556+0000 7f65796a1700 1 --2- 172.21.15.12:0/1559285341 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f65600b2880 0x7f65600b4d90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6570009a10 tx=0x7f6570009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:34:32.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.560+0000 7f656a7fc700 1 -- 172.21.15.12:0/1559285341 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f656401f020 con 0x7f6574107730 2024-05-17T22:34:32.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.778+0000 7f657b905700 1 -- 172.21.15.12:0/1559285341 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f657406bc70 con 0x7f6574107730 2024-05-17T22:34:32.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.779+0000 7f656a7fc700 1 -- 172.21.15.12:0/1559285341 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v71) ==== 84+0+65 (secure 0 0 0) 0x7f65640a9670 con 0x7f6574107730 2024-05-17T22:34:32.782 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:34:32.782 INFO:teuthology.orchestra.run.smithi012.stdout:[ 2024-05-17T22:34:32.782 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:34:32.782 INFO:teuthology.orchestra.run.smithi012.stdout: "poolnum": 1, 2024-05-17T22:34:32.782 INFO:teuthology.orchestra.run.smithi012.stdout: "poolname": ".mgr" 2024-05-17T22:34:32.782 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:34:32.783 INFO:teuthology.orchestra.run.smithi012.stdout:] 2024-05-17T22:34:32.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.781+0000 7f657b905700 1 -- 172.21.15.12:0/1559285341 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f65600b2880 msgr2=0x7f65600b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:32.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.781+0000 7f657b905700 1 --2- 172.21.15.12:0/1559285341 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f65600b2880 0x7f65600b4d90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6570009a10 tx=0x7f6570009190 comp rx=0 tx=0).stop 2024-05-17T22:34:32.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.781+0000 7f657b905700 1 -- 172.21.15.12:0/1559285341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6574107730 msgr2=0x7f657419bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:32.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.781+0000 7f657b905700 1 --2- 172.21.15.12:0/1559285341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6574107730 0x7f657419bcc0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f656400b500 tx=0x7f656400b810 comp rx=0 tx=0).stop 2024-05-17T22:34:32.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.781+0000 7f657b905700 1 -- 172.21.15.12:0/1559285341 shutdown_connections 2024-05-17T22:34:32.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.781+0000 7f657b905700 1 --2- 172.21.15.12:0/1559285341 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f65600b2880 0x7f65600b4d90 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:32.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.781+0000 7f657b905700 1 --2- 172.21.15.12:0/1559285341 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6574108080 0x7f657419c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:32.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.781+0000 7f657b905700 1 --2- 172.21.15.12:0/1559285341 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6574102e00 0x7f657419b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:32.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.781+0000 7f657b905700 1 --2- 172.21.15.12:0/1559285341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6574107730 0x7f657419bcc0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:32.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.781+0000 7f657b905700 1 -- 172.21.15.12:0/1559285341 >> 172.21.15.12:0/1559285341 conn(0x7f6574079bd0 msgr2=0x7f6574105870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:34:32.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.781+0000 7f657b905700 1 -- 172.21.15.12:0/1559285341 shutdown_connections 2024-05-17T22:34:32.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:32.781+0000 7f657b905700 1 -- 172.21.15.12:0/1559285341 wait complete. 2024-05-17T22:34:33.401 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-17T22:34:33.401 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-17T22:34:33.662 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:34:33.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:33 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:33.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:33 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:33.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:33 smithi151 ceph-mon[36101]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:33.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:33 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/1559285341' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-17T22:34:33.786 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:33 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:33.786 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:33 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:33.786 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:33 smithi012 ceph-mon[31632]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:33.786 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:33 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/1559285341' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-17T22:34:33.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:33 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:33.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:33 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:33.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:33 smithi096 ceph-mon[36065]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:33.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:33 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/1559285341' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-17T22:34:34.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:34 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:34:34.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:34 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:34:34.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:34 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:34.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:34 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:34:34.694 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:34 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:34:34.694 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:34 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:34:34.694 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:34 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:34.694 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:34 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:34:34.722 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:34 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:34:34.722 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:34 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:34:34.722 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:34 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:34.722 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:34 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:34:35.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:35.264+0000 7fe0fa0a3700 1 -- 172.21.15.12:0/812904488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0f410e220 msgr2=0x7fe0f4110730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:35.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:35.264+0000 7fe0fa0a3700 1 --2- 172.21.15.12:0/812904488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0f410e220 0x7fe0f4110730 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec009990 tx=0x7fe0ec009ca0 comp rx=0 tx=0).stop 2024-05-17T22:34:35.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:35.264+0000 7fe0fa0a3700 1 -- 172.21.15.12:0/812904488 shutdown_connections 2024-05-17T22:34:35.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:35.264+0000 7fe0fa0a3700 1 --2- 172.21.15.12:0/812904488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0f410e220 0x7fe0f4110730 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:35.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:35.264+0000 7fe0fa0a3700 1 --2- 172.21.15.12:0/812904488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0f4104dc0 0x7fe0f41051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:35.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:35.264+0000 7fe0fa0a3700 1 --2- 172.21.15.12:0/812904488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0f4105710 0x7fe0f410dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:35.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:35.264+0000 7fe0fa0a3700 1 -- 172.21.15.12:0/812904488 >> 172.21.15.12:0/812904488 conn(0x7fe0f4079bd0 msgr2=0x7fe0f4079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:34:35.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:35.264+0000 7fe0fa0a3700 1 -- 172.21.15.12:0/812904488 shutdown_connections 2024-05-17T22:34:35.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:35.264+0000 7fe0fa0a3700 1 -- 172.21.15.12:0/812904488 wait complete. 2024-05-17T22:34:35.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:35.265+0000 7fe0fa0a3700 1 Processor -- start 2024-05-17T22:34:35.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:35.265+0000 7fe0fa0a3700 1 -- start start 2024-05-17T22:34:35.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:35.265+0000 7fe0fa0a3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0f4104dc0 0x7fe0f419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:35.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:35.265+0000 7fe0fa0a3700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0f4105710 0x7fe0f419acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:35.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:35.265+0000 7fe0fa0a3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0f410e220 0x7fe0f419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:35.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:35.265+0000 7fe0fa0a3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f4113da0 con 0x7fe0f4104dc0 2024-05-17T22:34:35.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:35.265+0000 7fe0fa0a3700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f4113c20 con 0x7fe0f410e220 2024-05-17T22:34:35.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:35.265+0000 7fe0fa0a3700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f4113f20 con 0x7fe0f4105710 2024-05-17T22:34:35.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:35.265+0000 7fe0f37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0f4104dc0 0x7fe0f419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:35.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:35.265+0000 7fe0f37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0f4104dc0 0x7fe0f419a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44264/0 (socket says 172.21.15.12:44264) 2024-05-17T22:34:35.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:35.265+0000 7fe0f37fe700 1 -- 172.21.15.12:0/3797997979 learned_addr learned my addr 172.21.15.12:0/3797997979 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:34:35.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:35.265+0000 7fe0f2ffd700 1 --2- 172.21.15.12:0/3797997979 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0f4105710 0x7fe0f419acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:35.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:35.265+0000 7fe0f3fff700 1 --2- 172.21.15.12:0/3797997979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0f410e220 0x7fe0f419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:35.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:35.266+0000 7fe0f2ffd700 1 -- 172.21.15.12:0/3797997979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0f410e220 msgr2=0x7fe0f419ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:35.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:35.266+0000 7fe0f2ffd700 1 --2- 172.21.15.12:0/3797997979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0f410e220 0x7fe0f419ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:35.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:35.266+0000 7fe0f2ffd700 1 -- 172.21.15.12:0/3797997979 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0f4104dc0 msgr2=0x7fe0f419a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:35.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:35.266+0000 7fe0f2ffd700 1 --2- 172.21.15.12:0/3797997979 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0f4104dc0 0x7fe0f419a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:35.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:35.266+0000 7fe0f2ffd700 1 -- 172.21.15.12:0/3797997979 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0f419f5f0 con 0x7fe0f4105710 2024-05-17T22:34:35.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:35.266+0000 7fe0f37fe700 1 --2- 172.21.15.12:0/3797997979 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0f4104dc0 0x7fe0f419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:34:35.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:35.266+0000 7fe0f2ffd700 1 --2- 172.21.15.12:0/3797997979 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0f4105710 0x7fe0f419acc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe0e800e9f0 tx=0x7fe0e800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:34:35.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:35.266+0000 7fe0f0ff9700 1 -- 172.21.15.12:0/3797997979 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e800c9c0 con 0x7fe0f4105710 2024-05-17T22:34:35.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:35.266+0000 7fe0f0ff9700 1 -- 172.21.15.12:0/3797997979 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0e800cb60 con 0x7fe0f4105710 2024-05-17T22:34:35.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:35.266+0000 7fe0fa0a3700 1 -- 172.21.15.12:0/3797997979 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0f419f8e0 con 0x7fe0f4105710 2024-05-17T22:34:35.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:35.266+0000 7fe0f0ff9700 1 -- 172.21.15.12:0/3797997979 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e8018680 con 0x7fe0f4105710 2024-05-17T22:34:35.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:35.266+0000 7fe0fa0a3700 1 -- 172.21.15.12:0/3797997979 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0f41a7200 con 0x7fe0f4105710 2024-05-17T22:34:35.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:35.267+0000 7fe0fa0a3700 1 -- 172.21.15.12:0/3797997979 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0f41051d0 con 0x7fe0f4105710 2024-05-17T22:34:35.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:35.269+0000 7fe0f0ff9700 1 -- 172.21.15.12:0/3797997979 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fe0e8010430 con 0x7fe0f4105710 2024-05-17T22:34:35.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:35.270+0000 7fe0f0ff9700 1 --2- 172.21.15.12:0/3797997979 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe0dc0b2880 0x7fe0dc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:35.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:35.270+0000 7fe0f37fe700 1 --2- 172.21.15.12:0/3797997979 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe0dc0b2880 0x7fe0dc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:35.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:35.270+0000 7fe0f0ff9700 1 -- 172.21.15.12:0/3797997979 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fe0e80e2270 con 0x7fe0f4105710 2024-05-17T22:34:35.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:35.271+0000 7fe0f37fe700 1 --2- 172.21.15.12:0/3797997979 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe0dc0b2880 0x7fe0dc0b4d90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4009c10 tx=0x7fe0e400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:34:35.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:35.274+0000 7fe0f0ff9700 1 -- 172.21.15.12:0/3797997979 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe0e80aa390 con 0x7fe0f4105710 2024-05-17T22:34:35.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:35.498+0000 7fe0fa0a3700 1 -- 172.21.15.12:0/3797997979 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe0f419ba80 con 0x7fe0f4105710 2024-05-17T22:34:35.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:35 smithi151 ceph-mon[36101]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:35.761 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:35 smithi012 ceph-mon[31632]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:35.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:35 smithi096 ceph-mon[36065]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:36.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:36.388+0000 7fe0f0ff9700 1 -- 172.21.15.12:0/3797997979 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v72) ==== 138+0+0 (secure 0 0 0) 0x7fe0e80b2410 con 0x7fe0f4105710 2024-05-17T22:34:36.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:36.391+0000 7fe0fa0a3700 1 -- 172.21.15.12:0/3797997979 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe0dc0b2880 msgr2=0x7fe0dc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:36.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:36.391+0000 7fe0fa0a3700 1 --2- 172.21.15.12:0/3797997979 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe0dc0b2880 0x7fe0dc0b4d90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4009c10 tx=0x7fe0e400b040 comp rx=0 tx=0).stop 2024-05-17T22:34:36.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:36.391+0000 7fe0fa0a3700 1 -- 172.21.15.12:0/3797997979 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0f4105710 msgr2=0x7fe0f419acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:36.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:36.391+0000 7fe0fa0a3700 1 --2- 172.21.15.12:0/3797997979 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0f4105710 0x7fe0f419acc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe0e800e9f0 tx=0x7fe0e800ed00 comp rx=0 tx=0).stop 2024-05-17T22:34:36.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:36.391+0000 7fe0fa0a3700 1 -- 172.21.15.12:0/3797997979 shutdown_connections 2024-05-17T22:34:36.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:36.391+0000 7fe0fa0a3700 1 --2- 172.21.15.12:0/3797997979 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe0dc0b2880 0x7fe0dc0b4d90 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:36.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:36.391+0000 7fe0fa0a3700 1 --2- 172.21.15.12:0/3797997979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0f410e220 0x7fe0f419ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:36.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:36.391+0000 7fe0fa0a3700 1 --2- 172.21.15.12:0/3797997979 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0f4104dc0 0x7fe0f419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:36.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:36.391+0000 7fe0fa0a3700 1 --2- 172.21.15.12:0/3797997979 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0f4105710 0x7fe0f419acc0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:36.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:36.391+0000 7fe0fa0a3700 1 -- 172.21.15.12:0/3797997979 >> 172.21.15.12:0/3797997979 conn(0x7fe0f4079bd0 msgr2=0x7fe0f40ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:34:36.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:36.391+0000 7fe0fa0a3700 1 -- 172.21.15.12:0/3797997979 shutdown_connections 2024-05-17T22:34:36.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:36.391+0000 7fe0fa0a3700 1 -- 172.21.15.12:0/3797997979 wait complete. 2024-05-17T22:34:36.395 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_metadata' created 2024-05-17T22:34:36.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:36 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/3797997979' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-17T22:34:36.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:36 smithi151 ceph-mon[36101]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-17T22:34:36.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:36 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:36.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:36 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:36.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:36 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:36.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:36 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:36.769 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:36 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/3797997979' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-17T22:34:36.770 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:36 smithi012 ceph-mon[31632]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-17T22:34:36.770 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:36 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:36.770 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:36 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:36.770 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:36 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:36.770 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:36 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:36.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:36 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/3797997979' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-17T22:34:36.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:36 smithi096 ceph-mon[36065]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-17T22:34:36.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:36 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:36.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:36 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:36.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:36 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:36.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:36 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:37.108 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-05-17T22:34:37.367 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:34:37.640 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:37 smithi012 ceph-mon[31632]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-17T22:34:37.640 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:37 smithi012 ceph-mon[31632]: osdmap e72: 12 total, 12 up, 12 in 2024-05-17T22:34:37.640 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:37 smithi012 ceph-mon[31632]: pgmap v283: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:37.641 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:37 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:37.641 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:37 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:37.641 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:37 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:34:37.641 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:37 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:34:37.641 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:37 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:37.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:37 smithi151 ceph-mon[36101]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-17T22:34:37.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:37 smithi151 ceph-mon[36101]: osdmap e72: 12 total, 12 up, 12 in 2024-05-17T22:34:37.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:37 smithi151 ceph-mon[36101]: pgmap v283: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:37.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:37 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:37.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:37 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:37.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:37 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:34:37.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:37 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:34:37.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:37 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:37.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:37 smithi096 ceph-mon[36065]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-17T22:34:37.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:37 smithi096 ceph-mon[36065]: osdmap e72: 12 total, 12 up, 12 in 2024-05-17T22:34:37.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:37 smithi096 ceph-mon[36065]: pgmap v283: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:37.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:37 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:37.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:37 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:37.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:37 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:34:37.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:37 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:34:37.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:37 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:34:38.607 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:38 smithi012 ceph-mon[31632]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-17T22:34:38.607 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:38 smithi012 ceph-mon[31632]: osdmap e73: 12 total, 12 up, 12 in 2024-05-17T22:34:38.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:38 smithi151 ceph-mon[36101]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-17T22:34:38.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:38 smithi151 ceph-mon[36101]: osdmap e73: 12 total, 12 up, 12 in 2024-05-17T22:34:38.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:38 smithi096 ceph-mon[36065]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-17T22:34:38.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:38 smithi096 ceph-mon[36065]: osdmap e73: 12 total, 12 up, 12 in 2024-05-17T22:34:39.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.148+0000 7f94603ed700 1 -- 172.21.15.12:0/2815287386 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94581080a0 msgr2=0x7f945810fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:39.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.148+0000 7f94603ed700 1 --2- 172.21.15.12:0/2815287386 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94581080a0 0x7f945810fa00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f945000b0a0 tx=0x7f945000b3b0 comp rx=0 tx=0).stop 2024-05-17T22:34:39.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.148+0000 7f94603ed700 1 -- 172.21.15.12:0/2815287386 shutdown_connections 2024-05-17T22:34:39.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.148+0000 7f94603ed700 1 --2- 172.21.15.12:0/2815287386 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94581080a0 0x7f945810fa00 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:39.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.148+0000 7f94603ed700 1 --2- 172.21.15.12:0/2815287386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9458102e20 0x7f9458103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:39.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.148+0000 7f94603ed700 1 --2- 172.21.15.12:0/2815287386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9458107750 0x7f9458107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:39.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.148+0000 7f94603ed700 1 -- 172.21.15.12:0/2815287386 >> 172.21.15.12:0/2815287386 conn(0x7f9458079af0 msgr2=0x7f9458079f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:34:39.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.148+0000 7f94603ed700 1 -- 172.21.15.12:0/2815287386 shutdown_connections 2024-05-17T22:34:39.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.148+0000 7f94603ed700 1 -- 172.21.15.12:0/2815287386 wait complete. 2024-05-17T22:34:39.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.149+0000 7f94603ed700 1 Processor -- start 2024-05-17T22:34:39.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.149+0000 7f94603ed700 1 -- start start 2024-05-17T22:34:39.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.149+0000 7f94603ed700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9458102e20 0x7f945819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:39.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.149+0000 7f94603ed700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9458107750 0x7f945819ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:39.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.149+0000 7f94603ed700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94581080a0 0x7f945819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:39.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.149+0000 7f94603ed700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9458112ce0 con 0x7f94581080a0 2024-05-17T22:34:39.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.149+0000 7f94603ed700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9458112b60 con 0x7f9458102e20 2024-05-17T22:34:39.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.149+0000 7f94603ed700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9458112e60 con 0x7f9458107750 2024-05-17T22:34:39.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.149+0000 7f945d988700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9458107750 0x7f945819ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:39.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.149+0000 7f945d988700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9458107750 0x7f945819ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:38406/0 (socket says 172.21.15.12:38406) 2024-05-17T22:34:39.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.149+0000 7f945d988700 1 -- 172.21.15.12:0/602278380 learned_addr learned my addr 172.21.15.12:0/602278380 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:34:39.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.149+0000 7f945e98a700 1 --2- 172.21.15.12:0/602278380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94581080a0 0x7f945819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:39.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.150+0000 7f945e189700 1 --2- 172.21.15.12:0/602278380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9458102e20 0x7f945819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:39.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.150+0000 7f945d988700 1 -- 172.21.15.12:0/602278380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9458102e20 msgr2=0x7f945819a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:39.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.150+0000 7f945d988700 1 --2- 172.21.15.12:0/602278380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9458102e20 0x7f945819a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:39.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.150+0000 7f945d988700 1 -- 172.21.15.12:0/602278380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94581080a0 msgr2=0x7f945819ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:39.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.150+0000 7f945d988700 1 --2- 172.21.15.12:0/602278380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94581080a0 0x7f945819ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:39.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.150+0000 7f945d988700 1 -- 172.21.15.12:0/602278380 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f945819f510 con 0x7f9458107750 2024-05-17T22:34:39.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.150+0000 7f945e98a700 1 --2- 172.21.15.12:0/602278380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94581080a0 0x7f945819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:34:39.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.150+0000 7f945d988700 1 --2- 172.21.15.12:0/602278380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9458107750 0x7f945819ac90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f944800c9e0 tx=0x7f944800ccf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:34:39.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.150+0000 7f944f7fe700 1 -- 172.21.15.12:0/602278380 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9448007aa0 con 0x7f9458107750 2024-05-17T22:34:39.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.150+0000 7f94603ed700 1 -- 172.21.15.12:0/602278380 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f945819f800 con 0x7f9458107750 2024-05-17T22:34:39.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.150+0000 7f944f7fe700 1 -- 172.21.15.12:0/602278380 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9448007c40 con 0x7f9458107750 2024-05-17T22:34:39.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.150+0000 7f944f7fe700 1 -- 172.21.15.12:0/602278380 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944800fe60 con 0x7f9458107750 2024-05-17T22:34:39.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.150+0000 7f94603ed700 1 -- 172.21.15.12:0/602278380 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94581a71d0 con 0x7f9458107750 2024-05-17T22:34:39.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.152+0000 7f94603ed700 1 -- 172.21.15.12:0/602278380 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9458103230 con 0x7f9458107750 2024-05-17T22:34:39.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.153+0000 7f944f7fe700 1 -- 172.21.15.12:0/602278380 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f944800f450 con 0x7f9458107750 2024-05-17T22:34:39.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.154+0000 7f944f7fe700 1 --2- 172.21.15.12:0/602278380 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f94440b2880 0x7f94440b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:39.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.154+0000 7f945e189700 1 --2- 172.21.15.12:0/602278380 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f94440b2880 0x7f94440b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:39.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.154+0000 7f944f7fe700 1 -- 172.21.15.12:0/602278380 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7497+0+0 (secure 0 0 0) 0x7f94480e2000 con 0x7f9458107750 2024-05-17T22:34:39.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.155+0000 7f945e189700 1 --2- 172.21.15.12:0/602278380 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f94440b2880 0x7f94440b4d90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9454009c60 tx=0x7f9454009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:34:39.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.157+0000 7f944f7fe700 1 -- 172.21.15.12:0/602278380 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9448014030 con 0x7f9458107750 2024-05-17T22:34:39.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.370+0000 7f94603ed700 1 -- 172.21.15.12:0/602278380 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f945802ea80 con 0x7f9458107750 2024-05-17T22:34:39.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.405+0000 7f944f7fe700 1 -- 172.21.15.12:0/602278380 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v75) ==== 171+0+0 (secure 0 0 0) 0x7f94480aa0e0 con 0x7f9458107750 2024-05-17T22:34:39.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.409+0000 7f94603ed700 1 -- 172.21.15.12:0/602278380 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f94440b2880 msgr2=0x7f94440b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:39.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.409+0000 7f94603ed700 1 --2- 172.21.15.12:0/602278380 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f94440b2880 0x7f94440b4d90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9454009c60 tx=0x7f9454009260 comp rx=0 tx=0).stop 2024-05-17T22:34:39.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.409+0000 7f94603ed700 1 -- 172.21.15.12:0/602278380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9458107750 msgr2=0x7f945819ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:39.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.409+0000 7f94603ed700 1 --2- 172.21.15.12:0/602278380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9458107750 0x7f945819ac90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f944800c9e0 tx=0x7f944800ccf0 comp rx=0 tx=0).stop 2024-05-17T22:34:39.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.410+0000 7f94603ed700 1 -- 172.21.15.12:0/602278380 shutdown_connections 2024-05-17T22:34:39.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.410+0000 7f94603ed700 1 --2- 172.21.15.12:0/602278380 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f94440b2880 0x7f94440b4d90 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:39.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.410+0000 7f94603ed700 1 --2- 172.21.15.12:0/602278380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94581080a0 0x7f945819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:39.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.410+0000 7f94603ed700 1 --2- 172.21.15.12:0/602278380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9458102e20 0x7f945819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:39.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.410+0000 7f94603ed700 1 --2- 172.21.15.12:0/602278380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9458107750 0x7f945819ac90 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:39.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.410+0000 7f94603ed700 1 -- 172.21.15.12:0/602278380 >> 172.21.15.12:0/602278380 conn(0x7f9458079af0 msgr2=0x7f945806c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:34:39.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.411+0000 7f94603ed700 1 -- 172.21.15.12:0/602278380 shutdown_connections 2024-05-17T22:34:39.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:39.411+0000 7f94603ed700 1 -- 172.21.15.12:0/602278380 wait complete. 2024-05-17T22:34:39.415 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_data' created 2024-05-17T22:34:39.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:39 smithi151 ceph-mon[36101]: osdmap e74: 12 total, 12 up, 12 in 2024-05-17T22:34:39.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:39 smithi151 ceph-mon[36101]: pgmap v286: 33 pgs: 1 creating+activating, 3 creating+peering, 14 active+clean, 15 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:39.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:39 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/602278380' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-17T22:34:39.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:39 smithi151 ceph-mon[36101]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-17T22:34:39.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:39 smithi096 ceph-mon[36065]: osdmap e74: 12 total, 12 up, 12 in 2024-05-17T22:34:39.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:39 smithi096 ceph-mon[36065]: pgmap v286: 33 pgs: 1 creating+activating, 3 creating+peering, 14 active+clean, 15 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:39.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:39 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/602278380' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-17T22:34:39.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:39 smithi096 ceph-mon[36065]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-17T22:34:39.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:39 smithi012 ceph-mon[31632]: osdmap e74: 12 total, 12 up, 12 in 2024-05-17T22:34:39.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:39 smithi012 ceph-mon[31632]: pgmap v286: 33 pgs: 1 creating+activating, 3 creating+peering, 14 active+clean, 15 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:39.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:39 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/602278380' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-17T22:34:39.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:39 smithi012 ceph-mon[31632]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-17T22:34:40.323 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-17T22:34:40.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:40 smithi096 ceph-mon[36065]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-17T22:34:40.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:40 smithi096 ceph-mon[36065]: osdmap e75: 12 total, 12 up, 12 in 2024-05-17T22:34:40.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:40 smithi012 ceph-mon[31632]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-17T22:34:40.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:40 smithi012 ceph-mon[31632]: osdmap e75: 12 total, 12 up, 12 in 2024-05-17T22:34:40.593 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:34:40.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:40 smithi151 ceph-mon[36101]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-17T22:34:40.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:40 smithi151 ceph-mon[36101]: osdmap e75: 12 total, 12 up, 12 in 2024-05-17T22:34:41.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:41 smithi012 ceph-mon[31632]: osdmap e76: 12 total, 12 up, 12 in 2024-05-17T22:34:41.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:41 smithi012 ceph-mon[31632]: pgmap v289: 97 pgs: 1 creating+activating, 7 creating+peering, 23 active+clean, 66 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:41.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:41 smithi151 ceph-mon[36101]: osdmap e76: 12 total, 12 up, 12 in 2024-05-17T22:34:41.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:41 smithi151 ceph-mon[36101]: pgmap v289: 97 pgs: 1 creating+activating, 7 creating+peering, 23 active+clean, 66 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:41.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:41 smithi096 ceph-mon[36065]: osdmap e76: 12 total, 12 up, 12 in 2024-05-17T22:34:41.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:41 smithi096 ceph-mon[36065]: pgmap v289: 97 pgs: 1 creating+activating, 7 creating+peering, 23 active+clean, 66 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:42.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:42.361+0000 7f90106a3700 1 -- 172.21.15.12:0/4256722036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f900810e2a0 msgr2=0x7f9008110740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:42.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:42.361+0000 7f90106a3700 1 --2- 172.21.15.12:0/4256722036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f900810e2a0 0x7f9008110740 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f900000b0a0 tx=0x7f900000b3b0 comp rx=0 tx=0).stop 2024-05-17T22:34:42.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:42.361+0000 7f90106a3700 1 -- 172.21.15.12:0/4256722036 shutdown_connections 2024-05-17T22:34:42.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:42.361+0000 7f90106a3700 1 --2- 172.21.15.12:0/4256722036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f900810e2a0 0x7f9008110740 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:42.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:42.361+0000 7f90106a3700 1 --2- 172.21.15.12:0/4256722036 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9008104e70 0x7f9008105280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:42.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:42.361+0000 7f90106a3700 1 --2- 172.21.15.12:0/4256722036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90081057c0 0x7f900810dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:42.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:42.361+0000 7f90106a3700 1 -- 172.21.15.12:0/4256722036 >> 172.21.15.12:0/4256722036 conn(0x7f9008079ae0 msgr2=0x7f9008079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:34:42.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:42.361+0000 7f90106a3700 1 -- 172.21.15.12:0/4256722036 shutdown_connections 2024-05-17T22:34:42.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:42.361+0000 7f90106a3700 1 -- 172.21.15.12:0/4256722036 wait complete. 2024-05-17T22:34:42.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:42.362+0000 7f90106a3700 1 Processor -- start 2024-05-17T22:34:42.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:42.362+0000 7f90106a3700 1 -- start start 2024-05-17T22:34:42.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:42.362+0000 7f90106a3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9008104e70 0x7f9008102b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:42.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:42.362+0000 7f90106a3700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90081057c0 0x7f90081030a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:42.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:42.362+0000 7f90106a3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f900810e2a0 0x7f90081035e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:42.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:42.362+0000 7f90106a3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9008113db0 con 0x7f9008104e70 2024-05-17T22:34:42.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:42.362+0000 7f90106a3700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9008113c30 con 0x7f900810e2a0 2024-05-17T22:34:42.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:42.362+0000 7f90106a3700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9008113f30 con 0x7f90081057c0 2024-05-17T22:34:42.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:42.362+0000 7f900e43f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9008104e70 0x7f9008102b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:42.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:42.362+0000 7f900e43f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9008104e70 0x7f9008102b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54350/0 (socket says 172.21.15.12:54350) 2024-05-17T22:34:42.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:42.362+0000 7f900e43f700 1 -- 172.21.15.12:0/466520587 learned_addr learned my addr 172.21.15.12:0/466520587 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:34:42.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:42.362+0000 7f900ec40700 1 --2- 172.21.15.12:0/466520587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f900810e2a0 0x7f90081035e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:42.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:42.363+0000 7f900dc3e700 1 --2- 172.21.15.12:0/466520587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90081057c0 0x7f90081030a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:42.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:42.363+0000 7f900e43f700 1 -- 172.21.15.12:0/466520587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90081057c0 msgr2=0x7f90081030a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:42.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:42.363+0000 7f900e43f700 1 --2- 172.21.15.12:0/466520587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90081057c0 0x7f90081030a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:42.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:42.363+0000 7f900e43f700 1 -- 172.21.15.12:0/466520587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f900810e2a0 msgr2=0x7f90081035e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:42.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:42.363+0000 7f900e43f700 1 --2- 172.21.15.12:0/466520587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f900810e2a0 0x7f90081035e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:42.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:42.363+0000 7f900e43f700 1 -- 172.21.15.12:0/466520587 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90081a8ab0 con 0x7f9008104e70 2024-05-17T22:34:42.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:42.363+0000 7f900e43f700 1 --2- 172.21.15.12:0/466520587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9008104e70 0x7f9008102b60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f900400c920 tx=0x7f900400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:34:42.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:42.363+0000 7f8ffb7fe700 1 -- 172.21.15.12:0/466520587 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9004007b00 con 0x7f9008104e70 2024-05-17T22:34:42.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:42.363+0000 7f90106a3700 1 -- 172.21.15.12:0/466520587 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90081a8da0 con 0x7f9008104e70 2024-05-17T22:34:42.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:42.363+0000 7f8ffb7fe700 1 -- 172.21.15.12:0/466520587 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9004004ce0 con 0x7f9008104e70 2024-05-17T22:34:42.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:42.363+0000 7f8ffb7fe700 1 -- 172.21.15.12:0/466520587 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9004005500 con 0x7f9008104e70 2024-05-17T22:34:42.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:42.363+0000 7f90106a3700 1 -- 172.21.15.12:0/466520587 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90081a9260 con 0x7f9008104e70 2024-05-17T22:34:42.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:42.364+0000 7f90106a3700 1 -- 172.21.15.12:0/466520587 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9008105280 con 0x7f9008104e70 2024-05-17T22:34:42.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:42.366+0000 7f8ffb7fe700 1 -- 172.21.15.12:0/466520587 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f9004007ca0 con 0x7f9008104e70 2024-05-17T22:34:42.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:42.367+0000 7f8ffb7fe700 1 --2- 172.21.15.12:0/466520587 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8ff40b2880 0x7f8ff40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:42.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:42.367+0000 7f900dc3e700 1 --2- 172.21.15.12:0/466520587 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8ff40b2880 0x7f8ff40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:42.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:42.367+0000 7f8ffb7fe700 1 -- 172.21.15.12:0/466520587 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7879+0+0 (secure 0 0 0) 0x7f90040e1ff0 con 0x7f9008104e70 2024-05-17T22:34:42.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:42.367+0000 7f900dc3e700 1 --2- 172.21.15.12:0/466520587 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8ff40b2880 0x7f8ff40b4d90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc005950 tx=0x7f8ffc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:34:42.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:42.370+0000 7f8ffb7fe700 1 -- 172.21.15.12:0/466520587 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9004014030 con 0x7f9008104e70 2024-05-17T22:34:42.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:42.640+0000 7f90106a3700 1 -- 172.21.15.12:0/466520587 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f9008104090 con 0x7f9008104e70 2024-05-17T22:34:42.643 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:42 smithi012 ceph-mon[31632]: osdmap e77: 12 total, 12 up, 12 in 2024-05-17T22:34:42.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:42 smithi151 ceph-mon[36101]: osdmap e77: 12 total, 12 up, 12 in 2024-05-17T22:34:42.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:42 smithi096 ceph-mon[36065]: osdmap e77: 12 total, 12 up, 12 in 2024-05-17T22:34:43.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:43.424+0000 7f8ffb7fe700 1 -- 172.21.15.12:0/466520587 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f90040a9ec0 con 0x7f9008104e70 2024-05-17T22:34:43.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:43.428+0000 7f90106a3700 1 -- 172.21.15.12:0/466520587 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8ff40b2880 msgr2=0x7f8ff40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:43.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:43.428+0000 7f90106a3700 1 --2- 172.21.15.12:0/466520587 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8ff40b2880 0x7f8ff40b4d90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc005950 tx=0x7f8ffc0058e0 comp rx=0 tx=0).stop 2024-05-17T22:34:43.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:43.428+0000 7f90106a3700 1 -- 172.21.15.12:0/466520587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9008104e70 msgr2=0x7f9008102b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:43.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:43.428+0000 7f90106a3700 1 --2- 172.21.15.12:0/466520587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9008104e70 0x7f9008102b60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f900400c920 tx=0x7f900400cc30 comp rx=0 tx=0).stop 2024-05-17T22:34:43.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:43.428+0000 7f90106a3700 1 -- 172.21.15.12:0/466520587 shutdown_connections 2024-05-17T22:34:43.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:43.428+0000 7f90106a3700 1 --2- 172.21.15.12:0/466520587 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8ff40b2880 0x7f8ff40b4d90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:43.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:43.428+0000 7f90106a3700 1 --2- 172.21.15.12:0/466520587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f900810e2a0 0x7f90081035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:43.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:43.428+0000 7f90106a3700 1 --2- 172.21.15.12:0/466520587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9008104e70 0x7f9008102b60 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:43.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:43.428+0000 7f90106a3700 1 --2- 172.21.15.12:0/466520587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90081057c0 0x7f90081030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:43.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:43.428+0000 7f90106a3700 1 -- 172.21.15.12:0/466520587 >> 172.21.15.12:0/466520587 conn(0x7f9008079ae0 msgr2=0x7f90080ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:34:43.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:43.428+0000 7f90106a3700 1 -- 172.21.15.12:0/466520587 shutdown_connections 2024-05-17T22:34:43.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:43.428+0000 7f90106a3700 1 -- 172.21.15.12:0/466520587 wait complete. 2024-05-17T22:34:43.432 INFO:teuthology.orchestra.run.smithi012.stderr:new fs with metadata pool 2 and data pool 3 2024-05-17T22:34:43.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:43 smithi151 ceph-mon[36101]: pgmap v291: 97 pgs: 1 creating+activating, 4 creating+peering, 47 active+clean, 45 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:43.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:43 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/466520587' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-17T22:34:43.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:43 smithi151 ceph-mon[36101]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-17T22:34:43.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:43 smithi151 ceph-mon[36101]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-17T22:34:43.765 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:43 smithi012 ceph-mon[31632]: pgmap v291: 97 pgs: 1 creating+activating, 4 creating+peering, 47 active+clean, 45 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:43.765 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:43 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/466520587' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-17T22:34:43.766 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:43 smithi012 ceph-mon[31632]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-17T22:34:43.766 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:43 smithi012 ceph-mon[31632]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-17T22:34:43.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:43 smithi096 ceph-mon[36065]: pgmap v291: 97 pgs: 1 creating+activating, 4 creating+peering, 47 active+clean, 45 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:43.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:43 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/466520587' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-17T22:34:43.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:43 smithi096 ceph-mon[36065]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-17T22:34:43.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:43 smithi096 ceph-mon[36065]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-17T22:34:44.219 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-05-17T22:34:44.219 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-05-17T22:34:44.482 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:34:44.737 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:44 smithi012 ceph-mon[31632]: osdmap e78: 12 total, 12 up, 12 in 2024-05-17T22:34:44.737 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:44 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/466520587' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-17T22:34:44.737 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:44 smithi012 ceph-mon[31632]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-17T22:34:44.737 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:44 smithi012 ceph-mon[31632]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-17T22:34:44.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:44 smithi151 ceph-mon[36101]: osdmap e78: 12 total, 12 up, 12 in 2024-05-17T22:34:44.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:44 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/466520587' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-17T22:34:44.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:44 smithi151 ceph-mon[36101]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-17T22:34:44.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:44 smithi151 ceph-mon[36101]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-17T22:34:44.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:44 smithi096 ceph-mon[36065]: osdmap e78: 12 total, 12 up, 12 in 2024-05-17T22:34:44.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:44 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/466520587' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-17T22:34:44.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:44 smithi096 ceph-mon[36065]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-17T22:34:44.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:44 smithi096 ceph-mon[36065]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-17T22:34:45.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:45 smithi012 ceph-mon[31632]: mds.? [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] up:active 2024-05-17T22:34:45.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:45 smithi012 ceph-mon[31632]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T22:34:45.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:45 smithi012 ceph-mon[31632]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T22:34:45.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:45 smithi012 ceph-mon[31632]: pgmap v293: 97 pgs: 4 creating+peering, 93 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-17T22:34:45.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:45 smithi151 ceph-mon[36101]: mds.? [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] up:active 2024-05-17T22:34:45.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:45 smithi151 ceph-mon[36101]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T22:34:45.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:45 smithi151 ceph-mon[36101]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T22:34:45.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:45 smithi151 ceph-mon[36101]: pgmap v293: 97 pgs: 4 creating+peering, 93 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-17T22:34:45.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:45 smithi096 ceph-mon[36065]: mds.? [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] up:active 2024-05-17T22:34:45.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:45 smithi096 ceph-mon[36065]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T22:34:45.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:45 smithi096 ceph-mon[36065]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T22:34:45.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:45 smithi096 ceph-mon[36065]: pgmap v293: 97 pgs: 4 creating+peering, 93 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-17T22:34:46.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:46.285+0000 7f9b60384700 1 -- 172.21.15.12:0/937739479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b581091b0 msgr2=0x7f9b58110b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:46.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:46.285+0000 7f9b60384700 1 --2- 172.21.15.12:0/937739479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b581091b0 0x7f9b58110b10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9b5000b0a0 tx=0x7f9b5000b3b0 comp rx=0 tx=0).stop 2024-05-17T22:34:46.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:46.285+0000 7f9b60384700 1 -- 172.21.15.12:0/937739479 shutdown_connections 2024-05-17T22:34:46.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:46.285+0000 7f9b60384700 1 --2- 172.21.15.12:0/937739479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b581091b0 0x7f9b58110b10 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:46.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:46.285+0000 7f9b60384700 1 --2- 172.21.15.12:0/937739479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b58102440 0x7f9b58102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:46.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:46.285+0000 7f9b60384700 1 --2- 172.21.15.12:0/937739479 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b58108860 0x7f9b58108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:46.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:46.285+0000 7f9b60384700 1 -- 172.21.15.12:0/937739479 >> 172.21.15.12:0/937739479 conn(0x7f9b580fe1a0 msgr2=0x7f9b581005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:34:46.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:46.285+0000 7f9b60384700 1 -- 172.21.15.12:0/937739479 shutdown_connections 2024-05-17T22:34:46.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:46.285+0000 7f9b60384700 1 -- 172.21.15.12:0/937739479 wait complete. 2024-05-17T22:34:46.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:46.286+0000 7f9b60384700 1 Processor -- start 2024-05-17T22:34:46.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:46.286+0000 7f9b60384700 1 -- start start 2024-05-17T22:34:46.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:46.286+0000 7f9b60384700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b58102440 0x7f9b58103390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:46.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:46.286+0000 7f9b60384700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b58108860 0x7f9b581038d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:46.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:46.286+0000 7f9b60384700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b581091b0 0x7f9b58103e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:46.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:46.286+0000 7f9b60384700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9b58113e60 con 0x7f9b58102440 2024-05-17T22:34:46.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:46.286+0000 7f9b60384700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9b58113ce0 con 0x7f9b581091b0 2024-05-17T22:34:46.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:46.286+0000 7f9b60384700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9b58113fe0 con 0x7f9b58108860 2024-05-17T22:34:46.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:46.286+0000 7f9b5e120700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b58102440 0x7f9b58103390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:46.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:46.286+0000 7f9b5e120700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b58102440 0x7f9b58103390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54378/0 (socket says 172.21.15.12:54378) 2024-05-17T22:34:46.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:46.286+0000 7f9b5e120700 1 -- 172.21.15.12:0/3454752244 learned_addr learned my addr 172.21.15.12:0/3454752244 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:34:46.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:46.286+0000 7f9b5e921700 1 --2- 172.21.15.12:0/3454752244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b581091b0 0x7f9b58103e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:46.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:46.286+0000 7f9b5d91f700 1 --2- 172.21.15.12:0/3454752244 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b58108860 0x7f9b581038d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:46.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:46.287+0000 7f9b5e120700 1 -- 172.21.15.12:0/3454752244 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b58108860 msgr2=0x7f9b581038d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:46.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:46.287+0000 7f9b5e120700 1 --2- 172.21.15.12:0/3454752244 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b58108860 0x7f9b581038d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:46.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:46.287+0000 7f9b5e120700 1 -- 172.21.15.12:0/3454752244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b581091b0 msgr2=0x7f9b58103e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:46.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:46.287+0000 7f9b5e120700 1 --2- 172.21.15.12:0/3454752244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b581091b0 0x7f9b58103e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:46.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:46.287+0000 7f9b5e120700 1 -- 172.21.15.12:0/3454752244 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b581a8b30 con 0x7f9b58102440 2024-05-17T22:34:46.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:46.287+0000 7f9b5e120700 1 --2- 172.21.15.12:0/3454752244 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b58102440 0x7f9b58103390 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9b5400b530 tx=0x7f9b5400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:34:46.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:46.287+0000 7f9b4f7fe700 1 -- 172.21.15.12:0/3454752244 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b54015020 con 0x7f9b58102440 2024-05-17T22:34:46.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:46.287+0000 7f9b60384700 1 -- 172.21.15.12:0/3454752244 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b581a8e20 con 0x7f9b58102440 2024-05-17T22:34:46.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:46.287+0000 7f9b4f7fe700 1 -- 172.21.15.12:0/3454752244 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b54011ce0 con 0x7f9b58102440 2024-05-17T22:34:46.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:46.287+0000 7f9b60384700 1 -- 172.21.15.12:0/3454752244 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b581a92d0 con 0x7f9b58102440 2024-05-17T22:34:46.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:46.287+0000 7f9b4f7fe700 1 -- 172.21.15.12:0/3454752244 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b54010dd0 con 0x7f9b58102440 2024-05-17T22:34:46.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:46.288+0000 7f9b60384700 1 -- 172.21.15.12:0/3454752244 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b58102850 con 0x7f9b58102440 2024-05-17T22:34:46.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:46.291+0000 7f9b4f7fe700 1 -- 172.21.15.12:0/3454752244 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f9b54018020 con 0x7f9b58102440 2024-05-17T22:34:46.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:46.292+0000 7f9b4f7fe700 1 --2- 172.21.15.12:0/3454752244 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9b440b2880 0x7f9b440b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:46.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:46.292+0000 7f9b5d91f700 1 --2- 172.21.15.12:0/3454752244 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9b440b2880 0x7f9b440b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:46.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:46.292+0000 7f9b4f7fe700 1 -- 172.21.15.12:0/3454752244 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f9b540e1e10 con 0x7f9b58102440 2024-05-17T22:34:46.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:46.292+0000 7f9b5d91f700 1 --2- 172.21.15.12:0/3454752244 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9b440b2880 0x7f9b440b4d90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9b58107a80 tx=0x7f9b4800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:34:46.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:46.295+0000 7f9b4f7fe700 1 -- 172.21.15.12:0/3454752244 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9b5401f020 con 0x7f9b58102440 2024-05-17T22:34:46.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:46.520+0000 7f9b60384700 1 -- 172.21.15.12:0/3454752244 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f9b581048c0 con 0x7f9b58102440 2024-05-17T22:34:46.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:46 smithi012 ceph-mon[31632]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-17T22:34:46.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:46 smithi012 ceph-mon[31632]: Cluster is now healthy 2024-05-17T22:34:46.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:46 smithi151 ceph-mon[36101]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-17T22:34:46.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:46 smithi151 ceph-mon[36101]: Cluster is now healthy 2024-05-17T22:34:46.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:46 smithi096 ceph-mon[36065]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-17T22:34:46.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:46 smithi096 ceph-mon[36065]: Cluster is now healthy 2024-05-17T22:34:47.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:47.445+0000 7f9b4f7fe700 1 -- 172.21.15.12:0/3454752244 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v79) ==== 157+0+0 (secure 0 0 0) 0x7f9b540a9be0 con 0x7f9b58102440 2024-05-17T22:34:47.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:47.449+0000 7f9b60384700 1 -- 172.21.15.12:0/3454752244 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9b440b2880 msgr2=0x7f9b440b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:47.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:47.449+0000 7f9b60384700 1 --2- 172.21.15.12:0/3454752244 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9b440b2880 0x7f9b440b4d90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9b58107a80 tx=0x7f9b4800a380 comp rx=0 tx=0).stop 2024-05-17T22:34:47.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:47.449+0000 7f9b60384700 1 -- 172.21.15.12:0/3454752244 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b58102440 msgr2=0x7f9b58103390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:47.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:47.449+0000 7f9b60384700 1 --2- 172.21.15.12:0/3454752244 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b58102440 0x7f9b58103390 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9b5400b530 tx=0x7f9b5400b840 comp rx=0 tx=0).stop 2024-05-17T22:34:47.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:47.449+0000 7f9b60384700 1 -- 172.21.15.12:0/3454752244 shutdown_connections 2024-05-17T22:34:47.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:47.449+0000 7f9b60384700 1 --2- 172.21.15.12:0/3454752244 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9b440b2880 0x7f9b440b4d90 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:47.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:47.449+0000 7f9b60384700 1 --2- 172.21.15.12:0/3454752244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b581091b0 0x7f9b58103e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:47.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:47.449+0000 7f9b60384700 1 --2- 172.21.15.12:0/3454752244 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b58102440 0x7f9b58103390 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:47.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:47.449+0000 7f9b60384700 1 --2- 172.21.15.12:0/3454752244 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b58108860 0x7f9b581038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:47.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:47.449+0000 7f9b60384700 1 -- 172.21.15.12:0/3454752244 >> 172.21.15.12:0/3454752244 conn(0x7f9b580fe1a0 msgr2=0x7f9b58108020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:34:47.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:47.449+0000 7f9b60384700 1 -- 172.21.15.12:0/3454752244 shutdown_connections 2024-05-17T22:34:47.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:47.450+0000 7f9b60384700 1 -- 172.21.15.12:0/3454752244 wait complete. 2024-05-17T22:34:47.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:47 smithi151 ceph-mon[36101]: pgmap v294: 97 pgs: 97 active+clean; 578 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 674 B/s wr, 4 op/s 2024-05-17T22:34:47.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:47 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/3454752244' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-17T22:34:47.759 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:47 smithi012 ceph-mon[31632]: pgmap v294: 97 pgs: 97 active+clean; 578 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 674 B/s wr, 4 op/s 2024-05-17T22:34:47.759 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:47 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/3454752244' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-17T22:34:47.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:47 smithi096 ceph-mon[36065]: pgmap v294: 97 pgs: 97 active+clean; 578 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 674 B/s wr, 4 op/s 2024-05-17T22:34:47.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:47 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/3454752244' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-17T22:34:48.068 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-05-17T22:34:48.324 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:34:48.705 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:48 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/3454752244' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-17T22:34:48.705 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:48 smithi012 ceph-mon[31632]: osdmap e79: 12 total, 12 up, 12 in 2024-05-17T22:34:48.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:48 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/3454752244' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-17T22:34:48.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:48 smithi151 ceph-mon[36101]: osdmap e79: 12 total, 12 up, 12 in 2024-05-17T22:34:48.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:48 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/3454752244' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-17T22:34:48.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:48 smithi096 ceph-mon[36065]: osdmap e79: 12 total, 12 up, 12 in 2024-05-17T22:34:48.799 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 17 22:34:48 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-b[66490]: 2024-05-17T22:34:48.562+0000 7f9af454c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-17T22:34:49.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:49 smithi151 ceph-mon[36101]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-05-17T22:34:49.773 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:49 smithi012 ceph-mon[31632]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-05-17T22:34:49.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:49 smithi096 ceph-mon[36065]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-05-17T22:34:50.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:50.020+0000 7f5fee8fb700 1 -- 172.21.15.12:0/2126700962 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fe8104e10 msgr2=0x7f5fe8105220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:50.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:50.020+0000 7f5fee8fb700 1 --2- 172.21.15.12:0/2126700962 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fe8104e10 0x7f5fe8105220 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8009990 tx=0x7f5fd8009ca0 comp rx=0 tx=0).stop 2024-05-17T22:34:50.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:50.020+0000 7f5fee8fb700 1 -- 172.21.15.12:0/2126700962 shutdown_connections 2024-05-17T22:34:50.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:50.020+0000 7f5fee8fb700 1 --2- 172.21.15.12:0/2126700962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5fe810e240 0x7f5fe8110750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:50.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:50.020+0000 7f5fee8fb700 1 --2- 172.21.15.12:0/2126700962 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fe8104e10 0x7f5fe8105220 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:50.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:50.020+0000 7f5fee8fb700 1 --2- 172.21.15.12:0/2126700962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5fe8105760 0x7f5fe810dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:50.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:50.020+0000 7f5fee8fb700 1 -- 172.21.15.12:0/2126700962 >> 172.21.15.12:0/2126700962 conn(0x7f5fe8079ae0 msgr2=0x7f5fe8079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:34:50.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:50.020+0000 7f5fee8fb700 1 -- 172.21.15.12:0/2126700962 shutdown_connections 2024-05-17T22:34:50.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:50.020+0000 7f5fee8fb700 1 -- 172.21.15.12:0/2126700962 wait complete. 2024-05-17T22:34:50.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:50.021+0000 7f5fee8fb700 1 Processor -- start 2024-05-17T22:34:50.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:50.021+0000 7f5fee8fb700 1 -- start start 2024-05-17T22:34:50.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:50.021+0000 7f5fee8fb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fe8104e10 0x7f5fe81a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:50.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:50.021+0000 7f5fee8fb700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5fe8105760 0x7f5fe81a2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:50.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:50.021+0000 7f5fee8fb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5fe810e240 0x7f5fe819c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:50.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:50.021+0000 7f5fee8fb700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe8113e40 con 0x7f5fe8104e10 2024-05-17T22:34:50.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:50.021+0000 7f5fee8fb700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe8113cc0 con 0x7f5fe810e240 2024-05-17T22:34:50.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:50.021+0000 7f5fee8fb700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe8113fc0 con 0x7f5fe8105760 2024-05-17T22:34:50.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:50.021+0000 7f5fe7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fe8104e10 0x7f5fe81a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:50.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:50.021+0000 7f5fe7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fe8104e10 0x7f5fe81a2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53144/0 (socket says 172.21.15.12:53144) 2024-05-17T22:34:50.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:50.021+0000 7f5fe7fff700 1 -- 172.21.15.12:0/2251516127 learned_addr learned my addr 172.21.15.12:0/2251516127 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:34:50.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:50.021+0000 7f5fdffff700 1 --2- 172.21.15.12:0/2251516127 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5fe8105760 0x7f5fe81a2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:50.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:50.021+0000 7f5fece98700 1 --2- 172.21.15.12:0/2251516127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5fe810e240 0x7f5fe819c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:50.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:50.022+0000 7f5fe7fff700 1 -- 172.21.15.12:0/2251516127 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5fe8105760 msgr2=0x7f5fe81a2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:50.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:50.022+0000 7f5fe7fff700 1 --2- 172.21.15.12:0/2251516127 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5fe8105760 0x7f5fe81a2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:50.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:50.022+0000 7f5fe7fff700 1 -- 172.21.15.12:0/2251516127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5fe810e240 msgr2=0x7f5fe819c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:50.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:50.022+0000 7f5fe7fff700 1 --2- 172.21.15.12:0/2251516127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5fe810e240 0x7f5fe819c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:50.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:50.022+0000 7f5fe7fff700 1 -- 172.21.15.12:0/2251516127 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe819cf80 con 0x7f5fe8104e10 2024-05-17T22:34:50.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:50.022+0000 7f5fe7fff700 1 --2- 172.21.15.12:0/2251516127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fe8104e10 0x7f5fe81a2610 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8000c00 tx=0x7f5fd80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:34:50.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:50.022+0000 7f5fe5ffb700 1 -- 172.21.15.12:0/2251516127 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8027070 con 0x7f5fe8104e10 2024-05-17T22:34:50.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:50.022+0000 7f5fee8fb700 1 -- 172.21.15.12:0/2251516127 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fe819d210 con 0x7f5fe8104e10 2024-05-17T22:34:50.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:50.022+0000 7f5fe5ffb700 1 -- 172.21.15.12:0/2251516127 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fd8003f20 con 0x7f5fe8104e10 2024-05-17T22:34:50.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:50.022+0000 7f5fe5ffb700 1 -- 172.21.15.12:0/2251516127 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8022ca0 con 0x7f5fe8104e10 2024-05-17T22:34:50.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:50.022+0000 7f5fee8fb700 1 -- 172.21.15.12:0/2251516127 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fe81a9470 con 0x7f5fe8104e10 2024-05-17T22:34:50.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:50.023+0000 7f5fee8fb700 1 -- 172.21.15.12:0/2251516127 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fe806c8d0 con 0x7f5fe8104e10 2024-05-17T22:34:50.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:50.025+0000 7f5fe5ffb700 1 -- 172.21.15.12:0/2251516127 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f5fd802a070 con 0x7f5fe8104e10 2024-05-17T22:34:50.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:50.026+0000 7f5fe5ffb700 1 --2- 172.21.15.12:0/2251516127 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5fd40b2900 0x7f5fd40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:50.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:50.026+0000 7f5fdffff700 1 --2- 172.21.15.12:0/2251516127 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5fd40b2900 0x7f5fd40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:50.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:50.026+0000 7f5fe5ffb700 1 -- 172.21.15.12:0/2251516127 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8277+0+0 (secure 0 0 0) 0x7f5fd80e3440 con 0x7f5fe8104e10 2024-05-17T22:34:50.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:50.026+0000 7f5fdffff700 1 --2- 172.21.15.12:0/2251516127 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5fd40b2900 0x7f5fd40b4e10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5fe810da30 tx=0x7f5fd0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:34:50.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:50.029+0000 7f5fe5ffb700 1 -- 172.21.15.12:0/2251516127 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5fd8031080 con 0x7f5fe8104e10 2024-05-17T22:34:50.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:50.243+0000 7f5fee8fb700 1 -- 172.21.15.12:0/2251516127 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f5fe8105220 con 0x7f5fe8104e10 2024-05-17T22:34:50.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:50 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/2251516127' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-17T22:34:50.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:50 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/2251516127' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-17T22:34:50.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:50 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/2251516127' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-17T22:34:51.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:51.466+0000 7f5fe5ffb700 1 -- 172.21.15.12:0/2251516127 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v81) ==== 228+0+0 (secure 0 0 0) 0x7f5fd8013070 con 0x7f5fe8104e10 2024-05-17T22:34:51.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:51.470+0000 7f5fee8fb700 1 -- 172.21.15.12:0/2251516127 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5fd40b2900 msgr2=0x7f5fd40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:51.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:51.470+0000 7f5fee8fb700 1 --2- 172.21.15.12:0/2251516127 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5fd40b2900 0x7f5fd40b4e10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5fe810da30 tx=0x7f5fd0009190 comp rx=0 tx=0).stop 2024-05-17T22:34:51.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:51.470+0000 7f5fee8fb700 1 -- 172.21.15.12:0/2251516127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fe8104e10 msgr2=0x7f5fe81a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:51.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:51.470+0000 7f5fee8fb700 1 --2- 172.21.15.12:0/2251516127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fe8104e10 0x7f5fe81a2610 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8000c00 tx=0x7f5fd80047b0 comp rx=0 tx=0).stop 2024-05-17T22:34:51.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:51.472+0000 7f5fee8fb700 1 -- 172.21.15.12:0/2251516127 shutdown_connections 2024-05-17T22:34:51.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:51.472+0000 7f5fee8fb700 1 --2- 172.21.15.12:0/2251516127 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5fd40b2900 0x7f5fd40b4e10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:51.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:51.472+0000 7f5fee8fb700 1 --2- 172.21.15.12:0/2251516127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5fe810e240 0x7f5fe819c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:51.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:51.472+0000 7f5fee8fb700 1 --2- 172.21.15.12:0/2251516127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fe8104e10 0x7f5fe81a2610 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:51.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:51.472+0000 7f5fee8fb700 1 --2- 172.21.15.12:0/2251516127 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5fe8105760 0x7f5fe81a2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:51.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:51.472+0000 7f5fee8fb700 1 -- 172.21.15.12:0/2251516127 >> 172.21.15.12:0/2251516127 conn(0x7f5fe8079ae0 msgr2=0x7f5fe810a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:34:51.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:51.473+0000 7f5fee8fb700 1 -- 172.21.15.12:0/2251516127 shutdown_connections 2024-05-17T22:34:51.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:51.473+0000 7f5fee8fb700 1 -- 172.21.15.12:0/2251516127 wait complete. 2024-05-17T22:34:51.478 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_data_ec' created 2024-05-17T22:34:51.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:51 smithi151 ceph-mon[36101]: osdmap e80: 12 total, 12 up, 12 in 2024-05-17T22:34:51.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:51 smithi151 ceph-mon[36101]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-05-17T22:34:51.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:51 smithi096 ceph-mon[36065]: osdmap e80: 12 total, 12 up, 12 in 2024-05-17T22:34:51.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:51 smithi096 ceph-mon[36065]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-05-17T22:34:51.819 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:51 smithi012 ceph-mon[31632]: osdmap e80: 12 total, 12 up, 12 in 2024-05-17T22:34:51.819 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:51 smithi012 ceph-mon[31632]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-05-17T22:34:52.306 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-05-17T22:34:52.580 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:34:52.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:52 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/2251516127' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-17T22:34:52.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:52 smithi012 ceph-mon[31632]: osdmap e81: 12 total, 12 up, 12 in 2024-05-17T22:34:52.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:52 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/2251516127' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-17T22:34:52.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:52 smithi151 ceph-mon[36101]: osdmap e81: 12 total, 12 up, 12 in 2024-05-17T22:34:52.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:52 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/2251516127' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-17T22:34:52.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:52 smithi096 ceph-mon[36065]: osdmap e81: 12 total, 12 up, 12 in 2024-05-17T22:34:53.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:53 smithi151 ceph-mon[36101]: osdmap e82: 12 total, 12 up, 12 in 2024-05-17T22:34:53.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:53 smithi151 ceph-mon[36101]: pgmap v301: 129 pgs: 7 creating+peering, 25 unknown, 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 610 B/s wr, 2 op/s 2024-05-17T22:34:53.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:53 smithi096 ceph-mon[36065]: osdmap e82: 12 total, 12 up, 12 in 2024-05-17T22:34:53.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:53 smithi096 ceph-mon[36065]: pgmap v301: 129 pgs: 7 creating+peering, 25 unknown, 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 610 B/s wr, 2 op/s 2024-05-17T22:34:53.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:53 smithi012 ceph-mon[31632]: osdmap e82: 12 total, 12 up, 12 in 2024-05-17T22:34:53.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:53 smithi012 ceph-mon[31632]: pgmap v301: 129 pgs: 7 creating+peering, 25 unknown, 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 610 B/s wr, 2 op/s 2024-05-17T22:34:54.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:54.311+0000 7fd0692d3700 1 -- 172.21.15.12:0/412329476 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd064108840 msgr2=0x7fd064108c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:54.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:54.311+0000 7fd0692d3700 1 --2- 172.21.15.12:0/412329476 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd064108840 0x7fd064108c50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd058009990 tx=0x7fd058009ca0 comp rx=0 tx=0).stop 2024-05-17T22:34:54.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:54.311+0000 7fd0692d3700 1 -- 172.21.15.12:0/412329476 shutdown_connections 2024-05-17T22:34:54.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:54.311+0000 7fd0692d3700 1 --2- 172.21.15.12:0/412329476 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd064109190 0x7fd064110af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:54.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:54.311+0000 7fd0692d3700 1 --2- 172.21.15.12:0/412329476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd064103f10 0x7fd064104320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:54.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:54.311+0000 7fd0692d3700 1 --2- 172.21.15.12:0/412329476 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd064108840 0x7fd064108c50 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:54.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:54.311+0000 7fd0692d3700 1 -- 172.21.15.12:0/412329476 >> 172.21.15.12:0/412329476 conn(0x7fd0640ff9b0 msgr2=0x7fd064101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:34:54.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:54.311+0000 7fd0692d3700 1 -- 172.21.15.12:0/412329476 shutdown_connections 2024-05-17T22:34:54.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:54.311+0000 7fd0692d3700 1 -- 172.21.15.12:0/412329476 wait complete. 2024-05-17T22:34:54.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:54.311+0000 7fd0692d3700 1 Processor -- start 2024-05-17T22:34:54.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:54.312+0000 7fd0692d3700 1 -- start start 2024-05-17T22:34:54.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:54.312+0000 7fd0692d3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd064103f10 0x7fd064075b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:54.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:54.312+0000 7fd0692d3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd064108840 0x7fd064076070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:54.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:54.312+0000 7fd0692d3700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd064109190 0x7fd0640765b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:54.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:54.312+0000 7fd0692d3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd064113da0 con 0x7fd064108840 2024-05-17T22:34:54.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:54.312+0000 7fd0692d3700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd064113c20 con 0x7fd064103f10 2024-05-17T22:34:54.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:54.312+0000 7fd0692d3700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd064113f20 con 0x7fd064109190 2024-05-17T22:34:54.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:54.312+0000 7fd0627fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd064108840 0x7fd064076070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:54.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:54.312+0000 7fd0627fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd064108840 0x7fd064076070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53158/0 (socket says 172.21.15.12:53158) 2024-05-17T22:34:54.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:54.312+0000 7fd0627fc700 1 -- 172.21.15.12:0/2259751048 learned_addr learned my addr 172.21.15.12:0/2259751048 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:34:54.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:54.312+0000 7fd062ffd700 1 --2- 172.21.15.12:0/2259751048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd064103f10 0x7fd064075b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:54.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:54.312+0000 7fd0637fe700 1 --2- 172.21.15.12:0/2259751048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd064109190 0x7fd0640765b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:54.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:54.313+0000 7fd0627fc700 1 -- 172.21.15.12:0/2259751048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd064109190 msgr2=0x7fd0640765b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:54.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:54.313+0000 7fd0627fc700 1 --2- 172.21.15.12:0/2259751048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd064109190 0x7fd0640765b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:54.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:54.313+0000 7fd0627fc700 1 -- 172.21.15.12:0/2259751048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd064103f10 msgr2=0x7fd064075b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:54.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:54.313+0000 7fd0627fc700 1 --2- 172.21.15.12:0/2259751048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd064103f10 0x7fd064075b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:54.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:54.313+0000 7fd0627fc700 1 -- 172.21.15.12:0/2259751048 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0641a8bc0 con 0x7fd064108840 2024-05-17T22:34:54.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:54.313+0000 7fd062ffd700 1 --2- 172.21.15.12:0/2259751048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd064103f10 0x7fd064075b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:34:54.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:54.313+0000 7fd0627fc700 1 --2- 172.21.15.12:0/2259751048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd064108840 0x7fd064076070 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd058000c00 tx=0x7fd0580047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:34:54.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:54.313+0000 7fd04bfff700 1 -- 172.21.15.12:0/2259751048 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd058027070 con 0x7fd064108840 2024-05-17T22:34:54.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:54.313+0000 7fd04bfff700 1 -- 172.21.15.12:0/2259751048 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0580041d0 con 0x7fd064108840 2024-05-17T22:34:54.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:54.313+0000 7fd0692d3700 1 -- 172.21.15.12:0/2259751048 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0641a8e50 con 0x7fd064108840 2024-05-17T22:34:54.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:54.313+0000 7fd04bfff700 1 -- 172.21.15.12:0/2259751048 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05800f460 con 0x7fd064108840 2024-05-17T22:34:54.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:54.313+0000 7fd0692d3700 1 -- 172.21.15.12:0/2259751048 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0641a9250 con 0x7fd064108840 2024-05-17T22:34:54.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:54.314+0000 7fd0692d3700 1 -- 172.21.15.12:0/2259751048 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd064104320 con 0x7fd064108840 2024-05-17T22:34:54.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:54.316+0000 7fd04bfff700 1 -- 172.21.15.12:0/2259751048 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fd05802a070 con 0x7fd064108840 2024-05-17T22:34:54.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:54.317+0000 7fd04bfff700 1 --2- 172.21.15.12:0/2259751048 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd04c0b2900 0x7fd04c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:54.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:54.317+0000 7fd062ffd700 1 --2- 172.21.15.12:0/2259751048 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd04c0b2900 0x7fd04c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:54.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:54.317+0000 7fd04bfff700 1 -- 172.21.15.12:0/2259751048 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8770+0+0 (secure 0 0 0) 0x7fd0580e38b0 con 0x7fd064108840 2024-05-17T22:34:54.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:54.317+0000 7fd062ffd700 1 --2- 172.21.15.12:0/2259751048 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd04c0b2900 0x7fd04c0b4e10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd054005950 tx=0x7fd054005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:34:54.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:54.320+0000 7fd04bfff700 1 -- 172.21.15.12:0/2259751048 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd058031080 con 0x7fd064108840 2024-05-17T22:34:54.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:54.536+0000 7fd0692d3700 1 -- 172.21.15.12:0/2259751048 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fd064064440 con 0x7fd064108840 2024-05-17T22:34:54.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:54 smithi151 ceph-mon[36101]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-17T22:34:54.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:54 smithi151 ceph-mon[36101]: osdmap e83: 12 total, 12 up, 12 in 2024-05-17T22:34:54.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:54 smithi096 ceph-mon[36065]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-17T22:34:54.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:54 smithi096 ceph-mon[36065]: osdmap e83: 12 total, 12 up, 12 in 2024-05-17T22:34:54.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:54 smithi012 ceph-mon[31632]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-17T22:34:54.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:54 smithi012 ceph-mon[31632]: osdmap e83: 12 total, 12 up, 12 in 2024-05-17T22:34:55.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:55.494+0000 7fd04bfff700 1 -- 172.21.15.12:0/2259751048 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v84) ==== 169+0+0 (secure 0 0 0) 0x7fd058013070 con 0x7fd064108840 2024-05-17T22:34:55.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:55.498+0000 7fd0692d3700 1 -- 172.21.15.12:0/2259751048 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd04c0b2900 msgr2=0x7fd04c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:55.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:55.498+0000 7fd0692d3700 1 --2- 172.21.15.12:0/2259751048 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd04c0b2900 0x7fd04c0b4e10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd054005950 tx=0x7fd054005f50 comp rx=0 tx=0).stop 2024-05-17T22:34:55.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:55.498+0000 7fd0692d3700 1 -- 172.21.15.12:0/2259751048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd064108840 msgr2=0x7fd064076070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:55.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:55.498+0000 7fd0692d3700 1 --2- 172.21.15.12:0/2259751048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd064108840 0x7fd064076070 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd058000c00 tx=0x7fd0580047b0 comp rx=0 tx=0).stop 2024-05-17T22:34:55.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:55.498+0000 7fd0692d3700 1 -- 172.21.15.12:0/2259751048 shutdown_connections 2024-05-17T22:34:55.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:55.498+0000 7fd0692d3700 1 --2- 172.21.15.12:0/2259751048 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd04c0b2900 0x7fd04c0b4e10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:55.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:55.498+0000 7fd0692d3700 1 --2- 172.21.15.12:0/2259751048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd064109190 0x7fd0640765b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:55.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:55.498+0000 7fd0692d3700 1 --2- 172.21.15.12:0/2259751048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd064103f10 0x7fd064075b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:55.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:55.498+0000 7fd0692d3700 1 --2- 172.21.15.12:0/2259751048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd064108840 0x7fd064076070 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:55.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:55.498+0000 7fd0692d3700 1 -- 172.21.15.12:0/2259751048 >> 172.21.15.12:0/2259751048 conn(0x7fd0640ff9b0 msgr2=0x7fd064101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:34:55.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:55.499+0000 7fd0692d3700 1 -- 172.21.15.12:0/2259751048 shutdown_connections 2024-05-17T22:34:55.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:55.499+0000 7fd0692d3700 1 -- 172.21.15.12:0/2259751048 wait complete. 2024-05-17T22:34:55.502 INFO:teuthology.orchestra.run.smithi012.stderr:set pool 4 allow_ec_overwrites to true 2024-05-17T22:34:55.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:55 smithi096 ceph-mon[36065]: pgmap v303: 129 pgs: 7 creating+peering, 1 unknown, 121 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:55.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:55 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/2259751048' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-17T22:34:55.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:55 smithi012 ceph-mon[31632]: pgmap v303: 129 pgs: 7 creating+peering, 1 unknown, 121 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:55.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:55 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/2259751048' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-17T22:34:55.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:55 smithi151 ceph-mon[36101]: pgmap v303: 129 pgs: 7 creating+peering, 1 unknown, 121 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:55.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:55 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/2259751048' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-17T22:34:56.289 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-05-17T22:34:56.550 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:34:56.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:56 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/2259751048' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-17T22:34:56.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:56 smithi012 ceph-mon[31632]: osdmap e84: 12 total, 12 up, 12 in 2024-05-17T22:34:56.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:56 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/2259751048' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-17T22:34:56.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:56 smithi096 ceph-mon[36065]: osdmap e84: 12 total, 12 up, 12 in 2024-05-17T22:34:56.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:56 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/2259751048' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-17T22:34:56.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:56 smithi151 ceph-mon[36101]: osdmap e84: 12 total, 12 up, 12 in 2024-05-17T22:34:57.798 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:57 smithi012 ceph-mon[31632]: pgmap v305: 129 pgs: 7 creating+peering, 122 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:57.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:57 smithi096 ceph-mon[36065]: pgmap v305: 129 pgs: 7 creating+peering, 122 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:57.892 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:57 smithi151 ceph-mon[36101]: pgmap v305: 129 pgs: 7 creating+peering, 122 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:58.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:58.336+0000 7f4e8df85700 1 -- 172.21.15.12:0/54087043 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e88108840 msgr2=0x7f4e88108c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:58.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:58.336+0000 7f4e8df85700 1 --2- 172.21.15.12:0/54087043 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e88108840 0x7f4e88108c50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c009990 tx=0x7f4e7c009ca0 comp rx=0 tx=0).stop 2024-05-17T22:34:58.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:58.336+0000 7f4e8df85700 1 -- 172.21.15.12:0/54087043 shutdown_connections 2024-05-17T22:34:58.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:58.336+0000 7f4e8df85700 1 --2- 172.21.15.12:0/54087043 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e88109190 0x7f4e88110af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:58.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:58.336+0000 7f4e8df85700 1 --2- 172.21.15.12:0/54087043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e88103f10 0x7f4e88104320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:58.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:58.336+0000 7f4e8df85700 1 --2- 172.21.15.12:0/54087043 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e88108840 0x7f4e88108c50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:58.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:58.336+0000 7f4e8df85700 1 -- 172.21.15.12:0/54087043 >> 172.21.15.12:0/54087043 conn(0x7f4e880ff990 msgr2=0x7f4e88101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:34:58.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:58.336+0000 7f4e8df85700 1 -- 172.21.15.12:0/54087043 shutdown_connections 2024-05-17T22:34:58.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:58.336+0000 7f4e8df85700 1 -- 172.21.15.12:0/54087043 wait complete. 2024-05-17T22:34:58.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:58.337+0000 7f4e8df85700 1 Processor -- start 2024-05-17T22:34:58.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:58.337+0000 7f4e8df85700 1 -- start start 2024-05-17T22:34:58.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:58.337+0000 7f4e8df85700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e88103f10 0x7f4e8819a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:58.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:58.337+0000 7f4e8df85700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e88108840 0x7f4e8819ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:58.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:58.337+0000 7f4e8df85700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e88109190 0x7f4e8819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:58.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:58.337+0000 7f4e8df85700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4e88113da0 con 0x7f4e88109190 2024-05-17T22:34:58.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:58.337+0000 7f4e8df85700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4e88113c20 con 0x7f4e88108840 2024-05-17T22:34:58.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:58.337+0000 7f4e8df85700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4e88113f20 con 0x7f4e88103f10 2024-05-17T22:34:58.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:58.337+0000 7f4e87fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e88109190 0x7f4e8819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:58.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:58.337+0000 7f4e87fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e88109190 0x7f4e8819ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53176/0 (socket says 172.21.15.12:53176) 2024-05-17T22:34:58.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:58.337+0000 7f4e87fff700 1 -- 172.21.15.12:0/2031811533 learned_addr learned my addr 172.21.15.12:0/2031811533 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:34:58.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:58.337+0000 7f4e86ffd700 1 --2- 172.21.15.12:0/2031811533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e88108840 0x7f4e8819ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:58.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:58.337+0000 7f4e877fe700 1 --2- 172.21.15.12:0/2031811533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e88103f10 0x7f4e8819a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:58.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:58.338+0000 7f4e87fff700 1 -- 172.21.15.12:0/2031811533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e88103f10 msgr2=0x7f4e8819a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:58.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:58.338+0000 7f4e87fff700 1 --2- 172.21.15.12:0/2031811533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e88103f10 0x7f4e8819a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:58.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:58.338+0000 7f4e87fff700 1 -- 172.21.15.12:0/2031811533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e88108840 msgr2=0x7f4e8819ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:58.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:58.338+0000 7f4e87fff700 1 --2- 172.21.15.12:0/2031811533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e88108840 0x7f4e8819ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:58.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:58.338+0000 7f4e87fff700 1 -- 172.21.15.12:0/2031811533 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e8819f580 con 0x7f4e88109190 2024-05-17T22:34:58.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:58.338+0000 7f4e877fe700 1 --2- 172.21.15.12:0/2031811533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e88103f10 0x7f4e8819a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:34:58.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:58.338+0000 7f4e87fff700 1 --2- 172.21.15.12:0/2031811533 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e88109190 0x7f4e8819ef90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4e8000c900 tx=0x7f4e8000cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:34:58.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:58.338+0000 7f4e84ff9700 1 -- 172.21.15.12:0/2031811533 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e80007b00 con 0x7f4e88109190 2024-05-17T22:34:58.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:58.338+0000 7f4e8df85700 1 -- 172.21.15.12:0/2031811533 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e8819f870 con 0x7f4e88109190 2024-05-17T22:34:58.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:58.338+0000 7f4e84ff9700 1 -- 172.21.15.12:0/2031811533 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e8000ce50 con 0x7f4e88109190 2024-05-17T22:34:58.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:58.338+0000 7f4e84ff9700 1 -- 172.21.15.12:0/2031811533 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e80018660 con 0x7f4e88109190 2024-05-17T22:34:58.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:58.338+0000 7f4e8df85700 1 -- 172.21.15.12:0/2031811533 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e8819fd80 con 0x7f4e88109190 2024-05-17T22:34:58.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:58.339+0000 7f4e8df85700 1 -- 172.21.15.12:0/2031811533 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e88104320 con 0x7f4e88109190 2024-05-17T22:34:58.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:58.341+0000 7f4e84ff9700 1 -- 172.21.15.12:0/2031811533 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f4e8000f450 con 0x7f4e88109190 2024-05-17T22:34:58.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:58.342+0000 7f4e84ff9700 1 --2- 172.21.15.12:0/2031811533 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f4e700b2900 0x7f4e700b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:34:58.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:58.342+0000 7f4e877fe700 1 --2- 172.21.15.12:0/2031811533 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f4e700b2900 0x7f4e700b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:34:58.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:58.342+0000 7f4e84ff9700 1 -- 172.21.15.12:0/2031811533 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8770+0+0 (secure 0 0 0) 0x7f4e800e22e0 con 0x7f4e88109190 2024-05-17T22:34:58.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:58.343+0000 7f4e877fe700 1 --2- 172.21.15.12:0/2031811533 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f4e700b2900 0x7f4e700b4e10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4e7800e300 tx=0x7f4e78009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:34:58.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:58.345+0000 7f4e84ff9700 1 -- 172.21.15.12:0/2031811533 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4e80014030 con 0x7f4e88109190 2024-05-17T22:34:58.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:58.611+0000 7f4e8df85700 1 -- 172.21.15.12:0/2031811533 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f4e8819bac0 con 0x7f4e88109190 2024-05-17T22:34:59.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:59.511+0000 7f4e84ff9700 1 -- 172.21.15.12:0/2031811533 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f4e800a9e90 con 0x7f4e88109190 2024-05-17T22:34:59.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:59.516+0000 7f4e8df85700 1 -- 172.21.15.12:0/2031811533 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f4e700b2900 msgr2=0x7f4e700b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:59.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:59.516+0000 7f4e8df85700 1 --2- 172.21.15.12:0/2031811533 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f4e700b2900 0x7f4e700b4e10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4e7800e300 tx=0x7f4e78009040 comp rx=0 tx=0).stop 2024-05-17T22:34:59.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:59.516+0000 7f4e8df85700 1 -- 172.21.15.12:0/2031811533 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e88109190 msgr2=0x7f4e8819ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:34:59.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:59.516+0000 7f4e8df85700 1 --2- 172.21.15.12:0/2031811533 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e88109190 0x7f4e8819ef90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4e8000c900 tx=0x7f4e8000cc10 comp rx=0 tx=0).stop 2024-05-17T22:34:59.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:59.516+0000 7f4e8df85700 1 -- 172.21.15.12:0/2031811533 shutdown_connections 2024-05-17T22:34:59.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:59.516+0000 7f4e8df85700 1 --2- 172.21.15.12:0/2031811533 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f4e700b2900 0x7f4e700b4e10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:59.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:59.516+0000 7f4e8df85700 1 --2- 172.21.15.12:0/2031811533 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e88109190 0x7f4e8819ef90 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:59.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:59.516+0000 7f4e8df85700 1 --2- 172.21.15.12:0/2031811533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e88103f10 0x7f4e8819a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:59.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:59.516+0000 7f4e8df85700 1 --2- 172.21.15.12:0/2031811533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e88108840 0x7f4e8819ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:34:59.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:59.516+0000 7f4e8df85700 1 -- 172.21.15.12:0/2031811533 >> 172.21.15.12:0/2031811533 conn(0x7f4e880ff990 msgr2=0x7f4e8810a290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:34:59.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:59.516+0000 7f4e8df85700 1 -- 172.21.15.12:0/2031811533 shutdown_connections 2024-05-17T22:34:59.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:34:59.516+0000 7f4e8df85700 1 -- 172.21.15.12:0/2031811533 wait complete. 2024-05-17T22:34:59.520 INFO:teuthology.orchestra.run.smithi012.stderr:added data pool 4 to fsmap 2024-05-17T22:34:59.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:59 smithi096 ceph-mon[36065]: pgmap v306: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:59.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:34:59 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/2031811533' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-17T22:34:59.831 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:59 smithi012 ceph-mon[31632]: pgmap v306: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:59.831 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:34:59 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/2031811533' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-17T22:34:59.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:59 smithi151 ceph-mon[36101]: pgmap v306: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:34:59.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:34:59 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/2031811533' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-17T22:35:00.201 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T22:35:00.461 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:35:00.795 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:00 smithi012 ceph-mon[31632]: osdmap e85: 12 total, 12 up, 12 in 2024-05-17T22:35:00.795 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:00 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/2031811533' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-17T22:35:00.795 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:00 smithi012 ceph-mon[31632]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T22:35:00.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:00 smithi096 ceph-mon[36065]: osdmap e85: 12 total, 12 up, 12 in 2024-05-17T22:35:00.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:00 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/2031811533' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-17T22:35:00.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:00 smithi096 ceph-mon[36065]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T22:35:00.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:00 smithi151 ceph-mon[36101]: osdmap e85: 12 total, 12 up, 12 in 2024-05-17T22:35:00.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:00 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/2031811533' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-17T22:35:00.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:00 smithi151 ceph-mon[36101]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T22:35:01.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:01 smithi012 ceph-mon[31632]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:35:01.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:01 smithi012 ceph-mon[31632]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-17T22:35:01.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:01 smithi012 ceph-mon[31632]: Cluster is now healthy 2024-05-17T22:35:01.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:01 smithi096 ceph-mon[36065]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:35:01.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:01 smithi096 ceph-mon[36065]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-17T22:35:01.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:01 smithi096 ceph-mon[36065]: Cluster is now healthy 2024-05-17T22:35:01.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:01 smithi151 ceph-mon[36101]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:35:01.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:01 smithi151 ceph-mon[36101]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-17T22:35:01.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:01 smithi151 ceph-mon[36101]: Cluster is now healthy 2024-05-17T22:35:02.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.238+0000 7fbcba849700 1 -- 172.21.15.12:0/1403462639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcb4104e10 msgr2=0x7fbcb4105220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:02.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.238+0000 7fbcba849700 1 --2- 172.21.15.12:0/1403462639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcb4104e10 0x7fbcb4105220 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbca4009990 tx=0x7fbca4009ca0 comp rx=0 tx=0).stop 2024-05-17T22:35:02.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.238+0000 7fbcba849700 1 -- 172.21.15.12:0/1403462639 shutdown_connections 2024-05-17T22:35:02.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.238+0000 7fbcba849700 1 --2- 172.21.15.12:0/1403462639 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbcb410e240 0x7fbcb4110750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:02.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.238+0000 7fbcba849700 1 --2- 172.21.15.12:0/1403462639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcb4104e10 0x7fbcb4105220 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:02.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.238+0000 7fbcba849700 1 --2- 172.21.15.12:0/1403462639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbcb4105760 0x7fbcb410dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:02.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.238+0000 7fbcba849700 1 -- 172.21.15.12:0/1403462639 >> 172.21.15.12:0/1403462639 conn(0x7fbcb4079ae0 msgr2=0x7fbcb4079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:35:02.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.238+0000 7fbcba849700 1 -- 172.21.15.12:0/1403462639 shutdown_connections 2024-05-17T22:35:02.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.238+0000 7fbcba849700 1 -- 172.21.15.12:0/1403462639 wait complete. 2024-05-17T22:35:02.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.239+0000 7fbcba849700 1 Processor -- start 2024-05-17T22:35:02.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.239+0000 7fbcba849700 1 -- start start 2024-05-17T22:35:02.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.240+0000 7fbcba849700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcb4104e10 0x7fbcb419a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:02.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.240+0000 7fbcba849700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbcb4105760 0x7fbcb419ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:02.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.240+0000 7fbcba849700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbcb410e240 0x7fbcb419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:02.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.240+0000 7fbcba849700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb4113dc0 con 0x7fbcb4104e10 2024-05-17T22:35:02.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.240+0000 7fbcba849700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb4113c40 con 0x7fbcb4105760 2024-05-17T22:35:02.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.240+0000 7fbcba849700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb4113f40 con 0x7fbcb410e240 2024-05-17T22:35:02.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.240+0000 7fbcb3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcb4104e10 0x7fbcb419a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:02.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.240+0000 7fbcb3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcb4104e10 0x7fbcb419a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49718/0 (socket says 172.21.15.12:49718) 2024-05-17T22:35:02.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.240+0000 7fbcb3fff700 1 -- 172.21.15.12:0/2568834490 learned_addr learned my addr 172.21.15.12:0/2568834490 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:35:02.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.240+0000 7fbcb8de6700 1 --2- 172.21.15.12:0/2568834490 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbcb410e240 0x7fbcb419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:02.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.240+0000 7fbcb37fe700 1 --2- 172.21.15.12:0/2568834490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbcb4105760 0x7fbcb419ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:02.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.240+0000 7fbcb3fff700 1 -- 172.21.15.12:0/2568834490 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbcb410e240 msgr2=0x7fbcb419efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:02.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.240+0000 7fbcb3fff700 1 --2- 172.21.15.12:0/2568834490 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbcb410e240 0x7fbcb419efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:02.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.240+0000 7fbcb3fff700 1 -- 172.21.15.12:0/2568834490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbcb4105760 msgr2=0x7fbcb419ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:02.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.240+0000 7fbcb3fff700 1 --2- 172.21.15.12:0/2568834490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbcb4105760 0x7fbcb419ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:02.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.240+0000 7fbcb3fff700 1 -- 172.21.15.12:0/2568834490 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcb419f5c0 con 0x7fbcb4104e10 2024-05-17T22:35:02.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.241+0000 7fbcb3fff700 1 --2- 172.21.15.12:0/2568834490 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcb4104e10 0x7fbcb419a7a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbca4000c00 tx=0x7fbca40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:35:02.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.241+0000 7fbcb17fa700 1 -- 172.21.15.12:0/2568834490 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca4027070 con 0x7fbcb4104e10 2024-05-17T22:35:02.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.241+0000 7fbcb17fa700 1 -- 172.21.15.12:0/2568834490 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbca4003f20 con 0x7fbcb4104e10 2024-05-17T22:35:02.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.241+0000 7fbcb17fa700 1 -- 172.21.15.12:0/2568834490 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca4022ca0 con 0x7fbcb4104e10 2024-05-17T22:35:02.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.241+0000 7fbcba849700 1 -- 172.21.15.12:0/2568834490 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcb419f850 con 0x7fbcb4104e10 2024-05-17T22:35:02.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.241+0000 7fbcba849700 1 -- 172.21.15.12:0/2568834490 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcb419fce0 con 0x7fbcb4104e10 2024-05-17T22:35:02.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.242+0000 7fbcba849700 1 -- 172.21.15.12:0/2568834490 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcb4105220 con 0x7fbcb4104e10 2024-05-17T22:35:02.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.244+0000 7fbcb17fa700 1 -- 172.21.15.12:0/2568834490 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fbca402a070 con 0x7fbcb4104e10 2024-05-17T22:35:02.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.245+0000 7fbcb17fa700 1 --2- 172.21.15.12:0/2568834490 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fbc9c0b2900 0x7fbc9c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:02.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.245+0000 7fbcb37fe700 1 --2- 172.21.15.12:0/2568834490 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fbc9c0b2900 0x7fbc9c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:02.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.245+0000 7fbcb17fa700 1 -- 172.21.15.12:0/2568834490 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7fbca40e3700 con 0x7fbcb4104e10 2024-05-17T22:35:02.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.246+0000 7fbcb37fe700 1 --2- 172.21.15.12:0/2568834490 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fbc9c0b2900 0x7fbc9c0b4e10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4102300 tx=0x7fbca800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:35:02.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.249+0000 7fbcb17fa700 1 -- 172.21.15.12:0/2568834490 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbca4013070 con 0x7fbcb4104e10 2024-05-17T22:35:02.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.516+0000 7fbcba849700 1 -- 172.21.15.12:0/2568834490 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbcb419bb00 con 0x7fbcb4104e10 2024-05-17T22:35:02.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.517+0000 7fbcb17fa700 1 -- 172.21.15.12:0/2568834490 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11797 (secure 0 0 0) 0x7fbca40ab270 con 0x7fbcb4104e10 2024-05-17T22:35:02.521 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:35:02.521 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1407219319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1407219319},{"type":"v1","addr":"172.21.15.12:6835","nonce":1407219319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/630977886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":630977886},{"type":"v1","addr":"172.21.15.12:6837","nonce":630977886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3731914742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3731914742},{"type":"v1","addr":"172.21.15.12:6839","nonce":3731914742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/3642142501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3642142501},{"type":"v1","addr":"172.21.15.12:6841","nonce":3642142501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3554053059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3554053059},{"type":"v1","addr":"172.21.15.151:6833","nonce":3554053059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/4228325569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":4228325569},{"type":"v1","addr":"172.21.15.96:6833","nonce":4228325569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/542323754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":542323754},{"type":"v1","addr":"172.21.15.151:6835","nonce":542323754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1758442232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1758442232},{"type":"v1","addr":"172.21.15.96:6835","nonce":1758442232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/3403324038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3403324038},{"type":"v1","addr":"172.21.15.151:6837","nonce":3403324038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/2338353849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2338353849},{"type":"v1","addr":"172.21.15.151:6839","nonce":2338353849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/4222574338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":4222574338},{"type":"v1","addr":"172.21.15.96:6837","nonce":4222574338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:34:42.642111+0000","modified":"2024-05-17T22:34:58.612723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/2986295519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2986295519},{"type":"v1","addr":"172.21.15.96:6839","nonce":2986295519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-05-17T22:35:02.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.520+0000 7fbcba849700 1 -- 172.21.15.12:0/2568834490 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fbc9c0b2900 msgr2=0x7fbc9c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:02.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.520+0000 7fbcba849700 1 --2- 172.21.15.12:0/2568834490 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fbc9c0b2900 0x7fbc9c0b4e10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4102300 tx=0x7fbca800a380 comp rx=0 tx=0).stop 2024-05-17T22:35:02.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.520+0000 7fbcba849700 1 -- 172.21.15.12:0/2568834490 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcb4104e10 msgr2=0x7fbcb419a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:02.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.520+0000 7fbcba849700 1 --2- 172.21.15.12:0/2568834490 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcb4104e10 0x7fbcb419a7a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbca4000c00 tx=0x7fbca40047b0 comp rx=0 tx=0).stop 2024-05-17T22:35:02.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.520+0000 7fbcba849700 1 -- 172.21.15.12:0/2568834490 shutdown_connections 2024-05-17T22:35:02.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.520+0000 7fbcba849700 1 --2- 172.21.15.12:0/2568834490 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fbc9c0b2900 0x7fbc9c0b4e10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:02.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.520+0000 7fbcba849700 1 --2- 172.21.15.12:0/2568834490 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbcb410e240 0x7fbcb419efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:02.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.520+0000 7fbcba849700 1 --2- 172.21.15.12:0/2568834490 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcb4104e10 0x7fbcb419a7a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:02.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.520+0000 7fbcba849700 1 --2- 172.21.15.12:0/2568834490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbcb4105760 0x7fbcb419ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:02.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.520+0000 7fbcba849700 1 -- 172.21.15.12:0/2568834490 >> 172.21.15.12:0/2568834490 conn(0x7fbcb4079ae0 msgr2=0x7fbcb40ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:35:02.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.520+0000 7fbcba849700 1 -- 172.21.15.12:0/2568834490 shutdown_connections 2024-05-17T22:35:02.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:02.520+0000 7fbcba849700 1 -- 172.21.15.12:0/2568834490 wait complete. 2024-05-17T22:35:02.524 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 17 2024-05-17T22:35:03.212 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-17T22:35:03.464 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:35:03.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:03 smithi096 ceph-mon[36065]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:35:03.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:03 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/2568834490' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:35:03.822 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:03 smithi012 ceph-mon[31632]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:35:03.822 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:03 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/2568834490' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:35:03.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:03 smithi151 ceph-mon[36101]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:35:03.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:03 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/2568834490' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:35:05.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.214+0000 7f1ccb4f0700 1 -- 172.21.15.12:0/3074130097 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cc4110540 msgr2=0x7f1cc4112960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:05.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.214+0000 7f1ccb4f0700 1 --2- 172.21.15.12:0/3074130097 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cc4110540 0x7f1cc4112960 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc00b0a0 tx=0x7f1cbc00b3b0 comp rx=0 tx=0).stop 2024-05-17T22:35:05.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.214+0000 7f1ccb4f0700 1 -- 172.21.15.12:0/3074130097 shutdown_connections 2024-05-17T22:35:05.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.214+0000 7f1ccb4f0700 1 --2- 172.21.15.12:0/3074130097 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cc4110540 0x7f1cc4112960 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:05.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.214+0000 7f1ccb4f0700 1 --2- 172.21.15.12:0/3074130097 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1cc4075b50 0x7f1cc4075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:05.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.214+0000 7f1ccb4f0700 1 --2- 172.21.15.12:0/3074130097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1cc40764a0 0x7f1cc4110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:05.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.214+0000 7f1ccb4f0700 1 -- 172.21.15.12:0/3074130097 >> 172.21.15.12:0/3074130097 conn(0x7f1cc40ff9b0 msgr2=0x7f1cc4101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:35:05.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.214+0000 7f1ccb4f0700 1 -- 172.21.15.12:0/3074130097 shutdown_connections 2024-05-17T22:35:05.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.214+0000 7f1ccb4f0700 1 -- 172.21.15.12:0/3074130097 wait complete. 2024-05-17T22:35:05.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.215+0000 7f1ccb4f0700 1 Processor -- start 2024-05-17T22:35:05.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.215+0000 7f1ccb4f0700 1 -- start start 2024-05-17T22:35:05.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.215+0000 7f1ccb4f0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cc4075b50 0x7f1cc41a25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:05.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.215+0000 7f1ccb4f0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1cc40764a0 0x7f1cc41a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:05.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.215+0000 7f1ccb4f0700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1cc4110540 0x7f1cc419c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:05.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.215+0000 7f1ccb4f0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc4116060 con 0x7f1cc4075b50 2024-05-17T22:35:05.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.215+0000 7f1ccb4f0700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc4115ee0 con 0x7f1cc40764a0 2024-05-17T22:35:05.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.215+0000 7f1ccb4f0700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41161e0 con 0x7f1cc4110540 2024-05-17T22:35:05.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.215+0000 7f1cc928c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cc4075b50 0x7f1cc41a25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:05.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.215+0000 7f1cc928c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cc4075b50 0x7f1cc41a25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49732/0 (socket says 172.21.15.12:49732) 2024-05-17T22:35:05.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.215+0000 7f1cc928c700 1 -- 172.21.15.12:0/169767002 learned_addr learned my addr 172.21.15.12:0/169767002 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:35:05.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.215+0000 7f1cc9a8d700 1 --2- 172.21.15.12:0/169767002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1cc4110540 0x7f1cc419c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:05.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.215+0000 7f1cc8a8b700 1 --2- 172.21.15.12:0/169767002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1cc40764a0 0x7f1cc41a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:05.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.216+0000 7f1cc928c700 1 -- 172.21.15.12:0/169767002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1cc4110540 msgr2=0x7f1cc419c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:05.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.216+0000 7f1cc928c700 1 --2- 172.21.15.12:0/169767002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1cc4110540 0x7f1cc419c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:05.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.216+0000 7f1cc928c700 1 -- 172.21.15.12:0/169767002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1cc40764a0 msgr2=0x7f1cc41a2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:05.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.216+0000 7f1cc928c700 1 --2- 172.21.15.12:0/169767002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1cc40764a0 0x7f1cc41a2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:05.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.216+0000 7f1cc928c700 1 -- 172.21.15.12:0/169767002 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc419cf60 con 0x7f1cc4075b50 2024-05-17T22:35:05.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.216+0000 7f1cc9a8d700 1 --2- 172.21.15.12:0/169767002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1cc4110540 0x7f1cc419c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:35:05.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.216+0000 7f1cc928c700 1 --2- 172.21.15.12:0/169767002 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cc4075b50 0x7f1cc41a25f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1cc000c920 tx=0x7f1cc000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:35:05.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.216+0000 7f1cba7fc700 1 -- 172.21.15.12:0/169767002 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0007b00 con 0x7f1cc4075b50 2024-05-17T22:35:05.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.216+0000 7f1cba7fc700 1 -- 172.21.15.12:0/169767002 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cc0004ce0 con 0x7f1cc4075b50 2024-05-17T22:35:05.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.216+0000 7f1ccb4f0700 1 -- 172.21.15.12:0/169767002 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cc419d250 con 0x7f1cc4075b50 2024-05-17T22:35:05.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.216+0000 7f1cba7fc700 1 -- 172.21.15.12:0/169767002 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0005650 con 0x7f1cc4075b50 2024-05-17T22:35:05.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.216+0000 7f1ccb4f0700 1 -- 172.21.15.12:0/169767002 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cc41a9450 con 0x7f1cc4075b50 2024-05-17T22:35:05.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.217+0000 7f1ccb4f0700 1 -- 172.21.15.12:0/169767002 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cc4077290 con 0x7f1cc4075b50 2024-05-17T22:35:05.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.220+0000 7f1cba7fc700 1 -- 172.21.15.12:0/169767002 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f1cc0004750 con 0x7f1cc4075b50 2024-05-17T22:35:05.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.221+0000 7f1cba7fc700 1 --2- 172.21.15.12:0/169767002 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f1cb00b2880 0x7f1cb00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:05.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.221+0000 7f1cc8a8b700 1 --2- 172.21.15.12:0/169767002 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f1cb00b2880 0x7f1cb00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:05.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.221+0000 7f1cba7fc700 1 -- 172.21.15.12:0/169767002 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f1cc00e2500 con 0x7f1cc4075b50 2024-05-17T22:35:05.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.221+0000 7f1cc8a8b700 1 --2- 172.21.15.12:0/169767002 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f1cb00b2880 0x7f1cb00b4d90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4006fd0 tx=0x7f1cb4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:35:05.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.224+0000 7f1cba7fc700 1 -- 172.21.15.12:0/169767002 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1cc0014030 con 0x7f1cc4075b50 2024-05-17T22:35:05.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.439+0000 7f1ccb4f0700 1 -- 172.21.15.12:0/169767002 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1cc419dd40 con 0x7f1cc4075b50 2024-05-17T22:35:05.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.441+0000 7f1cba7fc700 1 -- 172.21.15.12:0/169767002 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24094 (secure 0 0 0) 0x7f1cc00a9f80 con 0x7f1cc4075b50 2024-05-17T22:35:05.444 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:35:05.444 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":85,"fsid":"4d593780-149c-11ef-bc9a-c7b262605968","created":"2024-05-17T22:26:16.955626+0000","modified":"2024-05-17T22:34:58.612542+0000","last_up_change":"2024-05-17T22:33:36.551093+0000","last_in_change":"2024-05-17T22:33:15.032677+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T22:29:42.475513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T22:34:35.504697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T22:34:39.377097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-17T22:34:50.467684+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5d42798f-dc5f-4e66-bce2-ec33dc789e4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6803","nonce":2015605593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6805","nonce":2015605593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6809","nonce":2015605593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6807","nonce":2015605593}]},"public_addr":"172.21.15.12:6803/2015605593","cluster_addr":"172.21.15.12:6805/2015605593","heartbeat_back_addr":"172.21.15.12:6809/2015605593","heartbeat_front_addr":"172.21.15.12:6807/2015605593","state":["exists","up"]},{"osd":1,"uuid":"58afecc7-119b-4c16-a401-640fb77d4714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6801","nonce":1238316358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6803","nonce":1238316358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6807","nonce":1238316358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6805","nonce":1238316358}]},"public_addr":"172.21.15.96:6801/1238316358","cluster_addr":"172.21.15.96:6803/1238316358","heartbeat_back_addr":"172.21.15.96:6807/1238316358","heartbeat_front_addr":"172.21.15.96:6805/1238316358","state":["exists","up"]},{"osd":2,"uuid":"4bef4049-482f-49fb-82fb-38052400eb2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6801","nonce":148729815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6803","nonce":148729815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6807","nonce":148729815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6805","nonce":148729815}]},"public_addr":"172.21.15.151:6801/148729815","cluster_addr":"172.21.15.151:6803/148729815","heartbeat_back_addr":"172.21.15.151:6807/148729815","heartbeat_front_addr":"172.21.15.151:6805/148729815","state":["exists","up"]},{"osd":3,"uuid":"6fed8e36-82a2-49e4-82a3-16e6d9de87f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6811","nonce":3563309320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6813","nonce":3563309320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6817","nonce":3563309320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6815","nonce":3563309320}]},"public_addr":"172.21.15.12:6811/3563309320","cluster_addr":"172.21.15.12:6813/3563309320","heartbeat_back_addr":"172.21.15.12:6817/3563309320","heartbeat_front_addr":"172.21.15.12:6815/3563309320","state":["exists","up"]},{"osd":4,"uuid":"c0e8689a-2816-4089-ade3-931960bae4f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6809","nonce":2219103166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6811","nonce":2219103166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6815","nonce":2219103166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6813","nonce":2219103166}]},"public_addr":"172.21.15.96:6809/2219103166","cluster_addr":"172.21.15.96:6811/2219103166","heartbeat_back_addr":"172.21.15.96:6815/2219103166","heartbeat_front_addr":"172.21.15.96:6813/2219103166","state":["exists","up"]},{"osd":5,"uuid":"e76ba432-94ff-4998-aaba-2645d9e6055d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6809","nonce":2589835078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6811","nonce":2589835078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6815","nonce":2589835078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6813","nonce":2589835078}]},"public_addr":"172.21.15.151:6809/2589835078","cluster_addr":"172.21.15.151:6811/2589835078","heartbeat_back_addr":"172.21.15.151:6815/2589835078","heartbeat_front_addr":"172.21.15.151:6813/2589835078","state":["exists","up"]},{"osd":6,"uuid":"b7ece35c-efd1-40df-95da-05a0cc147372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6819","nonce":3112661724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6821","nonce":3112661724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6825","nonce":3112661724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6823","nonce":3112661724}]},"public_addr":"172.21.15.12:6819/3112661724","cluster_addr":"172.21.15.12:6821/3112661724","heartbeat_back_addr":"172.21.15.12:6825/3112661724","heartbeat_front_addr":"172.21.15.12:6823/3112661724","state":["exists","up"]},{"osd":7,"uuid":"6a65bf30-daf5-4fba-b67e-8e513d88a971","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6817","nonce":2449989110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6819","nonce":2449989110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6823","nonce":2449989110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6821","nonce":2449989110}]},"public_addr":"172.21.15.96:6817/2449989110","cluster_addr":"172.21.15.96:6819/2449989110","heartbeat_back_addr":"172.21.15.96:6823/2449989110","heartbeat_front_addr":"172.21.15.96:6821/2449989110","state":["exists","up"]},{"osd":8,"uuid":"ad62ea44-e416-4a3e-bbbe-93f2a710dbc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6817","nonce":2545224628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6819","nonce":2545224628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6823","nonce":2545224628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6821","nonce":2545224628}]},"public_addr":"172.21.15.151:6817/2545224628","cluster_addr":"172.21.15.151:6819/2545224628","heartbeat_back_addr":"172.21.15.151:6823/2545224628","heartbeat_front_addr":"172.21.15.151:6821/2545224628","state":["exists","up"]},{"osd":9,"uuid":"1275de08-2a3c-48f3-920b-dd68b4024d42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6827","nonce":2100661273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6829","nonce":2100661273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6833","nonce":2100661273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6831","nonce":2100661273}]},"public_addr":"172.21.15.12:6827/2100661273","cluster_addr":"172.21.15.12:6829/2100661273","heartbeat_back_addr":"172.21.15.12:6833/2100661273","heartbeat_front_addr":"172.21.15.12:6831/2100661273","state":["exists","up"]},{"osd":10,"uuid":"347e4e40-9a9d-43fe-8f52-fdd3199a2a5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6825","nonce":304016309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6827","nonce":304016309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6831","nonce":304016309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6829","nonce":304016309}]},"public_addr":"172.21.15.96:6825/304016309","cluster_addr":"172.21.15.96:6827/304016309","heartbeat_back_addr":"172.21.15.96:6831/304016309","heartbeat_front_addr":"172.21.15.96:6829/304016309","state":["exists","up"]},{"osd":11,"uuid":"ee8d3502-8b3c-4e6f-824f-3a97e19111ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6825","nonce":3822307314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6827","nonce":3822307314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6831","nonce":3822307314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6829","nonce":3822307314}]},"public_addr":"172.21.15.151:6825/3822307314","cluster_addr":"172.21.15.151:6827/3822307314","heartbeat_back_addr":"172.21.15.151:6831/3822307314","heartbeat_front_addr":"172.21.15.151:6829/3822307314","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:28:51.542055+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:29:15.566424+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:29:39.500981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:04.797907+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:29.655650+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:55.211880+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:31:21.559462+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:31:47.154852+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:32:13.114892+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:32:40.529708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:33:06.930942+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:33:34.061783+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/2682700253":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6801/2250046521":"2024-05-18T22:27:14.372972+0000","172.21.15.12:0/4241094239":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/4172568611":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/3512529242":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6800/1039789215":"2024-05-18T22:26:45.466719+0000","172.21.15.12:6800/2250046521":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6801/1039789215":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/2043758453":"2024-05-18T22:27:14.372972+0000","172.21.15.12:0/2694593732":"2024-05-18T22:26:45.466719+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T22:35:05.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.443+0000 7f1ccb4f0700 1 -- 172.21.15.12:0/169767002 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f1cb00b2880 msgr2=0x7f1cb00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:05.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.443+0000 7f1ccb4f0700 1 --2- 172.21.15.12:0/169767002 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f1cb00b2880 0x7f1cb00b4d90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4006fd0 tx=0x7f1cb4008040 comp rx=0 tx=0).stop 2024-05-17T22:35:05.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.443+0000 7f1ccb4f0700 1 -- 172.21.15.12:0/169767002 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cc4075b50 msgr2=0x7f1cc41a25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:05.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.443+0000 7f1ccb4f0700 1 --2- 172.21.15.12:0/169767002 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cc4075b50 0x7f1cc41a25f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1cc000c920 tx=0x7f1cc000cce0 comp rx=0 tx=0).stop 2024-05-17T22:35:05.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.443+0000 7f1ccb4f0700 1 -- 172.21.15.12:0/169767002 shutdown_connections 2024-05-17T22:35:05.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.443+0000 7f1ccb4f0700 1 --2- 172.21.15.12:0/169767002 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f1cb00b2880 0x7f1cb00b4d90 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:05.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.443+0000 7f1ccb4f0700 1 --2- 172.21.15.12:0/169767002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1cc4110540 0x7f1cc419c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:05.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.443+0000 7f1ccb4f0700 1 --2- 172.21.15.12:0/169767002 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cc4075b50 0x7f1cc41a25f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:05.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.443+0000 7f1ccb4f0700 1 --2- 172.21.15.12:0/169767002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1cc40764a0 0x7f1cc41a2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:05.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.443+0000 7f1ccb4f0700 1 -- 172.21.15.12:0/169767002 >> 172.21.15.12:0/169767002 conn(0x7f1cc40ff9b0 msgr2=0x7f1cc4101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:35:05.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.443+0000 7f1ccb4f0700 1 -- 172.21.15.12:0/169767002 shutdown_connections 2024-05-17T22:35:05.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:05.443+0000 7f1ccb4f0700 1 -- 172.21.15.12:0/169767002 wait complete. 2024-05-17T22:35:05.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:05 smithi096 ceph-mon[36065]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:35:05.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:05 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/169767002' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:35:05.811 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:05 smithi012 ceph-mon[31632]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:35:05.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:05 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/169767002' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:35:05.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:05 smithi151 ceph-mon[36101]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:35:05.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:05 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/169767002' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:35:06.258 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-17T22:35:06.520 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:35:07.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:07 smithi096 ceph-mon[36065]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:35:07.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:07 smithi012 ceph-mon[31632]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:35:07.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:07 smithi151 ceph-mon[36101]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:35:08.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.204+0000 7f197b3bf700 1 -- 172.21.15.12:0/1814496089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1974108080 msgr2=0x7f197410f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:08.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.204+0000 7f197b3bf700 1 --2- 172.21.15.12:0/1814496089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1974108080 0x7f197410f9e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f196c00b0a0 tx=0x7f196c00b3b0 comp rx=0 tx=0).stop 2024-05-17T22:35:08.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.205+0000 7f197b3bf700 1 -- 172.21.15.12:0/1814496089 shutdown_connections 2024-05-17T22:35:08.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.205+0000 7f197b3bf700 1 --2- 172.21.15.12:0/1814496089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1974108080 0x7f197410f9e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:08.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.205+0000 7f197b3bf700 1 --2- 172.21.15.12:0/1814496089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1974102e00 0x7f1974103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:08.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.205+0000 7f197b3bf700 1 --2- 172.21.15.12:0/1814496089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1974107730 0x7f1974107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:08.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.205+0000 7f197b3bf700 1 -- 172.21.15.12:0/1814496089 >> 172.21.15.12:0/1814496089 conn(0x7f1974079bd0 msgr2=0x7f1974079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:35:08.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.205+0000 7f197b3bf700 1 -- 172.21.15.12:0/1814496089 shutdown_connections 2024-05-17T22:35:08.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.205+0000 7f197b3bf700 1 -- 172.21.15.12:0/1814496089 wait complete. 2024-05-17T22:35:08.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.205+0000 7f197b3bf700 1 Processor -- start 2024-05-17T22:35:08.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.205+0000 7f197b3bf700 1 -- start start 2024-05-17T22:35:08.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.206+0000 7f197b3bf700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1974102e00 0x7f19741a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:08.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.206+0000 7f197b3bf700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1974107730 0x7f19741a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:08.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.206+0000 7f197b3bf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1974108080 0x7f197419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:08.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.206+0000 7f197b3bf700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1974112d30 con 0x7f1974108080 2024-05-17T22:35:08.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.206+0000 7f197b3bf700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1974112bb0 con 0x7f1974107730 2024-05-17T22:35:08.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.206+0000 7f197b3bf700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1974112eb0 con 0x7f1974102e00 2024-05-17T22:35:08.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.206+0000 7f197995c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1974108080 0x7f197419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:08.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.206+0000 7f197995c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1974108080 0x7f197419a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49748/0 (socket says 172.21.15.12:49748) 2024-05-17T22:35:08.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.206+0000 7f197995c700 1 -- 172.21.15.12:0/1371842934 learned_addr learned my addr 172.21.15.12:0/1371842934 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:35:08.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.206+0000 7f197895a700 1 --2- 172.21.15.12:0/1371842934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1974107730 0x7f19741a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:08.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.206+0000 7f197915b700 1 --2- 172.21.15.12:0/1371842934 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1974102e00 0x7f19741a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:08.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.206+0000 7f197995c700 1 -- 172.21.15.12:0/1371842934 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1974102e00 msgr2=0x7f19741a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:08.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.206+0000 7f197995c700 1 --2- 172.21.15.12:0/1371842934 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1974102e00 0x7f19741a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:08.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.206+0000 7f197995c700 1 -- 172.21.15.12:0/1371842934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1974107730 msgr2=0x7f19741a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:08.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.206+0000 7f197995c700 1 --2- 172.21.15.12:0/1371842934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1974107730 0x7f19741a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:08.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.206+0000 7f197995c700 1 -- 172.21.15.12:0/1371842934 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f197419ae30 con 0x7f1974108080 2024-05-17T22:35:08.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.206+0000 7f197915b700 1 --2- 172.21.15.12:0/1371842934 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1974102e00 0x7f19741a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:35:08.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.206+0000 7f197995c700 1 --2- 172.21.15.12:0/1371842934 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1974108080 0x7f197419a700 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f196c0062a0 tx=0x7f196c004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:35:08.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.207+0000 7f196a7fc700 1 -- 172.21.15.12:0/1371842934 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c021070 con 0x7f1974108080 2024-05-17T22:35:08.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.207+0000 7f196a7fc700 1 -- 172.21.15.12:0/1371842934 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f196c007c80 con 0x7f1974108080 2024-05-17T22:35:08.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.207+0000 7f197b3bf700 1 -- 172.21.15.12:0/1371842934 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f197419b0c0 con 0x7f1974108080 2024-05-17T22:35:08.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.207+0000 7f196a7fc700 1 -- 172.21.15.12:0/1371842934 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c01d450 con 0x7f1974108080 2024-05-17T22:35:08.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.207+0000 7f197b3bf700 1 -- 172.21.15.12:0/1371842934 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19741a7320 con 0x7f1974108080 2024-05-17T22:35:08.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.208+0000 7f197b3bf700 1 -- 172.21.15.12:0/1371842934 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19741042b0 con 0x7f1974108080 2024-05-17T22:35:08.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.209+0000 7f196a7fc700 1 -- 172.21.15.12:0/1371842934 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f196c004000 con 0x7f1974108080 2024-05-17T22:35:08.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.210+0000 7f196a7fc700 1 --2- 172.21.15.12:0/1371842934 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f19600b2880 0x7f19600b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:08.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.211+0000 7f197915b700 1 --2- 172.21.15.12:0/1371842934 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f19600b2880 0x7f19600b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:08.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.211+0000 7f196a7fc700 1 -- 172.21.15.12:0/1371842934 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f196c00f020 con 0x7f1974108080 2024-05-17T22:35:08.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.211+0000 7f197915b700 1 --2- 172.21.15.12:0/1371842934 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f19600b2880 0x7f19600b4d90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1970007900 tx=0x7f1970008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:35:08.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.214+0000 7f196a7fc700 1 -- 172.21.15.12:0/1371842934 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f196c02a020 con 0x7f1974108080 2024-05-17T22:35:08.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.429+0000 7f197b3bf700 1 -- 172.21.15.12:0/1371842934 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f197419bc10 con 0x7f1974108080 2024-05-17T22:35:08.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.430+0000 7f196a7fc700 1 -- 172.21.15.12:0/1371842934 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24094 (secure 0 0 0) 0x7f196c0abac0 con 0x7f1974108080 2024-05-17T22:35:08.433 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:35:08.433 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":85,"fsid":"4d593780-149c-11ef-bc9a-c7b262605968","created":"2024-05-17T22:26:16.955626+0000","modified":"2024-05-17T22:34:58.612542+0000","last_up_change":"2024-05-17T22:33:36.551093+0000","last_in_change":"2024-05-17T22:33:15.032677+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T22:29:42.475513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T22:34:35.504697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T22:34:39.377097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-17T22:34:50.467684+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5d42798f-dc5f-4e66-bce2-ec33dc789e4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6803","nonce":2015605593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6805","nonce":2015605593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6809","nonce":2015605593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6807","nonce":2015605593}]},"public_addr":"172.21.15.12:6803/2015605593","cluster_addr":"172.21.15.12:6805/2015605593","heartbeat_back_addr":"172.21.15.12:6809/2015605593","heartbeat_front_addr":"172.21.15.12:6807/2015605593","state":["exists","up"]},{"osd":1,"uuid":"58afecc7-119b-4c16-a401-640fb77d4714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6801","nonce":1238316358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6803","nonce":1238316358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6807","nonce":1238316358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6805","nonce":1238316358}]},"public_addr":"172.21.15.96:6801/1238316358","cluster_addr":"172.21.15.96:6803/1238316358","heartbeat_back_addr":"172.21.15.96:6807/1238316358","heartbeat_front_addr":"172.21.15.96:6805/1238316358","state":["exists","up"]},{"osd":2,"uuid":"4bef4049-482f-49fb-82fb-38052400eb2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6801","nonce":148729815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6803","nonce":148729815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6807","nonce":148729815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6805","nonce":148729815}]},"public_addr":"172.21.15.151:6801/148729815","cluster_addr":"172.21.15.151:6803/148729815","heartbeat_back_addr":"172.21.15.151:6807/148729815","heartbeat_front_addr":"172.21.15.151:6805/148729815","state":["exists","up"]},{"osd":3,"uuid":"6fed8e36-82a2-49e4-82a3-16e6d9de87f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6811","nonce":3563309320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6813","nonce":3563309320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6817","nonce":3563309320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6815","nonce":3563309320}]},"public_addr":"172.21.15.12:6811/3563309320","cluster_addr":"172.21.15.12:6813/3563309320","heartbeat_back_addr":"172.21.15.12:6817/3563309320","heartbeat_front_addr":"172.21.15.12:6815/3563309320","state":["exists","up"]},{"osd":4,"uuid":"c0e8689a-2816-4089-ade3-931960bae4f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6809","nonce":2219103166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6811","nonce":2219103166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6815","nonce":2219103166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6813","nonce":2219103166}]},"public_addr":"172.21.15.96:6809/2219103166","cluster_addr":"172.21.15.96:6811/2219103166","heartbeat_back_addr":"172.21.15.96:6815/2219103166","heartbeat_front_addr":"172.21.15.96:6813/2219103166","state":["exists","up"]},{"osd":5,"uuid":"e76ba432-94ff-4998-aaba-2645d9e6055d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6809","nonce":2589835078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6811","nonce":2589835078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6815","nonce":2589835078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6813","nonce":2589835078}]},"public_addr":"172.21.15.151:6809/2589835078","cluster_addr":"172.21.15.151:6811/2589835078","heartbeat_back_addr":"172.21.15.151:6815/2589835078","heartbeat_front_addr":"172.21.15.151:6813/2589835078","state":["exists","up"]},{"osd":6,"uuid":"b7ece35c-efd1-40df-95da-05a0cc147372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6819","nonce":3112661724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6821","nonce":3112661724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6825","nonce":3112661724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6823","nonce":3112661724}]},"public_addr":"172.21.15.12:6819/3112661724","cluster_addr":"172.21.15.12:6821/3112661724","heartbeat_back_addr":"172.21.15.12:6825/3112661724","heartbeat_front_addr":"172.21.15.12:6823/3112661724","state":["exists","up"]},{"osd":7,"uuid":"6a65bf30-daf5-4fba-b67e-8e513d88a971","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6817","nonce":2449989110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6819","nonce":2449989110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6823","nonce":2449989110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6821","nonce":2449989110}]},"public_addr":"172.21.15.96:6817/2449989110","cluster_addr":"172.21.15.96:6819/2449989110","heartbeat_back_addr":"172.21.15.96:6823/2449989110","heartbeat_front_addr":"172.21.15.96:6821/2449989110","state":["exists","up"]},{"osd":8,"uuid":"ad62ea44-e416-4a3e-bbbe-93f2a710dbc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6817","nonce":2545224628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6819","nonce":2545224628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6823","nonce":2545224628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6821","nonce":2545224628}]},"public_addr":"172.21.15.151:6817/2545224628","cluster_addr":"172.21.15.151:6819/2545224628","heartbeat_back_addr":"172.21.15.151:6823/2545224628","heartbeat_front_addr":"172.21.15.151:6821/2545224628","state":["exists","up"]},{"osd":9,"uuid":"1275de08-2a3c-48f3-920b-dd68b4024d42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6827","nonce":2100661273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6829","nonce":2100661273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6833","nonce":2100661273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6831","nonce":2100661273}]},"public_addr":"172.21.15.12:6827/2100661273","cluster_addr":"172.21.15.12:6829/2100661273","heartbeat_back_addr":"172.21.15.12:6833/2100661273","heartbeat_front_addr":"172.21.15.12:6831/2100661273","state":["exists","up"]},{"osd":10,"uuid":"347e4e40-9a9d-43fe-8f52-fdd3199a2a5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6825","nonce":304016309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6827","nonce":304016309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6831","nonce":304016309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6829","nonce":304016309}]},"public_addr":"172.21.15.96:6825/304016309","cluster_addr":"172.21.15.96:6827/304016309","heartbeat_back_addr":"172.21.15.96:6831/304016309","heartbeat_front_addr":"172.21.15.96:6829/304016309","state":["exists","up"]},{"osd":11,"uuid":"ee8d3502-8b3c-4e6f-824f-3a97e19111ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6825","nonce":3822307314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6827","nonce":3822307314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6831","nonce":3822307314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6829","nonce":3822307314}]},"public_addr":"172.21.15.151:6825/3822307314","cluster_addr":"172.21.15.151:6827/3822307314","heartbeat_back_addr":"172.21.15.151:6831/3822307314","heartbeat_front_addr":"172.21.15.151:6829/3822307314","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:28:51.542055+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:29:15.566424+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:29:39.500981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:04.797907+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:29.655650+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:55.211880+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:31:21.559462+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:31:47.154852+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:32:13.114892+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:32:40.529708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:33:06.930942+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:33:34.061783+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/2682700253":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6801/2250046521":"2024-05-18T22:27:14.372972+0000","172.21.15.12:0/4241094239":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/4172568611":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/3512529242":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6800/1039789215":"2024-05-18T22:26:45.466719+0000","172.21.15.12:6800/2250046521":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6801/1039789215":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/2043758453":"2024-05-18T22:27:14.372972+0000","172.21.15.12:0/2694593732":"2024-05-18T22:26:45.466719+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T22:35:08.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.433+0000 7f197b3bf700 1 -- 172.21.15.12:0/1371842934 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f19600b2880 msgr2=0x7f19600b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:08.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.433+0000 7f197b3bf700 1 --2- 172.21.15.12:0/1371842934 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f19600b2880 0x7f19600b4d90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1970007900 tx=0x7f1970008040 comp rx=0 tx=0).stop 2024-05-17T22:35:08.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.433+0000 7f197b3bf700 1 -- 172.21.15.12:0/1371842934 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1974108080 msgr2=0x7f197419a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:08.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.433+0000 7f197b3bf700 1 --2- 172.21.15.12:0/1371842934 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1974108080 0x7f197419a700 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f196c0062a0 tx=0x7f196c004e20 comp rx=0 tx=0).stop 2024-05-17T22:35:08.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.433+0000 7f197b3bf700 1 -- 172.21.15.12:0/1371842934 shutdown_connections 2024-05-17T22:35:08.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.433+0000 7f197b3bf700 1 --2- 172.21.15.12:0/1371842934 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f19600b2880 0x7f19600b4d90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:08.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.433+0000 7f197b3bf700 1 --2- 172.21.15.12:0/1371842934 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1974108080 0x7f197419a700 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:08.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.433+0000 7f197b3bf700 1 --2- 172.21.15.12:0/1371842934 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1974102e00 0x7f19741a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:08.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.433+0000 7f197b3bf700 1 --2- 172.21.15.12:0/1371842934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1974107730 0x7f19741a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:08.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.433+0000 7f197b3bf700 1 -- 172.21.15.12:0/1371842934 >> 172.21.15.12:0/1371842934 conn(0x7f1974079bd0 msgr2=0x7f1974101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:35:08.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.433+0000 7f197b3bf700 1 -- 172.21.15.12:0/1371842934 shutdown_connections 2024-05-17T22:35:08.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:08.433+0000 7f197b3bf700 1 -- 172.21.15.12:0/1371842934 wait complete. 2024-05-17T22:35:08.801 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:08 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/1371842934' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:35:08.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:08 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/1371842934' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:35:08.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:08 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/1371842934' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:35:09.204 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T22:35:09.460 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:35:09.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:09 smithi096 ceph-mon[36065]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:35:09.801 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:09 smithi012 ceph-mon[31632]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:35:09.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:09 smithi151 ceph-mon[36101]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:35:11.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.285+0000 7f1cd12b1700 1 -- 172.21.15.12:0/2695177653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ccc108840 msgr2=0x7f1ccc108c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:11.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.285+0000 7f1cd12b1700 1 --2- 172.21.15.12:0/2695177653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ccc108840 0x7f1ccc108c50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0009990 tx=0x7f1cc0009ca0 comp rx=0 tx=0).stop 2024-05-17T22:35:11.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.285+0000 7f1cd12b1700 1 -- 172.21.15.12:0/2695177653 shutdown_connections 2024-05-17T22:35:11.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.285+0000 7f1cd12b1700 1 --2- 172.21.15.12:0/2695177653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ccc109190 0x7f1ccc110af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:11.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.285+0000 7f1cd12b1700 1 --2- 172.21.15.12:0/2695177653 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ccc103f10 0x7f1ccc104320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:11.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.285+0000 7f1cd12b1700 1 --2- 172.21.15.12:0/2695177653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ccc108840 0x7f1ccc108c50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:11.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.285+0000 7f1cd12b1700 1 -- 172.21.15.12:0/2695177653 >> 172.21.15.12:0/2695177653 conn(0x7f1ccc0ff9b0 msgr2=0x7f1ccc101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:35:11.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.285+0000 7f1cd12b1700 1 -- 172.21.15.12:0/2695177653 shutdown_connections 2024-05-17T22:35:11.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.285+0000 7f1cd12b1700 1 -- 172.21.15.12:0/2695177653 wait complete. 2024-05-17T22:35:11.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.286+0000 7f1cd12b1700 1 Processor -- start 2024-05-17T22:35:11.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.286+0000 7f1cd12b1700 1 -- start start 2024-05-17T22:35:11.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.286+0000 7f1cd12b1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ccc103f10 0x7f1ccc1a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:11.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.286+0000 7f1cd12b1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ccc108840 0x7f1ccc1a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:11.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.286+0000 7f1cd12b1700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ccc109190 0x7f1ccc19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:11.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.286+0000 7f1cd12b1700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc113df0 con 0x7f1ccc103f10 2024-05-17T22:35:11.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.287+0000 7f1cd12b1700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc113c70 con 0x7f1ccc108840 2024-05-17T22:35:11.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.287+0000 7f1cd12b1700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc113f70 con 0x7f1ccc109190 2024-05-17T22:35:11.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.287+0000 7f1ccaffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ccc103f10 0x7f1ccc1a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:11.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.287+0000 7f1ccaffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ccc103f10 0x7f1ccc1a04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39184/0 (socket says 172.21.15.12:39184) 2024-05-17T22:35:11.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.287+0000 7f1ccaffd700 1 -- 172.21.15.12:0/2865358510 learned_addr learned my addr 172.21.15.12:0/2865358510 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:35:11.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.287+0000 7f1cca7fc700 1 --2- 172.21.15.12:0/2865358510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ccc108840 0x7f1ccc1a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:11.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.287+0000 7f1ccb7fe700 1 --2- 172.21.15.12:0/2865358510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ccc109190 0x7f1ccc19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:11.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.287+0000 7f1ccaffd700 1 -- 172.21.15.12:0/2865358510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ccc109190 msgr2=0x7f1ccc19a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:11.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.287+0000 7f1ccaffd700 1 --2- 172.21.15.12:0/2865358510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ccc109190 0x7f1ccc19a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:11.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.287+0000 7f1ccaffd700 1 -- 172.21.15.12:0/2865358510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ccc108840 msgr2=0x7f1ccc1a0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:11.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.287+0000 7f1ccaffd700 1 --2- 172.21.15.12:0/2865358510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ccc108840 0x7f1ccc1a0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:11.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.287+0000 7f1ccaffd700 1 -- 172.21.15.12:0/2865358510 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ccc19ad90 con 0x7f1ccc103f10 2024-05-17T22:35:11.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.287+0000 7f1ccaffd700 1 --2- 172.21.15.12:0/2865358510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ccc103f10 0x7f1ccc1a04d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc00d950 tx=0x7f1cbc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:35:11.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.287+0000 7f1cb3fff700 1 -- 172.21.15.12:0/2865358510 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc016070 con 0x7f1ccc103f10 2024-05-17T22:35:11.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.288+0000 7f1cd12b1700 1 -- 172.21.15.12:0/2865358510 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ccc19b080 con 0x7f1ccc103f10 2024-05-17T22:35:11.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.288+0000 7f1cd12b1700 1 -- 172.21.15.12:0/2865358510 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ccc1a7280 con 0x7f1ccc103f10 2024-05-17T22:35:11.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.288+0000 7f1cb3fff700 1 -- 172.21.15.12:0/2865358510 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cbc00fd40 con 0x7f1ccc103f10 2024-05-17T22:35:11.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.288+0000 7f1cb3fff700 1 -- 172.21.15.12:0/2865358510 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc00e3b0 con 0x7f1ccc103f10 2024-05-17T22:35:11.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.289+0000 7f1cd12b1700 1 -- 172.21.15.12:0/2865358510 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ccc1053c0 con 0x7f1ccc103f10 2024-05-17T22:35:11.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.290+0000 7f1cb3fff700 1 -- 172.21.15.12:0/2865358510 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f1cbc019070 con 0x7f1ccc103f10 2024-05-17T22:35:11.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.291+0000 7f1cb3fff700 1 --2- 172.21.15.12:0/2865358510 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f1cb40b2900 0x7f1cb40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:11.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.291+0000 7f1cca7fc700 1 --2- 172.21.15.12:0/2865358510 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f1cb40b2900 0x7f1cb40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:11.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.291+0000 7f1cb3fff700 1 -- 172.21.15.12:0/2865358510 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f1cbc0e2200 con 0x7f1ccc103f10 2024-05-17T22:35:11.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.292+0000 7f1cca7fc700 1 --2- 172.21.15.12:0/2865358510 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f1cb40b2900 0x7f1cb40b4e10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc1074f0 tx=0x7f1cc001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:35:11.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.296+0000 7f1cb3fff700 1 -- 172.21.15.12:0/2865358510 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1cbc0a9ce0 con 0x7f1ccc103f10 2024-05-17T22:35:11.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.562+0000 7f1cd12b1700 1 -- 172.21.15.12:0/2865358510 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ccc104320 con 0x7f1ccc103f10 2024-05-17T22:35:11.566 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:11 smithi012 ceph-mon[31632]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:35:11.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.563+0000 7f1cb3fff700 1 -- 172.21.15.12:0/2865358510 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11797 (secure 0 0 0) 0x7f1cbc0a95e0 con 0x7f1ccc103f10 2024-05-17T22:35:11.566 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:35:11.567 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1407219319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1407219319},{"type":"v1","addr":"172.21.15.12:6835","nonce":1407219319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/630977886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":630977886},{"type":"v1","addr":"172.21.15.12:6837","nonce":630977886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3731914742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3731914742},{"type":"v1","addr":"172.21.15.12:6839","nonce":3731914742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/3642142501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3642142501},{"type":"v1","addr":"172.21.15.12:6841","nonce":3642142501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3554053059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3554053059},{"type":"v1","addr":"172.21.15.151:6833","nonce":3554053059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/4228325569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":4228325569},{"type":"v1","addr":"172.21.15.96:6833","nonce":4228325569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/542323754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":542323754},{"type":"v1","addr":"172.21.15.151:6835","nonce":542323754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1758442232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1758442232},{"type":"v1","addr":"172.21.15.96:6835","nonce":1758442232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/3403324038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3403324038},{"type":"v1","addr":"172.21.15.151:6837","nonce":3403324038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/2338353849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2338353849},{"type":"v1","addr":"172.21.15.151:6839","nonce":2338353849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/4222574338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":4222574338},{"type":"v1","addr":"172.21.15.96:6837","nonce":4222574338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:34:42.642111+0000","modified":"2024-05-17T22:34:58.612723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/2986295519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2986295519},{"type":"v1","addr":"172.21.15.96:6839","nonce":2986295519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-05-17T22:35:11.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.566+0000 7f1cd12b1700 1 -- 172.21.15.12:0/2865358510 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f1cb40b2900 msgr2=0x7f1cb40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:11.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.566+0000 7f1cd12b1700 1 --2- 172.21.15.12:0/2865358510 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f1cb40b2900 0x7f1cb40b4e10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc1074f0 tx=0x7f1cc001a040 comp rx=0 tx=0).stop 2024-05-17T22:35:11.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.566+0000 7f1cd12b1700 1 -- 172.21.15.12:0/2865358510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ccc103f10 msgr2=0x7f1ccc1a04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:11.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.566+0000 7f1cd12b1700 1 --2- 172.21.15.12:0/2865358510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ccc103f10 0x7f1ccc1a04d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc00d950 tx=0x7f1cbc00dc60 comp rx=0 tx=0).stop 2024-05-17T22:35:11.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.566+0000 7f1cd12b1700 1 -- 172.21.15.12:0/2865358510 shutdown_connections 2024-05-17T22:35:11.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.566+0000 7f1cd12b1700 1 --2- 172.21.15.12:0/2865358510 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f1cb40b2900 0x7f1cb40b4e10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:11.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.566+0000 7f1cd12b1700 1 --2- 172.21.15.12:0/2865358510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ccc109190 0x7f1ccc19a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:11.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.566+0000 7f1cd12b1700 1 --2- 172.21.15.12:0/2865358510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ccc103f10 0x7f1ccc1a04d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:11.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.566+0000 7f1cd12b1700 1 --2- 172.21.15.12:0/2865358510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ccc108840 0x7f1ccc1a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:11.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.566+0000 7f1cd12b1700 1 -- 172.21.15.12:0/2865358510 >> 172.21.15.12:0/2865358510 conn(0x7f1ccc0ff9b0 msgr2=0x7f1ccc101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:35:11.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.566+0000 7f1cd12b1700 1 -- 172.21.15.12:0/2865358510 shutdown_connections 2024-05-17T22:35:11.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:11.566+0000 7f1cd12b1700 1 -- 172.21.15.12:0/2865358510 wait complete. 2024-05-17T22:35:11.570 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 17 2024-05-17T22:35:11.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:11 smithi096 ceph-mon[36065]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:35:11.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:11 smithi151 ceph-mon[36101]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:35:12.266 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-17T22:35:12.529 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:35:12.787 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:12 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/2865358510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:35:12.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:12 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/2865358510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:35:12.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:12 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/2865358510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:35:13.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:13 smithi096 ceph-mon[36065]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:35:13.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:13 smithi012 ceph-mon[31632]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:35:13.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:13 smithi151 ceph-mon[36101]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:35:14.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.231+0000 7f8ec29fd700 1 -- 172.21.15.12:0/1614281814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ebc107740 msgr2=0x7f8ebc107b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:14.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.231+0000 7f8ec29fd700 1 --2- 172.21.15.12:0/1614281814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ebc107740 0x7f8ebc107b50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4009990 tx=0x7f8ea4009ca0 comp rx=0 tx=0).stop 2024-05-17T22:35:14.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.231+0000 7f8ec29fd700 1 -- 172.21.15.12:0/1614281814 shutdown_connections 2024-05-17T22:35:14.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.231+0000 7f8ec29fd700 1 --2- 172.21.15.12:0/1614281814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ebc108090 0x7f8ebc10f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:14.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.231+0000 7f8ec29fd700 1 --2- 172.21.15.12:0/1614281814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ebc102e10 0x7f8ebc103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:14.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.231+0000 7f8ec29fd700 1 --2- 172.21.15.12:0/1614281814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ebc107740 0x7f8ebc107b50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:14.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.231+0000 7f8ec29fd700 1 -- 172.21.15.12:0/1614281814 >> 172.21.15.12:0/1614281814 conn(0x7f8ebc079b40 msgr2=0x7f8ebc079f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:35:14.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.231+0000 7f8ec29fd700 1 -- 172.21.15.12:0/1614281814 shutdown_connections 2024-05-17T22:35:14.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.231+0000 7f8ec29fd700 1 -- 172.21.15.12:0/1614281814 wait complete. 2024-05-17T22:35:14.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.232+0000 7f8ec29fd700 1 Processor -- start 2024-05-17T22:35:14.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.232+0000 7f8ec29fd700 1 -- start start 2024-05-17T22:35:14.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.232+0000 7f8ec29fd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ebc102e10 0x7f8ebc105190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:14.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.232+0000 7f8ec29fd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ebc107740 0x7f8ebc1056d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:14.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.232+0000 7f8ec29fd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ebc108090 0x7f8ebc105c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:14.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.232+0000 7f8ec29fd700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8ebc112ca0 con 0x7f8ebc107740 2024-05-17T22:35:14.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.232+0000 7f8ec29fd700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8ebc112b20 con 0x7f8ebc102e10 2024-05-17T22:35:14.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.232+0000 7f8ec29fd700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8ebc112e20 con 0x7f8ebc108090 2024-05-17T22:35:14.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.232+0000 7f8eb3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ebc107740 0x7f8ebc1056d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:14.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.232+0000 7f8eb3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ebc107740 0x7f8ebc1056d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39192/0 (socket says 172.21.15.12:39192) 2024-05-17T22:35:14.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.232+0000 7f8eb3fff700 1 -- 172.21.15.12:0/913641295 learned_addr learned my addr 172.21.15.12:0/913641295 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:35:14.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.232+0000 7f8ebbfff700 1 --2- 172.21.15.12:0/913641295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ebc102e10 0x7f8ebc105190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:14.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.232+0000 7f8ec0f9a700 1 --2- 172.21.15.12:0/913641295 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ebc108090 0x7f8ebc105c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:14.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.232+0000 7f8eb3fff700 1 -- 172.21.15.12:0/913641295 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ebc108090 msgr2=0x7f8ebc105c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:14.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.232+0000 7f8eb3fff700 1 --2- 172.21.15.12:0/913641295 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ebc108090 0x7f8ebc105c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:14.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.232+0000 7f8eb3fff700 1 -- 172.21.15.12:0/913641295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ebc102e10 msgr2=0x7f8ebc105190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:14.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.232+0000 7f8eb3fff700 1 --2- 172.21.15.12:0/913641295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ebc102e10 0x7f8ebc105190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:14.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.232+0000 7f8eb3fff700 1 -- 172.21.15.12:0/913641295 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ebc1a8bf0 con 0x7f8ebc107740 2024-05-17T22:35:14.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.232+0000 7f8ec0f9a700 1 --2- 172.21.15.12:0/913641295 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ebc108090 0x7f8ebc105c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:35:14.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.233+0000 7f8eb3fff700 1 --2- 172.21.15.12:0/913641295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ebc107740 0x7f8ebc1056d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4000c00 tx=0x7f8ea40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:35:14.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.233+0000 7f8eb9ffb700 1 -- 172.21.15.12:0/913641295 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4027070 con 0x7f8ebc107740 2024-05-17T22:35:14.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.233+0000 7f8eb9ffb700 1 -- 172.21.15.12:0/913641295 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ea4003f20 con 0x7f8ebc107740 2024-05-17T22:35:14.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.233+0000 7f8eb9ffb700 1 -- 172.21.15.12:0/913641295 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4022cd0 con 0x7f8ebc107740 2024-05-17T22:35:14.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.233+0000 7f8ec29fd700 1 -- 172.21.15.12:0/913641295 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ebc1a8e80 con 0x7f8ebc107740 2024-05-17T22:35:14.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.233+0000 7f8ec29fd700 1 -- 172.21.15.12:0/913641295 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ebc1a9220 con 0x7f8ebc107740 2024-05-17T22:35:14.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.234+0000 7f8ec29fd700 1 -- 172.21.15.12:0/913641295 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ebc103220 con 0x7f8ebc107740 2024-05-17T22:35:14.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.236+0000 7f8eb9ffb700 1 -- 172.21.15.12:0/913641295 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f8ea402a070 con 0x7f8ebc107740 2024-05-17T22:35:14.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.236+0000 7f8eb9ffb700 1 --2- 172.21.15.12:0/913641295 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8ea80b2880 0x7f8ea80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:14.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.237+0000 7f8ebbfff700 1 --2- 172.21.15.12:0/913641295 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8ea80b2880 0x7f8ea80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:14.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.237+0000 7f8eb9ffb700 1 -- 172.21.15.12:0/913641295 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f8ea40e35e0 con 0x7f8ebc107740 2024-05-17T22:35:14.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.237+0000 7f8ebbfff700 1 --2- 172.21.15.12:0/913641295 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8ea80b2880 0x7f8ea80b4d90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8eac005950 tx=0x7f8eac00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:35:14.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.240+0000 7f8eb9ffb700 1 -- 172.21.15.12:0/913641295 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8ea4013070 con 0x7f8ebc107740 2024-05-17T22:35:14.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.454+0000 7f8ec29fd700 1 -- 172.21.15.12:0/913641295 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8ebc1066c0 con 0x7f8ebc107740 2024-05-17T22:35:14.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.456+0000 7f8eb9ffb700 1 -- 172.21.15.12:0/913641295 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24094 (secure 0 0 0) 0x7f8ea40ab150 con 0x7f8ebc107740 2024-05-17T22:35:14.460 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:35:14.460 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":85,"fsid":"4d593780-149c-11ef-bc9a-c7b262605968","created":"2024-05-17T22:26:16.955626+0000","modified":"2024-05-17T22:34:58.612542+0000","last_up_change":"2024-05-17T22:33:36.551093+0000","last_in_change":"2024-05-17T22:33:15.032677+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T22:29:42.475513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T22:34:35.504697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T22:34:39.377097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-17T22:34:50.467684+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5d42798f-dc5f-4e66-bce2-ec33dc789e4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6803","nonce":2015605593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6805","nonce":2015605593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6809","nonce":2015605593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6807","nonce":2015605593}]},"public_addr":"172.21.15.12:6803/2015605593","cluster_addr":"172.21.15.12:6805/2015605593","heartbeat_back_addr":"172.21.15.12:6809/2015605593","heartbeat_front_addr":"172.21.15.12:6807/2015605593","state":["exists","up"]},{"osd":1,"uuid":"58afecc7-119b-4c16-a401-640fb77d4714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6801","nonce":1238316358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6803","nonce":1238316358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6807","nonce":1238316358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6805","nonce":1238316358}]},"public_addr":"172.21.15.96:6801/1238316358","cluster_addr":"172.21.15.96:6803/1238316358","heartbeat_back_addr":"172.21.15.96:6807/1238316358","heartbeat_front_addr":"172.21.15.96:6805/1238316358","state":["exists","up"]},{"osd":2,"uuid":"4bef4049-482f-49fb-82fb-38052400eb2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6801","nonce":148729815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6803","nonce":148729815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6807","nonce":148729815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6805","nonce":148729815}]},"public_addr":"172.21.15.151:6801/148729815","cluster_addr":"172.21.15.151:6803/148729815","heartbeat_back_addr":"172.21.15.151:6807/148729815","heartbeat_front_addr":"172.21.15.151:6805/148729815","state":["exists","up"]},{"osd":3,"uuid":"6fed8e36-82a2-49e4-82a3-16e6d9de87f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6811","nonce":3563309320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6813","nonce":3563309320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6817","nonce":3563309320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6815","nonce":3563309320}]},"public_addr":"172.21.15.12:6811/3563309320","cluster_addr":"172.21.15.12:6813/3563309320","heartbeat_back_addr":"172.21.15.12:6817/3563309320","heartbeat_front_addr":"172.21.15.12:6815/3563309320","state":["exists","up"]},{"osd":4,"uuid":"c0e8689a-2816-4089-ade3-931960bae4f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6809","nonce":2219103166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6811","nonce":2219103166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6815","nonce":2219103166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6813","nonce":2219103166}]},"public_addr":"172.21.15.96:6809/2219103166","cluster_addr":"172.21.15.96:6811/2219103166","heartbeat_back_addr":"172.21.15.96:6815/2219103166","heartbeat_front_addr":"172.21.15.96:6813/2219103166","state":["exists","up"]},{"osd":5,"uuid":"e76ba432-94ff-4998-aaba-2645d9e6055d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6809","nonce":2589835078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6811","nonce":2589835078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6815","nonce":2589835078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6813","nonce":2589835078}]},"public_addr":"172.21.15.151:6809/2589835078","cluster_addr":"172.21.15.151:6811/2589835078","heartbeat_back_addr":"172.21.15.151:6815/2589835078","heartbeat_front_addr":"172.21.15.151:6813/2589835078","state":["exists","up"]},{"osd":6,"uuid":"b7ece35c-efd1-40df-95da-05a0cc147372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6819","nonce":3112661724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6821","nonce":3112661724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6825","nonce":3112661724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6823","nonce":3112661724}]},"public_addr":"172.21.15.12:6819/3112661724","cluster_addr":"172.21.15.12:6821/3112661724","heartbeat_back_addr":"172.21.15.12:6825/3112661724","heartbeat_front_addr":"172.21.15.12:6823/3112661724","state":["exists","up"]},{"osd":7,"uuid":"6a65bf30-daf5-4fba-b67e-8e513d88a971","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6817","nonce":2449989110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6819","nonce":2449989110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6823","nonce":2449989110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6821","nonce":2449989110}]},"public_addr":"172.21.15.96:6817/2449989110","cluster_addr":"172.21.15.96:6819/2449989110","heartbeat_back_addr":"172.21.15.96:6823/2449989110","heartbeat_front_addr":"172.21.15.96:6821/2449989110","state":["exists","up"]},{"osd":8,"uuid":"ad62ea44-e416-4a3e-bbbe-93f2a710dbc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6817","nonce":2545224628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6819","nonce":2545224628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6823","nonce":2545224628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6821","nonce":2545224628}]},"public_addr":"172.21.15.151:6817/2545224628","cluster_addr":"172.21.15.151:6819/2545224628","heartbeat_back_addr":"172.21.15.151:6823/2545224628","heartbeat_front_addr":"172.21.15.151:6821/2545224628","state":["exists","up"]},{"osd":9,"uuid":"1275de08-2a3c-48f3-920b-dd68b4024d42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6827","nonce":2100661273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6829","nonce":2100661273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6833","nonce":2100661273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6831","nonce":2100661273}]},"public_addr":"172.21.15.12:6827/2100661273","cluster_addr":"172.21.15.12:6829/2100661273","heartbeat_back_addr":"172.21.15.12:6833/2100661273","heartbeat_front_addr":"172.21.15.12:6831/2100661273","state":["exists","up"]},{"osd":10,"uuid":"347e4e40-9a9d-43fe-8f52-fdd3199a2a5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6825","nonce":304016309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6827","nonce":304016309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6831","nonce":304016309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6829","nonce":304016309}]},"public_addr":"172.21.15.96:6825/304016309","cluster_addr":"172.21.15.96:6827/304016309","heartbeat_back_addr":"172.21.15.96:6831/304016309","heartbeat_front_addr":"172.21.15.96:6829/304016309","state":["exists","up"]},{"osd":11,"uuid":"ee8d3502-8b3c-4e6f-824f-3a97e19111ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6825","nonce":3822307314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6827","nonce":3822307314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6831","nonce":3822307314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6829","nonce":3822307314}]},"public_addr":"172.21.15.151:6825/3822307314","cluster_addr":"172.21.15.151:6827/3822307314","heartbeat_back_addr":"172.21.15.151:6831/3822307314","heartbeat_front_addr":"172.21.15.151:6829/3822307314","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:28:51.542055+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:29:15.566424+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:29:39.500981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:04.797907+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:29.655650+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:55.211880+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:31:21.559462+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:31:47.154852+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:32:13.114892+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:32:40.529708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:33:06.930942+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:33:34.061783+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/2682700253":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6801/2250046521":"2024-05-18T22:27:14.372972+0000","172.21.15.12:0/4241094239":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/4172568611":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/3512529242":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6800/1039789215":"2024-05-18T22:26:45.466719+0000","172.21.15.12:6800/2250046521":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6801/1039789215":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/2043758453":"2024-05-18T22:27:14.372972+0000","172.21.15.12:0/2694593732":"2024-05-18T22:26:45.466719+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T22:35:14.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.459+0000 7f8ec29fd700 1 -- 172.21.15.12:0/913641295 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8ea80b2880 msgr2=0x7f8ea80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:14.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.459+0000 7f8ec29fd700 1 --2- 172.21.15.12:0/913641295 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8ea80b2880 0x7f8ea80b4d90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8eac005950 tx=0x7f8eac00b410 comp rx=0 tx=0).stop 2024-05-17T22:35:14.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.459+0000 7f8ec29fd700 1 -- 172.21.15.12:0/913641295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ebc107740 msgr2=0x7f8ebc1056d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:14.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.459+0000 7f8ec29fd700 1 --2- 172.21.15.12:0/913641295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ebc107740 0x7f8ebc1056d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4000c00 tx=0x7f8ea40047b0 comp rx=0 tx=0).stop 2024-05-17T22:35:14.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.459+0000 7f8ec29fd700 1 -- 172.21.15.12:0/913641295 shutdown_connections 2024-05-17T22:35:14.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.459+0000 7f8ec29fd700 1 --2- 172.21.15.12:0/913641295 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8ea80b2880 0x7f8ea80b4d90 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:14.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.459+0000 7f8ec29fd700 1 --2- 172.21.15.12:0/913641295 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ebc108090 0x7f8ebc105c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:14.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.459+0000 7f8ec29fd700 1 --2- 172.21.15.12:0/913641295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ebc102e10 0x7f8ebc105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:14.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.459+0000 7f8ec29fd700 1 --2- 172.21.15.12:0/913641295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ebc107740 0x7f8ebc1056d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:14.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.459+0000 7f8ec29fd700 1 -- 172.21.15.12:0/913641295 >> 172.21.15.12:0/913641295 conn(0x7f8ebc079b40 msgr2=0x7f8ebc06c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:35:14.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.459+0000 7f8ec29fd700 1 -- 172.21.15.12:0/913641295 shutdown_connections 2024-05-17T22:35:14.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:14.459+0000 7f8ec29fd700 1 -- 172.21.15.12:0/913641295 wait complete. 2024-05-17T22:35:14.786 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:14 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-05-17T22:35:14.786 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:14 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-05-17T22:35:14.786 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:14 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-05-17T22:35:14.787 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:14 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-05-17T22:35:14.787 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:14 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2024-05-17T22:35:14.787 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:14 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]: dispatch 2024-05-17T22:35:14.787 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:14 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-05-17T22:35:14.787 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:14 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-05-17T22:35:14.787 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:14 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2024-05-17T22:35:14.787 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:14 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-05-17T22:35:14.787 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:14 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/913641295' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:35:14.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:14 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-05-17T22:35:14.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:14 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-05-17T22:35:14.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:14 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-05-17T22:35:14.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:14 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-05-17T22:35:14.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:14 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2024-05-17T22:35:14.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:14 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]: dispatch 2024-05-17T22:35:14.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:14 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-05-17T22:35:14.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:14 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-05-17T22:35:14.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:14 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2024-05-17T22:35:14.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:14 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-05-17T22:35:14.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:14 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/913641295' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:35:14.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:14 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-05-17T22:35:14.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:14 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-05-17T22:35:14.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:14 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-05-17T22:35:14.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:14 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-05-17T22:35:14.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:14 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2024-05-17T22:35:14.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:14 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]: dispatch 2024-05-17T22:35:14.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:14 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-05-17T22:35:14.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:14 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-05-17T22:35:14.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:14 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2024-05-17T22:35:14.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:14 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-05-17T22:35:14.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:14 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/913641295' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:35:15.106 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-17T22:34:42.642111+0000', 'modified': '2024-05-17T22:34:58.612723+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24482}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.96:6839/2986295519', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6838', 'nonce': 2986295519}, {'type': 'v1', 'addr': '172.21.15.96:6839', 'nonce': 2986295519}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24482, 'qdb_cluster': [24482]} 2024-05-17T22:35:15.106 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-17T22:35:15.106 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.b status 2024-05-17T22:35:15.357 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:35:15.731 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:15 smithi012 ceph-mon[31632]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:35:15.731 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:15 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-05-17T22:35:15.732 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:15 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-05-17T22:35:15.732 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:15 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-05-17T22:35:15.732 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:15 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-05-17T22:35:15.732 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:15 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2024-05-17T22:35:15.732 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:15 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]': finished 2024-05-17T22:35:15.732 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:15 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-05-17T22:35:15.733 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:15 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-05-17T22:35:15.733 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:15 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-05-17T22:35:15.733 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:15 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-05-17T22:35:15.733 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:15 smithi012 ceph-mon[31632]: osdmap e86: 12 total, 12 up, 12 in 2024-05-17T22:35:15.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:15 smithi151 ceph-mon[36101]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:35:15.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:15 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-05-17T22:35:15.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:15 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-05-17T22:35:15.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:15 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-05-17T22:35:15.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:15 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-05-17T22:35:15.992 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:15 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2024-05-17T22:35:15.992 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:15 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]': finished 2024-05-17T22:35:15.992 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:15 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-05-17T22:35:15.992 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:15 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-05-17T22:35:15.992 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:15 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-05-17T22:35:15.992 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:15 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-05-17T22:35:15.992 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:15 smithi151 ceph-mon[36101]: osdmap e86: 12 total, 12 up, 12 in 2024-05-17T22:35:16.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:15 smithi096 ceph-mon[36065]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:35:16.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:15 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-05-17T22:35:16.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:15 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-05-17T22:35:16.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:15 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-05-17T22:35:16.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:15 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-05-17T22:35:16.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:15 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2024-05-17T22:35:16.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:15 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]': finished 2024-05-17T22:35:16.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:15 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-05-17T22:35:16.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:15 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-05-17T22:35:16.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:15 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-05-17T22:35:16.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:15 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-05-17T22:35:16.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:15 smithi096 ceph-mon[36065]: osdmap e86: 12 total, 12 up, 12 in 2024-05-17T22:35:16.798 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:16 smithi012 ceph-mon[31632]: osdmap e87: 12 total, 12 up, 12 in 2024-05-17T22:35:16.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:16 smithi151 ceph-mon[36101]: osdmap e87: 12 total, 12 up, 12 in 2024-05-17T22:35:17.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:16 smithi096 ceph-mon[36065]: osdmap e87: 12 total, 12 up, 12 in 2024-05-17T22:35:17.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.056+0000 7f56f646b700 1 -- 172.21.15.12:0/260778401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56f0102410 msgr2=0x7f56f0102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:17.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.056+0000 7f56f646b700 1 --2- 172.21.15.12:0/260778401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56f0102410 0x7f56f0102820 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f56e0009990 tx=0x7f56e0009ca0 comp rx=0 tx=0).stop 2024-05-17T22:35:17.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.056+0000 7f56f646b700 1 -- 172.21.15.12:0/260778401 shutdown_connections 2024-05-17T22:35:17.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.056+0000 7f56f646b700 1 --2- 172.21.15.12:0/260778401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56f0109180 0x7f56f0110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:17.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.056+0000 7f56f646b700 1 --2- 172.21.15.12:0/260778401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56f0102410 0x7f56f0102820 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:17.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.056+0000 7f56f646b700 1 --2- 172.21.15.12:0/260778401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56f0108830 0x7f56f0108c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:17.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.056+0000 7f56f646b700 1 -- 172.21.15.12:0/260778401 >> 172.21.15.12:0/260778401 conn(0x7f56f00fe030 msgr2=0x7f56f0100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:35:17.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.057+0000 7f56f646b700 1 -- 172.21.15.12:0/260778401 shutdown_connections 2024-05-17T22:35:17.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.057+0000 7f56f646b700 1 -- 172.21.15.12:0/260778401 wait complete. 2024-05-17T22:35:17.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.057+0000 7f56f646b700 1 Processor -- start 2024-05-17T22:35:17.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.057+0000 7f56f646b700 1 -- start start 2024-05-17T22:35:17.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.058+0000 7f56f646b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56f0102410 0x7f56f019a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:17.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.058+0000 7f56f646b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56f0108830 0x7f56f019ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:17.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.058+0000 7f56f646b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56f0109180 0x7f56f019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:17.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.058+0000 7f56f646b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f56f0113db0 con 0x7f56f0102410 2024-05-17T22:35:17.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.058+0000 7f56f646b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f56f0113c30 con 0x7f56f0109180 2024-05-17T22:35:17.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.058+0000 7f56f646b700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f56f0113f30 con 0x7f56f0108830 2024-05-17T22:35:17.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.058+0000 7f56effff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56f0102410 0x7f56f019a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:17.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.058+0000 7f56effff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56f0102410 0x7f56f019a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39204/0 (socket says 172.21.15.12:39204) 2024-05-17T22:35:17.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.058+0000 7f56effff700 1 -- 172.21.15.12:0/4050672543 learned_addr learned my addr 172.21.15.12:0/4050672543 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:35:17.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.058+0000 7f56f4a08700 1 --2- 172.21.15.12:0/4050672543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56f0109180 0x7f56f019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:17.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.058+0000 7f56ef7fe700 1 --2- 172.21.15.12:0/4050672543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56f0108830 0x7f56f019ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:17.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.058+0000 7f56effff700 1 -- 172.21.15.12:0/4050672543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56f0108830 msgr2=0x7f56f019ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:17.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.058+0000 7f56effff700 1 --2- 172.21.15.12:0/4050672543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56f0108830 0x7f56f019ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:17.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.058+0000 7f56effff700 1 -- 172.21.15.12:0/4050672543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56f0109180 msgr2=0x7f56f019efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:17.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.058+0000 7f56effff700 1 --2- 172.21.15.12:0/4050672543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56f0109180 0x7f56f019efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:17.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.058+0000 7f56effff700 1 -- 172.21.15.12:0/4050672543 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f019f680 con 0x7f56f0102410 2024-05-17T22:35:17.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.058+0000 7f56f4a08700 1 --2- 172.21.15.12:0/4050672543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56f0109180 0x7f56f019efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:35:17.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.059+0000 7f56effff700 1 --2- 172.21.15.12:0/4050672543 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56f0102410 0x7f56f019a7b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f56e0000c00 tx=0x7f56e00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:35:17.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.059+0000 7f56ed7fa700 1 -- 172.21.15.12:0/4050672543 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e0027070 con 0x7f56f0102410 2024-05-17T22:35:17.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.059+0000 7f56f646b700 1 -- 172.21.15.12:0/4050672543 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56f019f910 con 0x7f56f0102410 2024-05-17T22:35:17.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.059+0000 7f56ed7fa700 1 -- 172.21.15.12:0/4050672543 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56e0003fa0 con 0x7f56f0102410 2024-05-17T22:35:17.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.059+0000 7f56ed7fa700 1 -- 172.21.15.12:0/4050672543 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e0022db0 con 0x7f56f0102410 2024-05-17T22:35:17.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.059+0000 7f56f646b700 1 -- 172.21.15.12:0/4050672543 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56f01a7290 con 0x7f56f0102410 2024-05-17T22:35:17.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.062+0000 7f56ed7fa700 1 -- 172.21.15.12:0/4050672543 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f56e002a070 con 0x7f56f0102410 2024-05-17T22:35:17.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.063+0000 7f56ed7fa700 1 --2- 172.21.15.12:0/4050672543 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f56d80b2880 0x7f56d80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:17.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.063+0000 7f56ef7fe700 1 --2- 172.21.15.12:0/4050672543 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f56d80b2880 0x7f56d80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:17.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.063+0000 7f5704e46b80 1 Processor -- start 2024-05-17T22:35:17.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.063+0000 7f56ed7fa700 1 -- 172.21.15.12:0/4050672543 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f56e00e32c0 con 0x7f56f0102410 2024-05-17T22:35:17.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.063+0000 7f5704e46b80 1 -- start start 2024-05-17T22:35:17.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.063+0000 7f5704e46b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ad393b9a0 0x555ad3abf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:17.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.063+0000 7f5704e46b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ad3abfa50 0x555ad3ab9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:17.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.063+0000 7f5704e46b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ad3aba120 0x555ad3aba530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:17.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.063+0000 7f5704e46b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x555ad39e8240 con 0x555ad3abfa50 2024-05-17T22:35:17.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.063+0000 7f5704e46b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555ad39d7f80 con 0x555ad3aba120 2024-05-17T22:35:17.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.063+0000 7f5704e46b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555ad39dbe20 con 0x555ad393b9a0 2024-05-17T22:35:17.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.063+0000 7f56ef7fe700 1 --2- 172.21.15.12:0/4050672543 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f56d80b2880 0x7f56d80b4d90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f56e4005fd0 tx=0x7f56e400a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:35:17.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.063+0000 7f56ef7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ad3abfa50 0x555ad3ab9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:17.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.063+0000 7f56ef7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ad3abfa50 0x555ad3ab9be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39206/0 (socket says 172.21.15.12:39206) 2024-05-17T22:35:17.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.063+0000 7f56ef7fe700 1 -- 172.21.15.12:0/4030107787 learned_addr learned my addr 172.21.15.12:0/4030107787 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:35:17.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.064+0000 7f56effff700 1 --2- 172.21.15.12:0/4030107787 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ad393b9a0 0x555ad3abf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:17.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.064+0000 7f56f4a08700 1 --2- 172.21.15.12:0/4030107787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ad3aba120 0x555ad3aba530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:17.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.064+0000 7f56ef7fe700 1 -- 172.21.15.12:0/4030107787 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ad393b9a0 msgr2=0x555ad3abf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:35:17.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.064+0000 7f56ef7fe700 1 --2- 172.21.15.12:0/4030107787 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ad393b9a0 0x555ad3abf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:17.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.064+0000 7f56ef7fe700 1 -- 172.21.15.12:0/4030107787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ad3aba120 msgr2=0x555ad3aba530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:35:17.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.064+0000 7f56ef7fe700 1 --2- 172.21.15.12:0/4030107787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ad3aba120 0x555ad3aba530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:17.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.064+0000 7f56ef7fe700 1 -- 172.21.15.12:0/4030107787 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ad3981540 con 0x555ad3abfa50 2024-05-17T22:35:17.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.064+0000 7f56ef7fe700 1 --2- 172.21.15.12:0/4030107787 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ad3abfa50 0x555ad3ab9be0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f56e400ec30 tx=0x7f56e4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1abbc8ecd219b895 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:35:17.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.064+0000 7f56d67fc700 1 -- 172.21.15.12:0/4030107787 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e4021070 con 0x555ad3abfa50 2024-05-17T22:35:17.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.064+0000 7f56d67fc700 1 -- 172.21.15.12:0/4030107787 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56e4010cd0 con 0x555ad3abfa50 2024-05-17T22:35:17.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.064+0000 7f56d67fc700 1 -- 172.21.15.12:0/4030107787 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e400f620 con 0x555ad3abfa50 2024-05-17T22:35:17.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.065+0000 7f5704e46b80 1 -- 172.21.15.12:0/4030107787 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ad3abfa50 msgr2=0x555ad3ab9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:17.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.065+0000 7f5704e46b80 1 --2- 172.21.15.12:0/4030107787 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ad3abfa50 0x555ad3ab9be0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f56e400ec30 tx=0x7f56e4011000 comp rx=0 tx=0).stop 2024-05-17T22:35:17.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.065+0000 7f5704e46b80 1 -- 172.21.15.12:0/4030107787 shutdown_connections 2024-05-17T22:35:17.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.065+0000 7f5704e46b80 1 --2- 172.21.15.12:0/4030107787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ad3aba120 0x555ad3aba530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:17.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.065+0000 7f5704e46b80 1 --2- 172.21.15.12:0/4030107787 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ad393b9a0 0x555ad3abf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:17.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.065+0000 7f5704e46b80 1 --2- 172.21.15.12:0/4030107787 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ad3abfa50 0x555ad3ab9be0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:17.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.065+0000 7f5704e46b80 1 -- 172.21.15.12:0/4030107787 >> 172.21.15.12:0/4030107787 conn(0x555ad397feb0 msgr2=0x555ad3ab4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:35:17.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.065+0000 7f5704e46b80 1 -- 172.21.15.12:0/4030107787 shutdown_connections 2024-05-17T22:35:17.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.065+0000 7f5704e46b80 1 -- 172.21.15.12:0/4030107787 wait complete. 2024-05-17T22:35:17.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.066+0000 7f5704e46b80 1 -- start start 2024-05-17T22:35:17.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.066+0000 7f5704e46b80 1 Processor -- start 2024-05-17T22:35:17.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.066+0000 7f5704e46b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ad393b9a0 0x555ad3ac3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:17.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.066+0000 7f5704e46b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ad3aba120 0x555ad3ac36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:17.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.066+0000 7f5704e46b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ad3abfa50 0x555ad3ac3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:17.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.066+0000 7f5704e46b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x555ad38df100 con 0x555ad393b9a0 2024-05-17T22:35:17.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.066+0000 7f5704e46b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555ad3978e30 con 0x555ad3aba120 2024-05-17T22:35:17.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.066+0000 7f5704e46b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555ad3ac6340 con 0x555ad3abfa50 2024-05-17T22:35:17.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.066+0000 7f5704e46b80 10 client.0 _ll_register_callbacks cb 0x555ad3972ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:35:17.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.066+0000 7f56effff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ad393b9a0 0x555ad3ac3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:17.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.067+0000 7f56effff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ad393b9a0 0x555ad3ac3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39222/0 (socket says 172.21.15.12:39222) 2024-05-17T22:35:17.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.067+0000 7f56effff700 1 -- 172.21.15.12:0/1690417930 learned_addr learned my addr 172.21.15.12:0/1690417930 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:35:17.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.067+0000 7f56ef7fe700 1 --2- 172.21.15.12:0/1690417930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ad3aba120 0x555ad3ac36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:17.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.067+0000 7f56f4a08700 1 --2- 172.21.15.12:0/1690417930 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ad3abfa50 0x555ad3ac3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:17.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.067+0000 7f56effff700 1 -- 172.21.15.12:0/1690417930 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ad3abfa50 msgr2=0x555ad3ac3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:35:17.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.067+0000 7f56effff700 1 --2- 172.21.15.12:0/1690417930 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ad3abfa50 0x555ad3ac3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:17.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.067+0000 7f56effff700 1 -- 172.21.15.12:0/1690417930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ad3aba120 msgr2=0x555ad3ac36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:35:17.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.067+0000 7f56effff700 1 --2- 172.21.15.12:0/1690417930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ad3aba120 0x555ad3ac36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:17.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.067+0000 7f56effff700 1 -- 172.21.15.12:0/1690417930 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ad3928430 con 0x555ad393b9a0 2024-05-17T22:35:17.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.067+0000 7f56effff700 1 --2- 172.21.15.12:0/1690417930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ad393b9a0 0x555ad3ac3160 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f56e00e0dd0 tx=0x7f56e00aab50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d04c854b5e67b86 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:35:17.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.067+0000 7f56cf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:35:17.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.067+0000 7f56cf7fe700 1 -- 172.21.15.12:0/1690417930 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e00aad10 con 0x555ad393b9a0 2024-05-17T22:35:17.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.067+0000 7f56cf7fe700 1 -- 172.21.15.12:0/1690417930 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f56e0004150 con 0x555ad393b9a0 2024-05-17T22:35:17.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.067+0000 7f5704e46b80 1 -- 172.21.15.12:0/1690417930 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56e4010cd0 con 0x555ad393b9a0 2024-05-17T22:35:17.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.067+0000 7f56cf7fe700 1 -- 172.21.15.12:0/1690417930 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f56e00aa610 con 0x555ad393b9a0 2024-05-17T22:35:17.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.067+0000 7f5704e46b80 10 client.14745 fetch_fsmap learned FSMap version 17 2024-05-17T22:35:17.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.067+0000 7f5704e46b80 1 -- 172.21.15.12:0/1690417930 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x555ad3ad0d30 con 0x555ad393b9a0 2024-05-17T22:35:17.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.068+0000 7f56cf7fe700 1 -- 172.21.15.12:0/1690417930 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56e4010cd0 con 0x555ad393b9a0 2024-05-17T22:35:17.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.068+0000 7f56cf7fe700 1 -- 172.21.15.12:0/1690417930 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f56e00a9870 con 0x555ad393b9a0 2024-05-17T22:35:17.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.068+0000 7f5704e46b80 10 client.14745 fetch_fsmap finished waiting for FSMap version 17 2024-05-17T22:35:17.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.068+0000 7f5704e46b80 10 client.14745 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-17T22:35:17.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.068+0000 7f5704e46b80 20 client.14745 populate_metadata read hostname 'smithi012' 2024-05-17T22:35:17.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.068+0000 7f5704e46b80 1 --2- 172.21.15.12:0/1690417930 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555ad3ad32f0 0x555ad3ad4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:17.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.068+0000 7f5704e46b80 4 client.14745 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:35:17.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.068+0000 7f5704e46b80 1 -- 172.21.15.12:0/1690417930 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ad39ea190 con 0x555ad3ad32f0 2024-05-17T22:35:17.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.068+0000 7f56ef7fe700 1 --2- 172.21.15.12:0/1690417930 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555ad3ad32f0 0x555ad3ad4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:17.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.069+0000 7f56ef7fe700 1 --2- 172.21.15.12:0/1690417930 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555ad3ad32f0 0x555ad3ad4760 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c13abaf8e8aa8c06 server_cookie=b454147f26cdafdf in_seq=0 out_seq=0 2024-05-17T22:35:17.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.069+0000 7f56cf7fe700 10 client.14745 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:35:17.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.070+0000 7f56cf7fe700 1 -- 172.21.15.12:0/1690417930 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555ad39ea190 con 0x555ad3ad32f0 2024-05-17T22:35:17.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.070+0000 7f56cf7fe700 10 client.14745 handle_command_reply: tid=0 2024-05-17T22:35:17.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.070+0000 7f5704e46b80 1 client.14745 shutdown 2024-05-17T22:35:17.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.070+0000 7f5704e46b80 1 -- 172.21.15.12:0/1690417930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ad393b9a0 msgr2=0x555ad3ac3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:17.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.070+0000 7f5704e46b80 1 --2- 172.21.15.12:0/1690417930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ad393b9a0 0x555ad3ac3160 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f56e00e0dd0 tx=0x7f56e00aab50 comp rx=0 tx=0).stop 2024-05-17T22:35:17.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.070+0000 7f5704e46b80 1 -- 172.21.15.12:0/1690417930 shutdown_connections 2024-05-17T22:35:17.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.070+0000 7f5704e46b80 1 --2- 172.21.15.12:0/1690417930 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ad3abfa50 0x555ad3ac3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:17.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.070+0000 7f5704e46b80 1 --2- 172.21.15.12:0/1690417930 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555ad3ad32f0 0x555ad3ad4760 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:17.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.070+0000 7f5704e46b80 1 --2- 172.21.15.12:0/1690417930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ad393b9a0 0x555ad3ac3160 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:17.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.070+0000 7f5704e46b80 1 --2- 172.21.15.12:0/1690417930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ad3aba120 0x555ad3ac36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:17.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.070+0000 7f5704e46b80 1 -- 172.21.15.12:0/1690417930 >> 172.21.15.12:0/1690417930 conn(0x555ad397feb0 msgr2=0x555ad3aaee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:35:17.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.070+0000 7f56cf7fe700 0 client.14745 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:35:17.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.070+0000 7f5704e46b80 1 -- 172.21.15.12:0/1690417930 shutdown_connections 2024-05-17T22:35:17.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.070+0000 7f5704e46b80 1 -- 172.21.15.12:0/1690417930 wait complete. 2024-05-17T22:35:17.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.070+0000 7f5704e46b80 20 client.14745 trim_cache size 0 max 16384 2024-05-17T22:35:17.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.092+0000 7f5704e46b80 1 Processor -- start 2024-05-17T22:35:17.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.092+0000 7f5704e46b80 1 -- start start 2024-05-17T22:35:17.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.092+0000 7f5704e46b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ad393b9a0 0x555ad3ab9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:17.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.092+0000 7f5704e46b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ad3aba120 0x555ad3ac2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:17.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.092+0000 7f5704e46b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ad3abfa50 0x555ad3acf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:17.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.092+0000 7f5704e46b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x555ad3ad2aa0 con 0x555ad3aba120 2024-05-17T22:35:17.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.092+0000 7f5704e46b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555ad3ad3170 con 0x555ad393b9a0 2024-05-17T22:35:17.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.092+0000 7f5704e46b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555ad3ad3840 con 0x555ad3abfa50 2024-05-17T22:35:17.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.092+0000 7f56ef7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ad3aba120 0x555ad3ac2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:17.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.092+0000 7f56ef7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ad3aba120 0x555ad3ac2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39230/0 (socket says 172.21.15.12:39230) 2024-05-17T22:35:17.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.092+0000 7f56ef7fe700 1 -- 172.21.15.12:0/757361372 learned_addr learned my addr 172.21.15.12:0/757361372 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:35:17.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.092+0000 7f56effff700 1 --2- 172.21.15.12:0/757361372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ad393b9a0 0x555ad3ab9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:17.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.092+0000 7f56ef7fe700 1 -- 172.21.15.12:0/757361372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ad3abfa50 msgr2=0x555ad3acf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:35:17.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.092+0000 7f56ef7fe700 1 --2- 172.21.15.12:0/757361372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ad3abfa50 0x555ad3acf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:17.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.092+0000 7f56ef7fe700 1 -- 172.21.15.12:0/757361372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ad393b9a0 msgr2=0x555ad3ab9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:35:17.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.092+0000 7f56ef7fe700 1 --2- 172.21.15.12:0/757361372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ad393b9a0 0x555ad3ab9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:17.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.092+0000 7f56ef7fe700 1 -- 172.21.15.12:0/757361372 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ad3ac16d0 con 0x555ad3aba120 2024-05-17T22:35:17.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.093+0000 7f56ef7fe700 1 --2- 172.21.15.12:0/757361372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ad3aba120 0x555ad3ac2410 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f56e4010710 tx=0x7f56e400a400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f1867d57a30a5c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:35:17.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.093+0000 7f56d5ffb700 1 -- 172.21.15.12:0/757361372 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e4021040 con 0x555ad3aba120 2024-05-17T22:35:17.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.093+0000 7f56d5ffb700 1 -- 172.21.15.12:0/757361372 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56e403b070 con 0x555ad3aba120 2024-05-17T22:35:17.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.093+0000 7f56d5ffb700 1 -- 172.21.15.12:0/757361372 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e4004e80 con 0x555ad3aba120 2024-05-17T22:35:17.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.094+0000 7f5704e46b80 1 -- 172.21.15.12:0/757361372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ad3aba120 msgr2=0x555ad3ac2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:17.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.094+0000 7f5704e46b80 1 --2- 172.21.15.12:0/757361372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ad3aba120 0x555ad3ac2410 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f56e4010710 tx=0x7f56e400a400 comp rx=0 tx=0).stop 2024-05-17T22:35:17.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.094+0000 7f5704e46b80 1 -- 172.21.15.12:0/757361372 shutdown_connections 2024-05-17T22:35:17.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.094+0000 7f5704e46b80 1 --2- 172.21.15.12:0/757361372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ad3abfa50 0x555ad3acf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:17.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.094+0000 7f5704e46b80 1 --2- 172.21.15.12:0/757361372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ad393b9a0 0x555ad3ab9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:17.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.094+0000 7f5704e46b80 1 --2- 172.21.15.12:0/757361372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ad3aba120 0x555ad3ac2410 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:17.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.094+0000 7f5704e46b80 1 -- 172.21.15.12:0/757361372 >> 172.21.15.12:0/757361372 conn(0x555ad397feb0 msgr2=0x555ad3abcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:35:17.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.094+0000 7f5704e46b80 1 -- 172.21.15.12:0/757361372 shutdown_connections 2024-05-17T22:35:17.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.094+0000 7f5704e46b80 1 -- 172.21.15.12:0/757361372 wait complete. 2024-05-17T22:35:17.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.094+0000 7f5704e46b80 1 -- start start 2024-05-17T22:35:17.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.094+0000 7f5704e46b80 1 Processor -- start 2024-05-17T22:35:17.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.095+0000 7f5704e46b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ad393b9a0 0x555ad3addbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:17.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.095+0000 7f5704e46b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ad3aba120 0x555ad3ade120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:17.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.095+0000 7f5704e46b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ad3abfa50 0x555ad3ad8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:17.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.095+0000 7f5704e46b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x555ad3ad45e0 con 0x555ad3aba120 2024-05-17T22:35:17.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.095+0000 7f5704e46b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555ad3ad3f10 con 0x555ad393b9a0 2024-05-17T22:35:17.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.095+0000 7f5704e46b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555ad3ad4cb0 con 0x555ad3abfa50 2024-05-17T22:35:17.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.095+0000 7f5704e46b80 10 client.0 _ll_register_callbacks cb 0x555ad3972ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:35:17.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.095+0000 7f56ef7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ad3aba120 0x555ad3ade120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:17.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.095+0000 7f56effff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ad393b9a0 0x555ad3addbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:17.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.095+0000 7f56effff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ad393b9a0 0x555ad3addbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:52186/0 (socket says 172.21.15.12:52186) 2024-05-17T22:35:17.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.095+0000 7f56effff700 1 -- 172.21.15.12:0/2092723866 learned_addr learned my addr 172.21.15.12:0/2092723866 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:35:17.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.095+0000 7f56f4a08700 1 --2- 172.21.15.12:0/2092723866 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ad3abfa50 0x555ad3ad8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:17.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.095+0000 7f56effff700 1 -- 172.21.15.12:0/2092723866 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ad3abfa50 msgr2=0x555ad3ad8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:35:17.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.095+0000 7f56effff700 1 --2- 172.21.15.12:0/2092723866 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ad3abfa50 0x555ad3ad8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:17.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.095+0000 7f56effff700 1 -- 172.21.15.12:0/2092723866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ad3aba120 msgr2=0x555ad3ade120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:35:17.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.095+0000 7f56effff700 1 --2- 172.21.15.12:0/2092723866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ad3aba120 0x555ad3ade120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:17.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.095+0000 7f56effff700 1 -- 172.21.15.12:0/2092723866 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ad3ad90d0 con 0x555ad393b9a0 2024-05-17T22:35:17.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.095+0000 7f56ef7fe700 1 --2- 172.21.15.12:0/2092723866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ad3aba120 0x555ad3ade120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:35:17.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.095+0000 7f56effff700 1 --2- 172.21.15.12:0/2092723866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ad393b9a0 0x555ad3addbe0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f56e00a9c10 tx=0x7f56e0090e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=373a58435208c9f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:35:17.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.095+0000 7f56cf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:35:17.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.095+0000 7f56cf7fe700 1 -- 172.21.15.12:0/2092723866 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e00aad10 con 0x555ad393b9a0 2024-05-17T22:35:17.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.096+0000 7f5704e46b80 1 -- 172.21.15.12:0/2092723866 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56e403b070 con 0x555ad393b9a0 2024-05-17T22:35:17.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.096+0000 7f56cf7fe700 1 -- 172.21.15.12:0/2092723866 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f56e00a6c00 con 0x555ad393b9a0 2024-05-17T22:35:17.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.096+0000 7f56cf7fe700 1 -- 172.21.15.12:0/2092723866 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f56e00ae9c0 con 0x555ad393b9a0 2024-05-17T22:35:17.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.096+0000 7f5704e46b80 10 client.24557 fetch_fsmap learned FSMap version 17 2024-05-17T22:35:17.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.096+0000 7f5704e46b80 1 -- 172.21.15.12:0/2092723866 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x555ad3b3eda0 con 0x555ad393b9a0 2024-05-17T22:35:17.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.096+0000 7f56cf7fe700 1 -- 172.21.15.12:0/2092723866 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56e403b070 con 0x555ad393b9a0 2024-05-17T22:35:17.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.096+0000 7f56cf7fe700 1 -- 172.21.15.12:0/2092723866 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f56e00a5f80 con 0x555ad393b9a0 2024-05-17T22:35:17.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.096+0000 7f5704e46b80 10 client.24557 fetch_fsmap finished waiting for FSMap version 17 2024-05-17T22:35:17.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.096+0000 7f5704e46b80 10 client.24557 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-17T22:35:17.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.096+0000 7f5704e46b80 20 client.24557 populate_metadata read hostname 'smithi012' 2024-05-17T22:35:17.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.096+0000 7f5704e46b80 1 --2- 172.21.15.12:0/2092723866 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555ad3b41070 0x555ad3b424e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:17.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.096+0000 7f5704e46b80 4 client.24557 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-17T22:35:17.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.096+0000 7f5704e46b80 1 -- 172.21.15.12:0/2092723866 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "status"}) -- 0x555ad39e3970 con 0x555ad3b41070 2024-05-17T22:35:17.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.096+0000 7f56ef7fe700 1 --2- 172.21.15.12:0/2092723866 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555ad3b41070 0x555ad3b424e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:17.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.097+0000 7f56cf7fe700 10 client.24557 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:35:17.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.097+0000 7f56ef7fe700 1 --2- 172.21.15.12:0/2092723866 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555ad3b41070 0x555ad3b424e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d81f1c30e46ea757 server_cookie=681c52349642d71 in_seq=0 out_seq=0 2024-05-17T22:35:17.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.097+0000 7f56cf7fe700 1 -- 172.21.15.12:0/2092723866 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+320 (crc 0 0 0) 0x555ad39e3970 con 0x555ad3b41070 2024-05-17T22:35:17.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.097+0000 7f56cf7fe700 10 client.24557 handle_command_reply: tid=0 2024-05-17T22:35:17.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.097+0000 7f5704e46b80 1 client.24557 shutdown 2024-05-17T22:35:17.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.097+0000 7f5704e46b80 1 -- 172.21.15.12:0/2092723866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ad393b9a0 msgr2=0x555ad3addbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:17.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.097+0000 7f5704e46b80 1 --2- 172.21.15.12:0/2092723866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ad393b9a0 0x555ad3addbe0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f56e00a9c10 tx=0x7f56e0090e30 comp rx=0 tx=0).stop 2024-05-17T22:35:17.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.097+0000 7f5704e46b80 1 -- 172.21.15.12:0/2092723866 shutdown_connections 2024-05-17T22:35:17.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.097+0000 7f5704e46b80 1 --2- 172.21.15.12:0/2092723866 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ad3abfa50 0x555ad3ad8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:17.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.097+0000 7f5704e46b80 1 --2- 172.21.15.12:0/2092723866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ad393b9a0 0x555ad3addbe0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:17.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.097+0000 7f5704e46b80 1 --2- 172.21.15.12:0/2092723866 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555ad3b41070 0x555ad3b424e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:17.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.097+0000 7f5704e46b80 1 --2- 172.21.15.12:0/2092723866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ad3aba120 0x555ad3ade120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:17.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.097+0000 7f56cf7fe700 0 client.24557 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:35:17.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.097+0000 7f5704e46b80 1 -- 172.21.15.12:0/2092723866 >> 172.21.15.12:0/2092723866 conn(0x555ad397feb0 msgr2=0x555ad3ac21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:35:17.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.098+0000 7f5704e46b80 1 -- 172.21.15.12:0/2092723866 shutdown_connections 2024-05-17T22:35:17.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.098+0000 7f5704e46b80 1 -- 172.21.15.12:0/2092723866 wait complete. 2024-05-17T22:35:17.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.098+0000 7f5704e46b80 20 client.24557 trim_cache size 0 max 16384 2024-05-17T22:35:17.107 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:35:17.107 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "4d593780-149c-11ef-bc9a-c7b262605968", 2024-05-17T22:35:17.107 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 0, 2024-05-17T22:35:17.107 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24482, 2024-05-17T22:35:17.107 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-05-17T22:35:17.107 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-05-17T22:35:17.107 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-05-17T22:35:17.107 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 33.671305982, 2024-05-17T22:35:17.107 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 17, 2024-05-17T22:35:17.107 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 87, 2024-05-17T22:35:17.107 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 85, 2024-05-17T22:35:17.107 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 48.967832848 2024-05-17T22:35:17.108 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:35:17.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.098+0000 7f56f646b700 1 -- 172.21.15.12:0/4050672543 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f56d80b2880 msgr2=0x7f56d80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:17.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.098+0000 7f56f646b700 1 --2- 172.21.15.12:0/4050672543 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f56d80b2880 0x7f56d80b4d90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f56e4005fd0 tx=0x7f56e400a560 comp rx=0 tx=0).stop 2024-05-17T22:35:17.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.098+0000 7f56f646b700 1 -- 172.21.15.12:0/4050672543 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56f0102410 msgr2=0x7f56f019a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:17.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.098+0000 7f56f646b700 1 --2- 172.21.15.12:0/4050672543 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56f0102410 0x7f56f019a7b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f56e0000c00 tx=0x7f56e00047b0 comp rx=0 tx=0).stop 2024-05-17T22:35:17.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.099+0000 7f56f646b700 1 -- 172.21.15.12:0/4050672543 shutdown_connections 2024-05-17T22:35:17.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.099+0000 7f56f646b700 1 --2- 172.21.15.12:0/4050672543 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f56d80b2880 0x7f56d80b4d90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:17.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.099+0000 7f56f646b700 1 --2- 172.21.15.12:0/4050672543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56f0109180 0x7f56f019efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:17.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.099+0000 7f56f646b700 1 --2- 172.21.15.12:0/4050672543 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56f0102410 0x7f56f019a7b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:17.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.099+0000 7f56f646b700 1 --2- 172.21.15.12:0/4050672543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56f0108830 0x7f56f019ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:17.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.099+0000 7f56f646b700 1 -- 172.21.15.12:0/4050672543 >> 172.21.15.12:0/4050672543 conn(0x7f56f00fe030 msgr2=0x7f56f0100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:35:17.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.099+0000 7f56f646b700 1 -- 172.21.15.12:0/4050672543 shutdown_connections 2024-05-17T22:35:17.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:17.099+0000 7f56f646b700 1 -- 172.21.15.12:0/4050672543 wait complete. 2024-05-17T22:35:17.748 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-17T22:35:17.748 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-17T22:35:17.749 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs ls 2024-05-17T22:35:17.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:17 smithi012 ceph-mon[31632]: pgmap v318: 129 pgs: 1 peering, 128 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:35:17.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:17 smithi151 ceph-mon[36101]: pgmap v318: 129 pgs: 1 peering, 128 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:35:17.998 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:35:18.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:17 smithi096 ceph-mon[36065]: pgmap v318: 129 pgs: 1 peering, 128 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:35:19.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.572+0000 7f9a3438d700 1 -- 172.21.15.12:0/1575637007 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a2c107750 msgr2=0x7f9a2c107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:19.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.572+0000 7f9a3438d700 1 --2- 172.21.15.12:0/1575637007 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a2c107750 0x7f9a2c107b60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c009990 tx=0x7f9a1c009ca0 comp rx=0 tx=0).stop 2024-05-17T22:35:19.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.572+0000 7f9a3438d700 1 -- 172.21.15.12:0/1575637007 shutdown_connections 2024-05-17T22:35:19.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.572+0000 7f9a3438d700 1 --2- 172.21.15.12:0/1575637007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a2c1080a0 0x7f9a2c10fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:19.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.572+0000 7f9a3438d700 1 --2- 172.21.15.12:0/1575637007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a2c102e20 0x7f9a2c103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:19.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.572+0000 7f9a3438d700 1 --2- 172.21.15.12:0/1575637007 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a2c107750 0x7f9a2c107b60 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:19.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.572+0000 7f9a3438d700 1 -- 172.21.15.12:0/1575637007 >> 172.21.15.12:0/1575637007 conn(0x7f9a2c079af0 msgr2=0x7f9a2c079f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:35:19.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.572+0000 7f9a3438d700 1 -- 172.21.15.12:0/1575637007 shutdown_connections 2024-05-17T22:35:19.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.572+0000 7f9a3438d700 1 -- 172.21.15.12:0/1575637007 wait complete. 2024-05-17T22:35:19.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.573+0000 7f9a3438d700 1 Processor -- start 2024-05-17T22:35:19.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.573+0000 7f9a3438d700 1 -- start start 2024-05-17T22:35:19.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.573+0000 7f9a3438d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a2c102e20 0x7f9a2c1a0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:19.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.573+0000 7f9a3438d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a2c107750 0x7f9a2c1a0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:19.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.573+0000 7f9a3438d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a2c1080a0 0x7f9a2c19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:19.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.573+0000 7f9a3438d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c113e50 con 0x7f9a2c1080a0 2024-05-17T22:35:19.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.573+0000 7f9a3438d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c113cd0 con 0x7f9a2c102e20 2024-05-17T22:35:19.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.573+0000 7f9a3438d700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c113fd0 con 0x7f9a2c107750 2024-05-17T22:35:19.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.573+0000 7f9a3292a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a2c1080a0 0x7f9a2c19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:19.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.573+0000 7f9a3292a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a2c1080a0 0x7f9a2c19a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39256/0 (socket says 172.21.15.12:39256) 2024-05-17T22:35:19.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.573+0000 7f9a3292a700 1 -- 172.21.15.12:0/53674822 learned_addr learned my addr 172.21.15.12:0/53674822 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:35:19.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.573+0000 7f9a32129700 1 --2- 172.21.15.12:0/53674822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a2c102e20 0x7f9a2c1a0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:19.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.573+0000 7f9a31928700 1 --2- 172.21.15.12:0/53674822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a2c107750 0x7f9a2c1a0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:19.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.574+0000 7f9a3292a700 1 -- 172.21.15.12:0/53674822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a2c107750 msgr2=0x7f9a2c1a0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:19.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.574+0000 7f9a3292a700 1 --2- 172.21.15.12:0/53674822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a2c107750 0x7f9a2c1a0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:19.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.574+0000 7f9a3292a700 1 -- 172.21.15.12:0/53674822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a2c102e20 msgr2=0x7f9a2c1a0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:19.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.574+0000 7f9a3292a700 1 --2- 172.21.15.12:0/53674822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a2c102e20 0x7f9a2c1a0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:19.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.574+0000 7f9a3292a700 1 -- 172.21.15.12:0/53674822 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a2c19ae20 con 0x7f9a2c1080a0 2024-05-17T22:35:19.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.574+0000 7f9a3292a700 1 --2- 172.21.15.12:0/53674822 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a2c1080a0 0x7f9a2c19a720 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9a24007ee0 tx=0x7f9a2400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:35:19.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.574+0000 7f9a237fe700 1 -- 172.21.15.12:0/53674822 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a24004bb0 con 0x7f9a2c1080a0 2024-05-17T22:35:19.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.574+0000 7f9a237fe700 1 -- 172.21.15.12:0/53674822 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a24004d50 con 0x7f9a2c1080a0 2024-05-17T22:35:19.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.574+0000 7f9a237fe700 1 -- 172.21.15.12:0/53674822 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a24005920 con 0x7f9a2c1080a0 2024-05-17T22:35:19.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.574+0000 7f9a3438d700 1 -- 172.21.15.12:0/53674822 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a2c19b110 con 0x7f9a2c1080a0 2024-05-17T22:35:19.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.574+0000 7f9a3438d700 1 -- 172.21.15.12:0/53674822 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a2c1a7340 con 0x7f9a2c1080a0 2024-05-17T22:35:19.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.575+0000 7f9a3438d700 1 -- 172.21.15.12:0/53674822 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a2c103230 con 0x7f9a2c1080a0 2024-05-17T22:35:19.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.577+0000 7f9a237fe700 1 -- 172.21.15.12:0/53674822 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f9a2401a020 con 0x7f9a2c1080a0 2024-05-17T22:35:19.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.578+0000 7f9a237fe700 1 --2- 172.21.15.12:0/53674822 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9a180b2880 0x7f9a180b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:19.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.578+0000 7f9a32129700 1 --2- 172.21.15.12:0/53674822 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9a180b2880 0x7f9a180b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:19.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.578+0000 7f9a237fe700 1 -- 172.21.15.12:0/53674822 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f9a240e2200 con 0x7f9a2c1080a0 2024-05-17T22:35:19.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.578+0000 7f9a32129700 1 --2- 172.21.15.12:0/53674822 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9a180b2880 0x7f9a180b4d90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c19bb90 tx=0x7f9a2800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:35:19.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.581+0000 7f9a237fe700 1 -- 172.21.15.12:0/53674822 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9a24010070 con 0x7f9a2c1080a0 2024-05-17T22:35:19.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:19 smithi012 ceph-mon[31632]: pgmap v319: 129 pgs: 1 peering, 128 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-05-17T22:35:19.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.852+0000 7f9a3438d700 1 -- 172.21.15.12:0/53674822 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9a2c19bc30 con 0x7f9a2c1080a0 2024-05-17T22:35:19.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.852+0000 7f9a237fe700 1 -- 172.21.15.12:0/53674822 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f9a240a9bc0 con 0x7f9a2c1080a0 2024-05-17T22:35:19.856 INFO:teuthology.orchestra.run.smithi012.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-17T22:35:19.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.855+0000 7f9a3438d700 1 -- 172.21.15.12:0/53674822 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9a180b2880 msgr2=0x7f9a180b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:19.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.855+0000 7f9a3438d700 1 --2- 172.21.15.12:0/53674822 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9a180b2880 0x7f9a180b4d90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c19bb90 tx=0x7f9a2800b410 comp rx=0 tx=0).stop 2024-05-17T22:35:19.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.855+0000 7f9a3438d700 1 -- 172.21.15.12:0/53674822 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a2c1080a0 msgr2=0x7f9a2c19a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:19.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.855+0000 7f9a3438d700 1 --2- 172.21.15.12:0/53674822 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a2c1080a0 0x7f9a2c19a720 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9a24007ee0 tx=0x7f9a2400d3b0 comp rx=0 tx=0).stop 2024-05-17T22:35:19.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.855+0000 7f9a3438d700 1 -- 172.21.15.12:0/53674822 shutdown_connections 2024-05-17T22:35:19.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.855+0000 7f9a3438d700 1 --2- 172.21.15.12:0/53674822 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9a180b2880 0x7f9a180b4d90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:19.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.855+0000 7f9a3438d700 1 --2- 172.21.15.12:0/53674822 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a2c1080a0 0x7f9a2c19a720 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:19.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.855+0000 7f9a3438d700 1 --2- 172.21.15.12:0/53674822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a2c102e20 0x7f9a2c1a0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:19.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.855+0000 7f9a3438d700 1 --2- 172.21.15.12:0/53674822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a2c107750 0x7f9a2c1a0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:19.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.855+0000 7f9a3438d700 1 -- 172.21.15.12:0/53674822 >> 172.21.15.12:0/53674822 conn(0x7f9a2c079af0 msgr2=0x7f9a2c105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:35:19.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.855+0000 7f9a3438d700 1 -- 172.21.15.12:0/53674822 shutdown_connections 2024-05-17T22:35:19.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:19.855+0000 7f9a3438d700 1 -- 172.21.15.12:0/53674822 wait complete. 2024-05-17T22:35:19.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:19 smithi151 ceph-mon[36101]: pgmap v319: 129 pgs: 1 peering, 128 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-05-17T22:35:20.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:19 smithi096 ceph-mon[36065]: pgmap v319: 129 pgs: 1 peering, 128 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-05-17T22:35:20.529 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-17T22:35:20.530 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-17T22:35:20.530 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi012.front.sepia.ceph.com 2024-05-17T22:35:20.530 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-05-17T22:35:20.530 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-17T22:35:20.530 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-17T22:35:20.530 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-17T22:35:20.530 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-17T22:35:20.530 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-05-17T22:35:20.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:35:20.531 DEBUG:teuthology.orchestra.run.smithi012:> ip addr 2024-05-17T22:35:20.549 INFO:teuthology.orchestra.run.smithi012.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-17T22:35:20.549 INFO:teuthology.orchestra.run.smithi012.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-17T22:35:20.549 INFO:teuthology.orchestra.run.smithi012.stdout: inet 127.0.0.1/8 scope host lo 2024-05-17T22:35:20.550 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:35:20.550 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 ::1/128 scope host 2024-05-17T22:35:20.550 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:35:20.550 INFO:teuthology.orchestra.run.smithi012.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-17T22:35:20.550 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 0c:c4:7a:6b:fd:a4 brd ff:ff:ff:ff:ff:ff 2024-05-17T22:35:20.550 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp7s0f0 2024-05-17T22:35:20.550 INFO:teuthology.orchestra.run.smithi012.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-17T22:35:20.550 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether ac:1f:6b:f8:25:ea brd ff:ff:ff:ff:ff:ff 2024-05-17T22:35:20.550 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp1s0f0 2024-05-17T22:35:20.550 INFO:teuthology.orchestra.run.smithi012.stdout: inet 172.21.15.12/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-17T22:35:20.551 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft 42184sec preferred_lft 42184sec 2024-05-17T22:35:20.551 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 fe80::ae1f:6bff:fef8:25ea/64 scope link noprefixroute 2024-05-17T22:35:20.551 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:35:20.551 INFO:teuthology.orchestra.run.smithi012.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-17T22:35:20.551 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 0c:c4:7a:6b:fd:a5 brd ff:ff:ff:ff:ff:ff 2024-05-17T22:35:20.551 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp7s0f1 2024-05-17T22:35:20.551 INFO:teuthology.orchestra.run.smithi012.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-17T22:35:20.551 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether ac:1f:6b:f8:25:eb brd ff:ff:ff:ff:ff:ff 2024-05-17T22:35:20.551 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp1s0f1 2024-05-17T22:35:20.553 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-17T22:35:20.553 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T22:35:20.553 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-05-17T22:35:20.554 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link add name ceph-brx type bridge 2024-05-17T22:35:20.554 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip addr flush dev ceph-brx 2024-05-17T22:35:20.554 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set ceph-brx up 2024-05-17T22:35:20.554 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-17T22:35:20.554 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-05-17T22:35:20.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T22:35:20.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:35:20.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:35:20.750 DEBUG:teuthology.orchestra.run.smithi012:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-17T22:35:20.762 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:20 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/53674822' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-17T22:35:20.777 INFO:teuthology.orchestra.run.smithi012.stdout:1 2024-05-17T22:35:20.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:35:20.779 DEBUG:teuthology.orchestra.run.smithi012:> route 2024-05-17T22:35:20.851 INFO:teuthology.orchestra.run.smithi012.stdout:Kernel IP routing table 2024-05-17T22:35:20.852 INFO:teuthology.orchestra.run.smithi012.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-17T22:35:20.852 INFO:teuthology.orchestra.run.smithi012.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-17T22:35:20.852 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-17T22:35:20.852 INFO:teuthology.orchestra.run.smithi012.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-17T22:35:20.852 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T22:35:20.852 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-05-17T22:35:20.853 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-17T22:35:20.853 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-17T22:35:20.853 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-17T22:35:20.853 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-05-17T22:35:20.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T22:35:20.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:20 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/53674822' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-17T22:35:21.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:20 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/53674822' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-17T22:35:21.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:35:21.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:35:21.154 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2024-05-17T22:35:21.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:35:21.170 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list-id 2024-05-17T22:35:21.226 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T22:35:21.226 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-05-17T22:35:21.226 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-17T22:35:21.226 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-05-17T22:35:21.227 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-05-17T22:35:21.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T22:35:21.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:35:21.354 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-05-17T22:35:21.354 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T22:35:21.354 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-05-17T22:35:21.354 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-05-17T22:35:21.354 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-17T22:35:21.354 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-05-17T22:35:21.354 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-05-17T22:35:21.355 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-05-17T22:35:21.355 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-05-17T22:35:21.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T22:35:21.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:35:21.485 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T22:35:21.485 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-05-17T22:35:21.485 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set brx.0 up 2024-05-17T22:35:21.485 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set dev brx.0 master ceph-brx 2024-05-17T22:35:21.485 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-05-17T22:35:21.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T22:35:21.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:35:21.567 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-05-17T22:35:21.567 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T22:35:21.567 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-05-17T22:35:21.584 INFO:teuthology.orchestra.run.smithi012.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-05-17T22:35:21.584 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T22:35:21.584 DEBUG:teuthology.orchestra.run.smithi012:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-05-17T22:35:21.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:21 smithi012 ceph-mon[31632]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-05-17T22:35:21.607 DEBUG:teuthology.orchestra.run.smithi012:> sudo modprobe fuse 2024-05-17T22:35:21.673 DEBUG:teuthology.orchestra.run.smithi012:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T22:35:21.734 INFO:teuthology.orchestra.run.smithi012.stdout:/proc 2024-05-17T22:35:21.734 INFO:teuthology.orchestra.run.smithi012.stdout:/sys 2024-05-17T22:35:21.734 INFO:teuthology.orchestra.run.smithi012.stdout:/dev 2024-05-17T22:35:21.734 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/security 2024-05-17T22:35:21.734 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/shm 2024-05-17T22:35:21.734 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/pts 2024-05-17T22:35:21.734 INFO:teuthology.orchestra.run.smithi012.stdout:/run 2024-05-17T22:35:21.734 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/cgroup 2024-05-17T22:35:21.734 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/pstore 2024-05-17T22:35:21.734 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/bpf 2024-05-17T22:35:21.735 INFO:teuthology.orchestra.run.smithi012.stdout:/ 2024-05-17T22:35:21.735 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/selinux 2024-05-17T22:35:21.735 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T22:35:21.735 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/hugepages 2024-05-17T22:35:21.735 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/mqueue 2024-05-17T22:35:21.735 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/debug 2024-05-17T22:35:21.735 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/tracing 2024-05-17T22:35:21.735 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/fuse/connections 2024-05-17T22:35:21.735 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-sysctl.service 2024-05-17T22:35:21.735 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-17T22:35:21.735 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/config 2024-05-17T22:35:21.735 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-17T22:35:21.735 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/nfs/rpc_pipefs 2024-05-17T22:35:21.736 INFO:teuthology.orchestra.run.smithi012.stdout:/run/user/1000 2024-05-17T22:35:21.736 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph 2024-05-17T22:35:21.736 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay 2024-05-17T22:35:21.736 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/c027df5b444c0d4e7f5a44ee2efc1eec2a641fd5af546a770296d88db29f6fe4/merged 2024-05-17T22:35:21.736 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/7b2b12671e7c50004204ea56d1e622d0e7847ea734f5924ce13f1afa871308d9/merged 2024-05-17T22:35:21.736 INFO:teuthology.orchestra.run.smithi012.stdout:/run/user/0 2024-05-17T22:35:21.736 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T22:35:21.736 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/1115418b2557b0d102d0e33ca7a1061c19c4b6cc49685485f82d572382a8dbcf/merged 2024-05-17T22:35:21.736 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/04e2e816fe3c393b6c161c57f71595628209d69a04be77ad353ca40a326cec6f/merged 2024-05-17T22:35:21.736 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/97b996615a8e5503a599ab5ea9a7f659404001c419ccf1007c2af67b42a85d51/merged 2024-05-17T22:35:21.736 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/1abcce034a74b628266f3dac3d7b7c55929b7cf5e148518fc42037bd7787e7f5/merged 2024-05-17T22:35:21.736 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/1f743e8166dfc97758c5a789fab37f19e81dc3ed9417545b92000d0249775b55/merged 2024-05-17T22:35:21.736 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/f91fdddf87d2da2bf9d1bc0e7d02501ed34e045f0bbb9f242923a50758c26709/merged 2024-05-17T22:35:21.737 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/4f6829de00128440a7a9537d191acfd8fd7912be96c4e8fa8b7d85c4dc013a2f/merged 2024-05-17T22:35:21.737 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/ba928b8ad6d3063dc5f20b361a3d6dadeda4deccb9962ecccf3401150835e844/merged 2024-05-17T22:35:21.737 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns 2024-05-17T22:35:21.737 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-17T22:35:21.737 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-17T22:35:21.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:35:21.737 DEBUG:teuthology.orchestra.run.smithi012:> ls /sys/fs/fuse/connections 2024-05-17T22:35:21.791 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-17T22:35:21.791 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-05-17T22:35:21.833 DEBUG:teuthology.orchestra.run.smithi012:> sudo modprobe fuse 2024-05-17T22:35:21.858 DEBUG:teuthology.orchestra.run.smithi012:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T22:35:21.915 INFO:teuthology.orchestra.run.smithi012.stdout:/proc 2024-05-17T22:35:21.915 INFO:teuthology.orchestra.run.smithi012.stdout:/sys 2024-05-17T22:35:21.915 INFO:teuthology.orchestra.run.smithi012.stdout:/dev 2024-05-17T22:35:21.915 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/security 2024-05-17T22:35:21.916 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/shm 2024-05-17T22:35:21.916 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/pts 2024-05-17T22:35:21.916 INFO:teuthology.orchestra.run.smithi012.stdout:/run 2024-05-17T22:35:21.916 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/cgroup 2024-05-17T22:35:21.916 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/pstore 2024-05-17T22:35:21.916 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/bpf 2024-05-17T22:35:21.916 INFO:teuthology.orchestra.run.smithi012.stdout:/ 2024-05-17T22:35:21.916 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/selinux 2024-05-17T22:35:21.916 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T22:35:21.916 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/hugepages 2024-05-17T22:35:21.916 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/mqueue 2024-05-17T22:35:21.916 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/debug 2024-05-17T22:35:21.916 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/tracing 2024-05-17T22:35:21.916 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/fuse/connections 2024-05-17T22:35:21.917 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-sysctl.service 2024-05-17T22:35:21.917 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-17T22:35:21.917 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/config 2024-05-17T22:35:21.917 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-17T22:35:21.917 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/nfs/rpc_pipefs 2024-05-17T22:35:21.917 INFO:teuthology.orchestra.run.smithi012.stdout:/run/user/1000 2024-05-17T22:35:21.917 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph 2024-05-17T22:35:21.917 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay 2024-05-17T22:35:21.917 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/c027df5b444c0d4e7f5a44ee2efc1eec2a641fd5af546a770296d88db29f6fe4/merged 2024-05-17T22:35:21.917 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/7b2b12671e7c50004204ea56d1e622d0e7847ea734f5924ce13f1afa871308d9/merged 2024-05-17T22:35:21.917 INFO:teuthology.orchestra.run.smithi012.stdout:/run/user/0 2024-05-17T22:35:21.917 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T22:35:21.917 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/1115418b2557b0d102d0e33ca7a1061c19c4b6cc49685485f82d572382a8dbcf/merged 2024-05-17T22:35:21.917 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/04e2e816fe3c393b6c161c57f71595628209d69a04be77ad353ca40a326cec6f/merged 2024-05-17T22:35:21.918 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/97b996615a8e5503a599ab5ea9a7f659404001c419ccf1007c2af67b42a85d51/merged 2024-05-17T22:35:21.918 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/1abcce034a74b628266f3dac3d7b7c55929b7cf5e148518fc42037bd7787e7f5/merged 2024-05-17T22:35:21.918 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/1f743e8166dfc97758c5a789fab37f19e81dc3ed9417545b92000d0249775b55/merged 2024-05-17T22:35:21.918 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/f91fdddf87d2da2bf9d1bc0e7d02501ed34e045f0bbb9f242923a50758c26709/merged 2024-05-17T22:35:21.918 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/4f6829de00128440a7a9537d191acfd8fd7912be96c4e8fa8b7d85c4dc013a2f/merged 2024-05-17T22:35:21.918 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/ba928b8ad6d3063dc5f20b361a3d6dadeda4deccb9962ecccf3401150835e844/merged 2024-05-17T22:35:21.918 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns 2024-05-17T22:35:21.918 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-17T22:35:21.918 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-17T22:35:21.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:35:21.919 DEBUG:teuthology.orchestra.run.smithi012:> ls /sys/fs/fuse/connections 2024-05-17T22:35:21.957 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi012.stderr:2024-05-17T22:35:21.954+0000 7f7dcbdd7480 -1 init, newargv = 0x7f7db00046a0 newargc=15 2024-05-17T22:35:21.957 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi012.stderr:ceph-fuse[77079]: starting ceph client 2024-05-17T22:35:21.963 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi012.stderr:ceph-fuse[77079]: starting fuse 2024-05-17T22:35:21.972 INFO:teuthology.orchestra.run.smithi012.stdout:119 2024-05-17T22:35:21.973 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-05-17T22:35:21.973 DEBUG:teuthology.orchestra.run.smithi012:> sudo stdin-killer -- python3 -c ' 2024-05-17T22:35:21.973 DEBUG:teuthology.orchestra.run.smithi012:> import glob 2024-05-17T22:35:21.973 DEBUG:teuthology.orchestra.run.smithi012:> import re 2024-05-17T22:35:21.973 DEBUG:teuthology.orchestra.run.smithi012:> import os 2024-05-17T22:35:21.973 DEBUG:teuthology.orchestra.run.smithi012:> import subprocess 2024-05-17T22:35:21.973 DEBUG:teuthology.orchestra.run.smithi012:> 2024-05-17T22:35:21.973 DEBUG:teuthology.orchestra.run.smithi012:> def _find_admin_socket(client_name): 2024-05-17T22:35:21.973 DEBUG:teuthology.orchestra.run.smithi012:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-05-17T22:35:21.973 DEBUG:teuthology.orchestra.run.smithi012:> files = glob.glob(asok_path) 2024-05-17T22:35:21.973 DEBUG:teuthology.orchestra.run.smithi012:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-05-17T22:35:21.974 DEBUG:teuthology.orchestra.run.smithi012:> 2024-05-17T22:35:21.974 DEBUG:teuthology.orchestra.run.smithi012:> # Given a non-glob path, it better be there 2024-05-17T22:35:21.974 DEBUG:teuthology.orchestra.run.smithi012:> if "*" not in asok_path: 2024-05-17T22:35:21.974 DEBUG:teuthology.orchestra.run.smithi012:> assert(len(files) == 1) 2024-05-17T22:35:21.974 DEBUG:teuthology.orchestra.run.smithi012:> return files[0] 2024-05-17T22:35:21.974 DEBUG:teuthology.orchestra.run.smithi012:> 2024-05-17T22:35:21.974 DEBUG:teuthology.orchestra.run.smithi012:> for f in files: 2024-05-17T22:35:21.974 DEBUG:teuthology.orchestra.run.smithi012:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-17T22:35:21.974 DEBUG:teuthology.orchestra.run.smithi012:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-17T22:35:21.974 DEBUG:teuthology.orchestra.run.smithi012:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-17T22:35:21.974 DEBUG:teuthology.orchestra.run.smithi012:> contents = proc_f.read() 2024-05-17T22:35:21.974 DEBUG:teuthology.orchestra.run.smithi012:> if mountpoint in contents: 2024-05-17T22:35:21.974 DEBUG:teuthology.orchestra.run.smithi012:> return f 2024-05-17T22:35:21.974 DEBUG:teuthology.orchestra.run.smithi012:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-17T22:35:21.975 DEBUG:teuthology.orchestra.run.smithi012:> 2024-05-17T22:35:21.975 DEBUG:teuthology.orchestra.run.smithi012:> print(_find_admin_socket("client.admin")) 2024-05-17T22:35:21.975 DEBUG:teuthology.orchestra.run.smithi012:> ' 2024-05-17T22:35:21.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:21 smithi151 ceph-mon[36101]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-05-17T22:35:22.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:21 smithi096 ceph-mon[36065]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-05-17T22:35:22.095 INFO:teuthology.orchestra.run.smithi012.stdout:/var/run/ceph/ceph-client.admin.77079.asok 2024-05-17T22:35:22.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:35:22.105 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.77079.asok 2024-05-17T22:35:22.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:35:22.106 DEBUG:teuthology.orchestra.run.smithi012:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.77079.asok status 2024-05-17T22:35:22.253 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:35:22.254 INFO:teuthology.orchestra.run.smithi012.stdout: "metadata": { 2024-05-17T22:35:22.254 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_sha1": "931809a9e5d295bd61984693cac44fe131ded4bc", 2024-05-17T22:35:22.254 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_version": "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)", 2024-05-17T22:35:22.254 INFO:teuthology.orchestra.run.smithi012.stdout: "entity_id": "admin", 2024-05-17T22:35:22.254 INFO:teuthology.orchestra.run.smithi012.stdout: "hostname": "smithi012", 2024-05-17T22:35:22.254 INFO:teuthology.orchestra.run.smithi012.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-05-17T22:35:22.254 INFO:teuthology.orchestra.run.smithi012.stdout: "pid": "77079", 2024-05-17T22:35:22.254 INFO:teuthology.orchestra.run.smithi012.stdout: "root": "/" 2024-05-17T22:35:22.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:35:22.254 INFO:teuthology.orchestra.run.smithi012.stdout: "dentry_count": 0, 2024-05-17T22:35:22.254 INFO:teuthology.orchestra.run.smithi012.stdout: "dentry_pinned_count": 0, 2024-05-17T22:35:22.254 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 14763, 2024-05-17T22:35:22.255 INFO:teuthology.orchestra.run.smithi012.stdout: "inst": { 2024-05-17T22:35:22.255 INFO:teuthology.orchestra.run.smithi012.stdout: "name": { 2024-05-17T22:35:22.255 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "client", 2024-05-17T22:35:22.255 INFO:teuthology.orchestra.run.smithi012.stdout: "num": 14763 2024-05-17T22:35:22.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:35:22.255 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": { 2024-05-17T22:35:22.255 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-05-17T22:35:22.255 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "192.168.0.1:0", 2024-05-17T22:35:22.255 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2643774347 2024-05-17T22:35:22.255 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:35:22.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:35:22.255 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": { 2024-05-17T22:35:22.255 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-05-17T22:35:22.255 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "192.168.0.1:0", 2024-05-17T22:35:22.256 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2643774347 2024-05-17T22:35:22.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:35:22.256 INFO:teuthology.orchestra.run.smithi012.stdout: "inst_str": "client.14763 192.168.0.1:0/2643774347", 2024-05-17T22:35:22.256 INFO:teuthology.orchestra.run.smithi012.stdout: "addr_str": "192.168.0.1:0/2643774347", 2024-05-17T22:35:22.256 INFO:teuthology.orchestra.run.smithi012.stdout: "inode_count": 1, 2024-05-17T22:35:22.256 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_epoch": 17, 2024-05-17T22:35:22.256 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_epoch": 87, 2024-05-17T22:35:22.256 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_epoch_barrier": 0, 2024-05-17T22:35:22.256 INFO:teuthology.orchestra.run.smithi012.stdout: "blocklisted": false, 2024-05-17T22:35:22.256 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs" 2024-05-17T22:35:22.256 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:35:22.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:35:22.263 DEBUG:teuthology.orchestra.run.smithi012:> stat --file-system '--printf=%T 2024-05-17T22:35:22.263 DEBUG:teuthology.orchestra.run.smithi012:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-05-17T22:35:22.280 INFO:teuthology.orchestra.run.smithi012.stdout:fuseblk 2024-05-17T22:35:22.280 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-05-17T22:35:22.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:35:22.280 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-05-17T22:35:22.347 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-05-17T22:35:22.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:22 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-17T22:35:22.464 INFO:teuthology.orchestra.run.smithi012.stdout:# file: . 2024-05-17T22:35:22.464 INFO:teuthology.orchestra.run.smithi012.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-05-17T22:35:22.464 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:35:22.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:35:22.470 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-17T22:35:22.470 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:35:22.488 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph osd blocklist ls 2024-05-17T22:35:22.781 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:35:23.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:23 smithi012 ceph-mon[31632]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-05-17T22:35:23.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:23 smithi151 ceph-mon[36101]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-05-17T22:35:24.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:23 smithi096 ceph-mon[36065]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-05-17T22:35:24.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.539+0000 7f09fdaa5700 1 -- 172.21.15.12:0/3569876798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09f8107750 msgr2=0x7f09f8107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:24.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.539+0000 7f09fdaa5700 1 --2- 172.21.15.12:0/3569876798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09f8107750 0x7f09f8107b60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f09ec009990 tx=0x7f09ec009ca0 comp rx=0 tx=0).stop 2024-05-17T22:35:24.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.539+0000 7f09fdaa5700 1 -- 172.21.15.12:0/3569876798 shutdown_connections 2024-05-17T22:35:24.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.539+0000 7f09fdaa5700 1 --2- 172.21.15.12:0/3569876798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09f81080a0 0x7f09f810fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:24.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.539+0000 7f09fdaa5700 1 --2- 172.21.15.12:0/3569876798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09f8102e70 0x7f09f8103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:24.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.539+0000 7f09fdaa5700 1 --2- 172.21.15.12:0/3569876798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09f8107750 0x7f09f8107b60 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:24.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.539+0000 7f09fdaa5700 1 -- 172.21.15.12:0/3569876798 >> 172.21.15.12:0/3569876798 conn(0x7f09f8079ae0 msgr2=0x7f09f8079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:35:24.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.539+0000 7f09fdaa5700 1 -- 172.21.15.12:0/3569876798 shutdown_connections 2024-05-17T22:35:24.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.539+0000 7f09fdaa5700 1 -- 172.21.15.12:0/3569876798 wait complete. 2024-05-17T22:35:24.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.540+0000 7f09fdaa5700 1 Processor -- start 2024-05-17T22:35:24.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.540+0000 7f09fdaa5700 1 -- start start 2024-05-17T22:35:24.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.541+0000 7f09fdaa5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09f8102e70 0x7f09f81a04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:24.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.541+0000 7f09fdaa5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09f8107750 0x7f09f81a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:24.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.541+0000 7f09fdaa5700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09f81080a0 0x7f09f819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:24.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.541+0000 7f09fdaa5700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f09f8112cb0 con 0x7f09f8107750 2024-05-17T22:35:24.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.541+0000 7f09fdaa5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f09f8112b30 con 0x7f09f8102e70 2024-05-17T22:35:24.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.541+0000 7f09fdaa5700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f09f8112e30 con 0x7f09f81080a0 2024-05-17T22:35:24.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.541+0000 7f09f6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09f8107750 0x7f09f81a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:24.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.541+0000 7f09f6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09f8107750 0x7f09f81a0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38702/0 (socket says 172.21.15.12:38702) 2024-05-17T22:35:24.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.541+0000 7f09f6ffd700 1 -- 172.21.15.12:0/1225974882 learned_addr learned my addr 172.21.15.12:0/1225974882 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:35:24.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.541+0000 7f09f7fff700 1 --2- 172.21.15.12:0/1225974882 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09f81080a0 0x7f09f819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:24.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.541+0000 7f09f77fe700 1 --2- 172.21.15.12:0/1225974882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09f8102e70 0x7f09f81a04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:24.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.541+0000 7f09f6ffd700 1 -- 172.21.15.12:0/1225974882 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09f81080a0 msgr2=0x7f09f819a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:24.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.541+0000 7f09f6ffd700 1 --2- 172.21.15.12:0/1225974882 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09f81080a0 0x7f09f819a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:24.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.541+0000 7f09f6ffd700 1 -- 172.21.15.12:0/1225974882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09f8102e70 msgr2=0x7f09f81a04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:24.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.541+0000 7f09f6ffd700 1 --2- 172.21.15.12:0/1225974882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09f8102e70 0x7f09f81a04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:24.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.541+0000 7f09f6ffd700 1 -- 172.21.15.12:0/1225974882 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09f819ad80 con 0x7f09f8107750 2024-05-17T22:35:24.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.541+0000 7f09f77fe700 1 --2- 172.21.15.12:0/1225974882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09f8102e70 0x7f09f81a04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:35:24.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.541+0000 7f09f6ffd700 1 --2- 172.21.15.12:0/1225974882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09f8107750 0x7f09f81a0a00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f09ec000c00 tx=0x7f09ec009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:35:24.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.542+0000 7f09f4ff9700 1 -- 172.21.15.12:0/1225974882 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec027070 con 0x7f09f8107750 2024-05-17T22:35:24.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.542+0000 7f09fdaa5700 1 -- 172.21.15.12:0/1225974882 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09f819b010 con 0x7f09f8107750 2024-05-17T22:35:24.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.542+0000 7f09f4ff9700 1 -- 172.21.15.12:0/1225974882 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09ec022470 con 0x7f09f8107750 2024-05-17T22:35:24.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.542+0000 7f09f4ff9700 1 -- 172.21.15.12:0/1225974882 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec017650 con 0x7f09f8107750 2024-05-17T22:35:24.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.542+0000 7f09fdaa5700 1 -- 172.21.15.12:0/1225974882 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09f81a7270 con 0x7f09f8107750 2024-05-17T22:35:24.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.543+0000 7f09fdaa5700 1 -- 172.21.15.12:0/1225974882 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09f81042d0 con 0x7f09f8107750 2024-05-17T22:35:24.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.544+0000 7f09f4ff9700 1 -- 172.21.15.12:0/1225974882 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f09ec02a070 con 0x7f09f8107750 2024-05-17T22:35:24.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.545+0000 7f09f4ff9700 1 --2- 172.21.15.12:0/1225974882 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f09e00b2880 0x7f09e00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:24.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.545+0000 7f09f4ff9700 1 -- 172.21.15.12:0/1225974882 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f09ec0e3b10 con 0x7f09f8107750 2024-05-17T22:35:24.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.545+0000 7f09f77fe700 1 --2- 172.21.15.12:0/1225974882 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f09e00b2880 0x7f09e00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:24.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.546+0000 7f09f77fe700 1 --2- 172.21.15.12:0/1225974882 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f09e00b2880 0x7f09e00b4d90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f09e8009cb0 tx=0x7f09e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:35:24.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.549+0000 7f09f4ff9700 1 -- 172.21.15.12:0/1225974882 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f09ec031020 con 0x7f09f8107750 2024-05-17T22:35:24.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.781+0000 7f09fdaa5700 1 -- 172.21.15.12:0/1225974882 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f09f819bb60 con 0x7f09f8107750 2024-05-17T22:35:24.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.782+0000 7f09f4ff9700 1 -- 172.21.15.12:0/1225974882 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) ==== 81+0+592 (secure 0 0 0) 0x7f09ec013070 con 0x7f09f8107750 2024-05-17T22:35:24.785 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2682700253 2024-05-18T22:27:14.372972+0000 2024-05-17T22:35:24.785 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/2250046521 2024-05-18T22:27:14.372972+0000 2024-05-17T22:35:24.785 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/4241094239 2024-05-18T22:26:45.466719+0000 2024-05-17T22:35:24.785 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/4172568611 2024-05-18T22:26:45.466719+0000 2024-05-17T22:35:24.785 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/3512529242 2024-05-18T22:27:14.372972+0000 2024-05-17T22:35:24.785 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/1039789215 2024-05-18T22:26:45.466719+0000 2024-05-17T22:35:24.785 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/2250046521 2024-05-18T22:27:14.372972+0000 2024-05-17T22:35:24.785 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/1039789215 2024-05-18T22:26:45.466719+0000 2024-05-17T22:35:24.786 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2043758453 2024-05-18T22:27:14.372972+0000 2024-05-17T22:35:24.786 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2694593732 2024-05-18T22:26:45.466719+0000 2024-05-17T22:35:24.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.784+0000 7f09fdaa5700 1 -- 172.21.15.12:0/1225974882 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f09e00b2880 msgr2=0x7f09e00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:24.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.784+0000 7f09fdaa5700 1 --2- 172.21.15.12:0/1225974882 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f09e00b2880 0x7f09e00b4d90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f09e8009cb0 tx=0x7f09e8009260 comp rx=0 tx=0).stop 2024-05-17T22:35:24.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.784+0000 7f09fdaa5700 1 -- 172.21.15.12:0/1225974882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09f8107750 msgr2=0x7f09f81a0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:24.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.784+0000 7f09fdaa5700 1 --2- 172.21.15.12:0/1225974882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09f8107750 0x7f09f81a0a00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f09ec000c00 tx=0x7f09ec009580 comp rx=0 tx=0).stop 2024-05-17T22:35:24.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.784+0000 7f09fdaa5700 1 -- 172.21.15.12:0/1225974882 shutdown_connections 2024-05-17T22:35:24.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.784+0000 7f09fdaa5700 1 --2- 172.21.15.12:0/1225974882 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f09e00b2880 0x7f09e00b4d90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:24.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.784+0000 7f09fdaa5700 1 --2- 172.21.15.12:0/1225974882 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09f81080a0 0x7f09f819a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:24.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.784+0000 7f09fdaa5700 1 --2- 172.21.15.12:0/1225974882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09f8102e70 0x7f09f81a04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:24.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.784+0000 7f09fdaa5700 1 --2- 172.21.15.12:0/1225974882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09f8107750 0x7f09f81a0a00 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:24.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.784+0000 7f09fdaa5700 1 -- 172.21.15.12:0/1225974882 >> 172.21.15.12:0/1225974882 conn(0x7f09f8079ae0 msgr2=0x7f09f8101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:35:24.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.784+0000 7f09fdaa5700 1 -- 172.21.15.12:0/1225974882 shutdown_connections 2024-05-17T22:35:24.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:24.784+0000 7f09fdaa5700 1 -- 172.21.15.12:0/1225974882 wait complete. 2024-05-17T22:35:24.788 INFO:teuthology.orchestra.run.smithi012.stderr:listed 10 entries 2024-05-17T22:35:25.585 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-17T22:35:25.585 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:35:25.605 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph osd blocklist ls 2024-05-17T22:35:25.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:25 smithi012 ceph-mon[31632]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-05-17T22:35:25.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:25 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/1225974882' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-17T22:35:25.905 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:35:25.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:25 smithi151 ceph-mon[36101]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-05-17T22:35:25.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:25 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/1225974882' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-17T22:35:26.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:25 smithi096 ceph-mon[36065]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-05-17T22:35:26.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:25 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/1225974882' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-17T22:35:27.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.633+0000 7f522bb84700 1 -- 172.21.15.12:0/3624593991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52241082b0 msgr2=0x7f522410fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:27.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.633+0000 7f522bb84700 1 --2- 172.21.15.12:0/3624593991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52241082b0 0x7f522410fc10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f521c00b0a0 tx=0x7f521c00b3b0 comp rx=0 tx=0).stop 2024-05-17T22:35:27.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.633+0000 7f522bb84700 1 -- 172.21.15.12:0/3624593991 shutdown_connections 2024-05-17T22:35:27.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.633+0000 7f522bb84700 1 --2- 172.21.15.12:0/3624593991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52241082b0 0x7f522410fc10 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:27.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.633+0000 7f522bb84700 1 --2- 172.21.15.12:0/3624593991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5224103550 0x7f5224103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:27.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.633+0000 7f522bb84700 1 --2- 172.21.15.12:0/3624593991 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5224107960 0x7f5224107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:27.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.633+0000 7f522bb84700 1 -- 172.21.15.12:0/3624593991 >> 172.21.15.12:0/3624593991 conn(0x7f5224078040 msgr2=0x7f52240784a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:35:27.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.633+0000 7f522bb84700 1 -- 172.21.15.12:0/3624593991 shutdown_connections 2024-05-17T22:35:27.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.633+0000 7f522bb84700 1 -- 172.21.15.12:0/3624593991 wait complete. 2024-05-17T22:35:27.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.634+0000 7f522bb84700 1 Processor -- start 2024-05-17T22:35:27.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.634+0000 7f522bb84700 1 -- start start 2024-05-17T22:35:27.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.634+0000 7f522bb84700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5224103550 0x7f522419b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:27.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.634+0000 7f522bb84700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5224107960 0x7f522419bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:27.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.634+0000 7f522bb84700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52241082b0 0x7f52241a6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:27.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.634+0000 7f522bb84700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5224112ef0 con 0x7f5224107960 2024-05-17T22:35:27.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.634+0000 7f522bb84700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5224112d70 con 0x7f52241082b0 2024-05-17T22:35:27.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.634+0000 7f522bb84700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5224113070 con 0x7f5224103550 2024-05-17T22:35:27.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.634+0000 7f522911f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5224107960 0x7f522419bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:27.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.634+0000 7f522911f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5224107960 0x7f522419bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38716/0 (socket says 172.21.15.12:38716) 2024-05-17T22:35:27.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.634+0000 7f522911f700 1 -- 172.21.15.12:0/2376188747 learned_addr learned my addr 172.21.15.12:0/2376188747 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:35:27.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.634+0000 7f5229920700 1 --2- 172.21.15.12:0/2376188747 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5224103550 0x7f522419b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:27.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.634+0000 7f522a121700 1 --2- 172.21.15.12:0/2376188747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52241082b0 0x7f52241a6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:27.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.635+0000 7f522911f700 1 -- 172.21.15.12:0/2376188747 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5224103550 msgr2=0x7f522419b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:27.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.635+0000 7f522911f700 1 --2- 172.21.15.12:0/2376188747 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5224103550 0x7f522419b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:27.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.635+0000 7f522911f700 1 -- 172.21.15.12:0/2376188747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52241082b0 msgr2=0x7f52241a6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:27.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.635+0000 7f522911f700 1 --2- 172.21.15.12:0/2376188747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52241082b0 0x7f52241a6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:27.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.635+0000 7f522911f700 1 -- 172.21.15.12:0/2376188747 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f522419c570 con 0x7f5224107960 2024-05-17T22:35:27.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.635+0000 7f522a121700 1 --2- 172.21.15.12:0/2376188747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52241082b0 0x7f52241a6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:35:27.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.635+0000 7f522911f700 1 --2- 172.21.15.12:0/2376188747 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5224107960 0x7f522419bce0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f521400d980 tx=0x7f521400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:35:27.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.635+0000 7f521affd700 1 -- 172.21.15.12:0/2376188747 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5214017070 con 0x7f5224107960 2024-05-17T22:35:27.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.635+0000 7f522bb84700 1 -- 172.21.15.12:0/2376188747 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52241a6e20 con 0x7f5224107960 2024-05-17T22:35:27.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.635+0000 7f521affd700 1 -- 172.21.15.12:0/2376188747 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f521400fd40 con 0x7f5224107960 2024-05-17T22:35:27.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.635+0000 7f521affd700 1 -- 172.21.15.12:0/2376188747 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521400e3b0 con 0x7f5224107960 2024-05-17T22:35:27.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.635+0000 7f522bb84700 1 -- 172.21.15.12:0/2376188747 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52241a72a0 con 0x7f5224107960 2024-05-17T22:35:27.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.636+0000 7f522bb84700 1 -- 172.21.15.12:0/2376188747 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5224103960 con 0x7f5224107960 2024-05-17T22:35:27.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.638+0000 7f521affd700 1 -- 172.21.15.12:0/2376188747 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f5214010610 con 0x7f5224107960 2024-05-17T22:35:27.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.639+0000 7f521affd700 1 --2- 172.21.15.12:0/2376188747 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f52100b2880 0x7f52100b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:27.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.639+0000 7f5229920700 1 --2- 172.21.15.12:0/2376188747 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f52100b2880 0x7f52100b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:27.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.639+0000 7f521affd700 1 -- 172.21.15.12:0/2376188747 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f52140e1f90 con 0x7f5224107960 2024-05-17T22:35:27.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.639+0000 7f5229920700 1 --2- 172.21.15.12:0/2376188747 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f52100b2880 0x7f52100b4d90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5220009c60 tx=0x7f5220009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:35:27.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.642+0000 7f521affd700 1 -- 172.21.15.12:0/2376188747 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f52140a98f0 con 0x7f5224107960 2024-05-17T22:35:27.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:27 smithi012 ceph-mon[31632]: pgmap v323: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 374 B/s wr, 0 op/s; 2 B/s, 0 objects/s recovering 2024-05-17T22:35:27.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.879+0000 7f522bb84700 1 -- 172.21.15.12:0/2376188747 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f52241027e0 con 0x7f5224107960 2024-05-17T22:35:27.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.879+0000 7f521affd700 1 -- 172.21.15.12:0/2376188747 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) ==== 81+0+592 (secure 0 0 0) 0x7f52140a91f0 con 0x7f5224107960 2024-05-17T22:35:27.883 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2682700253 2024-05-18T22:27:14.372972+0000 2024-05-17T22:35:27.883 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/2250046521 2024-05-18T22:27:14.372972+0000 2024-05-17T22:35:27.883 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/4241094239 2024-05-18T22:26:45.466719+0000 2024-05-17T22:35:27.883 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/4172568611 2024-05-18T22:26:45.466719+0000 2024-05-17T22:35:27.883 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/3512529242 2024-05-18T22:27:14.372972+0000 2024-05-17T22:35:27.883 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/1039789215 2024-05-18T22:26:45.466719+0000 2024-05-17T22:35:27.883 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/2250046521 2024-05-18T22:27:14.372972+0000 2024-05-17T22:35:27.883 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/1039789215 2024-05-18T22:26:45.466719+0000 2024-05-17T22:35:27.883 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2043758453 2024-05-18T22:27:14.372972+0000 2024-05-17T22:35:27.883 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2694593732 2024-05-18T22:26:45.466719+0000 2024-05-17T22:35:27.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.882+0000 7f522bb84700 1 -- 172.21.15.12:0/2376188747 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f52100b2880 msgr2=0x7f52100b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:27.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.882+0000 7f522bb84700 1 --2- 172.21.15.12:0/2376188747 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f52100b2880 0x7f52100b4d90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5220009c60 tx=0x7f5220009260 comp rx=0 tx=0).stop 2024-05-17T22:35:27.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.882+0000 7f522bb84700 1 -- 172.21.15.12:0/2376188747 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5224107960 msgr2=0x7f522419bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:27.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.882+0000 7f522bb84700 1 --2- 172.21.15.12:0/2376188747 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5224107960 0x7f522419bce0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f521400d980 tx=0x7f521400dc90 comp rx=0 tx=0).stop 2024-05-17T22:35:27.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.882+0000 7f522bb84700 1 -- 172.21.15.12:0/2376188747 shutdown_connections 2024-05-17T22:35:27.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.882+0000 7f522bb84700 1 --2- 172.21.15.12:0/2376188747 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f52100b2880 0x7f52100b4d90 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:27.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.882+0000 7f522bb84700 1 --2- 172.21.15.12:0/2376188747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52241082b0 0x7f52241a6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:27.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.882+0000 7f522bb84700 1 --2- 172.21.15.12:0/2376188747 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5224103550 0x7f522419b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:27.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.882+0000 7f522bb84700 1 --2- 172.21.15.12:0/2376188747 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5224107960 0x7f522419bce0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:27.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.882+0000 7f522bb84700 1 -- 172.21.15.12:0/2376188747 >> 172.21.15.12:0/2376188747 conn(0x7f5224078040 msgr2=0x7f52241054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:35:27.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.882+0000 7f522bb84700 1 -- 172.21.15.12:0/2376188747 shutdown_connections 2024-05-17T22:35:27.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:27.882+0000 7f522bb84700 1 -- 172.21.15.12:0/2376188747 wait complete. 2024-05-17T22:35:27.886 INFO:teuthology.orchestra.run.smithi012.stderr:listed 10 entries 2024-05-17T22:35:27.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:27 smithi151 ceph-mon[36101]: pgmap v323: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 374 B/s wr, 0 op/s; 2 B/s, 0 objects/s recovering 2024-05-17T22:35:28.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:27 smithi096 ceph-mon[36065]: pgmap v323: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 374 B/s wr, 0 op/s; 2 B/s, 0 objects/s recovering 2024-05-17T22:35:28.566 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi012.front.sepia.ceph.com... 2024-05-17T22:35:28.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:35:28.566 DEBUG:teuthology.orchestra.run.smithi012:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-05-17T22:35:28.597 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi012.stderr:ceph-fuse[77079]: fuse finished with error 0 and tester_r 0 2024-05-17T22:35:28.599 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:35:28.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:28 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/2376188747' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-17T22:35:28.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:28 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/2376188747' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-17T22:35:29.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:28 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/2376188747' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-17T22:35:29.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:29 smithi012 ceph-mon[31632]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s; 1 B/s, 0 objects/s recovering 2024-05-17T22:35:29.950 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:29 smithi096 ceph-mon[36065]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s; 1 B/s, 0 objects/s recovering 2024-05-17T22:35:29.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:29 smithi151 ceph-mon[36101]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s; 1 B/s, 0 objects/s recovering 2024-05-17T22:35:31.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:31 smithi151 ceph-mon[36101]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-17T22:35:32.018 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:31 smithi012 ceph-mon[31632]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-17T22:35:32.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:31 smithi096 ceph-mon[36065]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-17T22:35:33.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:33 smithi151 ceph-mon[36101]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-17T22:35:34.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:33 smithi096 ceph-mon[36065]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-17T22:35:34.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:33 smithi012 ceph-mon[31632]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-17T22:35:34.701 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi012.front.sepia.ceph.com 2024-05-17T22:35:34.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:35:34.701 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-05-17T22:35:34.730 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-17T22:35:35.035 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:35:35.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:35 smithi012 ceph-mon[31632]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-17T22:35:35.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:35 smithi151 ceph-mon[36101]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-17T22:35:36.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:35 smithi096 ceph-mon[36065]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-17T22:35:36.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.752+0000 7f7208bd0700 1 -- 172.21.15.12:0/4180352551 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7204103f10 msgr2=0x7f7204104320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:36.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.752+0000 7f7208bd0700 1 --2- 172.21.15.12:0/4180352551 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7204103f10 0x7f7204104320 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f71f4009990 tx=0x7f71f4009ca0 comp rx=0 tx=0).stop 2024-05-17T22:35:36.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.752+0000 7f7208bd0700 1 -- 172.21.15.12:0/4180352551 shutdown_connections 2024-05-17T22:35:36.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.752+0000 7f7208bd0700 1 --2- 172.21.15.12:0/4180352551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7204109190 0x7f7204110af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:36.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.752+0000 7f7208bd0700 1 --2- 172.21.15.12:0/4180352551 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7204103f10 0x7f7204104320 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:36.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.752+0000 7f7208bd0700 1 --2- 172.21.15.12:0/4180352551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7204108840 0x7f7204108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:36.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.752+0000 7f7208bd0700 1 -- 172.21.15.12:0/4180352551 >> 172.21.15.12:0/4180352551 conn(0x7f72040ff9b0 msgr2=0x7f7204101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:35:36.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.752+0000 7f7208bd0700 1 -- 172.21.15.12:0/4180352551 shutdown_connections 2024-05-17T22:35:36.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.752+0000 7f7208bd0700 1 -- 172.21.15.12:0/4180352551 wait complete. 2024-05-17T22:35:36.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.753+0000 7f7208bd0700 1 Processor -- start 2024-05-17T22:35:36.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.753+0000 7f7208bd0700 1 -- start start 2024-05-17T22:35:36.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.753+0000 7f7208bd0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7204103f10 0x7f72041a25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:36.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.754+0000 7f7208bd0700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7204108840 0x7f72041a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:36.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.754+0000 7f7208bd0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7204109190 0x7f720419c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:36.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.754+0000 7f7208bd0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7204113dc0 con 0x7f7204103f10 2024-05-17T22:35:36.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.754+0000 7f7208bd0700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7204113c40 con 0x7f7204109190 2024-05-17T22:35:36.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.754+0000 7f7208bd0700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7204113f40 con 0x7f7204108840 2024-05-17T22:35:36.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.754+0000 7f720259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7204103f10 0x7f72041a25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:36.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.754+0000 7f720259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7204103f10 0x7f72041a25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49114/0 (socket says 172.21.15.12:49114) 2024-05-17T22:35:36.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.754+0000 7f720259c700 1 -- 172.21.15.12:0/310836728 learned_addr learned my addr 172.21.15.12:0/310836728 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:35:36.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.754+0000 7f7202d9d700 1 --2- 172.21.15.12:0/310836728 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7204109190 0x7f720419c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:36.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.754+0000 7f7201d9b700 1 --2- 172.21.15.12:0/310836728 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7204108840 0x7f72041a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:36.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.754+0000 7f720259c700 1 -- 172.21.15.12:0/310836728 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7204108840 msgr2=0x7f72041a2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:36.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.754+0000 7f720259c700 1 --2- 172.21.15.12:0/310836728 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7204108840 0x7f72041a2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:36.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.754+0000 7f720259c700 1 -- 172.21.15.12:0/310836728 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7204109190 msgr2=0x7f720419c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:36.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.754+0000 7f720259c700 1 --2- 172.21.15.12:0/310836728 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7204109190 0x7f720419c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:36.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.754+0000 7f720259c700 1 -- 172.21.15.12:0/310836728 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f720419cf60 con 0x7f7204103f10 2024-05-17T22:35:36.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.754+0000 7f7201d9b700 1 --2- 172.21.15.12:0/310836728 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7204108840 0x7f72041a2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:35:36.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.754+0000 7f720259c700 1 --2- 172.21.15.12:0/310836728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7204103f10 0x7f72041a25f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f71f4000c00 tx=0x7f71f40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:35:36.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.755+0000 7f71f37fe700 1 -- 172.21.15.12:0/310836728 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f4027070 con 0x7f7204103f10 2024-05-17T22:35:36.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.755+0000 7f71f37fe700 1 -- 172.21.15.12:0/310836728 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71f4003f20 con 0x7f7204103f10 2024-05-17T22:35:36.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.755+0000 7f7208bd0700 1 -- 172.21.15.12:0/310836728 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f720419d1f0 con 0x7f7204103f10 2024-05-17T22:35:36.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.755+0000 7f71f37fe700 1 -- 172.21.15.12:0/310836728 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f4022cf0 con 0x7f7204103f10 2024-05-17T22:35:36.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.755+0000 7f7208bd0700 1 -- 172.21.15.12:0/310836728 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72041a9450 con 0x7f7204103f10 2024-05-17T22:35:36.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.756+0000 7f7208bd0700 1 -- 172.21.15.12:0/310836728 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f720419d6d0 con 0x7f7204103f10 2024-05-17T22:35:36.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.758+0000 7f71f37fe700 1 -- 172.21.15.12:0/310836728 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f71f402a070 con 0x7f7204103f10 2024-05-17T22:35:36.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.758+0000 7f71f37fe700 1 --2- 172.21.15.12:0/310836728 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f71ec0b2880 0x7f71ec0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:36.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.758+0000 7f71f37fe700 1 -- 172.21.15.12:0/310836728 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f71f40e3650 con 0x7f7204103f10 2024-05-17T22:35:36.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.758+0000 7f7201d9b700 1 --2- 172.21.15.12:0/310836728 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f71ec0b2880 0x7f71ec0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:36.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.759+0000 7f7201d9b700 1 --2- 172.21.15.12:0/310836728 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f71ec0b2880 0x7f71ec0b4d90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f72041087f0 tx=0x7f71f8005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:35:36.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.762+0000 7f71f37fe700 1 -- 172.21.15.12:0/310836728 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f71f4013070 con 0x7f7204103f10 2024-05-17T22:35:36.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.977+0000 7f7208bd0700 1 -- 172.21.15.12:0/310836728 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f720419dd40 con 0x7f7204103f10 2024-05-17T22:35:36.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.979+0000 7f71f37fe700 1 -- 172.21.15.12:0/310836728 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24621 (secure 0 0 0) 0x7f71f40ab0a0 con 0x7f7204103f10 2024-05-17T22:35:36.982 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:35:36.982 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":87,"fsid":"4d593780-149c-11ef-bc9a-c7b262605968","created":"2024-05-17T22:26:16.955626+0000","modified":"2024-05-17T22:35:15.541373+0000","last_up_change":"2024-05-17T22:33:36.551093+0000","last_in_change":"2024-05-17T22:33:15.032677+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T22:29:42.475513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T22:34:35.504697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T22:34:39.377097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-17T22:34:50.467684+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5d42798f-dc5f-4e66-bce2-ec33dc789e4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6803","nonce":2015605593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6805","nonce":2015605593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6809","nonce":2015605593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6807","nonce":2015605593}]},"public_addr":"172.21.15.12:6803/2015605593","cluster_addr":"172.21.15.12:6805/2015605593","heartbeat_back_addr":"172.21.15.12:6809/2015605593","heartbeat_front_addr":"172.21.15.12:6807/2015605593","state":["exists","up"]},{"osd":1,"uuid":"58afecc7-119b-4c16-a401-640fb77d4714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6801","nonce":1238316358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6803","nonce":1238316358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6807","nonce":1238316358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6805","nonce":1238316358}]},"public_addr":"172.21.15.96:6801/1238316358","cluster_addr":"172.21.15.96:6803/1238316358","heartbeat_back_addr":"172.21.15.96:6807/1238316358","heartbeat_front_addr":"172.21.15.96:6805/1238316358","state":["exists","up"]},{"osd":2,"uuid":"4bef4049-482f-49fb-82fb-38052400eb2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6801","nonce":148729815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6803","nonce":148729815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6807","nonce":148729815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6805","nonce":148729815}]},"public_addr":"172.21.15.151:6801/148729815","cluster_addr":"172.21.15.151:6803/148729815","heartbeat_back_addr":"172.21.15.151:6807/148729815","heartbeat_front_addr":"172.21.15.151:6805/148729815","state":["exists","up"]},{"osd":3,"uuid":"6fed8e36-82a2-49e4-82a3-16e6d9de87f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6811","nonce":3563309320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6813","nonce":3563309320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6817","nonce":3563309320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6815","nonce":3563309320}]},"public_addr":"172.21.15.12:6811/3563309320","cluster_addr":"172.21.15.12:6813/3563309320","heartbeat_back_addr":"172.21.15.12:6817/3563309320","heartbeat_front_addr":"172.21.15.12:6815/3563309320","state":["exists","up"]},{"osd":4,"uuid":"c0e8689a-2816-4089-ade3-931960bae4f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6809","nonce":2219103166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6811","nonce":2219103166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6815","nonce":2219103166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6813","nonce":2219103166}]},"public_addr":"172.21.15.96:6809/2219103166","cluster_addr":"172.21.15.96:6811/2219103166","heartbeat_back_addr":"172.21.15.96:6815/2219103166","heartbeat_front_addr":"172.21.15.96:6813/2219103166","state":["exists","up"]},{"osd":5,"uuid":"e76ba432-94ff-4998-aaba-2645d9e6055d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6809","nonce":2589835078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6811","nonce":2589835078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6815","nonce":2589835078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6813","nonce":2589835078}]},"public_addr":"172.21.15.151:6809/2589835078","cluster_addr":"172.21.15.151:6811/2589835078","heartbeat_back_addr":"172.21.15.151:6815/2589835078","heartbeat_front_addr":"172.21.15.151:6813/2589835078","state":["exists","up"]},{"osd":6,"uuid":"b7ece35c-efd1-40df-95da-05a0cc147372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6819","nonce":3112661724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6821","nonce":3112661724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6825","nonce":3112661724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6823","nonce":3112661724}]},"public_addr":"172.21.15.12:6819/3112661724","cluster_addr":"172.21.15.12:6821/3112661724","heartbeat_back_addr":"172.21.15.12:6825/3112661724","heartbeat_front_addr":"172.21.15.12:6823/3112661724","state":["exists","up"]},{"osd":7,"uuid":"6a65bf30-daf5-4fba-b67e-8e513d88a971","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6817","nonce":2449989110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6819","nonce":2449989110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6823","nonce":2449989110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6821","nonce":2449989110}]},"public_addr":"172.21.15.96:6817/2449989110","cluster_addr":"172.21.15.96:6819/2449989110","heartbeat_back_addr":"172.21.15.96:6823/2449989110","heartbeat_front_addr":"172.21.15.96:6821/2449989110","state":["exists","up"]},{"osd":8,"uuid":"ad62ea44-e416-4a3e-bbbe-93f2a710dbc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6817","nonce":2545224628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6819","nonce":2545224628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6823","nonce":2545224628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6821","nonce":2545224628}]},"public_addr":"172.21.15.151:6817/2545224628","cluster_addr":"172.21.15.151:6819/2545224628","heartbeat_back_addr":"172.21.15.151:6823/2545224628","heartbeat_front_addr":"172.21.15.151:6821/2545224628","state":["exists","up"]},{"osd":9,"uuid":"1275de08-2a3c-48f3-920b-dd68b4024d42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6827","nonce":2100661273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6829","nonce":2100661273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6833","nonce":2100661273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6831","nonce":2100661273}]},"public_addr":"172.21.15.12:6827/2100661273","cluster_addr":"172.21.15.12:6829/2100661273","heartbeat_back_addr":"172.21.15.12:6833/2100661273","heartbeat_front_addr":"172.21.15.12:6831/2100661273","state":["exists","up"]},{"osd":10,"uuid":"347e4e40-9a9d-43fe-8f52-fdd3199a2a5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6825","nonce":304016309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6827","nonce":304016309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6831","nonce":304016309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6829","nonce":304016309}]},"public_addr":"172.21.15.96:6825/304016309","cluster_addr":"172.21.15.96:6827/304016309","heartbeat_back_addr":"172.21.15.96:6831/304016309","heartbeat_front_addr":"172.21.15.96:6829/304016309","state":["exists","up"]},{"osd":11,"uuid":"ee8d3502-8b3c-4e6f-824f-3a97e19111ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6825","nonce":3822307314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6827","nonce":3822307314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6831","nonce":3822307314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6829","nonce":3822307314}]},"public_addr":"172.21.15.151:6825/3822307314","cluster_addr":"172.21.15.151:6827/3822307314","heartbeat_back_addr":"172.21.15.151:6831/3822307314","heartbeat_front_addr":"172.21.15.151:6829/3822307314","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:28:51.542055+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:29:15.566424+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:29:39.500981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:04.797907+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:29.655650+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:55.211880+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:31:21.559462+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:31:47.154852+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:32:13.114892+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:32:40.529708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:33:06.930942+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:33:34.061783+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/2682700253":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6801/2250046521":"2024-05-18T22:27:14.372972+0000","172.21.15.12:0/4241094239":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/4172568611":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/3512529242":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6800/1039789215":"2024-05-18T22:26:45.466719+0000","172.21.15.12:6800/2250046521":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6801/1039789215":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/2043758453":"2024-05-18T22:27:14.372972+0000","172.21.15.12:0/2694593732":"2024-05-18T22:26:45.466719+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T22:35:36.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.981+0000 7f7208bd0700 1 -- 172.21.15.12:0/310836728 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f71ec0b2880 msgr2=0x7f71ec0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:36.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.981+0000 7f7208bd0700 1 --2- 172.21.15.12:0/310836728 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f71ec0b2880 0x7f71ec0b4d90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f72041087f0 tx=0x7f71f8005f50 comp rx=0 tx=0).stop 2024-05-17T22:35:36.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.981+0000 7f7208bd0700 1 -- 172.21.15.12:0/310836728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7204103f10 msgr2=0x7f72041a25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:36.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.981+0000 7f7208bd0700 1 --2- 172.21.15.12:0/310836728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7204103f10 0x7f72041a25f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f71f4000c00 tx=0x7f71f40047b0 comp rx=0 tx=0).stop 2024-05-17T22:35:36.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.981+0000 7f7208bd0700 1 -- 172.21.15.12:0/310836728 shutdown_connections 2024-05-17T22:35:36.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.981+0000 7f7208bd0700 1 --2- 172.21.15.12:0/310836728 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f71ec0b2880 0x7f71ec0b4d90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:36.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.981+0000 7f7208bd0700 1 --2- 172.21.15.12:0/310836728 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7204109190 0x7f720419c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:36.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.981+0000 7f7208bd0700 1 --2- 172.21.15.12:0/310836728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7204103f10 0x7f72041a25f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:36.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.981+0000 7f7208bd0700 1 --2- 172.21.15.12:0/310836728 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7204108840 0x7f72041a2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:36.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.981+0000 7f7208bd0700 1 -- 172.21.15.12:0/310836728 >> 172.21.15.12:0/310836728 conn(0x7f72040ff9b0 msgr2=0x7f720410a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:35:36.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.981+0000 7f7208bd0700 1 -- 172.21.15.12:0/310836728 shutdown_connections 2024-05-17T22:35:36.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:36.981+0000 7f7208bd0700 1 -- 172.21.15.12:0/310836728 wait complete. 2024-05-17T22:35:37.662 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-17T22:35:37.920 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:35:37.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:37 smithi151 ceph-mon[36101]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-17T22:35:37.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:37 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/310836728' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:35:37.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:37 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:35:38.044 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:37 smithi012 ceph-mon[31632]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-17T22:35:38.045 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:37 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/310836728' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:35:38.045 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:37 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:35:38.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:37 smithi096 ceph-mon[36065]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-17T22:35:38.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:37 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/310836728' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:35:38.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:37 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:35:39.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.549+0000 7fd291007700 1 -- 172.21.15.12:0/988234712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd28c1091c0 msgr2=0x7fd28c110b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:39.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.549+0000 7fd291007700 1 --2- 172.21.15.12:0/988234712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd28c1091c0 0x7fd28c110b20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd28400b0a0 tx=0x7fd28400b3b0 comp rx=0 tx=0).stop 2024-05-17T22:35:39.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.549+0000 7fd291007700 1 -- 172.21.15.12:0/988234712 shutdown_connections 2024-05-17T22:35:39.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.549+0000 7fd291007700 1 --2- 172.21.15.12:0/988234712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd28c1091c0 0x7fd28c110b20 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:39.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.549+0000 7fd291007700 1 --2- 172.21.15.12:0/988234712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd28c102450 0x7fd28c102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:39.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.549+0000 7fd291007700 1 --2- 172.21.15.12:0/988234712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd28c108870 0x7fd28c108c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:39.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.549+0000 7fd291007700 1 -- 172.21.15.12:0/988234712 >> 172.21.15.12:0/988234712 conn(0x7fd28c0fe030 msgr2=0x7fd28c100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:35:39.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.549+0000 7fd291007700 1 -- 172.21.15.12:0/988234712 shutdown_connections 2024-05-17T22:35:39.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.549+0000 7fd291007700 1 -- 172.21.15.12:0/988234712 wait complete. 2024-05-17T22:35:39.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.550+0000 7fd291007700 1 Processor -- start 2024-05-17T22:35:39.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.550+0000 7fd291007700 1 -- start start 2024-05-17T22:35:39.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.550+0000 7fd291007700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd28c102450 0x7fd28c19a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:39.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.551+0000 7fd291007700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd28c108870 0x7fd28c19ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:39.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.551+0000 7fd291007700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd28c1091c0 0x7fd28c19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:39.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.551+0000 7fd291007700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c113e20 con 0x7fd28c108870 2024-05-17T22:35:39.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.551+0000 7fd291007700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c113ca0 con 0x7fd28c1091c0 2024-05-17T22:35:39.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.551+0000 7fd291007700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c113fa0 con 0x7fd28c102450 2024-05-17T22:35:39.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.551+0000 7fd28a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd28c108870 0x7fd28c19ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:39.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.551+0000 7fd28a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd28c108870 0x7fd28c19ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49132/0 (socket says 172.21.15.12:49132) 2024-05-17T22:35:39.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.551+0000 7fd28a59c700 1 -- 172.21.15.12:0/494881704 learned_addr learned my addr 172.21.15.12:0/494881704 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:35:39.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.551+0000 7fd28ad9d700 1 --2- 172.21.15.12:0/494881704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd28c102450 0x7fd28c19a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:39.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.551+0000 7fd28b59e700 1 --2- 172.21.15.12:0/494881704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd28c1091c0 0x7fd28c19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:39.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.551+0000 7fd28a59c700 1 -- 172.21.15.12:0/494881704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd28c102450 msgr2=0x7fd28c19a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:39.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.551+0000 7fd28a59c700 1 --2- 172.21.15.12:0/494881704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd28c102450 0x7fd28c19a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:39.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.551+0000 7fd28a59c700 1 -- 172.21.15.12:0/494881704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd28c1091c0 msgr2=0x7fd28c19efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:39.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.551+0000 7fd28a59c700 1 --2- 172.21.15.12:0/494881704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd28c1091c0 0x7fd28c19efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:39.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.551+0000 7fd28a59c700 1 -- 172.21.15.12:0/494881704 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd28c19f670 con 0x7fd28c108870 2024-05-17T22:35:39.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.551+0000 7fd28ad9d700 1 --2- 172.21.15.12:0/494881704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd28c102450 0x7fd28c19a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:35:39.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.551+0000 7fd28a59c700 1 --2- 172.21.15.12:0/494881704 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd28c108870 0x7fd28c19ad40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd28000e940 tx=0x7fd28000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:35:39.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.551+0000 7fd273fff700 1 -- 172.21.15.12:0/494881704 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28000c9e0 con 0x7fd28c108870 2024-05-17T22:35:39.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.551+0000 7fd291007700 1 -- 172.21.15.12:0/494881704 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd28c19f960 con 0x7fd28c108870 2024-05-17T22:35:39.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.551+0000 7fd273fff700 1 -- 172.21.15.12:0/494881704 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd28000cb80 con 0x7fd28c108870 2024-05-17T22:35:39.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.551+0000 7fd273fff700 1 -- 172.21.15.12:0/494881704 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd280010e60 con 0x7fd28c108870 2024-05-17T22:35:39.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.552+0000 7fd291007700 1 -- 172.21.15.12:0/494881704 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd28c1a7280 con 0x7fd28c108870 2024-05-17T22:35:39.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.553+0000 7fd291007700 1 -- 172.21.15.12:0/494881704 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd28c102860 con 0x7fd28c108870 2024-05-17T22:35:39.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.554+0000 7fd273fff700 1 -- 172.21.15.12:0/494881704 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fd280010430 con 0x7fd28c108870 2024-05-17T22:35:39.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.555+0000 7fd273fff700 1 --2- 172.21.15.12:0/494881704 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd2740b2880 0x7fd2740b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:39.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.555+0000 7fd28ad9d700 1 --2- 172.21.15.12:0/494881704 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd2740b2880 0x7fd2740b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:39.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.555+0000 7fd273fff700 1 -- 172.21.15.12:0/494881704 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fd2800e2b90 con 0x7fd28c108870 2024-05-17T22:35:39.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.556+0000 7fd28ad9d700 1 --2- 172.21.15.12:0/494881704 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd2740b2880 0x7fd2740b4d90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd27c011490 tx=0x7fd27c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:35:39.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.559+0000 7fd273fff700 1 -- 172.21.15.12:0/494881704 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd2800aa550 con 0x7fd28c108870 2024-05-17T22:35:39.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.779+0000 7fd291007700 1 -- 172.21.15.12:0/494881704 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd28c19bb00 con 0x7fd28c108870 2024-05-17T22:35:39.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.780+0000 7fd273fff700 1 -- 172.21.15.12:0/494881704 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24621 (secure 0 0 0) 0x7fd2800a9e50 con 0x7fd28c108870 2024-05-17T22:35:39.784 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:35:39.784 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":87,"fsid":"4d593780-149c-11ef-bc9a-c7b262605968","created":"2024-05-17T22:26:16.955626+0000","modified":"2024-05-17T22:35:15.541373+0000","last_up_change":"2024-05-17T22:33:36.551093+0000","last_in_change":"2024-05-17T22:33:15.032677+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T22:29:42.475513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T22:34:35.504697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T22:34:39.377097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-17T22:34:50.467684+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5d42798f-dc5f-4e66-bce2-ec33dc789e4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6803","nonce":2015605593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6805","nonce":2015605593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6809","nonce":2015605593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6807","nonce":2015605593}]},"public_addr":"172.21.15.12:6803/2015605593","cluster_addr":"172.21.15.12:6805/2015605593","heartbeat_back_addr":"172.21.15.12:6809/2015605593","heartbeat_front_addr":"172.21.15.12:6807/2015605593","state":["exists","up"]},{"osd":1,"uuid":"58afecc7-119b-4c16-a401-640fb77d4714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6801","nonce":1238316358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6803","nonce":1238316358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6807","nonce":1238316358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6805","nonce":1238316358}]},"public_addr":"172.21.15.96:6801/1238316358","cluster_addr":"172.21.15.96:6803/1238316358","heartbeat_back_addr":"172.21.15.96:6807/1238316358","heartbeat_front_addr":"172.21.15.96:6805/1238316358","state":["exists","up"]},{"osd":2,"uuid":"4bef4049-482f-49fb-82fb-38052400eb2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6801","nonce":148729815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6803","nonce":148729815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6807","nonce":148729815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6805","nonce":148729815}]},"public_addr":"172.21.15.151:6801/148729815","cluster_addr":"172.21.15.151:6803/148729815","heartbeat_back_addr":"172.21.15.151:6807/148729815","heartbeat_front_addr":"172.21.15.151:6805/148729815","state":["exists","up"]},{"osd":3,"uuid":"6fed8e36-82a2-49e4-82a3-16e6d9de87f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6811","nonce":3563309320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6813","nonce":3563309320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6817","nonce":3563309320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6815","nonce":3563309320}]},"public_addr":"172.21.15.12:6811/3563309320","cluster_addr":"172.21.15.12:6813/3563309320","heartbeat_back_addr":"172.21.15.12:6817/3563309320","heartbeat_front_addr":"172.21.15.12:6815/3563309320","state":["exists","up"]},{"osd":4,"uuid":"c0e8689a-2816-4089-ade3-931960bae4f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6809","nonce":2219103166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6811","nonce":2219103166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6815","nonce":2219103166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6813","nonce":2219103166}]},"public_addr":"172.21.15.96:6809/2219103166","cluster_addr":"172.21.15.96:6811/2219103166","heartbeat_back_addr":"172.21.15.96:6815/2219103166","heartbeat_front_addr":"172.21.15.96:6813/2219103166","state":["exists","up"]},{"osd":5,"uuid":"e76ba432-94ff-4998-aaba-2645d9e6055d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6809","nonce":2589835078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6811","nonce":2589835078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6815","nonce":2589835078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6813","nonce":2589835078}]},"public_addr":"172.21.15.151:6809/2589835078","cluster_addr":"172.21.15.151:6811/2589835078","heartbeat_back_addr":"172.21.15.151:6815/2589835078","heartbeat_front_addr":"172.21.15.151:6813/2589835078","state":["exists","up"]},{"osd":6,"uuid":"b7ece35c-efd1-40df-95da-05a0cc147372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6819","nonce":3112661724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6821","nonce":3112661724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6825","nonce":3112661724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6823","nonce":3112661724}]},"public_addr":"172.21.15.12:6819/3112661724","cluster_addr":"172.21.15.12:6821/3112661724","heartbeat_back_addr":"172.21.15.12:6825/3112661724","heartbeat_front_addr":"172.21.15.12:6823/3112661724","state":["exists","up"]},{"osd":7,"uuid":"6a65bf30-daf5-4fba-b67e-8e513d88a971","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6817","nonce":2449989110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6819","nonce":2449989110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6823","nonce":2449989110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6821","nonce":2449989110}]},"public_addr":"172.21.15.96:6817/2449989110","cluster_addr":"172.21.15.96:6819/2449989110","heartbeat_back_addr":"172.21.15.96:6823/2449989110","heartbeat_front_addr":"172.21.15.96:6821/2449989110","state":["exists","up"]},{"osd":8,"uuid":"ad62ea44-e416-4a3e-bbbe-93f2a710dbc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6817","nonce":2545224628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6819","nonce":2545224628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6823","nonce":2545224628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6821","nonce":2545224628}]},"public_addr":"172.21.15.151:6817/2545224628","cluster_addr":"172.21.15.151:6819/2545224628","heartbeat_back_addr":"172.21.15.151:6823/2545224628","heartbeat_front_addr":"172.21.15.151:6821/2545224628","state":["exists","up"]},{"osd":9,"uuid":"1275de08-2a3c-48f3-920b-dd68b4024d42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6827","nonce":2100661273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6829","nonce":2100661273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6833","nonce":2100661273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6831","nonce":2100661273}]},"public_addr":"172.21.15.12:6827/2100661273","cluster_addr":"172.21.15.12:6829/2100661273","heartbeat_back_addr":"172.21.15.12:6833/2100661273","heartbeat_front_addr":"172.21.15.12:6831/2100661273","state":["exists","up"]},{"osd":10,"uuid":"347e4e40-9a9d-43fe-8f52-fdd3199a2a5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6825","nonce":304016309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6827","nonce":304016309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6831","nonce":304016309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6829","nonce":304016309}]},"public_addr":"172.21.15.96:6825/304016309","cluster_addr":"172.21.15.96:6827/304016309","heartbeat_back_addr":"172.21.15.96:6831/304016309","heartbeat_front_addr":"172.21.15.96:6829/304016309","state":["exists","up"]},{"osd":11,"uuid":"ee8d3502-8b3c-4e6f-824f-3a97e19111ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6825","nonce":3822307314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6827","nonce":3822307314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6831","nonce":3822307314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6829","nonce":3822307314}]},"public_addr":"172.21.15.151:6825/3822307314","cluster_addr":"172.21.15.151:6827/3822307314","heartbeat_back_addr":"172.21.15.151:6831/3822307314","heartbeat_front_addr":"172.21.15.151:6829/3822307314","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:28:51.542055+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:29:15.566424+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:29:39.500981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:04.797907+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:29.655650+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:55.211880+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:31:21.559462+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:31:47.154852+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:32:13.114892+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:32:40.529708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:33:06.930942+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:33:34.061783+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/2682700253":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6801/2250046521":"2024-05-18T22:27:14.372972+0000","172.21.15.12:0/4241094239":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/4172568611":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/3512529242":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6800/1039789215":"2024-05-18T22:26:45.466719+0000","172.21.15.12:6800/2250046521":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6801/1039789215":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/2043758453":"2024-05-18T22:27:14.372972+0000","172.21.15.12:0/2694593732":"2024-05-18T22:26:45.466719+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T22:35:39.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.783+0000 7fd291007700 1 -- 172.21.15.12:0/494881704 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd2740b2880 msgr2=0x7fd2740b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:39.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.783+0000 7fd291007700 1 --2- 172.21.15.12:0/494881704 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd2740b2880 0x7fd2740b4d90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd27c011490 tx=0x7fd27c009260 comp rx=0 tx=0).stop 2024-05-17T22:35:39.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.783+0000 7fd291007700 1 -- 172.21.15.12:0/494881704 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd28c108870 msgr2=0x7fd28c19ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:39.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.783+0000 7fd291007700 1 --2- 172.21.15.12:0/494881704 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd28c108870 0x7fd28c19ad40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd28000e940 tx=0x7fd28000ec50 comp rx=0 tx=0).stop 2024-05-17T22:35:39.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.783+0000 7fd291007700 1 -- 172.21.15.12:0/494881704 shutdown_connections 2024-05-17T22:35:39.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.783+0000 7fd291007700 1 --2- 172.21.15.12:0/494881704 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd2740b2880 0x7fd2740b4d90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:39.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.783+0000 7fd291007700 1 --2- 172.21.15.12:0/494881704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd28c1091c0 0x7fd28c19efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:39.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.783+0000 7fd291007700 1 --2- 172.21.15.12:0/494881704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd28c102450 0x7fd28c19a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:39.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.783+0000 7fd291007700 1 --2- 172.21.15.12:0/494881704 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd28c108870 0x7fd28c19ad40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:39.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.783+0000 7fd291007700 1 -- 172.21.15.12:0/494881704 >> 172.21.15.12:0/494881704 conn(0x7fd28c0fe030 msgr2=0x7fd28c100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:35:39.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.783+0000 7fd291007700 1 -- 172.21.15.12:0/494881704 shutdown_connections 2024-05-17T22:35:39.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:39.783+0000 7fd291007700 1 -- 172.21.15.12:0/494881704 wait complete. 2024-05-17T22:35:39.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:39 smithi012 ceph-mon[31632]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-17T22:35:39.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:39 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:35:39.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:39 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:35:39.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:39 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:35:39.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:39 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:35:39.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:39 smithi151 ceph-mon[36101]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-17T22:35:39.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:39 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:35:39.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:39 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:35:39.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:39 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:35:39.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:39 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:35:40.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:39 smithi096 ceph-mon[36065]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-17T22:35:40.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:39 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:35:40.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:39 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:35:40.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:39 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:35:40.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:39 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:35:40.430 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-05-17T22:35:40.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:40 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/494881704' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:35:40.922 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:35:40.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:40 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/494881704' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:35:41.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:40 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/494881704' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:35:41.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:41 smithi151 ceph-mon[36101]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-17T22:35:42.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:41 smithi096 ceph-mon[36065]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-17T22:35:42.085 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:41 smithi012 ceph-mon[31632]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-17T22:35:42.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:42.809+0000 7fcb92e9a700 1 -- 172.21.15.12:0/3724521381 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb8c107750 msgr2=0x7fcb8c107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:42.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:42.809+0000 7fcb92e9a700 1 --2- 172.21.15.12:0/3724521381 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb8c107750 0x7fcb8c107b60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcb78009990 tx=0x7fcb78009ca0 comp rx=0 tx=0).stop 2024-05-17T22:35:42.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:42.809+0000 7fcb92e9a700 1 -- 172.21.15.12:0/3724521381 shutdown_connections 2024-05-17T22:35:42.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:42.809+0000 7fcb92e9a700 1 --2- 172.21.15.12:0/3724521381 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb8c1080a0 0x7fcb8c10fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:42.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:42.809+0000 7fcb92e9a700 1 --2- 172.21.15.12:0/3724521381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb8c102e20 0x7fcb8c103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:42.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:42.809+0000 7fcb92e9a700 1 --2- 172.21.15.12:0/3724521381 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb8c107750 0x7fcb8c107b60 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:42.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:42.809+0000 7fcb92e9a700 1 -- 172.21.15.12:0/3724521381 >> 172.21.15.12:0/3724521381 conn(0x7fcb8c079bd0 msgr2=0x7fcb8c079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:35:42.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:42.809+0000 7fcb92e9a700 1 -- 172.21.15.12:0/3724521381 shutdown_connections 2024-05-17T22:35:42.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:42.809+0000 7fcb92e9a700 1 -- 172.21.15.12:0/3724521381 wait complete. 2024-05-17T22:35:42.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:42.810+0000 7fcb92e9a700 1 Processor -- start 2024-05-17T22:35:42.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:42.810+0000 7fcb92e9a700 1 -- start start 2024-05-17T22:35:42.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:42.810+0000 7fcb92e9a700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb8c102e20 0x7fcb8c19b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:42.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:42.810+0000 7fcb92e9a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb8c107750 0x7fcb8c19bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:42.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:42.810+0000 7fcb92e9a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb8c1080a0 0x7fcb8c1a6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:42.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:42.810+0000 7fcb92e9a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c112cb0 con 0x7fcb8c1080a0 2024-05-17T22:35:42.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:42.810+0000 7fcb92e9a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c112b30 con 0x7fcb8c107750 2024-05-17T22:35:42.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:42.810+0000 7fcb92e9a700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c112e30 con 0x7fcb8c102e20 2024-05-17T22:35:42.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:42.810+0000 7fcb91437700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb8c1080a0 0x7fcb8c1a6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:42.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:42.810+0000 7fcb91437700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb8c1080a0 0x7fcb8c1a6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36362/0 (socket says 172.21.15.12:36362) 2024-05-17T22:35:42.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:42.810+0000 7fcb91437700 1 -- 172.21.15.12:0/1768577024 learned_addr learned my addr 172.21.15.12:0/1768577024 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:35:42.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:42.810+0000 7fcb90c36700 1 --2- 172.21.15.12:0/1768577024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb8c102e20 0x7fcb8c19b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:42.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:42.810+0000 7fcb8bfff700 1 --2- 172.21.15.12:0/1768577024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb8c107750 0x7fcb8c19bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:42.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:42.811+0000 7fcb91437700 1 -- 172.21.15.12:0/1768577024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb8c102e20 msgr2=0x7fcb8c19b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:42.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:42.811+0000 7fcb91437700 1 --2- 172.21.15.12:0/1768577024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb8c102e20 0x7fcb8c19b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:42.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:42.811+0000 7fcb91437700 1 -- 172.21.15.12:0/1768577024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb8c107750 msgr2=0x7fcb8c19bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:42.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:42.811+0000 7fcb91437700 1 --2- 172.21.15.12:0/1768577024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb8c107750 0x7fcb8c19bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:42.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:42.811+0000 7fcb91437700 1 -- 172.21.15.12:0/1768577024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb8c19c300 con 0x7fcb8c1080a0 2024-05-17T22:35:42.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:42.811+0000 7fcb91437700 1 --2- 172.21.15.12:0/1768577024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb8c1080a0 0x7fcb8c1a6640 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400ada0 tx=0x7fcb8400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:35:42.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:42.811+0000 7fcb89ffb700 1 -- 172.21.15.12:0/1768577024 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb84017020 con 0x7fcb8c1080a0 2024-05-17T22:35:42.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:42.811+0000 7fcb89ffb700 1 -- 172.21.15.12:0/1768577024 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb8400cd80 con 0x7fcb8c1080a0 2024-05-17T22:35:42.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:42.811+0000 7fcb92e9a700 1 -- 172.21.15.12:0/1768577024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb8c1a6be0 con 0x7fcb8c1080a0 2024-05-17T22:35:42.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:42.811+0000 7fcb89ffb700 1 -- 172.21.15.12:0/1768577024 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb84005260 con 0x7fcb8c1080a0 2024-05-17T22:35:42.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:42.812+0000 7fcb92e9a700 1 -- 172.21.15.12:0/1768577024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb8c1a70f0 con 0x7fcb8c1080a0 2024-05-17T22:35:42.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:42.812+0000 7fcb92e9a700 1 -- 172.21.15.12:0/1768577024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb8c103230 con 0x7fcb8c1080a0 2024-05-17T22:35:42.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:42.814+0000 7fcb89ffb700 1 -- 172.21.15.12:0/1768577024 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fcb8400c8d0 con 0x7fcb8c1080a0 2024-05-17T22:35:42.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:42.815+0000 7fcb89ffb700 1 --2- 172.21.15.12:0/1768577024 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fcb7c0b2900 0x7fcb7c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:42.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:42.815+0000 7fcb90c36700 1 --2- 172.21.15.12:0/1768577024 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fcb7c0b2900 0x7fcb7c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:42.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:42.815+0000 7fcb89ffb700 1 -- 172.21.15.12:0/1768577024 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fcb840e1bf0 con 0x7fcb8c1080a0 2024-05-17T22:35:42.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:42.815+0000 7fcb90c36700 1 --2- 172.21.15.12:0/1768577024 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fcb7c0b2900 0x7fcb7c0b4e10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcb8000ac50 tx=0x7fcb8000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:35:42.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:42.819+0000 7fcb89ffb700 1 -- 172.21.15.12:0/1768577024 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcb84021020 con 0x7fcb8c1080a0 2024-05-17T22:35:43.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:43.108+0000 7fcb92e9a700 1 -- 172.21.15.12:0/1768577024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fcb8c06c670 con 0x7fcb8c1080a0 2024-05-17T22:35:43.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:43.982+0000 7fcb89ffb700 1 -- 172.21.15.12:0/1768577024 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7fcb840a95e0 con 0x7fcb8c1080a0 2024-05-17T22:35:43.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:43.984+0000 7fcb92e9a700 1 -- 172.21.15.12:0/1768577024 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fcb7c0b2900 msgr2=0x7fcb7c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:43.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:43.984+0000 7fcb92e9a700 1 --2- 172.21.15.12:0/1768577024 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fcb7c0b2900 0x7fcb7c0b4e10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcb8000ac50 tx=0x7fcb8000a380 comp rx=0 tx=0).stop 2024-05-17T22:35:43.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:43.984+0000 7fcb92e9a700 1 -- 172.21.15.12:0/1768577024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb8c1080a0 msgr2=0x7fcb8c1a6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:43.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:43.984+0000 7fcb92e9a700 1 --2- 172.21.15.12:0/1768577024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb8c1080a0 0x7fcb8c1a6640 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400ada0 tx=0x7fcb8400c330 comp rx=0 tx=0).stop 2024-05-17T22:35:43.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:43.985+0000 7fcb92e9a700 1 -- 172.21.15.12:0/1768577024 shutdown_connections 2024-05-17T22:35:43.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:43.985+0000 7fcb92e9a700 1 --2- 172.21.15.12:0/1768577024 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fcb7c0b2900 0x7fcb7c0b4e10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:43.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:43.985+0000 7fcb92e9a700 1 --2- 172.21.15.12:0/1768577024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb8c1080a0 0x7fcb8c1a6640 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:43.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:43.985+0000 7fcb92e9a700 1 --2- 172.21.15.12:0/1768577024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb8c102e20 0x7fcb8c19b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:43.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:43.985+0000 7fcb92e9a700 1 --2- 172.21.15.12:0/1768577024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb8c107750 0x7fcb8c19bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:43.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:43.985+0000 7fcb92e9a700 1 -- 172.21.15.12:0/1768577024 >> 172.21.15.12:0/1768577024 conn(0x7fcb8c079bd0 msgr2=0x7fcb8c1054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:35:43.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:43.985+0000 7fcb92e9a700 1 -- 172.21.15.12:0/1768577024 shutdown_connections 2024-05-17T22:35:43.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:43.985+0000 7fcb92e9a700 1 -- 172.21.15.12:0/1768577024 wait complete. 2024-05-17T22:35:44.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:43 smithi151 ceph-mon[36101]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:35:44.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:43 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:35:44.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:43 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:35:44.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:43 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/1768577024' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-17T22:35:44.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:43 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:35:44.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:43 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:35:44.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:43 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:35:44.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:43 smithi096 ceph-mon[36065]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:35:44.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:43 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:35:44.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:43 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:35:44.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:43 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/1768577024' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-17T22:35:44.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:43 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:35:44.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:43 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:35:44.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:43 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:35:44.300 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:43 smithi012 ceph-mon[31632]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:35:44.300 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:43 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:35:44.300 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:43 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:35:44.300 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:43 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/1768577024' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-17T22:35:44.300 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:43 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:35:44.300 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:43 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:35:44.300 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:43 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:35:44.632 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-05-17T22:35:44.633 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-05-17T22:35:44.886 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:35:45.251 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:44 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/1768577024' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-17T22:35:45.251 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:44 smithi012 ceph-mon[31632]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T22:35:45.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:44 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/1768577024' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-17T22:35:45.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:44 smithi096 ceph-mon[36065]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T22:35:45.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:44 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/1768577024' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-17T22:35:45.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:44 smithi151 ceph-mon[36101]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T22:35:46.301 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:45 smithi096 ceph-mon[36065]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:35:46.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:45 smithi012 ceph-mon[31632]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:35:46.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:45 smithi151 ceph-mon[36101]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:35:46.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.613+0000 7f6ae8742700 1 -- 172.21.15.12:0/2990843086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ae0107730 msgr2=0x7f6ae0107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:46.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.613+0000 7f6ae8742700 1 --2- 172.21.15.12:0/2990843086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ae0107730 0x7f6ae0107b40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4009990 tx=0x7f6ad4009ca0 comp rx=0 tx=0).stop 2024-05-17T22:35:46.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.613+0000 7f6ae8742700 1 -- 172.21.15.12:0/2990843086 shutdown_connections 2024-05-17T22:35:46.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.613+0000 7f6ae8742700 1 --2- 172.21.15.12:0/2990843086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ae0108080 0x7f6ae010f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:46.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.613+0000 7f6ae8742700 1 --2- 172.21.15.12:0/2990843086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ae0102e00 0x7f6ae0103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:46.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.613+0000 7f6ae8742700 1 --2- 172.21.15.12:0/2990843086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ae0107730 0x7f6ae0107b40 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:46.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.613+0000 7f6ae8742700 1 -- 172.21.15.12:0/2990843086 >> 172.21.15.12:0/2990843086 conn(0x7f6ae0079bd0 msgr2=0x7f6ae0079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:35:46.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.613+0000 7f6ae8742700 1 -- 172.21.15.12:0/2990843086 shutdown_connections 2024-05-17T22:35:46.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.613+0000 7f6ae8742700 1 -- 172.21.15.12:0/2990843086 wait complete. 2024-05-17T22:35:46.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.614+0000 7f6ae8742700 1 Processor -- start 2024-05-17T22:35:46.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.614+0000 7f6ae8742700 1 -- start start 2024-05-17T22:35:46.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.615+0000 7f6ae8742700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ae0102e00 0x7f6ae01993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:46.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.615+0000 7f6ae8742700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ae0107730 0x7f6ae0199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:46.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.615+0000 7f6ae8742700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ae0108080 0x7f6ae019e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:46.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.615+0000 7f6ae8742700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae0112c90 con 0x7f6ae0102e00 2024-05-17T22:35:46.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.615+0000 7f6ae8742700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae0112b10 con 0x7f6ae0107730 2024-05-17T22:35:46.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.615+0000 7f6ae8742700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae0112e10 con 0x7f6ae0108080 2024-05-17T22:35:46.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.615+0000 7f6ae64de700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ae0102e00 0x7f6ae01993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:46.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.615+0000 7f6ae64de700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ae0102e00 0x7f6ae01993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36374/0 (socket says 172.21.15.12:36374) 2024-05-17T22:35:46.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.615+0000 7f6ae64de700 1 -- 172.21.15.12:0/3719734136 learned_addr learned my addr 172.21.15.12:0/3719734136 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:35:46.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.615+0000 7f6ae5cdd700 1 --2- 172.21.15.12:0/3719734136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ae0107730 0x7f6ae0199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:46.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.615+0000 7f6ae6cdf700 1 --2- 172.21.15.12:0/3719734136 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ae0108080 0x7f6ae019e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:46.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.615+0000 7f6ae64de700 1 -- 172.21.15.12:0/3719734136 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ae0108080 msgr2=0x7f6ae019e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:46.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.615+0000 7f6ae64de700 1 --2- 172.21.15.12:0/3719734136 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ae0108080 0x7f6ae019e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:46.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.615+0000 7f6ae64de700 1 -- 172.21.15.12:0/3719734136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ae0107730 msgr2=0x7f6ae0199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:46.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.615+0000 7f6ae64de700 1 --2- 172.21.15.12:0/3719734136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ae0107730 0x7f6ae0199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:46.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.615+0000 7f6ae64de700 1 -- 172.21.15.12:0/3719734136 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ae019a120 con 0x7f6ae0102e00 2024-05-17T22:35:46.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.615+0000 7f6ae64de700 1 --2- 172.21.15.12:0/3719734136 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ae0102e00 0x7f6ae01993e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0007e20 tx=0x7f6ad000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:35:46.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.616+0000 7f6adf7fe700 1 -- 172.21.15.12:0/3719734136 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0016020 con 0x7f6ae0102e00 2024-05-17T22:35:46.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.616+0000 7f6adf7fe700 1 -- 172.21.15.12:0/3719734136 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ad000ad40 con 0x7f6ae0102e00 2024-05-17T22:35:46.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.616+0000 7f6ae8742700 1 -- 172.21.15.12:0/3719734136 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ae019e9d0 con 0x7f6ae0102e00 2024-05-17T22:35:46.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.616+0000 7f6adf7fe700 1 -- 172.21.15.12:0/3719734136 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0011670 con 0x7f6ae0102e00 2024-05-17T22:35:46.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.616+0000 7f6ae8742700 1 -- 172.21.15.12:0/3719734136 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ae019ef30 con 0x7f6ae0102e00 2024-05-17T22:35:46.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.617+0000 7f6ae8742700 1 -- 172.21.15.12:0/3719734136 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ae0103210 con 0x7f6ae0102e00 2024-05-17T22:35:46.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.619+0000 7f6adf7fe700 1 -- 172.21.15.12:0/3719734136 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f6ad0019020 con 0x7f6ae0102e00 2024-05-17T22:35:46.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.619+0000 7f6adf7fe700 1 --2- 172.21.15.12:0/3719734136 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f6ac80b2900 0x7f6ac80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:46.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.619+0000 7f6ae5cdd700 1 --2- 172.21.15.12:0/3719734136 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f6ac80b2900 0x7f6ac80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:46.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.619+0000 7f6adf7fe700 1 -- 172.21.15.12:0/3719734136 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f6ad00e26a0 con 0x7f6ae0102e00 2024-05-17T22:35:46.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.620+0000 7f6ae5cdd700 1 --2- 172.21.15.12:0/3719734136 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f6ac80b2900 0x7f6ac80b4e10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6ad400b5c0 tx=0x7f6ad4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:35:46.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.622+0000 7f6adf7fe700 1 -- 172.21.15.12:0/3719734136 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6ad0020070 con 0x7f6ae0102e00 2024-05-17T22:35:46.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.887+0000 7f6ae8742700 1 -- 172.21.15.12:0/3719734136 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f6ae006bcb0 con 0x7f6ae0102e00 2024-05-17T22:35:46.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.995+0000 7f6adf7fe700 1 -- 172.21.15.12:0/3719734136 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f6ad00aa060 con 0x7f6ae0102e00 2024-05-17T22:35:47.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.998+0000 7f6ae8742700 1 -- 172.21.15.12:0/3719734136 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f6ac80b2900 msgr2=0x7f6ac80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:47.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.998+0000 7f6ae8742700 1 --2- 172.21.15.12:0/3719734136 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f6ac80b2900 0x7f6ac80b4e10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6ad400b5c0 tx=0x7f6ad4005960 comp rx=0 tx=0).stop 2024-05-17T22:35:47.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.998+0000 7f6ae8742700 1 -- 172.21.15.12:0/3719734136 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ae0102e00 msgr2=0x7f6ae01993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:47.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.998+0000 7f6ae8742700 1 --2- 172.21.15.12:0/3719734136 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ae0102e00 0x7f6ae01993e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0007e20 tx=0x7f6ad000a340 comp rx=0 tx=0).stop 2024-05-17T22:35:47.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.998+0000 7f6ae8742700 1 -- 172.21.15.12:0/3719734136 shutdown_connections 2024-05-17T22:35:47.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.998+0000 7f6ae8742700 1 --2- 172.21.15.12:0/3719734136 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f6ac80b2900 0x7f6ac80b4e10 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:47.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.998+0000 7f6ae8742700 1 --2- 172.21.15.12:0/3719734136 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ae0108080 0x7f6ae019e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:47.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.998+0000 7f6ae8742700 1 --2- 172.21.15.12:0/3719734136 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ae0102e00 0x7f6ae01993e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:47.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.998+0000 7f6ae8742700 1 --2- 172.21.15.12:0/3719734136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ae0107730 0x7f6ae0199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:47.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.998+0000 7f6ae8742700 1 -- 172.21.15.12:0/3719734136 >> 172.21.15.12:0/3719734136 conn(0x7f6ae0079bd0 msgr2=0x7f6ae0105870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:35:47.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.998+0000 7f6ae8742700 1 -- 172.21.15.12:0/3719734136 shutdown_connections 2024-05-17T22:35:47.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:46.998+0000 7f6ae8742700 1 -- 172.21.15.12:0/3719734136 wait complete. 2024-05-17T22:35:47.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:46 smithi096 ceph-mon[36065]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:35:47.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:46 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/3719734136' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-17T22:35:47.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:46 smithi012 ceph-mon[31632]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:35:47.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:46 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/3719734136' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-17T22:35:47.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:46 smithi151 ceph-mon[36101]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:35:47.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:46 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/3719734136' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-17T22:35:47.655 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-05-17T22:35:47.910 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:35:48.268 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:47 smithi012 ceph-mon[31632]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-17T22:35:48.268 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:47 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/3719734136' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-17T22:35:48.269 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:47 smithi012 ceph-mon[31632]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-17T22:35:48.269 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:47 smithi012 ceph-mon[31632]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T22:35:48.269 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:47 smithi012 ceph-mon[31632]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-17T22:35:48.269 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:47 smithi012 ceph-mon[31632]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-17T22:35:48.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:47 smithi096 ceph-mon[36065]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-17T22:35:48.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:47 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/3719734136' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-17T22:35:48.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:47 smithi096 ceph-mon[36065]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-17T22:35:48.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:47 smithi096 ceph-mon[36065]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T22:35:48.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:47 smithi096 ceph-mon[36065]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-17T22:35:48.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:47 smithi096 ceph-mon[36065]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-17T22:35:48.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:48 smithi151 ceph-mon[36101]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-17T22:35:48.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:48 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/3719734136' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-17T22:35:48.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:48 smithi151 ceph-mon[36101]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-17T22:35:48.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:48 smithi151 ceph-mon[36101]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T22:35:48.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:48 smithi151 ceph-mon[36101]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-17T22:35:48.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:48 smithi151 ceph-mon[36101]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-17T22:35:49.012 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 17 22:35:48 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-b[66490]: 2024-05-17T22:35:48.567+0000 7f9af454c700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-17T22:35:49.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:49 smithi096 ceph-mon[36065]: mds.? [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] up:active 2024-05-17T22:35:49.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:49 smithi096 ceph-mon[36065]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-17T22:35:49.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:49 smithi096 ceph-mon[36065]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-17T22:35:49.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:49 smithi096 ceph-mon[36065]: Cluster is now healthy 2024-05-17T22:35:49.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:49 smithi096 ceph-mon[36065]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-17T22:35:49.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:49 smithi096 ceph-mon[36065]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-05-17T22:35:49.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:49 smithi096 ceph-mon[36065]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-17T22:35:49.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:49 smithi096 ceph-mon[36065]: pgmap v334: 129 pgs: 129 active+clean; 582 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 2 op/s 2024-05-17T22:35:49.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:49 smithi012 ceph-mon[31632]: mds.? [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] up:active 2024-05-17T22:35:49.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:49 smithi012 ceph-mon[31632]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-17T22:35:49.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:49 smithi012 ceph-mon[31632]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-17T22:35:49.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:49 smithi012 ceph-mon[31632]: Cluster is now healthy 2024-05-17T22:35:49.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:49 smithi012 ceph-mon[31632]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-17T22:35:49.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:49 smithi012 ceph-mon[31632]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-05-17T22:35:49.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:49 smithi012 ceph-mon[31632]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-17T22:35:49.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:49 smithi012 ceph-mon[31632]: pgmap v334: 129 pgs: 129 active+clean; 582 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 2 op/s 2024-05-17T22:35:49.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:49 smithi151 ceph-mon[36101]: mds.? [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] up:active 2024-05-17T22:35:49.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:49 smithi151 ceph-mon[36101]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-17T22:35:49.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:49 smithi151 ceph-mon[36101]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-17T22:35:49.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:49 smithi151 ceph-mon[36101]: Cluster is now healthy 2024-05-17T22:35:49.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:49 smithi151 ceph-mon[36101]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-17T22:35:49.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:49 smithi151 ceph-mon[36101]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-05-17T22:35:49.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:49 smithi151 ceph-mon[36101]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-17T22:35:49.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:49 smithi151 ceph-mon[36101]: pgmap v334: 129 pgs: 129 active+clean; 582 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 2 op/s 2024-05-17T22:35:49.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:49.592+0000 7f51f12d3700 1 -- 172.21.15.12:0/1161804301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51ec1080a0 msgr2=0x7f51ec10fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:49.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:49.592+0000 7f51f12d3700 1 --2- 172.21.15.12:0/1161804301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51ec1080a0 0x7f51ec10fa00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f51e400b0a0 tx=0x7f51e400b3b0 comp rx=0 tx=0).stop 2024-05-17T22:35:49.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:49.592+0000 7f51f12d3700 1 -- 172.21.15.12:0/1161804301 shutdown_connections 2024-05-17T22:35:49.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:49.592+0000 7f51f12d3700 1 --2- 172.21.15.12:0/1161804301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51ec1080a0 0x7f51ec10fa00 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:49.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:49.592+0000 7f51f12d3700 1 --2- 172.21.15.12:0/1161804301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51ec102e70 0x7f51ec103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:49.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:49.592+0000 7f51f12d3700 1 --2- 172.21.15.12:0/1161804301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51ec107750 0x7f51ec107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:49.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:49.592+0000 7f51f12d3700 1 -- 172.21.15.12:0/1161804301 >> 172.21.15.12:0/1161804301 conn(0x7f51ec079ae0 msgr2=0x7f51ec079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:35:49.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:49.592+0000 7f51f12d3700 1 -- 172.21.15.12:0/1161804301 shutdown_connections 2024-05-17T22:35:49.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:49.592+0000 7f51f12d3700 1 -- 172.21.15.12:0/1161804301 wait complete. 2024-05-17T22:35:49.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:49.593+0000 7f51f12d3700 1 Processor -- start 2024-05-17T22:35:49.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:49.593+0000 7f51f12d3700 1 -- start start 2024-05-17T22:35:49.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:49.593+0000 7f51f12d3700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51ec102e70 0x7f51ec19b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:49.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:49.593+0000 7f51f12d3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51ec107750 0x7f51ec19bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:49.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:49.593+0000 7f51f12d3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51ec1080a0 0x7f51ec19c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:49.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:49.593+0000 7f51f12d3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f51ec112d50 con 0x7f51ec1080a0 2024-05-17T22:35:49.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:49.593+0000 7f51f12d3700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f51ec112bd0 con 0x7f51ec107750 2024-05-17T22:35:49.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:49.593+0000 7f51f12d3700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f51ec112ed0 con 0x7f51ec102e70 2024-05-17T22:35:49.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:49.594+0000 7f51eb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51ec1080a0 0x7f51ec19c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:49.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:49.594+0000 7f51eb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51ec1080a0 0x7f51ec19c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36400/0 (socket says 172.21.15.12:36400) 2024-05-17T22:35:49.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:49.594+0000 7f51eb7fe700 1 -- 172.21.15.12:0/3449300814 learned_addr learned my addr 172.21.15.12:0/3449300814 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:35:49.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:49.594+0000 7f51eaffd700 1 --2- 172.21.15.12:0/3449300814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51ec102e70 0x7f51ec19b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:49.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:49.594+0000 7f51ea7fc700 1 --2- 172.21.15.12:0/3449300814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51ec107750 0x7f51ec19bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:49.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:49.594+0000 7f51eb7fe700 1 -- 172.21.15.12:0/3449300814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51ec102e70 msgr2=0x7f51ec19b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:49.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:49.594+0000 7f51eb7fe700 1 --2- 172.21.15.12:0/3449300814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51ec102e70 0x7f51ec19b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:49.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:49.594+0000 7f51eb7fe700 1 -- 172.21.15.12:0/3449300814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51ec107750 msgr2=0x7f51ec19bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:49.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:49.594+0000 7f51eb7fe700 1 --2- 172.21.15.12:0/3449300814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51ec107750 0x7f51ec19bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:49.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:49.594+0000 7f51eb7fe700 1 -- 172.21.15.12:0/3449300814 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51ec1a6950 con 0x7f51ec1080a0 2024-05-17T22:35:49.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:49.594+0000 7f51eb7fe700 1 --2- 172.21.15.12:0/3449300814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51ec1080a0 0x7f51ec19c0a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f51e400b830 tx=0x7f51e4004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:35:49.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:49.594+0000 7f51d3fff700 1 -- 172.21.15.12:0/3449300814 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e4021070 con 0x7f51ec1080a0 2024-05-17T22:35:49.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:49.594+0000 7f51d3fff700 1 -- 172.21.15.12:0/3449300814 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51e4007e30 con 0x7f51ec1080a0 2024-05-17T22:35:49.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:49.594+0000 7f51f12d3700 1 -- 172.21.15.12:0/3449300814 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51ec1a6be0 con 0x7f51ec1080a0 2024-05-17T22:35:49.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:49.594+0000 7f51d3fff700 1 -- 172.21.15.12:0/3449300814 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e4004700 con 0x7f51ec1080a0 2024-05-17T22:35:49.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:49.595+0000 7f51f12d3700 1 -- 172.21.15.12:0/3449300814 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51ec1a7090 con 0x7f51ec1080a0 2024-05-17T22:35:49.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:49.595+0000 7f51f12d3700 1 -- 172.21.15.12:0/3449300814 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51ec103280 con 0x7f51ec1080a0 2024-05-17T22:35:49.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:49.598+0000 7f51d3fff700 1 -- 172.21.15.12:0/3449300814 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f51e4003d70 con 0x7f51ec1080a0 2024-05-17T22:35:49.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:49.599+0000 7f51d3fff700 1 --2- 172.21.15.12:0/3449300814 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f51d40b2880 0x7f51d40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:49.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:49.599+0000 7f51eaffd700 1 --2- 172.21.15.12:0/3449300814 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f51d40b2880 0x7f51d40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:49.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:49.599+0000 7f51d3fff700 1 -- 172.21.15.12:0/3449300814 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f51e400f020 con 0x7f51ec1080a0 2024-05-17T22:35:49.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:49.599+0000 7f51eaffd700 1 --2- 172.21.15.12:0/3449300814 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f51d40b2880 0x7f51d40b4d90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f51dc005950 tx=0x7f51dc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:35:49.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:49.602+0000 7f51d3fff700 1 -- 172.21.15.12:0/3449300814 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f51e402a020 con 0x7f51ec1080a0 2024-05-17T22:35:49.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:49.862+0000 7f51f12d3700 1 -- 172.21.15.12:0/3449300814 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f51ec06c630 con 0x7f51ec1080a0 2024-05-17T22:35:50.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:50.028+0000 7f51d3fff700 1 -- 172.21.15.12:0/3449300814 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v24) ==== 121+0+0 (secure 0 0 0) 0x7f51e40ab8e0 con 0x7f51ec1080a0 2024-05-17T22:35:50.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:50.031+0000 7f51f12d3700 1 -- 172.21.15.12:0/3449300814 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f51d40b2880 msgr2=0x7f51d40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:50.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:50.031+0000 7f51f12d3700 1 --2- 172.21.15.12:0/3449300814 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f51d40b2880 0x7f51d40b4d90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f51dc005950 tx=0x7f51dc00b410 comp rx=0 tx=0).stop 2024-05-17T22:35:50.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:50.031+0000 7f51f12d3700 1 -- 172.21.15.12:0/3449300814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51ec1080a0 msgr2=0x7f51ec19c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:50.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:50.031+0000 7f51f12d3700 1 --2- 172.21.15.12:0/3449300814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51ec1080a0 0x7f51ec19c0a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f51e400b830 tx=0x7f51e4004ce0 comp rx=0 tx=0).stop 2024-05-17T22:35:50.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:50.031+0000 7f51f12d3700 1 -- 172.21.15.12:0/3449300814 shutdown_connections 2024-05-17T22:35:50.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:50.031+0000 7f51f12d3700 1 --2- 172.21.15.12:0/3449300814 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f51d40b2880 0x7f51d40b4d90 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:50.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:50.031+0000 7f51f12d3700 1 --2- 172.21.15.12:0/3449300814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51ec1080a0 0x7f51ec19c0a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:50.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:50.031+0000 7f51f12d3700 1 --2- 172.21.15.12:0/3449300814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51ec102e70 0x7f51ec19b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:50.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:50.031+0000 7f51f12d3700 1 --2- 172.21.15.12:0/3449300814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51ec107750 0x7f51ec19bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:50.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:50.031+0000 7f51f12d3700 1 -- 172.21.15.12:0/3449300814 >> 172.21.15.12:0/3449300814 conn(0x7f51ec079ae0 msgr2=0x7f51ec105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:35:50.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:50.031+0000 7f51f12d3700 1 -- 172.21.15.12:0/3449300814 shutdown_connections 2024-05-17T22:35:50.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:50.031+0000 7f51f12d3700 1 -- 172.21.15.12:0/3449300814 wait complete. 2024-05-17T22:35:50.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:50 smithi096 ceph-mon[36065]: mds.? [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] up:active 2024-05-17T22:35:50.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:50 smithi096 ceph-mon[36065]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-05-17T22:35:50.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:50 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/3449300814' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-17T22:35:50.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:50 smithi012 ceph-mon[31632]: mds.? [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] up:active 2024-05-17T22:35:50.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:50 smithi012 ceph-mon[31632]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-05-17T22:35:50.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:50 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/3449300814' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-17T22:35:50.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:50 smithi151 ceph-mon[36101]: mds.? [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] up:active 2024-05-17T22:35:50.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:50 smithi151 ceph-mon[36101]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-05-17T22:35:50.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:50 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/3449300814' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-17T22:35:50.704 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-05-17T22:35:50.956 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:35:51.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:51 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/3449300814' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-17T22:35:51.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:51 smithi096 ceph-mon[36065]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-05-17T22:35:51.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:51 smithi096 ceph-mon[36065]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:35:51.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:51 smithi096 ceph-mon[36065]: pgmap v335: 129 pgs: 129 active+clean; 583 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 4 op/s 2024-05-17T22:35:51.311 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:51 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/3449300814' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-17T22:35:51.311 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:51 smithi012 ceph-mon[31632]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-05-17T22:35:51.311 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:51 smithi012 ceph-mon[31632]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:35:51.311 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:51 smithi012 ceph-mon[31632]: pgmap v335: 129 pgs: 129 active+clean; 583 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 4 op/s 2024-05-17T22:35:51.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:51 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/3449300814' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-17T22:35:51.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:51 smithi151 ceph-mon[36101]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-05-17T22:35:51.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:51 smithi151 ceph-mon[36101]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:35:51.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:51 smithi151 ceph-mon[36101]: pgmap v335: 129 pgs: 129 active+clean; 583 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 4 op/s 2024-05-17T22:35:52.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:52.722+0000 7f40f717a700 1 -- 172.21.15.12:0/2684064762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40f0108850 msgr2=0x7f40f0108c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:52.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:52.722+0000 7f40f717a700 1 --2- 172.21.15.12:0/2684064762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40f0108850 0x7f40f0108c60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f40e4009990 tx=0x7f40e4009ca0 comp rx=0 tx=0).stop 2024-05-17T22:35:52.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:52.722+0000 7f40f717a700 1 -- 172.21.15.12:0/2684064762 shutdown_connections 2024-05-17T22:35:52.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:52.722+0000 7f40f717a700 1 --2- 172.21.15.12:0/2684064762 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40f01091a0 0x7f40f0110b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:52.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:52.722+0000 7f40f717a700 1 --2- 172.21.15.12:0/2684064762 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40f0103f20 0x7f40f0104330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:52.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:52.722+0000 7f40f717a700 1 --2- 172.21.15.12:0/2684064762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40f0108850 0x7f40f0108c60 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:52.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:52.722+0000 7f40f717a700 1 -- 172.21.15.12:0/2684064762 >> 172.21.15.12:0/2684064762 conn(0x7f40f00ff9c0 msgr2=0x7f40f0101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:35:52.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:52.722+0000 7f40f717a700 1 -- 172.21.15.12:0/2684064762 shutdown_connections 2024-05-17T22:35:52.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:52.722+0000 7f40f717a700 1 -- 172.21.15.12:0/2684064762 wait complete. 2024-05-17T22:35:52.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:52.723+0000 7f40f717a700 1 Processor -- start 2024-05-17T22:35:52.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:52.723+0000 7f40f717a700 1 -- start start 2024-05-17T22:35:52.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:52.723+0000 7f40f717a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40f0103f20 0x7f40f01a0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:52.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:52.723+0000 7f40f717a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40f0108850 0x7f40f01a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:52.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:52.724+0000 7f40f717a700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40f01091a0 0x7f40f019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:52.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:52.724+0000 7f40f717a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f40f0113db0 con 0x7f40f0108850 2024-05-17T22:35:52.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:52.724+0000 7f40f717a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f40f0113c30 con 0x7f40f0103f20 2024-05-17T22:35:52.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:52.724+0000 7f40f717a700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f40f0113f30 con 0x7f40f01091a0 2024-05-17T22:35:52.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:52.724+0000 7f40effff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40f0108850 0x7f40f01a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:52.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:52.724+0000 7f40effff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40f0108850 0x7f40f01a0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37764/0 (socket says 172.21.15.12:37764) 2024-05-17T22:35:52.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:52.724+0000 7f40effff700 1 -- 172.21.15.12:0/4186015452 learned_addr learned my addr 172.21.15.12:0/4186015452 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:35:52.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:52.724+0000 7f40f4f16700 1 --2- 172.21.15.12:0/4186015452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40f0103f20 0x7f40f01a0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:52.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:52.724+0000 7f40f5717700 1 --2- 172.21.15.12:0/4186015452 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40f01091a0 0x7f40f019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:52.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:52.724+0000 7f40effff700 1 -- 172.21.15.12:0/4186015452 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40f01091a0 msgr2=0x7f40f019a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:52.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:52.724+0000 7f40effff700 1 --2- 172.21.15.12:0/4186015452 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40f01091a0 0x7f40f019a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:52.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:52.724+0000 7f40effff700 1 -- 172.21.15.12:0/4186015452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40f0103f20 msgr2=0x7f40f01a0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:52.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:52.724+0000 7f40effff700 1 --2- 172.21.15.12:0/4186015452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40f0103f20 0x7f40f01a0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:52.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:52.724+0000 7f40effff700 1 -- 172.21.15.12:0/4186015452 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f019ad40 con 0x7f40f0108850 2024-05-17T22:35:52.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:52.724+0000 7f40effff700 1 --2- 172.21.15.12:0/4186015452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40f0108850 0x7f40f01a0a50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f40e4000c00 tx=0x7f40e4005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:35:52.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:52.724+0000 7f40edffb700 1 -- 172.21.15.12:0/4186015452 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e4027070 con 0x7f40f0108850 2024-05-17T22:35:52.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:52.724+0000 7f40edffb700 1 -- 172.21.15.12:0/4186015452 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40e4003fe0 con 0x7f40f0108850 2024-05-17T22:35:52.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:52.724+0000 7f40f717a700 1 -- 172.21.15.12:0/4186015452 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40f019afd0 con 0x7f40f0108850 2024-05-17T22:35:52.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:52.724+0000 7f40edffb700 1 -- 172.21.15.12:0/4186015452 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e4022c30 con 0x7f40f0108850 2024-05-17T22:35:52.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:52.725+0000 7f40f717a700 1 -- 172.21.15.12:0/4186015452 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40f019b500 con 0x7f40f0108850 2024-05-17T22:35:52.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:52.726+0000 7f40f717a700 1 -- 172.21.15.12:0/4186015452 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40f01053d0 con 0x7f40f0108850 2024-05-17T22:35:52.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:52.727+0000 7f40edffb700 1 -- 172.21.15.12:0/4186015452 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f40e402a070 con 0x7f40f0108850 2024-05-17T22:35:52.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:52.728+0000 7f40edffb700 1 --2- 172.21.15.12:0/4186015452 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f40d80b2900 0x7f40d80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:52.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:52.728+0000 7f40f4f16700 1 --2- 172.21.15.12:0/4186015452 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f40d80b2900 0x7f40d80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:52.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:52.728+0000 7f40edffb700 1 -- 172.21.15.12:0/4186015452 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f40e40e3420 con 0x7f40f0108850 2024-05-17T22:35:52.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:52.728+0000 7f40f4f16700 1 --2- 172.21.15.12:0/4186015452 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f40d80b2900 0x7f40d80b4e10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f40e0005950 tx=0x7f40e000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:35:52.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:52.732+0000 7f40edffb700 1 -- 172.21.15.12:0/4186015452 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f40f019b500 con 0x7f40f0108850 2024-05-17T22:35:53.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:52.998+0000 7f40f717a700 1 -- 172.21.15.12:0/4186015452 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f40f0104330 con 0x7f40f0108850 2024-05-17T22:35:53.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:53.549+0000 7f40edffb700 1 -- 172.21.15.12:0/4186015452 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v26) ==== 115+0+0 (secure 0 0 0) 0x7f40e4013070 con 0x7f40f0108850 2024-05-17T22:35:53.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:53.552+0000 7f40f717a700 1 -- 172.21.15.12:0/4186015452 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f40d80b2900 msgr2=0x7f40d80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:53.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:53.552+0000 7f40f717a700 1 --2- 172.21.15.12:0/4186015452 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f40d80b2900 0x7f40d80b4e10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f40e0005950 tx=0x7f40e000b410 comp rx=0 tx=0).stop 2024-05-17T22:35:53.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:53.552+0000 7f40f717a700 1 -- 172.21.15.12:0/4186015452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40f0108850 msgr2=0x7f40f01a0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:53.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:53.552+0000 7f40f717a700 1 --2- 172.21.15.12:0/4186015452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40f0108850 0x7f40f01a0a50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f40e4000c00 tx=0x7f40e4005ef0 comp rx=0 tx=0).stop 2024-05-17T22:35:53.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:53.552+0000 7f40f717a700 1 -- 172.21.15.12:0/4186015452 shutdown_connections 2024-05-17T22:35:53.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:53.552+0000 7f40f717a700 1 --2- 172.21.15.12:0/4186015452 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f40d80b2900 0x7f40d80b4e10 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:53.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:53.552+0000 7f40f717a700 1 --2- 172.21.15.12:0/4186015452 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40f01091a0 0x7f40f019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:53.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:53.552+0000 7f40f717a700 1 --2- 172.21.15.12:0/4186015452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40f0103f20 0x7f40f01a0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:53.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:53.552+0000 7f40f717a700 1 --2- 172.21.15.12:0/4186015452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40f0108850 0x7f40f01a0a50 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:53.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:53.552+0000 7f40f717a700 1 -- 172.21.15.12:0/4186015452 >> 172.21.15.12:0/4186015452 conn(0x7f40f00ff9c0 msgr2=0x7f40f0101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:35:53.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:53.553+0000 7f40f717a700 1 -- 172.21.15.12:0/4186015452 shutdown_connections 2024-05-17T22:35:53.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:53.553+0000 7f40f717a700 1 -- 172.21.15.12:0/4186015452 wait complete. 2024-05-17T22:35:53.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:53 smithi012 ceph-mon[31632]: pgmap v336: 129 pgs: 129 active+clean; 583 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 1.8 KiB/s wr, 6 op/s 2024-05-17T22:35:53.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:53 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/4186015452' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-17T22:35:53.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:53 smithi151 ceph-mon[36101]: pgmap v336: 129 pgs: 129 active+clean; 583 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 1.8 KiB/s wr, 6 op/s 2024-05-17T22:35:53.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:53 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/4186015452' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-17T22:35:54.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:53 smithi096 ceph-mon[36065]: pgmap v336: 129 pgs: 129 active+clean; 583 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 1.8 KiB/s wr, 6 op/s 2024-05-17T22:35:54.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:53 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/4186015452' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-17T22:35:54.252 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-05-17T22:35:54.252 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-05-17T22:35:54.517 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:35:54.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:54 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/4186015452' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-17T22:35:54.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:54 smithi012 ceph-mon[31632]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:35:54.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:54 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/4186015452' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-17T22:35:54.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:54 smithi151 ceph-mon[36101]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:35:55.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:54 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/4186015452' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-17T22:35:55.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:54 smithi096 ceph-mon[36065]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:35:55.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:55 smithi012 ceph-mon[31632]: pgmap v337: 129 pgs: 129 active+clean; 584 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-17T22:35:55.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:55 smithi151 ceph-mon[36101]: pgmap v337: 129 pgs: 129 active+clean; 584 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-17T22:35:56.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:55 smithi096 ceph-mon[36065]: pgmap v337: 129 pgs: 129 active+clean; 584 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-17T22:35:56.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.266+0000 7fc77cd91700 1 -- 172.21.15.12:0/4097742587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc778107730 msgr2=0x7fc778107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:56.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.266+0000 7fc77cd91700 1 --2- 172.21.15.12:0/4097742587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc778107730 0x7fc778107b40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc76c009990 tx=0x7fc76c009ca0 comp rx=0 tx=0).stop 2024-05-17T22:35:56.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.266+0000 7fc77cd91700 1 -- 172.21.15.12:0/4097742587 shutdown_connections 2024-05-17T22:35:56.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.266+0000 7fc77cd91700 1 --2- 172.21.15.12:0/4097742587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc778108080 0x7fc77810f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:56.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.266+0000 7fc77cd91700 1 --2- 172.21.15.12:0/4097742587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc778102e00 0x7fc778103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:56.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.266+0000 7fc77cd91700 1 --2- 172.21.15.12:0/4097742587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc778107730 0x7fc778107b40 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:56.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.266+0000 7fc77cd91700 1 -- 172.21.15.12:0/4097742587 >> 172.21.15.12:0/4097742587 conn(0x7fc778079bd0 msgr2=0x7fc778079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:35:56.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.266+0000 7fc77cd91700 1 -- 172.21.15.12:0/4097742587 shutdown_connections 2024-05-17T22:35:56.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.266+0000 7fc77cd91700 1 -- 172.21.15.12:0/4097742587 wait complete. 2024-05-17T22:35:56.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.268+0000 7fc77cd91700 1 Processor -- start 2024-05-17T22:35:56.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.268+0000 7fc77cd91700 1 -- start start 2024-05-17T22:35:56.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.268+0000 7fc77cd91700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc778102e00 0x7fc77819b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:56.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.268+0000 7fc77cd91700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc778107730 0x7fc77819bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:56.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.268+0000 7fc77cd91700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc778108080 0x7fc77819c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:56.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.268+0000 7fc77cd91700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc778112ce0 con 0x7fc778102e00 2024-05-17T22:35:56.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.268+0000 7fc77cd91700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc778112b60 con 0x7fc778108080 2024-05-17T22:35:56.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.268+0000 7fc77cd91700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc778112e60 con 0x7fc778107730 2024-05-17T22:35:56.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.268+0000 7fc77659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc778102e00 0x7fc77819b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:56.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.268+0000 7fc77659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc778102e00 0x7fc77819b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37780/0 (socket says 172.21.15.12:37780) 2024-05-17T22:35:56.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.268+0000 7fc77659c700 1 -- 172.21.15.12:0/3029638649 learned_addr learned my addr 172.21.15.12:0/3029638649 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:35:56.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.268+0000 7fc775d9b700 1 --2- 172.21.15.12:0/3029638649 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc778107730 0x7fc77819bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:56.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.268+0000 7fc776d9d700 1 --2- 172.21.15.12:0/3029638649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc778108080 0x7fc77819c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:56.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.268+0000 7fc77659c700 1 -- 172.21.15.12:0/3029638649 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc778107730 msgr2=0x7fc77819bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:56.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.268+0000 7fc77659c700 1 --2- 172.21.15.12:0/3029638649 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc778107730 0x7fc77819bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:56.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.268+0000 7fc77659c700 1 -- 172.21.15.12:0/3029638649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc778108080 msgr2=0x7fc77819c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:56.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.268+0000 7fc77659c700 1 --2- 172.21.15.12:0/3029638649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc778108080 0x7fc77819c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:56.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.268+0000 7fc77659c700 1 -- 172.21.15.12:0/3029638649 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7781a69c0 con 0x7fc778102e00 2024-05-17T22:35:56.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.268+0000 7fc776d9d700 1 --2- 172.21.15.12:0/3029638649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc778108080 0x7fc77819c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:35:56.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.268+0000 7fc775d9b700 1 --2- 172.21.15.12:0/3029638649 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc778107730 0x7fc77819bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:35:56.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.269+0000 7fc77659c700 1 --2- 172.21.15.12:0/3029638649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc778102e00 0x7fc77819b670 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc76800c920 tx=0x7fc76800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:35:56.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.269+0000 7fc7677fe700 1 -- 172.21.15.12:0/3029638649 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc768007b00 con 0x7fc778102e00 2024-05-17T22:35:56.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.269+0000 7fc7677fe700 1 -- 172.21.15.12:0/3029638649 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc768004ce0 con 0x7fc778102e00 2024-05-17T22:35:56.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.269+0000 7fc77cd91700 1 -- 172.21.15.12:0/3029638649 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7781a6cb0 con 0x7fc778102e00 2024-05-17T22:35:56.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.269+0000 7fc7677fe700 1 -- 172.21.15.12:0/3029638649 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc768005650 con 0x7fc778102e00 2024-05-17T22:35:56.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.269+0000 7fc77cd91700 1 -- 172.21.15.12:0/3029638649 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7781a71a0 con 0x7fc778102e00 2024-05-17T22:35:56.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.270+0000 7fc77cd91700 1 -- 172.21.15.12:0/3029638649 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc778103210 con 0x7fc778102e00 2024-05-17T22:35:56.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.272+0000 7fc7677fe700 1 -- 172.21.15.12:0/3029638649 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc768004750 con 0x7fc778102e00 2024-05-17T22:35:56.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.273+0000 7fc7677fe700 1 --2- 172.21.15.12:0/3029638649 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fc7600b2880 0x7fc7600b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:56.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.273+0000 7fc775d9b700 1 --2- 172.21.15.12:0/3029638649 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fc7600b2880 0x7fc7600b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:56.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.273+0000 7fc7677fe700 1 -- 172.21.15.12:0/3029638649 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fc7680e25b0 con 0x7fc778102e00 2024-05-17T22:35:56.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.273+0000 7fc775d9b700 1 --2- 172.21.15.12:0/3029638649 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fc7600b2880 0x7fc7600b4d90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc76c00b5c0 tx=0x7fc76c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:35:56.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.276+0000 7fc7677fe700 1 -- 172.21.15.12:0/3029638649 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc768014030 con 0x7fc778102e00 2024-05-17T22:35:56.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.528+0000 7fc77cd91700 1 -- 172.21.15.12:0/3029638649 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fc77806bc70 con 0x7fc7600b2880 2024-05-17T22:35:56.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.559+0000 7fc7677fe700 1 -- 172.21.15.12:0/3029638649 <== mgr.14152 v2:172.21.15.12:6800/1388514047 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc77806bc70 con 0x7fc7600b2880 2024-05-17T22:35:56.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.561+0000 7fc77cd91700 1 -- 172.21.15.12:0/3029638649 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fc7600b2880 msgr2=0x7fc7600b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:56.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.561+0000 7fc77cd91700 1 --2- 172.21.15.12:0/3029638649 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fc7600b2880 0x7fc7600b4d90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc76c00b5c0 tx=0x7fc76c01a040 comp rx=0 tx=0).stop 2024-05-17T22:35:56.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.561+0000 7fc77cd91700 1 -- 172.21.15.12:0/3029638649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc778102e00 msgr2=0x7fc77819b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:56.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.561+0000 7fc77cd91700 1 --2- 172.21.15.12:0/3029638649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc778102e00 0x7fc77819b670 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc76800c920 tx=0x7fc76800cce0 comp rx=0 tx=0).stop 2024-05-17T22:35:56.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.561+0000 7fc77cd91700 1 -- 172.21.15.12:0/3029638649 shutdown_connections 2024-05-17T22:35:56.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.561+0000 7fc77cd91700 1 --2- 172.21.15.12:0/3029638649 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fc7600b2880 0x7fc7600b4d90 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:56.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.561+0000 7fc77cd91700 1 --2- 172.21.15.12:0/3029638649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc778108080 0x7fc77819c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:56.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.561+0000 7fc77cd91700 1 --2- 172.21.15.12:0/3029638649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc778102e00 0x7fc77819b670 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:56.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.561+0000 7fc77cd91700 1 --2- 172.21.15.12:0/3029638649 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc778107730 0x7fc77819bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:56.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.561+0000 7fc77cd91700 1 -- 172.21.15.12:0/3029638649 >> 172.21.15.12:0/3029638649 conn(0x7fc778079bd0 msgr2=0x7fc778105810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:35:56.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.561+0000 7fc77cd91700 1 -- 172.21.15.12:0/3029638649 shutdown_connections 2024-05-17T22:35:56.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:56.562+0000 7fc77cd91700 1 -- 172.21.15.12:0/3029638649 wait complete. 2024-05-17T22:35:56.842 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:35:56 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:35:56.534+0000 7f3b650c7700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-17T22:35:56.842 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:35:56 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:35:56.534+0000 7f3b650c7700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-17T22:35:56.842 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:35:56 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:35:56.534+0000 7f3b650c7700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-17T22:35:56.842 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:35:56 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:35:56.534+0000 7f3b650c7700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-17T22:35:56.842 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:35:56 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:35:56.534+0000 7f3b650c7700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-17T22:35:57.265 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-17T22:35:57.521 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:35:57.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:57 smithi012 ceph-mon[31632]: pgmap v338: 129 pgs: 129 active+clean; 584 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-17T22:35:57.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:57 smithi012 ceph-mon[31632]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:35:57.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:57 smithi012 ceph-mon[31632]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-05-17T22:35:57.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:57 smithi151 ceph-mon[36101]: pgmap v338: 129 pgs: 129 active+clean; 584 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-17T22:35:57.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:57 smithi151 ceph-mon[36101]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:35:57.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:57 smithi151 ceph-mon[36101]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-05-17T22:35:58.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:57 smithi096 ceph-mon[36065]: pgmap v338: 129 pgs: 129 active+clean; 584 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-17T22:35:58.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:57 smithi096 ceph-mon[36065]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:35:58.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:57 smithi096 ceph-mon[36065]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-05-17T22:35:59.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.377+0000 7f61a23fb700 1 -- 172.21.15.12:0/1588212822 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f619c1080a0 msgr2=0x7f619c10fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:59.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.377+0000 7f61a23fb700 1 --2- 172.21.15.12:0/1588212822 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f619c1080a0 0x7f619c10fa00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f619400b0a0 tx=0x7f619400b3b0 comp rx=0 tx=0).stop 2024-05-17T22:35:59.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.377+0000 7f61a23fb700 1 -- 172.21.15.12:0/1588212822 shutdown_connections 2024-05-17T22:35:59.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.377+0000 7f61a23fb700 1 --2- 172.21.15.12:0/1588212822 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f619c1080a0 0x7f619c10fa00 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:59.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.377+0000 7f61a23fb700 1 --2- 172.21.15.12:0/1588212822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f619c102e70 0x7f619c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:59.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.377+0000 7f61a23fb700 1 --2- 172.21.15.12:0/1588212822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f619c107750 0x7f619c107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:59.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.377+0000 7f61a23fb700 1 -- 172.21.15.12:0/1588212822 >> 172.21.15.12:0/1588212822 conn(0x7f619c079ae0 msgr2=0x7f619c079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:35:59.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.377+0000 7f61a23fb700 1 -- 172.21.15.12:0/1588212822 shutdown_connections 2024-05-17T22:35:59.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.377+0000 7f61a23fb700 1 -- 172.21.15.12:0/1588212822 wait complete. 2024-05-17T22:35:59.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.378+0000 7f61a23fb700 1 Processor -- start 2024-05-17T22:35:59.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.378+0000 7f61a23fb700 1 -- start start 2024-05-17T22:35:59.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.378+0000 7f61a23fb700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f619c102e70 0x7f619c199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:59.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.378+0000 7f61a23fb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f619c107750 0x7f619c1998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:59.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.378+0000 7f61a23fb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f619c1080a0 0x7f619c199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:59.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.378+0000 7f61a23fb700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f619c112ce0 con 0x7f619c1080a0 2024-05-17T22:35:59.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.378+0000 7f61a23fb700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f619c112b60 con 0x7f619c107750 2024-05-17T22:35:59.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.378+0000 7f61a23fb700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f619c112e60 con 0x7f619c102e70 2024-05-17T22:35:59.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.379+0000 7f61a0998700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f619c1080a0 0x7f619c199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:59.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.379+0000 7f61a0998700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f619c1080a0 0x7f619c199e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37814/0 (socket says 172.21.15.12:37814) 2024-05-17T22:35:59.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.379+0000 7f619bfff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f619c102e70 0x7f619c199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:59.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.379+0000 7f61a0998700 1 -- 172.21.15.12:0/2847536771 learned_addr learned my addr 172.21.15.12:0/2847536771 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:35:59.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.379+0000 7f619b7fe700 1 --2- 172.21.15.12:0/2847536771 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f619c107750 0x7f619c1998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:59.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.379+0000 7f619bfff700 1 -- 172.21.15.12:0/2847536771 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f619c107750 msgr2=0x7f619c1998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:59.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.379+0000 7f619bfff700 1 --2- 172.21.15.12:0/2847536771 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f619c107750 0x7f619c1998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:59.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.379+0000 7f619bfff700 1 -- 172.21.15.12:0/2847536771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f619c1080a0 msgr2=0x7f619c199e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:59.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.379+0000 7f619bfff700 1 --2- 172.21.15.12:0/2847536771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f619c1080a0 0x7f619c199e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:59.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.379+0000 7f619bfff700 1 -- 172.21.15.12:0/2847536771 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f619c1a8a80 con 0x7f619c102e70 2024-05-17T22:35:59.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.379+0000 7f61a0998700 1 --2- 172.21.15.12:0/2847536771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f619c1080a0 0x7f619c199e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:35:59.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.379+0000 7f619bfff700 1 --2- 172.21.15.12:0/2847536771 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f619c102e70 0x7f619c199380 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f618c00e9f0 tx=0x7f618c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:35:59.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.380+0000 7f61997fa700 1 -- 172.21.15.12:0/2847536771 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618c00c9c0 con 0x7f619c102e70 2024-05-17T22:35:59.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.380+0000 7f61997fa700 1 -- 172.21.15.12:0/2847536771 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f618c00cb60 con 0x7f619c102e70 2024-05-17T22:35:59.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.380+0000 7f61a23fb700 1 -- 172.21.15.12:0/2847536771 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f619c1a8d70 con 0x7f619c102e70 2024-05-17T22:35:59.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.380+0000 7f61997fa700 1 -- 172.21.15.12:0/2847536771 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618c018680 con 0x7f619c102e70 2024-05-17T22:35:59.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.380+0000 7f61a23fb700 1 -- 172.21.15.12:0/2847536771 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f619c1a9270 con 0x7f619c102e70 2024-05-17T22:35:59.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.381+0000 7f61a23fb700 1 -- 172.21.15.12:0/2847536771 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f619c1042d0 con 0x7f619c102e70 2024-05-17T22:35:59.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.382+0000 7f61997fa700 1 -- 172.21.15.12:0/2847536771 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f618c010430 con 0x7f619c102e70 2024-05-17T22:35:59.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.383+0000 7f61997fa700 1 --2- 172.21.15.12:0/2847536771 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f61840b2990 0x7f61840b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:35:59.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.383+0000 7f619b7fe700 1 --2- 172.21.15.12:0/2847536771 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f61840b2990 0x7f61840b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:35:59.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.383+0000 7f61997fa700 1 -- 172.21.15.12:0/2847536771 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f618c0e2b30 con 0x7f619c102e70 2024-05-17T22:35:59.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.384+0000 7f619b7fe700 1 --2- 172.21.15.12:0/2847536771 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f61840b2990 0x7f61840b4ea0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f619c104840 tx=0x7f6190009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:35:59.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.386+0000 7f61997fa700 1 -- 172.21.15.12:0/2847536771 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f618c0aa390 con 0x7f619c102e70 2024-05-17T22:35:59.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.641+0000 7f61a23fb700 1 -- 172.21.15.12:0/2847536771 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f619c06c630 con 0x7f61840b2990 2024-05-17T22:35:59.645 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:35:59 smithi012 ceph-mon[31632]: pgmap v339: 129 pgs: 129 active+clean; 585 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-05-17T22:35:59.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.669+0000 7f61997fa700 1 -- 172.21.15.12:0/2847536771 <== mgr.14152 v2:172.21.15.12:6800/1388514047 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f619c06c630 con 0x7f61840b2990 2024-05-17T22:35:59.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.671+0000 7f61a23fb700 1 -- 172.21.15.12:0/2847536771 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f61840b2990 msgr2=0x7f61840b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:59.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.671+0000 7f61a23fb700 1 --2- 172.21.15.12:0/2847536771 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f61840b2990 0x7f61840b4ea0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f619c104840 tx=0x7f6190009260 comp rx=0 tx=0).stop 2024-05-17T22:35:59.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.671+0000 7f61a23fb700 1 -- 172.21.15.12:0/2847536771 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f619c102e70 msgr2=0x7f619c199380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:35:59.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.671+0000 7f61a23fb700 1 --2- 172.21.15.12:0/2847536771 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f619c102e70 0x7f619c199380 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f618c00e9f0 tx=0x7f618c00ed00 comp rx=0 tx=0).stop 2024-05-17T22:35:59.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.671+0000 7f61a23fb700 1 -- 172.21.15.12:0/2847536771 shutdown_connections 2024-05-17T22:35:59.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.671+0000 7f61a23fb700 1 --2- 172.21.15.12:0/2847536771 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f61840b2990 0x7f61840b4ea0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:59.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.671+0000 7f61a23fb700 1 --2- 172.21.15.12:0/2847536771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f619c1080a0 0x7f619c199e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:59.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.671+0000 7f61a23fb700 1 --2- 172.21.15.12:0/2847536771 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f619c102e70 0x7f619c199380 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:59.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.671+0000 7f61a23fb700 1 --2- 172.21.15.12:0/2847536771 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f619c107750 0x7f619c1998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:35:59.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.671+0000 7f61a23fb700 1 -- 172.21.15.12:0/2847536771 >> 172.21.15.12:0/2847536771 conn(0x7f619c079ae0 msgr2=0x7f619c1052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:35:59.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.671+0000 7f61a23fb700 1 -- 172.21.15.12:0/2847536771 shutdown_connections 2024-05-17T22:35:59.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:35:59.672+0000 7f61a23fb700 1 -- 172.21.15.12:0/2847536771 wait complete. 2024-05-17T22:35:59.851 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:35:59 smithi096 ceph-mon[36065]: pgmap v339: 129 pgs: 129 active+clean; 585 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-05-17T22:35:59.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:35:59 smithi151 ceph-mon[36101]: pgmap v339: 129 pgs: 129 active+clean; 585 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-05-17T22:36:00.373 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-17T22:36:00.633 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:36:00.756 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:00 smithi012 ceph-mon[31632]: from='client.24604 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:36:00.871 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:00 smithi096 ceph-mon[36065]: from='client.24604 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:36:00.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:00 smithi151 ceph-mon[36101]: from='client.24604 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:36:01.804 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:01 smithi012 ceph-mon[31632]: pgmap v340: 129 pgs: 129 active+clean; 591 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-05-17T22:36:01.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:01 smithi151 ceph-mon[36101]: pgmap v340: 129 pgs: 129 active+clean; 591 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-05-17T22:36:02.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:01 smithi096 ceph-mon[36065]: pgmap v340: 129 pgs: 129 active+clean; 591 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-05-17T22:36:02.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.379+0000 7f6aef99b700 1 -- 172.21.15.12:0/3073268404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ae8109190 msgr2=0x7f6ae8110af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:02.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.379+0000 7f6aef99b700 1 --2- 172.21.15.12:0/3073268404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ae8109190 0x7f6ae8110af0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6ae000b0a0 tx=0x7f6ae000b3b0 comp rx=0 tx=0).stop 2024-05-17T22:36:02.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.379+0000 7f6aef99b700 1 -- 172.21.15.12:0/3073268404 shutdown_connections 2024-05-17T22:36:02.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.379+0000 7f6aef99b700 1 --2- 172.21.15.12:0/3073268404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ae8109190 0x7f6ae8110af0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:02.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.379+0000 7f6aef99b700 1 --2- 172.21.15.12:0/3073268404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ae8103f10 0x7f6ae8104320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:02.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.379+0000 7f6aef99b700 1 --2- 172.21.15.12:0/3073268404 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ae8108840 0x7f6ae8108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:02.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.379+0000 7f6aef99b700 1 -- 172.21.15.12:0/3073268404 >> 172.21.15.12:0/3073268404 conn(0x7f6ae80ff990 msgr2=0x7f6ae8101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:36:02.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.379+0000 7f6aef99b700 1 -- 172.21.15.12:0/3073268404 shutdown_connections 2024-05-17T22:36:02.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.379+0000 7f6aef99b700 1 -- 172.21.15.12:0/3073268404 wait complete. 2024-05-17T22:36:02.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.380+0000 7f6aef99b700 1 Processor -- start 2024-05-17T22:36:02.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.380+0000 7f6aef99b700 1 -- start start 2024-05-17T22:36:02.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.380+0000 7f6aef99b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ae8103f10 0x7f6ae8075b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:02.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.380+0000 7f6aef99b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ae8108840 0x7f6ae8076070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:02.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.380+0000 7f6aef99b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ae8109190 0x7f6ae80765b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:02.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.380+0000 7f6aef99b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae8113e40 con 0x7f6ae8103f10 2024-05-17T22:36:02.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.380+0000 7f6aef99b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae8113cc0 con 0x7f6ae8108840 2024-05-17T22:36:02.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.380+0000 7f6aef99b700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae8113fc0 con 0x7f6ae8109190 2024-05-17T22:36:02.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.380+0000 7f6aed737700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ae8103f10 0x7f6ae8075b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:02.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.380+0000 7f6aed737700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ae8103f10 0x7f6ae8075b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40868/0 (socket says 172.21.15.12:40868) 2024-05-17T22:36:02.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.380+0000 7f6aed737700 1 -- 172.21.15.12:0/1184107390 learned_addr learned my addr 172.21.15.12:0/1184107390 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:36:02.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.380+0000 7f6aecf36700 1 --2- 172.21.15.12:0/1184107390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ae8108840 0x7f6ae8076070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:02.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.380+0000 7f6aedf38700 1 --2- 172.21.15.12:0/1184107390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ae8109190 0x7f6ae80765b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:02.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.381+0000 7f6aed737700 1 -- 172.21.15.12:0/1184107390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ae8109190 msgr2=0x7f6ae80765b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:02.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.381+0000 7f6aed737700 1 --2- 172.21.15.12:0/1184107390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ae8109190 0x7f6ae80765b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:02.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.381+0000 7f6aed737700 1 -- 172.21.15.12:0/1184107390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ae8108840 msgr2=0x7f6ae8076070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:02.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.381+0000 7f6aed737700 1 --2- 172.21.15.12:0/1184107390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ae8108840 0x7f6ae8076070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:02.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.381+0000 7f6aed737700 1 -- 172.21.15.12:0/1184107390 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ae81a8b40 con 0x7f6ae8103f10 2024-05-17T22:36:02.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.381+0000 7f6aed737700 1 --2- 172.21.15.12:0/1184107390 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ae8103f10 0x7f6ae8075b30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6ae400d950 tx=0x7f6ae400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:36:02.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.381+0000 7f6ade7fc700 1 -- 172.21.15.12:0/1184107390 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4016070 con 0x7f6ae8103f10 2024-05-17T22:36:02.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.381+0000 7f6ade7fc700 1 -- 172.21.15.12:0/1184107390 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ae400fd40 con 0x7f6ae8103f10 2024-05-17T22:36:02.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.381+0000 7f6aef99b700 1 -- 172.21.15.12:0/1184107390 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ae81a8e30 con 0x7f6ae8103f10 2024-05-17T22:36:02.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.381+0000 7f6ade7fc700 1 -- 172.21.15.12:0/1184107390 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae400e3b0 con 0x7f6ae8103f10 2024-05-17T22:36:02.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.381+0000 7f6aef99b700 1 -- 172.21.15.12:0/1184107390 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ae81a92e0 con 0x7f6ae8103f10 2024-05-17T22:36:02.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.382+0000 7f6aef99b700 1 -- 172.21.15.12:0/1184107390 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ae8104320 con 0x7f6ae8103f10 2024-05-17T22:36:02.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.384+0000 7f6ade7fc700 1 -- 172.21.15.12:0/1184107390 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f6ae4019070 con 0x7f6ae8103f10 2024-05-17T22:36:02.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.385+0000 7f6ade7fc700 1 --2- 172.21.15.12:0/1184107390 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f6ad40b2990 0x7f6ad40b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:02.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.385+0000 7f6aecf36700 1 --2- 172.21.15.12:0/1184107390 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f6ad40b2990 0x7f6ad40b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:02.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.385+0000 7f6ade7fc700 1 -- 172.21.15.12:0/1184107390 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f6ae40e21b0 con 0x7f6ae8103f10 2024-05-17T22:36:02.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.385+0000 7f6aecf36700 1 --2- 172.21.15.12:0/1184107390 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f6ad40b2990 0x7f6ad40b4ea0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8107aa0 tx=0x7f6ad8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:36:02.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.388+0000 7f6ade7fc700 1 -- 172.21.15.12:0/1184107390 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6ae40a99b0 con 0x7f6ae8103f10 2024-05-17T22:36:02.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.641+0000 7f6aef99b700 1 -- 172.21.15.12:0/1184107390 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f6ae8077060 con 0x7f6ad40b2990 2024-05-17T22:36:02.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.648+0000 7f6ade7fc700 1 -- 172.21.15.12:0/1184107390 <== mgr.14152 v2:172.21.15.12:6800/1388514047 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f6ae8077060 con 0x7f6ad40b2990 2024-05-17T22:36:02.651 INFO:teuthology.orchestra.run.smithi012.stdout:/volumes/qa/sv_0/d3056385-f987-4861-8b9d-26ebb4500975 2024-05-17T22:36:02.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.651+0000 7f6aef99b700 1 -- 172.21.15.12:0/1184107390 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f6ad40b2990 msgr2=0x7f6ad40b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:02.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.651+0000 7f6aef99b700 1 --2- 172.21.15.12:0/1184107390 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f6ad40b2990 0x7f6ad40b4ea0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8107aa0 tx=0x7f6ad8009190 comp rx=0 tx=0).stop 2024-05-17T22:36:02.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.651+0000 7f6aef99b700 1 -- 172.21.15.12:0/1184107390 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ae8103f10 msgr2=0x7f6ae8075b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:02.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.651+0000 7f6aef99b700 1 --2- 172.21.15.12:0/1184107390 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ae8103f10 0x7f6ae8075b30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6ae400d950 tx=0x7f6ae400dc60 comp rx=0 tx=0).stop 2024-05-17T22:36:02.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.651+0000 7f6aef99b700 1 -- 172.21.15.12:0/1184107390 shutdown_connections 2024-05-17T22:36:02.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.651+0000 7f6aef99b700 1 --2- 172.21.15.12:0/1184107390 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f6ad40b2990 0x7f6ad40b4ea0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:02.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.651+0000 7f6aef99b700 1 --2- 172.21.15.12:0/1184107390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ae8109190 0x7f6ae80765b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:02.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.651+0000 7f6aef99b700 1 --2- 172.21.15.12:0/1184107390 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ae8103f10 0x7f6ae8075b30 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:02.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.651+0000 7f6aef99b700 1 --2- 172.21.15.12:0/1184107390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ae8108840 0x7f6ae8076070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:02.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.651+0000 7f6aef99b700 1 -- 172.21.15.12:0/1184107390 >> 172.21.15.12:0/1184107390 conn(0x7f6ae80ff990 msgr2=0x7f6ae810a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:36:02.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.651+0000 7f6aef99b700 1 -- 172.21.15.12:0/1184107390 shutdown_connections 2024-05-17T22:36:02.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:02.651+0000 7f6aef99b700 1 -- 172.21.15.12:0/1184107390 wait complete. 2024-05-17T22:36:03.380 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-17T22:36:03.639 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:36:03.762 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:03 smithi012 ceph-mon[31632]: pgmap v341: 129 pgs: 129 active+clean; 591 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-17T22:36:03.762 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:03 smithi012 ceph-mon[31632]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:36:03.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:03 smithi151 ceph-mon[36101]: pgmap v341: 129 pgs: 129 active+clean; 591 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-17T22:36:03.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:03 smithi151 ceph-mon[36101]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:36:04.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:03 smithi096 ceph-mon[36065]: pgmap v341: 129 pgs: 129 active+clean; 591 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-17T22:36:04.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:03 smithi096 ceph-mon[36065]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:36:05.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.313+0000 7fee30ed1700 1 -- 172.21.15.12:0/4259345378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee2c102e70 msgr2=0x7fee2c103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:05.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.313+0000 7fee30ed1700 1 --2- 172.21.15.12:0/4259345378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee2c102e70 0x7fee2c103280 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fee1c00b0a0 tx=0x7fee1c00b3b0 comp rx=0 tx=0).stop 2024-05-17T22:36:05.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.314+0000 7fee30ed1700 1 -- 172.21.15.12:0/4259345378 shutdown_connections 2024-05-17T22:36:05.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.314+0000 7fee30ed1700 1 --2- 172.21.15.12:0/4259345378 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee2c1080a0 0x7fee2c10fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:05.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.314+0000 7fee30ed1700 1 --2- 172.21.15.12:0/4259345378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee2c102e70 0x7fee2c103280 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:05.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.314+0000 7fee30ed1700 1 --2- 172.21.15.12:0/4259345378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee2c107750 0x7fee2c107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:05.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.314+0000 7fee30ed1700 1 -- 172.21.15.12:0/4259345378 >> 172.21.15.12:0/4259345378 conn(0x7fee2c079ae0 msgr2=0x7fee2c079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:36:05.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.314+0000 7fee30ed1700 1 -- 172.21.15.12:0/4259345378 shutdown_connections 2024-05-17T22:36:05.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.314+0000 7fee30ed1700 1 -- 172.21.15.12:0/4259345378 wait complete. 2024-05-17T22:36:05.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.314+0000 7fee30ed1700 1 Processor -- start 2024-05-17T22:36:05.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.314+0000 7fee30ed1700 1 -- start start 2024-05-17T22:36:05.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.315+0000 7fee30ed1700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee2c102e70 0x7fee2c105240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:05.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.315+0000 7fee30ed1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee2c107750 0x7fee2c105780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:05.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.315+0000 7fee30ed1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee2c1080a0 0x7fee2c105cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:05.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.315+0000 7fee30ed1700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fee2c112cd0 con 0x7fee2c1080a0 2024-05-17T22:36:05.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.315+0000 7fee30ed1700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fee2c112b50 con 0x7fee2c107750 2024-05-17T22:36:05.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.315+0000 7fee30ed1700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fee2c112e50 con 0x7fee2c102e70 2024-05-17T22:36:05.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.315+0000 7fee2ad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee2c1080a0 0x7fee2c105cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:05.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.315+0000 7fee2ad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee2c1080a0 0x7fee2c105cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40882/0 (socket says 172.21.15.12:40882) 2024-05-17T22:36:05.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.315+0000 7fee2ad9d700 1 -- 172.21.15.12:0/3422939557 learned_addr learned my addr 172.21.15.12:0/3422939557 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:36:05.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.315+0000 7fee2a59c700 1 --2- 172.21.15.12:0/3422939557 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee2c102e70 0x7fee2c105240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:05.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.315+0000 7fee29d9b700 1 --2- 172.21.15.12:0/3422939557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee2c107750 0x7fee2c105780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:05.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.315+0000 7fee2ad9d700 1 -- 172.21.15.12:0/3422939557 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee2c102e70 msgr2=0x7fee2c105240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:05.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.315+0000 7fee2ad9d700 1 --2- 172.21.15.12:0/3422939557 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee2c102e70 0x7fee2c105240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:05.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.315+0000 7fee2ad9d700 1 -- 172.21.15.12:0/3422939557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee2c107750 msgr2=0x7fee2c105780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:05.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.315+0000 7fee2ad9d700 1 --2- 172.21.15.12:0/3422939557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee2c107750 0x7fee2c105780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:05.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.315+0000 7fee2ad9d700 1 -- 172.21.15.12:0/3422939557 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee2c1a8b20 con 0x7fee2c1080a0 2024-05-17T22:36:05.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.316+0000 7fee2ad9d700 1 --2- 172.21.15.12:0/3422939557 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee2c1080a0 0x7fee2c105cc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fee2400d980 tx=0x7fee2400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:36:05.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.316+0000 7fee237fe700 1 -- 172.21.15.12:0/3422939557 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee24017070 con 0x7fee2c1080a0 2024-05-17T22:36:05.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.316+0000 7fee237fe700 1 -- 172.21.15.12:0/3422939557 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee24005020 con 0x7fee2c1080a0 2024-05-17T22:36:05.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.316+0000 7fee30ed1700 1 -- 172.21.15.12:0/3422939557 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee2c1a8e10 con 0x7fee2c1080a0 2024-05-17T22:36:05.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.316+0000 7fee237fe700 1 -- 172.21.15.12:0/3422939557 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2400fa10 con 0x7fee2c1080a0 2024-05-17T22:36:05.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.316+0000 7fee30ed1700 1 -- 172.21.15.12:0/3422939557 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee2c1a9240 con 0x7fee2c1080a0 2024-05-17T22:36:05.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.317+0000 7fee30ed1700 1 -- 172.21.15.12:0/3422939557 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee2c103280 con 0x7fee2c1080a0 2024-05-17T22:36:05.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.318+0000 7fee237fe700 1 -- 172.21.15.12:0/3422939557 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fee2400fbb0 con 0x7fee2c1080a0 2024-05-17T22:36:05.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.319+0000 7fee237fe700 1 --2- 172.21.15.12:0/3422939557 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fee180b2910 0x7fee180b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:05.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.319+0000 7fee2a59c700 1 --2- 172.21.15.12:0/3422939557 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fee180b2910 0x7fee180b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:05.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.319+0000 7fee237fe700 1 -- 172.21.15.12:0/3422939557 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fee240e2570 con 0x7fee2c1080a0 2024-05-17T22:36:05.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.320+0000 7fee2a59c700 1 --2- 172.21.15.12:0/3422939557 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fee180b2910 0x7fee180b4e20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fee1c009900 tx=0x7fee1c00b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:36:05.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.324+0000 7fee237fe700 1 -- 172.21.15.12:0/3422939557 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fee24020020 con 0x7fee2c1080a0 2024-05-17T22:36:05.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.574+0000 7fee30ed1700 1 -- 172.21.15.12:0/3422939557 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fee2c106770 con 0x7fee180b2910 2024-05-17T22:36:05.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.602+0000 7fee237fe700 1 -- 172.21.15.12:0/3422939557 <== mgr.14152 v2:172.21.15.12:6800/1388514047 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fee2c106770 con 0x7fee180b2910 2024-05-17T22:36:05.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.605+0000 7fee30ed1700 1 -- 172.21.15.12:0/3422939557 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fee180b2910 msgr2=0x7fee180b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:05.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.605+0000 7fee30ed1700 1 --2- 172.21.15.12:0/3422939557 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fee180b2910 0x7fee180b4e20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fee1c009900 tx=0x7fee1c00b7b0 comp rx=0 tx=0).stop 2024-05-17T22:36:05.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.605+0000 7fee30ed1700 1 -- 172.21.15.12:0/3422939557 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee2c1080a0 msgr2=0x7fee2c105cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:05.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.605+0000 7fee30ed1700 1 --2- 172.21.15.12:0/3422939557 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee2c1080a0 0x7fee2c105cc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fee2400d980 tx=0x7fee2400dc90 comp rx=0 tx=0).stop 2024-05-17T22:36:05.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.605+0000 7fee30ed1700 1 -- 172.21.15.12:0/3422939557 shutdown_connections 2024-05-17T22:36:05.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.605+0000 7fee30ed1700 1 --2- 172.21.15.12:0/3422939557 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fee180b2910 0x7fee180b4e20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:05.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.605+0000 7fee30ed1700 1 --2- 172.21.15.12:0/3422939557 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee2c1080a0 0x7fee2c105cc0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:05.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.605+0000 7fee30ed1700 1 --2- 172.21.15.12:0/3422939557 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee2c102e70 0x7fee2c105240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:05.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.605+0000 7fee30ed1700 1 --2- 172.21.15.12:0/3422939557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee2c107750 0x7fee2c105780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:05.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.605+0000 7fee30ed1700 1 -- 172.21.15.12:0/3422939557 >> 172.21.15.12:0/3422939557 conn(0x7fee2c079ae0 msgr2=0x7fee2c06c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:36:05.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.605+0000 7fee30ed1700 1 -- 172.21.15.12:0/3422939557 shutdown_connections 2024-05-17T22:36:05.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:05.605+0000 7fee30ed1700 1 -- 172.21.15.12:0/3422939557 wait complete. 2024-05-17T22:36:05.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:05 smithi012 ceph-mon[31632]: pgmap v342: 129 pgs: 129 active+clean; 591 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-05-17T22:36:05.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:05 smithi151 ceph-mon[36101]: pgmap v342: 129 pgs: 129 active+clean; 591 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-05-17T22:36:06.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:05 smithi096 ceph-mon[36065]: pgmap v342: 129 pgs: 129 active+clean; 591 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-05-17T22:36:06.359 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-17T22:36:06.615 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:36:06.900 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:06 smithi012 ceph-mon[31632]: from='client.14841 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:36:06.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:06 smithi151 ceph-mon[36101]: from='client.14841 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:36:07.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:06 smithi096 ceph-mon[36065]: from='client.14841 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:36:07.725 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:07 smithi012 ceph-mon[31632]: pgmap v343: 129 pgs: 129 active+clean; 611 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-05-17T22:36:07.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:07 smithi151 ceph-mon[36101]: pgmap v343: 129 pgs: 129 active+clean; 611 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-05-17T22:36:08.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:07 smithi096 ceph-mon[36065]: pgmap v343: 129 pgs: 129 active+clean; 611 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-05-17T22:36:08.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.299+0000 7ff1103fb700 1 -- 172.21.15.12:0/2287092751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff108107750 msgr2=0x7ff108107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:08.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.299+0000 7ff1103fb700 1 --2- 172.21.15.12:0/2287092751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff108107750 0x7ff108107b60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8009990 tx=0x7ff0f8009ca0 comp rx=0 tx=0).stop 2024-05-17T22:36:08.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.299+0000 7ff1103fb700 1 -- 172.21.15.12:0/2287092751 shutdown_connections 2024-05-17T22:36:08.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.299+0000 7ff1103fb700 1 --2- 172.21.15.12:0/2287092751 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff1081080a0 0x7ff10810fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:08.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.299+0000 7ff1103fb700 1 --2- 172.21.15.12:0/2287092751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff108102e20 0x7ff108103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:08.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.299+0000 7ff1103fb700 1 --2- 172.21.15.12:0/2287092751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff108107750 0x7ff108107b60 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:08.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.299+0000 7ff1103fb700 1 -- 172.21.15.12:0/2287092751 >> 172.21.15.12:0/2287092751 conn(0x7ff108079af0 msgr2=0x7ff108079f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:36:08.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.299+0000 7ff1103fb700 1 -- 172.21.15.12:0/2287092751 shutdown_connections 2024-05-17T22:36:08.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.299+0000 7ff1103fb700 1 -- 172.21.15.12:0/2287092751 wait complete. 2024-05-17T22:36:08.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.300+0000 7ff1103fb700 1 Processor -- start 2024-05-17T22:36:08.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.300+0000 7ff1103fb700 1 -- start start 2024-05-17T22:36:08.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.300+0000 7ff1103fb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff108102e20 0x7ff108109aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:08.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.300+0000 7ff1103fb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff108107750 0x7ff108109fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:08.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.300+0000 7ff1103fb700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff1081080a0 0x7ff10810a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:08.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.300+0000 7ff1103fb700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff108112d40 con 0x7ff108107750 2024-05-17T22:36:08.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.300+0000 7ff1103fb700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff108112bc0 con 0x7ff108102e20 2024-05-17T22:36:08.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.300+0000 7ff1103fb700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff108112ec0 con 0x7ff1081080a0 2024-05-17T22:36:08.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.301+0000 7ff10d996700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff108107750 0x7ff108109fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:08.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.301+0000 7ff10d996700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff108107750 0x7ff108109fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40902/0 (socket says 172.21.15.12:40902) 2024-05-17T22:36:08.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.301+0000 7ff10d996700 1 -- 172.21.15.12:0/3741912810 learned_addr learned my addr 172.21.15.12:0/3741912810 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:36:08.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.301+0000 7ff10e998700 1 --2- 172.21.15.12:0/3741912810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff1081080a0 0x7ff10810a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:08.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.301+0000 7ff10e197700 1 --2- 172.21.15.12:0/3741912810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff108102e20 0x7ff108109aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:08.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.301+0000 7ff10d996700 1 -- 172.21.15.12:0/3741912810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff1081080a0 msgr2=0x7ff10810a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:08.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.301+0000 7ff10d996700 1 --2- 172.21.15.12:0/3741912810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff1081080a0 0x7ff10810a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:08.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.301+0000 7ff10d996700 1 -- 172.21.15.12:0/3741912810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff108102e20 msgr2=0x7ff108109aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:08.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.301+0000 7ff10d996700 1 --2- 172.21.15.12:0/3741912810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff108102e20 0x7ff108109aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:08.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.301+0000 7ff10d996700 1 -- 172.21.15.12:0/3741912810 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1081a8bb0 con 0x7ff108107750 2024-05-17T22:36:08.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.301+0000 7ff10d996700 1 --2- 172.21.15.12:0/3741912810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff108107750 0x7ff108109fe0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8000c00 tx=0x7ff0f80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:36:08.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.301+0000 7ff0ff7fe700 1 -- 172.21.15.12:0/3741912810 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8027070 con 0x7ff108107750 2024-05-17T22:36:08.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.301+0000 7ff1103fb700 1 -- 172.21.15.12:0/3741912810 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1081a8e40 con 0x7ff108107750 2024-05-17T22:36:08.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.301+0000 7ff0ff7fe700 1 -- 172.21.15.12:0/3741912810 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0f8003f20 con 0x7ff108107750 2024-05-17T22:36:08.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.302+0000 7ff0ff7fe700 1 -- 172.21.15.12:0/3741912810 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8022c70 con 0x7ff108107750 2024-05-17T22:36:08.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.302+0000 7ff1103fb700 1 -- 172.21.15.12:0/3741912810 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1081a9240 con 0x7ff108107750 2024-05-17T22:36:08.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.303+0000 7ff1103fb700 1 -- 172.21.15.12:0/3741912810 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff108103230 con 0x7ff108107750 2024-05-17T22:36:08.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.304+0000 7ff0ff7fe700 1 -- 172.21.15.12:0/3741912810 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ff0f802a070 con 0x7ff108107750 2024-05-17T22:36:08.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.306+0000 7ff0ff7fe700 1 --2- 172.21.15.12:0/3741912810 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff0f40b2910 0x7ff0f40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:08.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.306+0000 7ff10e197700 1 --2- 172.21.15.12:0/3741912810 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff0f40b2910 0x7ff0f40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:08.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.306+0000 7ff0ff7fe700 1 -- 172.21.15.12:0/3741912810 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7ff0f80e3ba0 con 0x7ff108107750 2024-05-17T22:36:08.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.306+0000 7ff10e197700 1 --2- 172.21.15.12:0/3741912810 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff0f40b2910 0x7ff0f40b4e20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff104009a10 tx=0x7ff104009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:36:08.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.309+0000 7ff0ff7fe700 1 -- 172.21.15.12:0/3741912810 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff0f8031020 con 0x7ff108107750 2024-05-17T22:36:08.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.561+0000 7ff1103fb700 1 -- 172.21.15.12:0/3741912810 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7ff108101970 con 0x7ff0f40b2910 2024-05-17T22:36:08.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.567+0000 7ff0ff7fe700 1 -- 172.21.15.12:0/3741912810 <== mgr.14152 v2:172.21.15.12:6800/1388514047 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7ff108101970 con 0x7ff0f40b2910 2024-05-17T22:36:08.570 INFO:teuthology.orchestra.run.smithi012.stdout:/volumes/qa/sv_1/682b7b25-6ede-403c-acf6-bc1f5a3307a1 2024-05-17T22:36:08.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.569+0000 7ff1103fb700 1 -- 172.21.15.12:0/3741912810 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff0f40b2910 msgr2=0x7ff0f40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:08.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.569+0000 7ff1103fb700 1 --2- 172.21.15.12:0/3741912810 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff0f40b2910 0x7ff0f40b4e20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff104009a10 tx=0x7ff104009190 comp rx=0 tx=0).stop 2024-05-17T22:36:08.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.569+0000 7ff1103fb700 1 -- 172.21.15.12:0/3741912810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff108107750 msgr2=0x7ff108109fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:08.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.569+0000 7ff1103fb700 1 --2- 172.21.15.12:0/3741912810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff108107750 0x7ff108109fe0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8000c00 tx=0x7ff0f80047b0 comp rx=0 tx=0).stop 2024-05-17T22:36:08.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.569+0000 7ff1103fb700 1 -- 172.21.15.12:0/3741912810 shutdown_connections 2024-05-17T22:36:08.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.569+0000 7ff1103fb700 1 --2- 172.21.15.12:0/3741912810 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff0f40b2910 0x7ff0f40b4e20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:08.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.569+0000 7ff1103fb700 1 --2- 172.21.15.12:0/3741912810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff1081080a0 0x7ff10810a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:08.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.569+0000 7ff1103fb700 1 --2- 172.21.15.12:0/3741912810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff108102e20 0x7ff108109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:08.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.569+0000 7ff1103fb700 1 --2- 172.21.15.12:0/3741912810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff108107750 0x7ff108109fe0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:08.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.569+0000 7ff1103fb700 1 -- 172.21.15.12:0/3741912810 >> 172.21.15.12:0/3741912810 conn(0x7ff108079af0 msgr2=0x7ff108104860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:36:08.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.569+0000 7ff1103fb700 1 -- 172.21.15.12:0/3741912810 shutdown_connections 2024-05-17T22:36:08.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:08.569+0000 7ff1103fb700 1 -- 172.21.15.12:0/3741912810 wait complete. 2024-05-17T22:36:09.308 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T22:36:09.563 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:36:09.862 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:09 smithi012 ceph-mon[31632]: pgmap v344: 129 pgs: 129 active+clean; 611 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-05-17T22:36:09.862 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:09 smithi012 ceph-mon[31632]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:36:09.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:09 smithi151 ceph-mon[36101]: pgmap v344: 129 pgs: 129 active+clean; 611 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-05-17T22:36:09.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:09 smithi151 ceph-mon[36101]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:36:10.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:09 smithi096 ceph-mon[36065]: pgmap v344: 129 pgs: 129 active+clean; 611 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-05-17T22:36:10.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:09 smithi096 ceph-mon[36065]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:36:11.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.344+0000 7f8731ac1700 1 -- 172.21.15.12:0/4046728232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f872c104e00 msgr2=0x7f872c105210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:11.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.344+0000 7f8731ac1700 1 --2- 172.21.15.12:0/4046728232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f872c104e00 0x7f872c105210 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f871c00b0a0 tx=0x7f871c00b3b0 comp rx=0 tx=0).stop 2024-05-17T22:36:11.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.344+0000 7f8731ac1700 1 -- 172.21.15.12:0/4046728232 shutdown_connections 2024-05-17T22:36:11.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.344+0000 7f8731ac1700 1 --2- 172.21.15.12:0/4046728232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f872c10e230 0x7f872c110740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:11.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.344+0000 7f8731ac1700 1 --2- 172.21.15.12:0/4046728232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f872c104e00 0x7f872c105210 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:11.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.344+0000 7f8731ac1700 1 --2- 172.21.15.12:0/4046728232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f872c105750 0x7f872c10dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:11.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.344+0000 7f8731ac1700 1 -- 172.21.15.12:0/4046728232 >> 172.21.15.12:0/4046728232 conn(0x7f872c079bd0 msgr2=0x7f872c079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:36:11.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.344+0000 7f8731ac1700 1 -- 172.21.15.12:0/4046728232 shutdown_connections 2024-05-17T22:36:11.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.344+0000 7f8731ac1700 1 -- 172.21.15.12:0/4046728232 wait complete. 2024-05-17T22:36:11.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.345+0000 7f8731ac1700 1 Processor -- start 2024-05-17T22:36:11.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.345+0000 7f8731ac1700 1 -- start start 2024-05-17T22:36:11.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.346+0000 7f8731ac1700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f872c104e00 0x7f872c19a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:11.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.346+0000 7f8731ac1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f872c105750 0x7f872c19ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:11.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.346+0000 7f8731ac1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f872c10e230 0x7f872c19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:11.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.346+0000 7f8731ac1700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f872c113db0 con 0x7f872c10e230 2024-05-17T22:36:11.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.346+0000 7f8731ac1700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f872c113c30 con 0x7f872c105750 2024-05-17T22:36:11.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.346+0000 7f8731ac1700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f872c113f30 con 0x7f872c104e00 2024-05-17T22:36:11.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.346+0000 7f872bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f872c10e230 0x7f872c19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:11.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.346+0000 7f872bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f872c10e230 0x7f872c19efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44882/0 (socket says 172.21.15.12:44882) 2024-05-17T22:36:11.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.346+0000 7f872bfff700 1 -- 172.21.15.12:0/2811659991 learned_addr learned my addr 172.21.15.12:0/2811659991 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:36:11.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.346+0000 7f872b7fe700 1 --2- 172.21.15.12:0/2811659991 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f872c104e00 0x7f872c19a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:11.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.346+0000 7f872affd700 1 --2- 172.21.15.12:0/2811659991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f872c105750 0x7f872c19ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:11.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.346+0000 7f872bfff700 1 -- 172.21.15.12:0/2811659991 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f872c104e00 msgr2=0x7f872c19a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:11.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.346+0000 7f872bfff700 1 --2- 172.21.15.12:0/2811659991 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f872c104e00 0x7f872c19a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:11.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.346+0000 7f872bfff700 1 -- 172.21.15.12:0/2811659991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f872c105750 msgr2=0x7f872c19ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:11.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.346+0000 7f872bfff700 1 --2- 172.21.15.12:0/2811659991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f872c105750 0x7f872c19ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:11.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.346+0000 7f872bfff700 1 -- 172.21.15.12:0/2811659991 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f872c19f660 con 0x7f872c10e230 2024-05-17T22:36:11.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.346+0000 7f872b7fe700 1 --2- 172.21.15.12:0/2811659991 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f872c104e00 0x7f872c19a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:36:11.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.346+0000 7f872bfff700 1 --2- 172.21.15.12:0/2811659991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f872c10e230 0x7f872c19efc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f872400c970 tx=0x7f872400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:36:11.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.347+0000 7f8728ff9700 1 -- 172.21.15.12:0/2811659991 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8724007b00 con 0x7f872c10e230 2024-05-17T22:36:11.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.347+0000 7f8728ff9700 1 -- 172.21.15.12:0/2811659991 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8724004ce0 con 0x7f872c10e230 2024-05-17T22:36:11.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.347+0000 7f8728ff9700 1 -- 172.21.15.12:0/2811659991 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8724005530 con 0x7f872c10e230 2024-05-17T22:36:11.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.347+0000 7f8731ac1700 1 -- 172.21.15.12:0/2811659991 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f872c19f950 con 0x7f872c10e230 2024-05-17T22:36:11.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.347+0000 7f8731ac1700 1 -- 172.21.15.12:0/2811659991 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f872c1a7270 con 0x7f872c10e230 2024-05-17T22:36:11.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.348+0000 7f8731ac1700 1 -- 172.21.15.12:0/2811659991 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f872c105210 con 0x7f872c10e230 2024-05-17T22:36:11.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.349+0000 7f8728ff9700 1 -- 172.21.15.12:0/2811659991 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f8724004750 con 0x7f872c10e230 2024-05-17T22:36:11.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.350+0000 7f8728ff9700 1 --2- 172.21.15.12:0/2811659991 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f87140b2990 0x7f87140b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:11.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.350+0000 7f872b7fe700 1 --2- 172.21.15.12:0/2811659991 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f87140b2990 0x7f87140b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:11.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.350+0000 7f8728ff9700 1 -- 172.21.15.12:0/2811659991 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f87240e26c0 con 0x7f872c10e230 2024-05-17T22:36:11.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.351+0000 7f872b7fe700 1 --2- 172.21.15.12:0/2811659991 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f87140b2990 0x7f87140b4ea0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f871c0062a0 tx=0x7f871c006230 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:36:11.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.353+0000 7f8728ff9700 1 -- 172.21.15.12:0/2811659991 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8724014030 con 0x7f872c10e230 2024-05-17T22:36:11.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.615+0000 7f8731ac1700 1 -- 172.21.15.12:0/2811659991 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f872c19baf0 con 0x7f872c10e230 2024-05-17T22:36:11.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.615+0000 7f8728ff9700 1 -- 172.21.15.12:0/2811659991 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11870 (secure 0 0 0) 0x7f87240a9fa0 con 0x7f872c10e230 2024-05-17T22:36:11.620 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:36:11.620 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1407219319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1407219319},{"type":"v1","addr":"172.21.15.12:6835","nonce":1407219319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/630977886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":630977886},{"type":"v1","addr":"172.21.15.12:6837","nonce":630977886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3731914742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3731914742},{"type":"v1","addr":"172.21.15.12:6839","nonce":3731914742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/3642142501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3642142501},{"type":"v1","addr":"172.21.15.12:6841","nonce":3642142501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3554053059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3554053059},{"type":"v1","addr":"172.21.15.151:6833","nonce":3554053059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/4228325569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":4228325569},{"type":"v1","addr":"172.21.15.96:6833","nonce":4228325569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T22:34:42.642111+0000","modified":"2024-05-17T22:35:52.999468+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24461,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/542323754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":542323754},{"type":"v1","addr":"172.21.15.151:6835","nonce":542323754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1758442232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1758442232},{"type":"v1","addr":"172.21.15.96:6835","nonce":1758442232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3403324038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3403324038},{"type":"v1","addr":"172.21.15.151:6837","nonce":3403324038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.151:6839/2338353849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2338353849},{"type":"v1","addr":"172.21.15.151:6839","nonce":2338353849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.96:6837/4222574338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":4222574338},{"type":"v1","addr":"172.21.15.96:6837","nonce":4222574338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/2986295519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2986295519},{"type":"v1","addr":"172.21.15.96:6839","nonce":2986295519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24457,24482,24461]},"id":1}]} 2024-05-17T22:36:11.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.618+0000 7f8731ac1700 1 -- 172.21.15.12:0/2811659991 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f87140b2990 msgr2=0x7f87140b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:11.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.618+0000 7f8731ac1700 1 --2- 172.21.15.12:0/2811659991 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f87140b2990 0x7f87140b4ea0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f871c0062a0 tx=0x7f871c006230 comp rx=0 tx=0).stop 2024-05-17T22:36:11.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.618+0000 7f8731ac1700 1 -- 172.21.15.12:0/2811659991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f872c10e230 msgr2=0x7f872c19efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:11.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.618+0000 7f8731ac1700 1 --2- 172.21.15.12:0/2811659991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f872c10e230 0x7f872c19efc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f872400c970 tx=0x7f872400cd30 comp rx=0 tx=0).stop 2024-05-17T22:36:11.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.618+0000 7f8731ac1700 1 -- 172.21.15.12:0/2811659991 shutdown_connections 2024-05-17T22:36:11.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.618+0000 7f8731ac1700 1 --2- 172.21.15.12:0/2811659991 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f87140b2990 0x7f87140b4ea0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:11.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.618+0000 7f8731ac1700 1 --2- 172.21.15.12:0/2811659991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f872c10e230 0x7f872c19efc0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:11.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.618+0000 7f8731ac1700 1 --2- 172.21.15.12:0/2811659991 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f872c104e00 0x7f872c19a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:11.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.618+0000 7f8731ac1700 1 --2- 172.21.15.12:0/2811659991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f872c105750 0x7f872c19ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:11.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.618+0000 7f8731ac1700 1 -- 172.21.15.12:0/2811659991 >> 172.21.15.12:0/2811659991 conn(0x7f872c079bd0 msgr2=0x7f872c0ffb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:36:11.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.618+0000 7f8731ac1700 1 -- 172.21.15.12:0/2811659991 shutdown_connections 2024-05-17T22:36:11.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:11.618+0000 7f8731ac1700 1 -- 172.21.15.12:0/2811659991 wait complete. 2024-05-17T22:36:11.623 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 26 2024-05-17T22:36:11.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:11 smithi151 ceph-mon[36101]: pgmap v345: 129 pgs: 129 active+clean; 633 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.1 KiB/s wr, 9 op/s 2024-05-17T22:36:12.019 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:11 smithi012 ceph-mon[31632]: pgmap v345: 129 pgs: 129 active+clean; 633 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.1 KiB/s wr, 9 op/s 2024-05-17T22:36:12.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:11 smithi096 ceph-mon[36065]: pgmap v345: 129 pgs: 129 active+clean; 633 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.1 KiB/s wr, 9 op/s 2024-05-17T22:36:12.473 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-17T22:36:12.732 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:36:12.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:12 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/2811659991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:36:12.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:12 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/2811659991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:36:13.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:12 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/2811659991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:36:13.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:13 smithi151 ceph-mon[36101]: pgmap v346: 129 pgs: 129 active+clean; 633 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-05-17T22:36:14.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:13 smithi096 ceph-mon[36065]: pgmap v346: 129 pgs: 129 active+clean; 633 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-05-17T22:36:14.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:13 smithi012 ceph-mon[31632]: pgmap v346: 129 pgs: 129 active+clean; 633 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-05-17T22:36:14.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.686+0000 7fa037425700 1 -- 172.21.15.12:0/3699794282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0301080a0 msgr2=0x7fa03010fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:14.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.686+0000 7fa037425700 1 --2- 172.21.15.12:0/3699794282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0301080a0 0x7fa03010fa00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa028009990 tx=0x7fa028009ca0 comp rx=0 tx=0).stop 2024-05-17T22:36:14.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.686+0000 7fa037425700 1 -- 172.21.15.12:0/3699794282 shutdown_connections 2024-05-17T22:36:14.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.686+0000 7fa037425700 1 --2- 172.21.15.12:0/3699794282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0301080a0 0x7fa03010fa00 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:14.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.686+0000 7fa037425700 1 --2- 172.21.15.12:0/3699794282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa030102e70 0x7fa030103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:14.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.686+0000 7fa037425700 1 --2- 172.21.15.12:0/3699794282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa030107750 0x7fa030107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:14.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.686+0000 7fa037425700 1 -- 172.21.15.12:0/3699794282 >> 172.21.15.12:0/3699794282 conn(0x7fa030079ae0 msgr2=0x7fa030079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:36:14.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.686+0000 7fa037425700 1 -- 172.21.15.12:0/3699794282 shutdown_connections 2024-05-17T22:36:14.691 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:14 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 10]}]: dispatch 2024-05-17T22:36:14.692 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:14 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-05-17T22:36:14.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.686+0000 7fa037425700 1 -- 172.21.15.12:0/3699794282 wait complete. 2024-05-17T22:36:14.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.687+0000 7fa037425700 1 Processor -- start 2024-05-17T22:36:14.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.687+0000 7fa037425700 1 -- start start 2024-05-17T22:36:14.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.687+0000 7fa037425700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa030102e70 0x7fa03019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:14.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.687+0000 7fa037425700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa030107750 0x7fa03019acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:14.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.687+0000 7fa037425700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0301080a0 0x7fa03019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:14.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.687+0000 7fa037425700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa030112ce0 con 0x7fa030102e70 2024-05-17T22:36:14.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.687+0000 7fa037425700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa030112b60 con 0x7fa0301080a0 2024-05-17T22:36:14.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.687+0000 7fa037425700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa030112e60 con 0x7fa030107750 2024-05-17T22:36:14.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.688+0000 7fa0351c1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa030102e70 0x7fa03019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:14.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.688+0000 7fa0351c1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa030102e70 0x7fa03019a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44904/0 (socket says 172.21.15.12:44904) 2024-05-17T22:36:14.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.688+0000 7fa0351c1700 1 -- 172.21.15.12:0/2612874218 learned_addr learned my addr 172.21.15.12:0/2612874218 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:36:14.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.688+0000 7fa0359c2700 1 --2- 172.21.15.12:0/2612874218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0301080a0 0x7fa03019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:14.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.688+0000 7fa0349c0700 1 --2- 172.21.15.12:0/2612874218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa030107750 0x7fa03019acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:14.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.688+0000 7fa0351c1700 1 -- 172.21.15.12:0/2612874218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa030107750 msgr2=0x7fa03019acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:14.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.688+0000 7fa0351c1700 1 --2- 172.21.15.12:0/2612874218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa030107750 0x7fa03019acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:14.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.688+0000 7fa0351c1700 1 -- 172.21.15.12:0/2612874218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0301080a0 msgr2=0x7fa03019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:14.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.688+0000 7fa0351c1700 1 --2- 172.21.15.12:0/2612874218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0301080a0 0x7fa03019ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:14.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.688+0000 7fa0351c1700 1 -- 172.21.15.12:0/2612874218 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa03019f620 con 0x7fa030102e70 2024-05-17T22:36:14.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.688+0000 7fa0359c2700 1 --2- 172.21.15.12:0/2612874218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0301080a0 0x7fa03019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:36:14.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.688+0000 7fa0351c1700 1 --2- 172.21.15.12:0/2612874218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa030102e70 0x7fa03019a750 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa02c00ef10 tx=0x7fa02c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:36:14.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.688+0000 7fa0267fc700 1 -- 172.21.15.12:0/2612874218 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02c00ca70 con 0x7fa030102e70 2024-05-17T22:36:14.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.688+0000 7fa0267fc700 1 -- 172.21.15.12:0/2612874218 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa02c019070 con 0x7fa030102e70 2024-05-17T22:36:14.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.688+0000 7fa037425700 1 -- 172.21.15.12:0/2612874218 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa03019f8b0 con 0x7fa030102e70 2024-05-17T22:36:14.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.688+0000 7fa0267fc700 1 -- 172.21.15.12:0/2612874218 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02c014480 con 0x7fa030102e70 2024-05-17T22:36:14.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.688+0000 7fa037425700 1 -- 172.21.15.12:0/2612874218 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0301a7230 con 0x7fa030102e70 2024-05-17T22:36:14.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.690+0000 7fa037425700 1 -- 172.21.15.12:0/2612874218 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa030103280 con 0x7fa030102e70 2024-05-17T22:36:14.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.691+0000 7fa0267fc700 1 -- 172.21.15.12:0/2612874218 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa02c012050 con 0x7fa030102e70 2024-05-17T22:36:14.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.692+0000 7fa0267fc700 1 --2- 172.21.15.12:0/2612874218 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa01c0b2910 0x7fa01c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:14.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.692+0000 7fa0349c0700 1 --2- 172.21.15.12:0/2612874218 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa01c0b2910 0x7fa01c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:14.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.692+0000 7fa0267fc700 1 -- 172.21.15.12:0/2612874218 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9150+0+0 (secure 0 0 0) 0x7fa02c0e27b0 con 0x7fa030102e70 2024-05-17T22:36:14.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.692+0000 7fa0349c0700 1 --2- 172.21.15.12:0/2612874218 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa01c0b2910 0x7fa01c0b4e20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa0301048b0 tx=0x7fa020009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:36:14.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.696+0000 7fa0267fc700 1 -- 172.21.15.12:0/2612874218 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa02c010040 con 0x7fa030102e70 2024-05-17T22:36:14.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.910+0000 7fa037425700 1 -- 172.21.15.12:0/2612874218 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa03019bab0 con 0x7fa030102e70 2024-05-17T22:36:14.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.911+0000 7fa0267fc700 1 -- 172.21.15.12:0/2612874218 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24715 (secure 0 0 0) 0x7fa02c0a9fe0 con 0x7fa030102e70 2024-05-17T22:36:14.916 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:36:14.916 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":88,"fsid":"4d593780-149c-11ef-bc9a-c7b262605968","created":"2024-05-17T22:26:16.955626+0000","modified":"2024-05-17T22:36:14.610878+0000","last_up_change":"2024-05-17T22:33:36.551093+0000","last_in_change":"2024-05-17T22:33:15.032677+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T22:29:42.475513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T22:34:35.504697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T22:34:39.377097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-17T22:34:50.467684+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5d42798f-dc5f-4e66-bce2-ec33dc789e4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6803","nonce":2015605593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6805","nonce":2015605593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6809","nonce":2015605593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6807","nonce":2015605593}]},"public_addr":"172.21.15.12:6803/2015605593","cluster_addr":"172.21.15.12:6805/2015605593","heartbeat_back_addr":"172.21.15.12:6809/2015605593","heartbeat_front_addr":"172.21.15.12:6807/2015605593","state":["exists","up"]},{"osd":1,"uuid":"58afecc7-119b-4c16-a401-640fb77d4714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6801","nonce":1238316358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6803","nonce":1238316358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6807","nonce":1238316358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6805","nonce":1238316358}]},"public_addr":"172.21.15.96:6801/1238316358","cluster_addr":"172.21.15.96:6803/1238316358","heartbeat_back_addr":"172.21.15.96:6807/1238316358","heartbeat_front_addr":"172.21.15.96:6805/1238316358","state":["exists","up"]},{"osd":2,"uuid":"4bef4049-482f-49fb-82fb-38052400eb2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6801","nonce":148729815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6803","nonce":148729815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6807","nonce":148729815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6805","nonce":148729815}]},"public_addr":"172.21.15.151:6801/148729815","cluster_addr":"172.21.15.151:6803/148729815","heartbeat_back_addr":"172.21.15.151:6807/148729815","heartbeat_front_addr":"172.21.15.151:6805/148729815","state":["exists","up"]},{"osd":3,"uuid":"6fed8e36-82a2-49e4-82a3-16e6d9de87f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6811","nonce":3563309320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6813","nonce":3563309320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6817","nonce":3563309320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6815","nonce":3563309320}]},"public_addr":"172.21.15.12:6811/3563309320","cluster_addr":"172.21.15.12:6813/3563309320","heartbeat_back_addr":"172.21.15.12:6817/3563309320","heartbeat_front_addr":"172.21.15.12:6815/3563309320","state":["exists","up"]},{"osd":4,"uuid":"c0e8689a-2816-4089-ade3-931960bae4f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6809","nonce":2219103166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6811","nonce":2219103166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6815","nonce":2219103166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6813","nonce":2219103166}]},"public_addr":"172.21.15.96:6809/2219103166","cluster_addr":"172.21.15.96:6811/2219103166","heartbeat_back_addr":"172.21.15.96:6815/2219103166","heartbeat_front_addr":"172.21.15.96:6813/2219103166","state":["exists","up"]},{"osd":5,"uuid":"e76ba432-94ff-4998-aaba-2645d9e6055d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6809","nonce":2589835078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6811","nonce":2589835078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6815","nonce":2589835078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6813","nonce":2589835078}]},"public_addr":"172.21.15.151:6809/2589835078","cluster_addr":"172.21.15.151:6811/2589835078","heartbeat_back_addr":"172.21.15.151:6815/2589835078","heartbeat_front_addr":"172.21.15.151:6813/2589835078","state":["exists","up"]},{"osd":6,"uuid":"b7ece35c-efd1-40df-95da-05a0cc147372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6819","nonce":3112661724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6821","nonce":3112661724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6825","nonce":3112661724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6823","nonce":3112661724}]},"public_addr":"172.21.15.12:6819/3112661724","cluster_addr":"172.21.15.12:6821/3112661724","heartbeat_back_addr":"172.21.15.12:6825/3112661724","heartbeat_front_addr":"172.21.15.12:6823/3112661724","state":["exists","up"]},{"osd":7,"uuid":"6a65bf30-daf5-4fba-b67e-8e513d88a971","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6817","nonce":2449989110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6819","nonce":2449989110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6823","nonce":2449989110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6821","nonce":2449989110}]},"public_addr":"172.21.15.96:6817/2449989110","cluster_addr":"172.21.15.96:6819/2449989110","heartbeat_back_addr":"172.21.15.96:6823/2449989110","heartbeat_front_addr":"172.21.15.96:6821/2449989110","state":["exists","up"]},{"osd":8,"uuid":"ad62ea44-e416-4a3e-bbbe-93f2a710dbc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6817","nonce":2545224628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6819","nonce":2545224628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6823","nonce":2545224628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6821","nonce":2545224628}]},"public_addr":"172.21.15.151:6817/2545224628","cluster_addr":"172.21.15.151:6819/2545224628","heartbeat_back_addr":"172.21.15.151:6823/2545224628","heartbeat_front_addr":"172.21.15.151:6821/2545224628","state":["exists","up"]},{"osd":9,"uuid":"1275de08-2a3c-48f3-920b-dd68b4024d42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6827","nonce":2100661273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6829","nonce":2100661273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6833","nonce":2100661273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6831","nonce":2100661273}]},"public_addr":"172.21.15.12:6827/2100661273","cluster_addr":"172.21.15.12:6829/2100661273","heartbeat_back_addr":"172.21.15.12:6833/2100661273","heartbeat_front_addr":"172.21.15.12:6831/2100661273","state":["exists","up"]},{"osd":10,"uuid":"347e4e40-9a9d-43fe-8f52-fdd3199a2a5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6825","nonce":304016309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6827","nonce":304016309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6831","nonce":304016309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6829","nonce":304016309}]},"public_addr":"172.21.15.96:6825/304016309","cluster_addr":"172.21.15.96:6827/304016309","heartbeat_back_addr":"172.21.15.96:6831/304016309","heartbeat_front_addr":"172.21.15.96:6829/304016309","state":["exists","up"]},{"osd":11,"uuid":"ee8d3502-8b3c-4e6f-824f-3a97e19111ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6825","nonce":3822307314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6827","nonce":3822307314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6831","nonce":3822307314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6829","nonce":3822307314}]},"public_addr":"172.21.15.151:6825/3822307314","cluster_addr":"172.21.15.151:6827/3822307314","heartbeat_back_addr":"172.21.15.151:6831/3822307314","heartbeat_front_addr":"172.21.15.151:6829/3822307314","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:28:51.542055+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:29:15.566424+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:29:39.500981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:04.797907+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:29.655650+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:55.211880+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:31:21.559462+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:31:47.154852+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:32:13.114892+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:32:40.529708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:33:06.930942+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:33:34.061783+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":10}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/2682700253":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6801/2250046521":"2024-05-18T22:27:14.372972+0000","172.21.15.12:0/4241094239":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/4172568611":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/3512529242":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6800/1039789215":"2024-05-18T22:26:45.466719+0000","172.21.15.12:6800/2250046521":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6801/1039789215":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/2043758453":"2024-05-18T22:27:14.372972+0000","172.21.15.12:0/2694593732":"2024-05-18T22:26:45.466719+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T22:36:14.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.913+0000 7fa037425700 1 -- 172.21.15.12:0/2612874218 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa01c0b2910 msgr2=0x7fa01c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:14.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.913+0000 7fa037425700 1 --2- 172.21.15.12:0/2612874218 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa01c0b2910 0x7fa01c0b4e20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa0301048b0 tx=0x7fa020009260 comp rx=0 tx=0).stop 2024-05-17T22:36:14.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.913+0000 7fa037425700 1 -- 172.21.15.12:0/2612874218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa030102e70 msgr2=0x7fa03019a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:14.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.913+0000 7fa037425700 1 --2- 172.21.15.12:0/2612874218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa030102e70 0x7fa03019a750 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa02c00ef10 tx=0x7fa02c00c3a0 comp rx=0 tx=0).stop 2024-05-17T22:36:14.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.913+0000 7fa037425700 1 -- 172.21.15.12:0/2612874218 shutdown_connections 2024-05-17T22:36:14.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.913+0000 7fa037425700 1 --2- 172.21.15.12:0/2612874218 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa01c0b2910 0x7fa01c0b4e20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:14.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.913+0000 7fa037425700 1 --2- 172.21.15.12:0/2612874218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0301080a0 0x7fa03019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:14.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.913+0000 7fa037425700 1 --2- 172.21.15.12:0/2612874218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa030102e70 0x7fa03019a750 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:14.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.913+0000 7fa037425700 1 --2- 172.21.15.12:0/2612874218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa030107750 0x7fa03019acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:14.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.913+0000 7fa037425700 1 -- 172.21.15.12:0/2612874218 >> 172.21.15.12:0/2612874218 conn(0x7fa030079ae0 msgr2=0x7fa03006c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:36:14.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.914+0000 7fa037425700 1 -- 172.21.15.12:0/2612874218 shutdown_connections 2024-05-17T22:36:14.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:14.914+0000 7fa037425700 1 -- 172.21.15.12:0/2612874218 wait complete. 2024-05-17T22:36:14.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:14 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 10]}]: dispatch 2024-05-17T22:36:14.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:14 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-05-17T22:36:15.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:14 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 10]}]: dispatch 2024-05-17T22:36:15.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:14 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-05-17T22:36:15.655 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-17T22:36:15.655 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-17T22:36:15.917 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:36:15.956 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:15 smithi012 ceph-mon[31632]: pgmap v347: 129 pgs: 129 active+clean; 633 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-05-17T22:36:15.956 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:15 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 10]}]': finished 2024-05-17T22:36:15.956 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:15 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-05-17T22:36:15.956 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:15 smithi012 ceph-mon[31632]: osdmap e88: 12 total, 12 up, 12 in 2024-05-17T22:36:15.957 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:15 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/2612874218' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:36:15.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:15 smithi151 ceph-mon[36101]: pgmap v347: 129 pgs: 129 active+clean; 633 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-05-17T22:36:15.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:15 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 10]}]': finished 2024-05-17T22:36:15.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:15 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-05-17T22:36:15.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:15 smithi151 ceph-mon[36101]: osdmap e88: 12 total, 12 up, 12 in 2024-05-17T22:36:15.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:15 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/2612874218' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:36:16.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:15 smithi096 ceph-mon[36065]: pgmap v347: 129 pgs: 129 active+clean; 633 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-05-17T22:36:16.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:15 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 10]}]': finished 2024-05-17T22:36:16.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:15 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-05-17T22:36:16.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:15 smithi096 ceph-mon[36065]: osdmap e88: 12 total, 12 up, 12 in 2024-05-17T22:36:16.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:15 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/2612874218' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:36:16.973 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:16 smithi012 ceph-mon[31632]: osdmap e89: 12 total, 12 up, 12 in 2024-05-17T22:36:16.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:16 smithi151 ceph-mon[36101]: osdmap e89: 12 total, 12 up, 12 in 2024-05-17T22:36:17.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:16 smithi096 ceph-mon[36065]: osdmap e89: 12 total, 12 up, 12 in 2024-05-17T22:36:17.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.614+0000 7fb54a522700 1 -- 172.21.15.12:0/182822717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5441080a0 msgr2=0x7fb54410fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:17.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.614+0000 7fb54a522700 1 --2- 172.21.15.12:0/182822717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5441080a0 0x7fb54410fa00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb53c00b0a0 tx=0x7fb53c00b3b0 comp rx=0 tx=0).stop 2024-05-17T22:36:17.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.614+0000 7fb54a522700 1 -- 172.21.15.12:0/182822717 shutdown_connections 2024-05-17T22:36:17.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.614+0000 7fb54a522700 1 --2- 172.21.15.12:0/182822717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5441080a0 0x7fb54410fa00 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:17.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.614+0000 7fb54a522700 1 --2- 172.21.15.12:0/182822717 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb544102e70 0x7fb544103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:17.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.614+0000 7fb54a522700 1 --2- 172.21.15.12:0/182822717 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb544107750 0x7fb544107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:17.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.614+0000 7fb54a522700 1 -- 172.21.15.12:0/182822717 >> 172.21.15.12:0/182822717 conn(0x7fb544079ae0 msgr2=0x7fb544079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:36:17.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.614+0000 7fb54a522700 1 -- 172.21.15.12:0/182822717 shutdown_connections 2024-05-17T22:36:17.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.614+0000 7fb54a522700 1 -- 172.21.15.12:0/182822717 wait complete. 2024-05-17T22:36:17.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.615+0000 7fb54a522700 1 Processor -- start 2024-05-17T22:36:17.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.615+0000 7fb54a522700 1 -- start start 2024-05-17T22:36:17.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.615+0000 7fb54a522700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb544102e70 0x7fb54419b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:17.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.615+0000 7fb54a522700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb544107750 0x7fb54419bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:17.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.615+0000 7fb54a522700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5441080a0 0x7fb54419c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:17.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.615+0000 7fb54a522700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb544112d10 con 0x7fb5441080a0 2024-05-17T22:36:17.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.615+0000 7fb54a522700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb544112b90 con 0x7fb544107750 2024-05-17T22:36:17.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.615+0000 7fb54a522700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb544112e90 con 0x7fb544102e70 2024-05-17T22:36:17.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.616+0000 7fb548abf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5441080a0 0x7fb54419c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:17.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.616+0000 7fb548abf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5441080a0 0x7fb54419c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44926/0 (socket says 172.21.15.12:44926) 2024-05-17T22:36:17.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.616+0000 7fb548abf700 1 -- 172.21.15.12:0/2540276625 learned_addr learned my addr 172.21.15.12:0/2540276625 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:36:17.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.616+0000 7fb5437fe700 1 --2- 172.21.15.12:0/2540276625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb544107750 0x7fb54419bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:17.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.616+0000 7fb543fff700 1 --2- 172.21.15.12:0/2540276625 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb544102e70 0x7fb54419b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:17.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.616+0000 7fb548abf700 1 -- 172.21.15.12:0/2540276625 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb544102e70 msgr2=0x7fb54419b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:17.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.616+0000 7fb548abf700 1 --2- 172.21.15.12:0/2540276625 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb544102e70 0x7fb54419b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:17.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.616+0000 7fb548abf700 1 -- 172.21.15.12:0/2540276625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb544107750 msgr2=0x7fb54419bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:17.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.616+0000 7fb548abf700 1 --2- 172.21.15.12:0/2540276625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb544107750 0x7fb54419bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:17.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.616+0000 7fb548abf700 1 -- 172.21.15.12:0/2540276625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5441a6970 con 0x7fb5441080a0 2024-05-17T22:36:17.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.616+0000 7fb548abf700 1 --2- 172.21.15.12:0/2540276625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5441080a0 0x7fb54419c0c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb53c00b830 tx=0x7fb53c004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:36:17.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.616+0000 7fb5417fa700 1 -- 172.21.15.12:0/2540276625 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53c021070 con 0x7fb5441080a0 2024-05-17T22:36:17.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.616+0000 7fb5417fa700 1 -- 172.21.15.12:0/2540276625 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb53c007e30 con 0x7fb5441080a0 2024-05-17T22:36:17.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.616+0000 7fb54a522700 1 -- 172.21.15.12:0/2540276625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5441a6c00 con 0x7fb5441080a0 2024-05-17T22:36:17.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.616+0000 7fb5417fa700 1 -- 172.21.15.12:0/2540276625 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53c004700 con 0x7fb5441080a0 2024-05-17T22:36:17.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.616+0000 7fb54a522700 1 -- 172.21.15.12:0/2540276625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5441a70b0 con 0x7fb5441080a0 2024-05-17T22:36:17.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.618+0000 7fb54a522700 1 -- 172.21.15.12:0/2540276625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb544103280 con 0x7fb5441080a0 2024-05-17T22:36:17.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.619+0000 7fb5417fa700 1 -- 172.21.15.12:0/2540276625 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fb53c003d70 con 0x7fb5441080a0 2024-05-17T22:36:17.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.620+0000 7fb5417fa700 1 --2- 172.21.15.12:0/2540276625 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb52c0b2910 0x7fb52c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:17.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.620+0000 7fb543fff700 1 --2- 172.21.15.12:0/2540276625 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb52c0b2910 0x7fb52c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:17.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.620+0000 7fb5417fa700 1 -- 172.21.15.12:0/2540276625 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fb53c00f020 con 0x7fb5441080a0 2024-05-17T22:36:17.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.621+0000 7fb543fff700 1 --2- 172.21.15.12:0/2540276625 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb52c0b2910 0x7fb52c0b4e20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb534005950 tx=0x7fb53400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:36:17.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.623+0000 7fb5417fa700 1 -- 172.21.15.12:0/2540276625 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb53c02a020 con 0x7fb5441080a0 2024-05-17T22:36:17.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.834+0000 7fb54a522700 1 -- 172.21.15.12:0/2540276625 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb54402ea80 con 0x7fb52c0b2910 2024-05-17T22:36:17.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:17 smithi012 ceph-mon[31632]: pgmap v350: 129 pgs: 1 peering, 128 active+clean; 638 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.1 KiB/s wr, 8 op/s 2024-05-17T22:36:17.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:17 smithi012 ceph-mon[31632]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-05-17T22:36:17.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.841+0000 7fb5417fa700 1 -- 172.21.15.12:0/2540276625 <== mgr.14152 v2:172.21.15.12:6800/1388514047 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340248 (secure 0 0 0) 0x7fb54402ea80 con 0x7fb52c0b2910 2024-05-17T22:36:17.849 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:36:17.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.844+0000 7fb54a522700 1 -- 172.21.15.12:0/2540276625 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb52c0b2910 msgr2=0x7fb52c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:17.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.844+0000 7fb54a522700 1 --2- 172.21.15.12:0/2540276625 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb52c0b2910 0x7fb52c0b4e20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb534005950 tx=0x7fb53400b410 comp rx=0 tx=0).stop 2024-05-17T22:36:17.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.844+0000 7fb54a522700 1 -- 172.21.15.12:0/2540276625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5441080a0 msgr2=0x7fb54419c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:17.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.844+0000 7fb54a522700 1 --2- 172.21.15.12:0/2540276625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5441080a0 0x7fb54419c0c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb53c00b830 tx=0x7fb53c004ce0 comp rx=0 tx=0).stop 2024-05-17T22:36:17.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.844+0000 7fb54a522700 1 -- 172.21.15.12:0/2540276625 shutdown_connections 2024-05-17T22:36:17.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.844+0000 7fb54a522700 1 --2- 172.21.15.12:0/2540276625 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb52c0b2910 0x7fb52c0b4e20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:17.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.844+0000 7fb54a522700 1 --2- 172.21.15.12:0/2540276625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5441080a0 0x7fb54419c0c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:17.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.844+0000 7fb54a522700 1 --2- 172.21.15.12:0/2540276625 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb544102e70 0x7fb54419b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:17.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.844+0000 7fb54a522700 1 --2- 172.21.15.12:0/2540276625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb544107750 0x7fb54419bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:17.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.844+0000 7fb54a522700 1 -- 172.21.15.12:0/2540276625 >> 172.21.15.12:0/2540276625 conn(0x7fb544079ae0 msgr2=0x7fb5441052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:36:17.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.845+0000 7fb54a522700 1 -- 172.21.15.12:0/2540276625 shutdown_connections 2024-05-17T22:36:17.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:17.845+0000 7fb54a522700 1 -- 172.21.15.12:0/2540276625 wait complete. 2024-05-17T22:36:17.853 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-05-17T22:36:17.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:17 smithi151 ceph-mon[36101]: pgmap v350: 129 pgs: 1 peering, 128 active+clean; 638 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.1 KiB/s wr, 8 op/s 2024-05-17T22:36:17.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:17 smithi151 ceph-mon[36101]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-05-17T22:36:18.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:17 smithi096 ceph-mon[36065]: pgmap v350: 129 pgs: 1 peering, 128 active+clean; 638 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.1 KiB/s wr, 8 op/s 2024-05-17T22:36:18.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:17 smithi096 ceph-mon[36065]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-05-17T22:36:18.638 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":350,"stamp":"2024-05-17T22:36:16.503767+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652801,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":249,"num_read_kb":218,"num_write":291,"num_write_kb":2265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":236,"ondisk_log_size":236,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":383652,"kb_used_data":13696,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124558172,"statfs":{"total":1151940427776,"available":1151547568128,"internally_reserved":0,"allocated":14024704,"data_stored":7888985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27462,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":58,"num_write":16,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002475"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.551612+0000","last_change":"2024-05-17T22:34:40.422850+0000","last_active":"2024-05-17T22:35:15.551612+0000","last_peered":"2024-05-17T22:35:15.551612+0000","last_clean":"2024-05-17T22:35:15.551612+0000","last_became_active":"2024-05-17T22:34:40.422474+0000","last_became_peered":"2024-05-17T22:34:40.422474+0000","last_unstale":"2024-05-17T22:35:15.551612+0000","last_undegraded":"2024-05-17T22:35:15.551612+0000","last_fullsized":"2024-05-17T22:35:15.551612+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:23:51.537128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.545559+0000","last_change":"2024-05-17T22:34:40.427907+0000","last_active":"2024-05-17T22:35:15.545559+0000","last_peered":"2024-05-17T22:35:15.545559+0000","last_clean":"2024-05-17T22:35:15.545559+0000","last_became_active":"2024-05-17T22:34:40.427511+0000","last_became_peered":"2024-05-17T22:34:40.427511+0000","last_unstale":"2024-05-17T22:35:15.545559+0000","last_undegraded":"2024-05-17T22:35:15.545559+0000","last_fullsized":"2024-05-17T22:35:15.545559+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:01:46.707820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.545402+0000","last_change":"2024-05-17T22:34:40.436370+0000","last_active":"2024-05-17T22:35:15.545402+0000","last_peered":"2024-05-17T22:35:15.545402+0000","last_clean":"2024-05-17T22:35:15.545402+0000","last_became_active":"2024-05-17T22:34:40.434089+0000","last_became_peered":"2024-05-17T22:34:40.434089+0000","last_unstale":"2024-05-17T22:35:15.545402+0000","last_undegraded":"2024-05-17T22:35:15.545402+0000","last_fullsized":"2024-05-17T22:35:15.545402+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:58:16.775471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.557050+0000","last_change":"2024-05-17T22:35:15.556681+0000","last_active":"2024-05-17T22:35:15.557050+0000","last_peered":"2024-05-17T22:35:15.557050+0000","last_clean":"2024-05-17T22:35:15.557050+0000","last_became_active":"2024-05-17T22:35:15.556427+0000","last_became_peered":"2024-05-17T22:35:15.556427+0000","last_unstale":"2024-05-17T22:35:15.557050+0000","last_undegraded":"2024-05-17T22:35:15.557050+0000","last_fullsized":"2024-05-17T22:35:15.557050+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:28:01.726262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T22:36:14.624343+0000","last_change":"2024-05-17T22:34:40.434394+0000","last_active":"2024-05-17T22:36:14.624343+0000","last_peered":"2024-05-17T22:36:14.624343+0000","last_clean":"2024-05-17T22:36:14.624343+0000","last_became_active":"2024-05-17T22:34:40.433489+0000","last_became_peered":"2024-05-17T22:34:40.433489+0000","last_unstale":"2024-05-17T22:36:14.624343+0000","last_undegraded":"2024-05-17T22:36:14.624343+0000","last_fullsized":"2024-05-17T22:36:14.624343+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:26:24.068814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.553601+0000","last_change":"2024-05-17T22:35:15.553601+0000","last_active":"2024-05-17T22:35:15.553601+0000","last_peered":"2024-05-17T22:35:15.553601+0000","last_clean":"2024-05-17T22:35:15.553601+0000","last_became_active":"2024-05-17T22:35:15.553116+0000","last_became_peered":"2024-05-17T22:35:15.553116+0000","last_unstale":"2024-05-17T22:35:15.553601+0000","last_undegraded":"2024-05-17T22:35:15.553601+0000","last_fullsized":"2024-05-17T22:35:15.553601+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:09:38.401107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.550878+0000","last_change":"2024-05-17T22:34:40.430479+0000","last_active":"2024-05-17T22:35:15.550878+0000","last_peered":"2024-05-17T22:35:15.550878+0000","last_clean":"2024-05-17T22:35:15.550878+0000","last_became_active":"2024-05-17T22:34:40.430025+0000","last_became_peered":"2024-05-17T22:34:40.430025+0000","last_unstale":"2024-05-17T22:35:15.550878+0000","last_undegraded":"2024-05-17T22:35:15.550878+0000","last_fullsized":"2024-05-17T22:35:15.550878+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:14:19.725096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.710439+0000","last_change":"2024-05-17T22:34:40.431711+0000","last_active":"2024-05-17T22:35:15.710439+0000","last_peered":"2024-05-17T22:35:15.710439+0000","last_clean":"2024-05-17T22:35:15.710439+0000","last_became_active":"2024-05-17T22:34:40.431368+0000","last_became_peered":"2024-05-17T22:34:40.431368+0000","last_unstale":"2024-05-17T22:35:15.710439+0000","last_undegraded":"2024-05-17T22:35:15.710439+0000","last_fullsized":"2024-05-17T22:35:15.710439+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:06:34.904652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.835208+0000","last_change":"2024-05-17T22:34:40.435593+0000","last_active":"2024-05-17T22:35:15.835208+0000","last_peered":"2024-05-17T22:35:15.835208+0000","last_clean":"2024-05-17T22:35:15.835208+0000","last_became_active":"2024-05-17T22:34:40.435241+0000","last_became_peered":"2024-05-17T22:34:40.435241+0000","last_unstale":"2024-05-17T22:35:15.835208+0000","last_undegraded":"2024-05-17T22:35:15.835208+0000","last_fullsized":"2024-05-17T22:35:15.835208+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:08:57.239998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.549564+0000","last_change":"2024-05-17T22:34:40.436248+0000","last_active":"2024-05-17T22:35:15.549564+0000","last_peered":"2024-05-17T22:35:15.549564+0000","last_clean":"2024-05-17T22:35:15.549564+0000","last_became_active":"2024-05-17T22:34:40.435824+0000","last_became_peered":"2024-05-17T22:34:40.435824+0000","last_unstale":"2024-05-17T22:35:15.549564+0000","last_undegraded":"2024-05-17T22:35:15.549564+0000","last_fullsized":"2024-05-17T22:35:15.549564+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:39:07.297470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.551323+0000","last_change":"2024-05-17T22:34:40.435843+0000","last_active":"2024-05-17T22:35:15.551323+0000","last_peered":"2024-05-17T22:35:15.551323+0000","last_clean":"2024-05-17T22:35:15.551323+0000","last_became_active":"2024-05-17T22:34:40.435500+0000","last_became_peered":"2024-05-17T22:34:40.435500+0000","last_unstale":"2024-05-17T22:35:15.551323+0000","last_undegraded":"2024-05-17T22:35:15.551323+0000","last_fullsized":"2024-05-17T22:35:15.551323+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:11:28.115850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.546620+0000","last_change":"2024-05-17T22:34:40.432044+0000","last_active":"2024-05-17T22:35:15.546620+0000","last_peered":"2024-05-17T22:35:15.546620+0000","last_clean":"2024-05-17T22:35:15.546620+0000","last_became_active":"2024-05-17T22:34:40.429141+0000","last_became_peered":"2024-05-17T22:34:40.429141+0000","last_unstale":"2024-05-17T22:35:15.546620+0000","last_undegraded":"2024-05-17T22:35:15.546620+0000","last_fullsized":"2024-05-17T22:35:15.546620+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:04:53.428681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.549412+0000","last_change":"2024-05-17T22:34:40.438474+0000","last_active":"2024-05-17T22:35:15.549412+0000","last_peered":"2024-05-17T22:35:15.549412+0000","last_clean":"2024-05-17T22:35:15.549412+0000","last_became_active":"2024-05-17T22:34:40.438109+0000","last_became_peered":"2024-05-17T22:34:40.438109+0000","last_unstale":"2024-05-17T22:35:15.549412+0000","last_undegraded":"2024-05-17T22:35:15.549412+0000","last_fullsized":"2024-05-17T22:35:15.549412+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:26:00.665246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.545176+0000","last_change":"2024-05-17T22:34:40.431563+0000","last_active":"2024-05-17T22:35:15.545176+0000","last_peered":"2024-05-17T22:35:15.545176+0000","last_clean":"2024-05-17T22:35:15.545176+0000","last_became_active":"2024-05-17T22:34:40.430992+0000","last_became_peered":"2024-05-17T22:34:40.430992+0000","last_unstale":"2024-05-17T22:35:15.545176+0000","last_undegraded":"2024-05-17T22:35:15.545176+0000","last_fullsized":"2024-05-17T22:35:15.545176+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:51:26.280440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.545167+0000","last_change":"2024-05-17T22:34:40.433629+0000","last_active":"2024-05-17T22:35:15.545167+0000","last_peered":"2024-05-17T22:35:15.545167+0000","last_clean":"2024-05-17T22:35:15.545167+0000","last_became_active":"2024-05-17T22:34:40.433270+0000","last_became_peered":"2024-05-17T22:34:40.433270+0000","last_unstale":"2024-05-17T22:35:15.545167+0000","last_undegraded":"2024-05-17T22:35:15.545167+0000","last_fullsized":"2024-05-17T22:35:15.545167+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:57:51.164078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.834304+0000","last_change":"2024-05-17T22:34:40.434218+0000","last_active":"2024-05-17T22:35:15.834304+0000","last_peered":"2024-05-17T22:35:15.834304+0000","last_clean":"2024-05-17T22:35:15.834304+0000","last_became_active":"2024-05-17T22:34:40.433849+0000","last_became_peered":"2024-05-17T22:34:40.433849+0000","last_unstale":"2024-05-17T22:35:15.834304+0000","last_undegraded":"2024-05-17T22:35:15.834304+0000","last_fullsized":"2024-05-17T22:35:15.834304+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:41:55.984109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.550739+0000","last_change":"2024-05-17T22:34:40.427892+0000","last_active":"2024-05-17T22:35:15.550739+0000","last_peered":"2024-05-17T22:35:15.550739+0000","last_clean":"2024-05-17T22:35:15.550739+0000","last_became_active":"2024-05-17T22:34:40.427303+0000","last_became_peered":"2024-05-17T22:34:40.427303+0000","last_unstale":"2024-05-17T22:35:15.550739+0000","last_undegraded":"2024-05-17T22:35:15.550739+0000","last_fullsized":"2024-05-17T22:35:15.550739+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:59:38.632123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.548517+0000","last_change":"2024-05-17T22:34:40.424784+0000","last_active":"2024-05-17T22:35:15.548517+0000","last_peered":"2024-05-17T22:35:15.548517+0000","last_clean":"2024-05-17T22:35:15.548517+0000","last_became_active":"2024-05-17T22:34:40.424439+0000","last_became_peered":"2024-05-17T22:34:40.424439+0000","last_unstale":"2024-05-17T22:35:15.548517+0000","last_undegraded":"2024-05-17T22:35:15.548517+0000","last_fullsized":"2024-05-17T22:35:15.548517+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:02:53.681600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.553263+0000","last_change":"2024-05-17T22:34:40.435023+0000","last_active":"2024-05-17T22:35:15.553263+0000","last_peered":"2024-05-17T22:35:15.553263+0000","last_clean":"2024-05-17T22:35:15.553263+0000","last_became_active":"2024-05-17T22:34:40.434441+0000","last_became_peered":"2024-05-17T22:34:40.434441+0000","last_unstale":"2024-05-17T22:35:15.553263+0000","last_undegraded":"2024-05-17T22:35:15.553263+0000","last_fullsized":"2024-05-17T22:35:15.553263+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:16:07.671897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.551344+0000","last_change":"2024-05-17T22:34:52.493192+0000","last_active":"2024-05-17T22:35:15.551344+0000","last_peered":"2024-05-17T22:35:15.551344+0000","last_clean":"2024-05-17T22:35:15.551344+0000","last_became_active":"2024-05-17T22:34:52.492786+0000","last_became_peered":"2024-05-17T22:34:52.492786+0000","last_unstale":"2024-05-17T22:35:15.551344+0000","last_undegraded":"2024-05-17T22:35:15.551344+0000","last_fullsized":"2024-05-17T22:35:15.551344+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:43:40.152240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":50,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:48.028306+0000","last_change":"2024-05-17T22:34:37.402360+0000","last_active":"2024-05-17T22:35:48.028306+0000","last_peered":"2024-05-17T22:35:48.028306+0000","last_clean":"2024-05-17T22:35:48.028306+0000","last_became_active":"2024-05-17T22:34:37.401882+0000","last_became_peered":"2024-05-17T22:34:37.401882+0000","last_unstale":"2024-05-17T22:35:48.028306+0000","last_undegraded":"2024-05-17T22:35:48.028306+0000","last_fullsized":"2024-05-17T22:35:48.028306+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:53:13.300626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.710475+0000","last_change":"2024-05-17T22:34:40.436259+0000","last_active":"2024-05-17T22:35:15.710475+0000","last_peered":"2024-05-17T22:35:15.710475+0000","last_clean":"2024-05-17T22:35:15.710475+0000","last_became_active":"2024-05-17T22:34:40.435899+0000","last_became_peered":"2024-05-17T22:34:40.435899+0000","last_unstale":"2024-05-17T22:35:15.710475+0000","last_undegraded":"2024-05-17T22:35:15.710475+0000","last_fullsized":"2024-05-17T22:35:15.710475+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:26:29.327513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.546155+0000","last_change":"2024-05-17T22:34:52.495236+0000","last_active":"2024-05-17T22:35:15.546155+0000","last_peered":"2024-05-17T22:35:15.546155+0000","last_clean":"2024-05-17T22:35:15.546155+0000","last_became_active":"2024-05-17T22:34:52.494783+0000","last_became_peered":"2024-05-17T22:34:52.494783+0000","last_unstale":"2024-05-17T22:35:15.546155+0000","last_undegraded":"2024-05-17T22:35:15.546155+0000","last_fullsized":"2024-05-17T22:35:15.546155+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:08:15.010706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.710726+0000","last_change":"2024-05-17T22:34:37.441129+0000","last_active":"2024-05-17T22:35:15.710726+0000","last_peered":"2024-05-17T22:35:15.710726+0000","last_clean":"2024-05-17T22:35:15.710726+0000","last_became_active":"2024-05-17T22:34:37.440738+0000","last_became_peered":"2024-05-17T22:34:37.440738+0000","last_unstale":"2024-05-17T22:35:15.710726+0000","last_undegraded":"2024-05-17T22:35:15.710726+0000","last_fullsized":"2024-05-17T22:35:15.710726+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:03:16.945076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.545621+0000","last_change":"2024-05-17T22:34:40.434146+0000","last_active":"2024-05-17T22:35:15.545621+0000","last_peered":"2024-05-17T22:35:15.545621+0000","last_clean":"2024-05-17T22:35:15.545621+0000","last_became_active":"2024-05-17T22:34:40.433802+0000","last_became_peered":"2024-05-17T22:34:40.433802+0000","last_unstale":"2024-05-17T22:35:15.545621+0000","last_undegraded":"2024-05-17T22:35:15.545621+0000","last_fullsized":"2024-05-17T22:35:15.545621+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:08:39.905409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.546179+0000","last_change":"2024-05-17T22:34:52.501134+0000","last_active":"2024-05-17T22:35:15.546179+0000","last_peered":"2024-05-17T22:35:15.546179+0000","last_clean":"2024-05-17T22:35:15.546179+0000","last_became_active":"2024-05-17T22:34:52.500547+0000","last_became_peered":"2024-05-17T22:34:52.500547+0000","last_unstale":"2024-05-17T22:35:15.546179+0000","last_undegraded":"2024-05-17T22:35:15.546179+0000","last_fullsized":"2024-05-17T22:35:15.546179+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:13:04.200865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T22:36:14.624749+0000","last_change":"2024-05-17T22:34:37.404019+0000","last_active":"2024-05-17T22:36:14.624749+0000","last_peered":"2024-05-17T22:36:14.624749+0000","last_clean":"2024-05-17T22:36:14.624749+0000","last_became_active":"2024-05-17T22:34:37.403607+0000","last_became_peered":"2024-05-17T22:34:37.403607+0000","last_unstale":"2024-05-17T22:36:14.624749+0000","last_undegraded":"2024-05-17T22:36:14.624749+0000","last_fullsized":"2024-05-17T22:36:14.624749+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:30:21.405195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.834685+0000","last_change":"2024-05-17T22:34:40.426889+0000","last_active":"2024-05-17T22:35:15.834685+0000","last_peered":"2024-05-17T22:35:15.834685+0000","last_clean":"2024-05-17T22:35:15.834685+0000","last_became_active":"2024-05-17T22:34:40.426512+0000","last_became_peered":"2024-05-17T22:34:40.426512+0000","last_unstale":"2024-05-17T22:35:15.834685+0000","last_undegraded":"2024-05-17T22:35:15.834685+0000","last_fullsized":"2024-05-17T22:35:15.834685+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:30:51.363177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.546057+0000","last_change":"2024-05-17T22:34:52.487514+0000","last_active":"2024-05-17T22:35:15.546057+0000","last_peered":"2024-05-17T22:35:15.546057+0000","last_clean":"2024-05-17T22:35:15.546057+0000","last_became_active":"2024-05-17T22:34:52.487068+0000","last_became_peered":"2024-05-17T22:34:52.487068+0000","last_unstale":"2024-05-17T22:35:15.546057+0000","last_undegraded":"2024-05-17T22:35:15.546057+0000","last_fullsized":"2024-05-17T22:35:15.546057+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:52:13.573986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T22:36:14.624545+0000","last_change":"2024-05-17T22:34:37.400518+0000","last_active":"2024-05-17T22:36:14.624545+0000","last_peered":"2024-05-17T22:36:14.624545+0000","last_clean":"2024-05-17T22:36:14.624545+0000","last_became_active":"2024-05-17T22:34:37.400145+0000","last_became_peered":"2024-05-17T22:34:37.400145+0000","last_unstale":"2024-05-17T22:36:14.624545+0000","last_undegraded":"2024-05-17T22:36:14.624545+0000","last_fullsized":"2024-05-17T22:36:14.624545+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:56:53.593624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.552295+0000","last_change":"2024-05-17T22:34:40.434148+0000","last_active":"2024-05-17T22:35:15.552295+0000","last_peered":"2024-05-17T22:35:15.552295+0000","last_clean":"2024-05-17T22:35:15.552295+0000","last_became_active":"2024-05-17T22:34:40.433613+0000","last_became_peered":"2024-05-17T22:34:40.433613+0000","last_unstale":"2024-05-17T22:35:15.552295+0000","last_undegraded":"2024-05-17T22:35:15.552295+0000","last_fullsized":"2024-05-17T22:35:15.552295+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:09:39.049852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.552964+0000","last_change":"2024-05-17T22:34:52.490417+0000","last_active":"2024-05-17T22:35:15.552964+0000","last_peered":"2024-05-17T22:35:15.552964+0000","last_clean":"2024-05-17T22:35:15.552964+0000","last_became_active":"2024-05-17T22:34:52.489953+0000","last_became_peered":"2024-05-17T22:34:52.489953+0000","last_unstale":"2024-05-17T22:35:15.552964+0000","last_undegraded":"2024-05-17T22:35:15.552964+0000","last_fullsized":"2024-05-17T22:35:15.552964+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:23:56.491899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":72,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:36:11.076130+0000","last_change":"2024-05-17T22:34:37.405304+0000","last_active":"2024-05-17T22:36:11.076130+0000","last_peered":"2024-05-17T22:36:11.076130+0000","last_clean":"2024-05-17T22:36:11.076130+0000","last_became_active":"2024-05-17T22:34:37.404552+0000","last_became_peered":"2024-05-17T22:34:37.404552+0000","last_unstale":"2024-05-17T22:36:11.076130+0000","last_undegraded":"2024-05-17T22:36:11.076130+0000","last_fullsized":"2024-05-17T22:36:11.076130+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:57:06.556886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T22:36:14.624060+0000","last_change":"2024-05-17T22:34:40.430537+0000","last_active":"2024-05-17T22:36:14.624060+0000","last_peered":"2024-05-17T22:36:14.624060+0000","last_clean":"2024-05-17T22:36:14.624060+0000","last_became_active":"2024-05-17T22:34:40.430193+0000","last_became_peered":"2024-05-17T22:34:40.430193+0000","last_unstale":"2024-05-17T22:36:14.624060+0000","last_undegraded":"2024-05-17T22:36:14.624060+0000","last_fullsized":"2024-05-17T22:36:14.624060+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:58:42.091445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.559077+0000","last_change":"2024-05-17T22:35:15.559077+0000","last_active":"2024-05-17T22:35:15.559077+0000","last_peered":"2024-05-17T22:35:15.559077+0000","last_clean":"2024-05-17T22:35:15.559077+0000","last_became_active":"2024-05-17T22:35:15.558382+0000","last_became_peered":"2024-05-17T22:35:15.558382+0000","last_unstale":"2024-05-17T22:35:15.559077+0000","last_undegraded":"2024-05-17T22:35:15.559077+0000","last_fullsized":"2024-05-17T22:35:15.559077+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:20:16.009620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:48.027593+0000","last_change":"2024-05-17T22:34:37.405332+0000","last_active":"2024-05-17T22:35:48.027593+0000","last_peered":"2024-05-17T22:35:48.027593+0000","last_clean":"2024-05-17T22:35:48.027593+0000","last_became_active":"2024-05-17T22:34:37.404970+0000","last_became_peered":"2024-05-17T22:34:37.404970+0000","last_unstale":"2024-05-17T22:35:48.027593+0000","last_undegraded":"2024-05-17T22:35:48.027593+0000","last_fullsized":"2024-05-17T22:35:48.027593+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:37:39.933686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.551829+0000","last_change":"2024-05-17T22:34:40.432690+0000","last_active":"2024-05-17T22:35:15.551829+0000","last_peered":"2024-05-17T22:35:15.551829+0000","last_clean":"2024-05-17T22:35:15.551829+0000","last_became_active":"2024-05-17T22:34:40.432274+0000","last_became_peered":"2024-05-17T22:34:40.432274+0000","last_unstale":"2024-05-17T22:35:15.551829+0000","last_undegraded":"2024-05-17T22:35:15.551829+0000","last_fullsized":"2024-05-17T22:35:15.551829+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:12:47.926865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.550457+0000","last_change":"2024-05-17T22:34:52.494645+0000","last_active":"2024-05-17T22:35:15.550457+0000","last_peered":"2024-05-17T22:35:15.550457+0000","last_clean":"2024-05-17T22:35:15.550457+0000","last_became_active":"2024-05-17T22:34:52.494255+0000","last_became_peered":"2024-05-17T22:34:52.494255+0000","last_unstale":"2024-05-17T22:35:15.550457+0000","last_undegraded":"2024-05-17T22:35:15.550457+0000","last_fullsized":"2024-05-17T22:35:15.550457+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:41:26.633571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:48.026285+0000","last_change":"2024-05-17T22:34:37.398582+0000","last_active":"2024-05-17T22:35:48.026285+0000","last_peered":"2024-05-17T22:35:48.026285+0000","last_clean":"2024-05-17T22:35:48.026285+0000","last_became_active":"2024-05-17T22:34:37.398156+0000","last_became_peered":"2024-05-17T22:34:37.398156+0000","last_unstale":"2024-05-17T22:35:48.026285+0000","last_undegraded":"2024-05-17T22:35:48.026285+0000","last_fullsized":"2024-05-17T22:35:48.026285+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:46:00.276214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.552024+0000","last_change":"2024-05-17T22:35:15.551402+0000","last_active":"2024-05-17T22:35:15.552024+0000","last_peered":"2024-05-17T22:35:15.552024+0000","last_clean":"2024-05-17T22:35:15.552024+0000","last_became_active":"2024-05-17T22:35:15.551021+0000","last_became_peered":"2024-05-17T22:35:15.551021+0000","last_unstale":"2024-05-17T22:35:15.552024+0000","last_undegraded":"2024-05-17T22:35:15.552024+0000","last_fullsized":"2024-05-17T22:35:15.552024+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:35:18.573620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.545564+0000","last_change":"2024-05-17T22:34:52.487413+0000","last_active":"2024-05-17T22:35:15.545564+0000","last_peered":"2024-05-17T22:35:15.545564+0000","last_clean":"2024-05-17T22:35:15.545564+0000","last_became_active":"2024-05-17T22:34:52.487032+0000","last_became_peered":"2024-05-17T22:34:52.487032+0000","last_unstale":"2024-05-17T22:35:15.545564+0000","last_undegraded":"2024-05-17T22:35:15.545564+0000","last_fullsized":"2024-05-17T22:35:15.545564+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:34:49.122008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.554440+0000","last_change":"2024-05-17T22:34:37.404894+0000","last_active":"2024-05-17T22:35:15.554440+0000","last_peered":"2024-05-17T22:35:15.554440+0000","last_clean":"2024-05-17T22:35:15.554440+0000","last_became_active":"2024-05-17T22:34:37.404567+0000","last_became_peered":"2024-05-17T22:34:37.404567+0000","last_unstale":"2024-05-17T22:35:15.554440+0000","last_undegraded":"2024-05-17T22:35:15.554440+0000","last_fullsized":"2024-05-17T22:35:15.554440+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:13:09.761556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.547819+0000","last_change":"2024-05-17T22:34:40.418267+0000","last_active":"2024-05-17T22:35:15.547819+0000","last_peered":"2024-05-17T22:35:15.547819+0000","last_clean":"2024-05-17T22:35:15.547819+0000","last_became_active":"2024-05-17T22:34:40.417928+0000","last_became_peered":"2024-05-17T22:34:40.417928+0000","last_unstale":"2024-05-17T22:35:15.547819+0000","last_undegraded":"2024-05-17T22:35:15.547819+0000","last_fullsized":"2024-05-17T22:35:15.547819+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:17:37.055004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.710253+0000","last_change":"2024-05-17T22:34:52.492126+0000","last_active":"2024-05-17T22:35:15.710253+0000","last_peered":"2024-05-17T22:35:15.710253+0000","last_clean":"2024-05-17T22:35:15.710253+0000","last_became_active":"2024-05-17T22:34:52.491695+0000","last_became_peered":"2024-05-17T22:34:52.491695+0000","last_unstale":"2024-05-17T22:35:15.710253+0000","last_undegraded":"2024-05-17T22:35:15.710253+0000","last_fullsized":"2024-05-17T22:35:15.710253+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:15:17.710279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":68,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:36:13.078370+0000","last_change":"2024-05-17T22:34:37.406989+0000","last_active":"2024-05-17T22:36:13.078370+0000","last_peered":"2024-05-17T22:36:13.078370+0000","last_clean":"2024-05-17T22:36:13.078370+0000","last_became_active":"2024-05-17T22:34:37.406612+0000","last_became_peered":"2024-05-17T22:34:37.406612+0000","last_unstale":"2024-05-17T22:36:13.078370+0000","last_undegraded":"2024-05-17T22:36:13.078370+0000","last_fullsized":"2024-05-17T22:36:13.078370+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:47:52.480804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.835242+0000","last_change":"2024-05-17T22:34:40.427221+0000","last_active":"2024-05-17T22:35:15.835242+0000","last_peered":"2024-05-17T22:35:15.835242+0000","last_clean":"2024-05-17T22:35:15.835242+0000","last_became_active":"2024-05-17T22:34:40.426703+0000","last_became_peered":"2024-05-17T22:34:40.426703+0000","last_unstale":"2024-05-17T22:35:15.835242+0000","last_undegraded":"2024-05-17T22:35:15.835242+0000","last_fullsized":"2024-05-17T22:35:15.835242+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:49:15.629048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.547080+0000","last_change":"2024-05-17T22:34:52.499086+0000","last_active":"2024-05-17T22:35:15.547080+0000","last_peered":"2024-05-17T22:35:15.547080+0000","last_clean":"2024-05-17T22:35:15.547080+0000","last_became_active":"2024-05-17T22:34:52.498666+0000","last_became_peered":"2024-05-17T22:34:52.498666+0000","last_unstale":"2024-05-17T22:35:15.547080+0000","last_undegraded":"2024-05-17T22:35:15.547080+0000","last_fullsized":"2024-05-17T22:35:15.547080+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:46:25.882964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":48,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:47.019833+0000","last_change":"2024-05-17T22:35:15.836294+0000","last_active":"2024-05-17T22:35:47.019833+0000","last_peered":"2024-05-17T22:35:47.019833+0000","last_clean":"2024-05-17T22:35:47.019833+0000","last_became_active":"2024-05-17T22:35:15.835759+0000","last_became_peered":"2024-05-17T22:35:15.835759+0000","last_unstale":"2024-05-17T22:35:47.019833+0000","last_undegraded":"2024-05-17T22:35:47.019833+0000","last_fullsized":"2024-05-17T22:35:47.019833+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:23:32.106692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.545615+0000","last_change":"2024-05-17T22:34:40.433331+0000","last_active":"2024-05-17T22:35:15.545615+0000","last_peered":"2024-05-17T22:35:15.545615+0000","last_clean":"2024-05-17T22:35:15.545615+0000","last_became_active":"2024-05-17T22:34:40.432989+0000","last_became_peered":"2024-05-17T22:34:40.432989+0000","last_unstale":"2024-05-17T22:35:15.545615+0000","last_undegraded":"2024-05-17T22:35:15.545615+0000","last_fullsized":"2024-05-17T22:35:15.545615+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:22:49.146148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.546406+0000","last_change":"2024-05-17T22:34:52.494418+0000","last_active":"2024-05-17T22:35:15.546406+0000","last_peered":"2024-05-17T22:35:15.546406+0000","last_clean":"2024-05-17T22:35:15.546406+0000","last_became_active":"2024-05-17T22:34:52.493953+0000","last_became_peered":"2024-05-17T22:34:52.493953+0000","last_unstale":"2024-05-17T22:35:15.546406+0000","last_undegraded":"2024-05-17T22:35:15.546406+0000","last_fullsized":"2024-05-17T22:35:15.546406+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:50:40.211770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:47.018796+0000","last_change":"2024-05-17T22:34:37.442986+0000","last_active":"2024-05-17T22:35:47.018796+0000","last_peered":"2024-05-17T22:35:47.018796+0000","last_clean":"2024-05-17T22:35:47.018796+0000","last_became_active":"2024-05-17T22:34:37.442399+0000","last_became_peered":"2024-05-17T22:34:37.442399+0000","last_unstale":"2024-05-17T22:35:47.018796+0000","last_undegraded":"2024-05-17T22:35:47.018796+0000","last_fullsized":"2024-05-17T22:35:47.018796+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:34:05.785811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.834687+0000","last_change":"2024-05-17T22:34:40.426793+0000","last_active":"2024-05-17T22:35:15.834687+0000","last_peered":"2024-05-17T22:35:15.834687+0000","last_clean":"2024-05-17T22:35:15.834687+0000","last_became_active":"2024-05-17T22:34:40.426420+0000","last_became_peered":"2024-05-17T22:34:40.426420+0000","last_unstale":"2024-05-17T22:35:15.834687+0000","last_undegraded":"2024-05-17T22:35:15.834687+0000","last_fullsized":"2024-05-17T22:35:15.834687+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:57:07.099254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.547288+0000","last_change":"2024-05-17T22:34:52.498762+0000","last_active":"2024-05-17T22:35:15.547288+0000","last_peered":"2024-05-17T22:35:15.547288+0000","last_clean":"2024-05-17T22:35:15.547288+0000","last_became_active":"2024-05-17T22:34:52.498306+0000","last_became_peered":"2024-05-17T22:34:52.498306+0000","last_unstale":"2024-05-17T22:35:15.547288+0000","last_undegraded":"2024-05-17T22:35:15.547288+0000","last_fullsized":"2024-05-17T22:35:15.547288+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:18:32.717563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'32","reported_seq":130,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:36:14.080602+0000","last_change":"2024-05-17T22:34:37.407885+0000","last_active":"2024-05-17T22:36:14.080602+0000","last_peered":"2024-05-17T22:36:14.080602+0000","last_clean":"2024-05-17T22:36:14.080602+0000","last_became_active":"2024-05-17T22:34:37.407463+0000","last_became_peered":"2024-05-17T22:34:37.407463+0000","last_unstale":"2024-05-17T22:36:14.080602+0000","last_undegraded":"2024-05-17T22:36:14.080602+0000","last_fullsized":"2024-05-17T22:36:14.080602+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:26:20.288557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57807,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":58,"num_read_kb":85,"num_write":33,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T22:36:14.624480+0000","last_change":"2024-05-17T22:34:40.435800+0000","last_active":"2024-05-17T22:36:14.624480+0000","last_peered":"2024-05-17T22:36:14.624480+0000","last_clean":"2024-05-17T22:36:14.624480+0000","last_became_active":"2024-05-17T22:34:40.435245+0000","last_became_peered":"2024-05-17T22:34:40.435245+0000","last_unstale":"2024-05-17T22:36:14.624480+0000","last_undegraded":"2024-05-17T22:36:14.624480+0000","last_fullsized":"2024-05-17T22:36:14.624480+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:09:58.699026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.552687+0000","last_change":"2024-05-17T22:34:52.487336+0000","last_active":"2024-05-17T22:35:15.552687+0000","last_peered":"2024-05-17T22:35:15.552687+0000","last_clean":"2024-05-17T22:35:15.552687+0000","last_became_active":"2024-05-17T22:34:52.486883+0000","last_became_peered":"2024-05-17T22:34:52.486883+0000","last_unstale":"2024-05-17T22:35:15.552687+0000","last_undegraded":"2024-05-17T22:35:15.552687+0000","last_fullsized":"2024-05-17T22:35:15.552687+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:44:41.264072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:48.026502+0000","last_change":"2024-05-17T22:34:37.405044+0000","last_active":"2024-05-17T22:35:48.026502+0000","last_peered":"2024-05-17T22:35:48.026502+0000","last_clean":"2024-05-17T22:35:48.026502+0000","last_became_active":"2024-05-17T22:34:37.404610+0000","last_became_peered":"2024-05-17T22:34:37.404610+0000","last_unstale":"2024-05-17T22:35:48.026502+0000","last_undegraded":"2024-05-17T22:35:48.026502+0000","last_fullsized":"2024-05-17T22:35:48.026502+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:20:37.939069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.548602+0000","last_change":"2024-05-17T22:34:40.430655+0000","last_active":"2024-05-17T22:35:15.548602+0000","last_peered":"2024-05-17T22:35:15.548602+0000","last_clean":"2024-05-17T22:35:15.548602+0000","last_became_active":"2024-05-17T22:34:40.430307+0000","last_became_peered":"2024-05-17T22:34:40.430307+0000","last_unstale":"2024-05-17T22:35:15.548602+0000","last_undegraded":"2024-05-17T22:35:15.548602+0000","last_fullsized":"2024-05-17T22:35:15.548602+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:57:19.036117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.549515+0000","last_change":"2024-05-17T22:34:52.494709+0000","last_active":"2024-05-17T22:35:15.549515+0000","last_peered":"2024-05-17T22:35:15.549515+0000","last_clean":"2024-05-17T22:35:15.549515+0000","last_became_active":"2024-05-17T22:34:52.494195+0000","last_became_peered":"2024-05-17T22:34:52.494195+0000","last_unstale":"2024-05-17T22:35:15.549515+0000","last_undegraded":"2024-05-17T22:35:15.549515+0000","last_fullsized":"2024-05-17T22:35:15.549515+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:35:41.188459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:47.020572+0000","last_change":"2024-05-17T22:34:37.402494+0000","last_active":"2024-05-17T22:35:47.020572+0000","last_peered":"2024-05-17T22:35:47.020572+0000","last_clean":"2024-05-17T22:35:47.020572+0000","last_became_active":"2024-05-17T22:34:37.402102+0000","last_became_peered":"2024-05-17T22:34:37.402102+0000","last_unstale":"2024-05-17T22:35:47.020572+0000","last_undegraded":"2024-05-17T22:35:47.020572+0000","last_fullsized":"2024-05-17T22:35:47.020572+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:22:30.915780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.551307+0000","last_change":"2024-05-17T22:34:40.440731+0000","last_active":"2024-05-17T22:35:15.551307+0000","last_peered":"2024-05-17T22:35:15.551307+0000","last_clean":"2024-05-17T22:35:15.551307+0000","last_became_active":"2024-05-17T22:34:40.440374+0000","last_became_peered":"2024-05-17T22:34:40.440374+0000","last_unstale":"2024-05-17T22:35:15.551307+0000","last_undegraded":"2024-05-17T22:35:15.551307+0000","last_fullsized":"2024-05-17T22:35:15.551307+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:26:40.029155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":2,"reported_epoch":88,"state":"peering","last_fresh":"2024-05-17T22:34:52.479189+0000","last_change":"2024-05-17T22:36:15.566584+0000","last_active":"2024-05-17T22:34:51.460688+0000","last_peered":"2024-05-17T22:34:51.460688+0000","last_clean":"2024-05-17T22:34:51.460688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T22:34:52.479189+0000","last_undegraded":"2024-05-17T22:34:52.479189+0000","last_fullsized":"2024-05-17T22:34:52.479189+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"87'4","reported_seq":69,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:36:11.076367+0000","last_change":"2024-05-17T22:34:37.403123+0000","last_active":"2024-05-17T22:36:11.076367+0000","last_peered":"2024-05-17T22:36:11.076367+0000","last_clean":"2024-05-17T22:36:11.076367+0000","last_became_active":"2024-05-17T22:34:37.402702+0000","last_became_peered":"2024-05-17T22:34:37.402702+0000","last_unstale":"2024-05-17T22:36:11.076367+0000","last_undegraded":"2024-05-17T22:36:11.076367+0000","last_fullsized":"2024-05-17T22:36:11.076367+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:50:02.031651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.550950+0000","last_change":"2024-05-17T22:34:40.431895+0000","last_active":"2024-05-17T22:35:15.550950+0000","last_peered":"2024-05-17T22:35:15.550950+0000","last_clean":"2024-05-17T22:35:15.550950+0000","last_became_active":"2024-05-17T22:34:40.431527+0000","last_became_peered":"2024-05-17T22:34:40.431527+0000","last_unstale":"2024-05-17T22:35:15.550950+0000","last_undegraded":"2024-05-17T22:35:15.550950+0000","last_fullsized":"2024-05-17T22:35:15.550950+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:30:09.404255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.550050+0000","last_change":"2024-05-17T22:34:52.495583+0000","last_active":"2024-05-17T22:35:15.550050+0000","last_peered":"2024-05-17T22:35:15.550050+0000","last_clean":"2024-05-17T22:35:15.550050+0000","last_became_active":"2024-05-17T22:34:52.495030+0000","last_became_peered":"2024-05-17T22:34:52.495030+0000","last_unstale":"2024-05-17T22:35:15.550050+0000","last_undegraded":"2024-05-17T22:35:15.550050+0000","last_fullsized":"2024-05-17T22:35:15.550050+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:04:54.046891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":58,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T22:36:14.623833+0000","last_change":"2024-05-17T22:34:37.442154+0000","last_active":"2024-05-17T22:36:14.623833+0000","last_peered":"2024-05-17T22:36:14.623833+0000","last_clean":"2024-05-17T22:36:14.623833+0000","last_became_active":"2024-05-17T22:34:37.441691+0000","last_became_peered":"2024-05-17T22:34:37.441691+0000","last_unstale":"2024-05-17T22:36:14.623833+0000","last_undegraded":"2024-05-17T22:36:14.623833+0000","last_fullsized":"2024-05-17T22:36:14.623833+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:47:29.862031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.554183+0000","last_change":"2024-05-17T22:34:40.430894+0000","last_active":"2024-05-17T22:35:15.554183+0000","last_peered":"2024-05-17T22:35:15.554183+0000","last_clean":"2024-05-17T22:35:15.554183+0000","last_became_active":"2024-05-17T22:34:40.430328+0000","last_became_peered":"2024-05-17T22:34:40.430328+0000","last_unstale":"2024-05-17T22:35:15.554183+0000","last_undegraded":"2024-05-17T22:35:15.554183+0000","last_fullsized":"2024-05-17T22:35:15.554183+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:36:46.619111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.554056+0000","last_change":"2024-05-17T22:34:52.487555+0000","last_active":"2024-05-17T22:35:15.554056+0000","last_peered":"2024-05-17T22:35:15.554056+0000","last_clean":"2024-05-17T22:35:15.554056+0000","last_became_active":"2024-05-17T22:34:52.487076+0000","last_became_peered":"2024-05-17T22:34:52.487076+0000","last_unstale":"2024-05-17T22:35:15.554056+0000","last_undegraded":"2024-05-17T22:35:15.554056+0000","last_fullsized":"2024-05-17T22:35:15.554056+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:05:46.720665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:49.022880+0000","last_change":"2024-05-17T22:35:15.558860+0000","last_active":"2024-05-17T22:35:49.022880+0000","last_peered":"2024-05-17T22:35:49.022880+0000","last_clean":"2024-05-17T22:35:49.022880+0000","last_became_active":"2024-05-17T22:35:15.553683+0000","last_became_peered":"2024-05-17T22:35:15.553683+0000","last_unstale":"2024-05-17T22:35:49.022880+0000","last_undegraded":"2024-05-17T22:35:49.022880+0000","last_fullsized":"2024-05-17T22:35:49.022880+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:27:24.978282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.549938+0000","last_change":"2024-05-17T22:34:40.433499+0000","last_active":"2024-05-17T22:35:15.549938+0000","last_peered":"2024-05-17T22:35:15.549938+0000","last_clean":"2024-05-17T22:35:15.549938+0000","last_became_active":"2024-05-17T22:34:40.431370+0000","last_became_peered":"2024-05-17T22:34:40.431370+0000","last_unstale":"2024-05-17T22:35:15.549938+0000","last_undegraded":"2024-05-17T22:35:15.549938+0000","last_fullsized":"2024-05-17T22:35:15.549938+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:25:08.383021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T22:36:14.624198+0000","last_change":"2024-05-17T22:34:52.489627+0000","last_active":"2024-05-17T22:36:14.624198+0000","last_peered":"2024-05-17T22:36:14.624198+0000","last_clean":"2024-05-17T22:36:14.624198+0000","last_became_active":"2024-05-17T22:34:52.489173+0000","last_became_peered":"2024-05-17T22:34:52.489173+0000","last_unstale":"2024-05-17T22:36:14.624198+0000","last_undegraded":"2024-05-17T22:36:14.624198+0000","last_fullsized":"2024-05-17T22:36:14.624198+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:52:26.908817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.553325+0000","last_change":"2024-05-17T22:35:15.553325+0000","last_active":"2024-05-17T22:35:15.553325+0000","last_peered":"2024-05-17T22:35:15.553325+0000","last_clean":"2024-05-17T22:35:15.553325+0000","last_became_active":"2024-05-17T22:35:15.552867+0000","last_became_peered":"2024-05-17T22:35:15.552867+0000","last_unstale":"2024-05-17T22:35:15.553325+0000","last_undegraded":"2024-05-17T22:35:15.553325+0000","last_fullsized":"2024-05-17T22:35:15.553325+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:28:36.618331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.549029+0000","last_change":"2024-05-17T22:34:40.423221+0000","last_active":"2024-05-17T22:35:15.549029+0000","last_peered":"2024-05-17T22:35:15.549029+0000","last_clean":"2024-05-17T22:35:15.549029+0000","last_became_active":"2024-05-17T22:34:40.422838+0000","last_became_peered":"2024-05-17T22:34:40.422838+0000","last_unstale":"2024-05-17T22:35:15.549029+0000","last_undegraded":"2024-05-17T22:35:15.549029+0000","last_fullsized":"2024-05-17T22:35:15.549029+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:39:51.873212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.549874+0000","last_change":"2024-05-17T22:34:52.484802+0000","last_active":"2024-05-17T22:35:15.549874+0000","last_peered":"2024-05-17T22:35:15.549874+0000","last_clean":"2024-05-17T22:35:15.549874+0000","last_became_active":"2024-05-17T22:34:52.483766+0000","last_became_peered":"2024-05-17T22:34:52.483766+0000","last_unstale":"2024-05-17T22:35:15.549874+0000","last_undegraded":"2024-05-17T22:35:15.549874+0000","last_fullsized":"2024-05-17T22:35:15.549874+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:32:01.427078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.551509+0000","last_change":"2024-05-17T22:34:37.404797+0000","last_active":"2024-05-17T22:35:15.551509+0000","last_peered":"2024-05-17T22:35:15.551509+0000","last_clean":"2024-05-17T22:35:15.551509+0000","last_became_active":"2024-05-17T22:34:37.404393+0000","last_became_peered":"2024-05-17T22:34:37.404393+0000","last_unstale":"2024-05-17T22:35:15.551509+0000","last_undegraded":"2024-05-17T22:35:15.551509+0000","last_fullsized":"2024-05-17T22:35:15.551509+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:26:36.887919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.548770+0000","last_change":"2024-05-17T22:34:40.428210+0000","last_active":"2024-05-17T22:35:15.548770+0000","last_peered":"2024-05-17T22:35:15.548770+0000","last_clean":"2024-05-17T22:35:15.548770+0000","last_became_active":"2024-05-17T22:34:40.427845+0000","last_became_peered":"2024-05-17T22:34:40.427845+0000","last_unstale":"2024-05-17T22:35:15.548770+0000","last_undegraded":"2024-05-17T22:35:15.548770+0000","last_fullsized":"2024-05-17T22:35:15.548770+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:45:30.437139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.549883+0000","last_change":"2024-05-17T22:34:52.490502+0000","last_active":"2024-05-17T22:35:15.549883+0000","last_peered":"2024-05-17T22:35:15.549883+0000","last_clean":"2024-05-17T22:35:15.549883+0000","last_became_active":"2024-05-17T22:34:52.490008+0000","last_became_peered":"2024-05-17T22:34:52.490008+0000","last_unstale":"2024-05-17T22:35:15.549883+0000","last_undegraded":"2024-05-17T22:35:15.549883+0000","last_fullsized":"2024-05-17T22:35:15.549883+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:58:29.122668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.554667+0000","last_change":"2024-05-17T22:34:40.432248+0000","last_active":"2024-05-17T22:35:15.554667+0000","last_peered":"2024-05-17T22:35:15.554667+0000","last_clean":"2024-05-17T22:35:15.554667+0000","last_became_active":"2024-05-17T22:34:40.431863+0000","last_became_peered":"2024-05-17T22:34:40.431863+0000","last_unstale":"2024-05-17T22:35:15.554667+0000","last_undegraded":"2024-05-17T22:35:15.554667+0000","last_fullsized":"2024-05-17T22:35:15.554667+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:20:34.730542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'4","reported_seq":68,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:36:12.076964+0000","last_change":"2024-05-17T22:34:37.441186+0000","last_active":"2024-05-17T22:36:12.076964+0000","last_peered":"2024-05-17T22:36:12.076964+0000","last_clean":"2024-05-17T22:36:12.076964+0000","last_became_active":"2024-05-17T22:34:37.440781+0000","last_became_peered":"2024-05-17T22:34:37.440781+0000","last_unstale":"2024-05-17T22:36:12.076964+0000","last_undegraded":"2024-05-17T22:36:12.076964+0000","last_fullsized":"2024-05-17T22:36:12.076964+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:44:34.435147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.551193+0000","last_change":"2024-05-17T22:34:52.491771+0000","last_active":"2024-05-17T22:35:15.551193+0000","last_peered":"2024-05-17T22:35:15.551193+0000","last_clean":"2024-05-17T22:35:15.551193+0000","last_became_active":"2024-05-17T22:34:52.491172+0000","last_became_peered":"2024-05-17T22:34:52.491172+0000","last_unstale":"2024-05-17T22:35:15.551193+0000","last_undegraded":"2024-05-17T22:35:15.551193+0000","last_fullsized":"2024-05-17T22:35:15.551193+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:15:21.010592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.710262+0000","last_change":"2024-05-17T22:34:40.430860+0000","last_active":"2024-05-17T22:35:15.710262+0000","last_peered":"2024-05-17T22:35:15.710262+0000","last_clean":"2024-05-17T22:35:15.710262+0000","last_became_active":"2024-05-17T22:34:40.430542+0000","last_became_peered":"2024-05-17T22:34:40.430542+0000","last_unstale":"2024-05-17T22:35:15.710262+0000","last_undegraded":"2024-05-17T22:35:15.710262+0000","last_fullsized":"2024-05-17T22:35:15.710262+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:16:26.257380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:50.039616+0000","last_change":"2024-05-17T22:34:37.403026+0000","last_active":"2024-05-17T22:35:50.039616+0000","last_peered":"2024-05-17T22:35:50.039616+0000","last_clean":"2024-05-17T22:35:50.039616+0000","last_became_active":"2024-05-17T22:34:37.402560+0000","last_became_peered":"2024-05-17T22:34:37.402560+0000","last_unstale":"2024-05-17T22:35:50.039616+0000","last_undegraded":"2024-05-17T22:35:50.039616+0000","last_fullsized":"2024-05-17T22:35:50.039616+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:23:03.784411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":115,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.550794+0000","last_change":"2024-05-17T22:33:11.055878+0000","last_active":"2024-05-17T22:35:15.550794+0000","last_peered":"2024-05-17T22:35:15.550794+0000","last_clean":"2024-05-17T22:35:15.550794+0000","last_became_active":"2024-05-17T22:33:11.050409+0000","last_became_peered":"2024-05-17T22:33:11.050409+0000","last_unstale":"2024-05-17T22:35:15.550794+0000","last_undegraded":"2024-05-17T22:35:15.550794+0000","last_fullsized":"2024-05-17T22:35:15.550794+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:29:43.341170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:29:43.341170+0000","last_clean_scrub_stamp":"2024-05-17T22:29:43.341170+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:30:54.153865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.552081+0000","last_change":"2024-05-17T22:34:52.497064+0000","last_active":"2024-05-17T22:35:15.552081+0000","last_peered":"2024-05-17T22:35:15.552081+0000","last_clean":"2024-05-17T22:35:15.552081+0000","last_became_active":"2024-05-17T22:34:52.496611+0000","last_became_peered":"2024-05-17T22:34:52.496611+0000","last_unstale":"2024-05-17T22:35:15.552081+0000","last_undegraded":"2024-05-17T22:35:15.552081+0000","last_fullsized":"2024-05-17T22:35:15.552081+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:20:29.597589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.555413+0000","last_change":"2024-05-17T22:35:15.555413+0000","last_active":"2024-05-17T22:35:15.555413+0000","last_peered":"2024-05-17T22:35:15.555413+0000","last_clean":"2024-05-17T22:35:15.555413+0000","last_became_active":"2024-05-17T22:35:15.555005+0000","last_became_peered":"2024-05-17T22:35:15.555005+0000","last_unstale":"2024-05-17T22:35:15.555413+0000","last_undegraded":"2024-05-17T22:35:15.555413+0000","last_fullsized":"2024-05-17T22:35:15.555413+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:51:49.860673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:48.026226+0000","last_change":"2024-05-17T22:34:37.406546+0000","last_active":"2024-05-17T22:35:48.026226+0000","last_peered":"2024-05-17T22:35:48.026226+0000","last_clean":"2024-05-17T22:35:48.026226+0000","last_became_active":"2024-05-17T22:34:37.406118+0000","last_became_peered":"2024-05-17T22:34:37.406118+0000","last_unstale":"2024-05-17T22:35:48.026226+0000","last_undegraded":"2024-05-17T22:35:48.026226+0000","last_fullsized":"2024-05-17T22:35:48.026226+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:47:16.350091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.549358+0000","last_change":"2024-05-17T22:34:52.488739+0000","last_active":"2024-05-17T22:35:15.549358+0000","last_peered":"2024-05-17T22:35:15.549358+0000","last_clean":"2024-05-17T22:35:15.549358+0000","last_became_active":"2024-05-17T22:34:52.488220+0000","last_became_peered":"2024-05-17T22:34:52.488220+0000","last_unstale":"2024-05-17T22:35:15.549358+0000","last_undegraded":"2024-05-17T22:35:15.549358+0000","last_fullsized":"2024-05-17T22:35:15.549358+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:03:44.290361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T22:36:14.624604+0000","last_change":"2024-05-17T22:34:37.442676+0000","last_active":"2024-05-17T22:36:14.624604+0000","last_peered":"2024-05-17T22:36:14.624604+0000","last_clean":"2024-05-17T22:36:14.624604+0000","last_became_active":"2024-05-17T22:34:37.442275+0000","last_became_peered":"2024-05-17T22:34:37.442275+0000","last_unstale":"2024-05-17T22:36:14.624604+0000","last_undegraded":"2024-05-17T22:36:14.624604+0000","last_fullsized":"2024-05-17T22:36:14.624604+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:54:18.542987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.549807+0000","last_change":"2024-05-17T22:34:40.424266+0000","last_active":"2024-05-17T22:35:15.549807+0000","last_peered":"2024-05-17T22:35:15.549807+0000","last_clean":"2024-05-17T22:35:15.549807+0000","last_became_active":"2024-05-17T22:34:40.423905+0000","last_became_peered":"2024-05-17T22:34:40.423905+0000","last_unstale":"2024-05-17T22:35:15.549807+0000","last_undegraded":"2024-05-17T22:35:15.549807+0000","last_fullsized":"2024-05-17T22:35:15.549807+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:03:43.397871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.710561+0000","last_change":"2024-05-17T22:34:52.493386+0000","last_active":"2024-05-17T22:35:15.710561+0000","last_peered":"2024-05-17T22:35:15.710561+0000","last_clean":"2024-05-17T22:35:15.710561+0000","last_became_active":"2024-05-17T22:34:52.492997+0000","last_became_peered":"2024-05-17T22:34:52.492997+0000","last_unstale":"2024-05-17T22:35:15.710561+0000","last_undegraded":"2024-05-17T22:35:15.710561+0000","last_fullsized":"2024-05-17T22:35:15.710561+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:14:40.818271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'10","reported_seq":50,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:36:09.443991+0000","last_change":"2024-05-17T22:34:37.399397+0000","last_active":"2024-05-17T22:36:09.443991+0000","last_peered":"2024-05-17T22:36:09.443991+0000","last_clean":"2024-05-17T22:36:09.443991+0000","last_became_active":"2024-05-17T22:34:37.399062+0000","last_became_peered":"2024-05-17T22:34:37.399062+0000","last_unstale":"2024-05-17T22:36:09.443991+0000","last_undegraded":"2024-05-17T22:36:09.443991+0000","last_fullsized":"2024-05-17T22:36:09.443991+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:02:05.230580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.710693+0000","last_change":"2024-05-17T22:34:40.426995+0000","last_active":"2024-05-17T22:35:15.710693+0000","last_peered":"2024-05-17T22:35:15.710693+0000","last_clean":"2024-05-17T22:35:15.710693+0000","last_became_active":"2024-05-17T22:34:40.426226+0000","last_became_peered":"2024-05-17T22:34:40.426226+0000","last_unstale":"2024-05-17T22:35:15.710693+0000","last_undegraded":"2024-05-17T22:35:15.710693+0000","last_fullsized":"2024-05-17T22:35:15.710693+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:49:53.331076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.550930+0000","last_change":"2024-05-17T22:34:52.487098+0000","last_active":"2024-05-17T22:35:15.550930+0000","last_peered":"2024-05-17T22:35:15.550930+0000","last_clean":"2024-05-17T22:35:15.550930+0000","last_became_active":"2024-05-17T22:34:52.486605+0000","last_became_peered":"2024-05-17T22:34:52.486605+0000","last_unstale":"2024-05-17T22:35:15.550930+0000","last_undegraded":"2024-05-17T22:35:15.550930+0000","last_fullsized":"2024-05-17T22:35:15.550930+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:33:28.234208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:47.019076+0000","last_change":"2024-05-17T22:34:37.800830+0000","last_active":"2024-05-17T22:35:47.019076+0000","last_peered":"2024-05-17T22:35:47.019076+0000","last_clean":"2024-05-17T22:35:47.019076+0000","last_became_active":"2024-05-17T22:34:37.800405+0000","last_became_peered":"2024-05-17T22:34:37.800405+0000","last_unstale":"2024-05-17T22:35:47.019076+0000","last_undegraded":"2024-05-17T22:35:47.019076+0000","last_fullsized":"2024-05-17T22:35:47.019076+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:19:10.087442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.545676+0000","last_change":"2024-05-17T22:34:40.426307+0000","last_active":"2024-05-17T22:35:15.545676+0000","last_peered":"2024-05-17T22:35:15.545676+0000","last_clean":"2024-05-17T22:35:15.545676+0000","last_became_active":"2024-05-17T22:34:40.425969+0000","last_became_peered":"2024-05-17T22:34:40.425969+0000","last_unstale":"2024-05-17T22:35:15.545676+0000","last_undegraded":"2024-05-17T22:35:15.545676+0000","last_fullsized":"2024-05-17T22:35:15.545676+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:25:49.680886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.556857+0000","last_change":"2024-05-17T22:35:15.556857+0000","last_active":"2024-05-17T22:35:15.556857+0000","last_peered":"2024-05-17T22:35:15.556857+0000","last_clean":"2024-05-17T22:35:15.556857+0000","last_became_active":"2024-05-17T22:35:15.556298+0000","last_became_peered":"2024-05-17T22:35:15.556298+0000","last_unstale":"2024-05-17T22:35:15.556857+0000","last_undegraded":"2024-05-17T22:35:15.556857+0000","last_fullsized":"2024-05-17T22:35:15.556857+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:23:20.111544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:47.021217+0000","last_change":"2024-05-17T22:34:37.442971+0000","last_active":"2024-05-17T22:35:47.021217+0000","last_peered":"2024-05-17T22:35:47.021217+0000","last_clean":"2024-05-17T22:35:47.021217+0000","last_became_active":"2024-05-17T22:34:37.442536+0000","last_became_peered":"2024-05-17T22:34:37.442536+0000","last_unstale":"2024-05-17T22:35:47.021217+0000","last_undegraded":"2024-05-17T22:35:47.021217+0000","last_fullsized":"2024-05-17T22:35:47.021217+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:03:45.224071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T22:36:14.623867+0000","last_change":"2024-05-17T22:34:40.432056+0000","last_active":"2024-05-17T22:36:14.623867+0000","last_peered":"2024-05-17T22:36:14.623867+0000","last_clean":"2024-05-17T22:36:14.623867+0000","last_became_active":"2024-05-17T22:34:40.431626+0000","last_became_peered":"2024-05-17T22:34:40.431626+0000","last_unstale":"2024-05-17T22:36:14.623867+0000","last_undegraded":"2024-05-17T22:36:14.623867+0000","last_fullsized":"2024-05-17T22:36:14.623867+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:12:24.650746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.548436+0000","last_change":"2024-05-17T22:34:52.500451+0000","last_active":"2024-05-17T22:35:15.548436+0000","last_peered":"2024-05-17T22:35:15.548436+0000","last_clean":"2024-05-17T22:35:15.548436+0000","last_became_active":"2024-05-17T22:34:52.500031+0000","last_became_peered":"2024-05-17T22:34:52.500031+0000","last_unstale":"2024-05-17T22:35:15.548436+0000","last_undegraded":"2024-05-17T22:35:15.548436+0000","last_fullsized":"2024-05-17T22:35:15.548436+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:54:36.825116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:47.019097+0000","last_change":"2024-05-17T22:34:37.402335+0000","last_active":"2024-05-17T22:35:47.019097+0000","last_peered":"2024-05-17T22:35:47.019097+0000","last_clean":"2024-05-17T22:35:47.019097+0000","last_became_active":"2024-05-17T22:34:37.401982+0000","last_became_peered":"2024-05-17T22:34:37.401982+0000","last_unstale":"2024-05-17T22:35:47.019097+0000","last_undegraded":"2024-05-17T22:35:47.019097+0000","last_fullsized":"2024-05-17T22:35:47.019097+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:58:06.194335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.549177+0000","last_change":"2024-05-17T22:34:40.425524+0000","last_active":"2024-05-17T22:35:15.549177+0000","last_peered":"2024-05-17T22:35:15.549177+0000","last_clean":"2024-05-17T22:35:15.549177+0000","last_became_active":"2024-05-17T22:34:40.425135+0000","last_became_peered":"2024-05-17T22:34:40.425135+0000","last_unstale":"2024-05-17T22:35:15.549177+0000","last_undegraded":"2024-05-17T22:35:15.549177+0000","last_fullsized":"2024-05-17T22:35:15.549177+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:41:14.176477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.546559+0000","last_change":"2024-05-17T22:34:52.496290+0000","last_active":"2024-05-17T22:35:15.546559+0000","last_peered":"2024-05-17T22:35:15.546559+0000","last_clean":"2024-05-17T22:35:15.546559+0000","last_became_active":"2024-05-17T22:34:52.495817+0000","last_became_peered":"2024-05-17T22:34:52.495817+0000","last_unstale":"2024-05-17T22:35:15.546559+0000","last_undegraded":"2024-05-17T22:35:15.546559+0000","last_fullsized":"2024-05-17T22:35:15.546559+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:47:22.056372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'1","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.552448+0000","last_change":"2024-05-17T22:34:37.404234+0000","last_active":"2024-05-17T22:35:15.552448+0000","last_peered":"2024-05-17T22:35:15.552448+0000","last_clean":"2024-05-17T22:35:15.552448+0000","last_became_active":"2024-05-17T22:34:37.403561+0000","last_became_peered":"2024-05-17T22:34:37.403561+0000","last_unstale":"2024-05-17T22:35:15.552448+0000","last_undegraded":"2024-05-17T22:35:15.552448+0000","last_fullsized":"2024-05-17T22:35:15.552448+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:30:28.432641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.710286+0000","last_change":"2024-05-17T22:34:40.426522+0000","last_active":"2024-05-17T22:35:15.710286+0000","last_peered":"2024-05-17T22:35:15.710286+0000","last_clean":"2024-05-17T22:35:15.710286+0000","last_became_active":"2024-05-17T22:34:40.426139+0000","last_became_peered":"2024-05-17T22:34:40.426139+0000","last_unstale":"2024-05-17T22:35:15.710286+0000","last_undegraded":"2024-05-17T22:35:15.710286+0000","last_fullsized":"2024-05-17T22:35:15.710286+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:30:19.503513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.546382+0000","last_change":"2024-05-17T22:34:52.487131+0000","last_active":"2024-05-17T22:35:15.546382+0000","last_peered":"2024-05-17T22:35:15.546382+0000","last_clean":"2024-05-17T22:35:15.546382+0000","last_became_active":"2024-05-17T22:34:52.486654+0000","last_became_peered":"2024-05-17T22:34:52.486654+0000","last_unstale":"2024-05-17T22:35:15.546382+0000","last_undegraded":"2024-05-17T22:35:15.546382+0000","last_fullsized":"2024-05-17T22:35:15.546382+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:28:25.041746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T22:36:14.624452+0000","last_change":"2024-05-17T22:34:37.405020+0000","last_active":"2024-05-17T22:36:14.624452+0000","last_peered":"2024-05-17T22:36:14.624452+0000","last_clean":"2024-05-17T22:36:14.624452+0000","last_became_active":"2024-05-17T22:34:37.404633+0000","last_became_peered":"2024-05-17T22:34:37.404633+0000","last_unstale":"2024-05-17T22:36:14.624452+0000","last_undegraded":"2024-05-17T22:36:14.624452+0000","last_fullsized":"2024-05-17T22:36:14.624452+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:06:00.001190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.551211+0000","last_change":"2024-05-17T22:34:40.435374+0000","last_active":"2024-05-17T22:35:15.551211+0000","last_peered":"2024-05-17T22:35:15.551211+0000","last_clean":"2024-05-17T22:35:15.551211+0000","last_became_active":"2024-05-17T22:34:40.434522+0000","last_became_peered":"2024-05-17T22:34:40.434522+0000","last_unstale":"2024-05-17T22:35:15.551211+0000","last_undegraded":"2024-05-17T22:35:15.551211+0000","last_fullsized":"2024-05-17T22:35:15.551211+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:36:41.244970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.834397+0000","last_change":"2024-05-17T22:34:52.491708+0000","last_active":"2024-05-17T22:35:15.834397+0000","last_peered":"2024-05-17T22:35:15.834397+0000","last_clean":"2024-05-17T22:35:15.834397+0000","last_became_active":"2024-05-17T22:34:52.491293+0000","last_became_peered":"2024-05-17T22:34:52.491293+0000","last_unstale":"2024-05-17T22:35:15.834397+0000","last_undegraded":"2024-05-17T22:35:15.834397+0000","last_fullsized":"2024-05-17T22:35:15.834397+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:23:01.634026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:48.028017+0000","last_change":"2024-05-17T22:34:37.800525+0000","last_active":"2024-05-17T22:35:48.028017+0000","last_peered":"2024-05-17T22:35:48.028017+0000","last_clean":"2024-05-17T22:35:48.028017+0000","last_became_active":"2024-05-17T22:34:37.800093+0000","last_became_peered":"2024-05-17T22:34:37.800093+0000","last_unstale":"2024-05-17T22:35:48.028017+0000","last_undegraded":"2024-05-17T22:35:48.028017+0000","last_fullsized":"2024-05-17T22:35:48.028017+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:25:27.249889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.551600+0000","last_change":"2024-05-17T22:34:40.434891+0000","last_active":"2024-05-17T22:35:15.551600+0000","last_peered":"2024-05-17T22:35:15.551600+0000","last_clean":"2024-05-17T22:35:15.551600+0000","last_became_active":"2024-05-17T22:34:40.434543+0000","last_became_peered":"2024-05-17T22:34:40.434543+0000","last_unstale":"2024-05-17T22:35:15.551600+0000","last_undegraded":"2024-05-17T22:35:15.551600+0000","last_fullsized":"2024-05-17T22:35:15.551600+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:17:47.770225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.545179+0000","last_change":"2024-05-17T22:34:52.491544+0000","last_active":"2024-05-17T22:35:15.545179+0000","last_peered":"2024-05-17T22:35:15.545179+0000","last_clean":"2024-05-17T22:35:15.545179+0000","last_became_active":"2024-05-17T22:34:52.491138+0000","last_became_peered":"2024-05-17T22:34:52.491138+0000","last_unstale":"2024-05-17T22:35:15.545179+0000","last_undegraded":"2024-05-17T22:35:15.545179+0000","last_fullsized":"2024-05-17T22:35:15.545179+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:34:20.411299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.552474+0000","last_change":"2024-05-17T22:34:37.403220+0000","last_active":"2024-05-17T22:35:15.552474+0000","last_peered":"2024-05-17T22:35:15.552474+0000","last_clean":"2024-05-17T22:35:15.552474+0000","last_became_active":"2024-05-17T22:34:37.402856+0000","last_became_peered":"2024-05-17T22:34:37.402856+0000","last_unstale":"2024-05-17T22:35:15.552474+0000","last_undegraded":"2024-05-17T22:35:15.552474+0000","last_fullsized":"2024-05-17T22:35:15.552474+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:04:19.469294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.551094+0000","last_change":"2024-05-17T22:34:40.434487+0000","last_active":"2024-05-17T22:35:15.551094+0000","last_peered":"2024-05-17T22:35:15.551094+0000","last_clean":"2024-05-17T22:35:15.551094+0000","last_became_active":"2024-05-17T22:34:40.434160+0000","last_became_peered":"2024-05-17T22:34:40.434160+0000","last_unstale":"2024-05-17T22:35:15.551094+0000","last_undegraded":"2024-05-17T22:35:15.551094+0000","last_fullsized":"2024-05-17T22:35:15.551094+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:24:38.782999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.550847+0000","last_change":"2024-05-17T22:34:52.487887+0000","last_active":"2024-05-17T22:35:15.550847+0000","last_peered":"2024-05-17T22:35:15.550847+0000","last_clean":"2024-05-17T22:35:15.550847+0000","last_became_active":"2024-05-17T22:34:52.486837+0000","last_became_peered":"2024-05-17T22:34:52.486837+0000","last_unstale":"2024-05-17T22:35:15.550847+0000","last_undegraded":"2024-05-17T22:35:15.550847+0000","last_fullsized":"2024-05-17T22:35:15.550847+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:12:22.206536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.550184+0000","last_change":"2024-05-17T22:34:37.399498+0000","last_active":"2024-05-17T22:35:15.550184+0000","last_peered":"2024-05-17T22:35:15.550184+0000","last_clean":"2024-05-17T22:35:15.550184+0000","last_became_active":"2024-05-17T22:34:37.399144+0000","last_became_peered":"2024-05-17T22:34:37.399144+0000","last_unstale":"2024-05-17T22:35:15.550184+0000","last_undegraded":"2024-05-17T22:35:15.550184+0000","last_fullsized":"2024-05-17T22:35:15.550184+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:13:37.346006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.551674+0000","last_change":"2024-05-17T22:34:40.433839+0000","last_active":"2024-05-17T22:35:15.551674+0000","last_peered":"2024-05-17T22:35:15.551674+0000","last_clean":"2024-05-17T22:35:15.551674+0000","last_became_active":"2024-05-17T22:34:40.433489+0000","last_became_peered":"2024-05-17T22:34:40.433489+0000","last_unstale":"2024-05-17T22:35:15.551674+0000","last_undegraded":"2024-05-17T22:35:15.551674+0000","last_fullsized":"2024-05-17T22:35:15.551674+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:13:56.014094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.549312+0000","last_change":"2024-05-17T22:34:40.434953+0000","last_active":"2024-05-17T22:35:15.549312+0000","last_peered":"2024-05-17T22:35:15.549312+0000","last_clean":"2024-05-17T22:35:15.549312+0000","last_became_active":"2024-05-17T22:34:40.434593+0000","last_became_peered":"2024-05-17T22:34:40.434593+0000","last_unstale":"2024-05-17T22:35:15.549312+0000","last_undegraded":"2024-05-17T22:35:15.549312+0000","last_fullsized":"2024-05-17T22:35:15.549312+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:20:27.075382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.551216+0000","last_change":"2024-05-17T22:34:40.432787+0000","last_active":"2024-05-17T22:35:15.551216+0000","last_peered":"2024-05-17T22:35:15.551216+0000","last_clean":"2024-05-17T22:35:15.551216+0000","last_became_active":"2024-05-17T22:34:40.432402+0000","last_became_peered":"2024-05-17T22:34:40.432402+0000","last_unstale":"2024-05-17T22:35:15.551216+0000","last_undegraded":"2024-05-17T22:35:15.551216+0000","last_fullsized":"2024-05-17T22:35:15.551216+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:48:13.034276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.552274+0000","last_change":"2024-05-17T22:34:40.426423+0000","last_active":"2024-05-17T22:35:15.552274+0000","last_peered":"2024-05-17T22:35:15.552274+0000","last_clean":"2024-05-17T22:35:15.552274+0000","last_became_active":"2024-05-17T22:34:40.426049+0000","last_became_peered":"2024-05-17T22:34:40.426049+0000","last_unstale":"2024-05-17T22:35:15.552274+0000","last_undegraded":"2024-05-17T22:35:15.552274+0000","last_fullsized":"2024-05-17T22:35:15.552274+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:56:55.631322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.545413+0000","last_change":"2024-05-17T22:34:40.435706+0000","last_active":"2024-05-17T22:35:15.545413+0000","last_peered":"2024-05-17T22:35:15.545413+0000","last_clean":"2024-05-17T22:35:15.545413+0000","last_became_active":"2024-05-17T22:34:40.435330+0000","last_became_peered":"2024-05-17T22:34:40.435330+0000","last_unstale":"2024-05-17T22:35:15.545413+0000","last_undegraded":"2024-05-17T22:35:15.545413+0000","last_fullsized":"2024-05-17T22:35:15.545413+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:25:36.529454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.549735+0000","last_change":"2024-05-17T22:34:40.428997+0000","last_active":"2024-05-17T22:35:15.549735+0000","last_peered":"2024-05-17T22:35:15.549735+0000","last_clean":"2024-05-17T22:35:15.549735+0000","last_became_active":"2024-05-17T22:34:40.428594+0000","last_became_peered":"2024-05-17T22:34:40.428594+0000","last_unstale":"2024-05-17T22:35:15.549735+0000","last_undegraded":"2024-05-17T22:35:15.549735+0000","last_fullsized":"2024-05-17T22:35:15.549735+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:59:01.777751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.549908+0000","last_change":"2024-05-17T22:34:40.434241+0000","last_active":"2024-05-17T22:35:15.549908+0000","last_peered":"2024-05-17T22:35:15.549908+0000","last_clean":"2024-05-17T22:35:15.549908+0000","last_became_active":"2024-05-17T22:34:40.431652+0000","last_became_peered":"2024-05-17T22:34:40.431652+0000","last_unstale":"2024-05-17T22:35:15.549908+0000","last_undegraded":"2024-05-17T22:35:15.549908+0000","last_fullsized":"2024-05-17T22:35:15.549908+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:33:27.035284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.553548+0000","last_change":"2024-05-17T22:34:40.431485+0000","last_active":"2024-05-17T22:35:15.553548+0000","last_peered":"2024-05-17T22:35:15.553548+0000","last_clean":"2024-05-17T22:35:15.553548+0000","last_became_active":"2024-05-17T22:34:40.431120+0000","last_became_peered":"2024-05-17T22:34:40.431120+0000","last_unstale":"2024-05-17T22:35:15.553548+0000","last_undegraded":"2024-05-17T22:35:15.553548+0000","last_fullsized":"2024-05-17T22:35:15.553548+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:46:55.213602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.710879+0000","last_change":"2024-05-17T22:34:40.423117+0000","last_active":"2024-05-17T22:35:15.710879+0000","last_peered":"2024-05-17T22:35:15.710879+0000","last_clean":"2024-05-17T22:35:15.710879+0000","last_became_active":"2024-05-17T22:34:40.422694+0000","last_became_peered":"2024-05-17T22:34:40.422694+0000","last_unstale":"2024-05-17T22:35:15.710879+0000","last_undegraded":"2024-05-17T22:35:15.710879+0000","last_fullsized":"2024-05-17T22:35:15.710879+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:19:55.589532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.712333+0000","last_change":"2024-05-17T22:35:15.711796+0000","last_active":"2024-05-17T22:35:15.712333+0000","last_peered":"2024-05-17T22:35:15.712333+0000","last_clean":"2024-05-17T22:35:15.712333+0000","last_became_active":"2024-05-17T22:35:15.711327+0000","last_became_peered":"2024-05-17T22:35:15.711327+0000","last_unstale":"2024-05-17T22:35:15.712333+0000","last_undegraded":"2024-05-17T22:35:15.712333+0000","last_fullsized":"2024-05-17T22:35:15.712333+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:48:48.913312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.710486+0000","last_change":"2024-05-17T22:34:40.430393+0000","last_active":"2024-05-17T22:35:15.710486+0000","last_peered":"2024-05-17T22:35:15.710486+0000","last_clean":"2024-05-17T22:35:15.710486+0000","last_became_active":"2024-05-17T22:34:40.430030+0000","last_became_peered":"2024-05-17T22:34:40.430030+0000","last_unstale":"2024-05-17T22:35:15.710486+0000","last_undegraded":"2024-05-17T22:35:15.710486+0000","last_fullsized":"2024-05-17T22:35:15.710486+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:39:37.204246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.834719+0000","last_change":"2024-05-17T22:34:40.432443+0000","last_active":"2024-05-17T22:35:15.834719+0000","last_peered":"2024-05-17T22:35:15.834719+0000","last_clean":"2024-05-17T22:35:15.834719+0000","last_became_active":"2024-05-17T22:34:40.431991+0000","last_became_peered":"2024-05-17T22:34:40.431991+0000","last_unstale":"2024-05-17T22:35:15.834719+0000","last_undegraded":"2024-05-17T22:35:15.834719+0000","last_fullsized":"2024-05-17T22:35:15.834719+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:37:45.593575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.549284+0000","last_change":"2024-05-17T22:34:40.434388+0000","last_active":"2024-05-17T22:35:15.549284+0000","last_peered":"2024-05-17T22:35:15.549284+0000","last_clean":"2024-05-17T22:35:15.549284+0000","last_became_active":"2024-05-17T22:34:40.434044+0000","last_became_peered":"2024-05-17T22:34:40.434044+0000","last_unstale":"2024-05-17T22:35:15.549284+0000","last_undegraded":"2024-05-17T22:35:15.549284+0000","last_fullsized":"2024-05-17T22:35:15.549284+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:20:43.296190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T22:35:15.549977+0000","last_change":"2024-05-17T22:34:40.427137+0000","last_active":"2024-05-17T22:35:15.549977+0000","last_peered":"2024-05-17T22:35:15.549977+0000","last_clean":"2024-05-17T22:35:15.549977+0000","last_became_active":"2024-05-17T22:34:40.426703+0000","last_became_peered":"2024-05-17T22:34:40.426703+0000","last_unstale":"2024-05-17T22:35:15.549977+0000","last_undegraded":"2024-05-17T22:35:15.549977+0000","last_fullsized":"2024-05-17T22:35:15.549977+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:55:52.992817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62199,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":173,"num_read_kb":154,"num_write":166,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":176961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710754,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":496262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906984,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28652,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716500,"statfs":{"total":95995035648,"available":95965696000,"internally_reserved":0,"allocated":1675264,"data_stored":1141706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070507,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":500572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266739,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1019904,"data_stored":505782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462967,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28580,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716572,"statfs":{"total":95995035648,"available":95965769728,"internally_reserved":0,"allocated":1601536,"data_stored":1092696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626491,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":491652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1044480,"data_stored":544551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":493040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1069056,"data_stored":546292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":491846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986246,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":496693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822722,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36908,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708244,"statfs":{"total":95995035648,"available":95957241856,"internally_reserved":0,"allocated":1609728,"data_stored":1087893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":52989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":54730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":59776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T22:36:18.645 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-17T22:36:18.896 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:36:18.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:18 smithi151 ceph-mon[36101]: from='client.14865 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T22:36:19.019 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:18 smithi012 ceph-mon[31632]: from='client.14865 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T22:36:19.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:18 smithi096 ceph-mon[36065]: from='client.14865 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T22:36:19.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:19 smithi151 ceph-mon[36101]: pgmap v351: 129 pgs: 1 peering, 128 active+clean; 638 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1023 B/s wr, 6 op/s 2024-05-17T22:36:20.046 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:19 smithi012 ceph-mon[31632]: pgmap v351: 129 pgs: 1 peering, 128 active+clean; 638 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1023 B/s wr, 6 op/s 2024-05-17T22:36:20.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:19 smithi096 ceph-mon[36065]: pgmap v351: 129 pgs: 1 peering, 128 active+clean; 638 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1023 B/s wr, 6 op/s 2024-05-17T22:36:20.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.604+0000 7fdb89eec700 1 -- 172.21.15.12:0/2875524956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb84102e00 msgr2=0x7fdb84103210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:20.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.604+0000 7fdb89eec700 1 --2- 172.21.15.12:0/2875524956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb84102e00 0x7fdb84103210 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdb7400b0a0 tx=0x7fdb7400b3b0 comp rx=0 tx=0).stop 2024-05-17T22:36:20.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.604+0000 7fdb89eec700 1 -- 172.21.15.12:0/2875524956 shutdown_connections 2024-05-17T22:36:20.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.604+0000 7fdb89eec700 1 --2- 172.21.15.12:0/2875524956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb84108080 0x7fdb8410f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:20.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.604+0000 7fdb89eec700 1 --2- 172.21.15.12:0/2875524956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb84102e00 0x7fdb84103210 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:20.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.604+0000 7fdb89eec700 1 --2- 172.21.15.12:0/2875524956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb84107730 0x7fdb84107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:20.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.605+0000 7fdb89eec700 1 -- 172.21.15.12:0/2875524956 >> 172.21.15.12:0/2875524956 conn(0x7fdb84079bd0 msgr2=0x7fdb84079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:36:20.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.605+0000 7fdb89eec700 1 -- 172.21.15.12:0/2875524956 shutdown_connections 2024-05-17T22:36:20.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.605+0000 7fdb89eec700 1 -- 172.21.15.12:0/2875524956 wait complete. 2024-05-17T22:36:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.606+0000 7fdb89eec700 1 Processor -- start 2024-05-17T22:36:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.606+0000 7fdb89eec700 1 -- start start 2024-05-17T22:36:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.606+0000 7fdb89eec700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb84102e00 0x7fdb8419b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.606+0000 7fdb89eec700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb84107730 0x7fdb8419bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.606+0000 7fdb89eec700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb84108080 0x7fdb8419c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.606+0000 7fdb89eec700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdb84112cb0 con 0x7fdb84108080 2024-05-17T22:36:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.606+0000 7fdb89eec700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdb84112b30 con 0x7fdb84102e00 2024-05-17T22:36:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.606+0000 7fdb89eec700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdb84112e30 con 0x7fdb84107730 2024-05-17T22:36:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.606+0000 7fdb83fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb84108080 0x7fdb8419c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.606+0000 7fdb83fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb84108080 0x7fdb8419c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60276/0 (socket says 172.21.15.12:60276) 2024-05-17T22:36:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.606+0000 7fdb83fff700 1 -- 172.21.15.12:0/1989344867 learned_addr learned my addr 172.21.15.12:0/1989344867 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:36:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.606+0000 7fdb82ffd700 1 --2- 172.21.15.12:0/1989344867 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb84107730 0x7fdb8419bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.607+0000 7fdb837fe700 1 --2- 172.21.15.12:0/1989344867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb84102e00 0x7fdb8419b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.607+0000 7fdb83fff700 1 -- 172.21.15.12:0/1989344867 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb84107730 msgr2=0x7fdb8419bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.607+0000 7fdb83fff700 1 --2- 172.21.15.12:0/1989344867 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb84107730 0x7fdb8419bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.607+0000 7fdb83fff700 1 -- 172.21.15.12:0/1989344867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb84102e00 msgr2=0x7fdb8419b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.607+0000 7fdb83fff700 1 --2- 172.21.15.12:0/1989344867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb84102e00 0x7fdb8419b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.607+0000 7fdb83fff700 1 -- 172.21.15.12:0/1989344867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb841a6970 con 0x7fdb84108080 2024-05-17T22:36:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.607+0000 7fdb837fe700 1 --2- 172.21.15.12:0/1989344867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb84102e00 0x7fdb8419b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:36:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.607+0000 7fdb83fff700 1 --2- 172.21.15.12:0/1989344867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb84108080 0x7fdb8419c0c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c00d980 tx=0x7fdb7c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:36:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.607+0000 7fdb80ff9700 1 -- 172.21.15.12:0/1989344867 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c017070 con 0x7fdb84108080 2024-05-17T22:36:20.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.607+0000 7fdb80ff9700 1 -- 172.21.15.12:0/1989344867 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb7c005020 con 0x7fdb84108080 2024-05-17T22:36:20.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.607+0000 7fdb89eec700 1 -- 172.21.15.12:0/1989344867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb841a6c60 con 0x7fdb84108080 2024-05-17T22:36:20.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.607+0000 7fdb80ff9700 1 -- 172.21.15.12:0/1989344867 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c0099b0 con 0x7fdb84108080 2024-05-17T22:36:20.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.607+0000 7fdb89eec700 1 -- 172.21.15.12:0/1989344867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb841a7150 con 0x7fdb84108080 2024-05-17T22:36:20.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.608+0000 7fdb89eec700 1 -- 172.21.15.12:0/1989344867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb84103210 con 0x7fdb84108080 2024-05-17T22:36:20.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.610+0000 7fdb80ff9700 1 -- 172.21.15.12:0/1989344867 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fdb7c009b70 con 0x7fdb84108080 2024-05-17T22:36:20.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.610+0000 7fdb80ff9700 1 --2- 172.21.15.12:0/1989344867 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fdb700b2910 0x7fdb700b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:20.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.611+0000 7fdb837fe700 1 --2- 172.21.15.12:0/1989344867 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fdb700b2910 0x7fdb700b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:20.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.611+0000 7fdb80ff9700 1 -- 172.21.15.12:0/1989344867 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fdb7c0e29d0 con 0x7fdb84108080 2024-05-17T22:36:20.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.611+0000 7fdb837fe700 1 --2- 172.21.15.12:0/1989344867 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fdb700b2910 0x7fdb700b4e20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdb740062a0 tx=0x7fdb74006230 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:36:20.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.614+0000 7fdb80ff9700 1 -- 172.21.15.12:0/1989344867 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdb7c020020 con 0x7fdb84108080 2024-05-17T22:36:20.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.823+0000 7fdb89eec700 1 -- 172.21.15.12:0/1989344867 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdb8406bc70 con 0x7fdb700b2910 2024-05-17T22:36:20.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.829+0000 7fdb80ff9700 1 -- 172.21.15.12:0/1989344867 <== mgr.14152 v2:172.21.15.12:6800/1388514047 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340283 (secure 0 0 0) 0x7fdb8406bc70 con 0x7fdb700b2910 2024-05-17T22:36:20.836 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:36:20.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.834+0000 7fdb89eec700 1 -- 172.21.15.12:0/1989344867 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fdb700b2910 msgr2=0x7fdb700b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:20.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.834+0000 7fdb89eec700 1 --2- 172.21.15.12:0/1989344867 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fdb700b2910 0x7fdb700b4e20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdb740062a0 tx=0x7fdb74006230 comp rx=0 tx=0).stop 2024-05-17T22:36:20.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.834+0000 7fdb89eec700 1 -- 172.21.15.12:0/1989344867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb84108080 msgr2=0x7fdb8419c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:20.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.834+0000 7fdb89eec700 1 --2- 172.21.15.12:0/1989344867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb84108080 0x7fdb8419c0c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c00d980 tx=0x7fdb7c00dc90 comp rx=0 tx=0).stop 2024-05-17T22:36:20.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.834+0000 7fdb89eec700 1 -- 172.21.15.12:0/1989344867 shutdown_connections 2024-05-17T22:36:20.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.834+0000 7fdb89eec700 1 --2- 172.21.15.12:0/1989344867 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fdb700b2910 0x7fdb700b4e20 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:20.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.834+0000 7fdb89eec700 1 --2- 172.21.15.12:0/1989344867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb84108080 0x7fdb8419c0c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:20.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.834+0000 7fdb89eec700 1 --2- 172.21.15.12:0/1989344867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb84102e00 0x7fdb8419b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:20.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.834+0000 7fdb89eec700 1 --2- 172.21.15.12:0/1989344867 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb84107730 0x7fdb8419bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:20.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.834+0000 7fdb89eec700 1 -- 172.21.15.12:0/1989344867 >> 172.21.15.12:0/1989344867 conn(0x7fdb84079bd0 msgr2=0x7fdb84105870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:36:20.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.834+0000 7fdb89eec700 1 -- 172.21.15.12:0/1989344867 shutdown_connections 2024-05-17T22:36:20.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:20.835+0000 7fdb89eec700 1 -- 172.21.15.12:0/1989344867 wait complete. 2024-05-17T22:36:20.840 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-05-17T22:36:21.643 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2024-05-17T22:36:20.504791+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657066,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":280,"num_read_kb":238,"num_write":294,"num_write_kb":2270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":238,"ondisk_log_size":238,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":418,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":384088,"kb_used_data":14060,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124557736,"statfs":{"total":1151940427776,"available":1151547121664,"internally_reserved":0,"allocated":14397440,"data_stored":8109400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9083,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":40,"num_write":12,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002314"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628725+0000","last_change":"2024-05-17T22:34:40.422850+0000","last_active":"2024-05-17T22:36:15.628725+0000","last_peered":"2024-05-17T22:36:15.628725+0000","last_clean":"2024-05-17T22:36:15.628725+0000","last_became_active":"2024-05-17T22:34:40.422474+0000","last_became_peered":"2024-05-17T22:34:40.422474+0000","last_unstale":"2024-05-17T22:36:15.628725+0000","last_undegraded":"2024-05-17T22:36:15.628725+0000","last_fullsized":"2024-05-17T22:36:15.628725+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:23:51.537128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968741+0000","last_change":"2024-05-17T22:34:40.427907+0000","last_active":"2024-05-17T22:36:15.968741+0000","last_peered":"2024-05-17T22:36:15.968741+0000","last_clean":"2024-05-17T22:36:15.968741+0000","last_became_active":"2024-05-17T22:34:40.427511+0000","last_became_peered":"2024-05-17T22:34:40.427511+0000","last_unstale":"2024-05-17T22:36:15.968741+0000","last_undegraded":"2024-05-17T22:36:15.968741+0000","last_fullsized":"2024-05-17T22:36:15.968741+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:01:46.707820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968443+0000","last_change":"2024-05-17T22:34:40.436370+0000","last_active":"2024-05-17T22:36:15.968443+0000","last_peered":"2024-05-17T22:36:15.968443+0000","last_clean":"2024-05-17T22:36:15.968443+0000","last_became_active":"2024-05-17T22:34:40.434089+0000","last_became_peered":"2024-05-17T22:34:40.434089+0000","last_unstale":"2024-05-17T22:36:15.968443+0000","last_undegraded":"2024-05-17T22:36:15.968443+0000","last_fullsized":"2024-05-17T22:36:15.968443+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:58:16.775471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623409+0000","last_change":"2024-05-17T22:35:15.556681+0000","last_active":"2024-05-17T22:36:15.623409+0000","last_peered":"2024-05-17T22:36:15.623409+0000","last_clean":"2024-05-17T22:36:15.623409+0000","last_became_active":"2024-05-17T22:35:15.556427+0000","last_became_peered":"2024-05-17T22:35:15.556427+0000","last_unstale":"2024-05-17T22:36:15.623409+0000","last_undegraded":"2024-05-17T22:36:15.623409+0000","last_fullsized":"2024-05-17T22:36:15.623409+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:28:01.726262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T22:36:14.624343+0000","last_change":"2024-05-17T22:34:40.434394+0000","last_active":"2024-05-17T22:36:14.624343+0000","last_peered":"2024-05-17T22:36:14.624343+0000","last_clean":"2024-05-17T22:36:14.624343+0000","last_became_active":"2024-05-17T22:34:40.433489+0000","last_became_peered":"2024-05-17T22:34:40.433489+0000","last_unstale":"2024-05-17T22:36:14.624343+0000","last_undegraded":"2024-05-17T22:36:14.624343+0000","last_fullsized":"2024-05-17T22:36:14.624343+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:26:24.068814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625953+0000","last_change":"2024-05-17T22:35:15.553601+0000","last_active":"2024-05-17T22:36:15.625953+0000","last_peered":"2024-05-17T22:36:15.625953+0000","last_clean":"2024-05-17T22:36:15.625953+0000","last_became_active":"2024-05-17T22:35:15.553116+0000","last_became_peered":"2024-05-17T22:35:15.553116+0000","last_unstale":"2024-05-17T22:36:15.625953+0000","last_undegraded":"2024-05-17T22:36:15.625953+0000","last_fullsized":"2024-05-17T22:36:15.625953+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:09:38.401107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004274+0000","last_change":"2024-05-17T22:34:40.430479+0000","last_active":"2024-05-17T22:36:16.004274+0000","last_peered":"2024-05-17T22:36:16.004274+0000","last_clean":"2024-05-17T22:36:16.004274+0000","last_became_active":"2024-05-17T22:34:40.430025+0000","last_became_peered":"2024-05-17T22:34:40.430025+0000","last_unstale":"2024-05-17T22:36:16.004274+0000","last_undegraded":"2024-05-17T22:36:16.004274+0000","last_fullsized":"2024-05-17T22:36:16.004274+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:14:19.725096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984193+0000","last_change":"2024-05-17T22:34:40.431711+0000","last_active":"2024-05-17T22:36:15.984193+0000","last_peered":"2024-05-17T22:36:15.984193+0000","last_clean":"2024-05-17T22:36:15.984193+0000","last_became_active":"2024-05-17T22:34:40.431368+0000","last_became_peered":"2024-05-17T22:34:40.431368+0000","last_unstale":"2024-05-17T22:36:15.984193+0000","last_undegraded":"2024-05-17T22:36:15.984193+0000","last_fullsized":"2024-05-17T22:36:15.984193+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:06:34.904652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.805730+0000","last_change":"2024-05-17T22:34:40.435593+0000","last_active":"2024-05-17T22:36:15.805730+0000","last_peered":"2024-05-17T22:36:15.805730+0000","last_clean":"2024-05-17T22:36:15.805730+0000","last_became_active":"2024-05-17T22:34:40.435241+0000","last_became_peered":"2024-05-17T22:34:40.435241+0000","last_unstale":"2024-05-17T22:36:15.805730+0000","last_undegraded":"2024-05-17T22:36:15.805730+0000","last_fullsized":"2024-05-17T22:36:15.805730+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:08:57.239998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.626366+0000","last_change":"2024-05-17T22:34:40.436248+0000","last_active":"2024-05-17T22:36:15.626366+0000","last_peered":"2024-05-17T22:36:15.626366+0000","last_clean":"2024-05-17T22:36:15.626366+0000","last_became_active":"2024-05-17T22:34:40.435824+0000","last_became_peered":"2024-05-17T22:34:40.435824+0000","last_unstale":"2024-05-17T22:36:15.626366+0000","last_undegraded":"2024-05-17T22:36:15.626366+0000","last_fullsized":"2024-05-17T22:36:15.626366+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:39:07.297470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.005145+0000","last_change":"2024-05-17T22:34:40.435843+0000","last_active":"2024-05-17T22:36:16.005145+0000","last_peered":"2024-05-17T22:36:16.005145+0000","last_clean":"2024-05-17T22:36:16.005145+0000","last_became_active":"2024-05-17T22:34:40.435500+0000","last_became_peered":"2024-05-17T22:34:40.435500+0000","last_unstale":"2024-05-17T22:36:16.005145+0000","last_undegraded":"2024-05-17T22:36:16.005145+0000","last_fullsized":"2024-05-17T22:36:16.005145+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:11:28.115850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004758+0000","last_change":"2024-05-17T22:34:40.432044+0000","last_active":"2024-05-17T22:36:16.004758+0000","last_peered":"2024-05-17T22:36:16.004758+0000","last_clean":"2024-05-17T22:36:16.004758+0000","last_became_active":"2024-05-17T22:34:40.429141+0000","last_became_peered":"2024-05-17T22:34:40.429141+0000","last_unstale":"2024-05-17T22:36:16.004758+0000","last_undegraded":"2024-05-17T22:36:16.004758+0000","last_fullsized":"2024-05-17T22:36:16.004758+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:04:53.428681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685924+0000","last_change":"2024-05-17T22:34:40.438474+0000","last_active":"2024-05-17T22:36:15.685924+0000","last_peered":"2024-05-17T22:36:15.685924+0000","last_clean":"2024-05-17T22:36:15.685924+0000","last_became_active":"2024-05-17T22:34:40.438109+0000","last_became_peered":"2024-05-17T22:34:40.438109+0000","last_unstale":"2024-05-17T22:36:15.685924+0000","last_undegraded":"2024-05-17T22:36:15.685924+0000","last_fullsized":"2024-05-17T22:36:15.685924+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:26:00.665246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968785+0000","last_change":"2024-05-17T22:34:40.431563+0000","last_active":"2024-05-17T22:36:15.968785+0000","last_peered":"2024-05-17T22:36:15.968785+0000","last_clean":"2024-05-17T22:36:15.968785+0000","last_became_active":"2024-05-17T22:34:40.430992+0000","last_became_peered":"2024-05-17T22:34:40.430992+0000","last_unstale":"2024-05-17T22:36:15.968785+0000","last_undegraded":"2024-05-17T22:36:15.968785+0000","last_fullsized":"2024-05-17T22:36:15.968785+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:51:26.280440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968328+0000","last_change":"2024-05-17T22:34:40.433629+0000","last_active":"2024-05-17T22:36:15.968328+0000","last_peered":"2024-05-17T22:36:15.968328+0000","last_clean":"2024-05-17T22:36:15.968328+0000","last_became_active":"2024-05-17T22:34:40.433270+0000","last_became_peered":"2024-05-17T22:34:40.433270+0000","last_unstale":"2024-05-17T22:36:15.968328+0000","last_undegraded":"2024-05-17T22:36:15.968328+0000","last_fullsized":"2024-05-17T22:36:15.968328+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:57:51.164078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.805030+0000","last_change":"2024-05-17T22:34:40.434218+0000","last_active":"2024-05-17T22:36:15.805030+0000","last_peered":"2024-05-17T22:36:15.805030+0000","last_clean":"2024-05-17T22:36:15.805030+0000","last_became_active":"2024-05-17T22:34:40.433849+0000","last_became_peered":"2024-05-17T22:34:40.433849+0000","last_unstale":"2024-05-17T22:36:15.805030+0000","last_undegraded":"2024-05-17T22:36:15.805030+0000","last_fullsized":"2024-05-17T22:36:15.805030+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:41:55.984109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628531+0000","last_change":"2024-05-17T22:34:40.427892+0000","last_active":"2024-05-17T22:36:15.628531+0000","last_peered":"2024-05-17T22:36:15.628531+0000","last_clean":"2024-05-17T22:36:15.628531+0000","last_became_active":"2024-05-17T22:34:40.427303+0000","last_became_peered":"2024-05-17T22:34:40.427303+0000","last_unstale":"2024-05-17T22:36:15.628531+0000","last_undegraded":"2024-05-17T22:36:15.628531+0000","last_fullsized":"2024-05-17T22:36:15.628531+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:59:38.632123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623636+0000","last_change":"2024-05-17T22:34:40.424784+0000","last_active":"2024-05-17T22:36:15.623636+0000","last_peered":"2024-05-17T22:36:15.623636+0000","last_clean":"2024-05-17T22:36:15.623636+0000","last_became_active":"2024-05-17T22:34:40.424439+0000","last_became_peered":"2024-05-17T22:34:40.424439+0000","last_unstale":"2024-05-17T22:36:15.623636+0000","last_undegraded":"2024-05-17T22:36:15.623636+0000","last_fullsized":"2024-05-17T22:36:15.623636+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:02:53.681600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624433+0000","last_change":"2024-05-17T22:34:40.435023+0000","last_active":"2024-05-17T22:36:15.624433+0000","last_peered":"2024-05-17T22:36:15.624433+0000","last_clean":"2024-05-17T22:36:15.624433+0000","last_became_active":"2024-05-17T22:34:40.434441+0000","last_became_peered":"2024-05-17T22:34:40.434441+0000","last_unstale":"2024-05-17T22:36:15.624433+0000","last_undegraded":"2024-05-17T22:36:15.624433+0000","last_fullsized":"2024-05-17T22:36:15.624433+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:16:07.671897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623973+0000","last_change":"2024-05-17T22:34:52.493192+0000","last_active":"2024-05-17T22:36:15.623973+0000","last_peered":"2024-05-17T22:36:15.623973+0000","last_clean":"2024-05-17T22:36:15.623973+0000","last_became_active":"2024-05-17T22:34:52.492786+0000","last_became_peered":"2024-05-17T22:34:52.492786+0000","last_unstale":"2024-05-17T22:36:15.623973+0000","last_undegraded":"2024-05-17T22:36:15.623973+0000","last_fullsized":"2024-05-17T22:36:15.623973+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:43:40.152240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623754+0000","last_change":"2024-05-17T22:34:37.402360+0000","last_active":"2024-05-17T22:36:15.623754+0000","last_peered":"2024-05-17T22:36:15.623754+0000","last_clean":"2024-05-17T22:36:15.623754+0000","last_became_active":"2024-05-17T22:34:37.401882+0000","last_became_peered":"2024-05-17T22:34:37.401882+0000","last_unstale":"2024-05-17T22:36:15.623754+0000","last_undegraded":"2024-05-17T22:36:15.623754+0000","last_fullsized":"2024-05-17T22:36:15.623754+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:53:13.300626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984571+0000","last_change":"2024-05-17T22:34:40.436259+0000","last_active":"2024-05-17T22:36:15.984571+0000","last_peered":"2024-05-17T22:36:15.984571+0000","last_clean":"2024-05-17T22:36:15.984571+0000","last_became_active":"2024-05-17T22:34:40.435899+0000","last_became_peered":"2024-05-17T22:34:40.435899+0000","last_unstale":"2024-05-17T22:36:15.984571+0000","last_undegraded":"2024-05-17T22:36:15.984571+0000","last_fullsized":"2024-05-17T22:36:15.984571+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:26:29.327513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003655+0000","last_change":"2024-05-17T22:34:52.495236+0000","last_active":"2024-05-17T22:36:16.003655+0000","last_peered":"2024-05-17T22:36:16.003655+0000","last_clean":"2024-05-17T22:36:16.003655+0000","last_became_active":"2024-05-17T22:34:52.494783+0000","last_became_peered":"2024-05-17T22:34:52.494783+0000","last_unstale":"2024-05-17T22:36:16.003655+0000","last_undegraded":"2024-05-17T22:36:16.003655+0000","last_fullsized":"2024-05-17T22:36:16.003655+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:08:15.010706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984767+0000","last_change":"2024-05-17T22:34:37.441129+0000","last_active":"2024-05-17T22:36:15.984767+0000","last_peered":"2024-05-17T22:36:15.984767+0000","last_clean":"2024-05-17T22:36:15.984767+0000","last_became_active":"2024-05-17T22:34:37.440738+0000","last_became_peered":"2024-05-17T22:34:37.440738+0000","last_unstale":"2024-05-17T22:36:15.984767+0000","last_undegraded":"2024-05-17T22:36:15.984767+0000","last_fullsized":"2024-05-17T22:36:15.984767+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:03:16.945076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968555+0000","last_change":"2024-05-17T22:34:40.434146+0000","last_active":"2024-05-17T22:36:15.968555+0000","last_peered":"2024-05-17T22:36:15.968555+0000","last_clean":"2024-05-17T22:36:15.968555+0000","last_became_active":"2024-05-17T22:34:40.433802+0000","last_became_peered":"2024-05-17T22:34:40.433802+0000","last_unstale":"2024-05-17T22:36:15.968555+0000","last_undegraded":"2024-05-17T22:36:15.968555+0000","last_fullsized":"2024-05-17T22:36:15.968555+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:08:39.905409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003713+0000","last_change":"2024-05-17T22:34:52.501134+0000","last_active":"2024-05-17T22:36:16.003713+0000","last_peered":"2024-05-17T22:36:16.003713+0000","last_clean":"2024-05-17T22:36:16.003713+0000","last_became_active":"2024-05-17T22:34:52.500547+0000","last_became_peered":"2024-05-17T22:34:52.500547+0000","last_unstale":"2024-05-17T22:36:16.003713+0000","last_undegraded":"2024-05-17T22:36:16.003713+0000","last_fullsized":"2024-05-17T22:36:16.003713+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:13:04.200865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T22:36:14.624749+0000","last_change":"2024-05-17T22:34:37.404019+0000","last_active":"2024-05-17T22:36:14.624749+0000","last_peered":"2024-05-17T22:36:14.624749+0000","last_clean":"2024-05-17T22:36:14.624749+0000","last_became_active":"2024-05-17T22:34:37.403607+0000","last_became_peered":"2024-05-17T22:34:37.403607+0000","last_unstale":"2024-05-17T22:36:14.624749+0000","last_undegraded":"2024-05-17T22:36:14.624749+0000","last_fullsized":"2024-05-17T22:36:14.624749+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:30:21.405195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.805310+0000","last_change":"2024-05-17T22:34:40.426889+0000","last_active":"2024-05-17T22:36:15.805310+0000","last_peered":"2024-05-17T22:36:15.805310+0000","last_clean":"2024-05-17T22:36:15.805310+0000","last_became_active":"2024-05-17T22:34:40.426512+0000","last_became_peered":"2024-05-17T22:34:40.426512+0000","last_unstale":"2024-05-17T22:36:15.805310+0000","last_undegraded":"2024-05-17T22:36:15.805310+0000","last_fullsized":"2024-05-17T22:36:15.805310+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:30:51.363177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003876+0000","last_change":"2024-05-17T22:34:52.487514+0000","last_active":"2024-05-17T22:36:16.003876+0000","last_peered":"2024-05-17T22:36:16.003876+0000","last_clean":"2024-05-17T22:36:16.003876+0000","last_became_active":"2024-05-17T22:34:52.487068+0000","last_became_peered":"2024-05-17T22:34:52.487068+0000","last_unstale":"2024-05-17T22:36:16.003876+0000","last_undegraded":"2024-05-17T22:36:16.003876+0000","last_fullsized":"2024-05-17T22:36:16.003876+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:52:13.573986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T22:36:14.624545+0000","last_change":"2024-05-17T22:34:37.400518+0000","last_active":"2024-05-17T22:36:14.624545+0000","last_peered":"2024-05-17T22:36:14.624545+0000","last_clean":"2024-05-17T22:36:14.624545+0000","last_became_active":"2024-05-17T22:34:37.400145+0000","last_became_peered":"2024-05-17T22:34:37.400145+0000","last_unstale":"2024-05-17T22:36:14.624545+0000","last_undegraded":"2024-05-17T22:36:14.624545+0000","last_fullsized":"2024-05-17T22:36:14.624545+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:56:53.593624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004156+0000","last_change":"2024-05-17T22:34:40.434148+0000","last_active":"2024-05-17T22:36:16.004156+0000","last_peered":"2024-05-17T22:36:16.004156+0000","last_clean":"2024-05-17T22:36:16.004156+0000","last_became_active":"2024-05-17T22:34:40.433613+0000","last_became_peered":"2024-05-17T22:34:40.433613+0000","last_unstale":"2024-05-17T22:36:16.004156+0000","last_undegraded":"2024-05-17T22:36:16.004156+0000","last_fullsized":"2024-05-17T22:36:16.004156+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:09:39.049852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624284+0000","last_change":"2024-05-17T22:34:52.490417+0000","last_active":"2024-05-17T22:36:15.624284+0000","last_peered":"2024-05-17T22:36:15.624284+0000","last_clean":"2024-05-17T22:36:15.624284+0000","last_became_active":"2024-05-17T22:34:52.489953+0000","last_became_peered":"2024-05-17T22:34:52.489953+0000","last_unstale":"2024-05-17T22:36:15.624284+0000","last_undegraded":"2024-05-17T22:36:15.624284+0000","last_fullsized":"2024-05-17T22:36:15.624284+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:23:56.491899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":81,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.083741+0000","last_change":"2024-05-17T22:34:37.405304+0000","last_active":"2024-05-17T22:36:16.083741+0000","last_peered":"2024-05-17T22:36:16.083741+0000","last_clean":"2024-05-17T22:36:16.083741+0000","last_became_active":"2024-05-17T22:34:37.404552+0000","last_became_peered":"2024-05-17T22:34:37.404552+0000","last_unstale":"2024-05-17T22:36:16.083741+0000","last_undegraded":"2024-05-17T22:36:16.083741+0000","last_fullsized":"2024-05-17T22:36:16.083741+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:57:06.556886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":31,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T22:36:14.624060+0000","last_change":"2024-05-17T22:34:40.430537+0000","last_active":"2024-05-17T22:36:14.624060+0000","last_peered":"2024-05-17T22:36:14.624060+0000","last_clean":"2024-05-17T22:36:14.624060+0000","last_became_active":"2024-05-17T22:34:40.430193+0000","last_became_peered":"2024-05-17T22:34:40.430193+0000","last_unstale":"2024-05-17T22:36:14.624060+0000","last_undegraded":"2024-05-17T22:36:14.624060+0000","last_fullsized":"2024-05-17T22:36:14.624060+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:58:42.091445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624333+0000","last_change":"2024-05-17T22:35:15.559077+0000","last_active":"2024-05-17T22:36:15.624333+0000","last_peered":"2024-05-17T22:36:15.624333+0000","last_clean":"2024-05-17T22:36:15.624333+0000","last_became_active":"2024-05-17T22:35:15.558382+0000","last_became_peered":"2024-05-17T22:35:15.558382+0000","last_unstale":"2024-05-17T22:36:15.624333+0000","last_undegraded":"2024-05-17T22:36:15.624333+0000","last_fullsized":"2024-05-17T22:36:15.624333+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:20:16.009620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625951+0000","last_change":"2024-05-17T22:34:37.405332+0000","last_active":"2024-05-17T22:36:15.625951+0000","last_peered":"2024-05-17T22:36:15.625951+0000","last_clean":"2024-05-17T22:36:15.625951+0000","last_became_active":"2024-05-17T22:34:37.404970+0000","last_became_peered":"2024-05-17T22:34:37.404970+0000","last_unstale":"2024-05-17T22:36:15.625951+0000","last_undegraded":"2024-05-17T22:36:15.625951+0000","last_fullsized":"2024-05-17T22:36:15.625951+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:37:39.933686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003790+0000","last_change":"2024-05-17T22:34:40.432690+0000","last_active":"2024-05-17T22:36:16.003790+0000","last_peered":"2024-05-17T22:36:16.003790+0000","last_clean":"2024-05-17T22:36:16.003790+0000","last_became_active":"2024-05-17T22:34:40.432274+0000","last_became_peered":"2024-05-17T22:34:40.432274+0000","last_unstale":"2024-05-17T22:36:16.003790+0000","last_undegraded":"2024-05-17T22:36:16.003790+0000","last_fullsized":"2024-05-17T22:36:16.003790+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:12:47.926865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628515+0000","last_change":"2024-05-17T22:34:52.494645+0000","last_active":"2024-05-17T22:36:15.628515+0000","last_peered":"2024-05-17T22:36:15.628515+0000","last_clean":"2024-05-17T22:36:15.628515+0000","last_became_active":"2024-05-17T22:34:52.494255+0000","last_became_peered":"2024-05-17T22:34:52.494255+0000","last_unstale":"2024-05-17T22:36:15.628515+0000","last_undegraded":"2024-05-17T22:36:15.628515+0000","last_fullsized":"2024-05-17T22:36:15.628515+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:41:26.633571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685352+0000","last_change":"2024-05-17T22:34:37.398582+0000","last_active":"2024-05-17T22:36:15.685352+0000","last_peered":"2024-05-17T22:36:15.685352+0000","last_clean":"2024-05-17T22:36:15.685352+0000","last_became_active":"2024-05-17T22:34:37.398156+0000","last_became_peered":"2024-05-17T22:34:37.398156+0000","last_unstale":"2024-05-17T22:36:15.685352+0000","last_undegraded":"2024-05-17T22:36:15.685352+0000","last_fullsized":"2024-05-17T22:36:15.685352+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:46:00.276214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.622638+0000","last_change":"2024-05-17T22:35:15.551402+0000","last_active":"2024-05-17T22:36:15.622638+0000","last_peered":"2024-05-17T22:36:15.622638+0000","last_clean":"2024-05-17T22:36:15.622638+0000","last_became_active":"2024-05-17T22:35:15.551021+0000","last_became_peered":"2024-05-17T22:35:15.551021+0000","last_unstale":"2024-05-17T22:36:15.622638+0000","last_undegraded":"2024-05-17T22:36:15.622638+0000","last_fullsized":"2024-05-17T22:36:15.622638+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:35:18.573620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968732+0000","last_change":"2024-05-17T22:34:52.487413+0000","last_active":"2024-05-17T22:36:15.968732+0000","last_peered":"2024-05-17T22:36:15.968732+0000","last_clean":"2024-05-17T22:36:15.968732+0000","last_became_active":"2024-05-17T22:34:52.487032+0000","last_became_peered":"2024-05-17T22:34:52.487032+0000","last_unstale":"2024-05-17T22:36:15.968732+0000","last_undegraded":"2024-05-17T22:36:15.968732+0000","last_fullsized":"2024-05-17T22:36:15.968732+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:34:49.122008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003249+0000","last_change":"2024-05-17T22:34:37.404894+0000","last_active":"2024-05-17T22:36:16.003249+0000","last_peered":"2024-05-17T22:36:16.003249+0000","last_clean":"2024-05-17T22:36:16.003249+0000","last_became_active":"2024-05-17T22:34:37.404567+0000","last_became_peered":"2024-05-17T22:34:37.404567+0000","last_unstale":"2024-05-17T22:36:16.003249+0000","last_undegraded":"2024-05-17T22:36:16.003249+0000","last_fullsized":"2024-05-17T22:36:16.003249+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:13:09.761556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623009+0000","last_change":"2024-05-17T22:34:40.418267+0000","last_active":"2024-05-17T22:36:15.623009+0000","last_peered":"2024-05-17T22:36:15.623009+0000","last_clean":"2024-05-17T22:36:15.623009+0000","last_became_active":"2024-05-17T22:34:40.417928+0000","last_became_peered":"2024-05-17T22:34:40.417928+0000","last_unstale":"2024-05-17T22:36:15.623009+0000","last_undegraded":"2024-05-17T22:36:15.623009+0000","last_fullsized":"2024-05-17T22:36:15.623009+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:17:37.055004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984148+0000","last_change":"2024-05-17T22:34:52.492126+0000","last_active":"2024-05-17T22:36:15.984148+0000","last_peered":"2024-05-17T22:36:15.984148+0000","last_clean":"2024-05-17T22:36:15.984148+0000","last_became_active":"2024-05-17T22:34:52.491695+0000","last_became_peered":"2024-05-17T22:34:52.491695+0000","last_unstale":"2024-05-17T22:36:15.984148+0000","last_undegraded":"2024-05-17T22:36:15.984148+0000","last_fullsized":"2024-05-17T22:36:15.984148+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:15:17.710279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":77,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:18.086300+0000","last_change":"2024-05-17T22:34:37.406989+0000","last_active":"2024-05-17T22:36:18.086300+0000","last_peered":"2024-05-17T22:36:18.086300+0000","last_clean":"2024-05-17T22:36:18.086300+0000","last_became_active":"2024-05-17T22:34:37.406612+0000","last_became_peered":"2024-05-17T22:34:37.406612+0000","last_unstale":"2024-05-17T22:36:18.086300+0000","last_undegraded":"2024-05-17T22:36:18.086300+0000","last_fullsized":"2024-05-17T22:36:18.086300+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:47:52.480804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.805642+0000","last_change":"2024-05-17T22:34:40.427221+0000","last_active":"2024-05-17T22:36:15.805642+0000","last_peered":"2024-05-17T22:36:15.805642+0000","last_clean":"2024-05-17T22:36:15.805642+0000","last_became_active":"2024-05-17T22:34:40.426703+0000","last_became_peered":"2024-05-17T22:34:40.426703+0000","last_unstale":"2024-05-17T22:36:15.805642+0000","last_undegraded":"2024-05-17T22:36:15.805642+0000","last_fullsized":"2024-05-17T22:36:15.805642+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:49:15.629048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.622645+0000","last_change":"2024-05-17T22:34:52.499086+0000","last_active":"2024-05-17T22:36:15.622645+0000","last_peered":"2024-05-17T22:36:15.622645+0000","last_clean":"2024-05-17T22:36:15.622645+0000","last_became_active":"2024-05-17T22:34:52.498666+0000","last_became_peered":"2024-05-17T22:34:52.498666+0000","last_unstale":"2024-05-17T22:36:15.622645+0000","last_undegraded":"2024-05-17T22:36:15.622645+0000","last_fullsized":"2024-05-17T22:36:15.622645+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:46:25.882964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.005012+0000","last_change":"2024-05-17T22:35:15.836294+0000","last_active":"2024-05-17T22:36:16.005012+0000","last_peered":"2024-05-17T22:36:16.005012+0000","last_clean":"2024-05-17T22:36:16.005012+0000","last_became_active":"2024-05-17T22:35:15.835759+0000","last_became_peered":"2024-05-17T22:35:15.835759+0000","last_unstale":"2024-05-17T22:36:16.005012+0000","last_undegraded":"2024-05-17T22:36:16.005012+0000","last_fullsized":"2024-05-17T22:36:16.005012+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:23:32.106692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968497+0000","last_change":"2024-05-17T22:34:40.433331+0000","last_active":"2024-05-17T22:36:15.968497+0000","last_peered":"2024-05-17T22:36:15.968497+0000","last_clean":"2024-05-17T22:36:15.968497+0000","last_became_active":"2024-05-17T22:34:40.432989+0000","last_became_peered":"2024-05-17T22:34:40.432989+0000","last_unstale":"2024-05-17T22:36:15.968497+0000","last_undegraded":"2024-05-17T22:36:15.968497+0000","last_fullsized":"2024-05-17T22:36:15.968497+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:22:49.146148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004182+0000","last_change":"2024-05-17T22:34:52.494418+0000","last_active":"2024-05-17T22:36:16.004182+0000","last_peered":"2024-05-17T22:36:16.004182+0000","last_clean":"2024-05-17T22:36:16.004182+0000","last_became_active":"2024-05-17T22:34:52.493953+0000","last_became_peered":"2024-05-17T22:34:52.493953+0000","last_unstale":"2024-05-17T22:36:16.004182+0000","last_undegraded":"2024-05-17T22:36:16.004182+0000","last_fullsized":"2024-05-17T22:36:16.004182+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:50:40.211770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003645+0000","last_change":"2024-05-17T22:34:37.442986+0000","last_active":"2024-05-17T22:36:16.003645+0000","last_peered":"2024-05-17T22:36:16.003645+0000","last_clean":"2024-05-17T22:36:16.003645+0000","last_became_active":"2024-05-17T22:34:37.442399+0000","last_became_peered":"2024-05-17T22:34:37.442399+0000","last_unstale":"2024-05-17T22:36:16.003645+0000","last_undegraded":"2024-05-17T22:36:16.003645+0000","last_fullsized":"2024-05-17T22:36:16.003645+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:34:05.785811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.805334+0000","last_change":"2024-05-17T22:34:40.426793+0000","last_active":"2024-05-17T22:36:15.805334+0000","last_peered":"2024-05-17T22:36:15.805334+0000","last_clean":"2024-05-17T22:36:15.805334+0000","last_became_active":"2024-05-17T22:34:40.426420+0000","last_became_peered":"2024-05-17T22:34:40.426420+0000","last_unstale":"2024-05-17T22:36:15.805334+0000","last_undegraded":"2024-05-17T22:36:15.805334+0000","last_fullsized":"2024-05-17T22:36:15.805334+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:57:07.099254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.622889+0000","last_change":"2024-05-17T22:34:52.498762+0000","last_active":"2024-05-17T22:36:15.622889+0000","last_peered":"2024-05-17T22:36:15.622889+0000","last_clean":"2024-05-17T22:36:15.622889+0000","last_became_active":"2024-05-17T22:34:52.498306+0000","last_became_peered":"2024-05-17T22:34:52.498306+0000","last_unstale":"2024-05-17T22:36:15.622889+0000","last_undegraded":"2024-05-17T22:36:15.622889+0000","last_fullsized":"2024-05-17T22:36:15.622889+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:18:32.717563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'33","reported_seq":146,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:19.088665+0000","last_change":"2024-05-17T22:34:37.407885+0000","last_active":"2024-05-17T22:36:19.088665+0000","last_peered":"2024-05-17T22:36:19.088665+0000","last_clean":"2024-05-17T22:36:19.088665+0000","last_became_active":"2024-05-17T22:34:37.407463+0000","last_became_peered":"2024-05-17T22:34:37.407463+0000","last_unstale":"2024-05-17T22:36:19.088665+0000","last_undegraded":"2024-05-17T22:36:19.088665+0000","last_fullsized":"2024-05-17T22:36:19.088665+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:26:20.288557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62072,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":69,"num_read_kb":95,"num_write":34,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T22:36:14.624480+0000","last_change":"2024-05-17T22:34:40.435800+0000","last_active":"2024-05-17T22:36:14.624480+0000","last_peered":"2024-05-17T22:36:14.624480+0000","last_clean":"2024-05-17T22:36:14.624480+0000","last_became_active":"2024-05-17T22:34:40.435245+0000","last_became_peered":"2024-05-17T22:34:40.435245+0000","last_unstale":"2024-05-17T22:36:14.624480+0000","last_undegraded":"2024-05-17T22:36:14.624480+0000","last_fullsized":"2024-05-17T22:36:14.624480+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:09:58.699026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004407+0000","last_change":"2024-05-17T22:34:52.487336+0000","last_active":"2024-05-17T22:36:16.004407+0000","last_peered":"2024-05-17T22:36:16.004407+0000","last_clean":"2024-05-17T22:36:16.004407+0000","last_became_active":"2024-05-17T22:34:52.486883+0000","last_became_peered":"2024-05-17T22:34:52.486883+0000","last_unstale":"2024-05-17T22:36:16.004407+0000","last_undegraded":"2024-05-17T22:36:16.004407+0000","last_fullsized":"2024-05-17T22:36:16.004407+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:44:41.264072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623955+0000","last_change":"2024-05-17T22:34:37.405044+0000","last_active":"2024-05-17T22:36:15.623955+0000","last_peered":"2024-05-17T22:36:15.623955+0000","last_clean":"2024-05-17T22:36:15.623955+0000","last_became_active":"2024-05-17T22:34:37.404610+0000","last_became_peered":"2024-05-17T22:34:37.404610+0000","last_unstale":"2024-05-17T22:36:15.623955+0000","last_undegraded":"2024-05-17T22:36:15.623955+0000","last_fullsized":"2024-05-17T22:36:15.623955+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:20:37.939069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685489+0000","last_change":"2024-05-17T22:34:40.430655+0000","last_active":"2024-05-17T22:36:15.685489+0000","last_peered":"2024-05-17T22:36:15.685489+0000","last_clean":"2024-05-17T22:36:15.685489+0000","last_became_active":"2024-05-17T22:34:40.430307+0000","last_became_peered":"2024-05-17T22:34:40.430307+0000","last_unstale":"2024-05-17T22:36:15.685489+0000","last_undegraded":"2024-05-17T22:36:15.685489+0000","last_fullsized":"2024-05-17T22:36:15.685489+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:57:19.036117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628122+0000","last_change":"2024-05-17T22:34:52.494709+0000","last_active":"2024-05-17T22:36:15.628122+0000","last_peered":"2024-05-17T22:36:15.628122+0000","last_clean":"2024-05-17T22:36:15.628122+0000","last_became_active":"2024-05-17T22:34:52.494195+0000","last_became_peered":"2024-05-17T22:34:52.494195+0000","last_unstale":"2024-05-17T22:36:15.628122+0000","last_undegraded":"2024-05-17T22:36:15.628122+0000","last_fullsized":"2024-05-17T22:36:15.628122+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:35:41.188459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003638+0000","last_change":"2024-05-17T22:34:37.402494+0000","last_active":"2024-05-17T22:36:16.003638+0000","last_peered":"2024-05-17T22:36:16.003638+0000","last_clean":"2024-05-17T22:36:16.003638+0000","last_became_active":"2024-05-17T22:34:37.402102+0000","last_became_peered":"2024-05-17T22:34:37.402102+0000","last_unstale":"2024-05-17T22:36:16.003638+0000","last_undegraded":"2024-05-17T22:36:16.003638+0000","last_fullsized":"2024-05-17T22:36:16.003638+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:22:30.915780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004954+0000","last_change":"2024-05-17T22:34:40.440731+0000","last_active":"2024-05-17T22:36:16.004954+0000","last_peered":"2024-05-17T22:36:16.004954+0000","last_clean":"2024-05-17T22:36:16.004954+0000","last_became_active":"2024-05-17T22:34:40.440374+0000","last_became_peered":"2024-05-17T22:34:40.440374+0000","last_unstale":"2024-05-17T22:36:16.004954+0000","last_undegraded":"2024-05-17T22:36:16.004954+0000","last_fullsized":"2024-05-17T22:36:16.004954+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:26:40.029155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":2,"reported_epoch":88,"state":"peering","last_fresh":"2024-05-17T22:34:52.479189+0000","last_change":"2024-05-17T22:36:15.566584+0000","last_active":"2024-05-17T22:34:51.460688+0000","last_peered":"2024-05-17T22:34:51.460688+0000","last_clean":"2024-05-17T22:34:51.460688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T22:34:52.479189+0000","last_undegraded":"2024-05-17T22:34:52.479189+0000","last_fullsized":"2024-05-17T22:34:52.479189+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"87'4","reported_seq":78,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.084097+0000","last_change":"2024-05-17T22:34:37.403123+0000","last_active":"2024-05-17T22:36:16.084097+0000","last_peered":"2024-05-17T22:36:16.084097+0000","last_clean":"2024-05-17T22:36:16.084097+0000","last_became_active":"2024-05-17T22:34:37.402702+0000","last_became_peered":"2024-05-17T22:34:37.402702+0000","last_unstale":"2024-05-17T22:36:16.084097+0000","last_undegraded":"2024-05-17T22:36:16.084097+0000","last_fullsized":"2024-05-17T22:36:16.084097+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:50:02.031651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":32,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003779+0000","last_change":"2024-05-17T22:34:40.431895+0000","last_active":"2024-05-17T22:36:16.003779+0000","last_peered":"2024-05-17T22:36:16.003779+0000","last_clean":"2024-05-17T22:36:16.003779+0000","last_became_active":"2024-05-17T22:34:40.431527+0000","last_became_peered":"2024-05-17T22:34:40.431527+0000","last_unstale":"2024-05-17T22:36:16.003779+0000","last_undegraded":"2024-05-17T22:36:16.003779+0000","last_fullsized":"2024-05-17T22:36:16.003779+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:30:09.404255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.686085+0000","last_change":"2024-05-17T22:34:52.495583+0000","last_active":"2024-05-17T22:36:15.686085+0000","last_peered":"2024-05-17T22:36:15.686085+0000","last_clean":"2024-05-17T22:36:15.686085+0000","last_became_active":"2024-05-17T22:34:52.495030+0000","last_became_peered":"2024-05-17T22:34:52.495030+0000","last_unstale":"2024-05-17T22:36:15.686085+0000","last_undegraded":"2024-05-17T22:36:15.686085+0000","last_fullsized":"2024-05-17T22:36:15.686085+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:04:54.046891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":58,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T22:36:14.623833+0000","last_change":"2024-05-17T22:34:37.442154+0000","last_active":"2024-05-17T22:36:14.623833+0000","last_peered":"2024-05-17T22:36:14.623833+0000","last_clean":"2024-05-17T22:36:14.623833+0000","last_became_active":"2024-05-17T22:34:37.441691+0000","last_became_peered":"2024-05-17T22:34:37.441691+0000","last_unstale":"2024-05-17T22:36:14.623833+0000","last_undegraded":"2024-05-17T22:36:14.623833+0000","last_fullsized":"2024-05-17T22:36:14.623833+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:47:29.862031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003475+0000","last_change":"2024-05-17T22:34:40.430894+0000","last_active":"2024-05-17T22:36:16.003475+0000","last_peered":"2024-05-17T22:36:16.003475+0000","last_clean":"2024-05-17T22:36:16.003475+0000","last_became_active":"2024-05-17T22:34:40.430328+0000","last_became_peered":"2024-05-17T22:34:40.430328+0000","last_unstale":"2024-05-17T22:36:16.003475+0000","last_undegraded":"2024-05-17T22:36:16.003475+0000","last_fullsized":"2024-05-17T22:36:16.003475+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:36:46.619111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003404+0000","last_change":"2024-05-17T22:34:52.487555+0000","last_active":"2024-05-17T22:36:16.003404+0000","last_peered":"2024-05-17T22:36:16.003404+0000","last_clean":"2024-05-17T22:36:16.003404+0000","last_became_active":"2024-05-17T22:34:52.487076+0000","last_became_peered":"2024-05-17T22:34:52.487076+0000","last_unstale":"2024-05-17T22:36:16.003404+0000","last_undegraded":"2024-05-17T22:36:16.003404+0000","last_fullsized":"2024-05-17T22:36:16.003404+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:05:46.720665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628312+0000","last_change":"2024-05-17T22:35:15.558860+0000","last_active":"2024-05-17T22:36:15.628312+0000","last_peered":"2024-05-17T22:36:15.628312+0000","last_clean":"2024-05-17T22:36:15.628312+0000","last_became_active":"2024-05-17T22:35:15.553683+0000","last_became_peered":"2024-05-17T22:35:15.553683+0000","last_unstale":"2024-05-17T22:36:15.628312+0000","last_undegraded":"2024-05-17T22:36:15.628312+0000","last_fullsized":"2024-05-17T22:36:15.628312+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:27:24.978282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628360+0000","last_change":"2024-05-17T22:34:40.433499+0000","last_active":"2024-05-17T22:36:15.628360+0000","last_peered":"2024-05-17T22:36:15.628360+0000","last_clean":"2024-05-17T22:36:15.628360+0000","last_became_active":"2024-05-17T22:34:40.431370+0000","last_became_peered":"2024-05-17T22:34:40.431370+0000","last_unstale":"2024-05-17T22:36:15.628360+0000","last_undegraded":"2024-05-17T22:36:15.628360+0000","last_fullsized":"2024-05-17T22:36:15.628360+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:25:08.383021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T22:36:14.624198+0000","last_change":"2024-05-17T22:34:52.489627+0000","last_active":"2024-05-17T22:36:14.624198+0000","last_peered":"2024-05-17T22:36:14.624198+0000","last_clean":"2024-05-17T22:36:14.624198+0000","last_became_active":"2024-05-17T22:34:52.489173+0000","last_became_peered":"2024-05-17T22:34:52.489173+0000","last_unstale":"2024-05-17T22:36:14.624198+0000","last_undegraded":"2024-05-17T22:36:14.624198+0000","last_fullsized":"2024-05-17T22:36:14.624198+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:52:26.908817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623301+0000","last_change":"2024-05-17T22:35:15.553325+0000","last_active":"2024-05-17T22:36:15.623301+0000","last_peered":"2024-05-17T22:36:15.623301+0000","last_clean":"2024-05-17T22:36:15.623301+0000","last_became_active":"2024-05-17T22:35:15.552867+0000","last_became_peered":"2024-05-17T22:35:15.552867+0000","last_unstale":"2024-05-17T22:36:15.623301+0000","last_undegraded":"2024-05-17T22:36:15.623301+0000","last_fullsized":"2024-05-17T22:36:15.623301+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:28:36.618331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.626212+0000","last_change":"2024-05-17T22:34:40.423221+0000","last_active":"2024-05-17T22:36:15.626212+0000","last_peered":"2024-05-17T22:36:15.626212+0000","last_clean":"2024-05-17T22:36:15.626212+0000","last_became_active":"2024-05-17T22:34:40.422838+0000","last_became_peered":"2024-05-17T22:34:40.422838+0000","last_unstale":"2024-05-17T22:36:15.626212+0000","last_undegraded":"2024-05-17T22:36:15.626212+0000","last_fullsized":"2024-05-17T22:36:15.626212+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:39:51.873212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685849+0000","last_change":"2024-05-17T22:34:52.484802+0000","last_active":"2024-05-17T22:36:15.685849+0000","last_peered":"2024-05-17T22:36:15.685849+0000","last_clean":"2024-05-17T22:36:15.685849+0000","last_became_active":"2024-05-17T22:34:52.483766+0000","last_became_peered":"2024-05-17T22:34:52.483766+0000","last_unstale":"2024-05-17T22:36:15.685849+0000","last_undegraded":"2024-05-17T22:36:15.685849+0000","last_fullsized":"2024-05-17T22:36:15.685849+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:32:01.427078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004410+0000","last_change":"2024-05-17T22:34:37.404797+0000","last_active":"2024-05-17T22:36:16.004410+0000","last_peered":"2024-05-17T22:36:16.004410+0000","last_clean":"2024-05-17T22:36:16.004410+0000","last_became_active":"2024-05-17T22:34:37.404393+0000","last_became_peered":"2024-05-17T22:34:37.404393+0000","last_unstale":"2024-05-17T22:36:16.004410+0000","last_undegraded":"2024-05-17T22:36:16.004410+0000","last_fullsized":"2024-05-17T22:36:16.004410+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:26:36.887919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625922+0000","last_change":"2024-05-17T22:34:40.428210+0000","last_active":"2024-05-17T22:36:15.625922+0000","last_peered":"2024-05-17T22:36:15.625922+0000","last_clean":"2024-05-17T22:36:15.625922+0000","last_became_active":"2024-05-17T22:34:40.427845+0000","last_became_peered":"2024-05-17T22:34:40.427845+0000","last_unstale":"2024-05-17T22:36:15.625922+0000","last_undegraded":"2024-05-17T22:36:15.625922+0000","last_fullsized":"2024-05-17T22:36:15.625922+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:45:30.437139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685982+0000","last_change":"2024-05-17T22:34:52.490502+0000","last_active":"2024-05-17T22:36:15.685982+0000","last_peered":"2024-05-17T22:36:15.685982+0000","last_clean":"2024-05-17T22:36:15.685982+0000","last_became_active":"2024-05-17T22:34:52.490008+0000","last_became_peered":"2024-05-17T22:34:52.490008+0000","last_unstale":"2024-05-17T22:36:15.685982+0000","last_undegraded":"2024-05-17T22:36:15.685982+0000","last_fullsized":"2024-05-17T22:36:15.685982+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:58:29.122668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003400+0000","last_change":"2024-05-17T22:34:40.432248+0000","last_active":"2024-05-17T22:36:16.003400+0000","last_peered":"2024-05-17T22:36:16.003400+0000","last_clean":"2024-05-17T22:36:16.003400+0000","last_became_active":"2024-05-17T22:34:40.431863+0000","last_became_peered":"2024-05-17T22:34:40.431863+0000","last_unstale":"2024-05-17T22:36:16.003400+0000","last_undegraded":"2024-05-17T22:36:16.003400+0000","last_fullsized":"2024-05-17T22:36:16.003400+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:20:34.730542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'4","reported_seq":77,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:17.084645+0000","last_change":"2024-05-17T22:34:37.441186+0000","last_active":"2024-05-17T22:36:17.084645+0000","last_peered":"2024-05-17T22:36:17.084645+0000","last_clean":"2024-05-17T22:36:17.084645+0000","last_became_active":"2024-05-17T22:34:37.440781+0000","last_became_peered":"2024-05-17T22:34:37.440781+0000","last_unstale":"2024-05-17T22:36:17.084645+0000","last_undegraded":"2024-05-17T22:36:17.084645+0000","last_fullsized":"2024-05-17T22:36:17.084645+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:44:34.435147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.005029+0000","last_change":"2024-05-17T22:34:52.491771+0000","last_active":"2024-05-17T22:36:16.005029+0000","last_peered":"2024-05-17T22:36:16.005029+0000","last_clean":"2024-05-17T22:36:16.005029+0000","last_became_active":"2024-05-17T22:34:52.491172+0000","last_became_peered":"2024-05-17T22:34:52.491172+0000","last_unstale":"2024-05-17T22:36:16.005029+0000","last_undegraded":"2024-05-17T22:36:16.005029+0000","last_fullsized":"2024-05-17T22:36:16.005029+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:15:21.010592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984305+0000","last_change":"2024-05-17T22:34:40.430860+0000","last_active":"2024-05-17T22:36:15.984305+0000","last_peered":"2024-05-17T22:36:15.984305+0000","last_clean":"2024-05-17T22:36:15.984305+0000","last_became_active":"2024-05-17T22:34:40.430542+0000","last_became_peered":"2024-05-17T22:34:40.430542+0000","last_unstale":"2024-05-17T22:36:15.984305+0000","last_undegraded":"2024-05-17T22:36:15.984305+0000","last_fullsized":"2024-05-17T22:36:15.984305+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:16:26.257380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004059+0000","last_change":"2024-05-17T22:34:37.403026+0000","last_active":"2024-05-17T22:36:16.004059+0000","last_peered":"2024-05-17T22:36:16.004059+0000","last_clean":"2024-05-17T22:36:16.004059+0000","last_became_active":"2024-05-17T22:34:37.402560+0000","last_became_peered":"2024-05-17T22:34:37.402560+0000","last_unstale":"2024-05-17T22:36:16.004059+0000","last_undegraded":"2024-05-17T22:36:16.004059+0000","last_fullsized":"2024-05-17T22:36:16.004059+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:23:03.784411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":119,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003503+0000","last_change":"2024-05-17T22:33:11.055878+0000","last_active":"2024-05-17T22:36:16.003503+0000","last_peered":"2024-05-17T22:36:16.003503+0000","last_clean":"2024-05-17T22:36:16.003503+0000","last_became_active":"2024-05-17T22:33:11.050409+0000","last_became_peered":"2024-05-17T22:33:11.050409+0000","last_unstale":"2024-05-17T22:36:16.003503+0000","last_undegraded":"2024-05-17T22:36:16.003503+0000","last_fullsized":"2024-05-17T22:36:16.003503+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:29:43.341170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:29:43.341170+0000","last_clean_scrub_stamp":"2024-05-17T22:29:43.341170+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:30:54.153865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003650+0000","last_change":"2024-05-17T22:34:52.497064+0000","last_active":"2024-05-17T22:36:16.003650+0000","last_peered":"2024-05-17T22:36:16.003650+0000","last_clean":"2024-05-17T22:36:16.003650+0000","last_became_active":"2024-05-17T22:34:52.496611+0000","last_became_peered":"2024-05-17T22:34:52.496611+0000","last_unstale":"2024-05-17T22:36:16.003650+0000","last_undegraded":"2024-05-17T22:36:16.003650+0000","last_fullsized":"2024-05-17T22:36:16.003650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:20:29.597589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685209+0000","last_change":"2024-05-17T22:35:15.555413+0000","last_active":"2024-05-17T22:36:15.685209+0000","last_peered":"2024-05-17T22:36:15.685209+0000","last_clean":"2024-05-17T22:36:15.685209+0000","last_became_active":"2024-05-17T22:35:15.555005+0000","last_became_peered":"2024-05-17T22:35:15.555005+0000","last_unstale":"2024-05-17T22:36:15.685209+0000","last_undegraded":"2024-05-17T22:36:15.685209+0000","last_fullsized":"2024-05-17T22:36:15.685209+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:51:49.860673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003644+0000","last_change":"2024-05-17T22:34:37.406546+0000","last_active":"2024-05-17T22:36:16.003644+0000","last_peered":"2024-05-17T22:36:16.003644+0000","last_clean":"2024-05-17T22:36:16.003644+0000","last_became_active":"2024-05-17T22:34:37.406118+0000","last_became_peered":"2024-05-17T22:34:37.406118+0000","last_unstale":"2024-05-17T22:36:16.003644+0000","last_undegraded":"2024-05-17T22:36:16.003644+0000","last_fullsized":"2024-05-17T22:36:16.003644+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:47:16.350091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.005647+0000","last_change":"2024-05-17T22:36:16.005647+0000","last_active":"2024-05-17T22:36:16.005647+0000","last_peered":"2024-05-17T22:36:16.005647+0000","last_clean":"2024-05-17T22:36:16.005647+0000","last_became_active":"2024-05-17T22:36:16.005128+0000","last_became_peered":"2024-05-17T22:36:16.005128+0000","last_unstale":"2024-05-17T22:36:16.005647+0000","last_undegraded":"2024-05-17T22:36:16.005647+0000","last_fullsized":"2024-05-17T22:36:16.005647+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:16:24.481236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,7,6],"acting":[11,10,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T22:36:14.624604+0000","last_change":"2024-05-17T22:34:37.442676+0000","last_active":"2024-05-17T22:36:14.624604+0000","last_peered":"2024-05-17T22:36:14.624604+0000","last_clean":"2024-05-17T22:36:14.624604+0000","last_became_active":"2024-05-17T22:34:37.442275+0000","last_became_peered":"2024-05-17T22:34:37.442275+0000","last_unstale":"2024-05-17T22:36:14.624604+0000","last_undegraded":"2024-05-17T22:36:14.624604+0000","last_fullsized":"2024-05-17T22:36:14.624604+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:54:18.542987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685755+0000","last_change":"2024-05-17T22:34:40.424266+0000","last_active":"2024-05-17T22:36:15.685755+0000","last_peered":"2024-05-17T22:36:15.685755+0000","last_clean":"2024-05-17T22:36:15.685755+0000","last_became_active":"2024-05-17T22:34:40.423905+0000","last_became_peered":"2024-05-17T22:34:40.423905+0000","last_unstale":"2024-05-17T22:36:15.685755+0000","last_undegraded":"2024-05-17T22:36:15.685755+0000","last_fullsized":"2024-05-17T22:36:15.685755+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:03:43.397871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984103+0000","last_change":"2024-05-17T22:34:52.493386+0000","last_active":"2024-05-17T22:36:15.984103+0000","last_peered":"2024-05-17T22:36:15.984103+0000","last_clean":"2024-05-17T22:36:15.984103+0000","last_became_active":"2024-05-17T22:34:52.492997+0000","last_became_peered":"2024-05-17T22:34:52.492997+0000","last_unstale":"2024-05-17T22:36:15.984103+0000","last_undegraded":"2024-05-17T22:36:15.984103+0000","last_fullsized":"2024-05-17T22:36:15.984103+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:14:40.818271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'11","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623586+0000","last_change":"2024-05-17T22:34:37.399397+0000","last_active":"2024-05-17T22:36:15.623586+0000","last_peered":"2024-05-17T22:36:15.623586+0000","last_clean":"2024-05-17T22:36:15.623586+0000","last_became_active":"2024-05-17T22:34:37.399062+0000","last_became_peered":"2024-05-17T22:34:37.399062+0000","last_unstale":"2024-05-17T22:36:15.623586+0000","last_undegraded":"2024-05-17T22:36:15.623586+0000","last_fullsized":"2024-05-17T22:36:15.623586+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:02:05.230580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984853+0000","last_change":"2024-05-17T22:34:40.426995+0000","last_active":"2024-05-17T22:36:15.984853+0000","last_peered":"2024-05-17T22:36:15.984853+0000","last_clean":"2024-05-17T22:36:15.984853+0000","last_became_active":"2024-05-17T22:34:40.426226+0000","last_became_peered":"2024-05-17T22:34:40.426226+0000","last_unstale":"2024-05-17T22:36:15.984853+0000","last_undegraded":"2024-05-17T22:36:15.984853+0000","last_fullsized":"2024-05-17T22:36:15.984853+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:49:53.331076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623635+0000","last_change":"2024-05-17T22:34:52.487098+0000","last_active":"2024-05-17T22:36:15.623635+0000","last_peered":"2024-05-17T22:36:15.623635+0000","last_clean":"2024-05-17T22:36:15.623635+0000","last_became_active":"2024-05-17T22:34:52.486605+0000","last_became_peered":"2024-05-17T22:34:52.486605+0000","last_unstale":"2024-05-17T22:36:15.623635+0000","last_undegraded":"2024-05-17T22:36:15.623635+0000","last_fullsized":"2024-05-17T22:36:15.623635+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:33:28.234208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004630+0000","last_change":"2024-05-17T22:34:37.800830+0000","last_active":"2024-05-17T22:36:16.004630+0000","last_peered":"2024-05-17T22:36:16.004630+0000","last_clean":"2024-05-17T22:36:16.004630+0000","last_became_active":"2024-05-17T22:34:37.800405+0000","last_became_peered":"2024-05-17T22:34:37.800405+0000","last_unstale":"2024-05-17T22:36:16.004630+0000","last_undegraded":"2024-05-17T22:36:16.004630+0000","last_fullsized":"2024-05-17T22:36:16.004630+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:19:10.087442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968862+0000","last_change":"2024-05-17T22:34:40.426307+0000","last_active":"2024-05-17T22:36:15.968862+0000","last_peered":"2024-05-17T22:36:15.968862+0000","last_clean":"2024-05-17T22:36:15.968862+0000","last_became_active":"2024-05-17T22:34:40.425969+0000","last_became_peered":"2024-05-17T22:34:40.425969+0000","last_unstale":"2024-05-17T22:36:15.968862+0000","last_undegraded":"2024-05-17T22:36:15.968862+0000","last_fullsized":"2024-05-17T22:36:15.968862+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:25:49.680886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004451+0000","last_change":"2024-05-17T22:35:15.556857+0000","last_active":"2024-05-17T22:36:16.004451+0000","last_peered":"2024-05-17T22:36:16.004451+0000","last_clean":"2024-05-17T22:36:16.004451+0000","last_became_active":"2024-05-17T22:35:15.556298+0000","last_became_peered":"2024-05-17T22:35:15.556298+0000","last_unstale":"2024-05-17T22:36:16.004451+0000","last_undegraded":"2024-05-17T22:36:16.004451+0000","last_fullsized":"2024-05-17T22:36:16.004451+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:23:20.111544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.805120+0000","last_change":"2024-05-17T22:34:37.442971+0000","last_active":"2024-05-17T22:36:15.805120+0000","last_peered":"2024-05-17T22:36:15.805120+0000","last_clean":"2024-05-17T22:36:15.805120+0000","last_became_active":"2024-05-17T22:34:37.442536+0000","last_became_peered":"2024-05-17T22:34:37.442536+0000","last_unstale":"2024-05-17T22:36:15.805120+0000","last_undegraded":"2024-05-17T22:36:15.805120+0000","last_fullsized":"2024-05-17T22:36:15.805120+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:03:45.224071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T22:36:14.623867+0000","last_change":"2024-05-17T22:34:40.432056+0000","last_active":"2024-05-17T22:36:14.623867+0000","last_peered":"2024-05-17T22:36:14.623867+0000","last_clean":"2024-05-17T22:36:14.623867+0000","last_became_active":"2024-05-17T22:34:40.431626+0000","last_became_peered":"2024-05-17T22:34:40.431626+0000","last_unstale":"2024-05-17T22:36:14.623867+0000","last_undegraded":"2024-05-17T22:36:14.623867+0000","last_fullsized":"2024-05-17T22:36:14.623867+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:12:24.650746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623590+0000","last_change":"2024-05-17T22:34:52.500451+0000","last_active":"2024-05-17T22:36:15.623590+0000","last_peered":"2024-05-17T22:36:15.623590+0000","last_clean":"2024-05-17T22:36:15.623590+0000","last_became_active":"2024-05-17T22:34:52.500031+0000","last_became_peered":"2024-05-17T22:34:52.500031+0000","last_unstale":"2024-05-17T22:36:15.623590+0000","last_undegraded":"2024-05-17T22:36:15.623590+0000","last_fullsized":"2024-05-17T22:36:15.623590+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:54:36.825116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984349+0000","last_change":"2024-05-17T22:34:37.402335+0000","last_active":"2024-05-17T22:36:15.984349+0000","last_peered":"2024-05-17T22:36:15.984349+0000","last_clean":"2024-05-17T22:36:15.984349+0000","last_became_active":"2024-05-17T22:34:37.401982+0000","last_became_peered":"2024-05-17T22:34:37.401982+0000","last_unstale":"2024-05-17T22:36:15.984349+0000","last_undegraded":"2024-05-17T22:36:15.984349+0000","last_fullsized":"2024-05-17T22:36:15.984349+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:58:06.194335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.627919+0000","last_change":"2024-05-17T22:34:40.425524+0000","last_active":"2024-05-17T22:36:15.627919+0000","last_peered":"2024-05-17T22:36:15.627919+0000","last_clean":"2024-05-17T22:36:15.627919+0000","last_became_active":"2024-05-17T22:34:40.425135+0000","last_became_peered":"2024-05-17T22:34:40.425135+0000","last_unstale":"2024-05-17T22:36:15.627919+0000","last_undegraded":"2024-05-17T22:36:15.627919+0000","last_fullsized":"2024-05-17T22:36:15.627919+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:41:14.176477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004602+0000","last_change":"2024-05-17T22:34:52.496290+0000","last_active":"2024-05-17T22:36:16.004602+0000","last_peered":"2024-05-17T22:36:16.004602+0000","last_clean":"2024-05-17T22:36:16.004602+0000","last_became_active":"2024-05-17T22:34:52.495817+0000","last_became_peered":"2024-05-17T22:34:52.495817+0000","last_unstale":"2024-05-17T22:36:16.004602+0000","last_undegraded":"2024-05-17T22:36:16.004602+0000","last_fullsized":"2024-05-17T22:36:16.004602+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:47:22.056372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'1","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623936+0000","last_change":"2024-05-17T22:34:37.404234+0000","last_active":"2024-05-17T22:36:15.623936+0000","last_peered":"2024-05-17T22:36:15.623936+0000","last_clean":"2024-05-17T22:36:15.623936+0000","last_became_active":"2024-05-17T22:34:37.403561+0000","last_became_peered":"2024-05-17T22:34:37.403561+0000","last_unstale":"2024-05-17T22:36:15.623936+0000","last_undegraded":"2024-05-17T22:36:15.623936+0000","last_fullsized":"2024-05-17T22:36:15.623936+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:30:28.432641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984498+0000","last_change":"2024-05-17T22:34:40.426522+0000","last_active":"2024-05-17T22:36:15.984498+0000","last_peered":"2024-05-17T22:36:15.984498+0000","last_clean":"2024-05-17T22:36:15.984498+0000","last_became_active":"2024-05-17T22:34:40.426139+0000","last_became_peered":"2024-05-17T22:34:40.426139+0000","last_unstale":"2024-05-17T22:36:15.984498+0000","last_undegraded":"2024-05-17T22:36:15.984498+0000","last_fullsized":"2024-05-17T22:36:15.984498+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:30:19.503513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004016+0000","last_change":"2024-05-17T22:34:52.487131+0000","last_active":"2024-05-17T22:36:16.004016+0000","last_peered":"2024-05-17T22:36:16.004016+0000","last_clean":"2024-05-17T22:36:16.004016+0000","last_became_active":"2024-05-17T22:34:52.486654+0000","last_became_peered":"2024-05-17T22:34:52.486654+0000","last_unstale":"2024-05-17T22:36:16.004016+0000","last_undegraded":"2024-05-17T22:36:16.004016+0000","last_fullsized":"2024-05-17T22:36:16.004016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:28:25.041746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T22:36:14.624452+0000","last_change":"2024-05-17T22:34:37.405020+0000","last_active":"2024-05-17T22:36:14.624452+0000","last_peered":"2024-05-17T22:36:14.624452+0000","last_clean":"2024-05-17T22:36:14.624452+0000","last_became_active":"2024-05-17T22:34:37.404633+0000","last_became_peered":"2024-05-17T22:34:37.404633+0000","last_unstale":"2024-05-17T22:36:14.624452+0000","last_undegraded":"2024-05-17T22:36:14.624452+0000","last_fullsized":"2024-05-17T22:36:14.624452+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:06:00.001190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004601+0000","last_change":"2024-05-17T22:34:40.435374+0000","last_active":"2024-05-17T22:36:16.004601+0000","last_peered":"2024-05-17T22:36:16.004601+0000","last_clean":"2024-05-17T22:36:16.004601+0000","last_became_active":"2024-05-17T22:34:40.434522+0000","last_became_peered":"2024-05-17T22:34:40.434522+0000","last_unstale":"2024-05-17T22:36:16.004601+0000","last_undegraded":"2024-05-17T22:36:16.004601+0000","last_fullsized":"2024-05-17T22:36:16.004601+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:36:41.244970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.805127+0000","last_change":"2024-05-17T22:34:52.491708+0000","last_active":"2024-05-17T22:36:15.805127+0000","last_peered":"2024-05-17T22:36:15.805127+0000","last_clean":"2024-05-17T22:36:15.805127+0000","last_became_active":"2024-05-17T22:34:52.491293+0000","last_became_peered":"2024-05-17T22:34:52.491293+0000","last_unstale":"2024-05-17T22:36:15.805127+0000","last_undegraded":"2024-05-17T22:36:15.805127+0000","last_fullsized":"2024-05-17T22:36:15.805127+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:23:01.634026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624618+0000","last_change":"2024-05-17T22:34:37.800525+0000","last_active":"2024-05-17T22:36:15.624618+0000","last_peered":"2024-05-17T22:36:15.624618+0000","last_clean":"2024-05-17T22:36:15.624618+0000","last_became_active":"2024-05-17T22:34:37.800093+0000","last_became_peered":"2024-05-17T22:34:37.800093+0000","last_unstale":"2024-05-17T22:36:15.624618+0000","last_undegraded":"2024-05-17T22:36:15.624618+0000","last_fullsized":"2024-05-17T22:36:15.624618+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:25:27.249889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.005319+0000","last_change":"2024-05-17T22:34:40.434891+0000","last_active":"2024-05-17T22:36:16.005319+0000","last_peered":"2024-05-17T22:36:16.005319+0000","last_clean":"2024-05-17T22:36:16.005319+0000","last_became_active":"2024-05-17T22:34:40.434543+0000","last_became_peered":"2024-05-17T22:34:40.434543+0000","last_unstale":"2024-05-17T22:36:16.005319+0000","last_undegraded":"2024-05-17T22:36:16.005319+0000","last_fullsized":"2024-05-17T22:36:16.005319+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:17:47.770225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968349+0000","last_change":"2024-05-17T22:34:52.491544+0000","last_active":"2024-05-17T22:36:15.968349+0000","last_peered":"2024-05-17T22:36:15.968349+0000","last_clean":"2024-05-17T22:36:15.968349+0000","last_became_active":"2024-05-17T22:34:52.491138+0000","last_became_peered":"2024-05-17T22:34:52.491138+0000","last_unstale":"2024-05-17T22:36:15.968349+0000","last_undegraded":"2024-05-17T22:36:15.968349+0000","last_fullsized":"2024-05-17T22:36:15.968349+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:34:20.411299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004214+0000","last_change":"2024-05-17T22:34:37.403220+0000","last_active":"2024-05-17T22:36:16.004214+0000","last_peered":"2024-05-17T22:36:16.004214+0000","last_clean":"2024-05-17T22:36:16.004214+0000","last_became_active":"2024-05-17T22:34:37.402856+0000","last_became_peered":"2024-05-17T22:34:37.402856+0000","last_unstale":"2024-05-17T22:36:16.004214+0000","last_undegraded":"2024-05-17T22:36:16.004214+0000","last_fullsized":"2024-05-17T22:36:16.004214+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:04:19.469294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.005034+0000","last_change":"2024-05-17T22:34:40.434487+0000","last_active":"2024-05-17T22:36:16.005034+0000","last_peered":"2024-05-17T22:36:16.005034+0000","last_clean":"2024-05-17T22:36:16.005034+0000","last_became_active":"2024-05-17T22:34:40.434160+0000","last_became_peered":"2024-05-17T22:34:40.434160+0000","last_unstale":"2024-05-17T22:36:16.005034+0000","last_undegraded":"2024-05-17T22:36:16.005034+0000","last_fullsized":"2024-05-17T22:36:16.005034+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:24:38.782999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003123+0000","last_change":"2024-05-17T22:34:52.487887+0000","last_active":"2024-05-17T22:36:16.003123+0000","last_peered":"2024-05-17T22:36:16.003123+0000","last_clean":"2024-05-17T22:36:16.003123+0000","last_became_active":"2024-05-17T22:34:52.486837+0000","last_became_peered":"2024-05-17T22:34:52.486837+0000","last_unstale":"2024-05-17T22:36:16.003123+0000","last_undegraded":"2024-05-17T22:36:16.003123+0000","last_fullsized":"2024-05-17T22:36:16.003123+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:12:22.206536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.686166+0000","last_change":"2024-05-17T22:34:37.399498+0000","last_active":"2024-05-17T22:36:15.686166+0000","last_peered":"2024-05-17T22:36:15.686166+0000","last_clean":"2024-05-17T22:36:15.686166+0000","last_became_active":"2024-05-17T22:34:37.399144+0000","last_became_peered":"2024-05-17T22:34:37.399144+0000","last_unstale":"2024-05-17T22:36:15.686166+0000","last_undegraded":"2024-05-17T22:36:15.686166+0000","last_fullsized":"2024-05-17T22:36:15.686166+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:13:37.346006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.005111+0000","last_change":"2024-05-17T22:34:40.433839+0000","last_active":"2024-05-17T22:36:16.005111+0000","last_peered":"2024-05-17T22:36:16.005111+0000","last_clean":"2024-05-17T22:36:16.005111+0000","last_became_active":"2024-05-17T22:34:40.433489+0000","last_became_peered":"2024-05-17T22:34:40.433489+0000","last_unstale":"2024-05-17T22:36:16.005111+0000","last_undegraded":"2024-05-17T22:36:16.005111+0000","last_fullsized":"2024-05-17T22:36:16.005111+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:13:56.014094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628026+0000","last_change":"2024-05-17T22:34:40.434953+0000","last_active":"2024-05-17T22:36:15.628026+0000","last_peered":"2024-05-17T22:36:15.628026+0000","last_clean":"2024-05-17T22:36:15.628026+0000","last_became_active":"2024-05-17T22:34:40.434593+0000","last_became_peered":"2024-05-17T22:34:40.434593+0000","last_unstale":"2024-05-17T22:36:15.628026+0000","last_undegraded":"2024-05-17T22:36:15.628026+0000","last_fullsized":"2024-05-17T22:36:15.628026+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:20:27.075382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623850+0000","last_change":"2024-05-17T22:34:40.432787+0000","last_active":"2024-05-17T22:36:15.623850+0000","last_peered":"2024-05-17T22:36:15.623850+0000","last_clean":"2024-05-17T22:36:15.623850+0000","last_became_active":"2024-05-17T22:34:40.432402+0000","last_became_peered":"2024-05-17T22:34:40.432402+0000","last_unstale":"2024-05-17T22:36:15.623850+0000","last_undegraded":"2024-05-17T22:36:15.623850+0000","last_fullsized":"2024-05-17T22:36:15.623850+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:48:13.034276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623724+0000","last_change":"2024-05-17T22:34:40.426423+0000","last_active":"2024-05-17T22:36:15.623724+0000","last_peered":"2024-05-17T22:36:15.623724+0000","last_clean":"2024-05-17T22:36:15.623724+0000","last_became_active":"2024-05-17T22:34:40.426049+0000","last_became_peered":"2024-05-17T22:34:40.426049+0000","last_unstale":"2024-05-17T22:36:15.623724+0000","last_undegraded":"2024-05-17T22:36:15.623724+0000","last_fullsized":"2024-05-17T22:36:15.623724+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:56:55.631322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968631+0000","last_change":"2024-05-17T22:34:40.435706+0000","last_active":"2024-05-17T22:36:15.968631+0000","last_peered":"2024-05-17T22:36:15.968631+0000","last_clean":"2024-05-17T22:36:15.968631+0000","last_became_active":"2024-05-17T22:34:40.435330+0000","last_became_peered":"2024-05-17T22:34:40.435330+0000","last_unstale":"2024-05-17T22:36:15.968631+0000","last_undegraded":"2024-05-17T22:36:15.968631+0000","last_fullsized":"2024-05-17T22:36:15.968631+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:25:36.529454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685705+0000","last_change":"2024-05-17T22:34:40.428997+0000","last_active":"2024-05-17T22:36:15.685705+0000","last_peered":"2024-05-17T22:36:15.685705+0000","last_clean":"2024-05-17T22:36:15.685705+0000","last_became_active":"2024-05-17T22:34:40.428594+0000","last_became_peered":"2024-05-17T22:34:40.428594+0000","last_unstale":"2024-05-17T22:36:15.685705+0000","last_undegraded":"2024-05-17T22:36:15.685705+0000","last_fullsized":"2024-05-17T22:36:15.685705+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:59:01.777751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628370+0000","last_change":"2024-05-17T22:34:40.434241+0000","last_active":"2024-05-17T22:36:15.628370+0000","last_peered":"2024-05-17T22:36:15.628370+0000","last_clean":"2024-05-17T22:36:15.628370+0000","last_became_active":"2024-05-17T22:34:40.431652+0000","last_became_peered":"2024-05-17T22:34:40.431652+0000","last_unstale":"2024-05-17T22:36:15.628370+0000","last_undegraded":"2024-05-17T22:36:15.628370+0000","last_fullsized":"2024-05-17T22:36:15.628370+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:33:27.035284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624649+0000","last_change":"2024-05-17T22:34:40.431485+0000","last_active":"2024-05-17T22:36:15.624649+0000","last_peered":"2024-05-17T22:36:15.624649+0000","last_clean":"2024-05-17T22:36:15.624649+0000","last_became_active":"2024-05-17T22:34:40.431120+0000","last_became_peered":"2024-05-17T22:34:40.431120+0000","last_unstale":"2024-05-17T22:36:15.624649+0000","last_undegraded":"2024-05-17T22:36:15.624649+0000","last_fullsized":"2024-05-17T22:36:15.624649+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:46:55.213602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984947+0000","last_change":"2024-05-17T22:34:40.423117+0000","last_active":"2024-05-17T22:36:15.984947+0000","last_peered":"2024-05-17T22:36:15.984947+0000","last_clean":"2024-05-17T22:36:15.984947+0000","last_became_active":"2024-05-17T22:34:40.422694+0000","last_became_peered":"2024-05-17T22:34:40.422694+0000","last_unstale":"2024-05-17T22:36:15.984947+0000","last_undegraded":"2024-05-17T22:36:15.984947+0000","last_fullsized":"2024-05-17T22:36:15.984947+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:19:55.589532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685170+0000","last_change":"2024-05-17T22:35:15.711796+0000","last_active":"2024-05-17T22:36:15.685170+0000","last_peered":"2024-05-17T22:36:15.685170+0000","last_clean":"2024-05-17T22:36:15.685170+0000","last_became_active":"2024-05-17T22:35:15.711327+0000","last_became_peered":"2024-05-17T22:35:15.711327+0000","last_unstale":"2024-05-17T22:36:15.685170+0000","last_undegraded":"2024-05-17T22:36:15.685170+0000","last_fullsized":"2024-05-17T22:36:15.685170+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:48:48.913312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984270+0000","last_change":"2024-05-17T22:34:40.430393+0000","last_active":"2024-05-17T22:36:15.984270+0000","last_peered":"2024-05-17T22:36:15.984270+0000","last_clean":"2024-05-17T22:36:15.984270+0000","last_became_active":"2024-05-17T22:34:40.430030+0000","last_became_peered":"2024-05-17T22:34:40.430030+0000","last_unstale":"2024-05-17T22:36:15.984270+0000","last_undegraded":"2024-05-17T22:36:15.984270+0000","last_fullsized":"2024-05-17T22:36:15.984270+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:39:37.204246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.805401+0000","last_change":"2024-05-17T22:34:40.432443+0000","last_active":"2024-05-17T22:36:15.805401+0000","last_peered":"2024-05-17T22:36:15.805401+0000","last_clean":"2024-05-17T22:36:15.805401+0000","last_became_active":"2024-05-17T22:34:40.431991+0000","last_became_peered":"2024-05-17T22:34:40.431991+0000","last_unstale":"2024-05-17T22:36:15.805401+0000","last_undegraded":"2024-05-17T22:36:15.805401+0000","last_fullsized":"2024-05-17T22:36:15.805401+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:37:45.593575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685767+0000","last_change":"2024-05-17T22:34:40.434388+0000","last_active":"2024-05-17T22:36:15.685767+0000","last_peered":"2024-05-17T22:36:15.685767+0000","last_clean":"2024-05-17T22:36:15.685767+0000","last_became_active":"2024-05-17T22:34:40.434044+0000","last_became_peered":"2024-05-17T22:34:40.434044+0000","last_unstale":"2024-05-17T22:36:15.685767+0000","last_undegraded":"2024-05-17T22:36:15.685767+0000","last_fullsized":"2024-05-17T22:36:15.685767+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:20:43.296190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628303+0000","last_change":"2024-05-17T22:34:40.427137+0000","last_active":"2024-05-17T22:36:15.628303+0000","last_peered":"2024-05-17T22:36:15.628303+0000","last_clean":"2024-05-17T22:36:15.628303+0000","last_became_active":"2024-05-17T22:34:40.426703+0000","last_became_peered":"2024-05-17T22:34:40.426703+0000","last_unstale":"2024-05-17T22:36:15.628303+0000","last_undegraded":"2024-05-17T22:36:15.628303+0000","last_fullsized":"2024-05-17T22:36:15.628303+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:55:52.992817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66464,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":204,"num_read_kb":174,"num_write":169,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":190862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710755,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1028096,"data_stored":515036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906985,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28692,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716460,"statfs":{"total":95995035648,"available":95965655040,"internally_reserved":0,"allocated":1712128,"data_stored":1164745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070508,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":519346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266739,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1019904,"data_stored":505782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462968,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1634304,"data_stored":1111470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626492,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":510426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1081344,"data_stored":568143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":511814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1105920,"data_stored":569884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1032192,"data_stored":510620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986247,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1032192,"data_stored":515467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822723,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36944,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708208,"statfs":{"total":95995035648,"available":95957204992,"internally_reserved":0,"allocated":1642496,"data_stored":1106667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":57807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":59548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T22:36:21.650 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-17T22:36:21.942 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:36:21.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:21 smithi151 ceph-mon[36101]: pgmap v352: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-05-17T22:36:21.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:21 smithi151 ceph-mon[36101]: from='client.14871 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T22:36:22.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:21 smithi096 ceph-mon[36065]: pgmap v352: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-05-17T22:36:22.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:21 smithi096 ceph-mon[36065]: from='client.14871 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T22:36:22.065 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:21 smithi012 ceph-mon[31632]: pgmap v352: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-05-17T22:36:22.065 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:21 smithi012 ceph-mon[31632]: from='client.14871 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T22:36:22.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:22 smithi151 ceph-mon[36101]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-05-17T22:36:22.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:22 smithi151 ceph-mon[36101]: Cluster is now healthy 2024-05-17T22:36:23.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:22 smithi096 ceph-mon[36065]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-05-17T22:36:23.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:22 smithi096 ceph-mon[36065]: Cluster is now healthy 2024-05-17T22:36:23.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:22 smithi012 ceph-mon[31632]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-05-17T22:36:23.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:22 smithi012 ceph-mon[31632]: Cluster is now healthy 2024-05-17T22:36:23.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.670+0000 7f504b366700 1 -- 172.21.15.12:0/112328752 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5044108840 msgr2=0x7f5044108c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:23.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.670+0000 7f504b366700 1 --2- 172.21.15.12:0/112328752 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5044108840 0x7f5044108c50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5038009990 tx=0x7f5038009ca0 comp rx=0 tx=0).stop 2024-05-17T22:36:23.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.670+0000 7f504b366700 1 -- 172.21.15.12:0/112328752 shutdown_connections 2024-05-17T22:36:23.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.670+0000 7f504b366700 1 --2- 172.21.15.12:0/112328752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5044109190 0x7f5044110af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:23.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.670+0000 7f504b366700 1 --2- 172.21.15.12:0/112328752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5044103f10 0x7f5044104320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:23.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.670+0000 7f504b366700 1 --2- 172.21.15.12:0/112328752 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5044108840 0x7f5044108c50 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:23.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.670+0000 7f504b366700 1 -- 172.21.15.12:0/112328752 >> 172.21.15.12:0/112328752 conn(0x7f50440ff990 msgr2=0x7f5044101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:36:23.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.670+0000 7f504b366700 1 -- 172.21.15.12:0/112328752 shutdown_connections 2024-05-17T22:36:23.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.670+0000 7f504b366700 1 -- 172.21.15.12:0/112328752 wait complete. 2024-05-17T22:36:23.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.671+0000 7f504b366700 1 Processor -- start 2024-05-17T22:36:23.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.671+0000 7f504b366700 1 -- start start 2024-05-17T22:36:23.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.672+0000 7f504b366700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5044103f10 0x7f50441a04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:23.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.672+0000 7f504b366700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5044108840 0x7f50441a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:23.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.672+0000 7f504b366700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5044109190 0x7f504419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:23.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.672+0000 7f504b366700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5044113de0 con 0x7f5044109190 2024-05-17T22:36:23.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.672+0000 7f504b366700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5044113c60 con 0x7f5044108840 2024-05-17T22:36:23.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.672+0000 7f504b366700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5044113f60 con 0x7f5044103f10 2024-05-17T22:36:23.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.672+0000 7f5049903700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5044109190 0x7f504419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:23.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.672+0000 7f5049903700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5044109190 0x7f504419a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60306/0 (socket says 172.21.15.12:60306) 2024-05-17T22:36:23.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.672+0000 7f5049903700 1 -- 172.21.15.12:0/2343163791 learned_addr learned my addr 172.21.15.12:0/2343163791 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:36:23.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.672+0000 7f5049102700 1 --2- 172.21.15.12:0/2343163791 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5044103f10 0x7f50441a04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:23.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.672+0000 7f5048901700 1 --2- 172.21.15.12:0/2343163791 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5044108840 0x7f50441a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:23.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.672+0000 7f5049903700 1 -- 172.21.15.12:0/2343163791 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5044103f10 msgr2=0x7f50441a04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:23.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.672+0000 7f5049903700 1 --2- 172.21.15.12:0/2343163791 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5044103f10 0x7f50441a04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:23.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.672+0000 7f5049903700 1 -- 172.21.15.12:0/2343163791 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5044108840 msgr2=0x7f50441a0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:23.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.672+0000 7f5049903700 1 --2- 172.21.15.12:0/2343163791 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5044108840 0x7f50441a0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:23.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.672+0000 7f5049903700 1 -- 172.21.15.12:0/2343163791 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f504419ada0 con 0x7f5044109190 2024-05-17T22:36:23.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.673+0000 7f5049903700 1 --2- 172.21.15.12:0/2343163791 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5044109190 0x7f504419a670 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f503c00c970 tx=0x7f503c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:36:23.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.673+0000 7f50427fc700 1 -- 172.21.15.12:0/2343163791 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503c004a30 con 0x7f5044109190 2024-05-17T22:36:23.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.673+0000 7f504b366700 1 -- 172.21.15.12:0/2343163791 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f504419b090 con 0x7f5044109190 2024-05-17T22:36:23.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.673+0000 7f50427fc700 1 -- 172.21.15.12:0/2343163791 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f503c0054d0 con 0x7f5044109190 2024-05-17T22:36:23.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.673+0000 7f50427fc700 1 -- 172.21.15.12:0/2343163791 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503c00ec20 con 0x7f5044109190 2024-05-17T22:36:23.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.673+0000 7f504b366700 1 -- 172.21.15.12:0/2343163791 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5044075a60 con 0x7f5044109190 2024-05-17T22:36:23.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.674+0000 7f504b366700 1 -- 172.21.15.12:0/2343163791 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50441053c0 con 0x7f5044109190 2024-05-17T22:36:23.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.676+0000 7f50427fc700 1 -- 172.21.15.12:0/2343163791 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f503c004fc0 con 0x7f5044109190 2024-05-17T22:36:23.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.676+0000 7f50427fc700 1 --2- 172.21.15.12:0/2343163791 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f502c0b2910 0x7f502c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:23.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.676+0000 7f50427fc700 1 -- 172.21.15.12:0/2343163791 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f503c0e2580 con 0x7f5044109190 2024-05-17T22:36:23.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.676+0000 7f5049102700 1 --2- 172.21.15.12:0/2343163791 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f502c0b2910 0x7f502c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:23.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.677+0000 7f5049102700 1 --2- 172.21.15.12:0/2343163791 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f502c0b2910 0x7f502c0b4e20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5034003f90 tx=0x7f5034009180 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:36:23.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.680+0000 7f50427fc700 1 -- 172.21.15.12:0/2343163791 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f503c014030 con 0x7f5044109190 2024-05-17T22:36:23.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.891+0000 7f504b366700 1 -- 172.21.15.12:0/2343163791 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5044104320 con 0x7f502c0b2910 2024-05-17T22:36:23.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.897+0000 7f50427fc700 1 -- 172.21.15.12:0/2343163791 <== mgr.14152 v2:172.21.15.12:6800/1388514047 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340372 (secure 0 0 0) 0x7f5044104320 con 0x7f502c0b2910 2024-05-17T22:36:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:36:23.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.900+0000 7f504b366700 1 -- 172.21.15.12:0/2343163791 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f502c0b2910 msgr2=0x7f502c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:23.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.900+0000 7f504b366700 1 --2- 172.21.15.12:0/2343163791 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f502c0b2910 0x7f502c0b4e20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5034003f90 tx=0x7f5034009180 comp rx=0 tx=0).stop 2024-05-17T22:36:23.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.900+0000 7f504b366700 1 -- 172.21.15.12:0/2343163791 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5044109190 msgr2=0x7f504419a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:23.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.900+0000 7f504b366700 1 --2- 172.21.15.12:0/2343163791 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5044109190 0x7f504419a670 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f503c00c970 tx=0x7f503c00cc80 comp rx=0 tx=0).stop 2024-05-17T22:36:23.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.900+0000 7f504b366700 1 -- 172.21.15.12:0/2343163791 shutdown_connections 2024-05-17T22:36:23.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.900+0000 7f504b366700 1 --2- 172.21.15.12:0/2343163791 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f502c0b2910 0x7f502c0b4e20 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:23.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.900+0000 7f504b366700 1 --2- 172.21.15.12:0/2343163791 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5044109190 0x7f504419a670 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:23.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.900+0000 7f504b366700 1 --2- 172.21.15.12:0/2343163791 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5044103f10 0x7f50441a04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:23.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.900+0000 7f504b366700 1 --2- 172.21.15.12:0/2343163791 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5044108840 0x7f50441a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:23.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.900+0000 7f504b366700 1 -- 172.21.15.12:0/2343163791 >> 172.21.15.12:0/2343163791 conn(0x7f50440ff990 msgr2=0x7f5044101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:36:23.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.901+0000 7f504b366700 1 -- 172.21.15.12:0/2343163791 shutdown_connections 2024-05-17T22:36:23.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:23.901+0000 7f504b366700 1 -- 172.21.15.12:0/2343163791 wait complete. 2024-05-17T22:36:23.908 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-05-17T22:36:23.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:23 smithi151 ceph-mon[36101]: pgmap v353: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-17T22:36:24.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:23 smithi096 ceph-mon[36065]: pgmap v353: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-17T22:36:24.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:23 smithi012 ceph-mon[31632]: pgmap v353: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-17T22:36:24.659 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":353,"stamp":"2024-05-17T22:36:22.505398+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657066,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":280,"num_read_kb":238,"num_write":294,"num_write_kb":2270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":238,"ondisk_log_size":238,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":384100,"kb_used_data":14080,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124557724,"statfs":{"total":1151940427776,"available":1151547109376,"internally_reserved":0,"allocated":14417920,"data_stored":8123031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9083,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":47,"num_read_kb":35,"num_write":6,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002240"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628725+0000","last_change":"2024-05-17T22:34:40.422850+0000","last_active":"2024-05-17T22:36:15.628725+0000","last_peered":"2024-05-17T22:36:15.628725+0000","last_clean":"2024-05-17T22:36:15.628725+0000","last_became_active":"2024-05-17T22:34:40.422474+0000","last_became_peered":"2024-05-17T22:34:40.422474+0000","last_unstale":"2024-05-17T22:36:15.628725+0000","last_undegraded":"2024-05-17T22:36:15.628725+0000","last_fullsized":"2024-05-17T22:36:15.628725+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:23:51.537128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968741+0000","last_change":"2024-05-17T22:34:40.427907+0000","last_active":"2024-05-17T22:36:15.968741+0000","last_peered":"2024-05-17T22:36:15.968741+0000","last_clean":"2024-05-17T22:36:15.968741+0000","last_became_active":"2024-05-17T22:34:40.427511+0000","last_became_peered":"2024-05-17T22:34:40.427511+0000","last_unstale":"2024-05-17T22:36:15.968741+0000","last_undegraded":"2024-05-17T22:36:15.968741+0000","last_fullsized":"2024-05-17T22:36:15.968741+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:01:46.707820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968443+0000","last_change":"2024-05-17T22:34:40.436370+0000","last_active":"2024-05-17T22:36:15.968443+0000","last_peered":"2024-05-17T22:36:15.968443+0000","last_clean":"2024-05-17T22:36:15.968443+0000","last_became_active":"2024-05-17T22:34:40.434089+0000","last_became_peered":"2024-05-17T22:34:40.434089+0000","last_unstale":"2024-05-17T22:36:15.968443+0000","last_undegraded":"2024-05-17T22:36:15.968443+0000","last_fullsized":"2024-05-17T22:36:15.968443+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:58:16.775471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623409+0000","last_change":"2024-05-17T22:35:15.556681+0000","last_active":"2024-05-17T22:36:15.623409+0000","last_peered":"2024-05-17T22:36:15.623409+0000","last_clean":"2024-05-17T22:36:15.623409+0000","last_became_active":"2024-05-17T22:35:15.556427+0000","last_became_peered":"2024-05-17T22:35:15.556427+0000","last_unstale":"2024-05-17T22:36:15.623409+0000","last_undegraded":"2024-05-17T22:36:15.623409+0000","last_fullsized":"2024-05-17T22:36:15.623409+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:28:01.726262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625720+0000","last_change":"2024-05-17T22:34:40.434394+0000","last_active":"2024-05-17T22:36:15.625720+0000","last_peered":"2024-05-17T22:36:15.625720+0000","last_clean":"2024-05-17T22:36:15.625720+0000","last_became_active":"2024-05-17T22:34:40.433489+0000","last_became_peered":"2024-05-17T22:34:40.433489+0000","last_unstale":"2024-05-17T22:36:15.625720+0000","last_undegraded":"2024-05-17T22:36:15.625720+0000","last_fullsized":"2024-05-17T22:36:15.625720+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:26:24.068814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625953+0000","last_change":"2024-05-17T22:35:15.553601+0000","last_active":"2024-05-17T22:36:15.625953+0000","last_peered":"2024-05-17T22:36:15.625953+0000","last_clean":"2024-05-17T22:36:15.625953+0000","last_became_active":"2024-05-17T22:35:15.553116+0000","last_became_peered":"2024-05-17T22:35:15.553116+0000","last_unstale":"2024-05-17T22:36:15.625953+0000","last_undegraded":"2024-05-17T22:36:15.625953+0000","last_fullsized":"2024-05-17T22:36:15.625953+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:09:38.401107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004274+0000","last_change":"2024-05-17T22:34:40.430479+0000","last_active":"2024-05-17T22:36:16.004274+0000","last_peered":"2024-05-17T22:36:16.004274+0000","last_clean":"2024-05-17T22:36:16.004274+0000","last_became_active":"2024-05-17T22:34:40.430025+0000","last_became_peered":"2024-05-17T22:34:40.430025+0000","last_unstale":"2024-05-17T22:36:16.004274+0000","last_undegraded":"2024-05-17T22:36:16.004274+0000","last_fullsized":"2024-05-17T22:36:16.004274+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:14:19.725096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984193+0000","last_change":"2024-05-17T22:34:40.431711+0000","last_active":"2024-05-17T22:36:15.984193+0000","last_peered":"2024-05-17T22:36:15.984193+0000","last_clean":"2024-05-17T22:36:15.984193+0000","last_became_active":"2024-05-17T22:34:40.431368+0000","last_became_peered":"2024-05-17T22:34:40.431368+0000","last_unstale":"2024-05-17T22:36:15.984193+0000","last_undegraded":"2024-05-17T22:36:15.984193+0000","last_fullsized":"2024-05-17T22:36:15.984193+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:06:34.904652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.805730+0000","last_change":"2024-05-17T22:34:40.435593+0000","last_active":"2024-05-17T22:36:15.805730+0000","last_peered":"2024-05-17T22:36:15.805730+0000","last_clean":"2024-05-17T22:36:15.805730+0000","last_became_active":"2024-05-17T22:34:40.435241+0000","last_became_peered":"2024-05-17T22:34:40.435241+0000","last_unstale":"2024-05-17T22:36:15.805730+0000","last_undegraded":"2024-05-17T22:36:15.805730+0000","last_fullsized":"2024-05-17T22:36:15.805730+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:08:57.239998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.626366+0000","last_change":"2024-05-17T22:34:40.436248+0000","last_active":"2024-05-17T22:36:15.626366+0000","last_peered":"2024-05-17T22:36:15.626366+0000","last_clean":"2024-05-17T22:36:15.626366+0000","last_became_active":"2024-05-17T22:34:40.435824+0000","last_became_peered":"2024-05-17T22:34:40.435824+0000","last_unstale":"2024-05-17T22:36:15.626366+0000","last_undegraded":"2024-05-17T22:36:15.626366+0000","last_fullsized":"2024-05-17T22:36:15.626366+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:39:07.297470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.005145+0000","last_change":"2024-05-17T22:34:40.435843+0000","last_active":"2024-05-17T22:36:16.005145+0000","last_peered":"2024-05-17T22:36:16.005145+0000","last_clean":"2024-05-17T22:36:16.005145+0000","last_became_active":"2024-05-17T22:34:40.435500+0000","last_became_peered":"2024-05-17T22:34:40.435500+0000","last_unstale":"2024-05-17T22:36:16.005145+0000","last_undegraded":"2024-05-17T22:36:16.005145+0000","last_fullsized":"2024-05-17T22:36:16.005145+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:11:28.115850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004758+0000","last_change":"2024-05-17T22:34:40.432044+0000","last_active":"2024-05-17T22:36:16.004758+0000","last_peered":"2024-05-17T22:36:16.004758+0000","last_clean":"2024-05-17T22:36:16.004758+0000","last_became_active":"2024-05-17T22:34:40.429141+0000","last_became_peered":"2024-05-17T22:34:40.429141+0000","last_unstale":"2024-05-17T22:36:16.004758+0000","last_undegraded":"2024-05-17T22:36:16.004758+0000","last_fullsized":"2024-05-17T22:36:16.004758+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:04:53.428681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685924+0000","last_change":"2024-05-17T22:34:40.438474+0000","last_active":"2024-05-17T22:36:15.685924+0000","last_peered":"2024-05-17T22:36:15.685924+0000","last_clean":"2024-05-17T22:36:15.685924+0000","last_became_active":"2024-05-17T22:34:40.438109+0000","last_became_peered":"2024-05-17T22:34:40.438109+0000","last_unstale":"2024-05-17T22:36:15.685924+0000","last_undegraded":"2024-05-17T22:36:15.685924+0000","last_fullsized":"2024-05-17T22:36:15.685924+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:26:00.665246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968785+0000","last_change":"2024-05-17T22:34:40.431563+0000","last_active":"2024-05-17T22:36:15.968785+0000","last_peered":"2024-05-17T22:36:15.968785+0000","last_clean":"2024-05-17T22:36:15.968785+0000","last_became_active":"2024-05-17T22:34:40.430992+0000","last_became_peered":"2024-05-17T22:34:40.430992+0000","last_unstale":"2024-05-17T22:36:15.968785+0000","last_undegraded":"2024-05-17T22:36:15.968785+0000","last_fullsized":"2024-05-17T22:36:15.968785+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:51:26.280440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968328+0000","last_change":"2024-05-17T22:34:40.433629+0000","last_active":"2024-05-17T22:36:15.968328+0000","last_peered":"2024-05-17T22:36:15.968328+0000","last_clean":"2024-05-17T22:36:15.968328+0000","last_became_active":"2024-05-17T22:34:40.433270+0000","last_became_peered":"2024-05-17T22:34:40.433270+0000","last_unstale":"2024-05-17T22:36:15.968328+0000","last_undegraded":"2024-05-17T22:36:15.968328+0000","last_fullsized":"2024-05-17T22:36:15.968328+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:57:51.164078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.805030+0000","last_change":"2024-05-17T22:34:40.434218+0000","last_active":"2024-05-17T22:36:15.805030+0000","last_peered":"2024-05-17T22:36:15.805030+0000","last_clean":"2024-05-17T22:36:15.805030+0000","last_became_active":"2024-05-17T22:34:40.433849+0000","last_became_peered":"2024-05-17T22:34:40.433849+0000","last_unstale":"2024-05-17T22:36:15.805030+0000","last_undegraded":"2024-05-17T22:36:15.805030+0000","last_fullsized":"2024-05-17T22:36:15.805030+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:41:55.984109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628531+0000","last_change":"2024-05-17T22:34:40.427892+0000","last_active":"2024-05-17T22:36:15.628531+0000","last_peered":"2024-05-17T22:36:15.628531+0000","last_clean":"2024-05-17T22:36:15.628531+0000","last_became_active":"2024-05-17T22:34:40.427303+0000","last_became_peered":"2024-05-17T22:34:40.427303+0000","last_unstale":"2024-05-17T22:36:15.628531+0000","last_undegraded":"2024-05-17T22:36:15.628531+0000","last_fullsized":"2024-05-17T22:36:15.628531+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:59:38.632123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623636+0000","last_change":"2024-05-17T22:34:40.424784+0000","last_active":"2024-05-17T22:36:15.623636+0000","last_peered":"2024-05-17T22:36:15.623636+0000","last_clean":"2024-05-17T22:36:15.623636+0000","last_became_active":"2024-05-17T22:34:40.424439+0000","last_became_peered":"2024-05-17T22:34:40.424439+0000","last_unstale":"2024-05-17T22:36:15.623636+0000","last_undegraded":"2024-05-17T22:36:15.623636+0000","last_fullsized":"2024-05-17T22:36:15.623636+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:02:53.681600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624433+0000","last_change":"2024-05-17T22:34:40.435023+0000","last_active":"2024-05-17T22:36:15.624433+0000","last_peered":"2024-05-17T22:36:15.624433+0000","last_clean":"2024-05-17T22:36:15.624433+0000","last_became_active":"2024-05-17T22:34:40.434441+0000","last_became_peered":"2024-05-17T22:34:40.434441+0000","last_unstale":"2024-05-17T22:36:15.624433+0000","last_undegraded":"2024-05-17T22:36:15.624433+0000","last_fullsized":"2024-05-17T22:36:15.624433+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:16:07.671897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623973+0000","last_change":"2024-05-17T22:34:52.493192+0000","last_active":"2024-05-17T22:36:15.623973+0000","last_peered":"2024-05-17T22:36:15.623973+0000","last_clean":"2024-05-17T22:36:15.623973+0000","last_became_active":"2024-05-17T22:34:52.492786+0000","last_became_peered":"2024-05-17T22:34:52.492786+0000","last_unstale":"2024-05-17T22:36:15.623973+0000","last_undegraded":"2024-05-17T22:36:15.623973+0000","last_fullsized":"2024-05-17T22:36:15.623973+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:43:40.152240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623754+0000","last_change":"2024-05-17T22:34:37.402360+0000","last_active":"2024-05-17T22:36:15.623754+0000","last_peered":"2024-05-17T22:36:15.623754+0000","last_clean":"2024-05-17T22:36:15.623754+0000","last_became_active":"2024-05-17T22:34:37.401882+0000","last_became_peered":"2024-05-17T22:34:37.401882+0000","last_unstale":"2024-05-17T22:36:15.623754+0000","last_undegraded":"2024-05-17T22:36:15.623754+0000","last_fullsized":"2024-05-17T22:36:15.623754+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:53:13.300626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984571+0000","last_change":"2024-05-17T22:34:40.436259+0000","last_active":"2024-05-17T22:36:15.984571+0000","last_peered":"2024-05-17T22:36:15.984571+0000","last_clean":"2024-05-17T22:36:15.984571+0000","last_became_active":"2024-05-17T22:34:40.435899+0000","last_became_peered":"2024-05-17T22:34:40.435899+0000","last_unstale":"2024-05-17T22:36:15.984571+0000","last_undegraded":"2024-05-17T22:36:15.984571+0000","last_fullsized":"2024-05-17T22:36:15.984571+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:26:29.327513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003655+0000","last_change":"2024-05-17T22:34:52.495236+0000","last_active":"2024-05-17T22:36:16.003655+0000","last_peered":"2024-05-17T22:36:16.003655+0000","last_clean":"2024-05-17T22:36:16.003655+0000","last_became_active":"2024-05-17T22:34:52.494783+0000","last_became_peered":"2024-05-17T22:34:52.494783+0000","last_unstale":"2024-05-17T22:36:16.003655+0000","last_undegraded":"2024-05-17T22:36:16.003655+0000","last_fullsized":"2024-05-17T22:36:16.003655+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:08:15.010706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984767+0000","last_change":"2024-05-17T22:34:37.441129+0000","last_active":"2024-05-17T22:36:15.984767+0000","last_peered":"2024-05-17T22:36:15.984767+0000","last_clean":"2024-05-17T22:36:15.984767+0000","last_became_active":"2024-05-17T22:34:37.440738+0000","last_became_peered":"2024-05-17T22:34:37.440738+0000","last_unstale":"2024-05-17T22:36:15.984767+0000","last_undegraded":"2024-05-17T22:36:15.984767+0000","last_fullsized":"2024-05-17T22:36:15.984767+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:03:16.945076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968555+0000","last_change":"2024-05-17T22:34:40.434146+0000","last_active":"2024-05-17T22:36:15.968555+0000","last_peered":"2024-05-17T22:36:15.968555+0000","last_clean":"2024-05-17T22:36:15.968555+0000","last_became_active":"2024-05-17T22:34:40.433802+0000","last_became_peered":"2024-05-17T22:34:40.433802+0000","last_unstale":"2024-05-17T22:36:15.968555+0000","last_undegraded":"2024-05-17T22:36:15.968555+0000","last_fullsized":"2024-05-17T22:36:15.968555+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:08:39.905409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003713+0000","last_change":"2024-05-17T22:34:52.501134+0000","last_active":"2024-05-17T22:36:16.003713+0000","last_peered":"2024-05-17T22:36:16.003713+0000","last_clean":"2024-05-17T22:36:16.003713+0000","last_became_active":"2024-05-17T22:34:52.500547+0000","last_became_peered":"2024-05-17T22:34:52.500547+0000","last_unstale":"2024-05-17T22:36:16.003713+0000","last_undegraded":"2024-05-17T22:36:16.003713+0000","last_fullsized":"2024-05-17T22:36:16.003713+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:13:04.200865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.626343+0000","last_change":"2024-05-17T22:34:37.404019+0000","last_active":"2024-05-17T22:36:15.626343+0000","last_peered":"2024-05-17T22:36:15.626343+0000","last_clean":"2024-05-17T22:36:15.626343+0000","last_became_active":"2024-05-17T22:34:37.403607+0000","last_became_peered":"2024-05-17T22:34:37.403607+0000","last_unstale":"2024-05-17T22:36:15.626343+0000","last_undegraded":"2024-05-17T22:36:15.626343+0000","last_fullsized":"2024-05-17T22:36:15.626343+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:30:21.405195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.805310+0000","last_change":"2024-05-17T22:34:40.426889+0000","last_active":"2024-05-17T22:36:15.805310+0000","last_peered":"2024-05-17T22:36:15.805310+0000","last_clean":"2024-05-17T22:36:15.805310+0000","last_became_active":"2024-05-17T22:34:40.426512+0000","last_became_peered":"2024-05-17T22:34:40.426512+0000","last_unstale":"2024-05-17T22:36:15.805310+0000","last_undegraded":"2024-05-17T22:36:15.805310+0000","last_fullsized":"2024-05-17T22:36:15.805310+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:30:51.363177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003876+0000","last_change":"2024-05-17T22:34:52.487514+0000","last_active":"2024-05-17T22:36:16.003876+0000","last_peered":"2024-05-17T22:36:16.003876+0000","last_clean":"2024-05-17T22:36:16.003876+0000","last_became_active":"2024-05-17T22:34:52.487068+0000","last_became_peered":"2024-05-17T22:34:52.487068+0000","last_unstale":"2024-05-17T22:36:16.003876+0000","last_undegraded":"2024-05-17T22:36:16.003876+0000","last_fullsized":"2024-05-17T22:36:16.003876+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:52:13.573986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.626041+0000","last_change":"2024-05-17T22:34:37.400518+0000","last_active":"2024-05-17T22:36:15.626041+0000","last_peered":"2024-05-17T22:36:15.626041+0000","last_clean":"2024-05-17T22:36:15.626041+0000","last_became_active":"2024-05-17T22:34:37.400145+0000","last_became_peered":"2024-05-17T22:34:37.400145+0000","last_unstale":"2024-05-17T22:36:15.626041+0000","last_undegraded":"2024-05-17T22:36:15.626041+0000","last_fullsized":"2024-05-17T22:36:15.626041+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:56:53.593624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004156+0000","last_change":"2024-05-17T22:34:40.434148+0000","last_active":"2024-05-17T22:36:16.004156+0000","last_peered":"2024-05-17T22:36:16.004156+0000","last_clean":"2024-05-17T22:36:16.004156+0000","last_became_active":"2024-05-17T22:34:40.433613+0000","last_became_peered":"2024-05-17T22:34:40.433613+0000","last_unstale":"2024-05-17T22:36:16.004156+0000","last_undegraded":"2024-05-17T22:36:16.004156+0000","last_fullsized":"2024-05-17T22:36:16.004156+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:09:39.049852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624284+0000","last_change":"2024-05-17T22:34:52.490417+0000","last_active":"2024-05-17T22:36:15.624284+0000","last_peered":"2024-05-17T22:36:15.624284+0000","last_clean":"2024-05-17T22:36:15.624284+0000","last_became_active":"2024-05-17T22:34:52.489953+0000","last_became_peered":"2024-05-17T22:34:52.489953+0000","last_unstale":"2024-05-17T22:36:15.624284+0000","last_undegraded":"2024-05-17T22:36:15.624284+0000","last_fullsized":"2024-05-17T22:36:15.624284+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:23:56.491899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":81,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.083741+0000","last_change":"2024-05-17T22:34:37.405304+0000","last_active":"2024-05-17T22:36:16.083741+0000","last_peered":"2024-05-17T22:36:16.083741+0000","last_clean":"2024-05-17T22:36:16.083741+0000","last_became_active":"2024-05-17T22:34:37.404552+0000","last_became_peered":"2024-05-17T22:34:37.404552+0000","last_unstale":"2024-05-17T22:36:16.083741+0000","last_undegraded":"2024-05-17T22:36:16.083741+0000","last_fullsized":"2024-05-17T22:36:16.083741+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:57:06.556886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":31,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625732+0000","last_change":"2024-05-17T22:34:40.430537+0000","last_active":"2024-05-17T22:36:15.625732+0000","last_peered":"2024-05-17T22:36:15.625732+0000","last_clean":"2024-05-17T22:36:15.625732+0000","last_became_active":"2024-05-17T22:34:40.430193+0000","last_became_peered":"2024-05-17T22:34:40.430193+0000","last_unstale":"2024-05-17T22:36:15.625732+0000","last_undegraded":"2024-05-17T22:36:15.625732+0000","last_fullsized":"2024-05-17T22:36:15.625732+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:58:42.091445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624333+0000","last_change":"2024-05-17T22:35:15.559077+0000","last_active":"2024-05-17T22:36:15.624333+0000","last_peered":"2024-05-17T22:36:15.624333+0000","last_clean":"2024-05-17T22:36:15.624333+0000","last_became_active":"2024-05-17T22:35:15.558382+0000","last_became_peered":"2024-05-17T22:35:15.558382+0000","last_unstale":"2024-05-17T22:36:15.624333+0000","last_undegraded":"2024-05-17T22:36:15.624333+0000","last_fullsized":"2024-05-17T22:36:15.624333+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:20:16.009620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625951+0000","last_change":"2024-05-17T22:34:37.405332+0000","last_active":"2024-05-17T22:36:15.625951+0000","last_peered":"2024-05-17T22:36:15.625951+0000","last_clean":"2024-05-17T22:36:15.625951+0000","last_became_active":"2024-05-17T22:34:37.404970+0000","last_became_peered":"2024-05-17T22:34:37.404970+0000","last_unstale":"2024-05-17T22:36:15.625951+0000","last_undegraded":"2024-05-17T22:36:15.625951+0000","last_fullsized":"2024-05-17T22:36:15.625951+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:37:39.933686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003790+0000","last_change":"2024-05-17T22:34:40.432690+0000","last_active":"2024-05-17T22:36:16.003790+0000","last_peered":"2024-05-17T22:36:16.003790+0000","last_clean":"2024-05-17T22:36:16.003790+0000","last_became_active":"2024-05-17T22:34:40.432274+0000","last_became_peered":"2024-05-17T22:34:40.432274+0000","last_unstale":"2024-05-17T22:36:16.003790+0000","last_undegraded":"2024-05-17T22:36:16.003790+0000","last_fullsized":"2024-05-17T22:36:16.003790+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:12:47.926865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628515+0000","last_change":"2024-05-17T22:34:52.494645+0000","last_active":"2024-05-17T22:36:15.628515+0000","last_peered":"2024-05-17T22:36:15.628515+0000","last_clean":"2024-05-17T22:36:15.628515+0000","last_became_active":"2024-05-17T22:34:52.494255+0000","last_became_peered":"2024-05-17T22:34:52.494255+0000","last_unstale":"2024-05-17T22:36:15.628515+0000","last_undegraded":"2024-05-17T22:36:15.628515+0000","last_fullsized":"2024-05-17T22:36:15.628515+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:41:26.633571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685352+0000","last_change":"2024-05-17T22:34:37.398582+0000","last_active":"2024-05-17T22:36:15.685352+0000","last_peered":"2024-05-17T22:36:15.685352+0000","last_clean":"2024-05-17T22:36:15.685352+0000","last_became_active":"2024-05-17T22:34:37.398156+0000","last_became_peered":"2024-05-17T22:34:37.398156+0000","last_unstale":"2024-05-17T22:36:15.685352+0000","last_undegraded":"2024-05-17T22:36:15.685352+0000","last_fullsized":"2024-05-17T22:36:15.685352+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:46:00.276214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.622638+0000","last_change":"2024-05-17T22:35:15.551402+0000","last_active":"2024-05-17T22:36:15.622638+0000","last_peered":"2024-05-17T22:36:15.622638+0000","last_clean":"2024-05-17T22:36:15.622638+0000","last_became_active":"2024-05-17T22:35:15.551021+0000","last_became_peered":"2024-05-17T22:35:15.551021+0000","last_unstale":"2024-05-17T22:36:15.622638+0000","last_undegraded":"2024-05-17T22:36:15.622638+0000","last_fullsized":"2024-05-17T22:36:15.622638+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:35:18.573620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968732+0000","last_change":"2024-05-17T22:34:52.487413+0000","last_active":"2024-05-17T22:36:15.968732+0000","last_peered":"2024-05-17T22:36:15.968732+0000","last_clean":"2024-05-17T22:36:15.968732+0000","last_became_active":"2024-05-17T22:34:52.487032+0000","last_became_peered":"2024-05-17T22:34:52.487032+0000","last_unstale":"2024-05-17T22:36:15.968732+0000","last_undegraded":"2024-05-17T22:36:15.968732+0000","last_fullsized":"2024-05-17T22:36:15.968732+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:34:49.122008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003249+0000","last_change":"2024-05-17T22:34:37.404894+0000","last_active":"2024-05-17T22:36:16.003249+0000","last_peered":"2024-05-17T22:36:16.003249+0000","last_clean":"2024-05-17T22:36:16.003249+0000","last_became_active":"2024-05-17T22:34:37.404567+0000","last_became_peered":"2024-05-17T22:34:37.404567+0000","last_unstale":"2024-05-17T22:36:16.003249+0000","last_undegraded":"2024-05-17T22:36:16.003249+0000","last_fullsized":"2024-05-17T22:36:16.003249+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:13:09.761556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623009+0000","last_change":"2024-05-17T22:34:40.418267+0000","last_active":"2024-05-17T22:36:15.623009+0000","last_peered":"2024-05-17T22:36:15.623009+0000","last_clean":"2024-05-17T22:36:15.623009+0000","last_became_active":"2024-05-17T22:34:40.417928+0000","last_became_peered":"2024-05-17T22:34:40.417928+0000","last_unstale":"2024-05-17T22:36:15.623009+0000","last_undegraded":"2024-05-17T22:36:15.623009+0000","last_fullsized":"2024-05-17T22:36:15.623009+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:17:37.055004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984148+0000","last_change":"2024-05-17T22:34:52.492126+0000","last_active":"2024-05-17T22:36:15.984148+0000","last_peered":"2024-05-17T22:36:15.984148+0000","last_clean":"2024-05-17T22:36:15.984148+0000","last_became_active":"2024-05-17T22:34:52.491695+0000","last_became_peered":"2024-05-17T22:34:52.491695+0000","last_unstale":"2024-05-17T22:36:15.984148+0000","last_undegraded":"2024-05-17T22:36:15.984148+0000","last_fullsized":"2024-05-17T22:36:15.984148+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:15:17.710279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":77,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:18.086300+0000","last_change":"2024-05-17T22:34:37.406989+0000","last_active":"2024-05-17T22:36:18.086300+0000","last_peered":"2024-05-17T22:36:18.086300+0000","last_clean":"2024-05-17T22:36:18.086300+0000","last_became_active":"2024-05-17T22:34:37.406612+0000","last_became_peered":"2024-05-17T22:34:37.406612+0000","last_unstale":"2024-05-17T22:36:18.086300+0000","last_undegraded":"2024-05-17T22:36:18.086300+0000","last_fullsized":"2024-05-17T22:36:18.086300+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:47:52.480804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.805642+0000","last_change":"2024-05-17T22:34:40.427221+0000","last_active":"2024-05-17T22:36:15.805642+0000","last_peered":"2024-05-17T22:36:15.805642+0000","last_clean":"2024-05-17T22:36:15.805642+0000","last_became_active":"2024-05-17T22:34:40.426703+0000","last_became_peered":"2024-05-17T22:34:40.426703+0000","last_unstale":"2024-05-17T22:36:15.805642+0000","last_undegraded":"2024-05-17T22:36:15.805642+0000","last_fullsized":"2024-05-17T22:36:15.805642+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:49:15.629048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.622645+0000","last_change":"2024-05-17T22:34:52.499086+0000","last_active":"2024-05-17T22:36:15.622645+0000","last_peered":"2024-05-17T22:36:15.622645+0000","last_clean":"2024-05-17T22:36:15.622645+0000","last_became_active":"2024-05-17T22:34:52.498666+0000","last_became_peered":"2024-05-17T22:34:52.498666+0000","last_unstale":"2024-05-17T22:36:15.622645+0000","last_undegraded":"2024-05-17T22:36:15.622645+0000","last_fullsized":"2024-05-17T22:36:15.622645+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:46:25.882964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.005012+0000","last_change":"2024-05-17T22:35:15.836294+0000","last_active":"2024-05-17T22:36:16.005012+0000","last_peered":"2024-05-17T22:36:16.005012+0000","last_clean":"2024-05-17T22:36:16.005012+0000","last_became_active":"2024-05-17T22:35:15.835759+0000","last_became_peered":"2024-05-17T22:35:15.835759+0000","last_unstale":"2024-05-17T22:36:16.005012+0000","last_undegraded":"2024-05-17T22:36:16.005012+0000","last_fullsized":"2024-05-17T22:36:16.005012+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:23:32.106692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968497+0000","last_change":"2024-05-17T22:34:40.433331+0000","last_active":"2024-05-17T22:36:15.968497+0000","last_peered":"2024-05-17T22:36:15.968497+0000","last_clean":"2024-05-17T22:36:15.968497+0000","last_became_active":"2024-05-17T22:34:40.432989+0000","last_became_peered":"2024-05-17T22:34:40.432989+0000","last_unstale":"2024-05-17T22:36:15.968497+0000","last_undegraded":"2024-05-17T22:36:15.968497+0000","last_fullsized":"2024-05-17T22:36:15.968497+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:22:49.146148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004182+0000","last_change":"2024-05-17T22:34:52.494418+0000","last_active":"2024-05-17T22:36:16.004182+0000","last_peered":"2024-05-17T22:36:16.004182+0000","last_clean":"2024-05-17T22:36:16.004182+0000","last_became_active":"2024-05-17T22:34:52.493953+0000","last_became_peered":"2024-05-17T22:34:52.493953+0000","last_unstale":"2024-05-17T22:36:16.004182+0000","last_undegraded":"2024-05-17T22:36:16.004182+0000","last_fullsized":"2024-05-17T22:36:16.004182+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:50:40.211770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003645+0000","last_change":"2024-05-17T22:34:37.442986+0000","last_active":"2024-05-17T22:36:16.003645+0000","last_peered":"2024-05-17T22:36:16.003645+0000","last_clean":"2024-05-17T22:36:16.003645+0000","last_became_active":"2024-05-17T22:34:37.442399+0000","last_became_peered":"2024-05-17T22:34:37.442399+0000","last_unstale":"2024-05-17T22:36:16.003645+0000","last_undegraded":"2024-05-17T22:36:16.003645+0000","last_fullsized":"2024-05-17T22:36:16.003645+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:34:05.785811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.805334+0000","last_change":"2024-05-17T22:34:40.426793+0000","last_active":"2024-05-17T22:36:15.805334+0000","last_peered":"2024-05-17T22:36:15.805334+0000","last_clean":"2024-05-17T22:36:15.805334+0000","last_became_active":"2024-05-17T22:34:40.426420+0000","last_became_peered":"2024-05-17T22:34:40.426420+0000","last_unstale":"2024-05-17T22:36:15.805334+0000","last_undegraded":"2024-05-17T22:36:15.805334+0000","last_fullsized":"2024-05-17T22:36:15.805334+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:57:07.099254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.622889+0000","last_change":"2024-05-17T22:34:52.498762+0000","last_active":"2024-05-17T22:36:15.622889+0000","last_peered":"2024-05-17T22:36:15.622889+0000","last_clean":"2024-05-17T22:36:15.622889+0000","last_became_active":"2024-05-17T22:34:52.498306+0000","last_became_peered":"2024-05-17T22:34:52.498306+0000","last_unstale":"2024-05-17T22:36:15.622889+0000","last_undegraded":"2024-05-17T22:36:15.622889+0000","last_fullsized":"2024-05-17T22:36:15.622889+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:18:32.717563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'33","reported_seq":146,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:19.088665+0000","last_change":"2024-05-17T22:34:37.407885+0000","last_active":"2024-05-17T22:36:19.088665+0000","last_peered":"2024-05-17T22:36:19.088665+0000","last_clean":"2024-05-17T22:36:19.088665+0000","last_became_active":"2024-05-17T22:34:37.407463+0000","last_became_peered":"2024-05-17T22:34:37.407463+0000","last_unstale":"2024-05-17T22:36:19.088665+0000","last_undegraded":"2024-05-17T22:36:19.088665+0000","last_fullsized":"2024-05-17T22:36:19.088665+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:26:20.288557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62072,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":69,"num_read_kb":95,"num_write":34,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.626156+0000","last_change":"2024-05-17T22:34:40.435800+0000","last_active":"2024-05-17T22:36:15.626156+0000","last_peered":"2024-05-17T22:36:15.626156+0000","last_clean":"2024-05-17T22:36:15.626156+0000","last_became_active":"2024-05-17T22:34:40.435245+0000","last_became_peered":"2024-05-17T22:34:40.435245+0000","last_unstale":"2024-05-17T22:36:15.626156+0000","last_undegraded":"2024-05-17T22:36:15.626156+0000","last_fullsized":"2024-05-17T22:36:15.626156+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:09:58.699026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004407+0000","last_change":"2024-05-17T22:34:52.487336+0000","last_active":"2024-05-17T22:36:16.004407+0000","last_peered":"2024-05-17T22:36:16.004407+0000","last_clean":"2024-05-17T22:36:16.004407+0000","last_became_active":"2024-05-17T22:34:52.486883+0000","last_became_peered":"2024-05-17T22:34:52.486883+0000","last_unstale":"2024-05-17T22:36:16.004407+0000","last_undegraded":"2024-05-17T22:36:16.004407+0000","last_fullsized":"2024-05-17T22:36:16.004407+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:44:41.264072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623955+0000","last_change":"2024-05-17T22:34:37.405044+0000","last_active":"2024-05-17T22:36:15.623955+0000","last_peered":"2024-05-17T22:36:15.623955+0000","last_clean":"2024-05-17T22:36:15.623955+0000","last_became_active":"2024-05-17T22:34:37.404610+0000","last_became_peered":"2024-05-17T22:34:37.404610+0000","last_unstale":"2024-05-17T22:36:15.623955+0000","last_undegraded":"2024-05-17T22:36:15.623955+0000","last_fullsized":"2024-05-17T22:36:15.623955+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:20:37.939069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685489+0000","last_change":"2024-05-17T22:34:40.430655+0000","last_active":"2024-05-17T22:36:15.685489+0000","last_peered":"2024-05-17T22:36:15.685489+0000","last_clean":"2024-05-17T22:36:15.685489+0000","last_became_active":"2024-05-17T22:34:40.430307+0000","last_became_peered":"2024-05-17T22:34:40.430307+0000","last_unstale":"2024-05-17T22:36:15.685489+0000","last_undegraded":"2024-05-17T22:36:15.685489+0000","last_fullsized":"2024-05-17T22:36:15.685489+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:57:19.036117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628122+0000","last_change":"2024-05-17T22:34:52.494709+0000","last_active":"2024-05-17T22:36:15.628122+0000","last_peered":"2024-05-17T22:36:15.628122+0000","last_clean":"2024-05-17T22:36:15.628122+0000","last_became_active":"2024-05-17T22:34:52.494195+0000","last_became_peered":"2024-05-17T22:34:52.494195+0000","last_unstale":"2024-05-17T22:36:15.628122+0000","last_undegraded":"2024-05-17T22:36:15.628122+0000","last_fullsized":"2024-05-17T22:36:15.628122+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:35:41.188459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003638+0000","last_change":"2024-05-17T22:34:37.402494+0000","last_active":"2024-05-17T22:36:16.003638+0000","last_peered":"2024-05-17T22:36:16.003638+0000","last_clean":"2024-05-17T22:36:16.003638+0000","last_became_active":"2024-05-17T22:34:37.402102+0000","last_became_peered":"2024-05-17T22:34:37.402102+0000","last_unstale":"2024-05-17T22:36:16.003638+0000","last_undegraded":"2024-05-17T22:36:16.003638+0000","last_fullsized":"2024-05-17T22:36:16.003638+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:22:30.915780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004954+0000","last_change":"2024-05-17T22:34:40.440731+0000","last_active":"2024-05-17T22:36:16.004954+0000","last_peered":"2024-05-17T22:36:16.004954+0000","last_clean":"2024-05-17T22:36:16.004954+0000","last_became_active":"2024-05-17T22:34:40.440374+0000","last_became_peered":"2024-05-17T22:34:40.440374+0000","last_unstale":"2024-05-17T22:36:16.004954+0000","last_undegraded":"2024-05-17T22:36:16.004954+0000","last_fullsized":"2024-05-17T22:36:16.004954+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:26:40.029155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.005856+0000","last_change":"2024-05-17T22:36:16.005856+0000","last_active":"2024-05-17T22:36:16.005856+0000","last_peered":"2024-05-17T22:36:16.005856+0000","last_clean":"2024-05-17T22:36:16.005856+0000","last_became_active":"2024-05-17T22:36:16.005303+0000","last_became_peered":"2024-05-17T22:36:16.005303+0000","last_unstale":"2024-05-17T22:36:16.005856+0000","last_undegraded":"2024-05-17T22:36:16.005856+0000","last_fullsized":"2024-05-17T22:36:16.005856+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:41:15.708879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"87'4","reported_seq":78,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.084097+0000","last_change":"2024-05-17T22:34:37.403123+0000","last_active":"2024-05-17T22:36:16.084097+0000","last_peered":"2024-05-17T22:36:16.084097+0000","last_clean":"2024-05-17T22:36:16.084097+0000","last_became_active":"2024-05-17T22:34:37.402702+0000","last_became_peered":"2024-05-17T22:34:37.402702+0000","last_unstale":"2024-05-17T22:36:16.084097+0000","last_undegraded":"2024-05-17T22:36:16.084097+0000","last_fullsized":"2024-05-17T22:36:16.084097+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:50:02.031651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":32,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003779+0000","last_change":"2024-05-17T22:34:40.431895+0000","last_active":"2024-05-17T22:36:16.003779+0000","last_peered":"2024-05-17T22:36:16.003779+0000","last_clean":"2024-05-17T22:36:16.003779+0000","last_became_active":"2024-05-17T22:34:40.431527+0000","last_became_peered":"2024-05-17T22:34:40.431527+0000","last_unstale":"2024-05-17T22:36:16.003779+0000","last_undegraded":"2024-05-17T22:36:16.003779+0000","last_fullsized":"2024-05-17T22:36:16.003779+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:30:09.404255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.686085+0000","last_change":"2024-05-17T22:34:52.495583+0000","last_active":"2024-05-17T22:36:15.686085+0000","last_peered":"2024-05-17T22:36:15.686085+0000","last_clean":"2024-05-17T22:36:15.686085+0000","last_became_active":"2024-05-17T22:34:52.495030+0000","last_became_peered":"2024-05-17T22:34:52.495030+0000","last_unstale":"2024-05-17T22:36:15.686085+0000","last_undegraded":"2024-05-17T22:36:15.686085+0000","last_fullsized":"2024-05-17T22:36:15.686085+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:04:54.046891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625526+0000","last_change":"2024-05-17T22:34:37.442154+0000","last_active":"2024-05-17T22:36:15.625526+0000","last_peered":"2024-05-17T22:36:15.625526+0000","last_clean":"2024-05-17T22:36:15.625526+0000","last_became_active":"2024-05-17T22:34:37.441691+0000","last_became_peered":"2024-05-17T22:34:37.441691+0000","last_unstale":"2024-05-17T22:36:15.625526+0000","last_undegraded":"2024-05-17T22:36:15.625526+0000","last_fullsized":"2024-05-17T22:36:15.625526+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:47:29.862031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003475+0000","last_change":"2024-05-17T22:34:40.430894+0000","last_active":"2024-05-17T22:36:16.003475+0000","last_peered":"2024-05-17T22:36:16.003475+0000","last_clean":"2024-05-17T22:36:16.003475+0000","last_became_active":"2024-05-17T22:34:40.430328+0000","last_became_peered":"2024-05-17T22:34:40.430328+0000","last_unstale":"2024-05-17T22:36:16.003475+0000","last_undegraded":"2024-05-17T22:36:16.003475+0000","last_fullsized":"2024-05-17T22:36:16.003475+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:36:46.619111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003404+0000","last_change":"2024-05-17T22:34:52.487555+0000","last_active":"2024-05-17T22:36:16.003404+0000","last_peered":"2024-05-17T22:36:16.003404+0000","last_clean":"2024-05-17T22:36:16.003404+0000","last_became_active":"2024-05-17T22:34:52.487076+0000","last_became_peered":"2024-05-17T22:34:52.487076+0000","last_unstale":"2024-05-17T22:36:16.003404+0000","last_undegraded":"2024-05-17T22:36:16.003404+0000","last_fullsized":"2024-05-17T22:36:16.003404+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:05:46.720665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628312+0000","last_change":"2024-05-17T22:35:15.558860+0000","last_active":"2024-05-17T22:36:15.628312+0000","last_peered":"2024-05-17T22:36:15.628312+0000","last_clean":"2024-05-17T22:36:15.628312+0000","last_became_active":"2024-05-17T22:35:15.553683+0000","last_became_peered":"2024-05-17T22:35:15.553683+0000","last_unstale":"2024-05-17T22:36:15.628312+0000","last_undegraded":"2024-05-17T22:36:15.628312+0000","last_fullsized":"2024-05-17T22:36:15.628312+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:27:24.978282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628360+0000","last_change":"2024-05-17T22:34:40.433499+0000","last_active":"2024-05-17T22:36:15.628360+0000","last_peered":"2024-05-17T22:36:15.628360+0000","last_clean":"2024-05-17T22:36:15.628360+0000","last_became_active":"2024-05-17T22:34:40.431370+0000","last_became_peered":"2024-05-17T22:34:40.431370+0000","last_unstale":"2024-05-17T22:36:15.628360+0000","last_undegraded":"2024-05-17T22:36:15.628360+0000","last_fullsized":"2024-05-17T22:36:15.628360+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:25:08.383021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625886+0000","last_change":"2024-05-17T22:34:52.489627+0000","last_active":"2024-05-17T22:36:15.625886+0000","last_peered":"2024-05-17T22:36:15.625886+0000","last_clean":"2024-05-17T22:36:15.625886+0000","last_became_active":"2024-05-17T22:34:52.489173+0000","last_became_peered":"2024-05-17T22:34:52.489173+0000","last_unstale":"2024-05-17T22:36:15.625886+0000","last_undegraded":"2024-05-17T22:36:15.625886+0000","last_fullsized":"2024-05-17T22:36:15.625886+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:52:26.908817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623301+0000","last_change":"2024-05-17T22:35:15.553325+0000","last_active":"2024-05-17T22:36:15.623301+0000","last_peered":"2024-05-17T22:36:15.623301+0000","last_clean":"2024-05-17T22:36:15.623301+0000","last_became_active":"2024-05-17T22:35:15.552867+0000","last_became_peered":"2024-05-17T22:35:15.552867+0000","last_unstale":"2024-05-17T22:36:15.623301+0000","last_undegraded":"2024-05-17T22:36:15.623301+0000","last_fullsized":"2024-05-17T22:36:15.623301+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:28:36.618331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.626212+0000","last_change":"2024-05-17T22:34:40.423221+0000","last_active":"2024-05-17T22:36:15.626212+0000","last_peered":"2024-05-17T22:36:15.626212+0000","last_clean":"2024-05-17T22:36:15.626212+0000","last_became_active":"2024-05-17T22:34:40.422838+0000","last_became_peered":"2024-05-17T22:34:40.422838+0000","last_unstale":"2024-05-17T22:36:15.626212+0000","last_undegraded":"2024-05-17T22:36:15.626212+0000","last_fullsized":"2024-05-17T22:36:15.626212+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:39:51.873212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685849+0000","last_change":"2024-05-17T22:34:52.484802+0000","last_active":"2024-05-17T22:36:15.685849+0000","last_peered":"2024-05-17T22:36:15.685849+0000","last_clean":"2024-05-17T22:36:15.685849+0000","last_became_active":"2024-05-17T22:34:52.483766+0000","last_became_peered":"2024-05-17T22:34:52.483766+0000","last_unstale":"2024-05-17T22:36:15.685849+0000","last_undegraded":"2024-05-17T22:36:15.685849+0000","last_fullsized":"2024-05-17T22:36:15.685849+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:32:01.427078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004410+0000","last_change":"2024-05-17T22:34:37.404797+0000","last_active":"2024-05-17T22:36:16.004410+0000","last_peered":"2024-05-17T22:36:16.004410+0000","last_clean":"2024-05-17T22:36:16.004410+0000","last_became_active":"2024-05-17T22:34:37.404393+0000","last_became_peered":"2024-05-17T22:34:37.404393+0000","last_unstale":"2024-05-17T22:36:16.004410+0000","last_undegraded":"2024-05-17T22:36:16.004410+0000","last_fullsized":"2024-05-17T22:36:16.004410+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:26:36.887919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625922+0000","last_change":"2024-05-17T22:34:40.428210+0000","last_active":"2024-05-17T22:36:15.625922+0000","last_peered":"2024-05-17T22:36:15.625922+0000","last_clean":"2024-05-17T22:36:15.625922+0000","last_became_active":"2024-05-17T22:34:40.427845+0000","last_became_peered":"2024-05-17T22:34:40.427845+0000","last_unstale":"2024-05-17T22:36:15.625922+0000","last_undegraded":"2024-05-17T22:36:15.625922+0000","last_fullsized":"2024-05-17T22:36:15.625922+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:45:30.437139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685982+0000","last_change":"2024-05-17T22:34:52.490502+0000","last_active":"2024-05-17T22:36:15.685982+0000","last_peered":"2024-05-17T22:36:15.685982+0000","last_clean":"2024-05-17T22:36:15.685982+0000","last_became_active":"2024-05-17T22:34:52.490008+0000","last_became_peered":"2024-05-17T22:34:52.490008+0000","last_unstale":"2024-05-17T22:36:15.685982+0000","last_undegraded":"2024-05-17T22:36:15.685982+0000","last_fullsized":"2024-05-17T22:36:15.685982+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:58:29.122668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003400+0000","last_change":"2024-05-17T22:34:40.432248+0000","last_active":"2024-05-17T22:36:16.003400+0000","last_peered":"2024-05-17T22:36:16.003400+0000","last_clean":"2024-05-17T22:36:16.003400+0000","last_became_active":"2024-05-17T22:34:40.431863+0000","last_became_peered":"2024-05-17T22:34:40.431863+0000","last_unstale":"2024-05-17T22:36:16.003400+0000","last_undegraded":"2024-05-17T22:36:16.003400+0000","last_fullsized":"2024-05-17T22:36:16.003400+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:20:34.730542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'4","reported_seq":77,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:17.084645+0000","last_change":"2024-05-17T22:34:37.441186+0000","last_active":"2024-05-17T22:36:17.084645+0000","last_peered":"2024-05-17T22:36:17.084645+0000","last_clean":"2024-05-17T22:36:17.084645+0000","last_became_active":"2024-05-17T22:34:37.440781+0000","last_became_peered":"2024-05-17T22:34:37.440781+0000","last_unstale":"2024-05-17T22:36:17.084645+0000","last_undegraded":"2024-05-17T22:36:17.084645+0000","last_fullsized":"2024-05-17T22:36:17.084645+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:44:34.435147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.005029+0000","last_change":"2024-05-17T22:34:52.491771+0000","last_active":"2024-05-17T22:36:16.005029+0000","last_peered":"2024-05-17T22:36:16.005029+0000","last_clean":"2024-05-17T22:36:16.005029+0000","last_became_active":"2024-05-17T22:34:52.491172+0000","last_became_peered":"2024-05-17T22:34:52.491172+0000","last_unstale":"2024-05-17T22:36:16.005029+0000","last_undegraded":"2024-05-17T22:36:16.005029+0000","last_fullsized":"2024-05-17T22:36:16.005029+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:15:21.010592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984305+0000","last_change":"2024-05-17T22:34:40.430860+0000","last_active":"2024-05-17T22:36:15.984305+0000","last_peered":"2024-05-17T22:36:15.984305+0000","last_clean":"2024-05-17T22:36:15.984305+0000","last_became_active":"2024-05-17T22:34:40.430542+0000","last_became_peered":"2024-05-17T22:34:40.430542+0000","last_unstale":"2024-05-17T22:36:15.984305+0000","last_undegraded":"2024-05-17T22:36:15.984305+0000","last_fullsized":"2024-05-17T22:36:15.984305+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:16:26.257380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004059+0000","last_change":"2024-05-17T22:34:37.403026+0000","last_active":"2024-05-17T22:36:16.004059+0000","last_peered":"2024-05-17T22:36:16.004059+0000","last_clean":"2024-05-17T22:36:16.004059+0000","last_became_active":"2024-05-17T22:34:37.402560+0000","last_became_peered":"2024-05-17T22:34:37.402560+0000","last_unstale":"2024-05-17T22:36:16.004059+0000","last_undegraded":"2024-05-17T22:36:16.004059+0000","last_fullsized":"2024-05-17T22:36:16.004059+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:23:03.784411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":119,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003503+0000","last_change":"2024-05-17T22:33:11.055878+0000","last_active":"2024-05-17T22:36:16.003503+0000","last_peered":"2024-05-17T22:36:16.003503+0000","last_clean":"2024-05-17T22:36:16.003503+0000","last_became_active":"2024-05-17T22:33:11.050409+0000","last_became_peered":"2024-05-17T22:33:11.050409+0000","last_unstale":"2024-05-17T22:36:16.003503+0000","last_undegraded":"2024-05-17T22:36:16.003503+0000","last_fullsized":"2024-05-17T22:36:16.003503+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:29:43.341170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:29:43.341170+0000","last_clean_scrub_stamp":"2024-05-17T22:29:43.341170+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:30:54.153865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003650+0000","last_change":"2024-05-17T22:34:52.497064+0000","last_active":"2024-05-17T22:36:16.003650+0000","last_peered":"2024-05-17T22:36:16.003650+0000","last_clean":"2024-05-17T22:36:16.003650+0000","last_became_active":"2024-05-17T22:34:52.496611+0000","last_became_peered":"2024-05-17T22:34:52.496611+0000","last_unstale":"2024-05-17T22:36:16.003650+0000","last_undegraded":"2024-05-17T22:36:16.003650+0000","last_fullsized":"2024-05-17T22:36:16.003650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:20:29.597589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685209+0000","last_change":"2024-05-17T22:35:15.555413+0000","last_active":"2024-05-17T22:36:15.685209+0000","last_peered":"2024-05-17T22:36:15.685209+0000","last_clean":"2024-05-17T22:36:15.685209+0000","last_became_active":"2024-05-17T22:35:15.555005+0000","last_became_peered":"2024-05-17T22:35:15.555005+0000","last_unstale":"2024-05-17T22:36:15.685209+0000","last_undegraded":"2024-05-17T22:36:15.685209+0000","last_fullsized":"2024-05-17T22:36:15.685209+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:51:49.860673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003644+0000","last_change":"2024-05-17T22:34:37.406546+0000","last_active":"2024-05-17T22:36:16.003644+0000","last_peered":"2024-05-17T22:36:16.003644+0000","last_clean":"2024-05-17T22:36:16.003644+0000","last_became_active":"2024-05-17T22:34:37.406118+0000","last_became_peered":"2024-05-17T22:34:37.406118+0000","last_unstale":"2024-05-17T22:36:16.003644+0000","last_undegraded":"2024-05-17T22:36:16.003644+0000","last_fullsized":"2024-05-17T22:36:16.003644+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:47:16.350091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.005647+0000","last_change":"2024-05-17T22:36:16.005647+0000","last_active":"2024-05-17T22:36:16.005647+0000","last_peered":"2024-05-17T22:36:16.005647+0000","last_clean":"2024-05-17T22:36:16.005647+0000","last_became_active":"2024-05-17T22:36:16.005128+0000","last_became_peered":"2024-05-17T22:36:16.005128+0000","last_unstale":"2024-05-17T22:36:16.005647+0000","last_undegraded":"2024-05-17T22:36:16.005647+0000","last_fullsized":"2024-05-17T22:36:16.005647+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:16:24.481236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,7,6],"acting":[11,10,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.626298+0000","last_change":"2024-05-17T22:34:37.442676+0000","last_active":"2024-05-17T22:36:15.626298+0000","last_peered":"2024-05-17T22:36:15.626298+0000","last_clean":"2024-05-17T22:36:15.626298+0000","last_became_active":"2024-05-17T22:34:37.442275+0000","last_became_peered":"2024-05-17T22:34:37.442275+0000","last_unstale":"2024-05-17T22:36:15.626298+0000","last_undegraded":"2024-05-17T22:36:15.626298+0000","last_fullsized":"2024-05-17T22:36:15.626298+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:54:18.542987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685755+0000","last_change":"2024-05-17T22:34:40.424266+0000","last_active":"2024-05-17T22:36:15.685755+0000","last_peered":"2024-05-17T22:36:15.685755+0000","last_clean":"2024-05-17T22:36:15.685755+0000","last_became_active":"2024-05-17T22:34:40.423905+0000","last_became_peered":"2024-05-17T22:34:40.423905+0000","last_unstale":"2024-05-17T22:36:15.685755+0000","last_undegraded":"2024-05-17T22:36:15.685755+0000","last_fullsized":"2024-05-17T22:36:15.685755+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:03:43.397871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984103+0000","last_change":"2024-05-17T22:34:52.493386+0000","last_active":"2024-05-17T22:36:15.984103+0000","last_peered":"2024-05-17T22:36:15.984103+0000","last_clean":"2024-05-17T22:36:15.984103+0000","last_became_active":"2024-05-17T22:34:52.492997+0000","last_became_peered":"2024-05-17T22:34:52.492997+0000","last_unstale":"2024-05-17T22:36:15.984103+0000","last_undegraded":"2024-05-17T22:36:15.984103+0000","last_fullsized":"2024-05-17T22:36:15.984103+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:14:40.818271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'11","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623586+0000","last_change":"2024-05-17T22:34:37.399397+0000","last_active":"2024-05-17T22:36:15.623586+0000","last_peered":"2024-05-17T22:36:15.623586+0000","last_clean":"2024-05-17T22:36:15.623586+0000","last_became_active":"2024-05-17T22:34:37.399062+0000","last_became_peered":"2024-05-17T22:34:37.399062+0000","last_unstale":"2024-05-17T22:36:15.623586+0000","last_undegraded":"2024-05-17T22:36:15.623586+0000","last_fullsized":"2024-05-17T22:36:15.623586+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:02:05.230580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984853+0000","last_change":"2024-05-17T22:34:40.426995+0000","last_active":"2024-05-17T22:36:15.984853+0000","last_peered":"2024-05-17T22:36:15.984853+0000","last_clean":"2024-05-17T22:36:15.984853+0000","last_became_active":"2024-05-17T22:34:40.426226+0000","last_became_peered":"2024-05-17T22:34:40.426226+0000","last_unstale":"2024-05-17T22:36:15.984853+0000","last_undegraded":"2024-05-17T22:36:15.984853+0000","last_fullsized":"2024-05-17T22:36:15.984853+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:49:53.331076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623635+0000","last_change":"2024-05-17T22:34:52.487098+0000","last_active":"2024-05-17T22:36:15.623635+0000","last_peered":"2024-05-17T22:36:15.623635+0000","last_clean":"2024-05-17T22:36:15.623635+0000","last_became_active":"2024-05-17T22:34:52.486605+0000","last_became_peered":"2024-05-17T22:34:52.486605+0000","last_unstale":"2024-05-17T22:36:15.623635+0000","last_undegraded":"2024-05-17T22:36:15.623635+0000","last_fullsized":"2024-05-17T22:36:15.623635+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:33:28.234208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004630+0000","last_change":"2024-05-17T22:34:37.800830+0000","last_active":"2024-05-17T22:36:16.004630+0000","last_peered":"2024-05-17T22:36:16.004630+0000","last_clean":"2024-05-17T22:36:16.004630+0000","last_became_active":"2024-05-17T22:34:37.800405+0000","last_became_peered":"2024-05-17T22:34:37.800405+0000","last_unstale":"2024-05-17T22:36:16.004630+0000","last_undegraded":"2024-05-17T22:36:16.004630+0000","last_fullsized":"2024-05-17T22:36:16.004630+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:19:10.087442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968862+0000","last_change":"2024-05-17T22:34:40.426307+0000","last_active":"2024-05-17T22:36:15.968862+0000","last_peered":"2024-05-17T22:36:15.968862+0000","last_clean":"2024-05-17T22:36:15.968862+0000","last_became_active":"2024-05-17T22:34:40.425969+0000","last_became_peered":"2024-05-17T22:34:40.425969+0000","last_unstale":"2024-05-17T22:36:15.968862+0000","last_undegraded":"2024-05-17T22:36:15.968862+0000","last_fullsized":"2024-05-17T22:36:15.968862+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:25:49.680886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004451+0000","last_change":"2024-05-17T22:35:15.556857+0000","last_active":"2024-05-17T22:36:16.004451+0000","last_peered":"2024-05-17T22:36:16.004451+0000","last_clean":"2024-05-17T22:36:16.004451+0000","last_became_active":"2024-05-17T22:35:15.556298+0000","last_became_peered":"2024-05-17T22:35:15.556298+0000","last_unstale":"2024-05-17T22:36:16.004451+0000","last_undegraded":"2024-05-17T22:36:16.004451+0000","last_fullsized":"2024-05-17T22:36:16.004451+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:23:20.111544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.805120+0000","last_change":"2024-05-17T22:34:37.442971+0000","last_active":"2024-05-17T22:36:15.805120+0000","last_peered":"2024-05-17T22:36:15.805120+0000","last_clean":"2024-05-17T22:36:15.805120+0000","last_became_active":"2024-05-17T22:34:37.442536+0000","last_became_peered":"2024-05-17T22:34:37.442536+0000","last_unstale":"2024-05-17T22:36:15.805120+0000","last_undegraded":"2024-05-17T22:36:15.805120+0000","last_fullsized":"2024-05-17T22:36:15.805120+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:03:45.224071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625559+0000","last_change":"2024-05-17T22:34:40.432056+0000","last_active":"2024-05-17T22:36:15.625559+0000","last_peered":"2024-05-17T22:36:15.625559+0000","last_clean":"2024-05-17T22:36:15.625559+0000","last_became_active":"2024-05-17T22:34:40.431626+0000","last_became_peered":"2024-05-17T22:34:40.431626+0000","last_unstale":"2024-05-17T22:36:15.625559+0000","last_undegraded":"2024-05-17T22:36:15.625559+0000","last_fullsized":"2024-05-17T22:36:15.625559+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:12:24.650746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623590+0000","last_change":"2024-05-17T22:34:52.500451+0000","last_active":"2024-05-17T22:36:15.623590+0000","last_peered":"2024-05-17T22:36:15.623590+0000","last_clean":"2024-05-17T22:36:15.623590+0000","last_became_active":"2024-05-17T22:34:52.500031+0000","last_became_peered":"2024-05-17T22:34:52.500031+0000","last_unstale":"2024-05-17T22:36:15.623590+0000","last_undegraded":"2024-05-17T22:36:15.623590+0000","last_fullsized":"2024-05-17T22:36:15.623590+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:54:36.825116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984349+0000","last_change":"2024-05-17T22:34:37.402335+0000","last_active":"2024-05-17T22:36:15.984349+0000","last_peered":"2024-05-17T22:36:15.984349+0000","last_clean":"2024-05-17T22:36:15.984349+0000","last_became_active":"2024-05-17T22:34:37.401982+0000","last_became_peered":"2024-05-17T22:34:37.401982+0000","last_unstale":"2024-05-17T22:36:15.984349+0000","last_undegraded":"2024-05-17T22:36:15.984349+0000","last_fullsized":"2024-05-17T22:36:15.984349+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:58:06.194335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.627919+0000","last_change":"2024-05-17T22:34:40.425524+0000","last_active":"2024-05-17T22:36:15.627919+0000","last_peered":"2024-05-17T22:36:15.627919+0000","last_clean":"2024-05-17T22:36:15.627919+0000","last_became_active":"2024-05-17T22:34:40.425135+0000","last_became_peered":"2024-05-17T22:34:40.425135+0000","last_unstale":"2024-05-17T22:36:15.627919+0000","last_undegraded":"2024-05-17T22:36:15.627919+0000","last_fullsized":"2024-05-17T22:36:15.627919+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:41:14.176477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004602+0000","last_change":"2024-05-17T22:34:52.496290+0000","last_active":"2024-05-17T22:36:16.004602+0000","last_peered":"2024-05-17T22:36:16.004602+0000","last_clean":"2024-05-17T22:36:16.004602+0000","last_became_active":"2024-05-17T22:34:52.495817+0000","last_became_peered":"2024-05-17T22:34:52.495817+0000","last_unstale":"2024-05-17T22:36:16.004602+0000","last_undegraded":"2024-05-17T22:36:16.004602+0000","last_fullsized":"2024-05-17T22:36:16.004602+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:47:22.056372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'1","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623936+0000","last_change":"2024-05-17T22:34:37.404234+0000","last_active":"2024-05-17T22:36:15.623936+0000","last_peered":"2024-05-17T22:36:15.623936+0000","last_clean":"2024-05-17T22:36:15.623936+0000","last_became_active":"2024-05-17T22:34:37.403561+0000","last_became_peered":"2024-05-17T22:34:37.403561+0000","last_unstale":"2024-05-17T22:36:15.623936+0000","last_undegraded":"2024-05-17T22:36:15.623936+0000","last_fullsized":"2024-05-17T22:36:15.623936+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:30:28.432641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984498+0000","last_change":"2024-05-17T22:34:40.426522+0000","last_active":"2024-05-17T22:36:15.984498+0000","last_peered":"2024-05-17T22:36:15.984498+0000","last_clean":"2024-05-17T22:36:15.984498+0000","last_became_active":"2024-05-17T22:34:40.426139+0000","last_became_peered":"2024-05-17T22:34:40.426139+0000","last_unstale":"2024-05-17T22:36:15.984498+0000","last_undegraded":"2024-05-17T22:36:15.984498+0000","last_fullsized":"2024-05-17T22:36:15.984498+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:30:19.503513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004016+0000","last_change":"2024-05-17T22:34:52.487131+0000","last_active":"2024-05-17T22:36:16.004016+0000","last_peered":"2024-05-17T22:36:16.004016+0000","last_clean":"2024-05-17T22:36:16.004016+0000","last_became_active":"2024-05-17T22:34:52.486654+0000","last_became_peered":"2024-05-17T22:34:52.486654+0000","last_unstale":"2024-05-17T22:36:16.004016+0000","last_undegraded":"2024-05-17T22:36:16.004016+0000","last_fullsized":"2024-05-17T22:36:16.004016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:28:25.041746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625819+0000","last_change":"2024-05-17T22:34:37.405020+0000","last_active":"2024-05-17T22:36:15.625819+0000","last_peered":"2024-05-17T22:36:15.625819+0000","last_clean":"2024-05-17T22:36:15.625819+0000","last_became_active":"2024-05-17T22:34:37.404633+0000","last_became_peered":"2024-05-17T22:34:37.404633+0000","last_unstale":"2024-05-17T22:36:15.625819+0000","last_undegraded":"2024-05-17T22:36:15.625819+0000","last_fullsized":"2024-05-17T22:36:15.625819+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:06:00.001190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004601+0000","last_change":"2024-05-17T22:34:40.435374+0000","last_active":"2024-05-17T22:36:16.004601+0000","last_peered":"2024-05-17T22:36:16.004601+0000","last_clean":"2024-05-17T22:36:16.004601+0000","last_became_active":"2024-05-17T22:34:40.434522+0000","last_became_peered":"2024-05-17T22:34:40.434522+0000","last_unstale":"2024-05-17T22:36:16.004601+0000","last_undegraded":"2024-05-17T22:36:16.004601+0000","last_fullsized":"2024-05-17T22:36:16.004601+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:36:41.244970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.805127+0000","last_change":"2024-05-17T22:34:52.491708+0000","last_active":"2024-05-17T22:36:15.805127+0000","last_peered":"2024-05-17T22:36:15.805127+0000","last_clean":"2024-05-17T22:36:15.805127+0000","last_became_active":"2024-05-17T22:34:52.491293+0000","last_became_peered":"2024-05-17T22:34:52.491293+0000","last_unstale":"2024-05-17T22:36:15.805127+0000","last_undegraded":"2024-05-17T22:36:15.805127+0000","last_fullsized":"2024-05-17T22:36:15.805127+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:23:01.634026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624618+0000","last_change":"2024-05-17T22:34:37.800525+0000","last_active":"2024-05-17T22:36:15.624618+0000","last_peered":"2024-05-17T22:36:15.624618+0000","last_clean":"2024-05-17T22:36:15.624618+0000","last_became_active":"2024-05-17T22:34:37.800093+0000","last_became_peered":"2024-05-17T22:34:37.800093+0000","last_unstale":"2024-05-17T22:36:15.624618+0000","last_undegraded":"2024-05-17T22:36:15.624618+0000","last_fullsized":"2024-05-17T22:36:15.624618+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:25:27.249889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.005319+0000","last_change":"2024-05-17T22:34:40.434891+0000","last_active":"2024-05-17T22:36:16.005319+0000","last_peered":"2024-05-17T22:36:16.005319+0000","last_clean":"2024-05-17T22:36:16.005319+0000","last_became_active":"2024-05-17T22:34:40.434543+0000","last_became_peered":"2024-05-17T22:34:40.434543+0000","last_unstale":"2024-05-17T22:36:16.005319+0000","last_undegraded":"2024-05-17T22:36:16.005319+0000","last_fullsized":"2024-05-17T22:36:16.005319+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:17:47.770225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968349+0000","last_change":"2024-05-17T22:34:52.491544+0000","last_active":"2024-05-17T22:36:15.968349+0000","last_peered":"2024-05-17T22:36:15.968349+0000","last_clean":"2024-05-17T22:36:15.968349+0000","last_became_active":"2024-05-17T22:34:52.491138+0000","last_became_peered":"2024-05-17T22:34:52.491138+0000","last_unstale":"2024-05-17T22:36:15.968349+0000","last_undegraded":"2024-05-17T22:36:15.968349+0000","last_fullsized":"2024-05-17T22:36:15.968349+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:34:20.411299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004214+0000","last_change":"2024-05-17T22:34:37.403220+0000","last_active":"2024-05-17T22:36:16.004214+0000","last_peered":"2024-05-17T22:36:16.004214+0000","last_clean":"2024-05-17T22:36:16.004214+0000","last_became_active":"2024-05-17T22:34:37.402856+0000","last_became_peered":"2024-05-17T22:34:37.402856+0000","last_unstale":"2024-05-17T22:36:16.004214+0000","last_undegraded":"2024-05-17T22:36:16.004214+0000","last_fullsized":"2024-05-17T22:36:16.004214+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:04:19.469294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.005034+0000","last_change":"2024-05-17T22:34:40.434487+0000","last_active":"2024-05-17T22:36:16.005034+0000","last_peered":"2024-05-17T22:36:16.005034+0000","last_clean":"2024-05-17T22:36:16.005034+0000","last_became_active":"2024-05-17T22:34:40.434160+0000","last_became_peered":"2024-05-17T22:34:40.434160+0000","last_unstale":"2024-05-17T22:36:16.005034+0000","last_undegraded":"2024-05-17T22:36:16.005034+0000","last_fullsized":"2024-05-17T22:36:16.005034+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:24:38.782999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003123+0000","last_change":"2024-05-17T22:34:52.487887+0000","last_active":"2024-05-17T22:36:16.003123+0000","last_peered":"2024-05-17T22:36:16.003123+0000","last_clean":"2024-05-17T22:36:16.003123+0000","last_became_active":"2024-05-17T22:34:52.486837+0000","last_became_peered":"2024-05-17T22:34:52.486837+0000","last_unstale":"2024-05-17T22:36:16.003123+0000","last_undegraded":"2024-05-17T22:36:16.003123+0000","last_fullsized":"2024-05-17T22:36:16.003123+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:12:22.206536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.686166+0000","last_change":"2024-05-17T22:34:37.399498+0000","last_active":"2024-05-17T22:36:15.686166+0000","last_peered":"2024-05-17T22:36:15.686166+0000","last_clean":"2024-05-17T22:36:15.686166+0000","last_became_active":"2024-05-17T22:34:37.399144+0000","last_became_peered":"2024-05-17T22:34:37.399144+0000","last_unstale":"2024-05-17T22:36:15.686166+0000","last_undegraded":"2024-05-17T22:36:15.686166+0000","last_fullsized":"2024-05-17T22:36:15.686166+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:13:37.346006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.005111+0000","last_change":"2024-05-17T22:34:40.433839+0000","last_active":"2024-05-17T22:36:16.005111+0000","last_peered":"2024-05-17T22:36:16.005111+0000","last_clean":"2024-05-17T22:36:16.005111+0000","last_became_active":"2024-05-17T22:34:40.433489+0000","last_became_peered":"2024-05-17T22:34:40.433489+0000","last_unstale":"2024-05-17T22:36:16.005111+0000","last_undegraded":"2024-05-17T22:36:16.005111+0000","last_fullsized":"2024-05-17T22:36:16.005111+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:13:56.014094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628026+0000","last_change":"2024-05-17T22:34:40.434953+0000","last_active":"2024-05-17T22:36:15.628026+0000","last_peered":"2024-05-17T22:36:15.628026+0000","last_clean":"2024-05-17T22:36:15.628026+0000","last_became_active":"2024-05-17T22:34:40.434593+0000","last_became_peered":"2024-05-17T22:34:40.434593+0000","last_unstale":"2024-05-17T22:36:15.628026+0000","last_undegraded":"2024-05-17T22:36:15.628026+0000","last_fullsized":"2024-05-17T22:36:15.628026+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:20:27.075382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623850+0000","last_change":"2024-05-17T22:34:40.432787+0000","last_active":"2024-05-17T22:36:15.623850+0000","last_peered":"2024-05-17T22:36:15.623850+0000","last_clean":"2024-05-17T22:36:15.623850+0000","last_became_active":"2024-05-17T22:34:40.432402+0000","last_became_peered":"2024-05-17T22:34:40.432402+0000","last_unstale":"2024-05-17T22:36:15.623850+0000","last_undegraded":"2024-05-17T22:36:15.623850+0000","last_fullsized":"2024-05-17T22:36:15.623850+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:48:13.034276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623724+0000","last_change":"2024-05-17T22:34:40.426423+0000","last_active":"2024-05-17T22:36:15.623724+0000","last_peered":"2024-05-17T22:36:15.623724+0000","last_clean":"2024-05-17T22:36:15.623724+0000","last_became_active":"2024-05-17T22:34:40.426049+0000","last_became_peered":"2024-05-17T22:34:40.426049+0000","last_unstale":"2024-05-17T22:36:15.623724+0000","last_undegraded":"2024-05-17T22:36:15.623724+0000","last_fullsized":"2024-05-17T22:36:15.623724+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:56:55.631322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968631+0000","last_change":"2024-05-17T22:34:40.435706+0000","last_active":"2024-05-17T22:36:15.968631+0000","last_peered":"2024-05-17T22:36:15.968631+0000","last_clean":"2024-05-17T22:36:15.968631+0000","last_became_active":"2024-05-17T22:34:40.435330+0000","last_became_peered":"2024-05-17T22:34:40.435330+0000","last_unstale":"2024-05-17T22:36:15.968631+0000","last_undegraded":"2024-05-17T22:36:15.968631+0000","last_fullsized":"2024-05-17T22:36:15.968631+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:25:36.529454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685705+0000","last_change":"2024-05-17T22:34:40.428997+0000","last_active":"2024-05-17T22:36:15.685705+0000","last_peered":"2024-05-17T22:36:15.685705+0000","last_clean":"2024-05-17T22:36:15.685705+0000","last_became_active":"2024-05-17T22:34:40.428594+0000","last_became_peered":"2024-05-17T22:34:40.428594+0000","last_unstale":"2024-05-17T22:36:15.685705+0000","last_undegraded":"2024-05-17T22:36:15.685705+0000","last_fullsized":"2024-05-17T22:36:15.685705+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:59:01.777751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628370+0000","last_change":"2024-05-17T22:34:40.434241+0000","last_active":"2024-05-17T22:36:15.628370+0000","last_peered":"2024-05-17T22:36:15.628370+0000","last_clean":"2024-05-17T22:36:15.628370+0000","last_became_active":"2024-05-17T22:34:40.431652+0000","last_became_peered":"2024-05-17T22:34:40.431652+0000","last_unstale":"2024-05-17T22:36:15.628370+0000","last_undegraded":"2024-05-17T22:36:15.628370+0000","last_fullsized":"2024-05-17T22:36:15.628370+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:33:27.035284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624649+0000","last_change":"2024-05-17T22:34:40.431485+0000","last_active":"2024-05-17T22:36:15.624649+0000","last_peered":"2024-05-17T22:36:15.624649+0000","last_clean":"2024-05-17T22:36:15.624649+0000","last_became_active":"2024-05-17T22:34:40.431120+0000","last_became_peered":"2024-05-17T22:34:40.431120+0000","last_unstale":"2024-05-17T22:36:15.624649+0000","last_undegraded":"2024-05-17T22:36:15.624649+0000","last_fullsized":"2024-05-17T22:36:15.624649+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:46:55.213602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984947+0000","last_change":"2024-05-17T22:34:40.423117+0000","last_active":"2024-05-17T22:36:15.984947+0000","last_peered":"2024-05-17T22:36:15.984947+0000","last_clean":"2024-05-17T22:36:15.984947+0000","last_became_active":"2024-05-17T22:34:40.422694+0000","last_became_peered":"2024-05-17T22:34:40.422694+0000","last_unstale":"2024-05-17T22:36:15.984947+0000","last_undegraded":"2024-05-17T22:36:15.984947+0000","last_fullsized":"2024-05-17T22:36:15.984947+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:19:55.589532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685170+0000","last_change":"2024-05-17T22:35:15.711796+0000","last_active":"2024-05-17T22:36:15.685170+0000","last_peered":"2024-05-17T22:36:15.685170+0000","last_clean":"2024-05-17T22:36:15.685170+0000","last_became_active":"2024-05-17T22:35:15.711327+0000","last_became_peered":"2024-05-17T22:35:15.711327+0000","last_unstale":"2024-05-17T22:36:15.685170+0000","last_undegraded":"2024-05-17T22:36:15.685170+0000","last_fullsized":"2024-05-17T22:36:15.685170+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:48:48.913312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984270+0000","last_change":"2024-05-17T22:34:40.430393+0000","last_active":"2024-05-17T22:36:15.984270+0000","last_peered":"2024-05-17T22:36:15.984270+0000","last_clean":"2024-05-17T22:36:15.984270+0000","last_became_active":"2024-05-17T22:34:40.430030+0000","last_became_peered":"2024-05-17T22:34:40.430030+0000","last_unstale":"2024-05-17T22:36:15.984270+0000","last_undegraded":"2024-05-17T22:36:15.984270+0000","last_fullsized":"2024-05-17T22:36:15.984270+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:39:37.204246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.805401+0000","last_change":"2024-05-17T22:34:40.432443+0000","last_active":"2024-05-17T22:36:15.805401+0000","last_peered":"2024-05-17T22:36:15.805401+0000","last_clean":"2024-05-17T22:36:15.805401+0000","last_became_active":"2024-05-17T22:34:40.431991+0000","last_became_peered":"2024-05-17T22:34:40.431991+0000","last_unstale":"2024-05-17T22:36:15.805401+0000","last_undegraded":"2024-05-17T22:36:15.805401+0000","last_fullsized":"2024-05-17T22:36:15.805401+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:37:45.593575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685767+0000","last_change":"2024-05-17T22:34:40.434388+0000","last_active":"2024-05-17T22:36:15.685767+0000","last_peered":"2024-05-17T22:36:15.685767+0000","last_clean":"2024-05-17T22:36:15.685767+0000","last_became_active":"2024-05-17T22:34:40.434044+0000","last_became_peered":"2024-05-17T22:34:40.434044+0000","last_unstale":"2024-05-17T22:36:15.685767+0000","last_undegraded":"2024-05-17T22:36:15.685767+0000","last_fullsized":"2024-05-17T22:36:15.685767+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:20:43.296190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628303+0000","last_change":"2024-05-17T22:34:40.427137+0000","last_active":"2024-05-17T22:36:15.628303+0000","last_peered":"2024-05-17T22:36:15.628303+0000","last_clean":"2024-05-17T22:36:15.628303+0000","last_became_active":"2024-05-17T22:34:40.426703+0000","last_became_peered":"2024-05-17T22:34:40.426703+0000","last_unstale":"2024-05-17T22:36:15.628303+0000","last_undegraded":"2024-05-17T22:36:15.628303+0000","last_fullsized":"2024-05-17T22:36:15.628303+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:55:52.992817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66464,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":204,"num_read_kb":174,"num_write":169,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":195127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710756,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1028096,"data_stored":515036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906985,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28692,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716460,"statfs":{"total":95995035648,"available":95965655040,"internally_reserved":0,"allocated":1712128,"data_stored":1164745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070508,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":519346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266740,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1036288,"data_stored":515148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462968,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1634304,"data_stored":1111470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626492,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":510426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1081344,"data_stored":568143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":511814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1110016,"data_stored":574149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1032192,"data_stored":510620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986247,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1032192,"data_stored":515467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822723,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36944,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708208,"statfs":{"total":95995035648,"available":95957204992,"internally_reserved":0,"allocated":1642496,"data_stored":1106667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":57807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T22:36:24.665 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph status --format=json 2024-05-17T22:36:24.919 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:36:24.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:24 smithi151 ceph-mon[36101]: from='client.14877 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T22:36:25.042 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:24 smithi012 ceph-mon[31632]: from='client.14877 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T22:36:25.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:24 smithi096 ceph-mon[36065]: from='client.14877 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T22:36:25.994 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:25 smithi151 ceph-mon[36101]: pgmap v354: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 517 B/s wr, 4 op/s 2024-05-17T22:36:26.025 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:25 smithi012 ceph-mon[31632]: pgmap v354: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 517 B/s wr, 4 op/s 2024-05-17T22:36:26.054 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:25 smithi096 ceph-mon[36065]: pgmap v354: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 517 B/s wr, 4 op/s 2024-05-17T22:36:26.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.597+0000 7f033494a700 1 -- 172.21.15.12:0/3264184275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0330102e20 msgr2=0x7f0330103230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:26.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.597+0000 7f033494a700 1 --2- 172.21.15.12:0/3264184275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0330102e20 0x7f0330103230 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0320009990 tx=0x7f0320009ca0 comp rx=0 tx=0).stop 2024-05-17T22:36:26.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.597+0000 7f033494a700 1 -- 172.21.15.12:0/3264184275 shutdown_connections 2024-05-17T22:36:26.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.597+0000 7f033494a700 1 --2- 172.21.15.12:0/3264184275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03301080a0 0x7f033010fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:26.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.597+0000 7f033494a700 1 --2- 172.21.15.12:0/3264184275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0330102e20 0x7f0330103230 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:26.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.597+0000 7f033494a700 1 --2- 172.21.15.12:0/3264184275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0330107750 0x7f0330107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:26.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.597+0000 7f033494a700 1 -- 172.21.15.12:0/3264184275 >> 172.21.15.12:0/3264184275 conn(0x7f0330079af0 msgr2=0x7f0330079f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:36:26.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.597+0000 7f033494a700 1 -- 172.21.15.12:0/3264184275 shutdown_connections 2024-05-17T22:36:26.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.597+0000 7f033494a700 1 -- 172.21.15.12:0/3264184275 wait complete. 2024-05-17T22:36:26.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.598+0000 7f033494a700 1 Processor -- start 2024-05-17T22:36:26.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.598+0000 7f033494a700 1 -- start start 2024-05-17T22:36:26.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.598+0000 7f033494a700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0330102e20 0x7f03301a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:26.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.598+0000 7f033494a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0330107750 0x7f03301a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:26.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.598+0000 7f033494a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03301080a0 0x7f033019a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:26.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.598+0000 7f033494a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0330112d20 con 0x7f03301080a0 2024-05-17T22:36:26.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.598+0000 7f033494a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0330112ba0 con 0x7f0330107750 2024-05-17T22:36:26.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.598+0000 7f033494a700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0330112ea0 con 0x7f0330102e20 2024-05-17T22:36:26.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.599+0000 7f032dd9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0330107750 0x7f03301a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:26.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.599+0000 7f032ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03301080a0 0x7f033019a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:26.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.599+0000 7f032e59c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0330102e20 0x7f03301a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:26.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.599+0000 7f032dd9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0330107750 0x7f03301a0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:49042/0 (socket says 172.21.15.12:49042) 2024-05-17T22:36:26.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.599+0000 7f032dd9b700 1 -- 172.21.15.12:0/1081638980 learned_addr learned my addr 172.21.15.12:0/1081638980 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:36:26.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.599+0000 7f032ed9d700 1 -- 172.21.15.12:0/1081638980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0330102e20 msgr2=0x7f03301a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:26.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.599+0000 7f032ed9d700 1 --2- 172.21.15.12:0/1081638980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0330102e20 0x7f03301a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:26.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.599+0000 7f032ed9d700 1 -- 172.21.15.12:0/1081638980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0330107750 msgr2=0x7f03301a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:26.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.599+0000 7f032ed9d700 1 --2- 172.21.15.12:0/1081638980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0330107750 0x7f03301a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:26.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.599+0000 7f032ed9d700 1 -- 172.21.15.12:0/1081638980 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f033019ae70 con 0x7f03301080a0 2024-05-17T22:36:26.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.599+0000 7f032ed9d700 1 --2- 172.21.15.12:0/1081638980 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03301080a0 0x7f033019a7a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f032800ab40 tx=0x7f032800ae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:36:26.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.599+0000 7f031f7fe700 1 -- 172.21.15.12:0/1081638980 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0328017020 con 0x7f03301080a0 2024-05-17T22:36:26.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.599+0000 7f033494a700 1 -- 172.21.15.12:0/1081638980 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f033019b160 con 0x7f03301080a0 2024-05-17T22:36:26.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.599+0000 7f031f7fe700 1 -- 172.21.15.12:0/1081638980 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0328004bb0 con 0x7f03301080a0 2024-05-17T22:36:26.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.599+0000 7f031f7fe700 1 -- 172.21.15.12:0/1081638980 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0328005280 con 0x7f03301080a0 2024-05-17T22:36:26.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.599+0000 7f033494a700 1 -- 172.21.15.12:0/1081638980 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03301a73c0 con 0x7f03301080a0 2024-05-17T22:36:26.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.601+0000 7f033494a700 1 -- 172.21.15.12:0/1081638980 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0330103230 con 0x7f03301080a0 2024-05-17T22:36:26.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.602+0000 7f031f7fe700 1 -- 172.21.15.12:0/1081638980 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f032800c580 con 0x7f03301080a0 2024-05-17T22:36:26.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.603+0000 7f031f7fe700 1 --2- 172.21.15.12:0/1081638980 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f03180b2910 0x7f03180b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:26.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.603+0000 7f032e59c700 1 --2- 172.21.15.12:0/1081638980 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f03180b2910 0x7f03180b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:26.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.603+0000 7f031f7fe700 1 -- 172.21.15.12:0/1081638980 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f03280e2250 con 0x7f03301080a0 2024-05-17T22:36:26.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.604+0000 7f032e59c700 1 --2- 172.21.15.12:0/1081638980 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f03180b2910 0x7f03180b4e20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0320009750 tx=0x7f032001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:36:26.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.606+0000 7f031f7fe700 1 -- 172.21.15.12:0/1081638980 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0328021020 con 0x7f03301080a0 2024-05-17T22:36:26.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.913+0000 7f033494a700 1 -- 172.21.15.12:0/1081638980 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f033019bcb0 con 0x7f03301080a0 2024-05-17T22:36:26.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.914+0000 7f031f7fe700 1 -- 172.21.15.12:0/1081638980 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1656 (secure 0 0 0) 0x7f03280a9ab0 con 0x7f03301080a0 2024-05-17T22:36:26.917 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:36:26.917 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"4d593780-149c-11ef-bc9a-c7b262605968","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":492,"monmap":{"epoch":3,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":89,"num_osds":12,"num_up_osds":12,"osd_up_since":1715985216,"num_in_osds":12,"osd_in_since":1715985195,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":129}],"num_pgs":129,"num_pools":4,"num_objects":64,"data_bytes":657066,"bytes_used":393314304,"bytes_avail":1151547113472,"bytes_total":1151940427776,"read_bytes_sec":3292,"write_bytes_sec":470,"read_op_per_sec":5,"write_op_per_sec":0},"fsmap":{"epoch":26,"id":1,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":1,"rank":2,"name":"f","status":"up:standby-replay","gid":24436},{"filesystem_id":1,"rank":1,"name":"e","status":"up:standby-replay","gid":24437},{"filesystem_id":1,"rank":0,"name":"c","status":"up:standby-replay","gid":24442},{"filesystem_id":1,"rank":2,"name":"i","status":"up:active","gid":24457},{"filesystem_id":1,"rank":1,"name":"h","status":"up:active","gid":24461},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":24482}],"up:standby":6},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.12:8443/"}},"servicemap":{"epoch":30,"modified":"2024-05-17T22:35:50.495746+0000","services":{"mds":{"daemons":{"summary":"","h":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-17T22:36:26.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.916+0000 7f033494a700 1 -- 172.21.15.12:0/1081638980 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f03180b2910 msgr2=0x7f03180b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:26.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.916+0000 7f033494a700 1 --2- 172.21.15.12:0/1081638980 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f03180b2910 0x7f03180b4e20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0320009750 tx=0x7f032001a040 comp rx=0 tx=0).stop 2024-05-17T22:36:26.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.916+0000 7f033494a700 1 -- 172.21.15.12:0/1081638980 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03301080a0 msgr2=0x7f033019a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:26.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.916+0000 7f033494a700 1 --2- 172.21.15.12:0/1081638980 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03301080a0 0x7f033019a7a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f032800ab40 tx=0x7f032800ae50 comp rx=0 tx=0).stop 2024-05-17T22:36:26.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.916+0000 7f033494a700 1 -- 172.21.15.12:0/1081638980 shutdown_connections 2024-05-17T22:36:26.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.916+0000 7f033494a700 1 --2- 172.21.15.12:0/1081638980 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f03180b2910 0x7f03180b4e20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:26.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.916+0000 7f033494a700 1 --2- 172.21.15.12:0/1081638980 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03301080a0 0x7f033019a7a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:26.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.916+0000 7f033494a700 1 --2- 172.21.15.12:0/1081638980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0330102e20 0x7f03301a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:26.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.916+0000 7f033494a700 1 --2- 172.21.15.12:0/1081638980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0330107750 0x7f03301a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:26.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.916+0000 7f033494a700 1 -- 172.21.15.12:0/1081638980 >> 172.21.15.12:0/1081638980 conn(0x7f0330079af0 msgr2=0x7f033006c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:36:26.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.916+0000 7f033494a700 1 -- 172.21.15.12:0/1081638980 shutdown_connections 2024-05-17T22:36:26.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:26.917+0000 7f033494a700 1 -- 172.21.15.12:0/1081638980 wait complete. 2024-05-17T22:36:27.536 INFO:tasks.cephadm.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-05-17T22:36:27.536 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-17T22:36:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:36:27.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:27 smithi012 ceph-mon[31632]: pgmap v355: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 470 B/s wr, 5 op/s 2024-05-17T22:36:27.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:27 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/1081638980' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-05-17T22:36:27.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:27 smithi151 ceph-mon[36101]: pgmap v355: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 470 B/s wr, 5 op/s 2024-05-17T22:36:27.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:27 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/1081638980' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-05-17T22:36:28.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:27 smithi096 ceph-mon[36065]: pgmap v355: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 470 B/s wr, 5 op/s 2024-05-17T22:36:28.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:27 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/1081638980' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-05-17T22:36:29.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.518+0000 7f7f911f8700 1 -- 172.21.15.12:0/3982522512 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f8c103f10 msgr2=0x7f7f8c104320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:29.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.518+0000 7f7f911f8700 1 --2- 172.21.15.12:0/3982522512 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f8c103f10 0x7f7f8c104320 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c009990 tx=0x7f7f7c009ca0 comp rx=0 tx=0).stop 2024-05-17T22:36:29.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.518+0000 7f7f911f8700 1 -- 172.21.15.12:0/3982522512 shutdown_connections 2024-05-17T22:36:29.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.518+0000 7f7f911f8700 1 --2- 172.21.15.12:0/3982522512 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7f8c109190 0x7f7f8c110af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:29.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.518+0000 7f7f911f8700 1 --2- 172.21.15.12:0/3982522512 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f8c103f10 0x7f7f8c104320 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:29.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.518+0000 7f7f911f8700 1 --2- 172.21.15.12:0/3982522512 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f8c108840 0x7f7f8c108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:29.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.518+0000 7f7f911f8700 1 -- 172.21.15.12:0/3982522512 >> 172.21.15.12:0/3982522512 conn(0x7f7f8c0ff990 msgr2=0x7f7f8c101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:36:29.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.518+0000 7f7f911f8700 1 -- 172.21.15.12:0/3982522512 shutdown_connections 2024-05-17T22:36:29.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.519+0000 7f7f911f8700 1 -- 172.21.15.12:0/3982522512 wait complete. 2024-05-17T22:36:29.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.520+0000 7f7f911f8700 1 Processor -- start 2024-05-17T22:36:29.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.520+0000 7f7f911f8700 1 -- start start 2024-05-17T22:36:29.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.520+0000 7f7f911f8700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7f8c103f10 0x7f7f8c075b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:29.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.520+0000 7f7f911f8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f8c108840 0x7f7f8c076070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:29.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.520+0000 7f7f911f8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f8c109190 0x7f7f8c0765b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:29.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.520+0000 7f7f911f8700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7f8c113dc0 con 0x7f7f8c108840 2024-05-17T22:36:29.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.520+0000 7f7f911f8700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7f8c113c40 con 0x7f7f8c109190 2024-05-17T22:36:29.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.520+0000 7f7f911f8700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7f8c113f40 con 0x7f7f8c103f10 2024-05-17T22:36:29.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.520+0000 7f7f8a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f8c108840 0x7f7f8c076070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:29.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.520+0000 7f7f8a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f8c108840 0x7f7f8c076070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60340/0 (socket says 172.21.15.12:60340) 2024-05-17T22:36:29.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.520+0000 7f7f8a59c700 1 -- 172.21.15.12:0/3734576992 learned_addr learned my addr 172.21.15.12:0/3734576992 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:36:29.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.521+0000 7f7f8b59e700 1 --2- 172.21.15.12:0/3734576992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f8c109190 0x7f7f8c0765b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:29.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.521+0000 7f7f8ad9d700 1 --2- 172.21.15.12:0/3734576992 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7f8c103f10 0x7f7f8c075b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:29.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.521+0000 7f7f8a59c700 1 -- 172.21.15.12:0/3734576992 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7f8c103f10 msgr2=0x7f7f8c075b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:29.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.521+0000 7f7f8a59c700 1 --2- 172.21.15.12:0/3734576992 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7f8c103f10 0x7f7f8c075b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:29.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.521+0000 7f7f8a59c700 1 -- 172.21.15.12:0/3734576992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f8c109190 msgr2=0x7f7f8c0765b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:29.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.521+0000 7f7f8a59c700 1 --2- 172.21.15.12:0/3734576992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f8c109190 0x7f7f8c0765b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:29.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.521+0000 7f7f8a59c700 1 -- 172.21.15.12:0/3734576992 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f8c1a8bc0 con 0x7f7f8c108840 2024-05-17T22:36:29.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.521+0000 7f7f8b59e700 1 --2- 172.21.15.12:0/3734576992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f8c109190 0x7f7f8c0765b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:36:29.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.521+0000 7f7f8ad9d700 1 --2- 172.21.15.12:0/3734576992 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7f8c103f10 0x7f7f8c075b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:36:29.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.521+0000 7f7f8a59c700 1 --2- 172.21.15.12:0/3734576992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f8c108840 0x7f7f8c076070 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7f7400d980 tx=0x7f7f7400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:36:29.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.521+0000 7f7f83fff700 1 -- 172.21.15.12:0/3734576992 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f74017070 con 0x7f7f8c108840 2024-05-17T22:36:29.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.521+0000 7f7f83fff700 1 -- 172.21.15.12:0/3734576992 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f74010460 con 0x7f7f8c108840 2024-05-17T22:36:29.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.521+0000 7f7f83fff700 1 -- 172.21.15.12:0/3734576992 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7400de60 con 0x7f7f8c108840 2024-05-17T22:36:29.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.521+0000 7f7f911f8700 1 -- 172.21.15.12:0/3734576992 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f8c1a8eb0 con 0x7f7f8c108840 2024-05-17T22:36:29.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.521+0000 7f7f911f8700 1 -- 172.21.15.12:0/3734576992 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f8c1a9310 con 0x7f7f8c108840 2024-05-17T22:36:29.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.523+0000 7f7f911f8700 1 -- 172.21.15.12:0/3734576992 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f8c104320 con 0x7f7f8c108840 2024-05-17T22:36:29.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.525+0000 7f7f83fff700 1 -- 172.21.15.12:0/3734576992 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f7f74010610 con 0x7f7f8c108840 2024-05-17T22:36:29.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.525+0000 7f7f83fff700 1 --2- 172.21.15.12:0/3734576992 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f7f780b2910 0x7f7f780b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:29.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.525+0000 7f7f8ad9d700 1 --2- 172.21.15.12:0/3734576992 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f7f780b2910 0x7f7f780b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:29.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.526+0000 7f7f83fff700 1 -- 172.21.15.12:0/3734576992 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f7f740e22b0 con 0x7f7f8c108840 2024-05-17T22:36:29.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.526+0000 7f7f8ad9d700 1 --2- 172.21.15.12:0/3734576992 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f7f780b2910 0x7f7f780b4e20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c003800 tx=0x7f7f7c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:36:29.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.528+0000 7f7f83fff700 1 -- 172.21.15.12:0/3734576992 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7f740a9ae0 con 0x7f7f8c108840 2024-05-17T22:36:29.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.738+0000 7f7f911f8700 1 -- 172.21.15.12:0/3734576992 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7f8c02ea80 con 0x7f7f780b2910 2024-05-17T22:36:29.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.744+0000 7f7f83fff700 1 -- 172.21.15.12:0/3734576992 <== mgr.14152 v2:172.21.15.12:6800/1388514047 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340349 (secure 0 0 0) 0x7f7f8c02ea80 con 0x7f7f780b2910 2024-05-17T22:36:29.751 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:36:29.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.747+0000 7f7f911f8700 1 -- 172.21.15.12:0/3734576992 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f7f780b2910 msgr2=0x7f7f780b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:29.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.747+0000 7f7f911f8700 1 --2- 172.21.15.12:0/3734576992 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f7f780b2910 0x7f7f780b4e20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c003800 tx=0x7f7f7c005fb0 comp rx=0 tx=0).stop 2024-05-17T22:36:29.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.747+0000 7f7f911f8700 1 -- 172.21.15.12:0/3734576992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f8c108840 msgr2=0x7f7f8c076070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:29.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.747+0000 7f7f911f8700 1 --2- 172.21.15.12:0/3734576992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f8c108840 0x7f7f8c076070 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7f7400d980 tx=0x7f7f7400dd40 comp rx=0 tx=0).stop 2024-05-17T22:36:29.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.747+0000 7f7f911f8700 1 -- 172.21.15.12:0/3734576992 shutdown_connections 2024-05-17T22:36:29.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.747+0000 7f7f911f8700 1 --2- 172.21.15.12:0/3734576992 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f7f780b2910 0x7f7f780b4e20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:29.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.747+0000 7f7f911f8700 1 --2- 172.21.15.12:0/3734576992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f8c109190 0x7f7f8c0765b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:29.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.747+0000 7f7f911f8700 1 --2- 172.21.15.12:0/3734576992 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7f8c103f10 0x7f7f8c075b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:29.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.747+0000 7f7f911f8700 1 --2- 172.21.15.12:0/3734576992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f8c108840 0x7f7f8c076070 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:29.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.748+0000 7f7f911f8700 1 -- 172.21.15.12:0/3734576992 >> 172.21.15.12:0/3734576992 conn(0x7f7f8c0ff990 msgr2=0x7f7f8c101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:36:29.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.748+0000 7f7f911f8700 1 -- 172.21.15.12:0/3734576992 shutdown_connections 2024-05-17T22:36:29.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:29.748+0000 7f7f911f8700 1 -- 172.21.15.12:0/3734576992 wait complete. 2024-05-17T22:36:29.755 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-05-17T22:36:29.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:29 smithi012 ceph-mon[31632]: pgmap v356: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-17T22:36:29.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:29 smithi151 ceph-mon[36101]: pgmap v356: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-17T22:36:30.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:29 smithi096 ceph-mon[36065]: pgmap v356: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-17T22:36:30.550 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":356,"stamp":"2024-05-17T22:36:28.507216+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657066,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":320,"num_read_kb":253,"num_write":294,"num_write_kb":2270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":238,"ondisk_log_size":238,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":384096,"kb_used_data":14084,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124557728,"statfs":{"total":1151940427776,"available":1151547113472,"internally_reserved":0,"allocated":14422016,"data_stored":8127296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4265,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":71,"num_read_kb":35,"num_write":3,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003449"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628725+0000","last_change":"2024-05-17T22:34:40.422850+0000","last_active":"2024-05-17T22:36:15.628725+0000","last_peered":"2024-05-17T22:36:15.628725+0000","last_clean":"2024-05-17T22:36:15.628725+0000","last_became_active":"2024-05-17T22:34:40.422474+0000","last_became_peered":"2024-05-17T22:34:40.422474+0000","last_unstale":"2024-05-17T22:36:15.628725+0000","last_undegraded":"2024-05-17T22:36:15.628725+0000","last_fullsized":"2024-05-17T22:36:15.628725+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:23:51.537128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968741+0000","last_change":"2024-05-17T22:34:40.427907+0000","last_active":"2024-05-17T22:36:15.968741+0000","last_peered":"2024-05-17T22:36:15.968741+0000","last_clean":"2024-05-17T22:36:15.968741+0000","last_became_active":"2024-05-17T22:34:40.427511+0000","last_became_peered":"2024-05-17T22:34:40.427511+0000","last_unstale":"2024-05-17T22:36:15.968741+0000","last_undegraded":"2024-05-17T22:36:15.968741+0000","last_fullsized":"2024-05-17T22:36:15.968741+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:01:46.707820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968443+0000","last_change":"2024-05-17T22:34:40.436370+0000","last_active":"2024-05-17T22:36:15.968443+0000","last_peered":"2024-05-17T22:36:15.968443+0000","last_clean":"2024-05-17T22:36:15.968443+0000","last_became_active":"2024-05-17T22:34:40.434089+0000","last_became_peered":"2024-05-17T22:34:40.434089+0000","last_unstale":"2024-05-17T22:36:15.968443+0000","last_undegraded":"2024-05-17T22:36:15.968443+0000","last_fullsized":"2024-05-17T22:36:15.968443+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:58:16.775471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623409+0000","last_change":"2024-05-17T22:35:15.556681+0000","last_active":"2024-05-17T22:36:15.623409+0000","last_peered":"2024-05-17T22:36:15.623409+0000","last_clean":"2024-05-17T22:36:15.623409+0000","last_became_active":"2024-05-17T22:35:15.556427+0000","last_became_peered":"2024-05-17T22:35:15.556427+0000","last_unstale":"2024-05-17T22:36:15.623409+0000","last_undegraded":"2024-05-17T22:36:15.623409+0000","last_fullsized":"2024-05-17T22:36:15.623409+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:28:01.726262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625720+0000","last_change":"2024-05-17T22:34:40.434394+0000","last_active":"2024-05-17T22:36:15.625720+0000","last_peered":"2024-05-17T22:36:15.625720+0000","last_clean":"2024-05-17T22:36:15.625720+0000","last_became_active":"2024-05-17T22:34:40.433489+0000","last_became_peered":"2024-05-17T22:34:40.433489+0000","last_unstale":"2024-05-17T22:36:15.625720+0000","last_undegraded":"2024-05-17T22:36:15.625720+0000","last_fullsized":"2024-05-17T22:36:15.625720+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:26:24.068814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625953+0000","last_change":"2024-05-17T22:35:15.553601+0000","last_active":"2024-05-17T22:36:15.625953+0000","last_peered":"2024-05-17T22:36:15.625953+0000","last_clean":"2024-05-17T22:36:15.625953+0000","last_became_active":"2024-05-17T22:35:15.553116+0000","last_became_peered":"2024-05-17T22:35:15.553116+0000","last_unstale":"2024-05-17T22:36:15.625953+0000","last_undegraded":"2024-05-17T22:36:15.625953+0000","last_fullsized":"2024-05-17T22:36:15.625953+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:09:38.401107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004274+0000","last_change":"2024-05-17T22:34:40.430479+0000","last_active":"2024-05-17T22:36:16.004274+0000","last_peered":"2024-05-17T22:36:16.004274+0000","last_clean":"2024-05-17T22:36:16.004274+0000","last_became_active":"2024-05-17T22:34:40.430025+0000","last_became_peered":"2024-05-17T22:34:40.430025+0000","last_unstale":"2024-05-17T22:36:16.004274+0000","last_undegraded":"2024-05-17T22:36:16.004274+0000","last_fullsized":"2024-05-17T22:36:16.004274+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:14:19.725096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984193+0000","last_change":"2024-05-17T22:34:40.431711+0000","last_active":"2024-05-17T22:36:15.984193+0000","last_peered":"2024-05-17T22:36:15.984193+0000","last_clean":"2024-05-17T22:36:15.984193+0000","last_became_active":"2024-05-17T22:34:40.431368+0000","last_became_peered":"2024-05-17T22:34:40.431368+0000","last_unstale":"2024-05-17T22:36:15.984193+0000","last_undegraded":"2024-05-17T22:36:15.984193+0000","last_fullsized":"2024-05-17T22:36:15.984193+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:06:34.904652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.805730+0000","last_change":"2024-05-17T22:34:40.435593+0000","last_active":"2024-05-17T22:36:15.805730+0000","last_peered":"2024-05-17T22:36:15.805730+0000","last_clean":"2024-05-17T22:36:15.805730+0000","last_became_active":"2024-05-17T22:34:40.435241+0000","last_became_peered":"2024-05-17T22:34:40.435241+0000","last_unstale":"2024-05-17T22:36:15.805730+0000","last_undegraded":"2024-05-17T22:36:15.805730+0000","last_fullsized":"2024-05-17T22:36:15.805730+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:08:57.239998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.626366+0000","last_change":"2024-05-17T22:34:40.436248+0000","last_active":"2024-05-17T22:36:15.626366+0000","last_peered":"2024-05-17T22:36:15.626366+0000","last_clean":"2024-05-17T22:36:15.626366+0000","last_became_active":"2024-05-17T22:34:40.435824+0000","last_became_peered":"2024-05-17T22:34:40.435824+0000","last_unstale":"2024-05-17T22:36:15.626366+0000","last_undegraded":"2024-05-17T22:36:15.626366+0000","last_fullsized":"2024-05-17T22:36:15.626366+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:39:07.297470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.005145+0000","last_change":"2024-05-17T22:34:40.435843+0000","last_active":"2024-05-17T22:36:16.005145+0000","last_peered":"2024-05-17T22:36:16.005145+0000","last_clean":"2024-05-17T22:36:16.005145+0000","last_became_active":"2024-05-17T22:34:40.435500+0000","last_became_peered":"2024-05-17T22:34:40.435500+0000","last_unstale":"2024-05-17T22:36:16.005145+0000","last_undegraded":"2024-05-17T22:36:16.005145+0000","last_fullsized":"2024-05-17T22:36:16.005145+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:11:28.115850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004758+0000","last_change":"2024-05-17T22:34:40.432044+0000","last_active":"2024-05-17T22:36:16.004758+0000","last_peered":"2024-05-17T22:36:16.004758+0000","last_clean":"2024-05-17T22:36:16.004758+0000","last_became_active":"2024-05-17T22:34:40.429141+0000","last_became_peered":"2024-05-17T22:34:40.429141+0000","last_unstale":"2024-05-17T22:36:16.004758+0000","last_undegraded":"2024-05-17T22:36:16.004758+0000","last_fullsized":"2024-05-17T22:36:16.004758+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:04:53.428681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685924+0000","last_change":"2024-05-17T22:34:40.438474+0000","last_active":"2024-05-17T22:36:15.685924+0000","last_peered":"2024-05-17T22:36:15.685924+0000","last_clean":"2024-05-17T22:36:15.685924+0000","last_became_active":"2024-05-17T22:34:40.438109+0000","last_became_peered":"2024-05-17T22:34:40.438109+0000","last_unstale":"2024-05-17T22:36:15.685924+0000","last_undegraded":"2024-05-17T22:36:15.685924+0000","last_fullsized":"2024-05-17T22:36:15.685924+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:26:00.665246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968785+0000","last_change":"2024-05-17T22:34:40.431563+0000","last_active":"2024-05-17T22:36:15.968785+0000","last_peered":"2024-05-17T22:36:15.968785+0000","last_clean":"2024-05-17T22:36:15.968785+0000","last_became_active":"2024-05-17T22:34:40.430992+0000","last_became_peered":"2024-05-17T22:34:40.430992+0000","last_unstale":"2024-05-17T22:36:15.968785+0000","last_undegraded":"2024-05-17T22:36:15.968785+0000","last_fullsized":"2024-05-17T22:36:15.968785+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:51:26.280440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968328+0000","last_change":"2024-05-17T22:34:40.433629+0000","last_active":"2024-05-17T22:36:15.968328+0000","last_peered":"2024-05-17T22:36:15.968328+0000","last_clean":"2024-05-17T22:36:15.968328+0000","last_became_active":"2024-05-17T22:34:40.433270+0000","last_became_peered":"2024-05-17T22:34:40.433270+0000","last_unstale":"2024-05-17T22:36:15.968328+0000","last_undegraded":"2024-05-17T22:36:15.968328+0000","last_fullsized":"2024-05-17T22:36:15.968328+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:57:51.164078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.805030+0000","last_change":"2024-05-17T22:34:40.434218+0000","last_active":"2024-05-17T22:36:15.805030+0000","last_peered":"2024-05-17T22:36:15.805030+0000","last_clean":"2024-05-17T22:36:15.805030+0000","last_became_active":"2024-05-17T22:34:40.433849+0000","last_became_peered":"2024-05-17T22:34:40.433849+0000","last_unstale":"2024-05-17T22:36:15.805030+0000","last_undegraded":"2024-05-17T22:36:15.805030+0000","last_fullsized":"2024-05-17T22:36:15.805030+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:41:55.984109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628531+0000","last_change":"2024-05-17T22:34:40.427892+0000","last_active":"2024-05-17T22:36:15.628531+0000","last_peered":"2024-05-17T22:36:15.628531+0000","last_clean":"2024-05-17T22:36:15.628531+0000","last_became_active":"2024-05-17T22:34:40.427303+0000","last_became_peered":"2024-05-17T22:34:40.427303+0000","last_unstale":"2024-05-17T22:36:15.628531+0000","last_undegraded":"2024-05-17T22:36:15.628531+0000","last_fullsized":"2024-05-17T22:36:15.628531+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:59:38.632123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623636+0000","last_change":"2024-05-17T22:34:40.424784+0000","last_active":"2024-05-17T22:36:15.623636+0000","last_peered":"2024-05-17T22:36:15.623636+0000","last_clean":"2024-05-17T22:36:15.623636+0000","last_became_active":"2024-05-17T22:34:40.424439+0000","last_became_peered":"2024-05-17T22:34:40.424439+0000","last_unstale":"2024-05-17T22:36:15.623636+0000","last_undegraded":"2024-05-17T22:36:15.623636+0000","last_fullsized":"2024-05-17T22:36:15.623636+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:02:53.681600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624433+0000","last_change":"2024-05-17T22:34:40.435023+0000","last_active":"2024-05-17T22:36:15.624433+0000","last_peered":"2024-05-17T22:36:15.624433+0000","last_clean":"2024-05-17T22:36:15.624433+0000","last_became_active":"2024-05-17T22:34:40.434441+0000","last_became_peered":"2024-05-17T22:34:40.434441+0000","last_unstale":"2024-05-17T22:36:15.624433+0000","last_undegraded":"2024-05-17T22:36:15.624433+0000","last_fullsized":"2024-05-17T22:36:15.624433+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:16:07.671897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623973+0000","last_change":"2024-05-17T22:34:52.493192+0000","last_active":"2024-05-17T22:36:15.623973+0000","last_peered":"2024-05-17T22:36:15.623973+0000","last_clean":"2024-05-17T22:36:15.623973+0000","last_became_active":"2024-05-17T22:34:52.492786+0000","last_became_peered":"2024-05-17T22:34:52.492786+0000","last_unstale":"2024-05-17T22:36:15.623973+0000","last_undegraded":"2024-05-17T22:36:15.623973+0000","last_fullsized":"2024-05-17T22:36:15.623973+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:43:40.152240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623754+0000","last_change":"2024-05-17T22:34:37.402360+0000","last_active":"2024-05-17T22:36:15.623754+0000","last_peered":"2024-05-17T22:36:15.623754+0000","last_clean":"2024-05-17T22:36:15.623754+0000","last_became_active":"2024-05-17T22:34:37.401882+0000","last_became_peered":"2024-05-17T22:34:37.401882+0000","last_unstale":"2024-05-17T22:36:15.623754+0000","last_undegraded":"2024-05-17T22:36:15.623754+0000","last_fullsized":"2024-05-17T22:36:15.623754+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:53:13.300626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984571+0000","last_change":"2024-05-17T22:34:40.436259+0000","last_active":"2024-05-17T22:36:15.984571+0000","last_peered":"2024-05-17T22:36:15.984571+0000","last_clean":"2024-05-17T22:36:15.984571+0000","last_became_active":"2024-05-17T22:34:40.435899+0000","last_became_peered":"2024-05-17T22:34:40.435899+0000","last_unstale":"2024-05-17T22:36:15.984571+0000","last_undegraded":"2024-05-17T22:36:15.984571+0000","last_fullsized":"2024-05-17T22:36:15.984571+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:26:29.327513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003655+0000","last_change":"2024-05-17T22:34:52.495236+0000","last_active":"2024-05-17T22:36:16.003655+0000","last_peered":"2024-05-17T22:36:16.003655+0000","last_clean":"2024-05-17T22:36:16.003655+0000","last_became_active":"2024-05-17T22:34:52.494783+0000","last_became_peered":"2024-05-17T22:34:52.494783+0000","last_unstale":"2024-05-17T22:36:16.003655+0000","last_undegraded":"2024-05-17T22:36:16.003655+0000","last_fullsized":"2024-05-17T22:36:16.003655+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:08:15.010706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984767+0000","last_change":"2024-05-17T22:34:37.441129+0000","last_active":"2024-05-17T22:36:15.984767+0000","last_peered":"2024-05-17T22:36:15.984767+0000","last_clean":"2024-05-17T22:36:15.984767+0000","last_became_active":"2024-05-17T22:34:37.440738+0000","last_became_peered":"2024-05-17T22:34:37.440738+0000","last_unstale":"2024-05-17T22:36:15.984767+0000","last_undegraded":"2024-05-17T22:36:15.984767+0000","last_fullsized":"2024-05-17T22:36:15.984767+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:03:16.945076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968555+0000","last_change":"2024-05-17T22:34:40.434146+0000","last_active":"2024-05-17T22:36:15.968555+0000","last_peered":"2024-05-17T22:36:15.968555+0000","last_clean":"2024-05-17T22:36:15.968555+0000","last_became_active":"2024-05-17T22:34:40.433802+0000","last_became_peered":"2024-05-17T22:34:40.433802+0000","last_unstale":"2024-05-17T22:36:15.968555+0000","last_undegraded":"2024-05-17T22:36:15.968555+0000","last_fullsized":"2024-05-17T22:36:15.968555+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:08:39.905409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003713+0000","last_change":"2024-05-17T22:34:52.501134+0000","last_active":"2024-05-17T22:36:16.003713+0000","last_peered":"2024-05-17T22:36:16.003713+0000","last_clean":"2024-05-17T22:36:16.003713+0000","last_became_active":"2024-05-17T22:34:52.500547+0000","last_became_peered":"2024-05-17T22:34:52.500547+0000","last_unstale":"2024-05-17T22:36:16.003713+0000","last_undegraded":"2024-05-17T22:36:16.003713+0000","last_fullsized":"2024-05-17T22:36:16.003713+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:13:04.200865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.626343+0000","last_change":"2024-05-17T22:34:37.404019+0000","last_active":"2024-05-17T22:36:15.626343+0000","last_peered":"2024-05-17T22:36:15.626343+0000","last_clean":"2024-05-17T22:36:15.626343+0000","last_became_active":"2024-05-17T22:34:37.403607+0000","last_became_peered":"2024-05-17T22:34:37.403607+0000","last_unstale":"2024-05-17T22:36:15.626343+0000","last_undegraded":"2024-05-17T22:36:15.626343+0000","last_fullsized":"2024-05-17T22:36:15.626343+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:30:21.405195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.805310+0000","last_change":"2024-05-17T22:34:40.426889+0000","last_active":"2024-05-17T22:36:15.805310+0000","last_peered":"2024-05-17T22:36:15.805310+0000","last_clean":"2024-05-17T22:36:15.805310+0000","last_became_active":"2024-05-17T22:34:40.426512+0000","last_became_peered":"2024-05-17T22:34:40.426512+0000","last_unstale":"2024-05-17T22:36:15.805310+0000","last_undegraded":"2024-05-17T22:36:15.805310+0000","last_fullsized":"2024-05-17T22:36:15.805310+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:30:51.363177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003876+0000","last_change":"2024-05-17T22:34:52.487514+0000","last_active":"2024-05-17T22:36:16.003876+0000","last_peered":"2024-05-17T22:36:16.003876+0000","last_clean":"2024-05-17T22:36:16.003876+0000","last_became_active":"2024-05-17T22:34:52.487068+0000","last_became_peered":"2024-05-17T22:34:52.487068+0000","last_unstale":"2024-05-17T22:36:16.003876+0000","last_undegraded":"2024-05-17T22:36:16.003876+0000","last_fullsized":"2024-05-17T22:36:16.003876+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:52:13.573986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.626041+0000","last_change":"2024-05-17T22:34:37.400518+0000","last_active":"2024-05-17T22:36:15.626041+0000","last_peered":"2024-05-17T22:36:15.626041+0000","last_clean":"2024-05-17T22:36:15.626041+0000","last_became_active":"2024-05-17T22:34:37.400145+0000","last_became_peered":"2024-05-17T22:34:37.400145+0000","last_unstale":"2024-05-17T22:36:15.626041+0000","last_undegraded":"2024-05-17T22:36:15.626041+0000","last_fullsized":"2024-05-17T22:36:15.626041+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:56:53.593624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004156+0000","last_change":"2024-05-17T22:34:40.434148+0000","last_active":"2024-05-17T22:36:16.004156+0000","last_peered":"2024-05-17T22:36:16.004156+0000","last_clean":"2024-05-17T22:36:16.004156+0000","last_became_active":"2024-05-17T22:34:40.433613+0000","last_became_peered":"2024-05-17T22:34:40.433613+0000","last_unstale":"2024-05-17T22:36:16.004156+0000","last_undegraded":"2024-05-17T22:36:16.004156+0000","last_fullsized":"2024-05-17T22:36:16.004156+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:09:39.049852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624284+0000","last_change":"2024-05-17T22:34:52.490417+0000","last_active":"2024-05-17T22:36:15.624284+0000","last_peered":"2024-05-17T22:36:15.624284+0000","last_clean":"2024-05-17T22:36:15.624284+0000","last_became_active":"2024-05-17T22:34:52.489953+0000","last_became_peered":"2024-05-17T22:34:52.489953+0000","last_unstale":"2024-05-17T22:36:15.624284+0000","last_undegraded":"2024-05-17T22:36:15.624284+0000","last_fullsized":"2024-05-17T22:36:15.624284+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:23:56.491899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":91,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:26.099583+0000","last_change":"2024-05-17T22:34:37.405304+0000","last_active":"2024-05-17T22:36:26.099583+0000","last_peered":"2024-05-17T22:36:26.099583+0000","last_clean":"2024-05-17T22:36:26.099583+0000","last_became_active":"2024-05-17T22:34:37.404552+0000","last_became_peered":"2024-05-17T22:34:37.404552+0000","last_unstale":"2024-05-17T22:36:26.099583+0000","last_undegraded":"2024-05-17T22:36:26.099583+0000","last_fullsized":"2024-05-17T22:36:26.099583+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:57:06.556886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":41,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625732+0000","last_change":"2024-05-17T22:34:40.430537+0000","last_active":"2024-05-17T22:36:15.625732+0000","last_peered":"2024-05-17T22:36:15.625732+0000","last_clean":"2024-05-17T22:36:15.625732+0000","last_became_active":"2024-05-17T22:34:40.430193+0000","last_became_peered":"2024-05-17T22:34:40.430193+0000","last_unstale":"2024-05-17T22:36:15.625732+0000","last_undegraded":"2024-05-17T22:36:15.625732+0000","last_fullsized":"2024-05-17T22:36:15.625732+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:58:42.091445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624333+0000","last_change":"2024-05-17T22:35:15.559077+0000","last_active":"2024-05-17T22:36:15.624333+0000","last_peered":"2024-05-17T22:36:15.624333+0000","last_clean":"2024-05-17T22:36:15.624333+0000","last_became_active":"2024-05-17T22:35:15.558382+0000","last_became_peered":"2024-05-17T22:35:15.558382+0000","last_unstale":"2024-05-17T22:36:15.624333+0000","last_undegraded":"2024-05-17T22:36:15.624333+0000","last_fullsized":"2024-05-17T22:36:15.624333+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:20:16.009620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625951+0000","last_change":"2024-05-17T22:34:37.405332+0000","last_active":"2024-05-17T22:36:15.625951+0000","last_peered":"2024-05-17T22:36:15.625951+0000","last_clean":"2024-05-17T22:36:15.625951+0000","last_became_active":"2024-05-17T22:34:37.404970+0000","last_became_peered":"2024-05-17T22:34:37.404970+0000","last_unstale":"2024-05-17T22:36:15.625951+0000","last_undegraded":"2024-05-17T22:36:15.625951+0000","last_fullsized":"2024-05-17T22:36:15.625951+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:37:39.933686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003790+0000","last_change":"2024-05-17T22:34:40.432690+0000","last_active":"2024-05-17T22:36:16.003790+0000","last_peered":"2024-05-17T22:36:16.003790+0000","last_clean":"2024-05-17T22:36:16.003790+0000","last_became_active":"2024-05-17T22:34:40.432274+0000","last_became_peered":"2024-05-17T22:34:40.432274+0000","last_unstale":"2024-05-17T22:36:16.003790+0000","last_undegraded":"2024-05-17T22:36:16.003790+0000","last_fullsized":"2024-05-17T22:36:16.003790+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:12:47.926865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628515+0000","last_change":"2024-05-17T22:34:52.494645+0000","last_active":"2024-05-17T22:36:15.628515+0000","last_peered":"2024-05-17T22:36:15.628515+0000","last_clean":"2024-05-17T22:36:15.628515+0000","last_became_active":"2024-05-17T22:34:52.494255+0000","last_became_peered":"2024-05-17T22:34:52.494255+0000","last_unstale":"2024-05-17T22:36:15.628515+0000","last_undegraded":"2024-05-17T22:36:15.628515+0000","last_fullsized":"2024-05-17T22:36:15.628515+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:41:26.633571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685352+0000","last_change":"2024-05-17T22:34:37.398582+0000","last_active":"2024-05-17T22:36:15.685352+0000","last_peered":"2024-05-17T22:36:15.685352+0000","last_clean":"2024-05-17T22:36:15.685352+0000","last_became_active":"2024-05-17T22:34:37.398156+0000","last_became_peered":"2024-05-17T22:34:37.398156+0000","last_unstale":"2024-05-17T22:36:15.685352+0000","last_undegraded":"2024-05-17T22:36:15.685352+0000","last_fullsized":"2024-05-17T22:36:15.685352+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:46:00.276214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.622638+0000","last_change":"2024-05-17T22:35:15.551402+0000","last_active":"2024-05-17T22:36:15.622638+0000","last_peered":"2024-05-17T22:36:15.622638+0000","last_clean":"2024-05-17T22:36:15.622638+0000","last_became_active":"2024-05-17T22:35:15.551021+0000","last_became_peered":"2024-05-17T22:35:15.551021+0000","last_unstale":"2024-05-17T22:36:15.622638+0000","last_undegraded":"2024-05-17T22:36:15.622638+0000","last_fullsized":"2024-05-17T22:36:15.622638+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:35:18.573620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968732+0000","last_change":"2024-05-17T22:34:52.487413+0000","last_active":"2024-05-17T22:36:15.968732+0000","last_peered":"2024-05-17T22:36:15.968732+0000","last_clean":"2024-05-17T22:36:15.968732+0000","last_became_active":"2024-05-17T22:34:52.487032+0000","last_became_peered":"2024-05-17T22:34:52.487032+0000","last_unstale":"2024-05-17T22:36:15.968732+0000","last_undegraded":"2024-05-17T22:36:15.968732+0000","last_fullsized":"2024-05-17T22:36:15.968732+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:34:49.122008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003249+0000","last_change":"2024-05-17T22:34:37.404894+0000","last_active":"2024-05-17T22:36:16.003249+0000","last_peered":"2024-05-17T22:36:16.003249+0000","last_clean":"2024-05-17T22:36:16.003249+0000","last_became_active":"2024-05-17T22:34:37.404567+0000","last_became_peered":"2024-05-17T22:34:37.404567+0000","last_unstale":"2024-05-17T22:36:16.003249+0000","last_undegraded":"2024-05-17T22:36:16.003249+0000","last_fullsized":"2024-05-17T22:36:16.003249+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:13:09.761556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623009+0000","last_change":"2024-05-17T22:34:40.418267+0000","last_active":"2024-05-17T22:36:15.623009+0000","last_peered":"2024-05-17T22:36:15.623009+0000","last_clean":"2024-05-17T22:36:15.623009+0000","last_became_active":"2024-05-17T22:34:40.417928+0000","last_became_peered":"2024-05-17T22:34:40.417928+0000","last_unstale":"2024-05-17T22:36:15.623009+0000","last_undegraded":"2024-05-17T22:36:15.623009+0000","last_fullsized":"2024-05-17T22:36:15.623009+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:17:37.055004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984148+0000","last_change":"2024-05-17T22:34:52.492126+0000","last_active":"2024-05-17T22:36:15.984148+0000","last_peered":"2024-05-17T22:36:15.984148+0000","last_clean":"2024-05-17T22:36:15.984148+0000","last_became_active":"2024-05-17T22:34:52.491695+0000","last_became_peered":"2024-05-17T22:34:52.491695+0000","last_unstale":"2024-05-17T22:36:15.984148+0000","last_undegraded":"2024-05-17T22:36:15.984148+0000","last_fullsized":"2024-05-17T22:36:15.984148+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:15:17.710279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":82,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:23.094276+0000","last_change":"2024-05-17T22:34:37.406989+0000","last_active":"2024-05-17T22:36:23.094276+0000","last_peered":"2024-05-17T22:36:23.094276+0000","last_clean":"2024-05-17T22:36:23.094276+0000","last_became_active":"2024-05-17T22:34:37.406612+0000","last_became_peered":"2024-05-17T22:34:37.406612+0000","last_unstale":"2024-05-17T22:36:23.094276+0000","last_undegraded":"2024-05-17T22:36:23.094276+0000","last_fullsized":"2024-05-17T22:36:23.094276+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:47:52.480804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.805642+0000","last_change":"2024-05-17T22:34:40.427221+0000","last_active":"2024-05-17T22:36:15.805642+0000","last_peered":"2024-05-17T22:36:15.805642+0000","last_clean":"2024-05-17T22:36:15.805642+0000","last_became_active":"2024-05-17T22:34:40.426703+0000","last_became_peered":"2024-05-17T22:34:40.426703+0000","last_unstale":"2024-05-17T22:36:15.805642+0000","last_undegraded":"2024-05-17T22:36:15.805642+0000","last_fullsized":"2024-05-17T22:36:15.805642+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:49:15.629048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.622645+0000","last_change":"2024-05-17T22:34:52.499086+0000","last_active":"2024-05-17T22:36:15.622645+0000","last_peered":"2024-05-17T22:36:15.622645+0000","last_clean":"2024-05-17T22:36:15.622645+0000","last_became_active":"2024-05-17T22:34:52.498666+0000","last_became_peered":"2024-05-17T22:34:52.498666+0000","last_unstale":"2024-05-17T22:36:15.622645+0000","last_undegraded":"2024-05-17T22:36:15.622645+0000","last_fullsized":"2024-05-17T22:36:15.622645+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:46:25.882964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.005012+0000","last_change":"2024-05-17T22:35:15.836294+0000","last_active":"2024-05-17T22:36:16.005012+0000","last_peered":"2024-05-17T22:36:16.005012+0000","last_clean":"2024-05-17T22:36:16.005012+0000","last_became_active":"2024-05-17T22:35:15.835759+0000","last_became_peered":"2024-05-17T22:35:15.835759+0000","last_unstale":"2024-05-17T22:36:16.005012+0000","last_undegraded":"2024-05-17T22:36:16.005012+0000","last_fullsized":"2024-05-17T22:36:16.005012+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:23:32.106692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968497+0000","last_change":"2024-05-17T22:34:40.433331+0000","last_active":"2024-05-17T22:36:15.968497+0000","last_peered":"2024-05-17T22:36:15.968497+0000","last_clean":"2024-05-17T22:36:15.968497+0000","last_became_active":"2024-05-17T22:34:40.432989+0000","last_became_peered":"2024-05-17T22:34:40.432989+0000","last_unstale":"2024-05-17T22:36:15.968497+0000","last_undegraded":"2024-05-17T22:36:15.968497+0000","last_fullsized":"2024-05-17T22:36:15.968497+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:22:49.146148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004182+0000","last_change":"2024-05-17T22:34:52.494418+0000","last_active":"2024-05-17T22:36:16.004182+0000","last_peered":"2024-05-17T22:36:16.004182+0000","last_clean":"2024-05-17T22:36:16.004182+0000","last_became_active":"2024-05-17T22:34:52.493953+0000","last_became_peered":"2024-05-17T22:34:52.493953+0000","last_unstale":"2024-05-17T22:36:16.004182+0000","last_undegraded":"2024-05-17T22:36:16.004182+0000","last_fullsized":"2024-05-17T22:36:16.004182+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:50:40.211770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003645+0000","last_change":"2024-05-17T22:34:37.442986+0000","last_active":"2024-05-17T22:36:16.003645+0000","last_peered":"2024-05-17T22:36:16.003645+0000","last_clean":"2024-05-17T22:36:16.003645+0000","last_became_active":"2024-05-17T22:34:37.442399+0000","last_became_peered":"2024-05-17T22:34:37.442399+0000","last_unstale":"2024-05-17T22:36:16.003645+0000","last_undegraded":"2024-05-17T22:36:16.003645+0000","last_fullsized":"2024-05-17T22:36:16.003645+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:34:05.785811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.805334+0000","last_change":"2024-05-17T22:34:40.426793+0000","last_active":"2024-05-17T22:36:15.805334+0000","last_peered":"2024-05-17T22:36:15.805334+0000","last_clean":"2024-05-17T22:36:15.805334+0000","last_became_active":"2024-05-17T22:34:40.426420+0000","last_became_peered":"2024-05-17T22:34:40.426420+0000","last_unstale":"2024-05-17T22:36:15.805334+0000","last_undegraded":"2024-05-17T22:36:15.805334+0000","last_fullsized":"2024-05-17T22:36:15.805334+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:57:07.099254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.622889+0000","last_change":"2024-05-17T22:34:52.498762+0000","last_active":"2024-05-17T22:36:15.622889+0000","last_peered":"2024-05-17T22:36:15.622889+0000","last_clean":"2024-05-17T22:36:15.622889+0000","last_became_active":"2024-05-17T22:34:52.498306+0000","last_became_peered":"2024-05-17T22:34:52.498306+0000","last_unstale":"2024-05-17T22:36:15.622889+0000","last_undegraded":"2024-05-17T22:36:15.622889+0000","last_fullsized":"2024-05-17T22:36:15.622889+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:18:32.717563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'33","reported_seq":156,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:24.095963+0000","last_change":"2024-05-17T22:34:37.407885+0000","last_active":"2024-05-17T22:36:24.095963+0000","last_peered":"2024-05-17T22:36:24.095963+0000","last_clean":"2024-05-17T22:36:24.095963+0000","last_became_active":"2024-05-17T22:34:37.407463+0000","last_became_peered":"2024-05-17T22:34:37.407463+0000","last_unstale":"2024-05-17T22:36:24.095963+0000","last_undegraded":"2024-05-17T22:36:24.095963+0000","last_fullsized":"2024-05-17T22:36:24.095963+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:26:20.288557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62072,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":79,"num_read_kb":100,"num_write":34,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.626156+0000","last_change":"2024-05-17T22:34:40.435800+0000","last_active":"2024-05-17T22:36:15.626156+0000","last_peered":"2024-05-17T22:36:15.626156+0000","last_clean":"2024-05-17T22:36:15.626156+0000","last_became_active":"2024-05-17T22:34:40.435245+0000","last_became_peered":"2024-05-17T22:34:40.435245+0000","last_unstale":"2024-05-17T22:36:15.626156+0000","last_undegraded":"2024-05-17T22:36:15.626156+0000","last_fullsized":"2024-05-17T22:36:15.626156+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:09:58.699026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004407+0000","last_change":"2024-05-17T22:34:52.487336+0000","last_active":"2024-05-17T22:36:16.004407+0000","last_peered":"2024-05-17T22:36:16.004407+0000","last_clean":"2024-05-17T22:36:16.004407+0000","last_became_active":"2024-05-17T22:34:52.486883+0000","last_became_peered":"2024-05-17T22:34:52.486883+0000","last_unstale":"2024-05-17T22:36:16.004407+0000","last_undegraded":"2024-05-17T22:36:16.004407+0000","last_fullsized":"2024-05-17T22:36:16.004407+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:44:41.264072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623955+0000","last_change":"2024-05-17T22:34:37.405044+0000","last_active":"2024-05-17T22:36:15.623955+0000","last_peered":"2024-05-17T22:36:15.623955+0000","last_clean":"2024-05-17T22:36:15.623955+0000","last_became_active":"2024-05-17T22:34:37.404610+0000","last_became_peered":"2024-05-17T22:34:37.404610+0000","last_unstale":"2024-05-17T22:36:15.623955+0000","last_undegraded":"2024-05-17T22:36:15.623955+0000","last_fullsized":"2024-05-17T22:36:15.623955+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:20:37.939069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685489+0000","last_change":"2024-05-17T22:34:40.430655+0000","last_active":"2024-05-17T22:36:15.685489+0000","last_peered":"2024-05-17T22:36:15.685489+0000","last_clean":"2024-05-17T22:36:15.685489+0000","last_became_active":"2024-05-17T22:34:40.430307+0000","last_became_peered":"2024-05-17T22:34:40.430307+0000","last_unstale":"2024-05-17T22:36:15.685489+0000","last_undegraded":"2024-05-17T22:36:15.685489+0000","last_fullsized":"2024-05-17T22:36:15.685489+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:57:19.036117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628122+0000","last_change":"2024-05-17T22:34:52.494709+0000","last_active":"2024-05-17T22:36:15.628122+0000","last_peered":"2024-05-17T22:36:15.628122+0000","last_clean":"2024-05-17T22:36:15.628122+0000","last_became_active":"2024-05-17T22:34:52.494195+0000","last_became_peered":"2024-05-17T22:34:52.494195+0000","last_unstale":"2024-05-17T22:36:15.628122+0000","last_undegraded":"2024-05-17T22:36:15.628122+0000","last_fullsized":"2024-05-17T22:36:15.628122+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:35:41.188459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003638+0000","last_change":"2024-05-17T22:34:37.402494+0000","last_active":"2024-05-17T22:36:16.003638+0000","last_peered":"2024-05-17T22:36:16.003638+0000","last_clean":"2024-05-17T22:36:16.003638+0000","last_became_active":"2024-05-17T22:34:37.402102+0000","last_became_peered":"2024-05-17T22:34:37.402102+0000","last_unstale":"2024-05-17T22:36:16.003638+0000","last_undegraded":"2024-05-17T22:36:16.003638+0000","last_fullsized":"2024-05-17T22:36:16.003638+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:22:30.915780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004954+0000","last_change":"2024-05-17T22:34:40.440731+0000","last_active":"2024-05-17T22:36:16.004954+0000","last_peered":"2024-05-17T22:36:16.004954+0000","last_clean":"2024-05-17T22:36:16.004954+0000","last_became_active":"2024-05-17T22:34:40.440374+0000","last_became_peered":"2024-05-17T22:34:40.440374+0000","last_unstale":"2024-05-17T22:36:16.004954+0000","last_undegraded":"2024-05-17T22:36:16.004954+0000","last_fullsized":"2024-05-17T22:36:16.004954+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:26:40.029155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.005856+0000","last_change":"2024-05-17T22:36:16.005856+0000","last_active":"2024-05-17T22:36:16.005856+0000","last_peered":"2024-05-17T22:36:16.005856+0000","last_clean":"2024-05-17T22:36:16.005856+0000","last_became_active":"2024-05-17T22:36:16.005303+0000","last_became_peered":"2024-05-17T22:36:16.005303+0000","last_unstale":"2024-05-17T22:36:16.005856+0000","last_undegraded":"2024-05-17T22:36:16.005856+0000","last_fullsized":"2024-05-17T22:36:16.005856+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:41:15.708879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"87'4","reported_seq":88,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:26.099659+0000","last_change":"2024-05-17T22:34:37.403123+0000","last_active":"2024-05-17T22:36:26.099659+0000","last_peered":"2024-05-17T22:36:26.099659+0000","last_clean":"2024-05-17T22:36:26.099659+0000","last_became_active":"2024-05-17T22:34:37.402702+0000","last_became_peered":"2024-05-17T22:34:37.402702+0000","last_unstale":"2024-05-17T22:36:26.099659+0000","last_undegraded":"2024-05-17T22:36:26.099659+0000","last_fullsized":"2024-05-17T22:36:26.099659+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:50:02.031651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":42,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003779+0000","last_change":"2024-05-17T22:34:40.431895+0000","last_active":"2024-05-17T22:36:16.003779+0000","last_peered":"2024-05-17T22:36:16.003779+0000","last_clean":"2024-05-17T22:36:16.003779+0000","last_became_active":"2024-05-17T22:34:40.431527+0000","last_became_peered":"2024-05-17T22:34:40.431527+0000","last_unstale":"2024-05-17T22:36:16.003779+0000","last_undegraded":"2024-05-17T22:36:16.003779+0000","last_fullsized":"2024-05-17T22:36:16.003779+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:30:09.404255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.686085+0000","last_change":"2024-05-17T22:34:52.495583+0000","last_active":"2024-05-17T22:36:15.686085+0000","last_peered":"2024-05-17T22:36:15.686085+0000","last_clean":"2024-05-17T22:36:15.686085+0000","last_became_active":"2024-05-17T22:34:52.495030+0000","last_became_peered":"2024-05-17T22:34:52.495030+0000","last_unstale":"2024-05-17T22:36:15.686085+0000","last_undegraded":"2024-05-17T22:36:15.686085+0000","last_fullsized":"2024-05-17T22:36:15.686085+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:04:54.046891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625526+0000","last_change":"2024-05-17T22:34:37.442154+0000","last_active":"2024-05-17T22:36:15.625526+0000","last_peered":"2024-05-17T22:36:15.625526+0000","last_clean":"2024-05-17T22:36:15.625526+0000","last_became_active":"2024-05-17T22:34:37.441691+0000","last_became_peered":"2024-05-17T22:34:37.441691+0000","last_unstale":"2024-05-17T22:36:15.625526+0000","last_undegraded":"2024-05-17T22:36:15.625526+0000","last_fullsized":"2024-05-17T22:36:15.625526+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:47:29.862031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003475+0000","last_change":"2024-05-17T22:34:40.430894+0000","last_active":"2024-05-17T22:36:16.003475+0000","last_peered":"2024-05-17T22:36:16.003475+0000","last_clean":"2024-05-17T22:36:16.003475+0000","last_became_active":"2024-05-17T22:34:40.430328+0000","last_became_peered":"2024-05-17T22:34:40.430328+0000","last_unstale":"2024-05-17T22:36:16.003475+0000","last_undegraded":"2024-05-17T22:36:16.003475+0000","last_fullsized":"2024-05-17T22:36:16.003475+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:36:46.619111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003404+0000","last_change":"2024-05-17T22:34:52.487555+0000","last_active":"2024-05-17T22:36:16.003404+0000","last_peered":"2024-05-17T22:36:16.003404+0000","last_clean":"2024-05-17T22:36:16.003404+0000","last_became_active":"2024-05-17T22:34:52.487076+0000","last_became_peered":"2024-05-17T22:34:52.487076+0000","last_unstale":"2024-05-17T22:36:16.003404+0000","last_undegraded":"2024-05-17T22:36:16.003404+0000","last_fullsized":"2024-05-17T22:36:16.003404+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:05:46.720665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628312+0000","last_change":"2024-05-17T22:35:15.558860+0000","last_active":"2024-05-17T22:36:15.628312+0000","last_peered":"2024-05-17T22:36:15.628312+0000","last_clean":"2024-05-17T22:36:15.628312+0000","last_became_active":"2024-05-17T22:35:15.553683+0000","last_became_peered":"2024-05-17T22:35:15.553683+0000","last_unstale":"2024-05-17T22:36:15.628312+0000","last_undegraded":"2024-05-17T22:36:15.628312+0000","last_fullsized":"2024-05-17T22:36:15.628312+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:27:24.978282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628360+0000","last_change":"2024-05-17T22:34:40.433499+0000","last_active":"2024-05-17T22:36:15.628360+0000","last_peered":"2024-05-17T22:36:15.628360+0000","last_clean":"2024-05-17T22:36:15.628360+0000","last_became_active":"2024-05-17T22:34:40.431370+0000","last_became_peered":"2024-05-17T22:34:40.431370+0000","last_unstale":"2024-05-17T22:36:15.628360+0000","last_undegraded":"2024-05-17T22:36:15.628360+0000","last_fullsized":"2024-05-17T22:36:15.628360+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:25:08.383021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625886+0000","last_change":"2024-05-17T22:34:52.489627+0000","last_active":"2024-05-17T22:36:15.625886+0000","last_peered":"2024-05-17T22:36:15.625886+0000","last_clean":"2024-05-17T22:36:15.625886+0000","last_became_active":"2024-05-17T22:34:52.489173+0000","last_became_peered":"2024-05-17T22:34:52.489173+0000","last_unstale":"2024-05-17T22:36:15.625886+0000","last_undegraded":"2024-05-17T22:36:15.625886+0000","last_fullsized":"2024-05-17T22:36:15.625886+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:52:26.908817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623301+0000","last_change":"2024-05-17T22:35:15.553325+0000","last_active":"2024-05-17T22:36:15.623301+0000","last_peered":"2024-05-17T22:36:15.623301+0000","last_clean":"2024-05-17T22:36:15.623301+0000","last_became_active":"2024-05-17T22:35:15.552867+0000","last_became_peered":"2024-05-17T22:35:15.552867+0000","last_unstale":"2024-05-17T22:36:15.623301+0000","last_undegraded":"2024-05-17T22:36:15.623301+0000","last_fullsized":"2024-05-17T22:36:15.623301+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:28:36.618331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.626212+0000","last_change":"2024-05-17T22:34:40.423221+0000","last_active":"2024-05-17T22:36:15.626212+0000","last_peered":"2024-05-17T22:36:15.626212+0000","last_clean":"2024-05-17T22:36:15.626212+0000","last_became_active":"2024-05-17T22:34:40.422838+0000","last_became_peered":"2024-05-17T22:34:40.422838+0000","last_unstale":"2024-05-17T22:36:15.626212+0000","last_undegraded":"2024-05-17T22:36:15.626212+0000","last_fullsized":"2024-05-17T22:36:15.626212+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:39:51.873212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685849+0000","last_change":"2024-05-17T22:34:52.484802+0000","last_active":"2024-05-17T22:36:15.685849+0000","last_peered":"2024-05-17T22:36:15.685849+0000","last_clean":"2024-05-17T22:36:15.685849+0000","last_became_active":"2024-05-17T22:34:52.483766+0000","last_became_peered":"2024-05-17T22:34:52.483766+0000","last_unstale":"2024-05-17T22:36:15.685849+0000","last_undegraded":"2024-05-17T22:36:15.685849+0000","last_fullsized":"2024-05-17T22:36:15.685849+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:32:01.427078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004410+0000","last_change":"2024-05-17T22:34:37.404797+0000","last_active":"2024-05-17T22:36:16.004410+0000","last_peered":"2024-05-17T22:36:16.004410+0000","last_clean":"2024-05-17T22:36:16.004410+0000","last_became_active":"2024-05-17T22:34:37.404393+0000","last_became_peered":"2024-05-17T22:34:37.404393+0000","last_unstale":"2024-05-17T22:36:16.004410+0000","last_undegraded":"2024-05-17T22:36:16.004410+0000","last_fullsized":"2024-05-17T22:36:16.004410+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:26:36.887919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625922+0000","last_change":"2024-05-17T22:34:40.428210+0000","last_active":"2024-05-17T22:36:15.625922+0000","last_peered":"2024-05-17T22:36:15.625922+0000","last_clean":"2024-05-17T22:36:15.625922+0000","last_became_active":"2024-05-17T22:34:40.427845+0000","last_became_peered":"2024-05-17T22:34:40.427845+0000","last_unstale":"2024-05-17T22:36:15.625922+0000","last_undegraded":"2024-05-17T22:36:15.625922+0000","last_fullsized":"2024-05-17T22:36:15.625922+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:45:30.437139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685982+0000","last_change":"2024-05-17T22:34:52.490502+0000","last_active":"2024-05-17T22:36:15.685982+0000","last_peered":"2024-05-17T22:36:15.685982+0000","last_clean":"2024-05-17T22:36:15.685982+0000","last_became_active":"2024-05-17T22:34:52.490008+0000","last_became_peered":"2024-05-17T22:34:52.490008+0000","last_unstale":"2024-05-17T22:36:15.685982+0000","last_undegraded":"2024-05-17T22:36:15.685982+0000","last_fullsized":"2024-05-17T22:36:15.685982+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:58:29.122668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003400+0000","last_change":"2024-05-17T22:34:40.432248+0000","last_active":"2024-05-17T22:36:16.003400+0000","last_peered":"2024-05-17T22:36:16.003400+0000","last_clean":"2024-05-17T22:36:16.003400+0000","last_became_active":"2024-05-17T22:34:40.431863+0000","last_became_peered":"2024-05-17T22:34:40.431863+0000","last_unstale":"2024-05-17T22:36:16.003400+0000","last_undegraded":"2024-05-17T22:36:16.003400+0000","last_fullsized":"2024-05-17T22:36:16.003400+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:20:34.730542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'4","reported_seq":82,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:22.092659+0000","last_change":"2024-05-17T22:34:37.441186+0000","last_active":"2024-05-17T22:36:22.092659+0000","last_peered":"2024-05-17T22:36:22.092659+0000","last_clean":"2024-05-17T22:36:22.092659+0000","last_became_active":"2024-05-17T22:34:37.440781+0000","last_became_peered":"2024-05-17T22:34:37.440781+0000","last_unstale":"2024-05-17T22:36:22.092659+0000","last_undegraded":"2024-05-17T22:36:22.092659+0000","last_fullsized":"2024-05-17T22:36:22.092659+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:44:34.435147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.005029+0000","last_change":"2024-05-17T22:34:52.491771+0000","last_active":"2024-05-17T22:36:16.005029+0000","last_peered":"2024-05-17T22:36:16.005029+0000","last_clean":"2024-05-17T22:36:16.005029+0000","last_became_active":"2024-05-17T22:34:52.491172+0000","last_became_peered":"2024-05-17T22:34:52.491172+0000","last_unstale":"2024-05-17T22:36:16.005029+0000","last_undegraded":"2024-05-17T22:36:16.005029+0000","last_fullsized":"2024-05-17T22:36:16.005029+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:15:21.010592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984305+0000","last_change":"2024-05-17T22:34:40.430860+0000","last_active":"2024-05-17T22:36:15.984305+0000","last_peered":"2024-05-17T22:36:15.984305+0000","last_clean":"2024-05-17T22:36:15.984305+0000","last_became_active":"2024-05-17T22:34:40.430542+0000","last_became_peered":"2024-05-17T22:34:40.430542+0000","last_unstale":"2024-05-17T22:36:15.984305+0000","last_undegraded":"2024-05-17T22:36:15.984305+0000","last_fullsized":"2024-05-17T22:36:15.984305+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:16:26.257380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004059+0000","last_change":"2024-05-17T22:34:37.403026+0000","last_active":"2024-05-17T22:36:16.004059+0000","last_peered":"2024-05-17T22:36:16.004059+0000","last_clean":"2024-05-17T22:36:16.004059+0000","last_became_active":"2024-05-17T22:34:37.402560+0000","last_became_peered":"2024-05-17T22:34:37.402560+0000","last_unstale":"2024-05-17T22:36:16.004059+0000","last_undegraded":"2024-05-17T22:36:16.004059+0000","last_fullsized":"2024-05-17T22:36:16.004059+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:23:03.784411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":119,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003503+0000","last_change":"2024-05-17T22:33:11.055878+0000","last_active":"2024-05-17T22:36:16.003503+0000","last_peered":"2024-05-17T22:36:16.003503+0000","last_clean":"2024-05-17T22:36:16.003503+0000","last_became_active":"2024-05-17T22:33:11.050409+0000","last_became_peered":"2024-05-17T22:33:11.050409+0000","last_unstale":"2024-05-17T22:36:16.003503+0000","last_undegraded":"2024-05-17T22:36:16.003503+0000","last_fullsized":"2024-05-17T22:36:16.003503+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:29:43.341170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:29:43.341170+0000","last_clean_scrub_stamp":"2024-05-17T22:29:43.341170+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:30:54.153865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003650+0000","last_change":"2024-05-17T22:34:52.497064+0000","last_active":"2024-05-17T22:36:16.003650+0000","last_peered":"2024-05-17T22:36:16.003650+0000","last_clean":"2024-05-17T22:36:16.003650+0000","last_became_active":"2024-05-17T22:34:52.496611+0000","last_became_peered":"2024-05-17T22:34:52.496611+0000","last_unstale":"2024-05-17T22:36:16.003650+0000","last_undegraded":"2024-05-17T22:36:16.003650+0000","last_fullsized":"2024-05-17T22:36:16.003650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:20:29.597589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685209+0000","last_change":"2024-05-17T22:35:15.555413+0000","last_active":"2024-05-17T22:36:15.685209+0000","last_peered":"2024-05-17T22:36:15.685209+0000","last_clean":"2024-05-17T22:36:15.685209+0000","last_became_active":"2024-05-17T22:35:15.555005+0000","last_became_peered":"2024-05-17T22:35:15.555005+0000","last_unstale":"2024-05-17T22:36:15.685209+0000","last_undegraded":"2024-05-17T22:36:15.685209+0000","last_fullsized":"2024-05-17T22:36:15.685209+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:51:49.860673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003644+0000","last_change":"2024-05-17T22:34:37.406546+0000","last_active":"2024-05-17T22:36:16.003644+0000","last_peered":"2024-05-17T22:36:16.003644+0000","last_clean":"2024-05-17T22:36:16.003644+0000","last_became_active":"2024-05-17T22:34:37.406118+0000","last_became_peered":"2024-05-17T22:34:37.406118+0000","last_unstale":"2024-05-17T22:36:16.003644+0000","last_undegraded":"2024-05-17T22:36:16.003644+0000","last_fullsized":"2024-05-17T22:36:16.003644+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:47:16.350091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.005647+0000","last_change":"2024-05-17T22:36:16.005647+0000","last_active":"2024-05-17T22:36:16.005647+0000","last_peered":"2024-05-17T22:36:16.005647+0000","last_clean":"2024-05-17T22:36:16.005647+0000","last_became_active":"2024-05-17T22:36:16.005128+0000","last_became_peered":"2024-05-17T22:36:16.005128+0000","last_unstale":"2024-05-17T22:36:16.005647+0000","last_undegraded":"2024-05-17T22:36:16.005647+0000","last_fullsized":"2024-05-17T22:36:16.005647+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:16:24.481236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,7,6],"acting":[11,10,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.626298+0000","last_change":"2024-05-17T22:34:37.442676+0000","last_active":"2024-05-17T22:36:15.626298+0000","last_peered":"2024-05-17T22:36:15.626298+0000","last_clean":"2024-05-17T22:36:15.626298+0000","last_became_active":"2024-05-17T22:34:37.442275+0000","last_became_peered":"2024-05-17T22:34:37.442275+0000","last_unstale":"2024-05-17T22:36:15.626298+0000","last_undegraded":"2024-05-17T22:36:15.626298+0000","last_fullsized":"2024-05-17T22:36:15.626298+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:54:18.542987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685755+0000","last_change":"2024-05-17T22:34:40.424266+0000","last_active":"2024-05-17T22:36:15.685755+0000","last_peered":"2024-05-17T22:36:15.685755+0000","last_clean":"2024-05-17T22:36:15.685755+0000","last_became_active":"2024-05-17T22:34:40.423905+0000","last_became_peered":"2024-05-17T22:34:40.423905+0000","last_unstale":"2024-05-17T22:36:15.685755+0000","last_undegraded":"2024-05-17T22:36:15.685755+0000","last_fullsized":"2024-05-17T22:36:15.685755+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:03:43.397871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984103+0000","last_change":"2024-05-17T22:34:52.493386+0000","last_active":"2024-05-17T22:36:15.984103+0000","last_peered":"2024-05-17T22:36:15.984103+0000","last_clean":"2024-05-17T22:36:15.984103+0000","last_became_active":"2024-05-17T22:34:52.492997+0000","last_became_peered":"2024-05-17T22:34:52.492997+0000","last_unstale":"2024-05-17T22:36:15.984103+0000","last_undegraded":"2024-05-17T22:36:15.984103+0000","last_fullsized":"2024-05-17T22:36:15.984103+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:14:40.818271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'11","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623586+0000","last_change":"2024-05-17T22:34:37.399397+0000","last_active":"2024-05-17T22:36:15.623586+0000","last_peered":"2024-05-17T22:36:15.623586+0000","last_clean":"2024-05-17T22:36:15.623586+0000","last_became_active":"2024-05-17T22:34:37.399062+0000","last_became_peered":"2024-05-17T22:34:37.399062+0000","last_unstale":"2024-05-17T22:36:15.623586+0000","last_undegraded":"2024-05-17T22:36:15.623586+0000","last_fullsized":"2024-05-17T22:36:15.623586+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:02:05.230580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984853+0000","last_change":"2024-05-17T22:34:40.426995+0000","last_active":"2024-05-17T22:36:15.984853+0000","last_peered":"2024-05-17T22:36:15.984853+0000","last_clean":"2024-05-17T22:36:15.984853+0000","last_became_active":"2024-05-17T22:34:40.426226+0000","last_became_peered":"2024-05-17T22:34:40.426226+0000","last_unstale":"2024-05-17T22:36:15.984853+0000","last_undegraded":"2024-05-17T22:36:15.984853+0000","last_fullsized":"2024-05-17T22:36:15.984853+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:49:53.331076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623635+0000","last_change":"2024-05-17T22:34:52.487098+0000","last_active":"2024-05-17T22:36:15.623635+0000","last_peered":"2024-05-17T22:36:15.623635+0000","last_clean":"2024-05-17T22:36:15.623635+0000","last_became_active":"2024-05-17T22:34:52.486605+0000","last_became_peered":"2024-05-17T22:34:52.486605+0000","last_unstale":"2024-05-17T22:36:15.623635+0000","last_undegraded":"2024-05-17T22:36:15.623635+0000","last_fullsized":"2024-05-17T22:36:15.623635+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:33:28.234208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004630+0000","last_change":"2024-05-17T22:34:37.800830+0000","last_active":"2024-05-17T22:36:16.004630+0000","last_peered":"2024-05-17T22:36:16.004630+0000","last_clean":"2024-05-17T22:36:16.004630+0000","last_became_active":"2024-05-17T22:34:37.800405+0000","last_became_peered":"2024-05-17T22:34:37.800405+0000","last_unstale":"2024-05-17T22:36:16.004630+0000","last_undegraded":"2024-05-17T22:36:16.004630+0000","last_fullsized":"2024-05-17T22:36:16.004630+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:19:10.087442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968862+0000","last_change":"2024-05-17T22:34:40.426307+0000","last_active":"2024-05-17T22:36:15.968862+0000","last_peered":"2024-05-17T22:36:15.968862+0000","last_clean":"2024-05-17T22:36:15.968862+0000","last_became_active":"2024-05-17T22:34:40.425969+0000","last_became_peered":"2024-05-17T22:34:40.425969+0000","last_unstale":"2024-05-17T22:36:15.968862+0000","last_undegraded":"2024-05-17T22:36:15.968862+0000","last_fullsized":"2024-05-17T22:36:15.968862+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:25:49.680886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004451+0000","last_change":"2024-05-17T22:35:15.556857+0000","last_active":"2024-05-17T22:36:16.004451+0000","last_peered":"2024-05-17T22:36:16.004451+0000","last_clean":"2024-05-17T22:36:16.004451+0000","last_became_active":"2024-05-17T22:35:15.556298+0000","last_became_peered":"2024-05-17T22:35:15.556298+0000","last_unstale":"2024-05-17T22:36:16.004451+0000","last_undegraded":"2024-05-17T22:36:16.004451+0000","last_fullsized":"2024-05-17T22:36:16.004451+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:23:20.111544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.805120+0000","last_change":"2024-05-17T22:34:37.442971+0000","last_active":"2024-05-17T22:36:15.805120+0000","last_peered":"2024-05-17T22:36:15.805120+0000","last_clean":"2024-05-17T22:36:15.805120+0000","last_became_active":"2024-05-17T22:34:37.442536+0000","last_became_peered":"2024-05-17T22:34:37.442536+0000","last_unstale":"2024-05-17T22:36:15.805120+0000","last_undegraded":"2024-05-17T22:36:15.805120+0000","last_fullsized":"2024-05-17T22:36:15.805120+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:03:45.224071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625559+0000","last_change":"2024-05-17T22:34:40.432056+0000","last_active":"2024-05-17T22:36:15.625559+0000","last_peered":"2024-05-17T22:36:15.625559+0000","last_clean":"2024-05-17T22:36:15.625559+0000","last_became_active":"2024-05-17T22:34:40.431626+0000","last_became_peered":"2024-05-17T22:34:40.431626+0000","last_unstale":"2024-05-17T22:36:15.625559+0000","last_undegraded":"2024-05-17T22:36:15.625559+0000","last_fullsized":"2024-05-17T22:36:15.625559+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:12:24.650746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623590+0000","last_change":"2024-05-17T22:34:52.500451+0000","last_active":"2024-05-17T22:36:15.623590+0000","last_peered":"2024-05-17T22:36:15.623590+0000","last_clean":"2024-05-17T22:36:15.623590+0000","last_became_active":"2024-05-17T22:34:52.500031+0000","last_became_peered":"2024-05-17T22:34:52.500031+0000","last_unstale":"2024-05-17T22:36:15.623590+0000","last_undegraded":"2024-05-17T22:36:15.623590+0000","last_fullsized":"2024-05-17T22:36:15.623590+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:54:36.825116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984349+0000","last_change":"2024-05-17T22:34:37.402335+0000","last_active":"2024-05-17T22:36:15.984349+0000","last_peered":"2024-05-17T22:36:15.984349+0000","last_clean":"2024-05-17T22:36:15.984349+0000","last_became_active":"2024-05-17T22:34:37.401982+0000","last_became_peered":"2024-05-17T22:34:37.401982+0000","last_unstale":"2024-05-17T22:36:15.984349+0000","last_undegraded":"2024-05-17T22:36:15.984349+0000","last_fullsized":"2024-05-17T22:36:15.984349+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:58:06.194335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.627919+0000","last_change":"2024-05-17T22:34:40.425524+0000","last_active":"2024-05-17T22:36:15.627919+0000","last_peered":"2024-05-17T22:36:15.627919+0000","last_clean":"2024-05-17T22:36:15.627919+0000","last_became_active":"2024-05-17T22:34:40.425135+0000","last_became_peered":"2024-05-17T22:34:40.425135+0000","last_unstale":"2024-05-17T22:36:15.627919+0000","last_undegraded":"2024-05-17T22:36:15.627919+0000","last_fullsized":"2024-05-17T22:36:15.627919+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:41:14.176477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004602+0000","last_change":"2024-05-17T22:34:52.496290+0000","last_active":"2024-05-17T22:36:16.004602+0000","last_peered":"2024-05-17T22:36:16.004602+0000","last_clean":"2024-05-17T22:36:16.004602+0000","last_became_active":"2024-05-17T22:34:52.495817+0000","last_became_peered":"2024-05-17T22:34:52.495817+0000","last_unstale":"2024-05-17T22:36:16.004602+0000","last_undegraded":"2024-05-17T22:36:16.004602+0000","last_fullsized":"2024-05-17T22:36:16.004602+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:47:22.056372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'1","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623936+0000","last_change":"2024-05-17T22:34:37.404234+0000","last_active":"2024-05-17T22:36:15.623936+0000","last_peered":"2024-05-17T22:36:15.623936+0000","last_clean":"2024-05-17T22:36:15.623936+0000","last_became_active":"2024-05-17T22:34:37.403561+0000","last_became_peered":"2024-05-17T22:34:37.403561+0000","last_unstale":"2024-05-17T22:36:15.623936+0000","last_undegraded":"2024-05-17T22:36:15.623936+0000","last_fullsized":"2024-05-17T22:36:15.623936+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:30:28.432641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984498+0000","last_change":"2024-05-17T22:34:40.426522+0000","last_active":"2024-05-17T22:36:15.984498+0000","last_peered":"2024-05-17T22:36:15.984498+0000","last_clean":"2024-05-17T22:36:15.984498+0000","last_became_active":"2024-05-17T22:34:40.426139+0000","last_became_peered":"2024-05-17T22:34:40.426139+0000","last_unstale":"2024-05-17T22:36:15.984498+0000","last_undegraded":"2024-05-17T22:36:15.984498+0000","last_fullsized":"2024-05-17T22:36:15.984498+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:30:19.503513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004016+0000","last_change":"2024-05-17T22:34:52.487131+0000","last_active":"2024-05-17T22:36:16.004016+0000","last_peered":"2024-05-17T22:36:16.004016+0000","last_clean":"2024-05-17T22:36:16.004016+0000","last_became_active":"2024-05-17T22:34:52.486654+0000","last_became_peered":"2024-05-17T22:34:52.486654+0000","last_unstale":"2024-05-17T22:36:16.004016+0000","last_undegraded":"2024-05-17T22:36:16.004016+0000","last_fullsized":"2024-05-17T22:36:16.004016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:28:25.041746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625819+0000","last_change":"2024-05-17T22:34:37.405020+0000","last_active":"2024-05-17T22:36:15.625819+0000","last_peered":"2024-05-17T22:36:15.625819+0000","last_clean":"2024-05-17T22:36:15.625819+0000","last_became_active":"2024-05-17T22:34:37.404633+0000","last_became_peered":"2024-05-17T22:34:37.404633+0000","last_unstale":"2024-05-17T22:36:15.625819+0000","last_undegraded":"2024-05-17T22:36:15.625819+0000","last_fullsized":"2024-05-17T22:36:15.625819+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:06:00.001190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004601+0000","last_change":"2024-05-17T22:34:40.435374+0000","last_active":"2024-05-17T22:36:16.004601+0000","last_peered":"2024-05-17T22:36:16.004601+0000","last_clean":"2024-05-17T22:36:16.004601+0000","last_became_active":"2024-05-17T22:34:40.434522+0000","last_became_peered":"2024-05-17T22:34:40.434522+0000","last_unstale":"2024-05-17T22:36:16.004601+0000","last_undegraded":"2024-05-17T22:36:16.004601+0000","last_fullsized":"2024-05-17T22:36:16.004601+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:36:41.244970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.805127+0000","last_change":"2024-05-17T22:34:52.491708+0000","last_active":"2024-05-17T22:36:15.805127+0000","last_peered":"2024-05-17T22:36:15.805127+0000","last_clean":"2024-05-17T22:36:15.805127+0000","last_became_active":"2024-05-17T22:34:52.491293+0000","last_became_peered":"2024-05-17T22:34:52.491293+0000","last_unstale":"2024-05-17T22:36:15.805127+0000","last_undegraded":"2024-05-17T22:36:15.805127+0000","last_fullsized":"2024-05-17T22:36:15.805127+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:23:01.634026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624618+0000","last_change":"2024-05-17T22:34:37.800525+0000","last_active":"2024-05-17T22:36:15.624618+0000","last_peered":"2024-05-17T22:36:15.624618+0000","last_clean":"2024-05-17T22:36:15.624618+0000","last_became_active":"2024-05-17T22:34:37.800093+0000","last_became_peered":"2024-05-17T22:34:37.800093+0000","last_unstale":"2024-05-17T22:36:15.624618+0000","last_undegraded":"2024-05-17T22:36:15.624618+0000","last_fullsized":"2024-05-17T22:36:15.624618+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:25:27.249889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.005319+0000","last_change":"2024-05-17T22:34:40.434891+0000","last_active":"2024-05-17T22:36:16.005319+0000","last_peered":"2024-05-17T22:36:16.005319+0000","last_clean":"2024-05-17T22:36:16.005319+0000","last_became_active":"2024-05-17T22:34:40.434543+0000","last_became_peered":"2024-05-17T22:34:40.434543+0000","last_unstale":"2024-05-17T22:36:16.005319+0000","last_undegraded":"2024-05-17T22:36:16.005319+0000","last_fullsized":"2024-05-17T22:36:16.005319+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:17:47.770225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968349+0000","last_change":"2024-05-17T22:34:52.491544+0000","last_active":"2024-05-17T22:36:15.968349+0000","last_peered":"2024-05-17T22:36:15.968349+0000","last_clean":"2024-05-17T22:36:15.968349+0000","last_became_active":"2024-05-17T22:34:52.491138+0000","last_became_peered":"2024-05-17T22:34:52.491138+0000","last_unstale":"2024-05-17T22:36:15.968349+0000","last_undegraded":"2024-05-17T22:36:15.968349+0000","last_fullsized":"2024-05-17T22:36:15.968349+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:34:20.411299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004214+0000","last_change":"2024-05-17T22:34:37.403220+0000","last_active":"2024-05-17T22:36:16.004214+0000","last_peered":"2024-05-17T22:36:16.004214+0000","last_clean":"2024-05-17T22:36:16.004214+0000","last_became_active":"2024-05-17T22:34:37.402856+0000","last_became_peered":"2024-05-17T22:34:37.402856+0000","last_unstale":"2024-05-17T22:36:16.004214+0000","last_undegraded":"2024-05-17T22:36:16.004214+0000","last_fullsized":"2024-05-17T22:36:16.004214+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:04:19.469294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.005034+0000","last_change":"2024-05-17T22:34:40.434487+0000","last_active":"2024-05-17T22:36:16.005034+0000","last_peered":"2024-05-17T22:36:16.005034+0000","last_clean":"2024-05-17T22:36:16.005034+0000","last_became_active":"2024-05-17T22:34:40.434160+0000","last_became_peered":"2024-05-17T22:34:40.434160+0000","last_unstale":"2024-05-17T22:36:16.005034+0000","last_undegraded":"2024-05-17T22:36:16.005034+0000","last_fullsized":"2024-05-17T22:36:16.005034+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:24:38.782999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003123+0000","last_change":"2024-05-17T22:34:52.487887+0000","last_active":"2024-05-17T22:36:16.003123+0000","last_peered":"2024-05-17T22:36:16.003123+0000","last_clean":"2024-05-17T22:36:16.003123+0000","last_became_active":"2024-05-17T22:34:52.486837+0000","last_became_peered":"2024-05-17T22:34:52.486837+0000","last_unstale":"2024-05-17T22:36:16.003123+0000","last_undegraded":"2024-05-17T22:36:16.003123+0000","last_fullsized":"2024-05-17T22:36:16.003123+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:12:22.206536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.686166+0000","last_change":"2024-05-17T22:34:37.399498+0000","last_active":"2024-05-17T22:36:15.686166+0000","last_peered":"2024-05-17T22:36:15.686166+0000","last_clean":"2024-05-17T22:36:15.686166+0000","last_became_active":"2024-05-17T22:34:37.399144+0000","last_became_peered":"2024-05-17T22:34:37.399144+0000","last_unstale":"2024-05-17T22:36:15.686166+0000","last_undegraded":"2024-05-17T22:36:15.686166+0000","last_fullsized":"2024-05-17T22:36:15.686166+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:13:37.346006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.005111+0000","last_change":"2024-05-17T22:34:40.433839+0000","last_active":"2024-05-17T22:36:16.005111+0000","last_peered":"2024-05-17T22:36:16.005111+0000","last_clean":"2024-05-17T22:36:16.005111+0000","last_became_active":"2024-05-17T22:34:40.433489+0000","last_became_peered":"2024-05-17T22:34:40.433489+0000","last_unstale":"2024-05-17T22:36:16.005111+0000","last_undegraded":"2024-05-17T22:36:16.005111+0000","last_fullsized":"2024-05-17T22:36:16.005111+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:13:56.014094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628026+0000","last_change":"2024-05-17T22:34:40.434953+0000","last_active":"2024-05-17T22:36:15.628026+0000","last_peered":"2024-05-17T22:36:15.628026+0000","last_clean":"2024-05-17T22:36:15.628026+0000","last_became_active":"2024-05-17T22:34:40.434593+0000","last_became_peered":"2024-05-17T22:34:40.434593+0000","last_unstale":"2024-05-17T22:36:15.628026+0000","last_undegraded":"2024-05-17T22:36:15.628026+0000","last_fullsized":"2024-05-17T22:36:15.628026+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:20:27.075382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623850+0000","last_change":"2024-05-17T22:34:40.432787+0000","last_active":"2024-05-17T22:36:15.623850+0000","last_peered":"2024-05-17T22:36:15.623850+0000","last_clean":"2024-05-17T22:36:15.623850+0000","last_became_active":"2024-05-17T22:34:40.432402+0000","last_became_peered":"2024-05-17T22:34:40.432402+0000","last_unstale":"2024-05-17T22:36:15.623850+0000","last_undegraded":"2024-05-17T22:36:15.623850+0000","last_fullsized":"2024-05-17T22:36:15.623850+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:48:13.034276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623724+0000","last_change":"2024-05-17T22:34:40.426423+0000","last_active":"2024-05-17T22:36:15.623724+0000","last_peered":"2024-05-17T22:36:15.623724+0000","last_clean":"2024-05-17T22:36:15.623724+0000","last_became_active":"2024-05-17T22:34:40.426049+0000","last_became_peered":"2024-05-17T22:34:40.426049+0000","last_unstale":"2024-05-17T22:36:15.623724+0000","last_undegraded":"2024-05-17T22:36:15.623724+0000","last_fullsized":"2024-05-17T22:36:15.623724+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:56:55.631322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968631+0000","last_change":"2024-05-17T22:34:40.435706+0000","last_active":"2024-05-17T22:36:15.968631+0000","last_peered":"2024-05-17T22:36:15.968631+0000","last_clean":"2024-05-17T22:36:15.968631+0000","last_became_active":"2024-05-17T22:34:40.435330+0000","last_became_peered":"2024-05-17T22:34:40.435330+0000","last_unstale":"2024-05-17T22:36:15.968631+0000","last_undegraded":"2024-05-17T22:36:15.968631+0000","last_fullsized":"2024-05-17T22:36:15.968631+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:25:36.529454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685705+0000","last_change":"2024-05-17T22:34:40.428997+0000","last_active":"2024-05-17T22:36:15.685705+0000","last_peered":"2024-05-17T22:36:15.685705+0000","last_clean":"2024-05-17T22:36:15.685705+0000","last_became_active":"2024-05-17T22:34:40.428594+0000","last_became_peered":"2024-05-17T22:34:40.428594+0000","last_unstale":"2024-05-17T22:36:15.685705+0000","last_undegraded":"2024-05-17T22:36:15.685705+0000","last_fullsized":"2024-05-17T22:36:15.685705+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:59:01.777751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628370+0000","last_change":"2024-05-17T22:34:40.434241+0000","last_active":"2024-05-17T22:36:15.628370+0000","last_peered":"2024-05-17T22:36:15.628370+0000","last_clean":"2024-05-17T22:36:15.628370+0000","last_became_active":"2024-05-17T22:34:40.431652+0000","last_became_peered":"2024-05-17T22:34:40.431652+0000","last_unstale":"2024-05-17T22:36:15.628370+0000","last_undegraded":"2024-05-17T22:36:15.628370+0000","last_fullsized":"2024-05-17T22:36:15.628370+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:33:27.035284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624649+0000","last_change":"2024-05-17T22:34:40.431485+0000","last_active":"2024-05-17T22:36:15.624649+0000","last_peered":"2024-05-17T22:36:15.624649+0000","last_clean":"2024-05-17T22:36:15.624649+0000","last_became_active":"2024-05-17T22:34:40.431120+0000","last_became_peered":"2024-05-17T22:34:40.431120+0000","last_unstale":"2024-05-17T22:36:15.624649+0000","last_undegraded":"2024-05-17T22:36:15.624649+0000","last_fullsized":"2024-05-17T22:36:15.624649+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:46:55.213602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984947+0000","last_change":"2024-05-17T22:34:40.423117+0000","last_active":"2024-05-17T22:36:15.984947+0000","last_peered":"2024-05-17T22:36:15.984947+0000","last_clean":"2024-05-17T22:36:15.984947+0000","last_became_active":"2024-05-17T22:34:40.422694+0000","last_became_peered":"2024-05-17T22:34:40.422694+0000","last_unstale":"2024-05-17T22:36:15.984947+0000","last_undegraded":"2024-05-17T22:36:15.984947+0000","last_fullsized":"2024-05-17T22:36:15.984947+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:19:55.589532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685170+0000","last_change":"2024-05-17T22:35:15.711796+0000","last_active":"2024-05-17T22:36:15.685170+0000","last_peered":"2024-05-17T22:36:15.685170+0000","last_clean":"2024-05-17T22:36:15.685170+0000","last_became_active":"2024-05-17T22:35:15.711327+0000","last_became_peered":"2024-05-17T22:35:15.711327+0000","last_unstale":"2024-05-17T22:36:15.685170+0000","last_undegraded":"2024-05-17T22:36:15.685170+0000","last_fullsized":"2024-05-17T22:36:15.685170+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:48:48.913312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984270+0000","last_change":"2024-05-17T22:34:40.430393+0000","last_active":"2024-05-17T22:36:15.984270+0000","last_peered":"2024-05-17T22:36:15.984270+0000","last_clean":"2024-05-17T22:36:15.984270+0000","last_became_active":"2024-05-17T22:34:40.430030+0000","last_became_peered":"2024-05-17T22:34:40.430030+0000","last_unstale":"2024-05-17T22:36:15.984270+0000","last_undegraded":"2024-05-17T22:36:15.984270+0000","last_fullsized":"2024-05-17T22:36:15.984270+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:39:37.204246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.805401+0000","last_change":"2024-05-17T22:34:40.432443+0000","last_active":"2024-05-17T22:36:15.805401+0000","last_peered":"2024-05-17T22:36:15.805401+0000","last_clean":"2024-05-17T22:36:15.805401+0000","last_became_active":"2024-05-17T22:34:40.431991+0000","last_became_peered":"2024-05-17T22:34:40.431991+0000","last_unstale":"2024-05-17T22:36:15.805401+0000","last_undegraded":"2024-05-17T22:36:15.805401+0000","last_fullsized":"2024-05-17T22:36:15.805401+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:37:45.593575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685767+0000","last_change":"2024-05-17T22:34:40.434388+0000","last_active":"2024-05-17T22:36:15.685767+0000","last_peered":"2024-05-17T22:36:15.685767+0000","last_clean":"2024-05-17T22:36:15.685767+0000","last_became_active":"2024-05-17T22:34:40.434044+0000","last_became_peered":"2024-05-17T22:34:40.434044+0000","last_unstale":"2024-05-17T22:36:15.685767+0000","last_undegraded":"2024-05-17T22:36:15.685767+0000","last_fullsized":"2024-05-17T22:36:15.685767+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:20:43.296190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628303+0000","last_change":"2024-05-17T22:34:40.427137+0000","last_active":"2024-05-17T22:36:15.628303+0000","last_peered":"2024-05-17T22:36:15.628303+0000","last_clean":"2024-05-17T22:36:15.628303+0000","last_became_active":"2024-05-17T22:34:40.426703+0000","last_became_peered":"2024-05-17T22:34:40.426703+0000","last_unstale":"2024-05-17T22:36:15.628303+0000","last_undegraded":"2024-05-17T22:36:15.628303+0000","last_fullsized":"2024-05-17T22:36:15.628303+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:55:52.992817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66464,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":244,"num_read_kb":189,"num_write":169,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":199392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710757,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1028096,"data_stored":515036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906986,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28692,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716460,"statfs":{"total":95995035648,"available":95965655040,"internally_reserved":0,"allocated":1712128,"data_stored":1164745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070510,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":519346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266741,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1036288,"data_stored":515148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462969,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1634304,"data_stored":1111470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626493,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":510426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1085440,"data_stored":572408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":511814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1110016,"data_stored":574149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182479,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1032192,"data_stored":510620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986249,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1032192,"data_stored":515467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822725,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36944,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708208,"statfs":{"total":95995035648,"available":95957204992,"internally_reserved":0,"allocated":1642496,"data_stored":1106667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":62072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T22:36:30.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:30 smithi012 ceph-mon[31632]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T22:36:30.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:30 smithi151 ceph-mon[36101]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T22:36:31.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:30 smithi096 ceph-mon[36065]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T22:36:31.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:31 smithi151 ceph-mon[36101]: pgmap v357: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-17T22:36:32.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:31 smithi096 ceph-mon[36065]: pgmap v357: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-17T22:36:32.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:31 smithi012 ceph-mon[31632]: pgmap v357: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-17T22:36:33.556 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-17T22:36:33.826 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:36:33.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:33 smithi012 ceph-mon[31632]: pgmap v358: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T22:36:33.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:33 smithi151 ceph-mon[36101]: pgmap v358: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T22:36:34.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:33 smithi096 ceph-mon[36065]: pgmap v358: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T22:36:35.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.479+0000 7fae52b9d700 1 -- 172.21.15.12:0/643414858 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae4c1080a0 msgr2=0x7fae4c10fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:35.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.479+0000 7fae52b9d700 1 --2- 172.21.15.12:0/643414858 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae4c1080a0 0x7fae4c10fa00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fae4400b0a0 tx=0x7fae4400b3b0 comp rx=0 tx=0).stop 2024-05-17T22:36:35.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.479+0000 7fae52b9d700 1 -- 172.21.15.12:0/643414858 shutdown_connections 2024-05-17T22:36:35.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.479+0000 7fae52b9d700 1 --2- 172.21.15.12:0/643414858 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae4c1080a0 0x7fae4c10fa00 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:35.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.479+0000 7fae52b9d700 1 --2- 172.21.15.12:0/643414858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae4c102e70 0x7fae4c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:35.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.479+0000 7fae52b9d700 1 --2- 172.21.15.12:0/643414858 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae4c107750 0x7fae4c107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:35.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.479+0000 7fae52b9d700 1 -- 172.21.15.12:0/643414858 >> 172.21.15.12:0/643414858 conn(0x7fae4c079ae0 msgr2=0x7fae4c079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:36:35.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.479+0000 7fae52b9d700 1 -- 172.21.15.12:0/643414858 shutdown_connections 2024-05-17T22:36:35.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.479+0000 7fae52b9d700 1 -- 172.21.15.12:0/643414858 wait complete. 2024-05-17T22:36:35.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.480+0000 7fae52b9d700 1 Processor -- start 2024-05-17T22:36:35.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.480+0000 7fae52b9d700 1 -- start start 2024-05-17T22:36:35.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.480+0000 7fae52b9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae4c102e70 0x7fae4c199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:35.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.480+0000 7fae52b9d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae4c107750 0x7fae4c199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:35.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.480+0000 7fae52b9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae4c1080a0 0x7fae4c19de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:35.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.480+0000 7fae52b9d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fae4c112ce0 con 0x7fae4c102e70 2024-05-17T22:36:35.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.480+0000 7fae52b9d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fae4c112b60 con 0x7fae4c1080a0 2024-05-17T22:36:35.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.480+0000 7fae52b9d700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fae4c112e60 con 0x7fae4c107750 2024-05-17T22:36:35.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.481+0000 7fae50939700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae4c102e70 0x7fae4c199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:35.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.481+0000 7fae50939700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae4c102e70 0x7fae4c199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54910/0 (socket says 172.21.15.12:54910) 2024-05-17T22:36:35.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.481+0000 7fae50939700 1 -- 172.21.15.12:0/705569140 learned_addr learned my addr 172.21.15.12:0/705569140 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:36:35.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.481+0000 7fae4bfff700 1 --2- 172.21.15.12:0/705569140 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae4c107750 0x7fae4c199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:35.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.481+0000 7fae5113a700 1 --2- 172.21.15.12:0/705569140 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae4c1080a0 0x7fae4c19de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:35.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.481+0000 7fae50939700 1 -- 172.21.15.12:0/705569140 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae4c107750 msgr2=0x7fae4c199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:35.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.481+0000 7fae50939700 1 --2- 172.21.15.12:0/705569140 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae4c107750 0x7fae4c199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:35.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.481+0000 7fae50939700 1 -- 172.21.15.12:0/705569140 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae4c1080a0 msgr2=0x7fae4c19de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:35.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.481+0000 7fae50939700 1 --2- 172.21.15.12:0/705569140 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae4c1080a0 0x7fae4c19de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:35.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.481+0000 7fae50939700 1 -- 172.21.15.12:0/705569140 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae4c19e510 con 0x7fae4c102e70 2024-05-17T22:36:35.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.481+0000 7fae50939700 1 --2- 172.21.15.12:0/705569140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae4c102e70 0x7fae4c199640 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fae4000d950 tx=0x7fae4000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:36:35.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.481+0000 7fae49ffb700 1 -- 172.21.15.12:0/705569140 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae40016070 con 0x7fae4c102e70 2024-05-17T22:36:35.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.481+0000 7fae52b9d700 1 -- 172.21.15.12:0/705569140 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae4c19e800 con 0x7fae4c102e70 2024-05-17T22:36:35.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.481+0000 7fae49ffb700 1 -- 172.21.15.12:0/705569140 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae4000fd40 con 0x7fae4c102e70 2024-05-17T22:36:35.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.481+0000 7fae49ffb700 1 -- 172.21.15.12:0/705569140 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4000e3b0 con 0x7fae4c102e70 2024-05-17T22:36:35.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.482+0000 7fae52b9d700 1 -- 172.21.15.12:0/705569140 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae4c06c040 con 0x7fae4c102e70 2024-05-17T22:36:35.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.483+0000 7fae52b9d700 1 -- 172.21.15.12:0/705569140 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae4c103280 con 0x7fae4c102e70 2024-05-17T22:36:35.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.484+0000 7fae49ffb700 1 -- 172.21.15.12:0/705569140 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fae40019070 con 0x7fae4c102e70 2024-05-17T22:36:35.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.485+0000 7fae49ffb700 1 --2- 172.21.15.12:0/705569140 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fae3c0b2910 0x7fae3c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:35.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.485+0000 7fae4bfff700 1 --2- 172.21.15.12:0/705569140 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fae3c0b2910 0x7fae3c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:35.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.485+0000 7fae49ffb700 1 -- 172.21.15.12:0/705569140 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fae400e2220 con 0x7fae4c102e70 2024-05-17T22:36:35.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.486+0000 7fae4bfff700 1 --2- 172.21.15.12:0/705569140 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fae3c0b2910 0x7fae3c0b4e20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fae4c104760 tx=0x7fae3800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:36:35.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.490+0000 7fae49ffb700 1 -- 172.21.15.12:0/705569140 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fae400a9a50 con 0x7fae4c102e70 2024-05-17T22:36:35.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.700+0000 7fae52b9d700 1 -- 172.21.15.12:0/705569140 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fae4c06ce90 con 0x7fae3c0b2910 2024-05-17T22:36:35.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.706+0000 7fae49ffb700 1 -- 172.21.15.12:0/705569140 <== mgr.14152 v2:172.21.15.12:6800/1388514047 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340346 (secure 0 0 0) 0x7fae4c06ce90 con 0x7fae3c0b2910 2024-05-17T22:36:35.713 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:36:35.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.710+0000 7fae52b9d700 1 -- 172.21.15.12:0/705569140 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fae3c0b2910 msgr2=0x7fae3c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:35.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.710+0000 7fae52b9d700 1 --2- 172.21.15.12:0/705569140 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fae3c0b2910 0x7fae3c0b4e20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fae4c104760 tx=0x7fae3800b410 comp rx=0 tx=0).stop 2024-05-17T22:36:35.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.710+0000 7fae52b9d700 1 -- 172.21.15.12:0/705569140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae4c102e70 msgr2=0x7fae4c199640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:35.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.710+0000 7fae52b9d700 1 --2- 172.21.15.12:0/705569140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae4c102e70 0x7fae4c199640 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fae4000d950 tx=0x7fae4000dc60 comp rx=0 tx=0).stop 2024-05-17T22:36:35.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.710+0000 7fae52b9d700 1 -- 172.21.15.12:0/705569140 shutdown_connections 2024-05-17T22:36:35.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.710+0000 7fae52b9d700 1 --2- 172.21.15.12:0/705569140 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fae3c0b2910 0x7fae3c0b4e20 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:35.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.710+0000 7fae52b9d700 1 --2- 172.21.15.12:0/705569140 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae4c1080a0 0x7fae4c19de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:35.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.710+0000 7fae52b9d700 1 --2- 172.21.15.12:0/705569140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae4c102e70 0x7fae4c199640 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:35.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.710+0000 7fae52b9d700 1 --2- 172.21.15.12:0/705569140 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae4c107750 0x7fae4c199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:35.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.710+0000 7fae52b9d700 1 -- 172.21.15.12:0/705569140 >> 172.21.15.12:0/705569140 conn(0x7fae4c079ae0 msgr2=0x7fae4c105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:36:35.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.710+0000 7fae52b9d700 1 -- 172.21.15.12:0/705569140 shutdown_connections 2024-05-17T22:36:35.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:35.710+0000 7fae52b9d700 1 -- 172.21.15.12:0/705569140 wait complete. 2024-05-17T22:36:35.716 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-05-17T22:36:35.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:35 smithi151 ceph-mon[36101]: pgmap v359: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T22:36:36.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:35 smithi096 ceph-mon[36065]: pgmap v359: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T22:36:36.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:35 smithi012 ceph-mon[31632]: pgmap v359: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T22:36:36.373 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":359,"stamp":"2024-05-17T22:36:34.509349+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657066,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":355,"num_read_kb":273,"num_write":294,"num_write_kb":2270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":238,"ondisk_log_size":238,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":384096,"kb_used_data":14084,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124557728,"statfs":{"total":1151940427776,"available":1151547113472,"internally_reserved":0,"allocated":14422016,"data_stored":8127296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003950"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628725+0000","last_change":"2024-05-17T22:34:40.422850+0000","last_active":"2024-05-17T22:36:15.628725+0000","last_peered":"2024-05-17T22:36:15.628725+0000","last_clean":"2024-05-17T22:36:15.628725+0000","last_became_active":"2024-05-17T22:34:40.422474+0000","last_became_peered":"2024-05-17T22:34:40.422474+0000","last_unstale":"2024-05-17T22:36:15.628725+0000","last_undegraded":"2024-05-17T22:36:15.628725+0000","last_fullsized":"2024-05-17T22:36:15.628725+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:23:51.537128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968741+0000","last_change":"2024-05-17T22:34:40.427907+0000","last_active":"2024-05-17T22:36:15.968741+0000","last_peered":"2024-05-17T22:36:15.968741+0000","last_clean":"2024-05-17T22:36:15.968741+0000","last_became_active":"2024-05-17T22:34:40.427511+0000","last_became_peered":"2024-05-17T22:34:40.427511+0000","last_unstale":"2024-05-17T22:36:15.968741+0000","last_undegraded":"2024-05-17T22:36:15.968741+0000","last_fullsized":"2024-05-17T22:36:15.968741+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:01:46.707820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968443+0000","last_change":"2024-05-17T22:34:40.436370+0000","last_active":"2024-05-17T22:36:15.968443+0000","last_peered":"2024-05-17T22:36:15.968443+0000","last_clean":"2024-05-17T22:36:15.968443+0000","last_became_active":"2024-05-17T22:34:40.434089+0000","last_became_peered":"2024-05-17T22:34:40.434089+0000","last_unstale":"2024-05-17T22:36:15.968443+0000","last_undegraded":"2024-05-17T22:36:15.968443+0000","last_fullsized":"2024-05-17T22:36:15.968443+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:58:16.775471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623409+0000","last_change":"2024-05-17T22:35:15.556681+0000","last_active":"2024-05-17T22:36:15.623409+0000","last_peered":"2024-05-17T22:36:15.623409+0000","last_clean":"2024-05-17T22:36:15.623409+0000","last_became_active":"2024-05-17T22:35:15.556427+0000","last_became_peered":"2024-05-17T22:35:15.556427+0000","last_unstale":"2024-05-17T22:36:15.623409+0000","last_undegraded":"2024-05-17T22:36:15.623409+0000","last_fullsized":"2024-05-17T22:36:15.623409+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:28:01.726262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625720+0000","last_change":"2024-05-17T22:34:40.434394+0000","last_active":"2024-05-17T22:36:15.625720+0000","last_peered":"2024-05-17T22:36:15.625720+0000","last_clean":"2024-05-17T22:36:15.625720+0000","last_became_active":"2024-05-17T22:34:40.433489+0000","last_became_peered":"2024-05-17T22:34:40.433489+0000","last_unstale":"2024-05-17T22:36:15.625720+0000","last_undegraded":"2024-05-17T22:36:15.625720+0000","last_fullsized":"2024-05-17T22:36:15.625720+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:26:24.068814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625953+0000","last_change":"2024-05-17T22:35:15.553601+0000","last_active":"2024-05-17T22:36:15.625953+0000","last_peered":"2024-05-17T22:36:15.625953+0000","last_clean":"2024-05-17T22:36:15.625953+0000","last_became_active":"2024-05-17T22:35:15.553116+0000","last_became_peered":"2024-05-17T22:35:15.553116+0000","last_unstale":"2024-05-17T22:36:15.625953+0000","last_undegraded":"2024-05-17T22:36:15.625953+0000","last_fullsized":"2024-05-17T22:36:15.625953+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:09:38.401107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004274+0000","last_change":"2024-05-17T22:34:40.430479+0000","last_active":"2024-05-17T22:36:16.004274+0000","last_peered":"2024-05-17T22:36:16.004274+0000","last_clean":"2024-05-17T22:36:16.004274+0000","last_became_active":"2024-05-17T22:34:40.430025+0000","last_became_peered":"2024-05-17T22:34:40.430025+0000","last_unstale":"2024-05-17T22:36:16.004274+0000","last_undegraded":"2024-05-17T22:36:16.004274+0000","last_fullsized":"2024-05-17T22:36:16.004274+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:14:19.725096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984193+0000","last_change":"2024-05-17T22:34:40.431711+0000","last_active":"2024-05-17T22:36:15.984193+0000","last_peered":"2024-05-17T22:36:15.984193+0000","last_clean":"2024-05-17T22:36:15.984193+0000","last_became_active":"2024-05-17T22:34:40.431368+0000","last_became_peered":"2024-05-17T22:34:40.431368+0000","last_unstale":"2024-05-17T22:36:15.984193+0000","last_undegraded":"2024-05-17T22:36:15.984193+0000","last_fullsized":"2024-05-17T22:36:15.984193+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:06:34.904652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.805730+0000","last_change":"2024-05-17T22:34:40.435593+0000","last_active":"2024-05-17T22:36:15.805730+0000","last_peered":"2024-05-17T22:36:15.805730+0000","last_clean":"2024-05-17T22:36:15.805730+0000","last_became_active":"2024-05-17T22:34:40.435241+0000","last_became_peered":"2024-05-17T22:34:40.435241+0000","last_unstale":"2024-05-17T22:36:15.805730+0000","last_undegraded":"2024-05-17T22:36:15.805730+0000","last_fullsized":"2024-05-17T22:36:15.805730+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:08:57.239998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.626366+0000","last_change":"2024-05-17T22:34:40.436248+0000","last_active":"2024-05-17T22:36:15.626366+0000","last_peered":"2024-05-17T22:36:15.626366+0000","last_clean":"2024-05-17T22:36:15.626366+0000","last_became_active":"2024-05-17T22:34:40.435824+0000","last_became_peered":"2024-05-17T22:34:40.435824+0000","last_unstale":"2024-05-17T22:36:15.626366+0000","last_undegraded":"2024-05-17T22:36:15.626366+0000","last_fullsized":"2024-05-17T22:36:15.626366+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:39:07.297470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.005145+0000","last_change":"2024-05-17T22:34:40.435843+0000","last_active":"2024-05-17T22:36:16.005145+0000","last_peered":"2024-05-17T22:36:16.005145+0000","last_clean":"2024-05-17T22:36:16.005145+0000","last_became_active":"2024-05-17T22:34:40.435500+0000","last_became_peered":"2024-05-17T22:34:40.435500+0000","last_unstale":"2024-05-17T22:36:16.005145+0000","last_undegraded":"2024-05-17T22:36:16.005145+0000","last_fullsized":"2024-05-17T22:36:16.005145+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:11:28.115850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004758+0000","last_change":"2024-05-17T22:34:40.432044+0000","last_active":"2024-05-17T22:36:16.004758+0000","last_peered":"2024-05-17T22:36:16.004758+0000","last_clean":"2024-05-17T22:36:16.004758+0000","last_became_active":"2024-05-17T22:34:40.429141+0000","last_became_peered":"2024-05-17T22:34:40.429141+0000","last_unstale":"2024-05-17T22:36:16.004758+0000","last_undegraded":"2024-05-17T22:36:16.004758+0000","last_fullsized":"2024-05-17T22:36:16.004758+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:04:53.428681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685924+0000","last_change":"2024-05-17T22:34:40.438474+0000","last_active":"2024-05-17T22:36:15.685924+0000","last_peered":"2024-05-17T22:36:15.685924+0000","last_clean":"2024-05-17T22:36:15.685924+0000","last_became_active":"2024-05-17T22:34:40.438109+0000","last_became_peered":"2024-05-17T22:34:40.438109+0000","last_unstale":"2024-05-17T22:36:15.685924+0000","last_undegraded":"2024-05-17T22:36:15.685924+0000","last_fullsized":"2024-05-17T22:36:15.685924+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:26:00.665246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968785+0000","last_change":"2024-05-17T22:34:40.431563+0000","last_active":"2024-05-17T22:36:15.968785+0000","last_peered":"2024-05-17T22:36:15.968785+0000","last_clean":"2024-05-17T22:36:15.968785+0000","last_became_active":"2024-05-17T22:34:40.430992+0000","last_became_peered":"2024-05-17T22:34:40.430992+0000","last_unstale":"2024-05-17T22:36:15.968785+0000","last_undegraded":"2024-05-17T22:36:15.968785+0000","last_fullsized":"2024-05-17T22:36:15.968785+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:51:26.280440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968328+0000","last_change":"2024-05-17T22:34:40.433629+0000","last_active":"2024-05-17T22:36:15.968328+0000","last_peered":"2024-05-17T22:36:15.968328+0000","last_clean":"2024-05-17T22:36:15.968328+0000","last_became_active":"2024-05-17T22:34:40.433270+0000","last_became_peered":"2024-05-17T22:34:40.433270+0000","last_unstale":"2024-05-17T22:36:15.968328+0000","last_undegraded":"2024-05-17T22:36:15.968328+0000","last_fullsized":"2024-05-17T22:36:15.968328+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:57:51.164078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.805030+0000","last_change":"2024-05-17T22:34:40.434218+0000","last_active":"2024-05-17T22:36:15.805030+0000","last_peered":"2024-05-17T22:36:15.805030+0000","last_clean":"2024-05-17T22:36:15.805030+0000","last_became_active":"2024-05-17T22:34:40.433849+0000","last_became_peered":"2024-05-17T22:34:40.433849+0000","last_unstale":"2024-05-17T22:36:15.805030+0000","last_undegraded":"2024-05-17T22:36:15.805030+0000","last_fullsized":"2024-05-17T22:36:15.805030+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:41:55.984109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628531+0000","last_change":"2024-05-17T22:34:40.427892+0000","last_active":"2024-05-17T22:36:15.628531+0000","last_peered":"2024-05-17T22:36:15.628531+0000","last_clean":"2024-05-17T22:36:15.628531+0000","last_became_active":"2024-05-17T22:34:40.427303+0000","last_became_peered":"2024-05-17T22:34:40.427303+0000","last_unstale":"2024-05-17T22:36:15.628531+0000","last_undegraded":"2024-05-17T22:36:15.628531+0000","last_fullsized":"2024-05-17T22:36:15.628531+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:59:38.632123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623636+0000","last_change":"2024-05-17T22:34:40.424784+0000","last_active":"2024-05-17T22:36:15.623636+0000","last_peered":"2024-05-17T22:36:15.623636+0000","last_clean":"2024-05-17T22:36:15.623636+0000","last_became_active":"2024-05-17T22:34:40.424439+0000","last_became_peered":"2024-05-17T22:34:40.424439+0000","last_unstale":"2024-05-17T22:36:15.623636+0000","last_undegraded":"2024-05-17T22:36:15.623636+0000","last_fullsized":"2024-05-17T22:36:15.623636+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:02:53.681600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624433+0000","last_change":"2024-05-17T22:34:40.435023+0000","last_active":"2024-05-17T22:36:15.624433+0000","last_peered":"2024-05-17T22:36:15.624433+0000","last_clean":"2024-05-17T22:36:15.624433+0000","last_became_active":"2024-05-17T22:34:40.434441+0000","last_became_peered":"2024-05-17T22:34:40.434441+0000","last_unstale":"2024-05-17T22:36:15.624433+0000","last_undegraded":"2024-05-17T22:36:15.624433+0000","last_fullsized":"2024-05-17T22:36:15.624433+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:16:07.671897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623973+0000","last_change":"2024-05-17T22:34:52.493192+0000","last_active":"2024-05-17T22:36:15.623973+0000","last_peered":"2024-05-17T22:36:15.623973+0000","last_clean":"2024-05-17T22:36:15.623973+0000","last_became_active":"2024-05-17T22:34:52.492786+0000","last_became_peered":"2024-05-17T22:34:52.492786+0000","last_unstale":"2024-05-17T22:36:15.623973+0000","last_undegraded":"2024-05-17T22:36:15.623973+0000","last_fullsized":"2024-05-17T22:36:15.623973+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:43:40.152240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623754+0000","last_change":"2024-05-17T22:34:37.402360+0000","last_active":"2024-05-17T22:36:15.623754+0000","last_peered":"2024-05-17T22:36:15.623754+0000","last_clean":"2024-05-17T22:36:15.623754+0000","last_became_active":"2024-05-17T22:34:37.401882+0000","last_became_peered":"2024-05-17T22:34:37.401882+0000","last_unstale":"2024-05-17T22:36:15.623754+0000","last_undegraded":"2024-05-17T22:36:15.623754+0000","last_fullsized":"2024-05-17T22:36:15.623754+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:53:13.300626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984571+0000","last_change":"2024-05-17T22:34:40.436259+0000","last_active":"2024-05-17T22:36:15.984571+0000","last_peered":"2024-05-17T22:36:15.984571+0000","last_clean":"2024-05-17T22:36:15.984571+0000","last_became_active":"2024-05-17T22:34:40.435899+0000","last_became_peered":"2024-05-17T22:34:40.435899+0000","last_unstale":"2024-05-17T22:36:15.984571+0000","last_undegraded":"2024-05-17T22:36:15.984571+0000","last_fullsized":"2024-05-17T22:36:15.984571+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:26:29.327513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003655+0000","last_change":"2024-05-17T22:34:52.495236+0000","last_active":"2024-05-17T22:36:16.003655+0000","last_peered":"2024-05-17T22:36:16.003655+0000","last_clean":"2024-05-17T22:36:16.003655+0000","last_became_active":"2024-05-17T22:34:52.494783+0000","last_became_peered":"2024-05-17T22:34:52.494783+0000","last_unstale":"2024-05-17T22:36:16.003655+0000","last_undegraded":"2024-05-17T22:36:16.003655+0000","last_fullsized":"2024-05-17T22:36:16.003655+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:08:15.010706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984767+0000","last_change":"2024-05-17T22:34:37.441129+0000","last_active":"2024-05-17T22:36:15.984767+0000","last_peered":"2024-05-17T22:36:15.984767+0000","last_clean":"2024-05-17T22:36:15.984767+0000","last_became_active":"2024-05-17T22:34:37.440738+0000","last_became_peered":"2024-05-17T22:34:37.440738+0000","last_unstale":"2024-05-17T22:36:15.984767+0000","last_undegraded":"2024-05-17T22:36:15.984767+0000","last_fullsized":"2024-05-17T22:36:15.984767+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:03:16.945076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968555+0000","last_change":"2024-05-17T22:34:40.434146+0000","last_active":"2024-05-17T22:36:15.968555+0000","last_peered":"2024-05-17T22:36:15.968555+0000","last_clean":"2024-05-17T22:36:15.968555+0000","last_became_active":"2024-05-17T22:34:40.433802+0000","last_became_peered":"2024-05-17T22:34:40.433802+0000","last_unstale":"2024-05-17T22:36:15.968555+0000","last_undegraded":"2024-05-17T22:36:15.968555+0000","last_fullsized":"2024-05-17T22:36:15.968555+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:08:39.905409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003713+0000","last_change":"2024-05-17T22:34:52.501134+0000","last_active":"2024-05-17T22:36:16.003713+0000","last_peered":"2024-05-17T22:36:16.003713+0000","last_clean":"2024-05-17T22:36:16.003713+0000","last_became_active":"2024-05-17T22:34:52.500547+0000","last_became_peered":"2024-05-17T22:34:52.500547+0000","last_unstale":"2024-05-17T22:36:16.003713+0000","last_undegraded":"2024-05-17T22:36:16.003713+0000","last_fullsized":"2024-05-17T22:36:16.003713+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:13:04.200865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.626343+0000","last_change":"2024-05-17T22:34:37.404019+0000","last_active":"2024-05-17T22:36:15.626343+0000","last_peered":"2024-05-17T22:36:15.626343+0000","last_clean":"2024-05-17T22:36:15.626343+0000","last_became_active":"2024-05-17T22:34:37.403607+0000","last_became_peered":"2024-05-17T22:34:37.403607+0000","last_unstale":"2024-05-17T22:36:15.626343+0000","last_undegraded":"2024-05-17T22:36:15.626343+0000","last_fullsized":"2024-05-17T22:36:15.626343+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:30:21.405195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.805310+0000","last_change":"2024-05-17T22:34:40.426889+0000","last_active":"2024-05-17T22:36:15.805310+0000","last_peered":"2024-05-17T22:36:15.805310+0000","last_clean":"2024-05-17T22:36:15.805310+0000","last_became_active":"2024-05-17T22:34:40.426512+0000","last_became_peered":"2024-05-17T22:34:40.426512+0000","last_unstale":"2024-05-17T22:36:15.805310+0000","last_undegraded":"2024-05-17T22:36:15.805310+0000","last_fullsized":"2024-05-17T22:36:15.805310+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:30:51.363177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003876+0000","last_change":"2024-05-17T22:34:52.487514+0000","last_active":"2024-05-17T22:36:16.003876+0000","last_peered":"2024-05-17T22:36:16.003876+0000","last_clean":"2024-05-17T22:36:16.003876+0000","last_became_active":"2024-05-17T22:34:52.487068+0000","last_became_peered":"2024-05-17T22:34:52.487068+0000","last_unstale":"2024-05-17T22:36:16.003876+0000","last_undegraded":"2024-05-17T22:36:16.003876+0000","last_fullsized":"2024-05-17T22:36:16.003876+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:52:13.573986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.626041+0000","last_change":"2024-05-17T22:34:37.400518+0000","last_active":"2024-05-17T22:36:15.626041+0000","last_peered":"2024-05-17T22:36:15.626041+0000","last_clean":"2024-05-17T22:36:15.626041+0000","last_became_active":"2024-05-17T22:34:37.400145+0000","last_became_peered":"2024-05-17T22:34:37.400145+0000","last_unstale":"2024-05-17T22:36:15.626041+0000","last_undegraded":"2024-05-17T22:36:15.626041+0000","last_fullsized":"2024-05-17T22:36:15.626041+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:56:53.593624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004156+0000","last_change":"2024-05-17T22:34:40.434148+0000","last_active":"2024-05-17T22:36:16.004156+0000","last_peered":"2024-05-17T22:36:16.004156+0000","last_clean":"2024-05-17T22:36:16.004156+0000","last_became_active":"2024-05-17T22:34:40.433613+0000","last_became_peered":"2024-05-17T22:34:40.433613+0000","last_unstale":"2024-05-17T22:36:16.004156+0000","last_undegraded":"2024-05-17T22:36:16.004156+0000","last_fullsized":"2024-05-17T22:36:16.004156+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:09:39.049852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624284+0000","last_change":"2024-05-17T22:34:52.490417+0000","last_active":"2024-05-17T22:36:15.624284+0000","last_peered":"2024-05-17T22:36:15.624284+0000","last_clean":"2024-05-17T22:36:15.624284+0000","last_became_active":"2024-05-17T22:34:52.489953+0000","last_became_peered":"2024-05-17T22:34:52.489953+0000","last_unstale":"2024-05-17T22:36:15.624284+0000","last_undegraded":"2024-05-17T22:36:15.624284+0000","last_fullsized":"2024-05-17T22:36:15.624284+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:23:56.491899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":96,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:31.106991+0000","last_change":"2024-05-17T22:34:37.405304+0000","last_active":"2024-05-17T22:36:31.106991+0000","last_peered":"2024-05-17T22:36:31.106991+0000","last_clean":"2024-05-17T22:36:31.106991+0000","last_became_active":"2024-05-17T22:34:37.404552+0000","last_became_peered":"2024-05-17T22:34:37.404552+0000","last_unstale":"2024-05-17T22:36:31.106991+0000","last_undegraded":"2024-05-17T22:36:31.106991+0000","last_fullsized":"2024-05-17T22:36:31.106991+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:57:06.556886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625732+0000","last_change":"2024-05-17T22:34:40.430537+0000","last_active":"2024-05-17T22:36:15.625732+0000","last_peered":"2024-05-17T22:36:15.625732+0000","last_clean":"2024-05-17T22:36:15.625732+0000","last_became_active":"2024-05-17T22:34:40.430193+0000","last_became_peered":"2024-05-17T22:34:40.430193+0000","last_unstale":"2024-05-17T22:36:15.625732+0000","last_undegraded":"2024-05-17T22:36:15.625732+0000","last_fullsized":"2024-05-17T22:36:15.625732+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:58:42.091445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624333+0000","last_change":"2024-05-17T22:35:15.559077+0000","last_active":"2024-05-17T22:36:15.624333+0000","last_peered":"2024-05-17T22:36:15.624333+0000","last_clean":"2024-05-17T22:36:15.624333+0000","last_became_active":"2024-05-17T22:35:15.558382+0000","last_became_peered":"2024-05-17T22:35:15.558382+0000","last_unstale":"2024-05-17T22:36:15.624333+0000","last_undegraded":"2024-05-17T22:36:15.624333+0000","last_fullsized":"2024-05-17T22:36:15.624333+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:20:16.009620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625951+0000","last_change":"2024-05-17T22:34:37.405332+0000","last_active":"2024-05-17T22:36:15.625951+0000","last_peered":"2024-05-17T22:36:15.625951+0000","last_clean":"2024-05-17T22:36:15.625951+0000","last_became_active":"2024-05-17T22:34:37.404970+0000","last_became_peered":"2024-05-17T22:34:37.404970+0000","last_unstale":"2024-05-17T22:36:15.625951+0000","last_undegraded":"2024-05-17T22:36:15.625951+0000","last_fullsized":"2024-05-17T22:36:15.625951+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:37:39.933686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003790+0000","last_change":"2024-05-17T22:34:40.432690+0000","last_active":"2024-05-17T22:36:16.003790+0000","last_peered":"2024-05-17T22:36:16.003790+0000","last_clean":"2024-05-17T22:36:16.003790+0000","last_became_active":"2024-05-17T22:34:40.432274+0000","last_became_peered":"2024-05-17T22:34:40.432274+0000","last_unstale":"2024-05-17T22:36:16.003790+0000","last_undegraded":"2024-05-17T22:36:16.003790+0000","last_fullsized":"2024-05-17T22:36:16.003790+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:12:47.926865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628515+0000","last_change":"2024-05-17T22:34:52.494645+0000","last_active":"2024-05-17T22:36:15.628515+0000","last_peered":"2024-05-17T22:36:15.628515+0000","last_clean":"2024-05-17T22:36:15.628515+0000","last_became_active":"2024-05-17T22:34:52.494255+0000","last_became_peered":"2024-05-17T22:34:52.494255+0000","last_unstale":"2024-05-17T22:36:15.628515+0000","last_undegraded":"2024-05-17T22:36:15.628515+0000","last_fullsized":"2024-05-17T22:36:15.628515+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:41:26.633571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685352+0000","last_change":"2024-05-17T22:34:37.398582+0000","last_active":"2024-05-17T22:36:15.685352+0000","last_peered":"2024-05-17T22:36:15.685352+0000","last_clean":"2024-05-17T22:36:15.685352+0000","last_became_active":"2024-05-17T22:34:37.398156+0000","last_became_peered":"2024-05-17T22:34:37.398156+0000","last_unstale":"2024-05-17T22:36:15.685352+0000","last_undegraded":"2024-05-17T22:36:15.685352+0000","last_fullsized":"2024-05-17T22:36:15.685352+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:46:00.276214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.622638+0000","last_change":"2024-05-17T22:35:15.551402+0000","last_active":"2024-05-17T22:36:15.622638+0000","last_peered":"2024-05-17T22:36:15.622638+0000","last_clean":"2024-05-17T22:36:15.622638+0000","last_became_active":"2024-05-17T22:35:15.551021+0000","last_became_peered":"2024-05-17T22:35:15.551021+0000","last_unstale":"2024-05-17T22:36:15.622638+0000","last_undegraded":"2024-05-17T22:36:15.622638+0000","last_fullsized":"2024-05-17T22:36:15.622638+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:35:18.573620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968732+0000","last_change":"2024-05-17T22:34:52.487413+0000","last_active":"2024-05-17T22:36:15.968732+0000","last_peered":"2024-05-17T22:36:15.968732+0000","last_clean":"2024-05-17T22:36:15.968732+0000","last_became_active":"2024-05-17T22:34:52.487032+0000","last_became_peered":"2024-05-17T22:34:52.487032+0000","last_unstale":"2024-05-17T22:36:15.968732+0000","last_undegraded":"2024-05-17T22:36:15.968732+0000","last_fullsized":"2024-05-17T22:36:15.968732+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:34:49.122008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003249+0000","last_change":"2024-05-17T22:34:37.404894+0000","last_active":"2024-05-17T22:36:16.003249+0000","last_peered":"2024-05-17T22:36:16.003249+0000","last_clean":"2024-05-17T22:36:16.003249+0000","last_became_active":"2024-05-17T22:34:37.404567+0000","last_became_peered":"2024-05-17T22:34:37.404567+0000","last_unstale":"2024-05-17T22:36:16.003249+0000","last_undegraded":"2024-05-17T22:36:16.003249+0000","last_fullsized":"2024-05-17T22:36:16.003249+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:13:09.761556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623009+0000","last_change":"2024-05-17T22:34:40.418267+0000","last_active":"2024-05-17T22:36:15.623009+0000","last_peered":"2024-05-17T22:36:15.623009+0000","last_clean":"2024-05-17T22:36:15.623009+0000","last_became_active":"2024-05-17T22:34:40.417928+0000","last_became_peered":"2024-05-17T22:34:40.417928+0000","last_unstale":"2024-05-17T22:36:15.623009+0000","last_undegraded":"2024-05-17T22:36:15.623009+0000","last_fullsized":"2024-05-17T22:36:15.623009+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:17:37.055004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984148+0000","last_change":"2024-05-17T22:34:52.492126+0000","last_active":"2024-05-17T22:36:15.984148+0000","last_peered":"2024-05-17T22:36:15.984148+0000","last_clean":"2024-05-17T22:36:15.984148+0000","last_became_active":"2024-05-17T22:34:52.491695+0000","last_became_peered":"2024-05-17T22:34:52.491695+0000","last_unstale":"2024-05-17T22:36:15.984148+0000","last_undegraded":"2024-05-17T22:36:15.984148+0000","last_fullsized":"2024-05-17T22:36:15.984148+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:15:17.710279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":87,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:28.101856+0000","last_change":"2024-05-17T22:34:37.406989+0000","last_active":"2024-05-17T22:36:28.101856+0000","last_peered":"2024-05-17T22:36:28.101856+0000","last_clean":"2024-05-17T22:36:28.101856+0000","last_became_active":"2024-05-17T22:34:37.406612+0000","last_became_peered":"2024-05-17T22:34:37.406612+0000","last_unstale":"2024-05-17T22:36:28.101856+0000","last_undegraded":"2024-05-17T22:36:28.101856+0000","last_fullsized":"2024-05-17T22:36:28.101856+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:47:52.480804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.805642+0000","last_change":"2024-05-17T22:34:40.427221+0000","last_active":"2024-05-17T22:36:15.805642+0000","last_peered":"2024-05-17T22:36:15.805642+0000","last_clean":"2024-05-17T22:36:15.805642+0000","last_became_active":"2024-05-17T22:34:40.426703+0000","last_became_peered":"2024-05-17T22:34:40.426703+0000","last_unstale":"2024-05-17T22:36:15.805642+0000","last_undegraded":"2024-05-17T22:36:15.805642+0000","last_fullsized":"2024-05-17T22:36:15.805642+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:49:15.629048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.622645+0000","last_change":"2024-05-17T22:34:52.499086+0000","last_active":"2024-05-17T22:36:15.622645+0000","last_peered":"2024-05-17T22:36:15.622645+0000","last_clean":"2024-05-17T22:36:15.622645+0000","last_became_active":"2024-05-17T22:34:52.498666+0000","last_became_peered":"2024-05-17T22:34:52.498666+0000","last_unstale":"2024-05-17T22:36:15.622645+0000","last_undegraded":"2024-05-17T22:36:15.622645+0000","last_fullsized":"2024-05-17T22:36:15.622645+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:46:25.882964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.005012+0000","last_change":"2024-05-17T22:35:15.836294+0000","last_active":"2024-05-17T22:36:16.005012+0000","last_peered":"2024-05-17T22:36:16.005012+0000","last_clean":"2024-05-17T22:36:16.005012+0000","last_became_active":"2024-05-17T22:35:15.835759+0000","last_became_peered":"2024-05-17T22:35:15.835759+0000","last_unstale":"2024-05-17T22:36:16.005012+0000","last_undegraded":"2024-05-17T22:36:16.005012+0000","last_fullsized":"2024-05-17T22:36:16.005012+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:23:32.106692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968497+0000","last_change":"2024-05-17T22:34:40.433331+0000","last_active":"2024-05-17T22:36:15.968497+0000","last_peered":"2024-05-17T22:36:15.968497+0000","last_clean":"2024-05-17T22:36:15.968497+0000","last_became_active":"2024-05-17T22:34:40.432989+0000","last_became_peered":"2024-05-17T22:34:40.432989+0000","last_unstale":"2024-05-17T22:36:15.968497+0000","last_undegraded":"2024-05-17T22:36:15.968497+0000","last_fullsized":"2024-05-17T22:36:15.968497+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:22:49.146148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004182+0000","last_change":"2024-05-17T22:34:52.494418+0000","last_active":"2024-05-17T22:36:16.004182+0000","last_peered":"2024-05-17T22:36:16.004182+0000","last_clean":"2024-05-17T22:36:16.004182+0000","last_became_active":"2024-05-17T22:34:52.493953+0000","last_became_peered":"2024-05-17T22:34:52.493953+0000","last_unstale":"2024-05-17T22:36:16.004182+0000","last_undegraded":"2024-05-17T22:36:16.004182+0000","last_fullsized":"2024-05-17T22:36:16.004182+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:50:40.211770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003645+0000","last_change":"2024-05-17T22:34:37.442986+0000","last_active":"2024-05-17T22:36:16.003645+0000","last_peered":"2024-05-17T22:36:16.003645+0000","last_clean":"2024-05-17T22:36:16.003645+0000","last_became_active":"2024-05-17T22:34:37.442399+0000","last_became_peered":"2024-05-17T22:34:37.442399+0000","last_unstale":"2024-05-17T22:36:16.003645+0000","last_undegraded":"2024-05-17T22:36:16.003645+0000","last_fullsized":"2024-05-17T22:36:16.003645+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:34:05.785811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.805334+0000","last_change":"2024-05-17T22:34:40.426793+0000","last_active":"2024-05-17T22:36:15.805334+0000","last_peered":"2024-05-17T22:36:15.805334+0000","last_clean":"2024-05-17T22:36:15.805334+0000","last_became_active":"2024-05-17T22:34:40.426420+0000","last_became_peered":"2024-05-17T22:34:40.426420+0000","last_unstale":"2024-05-17T22:36:15.805334+0000","last_undegraded":"2024-05-17T22:36:15.805334+0000","last_fullsized":"2024-05-17T22:36:15.805334+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:57:07.099254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.622889+0000","last_change":"2024-05-17T22:34:52.498762+0000","last_active":"2024-05-17T22:36:15.622889+0000","last_peered":"2024-05-17T22:36:15.622889+0000","last_clean":"2024-05-17T22:36:15.622889+0000","last_became_active":"2024-05-17T22:34:52.498306+0000","last_became_peered":"2024-05-17T22:34:52.498306+0000","last_unstale":"2024-05-17T22:36:15.622889+0000","last_undegraded":"2024-05-17T22:36:15.622889+0000","last_fullsized":"2024-05-17T22:36:15.622889+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:18:32.717563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'33","reported_seq":166,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:29.103413+0000","last_change":"2024-05-17T22:34:37.407885+0000","last_active":"2024-05-17T22:36:29.103413+0000","last_peered":"2024-05-17T22:36:29.103413+0000","last_clean":"2024-05-17T22:36:29.103413+0000","last_became_active":"2024-05-17T22:34:37.407463+0000","last_became_peered":"2024-05-17T22:34:37.407463+0000","last_unstale":"2024-05-17T22:36:29.103413+0000","last_undegraded":"2024-05-17T22:36:29.103413+0000","last_fullsized":"2024-05-17T22:36:29.103413+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:26:20.288557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62072,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":89,"num_read_kb":105,"num_write":34,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.626156+0000","last_change":"2024-05-17T22:34:40.435800+0000","last_active":"2024-05-17T22:36:15.626156+0000","last_peered":"2024-05-17T22:36:15.626156+0000","last_clean":"2024-05-17T22:36:15.626156+0000","last_became_active":"2024-05-17T22:34:40.435245+0000","last_became_peered":"2024-05-17T22:34:40.435245+0000","last_unstale":"2024-05-17T22:36:15.626156+0000","last_undegraded":"2024-05-17T22:36:15.626156+0000","last_fullsized":"2024-05-17T22:36:15.626156+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:09:58.699026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004407+0000","last_change":"2024-05-17T22:34:52.487336+0000","last_active":"2024-05-17T22:36:16.004407+0000","last_peered":"2024-05-17T22:36:16.004407+0000","last_clean":"2024-05-17T22:36:16.004407+0000","last_became_active":"2024-05-17T22:34:52.486883+0000","last_became_peered":"2024-05-17T22:34:52.486883+0000","last_unstale":"2024-05-17T22:36:16.004407+0000","last_undegraded":"2024-05-17T22:36:16.004407+0000","last_fullsized":"2024-05-17T22:36:16.004407+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:44:41.264072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623955+0000","last_change":"2024-05-17T22:34:37.405044+0000","last_active":"2024-05-17T22:36:15.623955+0000","last_peered":"2024-05-17T22:36:15.623955+0000","last_clean":"2024-05-17T22:36:15.623955+0000","last_became_active":"2024-05-17T22:34:37.404610+0000","last_became_peered":"2024-05-17T22:34:37.404610+0000","last_unstale":"2024-05-17T22:36:15.623955+0000","last_undegraded":"2024-05-17T22:36:15.623955+0000","last_fullsized":"2024-05-17T22:36:15.623955+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:20:37.939069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685489+0000","last_change":"2024-05-17T22:34:40.430655+0000","last_active":"2024-05-17T22:36:15.685489+0000","last_peered":"2024-05-17T22:36:15.685489+0000","last_clean":"2024-05-17T22:36:15.685489+0000","last_became_active":"2024-05-17T22:34:40.430307+0000","last_became_peered":"2024-05-17T22:34:40.430307+0000","last_unstale":"2024-05-17T22:36:15.685489+0000","last_undegraded":"2024-05-17T22:36:15.685489+0000","last_fullsized":"2024-05-17T22:36:15.685489+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:57:19.036117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628122+0000","last_change":"2024-05-17T22:34:52.494709+0000","last_active":"2024-05-17T22:36:15.628122+0000","last_peered":"2024-05-17T22:36:15.628122+0000","last_clean":"2024-05-17T22:36:15.628122+0000","last_became_active":"2024-05-17T22:34:52.494195+0000","last_became_peered":"2024-05-17T22:34:52.494195+0000","last_unstale":"2024-05-17T22:36:15.628122+0000","last_undegraded":"2024-05-17T22:36:15.628122+0000","last_fullsized":"2024-05-17T22:36:15.628122+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:35:41.188459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003638+0000","last_change":"2024-05-17T22:34:37.402494+0000","last_active":"2024-05-17T22:36:16.003638+0000","last_peered":"2024-05-17T22:36:16.003638+0000","last_clean":"2024-05-17T22:36:16.003638+0000","last_became_active":"2024-05-17T22:34:37.402102+0000","last_became_peered":"2024-05-17T22:34:37.402102+0000","last_unstale":"2024-05-17T22:36:16.003638+0000","last_undegraded":"2024-05-17T22:36:16.003638+0000","last_fullsized":"2024-05-17T22:36:16.003638+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:22:30.915780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004954+0000","last_change":"2024-05-17T22:34:40.440731+0000","last_active":"2024-05-17T22:36:16.004954+0000","last_peered":"2024-05-17T22:36:16.004954+0000","last_clean":"2024-05-17T22:36:16.004954+0000","last_became_active":"2024-05-17T22:34:40.440374+0000","last_became_peered":"2024-05-17T22:34:40.440374+0000","last_unstale":"2024-05-17T22:36:16.004954+0000","last_undegraded":"2024-05-17T22:36:16.004954+0000","last_fullsized":"2024-05-17T22:36:16.004954+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:26:40.029155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.005856+0000","last_change":"2024-05-17T22:36:16.005856+0000","last_active":"2024-05-17T22:36:16.005856+0000","last_peered":"2024-05-17T22:36:16.005856+0000","last_clean":"2024-05-17T22:36:16.005856+0000","last_became_active":"2024-05-17T22:36:16.005303+0000","last_became_peered":"2024-05-17T22:36:16.005303+0000","last_unstale":"2024-05-17T22:36:16.005856+0000","last_undegraded":"2024-05-17T22:36:16.005856+0000","last_fullsized":"2024-05-17T22:36:16.005856+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:41:15.708879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"87'4","reported_seq":93,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:31.107162+0000","last_change":"2024-05-17T22:34:37.403123+0000","last_active":"2024-05-17T22:36:31.107162+0000","last_peered":"2024-05-17T22:36:31.107162+0000","last_clean":"2024-05-17T22:36:31.107162+0000","last_became_active":"2024-05-17T22:34:37.402702+0000","last_became_peered":"2024-05-17T22:34:37.402702+0000","last_unstale":"2024-05-17T22:36:31.107162+0000","last_undegraded":"2024-05-17T22:36:31.107162+0000","last_fullsized":"2024-05-17T22:36:31.107162+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:50:02.031651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":47,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003779+0000","last_change":"2024-05-17T22:34:40.431895+0000","last_active":"2024-05-17T22:36:16.003779+0000","last_peered":"2024-05-17T22:36:16.003779+0000","last_clean":"2024-05-17T22:36:16.003779+0000","last_became_active":"2024-05-17T22:34:40.431527+0000","last_became_peered":"2024-05-17T22:34:40.431527+0000","last_unstale":"2024-05-17T22:36:16.003779+0000","last_undegraded":"2024-05-17T22:36:16.003779+0000","last_fullsized":"2024-05-17T22:36:16.003779+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:30:09.404255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.686085+0000","last_change":"2024-05-17T22:34:52.495583+0000","last_active":"2024-05-17T22:36:15.686085+0000","last_peered":"2024-05-17T22:36:15.686085+0000","last_clean":"2024-05-17T22:36:15.686085+0000","last_became_active":"2024-05-17T22:34:52.495030+0000","last_became_peered":"2024-05-17T22:34:52.495030+0000","last_unstale":"2024-05-17T22:36:15.686085+0000","last_undegraded":"2024-05-17T22:36:15.686085+0000","last_fullsized":"2024-05-17T22:36:15.686085+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:04:54.046891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625526+0000","last_change":"2024-05-17T22:34:37.442154+0000","last_active":"2024-05-17T22:36:15.625526+0000","last_peered":"2024-05-17T22:36:15.625526+0000","last_clean":"2024-05-17T22:36:15.625526+0000","last_became_active":"2024-05-17T22:34:37.441691+0000","last_became_peered":"2024-05-17T22:34:37.441691+0000","last_unstale":"2024-05-17T22:36:15.625526+0000","last_undegraded":"2024-05-17T22:36:15.625526+0000","last_fullsized":"2024-05-17T22:36:15.625526+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:47:29.862031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003475+0000","last_change":"2024-05-17T22:34:40.430894+0000","last_active":"2024-05-17T22:36:16.003475+0000","last_peered":"2024-05-17T22:36:16.003475+0000","last_clean":"2024-05-17T22:36:16.003475+0000","last_became_active":"2024-05-17T22:34:40.430328+0000","last_became_peered":"2024-05-17T22:34:40.430328+0000","last_unstale":"2024-05-17T22:36:16.003475+0000","last_undegraded":"2024-05-17T22:36:16.003475+0000","last_fullsized":"2024-05-17T22:36:16.003475+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:36:46.619111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003404+0000","last_change":"2024-05-17T22:34:52.487555+0000","last_active":"2024-05-17T22:36:16.003404+0000","last_peered":"2024-05-17T22:36:16.003404+0000","last_clean":"2024-05-17T22:36:16.003404+0000","last_became_active":"2024-05-17T22:34:52.487076+0000","last_became_peered":"2024-05-17T22:34:52.487076+0000","last_unstale":"2024-05-17T22:36:16.003404+0000","last_undegraded":"2024-05-17T22:36:16.003404+0000","last_fullsized":"2024-05-17T22:36:16.003404+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:05:46.720665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628312+0000","last_change":"2024-05-17T22:35:15.558860+0000","last_active":"2024-05-17T22:36:15.628312+0000","last_peered":"2024-05-17T22:36:15.628312+0000","last_clean":"2024-05-17T22:36:15.628312+0000","last_became_active":"2024-05-17T22:35:15.553683+0000","last_became_peered":"2024-05-17T22:35:15.553683+0000","last_unstale":"2024-05-17T22:36:15.628312+0000","last_undegraded":"2024-05-17T22:36:15.628312+0000","last_fullsized":"2024-05-17T22:36:15.628312+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:27:24.978282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628360+0000","last_change":"2024-05-17T22:34:40.433499+0000","last_active":"2024-05-17T22:36:15.628360+0000","last_peered":"2024-05-17T22:36:15.628360+0000","last_clean":"2024-05-17T22:36:15.628360+0000","last_became_active":"2024-05-17T22:34:40.431370+0000","last_became_peered":"2024-05-17T22:34:40.431370+0000","last_unstale":"2024-05-17T22:36:15.628360+0000","last_undegraded":"2024-05-17T22:36:15.628360+0000","last_fullsized":"2024-05-17T22:36:15.628360+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:25:08.383021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625886+0000","last_change":"2024-05-17T22:34:52.489627+0000","last_active":"2024-05-17T22:36:15.625886+0000","last_peered":"2024-05-17T22:36:15.625886+0000","last_clean":"2024-05-17T22:36:15.625886+0000","last_became_active":"2024-05-17T22:34:52.489173+0000","last_became_peered":"2024-05-17T22:34:52.489173+0000","last_unstale":"2024-05-17T22:36:15.625886+0000","last_undegraded":"2024-05-17T22:36:15.625886+0000","last_fullsized":"2024-05-17T22:36:15.625886+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:52:26.908817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623301+0000","last_change":"2024-05-17T22:35:15.553325+0000","last_active":"2024-05-17T22:36:15.623301+0000","last_peered":"2024-05-17T22:36:15.623301+0000","last_clean":"2024-05-17T22:36:15.623301+0000","last_became_active":"2024-05-17T22:35:15.552867+0000","last_became_peered":"2024-05-17T22:35:15.552867+0000","last_unstale":"2024-05-17T22:36:15.623301+0000","last_undegraded":"2024-05-17T22:36:15.623301+0000","last_fullsized":"2024-05-17T22:36:15.623301+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:28:36.618331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.626212+0000","last_change":"2024-05-17T22:34:40.423221+0000","last_active":"2024-05-17T22:36:15.626212+0000","last_peered":"2024-05-17T22:36:15.626212+0000","last_clean":"2024-05-17T22:36:15.626212+0000","last_became_active":"2024-05-17T22:34:40.422838+0000","last_became_peered":"2024-05-17T22:34:40.422838+0000","last_unstale":"2024-05-17T22:36:15.626212+0000","last_undegraded":"2024-05-17T22:36:15.626212+0000","last_fullsized":"2024-05-17T22:36:15.626212+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:39:51.873212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685849+0000","last_change":"2024-05-17T22:34:52.484802+0000","last_active":"2024-05-17T22:36:15.685849+0000","last_peered":"2024-05-17T22:36:15.685849+0000","last_clean":"2024-05-17T22:36:15.685849+0000","last_became_active":"2024-05-17T22:34:52.483766+0000","last_became_peered":"2024-05-17T22:34:52.483766+0000","last_unstale":"2024-05-17T22:36:15.685849+0000","last_undegraded":"2024-05-17T22:36:15.685849+0000","last_fullsized":"2024-05-17T22:36:15.685849+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:32:01.427078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004410+0000","last_change":"2024-05-17T22:34:37.404797+0000","last_active":"2024-05-17T22:36:16.004410+0000","last_peered":"2024-05-17T22:36:16.004410+0000","last_clean":"2024-05-17T22:36:16.004410+0000","last_became_active":"2024-05-17T22:34:37.404393+0000","last_became_peered":"2024-05-17T22:34:37.404393+0000","last_unstale":"2024-05-17T22:36:16.004410+0000","last_undegraded":"2024-05-17T22:36:16.004410+0000","last_fullsized":"2024-05-17T22:36:16.004410+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:26:36.887919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625922+0000","last_change":"2024-05-17T22:34:40.428210+0000","last_active":"2024-05-17T22:36:15.625922+0000","last_peered":"2024-05-17T22:36:15.625922+0000","last_clean":"2024-05-17T22:36:15.625922+0000","last_became_active":"2024-05-17T22:34:40.427845+0000","last_became_peered":"2024-05-17T22:34:40.427845+0000","last_unstale":"2024-05-17T22:36:15.625922+0000","last_undegraded":"2024-05-17T22:36:15.625922+0000","last_fullsized":"2024-05-17T22:36:15.625922+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:45:30.437139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685982+0000","last_change":"2024-05-17T22:34:52.490502+0000","last_active":"2024-05-17T22:36:15.685982+0000","last_peered":"2024-05-17T22:36:15.685982+0000","last_clean":"2024-05-17T22:36:15.685982+0000","last_became_active":"2024-05-17T22:34:52.490008+0000","last_became_peered":"2024-05-17T22:34:52.490008+0000","last_unstale":"2024-05-17T22:36:15.685982+0000","last_undegraded":"2024-05-17T22:36:15.685982+0000","last_fullsized":"2024-05-17T22:36:15.685982+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:58:29.122668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003400+0000","last_change":"2024-05-17T22:34:40.432248+0000","last_active":"2024-05-17T22:36:16.003400+0000","last_peered":"2024-05-17T22:36:16.003400+0000","last_clean":"2024-05-17T22:36:16.003400+0000","last_became_active":"2024-05-17T22:34:40.431863+0000","last_became_peered":"2024-05-17T22:34:40.431863+0000","last_unstale":"2024-05-17T22:36:16.003400+0000","last_undegraded":"2024-05-17T22:36:16.003400+0000","last_fullsized":"2024-05-17T22:36:16.003400+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:20:34.730542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'4","reported_seq":92,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:32.108099+0000","last_change":"2024-05-17T22:34:37.441186+0000","last_active":"2024-05-17T22:36:32.108099+0000","last_peered":"2024-05-17T22:36:32.108099+0000","last_clean":"2024-05-17T22:36:32.108099+0000","last_became_active":"2024-05-17T22:34:37.440781+0000","last_became_peered":"2024-05-17T22:34:37.440781+0000","last_unstale":"2024-05-17T22:36:32.108099+0000","last_undegraded":"2024-05-17T22:36:32.108099+0000","last_fullsized":"2024-05-17T22:36:32.108099+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:44:34.435147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":44,"num_read_kb":44,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.005029+0000","last_change":"2024-05-17T22:34:52.491771+0000","last_active":"2024-05-17T22:36:16.005029+0000","last_peered":"2024-05-17T22:36:16.005029+0000","last_clean":"2024-05-17T22:36:16.005029+0000","last_became_active":"2024-05-17T22:34:52.491172+0000","last_became_peered":"2024-05-17T22:34:52.491172+0000","last_unstale":"2024-05-17T22:36:16.005029+0000","last_undegraded":"2024-05-17T22:36:16.005029+0000","last_fullsized":"2024-05-17T22:36:16.005029+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:15:21.010592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984305+0000","last_change":"2024-05-17T22:34:40.430860+0000","last_active":"2024-05-17T22:36:15.984305+0000","last_peered":"2024-05-17T22:36:15.984305+0000","last_clean":"2024-05-17T22:36:15.984305+0000","last_became_active":"2024-05-17T22:34:40.430542+0000","last_became_peered":"2024-05-17T22:34:40.430542+0000","last_unstale":"2024-05-17T22:36:15.984305+0000","last_undegraded":"2024-05-17T22:36:15.984305+0000","last_fullsized":"2024-05-17T22:36:15.984305+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:16:26.257380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004059+0000","last_change":"2024-05-17T22:34:37.403026+0000","last_active":"2024-05-17T22:36:16.004059+0000","last_peered":"2024-05-17T22:36:16.004059+0000","last_clean":"2024-05-17T22:36:16.004059+0000","last_became_active":"2024-05-17T22:34:37.402560+0000","last_became_peered":"2024-05-17T22:34:37.402560+0000","last_unstale":"2024-05-17T22:36:16.004059+0000","last_undegraded":"2024-05-17T22:36:16.004059+0000","last_fullsized":"2024-05-17T22:36:16.004059+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:23:03.784411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":119,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003503+0000","last_change":"2024-05-17T22:33:11.055878+0000","last_active":"2024-05-17T22:36:16.003503+0000","last_peered":"2024-05-17T22:36:16.003503+0000","last_clean":"2024-05-17T22:36:16.003503+0000","last_became_active":"2024-05-17T22:33:11.050409+0000","last_became_peered":"2024-05-17T22:33:11.050409+0000","last_unstale":"2024-05-17T22:36:16.003503+0000","last_undegraded":"2024-05-17T22:36:16.003503+0000","last_fullsized":"2024-05-17T22:36:16.003503+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:29:43.341170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:29:43.341170+0000","last_clean_scrub_stamp":"2024-05-17T22:29:43.341170+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:30:54.153865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003650+0000","last_change":"2024-05-17T22:34:52.497064+0000","last_active":"2024-05-17T22:36:16.003650+0000","last_peered":"2024-05-17T22:36:16.003650+0000","last_clean":"2024-05-17T22:36:16.003650+0000","last_became_active":"2024-05-17T22:34:52.496611+0000","last_became_peered":"2024-05-17T22:34:52.496611+0000","last_unstale":"2024-05-17T22:36:16.003650+0000","last_undegraded":"2024-05-17T22:36:16.003650+0000","last_fullsized":"2024-05-17T22:36:16.003650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:20:29.597589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685209+0000","last_change":"2024-05-17T22:35:15.555413+0000","last_active":"2024-05-17T22:36:15.685209+0000","last_peered":"2024-05-17T22:36:15.685209+0000","last_clean":"2024-05-17T22:36:15.685209+0000","last_became_active":"2024-05-17T22:35:15.555005+0000","last_became_peered":"2024-05-17T22:35:15.555005+0000","last_unstale":"2024-05-17T22:36:15.685209+0000","last_undegraded":"2024-05-17T22:36:15.685209+0000","last_fullsized":"2024-05-17T22:36:15.685209+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:51:49.860673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003644+0000","last_change":"2024-05-17T22:34:37.406546+0000","last_active":"2024-05-17T22:36:16.003644+0000","last_peered":"2024-05-17T22:36:16.003644+0000","last_clean":"2024-05-17T22:36:16.003644+0000","last_became_active":"2024-05-17T22:34:37.406118+0000","last_became_peered":"2024-05-17T22:34:37.406118+0000","last_unstale":"2024-05-17T22:36:16.003644+0000","last_undegraded":"2024-05-17T22:36:16.003644+0000","last_fullsized":"2024-05-17T22:36:16.003644+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:47:16.350091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.005647+0000","last_change":"2024-05-17T22:36:16.005647+0000","last_active":"2024-05-17T22:36:16.005647+0000","last_peered":"2024-05-17T22:36:16.005647+0000","last_clean":"2024-05-17T22:36:16.005647+0000","last_became_active":"2024-05-17T22:36:16.005128+0000","last_became_peered":"2024-05-17T22:36:16.005128+0000","last_unstale":"2024-05-17T22:36:16.005647+0000","last_undegraded":"2024-05-17T22:36:16.005647+0000","last_fullsized":"2024-05-17T22:36:16.005647+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:16:24.481236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,7,6],"acting":[11,10,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.626298+0000","last_change":"2024-05-17T22:34:37.442676+0000","last_active":"2024-05-17T22:36:15.626298+0000","last_peered":"2024-05-17T22:36:15.626298+0000","last_clean":"2024-05-17T22:36:15.626298+0000","last_became_active":"2024-05-17T22:34:37.442275+0000","last_became_peered":"2024-05-17T22:34:37.442275+0000","last_unstale":"2024-05-17T22:36:15.626298+0000","last_undegraded":"2024-05-17T22:36:15.626298+0000","last_fullsized":"2024-05-17T22:36:15.626298+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:54:18.542987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685755+0000","last_change":"2024-05-17T22:34:40.424266+0000","last_active":"2024-05-17T22:36:15.685755+0000","last_peered":"2024-05-17T22:36:15.685755+0000","last_clean":"2024-05-17T22:36:15.685755+0000","last_became_active":"2024-05-17T22:34:40.423905+0000","last_became_peered":"2024-05-17T22:34:40.423905+0000","last_unstale":"2024-05-17T22:36:15.685755+0000","last_undegraded":"2024-05-17T22:36:15.685755+0000","last_fullsized":"2024-05-17T22:36:15.685755+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:03:43.397871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984103+0000","last_change":"2024-05-17T22:34:52.493386+0000","last_active":"2024-05-17T22:36:15.984103+0000","last_peered":"2024-05-17T22:36:15.984103+0000","last_clean":"2024-05-17T22:36:15.984103+0000","last_became_active":"2024-05-17T22:34:52.492997+0000","last_became_peered":"2024-05-17T22:34:52.492997+0000","last_unstale":"2024-05-17T22:36:15.984103+0000","last_undegraded":"2024-05-17T22:36:15.984103+0000","last_fullsized":"2024-05-17T22:36:15.984103+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:14:40.818271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'11","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623586+0000","last_change":"2024-05-17T22:34:37.399397+0000","last_active":"2024-05-17T22:36:15.623586+0000","last_peered":"2024-05-17T22:36:15.623586+0000","last_clean":"2024-05-17T22:36:15.623586+0000","last_became_active":"2024-05-17T22:34:37.399062+0000","last_became_peered":"2024-05-17T22:34:37.399062+0000","last_unstale":"2024-05-17T22:36:15.623586+0000","last_undegraded":"2024-05-17T22:36:15.623586+0000","last_fullsized":"2024-05-17T22:36:15.623586+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:02:05.230580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984853+0000","last_change":"2024-05-17T22:34:40.426995+0000","last_active":"2024-05-17T22:36:15.984853+0000","last_peered":"2024-05-17T22:36:15.984853+0000","last_clean":"2024-05-17T22:36:15.984853+0000","last_became_active":"2024-05-17T22:34:40.426226+0000","last_became_peered":"2024-05-17T22:34:40.426226+0000","last_unstale":"2024-05-17T22:36:15.984853+0000","last_undegraded":"2024-05-17T22:36:15.984853+0000","last_fullsized":"2024-05-17T22:36:15.984853+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:49:53.331076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623635+0000","last_change":"2024-05-17T22:34:52.487098+0000","last_active":"2024-05-17T22:36:15.623635+0000","last_peered":"2024-05-17T22:36:15.623635+0000","last_clean":"2024-05-17T22:36:15.623635+0000","last_became_active":"2024-05-17T22:34:52.486605+0000","last_became_peered":"2024-05-17T22:34:52.486605+0000","last_unstale":"2024-05-17T22:36:15.623635+0000","last_undegraded":"2024-05-17T22:36:15.623635+0000","last_fullsized":"2024-05-17T22:36:15.623635+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:33:28.234208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004630+0000","last_change":"2024-05-17T22:34:37.800830+0000","last_active":"2024-05-17T22:36:16.004630+0000","last_peered":"2024-05-17T22:36:16.004630+0000","last_clean":"2024-05-17T22:36:16.004630+0000","last_became_active":"2024-05-17T22:34:37.800405+0000","last_became_peered":"2024-05-17T22:34:37.800405+0000","last_unstale":"2024-05-17T22:36:16.004630+0000","last_undegraded":"2024-05-17T22:36:16.004630+0000","last_fullsized":"2024-05-17T22:36:16.004630+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:19:10.087442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968862+0000","last_change":"2024-05-17T22:34:40.426307+0000","last_active":"2024-05-17T22:36:15.968862+0000","last_peered":"2024-05-17T22:36:15.968862+0000","last_clean":"2024-05-17T22:36:15.968862+0000","last_became_active":"2024-05-17T22:34:40.425969+0000","last_became_peered":"2024-05-17T22:34:40.425969+0000","last_unstale":"2024-05-17T22:36:15.968862+0000","last_undegraded":"2024-05-17T22:36:15.968862+0000","last_fullsized":"2024-05-17T22:36:15.968862+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:25:49.680886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004451+0000","last_change":"2024-05-17T22:35:15.556857+0000","last_active":"2024-05-17T22:36:16.004451+0000","last_peered":"2024-05-17T22:36:16.004451+0000","last_clean":"2024-05-17T22:36:16.004451+0000","last_became_active":"2024-05-17T22:35:15.556298+0000","last_became_peered":"2024-05-17T22:35:15.556298+0000","last_unstale":"2024-05-17T22:36:16.004451+0000","last_undegraded":"2024-05-17T22:36:16.004451+0000","last_fullsized":"2024-05-17T22:36:16.004451+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:23:20.111544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.805120+0000","last_change":"2024-05-17T22:34:37.442971+0000","last_active":"2024-05-17T22:36:15.805120+0000","last_peered":"2024-05-17T22:36:15.805120+0000","last_clean":"2024-05-17T22:36:15.805120+0000","last_became_active":"2024-05-17T22:34:37.442536+0000","last_became_peered":"2024-05-17T22:34:37.442536+0000","last_unstale":"2024-05-17T22:36:15.805120+0000","last_undegraded":"2024-05-17T22:36:15.805120+0000","last_fullsized":"2024-05-17T22:36:15.805120+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:03:45.224071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625559+0000","last_change":"2024-05-17T22:34:40.432056+0000","last_active":"2024-05-17T22:36:15.625559+0000","last_peered":"2024-05-17T22:36:15.625559+0000","last_clean":"2024-05-17T22:36:15.625559+0000","last_became_active":"2024-05-17T22:34:40.431626+0000","last_became_peered":"2024-05-17T22:34:40.431626+0000","last_unstale":"2024-05-17T22:36:15.625559+0000","last_undegraded":"2024-05-17T22:36:15.625559+0000","last_fullsized":"2024-05-17T22:36:15.625559+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:12:24.650746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623590+0000","last_change":"2024-05-17T22:34:52.500451+0000","last_active":"2024-05-17T22:36:15.623590+0000","last_peered":"2024-05-17T22:36:15.623590+0000","last_clean":"2024-05-17T22:36:15.623590+0000","last_became_active":"2024-05-17T22:34:52.500031+0000","last_became_peered":"2024-05-17T22:34:52.500031+0000","last_unstale":"2024-05-17T22:36:15.623590+0000","last_undegraded":"2024-05-17T22:36:15.623590+0000","last_fullsized":"2024-05-17T22:36:15.623590+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:54:36.825116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984349+0000","last_change":"2024-05-17T22:34:37.402335+0000","last_active":"2024-05-17T22:36:15.984349+0000","last_peered":"2024-05-17T22:36:15.984349+0000","last_clean":"2024-05-17T22:36:15.984349+0000","last_became_active":"2024-05-17T22:34:37.401982+0000","last_became_peered":"2024-05-17T22:34:37.401982+0000","last_unstale":"2024-05-17T22:36:15.984349+0000","last_undegraded":"2024-05-17T22:36:15.984349+0000","last_fullsized":"2024-05-17T22:36:15.984349+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:58:06.194335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.627919+0000","last_change":"2024-05-17T22:34:40.425524+0000","last_active":"2024-05-17T22:36:15.627919+0000","last_peered":"2024-05-17T22:36:15.627919+0000","last_clean":"2024-05-17T22:36:15.627919+0000","last_became_active":"2024-05-17T22:34:40.425135+0000","last_became_peered":"2024-05-17T22:34:40.425135+0000","last_unstale":"2024-05-17T22:36:15.627919+0000","last_undegraded":"2024-05-17T22:36:15.627919+0000","last_fullsized":"2024-05-17T22:36:15.627919+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:41:14.176477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004602+0000","last_change":"2024-05-17T22:34:52.496290+0000","last_active":"2024-05-17T22:36:16.004602+0000","last_peered":"2024-05-17T22:36:16.004602+0000","last_clean":"2024-05-17T22:36:16.004602+0000","last_became_active":"2024-05-17T22:34:52.495817+0000","last_became_peered":"2024-05-17T22:34:52.495817+0000","last_unstale":"2024-05-17T22:36:16.004602+0000","last_undegraded":"2024-05-17T22:36:16.004602+0000","last_fullsized":"2024-05-17T22:36:16.004602+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:47:22.056372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'1","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623936+0000","last_change":"2024-05-17T22:34:37.404234+0000","last_active":"2024-05-17T22:36:15.623936+0000","last_peered":"2024-05-17T22:36:15.623936+0000","last_clean":"2024-05-17T22:36:15.623936+0000","last_became_active":"2024-05-17T22:34:37.403561+0000","last_became_peered":"2024-05-17T22:34:37.403561+0000","last_unstale":"2024-05-17T22:36:15.623936+0000","last_undegraded":"2024-05-17T22:36:15.623936+0000","last_fullsized":"2024-05-17T22:36:15.623936+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:30:28.432641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984498+0000","last_change":"2024-05-17T22:34:40.426522+0000","last_active":"2024-05-17T22:36:15.984498+0000","last_peered":"2024-05-17T22:36:15.984498+0000","last_clean":"2024-05-17T22:36:15.984498+0000","last_became_active":"2024-05-17T22:34:40.426139+0000","last_became_peered":"2024-05-17T22:34:40.426139+0000","last_unstale":"2024-05-17T22:36:15.984498+0000","last_undegraded":"2024-05-17T22:36:15.984498+0000","last_fullsized":"2024-05-17T22:36:15.984498+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:30:19.503513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004016+0000","last_change":"2024-05-17T22:34:52.487131+0000","last_active":"2024-05-17T22:36:16.004016+0000","last_peered":"2024-05-17T22:36:16.004016+0000","last_clean":"2024-05-17T22:36:16.004016+0000","last_became_active":"2024-05-17T22:34:52.486654+0000","last_became_peered":"2024-05-17T22:34:52.486654+0000","last_unstale":"2024-05-17T22:36:16.004016+0000","last_undegraded":"2024-05-17T22:36:16.004016+0000","last_fullsized":"2024-05-17T22:36:16.004016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:28:25.041746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625819+0000","last_change":"2024-05-17T22:34:37.405020+0000","last_active":"2024-05-17T22:36:15.625819+0000","last_peered":"2024-05-17T22:36:15.625819+0000","last_clean":"2024-05-17T22:36:15.625819+0000","last_became_active":"2024-05-17T22:34:37.404633+0000","last_became_peered":"2024-05-17T22:34:37.404633+0000","last_unstale":"2024-05-17T22:36:15.625819+0000","last_undegraded":"2024-05-17T22:36:15.625819+0000","last_fullsized":"2024-05-17T22:36:15.625819+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:06:00.001190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004601+0000","last_change":"2024-05-17T22:34:40.435374+0000","last_active":"2024-05-17T22:36:16.004601+0000","last_peered":"2024-05-17T22:36:16.004601+0000","last_clean":"2024-05-17T22:36:16.004601+0000","last_became_active":"2024-05-17T22:34:40.434522+0000","last_became_peered":"2024-05-17T22:34:40.434522+0000","last_unstale":"2024-05-17T22:36:16.004601+0000","last_undegraded":"2024-05-17T22:36:16.004601+0000","last_fullsized":"2024-05-17T22:36:16.004601+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:36:41.244970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.805127+0000","last_change":"2024-05-17T22:34:52.491708+0000","last_active":"2024-05-17T22:36:15.805127+0000","last_peered":"2024-05-17T22:36:15.805127+0000","last_clean":"2024-05-17T22:36:15.805127+0000","last_became_active":"2024-05-17T22:34:52.491293+0000","last_became_peered":"2024-05-17T22:34:52.491293+0000","last_unstale":"2024-05-17T22:36:15.805127+0000","last_undegraded":"2024-05-17T22:36:15.805127+0000","last_fullsized":"2024-05-17T22:36:15.805127+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:23:01.634026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624618+0000","last_change":"2024-05-17T22:34:37.800525+0000","last_active":"2024-05-17T22:36:15.624618+0000","last_peered":"2024-05-17T22:36:15.624618+0000","last_clean":"2024-05-17T22:36:15.624618+0000","last_became_active":"2024-05-17T22:34:37.800093+0000","last_became_peered":"2024-05-17T22:34:37.800093+0000","last_unstale":"2024-05-17T22:36:15.624618+0000","last_undegraded":"2024-05-17T22:36:15.624618+0000","last_fullsized":"2024-05-17T22:36:15.624618+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:25:27.249889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.005319+0000","last_change":"2024-05-17T22:34:40.434891+0000","last_active":"2024-05-17T22:36:16.005319+0000","last_peered":"2024-05-17T22:36:16.005319+0000","last_clean":"2024-05-17T22:36:16.005319+0000","last_became_active":"2024-05-17T22:34:40.434543+0000","last_became_peered":"2024-05-17T22:34:40.434543+0000","last_unstale":"2024-05-17T22:36:16.005319+0000","last_undegraded":"2024-05-17T22:36:16.005319+0000","last_fullsized":"2024-05-17T22:36:16.005319+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:17:47.770225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968349+0000","last_change":"2024-05-17T22:34:52.491544+0000","last_active":"2024-05-17T22:36:15.968349+0000","last_peered":"2024-05-17T22:36:15.968349+0000","last_clean":"2024-05-17T22:36:15.968349+0000","last_became_active":"2024-05-17T22:34:52.491138+0000","last_became_peered":"2024-05-17T22:34:52.491138+0000","last_unstale":"2024-05-17T22:36:15.968349+0000","last_undegraded":"2024-05-17T22:36:15.968349+0000","last_fullsized":"2024-05-17T22:36:15.968349+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:34:20.411299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004214+0000","last_change":"2024-05-17T22:34:37.403220+0000","last_active":"2024-05-17T22:36:16.004214+0000","last_peered":"2024-05-17T22:36:16.004214+0000","last_clean":"2024-05-17T22:36:16.004214+0000","last_became_active":"2024-05-17T22:34:37.402856+0000","last_became_peered":"2024-05-17T22:34:37.402856+0000","last_unstale":"2024-05-17T22:36:16.004214+0000","last_undegraded":"2024-05-17T22:36:16.004214+0000","last_fullsized":"2024-05-17T22:36:16.004214+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:04:19.469294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.005034+0000","last_change":"2024-05-17T22:34:40.434487+0000","last_active":"2024-05-17T22:36:16.005034+0000","last_peered":"2024-05-17T22:36:16.005034+0000","last_clean":"2024-05-17T22:36:16.005034+0000","last_became_active":"2024-05-17T22:34:40.434160+0000","last_became_peered":"2024-05-17T22:34:40.434160+0000","last_unstale":"2024-05-17T22:36:16.005034+0000","last_undegraded":"2024-05-17T22:36:16.005034+0000","last_fullsized":"2024-05-17T22:36:16.005034+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:24:38.782999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003123+0000","last_change":"2024-05-17T22:34:52.487887+0000","last_active":"2024-05-17T22:36:16.003123+0000","last_peered":"2024-05-17T22:36:16.003123+0000","last_clean":"2024-05-17T22:36:16.003123+0000","last_became_active":"2024-05-17T22:34:52.486837+0000","last_became_peered":"2024-05-17T22:34:52.486837+0000","last_unstale":"2024-05-17T22:36:16.003123+0000","last_undegraded":"2024-05-17T22:36:16.003123+0000","last_fullsized":"2024-05-17T22:36:16.003123+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:12:22.206536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.686166+0000","last_change":"2024-05-17T22:34:37.399498+0000","last_active":"2024-05-17T22:36:15.686166+0000","last_peered":"2024-05-17T22:36:15.686166+0000","last_clean":"2024-05-17T22:36:15.686166+0000","last_became_active":"2024-05-17T22:34:37.399144+0000","last_became_peered":"2024-05-17T22:34:37.399144+0000","last_unstale":"2024-05-17T22:36:15.686166+0000","last_undegraded":"2024-05-17T22:36:15.686166+0000","last_fullsized":"2024-05-17T22:36:15.686166+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:13:37.346006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.005111+0000","last_change":"2024-05-17T22:34:40.433839+0000","last_active":"2024-05-17T22:36:16.005111+0000","last_peered":"2024-05-17T22:36:16.005111+0000","last_clean":"2024-05-17T22:36:16.005111+0000","last_became_active":"2024-05-17T22:34:40.433489+0000","last_became_peered":"2024-05-17T22:34:40.433489+0000","last_unstale":"2024-05-17T22:36:16.005111+0000","last_undegraded":"2024-05-17T22:36:16.005111+0000","last_fullsized":"2024-05-17T22:36:16.005111+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:13:56.014094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628026+0000","last_change":"2024-05-17T22:34:40.434953+0000","last_active":"2024-05-17T22:36:15.628026+0000","last_peered":"2024-05-17T22:36:15.628026+0000","last_clean":"2024-05-17T22:36:15.628026+0000","last_became_active":"2024-05-17T22:34:40.434593+0000","last_became_peered":"2024-05-17T22:34:40.434593+0000","last_unstale":"2024-05-17T22:36:15.628026+0000","last_undegraded":"2024-05-17T22:36:15.628026+0000","last_fullsized":"2024-05-17T22:36:15.628026+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:20:27.075382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623850+0000","last_change":"2024-05-17T22:34:40.432787+0000","last_active":"2024-05-17T22:36:15.623850+0000","last_peered":"2024-05-17T22:36:15.623850+0000","last_clean":"2024-05-17T22:36:15.623850+0000","last_became_active":"2024-05-17T22:34:40.432402+0000","last_became_peered":"2024-05-17T22:34:40.432402+0000","last_unstale":"2024-05-17T22:36:15.623850+0000","last_undegraded":"2024-05-17T22:36:15.623850+0000","last_fullsized":"2024-05-17T22:36:15.623850+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:48:13.034276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623724+0000","last_change":"2024-05-17T22:34:40.426423+0000","last_active":"2024-05-17T22:36:15.623724+0000","last_peered":"2024-05-17T22:36:15.623724+0000","last_clean":"2024-05-17T22:36:15.623724+0000","last_became_active":"2024-05-17T22:34:40.426049+0000","last_became_peered":"2024-05-17T22:34:40.426049+0000","last_unstale":"2024-05-17T22:36:15.623724+0000","last_undegraded":"2024-05-17T22:36:15.623724+0000","last_fullsized":"2024-05-17T22:36:15.623724+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:56:55.631322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968631+0000","last_change":"2024-05-17T22:34:40.435706+0000","last_active":"2024-05-17T22:36:15.968631+0000","last_peered":"2024-05-17T22:36:15.968631+0000","last_clean":"2024-05-17T22:36:15.968631+0000","last_became_active":"2024-05-17T22:34:40.435330+0000","last_became_peered":"2024-05-17T22:34:40.435330+0000","last_unstale":"2024-05-17T22:36:15.968631+0000","last_undegraded":"2024-05-17T22:36:15.968631+0000","last_fullsized":"2024-05-17T22:36:15.968631+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:25:36.529454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685705+0000","last_change":"2024-05-17T22:34:40.428997+0000","last_active":"2024-05-17T22:36:15.685705+0000","last_peered":"2024-05-17T22:36:15.685705+0000","last_clean":"2024-05-17T22:36:15.685705+0000","last_became_active":"2024-05-17T22:34:40.428594+0000","last_became_peered":"2024-05-17T22:34:40.428594+0000","last_unstale":"2024-05-17T22:36:15.685705+0000","last_undegraded":"2024-05-17T22:36:15.685705+0000","last_fullsized":"2024-05-17T22:36:15.685705+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:59:01.777751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628370+0000","last_change":"2024-05-17T22:34:40.434241+0000","last_active":"2024-05-17T22:36:15.628370+0000","last_peered":"2024-05-17T22:36:15.628370+0000","last_clean":"2024-05-17T22:36:15.628370+0000","last_became_active":"2024-05-17T22:34:40.431652+0000","last_became_peered":"2024-05-17T22:34:40.431652+0000","last_unstale":"2024-05-17T22:36:15.628370+0000","last_undegraded":"2024-05-17T22:36:15.628370+0000","last_fullsized":"2024-05-17T22:36:15.628370+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:33:27.035284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624649+0000","last_change":"2024-05-17T22:34:40.431485+0000","last_active":"2024-05-17T22:36:15.624649+0000","last_peered":"2024-05-17T22:36:15.624649+0000","last_clean":"2024-05-17T22:36:15.624649+0000","last_became_active":"2024-05-17T22:34:40.431120+0000","last_became_peered":"2024-05-17T22:34:40.431120+0000","last_unstale":"2024-05-17T22:36:15.624649+0000","last_undegraded":"2024-05-17T22:36:15.624649+0000","last_fullsized":"2024-05-17T22:36:15.624649+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:46:55.213602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984947+0000","last_change":"2024-05-17T22:34:40.423117+0000","last_active":"2024-05-17T22:36:15.984947+0000","last_peered":"2024-05-17T22:36:15.984947+0000","last_clean":"2024-05-17T22:36:15.984947+0000","last_became_active":"2024-05-17T22:34:40.422694+0000","last_became_peered":"2024-05-17T22:34:40.422694+0000","last_unstale":"2024-05-17T22:36:15.984947+0000","last_undegraded":"2024-05-17T22:36:15.984947+0000","last_fullsized":"2024-05-17T22:36:15.984947+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:19:55.589532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685170+0000","last_change":"2024-05-17T22:35:15.711796+0000","last_active":"2024-05-17T22:36:15.685170+0000","last_peered":"2024-05-17T22:36:15.685170+0000","last_clean":"2024-05-17T22:36:15.685170+0000","last_became_active":"2024-05-17T22:35:15.711327+0000","last_became_peered":"2024-05-17T22:35:15.711327+0000","last_unstale":"2024-05-17T22:36:15.685170+0000","last_undegraded":"2024-05-17T22:36:15.685170+0000","last_fullsized":"2024-05-17T22:36:15.685170+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:48:48.913312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984270+0000","last_change":"2024-05-17T22:34:40.430393+0000","last_active":"2024-05-17T22:36:15.984270+0000","last_peered":"2024-05-17T22:36:15.984270+0000","last_clean":"2024-05-17T22:36:15.984270+0000","last_became_active":"2024-05-17T22:34:40.430030+0000","last_became_peered":"2024-05-17T22:34:40.430030+0000","last_unstale":"2024-05-17T22:36:15.984270+0000","last_undegraded":"2024-05-17T22:36:15.984270+0000","last_fullsized":"2024-05-17T22:36:15.984270+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:39:37.204246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.805401+0000","last_change":"2024-05-17T22:34:40.432443+0000","last_active":"2024-05-17T22:36:15.805401+0000","last_peered":"2024-05-17T22:36:15.805401+0000","last_clean":"2024-05-17T22:36:15.805401+0000","last_became_active":"2024-05-17T22:34:40.431991+0000","last_became_peered":"2024-05-17T22:34:40.431991+0000","last_unstale":"2024-05-17T22:36:15.805401+0000","last_undegraded":"2024-05-17T22:36:15.805401+0000","last_fullsized":"2024-05-17T22:36:15.805401+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:37:45.593575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685767+0000","last_change":"2024-05-17T22:34:40.434388+0000","last_active":"2024-05-17T22:36:15.685767+0000","last_peered":"2024-05-17T22:36:15.685767+0000","last_clean":"2024-05-17T22:36:15.685767+0000","last_became_active":"2024-05-17T22:34:40.434044+0000","last_became_peered":"2024-05-17T22:34:40.434044+0000","last_unstale":"2024-05-17T22:36:15.685767+0000","last_undegraded":"2024-05-17T22:36:15.685767+0000","last_fullsized":"2024-05-17T22:36:15.685767+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:20:43.296190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628303+0000","last_change":"2024-05-17T22:34:40.427137+0000","last_active":"2024-05-17T22:36:15.628303+0000","last_peered":"2024-05-17T22:36:15.628303+0000","last_clean":"2024-05-17T22:36:15.628303+0000","last_became_active":"2024-05-17T22:34:40.426703+0000","last_became_peered":"2024-05-17T22:34:40.426703+0000","last_unstale":"2024-05-17T22:36:15.628303+0000","last_undegraded":"2024-05-17T22:36:15.628303+0000","last_fullsized":"2024-05-17T22:36:15.628303+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:55:52.992817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66464,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":279,"num_read_kb":209,"num_write":169,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":199392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710758,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1028096,"data_stored":515036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906987,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28692,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716460,"statfs":{"total":95995035648,"available":95965655040,"internally_reserved":0,"allocated":1712128,"data_stored":1164745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070511,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":519346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266742,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1036288,"data_stored":515148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462971,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1634304,"data_stored":1111470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626495,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":510426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1085440,"data_stored":572408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":511814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1110016,"data_stored":574149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182480,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1032192,"data_stored":510620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986250,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1032192,"data_stored":515467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822726,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36944,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708208,"statfs":{"total":95995035648,"available":95957204992,"internally_reserved":0,"allocated":1642496,"data_stored":1106667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":62072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T22:36:36.378 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-17T22:36:36.379 INFO:tasks.cephadm:Setting up client nodes... 2024-05-17T22:36:36.380 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-17T22:36:36.638 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:36:36.992 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:36 smithi151 ceph-mon[36101]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T22:36:37.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:36 smithi096 ceph-mon[36065]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T22:36:37.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:36 smithi012 ceph-mon[31632]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T22:36:37.827 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:37 smithi012 ceph-mon[31632]: pgmap v360: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T22:36:38.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:37 smithi096 ceph-mon[36065]: pgmap v360: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T22:36:38.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:37 smithi151 ceph-mon[36101]: pgmap v360: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T22:36:38.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.413+0000 7fc221d9e700 1 -- 172.21.15.12:0/3073529230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc21c1029b0 msgr2=0x7fc21c102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:38.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.413+0000 7fc221d9e700 1 --2- 172.21.15.12:0/3073529230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc21c1029b0 0x7fc21c102dc0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc20c0099c0 tx=0x7fc20c009cd0 comp rx=0 tx=0).stop 2024-05-17T22:36:38.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.414+0000 7fc221d9e700 1 -- 172.21.15.12:0/3073529230 shutdown_connections 2024-05-17T22:36:38.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.414+0000 7fc221d9e700 1 --2- 172.21.15.12:0/3073529230 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc21c104480 0x7fc21c10ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:38.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.414+0000 7fc221d9e700 1 --2- 172.21.15.12:0/3073529230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc21c1029b0 0x7fc21c102dc0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:38.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.414+0000 7fc221d9e700 1 --2- 172.21.15.12:0/3073529230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc21c103ae0 0x7fc21c103ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:38.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.414+0000 7fc221d9e700 1 -- 172.21.15.12:0/3073529230 >> 172.21.15.12:0/3073529230 conn(0x7fc21c0fe1e0 msgr2=0x7fc21c100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:36:38.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.414+0000 7fc221d9e700 1 -- 172.21.15.12:0/3073529230 shutdown_connections 2024-05-17T22:36:38.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.414+0000 7fc221d9e700 1 -- 172.21.15.12:0/3073529230 wait complete. 2024-05-17T22:36:38.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.414+0000 7fc221d9e700 1 Processor -- start 2024-05-17T22:36:38.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.415+0000 7fc221d9e700 1 -- start start 2024-05-17T22:36:38.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.415+0000 7fc221d9e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc21c1029b0 0x7fc21c06caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:38.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.415+0000 7fc221d9e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc21c103ae0 0x7fc21c19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:38.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.415+0000 7fc221d9e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc21c104480 0x7fc21c1a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:38.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.415+0000 7fc221d9e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc21c10e420 con 0x7fc21c1029b0 2024-05-17T22:36:38.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.415+0000 7fc221d9e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc21c10e2a0 con 0x7fc21c103ae0 2024-05-17T22:36:38.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.415+0000 7fc221d9e700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc21c10e5a0 con 0x7fc21c104480 2024-05-17T22:36:38.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.415+0000 7fc220d9c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc21c1029b0 0x7fc21c06caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:38.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.415+0000 7fc220d9c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc21c1029b0 0x7fc21c06caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54926/0 (socket says 172.21.15.12:54926) 2024-05-17T22:36:38.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.415+0000 7fc220d9c700 1 -- 172.21.15.12:0/139929633 learned_addr learned my addr 172.21.15.12:0/139929633 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:36:38.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.415+0000 7fc21bfff700 1 --2- 172.21.15.12:0/139929633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc21c103ae0 0x7fc21c19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:38.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.415+0000 7fc22159d700 1 --2- 172.21.15.12:0/139929633 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc21c104480 0x7fc21c1a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:38.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.416+0000 7fc21bfff700 1 -- 172.21.15.12:0/139929633 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc21c104480 msgr2=0x7fc21c1a1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:38.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.416+0000 7fc21bfff700 1 --2- 172.21.15.12:0/139929633 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc21c104480 0x7fc21c1a1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:38.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.416+0000 7fc21bfff700 1 -- 172.21.15.12:0/139929633 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc21c1029b0 msgr2=0x7fc21c06caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:38.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.416+0000 7fc21bfff700 1 --2- 172.21.15.12:0/139929633 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc21c1029b0 0x7fc21c06caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:38.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.416+0000 7fc21bfff700 1 -- 172.21.15.12:0/139929633 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc21c1a1c90 con 0x7fc21c103ae0 2024-05-17T22:36:38.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.416+0000 7fc220d9c700 1 --2- 172.21.15.12:0/139929633 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc21c1029b0 0x7fc21c06caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:36:38.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.416+0000 7fc21bfff700 1 --2- 172.21.15.12:0/139929633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc21c103ae0 0x7fc21c19eef0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc21000d950 tx=0x7fc21000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:36:38.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.416+0000 7fc219ffb700 1 -- 172.21.15.12:0/139929633 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc210016070 con 0x7fc21c103ae0 2024-05-17T22:36:38.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.416+0000 7fc221d9e700 1 -- 172.21.15.12:0/139929633 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc21c1a1f80 con 0x7fc21c103ae0 2024-05-17T22:36:38.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.416+0000 7fc219ffb700 1 -- 172.21.15.12:0/139929633 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc21000fd40 con 0x7fc21c103ae0 2024-05-17T22:36:38.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.416+0000 7fc219ffb700 1 -- 172.21.15.12:0/139929633 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21000e3b0 con 0x7fc21c103ae0 2024-05-17T22:36:38.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.416+0000 7fc221d9e700 1 -- 172.21.15.12:0/139929633 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc21c1a2430 con 0x7fc21c103ae0 2024-05-17T22:36:38.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.417+0000 7fc221d9e700 1 -- 172.21.15.12:0/139929633 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc21c102dc0 con 0x7fc21c103ae0 2024-05-17T22:36:38.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.419+0000 7fc219ffb700 1 -- 172.21.15.12:0/139929633 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fc210019070 con 0x7fc21c103ae0 2024-05-17T22:36:38.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.420+0000 7fc219ffb700 1 --2- 172.21.15.12:0/139929633 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fc2040b2990 0x7fc2040b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:38.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.420+0000 7fc220d9c700 1 --2- 172.21.15.12:0/139929633 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fc2040b2990 0x7fc2040b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:38.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.420+0000 7fc219ffb700 1 -- 172.21.15.12:0/139929633 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fc2100e2610 con 0x7fc21c103ae0 2024-05-17T22:36:38.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.420+0000 7fc220d9c700 1 --2- 172.21.15.12:0/139929633 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fc2040b2990 0x7fc2040b4ea0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc20c000c00 tx=0x7fc20c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:36:38.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.423+0000 7fc219ffb700 1 -- 172.21.15.12:0/139929633 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc2100a9de0 con 0x7fc21c103ae0 2024-05-17T22:36:38.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.728+0000 7fc221d9e700 1 -- 172.21.15.12:0/139929633 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fc21c0647e0 con 0x7fc21c103ae0 2024-05-17T22:36:38.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.735+0000 7fc219ffb700 1 -- 172.21.15.12:0/139929633 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fc2100a96e0 con 0x7fc21c103ae0 2024-05-17T22:36:38.738 INFO:teuthology.orchestra.run.smithi012.stdout:[client.0] 2024-05-17T22:36:38.739 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQD220dmDkyKKxAA/2VYrXZ6+sEjl27cdaFvnA== 2024-05-17T22:36:38.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.738+0000 7fc221d9e700 1 -- 172.21.15.12:0/139929633 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fc2040b2990 msgr2=0x7fc2040b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:38.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.738+0000 7fc221d9e700 1 --2- 172.21.15.12:0/139929633 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fc2040b2990 0x7fc2040b4ea0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc20c000c00 tx=0x7fc20c01a040 comp rx=0 tx=0).stop 2024-05-17T22:36:38.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.738+0000 7fc221d9e700 1 -- 172.21.15.12:0/139929633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc21c103ae0 msgr2=0x7fc21c19eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:38.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.738+0000 7fc221d9e700 1 --2- 172.21.15.12:0/139929633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc21c103ae0 0x7fc21c19eef0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc21000d950 tx=0x7fc21000dc60 comp rx=0 tx=0).stop 2024-05-17T22:36:38.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.738+0000 7fc221d9e700 1 -- 172.21.15.12:0/139929633 shutdown_connections 2024-05-17T22:36:38.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.738+0000 7fc221d9e700 1 --2- 172.21.15.12:0/139929633 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fc2040b2990 0x7fc2040b4ea0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:38.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.738+0000 7fc221d9e700 1 --2- 172.21.15.12:0/139929633 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc21c104480 0x7fc21c1a1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:38.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.738+0000 7fc221d9e700 1 --2- 172.21.15.12:0/139929633 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc21c1029b0 0x7fc21c06caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:38.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.738+0000 7fc221d9e700 1 --2- 172.21.15.12:0/139929633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc21c103ae0 0x7fc21c19eef0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:38.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.738+0000 7fc221d9e700 1 -- 172.21.15.12:0/139929633 >> 172.21.15.12:0/139929633 conn(0x7fc21c0fe1e0 msgr2=0x7fc21c106de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:36:38.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.738+0000 7fc221d9e700 1 -- 172.21.15.12:0/139929633 shutdown_connections 2024-05-17T22:36:38.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:38.738+0000 7fc221d9e700 1 -- 172.21.15.12:0/139929633 wait complete. 2024-05-17T22:36:39.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:38 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/139929633' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-17T22:36:39.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:38 smithi096 ceph-mon[36065]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-17T22:36:39.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:38 smithi096 ceph-mon[36065]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-17T22:36:39.085 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:38 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/139929633' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-17T22:36:39.085 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:38 smithi012 ceph-mon[31632]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-17T22:36:39.085 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:38 smithi012 ceph-mon[31632]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-17T22:36:39.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:38 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/139929633' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-17T22:36:39.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:38 smithi151 ceph-mon[36101]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-17T22:36:39.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:38 smithi151 ceph-mon[36101]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-17T22:36:39.400 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-17T22:36:39.401 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-17T22:36:39.401 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-17T22:36:39.442 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-17T22:36:39.442 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-17T22:36:39.443 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph mgr dump --format=json 2024-05-17T22:36:39.732 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:36:40.005 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:39 smithi012 ceph-mon[31632]: pgmap v361: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:36:40.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:39 smithi096 ceph-mon[36065]: pgmap v361: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:36:40.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:39 smithi151 ceph-mon[36101]: pgmap v361: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:36:41.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.445+0000 7f847fb54700 1 -- 172.21.15.12:0/1759455743 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84781057d0 msgr2=0x7f847810dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:41.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.445+0000 7f847fb54700 1 --2- 172.21.15.12:0/1759455743 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84781057d0 0x7f847810dd20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8468009990 tx=0x7f8468009ca0 comp rx=0 tx=0).stop 2024-05-17T22:36:41.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.446+0000 7f847fb54700 1 -- 172.21.15.12:0/1759455743 shutdown_connections 2024-05-17T22:36:41.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.446+0000 7f847fb54700 1 --2- 172.21.15.12:0/1759455743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f847810e2b0 0x7f8478110750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:41.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.446+0000 7f847fb54700 1 --2- 172.21.15.12:0/1759455743 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8478104e80 0x7f8478105290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:41.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.446+0000 7f847fb54700 1 --2- 172.21.15.12:0/1759455743 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84781057d0 0x7f847810dd20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:41.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.446+0000 7f847fb54700 1 -- 172.21.15.12:0/1759455743 >> 172.21.15.12:0/1759455743 conn(0x7f8478079af0 msgr2=0x7f8478079f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:36:41.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.446+0000 7f847fb54700 1 -- 172.21.15.12:0/1759455743 shutdown_connections 2024-05-17T22:36:41.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.446+0000 7f847fb54700 1 -- 172.21.15.12:0/1759455743 wait complete. 2024-05-17T22:36:41.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.446+0000 7f847fb54700 1 Processor -- start 2024-05-17T22:36:41.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.446+0000 7f847fb54700 1 -- start start 2024-05-17T22:36:41.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.447+0000 7f847fb54700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8478104e80 0x7f847819a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:41.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.447+0000 7f847fb54700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84781057d0 0x7f847819ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:41.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.447+0000 7f847fb54700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f847810e2b0 0x7f847819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:41.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.447+0000 7f847fb54700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8478113dc0 con 0x7f847810e2b0 2024-05-17T22:36:41.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.447+0000 7f847fb54700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8478113c40 con 0x7f84781057d0 2024-05-17T22:36:41.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.447+0000 7f847fb54700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8478113f40 con 0x7f8478104e80 2024-05-17T22:36:41.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.447+0000 7f847e0f1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f847810e2b0 0x7f847819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:41.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.447+0000 7f847e0f1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f847810e2b0 0x7f847819efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49668/0 (socket says 172.21.15.12:49668) 2024-05-17T22:36:41.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.447+0000 7f847e0f1700 1 -- 172.21.15.12:0/142176169 learned_addr learned my addr 172.21.15.12:0/142176169 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:36:41.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.447+0000 7f847d0ef700 1 --2- 172.21.15.12:0/142176169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84781057d0 0x7f847819ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:41.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.447+0000 7f847d8f0700 1 --2- 172.21.15.12:0/142176169 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8478104e80 0x7f847819a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:41.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.447+0000 7f847e0f1700 1 -- 172.21.15.12:0/142176169 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8478104e80 msgr2=0x7f847819a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:41.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.447+0000 7f847e0f1700 1 --2- 172.21.15.12:0/142176169 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8478104e80 0x7f847819a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:41.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.447+0000 7f847e0f1700 1 -- 172.21.15.12:0/142176169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84781057d0 msgr2=0x7f847819ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:41.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.447+0000 7f847e0f1700 1 --2- 172.21.15.12:0/142176169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84781057d0 0x7f847819ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:41.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.447+0000 7f847e0f1700 1 -- 172.21.15.12:0/142176169 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f847819f5a0 con 0x7f847810e2b0 2024-05-17T22:36:41.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.447+0000 7f847d8f0700 1 --2- 172.21.15.12:0/142176169 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8478104e80 0x7f847819a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:36:41.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.447+0000 7f847e0f1700 1 --2- 172.21.15.12:0/142176169 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f847810e2b0 0x7f847819efb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f847000d980 tx=0x7f847000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:36:41.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.447+0000 7f846effd700 1 -- 172.21.15.12:0/142176169 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8470017070 con 0x7f847810e2b0 2024-05-17T22:36:41.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.448+0000 7f847fb54700 1 -- 172.21.15.12:0/142176169 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f847819f890 con 0x7f847810e2b0 2024-05-17T22:36:41.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.448+0000 7f846effd700 1 -- 172.21.15.12:0/142176169 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8470005020 con 0x7f847810e2b0 2024-05-17T22:36:41.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.448+0000 7f846effd700 1 -- 172.21.15.12:0/142176169 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84700099b0 con 0x7f847810e2b0 2024-05-17T22:36:41.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.448+0000 7f847fb54700 1 -- 172.21.15.12:0/142176169 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f847819fd50 con 0x7f847810e2b0 2024-05-17T22:36:41.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.449+0000 7f847fb54700 1 -- 172.21.15.12:0/142176169 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8478105290 con 0x7f847810e2b0 2024-05-17T22:36:41.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.450+0000 7f846effd700 1 -- 172.21.15.12:0/142176169 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f8470009b90 con 0x7f847810e2b0 2024-05-17T22:36:41.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.451+0000 7f846effd700 1 --2- 172.21.15.12:0/142176169 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f84640b2910 0x7f84640b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:41.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.451+0000 7f847d8f0700 1 --2- 172.21.15.12:0/142176169 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f84640b2910 0x7f84640b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:41.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.451+0000 7f846effd700 1 -- 172.21.15.12:0/142176169 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f84700e25f0 con 0x7f847810e2b0 2024-05-17T22:36:41.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.452+0000 7f847d8f0700 1 --2- 172.21.15.12:0/142176169 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f84640b2910 0x7f84640b4e20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f847400d340 tx=0x7f8474009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:36:41.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.456+0000 7f846effd700 1 -- 172.21.15.12:0/142176169 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8470020020 con 0x7f847810e2b0 2024-05-17T22:36:41.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.730+0000 7f847fb54700 1 -- 172.21.15.12:0/142176169 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f847819bae0 con 0x7f847810e2b0 2024-05-17T22:36:41.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.734+0000 7f846effd700 1 -- 172.21.15.12:0/142176169 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+288646 (secure 0 0 0) 0x7f84700a9dc0 con 0x7f847810e2b0 2024-05-17T22:36:41.741 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:36:41.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.739+0000 7f847fb54700 1 -- 172.21.15.12:0/142176169 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f84640b2910 msgr2=0x7f84640b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:41.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.739+0000 7f847fb54700 1 --2- 172.21.15.12:0/142176169 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f84640b2910 0x7f84640b4e20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f847400d340 tx=0x7f8474009040 comp rx=0 tx=0).stop 2024-05-17T22:36:41.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.739+0000 7f847fb54700 1 -- 172.21.15.12:0/142176169 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f847810e2b0 msgr2=0x7f847819efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:41.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.739+0000 7f847fb54700 1 --2- 172.21.15.12:0/142176169 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f847810e2b0 0x7f847819efb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f847000d980 tx=0x7f847000dc90 comp rx=0 tx=0).stop 2024-05-17T22:36:41.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.739+0000 7f847fb54700 1 -- 172.21.15.12:0/142176169 shutdown_connections 2024-05-17T22:36:41.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.739+0000 7f847fb54700 1 --2- 172.21.15.12:0/142176169 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f84640b2910 0x7f84640b4e20 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:41.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.739+0000 7f847fb54700 1 --2- 172.21.15.12:0/142176169 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f847810e2b0 0x7f847819efb0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:41.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.739+0000 7f847fb54700 1 --2- 172.21.15.12:0/142176169 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8478104e80 0x7f847819a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:41.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.739+0000 7f847fb54700 1 --2- 172.21.15.12:0/142176169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84781057d0 0x7f847819ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:41.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.739+0000 7f847fb54700 1 -- 172.21.15.12:0/142176169 >> 172.21.15.12:0/142176169 conn(0x7f8478079af0 msgr2=0x7f84780ffba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:36:41.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.739+0000 7f847fb54700 1 -- 172.21.15.12:0/142176169 shutdown_connections 2024-05-17T22:36:41.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:41.739+0000 7f847fb54700 1 -- 172.21.15.12:0/142176169 wait complete. 2024-05-17T22:36:42.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:41 smithi096 ceph-mon[36065]: pgmap v362: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T22:36:42.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:41 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/142176169' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-17T22:36:42.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:41 smithi012 ceph-mon[31632]: pgmap v362: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T22:36:42.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:41 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/142176169' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-17T22:36:42.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:41 smithi151 ceph-mon[36101]: pgmap v362: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T22:36:42.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:41 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/142176169' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-17T22:36:42.450 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":1388514047},{"type":"v1","addr":"172.21.15.12:6801","nonce":1388514047}]},"active_addr":"172.21.15.12:6801/1388514047","active_change":"2024-05-17T22:27:14.373097+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.12:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":3460962211}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":2647229663}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":2957487127}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":852753273}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":1170729706}]}]} 2024-05-17T22:36:42.456 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-17T22:36:42.456 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-17T22:36:42.456 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-17T22:36:42.708 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:36:43.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:43 smithi012 ceph-mon[31632]: pgmap v363: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T22:36:44.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:43 smithi096 ceph-mon[36065]: pgmap v363: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T22:36:44.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:43 smithi151 ceph-mon[36101]: pgmap v363: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T22:36:44.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.582+0000 7fb69eb6a700 1 -- 172.21.15.12:0/257872271 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb698107750 msgr2=0x7fb698107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:44.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.582+0000 7fb69eb6a700 1 --2- 172.21.15.12:0/257872271 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb698107750 0x7fb698107b60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb68c009990 tx=0x7fb68c009ca0 comp rx=0 tx=0).stop 2024-05-17T22:36:44.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.582+0000 7fb69eb6a700 1 -- 172.21.15.12:0/257872271 shutdown_connections 2024-05-17T22:36:44.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.582+0000 7fb69eb6a700 1 --2- 172.21.15.12:0/257872271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6981080a0 0x7fb69810fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:44.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.582+0000 7fb69eb6a700 1 --2- 172.21.15.12:0/257872271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb698102e20 0x7fb698103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:44.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.582+0000 7fb69eb6a700 1 --2- 172.21.15.12:0/257872271 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb698107750 0x7fb698107b60 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:44.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.582+0000 7fb69eb6a700 1 -- 172.21.15.12:0/257872271 >> 172.21.15.12:0/257872271 conn(0x7fb698079af0 msgr2=0x7fb698079f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:36:44.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.582+0000 7fb69eb6a700 1 -- 172.21.15.12:0/257872271 shutdown_connections 2024-05-17T22:36:44.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.583+0000 7fb69eb6a700 1 -- 172.21.15.12:0/257872271 wait complete. 2024-05-17T22:36:44.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.583+0000 7fb69eb6a700 1 Processor -- start 2024-05-17T22:36:44.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.583+0000 7fb69eb6a700 1 -- start start 2024-05-17T22:36:44.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.584+0000 7fb69eb6a700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb698102e20 0x7fb6981996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:44.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.584+0000 7fb69eb6a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb698107750 0x7fb698199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:44.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.584+0000 7fb69eb6a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6981080a0 0x7fb69819dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:44.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.584+0000 7fb69eb6a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb698112cb0 con 0x7fb6981080a0 2024-05-17T22:36:44.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.584+0000 7fb69eb6a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb698112b30 con 0x7fb698107750 2024-05-17T22:36:44.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.584+0000 7fb69eb6a700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb698112e30 con 0x7fb698102e20 2024-05-17T22:36:44.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.584+0000 7fb69d107700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6981080a0 0x7fb69819dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:44.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.584+0000 7fb69d107700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6981080a0 0x7fb69819dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49682/0 (socket says 172.21.15.12:49682) 2024-05-17T22:36:44.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.584+0000 7fb69d107700 1 -- 172.21.15.12:0/3465334292 learned_addr learned my addr 172.21.15.12:0/3465334292 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:36:44.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.584+0000 7fb697fff700 1 --2- 172.21.15.12:0/3465334292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb698107750 0x7fb698199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:44.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.584+0000 7fb69c906700 1 --2- 172.21.15.12:0/3465334292 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb698102e20 0x7fb6981996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:44.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.584+0000 7fb69d107700 1 -- 172.21.15.12:0/3465334292 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb698102e20 msgr2=0x7fb6981996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:44.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.584+0000 7fb69d107700 1 --2- 172.21.15.12:0/3465334292 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb698102e20 0x7fb6981996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:44.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.584+0000 7fb69d107700 1 -- 172.21.15.12:0/3465334292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb698107750 msgr2=0x7fb698199c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:44.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.584+0000 7fb69d107700 1 --2- 172.21.15.12:0/3465334292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb698107750 0x7fb698199c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:44.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.584+0000 7fb69d107700 1 -- 172.21.15.12:0/3465334292 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb69819e580 con 0x7fb6981080a0 2024-05-17T22:36:44.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.585+0000 7fb69d107700 1 --2- 172.21.15.12:0/3465334292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6981080a0 0x7fb69819dee0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb69000ef30 tx=0x7fb69000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:36:44.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.585+0000 7fb695ffb700 1 -- 172.21.15.12:0/3465334292 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6900092e0 con 0x7fb6981080a0 2024-05-17T22:36:44.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.585+0000 7fb69eb6a700 1 -- 172.21.15.12:0/3465334292 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb69819e810 con 0x7fb6981080a0 2024-05-17T22:36:44.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.585+0000 7fb695ffb700 1 -- 172.21.15.12:0/3465334292 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb690004bb0 con 0x7fb6981080a0 2024-05-17T22:36:44.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.585+0000 7fb695ffb700 1 -- 172.21.15.12:0/3465334292 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb690004020 con 0x7fb6981080a0 2024-05-17T22:36:44.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.585+0000 7fb69eb6a700 1 -- 172.21.15.12:0/3465334292 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb69819ecd0 con 0x7fb6981080a0 2024-05-17T22:36:44.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.586+0000 7fb69eb6a700 1 -- 172.21.15.12:0/3465334292 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb698103230 con 0x7fb6981080a0 2024-05-17T22:36:44.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.588+0000 7fb695ffb700 1 -- 172.21.15.12:0/3465334292 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fb69001b030 con 0x7fb6981080a0 2024-05-17T22:36:44.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.588+0000 7fb695ffb700 1 --2- 172.21.15.12:0/3465334292 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb6800b2910 0x7fb6800b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:44.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.589+0000 7fb69c906700 1 --2- 172.21.15.12:0/3465334292 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb6800b2910 0x7fb6800b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:44.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.589+0000 7fb695ffb700 1 -- 172.21.15.12:0/3465334292 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fb6900e2ba0 con 0x7fb6981080a0 2024-05-17T22:36:44.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.589+0000 7fb69c906700 1 --2- 172.21.15.12:0/3465334292 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb6800b2910 0x7fb6800b4e20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb688005950 tx=0x7fb68800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:36:44.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.591+0000 7fb695ffb700 1 -- 172.21.15.12:0/3465334292 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb690022020 con 0x7fb6981080a0 2024-05-17T22:36:44.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.821+0000 7fb69eb6a700 1 -- 172.21.15.12:0/3465334292 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb69819aa10 con 0x7fb6981080a0 2024-05-17T22:36:44.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.822+0000 7fb695ffb700 1 -- 172.21.15.12:0/3465334292 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24715 (secure 0 0 0) 0x7fb690010040 con 0x7fb6981080a0 2024-05-17T22:36:44.826 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:36:44.826 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":89,"fsid":"4d593780-149c-11ef-bc9a-c7b262605968","created":"2024-05-17T22:26:16.955626+0000","modified":"2024-05-17T22:36:15.618992+0000","last_up_change":"2024-05-17T22:33:36.551093+0000","last_in_change":"2024-05-17T22:33:15.032677+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T22:29:42.475513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T22:34:35.504697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T22:34:39.377097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-17T22:34:50.467684+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5d42798f-dc5f-4e66-bce2-ec33dc789e4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6803","nonce":2015605593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6805","nonce":2015605593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6809","nonce":2015605593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6807","nonce":2015605593}]},"public_addr":"172.21.15.12:6803/2015605593","cluster_addr":"172.21.15.12:6805/2015605593","heartbeat_back_addr":"172.21.15.12:6809/2015605593","heartbeat_front_addr":"172.21.15.12:6807/2015605593","state":["exists","up"]},{"osd":1,"uuid":"58afecc7-119b-4c16-a401-640fb77d4714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6801","nonce":1238316358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6803","nonce":1238316358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6807","nonce":1238316358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6805","nonce":1238316358}]},"public_addr":"172.21.15.96:6801/1238316358","cluster_addr":"172.21.15.96:6803/1238316358","heartbeat_back_addr":"172.21.15.96:6807/1238316358","heartbeat_front_addr":"172.21.15.96:6805/1238316358","state":["exists","up"]},{"osd":2,"uuid":"4bef4049-482f-49fb-82fb-38052400eb2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6801","nonce":148729815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6803","nonce":148729815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6807","nonce":148729815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6805","nonce":148729815}]},"public_addr":"172.21.15.151:6801/148729815","cluster_addr":"172.21.15.151:6803/148729815","heartbeat_back_addr":"172.21.15.151:6807/148729815","heartbeat_front_addr":"172.21.15.151:6805/148729815","state":["exists","up"]},{"osd":3,"uuid":"6fed8e36-82a2-49e4-82a3-16e6d9de87f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6811","nonce":3563309320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6813","nonce":3563309320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6817","nonce":3563309320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6815","nonce":3563309320}]},"public_addr":"172.21.15.12:6811/3563309320","cluster_addr":"172.21.15.12:6813/3563309320","heartbeat_back_addr":"172.21.15.12:6817/3563309320","heartbeat_front_addr":"172.21.15.12:6815/3563309320","state":["exists","up"]},{"osd":4,"uuid":"c0e8689a-2816-4089-ade3-931960bae4f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6809","nonce":2219103166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6811","nonce":2219103166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6815","nonce":2219103166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6813","nonce":2219103166}]},"public_addr":"172.21.15.96:6809/2219103166","cluster_addr":"172.21.15.96:6811/2219103166","heartbeat_back_addr":"172.21.15.96:6815/2219103166","heartbeat_front_addr":"172.21.15.96:6813/2219103166","state":["exists","up"]},{"osd":5,"uuid":"e76ba432-94ff-4998-aaba-2645d9e6055d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6809","nonce":2589835078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6811","nonce":2589835078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6815","nonce":2589835078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6813","nonce":2589835078}]},"public_addr":"172.21.15.151:6809/2589835078","cluster_addr":"172.21.15.151:6811/2589835078","heartbeat_back_addr":"172.21.15.151:6815/2589835078","heartbeat_front_addr":"172.21.15.151:6813/2589835078","state":["exists","up"]},{"osd":6,"uuid":"b7ece35c-efd1-40df-95da-05a0cc147372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6819","nonce":3112661724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6821","nonce":3112661724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6825","nonce":3112661724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6823","nonce":3112661724}]},"public_addr":"172.21.15.12:6819/3112661724","cluster_addr":"172.21.15.12:6821/3112661724","heartbeat_back_addr":"172.21.15.12:6825/3112661724","heartbeat_front_addr":"172.21.15.12:6823/3112661724","state":["exists","up"]},{"osd":7,"uuid":"6a65bf30-daf5-4fba-b67e-8e513d88a971","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6817","nonce":2449989110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6819","nonce":2449989110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6823","nonce":2449989110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6821","nonce":2449989110}]},"public_addr":"172.21.15.96:6817/2449989110","cluster_addr":"172.21.15.96:6819/2449989110","heartbeat_back_addr":"172.21.15.96:6823/2449989110","heartbeat_front_addr":"172.21.15.96:6821/2449989110","state":["exists","up"]},{"osd":8,"uuid":"ad62ea44-e416-4a3e-bbbe-93f2a710dbc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6817","nonce":2545224628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6819","nonce":2545224628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6823","nonce":2545224628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6821","nonce":2545224628}]},"public_addr":"172.21.15.151:6817/2545224628","cluster_addr":"172.21.15.151:6819/2545224628","heartbeat_back_addr":"172.21.15.151:6823/2545224628","heartbeat_front_addr":"172.21.15.151:6821/2545224628","state":["exists","up"]},{"osd":9,"uuid":"1275de08-2a3c-48f3-920b-dd68b4024d42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6827","nonce":2100661273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6829","nonce":2100661273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6833","nonce":2100661273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6831","nonce":2100661273}]},"public_addr":"172.21.15.12:6827/2100661273","cluster_addr":"172.21.15.12:6829/2100661273","heartbeat_back_addr":"172.21.15.12:6833/2100661273","heartbeat_front_addr":"172.21.15.12:6831/2100661273","state":["exists","up"]},{"osd":10,"uuid":"347e4e40-9a9d-43fe-8f52-fdd3199a2a5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6825","nonce":304016309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6827","nonce":304016309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6831","nonce":304016309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6829","nonce":304016309}]},"public_addr":"172.21.15.96:6825/304016309","cluster_addr":"172.21.15.96:6827/304016309","heartbeat_back_addr":"172.21.15.96:6831/304016309","heartbeat_front_addr":"172.21.15.96:6829/304016309","state":["exists","up"]},{"osd":11,"uuid":"ee8d3502-8b3c-4e6f-824f-3a97e19111ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6825","nonce":3822307314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6827","nonce":3822307314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6831","nonce":3822307314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6829","nonce":3822307314}]},"public_addr":"172.21.15.151:6825/3822307314","cluster_addr":"172.21.15.151:6827/3822307314","heartbeat_back_addr":"172.21.15.151:6831/3822307314","heartbeat_front_addr":"172.21.15.151:6829/3822307314","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:28:51.542055+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:29:15.566424+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:29:39.500981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:04.797907+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:29.655650+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:55.211880+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:31:21.559462+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:31:47.154852+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:32:13.114892+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:32:40.529708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:33:06.930942+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:33:34.061783+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":10}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/2682700253":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6801/2250046521":"2024-05-18T22:27:14.372972+0000","172.21.15.12:0/4241094239":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/4172568611":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/3512529242":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6800/1039789215":"2024-05-18T22:26:45.466719+0000","172.21.15.12:6800/2250046521":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6801/1039789215":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/2043758453":"2024-05-18T22:27:14.372972+0000","172.21.15.12:0/2694593732":"2024-05-18T22:26:45.466719+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T22:36:44.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.824+0000 7fb69eb6a700 1 -- 172.21.15.12:0/3465334292 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb6800b2910 msgr2=0x7fb6800b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:44.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.824+0000 7fb69eb6a700 1 --2- 172.21.15.12:0/3465334292 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb6800b2910 0x7fb6800b4e20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb688005950 tx=0x7fb68800b410 comp rx=0 tx=0).stop 2024-05-17T22:36:44.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.825+0000 7fb69eb6a700 1 -- 172.21.15.12:0/3465334292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6981080a0 msgr2=0x7fb69819dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:44.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.825+0000 7fb69eb6a700 1 --2- 172.21.15.12:0/3465334292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6981080a0 0x7fb69819dee0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb69000ef30 tx=0x7fb69000c3a0 comp rx=0 tx=0).stop 2024-05-17T22:36:44.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.825+0000 7fb69eb6a700 1 -- 172.21.15.12:0/3465334292 shutdown_connections 2024-05-17T22:36:44.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.825+0000 7fb69eb6a700 1 --2- 172.21.15.12:0/3465334292 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb6800b2910 0x7fb6800b4e20 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:44.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.825+0000 7fb69eb6a700 1 --2- 172.21.15.12:0/3465334292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6981080a0 0x7fb69819dee0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:44.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.825+0000 7fb69eb6a700 1 --2- 172.21.15.12:0/3465334292 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb698102e20 0x7fb6981996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:44.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.825+0000 7fb69eb6a700 1 --2- 172.21.15.12:0/3465334292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb698107750 0x7fb698199c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:44.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.825+0000 7fb69eb6a700 1 -- 172.21.15.12:0/3465334292 >> 172.21.15.12:0/3465334292 conn(0x7fb698079af0 msgr2=0x7fb698105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:36:44.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.825+0000 7fb69eb6a700 1 -- 172.21.15.12:0/3465334292 shutdown_connections 2024-05-17T22:36:44.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:44.825+0000 7fb69eb6a700 1 -- 172.21.15.12:0/3465334292 wait complete. 2024-05-17T22:36:44.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:44 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:36:44.959 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:44 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:36:45.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:44 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:36:46.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:45 smithi096 ceph-mon[36065]: pgmap v364: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T22:36:46.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:45 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/3465334292' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:36:46.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:45 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:36:46.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:45 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:36:46.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:45 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:36:46.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:45 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:36:46.077 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:45 smithi012 ceph-mon[31632]: pgmap v364: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T22:36:46.077 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:45 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/3465334292' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:36:46.078 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:45 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:36:46.078 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:45 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:36:46.078 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:45 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:36:46.078 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:45 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:36:46.111 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-17T22:36:46.111 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-17T22:36:46.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:45 smithi151 ceph-mon[36101]: pgmap v364: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T22:36:46.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:45 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/3465334292' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:36:46.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:45 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:36:46.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:45 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:36:46.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:45 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:36:46.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:45 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:36:46.395 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:36:47.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:47 smithi012 ceph-mon[31632]: pgmap v365: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T22:36:47.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:47 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:36:47.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:47 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:36:48.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:47 smithi096 ceph-mon[36065]: pgmap v365: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T22:36:48.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:47 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:36:48.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:47 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:36:48.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.136+0000 7fad157eb700 1 -- 172.21.15.12:0/2588616122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad10105790 msgr2=0x7fad1010dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:48.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.136+0000 7fad157eb700 1 --2- 172.21.15.12:0/2588616122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad10105790 0x7fad1010dce0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fad04009990 tx=0x7fad04009ca0 comp rx=0 tx=0).stop 2024-05-17T22:36:48.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.136+0000 7fad157eb700 1 -- 172.21.15.12:0/2588616122 shutdown_connections 2024-05-17T22:36:48.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.136+0000 7fad157eb700 1 --2- 172.21.15.12:0/2588616122 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fad1010e270 0x7fad10110740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:48.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.136+0000 7fad157eb700 1 --2- 172.21.15.12:0/2588616122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad10104e40 0x7fad10105250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:48.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.136+0000 7fad157eb700 1 --2- 172.21.15.12:0/2588616122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad10105790 0x7fad1010dce0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:48.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.136+0000 7fad157eb700 1 -- 172.21.15.12:0/2588616122 >> 172.21.15.12:0/2588616122 conn(0x7fad10079bd0 msgr2=0x7fad10079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:36:48.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.136+0000 7fad157eb700 1 -- 172.21.15.12:0/2588616122 shutdown_connections 2024-05-17T22:36:48.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.136+0000 7fad157eb700 1 -- 172.21.15.12:0/2588616122 wait complete. 2024-05-17T22:36:48.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.137+0000 7fad157eb700 1 Processor -- start 2024-05-17T22:36:48.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.137+0000 7fad157eb700 1 -- start start 2024-05-17T22:36:48.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.137+0000 7fad157eb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad10104e40 0x7fad101a0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:48.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.137+0000 7fad157eb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad10105790 0x7fad101a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:48.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.137+0000 7fad157eb700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fad1010e270 0x7fad1019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:48.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.137+0000 7fad157eb700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fad10113db0 con 0x7fad10105790 2024-05-17T22:36:48.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.137+0000 7fad157eb700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fad10113c30 con 0x7fad10104e40 2024-05-17T22:36:48.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.137+0000 7fad157eb700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fad10113f30 con 0x7fad1010e270 2024-05-17T22:36:48.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.137+0000 7fad0e7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad10105790 0x7fad101a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:48.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.137+0000 7fad0e7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad10105790 0x7fad101a0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49702/0 (socket says 172.21.15.12:49702) 2024-05-17T22:36:48.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.137+0000 7fad0e7fc700 1 -- 172.21.15.12:0/841506062 learned_addr learned my addr 172.21.15.12:0/841506062 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:36:48.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.137+0000 7fad0effd700 1 --2- 172.21.15.12:0/841506062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad10104e40 0x7fad101a0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:48.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.138+0000 7fad0f7fe700 1 --2- 172.21.15.12:0/841506062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fad1010e270 0x7fad1019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:48.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.138+0000 7fad0e7fc700 1 -- 172.21.15.12:0/841506062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fad1010e270 msgr2=0x7fad1019a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:48.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.138+0000 7fad0e7fc700 1 --2- 172.21.15.12:0/841506062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fad1010e270 0x7fad1019a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:48.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.138+0000 7fad0e7fc700 1 -- 172.21.15.12:0/841506062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad10104e40 msgr2=0x7fad101a0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:48.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.138+0000 7fad0e7fc700 1 --2- 172.21.15.12:0/841506062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad10104e40 0x7fad101a0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:48.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.138+0000 7fad0e7fc700 1 -- 172.21.15.12:0/841506062 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad1019add0 con 0x7fad10105790 2024-05-17T22:36:48.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.138+0000 7fad0e7fc700 1 --2- 172.21.15.12:0/841506062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad10105790 0x7fad101a0a50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fad04000c00 tx=0x7fad04009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:36:48.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.138+0000 7facf7fff700 1 -- 172.21.15.12:0/841506062 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad04027070 con 0x7fad10105790 2024-05-17T22:36:48.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.138+0000 7facf7fff700 1 -- 172.21.15.12:0/841506062 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad04022470 con 0x7fad10105790 2024-05-17T22:36:48.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.138+0000 7fad157eb700 1 -- 172.21.15.12:0/841506062 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad1019b060 con 0x7fad10105790 2024-05-17T22:36:48.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.138+0000 7facf7fff700 1 -- 172.21.15.12:0/841506062 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad04017650 con 0x7fad10105790 2024-05-17T22:36:48.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.138+0000 7fad157eb700 1 -- 172.21.15.12:0/841506062 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad101a72c0 con 0x7fad10105790 2024-05-17T22:36:48.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.139+0000 7fad157eb700 1 -- 172.21.15.12:0/841506062 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad1006c8a0 con 0x7fad10105790 2024-05-17T22:36:48.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.141+0000 7facf7fff700 1 -- 172.21.15.12:0/841506062 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fad0402a070 con 0x7fad10105790 2024-05-17T22:36:48.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.142+0000 7facf7fff700 1 --2- 172.21.15.12:0/841506062 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7facf80b2990 0x7facf80b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:48.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.142+0000 7fad0effd700 1 --2- 172.21.15.12:0/841506062 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7facf80b2990 0x7facf80b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:48.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.142+0000 7facf7fff700 1 -- 172.21.15.12:0/841506062 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fad040e38c0 con 0x7fad10105790 2024-05-17T22:36:48.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.143+0000 7fad0effd700 1 --2- 172.21.15.12:0/841506062 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7facf80b2990 0x7facf80b4ea0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fad00005950 tx=0x7fad0000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:36:48.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.146+0000 7facf7fff700 1 -- 172.21.15.12:0/841506062 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fad04013070 con 0x7fad10105790 2024-05-17T22:36:48.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:47 smithi151 ceph-mon[36101]: pgmap v365: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T22:36:48.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:47 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:36:48.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:47 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:36:48.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.363+0000 7fad157eb700 1 -- 172.21.15.12:0/841506062 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad10105250 con 0x7fad10105790 2024-05-17T22:36:48.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.364+0000 7facf7fff700 1 -- 172.21.15.12:0/841506062 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24715 (secure 0 0 0) 0x7fad040ab180 con 0x7fad10105790 2024-05-17T22:36:48.367 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:36:48.368 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":89,"fsid":"4d593780-149c-11ef-bc9a-c7b262605968","created":"2024-05-17T22:26:16.955626+0000","modified":"2024-05-17T22:36:15.618992+0000","last_up_change":"2024-05-17T22:33:36.551093+0000","last_in_change":"2024-05-17T22:33:15.032677+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T22:29:42.475513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T22:34:35.504697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T22:34:39.377097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-17T22:34:50.467684+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5d42798f-dc5f-4e66-bce2-ec33dc789e4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6803","nonce":2015605593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6805","nonce":2015605593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6809","nonce":2015605593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6807","nonce":2015605593}]},"public_addr":"172.21.15.12:6803/2015605593","cluster_addr":"172.21.15.12:6805/2015605593","heartbeat_back_addr":"172.21.15.12:6809/2015605593","heartbeat_front_addr":"172.21.15.12:6807/2015605593","state":["exists","up"]},{"osd":1,"uuid":"58afecc7-119b-4c16-a401-640fb77d4714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6801","nonce":1238316358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6803","nonce":1238316358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6807","nonce":1238316358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6805","nonce":1238316358}]},"public_addr":"172.21.15.96:6801/1238316358","cluster_addr":"172.21.15.96:6803/1238316358","heartbeat_back_addr":"172.21.15.96:6807/1238316358","heartbeat_front_addr":"172.21.15.96:6805/1238316358","state":["exists","up"]},{"osd":2,"uuid":"4bef4049-482f-49fb-82fb-38052400eb2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6801","nonce":148729815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6803","nonce":148729815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6807","nonce":148729815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6805","nonce":148729815}]},"public_addr":"172.21.15.151:6801/148729815","cluster_addr":"172.21.15.151:6803/148729815","heartbeat_back_addr":"172.21.15.151:6807/148729815","heartbeat_front_addr":"172.21.15.151:6805/148729815","state":["exists","up"]},{"osd":3,"uuid":"6fed8e36-82a2-49e4-82a3-16e6d9de87f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6811","nonce":3563309320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6813","nonce":3563309320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6817","nonce":3563309320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6815","nonce":3563309320}]},"public_addr":"172.21.15.12:6811/3563309320","cluster_addr":"172.21.15.12:6813/3563309320","heartbeat_back_addr":"172.21.15.12:6817/3563309320","heartbeat_front_addr":"172.21.15.12:6815/3563309320","state":["exists","up"]},{"osd":4,"uuid":"c0e8689a-2816-4089-ade3-931960bae4f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6809","nonce":2219103166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6811","nonce":2219103166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6815","nonce":2219103166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6813","nonce":2219103166}]},"public_addr":"172.21.15.96:6809/2219103166","cluster_addr":"172.21.15.96:6811/2219103166","heartbeat_back_addr":"172.21.15.96:6815/2219103166","heartbeat_front_addr":"172.21.15.96:6813/2219103166","state":["exists","up"]},{"osd":5,"uuid":"e76ba432-94ff-4998-aaba-2645d9e6055d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6809","nonce":2589835078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6811","nonce":2589835078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6815","nonce":2589835078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6813","nonce":2589835078}]},"public_addr":"172.21.15.151:6809/2589835078","cluster_addr":"172.21.15.151:6811/2589835078","heartbeat_back_addr":"172.21.15.151:6815/2589835078","heartbeat_front_addr":"172.21.15.151:6813/2589835078","state":["exists","up"]},{"osd":6,"uuid":"b7ece35c-efd1-40df-95da-05a0cc147372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6819","nonce":3112661724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6821","nonce":3112661724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6825","nonce":3112661724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6823","nonce":3112661724}]},"public_addr":"172.21.15.12:6819/3112661724","cluster_addr":"172.21.15.12:6821/3112661724","heartbeat_back_addr":"172.21.15.12:6825/3112661724","heartbeat_front_addr":"172.21.15.12:6823/3112661724","state":["exists","up"]},{"osd":7,"uuid":"6a65bf30-daf5-4fba-b67e-8e513d88a971","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6817","nonce":2449989110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6819","nonce":2449989110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6823","nonce":2449989110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6821","nonce":2449989110}]},"public_addr":"172.21.15.96:6817/2449989110","cluster_addr":"172.21.15.96:6819/2449989110","heartbeat_back_addr":"172.21.15.96:6823/2449989110","heartbeat_front_addr":"172.21.15.96:6821/2449989110","state":["exists","up"]},{"osd":8,"uuid":"ad62ea44-e416-4a3e-bbbe-93f2a710dbc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6817","nonce":2545224628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6819","nonce":2545224628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6823","nonce":2545224628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6821","nonce":2545224628}]},"public_addr":"172.21.15.151:6817/2545224628","cluster_addr":"172.21.15.151:6819/2545224628","heartbeat_back_addr":"172.21.15.151:6823/2545224628","heartbeat_front_addr":"172.21.15.151:6821/2545224628","state":["exists","up"]},{"osd":9,"uuid":"1275de08-2a3c-48f3-920b-dd68b4024d42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6827","nonce":2100661273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6829","nonce":2100661273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6833","nonce":2100661273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6831","nonce":2100661273}]},"public_addr":"172.21.15.12:6827/2100661273","cluster_addr":"172.21.15.12:6829/2100661273","heartbeat_back_addr":"172.21.15.12:6833/2100661273","heartbeat_front_addr":"172.21.15.12:6831/2100661273","state":["exists","up"]},{"osd":10,"uuid":"347e4e40-9a9d-43fe-8f52-fdd3199a2a5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6825","nonce":304016309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6827","nonce":304016309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6831","nonce":304016309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6829","nonce":304016309}]},"public_addr":"172.21.15.96:6825/304016309","cluster_addr":"172.21.15.96:6827/304016309","heartbeat_back_addr":"172.21.15.96:6831/304016309","heartbeat_front_addr":"172.21.15.96:6829/304016309","state":["exists","up"]},{"osd":11,"uuid":"ee8d3502-8b3c-4e6f-824f-3a97e19111ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6825","nonce":3822307314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6827","nonce":3822307314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6831","nonce":3822307314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6829","nonce":3822307314}]},"public_addr":"172.21.15.151:6825/3822307314","cluster_addr":"172.21.15.151:6827/3822307314","heartbeat_back_addr":"172.21.15.151:6831/3822307314","heartbeat_front_addr":"172.21.15.151:6829/3822307314","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:28:51.542055+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:29:15.566424+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:29:39.500981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:04.797907+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:29.655650+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:55.211880+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:31:21.559462+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:31:47.154852+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:32:13.114892+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:32:40.529708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:33:06.930942+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:33:34.061783+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":10}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/2682700253":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6801/2250046521":"2024-05-18T22:27:14.372972+0000","172.21.15.12:0/4241094239":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/4172568611":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/3512529242":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6800/1039789215":"2024-05-18T22:26:45.466719+0000","172.21.15.12:6800/2250046521":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6801/1039789215":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/2043758453":"2024-05-18T22:27:14.372972+0000","172.21.15.12:0/2694593732":"2024-05-18T22:26:45.466719+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T22:36:48.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.366+0000 7fad157eb700 1 -- 172.21.15.12:0/841506062 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7facf80b2990 msgr2=0x7facf80b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:48.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.366+0000 7fad157eb700 1 --2- 172.21.15.12:0/841506062 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7facf80b2990 0x7facf80b4ea0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fad00005950 tx=0x7fad0000b410 comp rx=0 tx=0).stop 2024-05-17T22:36:48.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.366+0000 7fad157eb700 1 -- 172.21.15.12:0/841506062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad10105790 msgr2=0x7fad101a0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:48.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.366+0000 7fad157eb700 1 --2- 172.21.15.12:0/841506062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad10105790 0x7fad101a0a50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fad04000c00 tx=0x7fad04009580 comp rx=0 tx=0).stop 2024-05-17T22:36:48.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.366+0000 7fad157eb700 1 -- 172.21.15.12:0/841506062 shutdown_connections 2024-05-17T22:36:48.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.366+0000 7fad157eb700 1 --2- 172.21.15.12:0/841506062 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7facf80b2990 0x7facf80b4ea0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:48.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.366+0000 7fad157eb700 1 --2- 172.21.15.12:0/841506062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fad1010e270 0x7fad1019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:48.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.366+0000 7fad157eb700 1 --2- 172.21.15.12:0/841506062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad10104e40 0x7fad101a0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:48.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.366+0000 7fad157eb700 1 --2- 172.21.15.12:0/841506062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad10105790 0x7fad101a0a50 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:48.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.366+0000 7fad157eb700 1 -- 172.21.15.12:0/841506062 >> 172.21.15.12:0/841506062 conn(0x7fad10079bd0 msgr2=0x7fad10100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:36:48.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.367+0000 7fad157eb700 1 -- 172.21.15.12:0/841506062 shutdown_connections 2024-05-17T22:36:48.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:48.367+0000 7fad157eb700 1 -- 172.21.15.12:0/841506062 wait complete. 2024-05-17T22:36:49.007 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:48 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:36:49.007 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:48 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:36:49.007 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:48 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:36:49.007 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:48 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/841506062' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:36:49.042 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-17T22:36:49.043 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-17T22:36:49.043 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-17T22:36:49.043 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-17T22:36:49.044 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-17T22:36:49.044 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-17T22:36:49.044 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-17T22:36:49.045 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-17T22:36:49.045 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-17T22:36:49.046 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-17T22:36:49.046 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-17T22:36:49.046 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-17T22:36:49.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:48 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:36:49.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:48 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:36:49.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:48 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:36:49.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:48 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/841506062' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:36:49.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:48 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:36:49.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:48 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:36:49.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:48 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:36:49.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:48 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/841506062' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:36:49.725 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:36:49.850 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:36:49.852 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:36:50.084 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:36:50.084 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:36:50.084 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:36:50.085 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:36:50.085 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:36:50.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:49 smithi012 ceph-mon[31632]: pgmap v366: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:36:50.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:49 smithi151 ceph-mon[36101]: pgmap v366: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:36:50.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:49 smithi096 ceph-mon[36065]: pgmap v366: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:36:51.648 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:36:51.882 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:51 smithi012 ceph-mon[31632]: pgmap v367: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T22:36:52.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:51 smithi151 ceph-mon[36101]: pgmap v367: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T22:36:52.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:51 smithi096 ceph-mon[36065]: pgmap v367: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T22:36:54.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:53 smithi012 ceph-mon[31632]: pgmap v368: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T22:36:54.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:53 smithi151 ceph-mon[36101]: pgmap v368: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T22:36:54.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:53 smithi096 ceph-mon[36065]: pgmap v368: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T22:36:55.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.722+0000 7fef54e48700 1 -- 172.21.15.12:0/1504343479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef5010e270 msgr2=0x7fef50110740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:55.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.722+0000 7fef54e48700 1 --2- 172.21.15.12:0/1504343479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef5010e270 0x7fef50110740 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fef4800b0a0 tx=0x7fef4800b3b0 comp rx=0 tx=0).stop 2024-05-17T22:36:55.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.722+0000 7fef54e48700 1 -- 172.21.15.12:0/1504343479 shutdown_connections 2024-05-17T22:36:55.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.722+0000 7fef54e48700 1 --2- 172.21.15.12:0/1504343479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef5010e270 0x7fef50110740 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:55.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.722+0000 7fef54e48700 1 --2- 172.21.15.12:0/1504343479 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef50104e40 0x7fef50105250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:55.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.722+0000 7fef54e48700 1 --2- 172.21.15.12:0/1504343479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef50105790 0x7fef5010dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:55.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.722+0000 7fef54e48700 1 -- 172.21.15.12:0/1504343479 >> 172.21.15.12:0/1504343479 conn(0x7fef50079bd0 msgr2=0x7fef50079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:36:55.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.722+0000 7fef54e48700 1 -- 172.21.15.12:0/1504343479 shutdown_connections 2024-05-17T22:36:55.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.722+0000 7fef54e48700 1 -- 172.21.15.12:0/1504343479 wait complete. 2024-05-17T22:36:55.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.723+0000 7fef54e48700 1 Processor -- start 2024-05-17T22:36:55.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.723+0000 7fef54e48700 1 -- start start 2024-05-17T22:36:55.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.724+0000 7fef54e48700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef50104e40 0x7fef5019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:55.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.724+0000 7fef54e48700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef50105790 0x7fef5019ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:55.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.724+0000 7fef54e48700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef5010e270 0x7fef5019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:55.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.724+0000 7fef54e48700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fef50113db0 con 0x7fef5010e270 2024-05-17T22:36:55.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.724+0000 7fef54e48700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fef50113c30 con 0x7fef50104e40 2024-05-17T22:36:55.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.724+0000 7fef54e48700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fef50113f30 con 0x7fef50105790 2024-05-17T22:36:55.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.724+0000 7fef4ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef5010e270 0x7fef5019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:55.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.724+0000 7fef4ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef5010e270 0x7fef5019ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53752/0 (socket says 172.21.15.12:53752) 2024-05-17T22:36:55.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.724+0000 7fef4ed9d700 1 -- 172.21.15.12:0/1878997166 learned_addr learned my addr 172.21.15.12:0/1878997166 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:36:55.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.724+0000 7fef4e59c700 1 --2- 172.21.15.12:0/1878997166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef50104e40 0x7fef5019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:55.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.724+0000 7fef4dd9b700 1 --2- 172.21.15.12:0/1878997166 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef50105790 0x7fef5019ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:55.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.724+0000 7fef4ed9d700 1 -- 172.21.15.12:0/1878997166 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef50105790 msgr2=0x7fef5019ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:55.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.724+0000 7fef4ed9d700 1 --2- 172.21.15.12:0/1878997166 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef50105790 0x7fef5019ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:55.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.724+0000 7fef4ed9d700 1 -- 172.21.15.12:0/1878997166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef50104e40 msgr2=0x7fef5019a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:55.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.724+0000 7fef4ed9d700 1 --2- 172.21.15.12:0/1878997166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef50104e40 0x7fef5019a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:55.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.724+0000 7fef4ed9d700 1 -- 172.21.15.12:0/1878997166 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef5019f5c0 con 0x7fef5010e270 2024-05-17T22:36:55.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.724+0000 7fef4e59c700 1 --2- 172.21.15.12:0/1878997166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef50104e40 0x7fef5019a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:36:55.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.724+0000 7fef4ed9d700 1 --2- 172.21.15.12:0/1878997166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef5010e270 0x7fef5019ef20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fef4800b830 tx=0x7fef48004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:36:55.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.725+0000 7fef3f7fe700 1 -- 172.21.15.12:0/1878997166 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef48021070 con 0x7fef5010e270 2024-05-17T22:36:55.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.725+0000 7fef3f7fe700 1 -- 172.21.15.12:0/1878997166 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef48007e30 con 0x7fef5010e270 2024-05-17T22:36:55.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.725+0000 7fef54e48700 1 -- 172.21.15.12:0/1878997166 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef5019f850 con 0x7fef5010e270 2024-05-17T22:36:55.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.725+0000 7fef3f7fe700 1 -- 172.21.15.12:0/1878997166 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef480047d0 con 0x7fef5010e270 2024-05-17T22:36:55.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.725+0000 7fef54e48700 1 -- 172.21.15.12:0/1878997166 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef501a71d0 con 0x7fef5010e270 2024-05-17T22:36:55.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.725+0000 7fef54e48700 1 -- 172.21.15.12:0/1878997166 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fef50063fc0 con 0x7fef5010e270 2024-05-17T22:36:55.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.727+0000 7fef3f7fe700 1 -- 172.21.15.12:0/1878997166 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fef48007980 con 0x7fef5010e270 2024-05-17T22:36:55.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.728+0000 7fef3f7fe700 1 --2- 172.21.15.12:0/1878997166 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fef380b2910 0x7fef380b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:55.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.728+0000 7fef4e59c700 1 --2- 172.21.15.12:0/1878997166 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fef380b2910 0x7fef380b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:55.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.728+0000 7fef3f7fe700 1 -- 172.21.15.12:0/1878997166 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fef4800f020 con 0x7fef5010e270 2024-05-17T22:36:55.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.728+0000 7fef3f7fe700 1 --2- 172.21.15.12:0/1878997166 >> [v2:172.21.15.12:6802/2015605593,v1:172.21.15.12:6803/2015605593] conn(0x7fef380bdbc0 0x7fef380c00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:55.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.728+0000 7fef3f7fe700 1 -- 172.21.15.12:0/1878997166 --> [v2:172.21.15.12:6802/2015605593,v1:172.21.15.12:6803/2015605593] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fef4801eec0 con 0x7fef380bdbc0 2024-05-17T22:36:55.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.728+0000 7fef3f7fe700 1 -- 172.21.15.12:0/1878997166 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fef480e45a0 con 0x7fef5010e270 2024-05-17T22:36:55.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.729+0000 7fef4dd9b700 1 --2- 172.21.15.12:0/1878997166 >> [v2:172.21.15.12:6802/2015605593,v1:172.21.15.12:6803/2015605593] conn(0x7fef380bdbc0 0x7fef380c00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:55.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.729+0000 7fef4e59c700 1 --2- 172.21.15.12:0/1878997166 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fef380b2910 0x7fef380b4e20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fef40005950 tx=0x7fef400058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:36:55.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.729+0000 7fef4dd9b700 1 --2- 172.21.15.12:0/1878997166 >> [v2:172.21.15.12:6802/2015605593,v1:172.21.15.12:6803/2015605593] conn(0x7fef380bdbc0 0x7fef380c00b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:36:55.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.730+0000 7fef3f7fe700 1 -- 172.21.15.12:0/1878997166 <== osd.0 v2:172.21.15.12:6802/2015605593 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fef4801eec0 con 0x7fef380bdbc0 2024-05-17T22:36:55.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.746+0000 7fef54e48700 1 -- 172.21.15.12:0/1878997166 --> [v2:172.21.15.12:6802/2015605593,v1:172.21.15.12:6803/2015605593] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fef500643c0 con 0x7fef380bdbc0 2024-05-17T22:36:55.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.746+0000 7fef3f7fe700 1 -- 172.21.15.12:0/1878997166 <== osd.0 v2:172.21.15.12:6802/2015605593 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fef500643c0 con 0x7fef380bdbc0 2024-05-17T22:36:55.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.747+0000 7fef54e48700 1 -- 172.21.15.12:0/1878997166 >> [v2:172.21.15.12:6802/2015605593,v1:172.21.15.12:6803/2015605593] conn(0x7fef380bdbc0 msgr2=0x7fef380c00b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:55.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.747+0000 7fef54e48700 1 --2- 172.21.15.12:0/1878997166 >> [v2:172.21.15.12:6802/2015605593,v1:172.21.15.12:6803/2015605593] conn(0x7fef380bdbc0 0x7fef380c00b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:55.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.747+0000 7fef54e48700 1 -- 172.21.15.12:0/1878997166 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fef380b2910 msgr2=0x7fef380b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:55.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.747+0000 7fef54e48700 1 --2- 172.21.15.12:0/1878997166 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fef380b2910 0x7fef380b4e20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fef40005950 tx=0x7fef400058e0 comp rx=0 tx=0).stop 2024-05-17T22:36:55.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.747+0000 7fef54e48700 1 -- 172.21.15.12:0/1878997166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef5010e270 msgr2=0x7fef5019ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:55.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.747+0000 7fef54e48700 1 --2- 172.21.15.12:0/1878997166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef5010e270 0x7fef5019ef20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fef4800b830 tx=0x7fef48004ce0 comp rx=0 tx=0).stop 2024-05-17T22:36:55.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.747+0000 7fef4ed9d700 1 -- 172.21.15.12:0/1878997166 reap_dead start 2024-05-17T22:36:55.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.747+0000 7fef54e48700 1 -- 172.21.15.12:0/1878997166 shutdown_connections 2024-05-17T22:36:55.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.747+0000 7fef54e48700 1 -- 172.21.15.12:0/1878997166 >> 172.21.15.12:0/1878997166 conn(0x7fef50079bd0 msgr2=0x7fef500ffad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:36:55.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.747+0000 7fef54e48700 1 -- 172.21.15.12:0/1878997166 shutdown_connections 2024-05-17T22:36:55.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.747+0000 7fef54e48700 1 -- 172.21.15.12:0/1878997166 wait complete. 2024-05-17T22:36:56.001 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:55 smithi012 ceph-mon[31632]: pgmap v369: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T22:36:56.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:55 smithi151 ceph-mon[36101]: pgmap v369: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T22:36:56.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:55 smithi096 ceph-mon[36065]: pgmap v369: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T22:36:57.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:57 smithi012 ceph-mon[31632]: pgmap v370: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T22:36:58.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.506+0000 7fc45eb76700 1 -- 172.21.15.12:0/3407639421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc458107750 msgr2=0x7fc458107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:58.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.506+0000 7fc45eb76700 1 --2- 172.21.15.12:0/3407639421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc458107750 0x7fc458107b60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc444009990 tx=0x7fc444009ca0 comp rx=0 tx=0).stop 2024-05-17T22:36:58.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.506+0000 7fc45eb76700 1 -- 172.21.15.12:0/3407639421 shutdown_connections 2024-05-17T22:36:58.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.506+0000 7fc45eb76700 1 --2- 172.21.15.12:0/3407639421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4581080a0 0x7fc45810fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:58.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.506+0000 7fc45eb76700 1 --2- 172.21.15.12:0/3407639421 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc458102e20 0x7fc458103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:58.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.506+0000 7fc45eb76700 1 --2- 172.21.15.12:0/3407639421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc458107750 0x7fc458107b60 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:58.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.506+0000 7fc45eb76700 1 -- 172.21.15.12:0/3407639421 >> 172.21.15.12:0/3407639421 conn(0x7fc458079bd0 msgr2=0x7fc458079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:36:58.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.506+0000 7fc45eb76700 1 -- 172.21.15.12:0/3407639421 shutdown_connections 2024-05-17T22:36:58.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.506+0000 7fc45eb76700 1 -- 172.21.15.12:0/3407639421 wait complete. 2024-05-17T22:36:58.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.507+0000 7fc45eb76700 1 Processor -- start 2024-05-17T22:36:58.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.507+0000 7fc45eb76700 1 -- start start 2024-05-17T22:36:58.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.508+0000 7fc45eb76700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc458102e20 0x7fc458199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:58.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.508+0000 7fc45eb76700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc458107750 0x7fc458199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:58.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.508+0000 7fc45eb76700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4581080a0 0x7fc458199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:58.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.508+0000 7fc45eb76700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc458112cb0 con 0x7fc458107750 2024-05-17T22:36:58.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.508+0000 7fc45eb76700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc458112b30 con 0x7fc4581080a0 2024-05-17T22:36:58.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.508+0000 7fc45eb76700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc458112e30 con 0x7fc458102e20 2024-05-17T22:36:58.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.508+0000 7fc457fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc458107750 0x7fc458199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:58.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.508+0000 7fc457fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc458107750 0x7fc458199950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53722/0 (socket says 172.21.15.12:53722) 2024-05-17T22:36:58.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.508+0000 7fc457fff700 1 -- 172.21.15.12:0/633768906 learned_addr learned my addr 172.21.15.12:0/633768906 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:36:58.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.508+0000 7fc45c912700 1 --2- 172.21.15.12:0/633768906 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc458102e20 0x7fc458199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:58.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.508+0000 7fc45d113700 1 --2- 172.21.15.12:0/633768906 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4581080a0 0x7fc458199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:58.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.508+0000 7fc457fff700 1 -- 172.21.15.12:0/633768906 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc458102e20 msgr2=0x7fc458199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:58.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.508+0000 7fc457fff700 1 --2- 172.21.15.12:0/633768906 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc458102e20 0x7fc458199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:58.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.508+0000 7fc457fff700 1 -- 172.21.15.12:0/633768906 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4581080a0 msgr2=0x7fc458199eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:58.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.508+0000 7fc457fff700 1 --2- 172.21.15.12:0/633768906 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4581080a0 0x7fc458199eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:58.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.508+0000 7fc457fff700 1 -- 172.21.15.12:0/633768906 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc45819e6a0 con 0x7fc458107750 2024-05-17T22:36:58.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.508+0000 7fc45d113700 1 --2- 172.21.15.12:0/633768906 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4581080a0 0x7fc458199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:36:58.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.509+0000 7fc457fff700 1 --2- 172.21.15.12:0/633768906 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc458107750 0x7fc458199950 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc444000c00 tx=0x7fc444005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:36:58.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.509+0000 7fc455ffb700 1 -- 172.21.15.12:0/633768906 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc444027070 con 0x7fc458107750 2024-05-17T22:36:58.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.509+0000 7fc455ffb700 1 -- 172.21.15.12:0/633768906 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc444003fe0 con 0x7fc458107750 2024-05-17T22:36:58.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.509+0000 7fc45eb76700 1 -- 172.21.15.12:0/633768906 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc45819e930 con 0x7fc458107750 2024-05-17T22:36:58.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.509+0000 7fc455ffb700 1 -- 172.21.15.12:0/633768906 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc444022c30 con 0x7fc458107750 2024-05-17T22:36:58.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.509+0000 7fc45eb76700 1 -- 172.21.15.12:0/633768906 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc45819ee10 con 0x7fc458107750 2024-05-17T22:36:58.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.510+0000 7fc45eb76700 1 -- 172.21.15.12:0/633768906 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc45819f340 con 0x7fc458107750 2024-05-17T22:36:58.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.511+0000 7fc455ffb700 1 -- 172.21.15.12:0/633768906 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fc44402a070 con 0x7fc458107750 2024-05-17T22:36:58.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.512+0000 7fc455ffb700 1 --2- 172.21.15.12:0/633768906 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fc4480b2910 0x7fc4480b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:58.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.512+0000 7fc45c912700 1 --2- 172.21.15.12:0/633768906 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fc4480b2910 0x7fc4480b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:58.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.512+0000 7fc455ffb700 1 -- 172.21.15.12:0/633768906 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fc4440e4370 con 0x7fc458107750 2024-05-17T22:36:58.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.513+0000 7fc455ffb700 1 --2- 172.21.15.12:0/633768906 >> [v2:172.21.15.12:6810/3563309320,v1:172.21.15.12:6811/3563309320] conn(0x7fc4480bda90 0x7fc4480bff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:36:58.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.513+0000 7fc455ffb700 1 -- 172.21.15.12:0/633768906 --> [v2:172.21.15.12:6810/3563309320,v1:172.21.15.12:6811/3563309320] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc4480c0650 con 0x7fc4480bda90 2024-05-17T22:36:58.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.513+0000 7fc455ffb700 1 -- 172.21.15.12:0/633768906 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fc4440e4720 con 0x7fc458107750 2024-05-17T22:36:58.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.513+0000 7fc45d113700 1 --2- 172.21.15.12:0/633768906 >> [v2:172.21.15.12:6810/3563309320,v1:172.21.15.12:6811/3563309320] conn(0x7fc4480bda90 0x7fc4480bff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:36:58.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.513+0000 7fc45c912700 1 --2- 172.21.15.12:0/633768906 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fc4480b2910 0x7fc4480b4e20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc44c00ac50 tx=0x7fc44c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:36:58.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.513+0000 7fc45d113700 1 --2- 172.21.15.12:0/633768906 >> [v2:172.21.15.12:6810/3563309320,v1:172.21.15.12:6811/3563309320] conn(0x7fc4480bda90 0x7fc4480bff80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:36:58.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.514+0000 7fc455ffb700 1 -- 172.21.15.12:0/633768906 <== osd.3 v2:172.21.15.12:6810/3563309320 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc4480c0650 con 0x7fc4480bda90 2024-05-17T22:36:58.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.530+0000 7fc45eb76700 1 -- 172.21.15.12:0/633768906 --> [v2:172.21.15.12:6810/3563309320,v1:172.21.15.12:6811/3563309320] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc45819f740 con 0x7fc4480bda90 2024-05-17T22:36:58.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.531+0000 7fc455ffb700 1 -- 172.21.15.12:0/633768906 <== osd.3 v2:172.21.15.12:6810/3563309320 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc45819f740 con 0x7fc4480bda90 2024-05-17T22:36:58.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.531+0000 7fc45eb76700 1 -- 172.21.15.12:0/633768906 >> [v2:172.21.15.12:6810/3563309320,v1:172.21.15.12:6811/3563309320] conn(0x7fc4480bda90 msgr2=0x7fc4480bff80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:58.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.531+0000 7fc45eb76700 1 --2- 172.21.15.12:0/633768906 >> [v2:172.21.15.12:6810/3563309320,v1:172.21.15.12:6811/3563309320] conn(0x7fc4480bda90 0x7fc4480bff80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:36:58.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.531+0000 7fc45eb76700 1 -- 172.21.15.12:0/633768906 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fc4480b2910 msgr2=0x7fc4480b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:58.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.531+0000 7fc45eb76700 1 --2- 172.21.15.12:0/633768906 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fc4480b2910 0x7fc4480b4e20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc44c00ac50 tx=0x7fc44c00a380 comp rx=0 tx=0).stop 2024-05-17T22:36:58.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.531+0000 7fc45eb76700 1 -- 172.21.15.12:0/633768906 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc458107750 msgr2=0x7fc458199950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:36:58.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.531+0000 7fc45eb76700 1 --2- 172.21.15.12:0/633768906 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc458107750 0x7fc458199950 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc444000c00 tx=0x7fc444005ef0 comp rx=0 tx=0).stop 2024-05-17T22:36:58.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.531+0000 7fc45d113700 1 -- 172.21.15.12:0/633768906 reap_dead start 2024-05-17T22:36:58.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.531+0000 7fc45eb76700 1 -- 172.21.15.12:0/633768906 shutdown_connections 2024-05-17T22:36:58.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.531+0000 7fc45eb76700 1 -- 172.21.15.12:0/633768906 >> 172.21.15.12:0/633768906 conn(0x7fc458079bd0 msgr2=0x7fc4581053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:36:58.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.531+0000 7fc45eb76700 1 -- 172.21.15.12:0/633768906 shutdown_connections 2024-05-17T22:36:58.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:36:55.531+0000 7fc45eb76700 1 -- 172.21.15.12:0/633768906 wait complete. 2024-05-17T22:36:58.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:57 smithi151 ceph-mon[36101]: pgmap v370: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T22:36:58.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:57 smithi096 ceph-mon[36065]: pgmap v370: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T22:36:58.688 INFO:teuthology.orchestra.run.smithi012.stdout:107374182485 2024-05-17T22:36:58.689 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-17T22:36:59.544 INFO:teuthology.orchestra.run.smithi012.stdout:34359738468 2024-05-17T22:36:59.544 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-17T22:37:00.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.028+0000 7fa323de5700 1 -- 172.21.15.12:0/2984358474 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa31c107730 msgr2=0x7fa31c107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:00.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.028+0000 7fa323de5700 1 --2- 172.21.15.12:0/2984358474 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa31c107730 0x7fa31c107b40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa30c009990 tx=0x7fa30c009ca0 comp rx=0 tx=0).stop 2024-05-17T22:37:00.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.028+0000 7fa323de5700 1 -- 172.21.15.12:0/2984358474 shutdown_connections 2024-05-17T22:37:00.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.028+0000 7fa323de5700 1 --2- 172.21.15.12:0/2984358474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa31c108080 0x7fa31c10f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:00.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.028+0000 7fa323de5700 1 --2- 172.21.15.12:0/2984358474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa31c102e00 0x7fa31c103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:00.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.028+0000 7fa323de5700 1 --2- 172.21.15.12:0/2984358474 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa31c107730 0x7fa31c107b40 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:00.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.028+0000 7fa323de5700 1 -- 172.21.15.12:0/2984358474 >> 172.21.15.12:0/2984358474 conn(0x7fa31c079bd0 msgr2=0x7fa31c079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:00.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.029+0000 7fa323de5700 1 -- 172.21.15.12:0/2984358474 shutdown_connections 2024-05-17T22:37:00.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.029+0000 7fa323de5700 1 -- 172.21.15.12:0/2984358474 wait complete. 2024-05-17T22:37:00.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.029+0000 7fa323de5700 1 Processor -- start 2024-05-17T22:37:00.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.029+0000 7fa323de5700 1 -- start start 2024-05-17T22:37:00.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.030+0000 7fa323de5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa31c102e00 0x7fa31c1a2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:00.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.030+0000 7fa323de5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa31c107730 0x7fa31c1a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:00.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.030+0000 7fa323de5700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa31c108080 0x7fa31c19c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:00.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.030+0000 7fa323de5700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c112c90 con 0x7fa31c107730 2024-05-17T22:37:00.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.030+0000 7fa323de5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c112b10 con 0x7fa31c102e00 2024-05-17T22:37:00.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.030+0000 7fa323de5700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c112e10 con 0x7fa31c108080 2024-05-17T22:37:00.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.030+0000 7fa321380700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa31c107730 0x7fa31c1a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:00.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.030+0000 7fa321380700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa31c107730 0x7fa31c1a2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44120/0 (socket says 172.21.15.12:44120) 2024-05-17T22:37:00.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.030+0000 7fa321380700 1 -- 172.21.15.12:0/2678782091 learned_addr learned my addr 172.21.15.12:0/2678782091 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:37:00.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.030+0000 7fa321b81700 1 --2- 172.21.15.12:0/2678782091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa31c102e00 0x7fa31c1a2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:00.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.030+0000 7fa322382700 1 --2- 172.21.15.12:0/2678782091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa31c108080 0x7fa31c19c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:00.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.030+0000 7fa321380700 1 -- 172.21.15.12:0/2678782091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa31c108080 msgr2=0x7fa31c19c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:00.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.030+0000 7fa321380700 1 --2- 172.21.15.12:0/2678782091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa31c108080 0x7fa31c19c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:00.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.030+0000 7fa321380700 1 -- 172.21.15.12:0/2678782091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa31c102e00 msgr2=0x7fa31c1a2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:00.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.030+0000 7fa321380700 1 --2- 172.21.15.12:0/2678782091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa31c102e00 0x7fa31c1a2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:00.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.030+0000 7fa321380700 1 -- 172.21.15.12:0/2678782091 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa31c19cf60 con 0x7fa31c107730 2024-05-17T22:37:00.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.030+0000 7fa321b81700 1 --2- 172.21.15.12:0/2678782091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa31c102e00 0x7fa31c1a2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:37:00.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.031+0000 7fa321380700 1 --2- 172.21.15.12:0/2678782091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa31c107730 0x7fa31c1a2bd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa30c000c00 tx=0x7fa30c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:00.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.031+0000 7fa312ffd700 1 -- 172.21.15.12:0/2678782091 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c027070 con 0x7fa31c107730 2024-05-17T22:37:00.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.031+0000 7fa323de5700 1 -- 172.21.15.12:0/2678782091 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa31c19d1f0 con 0x7fa31c107730 2024-05-17T22:37:00.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.031+0000 7fa312ffd700 1 -- 172.21.15.12:0/2678782091 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa30c003f20 con 0x7fa31c107730 2024-05-17T22:37:00.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.031+0000 7fa312ffd700 1 -- 172.21.15.12:0/2678782091 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c022c70 con 0x7fa31c107730 2024-05-17T22:37:00.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.031+0000 7fa323de5700 1 -- 172.21.15.12:0/2678782091 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa31c1a9520 con 0x7fa31c107730 2024-05-17T22:37:00.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.032+0000 7fa323de5700 1 -- 172.21.15.12:0/2678782091 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa31c063f40 con 0x7fa31c107730 2024-05-17T22:37:00.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.034+0000 7fa312ffd700 1 -- 172.21.15.12:0/2678782091 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa30c02a070 con 0x7fa31c107730 2024-05-17T22:37:00.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.034+0000 7fa312ffd700 1 --2- 172.21.15.12:0/2678782091 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa3080b2910 0x7fa3080b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:00.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.035+0000 7fa321b81700 1 --2- 172.21.15.12:0/2678782091 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa3080b2910 0x7fa3080b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:00.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.035+0000 7fa312ffd700 1 -- 172.21.15.12:0/2678782091 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fa30c0e3fb0 con 0x7fa31c107730 2024-05-17T22:37:00.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.035+0000 7fa312ffd700 1 --2- 172.21.15.12:0/2678782091 >> [v2:172.21.15.151:6816/2545224628,v1:172.21.15.151:6817/2545224628] conn(0x7fa3080bdbc0 0x7fa3080c00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:00.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.035+0000 7fa312ffd700 1 -- 172.21.15.12:0/2678782091 --> [v2:172.21.15.151:6816/2545224628,v1:172.21.15.151:6817/2545224628] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa30c00b0b0 con 0x7fa3080bdbc0 2024-05-17T22:37:00.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.035+0000 7fa312ffd700 1 -- 172.21.15.12:0/2678782091 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fa30c0e4400 con 0x7fa31c107730 2024-05-17T22:37:00.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.035+0000 7fa321b81700 1 --2- 172.21.15.12:0/2678782091 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa3080b2910 0x7fa3080b4e20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa31c19d9f0 tx=0x7fa31800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:00.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.035+0000 7fa322382700 1 --2- 172.21.15.12:0/2678782091 >> [v2:172.21.15.151:6816/2545224628,v1:172.21.15.151:6817/2545224628] conn(0x7fa3080bdbc0 0x7fa3080c00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:00.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.036+0000 7fa322382700 1 --2- 172.21.15.12:0/2678782091 >> [v2:172.21.15.151:6816/2545224628,v1:172.21.15.151:6817/2545224628] conn(0x7fa3080bdbc0 0x7fa3080c00b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:00.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.037+0000 7fa312ffd700 1 -- 172.21.15.12:0/2678782091 <== osd.8 v2:172.21.15.151:6816/2545224628 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa30c00b0b0 con 0x7fa3080bdbc0 2024-05-17T22:37:00.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.053+0000 7fa323de5700 1 -- 172.21.15.12:0/2678782091 --> [v2:172.21.15.151:6816/2545224628,v1:172.21.15.151:6817/2545224628] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa31c064340 con 0x7fa3080bdbc0 2024-05-17T22:37:00.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.054+0000 7fa312ffd700 1 -- 172.21.15.12:0/2678782091 <== osd.8 v2:172.21.15.151:6816/2545224628 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa31c064340 con 0x7fa3080bdbc0 2024-05-17T22:37:00.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.054+0000 7fa323de5700 1 -- 172.21.15.12:0/2678782091 >> [v2:172.21.15.151:6816/2545224628,v1:172.21.15.151:6817/2545224628] conn(0x7fa3080bdbc0 msgr2=0x7fa3080c00b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:00.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.054+0000 7fa323de5700 1 --2- 172.21.15.12:0/2678782091 >> [v2:172.21.15.151:6816/2545224628,v1:172.21.15.151:6817/2545224628] conn(0x7fa3080bdbc0 0x7fa3080c00b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:00.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.054+0000 7fa323de5700 1 -- 172.21.15.12:0/2678782091 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa3080b2910 msgr2=0x7fa3080b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:00.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.054+0000 7fa323de5700 1 --2- 172.21.15.12:0/2678782091 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa3080b2910 0x7fa3080b4e20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa31c19d9f0 tx=0x7fa31800b410 comp rx=0 tx=0).stop 2024-05-17T22:37:00.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.054+0000 7fa323de5700 1 -- 172.21.15.12:0/2678782091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa31c107730 msgr2=0x7fa31c1a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:00.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.054+0000 7fa323de5700 1 --2- 172.21.15.12:0/2678782091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa31c107730 0x7fa31c1a2bd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa30c000c00 tx=0x7fa30c0047b0 comp rx=0 tx=0).stop 2024-05-17T22:37:00.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.054+0000 7fa322382700 1 -- 172.21.15.12:0/2678782091 reap_dead start 2024-05-17T22:37:00.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.054+0000 7fa323de5700 1 -- 172.21.15.12:0/2678782091 shutdown_connections 2024-05-17T22:37:00.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.054+0000 7fa323de5700 1 -- 172.21.15.12:0/2678782091 >> 172.21.15.12:0/2678782091 conn(0x7fa31c079bd0 msgr2=0x7fa31c10e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:00.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.054+0000 7fa323de5700 1 -- 172.21.15.12:0/2678782091 shutdown_connections 2024-05-17T22:37:00.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:00.054+0000 7fa323de5700 1 -- 172.21.15.12:0/2678782091 wait complete. 2024-05-17T22:37:00.101 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:36:59 smithi096 ceph-mon[36065]: pgmap v371: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:37:00.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:36:59 smithi012 ceph-mon[31632]: pgmap v371: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:37:00.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:36:59 smithi151 ceph-mon[36101]: pgmap v371: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:37:01.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.919+0000 7f2a1ca27700 1 -- 172.21.15.12:0/753979067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a1810e290 msgr2=0x7f2a18110740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:01.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.919+0000 7f2a1ca27700 1 --2- 172.21.15.12:0/753979067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a1810e290 0x7f2a18110740 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2a1000b0a0 tx=0x7f2a1000b3b0 comp rx=0 tx=0).stop 2024-05-17T22:37:01.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.920+0000 7f2a1ca27700 1 -- 172.21.15.12:0/753979067 shutdown_connections 2024-05-17T22:37:01.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.920+0000 7f2a1ca27700 1 --2- 172.21.15.12:0/753979067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a1810e290 0x7f2a18110740 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:01.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.920+0000 7f2a1ca27700 1 --2- 172.21.15.12:0/753979067 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a18104eb0 0x7f2a181052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:01.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.920+0000 7f2a1ca27700 1 --2- 172.21.15.12:0/753979067 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a18105800 0x7f2a1810dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:01.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.920+0000 7f2a1ca27700 1 -- 172.21.15.12:0/753979067 >> 172.21.15.12:0/753979067 conn(0x7f2a18079ae0 msgr2=0x7f2a18079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:01.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.920+0000 7f2a1ca27700 1 -- 172.21.15.12:0/753979067 shutdown_connections 2024-05-17T22:37:01.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.920+0000 7f2a1ca27700 1 -- 172.21.15.12:0/753979067 wait complete. 2024-05-17T22:37:01.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.921+0000 7f2a1ca27700 1 Processor -- start 2024-05-17T22:37:01.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.921+0000 7f2a1ca27700 1 -- start start 2024-05-17T22:37:01.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.921+0000 7f2a1ca27700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a18104eb0 0x7f2a1819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:01.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.921+0000 7f2a1ca27700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a18105800 0x7f2a1819acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:01.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.921+0000 7f2a1ca27700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a1810e290 0x7f2a1819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:01.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.921+0000 7f2a1ca27700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2a18113db0 con 0x7f2a1810e290 2024-05-17T22:37:01.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.921+0000 7f2a1ca27700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2a18113c30 con 0x7f2a18104eb0 2024-05-17T22:37:01.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.921+0000 7f2a1ca27700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2a18113f30 con 0x7f2a18105800 2024-05-17T22:37:01.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.921+0000 7f2a16d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a1810e290 0x7f2a1819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:01.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.921+0000 7f2a16d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a1810e290 0x7f2a1819ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44158/0 (socket says 172.21.15.12:44158) 2024-05-17T22:37:01.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.921+0000 7f2a16d9d700 1 -- 172.21.15.12:0/2434072900 learned_addr learned my addr 172.21.15.12:0/2434072900 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:37:01.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.921+0000 7f2a1659c700 1 --2- 172.21.15.12:0/2434072900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a18104eb0 0x7f2a1819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:01.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.921+0000 7f2a15d9b700 1 --2- 172.21.15.12:0/2434072900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a18105800 0x7f2a1819acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:01.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.921+0000 7f2a16d9d700 1 -- 172.21.15.12:0/2434072900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a18105800 msgr2=0x7f2a1819acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:01.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.921+0000 7f2a16d9d700 1 --2- 172.21.15.12:0/2434072900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a18105800 0x7f2a1819acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:01.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.921+0000 7f2a16d9d700 1 -- 172.21.15.12:0/2434072900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a18104eb0 msgr2=0x7f2a1819a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:01.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.921+0000 7f2a16d9d700 1 --2- 172.21.15.12:0/2434072900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a18104eb0 0x7f2a1819a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:01.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.921+0000 7f2a16d9d700 1 -- 172.21.15.12:0/2434072900 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a1819f550 con 0x7f2a1810e290 2024-05-17T22:37:01.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.922+0000 7f2a16d9d700 1 --2- 172.21.15.12:0/2434072900 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a1810e290 0x7f2a1819ef60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2a100062a0 tx=0x7f2a10004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:01.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.922+0000 7f2a077fe700 1 -- 172.21.15.12:0/2434072900 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a10021070 con 0x7f2a1810e290 2024-05-17T22:37:01.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.922+0000 7f2a077fe700 1 -- 172.21.15.12:0/2434072900 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a10007e30 con 0x7f2a1810e290 2024-05-17T22:37:01.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.922+0000 7f2a077fe700 1 -- 172.21.15.12:0/2434072900 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a10004780 con 0x7f2a1810e290 2024-05-17T22:37:01.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.922+0000 7f2a1ca27700 1 -- 172.21.15.12:0/2434072900 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a1819f7e0 con 0x7f2a1810e290 2024-05-17T22:37:01.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.922+0000 7f2a1ca27700 1 -- 172.21.15.12:0/2434072900 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a1819fcd0 con 0x7f2a1810e290 2024-05-17T22:37:01.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.923+0000 7f2a1ca27700 1 -- 172.21.15.12:0/2434072900 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2a18063fc0 con 0x7f2a1810e290 2024-05-17T22:37:01.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.925+0000 7f2a077fe700 1 -- 172.21.15.12:0/2434072900 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f2a10007980 con 0x7f2a1810e290 2024-05-17T22:37:01.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.925+0000 7f2a077fe700 1 --2- 172.21.15.12:0/2434072900 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f2a000b2910 0x7f2a000b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:01.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.926+0000 7f2a1659c700 1 --2- 172.21.15.12:0/2434072900 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f2a000b2910 0x7f2a000b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:01.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.926+0000 7f2a077fe700 1 -- 172.21.15.12:0/2434072900 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f2a1000f020 con 0x7f2a1810e290 2024-05-17T22:37:01.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.926+0000 7f2a077fe700 1 --2- 172.21.15.12:0/2434072900 >> [v2:172.21.15.12:6826/2100661273,v1:172.21.15.12:6827/2100661273] conn(0x7f2a000bdbc0 0x7f2a000c00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:01.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.926+0000 7f2a077fe700 1 -- 172.21.15.12:0/2434072900 --> [v2:172.21.15.12:6826/2100661273,v1:172.21.15.12:6827/2100661273] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2a000c0780 con 0x7f2a000bdbc0 2024-05-17T22:37:01.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.926+0000 7f2a077fe700 1 -- 172.21.15.12:0/2434072900 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f2a100e4250 con 0x7f2a1810e290 2024-05-17T22:37:01.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.926+0000 7f2a15d9b700 1 --2- 172.21.15.12:0/2434072900 >> [v2:172.21.15.12:6826/2100661273,v1:172.21.15.12:6827/2100661273] conn(0x7f2a000bdbc0 0x7f2a000c00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:01.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.926+0000 7f2a1659c700 1 --2- 172.21.15.12:0/2434072900 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f2a000b2910 0x7f2a000b4e20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2a0800ac50 tx=0x7f2a0800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:01.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.926+0000 7f2a15d9b700 1 --2- 172.21.15.12:0/2434072900 >> [v2:172.21.15.12:6826/2100661273,v1:172.21.15.12:6827/2100661273] conn(0x7f2a000bdbc0 0x7f2a000c00b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:01.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.927+0000 7f2a077fe700 1 -- 172.21.15.12:0/2434072900 <== osd.9 v2:172.21.15.12:6826/2100661273 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2a000c0780 con 0x7f2a000bdbc0 2024-05-17T22:37:01.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.943+0000 7f2a1ca27700 1 -- 172.21.15.12:0/2434072900 --> [v2:172.21.15.12:6826/2100661273,v1:172.21.15.12:6827/2100661273] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2a18064440 con 0x7f2a000bdbc0 2024-05-17T22:37:01.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.944+0000 7f2a077fe700 1 -- 172.21.15.12:0/2434072900 <== osd.9 v2:172.21.15.12:6826/2100661273 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2a18064440 con 0x7f2a000bdbc0 2024-05-17T22:37:01.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.944+0000 7f2a1ca27700 1 -- 172.21.15.12:0/2434072900 >> [v2:172.21.15.12:6826/2100661273,v1:172.21.15.12:6827/2100661273] conn(0x7f2a000bdbc0 msgr2=0x7f2a000c00b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:01.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.944+0000 7f2a1ca27700 1 --2- 172.21.15.12:0/2434072900 >> [v2:172.21.15.12:6826/2100661273,v1:172.21.15.12:6827/2100661273] conn(0x7f2a000bdbc0 0x7f2a000c00b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:01.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.944+0000 7f2a1ca27700 1 -- 172.21.15.12:0/2434072900 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f2a000b2910 msgr2=0x7f2a000b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:01.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.944+0000 7f2a1ca27700 1 --2- 172.21.15.12:0/2434072900 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f2a000b2910 0x7f2a000b4e20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2a0800ac50 tx=0x7f2a0800a380 comp rx=0 tx=0).stop 2024-05-17T22:37:01.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.944+0000 7f2a1ca27700 1 -- 172.21.15.12:0/2434072900 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a1810e290 msgr2=0x7f2a1819ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:01.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.944+0000 7f2a1ca27700 1 --2- 172.21.15.12:0/2434072900 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a1810e290 0x7f2a1819ef60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2a100062a0 tx=0x7f2a10004c80 comp rx=0 tx=0).stop 2024-05-17T22:37:01.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.944+0000 7f2a16d9d700 1 -- 172.21.15.12:0/2434072900 reap_dead start 2024-05-17T22:37:01.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.944+0000 7f2a1ca27700 1 -- 172.21.15.12:0/2434072900 shutdown_connections 2024-05-17T22:37:01.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.944+0000 7f2a1ca27700 1 -- 172.21.15.12:0/2434072900 >> 172.21.15.12:0/2434072900 conn(0x7f2a18079ae0 msgr2=0x7f2a180ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:01.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.944+0000 7f2a1ca27700 1 -- 172.21.15.12:0/2434072900 shutdown_connections 2024-05-17T22:37:01.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.945+0000 7f2a1ca27700 1 -- 172.21.15.12:0/2434072900 wait complete. 2024-05-17T22:37:02.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:01 smithi012 ceph-mon[31632]: pgmap v372: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T22:37:02.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:01 smithi151 ceph-mon[36101]: pgmap v372: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T22:37:02.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:01 smithi096 ceph-mon[36065]: pgmap v372: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T22:37:02.692 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:37:02.693 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:37:03.133 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:37:03.553 INFO:teuthology.orchestra.run.smithi012.stdout:227633266748 2024-05-17T22:37:03.554 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-17T22:37:04.076 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:03 smithi012 ceph-mon[31632]: pgmap v373: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-17T22:37:04.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:03 smithi151 ceph-mon[36101]: pgmap v373: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-17T22:37:04.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:03 smithi096 ceph-mon[36065]: pgmap v373: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-17T22:37:04.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.795+0000 7fa00549f700 1 -- 172.21.15.12:0/2298685477 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa000103550 msgr2=0x7fa000103960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:04.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.795+0000 7fa00549f700 1 --2- 172.21.15.12:0/2298685477 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa000103550 0x7fa000103960 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0009990 tx=0x7f9ff0009ca0 comp rx=0 tx=0).stop 2024-05-17T22:37:04.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.795+0000 7fa00549f700 1 -- 172.21.15.12:0/2298685477 shutdown_connections 2024-05-17T22:37:04.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.795+0000 7fa00549f700 1 --2- 172.21.15.12:0/2298685477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0001082b0 0x7fa00010fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:04.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.795+0000 7fa00549f700 1 --2- 172.21.15.12:0/2298685477 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa000103550 0x7fa000103960 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:04.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.795+0000 7fa00549f700 1 --2- 172.21.15.12:0/2298685477 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa000107960 0x7fa000107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:04.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.795+0000 7fa00549f700 1 -- 172.21.15.12:0/2298685477 >> 172.21.15.12:0/2298685477 conn(0x7fa000078040 msgr2=0x7fa0000784a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:04.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.795+0000 7fa00549f700 1 -- 172.21.15.12:0/2298685477 shutdown_connections 2024-05-17T22:37:04.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.795+0000 7fa00549f700 1 -- 172.21.15.12:0/2298685477 wait complete. 2024-05-17T22:37:04.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.796+0000 7fa00549f700 1 Processor -- start 2024-05-17T22:37:04.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.796+0000 7fa00549f700 1 -- start start 2024-05-17T22:37:04.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.796+0000 7fa00549f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa000103550 0x7fa0001a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:04.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.796+0000 7fa00549f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa000107960 0x7fa0001a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:04.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.796+0000 7fa00549f700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0001082b0 0x7fa00019a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:04.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.796+0000 7fa00549f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa000112ef0 con 0x7fa000107960 2024-05-17T22:37:04.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.796+0000 7fa00549f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa000112d70 con 0x7fa000103550 2024-05-17T22:37:04.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.796+0000 7fa00549f700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa000113070 con 0x7fa0001082b0 2024-05-17T22:37:04.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.796+0000 7f9ff7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa000107960 0x7fa0001a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:04.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.796+0000 7f9ff7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa000107960 0x7fa0001a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44166/0 (socket says 172.21.15.12:44166) 2024-05-17T22:37:04.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.796+0000 7f9ff7fff700 1 -- 172.21.15.12:0/2066776924 learned_addr learned my addr 172.21.15.12:0/2066776924 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:37:04.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.796+0000 7f9ffeffd700 1 --2- 172.21.15.12:0/2066776924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa000103550 0x7fa0001a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:04.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.797+0000 7f9fff7fe700 1 --2- 172.21.15.12:0/2066776924 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0001082b0 0x7fa00019a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:04.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.797+0000 7f9ff7fff700 1 -- 172.21.15.12:0/2066776924 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0001082b0 msgr2=0x7fa00019a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:04.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.797+0000 7f9ff7fff700 1 --2- 172.21.15.12:0/2066776924 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0001082b0 0x7fa00019a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:04.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.797+0000 7f9ff7fff700 1 -- 172.21.15.12:0/2066776924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa000103550 msgr2=0x7fa0001a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:04.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.797+0000 7f9ff7fff700 1 --2- 172.21.15.12:0/2066776924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa000103550 0x7fa0001a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:04.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.797+0000 7f9ff7fff700 1 -- 172.21.15.12:0/2066776924 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa00019aed0 con 0x7fa000107960 2024-05-17T22:37:04.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.797+0000 7f9ff7fff700 1 --2- 172.21.15.12:0/2066776924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa000107960 0x7fa0001a0bb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9fe800c9e0 tx=0x7f9fe800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:04.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.797+0000 7f9ffcff9700 1 -- 172.21.15.12:0/2066776924 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe8007aa0 con 0x7fa000107960 2024-05-17T22:37:04.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.797+0000 7f9ffcff9700 1 -- 172.21.15.12:0/2066776924 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fe8007c40 con 0x7fa000107960 2024-05-17T22:37:04.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.797+0000 7f9ffcff9700 1 -- 172.21.15.12:0/2066776924 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe8018660 con 0x7fa000107960 2024-05-17T22:37:04.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.797+0000 7fa00549f700 1 -- 172.21.15.12:0/2066776924 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa00019b1c0 con 0x7fa000107960 2024-05-17T22:37:04.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.797+0000 7fa00549f700 1 -- 172.21.15.12:0/2066776924 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0001a7500 con 0x7fa000107960 2024-05-17T22:37:04.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.798+0000 7fa00549f700 1 -- 172.21.15.12:0/2066776924 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa000063f80 con 0x7fa000107960 2024-05-17T22:37:04.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.800+0000 7f9ffcff9700 1 -- 172.21.15.12:0/2066776924 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f9fe8018800 con 0x7fa000107960 2024-05-17T22:37:04.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.800+0000 7f9ffcff9700 1 --2- 172.21.15.12:0/2066776924 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9fec0b2990 0x7f9fec0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:04.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.801+0000 7f9ffeffd700 1 --2- 172.21.15.12:0/2066776924 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9fec0b2990 0x7f9fec0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:04.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.801+0000 7f9ffcff9700 1 -- 172.21.15.12:0/2066776924 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f9fe80e2aa0 con 0x7fa000107960 2024-05-17T22:37:04.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.801+0000 7f9ffcff9700 1 --2- 172.21.15.12:0/2066776924 >> [v2:172.21.15.12:6818/3112661724,v1:172.21.15.12:6819/3112661724] conn(0x7f9fec0bdc40 0x7f9fec0c0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:04.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.801+0000 7f9ffcff9700 1 -- 172.21.15.12:0/2066776924 --> [v2:172.21.15.12:6818/3112661724,v1:172.21.15.12:6819/3112661724] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9fe801fe50 con 0x7f9fec0bdc40 2024-05-17T22:37:04.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.801+0000 7f9ffcff9700 1 -- 172.21.15.12:0/2066776924 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f9fe80e2ef0 con 0x7fa000107960 2024-05-17T22:37:04.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.801+0000 7f9fff7fe700 1 --2- 172.21.15.12:0/2066776924 >> [v2:172.21.15.12:6818/3112661724,v1:172.21.15.12:6819/3112661724] conn(0x7f9fec0bdc40 0x7f9fec0c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:04.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.801+0000 7f9ffeffd700 1 --2- 172.21.15.12:0/2066776924 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9fec0b2990 0x7f9fec0b4ea0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa00019b9d0 tx=0x7f9ff0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:04.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.801+0000 7f9fff7fe700 1 --2- 172.21.15.12:0/2066776924 >> [v2:172.21.15.12:6818/3112661724,v1:172.21.15.12:6819/3112661724] conn(0x7f9fec0bdc40 0x7f9fec0c0130 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:04.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.802+0000 7f9ffcff9700 1 -- 172.21.15.12:0/2066776924 <== osd.6 v2:172.21.15.12:6818/3112661724 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9fe801fe50 con 0x7f9fec0bdc40 2024-05-17T22:37:04.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.820+0000 7fa00549f700 1 -- 172.21.15.12:0/2066776924 --> [v2:172.21.15.12:6818/3112661724,v1:172.21.15.12:6819/3112661724] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa0000643b0 con 0x7f9fec0bdc40 2024-05-17T22:37:04.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.820+0000 7f9ffcff9700 1 -- 172.21.15.12:0/2066776924 <== osd.6 v2:172.21.15.12:6818/3112661724 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa0000643b0 con 0x7f9fec0bdc40 2024-05-17T22:37:04.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.821+0000 7fa00549f700 1 -- 172.21.15.12:0/2066776924 >> [v2:172.21.15.12:6818/3112661724,v1:172.21.15.12:6819/3112661724] conn(0x7f9fec0bdc40 msgr2=0x7f9fec0c0130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:04.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.821+0000 7fa00549f700 1 --2- 172.21.15.12:0/2066776924 >> [v2:172.21.15.12:6818/3112661724,v1:172.21.15.12:6819/3112661724] conn(0x7f9fec0bdc40 0x7f9fec0c0130 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:04.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.821+0000 7fa00549f700 1 -- 172.21.15.12:0/2066776924 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9fec0b2990 msgr2=0x7f9fec0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:04.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.821+0000 7fa00549f700 1 --2- 172.21.15.12:0/2066776924 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9fec0b2990 0x7f9fec0b4ea0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa00019b9d0 tx=0x7f9ff0005960 comp rx=0 tx=0).stop 2024-05-17T22:37:04.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.821+0000 7fa00549f700 1 -- 172.21.15.12:0/2066776924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa000107960 msgr2=0x7fa0001a0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:04.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.821+0000 7fa00549f700 1 --2- 172.21.15.12:0/2066776924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa000107960 0x7fa0001a0bb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9fe800c9e0 tx=0x7f9fe800ccf0 comp rx=0 tx=0).stop 2024-05-17T22:37:04.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.821+0000 7f9fff7fe700 1 -- 172.21.15.12:0/2066776924 reap_dead start 2024-05-17T22:37:04.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.821+0000 7fa00549f700 1 -- 172.21.15.12:0/2066776924 shutdown_connections 2024-05-17T22:37:04.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.821+0000 7fa00549f700 1 -- 172.21.15.12:0/2066776924 >> 172.21.15.12:0/2066776924 conn(0x7fa000078040 msgr2=0x7fa000106770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:04.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.821+0000 7fa00549f700 1 -- 172.21.15.12:0/2066776924 shutdown_connections 2024-05-17T22:37:04.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:04.821+0000 7fa00549f700 1 -- 172.21.15.12:0/2066776924 wait complete. 2024-05-17T22:37:05.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.825+0000 7f9d820b5700 1 -- 172.21.15.12:0/53312487 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d7c1056f0 msgr2=0x7f9d7c10dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:05.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.825+0000 7f9d820b5700 1 --2- 172.21.15.12:0/53312487 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d7c1056f0 0x7f9d7c10dc40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9d64009990 tx=0x7f9d64009ca0 comp rx=0 tx=0).stop 2024-05-17T22:37:05.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.825+0000 7f9d820b5700 1 -- 172.21.15.12:0/53312487 shutdown_connections 2024-05-17T22:37:05.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.825+0000 7f9d820b5700 1 --2- 172.21.15.12:0/53312487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d7c10e200 0x7f9d7c110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:05.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.825+0000 7f9d820b5700 1 --2- 172.21.15.12:0/53312487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d7c104da0 0x7f9d7c1051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:05.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.825+0000 7f9d820b5700 1 --2- 172.21.15.12:0/53312487 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d7c1056f0 0x7f9d7c10dc40 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:05.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.825+0000 7f9d820b5700 1 -- 172.21.15.12:0/53312487 >> 172.21.15.12:0/53312487 conn(0x7f9d7c079bd0 msgr2=0x7f9d7c079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:05.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.825+0000 7f9d820b5700 1 -- 172.21.15.12:0/53312487 shutdown_connections 2024-05-17T22:37:05.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.825+0000 7f9d820b5700 1 -- 172.21.15.12:0/53312487 wait complete. 2024-05-17T22:37:05.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.825+0000 7f9d820b5700 1 Processor -- start 2024-05-17T22:37:05.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.826+0000 7f9d820b5700 1 -- start start 2024-05-17T22:37:05.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.826+0000 7f9d820b5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d7c104da0 0x7f9d7c102a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:05.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.826+0000 7f9d820b5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d7c1056f0 0x7f9d7c102fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:05.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.826+0000 7f9d820b5700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d7c10e200 0x7f9d7c103510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:05.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.826+0000 7f9d820b5700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7c113dd0 con 0x7f9d7c104da0 2024-05-17T22:37:05.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.826+0000 7f9d820b5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7c113c50 con 0x7f9d7c1056f0 2024-05-17T22:37:05.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.826+0000 7f9d820b5700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7c113f50 con 0x7f9d7c10e200 2024-05-17T22:37:05.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.826+0000 7f9d7b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d7c104da0 0x7f9d7c102a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:05.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.826+0000 7f9d7b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d7c104da0 0x7f9d7c102a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44140/0 (socket says 172.21.15.12:44140) 2024-05-17T22:37:05.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.826+0000 7f9d7b7fe700 1 -- 172.21.15.12:0/2493332034 learned_addr learned my addr 172.21.15.12:0/2493332034 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:37:05.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.826+0000 7f9d73fff700 1 --2- 172.21.15.12:0/2493332034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d7c1056f0 0x7f9d7c102fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:05.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.826+0000 7f9d7bfff700 1 --2- 172.21.15.12:0/2493332034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d7c10e200 0x7f9d7c103510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:05.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.826+0000 7f9d7b7fe700 1 -- 172.21.15.12:0/2493332034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d7c10e200 msgr2=0x7f9d7c103510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:05.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.826+0000 7f9d7b7fe700 1 --2- 172.21.15.12:0/2493332034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d7c10e200 0x7f9d7c103510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:05.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.826+0000 7f9d7b7fe700 1 -- 172.21.15.12:0/2493332034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d7c1056f0 msgr2=0x7f9d7c102fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:05.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.826+0000 7f9d7b7fe700 1 --2- 172.21.15.12:0/2493332034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d7c1056f0 0x7f9d7c102fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:05.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.826+0000 7f9d7b7fe700 1 -- 172.21.15.12:0/2493332034 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d7c1a8b50 con 0x7f9d7c104da0 2024-05-17T22:37:05.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.827+0000 7f9d7b7fe700 1 --2- 172.21.15.12:0/2493332034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d7c104da0 0x7f9d7c102a90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c00d950 tx=0x7f9d6c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:05.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.827+0000 7f9d797fa700 1 -- 172.21.15.12:0/2493332034 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c016070 con 0x7f9d7c104da0 2024-05-17T22:37:05.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.827+0000 7f9d797fa700 1 -- 172.21.15.12:0/2493332034 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d6c00fd40 con 0x7f9d7c104da0 2024-05-17T22:37:05.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.827+0000 7f9d820b5700 1 -- 172.21.15.12:0/2493332034 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d7c1a8e40 con 0x7f9d7c104da0 2024-05-17T22:37:05.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.827+0000 7f9d797fa700 1 -- 172.21.15.12:0/2493332034 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c00e3b0 con 0x7f9d7c104da0 2024-05-17T22:37:05.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.827+0000 7f9d820b5700 1 -- 172.21.15.12:0/2493332034 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d7c1a92f0 con 0x7f9d7c104da0 2024-05-17T22:37:05.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.828+0000 7f9d820b5700 1 -- 172.21.15.12:0/2493332034 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9d7c063f40 con 0x7f9d7c104da0 2024-05-17T22:37:05.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.830+0000 7f9d797fa700 1 -- 172.21.15.12:0/2493332034 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f9d6c019070 con 0x7f9d7c104da0 2024-05-17T22:37:05.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.830+0000 7f9d797fa700 1 --2- 172.21.15.12:0/2493332034 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9d680b2990 0x7f9d680b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:05.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.831+0000 7f9d73fff700 1 --2- 172.21.15.12:0/2493332034 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9d680b2990 0x7f9d680b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:05.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.831+0000 7f9d797fa700 1 -- 172.21.15.12:0/2493332034 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f9d6c0e26c0 con 0x7f9d7c104da0 2024-05-17T22:37:05.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.831+0000 7f9d797fa700 1 --2- 172.21.15.12:0/2493332034 >> [v2:172.21.15.96:6824/304016309,v1:172.21.15.96:6825/304016309] conn(0x7f9d680bdc40 0x7f9d680c0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:05.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.831+0000 7f9d797fa700 1 -- 172.21.15.12:0/2493332034 --> [v2:172.21.15.96:6824/304016309,v1:172.21.15.96:6825/304016309] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9d680c0800 con 0x7f9d680bdc40 2024-05-17T22:37:05.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.831+0000 7f9d797fa700 1 -- 172.21.15.12:0/2493332034 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f9d6c0e2b10 con 0x7f9d7c104da0 2024-05-17T22:37:05.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.831+0000 7f9d73fff700 1 --2- 172.21.15.12:0/2493332034 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9d680b2990 0x7f9d680b4ea0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c10fef0 tx=0x7f9d64005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:05.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.831+0000 7f9d7bfff700 1 --2- 172.21.15.12:0/2493332034 >> [v2:172.21.15.96:6824/304016309,v1:172.21.15.96:6825/304016309] conn(0x7f9d680bdc40 0x7f9d680c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:05.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.832+0000 7f9d7bfff700 1 --2- 172.21.15.12:0/2493332034 >> [v2:172.21.15.96:6824/304016309,v1:172.21.15.96:6825/304016309] conn(0x7f9d680bdc40 0x7f9d680c0130 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:05.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.833+0000 7f9d797fa700 1 -- 172.21.15.12:0/2493332034 <== osd.10 v2:172.21.15.96:6824/304016309 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9d680c0800 con 0x7f9d680bdc40 2024-05-17T22:37:05.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.850+0000 7f9d820b5700 1 -- 172.21.15.12:0/2493332034 --> [v2:172.21.15.96:6824/304016309,v1:172.21.15.96:6825/304016309] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9d7c064340 con 0x7f9d680bdc40 2024-05-17T22:37:05.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.850+0000 7f9d797fa700 1 -- 172.21.15.12:0/2493332034 <== osd.10 v2:172.21.15.96:6824/304016309 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9d7c064340 con 0x7f9d680bdc40 2024-05-17T22:37:05.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.851+0000 7f9d820b5700 1 -- 172.21.15.12:0/2493332034 >> [v2:172.21.15.96:6824/304016309,v1:172.21.15.96:6825/304016309] conn(0x7f9d680bdc40 msgr2=0x7f9d680c0130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:05.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.851+0000 7f9d820b5700 1 --2- 172.21.15.12:0/2493332034 >> [v2:172.21.15.96:6824/304016309,v1:172.21.15.96:6825/304016309] conn(0x7f9d680bdc40 0x7f9d680c0130 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:05.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.851+0000 7f9d820b5700 1 -- 172.21.15.12:0/2493332034 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9d680b2990 msgr2=0x7f9d680b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:05.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.851+0000 7f9d820b5700 1 --2- 172.21.15.12:0/2493332034 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9d680b2990 0x7f9d680b4ea0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c10fef0 tx=0x7f9d64005fd0 comp rx=0 tx=0).stop 2024-05-17T22:37:05.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.851+0000 7f9d820b5700 1 -- 172.21.15.12:0/2493332034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d7c104da0 msgr2=0x7f9d7c102a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:05.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.851+0000 7f9d820b5700 1 --2- 172.21.15.12:0/2493332034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d7c104da0 0x7f9d7c102a90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c00d950 tx=0x7f9d6c00dc60 comp rx=0 tx=0).stop 2024-05-17T22:37:05.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.851+0000 7f9d7bfff700 1 -- 172.21.15.12:0/2493332034 reap_dead start 2024-05-17T22:37:05.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.851+0000 7f9d820b5700 1 -- 172.21.15.12:0/2493332034 shutdown_connections 2024-05-17T22:37:05.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.851+0000 7f9d820b5700 1 -- 172.21.15.12:0/2493332034 >> 172.21.15.12:0/2493332034 conn(0x7f9d7c079bd0 msgr2=0x7f9d7c0ffb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:05.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.851+0000 7f9d820b5700 1 -- 172.21.15.12:0/2493332034 shutdown_connections 2024-05-17T22:37:05.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:01.851+0000 7f9d820b5700 1 -- 172.21.15.12:0/2493332034 wait complete. 2024-05-17T22:37:06.049 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:05 smithi012 ceph-mon[31632]: pgmap v374: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-17T22:37:06.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.174+0000 7f9b13ec2700 1 -- 172.21.15.12:0/3466414657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b0c102e20 msgr2=0x7f9b0c103230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:06.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.174+0000 7f9b13ec2700 1 --2- 172.21.15.12:0/3466414657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b0c102e20 0x7f9b0c103230 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9b08009990 tx=0x7f9b08009ca0 comp rx=0 tx=0).stop 2024-05-17T22:37:06.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.174+0000 7f9b13ec2700 1 -- 172.21.15.12:0/3466414657 shutdown_connections 2024-05-17T22:37:06.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.174+0000 7f9b13ec2700 1 --2- 172.21.15.12:0/3466414657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b0c1080a0 0x7f9b0c10fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:06.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.174+0000 7f9b13ec2700 1 --2- 172.21.15.12:0/3466414657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b0c102e20 0x7f9b0c103230 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:06.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.174+0000 7f9b13ec2700 1 --2- 172.21.15.12:0/3466414657 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b0c107750 0x7f9b0c107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:06.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.174+0000 7f9b13ec2700 1 -- 172.21.15.12:0/3466414657 >> 172.21.15.12:0/3466414657 conn(0x7f9b0c079af0 msgr2=0x7f9b0c079f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:06.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.174+0000 7f9b13ec2700 1 -- 172.21.15.12:0/3466414657 shutdown_connections 2024-05-17T22:37:06.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.174+0000 7f9b13ec2700 1 -- 172.21.15.12:0/3466414657 wait complete. 2024-05-17T22:37:06.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.175+0000 7f9b13ec2700 1 Processor -- start 2024-05-17T22:37:06.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.175+0000 7f9b13ec2700 1 -- start start 2024-05-17T22:37:06.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.175+0000 7f9b13ec2700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b0c102e20 0x7f9b0c19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:06.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.175+0000 7f9b13ec2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b0c107750 0x7f9b0c19ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:06.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.175+0000 7f9b13ec2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b0c1080a0 0x7f9b0c19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:06.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.175+0000 7f9b13ec2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0c112cd0 con 0x7f9b0c107750 2024-05-17T22:37:06.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.175+0000 7f9b13ec2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0c112b50 con 0x7f9b0c1080a0 2024-05-17T22:37:06.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.175+0000 7f9b13ec2700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0c112e50 con 0x7f9b0c102e20 2024-05-17T22:37:06.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.175+0000 7f9b1145d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b0c107750 0x7f9b0c19ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:06.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.175+0000 7f9b1145d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b0c107750 0x7f9b0c19ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44186/0 (socket says 172.21.15.12:44186) 2024-05-17T22:37:06.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.175+0000 7f9b1145d700 1 -- 172.21.15.12:0/1019696827 learned_addr learned my addr 172.21.15.12:0/1019696827 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:37:06.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.175+0000 7f9b1245f700 1 --2- 172.21.15.12:0/1019696827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b0c1080a0 0x7f9b0c19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:06.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.175+0000 7f9b11c5e700 1 --2- 172.21.15.12:0/1019696827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b0c102e20 0x7f9b0c19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:06.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.175+0000 7f9b1145d700 1 -- 172.21.15.12:0/1019696827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b0c102e20 msgr2=0x7f9b0c19a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:06.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.175+0000 7f9b1145d700 1 --2- 172.21.15.12:0/1019696827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b0c102e20 0x7f9b0c19a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:06.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.175+0000 7f9b1145d700 1 -- 172.21.15.12:0/1019696827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b0c1080a0 msgr2=0x7f9b0c19efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:06.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.175+0000 7f9b1145d700 1 --2- 172.21.15.12:0/1019696827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b0c1080a0 0x7f9b0c19efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:06.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.175+0000 7f9b1145d700 1 -- 172.21.15.12:0/1019696827 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b0c19f650 con 0x7f9b0c107750 2024-05-17T22:37:06.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.175+0000 7f9b1245f700 1 --2- 172.21.15.12:0/1019696827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b0c1080a0 0x7f9b0c19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:37:06.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.176+0000 7f9b1145d700 1 --2- 172.21.15.12:0/1019696827 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b0c107750 0x7f9b0c19ad20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9afc00d980 tx=0x7f9afc00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:06.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.176+0000 7f9b02ffd700 1 -- 172.21.15.12:0/1019696827 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9afc017070 con 0x7f9b0c107750 2024-05-17T22:37:06.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.176+0000 7f9b02ffd700 1 -- 172.21.15.12:0/1019696827 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9afc010460 con 0x7f9b0c107750 2024-05-17T22:37:06.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.176+0000 7f9b02ffd700 1 -- 172.21.15.12:0/1019696827 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9afc00de60 con 0x7f9b0c107750 2024-05-17T22:37:06.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.176+0000 7f9b13ec2700 1 -- 172.21.15.12:0/1019696827 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b0c19f940 con 0x7f9b0c107750 2024-05-17T22:37:06.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.176+0000 7f9b13ec2700 1 -- 172.21.15.12:0/1019696827 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b0c1a7260 con 0x7f9b0c107750 2024-05-17T22:37:06.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.177+0000 7f9b13ec2700 1 -- 172.21.15.12:0/1019696827 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9b0c063fc0 con 0x7f9b0c107750 2024-05-17T22:37:06.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.178+0000 7f9b02ffd700 1 -- 172.21.15.12:0/1019696827 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f9afc010610 con 0x7f9b0c107750 2024-05-17T22:37:06.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.179+0000 7f9b02ffd700 1 --2- 172.21.15.12:0/1019696827 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9af80b2990 0x7f9af80b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:06.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.179+0000 7f9b11c5e700 1 --2- 172.21.15.12:0/1019696827 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9af80b2990 0x7f9af80b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:06.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.179+0000 7f9b02ffd700 1 -- 172.21.15.12:0/1019696827 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f9afc0e26a0 con 0x7f9b0c107750 2024-05-17T22:37:06.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.179+0000 7f9b02ffd700 1 --2- 172.21.15.12:0/1019696827 >> [v2:172.21.15.151:6808/2589835078,v1:172.21.15.151:6809/2589835078] conn(0x7f9af80bdc40 0x7f9af80c0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:06.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.180+0000 7f9b02ffd700 1 -- 172.21.15.12:0/1019696827 --> [v2:172.21.15.151:6808/2589835078,v1:172.21.15.151:6809/2589835078] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9afc011e50 con 0x7f9af80bdc40 2024-05-17T22:37:06.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.180+0000 7f9b02ffd700 1 -- 172.21.15.12:0/1019696827 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f9afc0e2af0 con 0x7f9b0c107750 2024-05-17T22:37:06.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.180+0000 7f9b11c5e700 1 --2- 172.21.15.12:0/1019696827 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9af80b2990 0x7f9af80b4ea0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9b08003800 tx=0x7f9b08005ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:06.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.180+0000 7f9b1245f700 1 --2- 172.21.15.12:0/1019696827 >> [v2:172.21.15.151:6808/2589835078,v1:172.21.15.151:6809/2589835078] conn(0x7f9af80bdc40 0x7f9af80c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:06.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.180+0000 7f9b1245f700 1 --2- 172.21.15.12:0/1019696827 >> [v2:172.21.15.151:6808/2589835078,v1:172.21.15.151:6809/2589835078] conn(0x7f9af80bdc40 0x7f9af80c0130 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:06.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.181+0000 7f9b02ffd700 1 -- 172.21.15.12:0/1019696827 <== osd.5 v2:172.21.15.151:6808/2589835078 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9afc011e50 con 0x7f9af80bdc40 2024-05-17T22:37:06.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.198+0000 7f9b13ec2700 1 -- 172.21.15.12:0/1019696827 --> [v2:172.21.15.151:6808/2589835078,v1:172.21.15.151:6809/2589835078] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9b0c0643c0 con 0x7f9af80bdc40 2024-05-17T22:37:06.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.199+0000 7f9b02ffd700 1 -- 172.21.15.12:0/1019696827 <== osd.5 v2:172.21.15.151:6808/2589835078 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9b0c0643c0 con 0x7f9af80bdc40 2024-05-17T22:37:06.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.199+0000 7f9b13ec2700 1 -- 172.21.15.12:0/1019696827 >> [v2:172.21.15.151:6808/2589835078,v1:172.21.15.151:6809/2589835078] conn(0x7f9af80bdc40 msgr2=0x7f9af80c0130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:06.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.199+0000 7f9b13ec2700 1 --2- 172.21.15.12:0/1019696827 >> [v2:172.21.15.151:6808/2589835078,v1:172.21.15.151:6809/2589835078] conn(0x7f9af80bdc40 0x7f9af80c0130 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:06.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.199+0000 7f9b13ec2700 1 -- 172.21.15.12:0/1019696827 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9af80b2990 msgr2=0x7f9af80b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:06.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.199+0000 7f9b13ec2700 1 --2- 172.21.15.12:0/1019696827 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9af80b2990 0x7f9af80b4ea0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9b08003800 tx=0x7f9b08005ec0 comp rx=0 tx=0).stop 2024-05-17T22:37:06.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.199+0000 7f9b13ec2700 1 -- 172.21.15.12:0/1019696827 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b0c107750 msgr2=0x7f9b0c19ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:06.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.199+0000 7f9b13ec2700 1 --2- 172.21.15.12:0/1019696827 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b0c107750 0x7f9b0c19ad20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9afc00d980 tx=0x7f9afc00dd40 comp rx=0 tx=0).stop 2024-05-17T22:37:06.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.199+0000 7f9b1245f700 1 -- 172.21.15.12:0/1019696827 reap_dead start 2024-05-17T22:37:06.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.199+0000 7f9b13ec2700 1 -- 172.21.15.12:0/1019696827 shutdown_connections 2024-05-17T22:37:06.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.199+0000 7f9b13ec2700 1 -- 172.21.15.12:0/1019696827 >> 172.21.15.12:0/1019696827 conn(0x7f9b0c079af0 msgr2=0x7f9b0c106620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:06.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.199+0000 7f9b13ec2700 1 -- 172.21.15.12:0/1019696827 shutdown_connections 2024-05-17T22:37:06.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:06.199+0000 7f9b13ec2700 1 -- 172.21.15.12:0/1019696827 wait complete. 2024-05-17T22:37:06.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:05 smithi151 ceph-mon[36101]: pgmap v374: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-17T22:37:06.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:05 smithi096 ceph-mon[36065]: pgmap v374: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-17T22:37:08.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:07 smithi012 ceph-mon[31632]: pgmap v375: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-17T22:37:08.122 INFO:teuthology.orchestra.run.smithi012.stdout:253403070517 2024-05-17T22:37:08.123 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-17T22:37:08.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:07 smithi151 ceph-mon[36101]: pgmap v375: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-17T22:37:08.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:07 smithi096 ceph-mon[36065]: pgmap v375: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-17T22:37:08.525 INFO:teuthology.orchestra.run.smithi012.stdout:274877906994 2024-05-17T22:37:08.525 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-17T22:37:09.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.013+0000 7fb05d4f3700 1 -- 172.21.15.12:0/40936762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0581080a0 msgr2=0x7fb05810fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:09.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.013+0000 7fb05d4f3700 1 --2- 172.21.15.12:0/40936762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0581080a0 0x7fb05810fa00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb05000b0a0 tx=0x7fb05000b3b0 comp rx=0 tx=0).stop 2024-05-17T22:37:09.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.013+0000 7fb05d4f3700 1 -- 172.21.15.12:0/40936762 shutdown_connections 2024-05-17T22:37:09.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.013+0000 7fb05d4f3700 1 --2- 172.21.15.12:0/40936762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0581080a0 0x7fb05810fa00 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:09.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.013+0000 7fb05d4f3700 1 --2- 172.21.15.12:0/40936762 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb058102e70 0x7fb058103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:09.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.013+0000 7fb05d4f3700 1 --2- 172.21.15.12:0/40936762 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb058107750 0x7fb058107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:09.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.013+0000 7fb05d4f3700 1 -- 172.21.15.12:0/40936762 >> 172.21.15.12:0/40936762 conn(0x7fb058079ae0 msgr2=0x7fb058079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:09.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.013+0000 7fb05d4f3700 1 -- 172.21.15.12:0/40936762 shutdown_connections 2024-05-17T22:37:09.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.014+0000 7fb05d4f3700 1 -- 172.21.15.12:0/40936762 wait complete. 2024-05-17T22:37:09.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.014+0000 7fb05d4f3700 1 Processor -- start 2024-05-17T22:37:09.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.015+0000 7fb05d4f3700 1 -- start start 2024-05-17T22:37:09.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.015+0000 7fb05d4f3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb058102e70 0x7fb05819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:09.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.015+0000 7fb05d4f3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb058107750 0x7fb05819acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:09.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.015+0000 7fb05d4f3700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0581080a0 0x7fb05819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:09.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.015+0000 7fb05d4f3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb058112ce0 con 0x7fb058102e70 2024-05-17T22:37:09.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.015+0000 7fb05d4f3700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb058112b60 con 0x7fb058107750 2024-05-17T22:37:09.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.015+0000 7fb05d4f3700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb058112e60 con 0x7fb0581080a0 2024-05-17T22:37:09.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.015+0000 7fb056ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb058102e70 0x7fb05819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:09.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.015+0000 7fb056ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb058102e70 0x7fb05819a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44202/0 (socket says 172.21.15.12:44202) 2024-05-17T22:37:09.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.015+0000 7fb056ffd700 1 -- 172.21.15.12:0/1566419143 learned_addr learned my addr 172.21.15.12:0/1566419143 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:37:09.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.015+0000 7fb0567fc700 1 --2- 172.21.15.12:0/1566419143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb058107750 0x7fb05819acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:09.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.015+0000 7fb0577fe700 1 --2- 172.21.15.12:0/1566419143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0581080a0 0x7fb05819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:09.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.015+0000 7fb056ffd700 1 -- 172.21.15.12:0/1566419143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0581080a0 msgr2=0x7fb05819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:09.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.015+0000 7fb056ffd700 1 --2- 172.21.15.12:0/1566419143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0581080a0 0x7fb05819ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:09.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.015+0000 7fb056ffd700 1 -- 172.21.15.12:0/1566419143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb058107750 msgr2=0x7fb05819acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:09.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.015+0000 7fb056ffd700 1 --2- 172.21.15.12:0/1566419143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb058107750 0x7fb05819acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:09.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.015+0000 7fb056ffd700 1 -- 172.21.15.12:0/1566419143 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb05819f600 con 0x7fb058102e70 2024-05-17T22:37:09.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.016+0000 7fb056ffd700 1 --2- 172.21.15.12:0/1566419143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb058102e70 0x7fb05819a730 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb04800d980 tx=0x7fb04800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:09.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.016+0000 7fb04ffff700 1 -- 172.21.15.12:0/1566419143 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb048016070 con 0x7fb058102e70 2024-05-17T22:37:09.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.016+0000 7fb05d4f3700 1 -- 172.21.15.12:0/1566419143 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb05819f8f0 con 0x7fb058102e70 2024-05-17T22:37:09.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.016+0000 7fb04ffff700 1 -- 172.21.15.12:0/1566419143 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb048004830 con 0x7fb058102e70 2024-05-17T22:37:09.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.016+0000 7fb04ffff700 1 -- 172.21.15.12:0/1566419143 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0480054c0 con 0x7fb058102e70 2024-05-17T22:37:09.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.016+0000 7fb05d4f3700 1 -- 172.21.15.12:0/1566419143 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0581a7210 con 0x7fb058102e70 2024-05-17T22:37:09.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.017+0000 7fb05d4f3700 1 -- 172.21.15.12:0/1566419143 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb05806c390 con 0x7fb058102e70 2024-05-17T22:37:09.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.019+0000 7fb04ffff700 1 -- 172.21.15.12:0/1566419143 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fb048004220 con 0x7fb058102e70 2024-05-17T22:37:09.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.019+0000 7fb04ffff700 1 --2- 172.21.15.12:0/1566419143 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb0440b2910 0x7fb0440b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:09.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.019+0000 7fb0567fc700 1 --2- 172.21.15.12:0/1566419143 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb0440b2910 0x7fb0440b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:09.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.019+0000 7fb04ffff700 1 -- 172.21.15.12:0/1566419143 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fb0480e2bb0 con 0x7fb058102e70 2024-05-17T22:37:09.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.020+0000 7fb04ffff700 1 --2- 172.21.15.12:0/1566419143 >> [v2:172.21.15.96:6816/2449989110,v1:172.21.15.96:6817/2449989110] conn(0x7fb0440bdbc0 0x7fb0440c00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:09.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.020+0000 7fb04ffff700 1 -- 172.21.15.12:0/1566419143 --> [v2:172.21.15.96:6816/2449989110,v1:172.21.15.96:6817/2449989110] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb048027ea0 con 0x7fb0440bdbc0 2024-05-17T22:37:09.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.020+0000 7fb04ffff700 1 -- 172.21.15.12:0/1566419143 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fb0480e3000 con 0x7fb058102e70 2024-05-17T22:37:09.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.020+0000 7fb0567fc700 1 --2- 172.21.15.12:0/1566419143 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb0440b2910 0x7fb0440b4e20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb058108000 tx=0x7fb04000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:09.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.020+0000 7fb0577fe700 1 --2- 172.21.15.12:0/1566419143 >> [v2:172.21.15.96:6816/2449989110,v1:172.21.15.96:6817/2449989110] conn(0x7fb0440bdbc0 0x7fb0440c00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:09.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.020+0000 7fb0577fe700 1 --2- 172.21.15.12:0/1566419143 >> [v2:172.21.15.96:6816/2449989110,v1:172.21.15.96:6817/2449989110] conn(0x7fb0440bdbc0 0x7fb0440c00b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:09.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.021+0000 7fb04ffff700 1 -- 172.21.15.12:0/1566419143 <== osd.7 v2:172.21.15.96:6816/2449989110 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb048027ea0 con 0x7fb0440bdbc0 2024-05-17T22:37:09.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.038+0000 7fb05d4f3700 1 -- 172.21.15.12:0/1566419143 --> [v2:172.21.15.96:6816/2449989110,v1:172.21.15.96:6817/2449989110] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb0581042d0 con 0x7fb0440bdbc0 2024-05-17T22:37:09.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.038+0000 7fb04ffff700 1 -- 172.21.15.12:0/1566419143 <== osd.7 v2:172.21.15.96:6816/2449989110 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb0581042d0 con 0x7fb0440bdbc0 2024-05-17T22:37:09.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.039+0000 7fb05d4f3700 1 -- 172.21.15.12:0/1566419143 >> [v2:172.21.15.96:6816/2449989110,v1:172.21.15.96:6817/2449989110] conn(0x7fb0440bdbc0 msgr2=0x7fb0440c00b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:09.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.039+0000 7fb05d4f3700 1 --2- 172.21.15.12:0/1566419143 >> [v2:172.21.15.96:6816/2449989110,v1:172.21.15.96:6817/2449989110] conn(0x7fb0440bdbc0 0x7fb0440c00b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:09.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.039+0000 7fb05d4f3700 1 -- 172.21.15.12:0/1566419143 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb0440b2910 msgr2=0x7fb0440b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:09.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.039+0000 7fb05d4f3700 1 --2- 172.21.15.12:0/1566419143 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb0440b2910 0x7fb0440b4e20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb058108000 tx=0x7fb04000b410 comp rx=0 tx=0).stop 2024-05-17T22:37:09.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.039+0000 7fb05d4f3700 1 -- 172.21.15.12:0/1566419143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb058102e70 msgr2=0x7fb05819a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:09.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.039+0000 7fb05d4f3700 1 --2- 172.21.15.12:0/1566419143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb058102e70 0x7fb05819a730 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb04800d980 tx=0x7fb04800dc90 comp rx=0 tx=0).stop 2024-05-17T22:37:09.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.039+0000 7fb0577fe700 1 -- 172.21.15.12:0/1566419143 reap_dead start 2024-05-17T22:37:09.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.039+0000 7fb05d4f3700 1 -- 172.21.15.12:0/1566419143 shutdown_connections 2024-05-17T22:37:09.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.039+0000 7fb05d4f3700 1 -- 172.21.15.12:0/1566419143 >> 172.21.15.12:0/1566419143 conn(0x7fb058079ae0 msgr2=0x7fb058101c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:09.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.039+0000 7fb05d4f3700 1 -- 172.21.15.12:0/1566419143 shutdown_connections 2024-05-17T22:37:09.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.039+0000 7fb05d4f3700 1 -- 172.21.15.12:0/1566419143 wait complete. 2024-05-17T22:37:09.672 INFO:teuthology.orchestra.run.smithi012.stdout:154618822733 2024-05-17T22:37:09.672 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-17T22:37:09.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.884+0000 7f860cf85700 1 -- 172.21.15.12:0/787519803 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8608108080 msgr2=0x7f860810f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:09.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.884+0000 7f860cf85700 1 --2- 172.21.15.12:0/787519803 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8608108080 0x7f860810f9e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f860000b0a0 tx=0x7f860000b3b0 comp rx=0 tx=0).stop 2024-05-17T22:37:09.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.884+0000 7f860cf85700 1 -- 172.21.15.12:0/787519803 shutdown_connections 2024-05-17T22:37:09.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.884+0000 7f860cf85700 1 --2- 172.21.15.12:0/787519803 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8608108080 0x7f860810f9e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:09.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.884+0000 7f860cf85700 1 --2- 172.21.15.12:0/787519803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8608102e00 0x7f8608103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:09.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.884+0000 7f860cf85700 1 --2- 172.21.15.12:0/787519803 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8608107730 0x7f8608107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:09.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.884+0000 7f860cf85700 1 -- 172.21.15.12:0/787519803 >> 172.21.15.12:0/787519803 conn(0x7f8608079bd0 msgr2=0x7f8608079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:09.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.884+0000 7f860cf85700 1 -- 172.21.15.12:0/787519803 shutdown_connections 2024-05-17T22:37:09.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.884+0000 7f860cf85700 1 -- 172.21.15.12:0/787519803 wait complete. 2024-05-17T22:37:09.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.885+0000 7f860cf85700 1 Processor -- start 2024-05-17T22:37:09.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.885+0000 7f860cf85700 1 -- start start 2024-05-17T22:37:09.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.886+0000 7f860cf85700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8608102e00 0x7f86081a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:09.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.886+0000 7f860cf85700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8608107730 0x7f86081a2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:09.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.886+0000 7f860cf85700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8608108080 0x7f860819c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:09.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.886+0000 7f860cf85700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8608112ce0 con 0x7f8608102e00 2024-05-17T22:37:09.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.886+0000 7f860cf85700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8608112b60 con 0x7f8608107730 2024-05-17T22:37:09.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.886+0000 7f860cf85700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8608112e60 con 0x7f8608108080 2024-05-17T22:37:09.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.886+0000 7f860659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8608102e00 0x7f86081a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:09.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.886+0000 7f860659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8608102e00 0x7f86081a2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57046/0 (socket says 172.21.15.12:57046) 2024-05-17T22:37:09.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.886+0000 7f860659c700 1 -- 172.21.15.12:0/3704793209 learned_addr learned my addr 172.21.15.12:0/3704793209 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:37:09.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.886+0000 7f8605d9b700 1 --2- 172.21.15.12:0/3704793209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8608107730 0x7f86081a2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:09.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.886+0000 7f860659c700 1 -- 172.21.15.12:0/3704793209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8608108080 msgr2=0x7f860819c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:09.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.886+0000 7f860659c700 1 --2- 172.21.15.12:0/3704793209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8608108080 0x7f860819c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:09.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.886+0000 7f860659c700 1 -- 172.21.15.12:0/3704793209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8608107730 msgr2=0x7f86081a2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:09.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.886+0000 7f860659c700 1 --2- 172.21.15.12:0/3704793209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8608107730 0x7f86081a2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:09.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.886+0000 7f860659c700 1 -- 172.21.15.12:0/3704793209 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f860819cec0 con 0x7f8608102e00 2024-05-17T22:37:09.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.886+0000 7f8605d9b700 1 --2- 172.21.15.12:0/3704793209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8608107730 0x7f86081a2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:37:09.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.886+0000 7f860659c700 1 --2- 172.21.15.12:0/3704793209 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8608102e00 0x7f86081a2600 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f85f800d950 tx=0x7f85f800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:09.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.886+0000 7f85f77fe700 1 -- 172.21.15.12:0/3704793209 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f8016070 con 0x7f8608102e00 2024-05-17T22:37:09.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.886+0000 7f860cf85700 1 -- 172.21.15.12:0/3704793209 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f860819d1b0 con 0x7f8608102e00 2024-05-17T22:37:09.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.887+0000 7f85f77fe700 1 -- 172.21.15.12:0/3704793209 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85f8010460 con 0x7f8608102e00 2024-05-17T22:37:09.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.887+0000 7f860cf85700 1 -- 172.21.15.12:0/3704793209 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86081a93b0 con 0x7f8608102e00 2024-05-17T22:37:09.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.887+0000 7f85f77fe700 1 -- 172.21.15.12:0/3704793209 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f800f620 con 0x7f8608102e00 2024-05-17T22:37:09.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.887+0000 7f860cf85700 1 -- 172.21.15.12:0/3704793209 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f85e8001060 con 0x7f8608102e00 2024-05-17T22:37:09.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.889+0000 7f85f77fe700 1 -- 172.21.15.12:0/3704793209 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f85f8019070 con 0x7f8608102e00 2024-05-17T22:37:09.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.890+0000 7f85f77fe700 1 --2- 172.21.15.12:0/3704793209 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f85f00b2a10 0x7f85f00b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:09.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.890+0000 7f8605d9b700 1 --2- 172.21.15.12:0/3704793209 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f85f00b2a10 0x7f85f00b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:09.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.890+0000 7f85f77fe700 1 -- 172.21.15.12:0/3704793209 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f85f80e2840 con 0x7f8608102e00 2024-05-17T22:37:09.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.890+0000 7f85f77fe700 1 --2- 172.21.15.12:0/3704793209 >> [v2:172.21.15.151:6800/148729815,v1:172.21.15.151:6801/148729815] conn(0x7f85f00bdcc0 0x7f85f00c01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:09.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.890+0000 7f85f77fe700 1 -- 172.21.15.12:0/3704793209 --> [v2:172.21.15.151:6800/148729815,v1:172.21.15.151:6801/148729815] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f85f00c0880 con 0x7f85f00bdcc0 2024-05-17T22:37:09.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.890+0000 7f85f77fe700 1 -- 172.21.15.12:0/3704793209 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f85f80e2c30 con 0x7f8608102e00 2024-05-17T22:37:09.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.891+0000 7f8605d9b700 1 --2- 172.21.15.12:0/3704793209 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f85f00b2a10 0x7f85f00b4f20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f85fc005e10 tx=0x7f85fc005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:09.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.891+0000 7f8606d9d700 1 --2- 172.21.15.12:0/3704793209 >> [v2:172.21.15.151:6800/148729815,v1:172.21.15.151:6801/148729815] conn(0x7f85f00bdcc0 0x7f85f00c01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:09.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.891+0000 7f8606d9d700 1 --2- 172.21.15.12:0/3704793209 >> [v2:172.21.15.151:6800/148729815,v1:172.21.15.151:6801/148729815] conn(0x7f85f00bdcc0 0x7f85f00c01b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:09.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.892+0000 7f85f77fe700 1 -- 172.21.15.12:0/3704793209 <== osd.2 v2:172.21.15.151:6800/148729815 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f85f00c0880 con 0x7f85f00bdcc0 2024-05-17T22:37:09.895 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:09 smithi012 ceph-mon[31632]: pgmap v376: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-17T22:37:09.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.909+0000 7f860cf85700 1 -- 172.21.15.12:0/3704793209 --> [v2:172.21.15.151:6800/148729815,v1:172.21.15.151:6801/148729815] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f85e8002e90 con 0x7f85f00bdcc0 2024-05-17T22:37:09.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.909+0000 7f85f77fe700 1 -- 172.21.15.12:0/3704793209 <== osd.2 v2:172.21.15.151:6800/148729815 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f85e8002e90 con 0x7f85f00bdcc0 2024-05-17T22:37:09.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.910+0000 7f860cf85700 1 -- 172.21.15.12:0/3704793209 >> [v2:172.21.15.151:6800/148729815,v1:172.21.15.151:6801/148729815] conn(0x7f85f00bdcc0 msgr2=0x7f85f00c01b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:09.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.910+0000 7f860cf85700 1 --2- 172.21.15.12:0/3704793209 >> [v2:172.21.15.151:6800/148729815,v1:172.21.15.151:6801/148729815] conn(0x7f85f00bdcc0 0x7f85f00c01b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:09.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.910+0000 7f860cf85700 1 -- 172.21.15.12:0/3704793209 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f85f00b2a10 msgr2=0x7f85f00b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:09.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.910+0000 7f860cf85700 1 --2- 172.21.15.12:0/3704793209 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f85f00b2a10 0x7f85f00b4f20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f85fc005e10 tx=0x7f85fc005da0 comp rx=0 tx=0).stop 2024-05-17T22:37:09.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.910+0000 7f860cf85700 1 -- 172.21.15.12:0/3704793209 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8608102e00 msgr2=0x7f86081a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:09.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.910+0000 7f860cf85700 1 --2- 172.21.15.12:0/3704793209 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8608102e00 0x7f86081a2600 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f85f800d950 tx=0x7f85f800dd10 comp rx=0 tx=0).stop 2024-05-17T22:37:09.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.910+0000 7f8606d9d700 1 -- 172.21.15.12:0/3704793209 reap_dead start 2024-05-17T22:37:09.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.910+0000 7f860cf85700 1 -- 172.21.15.12:0/3704793209 shutdown_connections 2024-05-17T22:37:09.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.910+0000 7f860cf85700 1 -- 172.21.15.12:0/3704793209 >> 172.21.15.12:0/3704793209 conn(0x7f8608079bd0 msgr2=0x7f860806b780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:09.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.910+0000 7f860cf85700 1 -- 172.21.15.12:0/3704793209 shutdown_connections 2024-05-17T22:37:09.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:09.910+0000 7f860cf85700 1 -- 172.21.15.12:0/3704793209 wait complete. 2024-05-17T22:37:10.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:09 smithi151 ceph-mon[36101]: pgmap v376: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-17T22:37:10.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:09 smithi096 ceph-mon[36065]: pgmap v376: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-17T22:37:10.547 INFO:teuthology.orchestra.run.smithi012.stdout:201863462979 2024-05-17T22:37:10.547 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-17T22:37:11.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.158+0000 7f5462a6f700 1 -- 172.21.15.12:0/656741642 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f545c104e10 msgr2=0x7f545c105220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:11.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.158+0000 7f5462a6f700 1 --2- 172.21.15.12:0/656741642 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f545c104e10 0x7f545c105220 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5450009990 tx=0x7f5450009ca0 comp rx=0 tx=0).stop 2024-05-17T22:37:11.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.158+0000 7f5462a6f700 1 -- 172.21.15.12:0/656741642 shutdown_connections 2024-05-17T22:37:11.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.158+0000 7f5462a6f700 1 --2- 172.21.15.12:0/656741642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f545c10e240 0x7f545c110750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:11.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.158+0000 7f5462a6f700 1 --2- 172.21.15.12:0/656741642 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f545c104e10 0x7f545c105220 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:11.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.158+0000 7f5462a6f700 1 --2- 172.21.15.12:0/656741642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f545c105760 0x7f545c10dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:11.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.158+0000 7f5462a6f700 1 -- 172.21.15.12:0/656741642 >> 172.21.15.12:0/656741642 conn(0x7f545c079ae0 msgr2=0x7f545c079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:11.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.158+0000 7f5462a6f700 1 -- 172.21.15.12:0/656741642 shutdown_connections 2024-05-17T22:37:11.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.158+0000 7f5462a6f700 1 -- 172.21.15.12:0/656741642 wait complete. 2024-05-17T22:37:11.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.159+0000 7f5462a6f700 1 Processor -- start 2024-05-17T22:37:11.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.159+0000 7f5462a6f700 1 -- start start 2024-05-17T22:37:11.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.159+0000 7f5462a6f700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f545c104e10 0x7f545c102b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:11.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.159+0000 7f5462a6f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f545c105760 0x7f545c103040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:11.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.159+0000 7f5462a6f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f545c10e240 0x7f545c103580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:11.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.159+0000 7f5462a6f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f545c113e40 con 0x7f545c105760 2024-05-17T22:37:11.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.159+0000 7f5462a6f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f545c113cc0 con 0x7f545c10e240 2024-05-17T22:37:11.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.159+0000 7f5462a6f700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f545c113fc0 con 0x7f545c104e10 2024-05-17T22:37:11.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.160+0000 7f545bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f545c105760 0x7f545c103040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:11.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.160+0000 7f545bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f545c105760 0x7f545c103040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57054/0 (socket says 172.21.15.12:57054) 2024-05-17T22:37:11.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.160+0000 7f545bfff700 1 -- 172.21.15.12:0/1498083449 learned_addr learned my addr 172.21.15.12:0/1498083449 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:37:11.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.160+0000 7f546080b700 1 --2- 172.21.15.12:0/1498083449 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f545c104e10 0x7f545c102b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:11.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.160+0000 7f546100c700 1 --2- 172.21.15.12:0/1498083449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f545c10e240 0x7f545c103580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:11.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.160+0000 7f545bfff700 1 -- 172.21.15.12:0/1498083449 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f545c104e10 msgr2=0x7f545c102b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:11.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.160+0000 7f545bfff700 1 --2- 172.21.15.12:0/1498083449 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f545c104e10 0x7f545c102b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:11.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.160+0000 7f545bfff700 1 -- 172.21.15.12:0/1498083449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f545c10e240 msgr2=0x7f545c103580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:11.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.160+0000 7f545bfff700 1 --2- 172.21.15.12:0/1498083449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f545c10e240 0x7f545c103580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:11.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.160+0000 7f545bfff700 1 -- 172.21.15.12:0/1498083449 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f545c1a8b10 con 0x7f545c105760 2024-05-17T22:37:11.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.160+0000 7f545bfff700 1 --2- 172.21.15.12:0/1498083449 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f545c105760 0x7f545c103040 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f544800c9b0 tx=0x7f544800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:11.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.160+0000 7f5459ffb700 1 -- 172.21.15.12:0/1498083449 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448007b00 con 0x7f545c105760 2024-05-17T22:37:11.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.160+0000 7f5459ffb700 1 -- 172.21.15.12:0/1498083449 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f544800ce50 con 0x7f545c105760 2024-05-17T22:37:11.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.160+0000 7f5462a6f700 1 -- 172.21.15.12:0/1498083449 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f545c1a8e00 con 0x7f545c105760 2024-05-17T22:37:11.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.160+0000 7f5459ffb700 1 -- 172.21.15.12:0/1498083449 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448018660 con 0x7f545c105760 2024-05-17T22:37:11.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.161+0000 7f5462a6f700 1 -- 172.21.15.12:0/1498083449 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f545c1a92b0 con 0x7f545c105760 2024-05-17T22:37:11.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.162+0000 7f5462a6f700 1 -- 172.21.15.12:0/1498083449 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f545c063f40 con 0x7f545c105760 2024-05-17T22:37:11.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.163+0000 7f5459ffb700 1 -- 172.21.15.12:0/1498083449 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f544800f450 con 0x7f545c105760 2024-05-17T22:37:11.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.164+0000 7f5459ffb700 1 --2- 172.21.15.12:0/1498083449 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f544c0b2910 0x7f544c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:11.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.164+0000 7f546080b700 1 --2- 172.21.15.12:0/1498083449 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f544c0b2910 0x7f544c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:11.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.164+0000 7f5459ffb700 1 -- 172.21.15.12:0/1498083449 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f54480e2660 con 0x7f545c105760 2024-05-17T22:37:11.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.164+0000 7f5459ffb700 1 --2- 172.21.15.12:0/1498083449 >> [v2:172.21.15.96:6808/2219103166,v1:172.21.15.96:6809/2219103166] conn(0x7f544c0bdbc0 0x7f544c0c00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:11.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.164+0000 7f5459ffb700 1 -- 172.21.15.12:0/1498083449 --> [v2:172.21.15.96:6808/2219103166,v1:172.21.15.96:6809/2219103166] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5448012ec0 con 0x7f544c0bdbc0 2024-05-17T22:37:11.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.164+0000 7f5459ffb700 1 -- 172.21.15.12:0/1498083449 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f54480e9050 con 0x7f545c105760 2024-05-17T22:37:11.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.164+0000 7f546080b700 1 --2- 172.21.15.12:0/1498083449 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f544c0b2910 0x7f544c0b4e20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f54500098e0 tx=0x7f5450005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:11.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.164+0000 7f546100c700 1 --2- 172.21.15.12:0/1498083449 >> [v2:172.21.15.96:6808/2219103166,v1:172.21.15.96:6809/2219103166] conn(0x7f544c0bdbc0 0x7f544c0c00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:11.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.165+0000 7f546100c700 1 --2- 172.21.15.12:0/1498083449 >> [v2:172.21.15.96:6808/2219103166,v1:172.21.15.96:6809/2219103166] conn(0x7f544c0bdbc0 0x7f544c0c00b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:11.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.166+0000 7f5459ffb700 1 -- 172.21.15.12:0/1498083449 <== osd.4 v2:172.21.15.96:6808/2219103166 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5448012ec0 con 0x7f544c0bdbc0 2024-05-17T22:37:11.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.182+0000 7f5462a6f700 1 -- 172.21.15.12:0/1498083449 --> [v2:172.21.15.96:6808/2219103166,v1:172.21.15.96:6809/2219103166] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f545c0643f0 con 0x7f544c0bdbc0 2024-05-17T22:37:11.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.183+0000 7f5459ffb700 1 -- 172.21.15.12:0/1498083449 <== osd.4 v2:172.21.15.96:6808/2219103166 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f545c0643f0 con 0x7f544c0bdbc0 2024-05-17T22:37:11.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.183+0000 7f5462a6f700 1 -- 172.21.15.12:0/1498083449 >> [v2:172.21.15.96:6808/2219103166,v1:172.21.15.96:6809/2219103166] conn(0x7f544c0bdbc0 msgr2=0x7f544c0c00b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:11.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.183+0000 7f5462a6f700 1 --2- 172.21.15.12:0/1498083449 >> [v2:172.21.15.96:6808/2219103166,v1:172.21.15.96:6809/2219103166] conn(0x7f544c0bdbc0 0x7f544c0c00b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:11.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.183+0000 7f5462a6f700 1 -- 172.21.15.12:0/1498083449 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f544c0b2910 msgr2=0x7f544c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:11.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.183+0000 7f5462a6f700 1 --2- 172.21.15.12:0/1498083449 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f544c0b2910 0x7f544c0b4e20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f54500098e0 tx=0x7f5450005960 comp rx=0 tx=0).stop 2024-05-17T22:37:11.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.183+0000 7f5462a6f700 1 -- 172.21.15.12:0/1498083449 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f545c105760 msgr2=0x7f545c103040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:11.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.183+0000 7f5462a6f700 1 --2- 172.21.15.12:0/1498083449 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f545c105760 0x7f545c103040 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f544800c9b0 tx=0x7f544800ccc0 comp rx=0 tx=0).stop 2024-05-17T22:37:11.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.183+0000 7f546100c700 1 -- 172.21.15.12:0/1498083449 reap_dead start 2024-05-17T22:37:11.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.183+0000 7f5462a6f700 1 -- 172.21.15.12:0/1498083449 shutdown_connections 2024-05-17T22:37:11.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.184+0000 7f5462a6f700 1 -- 172.21.15.12:0/1498083449 >> 172.21.15.12:0/1498083449 conn(0x7f545c079ae0 msgr2=0x7f545c100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:11.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.184+0000 7f5462a6f700 1 -- 172.21.15.12:0/1498083449 shutdown_connections 2024-05-17T22:37:11.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:11.184+0000 7f5462a6f700 1 -- 172.21.15.12:0/1498083449 wait complete. 2024-05-17T22:37:11.940 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:11 smithi012 ceph-mon[31632]: pgmap v377: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T22:37:11.973 INFO:teuthology.orchestra.run.smithi012.stdout:133143986258 2024-05-17T22:37:11.973 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-17T22:37:12.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:11 smithi151 ceph-mon[36101]: pgmap v377: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T22:37:12.303 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:11 smithi096 ceph-mon[36065]: pgmap v377: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T22:37:12.556 INFO:teuthology.orchestra.run.smithi012.stdout:77309411421 2024-05-17T22:37:12.556 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-17T22:37:13.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.064+0000 7fa9b33f4700 1 -- 172.21.15.12:0/483009909 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa9ac107750 msgr2=0x7fa9ac107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:13.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.064+0000 7fa9b33f4700 1 --2- 172.21.15.12:0/483009909 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa9ac107750 0x7fa9ac107b60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa99c009990 tx=0x7fa99c009ca0 comp rx=0 tx=0).stop 2024-05-17T22:37:13.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.064+0000 7fa9b33f4700 1 -- 172.21.15.12:0/483009909 shutdown_connections 2024-05-17T22:37:13.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.064+0000 7fa9b33f4700 1 --2- 172.21.15.12:0/483009909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9ac1080a0 0x7fa9ac10fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:13.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.064+0000 7fa9b33f4700 1 --2- 172.21.15.12:0/483009909 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9ac102e70 0x7fa9ac103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:13.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.064+0000 7fa9b33f4700 1 --2- 172.21.15.12:0/483009909 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa9ac107750 0x7fa9ac107b60 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:13.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.064+0000 7fa9b33f4700 1 -- 172.21.15.12:0/483009909 >> 172.21.15.12:0/483009909 conn(0x7fa9ac079ae0 msgr2=0x7fa9ac079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:13.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.064+0000 7fa9b33f4700 1 -- 172.21.15.12:0/483009909 shutdown_connections 2024-05-17T22:37:13.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.064+0000 7fa9b33f4700 1 -- 172.21.15.12:0/483009909 wait complete. 2024-05-17T22:37:13.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.065+0000 7fa9b33f4700 1 Processor -- start 2024-05-17T22:37:13.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.065+0000 7fa9b33f4700 1 -- start start 2024-05-17T22:37:13.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.065+0000 7fa9b33f4700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9ac102e70 0x7fa9ac105190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:13.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.065+0000 7fa9b33f4700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9ac107750 0x7fa9ac1056d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:13.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.065+0000 7fa9b33f4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa9ac1080a0 0x7fa9ac105c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:13.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.065+0000 7fa9b33f4700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ac112cc0 con 0x7fa9ac1080a0 2024-05-17T22:37:13.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.065+0000 7fa9b33f4700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ac112b40 con 0x7fa9ac102e70 2024-05-17T22:37:13.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.065+0000 7fa9b33f4700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ac112e40 con 0x7fa9ac107750 2024-05-17T22:37:13.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.066+0000 7fa9b1991700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa9ac1080a0 0x7fa9ac105c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:13.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.066+0000 7fa9b1991700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa9ac1080a0 0x7fa9ac105c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57074/0 (socket says 172.21.15.12:57074) 2024-05-17T22:37:13.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.066+0000 7fa9b1190700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9ac102e70 0x7fa9ac105190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:13.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.066+0000 7fa9b1991700 1 -- 172.21.15.12:0/2438064112 learned_addr learned my addr 172.21.15.12:0/2438064112 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:37:13.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.066+0000 7fa9b098f700 1 --2- 172.21.15.12:0/2438064112 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9ac107750 0x7fa9ac1056d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:13.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.066+0000 7fa9b1991700 1 -- 172.21.15.12:0/2438064112 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9ac107750 msgr2=0x7fa9ac1056d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:13.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.066+0000 7fa9b1991700 1 --2- 172.21.15.12:0/2438064112 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9ac107750 0x7fa9ac1056d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:13.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.066+0000 7fa9b1991700 1 -- 172.21.15.12:0/2438064112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9ac102e70 msgr2=0x7fa9ac105190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:13.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.066+0000 7fa9b1991700 1 --2- 172.21.15.12:0/2438064112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9ac102e70 0x7fa9ac105190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:13.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.066+0000 7fa9b1991700 1 -- 172.21.15.12:0/2438064112 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9ac1a8c90 con 0x7fa9ac1080a0 2024-05-17T22:37:13.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.066+0000 7fa9b1991700 1 --2- 172.21.15.12:0/2438064112 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa9ac1080a0 0x7fa9ac105c10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4007ee0 tx=0x7fa9a400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:13.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.066+0000 7fa9a27fc700 1 -- 172.21.15.12:0/2438064112 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4004bb0 con 0x7fa9ac1080a0 2024-05-17T22:37:13.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.066+0000 7fa9a27fc700 1 -- 172.21.15.12:0/2438064112 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9a4004d50 con 0x7fa9ac1080a0 2024-05-17T22:37:13.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.066+0000 7fa9b33f4700 1 -- 172.21.15.12:0/2438064112 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9ac1a8f80 con 0x7fa9ac1080a0 2024-05-17T22:37:13.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.066+0000 7fa9a27fc700 1 -- 172.21.15.12:0/2438064112 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4005920 con 0x7fa9ac1080a0 2024-05-17T22:37:13.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.067+0000 7fa9b33f4700 1 -- 172.21.15.12:0/2438064112 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9ac1a93d0 con 0x7fa9ac1080a0 2024-05-17T22:37:13.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.067+0000 7fa9b33f4700 1 -- 172.21.15.12:0/2438064112 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa9ac063f40 con 0x7fa9ac1080a0 2024-05-17T22:37:13.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.069+0000 7fa9a27fc700 1 -- 172.21.15.12:0/2438064112 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa9a401a020 con 0x7fa9ac1080a0 2024-05-17T22:37:13.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.070+0000 7fa9a27fc700 1 --2- 172.21.15.12:0/2438064112 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa9980b2910 0x7fa9980b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:13.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.070+0000 7fa9b1190700 1 --2- 172.21.15.12:0/2438064112 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa9980b2910 0x7fa9980b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:13.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.070+0000 7fa9a27fc700 1 -- 172.21.15.12:0/2438064112 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fa9a40e2330 con 0x7fa9ac1080a0 2024-05-17T22:37:13.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.071+0000 7fa9a27fc700 1 --2- 172.21.15.12:0/2438064112 >> [v2:172.21.15.151:6824/3822307314,v1:172.21.15.151:6825/3822307314] conn(0x7fa9980bdbc0 0x7fa9980c00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:13.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.071+0000 7fa9a27fc700 1 -- 172.21.15.12:0/2438064112 --> [v2:172.21.15.151:6824/3822307314,v1:172.21.15.151:6825/3822307314] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa9a40020b0 con 0x7fa9980bdbc0 2024-05-17T22:37:13.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.071+0000 7fa9a27fc700 1 -- 172.21.15.12:0/2438064112 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fa9a40e2810 con 0x7fa9ac1080a0 2024-05-17T22:37:13.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.071+0000 7fa9b1190700 1 --2- 172.21.15.12:0/2438064112 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa9980b2910 0x7fa9980b4e20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8005950 tx=0x7fa9a800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:13.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.071+0000 7fa9b098f700 1 --2- 172.21.15.12:0/2438064112 >> [v2:172.21.15.151:6824/3822307314,v1:172.21.15.151:6825/3822307314] conn(0x7fa9980bdbc0 0x7fa9980c00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:13.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.072+0000 7fa9b098f700 1 --2- 172.21.15.12:0/2438064112 >> [v2:172.21.15.151:6824/3822307314,v1:172.21.15.151:6825/3822307314] conn(0x7fa9980bdbc0 0x7fa9980c00b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:13.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.073+0000 7fa9a27fc700 1 -- 172.21.15.12:0/2438064112 <== osd.11 v2:172.21.15.151:6824/3822307314 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa9a40020b0 con 0x7fa9980bdbc0 2024-05-17T22:37:13.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.089+0000 7fa9b33f4700 1 -- 172.21.15.12:0/2438064112 --> [v2:172.21.15.151:6824/3822307314,v1:172.21.15.151:6825/3822307314] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa9ac064340 con 0x7fa9980bdbc0 2024-05-17T22:37:13.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.090+0000 7fa9a27fc700 1 -- 172.21.15.12:0/2438064112 <== osd.11 v2:172.21.15.151:6824/3822307314 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa9ac064340 con 0x7fa9980bdbc0 2024-05-17T22:37:13.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.090+0000 7fa9b33f4700 1 -- 172.21.15.12:0/2438064112 >> [v2:172.21.15.151:6824/3822307314,v1:172.21.15.151:6825/3822307314] conn(0x7fa9980bdbc0 msgr2=0x7fa9980c00b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:13.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.090+0000 7fa9b33f4700 1 --2- 172.21.15.12:0/2438064112 >> [v2:172.21.15.151:6824/3822307314,v1:172.21.15.151:6825/3822307314] conn(0x7fa9980bdbc0 0x7fa9980c00b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:13.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.090+0000 7fa9b33f4700 1 -- 172.21.15.12:0/2438064112 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa9980b2910 msgr2=0x7fa9980b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:13.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.090+0000 7fa9b33f4700 1 --2- 172.21.15.12:0/2438064112 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa9980b2910 0x7fa9980b4e20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8005950 tx=0x7fa9a800a300 comp rx=0 tx=0).stop 2024-05-17T22:37:13.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.090+0000 7fa9b33f4700 1 -- 172.21.15.12:0/2438064112 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa9ac1080a0 msgr2=0x7fa9ac105c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:13.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.090+0000 7fa9b33f4700 1 --2- 172.21.15.12:0/2438064112 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa9ac1080a0 0x7fa9ac105c10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4007ee0 tx=0x7fa9a400d3b0 comp rx=0 tx=0).stop 2024-05-17T22:37:13.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.090+0000 7fa9b1991700 1 -- 172.21.15.12:0/2438064112 reap_dead start 2024-05-17T22:37:13.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.090+0000 7fa9b33f4700 1 -- 172.21.15.12:0/2438064112 shutdown_connections 2024-05-17T22:37:13.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.090+0000 7fa9b33f4700 1 -- 172.21.15.12:0/2438064112 >> 172.21.15.12:0/2438064112 conn(0x7fa9ac079ae0 msgr2=0x7fa9ac06c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:13.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.090+0000 7fa9b33f4700 1 -- 172.21.15.12:0/2438064112 shutdown_connections 2024-05-17T22:37:13.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:13.090+0000 7fa9b33f4700 1 -- 172.21.15.12:0/2438064112 wait complete. 2024-05-17T22:37:13.529 INFO:teuthology.orchestra.run.smithi012.stdout:180388626502 2024-05-17T22:37:13.529 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-17T22:37:14.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:13 smithi151 ceph-mon[36101]: pgmap v378: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:37:14.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:13 smithi096 ceph-mon[36065]: pgmap v378: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:37:14.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:13 smithi012 ceph-mon[31632]: pgmap v378: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:37:14.602 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:37:14.603 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:37:16.048 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:15 smithi012 ceph-mon[31632]: pgmap v379: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-17T22:37:16.048 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:15 smithi012 ceph-mon[31632]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-05-17T22:37:16.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:15 smithi151 ceph-mon[36101]: pgmap v379: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-17T22:37:16.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:15 smithi151 ceph-mon[36101]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-05-17T22:37:16.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:15 smithi096 ceph-mon[36065]: pgmap v379: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-17T22:37:16.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:15 smithi096 ceph-mon[36065]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-05-17T22:37:16.866 INFO:teuthology.orchestra.run.smithi012.stdout:300647710767 2024-05-17T22:37:16.866 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-17T22:37:16.892 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:37:16.893 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:37:16.895 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:37:16.898 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:37:17.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.157+0000 7fda69232700 1 -- 172.21.15.12:0/2314863472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda64103f10 msgr2=0x7fda64104320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:17.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.157+0000 7fda69232700 1 --2- 172.21.15.12:0/2314863472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda64103f10 0x7fda64104320 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fda54009990 tx=0x7fda54009ca0 comp rx=0 tx=0).stop 2024-05-17T22:37:17.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.157+0000 7fda69232700 1 -- 172.21.15.12:0/2314863472 shutdown_connections 2024-05-17T22:37:17.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.157+0000 7fda69232700 1 --2- 172.21.15.12:0/2314863472 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda64109190 0x7fda64110af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:17.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.157+0000 7fda69232700 1 --2- 172.21.15.12:0/2314863472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda64103f10 0x7fda64104320 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:17.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.157+0000 7fda69232700 1 --2- 172.21.15.12:0/2314863472 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda64108840 0x7fda64108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:17.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.157+0000 7fda69232700 1 -- 172.21.15.12:0/2314863472 >> 172.21.15.12:0/2314863472 conn(0x7fda640ff990 msgr2=0x7fda64101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:17.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.157+0000 7fda69232700 1 -- 172.21.15.12:0/2314863472 shutdown_connections 2024-05-17T22:37:17.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.157+0000 7fda69232700 1 -- 172.21.15.12:0/2314863472 wait complete. 2024-05-17T22:37:17.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.158+0000 7fda69232700 1 Processor -- start 2024-05-17T22:37:17.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.158+0000 7fda69232700 1 -- start start 2024-05-17T22:37:17.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.159+0000 7fda69232700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda64103f10 0x7fda64075b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:17.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.159+0000 7fda69232700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda64108840 0x7fda64076070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:17.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.159+0000 7fda69232700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda64109190 0x7fda640765b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:17.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.159+0000 7fda69232700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fda64113dc0 con 0x7fda64103f10 2024-05-17T22:37:17.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.159+0000 7fda69232700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fda64113c40 con 0x7fda64108840 2024-05-17T22:37:17.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.159+0000 7fda69232700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fda64113f40 con 0x7fda64109190 2024-05-17T22:37:17.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.159+0000 7fda62d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda64103f10 0x7fda64075b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:17.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.159+0000 7fda62d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda64103f10 0x7fda64075b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57086/0 (socket says 172.21.15.12:57086) 2024-05-17T22:37:17.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.159+0000 7fda62d9d700 1 -- 172.21.15.12:0/615070961 learned_addr learned my addr 172.21.15.12:0/615070961 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:37:17.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.159+0000 7fda6259c700 1 --2- 172.21.15.12:0/615070961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda64108840 0x7fda64076070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:17.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.159+0000 7fda6359e700 1 --2- 172.21.15.12:0/615070961 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda64109190 0x7fda640765b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:17.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.159+0000 7fda62d9d700 1 -- 172.21.15.12:0/615070961 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda64109190 msgr2=0x7fda640765b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:17.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.159+0000 7fda62d9d700 1 --2- 172.21.15.12:0/615070961 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda64109190 0x7fda640765b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:17.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.159+0000 7fda62d9d700 1 -- 172.21.15.12:0/615070961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda64108840 msgr2=0x7fda64076070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:17.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.159+0000 7fda62d9d700 1 --2- 172.21.15.12:0/615070961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda64108840 0x7fda64076070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:17.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.159+0000 7fda62d9d700 1 -- 172.21.15.12:0/615070961 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda641a8ab0 con 0x7fda64103f10 2024-05-17T22:37:17.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.159+0000 7fda6259c700 1 --2- 172.21.15.12:0/615070961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda64108840 0x7fda64076070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:37:17.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.159+0000 7fda6359e700 1 --2- 172.21.15.12:0/615070961 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda64109190 0x7fda640765b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:37:17.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.159+0000 7fda62d9d700 1 --2- 172.21.15.12:0/615070961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda64103f10 0x7fda64075b30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fda540057c0 tx=0x7fda540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:17.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.160+0000 7fda4bfff700 1 -- 172.21.15.12:0/615070961 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda54027070 con 0x7fda64103f10 2024-05-17T22:37:17.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.160+0000 7fda4bfff700 1 -- 172.21.15.12:0/615070961 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda54004020 con 0x7fda64103f10 2024-05-17T22:37:17.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.160+0000 7fda4bfff700 1 -- 172.21.15.12:0/615070961 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda54022bf0 con 0x7fda64103f10 2024-05-17T22:37:17.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.160+0000 7fda69232700 1 -- 172.21.15.12:0/615070961 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda641a8d40 con 0x7fda64103f10 2024-05-17T22:37:17.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.160+0000 7fda69232700 1 -- 172.21.15.12:0/615070961 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda641a9170 con 0x7fda64103f10 2024-05-17T22:37:17.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.160+0000 7fda69232700 1 -- 172.21.15.12:0/615070961 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fda64063f40 con 0x7fda64103f10 2024-05-17T22:37:17.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.162+0000 7fda4bfff700 1 -- 172.21.15.12:0/615070961 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fda5402a070 con 0x7fda64103f10 2024-05-17T22:37:17.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.163+0000 7fda4bfff700 1 --2- 172.21.15.12:0/615070961 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fda4c0b2900 0x7fda4c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:17.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.163+0000 7fda6259c700 1 --2- 172.21.15.12:0/615070961 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fda4c0b2900 0x7fda4c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:17.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.163+0000 7fda4bfff700 1 -- 172.21.15.12:0/615070961 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fda540e3ed0 con 0x7fda64103f10 2024-05-17T22:37:17.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.164+0000 7fda4bfff700 1 --2- 172.21.15.12:0/615070961 >> [v2:172.21.15.96:6800/1238316358,v1:172.21.15.96:6801/1238316358] conn(0x7fda4c0bdbb0 0x7fda4c0c00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:17.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.164+0000 7fda4bfff700 1 -- 172.21.15.12:0/615070961 --> [v2:172.21.15.96:6800/1238316358,v1:172.21.15.96:6801/1238316358] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fda4c0c0770 con 0x7fda4c0bdbb0 2024-05-17T22:37:17.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.164+0000 7fda4bfff700 1 -- 172.21.15.12:0/615070961 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fda540e4320 con 0x7fda64103f10 2024-05-17T22:37:17.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.164+0000 7fda6259c700 1 --2- 172.21.15.12:0/615070961 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fda4c0b2900 0x7fda4c0b4e10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fda58005fd0 tx=0x7fda58005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:17.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.164+0000 7fda6359e700 1 --2- 172.21.15.12:0/615070961 >> [v2:172.21.15.96:6800/1238316358,v1:172.21.15.96:6801/1238316358] conn(0x7fda4c0bdbb0 0x7fda4c0c00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:17.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.164+0000 7fda6359e700 1 --2- 172.21.15.12:0/615070961 >> [v2:172.21.15.96:6800/1238316358,v1:172.21.15.96:6801/1238316358] conn(0x7fda4c0bdbb0 0x7fda4c0c00a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:17.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.165+0000 7fda4bfff700 1 -- 172.21.15.12:0/615070961 <== osd.1 v2:172.21.15.96:6800/1238316358 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fda4c0c0770 con 0x7fda4c0bdbb0 2024-05-17T22:37:17.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.182+0000 7fda69232700 1 -- 172.21.15.12:0/615070961 --> [v2:172.21.15.96:6800/1238316358,v1:172.21.15.96:6801/1238316358] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fda640643c0 con 0x7fda4c0bdbb0 2024-05-17T22:37:17.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.182+0000 7fda4bfff700 1 -- 172.21.15.12:0/615070961 <== osd.1 v2:172.21.15.96:6800/1238316358 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fda640643c0 con 0x7fda4c0bdbb0 2024-05-17T22:37:17.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.183+0000 7fda69232700 1 -- 172.21.15.12:0/615070961 >> [v2:172.21.15.96:6800/1238316358,v1:172.21.15.96:6801/1238316358] conn(0x7fda4c0bdbb0 msgr2=0x7fda4c0c00a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:17.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.183+0000 7fda69232700 1 --2- 172.21.15.12:0/615070961 >> [v2:172.21.15.96:6800/1238316358,v1:172.21.15.96:6801/1238316358] conn(0x7fda4c0bdbb0 0x7fda4c0c00a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:17.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.183+0000 7fda69232700 1 -- 172.21.15.12:0/615070961 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fda4c0b2900 msgr2=0x7fda4c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:17.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.183+0000 7fda69232700 1 --2- 172.21.15.12:0/615070961 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fda4c0b2900 0x7fda4c0b4e10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fda58005fd0 tx=0x7fda58005e30 comp rx=0 tx=0).stop 2024-05-17T22:37:17.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.183+0000 7fda69232700 1 -- 172.21.15.12:0/615070961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda64103f10 msgr2=0x7fda64075b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:17.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.183+0000 7fda69232700 1 --2- 172.21.15.12:0/615070961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda64103f10 0x7fda64075b30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fda540057c0 tx=0x7fda540057f0 comp rx=0 tx=0).stop 2024-05-17T22:37:17.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.183+0000 7fda6359e700 1 -- 172.21.15.12:0/615070961 reap_dead start 2024-05-17T22:37:17.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.183+0000 7fda69232700 1 -- 172.21.15.12:0/615070961 shutdown_connections 2024-05-17T22:37:17.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.183+0000 7fda69232700 1 -- 172.21.15.12:0/615070961 >> 172.21.15.12:0/615070961 conn(0x7fda640ff990 msgr2=0x7fda641027d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:17.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.183+0000 7fda69232700 1 -- 172.21.15.12:0/615070961 shutdown_connections 2024-05-17T22:37:17.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:17.183+0000 7fda69232700 1 -- 172.21.15.12:0/615070961 wait complete. 2024-05-17T22:37:18.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:17 smithi012 ceph-mon[31632]: pgmap v380: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-17T22:37:18.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:17 smithi151 ceph-mon[36101]: pgmap v380: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-17T22:37:18.259 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:37:18.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:17 smithi096 ceph-mon[36065]: pgmap v380: 129 pgs: 129 active+clean; 642 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-17T22:37:20.043 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:19 smithi012 ceph-mon[31632]: pgmap v381: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-17T22:37:20.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:19 smithi151 ceph-mon[36101]: pgmap v381: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-17T22:37:20.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:19 smithi096 ceph-mon[36065]: pgmap v381: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-17T22:37:20.554 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:37:20.555 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:37:21.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:21.897+0000 7fcd42fa6700 1 -- 172.21.15.12:0/26394304 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd3c109190 msgr2=0x7fcd3c110af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:21.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:21.897+0000 7fcd42fa6700 1 --2- 172.21.15.12:0/26394304 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd3c109190 0x7fcd3c110af0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fcd3400b0a0 tx=0x7fcd3400b3b0 comp rx=0 tx=0).stop 2024-05-17T22:37:21.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:21.897+0000 7fcd42fa6700 1 -- 172.21.15.12:0/26394304 shutdown_connections 2024-05-17T22:37:21.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:21.897+0000 7fcd42fa6700 1 --2- 172.21.15.12:0/26394304 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd3c109190 0x7fcd3c110af0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:21.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:21.897+0000 7fcd42fa6700 1 --2- 172.21.15.12:0/26394304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd3c103f10 0x7fcd3c104320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:21.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:21.897+0000 7fcd42fa6700 1 --2- 172.21.15.12:0/26394304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd3c108840 0x7fcd3c108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:21.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:21.897+0000 7fcd42fa6700 1 -- 172.21.15.12:0/26394304 >> 172.21.15.12:0/26394304 conn(0x7fcd3c0ff990 msgr2=0x7fcd3c101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:21.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:21.897+0000 7fcd42fa6700 1 -- 172.21.15.12:0/26394304 shutdown_connections 2024-05-17T22:37:21.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:21.897+0000 7fcd42fa6700 1 -- 172.21.15.12:0/26394304 wait complete. 2024-05-17T22:37:21.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:21.898+0000 7fcd42fa6700 1 Processor -- start 2024-05-17T22:37:21.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:21.898+0000 7fcd42fa6700 1 -- start start 2024-05-17T22:37:21.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:21.899+0000 7fcd42fa6700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd3c103f10 0x7fcd3c19a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:21.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:21.899+0000 7fcd42fa6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd3c108840 0x7fcd3c19ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:21.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:21.899+0000 7fcd42fa6700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd3c109190 0x7fcd3c19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:21.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:21.899+0000 7fcd42fa6700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c113e50 con 0x7fcd3c108840 2024-05-17T22:37:21.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:21.899+0000 7fcd42fa6700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c113cd0 con 0x7fcd3c103f10 2024-05-17T22:37:21.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:21.899+0000 7fcd42fa6700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c113fd0 con 0x7fcd3c109190 2024-05-17T22:37:21.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:21.899+0000 7fcd3bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd3c108840 0x7fcd3c19ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:21.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:21.899+0000 7fcd3bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd3c108840 0x7fcd3c19ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34106/0 (socket says 172.21.15.12:34106) 2024-05-17T22:37:21.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:21.899+0000 7fcd3bfff700 1 -- 172.21.15.12:0/2242835627 learned_addr learned my addr 172.21.15.12:0/2242835627 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:37:21.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:21.899+0000 7fcd40d42700 1 --2- 172.21.15.12:0/2242835627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd3c103f10 0x7fcd3c19a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:21.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:21.899+0000 7fcd41543700 1 --2- 172.21.15.12:0/2242835627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd3c109190 0x7fcd3c19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:21.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:21.899+0000 7fcd3bfff700 1 -- 172.21.15.12:0/2242835627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd3c109190 msgr2=0x7fcd3c19efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:21.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:21.899+0000 7fcd3bfff700 1 --2- 172.21.15.12:0/2242835627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd3c109190 0x7fcd3c19efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:21.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:21.899+0000 7fcd3bfff700 1 -- 172.21.15.12:0/2242835627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd3c103f10 msgr2=0x7fcd3c19a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:21.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:21.899+0000 7fcd3bfff700 1 --2- 172.21.15.12:0/2242835627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd3c103f10 0x7fcd3c19a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:21.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:21.899+0000 7fcd3bfff700 1 -- 172.21.15.12:0/2242835627 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd3c19f660 con 0x7fcd3c108840 2024-05-17T22:37:21.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:21.899+0000 7fcd41543700 1 --2- 172.21.15.12:0/2242835627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd3c109190 0x7fcd3c19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:37:21.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:21.899+0000 7fcd3bfff700 1 --2- 172.21.15.12:0/2242835627 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd3c108840 0x7fcd3c19ad30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fcd2800d950 tx=0x7fcd2800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:21.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:21.899+0000 7fcd39ffb700 1 -- 172.21.15.12:0/2242835627 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd28016070 con 0x7fcd3c108840 2024-05-17T22:37:21.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:21.900+0000 7fcd42fa6700 1 -- 172.21.15.12:0/2242835627 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd3c19f950 con 0x7fcd3c108840 2024-05-17T22:37:21.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:21.900+0000 7fcd39ffb700 1 -- 172.21.15.12:0/2242835627 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd2800de30 con 0x7fcd3c108840 2024-05-17T22:37:21.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:21.900+0000 7fcd39ffb700 1 -- 172.21.15.12:0/2242835627 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd28005390 con 0x7fcd3c108840 2024-05-17T22:37:21.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:21.900+0000 7fcd42fa6700 1 -- 172.21.15.12:0/2242835627 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd3c1a7270 con 0x7fcd3c108840 2024-05-17T22:37:21.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:21.901+0000 7fcd42fa6700 1 -- 172.21.15.12:0/2242835627 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd3c104320 con 0x7fcd3c108840 2024-05-17T22:37:21.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:21.902+0000 7fcd39ffb700 1 -- 172.21.15.12:0/2242835627 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fcd28005530 con 0x7fcd3c108840 2024-05-17T22:37:21.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:21.903+0000 7fcd39ffb700 1 --2- 172.21.15.12:0/2242835627 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fcd2c0b2880 0x7fcd2c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:21.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:21.903+0000 7fcd40d42700 1 --2- 172.21.15.12:0/2242835627 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fcd2c0b2880 0x7fcd2c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:21.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:21.903+0000 7fcd39ffb700 1 -- 172.21.15.12:0/2242835627 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fcd280e2190 con 0x7fcd3c108840 2024-05-17T22:37:21.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:21.904+0000 7fcd40d42700 1 --2- 172.21.15.12:0/2242835627 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fcd2c0b2880 0x7fcd2c0b4d90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcd30005950 tx=0x7fcd300058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:21.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:21.906+0000 7fcd39ffb700 1 -- 172.21.15.12:0/2242835627 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcd2801f070 con 0x7fcd3c108840 2024-05-17T22:37:22.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.124+0000 7fcd42fa6700 1 -- 172.21.15.12:0/2242835627 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fcd3c19baf0 con 0x7fcd3c108840 2024-05-17T22:37:22.128 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:21 smithi012 ceph-mon[31632]: pgmap v382: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-17T22:37:22.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.125+0000 7fcd39ffb700 1 -- 172.21.15.12:0/2242835627 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcd280a9ac0 con 0x7fcd3c108840 2024-05-17T22:37:22.128 INFO:teuthology.orchestra.run.smithi012.stdout:34359738473 2024-05-17T22:37:22.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.127+0000 7fcd42fa6700 1 -- 172.21.15.12:0/2242835627 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fcd2c0b2880 msgr2=0x7fcd2c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:22.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.127+0000 7fcd42fa6700 1 --2- 172.21.15.12:0/2242835627 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fcd2c0b2880 0x7fcd2c0b4d90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcd30005950 tx=0x7fcd300058e0 comp rx=0 tx=0).stop 2024-05-17T22:37:22.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.127+0000 7fcd42fa6700 1 -- 172.21.15.12:0/2242835627 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd3c108840 msgr2=0x7fcd3c19ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:22.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.127+0000 7fcd42fa6700 1 --2- 172.21.15.12:0/2242835627 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd3c108840 0x7fcd3c19ad30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fcd2800d950 tx=0x7fcd2800dd10 comp rx=0 tx=0).stop 2024-05-17T22:37:22.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.127+0000 7fcd42fa6700 1 -- 172.21.15.12:0/2242835627 shutdown_connections 2024-05-17T22:37:22.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.127+0000 7fcd42fa6700 1 --2- 172.21.15.12:0/2242835627 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fcd2c0b2880 0x7fcd2c0b4d90 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:22.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.127+0000 7fcd42fa6700 1 --2- 172.21.15.12:0/2242835627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd3c109190 0x7fcd3c19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:22.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.127+0000 7fcd42fa6700 1 --2- 172.21.15.12:0/2242835627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd3c103f10 0x7fcd3c19a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:22.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.127+0000 7fcd42fa6700 1 --2- 172.21.15.12:0/2242835627 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd3c108840 0x7fcd3c19ad30 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:22.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.127+0000 7fcd42fa6700 1 -- 172.21.15.12:0/2242835627 >> 172.21.15.12:0/2242835627 conn(0x7fcd3c0ff990 msgr2=0x7fcd3c10a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:22.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.128+0000 7fcd42fa6700 1 -- 172.21.15.12:0/2242835627 shutdown_connections 2024-05-17T22:37:22.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.128+0000 7fcd42fa6700 1 -- 172.21.15.12:0/2242835627 wait complete. 2024-05-17T22:37:22.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:21 smithi151 ceph-mon[36101]: pgmap v382: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-17T22:37:22.287 INFO:teuthology.orchestra.run.smithi012.stdout:55834574947 2024-05-17T22:37:22.288 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-17T22:37:22.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:21 smithi096 ceph-mon[36065]: pgmap v382: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-17T22:37:22.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.477+0000 7fc188698700 1 -- 172.21.15.12:0/1427547721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc180105770 msgr2=0x7fc18010dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:22.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.477+0000 7fc188698700 1 --2- 172.21.15.12:0/1427547721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc180105770 0x7fc18010dcc0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc17c009990 tx=0x7fc17c009ca0 comp rx=0 tx=0).stop 2024-05-17T22:37:22.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.477+0000 7fc188698700 1 -- 172.21.15.12:0/1427547721 shutdown_connections 2024-05-17T22:37:22.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.477+0000 7fc188698700 1 --2- 172.21.15.12:0/1427547721 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc18010e250 0x7fc180110760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:22.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.477+0000 7fc188698700 1 --2- 172.21.15.12:0/1427547721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc180104e20 0x7fc180105230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:22.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.477+0000 7fc188698700 1 --2- 172.21.15.12:0/1427547721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc180105770 0x7fc18010dcc0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:22.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.477+0000 7fc188698700 1 -- 172.21.15.12:0/1427547721 >> 172.21.15.12:0/1427547721 conn(0x7fc180079b50 msgr2=0x7fc180079f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:22.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.477+0000 7fc188698700 1 -- 172.21.15.12:0/1427547721 shutdown_connections 2024-05-17T22:37:22.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.477+0000 7fc188698700 1 -- 172.21.15.12:0/1427547721 wait complete. 2024-05-17T22:37:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.478+0000 7fc188698700 1 Processor -- start 2024-05-17T22:37:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.478+0000 7fc188698700 1 -- start start 2024-05-17T22:37:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.478+0000 7fc188698700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc180104e20 0x7fc180102b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.478+0000 7fc188698700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc180105770 0x7fc180103050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.478+0000 7fc188698700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc18010e250 0x7fc180103590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.478+0000 7fc188698700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc180113df0 con 0x7fc180105770 2024-05-17T22:37:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.478+0000 7fc188698700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc180113c70 con 0x7fc18010e250 2024-05-17T22:37:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.478+0000 7fc188698700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc180113f70 con 0x7fc180104e20 2024-05-17T22:37:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.479+0000 7fc185c33700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc180105770 0x7fc180103050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.479+0000 7fc185c33700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc180105770 0x7fc180103050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34128/0 (socket says 172.21.15.12:34128) 2024-05-17T22:37:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.479+0000 7fc185c33700 1 -- 172.21.15.12:0/3857531268 learned_addr learned my addr 172.21.15.12:0/3857531268 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:37:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.479+0000 7fc186434700 1 --2- 172.21.15.12:0/3857531268 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc180104e20 0x7fc180102b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.479+0000 7fc186c35700 1 --2- 172.21.15.12:0/3857531268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc18010e250 0x7fc180103590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.479+0000 7fc185c33700 1 -- 172.21.15.12:0/3857531268 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc180104e20 msgr2=0x7fc180102b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.479+0000 7fc185c33700 1 --2- 172.21.15.12:0/3857531268 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc180104e20 0x7fc180102b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.479+0000 7fc185c33700 1 -- 172.21.15.12:0/3857531268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc18010e250 msgr2=0x7fc180103590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.479+0000 7fc185c33700 1 --2- 172.21.15.12:0/3857531268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc18010e250 0x7fc180103590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.479+0000 7fc185c33700 1 -- 172.21.15.12:0/3857531268 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1801a8ae0 con 0x7fc180105770 2024-05-17T22:37:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.479+0000 7fc186c35700 1 --2- 172.21.15.12:0/3857531268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc18010e250 0x7fc180103590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:37:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.479+0000 7fc185c33700 1 --2- 172.21.15.12:0/3857531268 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc180105770 0x7fc180103050 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc17c00b5c0 tx=0x7fc17c005990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.479+0000 7fc1737fe700 1 -- 172.21.15.12:0/3857531268 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c027070 con 0x7fc180105770 2024-05-17T22:37:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.479+0000 7fc1737fe700 1 -- 172.21.15.12:0/3857531268 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc17c003fb0 con 0x7fc180105770 2024-05-17T22:37:22.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.480+0000 7fc1737fe700 1 -- 172.21.15.12:0/3857531268 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c022d40 con 0x7fc180105770 2024-05-17T22:37:22.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.480+0000 7fc188698700 1 -- 172.21.15.12:0/3857531268 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1801a8d70 con 0x7fc180105770 2024-05-17T22:37:22.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.480+0000 7fc188698700 1 -- 172.21.15.12:0/3857531268 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1801a91a0 con 0x7fc180105770 2024-05-17T22:37:22.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.481+0000 7fc188698700 1 -- 172.21.15.12:0/3857531268 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc180105230 con 0x7fc180105770 2024-05-17T22:37:22.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.483+0000 7fc1737fe700 1 -- 172.21.15.12:0/3857531268 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc17c02a070 con 0x7fc180105770 2024-05-17T22:37:22.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.484+0000 7fc1737fe700 1 --2- 172.21.15.12:0/3857531268 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fc16c0b2880 0x7fc16c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:22.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.484+0000 7fc186434700 1 --2- 172.21.15.12:0/3857531268 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fc16c0b2880 0x7fc16c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:22.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.484+0000 7fc1737fe700 1 -- 172.21.15.12:0/3857531268 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fc17c0e3b30 con 0x7fc180105770 2024-05-17T22:37:22.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.485+0000 7fc186434700 1 --2- 172.21.15.12:0/3857531268 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fc16c0b2880 0x7fc16c0b4d90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc174005950 tx=0x7fc1740058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:22.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.487+0000 7fc1737fe700 1 -- 172.21.15.12:0/3857531268 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc17c031020 con 0x7fc180105770 2024-05-17T22:37:22.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.710+0000 7fc188698700 1 -- 172.21.15.12:0/3857531268 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fc180064440 con 0x7fc180105770 2024-05-17T22:37:22.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.710+0000 7fc1737fe700 1 -- 172.21.15.12:0/3857531268 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc17c013070 con 0x7fc180105770 2024-05-17T22:37:22.714 INFO:teuthology.orchestra.run.smithi012.stdout:154618822736 2024-05-17T22:37:22.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.713+0000 7fc188698700 1 -- 172.21.15.12:0/3857531268 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fc16c0b2880 msgr2=0x7fc16c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:22.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.713+0000 7fc188698700 1 --2- 172.21.15.12:0/3857531268 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fc16c0b2880 0x7fc16c0b4d90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc174005950 tx=0x7fc1740058e0 comp rx=0 tx=0).stop 2024-05-17T22:37:22.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.713+0000 7fc188698700 1 -- 172.21.15.12:0/3857531268 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc180105770 msgr2=0x7fc180103050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:22.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.713+0000 7fc188698700 1 --2- 172.21.15.12:0/3857531268 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc180105770 0x7fc180103050 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc17c00b5c0 tx=0x7fc17c005990 comp rx=0 tx=0).stop 2024-05-17T22:37:22.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.713+0000 7fc188698700 1 -- 172.21.15.12:0/3857531268 shutdown_connections 2024-05-17T22:37:22.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.713+0000 7fc188698700 1 --2- 172.21.15.12:0/3857531268 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fc16c0b2880 0x7fc16c0b4d90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:22.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.713+0000 7fc188698700 1 --2- 172.21.15.12:0/3857531268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc18010e250 0x7fc180103590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:22.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.713+0000 7fc188698700 1 --2- 172.21.15.12:0/3857531268 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc180104e20 0x7fc180102b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:22.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.713+0000 7fc188698700 1 --2- 172.21.15.12:0/3857531268 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc180105770 0x7fc180103050 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:22.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.713+0000 7fc188698700 1 -- 172.21.15.12:0/3857531268 >> 172.21.15.12:0/3857531268 conn(0x7fc180079b50 msgr2=0x7fc1800ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:22.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.713+0000 7fc188698700 1 -- 172.21.15.12:0/3857531268 shutdown_connections 2024-05-17T22:37:22.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:22.713+0000 7fc188698700 1 -- 172.21.15.12:0/3857531268 wait complete. 2024-05-17T22:37:23.025 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:22 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/2242835627' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-17T22:37:23.025 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:22 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/3857531268' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-17T22:37:23.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:22 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/2242835627' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-17T22:37:23.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:22 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/3857531268' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-17T22:37:23.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:22 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/2242835627' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-17T22:37:23.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:22 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/3857531268' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-17T22:37:24.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:23 smithi012 ceph-mon[31632]: pgmap v383: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-17T22:37:24.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:23 smithi151 ceph-mon[36101]: pgmap v383: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-17T22:37:24.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:23 smithi096 ceph-mon[36065]: pgmap v383: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-17T22:37:24.823 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738468 got 34359738473 for osd.0 2024-05-17T22:37:24.823 DEBUG:teuthology.parallel:result is None 2024-05-17T22:37:25.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.021+0000 7f925a168700 1 -- 172.21.15.12:0/2932978185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9254102e20 msgr2=0x7f9254103230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:25.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.021+0000 7f925a168700 1 --2- 172.21.15.12:0/2932978185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9254102e20 0x7f9254103230 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9244009990 tx=0x7f9244009ca0 comp rx=0 tx=0).stop 2024-05-17T22:37:25.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.021+0000 7f925a168700 1 -- 172.21.15.12:0/2932978185 shutdown_connections 2024-05-17T22:37:25.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.021+0000 7f925a168700 1 --2- 172.21.15.12:0/2932978185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92541080a0 0x7f925410fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:25.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.021+0000 7f925a168700 1 --2- 172.21.15.12:0/2932978185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9254102e20 0x7f9254103230 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:25.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.021+0000 7f925a168700 1 --2- 172.21.15.12:0/2932978185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9254107750 0x7f9254107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:25.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.021+0000 7f925a168700 1 -- 172.21.15.12:0/2932978185 >> 172.21.15.12:0/2932978185 conn(0x7f9254079bd0 msgr2=0x7f9254079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:25.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.021+0000 7f925a168700 1 -- 172.21.15.12:0/2932978185 shutdown_connections 2024-05-17T22:37:25.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.021+0000 7f925a168700 1 -- 172.21.15.12:0/2932978185 wait complete. 2024-05-17T22:37:25.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.022+0000 7f925a168700 1 Processor -- start 2024-05-17T22:37:25.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.022+0000 7f925a168700 1 -- start start 2024-05-17T22:37:25.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.022+0000 7f925a168700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9254102e20 0x7f9254199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:25.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.022+0000 7f925a168700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9254107750 0x7f92541999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:25.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.022+0000 7f925a168700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92541080a0 0x7f9254199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:25.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.022+0000 7f925a168700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9254112d50 con 0x7f92541080a0 2024-05-17T22:37:25.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.022+0000 7f925a168700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9254112bd0 con 0x7f9254102e20 2024-05-17T22:37:25.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.022+0000 7f925a168700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9254112ed0 con 0x7f9254107750 2024-05-17T22:37:25.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.023+0000 7f9253fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92541080a0 0x7f9254199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:25.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.023+0000 7f9253fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92541080a0 0x7f9254199f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34152/0 (socket says 172.21.15.12:34152) 2024-05-17T22:37:25.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.023+0000 7f9253fff700 1 -- 172.21.15.12:0/3104203695 learned_addr learned my addr 172.21.15.12:0/3104203695 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:37:25.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.023+0000 7f92537fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9254102e20 0x7f9254199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:25.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.023+0000 7f9252ffd700 1 --2- 172.21.15.12:0/3104203695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9254107750 0x7f92541999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:25.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.023+0000 7f9253fff700 1 -- 172.21.15.12:0/3104203695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9254107750 msgr2=0x7f92541999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:25.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.023+0000 7f9253fff700 1 --2- 172.21.15.12:0/3104203695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9254107750 0x7f92541999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:25.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.023+0000 7f9253fff700 1 -- 172.21.15.12:0/3104203695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9254102e20 msgr2=0x7f9254199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:25.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.023+0000 7f9253fff700 1 --2- 172.21.15.12:0/3104203695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9254102e20 0x7f9254199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:25.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.023+0000 7f9253fff700 1 -- 172.21.15.12:0/3104203695 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92541a8b90 con 0x7f92541080a0 2024-05-17T22:37:25.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.023+0000 7f9253fff700 1 --2- 172.21.15.12:0/3104203695 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92541080a0 0x7f9254199f10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f924c00ada0 tx=0x7f924c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:25.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.023+0000 7f9250ff9700 1 -- 172.21.15.12:0/3104203695 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924c017020 con 0x7f92541080a0 2024-05-17T22:37:25.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.023+0000 7f9250ff9700 1 -- 172.21.15.12:0/3104203695 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f924c00cd80 con 0x7f92541080a0 2024-05-17T22:37:25.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.023+0000 7f925a168700 1 -- 172.21.15.12:0/3104203695 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92541a8e80 con 0x7f92541080a0 2024-05-17T22:37:25.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.024+0000 7f9250ff9700 1 -- 172.21.15.12:0/3104203695 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924c005260 con 0x7f92541080a0 2024-05-17T22:37:25.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.024+0000 7f925a168700 1 -- 172.21.15.12:0/3104203695 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92541a92e0 con 0x7f92541080a0 2024-05-17T22:37:25.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.024+0000 7f925a168700 1 -- 172.21.15.12:0/3104203695 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92541042d0 con 0x7f92541080a0 2024-05-17T22:37:25.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.026+0000 7f9250ff9700 1 -- 172.21.15.12:0/3104203695 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f924c00a090 con 0x7f92541080a0 2024-05-17T22:37:25.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.027+0000 7f9250ff9700 1 --2- 172.21.15.12:0/3104203695 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f923c0b2880 0x7f923c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:25.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.027+0000 7f92537fe700 1 --2- 172.21.15.12:0/3104203695 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f923c0b2880 0x7f923c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:25.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.027+0000 7f9250ff9700 1 -- 172.21.15.12:0/3104203695 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f924c0e2260 con 0x7f92541080a0 2024-05-17T22:37:25.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.028+0000 7f92537fe700 1 --2- 172.21.15.12:0/3104203695 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f923c0b2880 0x7f923c0b4d90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9244009750 tx=0x7f924401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:25.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.031+0000 7f9250ff9700 1 -- 172.21.15.12:0/3104203695 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f924c021020 con 0x7f92541080a0 2024-05-17T22:37:25.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.255+0000 7f925a168700 1 -- 172.21.15.12:0/3104203695 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f925406c630 con 0x7f92541080a0 2024-05-17T22:37:25.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.255+0000 7f9250ff9700 1 -- 172.21.15.12:0/3104203695 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f924c0a9c80 con 0x7f92541080a0 2024-05-17T22:37:25.258 INFO:teuthology.orchestra.run.smithi012.stdout:107374182491 2024-05-17T22:37:25.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.258+0000 7f925a168700 1 -- 172.21.15.12:0/3104203695 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f923c0b2880 msgr2=0x7f923c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:25.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.258+0000 7f925a168700 1 --2- 172.21.15.12:0/3104203695 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f923c0b2880 0x7f923c0b4d90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9244009750 tx=0x7f924401a040 comp rx=0 tx=0).stop 2024-05-17T22:37:25.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.258+0000 7f925a168700 1 -- 172.21.15.12:0/3104203695 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92541080a0 msgr2=0x7f9254199f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:25.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.258+0000 7f925a168700 1 --2- 172.21.15.12:0/3104203695 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92541080a0 0x7f9254199f10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f924c00ada0 tx=0x7f924c00c330 comp rx=0 tx=0).stop 2024-05-17T22:37:25.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.258+0000 7f925a168700 1 -- 172.21.15.12:0/3104203695 shutdown_connections 2024-05-17T22:37:25.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.258+0000 7f925a168700 1 --2- 172.21.15.12:0/3104203695 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f923c0b2880 0x7f923c0b4d90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:25.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.258+0000 7f925a168700 1 --2- 172.21.15.12:0/3104203695 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92541080a0 0x7f9254199f10 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:25.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.258+0000 7f925a168700 1 --2- 172.21.15.12:0/3104203695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9254102e20 0x7f9254199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:25.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.258+0000 7f925a168700 1 --2- 172.21.15.12:0/3104203695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9254107750 0x7f92541999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:25.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.258+0000 7f925a168700 1 -- 172.21.15.12:0/3104203695 >> 172.21.15.12:0/3104203695 conn(0x7f9254079bd0 msgr2=0x7f9254105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:25.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.258+0000 7f925a168700 1 -- 172.21.15.12:0/3104203695 shutdown_connections 2024-05-17T22:37:25.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:25.258+0000 7f925a168700 1 -- 172.21.15.12:0/3104203695 wait complete. 2024-05-17T22:37:26.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:25 smithi012 ceph-mon[31632]: pgmap v384: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-17T22:37:26.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:25 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/3104203695' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-17T22:37:26.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:25 smithi151 ceph-mon[36101]: pgmap v384: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-17T22:37:26.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:25 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/3104203695' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-17T22:37:26.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:25 smithi096 ceph-mon[36065]: pgmap v384: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-17T22:37:26.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:25 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/3104203695' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-17T22:37:27.386 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182485 got 107374182491 for osd.3 2024-05-17T22:37:27.387 DEBUG:teuthology.parallel:result is None 2024-05-17T22:37:28.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:27 smithi151 ceph-mon[36101]: pgmap v385: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-17T22:37:28.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:27 smithi096 ceph-mon[36065]: pgmap v385: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-17T22:37:28.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:27 smithi012 ceph-mon[31632]: pgmap v385: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-17T22:37:30.039 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:29 smithi012 ceph-mon[31632]: pgmap v386: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T22:37:30.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.155+0000 7fa618f62700 1 -- 172.21.15.12:0/1801645457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa614108080 msgr2=0x7fa61410f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:30.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.155+0000 7fa618f62700 1 --2- 172.21.15.12:0/1801645457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa614108080 0x7fa61410f9e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa60c00b0a0 tx=0x7fa60c00b3b0 comp rx=0 tx=0).stop 2024-05-17T22:37:30.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.155+0000 7fa618f62700 1 -- 172.21.15.12:0/1801645457 shutdown_connections 2024-05-17T22:37:30.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.155+0000 7fa618f62700 1 --2- 172.21.15.12:0/1801645457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa614108080 0x7fa61410f9e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:30.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.155+0000 7fa618f62700 1 --2- 172.21.15.12:0/1801645457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa614102e00 0x7fa614103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:30.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.155+0000 7fa618f62700 1 --2- 172.21.15.12:0/1801645457 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa614107730 0x7fa614107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:30.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.155+0000 7fa618f62700 1 -- 172.21.15.12:0/1801645457 >> 172.21.15.12:0/1801645457 conn(0x7fa614079bd0 msgr2=0x7fa614079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:30.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.155+0000 7fa618f62700 1 -- 172.21.15.12:0/1801645457 shutdown_connections 2024-05-17T22:37:30.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.155+0000 7fa618f62700 1 -- 172.21.15.12:0/1801645457 wait complete. 2024-05-17T22:37:30.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.156+0000 7fa618f62700 1 Processor -- start 2024-05-17T22:37:30.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.156+0000 7fa618f62700 1 -- start start 2024-05-17T22:37:30.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.156+0000 7fa618f62700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa614102e00 0x7fa61419a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:30.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.156+0000 7fa618f62700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa614107730 0x7fa61419ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:30.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.156+0000 7fa618f62700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa614108080 0x7fa61419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:30.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.156+0000 7fa618f62700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa614112cc0 con 0x7fa614108080 2024-05-17T22:37:30.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.156+0000 7fa618f62700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa614112b40 con 0x7fa614102e00 2024-05-17T22:37:30.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.156+0000 7fa618f62700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa614112e40 con 0x7fa614107730 2024-05-17T22:37:30.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.156+0000 7fa612d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa614108080 0x7fa61419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:30.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.156+0000 7fa612d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa614108080 0x7fa61419efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40292/0 (socket says 172.21.15.12:40292) 2024-05-17T22:37:30.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.156+0000 7fa612d9d700 1 -- 172.21.15.12:0/991385104 learned_addr learned my addr 172.21.15.12:0/991385104 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:37:30.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.156+0000 7fa61259c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa614102e00 0x7fa61419a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:30.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.156+0000 7fa611d9b700 1 --2- 172.21.15.12:0/991385104 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa614107730 0x7fa61419ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:30.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.157+0000 7fa612d9d700 1 -- 172.21.15.12:0/991385104 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa614107730 msgr2=0x7fa61419ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:30.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.157+0000 7fa612d9d700 1 --2- 172.21.15.12:0/991385104 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa614107730 0x7fa61419ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:30.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.157+0000 7fa612d9d700 1 -- 172.21.15.12:0/991385104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa614102e00 msgr2=0x7fa61419a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:30.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.157+0000 7fa612d9d700 1 --2- 172.21.15.12:0/991385104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa614102e00 0x7fa61419a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:30.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.157+0000 7fa612d9d700 1 -- 172.21.15.12:0/991385104 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa61419f640 con 0x7fa614108080 2024-05-17T22:37:30.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.157+0000 7fa612d9d700 1 --2- 172.21.15.12:0/991385104 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa614108080 0x7fa61419efa0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa60c00b830 tx=0x7fa60c004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:30.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.157+0000 7fa6037fe700 1 -- 172.21.15.12:0/991385104 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60c021070 con 0x7fa614108080 2024-05-17T22:37:30.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.157+0000 7fa6037fe700 1 -- 172.21.15.12:0/991385104 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa60c007e30 con 0x7fa614108080 2024-05-17T22:37:30.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.157+0000 7fa6037fe700 1 -- 172.21.15.12:0/991385104 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60c004700 con 0x7fa614108080 2024-05-17T22:37:30.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.157+0000 7fa618f62700 1 -- 172.21.15.12:0/991385104 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa61419f8d0 con 0x7fa614108080 2024-05-17T22:37:30.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.157+0000 7fa618f62700 1 -- 172.21.15.12:0/991385104 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa61419fd50 con 0x7fa614108080 2024-05-17T22:37:30.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.158+0000 7fa618f62700 1 -- 172.21.15.12:0/991385104 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa614103210 con 0x7fa614108080 2024-05-17T22:37:30.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.160+0000 7fa6037fe700 1 -- 172.21.15.12:0/991385104 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa60c003d70 con 0x7fa614108080 2024-05-17T22:37:30.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.161+0000 7fa6037fe700 1 --2- 172.21.15.12:0/991385104 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa5fc0b2880 0x7fa5fc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:30.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.161+0000 7fa61259c700 1 --2- 172.21.15.12:0/991385104 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa5fc0b2880 0x7fa5fc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:30.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.161+0000 7fa6037fe700 1 -- 172.21.15.12:0/991385104 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fa60c00f020 con 0x7fa614108080 2024-05-17T22:37:30.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.162+0000 7fa61259c700 1 --2- 172.21.15.12:0/991385104 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa5fc0b2880 0x7fa5fc0b4d90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa604005950 tx=0x7fa60400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:30.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.164+0000 7fa6037fe700 1 -- 172.21.15.12:0/991385104 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa60c02a020 con 0x7fa614108080 2024-05-17T22:37:30.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:29 smithi151 ceph-mon[36101]: pgmap v386: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T22:37:30.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:29 smithi096 ceph-mon[36065]: pgmap v386: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T22:37:30.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.346+0000 7fdd17824700 1 -- 172.21.15.12:0/1262453232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd10102e00 msgr2=0x7fdd10103210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:30.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.346+0000 7fdd17824700 1 --2- 172.21.15.12:0/1262453232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd10102e00 0x7fdd10103210 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c009990 tx=0x7fdd0c009ca0 comp rx=0 tx=0).stop 2024-05-17T22:37:30.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.346+0000 7fdd17824700 1 -- 172.21.15.12:0/1262453232 shutdown_connections 2024-05-17T22:37:30.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.346+0000 7fdd17824700 1 --2- 172.21.15.12:0/1262453232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd10108080 0x7fdd1010f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:30.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.347+0000 7fdd17824700 1 --2- 172.21.15.12:0/1262453232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd10102e00 0x7fdd10103210 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:30.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.347+0000 7fdd17824700 1 --2- 172.21.15.12:0/1262453232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd10107730 0x7fdd10107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:30.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.347+0000 7fdd17824700 1 -- 172.21.15.12:0/1262453232 >> 172.21.15.12:0/1262453232 conn(0x7fdd10079bd0 msgr2=0x7fdd10079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:30.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.347+0000 7fdd17824700 1 -- 172.21.15.12:0/1262453232 shutdown_connections 2024-05-17T22:37:30.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.347+0000 7fdd17824700 1 -- 172.21.15.12:0/1262453232 wait complete. 2024-05-17T22:37:30.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.348+0000 7fdd17824700 1 Processor -- start 2024-05-17T22:37:30.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.348+0000 7fdd17824700 1 -- start start 2024-05-17T22:37:30.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.348+0000 7fdd17824700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd10102e00 0x7fdd101a25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:30.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.348+0000 7fdd17824700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd10107730 0x7fdd101a2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:30.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.348+0000 7fdd17824700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd10108080 0x7fdd1019c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:30.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.348+0000 7fdd17824700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdd10112cb0 con 0x7fdd10102e00 2024-05-17T22:37:30.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.348+0000 7fdd155c0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd10102e00 0x7fdd101a25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:30.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.349+0000 7fdd17824700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdd10112b30 con 0x7fdd10108080 2024-05-17T22:37:30.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.349+0000 7fdd17824700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdd10112e30 con 0x7fdd10107730 2024-05-17T22:37:30.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.349+0000 7fdd155c0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd10102e00 0x7fdd101a25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40300/0 (socket says 172.21.15.12:40300) 2024-05-17T22:37:30.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.349+0000 7fdd155c0700 1 -- 172.21.15.12:0/1310631587 learned_addr learned my addr 172.21.15.12:0/1310631587 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:37:30.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.349+0000 7fdd14dbf700 1 --2- 172.21.15.12:0/1310631587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd10107730 0x7fdd101a2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:30.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.349+0000 7fdd15dc1700 1 --2- 172.21.15.12:0/1310631587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd10108080 0x7fdd1019c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:30.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.349+0000 7fdd155c0700 1 -- 172.21.15.12:0/1310631587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd10107730 msgr2=0x7fdd101a2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:30.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.349+0000 7fdd155c0700 1 --2- 172.21.15.12:0/1310631587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd10107730 0x7fdd101a2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:30.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.349+0000 7fdd155c0700 1 -- 172.21.15.12:0/1310631587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd10108080 msgr2=0x7fdd1019c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:30.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.349+0000 7fdd155c0700 1 --2- 172.21.15.12:0/1310631587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd10108080 0x7fdd1019c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:30.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.349+0000 7fdd155c0700 1 -- 172.21.15.12:0/1310631587 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd1019cf40 con 0x7fdd10102e00 2024-05-17T22:37:30.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.349+0000 7fdd155c0700 1 --2- 172.21.15.12:0/1310631587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd10102e00 0x7fdd101a25d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c000c00 tx=0x7fdd0c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:30.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.350+0000 7fdd067fc700 1 -- 172.21.15.12:0/1310631587 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c027070 con 0x7fdd10102e00 2024-05-17T22:37:30.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.350+0000 7fdd067fc700 1 -- 172.21.15.12:0/1310631587 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd0c003f20 con 0x7fdd10102e00 2024-05-17T22:37:30.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.350+0000 7fdd17824700 1 -- 172.21.15.12:0/1310631587 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd1019d1d0 con 0x7fdd10102e00 2024-05-17T22:37:30.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.350+0000 7fdd067fc700 1 -- 172.21.15.12:0/1310631587 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c022ca0 con 0x7fdd10102e00 2024-05-17T22:37:30.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.350+0000 7fdd17824700 1 -- 172.21.15.12:0/1310631587 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd101a9430 con 0x7fdd10102e00 2024-05-17T22:37:30.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.351+0000 7fdd17824700 1 -- 172.21.15.12:0/1310631587 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd1019d6b0 con 0x7fdd10102e00 2024-05-17T22:37:30.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.354+0000 7fdd067fc700 1 -- 172.21.15.12:0/1310631587 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fdd0c02a070 con 0x7fdd10102e00 2024-05-17T22:37:30.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.354+0000 7fdd067fc700 1 --2- 172.21.15.12:0/1310631587 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fdcfc0b2880 0x7fdcfc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:30.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.355+0000 7fdd067fc700 1 -- 172.21.15.12:0/1310631587 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fdd0c0e3340 con 0x7fdd10102e00 2024-05-17T22:37:30.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.355+0000 7fdd14dbf700 1 --2- 172.21.15.12:0/1310631587 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fdcfc0b2880 0x7fdcfc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:30.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.355+0000 7fdd14dbf700 1 --2- 172.21.15.12:0/1310631587 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fdcfc0b2880 0x7fdcfc0b4d90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdd1007a090 tx=0x7fdd0000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:30.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.359+0000 7fdd067fc700 1 -- 172.21.15.12:0/1310631587 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdd0c013070 con 0x7fdd10102e00 2024-05-17T22:37:30.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.389+0000 7fa618f62700 1 -- 172.21.15.12:0/991385104 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fa61419bad0 con 0x7fa614108080 2024-05-17T22:37:30.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.389+0000 7fa6037fe700 1 -- 172.21.15.12:0/991385104 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa60c0ab8c0 con 0x7fa614108080 2024-05-17T22:37:30.393 INFO:teuthology.orchestra.run.smithi012.stdout:201863462982 2024-05-17T22:37:30.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.392+0000 7fa618f62700 1 -- 172.21.15.12:0/991385104 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa5fc0b2880 msgr2=0x7fa5fc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:30.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.392+0000 7fa618f62700 1 --2- 172.21.15.12:0/991385104 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa5fc0b2880 0x7fa5fc0b4d90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa604005950 tx=0x7fa60400b410 comp rx=0 tx=0).stop 2024-05-17T22:37:30.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.392+0000 7fa618f62700 1 -- 172.21.15.12:0/991385104 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa614108080 msgr2=0x7fa61419efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:30.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.392+0000 7fa618f62700 1 --2- 172.21.15.12:0/991385104 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa614108080 0x7fa61419efa0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa60c00b830 tx=0x7fa60c004ce0 comp rx=0 tx=0).stop 2024-05-17T22:37:30.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.392+0000 7fa618f62700 1 -- 172.21.15.12:0/991385104 shutdown_connections 2024-05-17T22:37:30.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.392+0000 7fa618f62700 1 --2- 172.21.15.12:0/991385104 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa5fc0b2880 0x7fa5fc0b4d90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:30.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.392+0000 7fa618f62700 1 --2- 172.21.15.12:0/991385104 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa614108080 0x7fa61419efa0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:30.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.392+0000 7fa618f62700 1 --2- 172.21.15.12:0/991385104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa614102e00 0x7fa61419a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:30.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.392+0000 7fa618f62700 1 --2- 172.21.15.12:0/991385104 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa614107730 0x7fa61419ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:30.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.392+0000 7fa618f62700 1 -- 172.21.15.12:0/991385104 >> 172.21.15.12:0/991385104 conn(0x7fa614079bd0 msgr2=0x7fa61406b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:30.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.392+0000 7fa618f62700 1 -- 172.21.15.12:0/991385104 shutdown_connections 2024-05-17T22:37:30.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.392+0000 7fa618f62700 1 -- 172.21.15.12:0/991385104 wait complete. 2024-05-17T22:37:30.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.591+0000 7fdd17824700 1 -- 172.21.15.12:0/1310631587 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fdd1019dd20 con 0x7fdd10102e00 2024-05-17T22:37:30.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.591+0000 7fdd067fc700 1 -- 172.21.15.12:0/1310631587 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fdd0c0a4e20 con 0x7fdd10102e00 2024-05-17T22:37:30.595 INFO:teuthology.orchestra.run.smithi012.stdout:274877906999 2024-05-17T22:37:30.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.594+0000 7fdd17824700 1 -- 172.21.15.12:0/1310631587 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fdcfc0b2880 msgr2=0x7fdcfc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:30.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.594+0000 7fdd17824700 1 --2- 172.21.15.12:0/1310631587 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fdcfc0b2880 0x7fdcfc0b4d90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdd1007a090 tx=0x7fdd0000b410 comp rx=0 tx=0).stop 2024-05-17T22:37:30.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.594+0000 7fdd17824700 1 -- 172.21.15.12:0/1310631587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd10102e00 msgr2=0x7fdd101a25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:30.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.594+0000 7fdd17824700 1 --2- 172.21.15.12:0/1310631587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd10102e00 0x7fdd101a25d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c000c00 tx=0x7fdd0c0047b0 comp rx=0 tx=0).stop 2024-05-17T22:37:30.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.594+0000 7fdd17824700 1 -- 172.21.15.12:0/1310631587 shutdown_connections 2024-05-17T22:37:30.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.594+0000 7fdd17824700 1 --2- 172.21.15.12:0/1310631587 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fdcfc0b2880 0x7fdcfc0b4d90 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:30.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.594+0000 7fdd17824700 1 --2- 172.21.15.12:0/1310631587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd10108080 0x7fdd1019c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:30.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.594+0000 7fdd17824700 1 --2- 172.21.15.12:0/1310631587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd10102e00 0x7fdd101a25d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:30.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.594+0000 7fdd17824700 1 --2- 172.21.15.12:0/1310631587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd10107730 0x7fdd101a2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:30.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.594+0000 7fdd17824700 1 -- 172.21.15.12:0/1310631587 >> 172.21.15.12:0/1310631587 conn(0x7fdd10079bd0 msgr2=0x7fdd1006b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:30.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.594+0000 7fdd17824700 1 -- 172.21.15.12:0/1310631587 shutdown_connections 2024-05-17T22:37:30.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.594+0000 7fdd17824700 1 -- 172.21.15.12:0/1310631587 wait complete. 2024-05-17T22:37:31.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:30 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/991385104' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-17T22:37:31.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:30 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/1310631587' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-17T22:37:31.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:30 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/484941370' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-17T22:37:31.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:30 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/991385104' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-17T22:37:31.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:30 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/1310631587' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-17T22:37:31.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:30 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/484941370' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-17T22:37:31.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:30 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/991385104' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-17T22:37:31.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:30 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/1310631587' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-17T22:37:31.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:30 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/484941370' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-17T22:37:31.892 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822733 got 154618822736 for osd.5 2024-05-17T22:37:31.892 DEBUG:teuthology.parallel:result is None 2024-05-17T22:37:32.181 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:31 smithi012 ceph-mon[31632]: pgmap v387: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-17T22:37:32.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:31 smithi151 ceph-mon[36101]: pgmap v387: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-17T22:37:32.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:31 smithi096 ceph-mon[36065]: pgmap v387: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-17T22:37:32.616 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462979 got 201863462982 for osd.7 2024-05-17T22:37:32.617 DEBUG:teuthology.parallel:result is None 2024-05-17T22:37:32.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.574+0000 7f5a33c6d700 1 -- 172.21.15.12:0/1157304570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a2c103f10 msgr2=0x7f5a2c104320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:32.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.574+0000 7f5a33c6d700 1 --2- 172.21.15.12:0/1157304570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a2c103f10 0x7f5a2c104320 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5a28009990 tx=0x7f5a28009ca0 comp rx=0 tx=0).stop 2024-05-17T22:37:32.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.574+0000 7f5a33c6d700 1 -- 172.21.15.12:0/1157304570 shutdown_connections 2024-05-17T22:37:32.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.574+0000 7f5a33c6d700 1 --2- 172.21.15.12:0/1157304570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a2c109190 0x7f5a2c110af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:32.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.574+0000 7f5a33c6d700 1 --2- 172.21.15.12:0/1157304570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a2c103f10 0x7f5a2c104320 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:32.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.574+0000 7f5a33c6d700 1 --2- 172.21.15.12:0/1157304570 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a2c108840 0x7f5a2c108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:32.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.574+0000 7f5a33c6d700 1 -- 172.21.15.12:0/1157304570 >> 172.21.15.12:0/1157304570 conn(0x7f5a2c0ff990 msgr2=0x7f5a2c101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:32.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.574+0000 7f5a33c6d700 1 -- 172.21.15.12:0/1157304570 shutdown_connections 2024-05-17T22:37:32.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.575+0000 7f5a33c6d700 1 -- 172.21.15.12:0/1157304570 wait complete. 2024-05-17T22:37:32.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.575+0000 7f5a33c6d700 1 Processor -- start 2024-05-17T22:37:32.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.575+0000 7f5a33c6d700 1 -- start start 2024-05-17T22:37:32.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.575+0000 7f5a33c6d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a2c103f10 0x7f5a2c1a04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:32.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.576+0000 7f5a33c6d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a2c108840 0x7f5a2c1a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:32.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.576+0000 7f5a33c6d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a2c109190 0x7f5a2c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:32.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.576+0000 7f5a33c6d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2c113e20 con 0x7f5a2c109190 2024-05-17T22:37:32.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.576+0000 7f5a33c6d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2c113ca0 con 0x7f5a2c103f10 2024-05-17T22:37:32.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.576+0000 7f5a33c6d700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2c113fa0 con 0x7f5a2c108840 2024-05-17T22:37:32.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.576+0000 7f5a3220a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a2c109190 0x7f5a2c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:32.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.576+0000 7f5a3220a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a2c109190 0x7f5a2c19a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40326/0 (socket says 172.21.15.12:40326) 2024-05-17T22:37:32.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.576+0000 7f5a3220a700 1 -- 172.21.15.12:0/484941370 learned_addr learned my addr 172.21.15.12:0/484941370 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:37:32.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.576+0000 7f5a31208700 1 --2- 172.21.15.12:0/484941370 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a2c108840 0x7f5a2c1a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:32.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.576+0000 7f5a31a09700 1 --2- 172.21.15.12:0/484941370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a2c103f10 0x7f5a2c1a04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:32.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.576+0000 7f5a3220a700 1 -- 172.21.15.12:0/484941370 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a2c108840 msgr2=0x7f5a2c1a0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:32.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.576+0000 7f5a3220a700 1 --2- 172.21.15.12:0/484941370 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a2c108840 0x7f5a2c1a0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:32.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.576+0000 7f5a3220a700 1 -- 172.21.15.12:0/484941370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a2c103f10 msgr2=0x7f5a2c1a04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:32.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.576+0000 7f5a3220a700 1 --2- 172.21.15.12:0/484941370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a2c103f10 0x7f5a2c1a04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:32.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.576+0000 7f5a3220a700 1 -- 172.21.15.12:0/484941370 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a2c19ad10 con 0x7f5a2c109190 2024-05-17T22:37:32.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.576+0000 7f5a3220a700 1 --2- 172.21.15.12:0/484941370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a2c109190 0x7f5a2c19a670 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5a24007d70 tx=0x7f5a2400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:32.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.576+0000 7f5a1effd700 1 -- 172.21.15.12:0/484941370 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a24016020 con 0x7f5a2c109190 2024-05-17T22:37:32.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.576+0000 7f5a1effd700 1 -- 172.21.15.12:0/484941370 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a2400ac90 con 0x7f5a2c109190 2024-05-17T22:37:32.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.576+0000 7f5a33c6d700 1 -- 172.21.15.12:0/484941370 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a2c19b000 con 0x7f5a2c109190 2024-05-17T22:37:32.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.577+0000 7f5a1effd700 1 -- 172.21.15.12:0/484941370 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a240115e0 con 0x7f5a2c109190 2024-05-17T22:37:32.694 INFO:teuthology.orchestra.run.smithi012.stdout:253403070523 2024-05-17T22:37:32.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.577+0000 7f5a33c6d700 1 -- 172.21.15.12:0/484941370 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a2c1a7290 con 0x7f5a2c109190 2024-05-17T22:37:32.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.578+0000 7f5a33c6d700 1 -- 172.21.15.12:0/484941370 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a2c1053c0 con 0x7f5a2c109190 2024-05-17T22:37:32.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.579+0000 7f5a1effd700 1 -- 172.21.15.12:0/484941370 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5a24019020 con 0x7f5a2c109190 2024-05-17T22:37:32.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.580+0000 7f5a1effd700 1 --2- 172.21.15.12:0/484941370 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5a180b2880 0x7f5a180b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:32.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.580+0000 7f5a31a09700 1 --2- 172.21.15.12:0/484941370 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5a180b2880 0x7f5a180b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:32.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.580+0000 7f5a1effd700 1 -- 172.21.15.12:0/484941370 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f5a240e2490 con 0x7f5a2c109190 2024-05-17T22:37:32.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.581+0000 7f5a31a09700 1 --2- 172.21.15.12:0/484941370 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5a180b2880 0x7f5a180b4d90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5a28003800 tx=0x7f5a2800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:32.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.585+0000 7f5a1effd700 1 -- 172.21.15.12:0/484941370 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5a24020020 con 0x7f5a2c109190 2024-05-17T22:37:32.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.805+0000 7f5a33c6d700 1 -- 172.21.15.12:0/484941370 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f5a2c104320 con 0x7f5a2c109190 2024-05-17T22:37:32.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.806+0000 7f5a1effd700 1 -- 172.21.15.12:0/484941370 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5a240a9e20 con 0x7f5a2c109190 2024-05-17T22:37:32.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.808+0000 7f5a33c6d700 1 -- 172.21.15.12:0/484941370 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5a180b2880 msgr2=0x7f5a180b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:32.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.808+0000 7f5a33c6d700 1 --2- 172.21.15.12:0/484941370 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5a180b2880 0x7f5a180b4d90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5a28003800 tx=0x7f5a2800b540 comp rx=0 tx=0).stop 2024-05-17T22:37:32.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.808+0000 7f5a33c6d700 1 -- 172.21.15.12:0/484941370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a2c109190 msgr2=0x7f5a2c19a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:32.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.808+0000 7f5a33c6d700 1 --2- 172.21.15.12:0/484941370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a2c109190 0x7f5a2c19a670 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5a24007d70 tx=0x7f5a2400a340 comp rx=0 tx=0).stop 2024-05-17T22:37:32.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.808+0000 7f5a33c6d700 1 -- 172.21.15.12:0/484941370 shutdown_connections 2024-05-17T22:37:32.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.808+0000 7f5a33c6d700 1 --2- 172.21.15.12:0/484941370 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5a180b2880 0x7f5a180b4d90 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:32.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.808+0000 7f5a33c6d700 1 --2- 172.21.15.12:0/484941370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a2c109190 0x7f5a2c19a670 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:32.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.808+0000 7f5a33c6d700 1 --2- 172.21.15.12:0/484941370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a2c103f10 0x7f5a2c1a04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:32.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.808+0000 7f5a33c6d700 1 --2- 172.21.15.12:0/484941370 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a2c108840 0x7f5a2c1a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:32.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.808+0000 7f5a33c6d700 1 -- 172.21.15.12:0/484941370 >> 172.21.15.12:0/484941370 conn(0x7f5a2c0ff990 msgr2=0x7f5a2c101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:32.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.808+0000 7f5a33c6d700 1 -- 172.21.15.12:0/484941370 shutdown_connections 2024-05-17T22:37:32.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:30.808+0000 7f5a33c6d700 1 -- 172.21.15.12:0/484941370 wait complete. 2024-05-17T22:37:33.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.446+0000 7fcda8abc700 1 -- 172.21.15.12:0/3752909745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcda4102e00 msgr2=0x7fcda4103210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:33.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.446+0000 7fcda8abc700 1 --2- 172.21.15.12:0/3752909745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcda4102e00 0x7fcda4103210 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fcd94009990 tx=0x7fcd94009ca0 comp rx=0 tx=0).stop 2024-05-17T22:37:33.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.446+0000 7fcda8abc700 1 -- 172.21.15.12:0/3752909745 shutdown_connections 2024-05-17T22:37:33.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.446+0000 7fcda8abc700 1 --2- 172.21.15.12:0/3752909745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcda4108080 0x7fcda410f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:33.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.446+0000 7fcda8abc700 1 --2- 172.21.15.12:0/3752909745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcda4102e00 0x7fcda4103210 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:33.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.446+0000 7fcda8abc700 1 --2- 172.21.15.12:0/3752909745 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcda4107730 0x7fcda4107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:33.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.446+0000 7fcda8abc700 1 -- 172.21.15.12:0/3752909745 >> 172.21.15.12:0/3752909745 conn(0x7fcda4079bd0 msgr2=0x7fcda4079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:33.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.447+0000 7fcda8abc700 1 -- 172.21.15.12:0/3752909745 shutdown_connections 2024-05-17T22:37:33.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.447+0000 7fcda8abc700 1 -- 172.21.15.12:0/3752909745 wait complete. 2024-05-17T22:37:33.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.447+0000 7fcda8abc700 1 Processor -- start 2024-05-17T22:37:33.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.447+0000 7fcda8abc700 1 -- start start 2024-05-17T22:37:33.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.448+0000 7fcda8abc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcda4102e00 0x7fcda41a0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:33.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.448+0000 7fcda8abc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcda4107730 0x7fcda41a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:33.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.448+0000 7fcda8abc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcda4108080 0x7fcda419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:33.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.448+0000 7fcda8abc700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcda4112d00 con 0x7fcda4108080 2024-05-17T22:37:33.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.448+0000 7fcda8abc700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcda4112b80 con 0x7fcda4102e00 2024-05-17T22:37:33.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.448+0000 7fcda8abc700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcda4112e80 con 0x7fcda4107730 2024-05-17T22:37:33.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.448+0000 7fcda2d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcda4108080 0x7fcda419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:33.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.448+0000 7fcda1d9b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcda4107730 0x7fcda41a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:33.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.448+0000 7fcda259c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcda4102e00 0x7fcda41a0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:33.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.448+0000 7fcda2d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcda4108080 0x7fcda419a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40344/0 (socket says 172.21.15.12:40344) 2024-05-17T22:37:33.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.448+0000 7fcda2d9d700 1 -- 172.21.15.12:0/1515103746 learned_addr learned my addr 172.21.15.12:0/1515103746 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:37:33.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.448+0000 7fcda1d9b700 1 -- 172.21.15.12:0/1515103746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcda4102e00 msgr2=0x7fcda41a0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:33.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.448+0000 7fcda1d9b700 1 --2- 172.21.15.12:0/1515103746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcda4102e00 0x7fcda41a0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:33.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.448+0000 7fcda1d9b700 1 -- 172.21.15.12:0/1515103746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcda4108080 msgr2=0x7fcda419a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:33.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.448+0000 7fcda1d9b700 1 --2- 172.21.15.12:0/1515103746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcda4108080 0x7fcda419a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:33.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.448+0000 7fcda1d9b700 1 -- 172.21.15.12:0/1515103746 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcda419add0 con 0x7fcda4107730 2024-05-17T22:37:33.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.448+0000 7fcda2d9d700 1 --2- 172.21.15.12:0/1515103746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcda4108080 0x7fcda419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:37:33.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.449+0000 7fcda259c700 1 --2- 172.21.15.12:0/1515103746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcda4102e00 0x7fcda41a0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:37:33.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.449+0000 7fcda1d9b700 1 --2- 172.21.15.12:0/1515103746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcda4107730 0x7fcda41a0a50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcda406b1a0 tx=0x7fcd9800dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:33.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.449+0000 7fcd937fe700 1 -- 172.21.15.12:0/1515103746 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd98016070 con 0x7fcda4107730 2024-05-17T22:37:33.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.449+0000 7fcda8abc700 1 -- 172.21.15.12:0/1515103746 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcda419b0c0 con 0x7fcda4107730 2024-05-17T22:37:33.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.449+0000 7fcd937fe700 1 -- 172.21.15.12:0/1515103746 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd9800fd40 con 0x7fcda4107730 2024-05-17T22:37:33.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.449+0000 7fcd937fe700 1 -- 172.21.15.12:0/1515103746 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9800e3b0 con 0x7fcda4107730 2024-05-17T22:37:33.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.449+0000 7fcda8abc700 1 -- 172.21.15.12:0/1515103746 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcda41a72c0 con 0x7fcda4107730 2024-05-17T22:37:33.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.450+0000 7fcda8abc700 1 -- 172.21.15.12:0/1515103746 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcda41042b0 con 0x7fcda4107730 2024-05-17T22:37:33.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.452+0000 7fcd937fe700 1 -- 172.21.15.12:0/1515103746 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fcd98019070 con 0x7fcda4107730 2024-05-17T22:37:33.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.453+0000 7fcd937fe700 1 --2- 172.21.15.12:0/1515103746 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fcd8c0b2880 0x7fcd8c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:33.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.453+0000 7fcda259c700 1 --2- 172.21.15.12:0/1515103746 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fcd8c0b2880 0x7fcd8c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:33.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.453+0000 7fcd937fe700 1 -- 172.21.15.12:0/1515103746 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fcd980e2490 con 0x7fcda4107730 2024-05-17T22:37:33.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.453+0000 7fcda259c700 1 --2- 172.21.15.12:0/1515103746 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fcd8c0b2880 0x7fcd8c0b4d90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcd94006010 tx=0x7fcd9401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:33.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.456+0000 7fcd937fe700 1 -- 172.21.15.12:0/1515103746 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcd980a9e20 con 0x7fcda4107730 2024-05-17T22:37:33.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.673+0000 7fcda8abc700 1 -- 172.21.15.12:0/1515103746 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fcda4103210 con 0x7fcda4107730 2024-05-17T22:37:33.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.674+0000 7fcd937fe700 1 -- 172.21.15.12:0/1515103746 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fcd980a9720 con 0x7fcda4107730 2024-05-17T22:37:33.677 INFO:teuthology.orchestra.run.smithi012.stdout:133143986262 2024-05-17T22:37:33.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.676+0000 7fcda8abc700 1 -- 172.21.15.12:0/1515103746 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fcd8c0b2880 msgr2=0x7fcd8c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:33.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.676+0000 7fcda8abc700 1 --2- 172.21.15.12:0/1515103746 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fcd8c0b2880 0x7fcd8c0b4d90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcd94006010 tx=0x7fcd9401a040 comp rx=0 tx=0).stop 2024-05-17T22:37:33.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.676+0000 7fcda8abc700 1 -- 172.21.15.12:0/1515103746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcda4107730 msgr2=0x7fcda41a0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:33.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.676+0000 7fcda8abc700 1 --2- 172.21.15.12:0/1515103746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcda4107730 0x7fcda41a0a50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcda406b1a0 tx=0x7fcd9800dc30 comp rx=0 tx=0).stop 2024-05-17T22:37:33.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.676+0000 7fcda8abc700 1 -- 172.21.15.12:0/1515103746 shutdown_connections 2024-05-17T22:37:33.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.676+0000 7fcda8abc700 1 --2- 172.21.15.12:0/1515103746 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fcd8c0b2880 0x7fcd8c0b4d90 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:33.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.676+0000 7fcda8abc700 1 --2- 172.21.15.12:0/1515103746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcda4108080 0x7fcda419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:33.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.676+0000 7fcda8abc700 1 --2- 172.21.15.12:0/1515103746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcda4102e00 0x7fcda41a0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:33.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.676+0000 7fcda8abc700 1 --2- 172.21.15.12:0/1515103746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcda4107730 0x7fcda41a0a50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:33.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.676+0000 7fcda8abc700 1 -- 172.21.15.12:0/1515103746 >> 172.21.15.12:0/1515103746 conn(0x7fcda4079bd0 msgr2=0x7fcda406ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:33.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.676+0000 7fcda8abc700 1 -- 172.21.15.12:0/1515103746 shutdown_connections 2024-05-17T22:37:33.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.676+0000 7fcda8abc700 1 -- 172.21.15.12:0/1515103746 wait complete. 2024-05-17T22:37:33.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.671+0000 7f30af7bc700 1 -- 172.21.15.12:0/1123979284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30a8102e00 msgr2=0x7f30a8103210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:33.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.671+0000 7f30af7bc700 1 --2- 172.21.15.12:0/1123979284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30a8102e00 0x7f30a8103210 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f30a400b0a0 tx=0x7f30a400b3b0 comp rx=0 tx=0).stop 2024-05-17T22:37:33.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.671+0000 7f30af7bc700 1 -- 172.21.15.12:0/1123979284 shutdown_connections 2024-05-17T22:37:33.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.671+0000 7f30af7bc700 1 --2- 172.21.15.12:0/1123979284 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30a8108080 0x7f30a810f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:33.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.671+0000 7f30af7bc700 1 --2- 172.21.15.12:0/1123979284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30a8102e00 0x7f30a8103210 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:33.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.671+0000 7f30af7bc700 1 --2- 172.21.15.12:0/1123979284 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30a8107730 0x7f30a8107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:33.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.671+0000 7f30af7bc700 1 -- 172.21.15.12:0/1123979284 >> 172.21.15.12:0/1123979284 conn(0x7f30a8079bd0 msgr2=0x7f30a8079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:33.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.671+0000 7f30af7bc700 1 -- 172.21.15.12:0/1123979284 shutdown_connections 2024-05-17T22:37:33.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.671+0000 7f30af7bc700 1 -- 172.21.15.12:0/1123979284 wait complete. 2024-05-17T22:37:33.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.672+0000 7f30af7bc700 1 Processor -- start 2024-05-17T22:37:33.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.672+0000 7f30af7bc700 1 -- start start 2024-05-17T22:37:33.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.672+0000 7f30af7bc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30a8102e00 0x7f30a819a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:33.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.672+0000 7f30af7bc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30a8107730 0x7f30a819ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:33.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.672+0000 7f30af7bc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30a8108080 0x7f30a819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:33.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.672+0000 7f30af7bc700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f30a8112cb0 con 0x7f30a8102e00 2024-05-17T22:37:33.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.672+0000 7f30af7bc700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f30a8112b30 con 0x7f30a8107730 2024-05-17T22:37:33.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.672+0000 7f30af7bc700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f30a8112e30 con 0x7f30a8108080 2024-05-17T22:37:33.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.672+0000 7f30ad558700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30a8102e00 0x7f30a819a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:33.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.672+0000 7f30ad558700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30a8102e00 0x7f30a819a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40368/0 (socket says 172.21.15.12:40368) 2024-05-17T22:37:33.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.672+0000 7f30ad558700 1 -- 172.21.15.12:0/1108428218 learned_addr learned my addr 172.21.15.12:0/1108428218 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:37:33.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.672+0000 7f30add59700 1 --2- 172.21.15.12:0/1108428218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30a8108080 0x7f30a819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:33.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.672+0000 7f30acd57700 1 --2- 172.21.15.12:0/1108428218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30a8107730 0x7f30a819ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:33.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.672+0000 7f30ad558700 1 -- 172.21.15.12:0/1108428218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30a8108080 msgr2=0x7f30a819efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:33.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.672+0000 7f30ad558700 1 --2- 172.21.15.12:0/1108428218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30a8108080 0x7f30a819efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:33.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.672+0000 7f30ad558700 1 -- 172.21.15.12:0/1108428218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30a8107730 msgr2=0x7f30a819ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:33.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.672+0000 7f30ad558700 1 --2- 172.21.15.12:0/1108428218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30a8107730 0x7f30a819ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:33.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.672+0000 7f30ad558700 1 -- 172.21.15.12:0/1108428218 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30a819f650 con 0x7f30a8102e00 2024-05-17T22:37:33.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.672+0000 7f30acd57700 1 --2- 172.21.15.12:0/1108428218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30a8107730 0x7f30a819ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:37:33.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.672+0000 7f30add59700 1 --2- 172.21.15.12:0/1108428218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30a8108080 0x7f30a819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:37:33.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.673+0000 7f30ad558700 1 --2- 172.21.15.12:0/1108428218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30a8102e00 0x7f30a819a780 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f30a400b070 tx=0x7f30a4004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:33.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.673+0000 7f309e7fc700 1 -- 172.21.15.12:0/1108428218 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a4021070 con 0x7f30a8102e00 2024-05-17T22:37:33.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.673+0000 7f30af7bc700 1 -- 172.21.15.12:0/1108428218 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30a819f8e0 con 0x7f30a8102e00 2024-05-17T22:37:33.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.673+0000 7f309e7fc700 1 -- 172.21.15.12:0/1108428218 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30a4007c80 con 0x7f30a8102e00 2024-05-17T22:37:33.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.673+0000 7f309e7fc700 1 -- 172.21.15.12:0/1108428218 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a401d450 con 0x7f30a8102e00 2024-05-17T22:37:33.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.673+0000 7f30af7bc700 1 -- 172.21.15.12:0/1108428218 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30a81a7260 con 0x7f30a8102e00 2024-05-17T22:37:33.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.674+0000 7f30af7bc700 1 -- 172.21.15.12:0/1108428218 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30a81042b0 con 0x7f30a8102e00 2024-05-17T22:37:33.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.676+0000 7f309e7fc700 1 -- 172.21.15.12:0/1108428218 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f30a4004020 con 0x7f30a8102e00 2024-05-17T22:37:33.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.676+0000 7f309e7fc700 1 --2- 172.21.15.12:0/1108428218 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f30940b2880 0x7f30940b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:33.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.677+0000 7f30acd57700 1 --2- 172.21.15.12:0/1108428218 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f30940b2880 0x7f30940b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:33.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.677+0000 7f309e7fc700 1 -- 172.21.15.12:0/1108428218 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f30a400f020 con 0x7f30a8102e00 2024-05-17T22:37:33.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.677+0000 7f30acd57700 1 --2- 172.21.15.12:0/1108428218 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f30940b2880 0x7f30940b4d90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3098005fd0 tx=0x7f3098005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:33.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.681+0000 7f309e7fc700 1 -- 172.21.15.12:0/1108428218 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f30a402a020 con 0x7f30a8102e00 2024-05-17T22:37:33.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.903+0000 7f30af7bc700 1 -- 172.21.15.12:0/1108428218 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f30a8103210 con 0x7f30a8102e00 2024-05-17T22:37:33.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.903+0000 7f309e7fc700 1 -- 172.21.15.12:0/1108428218 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f30a40ab5f0 con 0x7f30a8102e00 2024-05-17T22:37:33.907 INFO:teuthology.orchestra.run.smithi012.stdout:77309411426 2024-05-17T22:37:33.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.906+0000 7f30af7bc700 1 -- 172.21.15.12:0/1108428218 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f30940b2880 msgr2=0x7f30940b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:33.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.906+0000 7f30af7bc700 1 --2- 172.21.15.12:0/1108428218 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f30940b2880 0x7f30940b4d90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3098005fd0 tx=0x7f3098005ee0 comp rx=0 tx=0).stop 2024-05-17T22:37:33.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.906+0000 7f30af7bc700 1 -- 172.21.15.12:0/1108428218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30a8102e00 msgr2=0x7f30a819a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:33.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.906+0000 7f30af7bc700 1 --2- 172.21.15.12:0/1108428218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30a8102e00 0x7f30a819a780 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f30a400b070 tx=0x7f30a4004db0 comp rx=0 tx=0).stop 2024-05-17T22:37:33.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.906+0000 7f30af7bc700 1 -- 172.21.15.12:0/1108428218 shutdown_connections 2024-05-17T22:37:33.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.906+0000 7f30af7bc700 1 --2- 172.21.15.12:0/1108428218 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f30940b2880 0x7f30940b4d90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:33.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.906+0000 7f30af7bc700 1 --2- 172.21.15.12:0/1108428218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30a8108080 0x7f30a819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:33.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.906+0000 7f30af7bc700 1 --2- 172.21.15.12:0/1108428218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30a8102e00 0x7f30a819a780 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:33.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.906+0000 7f30af7bc700 1 --2- 172.21.15.12:0/1108428218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30a8107730 0x7f30a819ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:33.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.906+0000 7f30af7bc700 1 -- 172.21.15.12:0/1108428218 >> 172.21.15.12:0/1108428218 conn(0x7f30a8079bd0 msgr2=0x7f30a8102390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:33.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.906+0000 7f30af7bc700 1 -- 172.21.15.12:0/1108428218 shutdown_connections 2024-05-17T22:37:33.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:33.906+0000 7f30af7bc700 1 -- 172.21.15.12:0/1108428218 wait complete. 2024-05-17T22:37:34.000 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:33 smithi012 ceph-mon[31632]: pgmap v388: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T22:37:34.000 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:33 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/1515103746' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-17T22:37:34.000 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:33 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/1108428218' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-17T22:37:34.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:33 smithi151 ceph-mon[36101]: pgmap v388: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T22:37:34.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:33 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/1515103746' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-17T22:37:34.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:33 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/1108428218' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-17T22:37:34.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:33 smithi096 ceph-mon[36065]: pgmap v388: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T22:37:34.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:33 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/1515103746' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-17T22:37:34.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:33 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/1108428218' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-17T22:37:34.445 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986258 got 133143986262 for osd.4 2024-05-17T22:37:34.445 DEBUG:teuthology.parallel:result is None 2024-05-17T22:37:35.267 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070517 got 253403070523 for osd.9 2024-05-17T22:37:35.268 DEBUG:teuthology.parallel:result is None 2024-05-17T22:37:35.271 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:37:35.272 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:37:36.079 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411421 got 77309411426 for osd.2 2024-05-17T22:37:36.080 DEBUG:teuthology.parallel:result is None 2024-05-17T22:37:36.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:35 smithi012 ceph-mon[31632]: pgmap v389: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T22:37:36.176 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906994 got 274877906999 for osd.10 2024-05-17T22:37:36.177 DEBUG:teuthology.parallel:result is None 2024-05-17T22:37:36.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:35 smithi151 ceph-mon[36101]: pgmap v389: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T22:37:36.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:35 smithi096 ceph-mon[36065]: pgmap v389: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T22:37:36.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.427+0000 7fb70d342700 1 -- 172.21.15.12:0/681290205 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7081080a0 msgr2=0x7fb70810fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:36.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.427+0000 7fb70d342700 1 --2- 172.21.15.12:0/681290205 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7081080a0 0x7fb70810fa00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb70000b0a0 tx=0x7fb70000b3b0 comp rx=0 tx=0).stop 2024-05-17T22:37:36.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.427+0000 7fb70d342700 1 -- 172.21.15.12:0/681290205 shutdown_connections 2024-05-17T22:37:36.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.427+0000 7fb70d342700 1 --2- 172.21.15.12:0/681290205 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7081080a0 0x7fb70810fa00 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:36.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.427+0000 7fb70d342700 1 --2- 172.21.15.12:0/681290205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb708102e70 0x7fb708103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:36.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.427+0000 7fb70d342700 1 --2- 172.21.15.12:0/681290205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb708107750 0x7fb708107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:36.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.427+0000 7fb70d342700 1 -- 172.21.15.12:0/681290205 >> 172.21.15.12:0/681290205 conn(0x7fb708079ae0 msgr2=0x7fb708079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:36.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.427+0000 7fb70d342700 1 -- 172.21.15.12:0/681290205 shutdown_connections 2024-05-17T22:37:36.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.427+0000 7fb70d342700 1 -- 172.21.15.12:0/681290205 wait complete. 2024-05-17T22:37:36.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.428+0000 7fb70d342700 1 Processor -- start 2024-05-17T22:37:36.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.429+0000 7fb70d342700 1 -- start start 2024-05-17T22:37:36.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.429+0000 7fb70d342700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb708102e70 0x7fb7081a6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:36.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.429+0000 7fb70d342700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb708107750 0x7fb7081a6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:36.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.429+0000 7fb70d342700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7081080a0 0x7fb7081a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:36.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.429+0000 7fb70d342700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb708112d60 con 0x7fb708107750 2024-05-17T22:37:36.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.429+0000 7fb70d342700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb708112be0 con 0x7fb708102e70 2024-05-17T22:37:36.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.429+0000 7fb70d342700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb708112ee0 con 0x7fb7081080a0 2024-05-17T22:37:36.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.429+0000 7fb706ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb708102e70 0x7fb7081a6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:36.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.429+0000 7fb706ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb708102e70 0x7fb7081a6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:34364/0 (socket says 172.21.15.12:34364) 2024-05-17T22:37:36.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.429+0000 7fb706ffd700 1 -- 172.21.15.12:0/695169028 learned_addr learned my addr 172.21.15.12:0/695169028 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:37:36.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.429+0000 7fb7067fc700 1 --2- 172.21.15.12:0/695169028 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb708107750 0x7fb7081a6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:36.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.430+0000 7fb706ffd700 1 -- 172.21.15.12:0/695169028 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7081080a0 msgr2=0x7fb7081a0c50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-17T22:37:36.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.430+0000 7fb706ffd700 1 --2- 172.21.15.12:0/695169028 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7081080a0 0x7fb7081a0c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:36.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.430+0000 7fb706ffd700 1 -- 172.21.15.12:0/695169028 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb708107750 msgr2=0x7fb7081a6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:36.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.430+0000 7fb706ffd700 1 --2- 172.21.15.12:0/695169028 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb708107750 0x7fb7081a6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:36.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.430+0000 7fb706ffd700 1 -- 172.21.15.12:0/695169028 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7081a13c0 con 0x7fb708102e70 2024-05-17T22:37:36.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.430+0000 7fb7067fc700 1 --2- 172.21.15.12:0/695169028 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb708107750 0x7fb7081a6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:37:36.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.430+0000 7fb706ffd700 1 --2- 172.21.15.12:0/695169028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb708102e70 0x7fb7081a6aa0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb6f800d980 tx=0x7fb6f800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:36.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.431+0000 7fb6effff700 1 -- 172.21.15.12:0/695169028 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f8017070 con 0x7fb708102e70 2024-05-17T22:37:36.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.431+0000 7fb6effff700 1 -- 172.21.15.12:0/695169028 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6f8005020 con 0x7fb708102e70 2024-05-17T22:37:36.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.431+0000 7fb70d342700 1 -- 172.21.15.12:0/695169028 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7081a16b0 con 0x7fb708102e70 2024-05-17T22:37:36.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.431+0000 7fb6effff700 1 -- 172.21.15.12:0/695169028 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f8009b50 con 0x7fb708102e70 2024-05-17T22:37:36.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.431+0000 7fb70d342700 1 -- 172.21.15.12:0/695169028 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7081ad850 con 0x7fb708102e70 2024-05-17T22:37:36.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.432+0000 7fb70d342700 1 -- 172.21.15.12:0/695169028 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6f4005330 con 0x7fb708102e70 2024-05-17T22:37:36.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.434+0000 7fb6effff700 1 -- 172.21.15.12:0/695169028 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb6f8009cf0 con 0x7fb708102e70 2024-05-17T22:37:36.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.435+0000 7fb6effff700 1 --2- 172.21.15.12:0/695169028 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb6f00b2880 0x7fb6f00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:36.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.436+0000 7fb7067fc700 1 --2- 172.21.15.12:0/695169028 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb6f00b2880 0x7fb6f00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:36.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.436+0000 7fb6effff700 1 -- 172.21.15.12:0/695169028 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fb6f80e2760 con 0x7fb708102e70 2024-05-17T22:37:36.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.436+0000 7fb7067fc700 1 --2- 172.21.15.12:0/695169028 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb6f00b2880 0x7fb6f00b4d90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb70806ca80 tx=0x7fb6fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:36.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.439+0000 7fb6effff700 1 -- 172.21.15.12:0/695169028 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb6f8020020 con 0x7fb708102e70 2024-05-17T22:37:36.449 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:37:36.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.661+0000 7fb70d342700 1 -- 172.21.15.12:0/695169028 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fb6f4005d10 con 0x7fb708102e70 2024-05-17T22:37:36.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.661+0000 7fb6effff700 1 -- 172.21.15.12:0/695169028 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb6f80aa0f0 con 0x7fb708102e70 2024-05-17T22:37:36.664 INFO:teuthology.orchestra.run.smithi012.stdout:227633266755 2024-05-17T22:37:36.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.664+0000 7fb70d342700 1 -- 172.21.15.12:0/695169028 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb6f00b2880 msgr2=0x7fb6f00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:36.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.664+0000 7fb70d342700 1 --2- 172.21.15.12:0/695169028 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb6f00b2880 0x7fb6f00b4d90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb70806ca80 tx=0x7fb6fc009260 comp rx=0 tx=0).stop 2024-05-17T22:37:36.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.664+0000 7fb70d342700 1 -- 172.21.15.12:0/695169028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb708102e70 msgr2=0x7fb7081a6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:36.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.664+0000 7fb70d342700 1 --2- 172.21.15.12:0/695169028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb708102e70 0x7fb7081a6aa0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb6f800d980 tx=0x7fb6f800dc90 comp rx=0 tx=0).stop 2024-05-17T22:37:36.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.664+0000 7fb70d342700 1 -- 172.21.15.12:0/695169028 shutdown_connections 2024-05-17T22:37:36.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.664+0000 7fb70d342700 1 --2- 172.21.15.12:0/695169028 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb6f00b2880 0x7fb6f00b4d90 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:36.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.664+0000 7fb70d342700 1 --2- 172.21.15.12:0/695169028 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7081080a0 0x7fb7081a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:36.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.664+0000 7fb70d342700 1 --2- 172.21.15.12:0/695169028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb708102e70 0x7fb7081a6aa0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:36.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.664+0000 7fb70d342700 1 --2- 172.21.15.12:0/695169028 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb708107750 0x7fb7081a6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:36.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.664+0000 7fb70d342700 1 -- 172.21.15.12:0/695169028 >> 172.21.15.12:0/695169028 conn(0x7fb708079ae0 msgr2=0x7fb708104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:36.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.664+0000 7fb70d342700 1 -- 172.21.15.12:0/695169028 shutdown_connections 2024-05-17T22:37:36.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:36.664+0000 7fb70d342700 1 -- 172.21.15.12:0/695169028 wait complete. 2024-05-17T22:37:37.017 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:36 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/695169028' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-17T22:37:37.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:36 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/695169028' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-17T22:37:37.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:36 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/695169028' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-17T22:37:38.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:37 smithi012 ceph-mon[31632]: pgmap v390: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T22:37:38.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:37 smithi151 ceph-mon[36101]: pgmap v390: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T22:37:38.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:37 smithi096 ceph-mon[36065]: pgmap v390: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T22:37:39.153 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266748 got 227633266755 for osd.8 2024-05-17T22:37:39.153 DEBUG:teuthology.parallel:result is None 2024-05-17T22:37:40.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:39 smithi012 ceph-mon[31632]: pgmap v391: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:37:40.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:39 smithi151 ceph-mon[36101]: pgmap v391: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:37:40.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:39 smithi096 ceph-mon[36065]: pgmap v391: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:37:41.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.191+0000 7f8ba73a4700 1 -- 172.21.15.12:0/2461817411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ba0103f10 msgr2=0x7f8ba0104320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:41.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.191+0000 7f8ba73a4700 1 --2- 172.21.15.12:0/2461817411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ba0103f10 0x7f8ba0104320 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c009990 tx=0x7f8b9c009ca0 comp rx=0 tx=0).stop 2024-05-17T22:37:41.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.191+0000 7f8ba73a4700 1 -- 172.21.15.12:0/2461817411 shutdown_connections 2024-05-17T22:37:41.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.191+0000 7f8ba73a4700 1 --2- 172.21.15.12:0/2461817411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ba0109190 0x7f8ba0110af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:41.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.191+0000 7f8ba73a4700 1 --2- 172.21.15.12:0/2461817411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ba0103f10 0x7f8ba0104320 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:41.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.191+0000 7f8ba73a4700 1 --2- 172.21.15.12:0/2461817411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ba0108840 0x7f8ba0108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:41.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.191+0000 7f8ba73a4700 1 -- 172.21.15.12:0/2461817411 >> 172.21.15.12:0/2461817411 conn(0x7f8ba00ff9b0 msgr2=0x7f8ba0101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:41.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.191+0000 7f8ba73a4700 1 -- 172.21.15.12:0/2461817411 shutdown_connections 2024-05-17T22:37:41.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.191+0000 7f8ba73a4700 1 -- 172.21.15.12:0/2461817411 wait complete. 2024-05-17T22:37:41.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.192+0000 7f8ba73a4700 1 Processor -- start 2024-05-17T22:37:41.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.192+0000 7f8ba73a4700 1 -- start start 2024-05-17T22:37:41.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.192+0000 7f8ba73a4700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ba0103f10 0x7f8ba01a04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:41.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.192+0000 7f8ba73a4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ba0108840 0x7f8ba01a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:41.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.192+0000 7f8ba73a4700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ba0109190 0x7f8ba019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:41.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.192+0000 7f8ba73a4700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba0113dc0 con 0x7f8ba0108840 2024-05-17T22:37:41.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.192+0000 7f8ba73a4700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba0113c40 con 0x7f8ba0103f10 2024-05-17T22:37:41.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.192+0000 7f8ba73a4700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba0113f40 con 0x7f8ba0109190 2024-05-17T22:37:41.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.192+0000 7f8ba493f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ba0108840 0x7f8ba01a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:41.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.192+0000 7f8ba493f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ba0108840 0x7f8ba01a0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55264/0 (socket says 172.21.15.12:55264) 2024-05-17T22:37:41.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.192+0000 7f8ba493f700 1 -- 172.21.15.12:0/3248685127 learned_addr learned my addr 172.21.15.12:0/3248685127 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:37:41.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.192+0000 7f8ba5140700 1 --2- 172.21.15.12:0/3248685127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ba0103f10 0x7f8ba01a04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:41.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.192+0000 7f8ba5941700 1 --2- 172.21.15.12:0/3248685127 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ba0109190 0x7f8ba019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:41.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.193+0000 7f8ba493f700 1 -- 172.21.15.12:0/3248685127 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ba0109190 msgr2=0x7f8ba019a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:41.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.193+0000 7f8ba493f700 1 --2- 172.21.15.12:0/3248685127 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ba0109190 0x7f8ba019a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:41.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.193+0000 7f8ba493f700 1 -- 172.21.15.12:0/3248685127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ba0103f10 msgr2=0x7f8ba01a04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:41.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.193+0000 7f8ba493f700 1 --2- 172.21.15.12:0/3248685127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ba0103f10 0x7f8ba01a04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:41.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.193+0000 7f8ba493f700 1 -- 172.21.15.12:0/3248685127 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba019acf0 con 0x7f8ba0108840 2024-05-17T22:37:41.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.193+0000 7f8ba5941700 1 --2- 172.21.15.12:0/3248685127 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ba0109190 0x7f8ba019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:37:41.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.193+0000 7f8ba493f700 1 --2- 172.21.15.12:0/3248685127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ba0108840 0x7f8ba01a0a00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8b9000d980 tx=0x7f8b9000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:41.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.193+0000 7f8b967fc700 1 -- 172.21.15.12:0/3248685127 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b90017070 con 0x7f8ba0108840 2024-05-17T22:37:41.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.193+0000 7f8b967fc700 1 -- 172.21.15.12:0/3248685127 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b90009de0 con 0x7f8ba0108840 2024-05-17T22:37:41.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.193+0000 7f8ba73a4700 1 -- 172.21.15.12:0/3248685127 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ba019afe0 con 0x7f8ba0108840 2024-05-17T22:37:41.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.193+0000 7f8b967fc700 1 -- 172.21.15.12:0/3248685127 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9000f3c0 con 0x7f8ba0108840 2024-05-17T22:37:41.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.193+0000 7f8ba73a4700 1 -- 172.21.15.12:0/3248685127 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ba01a7270 con 0x7f8ba0108840 2024-05-17T22:37:41.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.194+0000 7f8ba73a4700 1 -- 172.21.15.12:0/3248685127 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ba01053c0 con 0x7f8ba0108840 2024-05-17T22:37:41.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.196+0000 7f8b967fc700 1 -- 172.21.15.12:0/3248685127 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8b90009930 con 0x7f8ba0108840 2024-05-17T22:37:41.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.196+0000 7f8b967fc700 1 --2- 172.21.15.12:0/3248685127 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8b8c0b2880 0x7f8b8c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:41.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.197+0000 7f8ba5140700 1 --2- 172.21.15.12:0/3248685127 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8b8c0b2880 0x7f8b8c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:41.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.197+0000 7f8b967fc700 1 -- 172.21.15.12:0/3248685127 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f8b900e2070 con 0x7f8ba0108840 2024-05-17T22:37:41.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.197+0000 7f8ba5140700 1 --2- 172.21.15.12:0/3248685127 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8b8c0b2880 0x7f8b8c0b4d90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c0098e0 tx=0x7f8b9c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:41.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.200+0000 7f8b967fc700 1 -- 172.21.15.12:0/3248685127 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8b900a9a00 con 0x7f8ba0108840 2024-05-17T22:37:41.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.418+0000 7f8ba73a4700 1 -- 172.21.15.12:0/3248685127 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f8ba0104320 con 0x7f8ba0108840 2024-05-17T22:37:41.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.418+0000 7f8b967fc700 1 -- 172.21.15.12:0/3248685127 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f8b900a9300 con 0x7f8ba0108840 2024-05-17T22:37:41.421 INFO:teuthology.orchestra.run.smithi012.stdout:300647710772 2024-05-17T22:37:41.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.420+0000 7f8ba73a4700 1 -- 172.21.15.12:0/3248685127 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8b8c0b2880 msgr2=0x7f8b8c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:41.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.420+0000 7f8ba73a4700 1 --2- 172.21.15.12:0/3248685127 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8b8c0b2880 0x7f8b8c0b4d90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c0098e0 tx=0x7f8b9c005960 comp rx=0 tx=0).stop 2024-05-17T22:37:41.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.420+0000 7f8ba73a4700 1 -- 172.21.15.12:0/3248685127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ba0108840 msgr2=0x7f8ba01a0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:41.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.420+0000 7f8ba73a4700 1 --2- 172.21.15.12:0/3248685127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ba0108840 0x7f8ba01a0a00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8b9000d980 tx=0x7f8b9000dc90 comp rx=0 tx=0).stop 2024-05-17T22:37:41.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.420+0000 7f8ba73a4700 1 -- 172.21.15.12:0/3248685127 shutdown_connections 2024-05-17T22:37:41.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.420+0000 7f8ba73a4700 1 --2- 172.21.15.12:0/3248685127 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8b8c0b2880 0x7f8b8c0b4d90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:41.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.420+0000 7f8ba73a4700 1 --2- 172.21.15.12:0/3248685127 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ba0109190 0x7f8ba019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:41.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.420+0000 7f8ba73a4700 1 --2- 172.21.15.12:0/3248685127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ba0103f10 0x7f8ba01a04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:41.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.420+0000 7f8ba73a4700 1 --2- 172.21.15.12:0/3248685127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ba0108840 0x7f8ba01a0a00 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:41.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.420+0000 7f8ba73a4700 1 -- 172.21.15.12:0/3248685127 >> 172.21.15.12:0/3248685127 conn(0x7f8ba00ff9b0 msgr2=0x7f8ba0101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:41.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.420+0000 7f8ba73a4700 1 -- 172.21.15.12:0/3248685127 shutdown_connections 2024-05-17T22:37:41.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.420+0000 7f8ba73a4700 1 -- 172.21.15.12:0/3248685127 wait complete. 2024-05-17T22:37:42.180 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:41 smithi012 ceph-mon[31632]: pgmap v392: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T22:37:42.180 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:41 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/3007889168' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-17T22:37:42.180 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:41 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/3248685127' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-17T22:37:42.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:41 smithi151 ceph-mon[36101]: pgmap v392: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T22:37:42.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:41 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/3007889168' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-17T22:37:42.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:41 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/3248685127' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-17T22:37:42.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:40.852+0000 7fe583563700 1 -- 172.21.15.12:0/1576224587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe57c103380 msgr2=0x7fe57c1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:42.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:40.852+0000 7fe583563700 1 --2- 172.21.15.12:0/1576224587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe57c103380 0x7fe57c1100a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe56c009990 tx=0x7fe56c009ca0 comp rx=0 tx=0).stop 2024-05-17T22:37:42.276 INFO:teuthology.orchestra.run.smithi012.stdout:180388626509 2024-05-17T22:37:42.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:40.852+0000 7fe583563700 1 -- 172.21.15.12:0/1576224587 shutdown_connections 2024-05-17T22:37:42.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:40.852+0000 7fe583563700 1 --2- 172.21.15.12:0/1576224587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe57c1105e0 0x7fe57c112a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:42.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:40.852+0000 7fe583563700 1 --2- 172.21.15.12:0/1576224587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe57c102a30 0x7fe57c102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:42.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:40.852+0000 7fe583563700 1 --2- 172.21.15.12:0/1576224587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe57c103380 0x7fe57c1100a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:42.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:40.852+0000 7fe583563700 1 -- 172.21.15.12:0/1576224587 >> 172.21.15.12:0/1576224587 conn(0x7fe57c079ae0 msgr2=0x7fe57c079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:42.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:40.852+0000 7fe583563700 1 -- 172.21.15.12:0/1576224587 shutdown_connections 2024-05-17T22:37:42.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:40.852+0000 7fe583563700 1 -- 172.21.15.12:0/1576224587 wait complete. 2024-05-17T22:37:42.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:40.853+0000 7fe583563700 1 Processor -- start 2024-05-17T22:37:42.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:40.853+0000 7fe583563700 1 -- start start 2024-05-17T22:37:42.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:40.854+0000 7fe583563700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe57c102a30 0x7fe57c1a25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:42.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:40.854+0000 7fe583563700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe57c103380 0x7fe57c1a2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:42.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:40.854+0000 7fe583563700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe57c1105e0 0x7fe57c19c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:42.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:40.854+0000 7fe583563700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe57c115fc0 con 0x7fe57c103380 2024-05-17T22:37:42.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:40.854+0000 7fe583563700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe57c115e40 con 0x7fe57c1105e0 2024-05-17T22:37:42.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:40.854+0000 7fe583563700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe57c116140 con 0x7fe57c102a30 2024-05-17T22:37:42.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:40.854+0000 7fe580afe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe57c103380 0x7fe57c1a2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:42.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:40.854+0000 7fe580afe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe57c103380 0x7fe57c1a2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55252/0 (socket says 172.21.15.12:55252) 2024-05-17T22:37:42.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:40.854+0000 7fe580afe700 1 -- 172.21.15.12:0/3007889168 learned_addr learned my addr 172.21.15.12:0/3007889168 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:37:42.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:40.854+0000 7fe5812ff700 1 --2- 172.21.15.12:0/3007889168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe57c102a30 0x7fe57c1a25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:42.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:40.854+0000 7fe581b00700 1 --2- 172.21.15.12:0/3007889168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe57c1105e0 0x7fe57c19c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:42.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:40.854+0000 7fe580afe700 1 -- 172.21.15.12:0/3007889168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe57c102a30 msgr2=0x7fe57c1a25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:42.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:40.854+0000 7fe580afe700 1 --2- 172.21.15.12:0/3007889168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe57c102a30 0x7fe57c1a25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:42.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:40.854+0000 7fe580afe700 1 -- 172.21.15.12:0/3007889168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe57c1105e0 msgr2=0x7fe57c19c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:42.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:40.854+0000 7fe580afe700 1 --2- 172.21.15.12:0/3007889168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe57c1105e0 0x7fe57c19c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:42.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:40.854+0000 7fe580afe700 1 -- 172.21.15.12:0/3007889168 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe57c19ce90 con 0x7fe57c103380 2024-05-17T22:37:42.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:40.854+0000 7fe581b00700 1 --2- 172.21.15.12:0/3007889168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe57c1105e0 0x7fe57c19c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:37:42.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:40.854+0000 7fe580afe700 1 --2- 172.21.15.12:0/3007889168 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe57c103380 0x7fe57c1a2b10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe56c000c00 tx=0x7fe56c003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:42.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:40.854+0000 7fe5727fc700 1 -- 172.21.15.12:0/3007889168 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c027070 con 0x7fe57c103380 2024-05-17T22:37:42.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:40.855+0000 7fe583563700 1 -- 172.21.15.12:0/3007889168 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe57c19d120 con 0x7fe57c103380 2024-05-17T22:37:42.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:40.855+0000 7fe5727fc700 1 -- 172.21.15.12:0/3007889168 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe56c004160 con 0x7fe57c103380 2024-05-17T22:37:42.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:40.855+0000 7fe5727fc700 1 -- 172.21.15.12:0/3007889168 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c022db0 con 0x7fe57c103380 2024-05-17T22:37:42.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:40.855+0000 7fe583563700 1 -- 172.21.15.12:0/3007889168 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe57c19d650 con 0x7fe57c103380 2024-05-17T22:37:42.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:40.856+0000 7fe583563700 1 -- 172.21.15.12:0/3007889168 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe57c06c8d0 con 0x7fe57c103380 2024-05-17T22:37:42.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:40.857+0000 7fe5727fc700 1 -- 172.21.15.12:0/3007889168 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe56c02a070 con 0x7fe57c103380 2024-05-17T22:37:42.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:40.858+0000 7fe5727fc700 1 --2- 172.21.15.12:0/3007889168 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe5680b2880 0x7fe5680b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:42.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:40.858+0000 7fe5727fc700 1 -- 172.21.15.12:0/3007889168 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe56c0e3b60 con 0x7fe57c103380 2024-05-17T22:37:42.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:40.858+0000 7fe5812ff700 1 --2- 172.21.15.12:0/3007889168 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe5680b2880 0x7fe5680b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:42.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:40.859+0000 7fe5812ff700 1 --2- 172.21.15.12:0/3007889168 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe5680b2880 0x7fe5680b4d90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe578005ea0 tx=0x7fe578005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:42.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:40.861+0000 7fe5727fc700 1 -- 172.21.15.12:0/3007889168 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe56c031080 con 0x7fe57c103380 2024-05-17T22:37:42.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.080+0000 7fe583563700 1 -- 172.21.15.12:0/3007889168 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fe57c102e40 con 0x7fe57c103380 2024-05-17T22:37:42.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.080+0000 7fe5727fc700 1 -- 172.21.15.12:0/3007889168 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe57c19d650 con 0x7fe57c103380 2024-05-17T22:37:42.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.083+0000 7fe583563700 1 -- 172.21.15.12:0/3007889168 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe5680b2880 msgr2=0x7fe5680b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:42.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.083+0000 7fe583563700 1 --2- 172.21.15.12:0/3007889168 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe5680b2880 0x7fe5680b4d90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe578005ea0 tx=0x7fe578005e30 comp rx=0 tx=0).stop 2024-05-17T22:37:42.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.083+0000 7fe583563700 1 -- 172.21.15.12:0/3007889168 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe57c103380 msgr2=0x7fe57c1a2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:42.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.083+0000 7fe583563700 1 --2- 172.21.15.12:0/3007889168 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe57c103380 0x7fe57c1a2b10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe56c000c00 tx=0x7fe56c003680 comp rx=0 tx=0).stop 2024-05-17T22:37:42.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.083+0000 7fe583563700 1 -- 172.21.15.12:0/3007889168 shutdown_connections 2024-05-17T22:37:42.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.083+0000 7fe583563700 1 --2- 172.21.15.12:0/3007889168 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe5680b2880 0x7fe5680b4d90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:42.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.083+0000 7fe583563700 1 --2- 172.21.15.12:0/3007889168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe57c1105e0 0x7fe57c19c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:42.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.083+0000 7fe583563700 1 --2- 172.21.15.12:0/3007889168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe57c102a30 0x7fe57c1a25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:42.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.083+0000 7fe583563700 1 --2- 172.21.15.12:0/3007889168 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe57c103380 0x7fe57c1a2b10 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:42.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.083+0000 7fe583563700 1 -- 172.21.15.12:0/3007889168 >> 172.21.15.12:0/3007889168 conn(0x7fe57c079ae0 msgr2=0x7fe57c111a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:42.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.083+0000 7fe583563700 1 -- 172.21.15.12:0/3007889168 shutdown_connections 2024-05-17T22:37:42.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:41.083+0000 7fe583563700 1 -- 172.21.15.12:0/3007889168 wait complete. 2024-05-17T22:37:42.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:41 smithi096 ceph-mon[36065]: pgmap v392: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T22:37:42.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:41 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/3007889168' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-17T22:37:42.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:41 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/3248685127' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-17T22:37:42.907 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626502 got 180388626509 for osd.6 2024-05-17T22:37:42.908 DEBUG:teuthology.parallel:result is None 2024-05-17T22:37:43.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.156+0000 7fe48e516700 1 -- 172.21.15.12:0/2379976417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe488102e20 msgr2=0x7fe488103230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:43.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.156+0000 7fe48e516700 1 --2- 172.21.15.12:0/2379976417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe488102e20 0x7fe488103230 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe478009990 tx=0x7fe478009ca0 comp rx=0 tx=0).stop 2024-05-17T22:37:43.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.156+0000 7fe48e516700 1 -- 172.21.15.12:0/2379976417 shutdown_connections 2024-05-17T22:37:43.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.156+0000 7fe48e516700 1 --2- 172.21.15.12:0/2379976417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4881080a0 0x7fe48810fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:43.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.156+0000 7fe48e516700 1 --2- 172.21.15.12:0/2379976417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe488102e20 0x7fe488103230 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:43.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.156+0000 7fe48e516700 1 --2- 172.21.15.12:0/2379976417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe488107750 0x7fe488107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:43.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.156+0000 7fe48e516700 1 -- 172.21.15.12:0/2379976417 >> 172.21.15.12:0/2379976417 conn(0x7fe488079bd0 msgr2=0x7fe488079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:43.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.156+0000 7fe48e516700 1 -- 172.21.15.12:0/2379976417 shutdown_connections 2024-05-17T22:37:43.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.156+0000 7fe48e516700 1 -- 172.21.15.12:0/2379976417 wait complete. 2024-05-17T22:37:43.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.157+0000 7fe48e516700 1 Processor -- start 2024-05-17T22:37:43.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.157+0000 7fe48e516700 1 -- start start 2024-05-17T22:37:43.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.158+0000 7fe48e516700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe488102e20 0x7fe4880ff9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:43.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.158+0000 7fe48e516700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe488107750 0x7fe4880fff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:43.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.158+0000 7fe48e516700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4881080a0 0x7fe488100440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:43.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.158+0000 7fe48e516700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe488112d50 con 0x7fe488102e20 2024-05-17T22:37:43.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.158+0000 7fe48e516700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe488112bd0 con 0x7fe4881080a0 2024-05-17T22:37:43.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.158+0000 7fe48e516700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe488112ed0 con 0x7fe488107750 2024-05-17T22:37:43.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.158+0000 7fe487fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe488102e20 0x7fe4880ff9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:43.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.158+0000 7fe487fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe488102e20 0x7fe4880ff9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55284/0 (socket says 172.21.15.12:55284) 2024-05-17T22:37:43.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.158+0000 7fe487fff700 1 -- 172.21.15.12:0/4146282496 learned_addr learned my addr 172.21.15.12:0/4146282496 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:37:43.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.158+0000 7fe48cab3700 1 --2- 172.21.15.12:0/4146282496 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4881080a0 0x7fe488100440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:43.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.158+0000 7fe4877fe700 1 --2- 172.21.15.12:0/4146282496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe488107750 0x7fe4880fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:43.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.158+0000 7fe487fff700 1 -- 172.21.15.12:0/4146282496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe488107750 msgr2=0x7fe4880fff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:43.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.158+0000 7fe487fff700 1 --2- 172.21.15.12:0/4146282496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe488107750 0x7fe4880fff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:43.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.158+0000 7fe487fff700 1 -- 172.21.15.12:0/4146282496 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4881080a0 msgr2=0x7fe488100440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:43.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.158+0000 7fe487fff700 1 --2- 172.21.15.12:0/4146282496 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4881080a0 0x7fe488100440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:43.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.158+0000 7fe487fff700 1 -- 172.21.15.12:0/4146282496 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4881a8c40 con 0x7fe488102e20 2024-05-17T22:37:43.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.158+0000 7fe487fff700 1 --2- 172.21.15.12:0/4146282496 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe488102e20 0x7fe4880ff9c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe4780057c0 tx=0x7fe4780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:43.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.159+0000 7fe4857fa700 1 -- 172.21.15.12:0/4146282496 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe478027070 con 0x7fe488102e20 2024-05-17T22:37:43.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.159+0000 7fe4857fa700 1 -- 172.21.15.12:0/4146282496 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe478003f20 con 0x7fe488102e20 2024-05-17T22:37:43.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.159+0000 7fe4857fa700 1 -- 172.21.15.12:0/4146282496 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe478022ba0 con 0x7fe488102e20 2024-05-17T22:37:43.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.159+0000 7fe48e516700 1 -- 172.21.15.12:0/4146282496 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4881a8ed0 con 0x7fe488102e20 2024-05-17T22:37:43.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.159+0000 7fe48e516700 1 -- 172.21.15.12:0/4146282496 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4881a9270 con 0x7fe488102e20 2024-05-17T22:37:43.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.160+0000 7fe48e516700 1 -- 172.21.15.12:0/4146282496 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe488103230 con 0x7fe488102e20 2024-05-17T22:37:43.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.161+0000 7fe4857fa700 1 -- 172.21.15.12:0/4146282496 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe47802a070 con 0x7fe488102e20 2024-05-17T22:37:43.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.162+0000 7fe4857fa700 1 --2- 172.21.15.12:0/4146282496 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe4700b2880 0x7fe4700b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:43.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.162+0000 7fe4877fe700 1 --2- 172.21.15.12:0/4146282496 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe4700b2880 0x7fe4700b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:43.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.162+0000 7fe4857fa700 1 -- 172.21.15.12:0/4146282496 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe4780e3540 con 0x7fe488102e20 2024-05-17T22:37:43.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.163+0000 7fe4877fe700 1 --2- 172.21.15.12:0/4146282496 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe4700b2880 0x7fe4700b4d90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe47c006fd0 tx=0x7fe47c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:43.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.167+0000 7fe4857fa700 1 -- 172.21.15.12:0/4146282496 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe478013070 con 0x7fe488102e20 2024-05-17T22:37:43.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.384+0000 7fe48e516700 1 -- 172.21.15.12:0/4146282496 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fe488064440 con 0x7fe488102e20 2024-05-17T22:37:43.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.385+0000 7fe4857fa700 1 -- 172.21.15.12:0/4146282496 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe4780aaf60 con 0x7fe488102e20 2024-05-17T22:37:43.388 INFO:teuthology.orchestra.run.smithi012.stdout:55834574952 2024-05-17T22:37:43.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.388+0000 7fe48e516700 1 -- 172.21.15.12:0/4146282496 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe4700b2880 msgr2=0x7fe4700b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:43.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.388+0000 7fe48e516700 1 --2- 172.21.15.12:0/4146282496 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe4700b2880 0x7fe4700b4d90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe47c006fd0 tx=0x7fe47c008040 comp rx=0 tx=0).stop 2024-05-17T22:37:43.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.388+0000 7fe48e516700 1 -- 172.21.15.12:0/4146282496 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe488102e20 msgr2=0x7fe4880ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:43.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.388+0000 7fe48e516700 1 --2- 172.21.15.12:0/4146282496 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe488102e20 0x7fe4880ff9c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe4780057c0 tx=0x7fe4780057f0 comp rx=0 tx=0).stop 2024-05-17T22:37:43.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.388+0000 7fe48e516700 1 -- 172.21.15.12:0/4146282496 shutdown_connections 2024-05-17T22:37:43.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.388+0000 7fe48e516700 1 --2- 172.21.15.12:0/4146282496 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe4700b2880 0x7fe4700b4d90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:43.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.388+0000 7fe48e516700 1 --2- 172.21.15.12:0/4146282496 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4881080a0 0x7fe488100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:43.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.388+0000 7fe48e516700 1 --2- 172.21.15.12:0/4146282496 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe488102e20 0x7fe4880ff9c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:43.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.388+0000 7fe48e516700 1 --2- 172.21.15.12:0/4146282496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe488107750 0x7fe4880fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:43.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.388+0000 7fe48e516700 1 -- 172.21.15.12:0/4146282496 >> 172.21.15.12:0/4146282496 conn(0x7fe488079bd0 msgr2=0x7fe488106630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:43.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.388+0000 7fe48e516700 1 -- 172.21.15.12:0/4146282496 shutdown_connections 2024-05-17T22:37:43.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:43.388+0000 7fe48e516700 1 -- 172.21.15.12:0/4146282496 wait complete. 2024-05-17T22:37:43.672 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710767 got 300647710772 for osd.11 2024-05-17T22:37:43.672 DEBUG:teuthology.parallel:result is None 2024-05-17T22:37:44.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:43 smithi151 ceph-mon[36101]: pgmap v393: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T22:37:44.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:43 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/4146282496' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-17T22:37:44.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:43 smithi096 ceph-mon[36065]: pgmap v393: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T22:37:44.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:43 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/4146282496' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-17T22:37:44.315 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:43 smithi012 ceph-mon[31632]: pgmap v393: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T22:37:44.316 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:43 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/4146282496' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-17T22:37:44.350 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574947 got 55834574952 for osd.1 2024-05-17T22:37:44.351 DEBUG:teuthology.parallel:result is None 2024-05-17T22:37:44.351 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-17T22:37:44.351 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-17T22:37:44.617 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:37:46.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:45 smithi151 ceph-mon[36101]: pgmap v394: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T22:37:46.273 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:45 smithi012 ceph-mon[31632]: pgmap v394: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T22:37:46.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:45 smithi096 ceph-mon[36065]: pgmap v394: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T22:37:46.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.535+0000 7f4aeeed7700 1 -- 172.21.15.12:0/1060134229 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ae810e220 msgr2=0x7f4ae8110730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:46.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.535+0000 7f4aeeed7700 1 --2- 172.21.15.12:0/1060134229 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ae810e220 0x7f4ae8110730 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4ae000b0a0 tx=0x7f4ae000b3b0 comp rx=0 tx=0).stop 2024-05-17T22:37:46.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.535+0000 7f4aeeed7700 1 -- 172.21.15.12:0/1060134229 shutdown_connections 2024-05-17T22:37:46.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.535+0000 7f4aeeed7700 1 --2- 172.21.15.12:0/1060134229 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ae810e220 0x7f4ae8110730 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:46.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.535+0000 7f4aeeed7700 1 --2- 172.21.15.12:0/1060134229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ae8104dc0 0x7f4ae81051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:46.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.535+0000 7f4aeeed7700 1 --2- 172.21.15.12:0/1060134229 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ae8105710 0x7f4ae810dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:46.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.535+0000 7f4aeeed7700 1 -- 172.21.15.12:0/1060134229 >> 172.21.15.12:0/1060134229 conn(0x7f4ae8079bd0 msgr2=0x7f4ae8079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:46.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.535+0000 7f4aeeed7700 1 -- 172.21.15.12:0/1060134229 shutdown_connections 2024-05-17T22:37:46.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.535+0000 7f4aeeed7700 1 -- 172.21.15.12:0/1060134229 wait complete. 2024-05-17T22:37:46.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.536+0000 7f4aeeed7700 1 Processor -- start 2024-05-17T22:37:46.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.536+0000 7f4aeeed7700 1 -- start start 2024-05-17T22:37:46.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.536+0000 7f4aeeed7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ae8104dc0 0x7f4ae81a26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:46.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.536+0000 7f4aeeed7700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ae8105710 0x7f4ae81a2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:46.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.536+0000 7f4aeeed7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ae810e220 0x7f4ae819c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:46.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.536+0000 7f4aeeed7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae8113ea0 con 0x7f4ae8104dc0 2024-05-17T22:37:46.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.536+0000 7f4aeeed7700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae8113d20 con 0x7f4ae810e220 2024-05-17T22:37:46.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.536+0000 7f4aeeed7700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae8114020 con 0x7f4ae8105710 2024-05-17T22:37:46.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.537+0000 7f4aecc73700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ae8104dc0 0x7f4ae81a26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:46.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.537+0000 7f4aecc73700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ae8104dc0 0x7f4ae81a26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55310/0 (socket says 172.21.15.12:55310) 2024-05-17T22:37:46.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.537+0000 7f4aecc73700 1 -- 172.21.15.12:0/501385698 learned_addr learned my addr 172.21.15.12:0/501385698 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:37:46.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.537+0000 7f4ae7fff700 1 --2- 172.21.15.12:0/501385698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ae8105710 0x7f4ae81a2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:46.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.537+0000 7f4aed474700 1 --2- 172.21.15.12:0/501385698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ae810e220 0x7f4ae819c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:46.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.537+0000 7f4aecc73700 1 -- 172.21.15.12:0/501385698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ae8105710 msgr2=0x7f4ae81a2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:46.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.537+0000 7f4aecc73700 1 --2- 172.21.15.12:0/501385698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ae8105710 0x7f4ae81a2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:46.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.537+0000 7f4aecc73700 1 -- 172.21.15.12:0/501385698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ae810e220 msgr2=0x7f4ae819c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:46.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.537+0000 7f4aecc73700 1 --2- 172.21.15.12:0/501385698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ae810e220 0x7f4ae819c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:46.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.537+0000 7f4aecc73700 1 -- 172.21.15.12:0/501385698 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ae819cf80 con 0x7f4ae8104dc0 2024-05-17T22:37:46.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.537+0000 7f4aecc73700 1 --2- 172.21.15.12:0/501385698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ae8104dc0 0x7f4ae81a26b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4adc00b530 tx=0x7f4adc00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:46.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.537+0000 7f4ae5ffb700 1 -- 172.21.15.12:0/501385698 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc015020 con 0x7f4ae8104dc0 2024-05-17T22:37:46.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.538+0000 7f4aeeed7700 1 -- 172.21.15.12:0/501385698 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ae819d270 con 0x7f4ae8104dc0 2024-05-17T22:37:46.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.538+0000 7f4ae5ffb700 1 -- 172.21.15.12:0/501385698 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4adc011ce0 con 0x7f4ae8104dc0 2024-05-17T22:37:46.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.538+0000 7f4ae5ffb700 1 -- 172.21.15.12:0/501385698 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc010d20 con 0x7f4ae8104dc0 2024-05-17T22:37:46.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.538+0000 7f4aeeed7700 1 -- 172.21.15.12:0/501385698 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ae81a9540 con 0x7f4ae8104dc0 2024-05-17T22:37:46.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.539+0000 7f4aeeed7700 1 -- 172.21.15.12:0/501385698 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ae8105250 con 0x7f4ae8104dc0 2024-05-17T22:37:46.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.541+0000 7f4ae5ffb700 1 -- 172.21.15.12:0/501385698 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4adc018020 con 0x7f4ae8104dc0 2024-05-17T22:37:46.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.541+0000 7f4ae5ffb700 1 --2- 172.21.15.12:0/501385698 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f4ad80b2880 0x7f4ad80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:46.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.541+0000 7f4ae7fff700 1 --2- 172.21.15.12:0/501385698 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f4ad80b2880 0x7f4ad80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:46.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.541+0000 7f4ae5ffb700 1 -- 172.21.15.12:0/501385698 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f4adc0e1e40 con 0x7f4ae8104dc0 2024-05-17T22:37:46.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.542+0000 7f4ae7fff700 1 --2- 172.21.15.12:0/501385698 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f4ad80b2880 0x7f4ad80b4d90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4006fd0 tx=0x7f4ad4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:46.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.546+0000 7f4ae5ffb700 1 -- 172.21.15.12:0/501385698 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4adc01f020 con 0x7f4ae8104dc0 2024-05-17T22:37:46.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.756+0000 7f4aeeed7700 1 -- 172.21.15.12:0/501385698 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4ae802ea80 con 0x7f4ad80b2880 2024-05-17T22:37:46.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.763+0000 7f4ae5ffb700 1 -- 172.21.15.12:0/501385698 <== mgr.14152 v2:172.21.15.12:6800/1388514047 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340465 (secure 0 0 0) 0x7f4ae802ea80 con 0x7f4ad80b2880 2024-05-17T22:37:46.771 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:37:46.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.767+0000 7f4aeeed7700 1 -- 172.21.15.12:0/501385698 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f4ad80b2880 msgr2=0x7f4ad80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:46.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.767+0000 7f4aeeed7700 1 --2- 172.21.15.12:0/501385698 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f4ad80b2880 0x7f4ad80b4d90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4006fd0 tx=0x7f4ad4008040 comp rx=0 tx=0).stop 2024-05-17T22:37:46.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.767+0000 7f4aeeed7700 1 -- 172.21.15.12:0/501385698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ae8104dc0 msgr2=0x7f4ae81a26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:46.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.767+0000 7f4aeeed7700 1 --2- 172.21.15.12:0/501385698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ae8104dc0 0x7f4ae81a26b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4adc00b530 tx=0x7f4adc00b8f0 comp rx=0 tx=0).stop 2024-05-17T22:37:46.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.768+0000 7f4aeeed7700 1 -- 172.21.15.12:0/501385698 shutdown_connections 2024-05-17T22:37:46.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.768+0000 7f4aeeed7700 1 --2- 172.21.15.12:0/501385698 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f4ad80b2880 0x7f4ad80b4d90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:46.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.768+0000 7f4aeeed7700 1 --2- 172.21.15.12:0/501385698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ae810e220 0x7f4ae819c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:46.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.768+0000 7f4aeeed7700 1 --2- 172.21.15.12:0/501385698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ae8104dc0 0x7f4ae81a26b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:46.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.768+0000 7f4aeeed7700 1 --2- 172.21.15.12:0/501385698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ae8105710 0x7f4ae81a2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:46.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.768+0000 7f4aeeed7700 1 -- 172.21.15.12:0/501385698 >> 172.21.15.12:0/501385698 conn(0x7f4ae8079bd0 msgr2=0x7f4ae810a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:46.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.768+0000 7f4aeeed7700 1 -- 172.21.15.12:0/501385698 shutdown_connections 2024-05-17T22:37:46.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:46.768+0000 7f4aeeed7700 1 -- 172.21.15.12:0/501385698 wait complete. 2024-05-17T22:37:46.776 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-05-17T22:37:47.579 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":395,"stamp":"2024-05-17T22:37:46.531986+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657597,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":793,"num_read_kb":496,"num_write":303,"num_write_kb":2277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":746,"ondisk_log_size":746,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":388208,"kb_used_data":14096,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553616,"statfs":{"total":1151940427776,"available":1151542902784,"internally_reserved":0,"allocated":14434304,"data_stored":8128889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003628"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.522395+0000","last_change":"2024-05-17T22:34:40.422850+0000","last_active":"2024-05-17T22:37:14.522395+0000","last_peered":"2024-05-17T22:37:14.522395+0000","last_clean":"2024-05-17T22:37:14.522395+0000","last_became_active":"2024-05-17T22:34:40.422474+0000","last_became_peered":"2024-05-17T22:34:40.422474+0000","last_unstale":"2024-05-17T22:37:14.522395+0000","last_undegraded":"2024-05-17T22:37:14.522395+0000","last_fullsized":"2024-05-17T22:37:14.522395+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:23:51.537128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.516491+0000","last_change":"2024-05-17T22:34:40.427907+0000","last_active":"2024-05-17T22:37:14.516491+0000","last_peered":"2024-05-17T22:37:14.516491+0000","last_clean":"2024-05-17T22:37:14.516491+0000","last_became_active":"2024-05-17T22:34:40.427511+0000","last_became_peered":"2024-05-17T22:34:40.427511+0000","last_unstale":"2024-05-17T22:37:14.516491+0000","last_undegraded":"2024-05-17T22:37:14.516491+0000","last_fullsized":"2024-05-17T22:37:14.516491+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:01:46.707820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.525653+0000","last_change":"2024-05-17T22:34:40.436370+0000","last_active":"2024-05-17T22:37:14.525653+0000","last_peered":"2024-05-17T22:37:14.525653+0000","last_clean":"2024-05-17T22:37:14.525653+0000","last_became_active":"2024-05-17T22:34:40.434089+0000","last_became_peered":"2024-05-17T22:34:40.434089+0000","last_unstale":"2024-05-17T22:37:14.525653+0000","last_undegraded":"2024-05-17T22:37:14.525653+0000","last_fullsized":"2024-05-17T22:37:14.525653+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:58:16.775471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.509106+0000","last_change":"2024-05-17T22:35:15.556681+0000","last_active":"2024-05-17T22:37:14.509106+0000","last_peered":"2024-05-17T22:37:14.509106+0000","last_clean":"2024-05-17T22:37:14.509106+0000","last_became_active":"2024-05-17T22:35:15.556427+0000","last_became_peered":"2024-05-17T22:35:15.556427+0000","last_unstale":"2024-05-17T22:37:14.509106+0000","last_undegraded":"2024-05-17T22:37:14.509106+0000","last_fullsized":"2024-05-17T22:37:14.509106+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:28:01.726262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.495402+0000","last_change":"2024-05-17T22:34:40.434394+0000","last_active":"2024-05-17T22:37:14.495402+0000","last_peered":"2024-05-17T22:37:14.495402+0000","last_clean":"2024-05-17T22:37:14.495402+0000","last_became_active":"2024-05-17T22:34:40.433489+0000","last_became_peered":"2024-05-17T22:34:40.433489+0000","last_unstale":"2024-05-17T22:37:14.495402+0000","last_undegraded":"2024-05-17T22:37:14.495402+0000","last_fullsized":"2024-05-17T22:37:14.495402+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:26:24.068814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.515238+0000","last_change":"2024-05-17T22:35:15.553601+0000","last_active":"2024-05-17T22:37:14.515238+0000","last_peered":"2024-05-17T22:37:14.515238+0000","last_clean":"2024-05-17T22:37:14.515238+0000","last_became_active":"2024-05-17T22:35:15.553116+0000","last_became_peered":"2024-05-17T22:35:15.553116+0000","last_unstale":"2024-05-17T22:37:14.515238+0000","last_undegraded":"2024-05-17T22:37:14.515238+0000","last_fullsized":"2024-05-17T22:37:14.515238+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:09:38.401107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.518979+0000","last_change":"2024-05-17T22:34:40.430479+0000","last_active":"2024-05-17T22:37:14.518979+0000","last_peered":"2024-05-17T22:37:14.518979+0000","last_clean":"2024-05-17T22:37:14.518979+0000","last_became_active":"2024-05-17T22:34:40.430025+0000","last_became_peered":"2024-05-17T22:34:40.430025+0000","last_unstale":"2024-05-17T22:37:14.518979+0000","last_undegraded":"2024-05-17T22:37:14.518979+0000","last_fullsized":"2024-05-17T22:37:14.518979+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:14:19.725096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.519153+0000","last_change":"2024-05-17T22:34:40.431711+0000","last_active":"2024-05-17T22:37:14.519153+0000","last_peered":"2024-05-17T22:37:14.519153+0000","last_clean":"2024-05-17T22:37:14.519153+0000","last_became_active":"2024-05-17T22:34:40.431368+0000","last_became_peered":"2024-05-17T22:34:40.431368+0000","last_unstale":"2024-05-17T22:37:14.519153+0000","last_undegraded":"2024-05-17T22:37:14.519153+0000","last_fullsized":"2024-05-17T22:37:14.519153+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:06:34.904652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.521909+0000","last_change":"2024-05-17T22:34:40.435593+0000","last_active":"2024-05-17T22:37:14.521909+0000","last_peered":"2024-05-17T22:37:14.521909+0000","last_clean":"2024-05-17T22:37:14.521909+0000","last_became_active":"2024-05-17T22:34:40.435241+0000","last_became_peered":"2024-05-17T22:34:40.435241+0000","last_unstale":"2024-05-17T22:37:14.521909+0000","last_undegraded":"2024-05-17T22:37:14.521909+0000","last_fullsized":"2024-05-17T22:37:14.521909+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:08:57.239998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.526427+0000","last_change":"2024-05-17T22:34:40.436248+0000","last_active":"2024-05-17T22:37:14.526427+0000","last_peered":"2024-05-17T22:37:14.526427+0000","last_clean":"2024-05-17T22:37:14.526427+0000","last_became_active":"2024-05-17T22:34:40.435824+0000","last_became_peered":"2024-05-17T22:34:40.435824+0000","last_unstale":"2024-05-17T22:37:14.526427+0000","last_undegraded":"2024-05-17T22:37:14.526427+0000","last_fullsized":"2024-05-17T22:37:14.526427+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:39:07.297470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.524640+0000","last_change":"2024-05-17T22:34:40.435843+0000","last_active":"2024-05-17T22:37:14.524640+0000","last_peered":"2024-05-17T22:37:14.524640+0000","last_clean":"2024-05-17T22:37:14.524640+0000","last_became_active":"2024-05-17T22:34:40.435500+0000","last_became_peered":"2024-05-17T22:34:40.435500+0000","last_unstale":"2024-05-17T22:37:14.524640+0000","last_undegraded":"2024-05-17T22:37:14.524640+0000","last_fullsized":"2024-05-17T22:37:14.524640+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:11:28.115850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.501798+0000","last_change":"2024-05-17T22:34:40.432044+0000","last_active":"2024-05-17T22:37:14.501798+0000","last_peered":"2024-05-17T22:37:14.501798+0000","last_clean":"2024-05-17T22:37:14.501798+0000","last_became_active":"2024-05-17T22:34:40.429141+0000","last_became_peered":"2024-05-17T22:34:40.429141+0000","last_unstale":"2024-05-17T22:37:14.501798+0000","last_undegraded":"2024-05-17T22:37:14.501798+0000","last_fullsized":"2024-05-17T22:37:14.501798+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:04:53.428681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.526534+0000","last_change":"2024-05-17T22:34:40.438474+0000","last_active":"2024-05-17T22:37:14.526534+0000","last_peered":"2024-05-17T22:37:14.526534+0000","last_clean":"2024-05-17T22:37:14.526534+0000","last_became_active":"2024-05-17T22:34:40.438109+0000","last_became_peered":"2024-05-17T22:34:40.438109+0000","last_unstale":"2024-05-17T22:37:14.526534+0000","last_undegraded":"2024-05-17T22:37:14.526534+0000","last_fullsized":"2024-05-17T22:37:14.526534+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:26:00.665246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.516486+0000","last_change":"2024-05-17T22:34:40.431563+0000","last_active":"2024-05-17T22:37:14.516486+0000","last_peered":"2024-05-17T22:37:14.516486+0000","last_clean":"2024-05-17T22:37:14.516486+0000","last_became_active":"2024-05-17T22:34:40.430992+0000","last_became_peered":"2024-05-17T22:34:40.430992+0000","last_unstale":"2024-05-17T22:37:14.516486+0000","last_undegraded":"2024-05-17T22:37:14.516486+0000","last_fullsized":"2024-05-17T22:37:14.516486+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:51:26.280440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.516587+0000","last_change":"2024-05-17T22:34:40.433629+0000","last_active":"2024-05-17T22:37:14.516587+0000","last_peered":"2024-05-17T22:37:14.516587+0000","last_clean":"2024-05-17T22:37:14.516587+0000","last_became_active":"2024-05-17T22:34:40.433270+0000","last_became_peered":"2024-05-17T22:34:40.433270+0000","last_unstale":"2024-05-17T22:37:14.516587+0000","last_undegraded":"2024-05-17T22:37:14.516587+0000","last_fullsized":"2024-05-17T22:37:14.516587+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:57:51.164078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.525064+0000","last_change":"2024-05-17T22:34:40.434218+0000","last_active":"2024-05-17T22:37:14.525064+0000","last_peered":"2024-05-17T22:37:14.525064+0000","last_clean":"2024-05-17T22:37:14.525064+0000","last_became_active":"2024-05-17T22:34:40.433849+0000","last_became_peered":"2024-05-17T22:34:40.433849+0000","last_unstale":"2024-05-17T22:37:14.525064+0000","last_undegraded":"2024-05-17T22:37:14.525064+0000","last_fullsized":"2024-05-17T22:37:14.525064+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:41:55.984109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.522605+0000","last_change":"2024-05-17T22:34:40.427892+0000","last_active":"2024-05-17T22:37:14.522605+0000","last_peered":"2024-05-17T22:37:14.522605+0000","last_clean":"2024-05-17T22:37:14.522605+0000","last_became_active":"2024-05-17T22:34:40.427303+0000","last_became_peered":"2024-05-17T22:34:40.427303+0000","last_unstale":"2024-05-17T22:37:14.522605+0000","last_undegraded":"2024-05-17T22:37:14.522605+0000","last_fullsized":"2024-05-17T22:37:14.522605+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:59:38.632123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.524943+0000","last_change":"2024-05-17T22:34:40.424784+0000","last_active":"2024-05-17T22:37:14.524943+0000","last_peered":"2024-05-17T22:37:14.524943+0000","last_clean":"2024-05-17T22:37:14.524943+0000","last_became_active":"2024-05-17T22:34:40.424439+0000","last_became_peered":"2024-05-17T22:34:40.424439+0000","last_unstale":"2024-05-17T22:37:14.524943+0000","last_undegraded":"2024-05-17T22:37:14.524943+0000","last_fullsized":"2024-05-17T22:37:14.524943+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:02:53.681600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.513305+0000","last_change":"2024-05-17T22:34:40.435023+0000","last_active":"2024-05-17T22:37:14.513305+0000","last_peered":"2024-05-17T22:37:14.513305+0000","last_clean":"2024-05-17T22:37:14.513305+0000","last_became_active":"2024-05-17T22:34:40.434441+0000","last_became_peered":"2024-05-17T22:34:40.434441+0000","last_unstale":"2024-05-17T22:37:14.513305+0000","last_undegraded":"2024-05-17T22:37:14.513305+0000","last_fullsized":"2024-05-17T22:37:14.513305+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:16:07.671897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624113+0000","last_change":"2024-05-17T22:34:52.493192+0000","last_active":"2024-05-17T22:36:15.624113+0000","last_peered":"2024-05-17T22:36:15.624113+0000","last_clean":"2024-05-17T22:36:15.624113+0000","last_became_active":"2024-05-17T22:34:52.492786+0000","last_became_peered":"2024-05-17T22:34:52.492786+0000","last_unstale":"2024-05-17T22:36:15.624113+0000","last_undegraded":"2024-05-17T22:36:15.624113+0000","last_fullsized":"2024-05-17T22:36:15.624113+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:43:40.152240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623812+0000","last_change":"2024-05-17T22:34:37.402360+0000","last_active":"2024-05-17T22:36:15.623812+0000","last_peered":"2024-05-17T22:36:15.623812+0000","last_clean":"2024-05-17T22:36:15.623812+0000","last_became_active":"2024-05-17T22:34:37.401882+0000","last_became_peered":"2024-05-17T22:34:37.401882+0000","last_unstale":"2024-05-17T22:36:15.623812+0000","last_undegraded":"2024-05-17T22:36:15.623812+0000","last_fullsized":"2024-05-17T22:36:15.623812+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:53:13.300626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.501991+0000","last_change":"2024-05-17T22:34:40.436259+0000","last_active":"2024-05-17T22:37:14.501991+0000","last_peered":"2024-05-17T22:37:14.501991+0000","last_clean":"2024-05-17T22:37:14.501991+0000","last_became_active":"2024-05-17T22:34:40.435899+0000","last_became_peered":"2024-05-17T22:34:40.435899+0000","last_unstale":"2024-05-17T22:37:14.501991+0000","last_undegraded":"2024-05-17T22:37:14.501991+0000","last_fullsized":"2024-05-17T22:37:14.501991+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:26:29.327513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003713+0000","last_change":"2024-05-17T22:34:52.495236+0000","last_active":"2024-05-17T22:36:16.003713+0000","last_peered":"2024-05-17T22:36:16.003713+0000","last_clean":"2024-05-17T22:36:16.003713+0000","last_became_active":"2024-05-17T22:34:52.494783+0000","last_became_peered":"2024-05-17T22:34:52.494783+0000","last_unstale":"2024-05-17T22:36:16.003713+0000","last_undegraded":"2024-05-17T22:36:16.003713+0000","last_fullsized":"2024-05-17T22:36:16.003713+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:08:15.010706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984822+0000","last_change":"2024-05-17T22:34:37.441129+0000","last_active":"2024-05-17T22:36:15.984822+0000","last_peered":"2024-05-17T22:36:15.984822+0000","last_clean":"2024-05-17T22:36:15.984822+0000","last_became_active":"2024-05-17T22:34:37.440738+0000","last_became_peered":"2024-05-17T22:34:37.440738+0000","last_unstale":"2024-05-17T22:36:15.984822+0000","last_undegraded":"2024-05-17T22:36:15.984822+0000","last_fullsized":"2024-05-17T22:36:15.984822+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:03:16.945076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.525419+0000","last_change":"2024-05-17T22:34:40.434146+0000","last_active":"2024-05-17T22:37:14.525419+0000","last_peered":"2024-05-17T22:37:14.525419+0000","last_clean":"2024-05-17T22:37:14.525419+0000","last_became_active":"2024-05-17T22:34:40.433802+0000","last_became_peered":"2024-05-17T22:34:40.433802+0000","last_unstale":"2024-05-17T22:37:14.525419+0000","last_undegraded":"2024-05-17T22:37:14.525419+0000","last_fullsized":"2024-05-17T22:37:14.525419+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:08:39.905409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004211+0000","last_change":"2024-05-17T22:34:52.501134+0000","last_active":"2024-05-17T22:36:16.004211+0000","last_peered":"2024-05-17T22:36:16.004211+0000","last_clean":"2024-05-17T22:36:16.004211+0000","last_became_active":"2024-05-17T22:34:52.500547+0000","last_became_peered":"2024-05-17T22:34:52.500547+0000","last_unstale":"2024-05-17T22:36:16.004211+0000","last_undegraded":"2024-05-17T22:36:16.004211+0000","last_fullsized":"2024-05-17T22:36:16.004211+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:13:04.200865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.626422+0000","last_change":"2024-05-17T22:34:37.404019+0000","last_active":"2024-05-17T22:36:15.626422+0000","last_peered":"2024-05-17T22:36:15.626422+0000","last_clean":"2024-05-17T22:36:15.626422+0000","last_became_active":"2024-05-17T22:34:37.403607+0000","last_became_peered":"2024-05-17T22:34:37.403607+0000","last_unstale":"2024-05-17T22:36:15.626422+0000","last_undegraded":"2024-05-17T22:36:15.626422+0000","last_fullsized":"2024-05-17T22:36:15.626422+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:30:21.405195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.525402+0000","last_change":"2024-05-17T22:34:40.426889+0000","last_active":"2024-05-17T22:37:14.525402+0000","last_peered":"2024-05-17T22:37:14.525402+0000","last_clean":"2024-05-17T22:37:14.525402+0000","last_became_active":"2024-05-17T22:34:40.426512+0000","last_became_peered":"2024-05-17T22:34:40.426512+0000","last_unstale":"2024-05-17T22:37:14.525402+0000","last_undegraded":"2024-05-17T22:37:14.525402+0000","last_fullsized":"2024-05-17T22:37:14.525402+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:30:51.363177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004528+0000","last_change":"2024-05-17T22:34:52.487514+0000","last_active":"2024-05-17T22:36:16.004528+0000","last_peered":"2024-05-17T22:36:16.004528+0000","last_clean":"2024-05-17T22:36:16.004528+0000","last_became_active":"2024-05-17T22:34:52.487068+0000","last_became_peered":"2024-05-17T22:34:52.487068+0000","last_unstale":"2024-05-17T22:36:16.004528+0000","last_undegraded":"2024-05-17T22:36:16.004528+0000","last_fullsized":"2024-05-17T22:36:16.004528+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:52:13.573986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.626123+0000","last_change":"2024-05-17T22:34:37.400518+0000","last_active":"2024-05-17T22:36:15.626123+0000","last_peered":"2024-05-17T22:36:15.626123+0000","last_clean":"2024-05-17T22:36:15.626123+0000","last_became_active":"2024-05-17T22:34:37.400145+0000","last_became_peered":"2024-05-17T22:34:37.400145+0000","last_unstale":"2024-05-17T22:36:15.626123+0000","last_undegraded":"2024-05-17T22:36:15.626123+0000","last_fullsized":"2024-05-17T22:36:15.626123+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:56:53.593624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.525513+0000","last_change":"2024-05-17T22:34:40.434148+0000","last_active":"2024-05-17T22:37:14.525513+0000","last_peered":"2024-05-17T22:37:14.525513+0000","last_clean":"2024-05-17T22:37:14.525513+0000","last_became_active":"2024-05-17T22:34:40.433613+0000","last_became_peered":"2024-05-17T22:34:40.433613+0000","last_unstale":"2024-05-17T22:37:14.525513+0000","last_undegraded":"2024-05-17T22:37:14.525513+0000","last_fullsized":"2024-05-17T22:37:14.525513+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:09:39.049852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624340+0000","last_change":"2024-05-17T22:34:52.490417+0000","last_active":"2024-05-17T22:36:15.624340+0000","last_peered":"2024-05-17T22:36:15.624340+0000","last_clean":"2024-05-17T22:36:15.624340+0000","last_became_active":"2024-05-17T22:34:52.489953+0000","last_became_peered":"2024-05-17T22:34:52.489953+0000","last_unstale":"2024-05-17T22:36:15.624340+0000","last_undegraded":"2024-05-17T22:36:15.624340+0000","last_fullsized":"2024-05-17T22:36:15.624340+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:23:56.491899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'7","reported_seq":169,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:41.216470+0000","last_change":"2024-05-17T22:34:37.405304+0000","last_active":"2024-05-17T22:37:41.216470+0000","last_peered":"2024-05-17T22:37:41.216470+0000","last_clean":"2024-05-17T22:37:41.216470+0000","last_became_active":"2024-05-17T22:34:37.404552+0000","last_became_peered":"2024-05-17T22:34:37.404552+0000","last_unstale":"2024-05-17T22:37:41.216470+0000","last_undegraded":"2024-05-17T22:37:41.216470+0000","last_fullsized":"2024-05-17T22:37:41.216470+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:57:06.556886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.512663+0000","last_change":"2024-05-17T22:34:40.430537+0000","last_active":"2024-05-17T22:37:14.512663+0000","last_peered":"2024-05-17T22:37:14.512663+0000","last_clean":"2024-05-17T22:37:14.512663+0000","last_became_active":"2024-05-17T22:34:40.430193+0000","last_became_peered":"2024-05-17T22:34:40.430193+0000","last_unstale":"2024-05-17T22:37:14.512663+0000","last_undegraded":"2024-05-17T22:37:14.512663+0000","last_fullsized":"2024-05-17T22:37:14.512663+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:58:42.091445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624546+0000","last_change":"2024-05-17T22:35:15.559077+0000","last_active":"2024-05-17T22:36:15.624546+0000","last_peered":"2024-05-17T22:36:15.624546+0000","last_clean":"2024-05-17T22:36:15.624546+0000","last_became_active":"2024-05-17T22:35:15.558382+0000","last_became_peered":"2024-05-17T22:35:15.558382+0000","last_unstale":"2024-05-17T22:36:15.624546+0000","last_undegraded":"2024-05-17T22:36:15.624546+0000","last_fullsized":"2024-05-17T22:36:15.624546+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:20:16.009620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.626030+0000","last_change":"2024-05-17T22:34:37.405332+0000","last_active":"2024-05-17T22:36:15.626030+0000","last_peered":"2024-05-17T22:36:15.626030+0000","last_clean":"2024-05-17T22:36:15.626030+0000","last_became_active":"2024-05-17T22:34:37.404970+0000","last_became_peered":"2024-05-17T22:34:37.404970+0000","last_unstale":"2024-05-17T22:36:15.626030+0000","last_undegraded":"2024-05-17T22:36:15.626030+0000","last_fullsized":"2024-05-17T22:36:15.626030+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:37:39.933686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.517661+0000","last_change":"2024-05-17T22:34:40.432690+0000","last_active":"2024-05-17T22:37:14.517661+0000","last_peered":"2024-05-17T22:37:14.517661+0000","last_clean":"2024-05-17T22:37:14.517661+0000","last_became_active":"2024-05-17T22:34:40.432274+0000","last_became_peered":"2024-05-17T22:34:40.432274+0000","last_unstale":"2024-05-17T22:37:14.517661+0000","last_undegraded":"2024-05-17T22:37:14.517661+0000","last_fullsized":"2024-05-17T22:37:14.517661+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:12:47.926865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628628+0000","last_change":"2024-05-17T22:34:52.494645+0000","last_active":"2024-05-17T22:36:15.628628+0000","last_peered":"2024-05-17T22:36:15.628628+0000","last_clean":"2024-05-17T22:36:15.628628+0000","last_became_active":"2024-05-17T22:34:52.494255+0000","last_became_peered":"2024-05-17T22:34:52.494255+0000","last_unstale":"2024-05-17T22:36:15.628628+0000","last_undegraded":"2024-05-17T22:36:15.628628+0000","last_fullsized":"2024-05-17T22:36:15.628628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:41:26.633571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685426+0000","last_change":"2024-05-17T22:34:37.398582+0000","last_active":"2024-05-17T22:36:15.685426+0000","last_peered":"2024-05-17T22:36:15.685426+0000","last_clean":"2024-05-17T22:36:15.685426+0000","last_became_active":"2024-05-17T22:34:37.398156+0000","last_became_peered":"2024-05-17T22:34:37.398156+0000","last_unstale":"2024-05-17T22:36:15.685426+0000","last_undegraded":"2024-05-17T22:36:15.685426+0000","last_fullsized":"2024-05-17T22:36:15.685426+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:46:00.276214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.518565+0000","last_change":"2024-05-17T22:35:15.551402+0000","last_active":"2024-05-17T22:37:14.518565+0000","last_peered":"2024-05-17T22:37:14.518565+0000","last_clean":"2024-05-17T22:37:14.518565+0000","last_became_active":"2024-05-17T22:35:15.551021+0000","last_became_peered":"2024-05-17T22:35:15.551021+0000","last_unstale":"2024-05-17T22:37:14.518565+0000","last_undegraded":"2024-05-17T22:37:14.518565+0000","last_fullsized":"2024-05-17T22:37:14.518565+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:35:18.573620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968809+0000","last_change":"2024-05-17T22:34:52.487413+0000","last_active":"2024-05-17T22:36:15.968809+0000","last_peered":"2024-05-17T22:36:15.968809+0000","last_clean":"2024-05-17T22:36:15.968809+0000","last_became_active":"2024-05-17T22:34:52.487032+0000","last_became_peered":"2024-05-17T22:34:52.487032+0000","last_unstale":"2024-05-17T22:36:15.968809+0000","last_undegraded":"2024-05-17T22:36:15.968809+0000","last_fullsized":"2024-05-17T22:36:15.968809+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:34:49.122008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003315+0000","last_change":"2024-05-17T22:34:37.404894+0000","last_active":"2024-05-17T22:36:16.003315+0000","last_peered":"2024-05-17T22:36:16.003315+0000","last_clean":"2024-05-17T22:36:16.003315+0000","last_became_active":"2024-05-17T22:34:37.404567+0000","last_became_peered":"2024-05-17T22:34:37.404567+0000","last_unstale":"2024-05-17T22:36:16.003315+0000","last_undegraded":"2024-05-17T22:36:16.003315+0000","last_fullsized":"2024-05-17T22:36:16.003315+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:13:09.761556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.525152+0000","last_change":"2024-05-17T22:34:40.418267+0000","last_active":"2024-05-17T22:37:14.525152+0000","last_peered":"2024-05-17T22:37:14.525152+0000","last_clean":"2024-05-17T22:37:14.525152+0000","last_became_active":"2024-05-17T22:34:40.417928+0000","last_became_peered":"2024-05-17T22:34:40.417928+0000","last_unstale":"2024-05-17T22:37:14.525152+0000","last_undegraded":"2024-05-17T22:37:14.525152+0000","last_fullsized":"2024-05-17T22:37:14.525152+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:17:37.055004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984205+0000","last_change":"2024-05-17T22:34:52.492126+0000","last_active":"2024-05-17T22:36:15.984205+0000","last_peered":"2024-05-17T22:36:15.984205+0000","last_clean":"2024-05-17T22:36:15.984205+0000","last_became_active":"2024-05-17T22:34:52.491695+0000","last_became_peered":"2024-05-17T22:34:52.491695+0000","last_unstale":"2024-05-17T22:36:15.984205+0000","last_undegraded":"2024-05-17T22:36:15.984205+0000","last_fullsized":"2024-05-17T22:36:15.984205+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:15:17.710279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":164,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:43.219274+0000","last_change":"2024-05-17T22:34:37.406989+0000","last_active":"2024-05-17T22:37:43.219274+0000","last_peered":"2024-05-17T22:37:43.219274+0000","last_clean":"2024-05-17T22:37:43.219274+0000","last_became_active":"2024-05-17T22:34:37.406612+0000","last_became_peered":"2024-05-17T22:34:37.406612+0000","last_unstale":"2024-05-17T22:37:43.219274+0000","last_undegraded":"2024-05-17T22:37:43.219274+0000","last_fullsized":"2024-05-17T22:37:43.219274+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:47:52.480804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.524255+0000","last_change":"2024-05-17T22:34:40.427221+0000","last_active":"2024-05-17T22:37:14.524255+0000","last_peered":"2024-05-17T22:37:14.524255+0000","last_clean":"2024-05-17T22:37:14.524255+0000","last_became_active":"2024-05-17T22:34:40.426703+0000","last_became_peered":"2024-05-17T22:34:40.426703+0000","last_unstale":"2024-05-17T22:37:14.524255+0000","last_undegraded":"2024-05-17T22:37:14.524255+0000","last_fullsized":"2024-05-17T22:37:14.524255+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:49:15.629048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.622725+0000","last_change":"2024-05-17T22:34:52.499086+0000","last_active":"2024-05-17T22:36:15.622725+0000","last_peered":"2024-05-17T22:36:15.622725+0000","last_clean":"2024-05-17T22:36:15.622725+0000","last_became_active":"2024-05-17T22:34:52.498666+0000","last_became_peered":"2024-05-17T22:34:52.498666+0000","last_unstale":"2024-05-17T22:36:15.622725+0000","last_undegraded":"2024-05-17T22:36:15.622725+0000","last_fullsized":"2024-05-17T22:36:15.622725+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:46:25.882964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.005213+0000","last_change":"2024-05-17T22:35:15.836294+0000","last_active":"2024-05-17T22:36:16.005213+0000","last_peered":"2024-05-17T22:36:16.005213+0000","last_clean":"2024-05-17T22:36:16.005213+0000","last_became_active":"2024-05-17T22:35:15.835759+0000","last_became_peered":"2024-05-17T22:35:15.835759+0000","last_unstale":"2024-05-17T22:36:16.005213+0000","last_undegraded":"2024-05-17T22:36:16.005213+0000","last_fullsized":"2024-05-17T22:36:16.005213+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:23:32.106692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.519861+0000","last_change":"2024-05-17T22:34:40.433331+0000","last_active":"2024-05-17T22:37:14.519861+0000","last_peered":"2024-05-17T22:37:14.519861+0000","last_clean":"2024-05-17T22:37:14.519861+0000","last_became_active":"2024-05-17T22:34:40.432989+0000","last_became_peered":"2024-05-17T22:34:40.432989+0000","last_unstale":"2024-05-17T22:37:14.519861+0000","last_undegraded":"2024-05-17T22:37:14.519861+0000","last_fullsized":"2024-05-17T22:37:14.519861+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:22:49.146148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004235+0000","last_change":"2024-05-17T22:34:52.494418+0000","last_active":"2024-05-17T22:36:16.004235+0000","last_peered":"2024-05-17T22:36:16.004235+0000","last_clean":"2024-05-17T22:36:16.004235+0000","last_became_active":"2024-05-17T22:34:52.493953+0000","last_became_peered":"2024-05-17T22:34:52.493953+0000","last_unstale":"2024-05-17T22:36:16.004235+0000","last_undegraded":"2024-05-17T22:36:16.004235+0000","last_fullsized":"2024-05-17T22:36:16.004235+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:50:40.211770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004010+0000","last_change":"2024-05-17T22:34:37.442986+0000","last_active":"2024-05-17T22:36:16.004010+0000","last_peered":"2024-05-17T22:36:16.004010+0000","last_clean":"2024-05-17T22:36:16.004010+0000","last_became_active":"2024-05-17T22:34:37.442399+0000","last_became_peered":"2024-05-17T22:34:37.442399+0000","last_unstale":"2024-05-17T22:36:16.004010+0000","last_undegraded":"2024-05-17T22:36:16.004010+0000","last_fullsized":"2024-05-17T22:36:16.004010+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:34:05.785811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.525214+0000","last_change":"2024-05-17T22:34:40.426793+0000","last_active":"2024-05-17T22:37:14.525214+0000","last_peered":"2024-05-17T22:37:14.525214+0000","last_clean":"2024-05-17T22:37:14.525214+0000","last_became_active":"2024-05-17T22:34:40.426420+0000","last_became_peered":"2024-05-17T22:34:40.426420+0000","last_unstale":"2024-05-17T22:37:14.525214+0000","last_undegraded":"2024-05-17T22:37:14.525214+0000","last_fullsized":"2024-05-17T22:37:14.525214+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:57:07.099254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.622953+0000","last_change":"2024-05-17T22:34:52.498762+0000","last_active":"2024-05-17T22:36:15.622953+0000","last_peered":"2024-05-17T22:36:15.622953+0000","last_clean":"2024-05-17T22:36:15.622953+0000","last_became_active":"2024-05-17T22:34:52.498306+0000","last_became_peered":"2024-05-17T22:34:52.498306+0000","last_unstale":"2024-05-17T22:36:15.622953+0000","last_undegraded":"2024-05-17T22:36:15.622953+0000","last_fullsized":"2024-05-17T22:36:15.622953+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:18:32.717563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'36","reported_seq":321,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:44.220856+0000","last_change":"2024-05-17T22:34:37.407885+0000","last_active":"2024-05-17T22:37:44.220856+0000","last_peered":"2024-05-17T22:37:44.220856+0000","last_clean":"2024-05-17T22:37:44.220856+0000","last_became_active":"2024-05-17T22:34:37.407463+0000","last_became_peered":"2024-05-17T22:34:37.407463+0000","last_unstale":"2024-05-17T22:37:44.220856+0000","last_undegraded":"2024-05-17T22:37:44.220856+0000","last_fullsized":"2024-05-17T22:37:44.220856+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:26:20.288557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62321,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":240,"num_read_kb":181,"num_write":37,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.501714+0000","last_change":"2024-05-17T22:34:40.435800+0000","last_active":"2024-05-17T22:37:14.501714+0000","last_peered":"2024-05-17T22:37:14.501714+0000","last_clean":"2024-05-17T22:37:14.501714+0000","last_became_active":"2024-05-17T22:34:40.435245+0000","last_became_peered":"2024-05-17T22:34:40.435245+0000","last_unstale":"2024-05-17T22:37:14.501714+0000","last_undegraded":"2024-05-17T22:37:14.501714+0000","last_fullsized":"2024-05-17T22:37:14.501714+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:09:58.699026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004448+0000","last_change":"2024-05-17T22:34:52.487336+0000","last_active":"2024-05-17T22:36:16.004448+0000","last_peered":"2024-05-17T22:36:16.004448+0000","last_clean":"2024-05-17T22:36:16.004448+0000","last_became_active":"2024-05-17T22:34:52.486883+0000","last_became_peered":"2024-05-17T22:34:52.486883+0000","last_unstale":"2024-05-17T22:36:16.004448+0000","last_undegraded":"2024-05-17T22:36:16.004448+0000","last_fullsized":"2024-05-17T22:36:16.004448+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:44:41.264072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624096+0000","last_change":"2024-05-17T22:34:37.405044+0000","last_active":"2024-05-17T22:36:15.624096+0000","last_peered":"2024-05-17T22:36:15.624096+0000","last_clean":"2024-05-17T22:36:15.624096+0000","last_became_active":"2024-05-17T22:34:37.404610+0000","last_became_peered":"2024-05-17T22:34:37.404610+0000","last_unstale":"2024-05-17T22:36:15.624096+0000","last_undegraded":"2024-05-17T22:36:15.624096+0000","last_fullsized":"2024-05-17T22:36:15.624096+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:20:37.939069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.521845+0000","last_change":"2024-05-17T22:34:40.430655+0000","last_active":"2024-05-17T22:37:14.521845+0000","last_peered":"2024-05-17T22:37:14.521845+0000","last_clean":"2024-05-17T22:37:14.521845+0000","last_became_active":"2024-05-17T22:34:40.430307+0000","last_became_peered":"2024-05-17T22:34:40.430307+0000","last_unstale":"2024-05-17T22:37:14.521845+0000","last_undegraded":"2024-05-17T22:37:14.521845+0000","last_fullsized":"2024-05-17T22:37:14.521845+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:57:19.036117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628175+0000","last_change":"2024-05-17T22:34:52.494709+0000","last_active":"2024-05-17T22:36:15.628175+0000","last_peered":"2024-05-17T22:36:15.628175+0000","last_clean":"2024-05-17T22:36:15.628175+0000","last_became_active":"2024-05-17T22:34:52.494195+0000","last_became_peered":"2024-05-17T22:34:52.494195+0000","last_unstale":"2024-05-17T22:36:15.628175+0000","last_undegraded":"2024-05-17T22:36:15.628175+0000","last_fullsized":"2024-05-17T22:36:15.628175+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:35:41.188459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003957+0000","last_change":"2024-05-17T22:34:37.402494+0000","last_active":"2024-05-17T22:36:16.003957+0000","last_peered":"2024-05-17T22:36:16.003957+0000","last_clean":"2024-05-17T22:36:16.003957+0000","last_became_active":"2024-05-17T22:34:37.402102+0000","last_became_peered":"2024-05-17T22:34:37.402102+0000","last_unstale":"2024-05-17T22:36:16.003957+0000","last_undegraded":"2024-05-17T22:36:16.003957+0000","last_fullsized":"2024-05-17T22:36:16.003957+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:22:30.915780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.522351+0000","last_change":"2024-05-17T22:34:40.440731+0000","last_active":"2024-05-17T22:37:14.522351+0000","last_peered":"2024-05-17T22:37:14.522351+0000","last_clean":"2024-05-17T22:37:14.522351+0000","last_became_active":"2024-05-17T22:34:40.440374+0000","last_became_peered":"2024-05-17T22:34:40.440374+0000","last_unstale":"2024-05-17T22:37:14.522351+0000","last_undegraded":"2024-05-17T22:37:14.522351+0000","last_fullsized":"2024-05-17T22:37:14.522351+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:26:40.029155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.005856+0000","last_change":"2024-05-17T22:36:16.005856+0000","last_active":"2024-05-17T22:36:16.005856+0000","last_peered":"2024-05-17T22:36:16.005856+0000","last_clean":"2024-05-17T22:36:16.005856+0000","last_became_active":"2024-05-17T22:36:16.005303+0000","last_became_peered":"2024-05-17T22:36:16.005303+0000","last_unstale":"2024-05-17T22:36:16.005856+0000","last_undegraded":"2024-05-17T22:36:16.005856+0000","last_fullsized":"2024-05-17T22:36:16.005856+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:41:15.708879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":166,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:41.237432+0000","last_change":"2024-05-17T22:34:37.403123+0000","last_active":"2024-05-17T22:37:41.237432+0000","last_peered":"2024-05-17T22:37:41.237432+0000","last_clean":"2024-05-17T22:37:41.237432+0000","last_became_active":"2024-05-17T22:34:37.402702+0000","last_became_peered":"2024-05-17T22:34:37.402702+0000","last_unstale":"2024-05-17T22:37:41.237432+0000","last_undegraded":"2024-05-17T22:37:41.237432+0000","last_fullsized":"2024-05-17T22:37:41.237432+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:50:02.031651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.512621+0000","last_change":"2024-05-17T22:34:40.431895+0000","last_active":"2024-05-17T22:37:14.512621+0000","last_peered":"2024-05-17T22:37:14.512621+0000","last_clean":"2024-05-17T22:37:14.512621+0000","last_became_active":"2024-05-17T22:34:40.431527+0000","last_became_peered":"2024-05-17T22:34:40.431527+0000","last_unstale":"2024-05-17T22:37:14.512621+0000","last_undegraded":"2024-05-17T22:37:14.512621+0000","last_fullsized":"2024-05-17T22:37:14.512621+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:30:09.404255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.686140+0000","last_change":"2024-05-17T22:34:52.495583+0000","last_active":"2024-05-17T22:36:15.686140+0000","last_peered":"2024-05-17T22:36:15.686140+0000","last_clean":"2024-05-17T22:36:15.686140+0000","last_became_active":"2024-05-17T22:34:52.495030+0000","last_became_peered":"2024-05-17T22:34:52.495030+0000","last_unstale":"2024-05-17T22:36:15.686140+0000","last_undegraded":"2024-05-17T22:36:15.686140+0000","last_fullsized":"2024-05-17T22:36:15.686140+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:04:54.046891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":61,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625592+0000","last_change":"2024-05-17T22:34:37.442154+0000","last_active":"2024-05-17T22:36:15.625592+0000","last_peered":"2024-05-17T22:36:15.625592+0000","last_clean":"2024-05-17T22:36:15.625592+0000","last_became_active":"2024-05-17T22:34:37.441691+0000","last_became_peered":"2024-05-17T22:34:37.441691+0000","last_unstale":"2024-05-17T22:36:15.625592+0000","last_undegraded":"2024-05-17T22:36:15.625592+0000","last_fullsized":"2024-05-17T22:36:15.625592+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:47:29.862031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.517818+0000","last_change":"2024-05-17T22:34:40.430894+0000","last_active":"2024-05-17T22:37:14.517818+0000","last_peered":"2024-05-17T22:37:14.517818+0000","last_clean":"2024-05-17T22:37:14.517818+0000","last_became_active":"2024-05-17T22:34:40.430328+0000","last_became_peered":"2024-05-17T22:34:40.430328+0000","last_unstale":"2024-05-17T22:37:14.517818+0000","last_undegraded":"2024-05-17T22:37:14.517818+0000","last_fullsized":"2024-05-17T22:37:14.517818+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:36:46.619111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003488+0000","last_change":"2024-05-17T22:34:52.487555+0000","last_active":"2024-05-17T22:36:16.003488+0000","last_peered":"2024-05-17T22:36:16.003488+0000","last_clean":"2024-05-17T22:36:16.003488+0000","last_became_active":"2024-05-17T22:34:52.487076+0000","last_became_peered":"2024-05-17T22:34:52.487076+0000","last_unstale":"2024-05-17T22:36:16.003488+0000","last_undegraded":"2024-05-17T22:36:16.003488+0000","last_fullsized":"2024-05-17T22:36:16.003488+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:05:46.720665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628417+0000","last_change":"2024-05-17T22:35:15.558860+0000","last_active":"2024-05-17T22:36:15.628417+0000","last_peered":"2024-05-17T22:36:15.628417+0000","last_clean":"2024-05-17T22:36:15.628417+0000","last_became_active":"2024-05-17T22:35:15.553683+0000","last_became_peered":"2024-05-17T22:35:15.553683+0000","last_unstale":"2024-05-17T22:36:15.628417+0000","last_undegraded":"2024-05-17T22:36:15.628417+0000","last_fullsized":"2024-05-17T22:36:15.628417+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:27:24.978282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.506967+0000","last_change":"2024-05-17T22:34:40.433499+0000","last_active":"2024-05-17T22:37:14.506967+0000","last_peered":"2024-05-17T22:37:14.506967+0000","last_clean":"2024-05-17T22:37:14.506967+0000","last_became_active":"2024-05-17T22:34:40.431370+0000","last_became_peered":"2024-05-17T22:34:40.431370+0000","last_unstale":"2024-05-17T22:37:14.506967+0000","last_undegraded":"2024-05-17T22:37:14.506967+0000","last_fullsized":"2024-05-17T22:37:14.506967+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:25:08.383021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625953+0000","last_change":"2024-05-17T22:34:52.489627+0000","last_active":"2024-05-17T22:36:15.625953+0000","last_peered":"2024-05-17T22:36:15.625953+0000","last_clean":"2024-05-17T22:36:15.625953+0000","last_became_active":"2024-05-17T22:34:52.489173+0000","last_became_peered":"2024-05-17T22:34:52.489173+0000","last_unstale":"2024-05-17T22:36:15.625953+0000","last_undegraded":"2024-05-17T22:36:15.625953+0000","last_fullsized":"2024-05-17T22:36:15.625953+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:52:26.908817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623360+0000","last_change":"2024-05-17T22:35:15.553325+0000","last_active":"2024-05-17T22:36:15.623360+0000","last_peered":"2024-05-17T22:36:15.623360+0000","last_clean":"2024-05-17T22:36:15.623360+0000","last_became_active":"2024-05-17T22:35:15.552867+0000","last_became_peered":"2024-05-17T22:35:15.552867+0000","last_unstale":"2024-05-17T22:36:15.623360+0000","last_undegraded":"2024-05-17T22:36:15.623360+0000","last_fullsized":"2024-05-17T22:36:15.623360+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:28:36.618331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.525936+0000","last_change":"2024-05-17T22:34:40.423221+0000","last_active":"2024-05-17T22:37:14.525936+0000","last_peered":"2024-05-17T22:37:14.525936+0000","last_clean":"2024-05-17T22:37:14.525936+0000","last_became_active":"2024-05-17T22:34:40.422838+0000","last_became_peered":"2024-05-17T22:34:40.422838+0000","last_unstale":"2024-05-17T22:37:14.525936+0000","last_undegraded":"2024-05-17T22:37:14.525936+0000","last_fullsized":"2024-05-17T22:37:14.525936+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:39:51.873212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685905+0000","last_change":"2024-05-17T22:34:52.484802+0000","last_active":"2024-05-17T22:36:15.685905+0000","last_peered":"2024-05-17T22:36:15.685905+0000","last_clean":"2024-05-17T22:36:15.685905+0000","last_became_active":"2024-05-17T22:34:52.483766+0000","last_became_peered":"2024-05-17T22:34:52.483766+0000","last_unstale":"2024-05-17T22:36:15.685905+0000","last_undegraded":"2024-05-17T22:36:15.685905+0000","last_fullsized":"2024-05-17T22:36:15.685905+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:32:01.427078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004472+0000","last_change":"2024-05-17T22:34:37.404797+0000","last_active":"2024-05-17T22:36:16.004472+0000","last_peered":"2024-05-17T22:36:16.004472+0000","last_clean":"2024-05-17T22:36:16.004472+0000","last_became_active":"2024-05-17T22:34:37.404393+0000","last_became_peered":"2024-05-17T22:34:37.404393+0000","last_unstale":"2024-05-17T22:36:16.004472+0000","last_undegraded":"2024-05-17T22:36:16.004472+0000","last_fullsized":"2024-05-17T22:36:16.004472+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:26:36.887919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.515773+0000","last_change":"2024-05-17T22:34:40.428210+0000","last_active":"2024-05-17T22:37:14.515773+0000","last_peered":"2024-05-17T22:37:14.515773+0000","last_clean":"2024-05-17T22:37:14.515773+0000","last_became_active":"2024-05-17T22:34:40.427845+0000","last_became_peered":"2024-05-17T22:34:40.427845+0000","last_unstale":"2024-05-17T22:37:14.515773+0000","last_undegraded":"2024-05-17T22:37:14.515773+0000","last_fullsized":"2024-05-17T22:37:14.515773+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:45:30.437139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.686040+0000","last_change":"2024-05-17T22:34:52.490502+0000","last_active":"2024-05-17T22:36:15.686040+0000","last_peered":"2024-05-17T22:36:15.686040+0000","last_clean":"2024-05-17T22:36:15.686040+0000","last_became_active":"2024-05-17T22:34:52.490008+0000","last_became_peered":"2024-05-17T22:34:52.490008+0000","last_unstale":"2024-05-17T22:36:15.686040+0000","last_undegraded":"2024-05-17T22:36:15.686040+0000","last_fullsized":"2024-05-17T22:36:15.686040+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:58:29.122668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.517916+0000","last_change":"2024-05-17T22:34:40.432248+0000","last_active":"2024-05-17T22:37:14.517916+0000","last_peered":"2024-05-17T22:37:14.517916+0000","last_clean":"2024-05-17T22:37:14.517916+0000","last_became_active":"2024-05-17T22:34:40.431863+0000","last_became_peered":"2024-05-17T22:34:40.431863+0000","last_unstale":"2024-05-17T22:37:14.517916+0000","last_undegraded":"2024-05-17T22:37:14.517916+0000","last_fullsized":"2024-05-17T22:37:14.517916+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:20:34.730542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":164,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:42.238039+0000","last_change":"2024-05-17T22:34:37.441186+0000","last_active":"2024-05-17T22:37:42.238039+0000","last_peered":"2024-05-17T22:37:42.238039+0000","last_clean":"2024-05-17T22:37:42.238039+0000","last_became_active":"2024-05-17T22:34:37.440781+0000","last_became_peered":"2024-05-17T22:34:37.440781+0000","last_unstale":"2024-05-17T22:37:42.238039+0000","last_undegraded":"2024-05-17T22:37:42.238039+0000","last_fullsized":"2024-05-17T22:37:42.238039+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:44:34.435147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.005134+0000","last_change":"2024-05-17T22:34:52.491771+0000","last_active":"2024-05-17T22:36:16.005134+0000","last_peered":"2024-05-17T22:36:16.005134+0000","last_clean":"2024-05-17T22:36:16.005134+0000","last_became_active":"2024-05-17T22:34:52.491172+0000","last_became_peered":"2024-05-17T22:34:52.491172+0000","last_unstale":"2024-05-17T22:36:16.005134+0000","last_undegraded":"2024-05-17T22:36:16.005134+0000","last_fullsized":"2024-05-17T22:36:16.005134+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:15:21.010592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.511507+0000","last_change":"2024-05-17T22:34:40.430860+0000","last_active":"2024-05-17T22:37:14.511507+0000","last_peered":"2024-05-17T22:37:14.511507+0000","last_clean":"2024-05-17T22:37:14.511507+0000","last_became_active":"2024-05-17T22:34:40.430542+0000","last_became_peered":"2024-05-17T22:34:40.430542+0000","last_unstale":"2024-05-17T22:37:14.511507+0000","last_undegraded":"2024-05-17T22:37:14.511507+0000","last_fullsized":"2024-05-17T22:37:14.511507+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:16:26.257380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004114+0000","last_change":"2024-05-17T22:34:37.403026+0000","last_active":"2024-05-17T22:36:16.004114+0000","last_peered":"2024-05-17T22:36:16.004114+0000","last_clean":"2024-05-17T22:36:16.004114+0000","last_became_active":"2024-05-17T22:34:37.402560+0000","last_became_peered":"2024-05-17T22:34:37.402560+0000","last_unstale":"2024-05-17T22:36:16.004114+0000","last_undegraded":"2024-05-17T22:36:16.004114+0000","last_fullsized":"2024-05-17T22:36:16.004114+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:23:03.784411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":119,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003503+0000","last_change":"2024-05-17T22:33:11.055878+0000","last_active":"2024-05-17T22:36:16.003503+0000","last_peered":"2024-05-17T22:36:16.003503+0000","last_clean":"2024-05-17T22:36:16.003503+0000","last_became_active":"2024-05-17T22:33:11.050409+0000","last_became_peered":"2024-05-17T22:33:11.050409+0000","last_unstale":"2024-05-17T22:36:16.003503+0000","last_undegraded":"2024-05-17T22:36:16.003503+0000","last_fullsized":"2024-05-17T22:36:16.003503+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:29:43.341170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:29:43.341170+0000","last_clean_scrub_stamp":"2024-05-17T22:29:43.341170+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:30:54.153865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003960+0000","last_change":"2024-05-17T22:34:52.497064+0000","last_active":"2024-05-17T22:36:16.003960+0000","last_peered":"2024-05-17T22:36:16.003960+0000","last_clean":"2024-05-17T22:36:16.003960+0000","last_became_active":"2024-05-17T22:34:52.496611+0000","last_became_peered":"2024-05-17T22:34:52.496611+0000","last_unstale":"2024-05-17T22:36:16.003960+0000","last_undegraded":"2024-05-17T22:36:16.003960+0000","last_fullsized":"2024-05-17T22:36:16.003960+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:20:29.597589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.519744+0000","last_change":"2024-05-17T22:35:15.555413+0000","last_active":"2024-05-17T22:37:14.519744+0000","last_peered":"2024-05-17T22:37:14.519744+0000","last_clean":"2024-05-17T22:37:14.519744+0000","last_became_active":"2024-05-17T22:35:15.555005+0000","last_became_peered":"2024-05-17T22:35:15.555005+0000","last_unstale":"2024-05-17T22:37:14.519744+0000","last_undegraded":"2024-05-17T22:37:14.519744+0000","last_fullsized":"2024-05-17T22:37:14.519744+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:51:49.860673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003705+0000","last_change":"2024-05-17T22:34:37.406546+0000","last_active":"2024-05-17T22:36:16.003705+0000","last_peered":"2024-05-17T22:36:16.003705+0000","last_clean":"2024-05-17T22:36:16.003705+0000","last_became_active":"2024-05-17T22:34:37.406118+0000","last_became_peered":"2024-05-17T22:34:37.406118+0000","last_unstale":"2024-05-17T22:36:16.003705+0000","last_undegraded":"2024-05-17T22:36:16.003705+0000","last_fullsized":"2024-05-17T22:36:16.003705+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:47:16.350091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.006276+0000","last_change":"2024-05-17T22:36:16.005647+0000","last_active":"2024-05-17T22:36:16.006276+0000","last_peered":"2024-05-17T22:36:16.006276+0000","last_clean":"2024-05-17T22:36:16.006276+0000","last_became_active":"2024-05-17T22:36:16.005128+0000","last_became_peered":"2024-05-17T22:36:16.005128+0000","last_unstale":"2024-05-17T22:36:16.006276+0000","last_undegraded":"2024-05-17T22:36:16.006276+0000","last_fullsized":"2024-05-17T22:36:16.006276+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:16:24.481236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,7,6],"acting":[11,10,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.626356+0000","last_change":"2024-05-17T22:34:37.442676+0000","last_active":"2024-05-17T22:36:15.626356+0000","last_peered":"2024-05-17T22:36:15.626356+0000","last_clean":"2024-05-17T22:36:15.626356+0000","last_became_active":"2024-05-17T22:34:37.442275+0000","last_became_peered":"2024-05-17T22:34:37.442275+0000","last_unstale":"2024-05-17T22:36:15.626356+0000","last_undegraded":"2024-05-17T22:36:15.626356+0000","last_fullsized":"2024-05-17T22:36:15.626356+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:54:18.542987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.513754+0000","last_change":"2024-05-17T22:34:40.424266+0000","last_active":"2024-05-17T22:37:14.513754+0000","last_peered":"2024-05-17T22:37:14.513754+0000","last_clean":"2024-05-17T22:37:14.513754+0000","last_became_active":"2024-05-17T22:34:40.423905+0000","last_became_peered":"2024-05-17T22:34:40.423905+0000","last_unstale":"2024-05-17T22:37:14.513754+0000","last_undegraded":"2024-05-17T22:37:14.513754+0000","last_fullsized":"2024-05-17T22:37:14.513754+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:03:43.397871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984174+0000","last_change":"2024-05-17T22:34:52.493386+0000","last_active":"2024-05-17T22:36:15.984174+0000","last_peered":"2024-05-17T22:36:15.984174+0000","last_clean":"2024-05-17T22:36:15.984174+0000","last_became_active":"2024-05-17T22:34:52.492997+0000","last_became_peered":"2024-05-17T22:34:52.492997+0000","last_unstale":"2024-05-17T22:36:15.984174+0000","last_undegraded":"2024-05-17T22:36:15.984174+0000","last_fullsized":"2024-05-17T22:36:15.984174+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:14:40.818271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'12","reported_seq":57,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:48.068053+0000","last_change":"2024-05-17T22:34:37.399397+0000","last_active":"2024-05-17T22:36:48.068053+0000","last_peered":"2024-05-17T22:36:48.068053+0000","last_clean":"2024-05-17T22:36:48.068053+0000","last_became_active":"2024-05-17T22:34:37.399062+0000","last_became_peered":"2024-05-17T22:34:37.399062+0000","last_unstale":"2024-05-17T22:36:48.068053+0000","last_undegraded":"2024-05-17T22:36:48.068053+0000","last_fullsized":"2024-05-17T22:36:48.068053+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:02:05.230580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.518907+0000","last_change":"2024-05-17T22:34:40.426995+0000","last_active":"2024-05-17T22:37:14.518907+0000","last_peered":"2024-05-17T22:37:14.518907+0000","last_clean":"2024-05-17T22:37:14.518907+0000","last_became_active":"2024-05-17T22:34:40.426226+0000","last_became_peered":"2024-05-17T22:34:40.426226+0000","last_unstale":"2024-05-17T22:37:14.518907+0000","last_undegraded":"2024-05-17T22:37:14.518907+0000","last_fullsized":"2024-05-17T22:37:14.518907+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:49:53.331076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623704+0000","last_change":"2024-05-17T22:34:52.487098+0000","last_active":"2024-05-17T22:36:15.623704+0000","last_peered":"2024-05-17T22:36:15.623704+0000","last_clean":"2024-05-17T22:36:15.623704+0000","last_became_active":"2024-05-17T22:34:52.486605+0000","last_became_peered":"2024-05-17T22:34:52.486605+0000","last_unstale":"2024-05-17T22:36:15.623704+0000","last_undegraded":"2024-05-17T22:36:15.623704+0000","last_fullsized":"2024-05-17T22:36:15.623704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:33:28.234208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004689+0000","last_change":"2024-05-17T22:34:37.800830+0000","last_active":"2024-05-17T22:36:16.004689+0000","last_peered":"2024-05-17T22:36:16.004689+0000","last_clean":"2024-05-17T22:36:16.004689+0000","last_became_active":"2024-05-17T22:34:37.800405+0000","last_became_peered":"2024-05-17T22:34:37.800405+0000","last_unstale":"2024-05-17T22:36:16.004689+0000","last_undegraded":"2024-05-17T22:36:16.004689+0000","last_fullsized":"2024-05-17T22:36:16.004689+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:19:10.087442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.520738+0000","last_change":"2024-05-17T22:34:40.426307+0000","last_active":"2024-05-17T22:37:14.520738+0000","last_peered":"2024-05-17T22:37:14.520738+0000","last_clean":"2024-05-17T22:37:14.520738+0000","last_became_active":"2024-05-17T22:34:40.425969+0000","last_became_peered":"2024-05-17T22:34:40.425969+0000","last_unstale":"2024-05-17T22:37:14.520738+0000","last_undegraded":"2024-05-17T22:37:14.520738+0000","last_fullsized":"2024-05-17T22:37:14.520738+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:25:49.680886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004511+0000","last_change":"2024-05-17T22:35:15.556857+0000","last_active":"2024-05-17T22:36:16.004511+0000","last_peered":"2024-05-17T22:36:16.004511+0000","last_clean":"2024-05-17T22:36:16.004511+0000","last_became_active":"2024-05-17T22:35:15.556298+0000","last_became_peered":"2024-05-17T22:35:15.556298+0000","last_unstale":"2024-05-17T22:36:16.004511+0000","last_undegraded":"2024-05-17T22:36:16.004511+0000","last_fullsized":"2024-05-17T22:36:16.004511+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:23:20.111544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.805197+0000","last_change":"2024-05-17T22:34:37.442971+0000","last_active":"2024-05-17T22:36:15.805197+0000","last_peered":"2024-05-17T22:36:15.805197+0000","last_clean":"2024-05-17T22:36:15.805197+0000","last_became_active":"2024-05-17T22:34:37.442536+0000","last_became_peered":"2024-05-17T22:34:37.442536+0000","last_unstale":"2024-05-17T22:36:15.805197+0000","last_undegraded":"2024-05-17T22:36:15.805197+0000","last_fullsized":"2024-05-17T22:36:15.805197+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:03:45.224071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.467874+0000","last_change":"2024-05-17T22:34:40.432056+0000","last_active":"2024-05-17T22:37:14.467874+0000","last_peered":"2024-05-17T22:37:14.467874+0000","last_clean":"2024-05-17T22:37:14.467874+0000","last_became_active":"2024-05-17T22:34:40.431626+0000","last_became_peered":"2024-05-17T22:34:40.431626+0000","last_unstale":"2024-05-17T22:37:14.467874+0000","last_undegraded":"2024-05-17T22:37:14.467874+0000","last_fullsized":"2024-05-17T22:37:14.467874+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:12:24.650746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623664+0000","last_change":"2024-05-17T22:34:52.500451+0000","last_active":"2024-05-17T22:36:15.623664+0000","last_peered":"2024-05-17T22:36:15.623664+0000","last_clean":"2024-05-17T22:36:15.623664+0000","last_became_active":"2024-05-17T22:34:52.500031+0000","last_became_peered":"2024-05-17T22:34:52.500031+0000","last_unstale":"2024-05-17T22:36:15.623664+0000","last_undegraded":"2024-05-17T22:36:15.623664+0000","last_fullsized":"2024-05-17T22:36:15.623664+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:54:36.825116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984386+0000","last_change":"2024-05-17T22:34:37.402335+0000","last_active":"2024-05-17T22:36:15.984386+0000","last_peered":"2024-05-17T22:36:15.984386+0000","last_clean":"2024-05-17T22:36:15.984386+0000","last_became_active":"2024-05-17T22:34:37.401982+0000","last_became_peered":"2024-05-17T22:34:37.401982+0000","last_unstale":"2024-05-17T22:36:15.984386+0000","last_undegraded":"2024-05-17T22:36:15.984386+0000","last_fullsized":"2024-05-17T22:36:15.984386+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:58:06.194335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.517775+0000","last_change":"2024-05-17T22:34:40.425524+0000","last_active":"2024-05-17T22:37:14.517775+0000","last_peered":"2024-05-17T22:37:14.517775+0000","last_clean":"2024-05-17T22:37:14.517775+0000","last_became_active":"2024-05-17T22:34:40.425135+0000","last_became_peered":"2024-05-17T22:34:40.425135+0000","last_unstale":"2024-05-17T22:37:14.517775+0000","last_undegraded":"2024-05-17T22:37:14.517775+0000","last_fullsized":"2024-05-17T22:37:14.517775+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:41:14.176477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004843+0000","last_change":"2024-05-17T22:34:52.496290+0000","last_active":"2024-05-17T22:36:16.004843+0000","last_peered":"2024-05-17T22:36:16.004843+0000","last_clean":"2024-05-17T22:36:16.004843+0000","last_became_active":"2024-05-17T22:34:52.495817+0000","last_became_peered":"2024-05-17T22:34:52.495817+0000","last_unstale":"2024-05-17T22:36:16.004843+0000","last_undegraded":"2024-05-17T22:36:16.004843+0000","last_fullsized":"2024-05-17T22:36:16.004843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:47:22.056372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624160+0000","last_change":"2024-05-17T22:34:37.404234+0000","last_active":"2024-05-17T22:36:15.624160+0000","last_peered":"2024-05-17T22:36:15.624160+0000","last_clean":"2024-05-17T22:36:15.624160+0000","last_became_active":"2024-05-17T22:34:37.403561+0000","last_became_peered":"2024-05-17T22:34:37.403561+0000","last_unstale":"2024-05-17T22:36:15.624160+0000","last_undegraded":"2024-05-17T22:36:15.624160+0000","last_fullsized":"2024-05-17T22:36:15.624160+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:30:28.432641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.517802+0000","last_change":"2024-05-17T22:34:40.426522+0000","last_active":"2024-05-17T22:37:14.517802+0000","last_peered":"2024-05-17T22:37:14.517802+0000","last_clean":"2024-05-17T22:37:14.517802+0000","last_became_active":"2024-05-17T22:34:40.426139+0000","last_became_peered":"2024-05-17T22:34:40.426139+0000","last_unstale":"2024-05-17T22:37:14.517802+0000","last_undegraded":"2024-05-17T22:37:14.517802+0000","last_fullsized":"2024-05-17T22:37:14.517802+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:30:19.503513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004076+0000","last_change":"2024-05-17T22:34:52.487131+0000","last_active":"2024-05-17T22:36:16.004076+0000","last_peered":"2024-05-17T22:36:16.004076+0000","last_clean":"2024-05-17T22:36:16.004076+0000","last_became_active":"2024-05-17T22:34:52.486654+0000","last_became_peered":"2024-05-17T22:34:52.486654+0000","last_unstale":"2024-05-17T22:36:16.004076+0000","last_undegraded":"2024-05-17T22:36:16.004076+0000","last_fullsized":"2024-05-17T22:36:16.004076+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:28:25.041746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625893+0000","last_change":"2024-05-17T22:34:37.405020+0000","last_active":"2024-05-17T22:36:15.625893+0000","last_peered":"2024-05-17T22:36:15.625893+0000","last_clean":"2024-05-17T22:36:15.625893+0000","last_became_active":"2024-05-17T22:34:37.404633+0000","last_became_peered":"2024-05-17T22:34:37.404633+0000","last_unstale":"2024-05-17T22:36:15.625893+0000","last_undegraded":"2024-05-17T22:36:15.625893+0000","last_fullsized":"2024-05-17T22:36:15.625893+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:06:00.001190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.525054+0000","last_change":"2024-05-17T22:34:40.435374+0000","last_active":"2024-05-17T22:37:14.525054+0000","last_peered":"2024-05-17T22:37:14.525054+0000","last_clean":"2024-05-17T22:37:14.525054+0000","last_became_active":"2024-05-17T22:34:40.434522+0000","last_became_peered":"2024-05-17T22:34:40.434522+0000","last_unstale":"2024-05-17T22:37:14.525054+0000","last_undegraded":"2024-05-17T22:37:14.525054+0000","last_fullsized":"2024-05-17T22:37:14.525054+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:36:41.244970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.805274+0000","last_change":"2024-05-17T22:34:52.491708+0000","last_active":"2024-05-17T22:36:15.805274+0000","last_peered":"2024-05-17T22:36:15.805274+0000","last_clean":"2024-05-17T22:36:15.805274+0000","last_became_active":"2024-05-17T22:34:52.491293+0000","last_became_peered":"2024-05-17T22:34:52.491293+0000","last_unstale":"2024-05-17T22:36:15.805274+0000","last_undegraded":"2024-05-17T22:36:15.805274+0000","last_fullsized":"2024-05-17T22:36:15.805274+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:23:01.634026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624679+0000","last_change":"2024-05-17T22:34:37.800525+0000","last_active":"2024-05-17T22:36:15.624679+0000","last_peered":"2024-05-17T22:36:15.624679+0000","last_clean":"2024-05-17T22:36:15.624679+0000","last_became_active":"2024-05-17T22:34:37.800093+0000","last_became_peered":"2024-05-17T22:34:37.800093+0000","last_unstale":"2024-05-17T22:36:15.624679+0000","last_undegraded":"2024-05-17T22:36:15.624679+0000","last_fullsized":"2024-05-17T22:36:15.624679+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:25:27.249889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.524773+0000","last_change":"2024-05-17T22:34:40.434891+0000","last_active":"2024-05-17T22:37:14.524773+0000","last_peered":"2024-05-17T22:37:14.524773+0000","last_clean":"2024-05-17T22:37:14.524773+0000","last_became_active":"2024-05-17T22:34:40.434543+0000","last_became_peered":"2024-05-17T22:34:40.434543+0000","last_unstale":"2024-05-17T22:37:14.524773+0000","last_undegraded":"2024-05-17T22:37:14.524773+0000","last_fullsized":"2024-05-17T22:37:14.524773+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:17:47.770225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968412+0000","last_change":"2024-05-17T22:34:52.491544+0000","last_active":"2024-05-17T22:36:15.968412+0000","last_peered":"2024-05-17T22:36:15.968412+0000","last_clean":"2024-05-17T22:36:15.968412+0000","last_became_active":"2024-05-17T22:34:52.491138+0000","last_became_peered":"2024-05-17T22:34:52.491138+0000","last_unstale":"2024-05-17T22:36:15.968412+0000","last_undegraded":"2024-05-17T22:36:15.968412+0000","last_fullsized":"2024-05-17T22:36:15.968412+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:34:20.411299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004302+0000","last_change":"2024-05-17T22:34:37.403220+0000","last_active":"2024-05-17T22:36:16.004302+0000","last_peered":"2024-05-17T22:36:16.004302+0000","last_clean":"2024-05-17T22:36:16.004302+0000","last_became_active":"2024-05-17T22:34:37.402856+0000","last_became_peered":"2024-05-17T22:34:37.402856+0000","last_unstale":"2024-05-17T22:36:16.004302+0000","last_undegraded":"2024-05-17T22:36:16.004302+0000","last_fullsized":"2024-05-17T22:36:16.004302+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:04:19.469294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.511989+0000","last_change":"2024-05-17T22:34:40.434487+0000","last_active":"2024-05-17T22:37:14.511989+0000","last_peered":"2024-05-17T22:37:14.511989+0000","last_clean":"2024-05-17T22:37:14.511989+0000","last_became_active":"2024-05-17T22:34:40.434160+0000","last_became_peered":"2024-05-17T22:34:40.434160+0000","last_unstale":"2024-05-17T22:37:14.511989+0000","last_undegraded":"2024-05-17T22:37:14.511989+0000","last_fullsized":"2024-05-17T22:37:14.511989+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:24:38.782999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003180+0000","last_change":"2024-05-17T22:34:52.487887+0000","last_active":"2024-05-17T22:36:16.003180+0000","last_peered":"2024-05-17T22:36:16.003180+0000","last_clean":"2024-05-17T22:36:16.003180+0000","last_became_active":"2024-05-17T22:34:52.486837+0000","last_became_peered":"2024-05-17T22:34:52.486837+0000","last_unstale":"2024-05-17T22:36:16.003180+0000","last_undegraded":"2024-05-17T22:36:16.003180+0000","last_fullsized":"2024-05-17T22:36:16.003180+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:12:22.206536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.686208+0000","last_change":"2024-05-17T22:34:37.399498+0000","last_active":"2024-05-17T22:36:15.686208+0000","last_peered":"2024-05-17T22:36:15.686208+0000","last_clean":"2024-05-17T22:36:15.686208+0000","last_became_active":"2024-05-17T22:34:37.399144+0000","last_became_peered":"2024-05-17T22:34:37.399144+0000","last_unstale":"2024-05-17T22:36:15.686208+0000","last_undegraded":"2024-05-17T22:36:15.686208+0000","last_fullsized":"2024-05-17T22:36:15.686208+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:13:37.346006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.514102+0000","last_change":"2024-05-17T22:34:40.433839+0000","last_active":"2024-05-17T22:37:14.514102+0000","last_peered":"2024-05-17T22:37:14.514102+0000","last_clean":"2024-05-17T22:37:14.514102+0000","last_became_active":"2024-05-17T22:34:40.433489+0000","last_became_peered":"2024-05-17T22:34:40.433489+0000","last_unstale":"2024-05-17T22:37:14.514102+0000","last_undegraded":"2024-05-17T22:37:14.514102+0000","last_fullsized":"2024-05-17T22:37:14.514102+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:13:56.014094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.512513+0000","last_change":"2024-05-17T22:34:40.434953+0000","last_active":"2024-05-17T22:37:14.512513+0000","last_peered":"2024-05-17T22:37:14.512513+0000","last_clean":"2024-05-17T22:37:14.512513+0000","last_became_active":"2024-05-17T22:34:40.434593+0000","last_became_peered":"2024-05-17T22:34:40.434593+0000","last_unstale":"2024-05-17T22:37:14.512513+0000","last_undegraded":"2024-05-17T22:37:14.512513+0000","last_fullsized":"2024-05-17T22:37:14.512513+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:20:27.075382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.513470+0000","last_change":"2024-05-17T22:34:40.432787+0000","last_active":"2024-05-17T22:37:14.513470+0000","last_peered":"2024-05-17T22:37:14.513470+0000","last_clean":"2024-05-17T22:37:14.513470+0000","last_became_active":"2024-05-17T22:34:40.432402+0000","last_became_peered":"2024-05-17T22:34:40.432402+0000","last_unstale":"2024-05-17T22:37:14.513470+0000","last_undegraded":"2024-05-17T22:37:14.513470+0000","last_fullsized":"2024-05-17T22:37:14.513470+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:48:13.034276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.515580+0000","last_change":"2024-05-17T22:34:40.426423+0000","last_active":"2024-05-17T22:37:14.515580+0000","last_peered":"2024-05-17T22:37:14.515580+0000","last_clean":"2024-05-17T22:37:14.515580+0000","last_became_active":"2024-05-17T22:34:40.426049+0000","last_became_peered":"2024-05-17T22:34:40.426049+0000","last_unstale":"2024-05-17T22:37:14.515580+0000","last_undegraded":"2024-05-17T22:37:14.515580+0000","last_fullsized":"2024-05-17T22:37:14.515580+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:56:55.631322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.520679+0000","last_change":"2024-05-17T22:34:40.435706+0000","last_active":"2024-05-17T22:37:14.520679+0000","last_peered":"2024-05-17T22:37:14.520679+0000","last_clean":"2024-05-17T22:37:14.520679+0000","last_became_active":"2024-05-17T22:34:40.435330+0000","last_became_peered":"2024-05-17T22:34:40.435330+0000","last_unstale":"2024-05-17T22:37:14.520679+0000","last_undegraded":"2024-05-17T22:37:14.520679+0000","last_fullsized":"2024-05-17T22:37:14.520679+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:25:36.529454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.517285+0000","last_change":"2024-05-17T22:34:40.428997+0000","last_active":"2024-05-17T22:37:14.517285+0000","last_peered":"2024-05-17T22:37:14.517285+0000","last_clean":"2024-05-17T22:37:14.517285+0000","last_became_active":"2024-05-17T22:34:40.428594+0000","last_became_peered":"2024-05-17T22:34:40.428594+0000","last_unstale":"2024-05-17T22:37:14.517285+0000","last_undegraded":"2024-05-17T22:37:14.517285+0000","last_fullsized":"2024-05-17T22:37:14.517285+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:59:01.777751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.509268+0000","last_change":"2024-05-17T22:34:40.434241+0000","last_active":"2024-05-17T22:37:14.509268+0000","last_peered":"2024-05-17T22:37:14.509268+0000","last_clean":"2024-05-17T22:37:14.509268+0000","last_became_active":"2024-05-17T22:34:40.431652+0000","last_became_peered":"2024-05-17T22:34:40.431652+0000","last_unstale":"2024-05-17T22:37:14.509268+0000","last_undegraded":"2024-05-17T22:37:14.509268+0000","last_fullsized":"2024-05-17T22:37:14.509268+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:33:27.035284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.505573+0000","last_change":"2024-05-17T22:34:40.431485+0000","last_active":"2024-05-17T22:37:14.505573+0000","last_peered":"2024-05-17T22:37:14.505573+0000","last_clean":"2024-05-17T22:37:14.505573+0000","last_became_active":"2024-05-17T22:34:40.431120+0000","last_became_peered":"2024-05-17T22:34:40.431120+0000","last_unstale":"2024-05-17T22:37:14.505573+0000","last_undegraded":"2024-05-17T22:37:14.505573+0000","last_fullsized":"2024-05-17T22:37:14.505573+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:46:55.213602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.501567+0000","last_change":"2024-05-17T22:34:40.423117+0000","last_active":"2024-05-17T22:37:14.501567+0000","last_peered":"2024-05-17T22:37:14.501567+0000","last_clean":"2024-05-17T22:37:14.501567+0000","last_became_active":"2024-05-17T22:34:40.422694+0000","last_became_peered":"2024-05-17T22:34:40.422694+0000","last_unstale":"2024-05-17T22:37:14.501567+0000","last_undegraded":"2024-05-17T22:37:14.501567+0000","last_fullsized":"2024-05-17T22:37:14.501567+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:19:55.589532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.518389+0000","last_change":"2024-05-17T22:35:15.711796+0000","last_active":"2024-05-17T22:37:14.518389+0000","last_peered":"2024-05-17T22:37:14.518389+0000","last_clean":"2024-05-17T22:37:14.518389+0000","last_became_active":"2024-05-17T22:35:15.711327+0000","last_became_peered":"2024-05-17T22:35:15.711327+0000","last_unstale":"2024-05-17T22:37:14.518389+0000","last_undegraded":"2024-05-17T22:37:14.518389+0000","last_fullsized":"2024-05-17T22:37:14.518389+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:48:48.913312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.512607+0000","last_change":"2024-05-17T22:34:40.430393+0000","last_active":"2024-05-17T22:37:14.512607+0000","last_peered":"2024-05-17T22:37:14.512607+0000","last_clean":"2024-05-17T22:37:14.512607+0000","last_became_active":"2024-05-17T22:34:40.430030+0000","last_became_peered":"2024-05-17T22:34:40.430030+0000","last_unstale":"2024-05-17T22:37:14.512607+0000","last_undegraded":"2024-05-17T22:37:14.512607+0000","last_fullsized":"2024-05-17T22:37:14.512607+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:39:37.204246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.526052+0000","last_change":"2024-05-17T22:34:40.432443+0000","last_active":"2024-05-17T22:37:14.526052+0000","last_peered":"2024-05-17T22:37:14.526052+0000","last_clean":"2024-05-17T22:37:14.526052+0000","last_became_active":"2024-05-17T22:34:40.431991+0000","last_became_peered":"2024-05-17T22:34:40.431991+0000","last_unstale":"2024-05-17T22:37:14.526052+0000","last_undegraded":"2024-05-17T22:37:14.526052+0000","last_fullsized":"2024-05-17T22:37:14.526052+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:37:45.593575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.523169+0000","last_change":"2024-05-17T22:34:40.434388+0000","last_active":"2024-05-17T22:37:14.523169+0000","last_peered":"2024-05-17T22:37:14.523169+0000","last_clean":"2024-05-17T22:37:14.523169+0000","last_became_active":"2024-05-17T22:34:40.434044+0000","last_became_peered":"2024-05-17T22:34:40.434044+0000","last_unstale":"2024-05-17T22:37:14.523169+0000","last_undegraded":"2024-05-17T22:37:14.523169+0000","last_fullsized":"2024-05-17T22:37:14.523169+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:20:43.296190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.507137+0000","last_change":"2024-05-17T22:34:40.427137+0000","last_active":"2024-05-17T22:37:14.507137+0000","last_peered":"2024-05-17T22:37:14.507137+0000","last_clean":"2024-05-17T22:37:14.507137+0000","last_became_active":"2024-05-17T22:34:40.426703+0000","last_became_peered":"2024-05-17T22:34:40.426703+0000","last_unstale":"2024-05-17T22:37:14.507137+0000","last_undegraded":"2024-05-17T22:37:14.507137+0000","last_fullsized":"2024-05-17T22:37:14.507137+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:55:52.992817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66995,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":717,"num_read_kb":432,"num_write":178,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":200985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710773,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":515036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907003,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28692,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716460,"statfs":{"total":95995035648,"available":95965655040,"internally_reserved":0,"allocated":1716224,"data_stored":1165135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070526,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":519346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266758,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":515148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462986,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1634304,"data_stored":1111752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626510,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":510426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1089536,"data_stored":572657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738478,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":511814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411428,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1114112,"data_stored":574539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182495,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":510620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986265,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":515608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822741,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36940,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708212,"statfs":{"total":95995035648,"available":95957209088,"internally_reserved":0,"allocated":1642496,"data_stored":1106808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T22:37:47.586 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-17T22:37:47.853 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:37:48.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:47 smithi012 ceph-mon[31632]: pgmap v395: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T22:37:48.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:47 smithi012 ceph-mon[31632]: from='client.15069 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T22:37:48.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:47 smithi151 ceph-mon[36101]: pgmap v395: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T22:37:48.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:47 smithi151 ceph-mon[36101]: from='client.15069 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T22:37:48.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:47 smithi096 ceph-mon[36065]: pgmap v395: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T22:37:48.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:47 smithi096 ceph-mon[36065]: from='client.15069 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T22:37:49.017 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:48 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:37:49.018 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:48 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:37:49.018 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:48 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:37:49.018 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:48 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:37:49.018 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:48 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:37:49.018 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:48 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:37:49.018 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:48 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:37:49.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:48 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:37:49.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:48 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:37:49.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:48 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:37:49.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:48 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:37:49.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:48 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:37:49.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:48 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:37:49.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:48 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:37:49.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:48 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:37:49.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:48 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:37:49.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:48 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:37:49.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:48 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:37:49.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:48 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:37:49.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:48 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:37:49.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:48 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:37:49.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.584+0000 7fe1d25c2700 1 -- 172.21.15.12:0/1643689869 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1cc108840 msgr2=0x7fe1cc108c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:49.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.584+0000 7fe1d25c2700 1 --2- 172.21.15.12:0/1643689869 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1cc108840 0x7fe1cc108c50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4009990 tx=0x7fe1b4009ca0 comp rx=0 tx=0).stop 2024-05-17T22:37:49.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.585+0000 7fe1d25c2700 1 -- 172.21.15.12:0/1643689869 shutdown_connections 2024-05-17T22:37:49.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.585+0000 7fe1d25c2700 1 --2- 172.21.15.12:0/1643689869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1cc109190 0x7fe1cc110af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:49.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.585+0000 7fe1d25c2700 1 --2- 172.21.15.12:0/1643689869 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe1cc103f10 0x7fe1cc104320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:49.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.585+0000 7fe1d25c2700 1 --2- 172.21.15.12:0/1643689869 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1cc108840 0x7fe1cc108c50 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:49.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.585+0000 7fe1d25c2700 1 -- 172.21.15.12:0/1643689869 >> 172.21.15.12:0/1643689869 conn(0x7fe1cc0ff9b0 msgr2=0x7fe1cc101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:49.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.585+0000 7fe1d25c2700 1 -- 172.21.15.12:0/1643689869 shutdown_connections 2024-05-17T22:37:49.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.585+0000 7fe1d25c2700 1 -- 172.21.15.12:0/1643689869 wait complete. 2024-05-17T22:37:49.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.585+0000 7fe1d25c2700 1 Processor -- start 2024-05-17T22:37:49.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.585+0000 7fe1d25c2700 1 -- start start 2024-05-17T22:37:49.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.586+0000 7fe1d25c2700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe1cc103f10 0x7fe1cc19a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:49.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.586+0000 7fe1d25c2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1cc108840 0x7fe1cc19ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:49.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.586+0000 7fe1d25c2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1cc109190 0x7fe1cc19eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:49.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.586+0000 7fe1d25c2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc113da0 con 0x7fe1cc108840 2024-05-17T22:37:49.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.586+0000 7fe1d25c2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc113c20 con 0x7fe1cc109190 2024-05-17T22:37:49.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.586+0000 7fe1d25c2700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc113f20 con 0x7fe1cc103f10 2024-05-17T22:37:49.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.586+0000 7fe1c3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1cc108840 0x7fe1cc19ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:49.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.586+0000 7fe1c3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1cc108840 0x7fe1cc19ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55342/0 (socket says 172.21.15.12:55342) 2024-05-17T22:37:49.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.586+0000 7fe1c3fff700 1 -- 172.21.15.12:0/4166396740 learned_addr learned my addr 172.21.15.12:0/4166396740 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:37:49.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.586+0000 7fe1cbfff700 1 --2- 172.21.15.12:0/4166396740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe1cc103f10 0x7fe1cc19a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:49.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.586+0000 7fe1d0b5f700 1 --2- 172.21.15.12:0/4166396740 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1cc109190 0x7fe1cc19eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:49.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.586+0000 7fe1c3fff700 1 -- 172.21.15.12:0/4166396740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe1cc103f10 msgr2=0x7fe1cc19a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:49.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.586+0000 7fe1c3fff700 1 --2- 172.21.15.12:0/4166396740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe1cc103f10 0x7fe1cc19a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:49.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.586+0000 7fe1c3fff700 1 -- 172.21.15.12:0/4166396740 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1cc109190 msgr2=0x7fe1cc19eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:49.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.586+0000 7fe1c3fff700 1 --2- 172.21.15.12:0/4166396740 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1cc109190 0x7fe1cc19eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:49.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.586+0000 7fe1c3fff700 1 -- 172.21.15.12:0/4166396740 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1cc19f690 con 0x7fe1cc108840 2024-05-17T22:37:49.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.586+0000 7fe1d0b5f700 1 --2- 172.21.15.12:0/4166396740 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1cc109190 0x7fe1cc19eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:37:49.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.586+0000 7fe1c3fff700 1 --2- 172.21.15.12:0/4166396740 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1cc108840 0x7fe1cc19ad60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4000c00 tx=0x7fe1b400bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:49.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.587+0000 7fe1c9ffb700 1 -- 172.21.15.12:0/4166396740 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4027070 con 0x7fe1cc108840 2024-05-17T22:37:49.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.587+0000 7fe1c9ffb700 1 -- 172.21.15.12:0/4166396740 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1b4022470 con 0x7fe1cc108840 2024-05-17T22:37:49.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.587+0000 7fe1d25c2700 1 -- 172.21.15.12:0/4166396740 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1cc19f920 con 0x7fe1cc108840 2024-05-17T22:37:49.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.587+0000 7fe1c9ffb700 1 -- 172.21.15.12:0/4166396740 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4017650 con 0x7fe1cc108840 2024-05-17T22:37:49.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.587+0000 7fe1d25c2700 1 -- 172.21.15.12:0/4166396740 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1cc075a60 con 0x7fe1cc108840 2024-05-17T22:37:49.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.588+0000 7fe1d25c2700 1 -- 172.21.15.12:0/4166396740 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1cc1053c0 con 0x7fe1cc108840 2024-05-17T22:37:49.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.589+0000 7fe1c9ffb700 1 -- 172.21.15.12:0/4166396740 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe1b402a070 con 0x7fe1cc108840 2024-05-17T22:37:49.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.590+0000 7fe1c9ffb700 1 --2- 172.21.15.12:0/4166396740 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe1b80b2900 0x7fe1b80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:49.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.590+0000 7fe1cbfff700 1 --2- 172.21.15.12:0/4166396740 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe1b80b2900 0x7fe1b80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:49.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.590+0000 7fe1c9ffb700 1 -- 172.21.15.12:0/4166396740 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe1b40e32a0 con 0x7fe1cc108840 2024-05-17T22:37:49.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.591+0000 7fe1cbfff700 1 --2- 172.21.15.12:0/4166396740 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe1b80b2900 0x7fe1b80b4e10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc009c60 tx=0x7fe1bc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:49.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.595+0000 7fe1c9ffb700 1 -- 172.21.15.12:0/4166396740 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe1b4013070 con 0x7fe1cc108840 2024-05-17T22:37:49.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.805+0000 7fe1d25c2700 1 -- 172.21.15.12:0/4166396740 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe1cc02ea80 con 0x7fe1b80b2900 2024-05-17T22:37:49.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.812+0000 7fe1c9ffb700 1 -- 172.21.15.12:0/4166396740 <== mgr.14152 v2:172.21.15.12:6800/1388514047 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340465 (secure 0 0 0) 0x7fe1cc02ea80 con 0x7fe1b80b2900 2024-05-17T22:37:49.822 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:37:49.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.815+0000 7fe1d25c2700 1 -- 172.21.15.12:0/4166396740 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe1b80b2900 msgr2=0x7fe1b80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:49.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.815+0000 7fe1d25c2700 1 --2- 172.21.15.12:0/4166396740 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe1b80b2900 0x7fe1b80b4e10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc009c60 tx=0x7fe1bc009260 comp rx=0 tx=0).stop 2024-05-17T22:37:49.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.815+0000 7fe1d25c2700 1 -- 172.21.15.12:0/4166396740 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1cc108840 msgr2=0x7fe1cc19ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:49.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.815+0000 7fe1d25c2700 1 --2- 172.21.15.12:0/4166396740 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1cc108840 0x7fe1cc19ad60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4000c00 tx=0x7fe1b400bfd0 comp rx=0 tx=0).stop 2024-05-17T22:37:49.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.815+0000 7fe1d25c2700 1 -- 172.21.15.12:0/4166396740 shutdown_connections 2024-05-17T22:37:49.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.815+0000 7fe1d25c2700 1 --2- 172.21.15.12:0/4166396740 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe1b80b2900 0x7fe1b80b4e10 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:49.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.815+0000 7fe1d25c2700 1 --2- 172.21.15.12:0/4166396740 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1cc109190 0x7fe1cc19eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:49.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.815+0000 7fe1d25c2700 1 --2- 172.21.15.12:0/4166396740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe1cc103f10 0x7fe1cc19a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:49.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.815+0000 7fe1d25c2700 1 --2- 172.21.15.12:0/4166396740 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1cc108840 0x7fe1cc19ad60 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:49.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.815+0000 7fe1d25c2700 1 -- 172.21.15.12:0/4166396740 >> 172.21.15.12:0/4166396740 conn(0x7fe1cc0ff9b0 msgr2=0x7fe1cc101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:49.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.816+0000 7fe1d25c2700 1 -- 172.21.15.12:0/4166396740 shutdown_connections 2024-05-17T22:37:49.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:49.816+0000 7fe1d25c2700 1 -- 172.21.15.12:0/4166396740 wait complete. 2024-05-17T22:37:49.826 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-05-17T22:37:50.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:49 smithi012 ceph-mon[31632]: pgmap v396: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:37:50.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:49 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:37:50.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:49 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:37:50.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:49 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:37:50.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:49 smithi151 ceph-mon[36101]: pgmap v396: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:37:50.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:49 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:37:50.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:49 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:37:50.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:49 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:37:50.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:49 smithi096 ceph-mon[36065]: pgmap v396: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:37:50.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:49 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:37:50.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:49 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:37:50.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:49 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:37:50.459 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":396,"stamp":"2024-05-17T22:37:48.532392+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657597,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":803,"num_read_kb":496,"num_write":303,"num_write_kb":2277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":746,"ondisk_log_size":746,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":388208,"kb_used_data":14096,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553616,"statfs":{"total":1151940427776,"available":1151542902784,"internally_reserved":0,"allocated":14434304,"data_stored":8128889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003279"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.522395+0000","last_change":"2024-05-17T22:34:40.422850+0000","last_active":"2024-05-17T22:37:14.522395+0000","last_peered":"2024-05-17T22:37:14.522395+0000","last_clean":"2024-05-17T22:37:14.522395+0000","last_became_active":"2024-05-17T22:34:40.422474+0000","last_became_peered":"2024-05-17T22:34:40.422474+0000","last_unstale":"2024-05-17T22:37:14.522395+0000","last_undegraded":"2024-05-17T22:37:14.522395+0000","last_fullsized":"2024-05-17T22:37:14.522395+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:23:51.537128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.516491+0000","last_change":"2024-05-17T22:34:40.427907+0000","last_active":"2024-05-17T22:37:14.516491+0000","last_peered":"2024-05-17T22:37:14.516491+0000","last_clean":"2024-05-17T22:37:14.516491+0000","last_became_active":"2024-05-17T22:34:40.427511+0000","last_became_peered":"2024-05-17T22:34:40.427511+0000","last_unstale":"2024-05-17T22:37:14.516491+0000","last_undegraded":"2024-05-17T22:37:14.516491+0000","last_fullsized":"2024-05-17T22:37:14.516491+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:01:46.707820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.525653+0000","last_change":"2024-05-17T22:34:40.436370+0000","last_active":"2024-05-17T22:37:14.525653+0000","last_peered":"2024-05-17T22:37:14.525653+0000","last_clean":"2024-05-17T22:37:14.525653+0000","last_became_active":"2024-05-17T22:34:40.434089+0000","last_became_peered":"2024-05-17T22:34:40.434089+0000","last_unstale":"2024-05-17T22:37:14.525653+0000","last_undegraded":"2024-05-17T22:37:14.525653+0000","last_fullsized":"2024-05-17T22:37:14.525653+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:58:16.775471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.509106+0000","last_change":"2024-05-17T22:35:15.556681+0000","last_active":"2024-05-17T22:37:14.509106+0000","last_peered":"2024-05-17T22:37:14.509106+0000","last_clean":"2024-05-17T22:37:14.509106+0000","last_became_active":"2024-05-17T22:35:15.556427+0000","last_became_peered":"2024-05-17T22:35:15.556427+0000","last_unstale":"2024-05-17T22:37:14.509106+0000","last_undegraded":"2024-05-17T22:37:14.509106+0000","last_fullsized":"2024-05-17T22:37:14.509106+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:28:01.726262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.495402+0000","last_change":"2024-05-17T22:34:40.434394+0000","last_active":"2024-05-17T22:37:14.495402+0000","last_peered":"2024-05-17T22:37:14.495402+0000","last_clean":"2024-05-17T22:37:14.495402+0000","last_became_active":"2024-05-17T22:34:40.433489+0000","last_became_peered":"2024-05-17T22:34:40.433489+0000","last_unstale":"2024-05-17T22:37:14.495402+0000","last_undegraded":"2024-05-17T22:37:14.495402+0000","last_fullsized":"2024-05-17T22:37:14.495402+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:26:24.068814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.515238+0000","last_change":"2024-05-17T22:35:15.553601+0000","last_active":"2024-05-17T22:37:14.515238+0000","last_peered":"2024-05-17T22:37:14.515238+0000","last_clean":"2024-05-17T22:37:14.515238+0000","last_became_active":"2024-05-17T22:35:15.553116+0000","last_became_peered":"2024-05-17T22:35:15.553116+0000","last_unstale":"2024-05-17T22:37:14.515238+0000","last_undegraded":"2024-05-17T22:37:14.515238+0000","last_fullsized":"2024-05-17T22:37:14.515238+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:09:38.401107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.518979+0000","last_change":"2024-05-17T22:34:40.430479+0000","last_active":"2024-05-17T22:37:14.518979+0000","last_peered":"2024-05-17T22:37:14.518979+0000","last_clean":"2024-05-17T22:37:14.518979+0000","last_became_active":"2024-05-17T22:34:40.430025+0000","last_became_peered":"2024-05-17T22:34:40.430025+0000","last_unstale":"2024-05-17T22:37:14.518979+0000","last_undegraded":"2024-05-17T22:37:14.518979+0000","last_fullsized":"2024-05-17T22:37:14.518979+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:14:19.725096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.519153+0000","last_change":"2024-05-17T22:34:40.431711+0000","last_active":"2024-05-17T22:37:14.519153+0000","last_peered":"2024-05-17T22:37:14.519153+0000","last_clean":"2024-05-17T22:37:14.519153+0000","last_became_active":"2024-05-17T22:34:40.431368+0000","last_became_peered":"2024-05-17T22:34:40.431368+0000","last_unstale":"2024-05-17T22:37:14.519153+0000","last_undegraded":"2024-05-17T22:37:14.519153+0000","last_fullsized":"2024-05-17T22:37:14.519153+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:06:34.904652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.521909+0000","last_change":"2024-05-17T22:34:40.435593+0000","last_active":"2024-05-17T22:37:14.521909+0000","last_peered":"2024-05-17T22:37:14.521909+0000","last_clean":"2024-05-17T22:37:14.521909+0000","last_became_active":"2024-05-17T22:34:40.435241+0000","last_became_peered":"2024-05-17T22:34:40.435241+0000","last_unstale":"2024-05-17T22:37:14.521909+0000","last_undegraded":"2024-05-17T22:37:14.521909+0000","last_fullsized":"2024-05-17T22:37:14.521909+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:08:57.239998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.526427+0000","last_change":"2024-05-17T22:34:40.436248+0000","last_active":"2024-05-17T22:37:14.526427+0000","last_peered":"2024-05-17T22:37:14.526427+0000","last_clean":"2024-05-17T22:37:14.526427+0000","last_became_active":"2024-05-17T22:34:40.435824+0000","last_became_peered":"2024-05-17T22:34:40.435824+0000","last_unstale":"2024-05-17T22:37:14.526427+0000","last_undegraded":"2024-05-17T22:37:14.526427+0000","last_fullsized":"2024-05-17T22:37:14.526427+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:39:07.297470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.524640+0000","last_change":"2024-05-17T22:34:40.435843+0000","last_active":"2024-05-17T22:37:14.524640+0000","last_peered":"2024-05-17T22:37:14.524640+0000","last_clean":"2024-05-17T22:37:14.524640+0000","last_became_active":"2024-05-17T22:34:40.435500+0000","last_became_peered":"2024-05-17T22:34:40.435500+0000","last_unstale":"2024-05-17T22:37:14.524640+0000","last_undegraded":"2024-05-17T22:37:14.524640+0000","last_fullsized":"2024-05-17T22:37:14.524640+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:11:28.115850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.501798+0000","last_change":"2024-05-17T22:34:40.432044+0000","last_active":"2024-05-17T22:37:14.501798+0000","last_peered":"2024-05-17T22:37:14.501798+0000","last_clean":"2024-05-17T22:37:14.501798+0000","last_became_active":"2024-05-17T22:34:40.429141+0000","last_became_peered":"2024-05-17T22:34:40.429141+0000","last_unstale":"2024-05-17T22:37:14.501798+0000","last_undegraded":"2024-05-17T22:37:14.501798+0000","last_fullsized":"2024-05-17T22:37:14.501798+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:04:53.428681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.526534+0000","last_change":"2024-05-17T22:34:40.438474+0000","last_active":"2024-05-17T22:37:14.526534+0000","last_peered":"2024-05-17T22:37:14.526534+0000","last_clean":"2024-05-17T22:37:14.526534+0000","last_became_active":"2024-05-17T22:34:40.438109+0000","last_became_peered":"2024-05-17T22:34:40.438109+0000","last_unstale":"2024-05-17T22:37:14.526534+0000","last_undegraded":"2024-05-17T22:37:14.526534+0000","last_fullsized":"2024-05-17T22:37:14.526534+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:26:00.665246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.516486+0000","last_change":"2024-05-17T22:34:40.431563+0000","last_active":"2024-05-17T22:37:14.516486+0000","last_peered":"2024-05-17T22:37:14.516486+0000","last_clean":"2024-05-17T22:37:14.516486+0000","last_became_active":"2024-05-17T22:34:40.430992+0000","last_became_peered":"2024-05-17T22:34:40.430992+0000","last_unstale":"2024-05-17T22:37:14.516486+0000","last_undegraded":"2024-05-17T22:37:14.516486+0000","last_fullsized":"2024-05-17T22:37:14.516486+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:51:26.280440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.516587+0000","last_change":"2024-05-17T22:34:40.433629+0000","last_active":"2024-05-17T22:37:14.516587+0000","last_peered":"2024-05-17T22:37:14.516587+0000","last_clean":"2024-05-17T22:37:14.516587+0000","last_became_active":"2024-05-17T22:34:40.433270+0000","last_became_peered":"2024-05-17T22:34:40.433270+0000","last_unstale":"2024-05-17T22:37:14.516587+0000","last_undegraded":"2024-05-17T22:37:14.516587+0000","last_fullsized":"2024-05-17T22:37:14.516587+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:57:51.164078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.525064+0000","last_change":"2024-05-17T22:34:40.434218+0000","last_active":"2024-05-17T22:37:14.525064+0000","last_peered":"2024-05-17T22:37:14.525064+0000","last_clean":"2024-05-17T22:37:14.525064+0000","last_became_active":"2024-05-17T22:34:40.433849+0000","last_became_peered":"2024-05-17T22:34:40.433849+0000","last_unstale":"2024-05-17T22:37:14.525064+0000","last_undegraded":"2024-05-17T22:37:14.525064+0000","last_fullsized":"2024-05-17T22:37:14.525064+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:41:55.984109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.522605+0000","last_change":"2024-05-17T22:34:40.427892+0000","last_active":"2024-05-17T22:37:14.522605+0000","last_peered":"2024-05-17T22:37:14.522605+0000","last_clean":"2024-05-17T22:37:14.522605+0000","last_became_active":"2024-05-17T22:34:40.427303+0000","last_became_peered":"2024-05-17T22:34:40.427303+0000","last_unstale":"2024-05-17T22:37:14.522605+0000","last_undegraded":"2024-05-17T22:37:14.522605+0000","last_fullsized":"2024-05-17T22:37:14.522605+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:59:38.632123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.524943+0000","last_change":"2024-05-17T22:34:40.424784+0000","last_active":"2024-05-17T22:37:14.524943+0000","last_peered":"2024-05-17T22:37:14.524943+0000","last_clean":"2024-05-17T22:37:14.524943+0000","last_became_active":"2024-05-17T22:34:40.424439+0000","last_became_peered":"2024-05-17T22:34:40.424439+0000","last_unstale":"2024-05-17T22:37:14.524943+0000","last_undegraded":"2024-05-17T22:37:14.524943+0000","last_fullsized":"2024-05-17T22:37:14.524943+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:02:53.681600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.513305+0000","last_change":"2024-05-17T22:34:40.435023+0000","last_active":"2024-05-17T22:37:14.513305+0000","last_peered":"2024-05-17T22:37:14.513305+0000","last_clean":"2024-05-17T22:37:14.513305+0000","last_became_active":"2024-05-17T22:34:40.434441+0000","last_became_peered":"2024-05-17T22:34:40.434441+0000","last_unstale":"2024-05-17T22:37:14.513305+0000","last_undegraded":"2024-05-17T22:37:14.513305+0000","last_fullsized":"2024-05-17T22:37:14.513305+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:16:07.671897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624113+0000","last_change":"2024-05-17T22:34:52.493192+0000","last_active":"2024-05-17T22:36:15.624113+0000","last_peered":"2024-05-17T22:36:15.624113+0000","last_clean":"2024-05-17T22:36:15.624113+0000","last_became_active":"2024-05-17T22:34:52.492786+0000","last_became_peered":"2024-05-17T22:34:52.492786+0000","last_unstale":"2024-05-17T22:36:15.624113+0000","last_undegraded":"2024-05-17T22:36:15.624113+0000","last_fullsized":"2024-05-17T22:36:15.624113+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:43:40.152240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623812+0000","last_change":"2024-05-17T22:34:37.402360+0000","last_active":"2024-05-17T22:36:15.623812+0000","last_peered":"2024-05-17T22:36:15.623812+0000","last_clean":"2024-05-17T22:36:15.623812+0000","last_became_active":"2024-05-17T22:34:37.401882+0000","last_became_peered":"2024-05-17T22:34:37.401882+0000","last_unstale":"2024-05-17T22:36:15.623812+0000","last_undegraded":"2024-05-17T22:36:15.623812+0000","last_fullsized":"2024-05-17T22:36:15.623812+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:53:13.300626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.501991+0000","last_change":"2024-05-17T22:34:40.436259+0000","last_active":"2024-05-17T22:37:14.501991+0000","last_peered":"2024-05-17T22:37:14.501991+0000","last_clean":"2024-05-17T22:37:14.501991+0000","last_became_active":"2024-05-17T22:34:40.435899+0000","last_became_peered":"2024-05-17T22:34:40.435899+0000","last_unstale":"2024-05-17T22:37:14.501991+0000","last_undegraded":"2024-05-17T22:37:14.501991+0000","last_fullsized":"2024-05-17T22:37:14.501991+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:26:29.327513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003713+0000","last_change":"2024-05-17T22:34:52.495236+0000","last_active":"2024-05-17T22:36:16.003713+0000","last_peered":"2024-05-17T22:36:16.003713+0000","last_clean":"2024-05-17T22:36:16.003713+0000","last_became_active":"2024-05-17T22:34:52.494783+0000","last_became_peered":"2024-05-17T22:34:52.494783+0000","last_unstale":"2024-05-17T22:36:16.003713+0000","last_undegraded":"2024-05-17T22:36:16.003713+0000","last_fullsized":"2024-05-17T22:36:16.003713+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:08:15.010706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984822+0000","last_change":"2024-05-17T22:34:37.441129+0000","last_active":"2024-05-17T22:36:15.984822+0000","last_peered":"2024-05-17T22:36:15.984822+0000","last_clean":"2024-05-17T22:36:15.984822+0000","last_became_active":"2024-05-17T22:34:37.440738+0000","last_became_peered":"2024-05-17T22:34:37.440738+0000","last_unstale":"2024-05-17T22:36:15.984822+0000","last_undegraded":"2024-05-17T22:36:15.984822+0000","last_fullsized":"2024-05-17T22:36:15.984822+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:03:16.945076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.525419+0000","last_change":"2024-05-17T22:34:40.434146+0000","last_active":"2024-05-17T22:37:14.525419+0000","last_peered":"2024-05-17T22:37:14.525419+0000","last_clean":"2024-05-17T22:37:14.525419+0000","last_became_active":"2024-05-17T22:34:40.433802+0000","last_became_peered":"2024-05-17T22:34:40.433802+0000","last_unstale":"2024-05-17T22:37:14.525419+0000","last_undegraded":"2024-05-17T22:37:14.525419+0000","last_fullsized":"2024-05-17T22:37:14.525419+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:08:39.905409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004211+0000","last_change":"2024-05-17T22:34:52.501134+0000","last_active":"2024-05-17T22:36:16.004211+0000","last_peered":"2024-05-17T22:36:16.004211+0000","last_clean":"2024-05-17T22:36:16.004211+0000","last_became_active":"2024-05-17T22:34:52.500547+0000","last_became_peered":"2024-05-17T22:34:52.500547+0000","last_unstale":"2024-05-17T22:36:16.004211+0000","last_undegraded":"2024-05-17T22:36:16.004211+0000","last_fullsized":"2024-05-17T22:36:16.004211+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:13:04.200865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.626422+0000","last_change":"2024-05-17T22:34:37.404019+0000","last_active":"2024-05-17T22:36:15.626422+0000","last_peered":"2024-05-17T22:36:15.626422+0000","last_clean":"2024-05-17T22:36:15.626422+0000","last_became_active":"2024-05-17T22:34:37.403607+0000","last_became_peered":"2024-05-17T22:34:37.403607+0000","last_unstale":"2024-05-17T22:36:15.626422+0000","last_undegraded":"2024-05-17T22:36:15.626422+0000","last_fullsized":"2024-05-17T22:36:15.626422+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:30:21.405195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.525402+0000","last_change":"2024-05-17T22:34:40.426889+0000","last_active":"2024-05-17T22:37:14.525402+0000","last_peered":"2024-05-17T22:37:14.525402+0000","last_clean":"2024-05-17T22:37:14.525402+0000","last_became_active":"2024-05-17T22:34:40.426512+0000","last_became_peered":"2024-05-17T22:34:40.426512+0000","last_unstale":"2024-05-17T22:37:14.525402+0000","last_undegraded":"2024-05-17T22:37:14.525402+0000","last_fullsized":"2024-05-17T22:37:14.525402+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:30:51.363177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004528+0000","last_change":"2024-05-17T22:34:52.487514+0000","last_active":"2024-05-17T22:36:16.004528+0000","last_peered":"2024-05-17T22:36:16.004528+0000","last_clean":"2024-05-17T22:36:16.004528+0000","last_became_active":"2024-05-17T22:34:52.487068+0000","last_became_peered":"2024-05-17T22:34:52.487068+0000","last_unstale":"2024-05-17T22:36:16.004528+0000","last_undegraded":"2024-05-17T22:36:16.004528+0000","last_fullsized":"2024-05-17T22:36:16.004528+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:52:13.573986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.626123+0000","last_change":"2024-05-17T22:34:37.400518+0000","last_active":"2024-05-17T22:36:15.626123+0000","last_peered":"2024-05-17T22:36:15.626123+0000","last_clean":"2024-05-17T22:36:15.626123+0000","last_became_active":"2024-05-17T22:34:37.400145+0000","last_became_peered":"2024-05-17T22:34:37.400145+0000","last_unstale":"2024-05-17T22:36:15.626123+0000","last_undegraded":"2024-05-17T22:36:15.626123+0000","last_fullsized":"2024-05-17T22:36:15.626123+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:56:53.593624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.525513+0000","last_change":"2024-05-17T22:34:40.434148+0000","last_active":"2024-05-17T22:37:14.525513+0000","last_peered":"2024-05-17T22:37:14.525513+0000","last_clean":"2024-05-17T22:37:14.525513+0000","last_became_active":"2024-05-17T22:34:40.433613+0000","last_became_peered":"2024-05-17T22:34:40.433613+0000","last_unstale":"2024-05-17T22:37:14.525513+0000","last_undegraded":"2024-05-17T22:37:14.525513+0000","last_fullsized":"2024-05-17T22:37:14.525513+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:09:39.049852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624340+0000","last_change":"2024-05-17T22:34:52.490417+0000","last_active":"2024-05-17T22:36:15.624340+0000","last_peered":"2024-05-17T22:36:15.624340+0000","last_clean":"2024-05-17T22:36:15.624340+0000","last_became_active":"2024-05-17T22:34:52.489953+0000","last_became_peered":"2024-05-17T22:34:52.489953+0000","last_unstale":"2024-05-17T22:36:15.624340+0000","last_undegraded":"2024-05-17T22:36:15.624340+0000","last_fullsized":"2024-05-17T22:36:15.624340+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:23:56.491899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'7","reported_seq":174,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:46.224773+0000","last_change":"2024-05-17T22:34:37.405304+0000","last_active":"2024-05-17T22:37:46.224773+0000","last_peered":"2024-05-17T22:37:46.224773+0000","last_clean":"2024-05-17T22:37:46.224773+0000","last_became_active":"2024-05-17T22:34:37.404552+0000","last_became_peered":"2024-05-17T22:34:37.404552+0000","last_unstale":"2024-05-17T22:37:46.224773+0000","last_undegraded":"2024-05-17T22:37:46.224773+0000","last_fullsized":"2024-05-17T22:37:46.224773+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:57:06.556886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.512663+0000","last_change":"2024-05-17T22:34:40.430537+0000","last_active":"2024-05-17T22:37:14.512663+0000","last_peered":"2024-05-17T22:37:14.512663+0000","last_clean":"2024-05-17T22:37:14.512663+0000","last_became_active":"2024-05-17T22:34:40.430193+0000","last_became_peered":"2024-05-17T22:34:40.430193+0000","last_unstale":"2024-05-17T22:37:14.512663+0000","last_undegraded":"2024-05-17T22:37:14.512663+0000","last_fullsized":"2024-05-17T22:37:14.512663+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:58:42.091445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624546+0000","last_change":"2024-05-17T22:35:15.559077+0000","last_active":"2024-05-17T22:36:15.624546+0000","last_peered":"2024-05-17T22:36:15.624546+0000","last_clean":"2024-05-17T22:36:15.624546+0000","last_became_active":"2024-05-17T22:35:15.558382+0000","last_became_peered":"2024-05-17T22:35:15.558382+0000","last_unstale":"2024-05-17T22:36:15.624546+0000","last_undegraded":"2024-05-17T22:36:15.624546+0000","last_fullsized":"2024-05-17T22:36:15.624546+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:20:16.009620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.626030+0000","last_change":"2024-05-17T22:34:37.405332+0000","last_active":"2024-05-17T22:36:15.626030+0000","last_peered":"2024-05-17T22:36:15.626030+0000","last_clean":"2024-05-17T22:36:15.626030+0000","last_became_active":"2024-05-17T22:34:37.404970+0000","last_became_peered":"2024-05-17T22:34:37.404970+0000","last_unstale":"2024-05-17T22:36:15.626030+0000","last_undegraded":"2024-05-17T22:36:15.626030+0000","last_fullsized":"2024-05-17T22:36:15.626030+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:37:39.933686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.517661+0000","last_change":"2024-05-17T22:34:40.432690+0000","last_active":"2024-05-17T22:37:14.517661+0000","last_peered":"2024-05-17T22:37:14.517661+0000","last_clean":"2024-05-17T22:37:14.517661+0000","last_became_active":"2024-05-17T22:34:40.432274+0000","last_became_peered":"2024-05-17T22:34:40.432274+0000","last_unstale":"2024-05-17T22:37:14.517661+0000","last_undegraded":"2024-05-17T22:37:14.517661+0000","last_fullsized":"2024-05-17T22:37:14.517661+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:12:47.926865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628628+0000","last_change":"2024-05-17T22:34:52.494645+0000","last_active":"2024-05-17T22:36:15.628628+0000","last_peered":"2024-05-17T22:36:15.628628+0000","last_clean":"2024-05-17T22:36:15.628628+0000","last_became_active":"2024-05-17T22:34:52.494255+0000","last_became_peered":"2024-05-17T22:34:52.494255+0000","last_unstale":"2024-05-17T22:36:15.628628+0000","last_undegraded":"2024-05-17T22:36:15.628628+0000","last_fullsized":"2024-05-17T22:36:15.628628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:41:26.633571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685426+0000","last_change":"2024-05-17T22:34:37.398582+0000","last_active":"2024-05-17T22:36:15.685426+0000","last_peered":"2024-05-17T22:36:15.685426+0000","last_clean":"2024-05-17T22:36:15.685426+0000","last_became_active":"2024-05-17T22:34:37.398156+0000","last_became_peered":"2024-05-17T22:34:37.398156+0000","last_unstale":"2024-05-17T22:36:15.685426+0000","last_undegraded":"2024-05-17T22:36:15.685426+0000","last_fullsized":"2024-05-17T22:36:15.685426+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:46:00.276214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.518565+0000","last_change":"2024-05-17T22:35:15.551402+0000","last_active":"2024-05-17T22:37:14.518565+0000","last_peered":"2024-05-17T22:37:14.518565+0000","last_clean":"2024-05-17T22:37:14.518565+0000","last_became_active":"2024-05-17T22:35:15.551021+0000","last_became_peered":"2024-05-17T22:35:15.551021+0000","last_unstale":"2024-05-17T22:37:14.518565+0000","last_undegraded":"2024-05-17T22:37:14.518565+0000","last_fullsized":"2024-05-17T22:37:14.518565+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:35:18.573620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968809+0000","last_change":"2024-05-17T22:34:52.487413+0000","last_active":"2024-05-17T22:36:15.968809+0000","last_peered":"2024-05-17T22:36:15.968809+0000","last_clean":"2024-05-17T22:36:15.968809+0000","last_became_active":"2024-05-17T22:34:52.487032+0000","last_became_peered":"2024-05-17T22:34:52.487032+0000","last_unstale":"2024-05-17T22:36:15.968809+0000","last_undegraded":"2024-05-17T22:36:15.968809+0000","last_fullsized":"2024-05-17T22:36:15.968809+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:34:49.122008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003315+0000","last_change":"2024-05-17T22:34:37.404894+0000","last_active":"2024-05-17T22:36:16.003315+0000","last_peered":"2024-05-17T22:36:16.003315+0000","last_clean":"2024-05-17T22:36:16.003315+0000","last_became_active":"2024-05-17T22:34:37.404567+0000","last_became_peered":"2024-05-17T22:34:37.404567+0000","last_unstale":"2024-05-17T22:36:16.003315+0000","last_undegraded":"2024-05-17T22:36:16.003315+0000","last_fullsized":"2024-05-17T22:36:16.003315+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:13:09.761556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.525152+0000","last_change":"2024-05-17T22:34:40.418267+0000","last_active":"2024-05-17T22:37:14.525152+0000","last_peered":"2024-05-17T22:37:14.525152+0000","last_clean":"2024-05-17T22:37:14.525152+0000","last_became_active":"2024-05-17T22:34:40.417928+0000","last_became_peered":"2024-05-17T22:34:40.417928+0000","last_unstale":"2024-05-17T22:37:14.525152+0000","last_undegraded":"2024-05-17T22:37:14.525152+0000","last_fullsized":"2024-05-17T22:37:14.525152+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:17:37.055004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984205+0000","last_change":"2024-05-17T22:34:52.492126+0000","last_active":"2024-05-17T22:36:15.984205+0000","last_peered":"2024-05-17T22:36:15.984205+0000","last_clean":"2024-05-17T22:36:15.984205+0000","last_became_active":"2024-05-17T22:34:52.491695+0000","last_became_peered":"2024-05-17T22:34:52.491695+0000","last_unstale":"2024-05-17T22:36:15.984205+0000","last_undegraded":"2024-05-17T22:36:15.984205+0000","last_fullsized":"2024-05-17T22:36:15.984205+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:15:17.710279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":164,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:43.219274+0000","last_change":"2024-05-17T22:34:37.406989+0000","last_active":"2024-05-17T22:37:43.219274+0000","last_peered":"2024-05-17T22:37:43.219274+0000","last_clean":"2024-05-17T22:37:43.219274+0000","last_became_active":"2024-05-17T22:34:37.406612+0000","last_became_peered":"2024-05-17T22:34:37.406612+0000","last_unstale":"2024-05-17T22:37:43.219274+0000","last_undegraded":"2024-05-17T22:37:43.219274+0000","last_fullsized":"2024-05-17T22:37:43.219274+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:47:52.480804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.524255+0000","last_change":"2024-05-17T22:34:40.427221+0000","last_active":"2024-05-17T22:37:14.524255+0000","last_peered":"2024-05-17T22:37:14.524255+0000","last_clean":"2024-05-17T22:37:14.524255+0000","last_became_active":"2024-05-17T22:34:40.426703+0000","last_became_peered":"2024-05-17T22:34:40.426703+0000","last_unstale":"2024-05-17T22:37:14.524255+0000","last_undegraded":"2024-05-17T22:37:14.524255+0000","last_fullsized":"2024-05-17T22:37:14.524255+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:49:15.629048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.622725+0000","last_change":"2024-05-17T22:34:52.499086+0000","last_active":"2024-05-17T22:36:15.622725+0000","last_peered":"2024-05-17T22:36:15.622725+0000","last_clean":"2024-05-17T22:36:15.622725+0000","last_became_active":"2024-05-17T22:34:52.498666+0000","last_became_peered":"2024-05-17T22:34:52.498666+0000","last_unstale":"2024-05-17T22:36:15.622725+0000","last_undegraded":"2024-05-17T22:36:15.622725+0000","last_fullsized":"2024-05-17T22:36:15.622725+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:46:25.882964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.005213+0000","last_change":"2024-05-17T22:35:15.836294+0000","last_active":"2024-05-17T22:36:16.005213+0000","last_peered":"2024-05-17T22:36:16.005213+0000","last_clean":"2024-05-17T22:36:16.005213+0000","last_became_active":"2024-05-17T22:35:15.835759+0000","last_became_peered":"2024-05-17T22:35:15.835759+0000","last_unstale":"2024-05-17T22:36:16.005213+0000","last_undegraded":"2024-05-17T22:36:16.005213+0000","last_fullsized":"2024-05-17T22:36:16.005213+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:23:32.106692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.519861+0000","last_change":"2024-05-17T22:34:40.433331+0000","last_active":"2024-05-17T22:37:14.519861+0000","last_peered":"2024-05-17T22:37:14.519861+0000","last_clean":"2024-05-17T22:37:14.519861+0000","last_became_active":"2024-05-17T22:34:40.432989+0000","last_became_peered":"2024-05-17T22:34:40.432989+0000","last_unstale":"2024-05-17T22:37:14.519861+0000","last_undegraded":"2024-05-17T22:37:14.519861+0000","last_fullsized":"2024-05-17T22:37:14.519861+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:22:49.146148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004235+0000","last_change":"2024-05-17T22:34:52.494418+0000","last_active":"2024-05-17T22:36:16.004235+0000","last_peered":"2024-05-17T22:36:16.004235+0000","last_clean":"2024-05-17T22:36:16.004235+0000","last_became_active":"2024-05-17T22:34:52.493953+0000","last_became_peered":"2024-05-17T22:34:52.493953+0000","last_unstale":"2024-05-17T22:36:16.004235+0000","last_undegraded":"2024-05-17T22:36:16.004235+0000","last_fullsized":"2024-05-17T22:36:16.004235+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:50:40.211770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004010+0000","last_change":"2024-05-17T22:34:37.442986+0000","last_active":"2024-05-17T22:36:16.004010+0000","last_peered":"2024-05-17T22:36:16.004010+0000","last_clean":"2024-05-17T22:36:16.004010+0000","last_became_active":"2024-05-17T22:34:37.442399+0000","last_became_peered":"2024-05-17T22:34:37.442399+0000","last_unstale":"2024-05-17T22:36:16.004010+0000","last_undegraded":"2024-05-17T22:36:16.004010+0000","last_fullsized":"2024-05-17T22:36:16.004010+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:34:05.785811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.525214+0000","last_change":"2024-05-17T22:34:40.426793+0000","last_active":"2024-05-17T22:37:14.525214+0000","last_peered":"2024-05-17T22:37:14.525214+0000","last_clean":"2024-05-17T22:37:14.525214+0000","last_became_active":"2024-05-17T22:34:40.426420+0000","last_became_peered":"2024-05-17T22:34:40.426420+0000","last_unstale":"2024-05-17T22:37:14.525214+0000","last_undegraded":"2024-05-17T22:37:14.525214+0000","last_fullsized":"2024-05-17T22:37:14.525214+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:57:07.099254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.622953+0000","last_change":"2024-05-17T22:34:52.498762+0000","last_active":"2024-05-17T22:36:15.622953+0000","last_peered":"2024-05-17T22:36:15.622953+0000","last_clean":"2024-05-17T22:36:15.622953+0000","last_became_active":"2024-05-17T22:34:52.498306+0000","last_became_peered":"2024-05-17T22:34:52.498306+0000","last_unstale":"2024-05-17T22:36:15.622953+0000","last_undegraded":"2024-05-17T22:36:15.622953+0000","last_fullsized":"2024-05-17T22:36:15.622953+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:18:32.717563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'36","reported_seq":321,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:44.220856+0000","last_change":"2024-05-17T22:34:37.407885+0000","last_active":"2024-05-17T22:37:44.220856+0000","last_peered":"2024-05-17T22:37:44.220856+0000","last_clean":"2024-05-17T22:37:44.220856+0000","last_became_active":"2024-05-17T22:34:37.407463+0000","last_became_peered":"2024-05-17T22:34:37.407463+0000","last_unstale":"2024-05-17T22:37:44.220856+0000","last_undegraded":"2024-05-17T22:37:44.220856+0000","last_fullsized":"2024-05-17T22:37:44.220856+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:26:20.288557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62321,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":240,"num_read_kb":181,"num_write":37,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.501714+0000","last_change":"2024-05-17T22:34:40.435800+0000","last_active":"2024-05-17T22:37:14.501714+0000","last_peered":"2024-05-17T22:37:14.501714+0000","last_clean":"2024-05-17T22:37:14.501714+0000","last_became_active":"2024-05-17T22:34:40.435245+0000","last_became_peered":"2024-05-17T22:34:40.435245+0000","last_unstale":"2024-05-17T22:37:14.501714+0000","last_undegraded":"2024-05-17T22:37:14.501714+0000","last_fullsized":"2024-05-17T22:37:14.501714+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:09:58.699026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004448+0000","last_change":"2024-05-17T22:34:52.487336+0000","last_active":"2024-05-17T22:36:16.004448+0000","last_peered":"2024-05-17T22:36:16.004448+0000","last_clean":"2024-05-17T22:36:16.004448+0000","last_became_active":"2024-05-17T22:34:52.486883+0000","last_became_peered":"2024-05-17T22:34:52.486883+0000","last_unstale":"2024-05-17T22:36:16.004448+0000","last_undegraded":"2024-05-17T22:36:16.004448+0000","last_fullsized":"2024-05-17T22:36:16.004448+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:44:41.264072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624096+0000","last_change":"2024-05-17T22:34:37.405044+0000","last_active":"2024-05-17T22:36:15.624096+0000","last_peered":"2024-05-17T22:36:15.624096+0000","last_clean":"2024-05-17T22:36:15.624096+0000","last_became_active":"2024-05-17T22:34:37.404610+0000","last_became_peered":"2024-05-17T22:34:37.404610+0000","last_unstale":"2024-05-17T22:36:15.624096+0000","last_undegraded":"2024-05-17T22:36:15.624096+0000","last_fullsized":"2024-05-17T22:36:15.624096+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:20:37.939069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.521845+0000","last_change":"2024-05-17T22:34:40.430655+0000","last_active":"2024-05-17T22:37:14.521845+0000","last_peered":"2024-05-17T22:37:14.521845+0000","last_clean":"2024-05-17T22:37:14.521845+0000","last_became_active":"2024-05-17T22:34:40.430307+0000","last_became_peered":"2024-05-17T22:34:40.430307+0000","last_unstale":"2024-05-17T22:37:14.521845+0000","last_undegraded":"2024-05-17T22:37:14.521845+0000","last_fullsized":"2024-05-17T22:37:14.521845+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:57:19.036117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628175+0000","last_change":"2024-05-17T22:34:52.494709+0000","last_active":"2024-05-17T22:36:15.628175+0000","last_peered":"2024-05-17T22:36:15.628175+0000","last_clean":"2024-05-17T22:36:15.628175+0000","last_became_active":"2024-05-17T22:34:52.494195+0000","last_became_peered":"2024-05-17T22:34:52.494195+0000","last_unstale":"2024-05-17T22:36:15.628175+0000","last_undegraded":"2024-05-17T22:36:15.628175+0000","last_fullsized":"2024-05-17T22:36:15.628175+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:35:41.188459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003957+0000","last_change":"2024-05-17T22:34:37.402494+0000","last_active":"2024-05-17T22:36:16.003957+0000","last_peered":"2024-05-17T22:36:16.003957+0000","last_clean":"2024-05-17T22:36:16.003957+0000","last_became_active":"2024-05-17T22:34:37.402102+0000","last_became_peered":"2024-05-17T22:34:37.402102+0000","last_unstale":"2024-05-17T22:36:16.003957+0000","last_undegraded":"2024-05-17T22:36:16.003957+0000","last_fullsized":"2024-05-17T22:36:16.003957+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:22:30.915780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.522351+0000","last_change":"2024-05-17T22:34:40.440731+0000","last_active":"2024-05-17T22:37:14.522351+0000","last_peered":"2024-05-17T22:37:14.522351+0000","last_clean":"2024-05-17T22:37:14.522351+0000","last_became_active":"2024-05-17T22:34:40.440374+0000","last_became_peered":"2024-05-17T22:34:40.440374+0000","last_unstale":"2024-05-17T22:37:14.522351+0000","last_undegraded":"2024-05-17T22:37:14.522351+0000","last_fullsized":"2024-05-17T22:37:14.522351+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:26:40.029155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.005856+0000","last_change":"2024-05-17T22:36:16.005856+0000","last_active":"2024-05-17T22:36:16.005856+0000","last_peered":"2024-05-17T22:36:16.005856+0000","last_clean":"2024-05-17T22:36:16.005856+0000","last_became_active":"2024-05-17T22:36:16.005303+0000","last_became_peered":"2024-05-17T22:36:16.005303+0000","last_unstale":"2024-05-17T22:36:16.005856+0000","last_undegraded":"2024-05-17T22:36:16.005856+0000","last_fullsized":"2024-05-17T22:36:16.005856+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:41:15.708879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":171,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:46.246983+0000","last_change":"2024-05-17T22:34:37.403123+0000","last_active":"2024-05-17T22:37:46.246983+0000","last_peered":"2024-05-17T22:37:46.246983+0000","last_clean":"2024-05-17T22:37:46.246983+0000","last_became_active":"2024-05-17T22:34:37.402702+0000","last_became_peered":"2024-05-17T22:34:37.402702+0000","last_unstale":"2024-05-17T22:37:46.246983+0000","last_undegraded":"2024-05-17T22:37:46.246983+0000","last_fullsized":"2024-05-17T22:37:46.246983+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:50:02.031651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.512621+0000","last_change":"2024-05-17T22:34:40.431895+0000","last_active":"2024-05-17T22:37:14.512621+0000","last_peered":"2024-05-17T22:37:14.512621+0000","last_clean":"2024-05-17T22:37:14.512621+0000","last_became_active":"2024-05-17T22:34:40.431527+0000","last_became_peered":"2024-05-17T22:34:40.431527+0000","last_unstale":"2024-05-17T22:37:14.512621+0000","last_undegraded":"2024-05-17T22:37:14.512621+0000","last_fullsized":"2024-05-17T22:37:14.512621+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:30:09.404255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.686140+0000","last_change":"2024-05-17T22:34:52.495583+0000","last_active":"2024-05-17T22:36:15.686140+0000","last_peered":"2024-05-17T22:36:15.686140+0000","last_clean":"2024-05-17T22:36:15.686140+0000","last_became_active":"2024-05-17T22:34:52.495030+0000","last_became_peered":"2024-05-17T22:34:52.495030+0000","last_unstale":"2024-05-17T22:36:15.686140+0000","last_undegraded":"2024-05-17T22:36:15.686140+0000","last_fullsized":"2024-05-17T22:36:15.686140+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:04:54.046891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":61,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625592+0000","last_change":"2024-05-17T22:34:37.442154+0000","last_active":"2024-05-17T22:36:15.625592+0000","last_peered":"2024-05-17T22:36:15.625592+0000","last_clean":"2024-05-17T22:36:15.625592+0000","last_became_active":"2024-05-17T22:34:37.441691+0000","last_became_peered":"2024-05-17T22:34:37.441691+0000","last_unstale":"2024-05-17T22:36:15.625592+0000","last_undegraded":"2024-05-17T22:36:15.625592+0000","last_fullsized":"2024-05-17T22:36:15.625592+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:47:29.862031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.517818+0000","last_change":"2024-05-17T22:34:40.430894+0000","last_active":"2024-05-17T22:37:14.517818+0000","last_peered":"2024-05-17T22:37:14.517818+0000","last_clean":"2024-05-17T22:37:14.517818+0000","last_became_active":"2024-05-17T22:34:40.430328+0000","last_became_peered":"2024-05-17T22:34:40.430328+0000","last_unstale":"2024-05-17T22:37:14.517818+0000","last_undegraded":"2024-05-17T22:37:14.517818+0000","last_fullsized":"2024-05-17T22:37:14.517818+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:36:46.619111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003488+0000","last_change":"2024-05-17T22:34:52.487555+0000","last_active":"2024-05-17T22:36:16.003488+0000","last_peered":"2024-05-17T22:36:16.003488+0000","last_clean":"2024-05-17T22:36:16.003488+0000","last_became_active":"2024-05-17T22:34:52.487076+0000","last_became_peered":"2024-05-17T22:34:52.487076+0000","last_unstale":"2024-05-17T22:36:16.003488+0000","last_undegraded":"2024-05-17T22:36:16.003488+0000","last_fullsized":"2024-05-17T22:36:16.003488+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:05:46.720665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628417+0000","last_change":"2024-05-17T22:35:15.558860+0000","last_active":"2024-05-17T22:36:15.628417+0000","last_peered":"2024-05-17T22:36:15.628417+0000","last_clean":"2024-05-17T22:36:15.628417+0000","last_became_active":"2024-05-17T22:35:15.553683+0000","last_became_peered":"2024-05-17T22:35:15.553683+0000","last_unstale":"2024-05-17T22:36:15.628417+0000","last_undegraded":"2024-05-17T22:36:15.628417+0000","last_fullsized":"2024-05-17T22:36:15.628417+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:27:24.978282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.506967+0000","last_change":"2024-05-17T22:34:40.433499+0000","last_active":"2024-05-17T22:37:14.506967+0000","last_peered":"2024-05-17T22:37:14.506967+0000","last_clean":"2024-05-17T22:37:14.506967+0000","last_became_active":"2024-05-17T22:34:40.431370+0000","last_became_peered":"2024-05-17T22:34:40.431370+0000","last_unstale":"2024-05-17T22:37:14.506967+0000","last_undegraded":"2024-05-17T22:37:14.506967+0000","last_fullsized":"2024-05-17T22:37:14.506967+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:25:08.383021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625953+0000","last_change":"2024-05-17T22:34:52.489627+0000","last_active":"2024-05-17T22:36:15.625953+0000","last_peered":"2024-05-17T22:36:15.625953+0000","last_clean":"2024-05-17T22:36:15.625953+0000","last_became_active":"2024-05-17T22:34:52.489173+0000","last_became_peered":"2024-05-17T22:34:52.489173+0000","last_unstale":"2024-05-17T22:36:15.625953+0000","last_undegraded":"2024-05-17T22:36:15.625953+0000","last_fullsized":"2024-05-17T22:36:15.625953+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:52:26.908817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623360+0000","last_change":"2024-05-17T22:35:15.553325+0000","last_active":"2024-05-17T22:36:15.623360+0000","last_peered":"2024-05-17T22:36:15.623360+0000","last_clean":"2024-05-17T22:36:15.623360+0000","last_became_active":"2024-05-17T22:35:15.552867+0000","last_became_peered":"2024-05-17T22:35:15.552867+0000","last_unstale":"2024-05-17T22:36:15.623360+0000","last_undegraded":"2024-05-17T22:36:15.623360+0000","last_fullsized":"2024-05-17T22:36:15.623360+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:28:36.618331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.525936+0000","last_change":"2024-05-17T22:34:40.423221+0000","last_active":"2024-05-17T22:37:14.525936+0000","last_peered":"2024-05-17T22:37:14.525936+0000","last_clean":"2024-05-17T22:37:14.525936+0000","last_became_active":"2024-05-17T22:34:40.422838+0000","last_became_peered":"2024-05-17T22:34:40.422838+0000","last_unstale":"2024-05-17T22:37:14.525936+0000","last_undegraded":"2024-05-17T22:37:14.525936+0000","last_fullsized":"2024-05-17T22:37:14.525936+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:39:51.873212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685905+0000","last_change":"2024-05-17T22:34:52.484802+0000","last_active":"2024-05-17T22:36:15.685905+0000","last_peered":"2024-05-17T22:36:15.685905+0000","last_clean":"2024-05-17T22:36:15.685905+0000","last_became_active":"2024-05-17T22:34:52.483766+0000","last_became_peered":"2024-05-17T22:34:52.483766+0000","last_unstale":"2024-05-17T22:36:15.685905+0000","last_undegraded":"2024-05-17T22:36:15.685905+0000","last_fullsized":"2024-05-17T22:36:15.685905+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:32:01.427078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004472+0000","last_change":"2024-05-17T22:34:37.404797+0000","last_active":"2024-05-17T22:36:16.004472+0000","last_peered":"2024-05-17T22:36:16.004472+0000","last_clean":"2024-05-17T22:36:16.004472+0000","last_became_active":"2024-05-17T22:34:37.404393+0000","last_became_peered":"2024-05-17T22:34:37.404393+0000","last_unstale":"2024-05-17T22:36:16.004472+0000","last_undegraded":"2024-05-17T22:36:16.004472+0000","last_fullsized":"2024-05-17T22:36:16.004472+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:26:36.887919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.515773+0000","last_change":"2024-05-17T22:34:40.428210+0000","last_active":"2024-05-17T22:37:14.515773+0000","last_peered":"2024-05-17T22:37:14.515773+0000","last_clean":"2024-05-17T22:37:14.515773+0000","last_became_active":"2024-05-17T22:34:40.427845+0000","last_became_peered":"2024-05-17T22:34:40.427845+0000","last_unstale":"2024-05-17T22:37:14.515773+0000","last_undegraded":"2024-05-17T22:37:14.515773+0000","last_fullsized":"2024-05-17T22:37:14.515773+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:45:30.437139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.686040+0000","last_change":"2024-05-17T22:34:52.490502+0000","last_active":"2024-05-17T22:36:15.686040+0000","last_peered":"2024-05-17T22:36:15.686040+0000","last_clean":"2024-05-17T22:36:15.686040+0000","last_became_active":"2024-05-17T22:34:52.490008+0000","last_became_peered":"2024-05-17T22:34:52.490008+0000","last_unstale":"2024-05-17T22:36:15.686040+0000","last_undegraded":"2024-05-17T22:36:15.686040+0000","last_fullsized":"2024-05-17T22:36:15.686040+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:58:29.122668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.517916+0000","last_change":"2024-05-17T22:34:40.432248+0000","last_active":"2024-05-17T22:37:14.517916+0000","last_peered":"2024-05-17T22:37:14.517916+0000","last_clean":"2024-05-17T22:37:14.517916+0000","last_became_active":"2024-05-17T22:34:40.431863+0000","last_became_peered":"2024-05-17T22:34:40.431863+0000","last_unstale":"2024-05-17T22:37:14.517916+0000","last_undegraded":"2024-05-17T22:37:14.517916+0000","last_fullsized":"2024-05-17T22:37:14.517916+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:20:34.730542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":164,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:42.238039+0000","last_change":"2024-05-17T22:34:37.441186+0000","last_active":"2024-05-17T22:37:42.238039+0000","last_peered":"2024-05-17T22:37:42.238039+0000","last_clean":"2024-05-17T22:37:42.238039+0000","last_became_active":"2024-05-17T22:34:37.440781+0000","last_became_peered":"2024-05-17T22:34:37.440781+0000","last_unstale":"2024-05-17T22:37:42.238039+0000","last_undegraded":"2024-05-17T22:37:42.238039+0000","last_fullsized":"2024-05-17T22:37:42.238039+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:44:34.435147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.005134+0000","last_change":"2024-05-17T22:34:52.491771+0000","last_active":"2024-05-17T22:36:16.005134+0000","last_peered":"2024-05-17T22:36:16.005134+0000","last_clean":"2024-05-17T22:36:16.005134+0000","last_became_active":"2024-05-17T22:34:52.491172+0000","last_became_peered":"2024-05-17T22:34:52.491172+0000","last_unstale":"2024-05-17T22:36:16.005134+0000","last_undegraded":"2024-05-17T22:36:16.005134+0000","last_fullsized":"2024-05-17T22:36:16.005134+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:15:21.010592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.511507+0000","last_change":"2024-05-17T22:34:40.430860+0000","last_active":"2024-05-17T22:37:14.511507+0000","last_peered":"2024-05-17T22:37:14.511507+0000","last_clean":"2024-05-17T22:37:14.511507+0000","last_became_active":"2024-05-17T22:34:40.430542+0000","last_became_peered":"2024-05-17T22:34:40.430542+0000","last_unstale":"2024-05-17T22:37:14.511507+0000","last_undegraded":"2024-05-17T22:37:14.511507+0000","last_fullsized":"2024-05-17T22:37:14.511507+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:16:26.257380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004114+0000","last_change":"2024-05-17T22:34:37.403026+0000","last_active":"2024-05-17T22:36:16.004114+0000","last_peered":"2024-05-17T22:36:16.004114+0000","last_clean":"2024-05-17T22:36:16.004114+0000","last_became_active":"2024-05-17T22:34:37.402560+0000","last_became_peered":"2024-05-17T22:34:37.402560+0000","last_unstale":"2024-05-17T22:36:16.004114+0000","last_undegraded":"2024-05-17T22:36:16.004114+0000","last_fullsized":"2024-05-17T22:36:16.004114+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:23:03.784411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":119,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003503+0000","last_change":"2024-05-17T22:33:11.055878+0000","last_active":"2024-05-17T22:36:16.003503+0000","last_peered":"2024-05-17T22:36:16.003503+0000","last_clean":"2024-05-17T22:36:16.003503+0000","last_became_active":"2024-05-17T22:33:11.050409+0000","last_became_peered":"2024-05-17T22:33:11.050409+0000","last_unstale":"2024-05-17T22:36:16.003503+0000","last_undegraded":"2024-05-17T22:36:16.003503+0000","last_fullsized":"2024-05-17T22:36:16.003503+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:29:43.341170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:29:43.341170+0000","last_clean_scrub_stamp":"2024-05-17T22:29:43.341170+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:30:54.153865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003960+0000","last_change":"2024-05-17T22:34:52.497064+0000","last_active":"2024-05-17T22:36:16.003960+0000","last_peered":"2024-05-17T22:36:16.003960+0000","last_clean":"2024-05-17T22:36:16.003960+0000","last_became_active":"2024-05-17T22:34:52.496611+0000","last_became_peered":"2024-05-17T22:34:52.496611+0000","last_unstale":"2024-05-17T22:36:16.003960+0000","last_undegraded":"2024-05-17T22:36:16.003960+0000","last_fullsized":"2024-05-17T22:36:16.003960+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:20:29.597589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.519744+0000","last_change":"2024-05-17T22:35:15.555413+0000","last_active":"2024-05-17T22:37:14.519744+0000","last_peered":"2024-05-17T22:37:14.519744+0000","last_clean":"2024-05-17T22:37:14.519744+0000","last_became_active":"2024-05-17T22:35:15.555005+0000","last_became_peered":"2024-05-17T22:35:15.555005+0000","last_unstale":"2024-05-17T22:37:14.519744+0000","last_undegraded":"2024-05-17T22:37:14.519744+0000","last_fullsized":"2024-05-17T22:37:14.519744+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:51:49.860673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003705+0000","last_change":"2024-05-17T22:34:37.406546+0000","last_active":"2024-05-17T22:36:16.003705+0000","last_peered":"2024-05-17T22:36:16.003705+0000","last_clean":"2024-05-17T22:36:16.003705+0000","last_became_active":"2024-05-17T22:34:37.406118+0000","last_became_peered":"2024-05-17T22:34:37.406118+0000","last_unstale":"2024-05-17T22:36:16.003705+0000","last_undegraded":"2024-05-17T22:36:16.003705+0000","last_fullsized":"2024-05-17T22:36:16.003705+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:47:16.350091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.006276+0000","last_change":"2024-05-17T22:36:16.005647+0000","last_active":"2024-05-17T22:36:16.006276+0000","last_peered":"2024-05-17T22:36:16.006276+0000","last_clean":"2024-05-17T22:36:16.006276+0000","last_became_active":"2024-05-17T22:36:16.005128+0000","last_became_peered":"2024-05-17T22:36:16.005128+0000","last_unstale":"2024-05-17T22:36:16.006276+0000","last_undegraded":"2024-05-17T22:36:16.006276+0000","last_fullsized":"2024-05-17T22:36:16.006276+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:16:24.481236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,7,6],"acting":[11,10,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.626356+0000","last_change":"2024-05-17T22:34:37.442676+0000","last_active":"2024-05-17T22:36:15.626356+0000","last_peered":"2024-05-17T22:36:15.626356+0000","last_clean":"2024-05-17T22:36:15.626356+0000","last_became_active":"2024-05-17T22:34:37.442275+0000","last_became_peered":"2024-05-17T22:34:37.442275+0000","last_unstale":"2024-05-17T22:36:15.626356+0000","last_undegraded":"2024-05-17T22:36:15.626356+0000","last_fullsized":"2024-05-17T22:36:15.626356+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:54:18.542987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.513754+0000","last_change":"2024-05-17T22:34:40.424266+0000","last_active":"2024-05-17T22:37:14.513754+0000","last_peered":"2024-05-17T22:37:14.513754+0000","last_clean":"2024-05-17T22:37:14.513754+0000","last_became_active":"2024-05-17T22:34:40.423905+0000","last_became_peered":"2024-05-17T22:34:40.423905+0000","last_unstale":"2024-05-17T22:37:14.513754+0000","last_undegraded":"2024-05-17T22:37:14.513754+0000","last_fullsized":"2024-05-17T22:37:14.513754+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:03:43.397871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984174+0000","last_change":"2024-05-17T22:34:52.493386+0000","last_active":"2024-05-17T22:36:15.984174+0000","last_peered":"2024-05-17T22:36:15.984174+0000","last_clean":"2024-05-17T22:36:15.984174+0000","last_became_active":"2024-05-17T22:34:52.492997+0000","last_became_peered":"2024-05-17T22:34:52.492997+0000","last_unstale":"2024-05-17T22:36:15.984174+0000","last_undegraded":"2024-05-17T22:36:15.984174+0000","last_fullsized":"2024-05-17T22:36:15.984174+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:14:40.818271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'12","reported_seq":57,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:48.068053+0000","last_change":"2024-05-17T22:34:37.399397+0000","last_active":"2024-05-17T22:36:48.068053+0000","last_peered":"2024-05-17T22:36:48.068053+0000","last_clean":"2024-05-17T22:36:48.068053+0000","last_became_active":"2024-05-17T22:34:37.399062+0000","last_became_peered":"2024-05-17T22:34:37.399062+0000","last_unstale":"2024-05-17T22:36:48.068053+0000","last_undegraded":"2024-05-17T22:36:48.068053+0000","last_fullsized":"2024-05-17T22:36:48.068053+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:02:05.230580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.518907+0000","last_change":"2024-05-17T22:34:40.426995+0000","last_active":"2024-05-17T22:37:14.518907+0000","last_peered":"2024-05-17T22:37:14.518907+0000","last_clean":"2024-05-17T22:37:14.518907+0000","last_became_active":"2024-05-17T22:34:40.426226+0000","last_became_peered":"2024-05-17T22:34:40.426226+0000","last_unstale":"2024-05-17T22:37:14.518907+0000","last_undegraded":"2024-05-17T22:37:14.518907+0000","last_fullsized":"2024-05-17T22:37:14.518907+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:49:53.331076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623704+0000","last_change":"2024-05-17T22:34:52.487098+0000","last_active":"2024-05-17T22:36:15.623704+0000","last_peered":"2024-05-17T22:36:15.623704+0000","last_clean":"2024-05-17T22:36:15.623704+0000","last_became_active":"2024-05-17T22:34:52.486605+0000","last_became_peered":"2024-05-17T22:34:52.486605+0000","last_unstale":"2024-05-17T22:36:15.623704+0000","last_undegraded":"2024-05-17T22:36:15.623704+0000","last_fullsized":"2024-05-17T22:36:15.623704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:33:28.234208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004689+0000","last_change":"2024-05-17T22:34:37.800830+0000","last_active":"2024-05-17T22:36:16.004689+0000","last_peered":"2024-05-17T22:36:16.004689+0000","last_clean":"2024-05-17T22:36:16.004689+0000","last_became_active":"2024-05-17T22:34:37.800405+0000","last_became_peered":"2024-05-17T22:34:37.800405+0000","last_unstale":"2024-05-17T22:36:16.004689+0000","last_undegraded":"2024-05-17T22:36:16.004689+0000","last_fullsized":"2024-05-17T22:36:16.004689+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:19:10.087442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.520738+0000","last_change":"2024-05-17T22:34:40.426307+0000","last_active":"2024-05-17T22:37:14.520738+0000","last_peered":"2024-05-17T22:37:14.520738+0000","last_clean":"2024-05-17T22:37:14.520738+0000","last_became_active":"2024-05-17T22:34:40.425969+0000","last_became_peered":"2024-05-17T22:34:40.425969+0000","last_unstale":"2024-05-17T22:37:14.520738+0000","last_undegraded":"2024-05-17T22:37:14.520738+0000","last_fullsized":"2024-05-17T22:37:14.520738+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:25:49.680886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004511+0000","last_change":"2024-05-17T22:35:15.556857+0000","last_active":"2024-05-17T22:36:16.004511+0000","last_peered":"2024-05-17T22:36:16.004511+0000","last_clean":"2024-05-17T22:36:16.004511+0000","last_became_active":"2024-05-17T22:35:15.556298+0000","last_became_peered":"2024-05-17T22:35:15.556298+0000","last_unstale":"2024-05-17T22:36:16.004511+0000","last_undegraded":"2024-05-17T22:36:16.004511+0000","last_fullsized":"2024-05-17T22:36:16.004511+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:23:20.111544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.805197+0000","last_change":"2024-05-17T22:34:37.442971+0000","last_active":"2024-05-17T22:36:15.805197+0000","last_peered":"2024-05-17T22:36:15.805197+0000","last_clean":"2024-05-17T22:36:15.805197+0000","last_became_active":"2024-05-17T22:34:37.442536+0000","last_became_peered":"2024-05-17T22:34:37.442536+0000","last_unstale":"2024-05-17T22:36:15.805197+0000","last_undegraded":"2024-05-17T22:36:15.805197+0000","last_fullsized":"2024-05-17T22:36:15.805197+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:03:45.224071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.467874+0000","last_change":"2024-05-17T22:34:40.432056+0000","last_active":"2024-05-17T22:37:14.467874+0000","last_peered":"2024-05-17T22:37:14.467874+0000","last_clean":"2024-05-17T22:37:14.467874+0000","last_became_active":"2024-05-17T22:34:40.431626+0000","last_became_peered":"2024-05-17T22:34:40.431626+0000","last_unstale":"2024-05-17T22:37:14.467874+0000","last_undegraded":"2024-05-17T22:37:14.467874+0000","last_fullsized":"2024-05-17T22:37:14.467874+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:12:24.650746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623664+0000","last_change":"2024-05-17T22:34:52.500451+0000","last_active":"2024-05-17T22:36:15.623664+0000","last_peered":"2024-05-17T22:36:15.623664+0000","last_clean":"2024-05-17T22:36:15.623664+0000","last_became_active":"2024-05-17T22:34:52.500031+0000","last_became_peered":"2024-05-17T22:34:52.500031+0000","last_unstale":"2024-05-17T22:36:15.623664+0000","last_undegraded":"2024-05-17T22:36:15.623664+0000","last_fullsized":"2024-05-17T22:36:15.623664+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:54:36.825116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984386+0000","last_change":"2024-05-17T22:34:37.402335+0000","last_active":"2024-05-17T22:36:15.984386+0000","last_peered":"2024-05-17T22:36:15.984386+0000","last_clean":"2024-05-17T22:36:15.984386+0000","last_became_active":"2024-05-17T22:34:37.401982+0000","last_became_peered":"2024-05-17T22:34:37.401982+0000","last_unstale":"2024-05-17T22:36:15.984386+0000","last_undegraded":"2024-05-17T22:36:15.984386+0000","last_fullsized":"2024-05-17T22:36:15.984386+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:58:06.194335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.517775+0000","last_change":"2024-05-17T22:34:40.425524+0000","last_active":"2024-05-17T22:37:14.517775+0000","last_peered":"2024-05-17T22:37:14.517775+0000","last_clean":"2024-05-17T22:37:14.517775+0000","last_became_active":"2024-05-17T22:34:40.425135+0000","last_became_peered":"2024-05-17T22:34:40.425135+0000","last_unstale":"2024-05-17T22:37:14.517775+0000","last_undegraded":"2024-05-17T22:37:14.517775+0000","last_fullsized":"2024-05-17T22:37:14.517775+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:41:14.176477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004843+0000","last_change":"2024-05-17T22:34:52.496290+0000","last_active":"2024-05-17T22:36:16.004843+0000","last_peered":"2024-05-17T22:36:16.004843+0000","last_clean":"2024-05-17T22:36:16.004843+0000","last_became_active":"2024-05-17T22:34:52.495817+0000","last_became_peered":"2024-05-17T22:34:52.495817+0000","last_unstale":"2024-05-17T22:36:16.004843+0000","last_undegraded":"2024-05-17T22:36:16.004843+0000","last_fullsized":"2024-05-17T22:36:16.004843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:47:22.056372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624160+0000","last_change":"2024-05-17T22:34:37.404234+0000","last_active":"2024-05-17T22:36:15.624160+0000","last_peered":"2024-05-17T22:36:15.624160+0000","last_clean":"2024-05-17T22:36:15.624160+0000","last_became_active":"2024-05-17T22:34:37.403561+0000","last_became_peered":"2024-05-17T22:34:37.403561+0000","last_unstale":"2024-05-17T22:36:15.624160+0000","last_undegraded":"2024-05-17T22:36:15.624160+0000","last_fullsized":"2024-05-17T22:36:15.624160+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:30:28.432641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.517802+0000","last_change":"2024-05-17T22:34:40.426522+0000","last_active":"2024-05-17T22:37:14.517802+0000","last_peered":"2024-05-17T22:37:14.517802+0000","last_clean":"2024-05-17T22:37:14.517802+0000","last_became_active":"2024-05-17T22:34:40.426139+0000","last_became_peered":"2024-05-17T22:34:40.426139+0000","last_unstale":"2024-05-17T22:37:14.517802+0000","last_undegraded":"2024-05-17T22:37:14.517802+0000","last_fullsized":"2024-05-17T22:37:14.517802+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:30:19.503513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004076+0000","last_change":"2024-05-17T22:34:52.487131+0000","last_active":"2024-05-17T22:36:16.004076+0000","last_peered":"2024-05-17T22:36:16.004076+0000","last_clean":"2024-05-17T22:36:16.004076+0000","last_became_active":"2024-05-17T22:34:52.486654+0000","last_became_peered":"2024-05-17T22:34:52.486654+0000","last_unstale":"2024-05-17T22:36:16.004076+0000","last_undegraded":"2024-05-17T22:36:16.004076+0000","last_fullsized":"2024-05-17T22:36:16.004076+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:28:25.041746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625893+0000","last_change":"2024-05-17T22:34:37.405020+0000","last_active":"2024-05-17T22:36:15.625893+0000","last_peered":"2024-05-17T22:36:15.625893+0000","last_clean":"2024-05-17T22:36:15.625893+0000","last_became_active":"2024-05-17T22:34:37.404633+0000","last_became_peered":"2024-05-17T22:34:37.404633+0000","last_unstale":"2024-05-17T22:36:15.625893+0000","last_undegraded":"2024-05-17T22:36:15.625893+0000","last_fullsized":"2024-05-17T22:36:15.625893+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:06:00.001190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.525054+0000","last_change":"2024-05-17T22:34:40.435374+0000","last_active":"2024-05-17T22:37:14.525054+0000","last_peered":"2024-05-17T22:37:14.525054+0000","last_clean":"2024-05-17T22:37:14.525054+0000","last_became_active":"2024-05-17T22:34:40.434522+0000","last_became_peered":"2024-05-17T22:34:40.434522+0000","last_unstale":"2024-05-17T22:37:14.525054+0000","last_undegraded":"2024-05-17T22:37:14.525054+0000","last_fullsized":"2024-05-17T22:37:14.525054+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:36:41.244970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.805274+0000","last_change":"2024-05-17T22:34:52.491708+0000","last_active":"2024-05-17T22:36:15.805274+0000","last_peered":"2024-05-17T22:36:15.805274+0000","last_clean":"2024-05-17T22:36:15.805274+0000","last_became_active":"2024-05-17T22:34:52.491293+0000","last_became_peered":"2024-05-17T22:34:52.491293+0000","last_unstale":"2024-05-17T22:36:15.805274+0000","last_undegraded":"2024-05-17T22:36:15.805274+0000","last_fullsized":"2024-05-17T22:36:15.805274+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:23:01.634026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624679+0000","last_change":"2024-05-17T22:34:37.800525+0000","last_active":"2024-05-17T22:36:15.624679+0000","last_peered":"2024-05-17T22:36:15.624679+0000","last_clean":"2024-05-17T22:36:15.624679+0000","last_became_active":"2024-05-17T22:34:37.800093+0000","last_became_peered":"2024-05-17T22:34:37.800093+0000","last_unstale":"2024-05-17T22:36:15.624679+0000","last_undegraded":"2024-05-17T22:36:15.624679+0000","last_fullsized":"2024-05-17T22:36:15.624679+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:25:27.249889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.524773+0000","last_change":"2024-05-17T22:34:40.434891+0000","last_active":"2024-05-17T22:37:14.524773+0000","last_peered":"2024-05-17T22:37:14.524773+0000","last_clean":"2024-05-17T22:37:14.524773+0000","last_became_active":"2024-05-17T22:34:40.434543+0000","last_became_peered":"2024-05-17T22:34:40.434543+0000","last_unstale":"2024-05-17T22:37:14.524773+0000","last_undegraded":"2024-05-17T22:37:14.524773+0000","last_fullsized":"2024-05-17T22:37:14.524773+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:17:47.770225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968412+0000","last_change":"2024-05-17T22:34:52.491544+0000","last_active":"2024-05-17T22:36:15.968412+0000","last_peered":"2024-05-17T22:36:15.968412+0000","last_clean":"2024-05-17T22:36:15.968412+0000","last_became_active":"2024-05-17T22:34:52.491138+0000","last_became_peered":"2024-05-17T22:34:52.491138+0000","last_unstale":"2024-05-17T22:36:15.968412+0000","last_undegraded":"2024-05-17T22:36:15.968412+0000","last_fullsized":"2024-05-17T22:36:15.968412+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:34:20.411299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004302+0000","last_change":"2024-05-17T22:34:37.403220+0000","last_active":"2024-05-17T22:36:16.004302+0000","last_peered":"2024-05-17T22:36:16.004302+0000","last_clean":"2024-05-17T22:36:16.004302+0000","last_became_active":"2024-05-17T22:34:37.402856+0000","last_became_peered":"2024-05-17T22:34:37.402856+0000","last_unstale":"2024-05-17T22:36:16.004302+0000","last_undegraded":"2024-05-17T22:36:16.004302+0000","last_fullsized":"2024-05-17T22:36:16.004302+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:04:19.469294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.511989+0000","last_change":"2024-05-17T22:34:40.434487+0000","last_active":"2024-05-17T22:37:14.511989+0000","last_peered":"2024-05-17T22:37:14.511989+0000","last_clean":"2024-05-17T22:37:14.511989+0000","last_became_active":"2024-05-17T22:34:40.434160+0000","last_became_peered":"2024-05-17T22:34:40.434160+0000","last_unstale":"2024-05-17T22:37:14.511989+0000","last_undegraded":"2024-05-17T22:37:14.511989+0000","last_fullsized":"2024-05-17T22:37:14.511989+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:24:38.782999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003180+0000","last_change":"2024-05-17T22:34:52.487887+0000","last_active":"2024-05-17T22:36:16.003180+0000","last_peered":"2024-05-17T22:36:16.003180+0000","last_clean":"2024-05-17T22:36:16.003180+0000","last_became_active":"2024-05-17T22:34:52.486837+0000","last_became_peered":"2024-05-17T22:34:52.486837+0000","last_unstale":"2024-05-17T22:36:16.003180+0000","last_undegraded":"2024-05-17T22:36:16.003180+0000","last_fullsized":"2024-05-17T22:36:16.003180+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:12:22.206536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.686208+0000","last_change":"2024-05-17T22:34:37.399498+0000","last_active":"2024-05-17T22:36:15.686208+0000","last_peered":"2024-05-17T22:36:15.686208+0000","last_clean":"2024-05-17T22:36:15.686208+0000","last_became_active":"2024-05-17T22:34:37.399144+0000","last_became_peered":"2024-05-17T22:34:37.399144+0000","last_unstale":"2024-05-17T22:36:15.686208+0000","last_undegraded":"2024-05-17T22:36:15.686208+0000","last_fullsized":"2024-05-17T22:36:15.686208+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:13:37.346006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.514102+0000","last_change":"2024-05-17T22:34:40.433839+0000","last_active":"2024-05-17T22:37:14.514102+0000","last_peered":"2024-05-17T22:37:14.514102+0000","last_clean":"2024-05-17T22:37:14.514102+0000","last_became_active":"2024-05-17T22:34:40.433489+0000","last_became_peered":"2024-05-17T22:34:40.433489+0000","last_unstale":"2024-05-17T22:37:14.514102+0000","last_undegraded":"2024-05-17T22:37:14.514102+0000","last_fullsized":"2024-05-17T22:37:14.514102+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:13:56.014094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.512513+0000","last_change":"2024-05-17T22:34:40.434953+0000","last_active":"2024-05-17T22:37:14.512513+0000","last_peered":"2024-05-17T22:37:14.512513+0000","last_clean":"2024-05-17T22:37:14.512513+0000","last_became_active":"2024-05-17T22:34:40.434593+0000","last_became_peered":"2024-05-17T22:34:40.434593+0000","last_unstale":"2024-05-17T22:37:14.512513+0000","last_undegraded":"2024-05-17T22:37:14.512513+0000","last_fullsized":"2024-05-17T22:37:14.512513+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:20:27.075382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.513470+0000","last_change":"2024-05-17T22:34:40.432787+0000","last_active":"2024-05-17T22:37:14.513470+0000","last_peered":"2024-05-17T22:37:14.513470+0000","last_clean":"2024-05-17T22:37:14.513470+0000","last_became_active":"2024-05-17T22:34:40.432402+0000","last_became_peered":"2024-05-17T22:34:40.432402+0000","last_unstale":"2024-05-17T22:37:14.513470+0000","last_undegraded":"2024-05-17T22:37:14.513470+0000","last_fullsized":"2024-05-17T22:37:14.513470+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:48:13.034276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.515580+0000","last_change":"2024-05-17T22:34:40.426423+0000","last_active":"2024-05-17T22:37:14.515580+0000","last_peered":"2024-05-17T22:37:14.515580+0000","last_clean":"2024-05-17T22:37:14.515580+0000","last_became_active":"2024-05-17T22:34:40.426049+0000","last_became_peered":"2024-05-17T22:34:40.426049+0000","last_unstale":"2024-05-17T22:37:14.515580+0000","last_undegraded":"2024-05-17T22:37:14.515580+0000","last_fullsized":"2024-05-17T22:37:14.515580+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:56:55.631322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.520679+0000","last_change":"2024-05-17T22:34:40.435706+0000","last_active":"2024-05-17T22:37:14.520679+0000","last_peered":"2024-05-17T22:37:14.520679+0000","last_clean":"2024-05-17T22:37:14.520679+0000","last_became_active":"2024-05-17T22:34:40.435330+0000","last_became_peered":"2024-05-17T22:34:40.435330+0000","last_unstale":"2024-05-17T22:37:14.520679+0000","last_undegraded":"2024-05-17T22:37:14.520679+0000","last_fullsized":"2024-05-17T22:37:14.520679+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:25:36.529454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.517285+0000","last_change":"2024-05-17T22:34:40.428997+0000","last_active":"2024-05-17T22:37:14.517285+0000","last_peered":"2024-05-17T22:37:14.517285+0000","last_clean":"2024-05-17T22:37:14.517285+0000","last_became_active":"2024-05-17T22:34:40.428594+0000","last_became_peered":"2024-05-17T22:34:40.428594+0000","last_unstale":"2024-05-17T22:37:14.517285+0000","last_undegraded":"2024-05-17T22:37:14.517285+0000","last_fullsized":"2024-05-17T22:37:14.517285+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:59:01.777751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.509268+0000","last_change":"2024-05-17T22:34:40.434241+0000","last_active":"2024-05-17T22:37:14.509268+0000","last_peered":"2024-05-17T22:37:14.509268+0000","last_clean":"2024-05-17T22:37:14.509268+0000","last_became_active":"2024-05-17T22:34:40.431652+0000","last_became_peered":"2024-05-17T22:34:40.431652+0000","last_unstale":"2024-05-17T22:37:14.509268+0000","last_undegraded":"2024-05-17T22:37:14.509268+0000","last_fullsized":"2024-05-17T22:37:14.509268+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:33:27.035284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.505573+0000","last_change":"2024-05-17T22:34:40.431485+0000","last_active":"2024-05-17T22:37:14.505573+0000","last_peered":"2024-05-17T22:37:14.505573+0000","last_clean":"2024-05-17T22:37:14.505573+0000","last_became_active":"2024-05-17T22:34:40.431120+0000","last_became_peered":"2024-05-17T22:34:40.431120+0000","last_unstale":"2024-05-17T22:37:14.505573+0000","last_undegraded":"2024-05-17T22:37:14.505573+0000","last_fullsized":"2024-05-17T22:37:14.505573+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:46:55.213602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.501567+0000","last_change":"2024-05-17T22:34:40.423117+0000","last_active":"2024-05-17T22:37:14.501567+0000","last_peered":"2024-05-17T22:37:14.501567+0000","last_clean":"2024-05-17T22:37:14.501567+0000","last_became_active":"2024-05-17T22:34:40.422694+0000","last_became_peered":"2024-05-17T22:34:40.422694+0000","last_unstale":"2024-05-17T22:37:14.501567+0000","last_undegraded":"2024-05-17T22:37:14.501567+0000","last_fullsized":"2024-05-17T22:37:14.501567+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:19:55.589532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.518389+0000","last_change":"2024-05-17T22:35:15.711796+0000","last_active":"2024-05-17T22:37:14.518389+0000","last_peered":"2024-05-17T22:37:14.518389+0000","last_clean":"2024-05-17T22:37:14.518389+0000","last_became_active":"2024-05-17T22:35:15.711327+0000","last_became_peered":"2024-05-17T22:35:15.711327+0000","last_unstale":"2024-05-17T22:37:14.518389+0000","last_undegraded":"2024-05-17T22:37:14.518389+0000","last_fullsized":"2024-05-17T22:37:14.518389+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:48:48.913312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.512607+0000","last_change":"2024-05-17T22:34:40.430393+0000","last_active":"2024-05-17T22:37:14.512607+0000","last_peered":"2024-05-17T22:37:14.512607+0000","last_clean":"2024-05-17T22:37:14.512607+0000","last_became_active":"2024-05-17T22:34:40.430030+0000","last_became_peered":"2024-05-17T22:34:40.430030+0000","last_unstale":"2024-05-17T22:37:14.512607+0000","last_undegraded":"2024-05-17T22:37:14.512607+0000","last_fullsized":"2024-05-17T22:37:14.512607+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:39:37.204246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.526052+0000","last_change":"2024-05-17T22:34:40.432443+0000","last_active":"2024-05-17T22:37:14.526052+0000","last_peered":"2024-05-17T22:37:14.526052+0000","last_clean":"2024-05-17T22:37:14.526052+0000","last_became_active":"2024-05-17T22:34:40.431991+0000","last_became_peered":"2024-05-17T22:34:40.431991+0000","last_unstale":"2024-05-17T22:37:14.526052+0000","last_undegraded":"2024-05-17T22:37:14.526052+0000","last_fullsized":"2024-05-17T22:37:14.526052+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:37:45.593575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.523169+0000","last_change":"2024-05-17T22:34:40.434388+0000","last_active":"2024-05-17T22:37:14.523169+0000","last_peered":"2024-05-17T22:37:14.523169+0000","last_clean":"2024-05-17T22:37:14.523169+0000","last_became_active":"2024-05-17T22:34:40.434044+0000","last_became_peered":"2024-05-17T22:34:40.434044+0000","last_unstale":"2024-05-17T22:37:14.523169+0000","last_undegraded":"2024-05-17T22:37:14.523169+0000","last_fullsized":"2024-05-17T22:37:14.523169+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:20:43.296190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.507137+0000","last_change":"2024-05-17T22:34:40.427137+0000","last_active":"2024-05-17T22:37:14.507137+0000","last_peered":"2024-05-17T22:37:14.507137+0000","last_clean":"2024-05-17T22:37:14.507137+0000","last_became_active":"2024-05-17T22:34:40.426703+0000","last_became_peered":"2024-05-17T22:34:40.426703+0000","last_unstale":"2024-05-17T22:37:14.507137+0000","last_undegraded":"2024-05-17T22:37:14.507137+0000","last_fullsized":"2024-05-17T22:37:14.507137+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:55:52.992817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66995,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":727,"num_read_kb":432,"num_write":178,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":200985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710774,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":515036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907003,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28692,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716460,"statfs":{"total":95995035648,"available":95965655040,"internally_reserved":0,"allocated":1716224,"data_stored":1165135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070527,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":519346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266758,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":515148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462986,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1634304,"data_stored":1111752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626510,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":510426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1089536,"data_stored":572657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738478,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":511814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411429,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1114112,"data_stored":574539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182496,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":510620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986266,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":515608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822742,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36940,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708212,"statfs":{"total":95995035648,"available":95957209088,"internally_reserved":0,"allocated":1642496,"data_stored":1106808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T22:37:50.464 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-17T22:37:50.464 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-17T22:37:50.465 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-17T22:37:50.465 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph health --format=json 2024-05-17T22:37:50.725 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:37:51.101 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:50 smithi012 ceph-mon[31632]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T22:37:51.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:50 smithi151 ceph-mon[36101]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T22:37:51.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:50 smithi096 ceph-mon[36065]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T22:37:52.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:51 smithi012 ceph-mon[31632]: pgmap v397: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T22:37:52.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:51 smithi151 ceph-mon[36101]: pgmap v397: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T22:37:52.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:51 smithi096 ceph-mon[36065]: pgmap v397: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T22:37:52.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.300+0000 7f28bcf16700 1 -- 172.21.15.12:0/3543713838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f28b8104e20 msgr2=0x7f28b8105230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:52.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.300+0000 7f28bcf16700 1 --2- 172.21.15.12:0/3543713838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f28b8104e20 0x7f28b8105230 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f28a8009990 tx=0x7f28a8009ca0 comp rx=0 tx=0).stop 2024-05-17T22:37:52.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.300+0000 7f28bcf16700 1 -- 172.21.15.12:0/3543713838 shutdown_connections 2024-05-17T22:37:52.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.300+0000 7f28bcf16700 1 --2- 172.21.15.12:0/3543713838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28b810e250 0x7f28b8110760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:52.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.300+0000 7f28bcf16700 1 --2- 172.21.15.12:0/3543713838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f28b8104e20 0x7f28b8105230 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:52.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.300+0000 7f28bcf16700 1 --2- 172.21.15.12:0/3543713838 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28b8105770 0x7f28b810dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:52.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.300+0000 7f28bcf16700 1 -- 172.21.15.12:0/3543713838 >> 172.21.15.12:0/3543713838 conn(0x7f28b8079af0 msgr2=0x7f28b8079f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:52.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.300+0000 7f28bcf16700 1 -- 172.21.15.12:0/3543713838 shutdown_connections 2024-05-17T22:37:52.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.300+0000 7f28bcf16700 1 -- 172.21.15.12:0/3543713838 wait complete. 2024-05-17T22:37:52.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.301+0000 7f28bcf16700 1 Processor -- start 2024-05-17T22:37:52.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.301+0000 7f28bcf16700 1 -- start start 2024-05-17T22:37:52.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.301+0000 7f28bcf16700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f28b8104e20 0x7f28b819a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:52.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.301+0000 7f28bcf16700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28b8105770 0x7f28b819ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:52.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.301+0000 7f28bcf16700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28b810e250 0x7f28b819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:52.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.301+0000 7f28bcf16700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f28b8113e50 con 0x7f28b8104e20 2024-05-17T22:37:52.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.301+0000 7f28bcf16700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f28b8113cd0 con 0x7f28b8105770 2024-05-17T22:37:52.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.301+0000 7f28bcf16700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f28b8113fd0 con 0x7f28b810e250 2024-05-17T22:37:52.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.301+0000 7f28b659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f28b8104e20 0x7f28b819a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:52.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.301+0000 7f28b659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f28b8104e20 0x7f28b819a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46138/0 (socket says 172.21.15.12:46138) 2024-05-17T22:37:52.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.301+0000 7f28b659c700 1 -- 172.21.15.12:0/2757680837 learned_addr learned my addr 172.21.15.12:0/2757680837 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:37:52.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.301+0000 7f28b6d9d700 1 --2- 172.21.15.12:0/2757680837 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28b810e250 0x7f28b819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:52.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.301+0000 7f28b5d9b700 1 --2- 172.21.15.12:0/2757680837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28b8105770 0x7f28b819ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:52.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.301+0000 7f28b659c700 1 -- 172.21.15.12:0/2757680837 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28b810e250 msgr2=0x7f28b819f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:52.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.301+0000 7f28b659c700 1 --2- 172.21.15.12:0/2757680837 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28b810e250 0x7f28b819f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:52.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.301+0000 7f28b659c700 1 -- 172.21.15.12:0/2757680837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28b8105770 msgr2=0x7f28b819ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:52.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.301+0000 7f28b659c700 1 --2- 172.21.15.12:0/2757680837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28b8105770 0x7f28b819ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:52.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.301+0000 7f28b659c700 1 -- 172.21.15.12:0/2757680837 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b819f6a0 con 0x7f28b8104e20 2024-05-17T22:37:52.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.301+0000 7f28b5d9b700 1 --2- 172.21.15.12:0/2757680837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28b8105770 0x7f28b819ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:37:52.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.301+0000 7f28b6d9d700 1 --2- 172.21.15.12:0/2757680837 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28b810e250 0x7f28b819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:37:52.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.302+0000 7f28b659c700 1 --2- 172.21.15.12:0/2757680837 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f28b8104e20 0x7f28b819a7d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f28a8000c00 tx=0x7f28a80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:52.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.302+0000 7f28a77fe700 1 -- 172.21.15.12:0/2757680837 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a8027070 con 0x7f28b8104e20 2024-05-17T22:37:52.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.302+0000 7f28bcf16700 1 -- 172.21.15.12:0/2757680837 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28b819f930 con 0x7f28b8104e20 2024-05-17T22:37:52.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.302+0000 7f28a77fe700 1 -- 172.21.15.12:0/2757680837 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28a8003fa0 con 0x7f28b8104e20 2024-05-17T22:37:52.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.302+0000 7f28a77fe700 1 -- 172.21.15.12:0/2757680837 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a8022db0 con 0x7f28b8104e20 2024-05-17T22:37:52.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.302+0000 7f28bcf16700 1 -- 172.21.15.12:0/2757680837 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28b819fe10 con 0x7f28b8104e20 2024-05-17T22:37:52.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.303+0000 7f28bcf16700 1 -- 172.21.15.12:0/2757680837 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28b8105230 con 0x7f28b8104e20 2024-05-17T22:37:52.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.306+0000 7f28a77fe700 1 -- 172.21.15.12:0/2757680837 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f28a802a070 con 0x7f28b8104e20 2024-05-17T22:37:52.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.306+0000 7f28a77fe700 1 --2- 172.21.15.12:0/2757680837 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f28a00b2880 0x7f28a00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:52.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.307+0000 7f28b5d9b700 1 --2- 172.21.15.12:0/2757680837 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f28a00b2880 0x7f28a00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:52.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.307+0000 7f28a77fe700 1 -- 172.21.15.12:0/2757680837 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f28a80e3770 con 0x7f28b8104e20 2024-05-17T22:37:52.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.307+0000 7f28b5d9b700 1 --2- 172.21.15.12:0/2757680837 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f28a00b2880 0x7f28a00b4d90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f28ac005950 tx=0x7f28ac0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:52.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.310+0000 7f28a77fe700 1 -- 172.21.15.12:0/2757680837 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f28a8013070 con 0x7f28b8104e20 2024-05-17T22:37:52.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.583+0000 7f28bcf16700 1 -- 172.21.15.12:0/2757680837 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f28b819bb30 con 0x7f28b8104e20 2024-05-17T22:37:52.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.583+0000 7f28a77fe700 1 -- 172.21.15.12:0/2757680837 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f28a80ab100 con 0x7f28b8104e20 2024-05-17T22:37:52.586 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:37:52.586 INFO:teuthology.orchestra.run.smithi012.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-17T22:37:52.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.585+0000 7f28bcf16700 1 -- 172.21.15.12:0/2757680837 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f28a00b2880 msgr2=0x7f28a00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:52.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.585+0000 7f28bcf16700 1 --2- 172.21.15.12:0/2757680837 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f28a00b2880 0x7f28a00b4d90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f28ac005950 tx=0x7f28ac0058e0 comp rx=0 tx=0).stop 2024-05-17T22:37:52.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.585+0000 7f28bcf16700 1 -- 172.21.15.12:0/2757680837 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f28b8104e20 msgr2=0x7f28b819a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:52.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.585+0000 7f28bcf16700 1 --2- 172.21.15.12:0/2757680837 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f28b8104e20 0x7f28b819a7d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f28a8000c00 tx=0x7f28a80047b0 comp rx=0 tx=0).stop 2024-05-17T22:37:52.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.585+0000 7f28bcf16700 1 -- 172.21.15.12:0/2757680837 shutdown_connections 2024-05-17T22:37:52.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.585+0000 7f28bcf16700 1 --2- 172.21.15.12:0/2757680837 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f28a00b2880 0x7f28a00b4d90 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:52.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.585+0000 7f28bcf16700 1 --2- 172.21.15.12:0/2757680837 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28b810e250 0x7f28b819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:52.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.585+0000 7f28bcf16700 1 --2- 172.21.15.12:0/2757680837 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f28b8104e20 0x7f28b819a7d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:52.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.585+0000 7f28bcf16700 1 --2- 172.21.15.12:0/2757680837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28b8105770 0x7f28b819ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:52.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.585+0000 7f28bcf16700 1 -- 172.21.15.12:0/2757680837 >> 172.21.15.12:0/2757680837 conn(0x7f28b8079af0 msgr2=0x7f28b80ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:52.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.585+0000 7f28bcf16700 1 -- 172.21.15.12:0/2757680837 shutdown_connections 2024-05-17T22:37:52.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:52.585+0000 7f28bcf16700 1 -- 172.21.15.12:0/2757680837 wait complete. 2024-05-17T22:37:53.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:52 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/2757680837' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-17T22:37:53.264 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:52 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/2757680837' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-17T22:37:53.295 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-17T22:37:53.296 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T22:37:53.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:52 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/2757680837' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-17T22:37:53.552 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:37:54.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:53 smithi151 ceph-mon[36101]: pgmap v398: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T22:37:54.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:53 smithi096 ceph-mon[36065]: pgmap v398: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T22:37:54.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:53 smithi012 ceph-mon[31632]: pgmap v398: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T22:37:55.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.283+0000 7f2926e08700 1 -- 172.21.15.12:0/2221801250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2920107750 msgr2=0x7f2920107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:55.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.283+0000 7f2926e08700 1 --2- 172.21.15.12:0/2221801250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2920107750 0x7f2920107b60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2908009990 tx=0x7f2908009ca0 comp rx=0 tx=0).stop 2024-05-17T22:37:55.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.283+0000 7f2926e08700 1 -- 172.21.15.12:0/2221801250 shutdown_connections 2024-05-17T22:37:55.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.283+0000 7f2926e08700 1 --2- 172.21.15.12:0/2221801250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f29201080a0 0x7f292010fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:55.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.283+0000 7f2926e08700 1 --2- 172.21.15.12:0/2221801250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2920102e70 0x7f2920103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:55.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.283+0000 7f2926e08700 1 --2- 172.21.15.12:0/2221801250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2920107750 0x7f2920107b60 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:55.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.283+0000 7f2926e08700 1 -- 172.21.15.12:0/2221801250 >> 172.21.15.12:0/2221801250 conn(0x7f2920079ae0 msgr2=0x7f2920079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:55.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.283+0000 7f2926e08700 1 -- 172.21.15.12:0/2221801250 shutdown_connections 2024-05-17T22:37:55.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.283+0000 7f2926e08700 1 -- 172.21.15.12:0/2221801250 wait complete. 2024-05-17T22:37:55.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.283+0000 7f2926e08700 1 Processor -- start 2024-05-17T22:37:55.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.284+0000 7f2926e08700 1 -- start start 2024-05-17T22:37:55.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.284+0000 7f2926e08700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2920102e70 0x7f292019b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:55.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.284+0000 7f2926e08700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2920107750 0x7f292019bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:55.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.284+0000 7f2926e08700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f29201080a0 0x7f292019c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:55.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.284+0000 7f2926e08700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2920112cb0 con 0x7f2920107750 2024-05-17T22:37:55.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.284+0000 7f2926e08700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2920112b30 con 0x7f2920102e70 2024-05-17T22:37:55.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.284+0000 7f2926e08700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2920112e30 con 0x7f29201080a0 2024-05-17T22:37:55.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.284+0000 7f291ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2920107750 0x7f292019bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:55.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.284+0000 7f291ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2920107750 0x7f292019bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46158/0 (socket says 172.21.15.12:46158) 2024-05-17T22:37:55.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.284+0000 7f291ffff700 1 -- 172.21.15.12:0/3873039047 learned_addr learned my addr 172.21.15.12:0/3873039047 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:37:55.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.284+0000 7f2924ba4700 1 --2- 172.21.15.12:0/3873039047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2920102e70 0x7f292019b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:55.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.284+0000 7f29253a5700 1 --2- 172.21.15.12:0/3873039047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f29201080a0 0x7f292019c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:55.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.285+0000 7f291ffff700 1 -- 172.21.15.12:0/3873039047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f29201080a0 msgr2=0x7f292019c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:55.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.285+0000 7f291ffff700 1 --2- 172.21.15.12:0/3873039047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f29201080a0 0x7f292019c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:55.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.285+0000 7f291ffff700 1 -- 172.21.15.12:0/3873039047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2920102e70 msgr2=0x7f292019b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:55.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.285+0000 7f291ffff700 1 --2- 172.21.15.12:0/3873039047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2920102e70 0x7f292019b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:55.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.285+0000 7f291ffff700 1 -- 172.21.15.12:0/3873039047 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29201a68e0 con 0x7f2920107750 2024-05-17T22:37:55.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.285+0000 7f2924ba4700 1 --2- 172.21.15.12:0/3873039047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2920102e70 0x7f292019b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:37:55.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.285+0000 7f291ffff700 1 --2- 172.21.15.12:0/3873039047 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2920107750 0x7f292019bb60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2908000c00 tx=0x7f2908005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:55.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.285+0000 7f291dffb700 1 -- 172.21.15.12:0/3873039047 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2908027070 con 0x7f2920107750 2024-05-17T22:37:55.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.285+0000 7f2926e08700 1 -- 172.21.15.12:0/3873039047 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29201a6b70 con 0x7f2920107750 2024-05-17T22:37:55.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.285+0000 7f291dffb700 1 -- 172.21.15.12:0/3873039047 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2908003fe0 con 0x7f2920107750 2024-05-17T22:37:55.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.285+0000 7f291dffb700 1 -- 172.21.15.12:0/3873039047 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2908022c30 con 0x7f2920107750 2024-05-17T22:37:55.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.285+0000 7f2926e08700 1 -- 172.21.15.12:0/3873039047 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29201a70a0 con 0x7f2920107750 2024-05-17T22:37:55.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.286+0000 7f2926e08700 1 -- 172.21.15.12:0/3873039047 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2920103280 con 0x7f2920107750 2024-05-17T22:37:55.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.289+0000 7f291dffb700 1 -- 172.21.15.12:0/3873039047 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f290802a070 con 0x7f2920107750 2024-05-17T22:37:55.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.290+0000 7f291dffb700 1 --2- 172.21.15.12:0/3873039047 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f290c0b2900 0x7f290c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:55.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.290+0000 7f2924ba4700 1 --2- 172.21.15.12:0/3873039047 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f290c0b2900 0x7f290c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:55.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.290+0000 7f291dffb700 1 -- 172.21.15.12:0/3873039047 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f29080e3a40 con 0x7f2920107750 2024-05-17T22:37:55.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.290+0000 7f2924ba4700 1 --2- 172.21.15.12:0/3873039047 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f290c0b2900 0x7f290c0b4e10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2910005950 tx=0x7f29100058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:55.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.293+0000 7f291dffb700 1 -- 172.21.15.12:0/3873039047 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2908031080 con 0x7f2920107750 2024-05-17T22:37:55.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.556+0000 7f2926e08700 1 -- 172.21.15.12:0/3873039047 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f292006c630 con 0x7f2920107750 2024-05-17T22:37:55.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.557+0000 7f291dffb700 1 -- 172.21.15.12:0/3873039047 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11870 (secure 0 0 0) 0x7f2908013070 con 0x7f2920107750 2024-05-17T22:37:55.560 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:37:55.560 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1407219319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1407219319},{"type":"v1","addr":"172.21.15.12:6835","nonce":1407219319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/630977886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":630977886},{"type":"v1","addr":"172.21.15.12:6837","nonce":630977886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3731914742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3731914742},{"type":"v1","addr":"172.21.15.12:6839","nonce":3731914742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/3642142501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3642142501},{"type":"v1","addr":"172.21.15.12:6841","nonce":3642142501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3554053059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3554053059},{"type":"v1","addr":"172.21.15.151:6833","nonce":3554053059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/4228325569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":4228325569},{"type":"v1","addr":"172.21.15.96:6833","nonce":4228325569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T22:34:42.642111+0000","modified":"2024-05-17T22:35:52.999468+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24461,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/542323754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":542323754},{"type":"v1","addr":"172.21.15.151:6835","nonce":542323754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1758442232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1758442232},{"type":"v1","addr":"172.21.15.96:6835","nonce":1758442232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3403324038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3403324038},{"type":"v1","addr":"172.21.15.151:6837","nonce":3403324038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.151:6839/2338353849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2338353849},{"type":"v1","addr":"172.21.15.151:6839","nonce":2338353849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.96:6837/4222574338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":4222574338},{"type":"v1","addr":"172.21.15.96:6837","nonce":4222574338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/2986295519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2986295519},{"type":"v1","addr":"172.21.15.96:6839","nonce":2986295519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24457,24482,24461]},"id":1}]} 2024-05-17T22:37:55.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.560+0000 7f2926e08700 1 -- 172.21.15.12:0/3873039047 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f290c0b2900 msgr2=0x7f290c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:55.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.560+0000 7f2926e08700 1 --2- 172.21.15.12:0/3873039047 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f290c0b2900 0x7f290c0b4e10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2910005950 tx=0x7f29100058e0 comp rx=0 tx=0).stop 2024-05-17T22:37:55.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.560+0000 7f2926e08700 1 -- 172.21.15.12:0/3873039047 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2920107750 msgr2=0x7f292019bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:55.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.560+0000 7f2926e08700 1 --2- 172.21.15.12:0/3873039047 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2920107750 0x7f292019bb60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2908000c00 tx=0x7f2908005ef0 comp rx=0 tx=0).stop 2024-05-17T22:37:55.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.560+0000 7f2926e08700 1 -- 172.21.15.12:0/3873039047 shutdown_connections 2024-05-17T22:37:55.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.560+0000 7f2926e08700 1 --2- 172.21.15.12:0/3873039047 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f290c0b2900 0x7f290c0b4e10 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:55.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.560+0000 7f2926e08700 1 --2- 172.21.15.12:0/3873039047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f29201080a0 0x7f292019c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:55.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.560+0000 7f2926e08700 1 --2- 172.21.15.12:0/3873039047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2920102e70 0x7f292019b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:55.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.560+0000 7f2926e08700 1 --2- 172.21.15.12:0/3873039047 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2920107750 0x7f292019bb60 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:55.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.560+0000 7f2926e08700 1 -- 172.21.15.12:0/3873039047 >> 172.21.15.12:0/3873039047 conn(0x7f2920079ae0 msgr2=0x7f2920105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:55.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.560+0000 7f2926e08700 1 -- 172.21.15.12:0/3873039047 shutdown_connections 2024-05-17T22:37:55.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:55.560+0000 7f2926e08700 1 -- 172.21.15.12:0/3873039047 wait complete. 2024-05-17T22:37:55.564 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 26 2024-05-17T22:37:56.132 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:55 smithi012 ceph-mon[31632]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T22:37:56.132 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:55 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/3873039047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:37:56.165 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T22:37:56.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:55 smithi151 ceph-mon[36101]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T22:37:56.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:55 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/3873039047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:37:56.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:55 smithi096 ceph-mon[36065]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T22:37:56.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:55 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/3873039047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:37:56.421 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:37:58.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.143+0000 7fa75bae2700 1 -- 172.21.15.12:0/3786424459 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7541080a0 msgr2=0x7fa75410fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:58.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.143+0000 7fa75bae2700 1 --2- 172.21.15.12:0/3786424459 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7541080a0 0x7fa75410fa00 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa74c009990 tx=0x7fa74c009ca0 comp rx=0 tx=0).stop 2024-05-17T22:37:58.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.143+0000 7fa75bae2700 1 -- 172.21.15.12:0/3786424459 shutdown_connections 2024-05-17T22:37:58.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.143+0000 7fa75bae2700 1 --2- 172.21.15.12:0/3786424459 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7541080a0 0x7fa75410fa00 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:58.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.143+0000 7fa75bae2700 1 --2- 172.21.15.12:0/3786424459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa754102e70 0x7fa754103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:58.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.143+0000 7fa75bae2700 1 --2- 172.21.15.12:0/3786424459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa754107750 0x7fa754107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:58.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.143+0000 7fa75bae2700 1 -- 172.21.15.12:0/3786424459 >> 172.21.15.12:0/3786424459 conn(0x7fa754079ae0 msgr2=0x7fa754079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:58.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.143+0000 7fa75bae2700 1 -- 172.21.15.12:0/3786424459 shutdown_connections 2024-05-17T22:37:58.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.143+0000 7fa75bae2700 1 -- 172.21.15.12:0/3786424459 wait complete. 2024-05-17T22:37:58.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.143+0000 7fa75bae2700 1 Processor -- start 2024-05-17T22:37:58.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.143+0000 7fa75bae2700 1 -- start start 2024-05-17T22:37:58.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.144+0000 7fa75bae2700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa754102e70 0x7fa7541a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:58.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.144+0000 7fa75bae2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa754107750 0x7fa7541a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:58.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.144+0000 7fa75bae2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7541080a0 0x7fa75419a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:58.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.144+0000 7fa75bae2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa754112dd0 con 0x7fa754107750 2024-05-17T22:37:58.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.144+0000 7fa75bae2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa754112c50 con 0x7fa7541080a0 2024-05-17T22:37:58.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.144+0000 7fa75bae2700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa754112f50 con 0x7fa754102e70 2024-05-17T22:37:58.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.144+0000 7fa75907d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa754107750 0x7fa7541a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:58.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.144+0000 7fa75907d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa754107750 0x7fa7541a0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46178/0 (socket says 172.21.15.12:46178) 2024-05-17T22:37:58.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.144+0000 7fa75907d700 1 -- 172.21.15.12:0/2095449160 learned_addr learned my addr 172.21.15.12:0/2095449160 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:37:58.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.144+0000 7fa75987e700 1 --2- 172.21.15.12:0/2095449160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa754102e70 0x7fa7541a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:58.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.144+0000 7fa75a07f700 1 --2- 172.21.15.12:0/2095449160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7541080a0 0x7fa75419a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:58.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.144+0000 7fa75907d700 1 -- 172.21.15.12:0/2095449160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa754102e70 msgr2=0x7fa7541a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:58.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.144+0000 7fa75907d700 1 --2- 172.21.15.12:0/2095449160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa754102e70 0x7fa7541a0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:58.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.144+0000 7fa75907d700 1 -- 172.21.15.12:0/2095449160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7541080a0 msgr2=0x7fa75419a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:58.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.144+0000 7fa75907d700 1 --2- 172.21.15.12:0/2095449160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7541080a0 0x7fa75419a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:58.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.144+0000 7fa75907d700 1 -- 172.21.15.12:0/2095449160 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa75419af00 con 0x7fa754107750 2024-05-17T22:37:58.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.144+0000 7fa75987e700 1 --2- 172.21.15.12:0/2095449160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa754102e70 0x7fa7541a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:37:58.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.144+0000 7fa75a07f700 1 --2- 172.21.15.12:0/2095449160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7541080a0 0x7fa75419a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:37:58.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.144+0000 7fa75907d700 1 --2- 172.21.15.12:0/2095449160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa754107750 0x7fa7541a0af0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa74400b500 tx=0x7fa74400b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:58.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.145+0000 7fa74affd700 1 -- 172.21.15.12:0/2095449160 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa744015020 con 0x7fa754107750 2024-05-17T22:37:58.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.145+0000 7fa74affd700 1 -- 172.21.15.12:0/2095449160 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa744011ce0 con 0x7fa754107750 2024-05-17T22:37:58.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.145+0000 7fa75bae2700 1 -- 172.21.15.12:0/2095449160 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa75419b1f0 con 0x7fa754107750 2024-05-17T22:37:58.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.145+0000 7fa74affd700 1 -- 172.21.15.12:0/2095449160 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa744010d40 con 0x7fa754107750 2024-05-17T22:37:58.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.145+0000 7fa75bae2700 1 -- 172.21.15.12:0/2095449160 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7541a73f0 con 0x7fa754107750 2024-05-17T22:37:58.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.147+0000 7fa75bae2700 1 -- 172.21.15.12:0/2095449160 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7541042d0 con 0x7fa754107750 2024-05-17T22:37:58.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.148+0000 7fa74affd700 1 -- 172.21.15.12:0/2095449160 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa744018020 con 0x7fa754107750 2024-05-17T22:37:58.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.149+0000 7fa74affd700 1 --2- 172.21.15.12:0/2095449160 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa7400b2880 0x7fa7400b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:37:58.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.149+0000 7fa75987e700 1 --2- 172.21.15.12:0/2095449160 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa7400b2880 0x7fa7400b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:37:58.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.149+0000 7fa74affd700 1 -- 172.21.15.12:0/2095449160 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fa7440e1db0 con 0x7fa754107750 2024-05-17T22:37:58.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.149+0000 7fa75987e700 1 --2- 172.21.15.12:0/2095449160 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa7400b2880 0x7fa7400b4d90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa75419bb00 tx=0x7fa7500073e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:37:58.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.153+0000 7fa74affd700 1 -- 172.21.15.12:0/2095449160 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa74401f020 con 0x7fa754107750 2024-05-17T22:37:58.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:57 smithi151 ceph-mon[36101]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T22:37:58.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:57 smithi096 ceph-mon[36065]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T22:37:58.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:57 smithi012 ceph-mon[31632]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T22:37:58.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.415+0000 7fa75bae2700 1 -- 172.21.15.12:0/2095449160 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa754103300 con 0x7fa754107750 2024-05-17T22:37:58.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.416+0000 7fa74affd700 1 -- 172.21.15.12:0/2095449160 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11870 (secure 0 0 0) 0x7fa7440a9740 con 0x7fa754107750 2024-05-17T22:37:58.419 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:37:58.419 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1407219319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1407219319},{"type":"v1","addr":"172.21.15.12:6835","nonce":1407219319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/630977886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":630977886},{"type":"v1","addr":"172.21.15.12:6837","nonce":630977886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3731914742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3731914742},{"type":"v1","addr":"172.21.15.12:6839","nonce":3731914742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/3642142501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3642142501},{"type":"v1","addr":"172.21.15.12:6841","nonce":3642142501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3554053059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3554053059},{"type":"v1","addr":"172.21.15.151:6833","nonce":3554053059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/4228325569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":4228325569},{"type":"v1","addr":"172.21.15.96:6833","nonce":4228325569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T22:34:42.642111+0000","modified":"2024-05-17T22:35:52.999468+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24461,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/542323754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":542323754},{"type":"v1","addr":"172.21.15.151:6835","nonce":542323754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1758442232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1758442232},{"type":"v1","addr":"172.21.15.96:6835","nonce":1758442232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3403324038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3403324038},{"type":"v1","addr":"172.21.15.151:6837","nonce":3403324038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.151:6839/2338353849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2338353849},{"type":"v1","addr":"172.21.15.151:6839","nonce":2338353849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.96:6837/4222574338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":4222574338},{"type":"v1","addr":"172.21.15.96:6837","nonce":4222574338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/2986295519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2986295519},{"type":"v1","addr":"172.21.15.96:6839","nonce":2986295519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24457,24482,24461]},"id":1}]} 2024-05-17T22:37:58.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.418+0000 7fa75bae2700 1 -- 172.21.15.12:0/2095449160 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa7400b2880 msgr2=0x7fa7400b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:58.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.418+0000 7fa75bae2700 1 --2- 172.21.15.12:0/2095449160 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa7400b2880 0x7fa7400b4d90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa75419bb00 tx=0x7fa7500073e0 comp rx=0 tx=0).stop 2024-05-17T22:37:58.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.418+0000 7fa75bae2700 1 -- 172.21.15.12:0/2095449160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa754107750 msgr2=0x7fa7541a0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:37:58.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.418+0000 7fa75bae2700 1 --2- 172.21.15.12:0/2095449160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa754107750 0x7fa7541a0af0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa74400b500 tx=0x7fa74400b8c0 comp rx=0 tx=0).stop 2024-05-17T22:37:58.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.418+0000 7fa75bae2700 1 -- 172.21.15.12:0/2095449160 shutdown_connections 2024-05-17T22:37:58.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.418+0000 7fa75bae2700 1 --2- 172.21.15.12:0/2095449160 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa7400b2880 0x7fa7400b4d90 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:58.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.418+0000 7fa75bae2700 1 --2- 172.21.15.12:0/2095449160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7541080a0 0x7fa75419a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:58.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.418+0000 7fa75bae2700 1 --2- 172.21.15.12:0/2095449160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa754102e70 0x7fa7541a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:58.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.418+0000 7fa75bae2700 1 --2- 172.21.15.12:0/2095449160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa754107750 0x7fa7541a0af0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:37:58.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.418+0000 7fa75bae2700 1 -- 172.21.15.12:0/2095449160 >> 172.21.15.12:0/2095449160 conn(0x7fa754079ae0 msgr2=0x7fa75406c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:37:58.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.418+0000 7fa75bae2700 1 -- 172.21.15.12:0/2095449160 shutdown_connections 2024-05-17T22:37:58.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:37:58.418+0000 7fa75bae2700 1 -- 172.21.15.12:0/2095449160 wait complete. 2024-05-17T22:37:58.422 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 26 2024-05-17T22:37:59.051 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:58 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/2095449160' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:37:59.083 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-17T22:37:59.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:58 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/2095449160' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:37:59.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:58 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/2095449160' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:37:59.339 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:38:00.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:37:59 smithi012 ceph-mon[31632]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:38:00.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:37:59 smithi151 ceph-mon[36101]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:38:00.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:37:59 smithi096 ceph-mon[36065]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:38:01.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.064+0000 7f89d637e700 1 -- 172.21.15.12:0/2713665649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89d010e200 msgr2=0x7f89d0110710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:01.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.064+0000 7f89d637e700 1 --2- 172.21.15.12:0/2713665649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89d010e200 0x7f89d0110710 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f89c800b0a0 tx=0x7f89c800b3b0 comp rx=0 tx=0).stop 2024-05-17T22:38:01.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.064+0000 7f89d637e700 1 -- 172.21.15.12:0/2713665649 shutdown_connections 2024-05-17T22:38:01.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.064+0000 7f89d637e700 1 --2- 172.21.15.12:0/2713665649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89d010e200 0x7f89d0110710 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:01.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.064+0000 7f89d637e700 1 --2- 172.21.15.12:0/2713665649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89d0104da0 0x7f89d01051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:01.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.064+0000 7f89d637e700 1 --2- 172.21.15.12:0/2713665649 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89d01056f0 0x7f89d010dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:01.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.064+0000 7f89d637e700 1 -- 172.21.15.12:0/2713665649 >> 172.21.15.12:0/2713665649 conn(0x7f89d0079bd0 msgr2=0x7f89d0079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:01.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.064+0000 7f89d637e700 1 -- 172.21.15.12:0/2713665649 shutdown_connections 2024-05-17T22:38:01.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.064+0000 7f89d637e700 1 -- 172.21.15.12:0/2713665649 wait complete. 2024-05-17T22:38:01.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.065+0000 7f89d637e700 1 Processor -- start 2024-05-17T22:38:01.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.065+0000 7f89d637e700 1 -- start start 2024-05-17T22:38:01.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.065+0000 7f89d637e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89d0104da0 0x7f89d01a0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:01.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.066+0000 7f89d637e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89d01056f0 0x7f89d01a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:01.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.066+0000 7f89d637e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89d010e200 0x7f89d019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:01.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.066+0000 7f89d637e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f89d0113e80 con 0x7f89d01056f0 2024-05-17T22:38:01.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.066+0000 7f89d637e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f89d0113d00 con 0x7f89d0104da0 2024-05-17T22:38:01.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.066+0000 7f89d637e700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f89d0114000 con 0x7f89d010e200 2024-05-17T22:38:01.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.066+0000 7f89cf7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89d01056f0 0x7f89d01a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:01.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.066+0000 7f89d491b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89d010e200 0x7f89d019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:01.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.066+0000 7f89cffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89d0104da0 0x7f89d01a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:01.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.066+0000 7f89cf7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89d01056f0 0x7f89d01a0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57524/0 (socket says 172.21.15.12:57524) 2024-05-17T22:38:01.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.066+0000 7f89d491b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89d010e200 0x7f89d019a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:46152/0 (socket says 172.21.15.12:46152) 2024-05-17T22:38:01.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.066+0000 7f89cffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89d0104da0 0x7f89d01a0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:39316/0 (socket says 172.21.15.12:39316) 2024-05-17T22:38:01.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.066+0000 7f89d491b700 1 -- 172.21.15.12:0/3477398692 learned_addr learned my addr 172.21.15.12:0/3477398692 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:01.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.066+0000 7f89d491b700 1 -- 172.21.15.12:0/3477398692 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89d0104da0 msgr2=0x7f89d01a0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:01.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.066+0000 7f89d491b700 1 --2- 172.21.15.12:0/3477398692 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89d0104da0 0x7f89d01a0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:01.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.066+0000 7f89d491b700 1 -- 172.21.15.12:0/3477398692 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89d01056f0 msgr2=0x7f89d01a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:01.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.066+0000 7f89d491b700 1 --2- 172.21.15.12:0/3477398692 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89d01056f0 0x7f89d01a0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:01.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.066+0000 7f89d491b700 1 -- 172.21.15.12:0/3477398692 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89d019ae20 con 0x7f89d010e200 2024-05-17T22:38:01.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.066+0000 7f89cffff700 1 --2- 172.21.15.12:0/3477398692 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89d0104da0 0x7f89d01a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:38:01.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.066+0000 7f89cf7fe700 1 --2- 172.21.15.12:0/3477398692 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89d01056f0 0x7f89d01a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:38:01.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.067+0000 7f89d491b700 1 --2- 172.21.15.12:0/3477398692 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89d010e200 0x7f89d019a6f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f89c80062a0 tx=0x7f89c8004e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:01.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.067+0000 7f89cd7fa700 1 -- 172.21.15.12:0/3477398692 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c8021070 con 0x7f89d010e200 2024-05-17T22:38:01.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.067+0000 7f89d637e700 1 -- 172.21.15.12:0/3477398692 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89d019b0b0 con 0x7f89d010e200 2024-05-17T22:38:01.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.067+0000 7f89cd7fa700 1 -- 172.21.15.12:0/3477398692 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89c8003c00 con 0x7f89d010e200 2024-05-17T22:38:01.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.067+0000 7f89cd7fa700 1 -- 172.21.15.12:0/3477398692 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c8004880 con 0x7f89d010e200 2024-05-17T22:38:01.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.067+0000 7f89d637e700 1 -- 172.21.15.12:0/3477398692 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89d01a7310 con 0x7f89d010e200 2024-05-17T22:38:01.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.068+0000 7f89d637e700 1 -- 172.21.15.12:0/3477398692 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89d006bf60 con 0x7f89d010e200 2024-05-17T22:38:01.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.070+0000 7f89cd7fa700 1 -- 172.21.15.12:0/3477398692 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f89c8007a00 con 0x7f89d010e200 2024-05-17T22:38:01.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.070+0000 7f89cd7fa700 1 --2- 172.21.15.12:0/3477398692 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f89b80b2880 0x7f89b80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:01.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.071+0000 7f89cffff700 1 --2- 172.21.15.12:0/3477398692 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f89b80b2880 0x7f89b80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:01.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.071+0000 7f89cd7fa700 1 -- 172.21.15.12:0/3477398692 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f89c800f020 con 0x7f89d010e200 2024-05-17T22:38:01.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.071+0000 7f89cffff700 1 --2- 172.21.15.12:0/3477398692 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f89b80b2880 0x7f89b80b4d90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f89c0009de0 tx=0x7f89c0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:01.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.074+0000 7f89cd7fa700 1 -- 172.21.15.12:0/3477398692 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f89c802a020 con 0x7f89d010e200 2024-05-17T22:38:01.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.290+0000 7f89d637e700 1 -- 172.21.15.12:0/3477398692 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f89d002eab0 con 0x7f89d010e200 2024-05-17T22:38:01.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.292+0000 7f89cd7fa700 1 -- 172.21.15.12:0/3477398692 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24715 (secure 0 0 0) 0x7f89c802e3e0 con 0x7f89d010e200 2024-05-17T22:38:01.296 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:38:01.296 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":89,"fsid":"4d593780-149c-11ef-bc9a-c7b262605968","created":"2024-05-17T22:26:16.955626+0000","modified":"2024-05-17T22:36:15.618992+0000","last_up_change":"2024-05-17T22:33:36.551093+0000","last_in_change":"2024-05-17T22:33:15.032677+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T22:29:42.475513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T22:34:35.504697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T22:34:39.377097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-17T22:34:50.467684+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5d42798f-dc5f-4e66-bce2-ec33dc789e4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6803","nonce":2015605593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6805","nonce":2015605593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6809","nonce":2015605593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6807","nonce":2015605593}]},"public_addr":"172.21.15.12:6803/2015605593","cluster_addr":"172.21.15.12:6805/2015605593","heartbeat_back_addr":"172.21.15.12:6809/2015605593","heartbeat_front_addr":"172.21.15.12:6807/2015605593","state":["exists","up"]},{"osd":1,"uuid":"58afecc7-119b-4c16-a401-640fb77d4714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6801","nonce":1238316358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6803","nonce":1238316358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6807","nonce":1238316358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6805","nonce":1238316358}]},"public_addr":"172.21.15.96:6801/1238316358","cluster_addr":"172.21.15.96:6803/1238316358","heartbeat_back_addr":"172.21.15.96:6807/1238316358","heartbeat_front_addr":"172.21.15.96:6805/1238316358","state":["exists","up"]},{"osd":2,"uuid":"4bef4049-482f-49fb-82fb-38052400eb2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6801","nonce":148729815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6803","nonce":148729815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6807","nonce":148729815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6805","nonce":148729815}]},"public_addr":"172.21.15.151:6801/148729815","cluster_addr":"172.21.15.151:6803/148729815","heartbeat_back_addr":"172.21.15.151:6807/148729815","heartbeat_front_addr":"172.21.15.151:6805/148729815","state":["exists","up"]},{"osd":3,"uuid":"6fed8e36-82a2-49e4-82a3-16e6d9de87f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6811","nonce":3563309320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6813","nonce":3563309320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6817","nonce":3563309320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6815","nonce":3563309320}]},"public_addr":"172.21.15.12:6811/3563309320","cluster_addr":"172.21.15.12:6813/3563309320","heartbeat_back_addr":"172.21.15.12:6817/3563309320","heartbeat_front_addr":"172.21.15.12:6815/3563309320","state":["exists","up"]},{"osd":4,"uuid":"c0e8689a-2816-4089-ade3-931960bae4f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6809","nonce":2219103166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6811","nonce":2219103166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6815","nonce":2219103166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6813","nonce":2219103166}]},"public_addr":"172.21.15.96:6809/2219103166","cluster_addr":"172.21.15.96:6811/2219103166","heartbeat_back_addr":"172.21.15.96:6815/2219103166","heartbeat_front_addr":"172.21.15.96:6813/2219103166","state":["exists","up"]},{"osd":5,"uuid":"e76ba432-94ff-4998-aaba-2645d9e6055d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6809","nonce":2589835078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6811","nonce":2589835078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6815","nonce":2589835078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6813","nonce":2589835078}]},"public_addr":"172.21.15.151:6809/2589835078","cluster_addr":"172.21.15.151:6811/2589835078","heartbeat_back_addr":"172.21.15.151:6815/2589835078","heartbeat_front_addr":"172.21.15.151:6813/2589835078","state":["exists","up"]},{"osd":6,"uuid":"b7ece35c-efd1-40df-95da-05a0cc147372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6819","nonce":3112661724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6821","nonce":3112661724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6825","nonce":3112661724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6823","nonce":3112661724}]},"public_addr":"172.21.15.12:6819/3112661724","cluster_addr":"172.21.15.12:6821/3112661724","heartbeat_back_addr":"172.21.15.12:6825/3112661724","heartbeat_front_addr":"172.21.15.12:6823/3112661724","state":["exists","up"]},{"osd":7,"uuid":"6a65bf30-daf5-4fba-b67e-8e513d88a971","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6817","nonce":2449989110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6819","nonce":2449989110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6823","nonce":2449989110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6821","nonce":2449989110}]},"public_addr":"172.21.15.96:6817/2449989110","cluster_addr":"172.21.15.96:6819/2449989110","heartbeat_back_addr":"172.21.15.96:6823/2449989110","heartbeat_front_addr":"172.21.15.96:6821/2449989110","state":["exists","up"]},{"osd":8,"uuid":"ad62ea44-e416-4a3e-bbbe-93f2a710dbc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6817","nonce":2545224628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6819","nonce":2545224628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6823","nonce":2545224628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6821","nonce":2545224628}]},"public_addr":"172.21.15.151:6817/2545224628","cluster_addr":"172.21.15.151:6819/2545224628","heartbeat_back_addr":"172.21.15.151:6823/2545224628","heartbeat_front_addr":"172.21.15.151:6821/2545224628","state":["exists","up"]},{"osd":9,"uuid":"1275de08-2a3c-48f3-920b-dd68b4024d42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6827","nonce":2100661273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6829","nonce":2100661273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6833","nonce":2100661273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6831","nonce":2100661273}]},"public_addr":"172.21.15.12:6827/2100661273","cluster_addr":"172.21.15.12:6829/2100661273","heartbeat_back_addr":"172.21.15.12:6833/2100661273","heartbeat_front_addr":"172.21.15.12:6831/2100661273","state":["exists","up"]},{"osd":10,"uuid":"347e4e40-9a9d-43fe-8f52-fdd3199a2a5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6825","nonce":304016309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6827","nonce":304016309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6831","nonce":304016309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6829","nonce":304016309}]},"public_addr":"172.21.15.96:6825/304016309","cluster_addr":"172.21.15.96:6827/304016309","heartbeat_back_addr":"172.21.15.96:6831/304016309","heartbeat_front_addr":"172.21.15.96:6829/304016309","state":["exists","up"]},{"osd":11,"uuid":"ee8d3502-8b3c-4e6f-824f-3a97e19111ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6825","nonce":3822307314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6827","nonce":3822307314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6831","nonce":3822307314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6829","nonce":3822307314}]},"public_addr":"172.21.15.151:6825/3822307314","cluster_addr":"172.21.15.151:6827/3822307314","heartbeat_back_addr":"172.21.15.151:6831/3822307314","heartbeat_front_addr":"172.21.15.151:6829/3822307314","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:28:51.542055+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:29:15.566424+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:29:39.500981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:04.797907+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:29.655650+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:55.211880+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:31:21.559462+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:31:47.154852+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:32:13.114892+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:32:40.529708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:33:06.930942+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:33:34.061783+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":10}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/2682700253":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6801/2250046521":"2024-05-18T22:27:14.372972+0000","172.21.15.12:0/4241094239":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/4172568611":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/3512529242":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6800/1039789215":"2024-05-18T22:26:45.466719+0000","172.21.15.12:6800/2250046521":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6801/1039789215":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/2043758453":"2024-05-18T22:27:14.372972+0000","172.21.15.12:0/2694593732":"2024-05-18T22:26:45.466719+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T22:38:01.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.295+0000 7f89d637e700 1 -- 172.21.15.12:0/3477398692 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f89b80b2880 msgr2=0x7f89b80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:01.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.295+0000 7f89d637e700 1 --2- 172.21.15.12:0/3477398692 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f89b80b2880 0x7f89b80b4d90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f89c0009de0 tx=0x7f89c0009190 comp rx=0 tx=0).stop 2024-05-17T22:38:01.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.295+0000 7f89d637e700 1 -- 172.21.15.12:0/3477398692 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89d010e200 msgr2=0x7f89d019a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:01.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.295+0000 7f89d637e700 1 --2- 172.21.15.12:0/3477398692 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89d010e200 0x7f89d019a6f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f89c80062a0 tx=0x7f89c8004e20 comp rx=0 tx=0).stop 2024-05-17T22:38:01.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.295+0000 7f89d637e700 1 -- 172.21.15.12:0/3477398692 shutdown_connections 2024-05-17T22:38:01.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.295+0000 7f89d637e700 1 --2- 172.21.15.12:0/3477398692 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f89b80b2880 0x7f89b80b4d90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:01.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.295+0000 7f89d637e700 1 --2- 172.21.15.12:0/3477398692 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89d010e200 0x7f89d019a6f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:01.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.295+0000 7f89d637e700 1 --2- 172.21.15.12:0/3477398692 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89d0104da0 0x7f89d01a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:01.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.295+0000 7f89d637e700 1 --2- 172.21.15.12:0/3477398692 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89d01056f0 0x7f89d01a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:01.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.295+0000 7f89d637e700 1 -- 172.21.15.12:0/3477398692 >> 172.21.15.12:0/3477398692 conn(0x7f89d0079bd0 msgr2=0x7f89d0100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:01.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.295+0000 7f89d637e700 1 -- 172.21.15.12:0/3477398692 shutdown_connections 2024-05-17T22:38:01.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:01.295+0000 7f89d637e700 1 -- 172.21.15.12:0/3477398692 wait complete. 2024-05-17T22:38:01.946 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T22:38:02.202 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:38:02.242 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:01 smithi151 ceph-mon[36101]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T22:38:02.242 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:01 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/3477398692' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:38:02.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:01 smithi096 ceph-mon[36065]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T22:38:02.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:01 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/3477398692' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:38:02.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:01 smithi012 ceph-mon[31632]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T22:38:02.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:01 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/3477398692' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:38:03.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:03.895+0000 7fd0290c8700 1 -- 172.21.15.12:0/565697896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd024103f10 msgr2=0x7fd024104320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:03.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:03.895+0000 7fd0290c8700 1 --2- 172.21.15.12:0/565697896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd024103f10 0x7fd024104320 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd014009990 tx=0x7fd014009ca0 comp rx=0 tx=0).stop 2024-05-17T22:38:03.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:03.895+0000 7fd0290c8700 1 -- 172.21.15.12:0/565697896 shutdown_connections 2024-05-17T22:38:03.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:03.895+0000 7fd0290c8700 1 --2- 172.21.15.12:0/565697896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd024109190 0x7fd024110af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:03.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:03.895+0000 7fd0290c8700 1 --2- 172.21.15.12:0/565697896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd024103f10 0x7fd024104320 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:03.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:03.895+0000 7fd0290c8700 1 --2- 172.21.15.12:0/565697896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd024108840 0x7fd024108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:03.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:03.895+0000 7fd0290c8700 1 -- 172.21.15.12:0/565697896 >> 172.21.15.12:0/565697896 conn(0x7fd0240ff990 msgr2=0x7fd024101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:03.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:03.895+0000 7fd0290c8700 1 -- 172.21.15.12:0/565697896 shutdown_connections 2024-05-17T22:38:03.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:03.895+0000 7fd0290c8700 1 -- 172.21.15.12:0/565697896 wait complete. 2024-05-17T22:38:03.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:03.896+0000 7fd0290c8700 1 Processor -- start 2024-05-17T22:38:03.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:03.896+0000 7fd0290c8700 1 -- start start 2024-05-17T22:38:03.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:03.897+0000 7fd0290c8700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd024103f10 0x7fd0241a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:03.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:03.897+0000 7fd0290c8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd024108840 0x7fd0241a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:03.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:03.897+0000 7fd0290c8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd024109190 0x7fd02419a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:03.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:03.897+0000 7fd0290c8700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd024113ec0 con 0x7fd024109190 2024-05-17T22:38:03.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:03.897+0000 7fd02359e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd024109190 0x7fd02419a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:03.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:03.897+0000 7fd0290c8700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd024113d40 con 0x7fd024108840 2024-05-17T22:38:03.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:03.897+0000 7fd0290c8700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd024114040 con 0x7fd024103f10 2024-05-17T22:38:03.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:03.897+0000 7fd02359e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd024109190 0x7fd02419a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57550/0 (socket says 172.21.15.12:57550) 2024-05-17T22:38:03.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:03.897+0000 7fd02359e700 1 -- 172.21.15.12:0/3613989814 learned_addr learned my addr 172.21.15.12:0/3613989814 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:03.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:03.897+0000 7fd022d9d700 1 --2- 172.21.15.12:0/3613989814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd024103f10 0x7fd0241a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:03.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:03.897+0000 7fd02259c700 1 --2- 172.21.15.12:0/3613989814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd024108840 0x7fd0241a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:03.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:03.897+0000 7fd02359e700 1 -- 172.21.15.12:0/3613989814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd024103f10 msgr2=0x7fd0241a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:03.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:03.897+0000 7fd02359e700 1 --2- 172.21.15.12:0/3613989814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd024103f10 0x7fd0241a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:03.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:03.897+0000 7fd02359e700 1 -- 172.21.15.12:0/3613989814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd024108840 msgr2=0x7fd0241a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:03.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:03.897+0000 7fd02359e700 1 --2- 172.21.15.12:0/3613989814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd024108840 0x7fd0241a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:03.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:03.897+0000 7fd02359e700 1 -- 172.21.15.12:0/3613989814 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd02419aec0 con 0x7fd024109190 2024-05-17T22:38:03.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:03.897+0000 7fd02259c700 1 --2- 172.21.15.12:0/3613989814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd024108840 0x7fd0241a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:38:03.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:03.898+0000 7fd02359e700 1 --2- 172.21.15.12:0/3613989814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd024109190 0x7fd02419a790 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd01c00b970 tx=0x7fd01c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:03.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:03.898+0000 7fd00bfff700 1 -- 172.21.15.12:0/3613989814 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01c019070 con 0x7fd024109190 2024-05-17T22:38:03.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:03.898+0000 7fd00bfff700 1 -- 172.21.15.12:0/3613989814 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd01c00cde0 con 0x7fd024109190 2024-05-17T22:38:03.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:03.898+0000 7fd0290c8700 1 -- 172.21.15.12:0/3613989814 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd02419b1b0 con 0x7fd024109190 2024-05-17T22:38:03.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:03.898+0000 7fd00bfff700 1 -- 172.21.15.12:0/3613989814 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01c007370 con 0x7fd024109190 2024-05-17T22:38:03.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:03.898+0000 7fd0290c8700 1 -- 172.21.15.12:0/3613989814 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0241a73b0 con 0x7fd024109190 2024-05-17T22:38:03.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:03.899+0000 7fd0290c8700 1 -- 172.21.15.12:0/3613989814 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0241053c0 con 0x7fd024109190 2024-05-17T22:38:03.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:03.901+0000 7fd00bfff700 1 -- 172.21.15.12:0/3613989814 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd01c01c070 con 0x7fd024109190 2024-05-17T22:38:03.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:03.901+0000 7fd00bfff700 1 --2- 172.21.15.12:0/3613989814 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd00c0b2900 0x7fd00c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:03.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:03.902+0000 7fd022d9d700 1 --2- 172.21.15.12:0/3613989814 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd00c0b2900 0x7fd00c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:03.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:03.902+0000 7fd00bfff700 1 -- 172.21.15.12:0/3613989814 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fd01c0e1ad0 con 0x7fd024109190 2024-05-17T22:38:03.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:03.902+0000 7fd022d9d700 1 --2- 172.21.15.12:0/3613989814 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd00c0b2900 0x7fd00c0b4e10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd01400b5c0 tx=0x7fd014005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:03.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:03.905+0000 7fd00bfff700 1 -- 172.21.15.12:0/3613989814 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd01c0a9460 con 0x7fd024109190 2024-05-17T22:38:04.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:04.166+0000 7fd0290c8700 1 -- 172.21.15.12:0/3613989814 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0241043d0 con 0x7fd024109190 2024-05-17T22:38:04.170 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:03 smithi012 ceph-mon[31632]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T22:38:04.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:04.168+0000 7fd00bfff700 1 -- 172.21.15.12:0/3613989814 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11870 (secure 0 0 0) 0x7fd01c0c1e70 con 0x7fd024109190 2024-05-17T22:38:04.171 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:38:04.171 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1407219319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1407219319},{"type":"v1","addr":"172.21.15.12:6835","nonce":1407219319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/630977886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":630977886},{"type":"v1","addr":"172.21.15.12:6837","nonce":630977886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3731914742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3731914742},{"type":"v1","addr":"172.21.15.12:6839","nonce":3731914742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/3642142501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3642142501},{"type":"v1","addr":"172.21.15.12:6841","nonce":3642142501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3554053059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3554053059},{"type":"v1","addr":"172.21.15.151:6833","nonce":3554053059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/4228325569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":4228325569},{"type":"v1","addr":"172.21.15.96:6833","nonce":4228325569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T22:34:42.642111+0000","modified":"2024-05-17T22:35:52.999468+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24461,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/542323754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":542323754},{"type":"v1","addr":"172.21.15.151:6835","nonce":542323754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1758442232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1758442232},{"type":"v1","addr":"172.21.15.96:6835","nonce":1758442232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3403324038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3403324038},{"type":"v1","addr":"172.21.15.151:6837","nonce":3403324038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.151:6839/2338353849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2338353849},{"type":"v1","addr":"172.21.15.151:6839","nonce":2338353849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.96:6837/4222574338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":4222574338},{"type":"v1","addr":"172.21.15.96:6837","nonce":4222574338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/2986295519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2986295519},{"type":"v1","addr":"172.21.15.96:6839","nonce":2986295519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24457,24482,24461]},"id":1}]} 2024-05-17T22:38:04.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:04.170+0000 7fd0290c8700 1 -- 172.21.15.12:0/3613989814 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd00c0b2900 msgr2=0x7fd00c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:04.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:04.170+0000 7fd0290c8700 1 --2- 172.21.15.12:0/3613989814 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd00c0b2900 0x7fd00c0b4e10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd01400b5c0 tx=0x7fd014005960 comp rx=0 tx=0).stop 2024-05-17T22:38:04.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:04.170+0000 7fd0290c8700 1 -- 172.21.15.12:0/3613989814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd024109190 msgr2=0x7fd02419a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:04.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:04.170+0000 7fd0290c8700 1 --2- 172.21.15.12:0/3613989814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd024109190 0x7fd02419a790 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd01c00b970 tx=0x7fd01c00bc80 comp rx=0 tx=0).stop 2024-05-17T22:38:04.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:04.170+0000 7fd0290c8700 1 -- 172.21.15.12:0/3613989814 shutdown_connections 2024-05-17T22:38:04.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:04.170+0000 7fd0290c8700 1 --2- 172.21.15.12:0/3613989814 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd00c0b2900 0x7fd00c0b4e10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:04.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:04.170+0000 7fd0290c8700 1 --2- 172.21.15.12:0/3613989814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd024109190 0x7fd02419a790 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:04.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:04.170+0000 7fd0290c8700 1 --2- 172.21.15.12:0/3613989814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd024103f10 0x7fd0241a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:04.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:04.170+0000 7fd0290c8700 1 --2- 172.21.15.12:0/3613989814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd024108840 0x7fd0241a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:04.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:04.170+0000 7fd0290c8700 1 -- 172.21.15.12:0/3613989814 >> 172.21.15.12:0/3613989814 conn(0x7fd0240ff990 msgr2=0x7fd0241028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:04.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:04.170+0000 7fd0290c8700 1 -- 172.21.15.12:0/3613989814 shutdown_connections 2024-05-17T22:38:04.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:04.170+0000 7fd0290c8700 1 -- 172.21.15.12:0/3613989814 wait complete. 2024-05-17T22:38:04.174 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 26 2024-05-17T22:38:04.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:03 smithi151 ceph-mon[36101]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T22:38:04.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:03 smithi096 ceph-mon[36065]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T22:38:04.842 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-17T22:34:42.642111+0000', 'modified': '2024-05-17T22:35:52.999468+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24482, 'mds_1': 24461, 'mds_2': 24457}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24436': {'gid': 24436, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.151:6835/542323754', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6834', 'nonce': 542323754}, {'type': 'v1', 'addr': '172.21.15.151:6835', 'nonce': 542323754}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24437': {'gid': 24437, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.96:6835/1758442232', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6834', 'nonce': 1758442232}, {'type': 'v1', 'addr': '172.21.15.96:6835', 'nonce': 1758442232}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.151:6837/3403324038', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6836', 'nonce': 3403324038}, {'type': 'v1', 'addr': '172.21.15.151:6837', 'nonce': 3403324038}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 24, 'addr': '172.21.15.151:6839/2338353849', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6838', 'nonce': 2338353849}, {'type': 'v1', 'addr': '172.21.15.151:6839', 'nonce': 2338353849}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24461': {'gid': 24461, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 25, 'addr': '172.21.15.96:6837/4222574338', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6836', 'nonce': 4222574338}, {'type': 'v1', 'addr': '172.21.15.96:6837', 'nonce': 4222574338}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.96:6839/2986295519', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6838', 'nonce': 2986295519}, {'type': 'v1', 'addr': '172.21.15.96:6839', 'nonce': 2986295519}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24482, 'qdb_cluster': [24457, 24482, 24461]} 2024-05-17T22:38:04.842 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-17T22:38:04.843 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.i status 2024-05-17T22:38:05.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:04 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/3613989814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:38:05.099 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:38:05.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:04 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/3613989814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:38:05.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:04 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/3613989814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:38:06.189 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:05 smithi012 ceph-mon[31632]: pgmap v404: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T22:38:06.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:05 smithi096 ceph-mon[36065]: pgmap v404: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T22:38:06.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:05 smithi151 ceph-mon[36101]: pgmap v404: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T22:38:06.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.734+0000 7f5bbe2fb700 1 -- 172.21.15.12:0/2815348990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bb8107730 msgr2=0x7f5bb8107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:06.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.734+0000 7f5bbe2fb700 1 --2- 172.21.15.12:0/2815348990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bb8107730 0x7f5bb8107b40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0009990 tx=0x7f5ba0009ca0 comp rx=0 tx=0).stop 2024-05-17T22:38:06.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.734+0000 7f5bbe2fb700 1 -- 172.21.15.12:0/2815348990 shutdown_connections 2024-05-17T22:38:06.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.734+0000 7f5bbe2fb700 1 --2- 172.21.15.12:0/2815348990 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bb8108080 0x7f5bb810f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:06.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.734+0000 7f5bbe2fb700 1 --2- 172.21.15.12:0/2815348990 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bb8102e00 0x7f5bb8103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:06.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.734+0000 7f5bbe2fb700 1 --2- 172.21.15.12:0/2815348990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bb8107730 0x7f5bb8107b40 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:06.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.734+0000 7f5bbe2fb700 1 -- 172.21.15.12:0/2815348990 >> 172.21.15.12:0/2815348990 conn(0x7f5bb8079bd0 msgr2=0x7f5bb8079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:06.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.734+0000 7f5bbe2fb700 1 -- 172.21.15.12:0/2815348990 shutdown_connections 2024-05-17T22:38:06.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.734+0000 7f5bbe2fb700 1 -- 172.21.15.12:0/2815348990 wait complete. 2024-05-17T22:38:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.735+0000 7f5bbe2fb700 1 Processor -- start 2024-05-17T22:38:06.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.735+0000 7f5bbe2fb700 1 -- start start 2024-05-17T22:38:06.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.735+0000 7f5bbe2fb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bb8102e00 0x7f5bb819b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:06.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.735+0000 7f5bbe2fb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bb8107730 0x7f5bb819bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:06.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.735+0000 7f5bbe2fb700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bb8108080 0x7f5bb81a66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:06.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.735+0000 7f5bbe2fb700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb8112cb0 con 0x7f5bb8102e00 2024-05-17T22:38:06.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.735+0000 7f5bbe2fb700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb8112b30 con 0x7f5bb8107730 2024-05-17T22:38:06.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.735+0000 7f5bbe2fb700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb8112e30 con 0x7f5bb8108080 2024-05-17T22:38:06.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.735+0000 7f5bb7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bb8102e00 0x7f5bb819b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:06.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.735+0000 7f5bb7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bb8102e00 0x7f5bb819b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57566/0 (socket says 172.21.15.12:57566) 2024-05-17T22:38:06.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.735+0000 7f5bb7fff700 1 -- 172.21.15.12:0/2462690109 learned_addr learned my addr 172.21.15.12:0/2462690109 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:06.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.735+0000 7f5bbc898700 1 --2- 172.21.15.12:0/2462690109 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bb8108080 0x7f5bb81a66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:06.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.736+0000 7f5baffff700 1 --2- 172.21.15.12:0/2462690109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bb8107730 0x7f5bb819bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:06.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.736+0000 7f5bb7fff700 1 -- 172.21.15.12:0/2462690109 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bb8108080 msgr2=0x7f5bb81a66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:06.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.736+0000 7f5bb7fff700 1 --2- 172.21.15.12:0/2462690109 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bb8108080 0x7f5bb81a66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:06.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.736+0000 7f5bb7fff700 1 -- 172.21.15.12:0/2462690109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bb8107730 msgr2=0x7f5bb819bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:06.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.736+0000 7f5bb7fff700 1 --2- 172.21.15.12:0/2462690109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bb8107730 0x7f5bb819bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:06.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.736+0000 7f5bb7fff700 1 -- 172.21.15.12:0/2462690109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb819c410 con 0x7f5bb8102e00 2024-05-17T22:38:06.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.736+0000 7f5bb7fff700 1 --2- 172.21.15.12:0/2462690109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bb8102e00 0x7f5bb819b640 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5ba800d950 tx=0x7f5ba800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:06.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.736+0000 7f5bb5ffb700 1 -- 172.21.15.12:0/2462690109 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba8016070 con 0x7f5bb8102e00 2024-05-17T22:38:06.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.736+0000 7f5bbe2fb700 1 -- 172.21.15.12:0/2462690109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bb81a6cc0 con 0x7f5bb8102e00 2024-05-17T22:38:06.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.736+0000 7f5bb5ffb700 1 -- 172.21.15.12:0/2462690109 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ba800fd40 con 0x7f5bb8102e00 2024-05-17T22:38:06.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.736+0000 7f5bb5ffb700 1 -- 172.21.15.12:0/2462690109 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba800e3b0 con 0x7f5bb8102e00 2024-05-17T22:38:06.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.736+0000 7f5bbe2fb700 1 -- 172.21.15.12:0/2462690109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bb81a7170 con 0x7f5bb8102e00 2024-05-17T22:38:06.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.739+0000 7f5bb5ffb700 1 -- 172.21.15.12:0/2462690109 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5ba8019070 con 0x7f5bb8102e00 2024-05-17T22:38:06.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.740+0000 7f5bb5ffb700 1 --2- 172.21.15.12:0/2462690109 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5ba40b2900 0x7f5ba40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:06.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.740+0000 7f5baffff700 1 --2- 172.21.15.12:0/2462690109 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5ba40b2900 0x7f5ba40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:06.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.740+0000 7f5bb5ffb700 1 -- 172.21.15.12:0/2462690109 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f5ba80e2010 con 0x7f5bb8102e00 2024-05-17T22:38:06.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.740+0000 7f5bcccd6b80 1 Processor -- start 2024-05-17T22:38:06.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.741+0000 7f5baffff700 1 --2- 172.21.15.12:0/2462690109 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5ba40b2900 0x7f5ba40b4e10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0006010 tx=0x7f5ba0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:06.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.741+0000 7f5bcccd6b80 1 -- start start 2024-05-17T22:38:06.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.741+0000 7f5bcccd6b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f1b839a0 0x55e9f1d07510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:06.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.741+0000 7f5bcccd6b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9f1d07a50 0x55e9f1d01be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:06.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.741+0000 7f5bcccd6b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9f1d02120 0x55e9f1d02530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:06.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.741+0000 7f5bcccd6b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e9f1c30240 con 0x55e9f1b839a0 2024-05-17T22:38:06.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.741+0000 7f5bcccd6b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e9f1c1ff80 con 0x55e9f1d07a50 2024-05-17T22:38:06.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.741+0000 7f5bcccd6b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e9f1c23e20 con 0x55e9f1d02120 2024-05-17T22:38:06.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.741+0000 7f5bb7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f1b839a0 0x55e9f1d07510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:06.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.741+0000 7f5bb7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f1b839a0 0x55e9f1d07510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57576/0 (socket says 172.21.15.12:57576) 2024-05-17T22:38:06.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.741+0000 7f5bb7fff700 1 -- 172.21.15.12:0/336721704 learned_addr learned my addr 172.21.15.12:0/336721704 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:06.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.741+0000 7f5baffff700 1 --2- 172.21.15.12:0/336721704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9f1d07a50 0x55e9f1d01be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:06.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.741+0000 7f5bbc898700 1 --2- 172.21.15.12:0/336721704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9f1d02120 0x55e9f1d02530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:06.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.741+0000 7f5bb7fff700 1 -- 172.21.15.12:0/336721704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9f1d02120 msgr2=0x55e9f1d02530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:06.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.741+0000 7f5bb7fff700 1 --2- 172.21.15.12:0/336721704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9f1d02120 0x55e9f1d02530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:06.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.741+0000 7f5bb7fff700 1 -- 172.21.15.12:0/336721704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9f1d07a50 msgr2=0x55e9f1d01be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:06.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.741+0000 7f5bb7fff700 1 --2- 172.21.15.12:0/336721704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9f1d07a50 0x55e9f1d01be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:06.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.741+0000 7f5bb7fff700 1 -- 172.21.15.12:0/336721704 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9f1bc9540 con 0x55e9f1b839a0 2024-05-17T22:38:06.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.741+0000 7f5bbc898700 1 --2- 172.21.15.12:0/336721704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9f1d02120 0x55e9f1d02530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:38:06.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.741+0000 7f5baffff700 1 --2- 172.21.15.12:0/336721704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9f1d07a50 0x55e9f1d01be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:38:06.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.742+0000 7f5bb7fff700 1 --2- 172.21.15.12:0/336721704 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f1b839a0 0x55e9f1d07510 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5ba80a8fa0 tx=0x7f5ba8010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa2f7b851df2d7db server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:06.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.742+0000 7f5bae7fc700 1 -- 172.21.15.12:0/336721704 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba8020040 con 0x55e9f1b839a0 2024-05-17T22:38:06.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.742+0000 7f5bae7fc700 1 -- 172.21.15.12:0/336721704 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ba80ad650 con 0x55e9f1b839a0 2024-05-17T22:38:06.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.742+0000 7f5bae7fc700 1 -- 172.21.15.12:0/336721704 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba80b1ed0 con 0x55e9f1b839a0 2024-05-17T22:38:06.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.743+0000 7f5bcccd6b80 1 -- 172.21.15.12:0/336721704 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f1b839a0 msgr2=0x55e9f1d07510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:06.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.743+0000 7f5bcccd6b80 1 --2- 172.21.15.12:0/336721704 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f1b839a0 0x55e9f1d07510 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5ba80a8fa0 tx=0x7f5ba8010000 comp rx=0 tx=0).stop 2024-05-17T22:38:06.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.743+0000 7f5bcccd6b80 1 -- 172.21.15.12:0/336721704 shutdown_connections 2024-05-17T22:38:06.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.743+0000 7f5bcccd6b80 1 --2- 172.21.15.12:0/336721704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9f1d02120 0x55e9f1d02530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:06.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.743+0000 7f5bcccd6b80 1 --2- 172.21.15.12:0/336721704 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f1b839a0 0x55e9f1d07510 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:06.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.743+0000 7f5bcccd6b80 1 --2- 172.21.15.12:0/336721704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9f1d07a50 0x55e9f1d01be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:06.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.743+0000 7f5bcccd6b80 1 -- 172.21.15.12:0/336721704 >> 172.21.15.12:0/336721704 conn(0x55e9f1bc7eb0 msgr2=0x55e9f1cfcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:06.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.743+0000 7f5bcccd6b80 1 -- 172.21.15.12:0/336721704 shutdown_connections 2024-05-17T22:38:06.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.743+0000 7f5bcccd6b80 1 -- 172.21.15.12:0/336721704 wait complete. 2024-05-17T22:38:06.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.743+0000 7f5bcccd6b80 1 -- start start 2024-05-17T22:38:06.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.744+0000 7f5bcccd6b80 1 Processor -- start 2024-05-17T22:38:06.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.744+0000 7f5bcccd6b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9f1b839a0 0x55e9f1d0b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:06.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.744+0000 7f5bcccd6b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f1d02120 0x55e9f1d0b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:06.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.744+0000 7f5bcccd6b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9f1d07a50 0x55e9f1d0bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:06.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.744+0000 7f5bcccd6b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e9f1b27100 con 0x55e9f1d02120 2024-05-17T22:38:06.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.744+0000 7f5bcccd6b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e9f1bc0e30 con 0x55e9f1d07a50 2024-05-17T22:38:06.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.744+0000 7f5bcccd6b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e9f1d0e2e0 con 0x55e9f1b839a0 2024-05-17T22:38:06.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.744+0000 7f5bcccd6b80 10 client.0 _ll_register_callbacks cb 0x55e9f1bbaec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:38:06.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.744+0000 7f5baffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f1d02120 0x55e9f1d0b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:06.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.744+0000 7f5baffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f1d02120 0x55e9f1d0b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57584/0 (socket says 172.21.15.12:57584) 2024-05-17T22:38:06.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.744+0000 7f5baffff700 1 -- 172.21.15.12:0/1901328963 learned_addr learned my addr 172.21.15.12:0/1901328963 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:06.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.744+0000 7f5bb7fff700 1 --2- 172.21.15.12:0/1901328963 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9f1b839a0 0x55e9f1d0b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:06.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.744+0000 7f5bbc898700 1 --2- 172.21.15.12:0/1901328963 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9f1d07a50 0x55e9f1d0bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:06.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.744+0000 7f5baffff700 1 -- 172.21.15.12:0/1901328963 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9f1b839a0 msgr2=0x55e9f1d0b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:06.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.744+0000 7f5baffff700 1 --2- 172.21.15.12:0/1901328963 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9f1b839a0 0x55e9f1d0b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:06.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.744+0000 7f5baffff700 1 -- 172.21.15.12:0/1901328963 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9f1d07a50 msgr2=0x55e9f1d0bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:06.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.744+0000 7f5baffff700 1 --2- 172.21.15.12:0/1901328963 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9f1d07a50 0x55e9f1d0bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:06.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.744+0000 7f5baffff700 1 -- 172.21.15.12:0/1901328963 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9f1b70430 con 0x55e9f1d02120 2024-05-17T22:38:06.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.744+0000 7f5bbc898700 1 --2- 172.21.15.12:0/1901328963 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9f1d07a50 0x55e9f1d0bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:38:06.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.745+0000 7f5bb7fff700 1 --2- 172.21.15.12:0/1901328963 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9f1b839a0 0x55e9f1d0b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:38:06.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.745+0000 7f5baffff700 1 --2- 172.21.15.12:0/1901328963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f1d02120 0x55e9f1d0b720 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5bb806b1a0 tx=0x7f5ba001a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae57bbd94a80a1b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:06.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.745+0000 7f5b977fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:38:06.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.745+0000 7f5b977fe700 1 -- 172.21.15.12:0/1901328963 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba000f660 con 0x55e9f1d02120 2024-05-17T22:38:06.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.745+0000 7f5b977fe700 1 -- 172.21.15.12:0/1901328963 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f5ba000f800 con 0x55e9f1d02120 2024-05-17T22:38:06.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.745+0000 7f5bcccd6b80 1 -- 172.21.15.12:0/1901328963 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ba80ad650 con 0x55e9f1d02120 2024-05-17T22:38:06.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.745+0000 7f5b977fe700 1 -- 172.21.15.12:0/1901328963 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5ba003d050 con 0x55e9f1d02120 2024-05-17T22:38:06.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.745+0000 7f5bcccd6b80 10 client.15117 fetch_fsmap learned FSMap version 26 2024-05-17T22:38:06.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.745+0000 7f5bcccd6b80 1 -- 172.21.15.12:0/1901328963 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e9f1d18d70 con 0x55e9f1d02120 2024-05-17T22:38:06.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.745+0000 7f5b977fe700 1 -- 172.21.15.12:0/1901328963 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ba80ad650 con 0x55e9f1d02120 2024-05-17T22:38:06.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.745+0000 7f5b977fe700 1 -- 172.21.15.12:0/1901328963 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f5ba0022070 con 0x55e9f1d02120 2024-05-17T22:38:06.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.746+0000 7f5bcccd6b80 10 client.15117 fetch_fsmap finished waiting for FSMap version 26 2024-05-17T22:38:06.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.746+0000 7f5bcccd6b80 10 client.15117 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-17T22:38:06.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.746+0000 7f5bcccd6b80 20 client.15117 populate_metadata read hostname 'smithi012' 2024-05-17T22:38:06.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.746+0000 7f5bcccd6b80 1 --2- 172.21.15.12:0/1901328963 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55e9f1d1b440 0x55e9f1d1c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:06.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.746+0000 7f5bcccd6b80 4 client.15117 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:38:06.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.746+0000 7f5bcccd6b80 1 -- 172.21.15.12:0/1901328963 --> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9f1c32190 con 0x55e9f1d1b440 2024-05-17T22:38:06.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.746+0000 7f5bb7fff700 1 --2- 172.21.15.12:0/1901328963 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55e9f1d1b440 0x55e9f1d1c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:06.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.747+0000 7f5bb7fff700 1 --2- 172.21.15.12:0/1901328963 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55e9f1d1b440 0x55e9f1d1c8b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a6a8bb3204936a98 server_cookie=947f297cfeff58ff in_seq=0 out_seq=0 2024-05-17T22:38:06.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.747+0000 7f5b977fe700 10 client.15117 ms_handle_connect on v2:172.21.15.151:6838/2338353849 2024-05-17T22:38:06.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.747+0000 7f5b977fe700 1 -- 172.21.15.12:0/1901328963 <== mds.2 v2:172.21.15.151:6838/2338353849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e9f1c32190 con 0x55e9f1d1b440 2024-05-17T22:38:06.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.747+0000 7f5b977fe700 10 client.15117 handle_command_reply: tid=0 2024-05-17T22:38:06.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.747+0000 7f5bcccd6b80 1 client.15117 shutdown 2024-05-17T22:38:06.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.748+0000 7f5bcccd6b80 1 -- 172.21.15.12:0/1901328963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f1d02120 msgr2=0x55e9f1d0b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:06.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.748+0000 7f5bcccd6b80 1 --2- 172.21.15.12:0/1901328963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f1d02120 0x55e9f1d0b720 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5bb806b1a0 tx=0x7f5ba001a050 comp rx=0 tx=0).stop 2024-05-17T22:38:06.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.748+0000 7f5bcccd6b80 1 -- 172.21.15.12:0/1901328963 shutdown_connections 2024-05-17T22:38:06.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.748+0000 7f5bcccd6b80 1 --2- 172.21.15.12:0/1901328963 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9f1d07a50 0x55e9f1d0bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:06.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.748+0000 7f5bcccd6b80 1 --2- 172.21.15.12:0/1901328963 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55e9f1d1b440 0x55e9f1d1c8b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:06.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.748+0000 7f5bcccd6b80 1 --2- 172.21.15.12:0/1901328963 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9f1b839a0 0x55e9f1d0b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:06.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.748+0000 7f5bcccd6b80 1 --2- 172.21.15.12:0/1901328963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f1d02120 0x55e9f1d0b720 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:06.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.748+0000 7f5bcccd6b80 1 -- 172.21.15.12:0/1901328963 >> 172.21.15.12:0/1901328963 conn(0x55e9f1bc7eb0 msgr2=0x55e9f1cf6e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:06.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.748+0000 7f5b977fe700 0 client.15117 ms_handle_reset on v2:172.21.15.151:6838/2338353849 2024-05-17T22:38:06.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.748+0000 7f5bcccd6b80 1 -- 172.21.15.12:0/1901328963 shutdown_connections 2024-05-17T22:38:06.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.748+0000 7f5bcccd6b80 1 -- 172.21.15.12:0/1901328963 wait complete. 2024-05-17T22:38:06.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.748+0000 7f5bcccd6b80 20 client.15117 trim_cache size 0 max 16384 2024-05-17T22:38:06.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.769+0000 7f5bcccd6b80 1 Processor -- start 2024-05-17T22:38:06.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.769+0000 7f5bcccd6b80 1 -- start start 2024-05-17T22:38:06.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.770+0000 7f5bcccd6b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9f1b839a0 0x55e9f1d01be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:06.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.770+0000 7f5bcccd6b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9f1d02120 0x55e9f1cf6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:06.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.770+0000 7f5bcccd6b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f1d07a50 0x55e9f1d0bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:06.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.770+0000 7f5bcccd6b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e9f1d1abf0 con 0x55e9f1d07a50 2024-05-17T22:38:06.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.770+0000 7f5bcccd6b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e9f1d1b2c0 con 0x55e9f1b839a0 2024-05-17T22:38:06.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.770+0000 7f5bcccd6b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e9f1d1b990 con 0x55e9f1d02120 2024-05-17T22:38:06.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.770+0000 7f5bbc898700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f1d07a50 0x55e9f1d0bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:06.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.770+0000 7f5bbc898700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f1d07a50 0x55e9f1d0bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57596/0 (socket says 172.21.15.12:57596) 2024-05-17T22:38:06.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.770+0000 7f5bbc898700 1 -- 172.21.15.12:0/2379173303 learned_addr learned my addr 172.21.15.12:0/2379173303 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:06.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.770+0000 7f5bb7fff700 1 --2- 172.21.15.12:0/2379173303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9f1b839a0 0x55e9f1d01be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:06.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.770+0000 7f5baffff700 1 --2- 172.21.15.12:0/2379173303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9f1d02120 0x55e9f1cf6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:06.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.770+0000 7f5bbc898700 1 -- 172.21.15.12:0/2379173303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9f1d02120 msgr2=0x55e9f1cf6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:06.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.770+0000 7f5bbc898700 1 --2- 172.21.15.12:0/2379173303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9f1d02120 0x55e9f1cf6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:06.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.770+0000 7f5bbc898700 1 -- 172.21.15.12:0/2379173303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9f1b839a0 msgr2=0x55e9f1d01be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:06.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.770+0000 7f5bbc898700 1 --2- 172.21.15.12:0/2379173303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9f1b839a0 0x55e9f1d01be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:06.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.770+0000 7f5bbc898700 1 -- 172.21.15.12:0/2379173303 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9f1cf8ee0 con 0x55e9f1d07a50 2024-05-17T22:38:06.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.770+0000 7f5bbc898700 1 --2- 172.21.15.12:0/2379173303 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f1d07a50 0x55e9f1d0bab0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0003f50 tx=0x7f5bb0009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa72fb2a1bc5d42b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:06.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.771+0000 7f5badffb700 1 -- 172.21.15.12:0/2379173303 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb001a020 con 0x55e9f1d07a50 2024-05-17T22:38:06.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.771+0000 7f5badffb700 1 -- 172.21.15.12:0/2379173303 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bb00046d0 con 0x55e9f1d07a50 2024-05-17T22:38:06.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.771+0000 7f5badffb700 1 -- 172.21.15.12:0/2379173303 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0016750 con 0x55e9f1d07a50 2024-05-17T22:38:06.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.771+0000 7f5bcccd6b80 1 -- 172.21.15.12:0/2379173303 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f1d07a50 msgr2=0x55e9f1d0bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:06.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.771+0000 7f5bcccd6b80 1 --2- 172.21.15.12:0/2379173303 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f1d07a50 0x55e9f1d0bab0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0003f50 tx=0x7f5bb0009ba0 comp rx=0 tx=0).stop 2024-05-17T22:38:06.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.771+0000 7f5bcccd6b80 1 -- 172.21.15.12:0/2379173303 shutdown_connections 2024-05-17T22:38:06.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.771+0000 7f5bcccd6b80 1 --2- 172.21.15.12:0/2379173303 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f1d07a50 0x55e9f1d0bab0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:06.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.771+0000 7f5bcccd6b80 1 --2- 172.21.15.12:0/2379173303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9f1b839a0 0x55e9f1d01be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:06.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.771+0000 7f5bcccd6b80 1 --2- 172.21.15.12:0/2379173303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9f1d02120 0x55e9f1cf6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:06.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.771+0000 7f5bcccd6b80 1 -- 172.21.15.12:0/2379173303 >> 172.21.15.12:0/2379173303 conn(0x55e9f1bc7eb0 msgr2=0x55e9f1d09ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:06.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.772+0000 7f5bcccd6b80 1 -- 172.21.15.12:0/2379173303 shutdown_connections 2024-05-17T22:38:06.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.772+0000 7f5bcccd6b80 1 -- 172.21.15.12:0/2379173303 wait complete. 2024-05-17T22:38:06.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.772+0000 7f5bcccd6b80 1 -- start start 2024-05-17T22:38:06.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.772+0000 7f5bcccd6b80 1 Processor -- start 2024-05-17T22:38:06.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.772+0000 7f5bcccd6b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f1b839a0 0x55e9f1d0b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:06.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.772+0000 7f5bcccd6b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9f1d02120 0x55e9f1d0b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:06.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.772+0000 7f5bcccd6b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9f1d07a50 0x55e9f1d0bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:06.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.772+0000 7f5bcccd6b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e9f1d1c730 con 0x55e9f1b839a0 2024-05-17T22:38:06.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.772+0000 7f5bcccd6b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e9f1d1c060 con 0x55e9f1d02120 2024-05-17T22:38:06.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.772+0000 7f5bcccd6b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e9f1d1ce00 con 0x55e9f1d07a50 2024-05-17T22:38:06.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.773+0000 7f5bcccd6b80 10 client.0 _ll_register_callbacks cb 0x55e9f1bbaec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:38:06.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.773+0000 7f5bb7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f1b839a0 0x55e9f1d0b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:06.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.773+0000 7f5bb7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f1b839a0 0x55e9f1d0b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57612/0 (socket says 172.21.15.12:57612) 2024-05-17T22:38:06.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.773+0000 7f5bb7fff700 1 -- 172.21.15.12:0/1320881108 learned_addr learned my addr 172.21.15.12:0/1320881108 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:06.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.773+0000 7f5baffff700 1 --2- 172.21.15.12:0/1320881108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9f1d02120 0x55e9f1d0b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:06.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.773+0000 7f5bbc898700 1 --2- 172.21.15.12:0/1320881108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9f1d07a50 0x55e9f1d0bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:06.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.773+0000 7f5bb7fff700 1 -- 172.21.15.12:0/1320881108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9f1d07a50 msgr2=0x55e9f1d0bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:06.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.773+0000 7f5bb7fff700 1 --2- 172.21.15.12:0/1320881108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9f1d07a50 0x55e9f1d0bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:06.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.773+0000 7f5bb7fff700 1 -- 172.21.15.12:0/1320881108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9f1d02120 msgr2=0x55e9f1d0b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:06.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.773+0000 7f5bb7fff700 1 --2- 172.21.15.12:0/1320881108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9f1d02120 0x55e9f1d0b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:06.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.773+0000 7f5bb7fff700 1 -- 172.21.15.12:0/1320881108 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9f1d07080 con 0x55e9f1b839a0 2024-05-17T22:38:06.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.773+0000 7f5bb7fff700 1 --2- 172.21.15.12:0/1320881108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f1b839a0 0x55e9f1d0b160 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5ba80a9230 tx=0x7f5ba80a8ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=130013e80835ce34 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:06.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.773+0000 7f5b977fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:38:06.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.773+0000 7f5b977fe700 1 -- 172.21.15.12:0/1320881108 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba8020040 con 0x55e9f1b839a0 2024-05-17T22:38:06.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.773+0000 7f5b977fe700 1 -- 172.21.15.12:0/1320881108 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f5ba80a9500 con 0x55e9f1b839a0 2024-05-17T22:38:06.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.773+0000 7f5bcccd6b80 1 -- 172.21.15.12:0/1320881108 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bb00046d0 con 0x55e9f1b839a0 2024-05-17T22:38:06.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.773+0000 7f5b977fe700 1 -- 172.21.15.12:0/1320881108 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5ba80ad650 con 0x55e9f1b839a0 2024-05-17T22:38:06.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.774+0000 7f5bcccd6b80 10 client.15123 fetch_fsmap learned FSMap version 26 2024-05-17T22:38:06.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.774+0000 7f5bcccd6b80 1 -- 172.21.15.12:0/1320881108 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e9f1d869a0 con 0x55e9f1b839a0 2024-05-17T22:38:06.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.774+0000 7f5b977fe700 1 -- 172.21.15.12:0/1320881108 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bb00046d0 con 0x55e9f1b839a0 2024-05-17T22:38:06.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.774+0000 7f5b977fe700 1 -- 172.21.15.12:0/1320881108 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f5ba80c3b90 con 0x55e9f1b839a0 2024-05-17T22:38:06.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.774+0000 7f5bcccd6b80 10 client.15123 fetch_fsmap finished waiting for FSMap version 26 2024-05-17T22:38:06.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.774+0000 7f5bcccd6b80 10 client.15123 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-17T22:38:06.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.774+0000 7f5bcccd6b80 20 client.15123 populate_metadata read hostname 'smithi012' 2024-05-17T22:38:06.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.774+0000 7f5bcccd6b80 1 --2- 172.21.15.12:0/1320881108 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55e9f1d88900 0x55e9f1d89d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:06.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.774+0000 7f5bcccd6b80 4 client.15123 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-17T22:38:06.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.774+0000 7f5bcccd6b80 1 -- 172.21.15.12:0/1320881108 --> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] -- command(tid 0: {"prefix": "status"}) -- 0x55e9f1c2b970 con 0x55e9f1d88900 2024-05-17T22:38:06.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.774+0000 7f5baffff700 1 --2- 172.21.15.12:0/1320881108 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55e9f1d88900 0x55e9f1d89d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:06.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.775+0000 7f5b977fe700 10 client.15123 ms_handle_connect on v2:172.21.15.151:6838/2338353849 2024-05-17T22:38:06.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.775+0000 7f5baffff700 1 --2- 172.21.15.12:0/1320881108 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55e9f1d88900 0x55e9f1d89d70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1f7f40e238fd0aef server_cookie=dabc795aac98162a in_seq=0 out_seq=0 2024-05-17T22:38:06.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.775+0000 7f5b977fe700 1 -- 172.21.15.12:0/1320881108 <== mds.2 v2:172.21.15.151:6838/2338353849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55e9f1c2b970 con 0x55e9f1d88900 2024-05-17T22:38:06.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.775+0000 7f5b977fe700 10 client.15123 handle_command_reply: tid=0 2024-05-17T22:38:06.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.775+0000 7f5bcccd6b80 1 client.15123 shutdown 2024-05-17T22:38:06.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.775+0000 7f5bcccd6b80 1 -- 172.21.15.12:0/1320881108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f1b839a0 msgr2=0x55e9f1d0b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:06.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.775+0000 7f5bcccd6b80 1 --2- 172.21.15.12:0/1320881108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f1b839a0 0x55e9f1d0b160 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5ba80a9230 tx=0x7f5ba80a8ef0 comp rx=0 tx=0).stop 2024-05-17T22:38:06.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.775+0000 7f5bcccd6b80 1 -- 172.21.15.12:0/1320881108 shutdown_connections 2024-05-17T22:38:06.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.775+0000 7f5bcccd6b80 1 --2- 172.21.15.12:0/1320881108 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55e9f1d88900 0x55e9f1d89d70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:06.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.775+0000 7f5bcccd6b80 1 --2- 172.21.15.12:0/1320881108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9f1d07a50 0x55e9f1d0bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:06.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.775+0000 7f5bcccd6b80 1 --2- 172.21.15.12:0/1320881108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f1b839a0 0x55e9f1d0b160 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:06.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.775+0000 7f5b977fe700 0 client.15123 ms_handle_reset on v2:172.21.15.151:6838/2338353849 2024-05-17T22:38:06.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.775+0000 7f5bcccd6b80 1 --2- 172.21.15.12:0/1320881108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9f1d02120 0x55e9f1d0b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:06.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.775+0000 7f5bcccd6b80 1 -- 172.21.15.12:0/1320881108 >> 172.21.15.12:0/1320881108 conn(0x55e9f1bc7eb0 msgr2=0x55e9f1d080f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:06.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.775+0000 7f5bcccd6b80 1 -- 172.21.15.12:0/1320881108 shutdown_connections 2024-05-17T22:38:06.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.775+0000 7f5bcccd6b80 1 -- 172.21.15.12:0/1320881108 wait complete. 2024-05-17T22:38:06.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.776+0000 7f5bcccd6b80 20 client.15123 trim_cache size 0 max 16384 2024-05-17T22:38:06.786 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:38:06.787 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "4d593780-149c-11ef-bc9a-c7b262605968", 2024-05-17T22:38:06.787 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 2, 2024-05-17T22:38:06.787 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24457, 2024-05-17T22:38:06.787 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-05-17T22:38:06.787 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-05-17T22:38:06.787 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-05-17T22:38:06.787 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 138.75834929999999, 2024-05-17T22:38:06.787 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 26, 2024-05-17T22:38:06.788 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 89, 2024-05-17T22:38:06.788 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 87, 2024-05-17T22:38:06.788 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 228.86145816499999 2024-05-17T22:38:06.788 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:38:06.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.776+0000 7f5bbe2fb700 1 -- 172.21.15.12:0/2462690109 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5ba40b2900 msgr2=0x7f5ba40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:06.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.776+0000 7f5bbe2fb700 1 --2- 172.21.15.12:0/2462690109 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5ba40b2900 0x7f5ba40b4e10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0006010 tx=0x7f5ba0005960 comp rx=0 tx=0).stop 2024-05-17T22:38:06.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.776+0000 7f5bbe2fb700 1 -- 172.21.15.12:0/2462690109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bb8102e00 msgr2=0x7f5bb819b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:06.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.776+0000 7f5bbe2fb700 1 --2- 172.21.15.12:0/2462690109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bb8102e00 0x7f5bb819b640 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5ba800d950 tx=0x7f5ba800dc60 comp rx=0 tx=0).stop 2024-05-17T22:38:06.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.776+0000 7f5bbe2fb700 1 -- 172.21.15.12:0/2462690109 shutdown_connections 2024-05-17T22:38:06.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.776+0000 7f5bbe2fb700 1 --2- 172.21.15.12:0/2462690109 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5ba40b2900 0x7f5ba40b4e10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:06.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.776+0000 7f5bbe2fb700 1 --2- 172.21.15.12:0/2462690109 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bb8108080 0x7f5bb81a66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:06.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.776+0000 7f5bbe2fb700 1 --2- 172.21.15.12:0/2462690109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bb8102e00 0x7f5bb819b640 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:06.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.776+0000 7f5bbe2fb700 1 --2- 172.21.15.12:0/2462690109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bb8107730 0x7f5bb819bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:06.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.776+0000 7f5bbe2fb700 1 -- 172.21.15.12:0/2462690109 >> 172.21.15.12:0/2462690109 conn(0x7f5bb8079bd0 msgr2=0x7f5bb8105870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:06.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.776+0000 7f5bbe2fb700 1 -- 172.21.15.12:0/2462690109 shutdown_connections 2024-05-17T22:38:06.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:06.776+0000 7f5bbe2fb700 1 -- 172.21.15.12:0/2462690109 wait complete. 2024-05-17T22:38:07.460 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.h status 2024-05-17T22:38:07.714 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:38:08.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:07 smithi012 ceph-mon[31632]: pgmap v405: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T22:38:08.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:07 smithi096 ceph-mon[36065]: pgmap v405: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T22:38:08.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:07 smithi151 ceph-mon[36101]: pgmap v405: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T22:38:09.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.441+0000 7f765870d700 1 -- 172.21.15.12:0/3200756094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7650104dc0 msgr2=0x7f76501051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:09.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.441+0000 7f765870d700 1 --2- 172.21.15.12:0/3200756094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7650104dc0 0x7f76501051d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f764c009990 tx=0x7f764c009ca0 comp rx=0 tx=0).stop 2024-05-17T22:38:09.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.441+0000 7f765870d700 1 -- 172.21.15.12:0/3200756094 shutdown_connections 2024-05-17T22:38:09.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.441+0000 7f765870d700 1 --2- 172.21.15.12:0/3200756094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f765010e220 0x7f7650110730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:09.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.441+0000 7f765870d700 1 --2- 172.21.15.12:0/3200756094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7650104dc0 0x7f76501051d0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:09.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.441+0000 7f765870d700 1 --2- 172.21.15.12:0/3200756094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7650105710 0x7f765010dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:09.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.441+0000 7f765870d700 1 -- 172.21.15.12:0/3200756094 >> 172.21.15.12:0/3200756094 conn(0x7f7650079af0 msgr2=0x7f7650079f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:09.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.441+0000 7f765870d700 1 -- 172.21.15.12:0/3200756094 shutdown_connections 2024-05-17T22:38:09.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.441+0000 7f765870d700 1 -- 172.21.15.12:0/3200756094 wait complete. 2024-05-17T22:38:09.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.442+0000 7f765870d700 1 Processor -- start 2024-05-17T22:38:09.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.442+0000 7f765870d700 1 -- start start 2024-05-17T22:38:09.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.443+0000 7f765870d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7650104dc0 0x7f765019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:09.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.443+0000 7f765870d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7650105710 0x7f765019ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:09.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.443+0000 7f765870d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f765010e220 0x7f765019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:09.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.443+0000 7f765870d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7650113da0 con 0x7f765010e220 2024-05-17T22:38:09.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.443+0000 7f765870d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7650113c20 con 0x7f7650104dc0 2024-05-17T22:38:09.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.443+0000 7f765870d700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7650113f20 con 0x7f7650105710 2024-05-17T22:38:09.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.443+0000 7f7656caa700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f765010e220 0x7f765019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:09.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.443+0000 7f7656caa700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f765010e220 0x7f765019efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57622/0 (socket says 172.21.15.12:57622) 2024-05-17T22:38:09.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.443+0000 7f7656caa700 1 -- 172.21.15.12:0/2926864560 learned_addr learned my addr 172.21.15.12:0/2926864560 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:09.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.443+0000 7f76564a9700 1 --2- 172.21.15.12:0/2926864560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7650104dc0 0x7f765019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:09.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.443+0000 7f7655ca8700 1 --2- 172.21.15.12:0/2926864560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7650105710 0x7f765019ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:09.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.443+0000 7f7656caa700 1 -- 172.21.15.12:0/2926864560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7650105710 msgr2=0x7f765019ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:09.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.443+0000 7f7656caa700 1 --2- 172.21.15.12:0/2926864560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7650105710 0x7f765019ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:09.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.443+0000 7f7656caa700 1 -- 172.21.15.12:0/2926864560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7650104dc0 msgr2=0x7f765019a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:09.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.443+0000 7f7656caa700 1 --2- 172.21.15.12:0/2926864560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7650104dc0 0x7f765019a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:09.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.443+0000 7f7656caa700 1 -- 172.21.15.12:0/2926864560 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f765019f650 con 0x7f765010e220 2024-05-17T22:38:09.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.443+0000 7f7655ca8700 1 --2- 172.21.15.12:0/2926864560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7650105710 0x7f765019ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:38:09.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.443+0000 7f76564a9700 1 --2- 172.21.15.12:0/2926864560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7650104dc0 0x7f765019a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:38:09.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.444+0000 7f7656caa700 1 --2- 172.21.15.12:0/2926864560 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f765010e220 0x7f765019efb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7648007ee0 tx=0x7f764800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:09.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.444+0000 7f76477fe700 1 -- 172.21.15.12:0/2926864560 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764800dbd0 con 0x7f765010e220 2024-05-17T22:38:09.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.444+0000 7f76477fe700 1 -- 172.21.15.12:0/2926864560 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f764800dd70 con 0x7f765010e220 2024-05-17T22:38:09.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.444+0000 7f765870d700 1 -- 172.21.15.12:0/2926864560 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f765019f940 con 0x7f765010e220 2024-05-17T22:38:09.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.444+0000 7f76477fe700 1 -- 172.21.15.12:0/2926864560 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7648005990 con 0x7f765010e220 2024-05-17T22:38:09.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.444+0000 7f765870d700 1 -- 172.21.15.12:0/2926864560 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76501a7260 con 0x7f765010e220 2024-05-17T22:38:09.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.446+0000 7f76477fe700 1 -- 172.21.15.12:0/2926864560 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f764801a020 con 0x7f765010e220 2024-05-17T22:38:09.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.447+0000 7f76477fe700 1 --2- 172.21.15.12:0/2926864560 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f763c0b2900 0x7f763c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:09.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.447+0000 7f76564a9700 1 --2- 172.21.15.12:0/2926864560 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f763c0b2900 0x7f763c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:09.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.447+0000 7f76477fe700 1 -- 172.21.15.12:0/2926864560 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f76480e1c70 con 0x7f765010e220 2024-05-17T22:38:09.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.448+0000 7f76564a9700 1 --2- 172.21.15.12:0/2926864560 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f763c0b2900 0x7f763c0b4e10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f764c0098e0 tx=0x7f764c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:09.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.448+0000 7f76670e8b80 1 Processor -- start 2024-05-17T22:38:09.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.448+0000 7f76670e8b80 1 -- start start 2024-05-17T22:38:09.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.448+0000 7f76670e8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55693736f9a0 0x5569374f3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:09.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.448+0000 7f76670e8b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569374f3a50 0x5569374edbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:09.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.449+0000 7f76670e8b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569374ee120 0x5569374ee530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:09.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.449+0000 7f76670e8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55693741c240 con 0x55693736f9a0 2024-05-17T22:38:09.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.449+0000 7f76670e8b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55693740bf80 con 0x5569374ee120 2024-05-17T22:38:09.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.449+0000 7f76670e8b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55693740fe20 con 0x5569374f3a50 2024-05-17T22:38:09.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.449+0000 7f76564a9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55693736f9a0 0x5569374f3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:09.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.449+0000 7f76564a9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55693736f9a0 0x5569374f3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57624/0 (socket says 172.21.15.12:57624) 2024-05-17T22:38:09.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.449+0000 7f76564a9700 1 -- 172.21.15.12:0/2801005502 learned_addr learned my addr 172.21.15.12:0/2801005502 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:09.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.449+0000 7f7656caa700 1 --2- 172.21.15.12:0/2801005502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569374ee120 0x5569374ee530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:09.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.449+0000 7f7655ca8700 1 --2- 172.21.15.12:0/2801005502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569374f3a50 0x5569374edbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:09.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.449+0000 7f76564a9700 1 -- 172.21.15.12:0/2801005502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569374f3a50 msgr2=0x5569374edbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:09.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.449+0000 7f76564a9700 1 --2- 172.21.15.12:0/2801005502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569374f3a50 0x5569374edbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:09.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.449+0000 7f76564a9700 1 -- 172.21.15.12:0/2801005502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569374ee120 msgr2=0x5569374ee530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:09.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.449+0000 7f76564a9700 1 --2- 172.21.15.12:0/2801005502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569374ee120 0x5569374ee530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:09.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.449+0000 7f76564a9700 1 -- 172.21.15.12:0/2801005502 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569373b5540 con 0x55693736f9a0 2024-05-17T22:38:09.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.449+0000 7f76564a9700 1 --2- 172.21.15.12:0/2801005502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55693736f9a0 0x5569374f3510 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f764c0095e0 tx=0x7f764c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee2b8a9da0887077 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:09.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.449+0000 7f7644ff9700 1 -- 172.21.15.12:0/2801005502 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764c005050 con 0x55693736f9a0 2024-05-17T22:38:09.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.450+0000 7f7644ff9700 1 -- 172.21.15.12:0/2801005502 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f764c0056a0 con 0x55693736f9a0 2024-05-17T22:38:09.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.450+0000 7f7644ff9700 1 -- 172.21.15.12:0/2801005502 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764c023400 con 0x55693736f9a0 2024-05-17T22:38:09.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.450+0000 7f76670e8b80 1 -- 172.21.15.12:0/2801005502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55693736f9a0 msgr2=0x5569374f3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:09.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.450+0000 7f76670e8b80 1 --2- 172.21.15.12:0/2801005502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55693736f9a0 0x5569374f3510 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f764c0095e0 tx=0x7f764c01e000 comp rx=0 tx=0).stop 2024-05-17T22:38:09.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.450+0000 7f76670e8b80 1 -- 172.21.15.12:0/2801005502 shutdown_connections 2024-05-17T22:38:09.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.450+0000 7f76670e8b80 1 --2- 172.21.15.12:0/2801005502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569374ee120 0x5569374ee530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:09.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.450+0000 7f76670e8b80 1 --2- 172.21.15.12:0/2801005502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55693736f9a0 0x5569374f3510 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:09.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.450+0000 7f76670e8b80 1 --2- 172.21.15.12:0/2801005502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569374f3a50 0x5569374edbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:09.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.451+0000 7f76670e8b80 1 -- 172.21.15.12:0/2801005502 >> 172.21.15.12:0/2801005502 conn(0x5569373b3eb0 msgr2=0x5569374e8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:09.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.451+0000 7f76670e8b80 1 -- 172.21.15.12:0/2801005502 shutdown_connections 2024-05-17T22:38:09.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.451+0000 7f76670e8b80 1 -- 172.21.15.12:0/2801005502 wait complete. 2024-05-17T22:38:09.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.451+0000 7f76670e8b80 1 -- start start 2024-05-17T22:38:09.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.451+0000 7f76670e8b80 1 Processor -- start 2024-05-17T22:38:09.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.451+0000 7f76670e8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55693736f9a0 0x5569374f71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:09.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.451+0000 7f76670e8b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569374ee120 0x5569374f7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:09.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.451+0000 7f76670e8b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569374f3a50 0x5569374f7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:09.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.451+0000 7f76670e8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x556937313100 con 0x55693736f9a0 2024-05-17T22:38:09.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.451+0000 7f76670e8b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5569373ace30 con 0x5569374ee120 2024-05-17T22:38:09.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.451+0000 7f76670e8b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5569374fa2e0 con 0x5569374f3a50 2024-05-17T22:38:09.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.452+0000 7f76670e8b80 10 client.0 _ll_register_callbacks cb 0x5569373a6ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:38:09.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.452+0000 7f76564a9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55693736f9a0 0x5569374f71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:09.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.452+0000 7f76564a9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55693736f9a0 0x5569374f71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57630/0 (socket says 172.21.15.12:57630) 2024-05-17T22:38:09.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.452+0000 7f76564a9700 1 -- 172.21.15.12:0/1457120306 learned_addr learned my addr 172.21.15.12:0/1457120306 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:09.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.452+0000 7f7655ca8700 1 --2- 172.21.15.12:0/1457120306 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569374ee120 0x5569374f7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:09.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.452+0000 7f7656caa700 1 --2- 172.21.15.12:0/1457120306 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569374f3a50 0x5569374f7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:09.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.452+0000 7f76564a9700 1 -- 172.21.15.12:0/1457120306 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569374f3a50 msgr2=0x5569374f7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:09.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.452+0000 7f76564a9700 1 --2- 172.21.15.12:0/1457120306 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569374f3a50 0x5569374f7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:09.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.452+0000 7f76564a9700 1 -- 172.21.15.12:0/1457120306 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569374ee120 msgr2=0x5569374f7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:09.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.452+0000 7f76564a9700 1 --2- 172.21.15.12:0/1457120306 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569374ee120 0x5569374f7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:09.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.452+0000 7f76564a9700 1 -- 172.21.15.12:0/1457120306 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55693735c430 con 0x55693736f9a0 2024-05-17T22:38:09.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.452+0000 7f7656caa700 1 --2- 172.21.15.12:0/1457120306 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569374f3a50 0x5569374f7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:38:09.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.452+0000 7f76564a9700 1 --2- 172.21.15.12:0/1457120306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55693736f9a0 0x5569374f71f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f764c00a010 tx=0x7f764c0234b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae0b64c7a3dff7e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:09.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.452+0000 7f7639ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:38:09.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.452+0000 7f7639ffb700 1 -- 172.21.15.12:0/1457120306 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764c03b060 con 0x55693736f9a0 2024-05-17T22:38:09.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.452+0000 7f7639ffb700 1 -- 172.21.15.12:0/1457120306 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f764c023b50 con 0x55693736f9a0 2024-05-17T22:38:09.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.452+0000 7f76670e8b80 1 -- 172.21.15.12:0/1457120306 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f764c0056a0 con 0x55693736f9a0 2024-05-17T22:38:09.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.453+0000 7f7639ffb700 1 -- 172.21.15.12:0/1457120306 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f764c042050 con 0x55693736f9a0 2024-05-17T22:38:09.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.453+0000 7f76670e8b80 10 client.15135 fetch_fsmap learned FSMap version 26 2024-05-17T22:38:09.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.453+0000 7f76670e8b80 1 -- 172.21.15.12:0/1457120306 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556937504da0 con 0x55693736f9a0 2024-05-17T22:38:09.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.453+0000 7f7639ffb700 1 -- 172.21.15.12:0/1457120306 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f764c0056a0 con 0x55693736f9a0 2024-05-17T22:38:09.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.453+0000 7f7639ffb700 1 -- 172.21.15.12:0/1457120306 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f764c022070 con 0x55693736f9a0 2024-05-17T22:38:09.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.453+0000 7f76670e8b80 10 client.15135 fetch_fsmap finished waiting for FSMap version 26 2024-05-17T22:38:09.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.453+0000 7f76670e8b80 10 client.15135 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-17T22:38:09.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.453+0000 7f76670e8b80 20 client.15135 populate_metadata read hostname 'smithi012' 2024-05-17T22:38:09.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.453+0000 7f76670e8b80 1 --2- 172.21.15.12:0/1457120306 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x556937506f00 0x556937508370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:09.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.453+0000 7f76670e8b80 4 client.15135 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:38:09.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.453+0000 7f76670e8b80 1 -- 172.21.15.12:0/1457120306 --> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55693741e190 con 0x556937506f00 2024-05-17T22:38:09.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.453+0000 7f7655ca8700 1 --2- 172.21.15.12:0/1457120306 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x556937506f00 0x556937508370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:09.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.454+0000 7f7655ca8700 1 --2- 172.21.15.12:0/1457120306 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x556937506f00 0x556937508370 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bc184db8cecef09b server_cookie=d6b9b9e41bd2d4a6 in_seq=0 out_seq=0 2024-05-17T22:38:09.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.454+0000 7f7639ffb700 10 client.15135 ms_handle_connect on v2:172.21.15.96:6836/4222574338 2024-05-17T22:38:09.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.455+0000 7f7639ffb700 1 -- 172.21.15.12:0/1457120306 <== mds.1 v2:172.21.15.96:6836/4222574338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55693741e190 con 0x556937506f00 2024-05-17T22:38:09.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.455+0000 7f7639ffb700 10 client.15135 handle_command_reply: tid=0 2024-05-17T22:38:09.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.455+0000 7f76670e8b80 1 client.15135 shutdown 2024-05-17T22:38:09.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.455+0000 7f76670e8b80 1 -- 172.21.15.12:0/1457120306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55693736f9a0 msgr2=0x5569374f71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:09.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.455+0000 7f76670e8b80 1 --2- 172.21.15.12:0/1457120306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55693736f9a0 0x5569374f71f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f764c00a010 tx=0x7f764c0234b0 comp rx=0 tx=0).stop 2024-05-17T22:38:09.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.455+0000 7f76670e8b80 1 -- 172.21.15.12:0/1457120306 shutdown_connections 2024-05-17T22:38:09.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.455+0000 7f76670e8b80 1 --2- 172.21.15.12:0/1457120306 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569374f3a50 0x5569374f7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:09.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.455+0000 7f76670e8b80 1 --2- 172.21.15.12:0/1457120306 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x556937506f00 0x556937508370 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:09.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.455+0000 7f76670e8b80 1 --2- 172.21.15.12:0/1457120306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55693736f9a0 0x5569374f71f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:09.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.455+0000 7f76670e8b80 1 --2- 172.21.15.12:0/1457120306 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569374ee120 0x5569374f7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:09.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.455+0000 7f76670e8b80 1 -- 172.21.15.12:0/1457120306 >> 172.21.15.12:0/1457120306 conn(0x5569373b3eb0 msgr2=0x5569374e2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:09.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.455+0000 7f7639ffb700 0 client.15135 ms_handle_reset on v2:172.21.15.96:6836/4222574338 2024-05-17T22:38:09.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.455+0000 7f76670e8b80 1 -- 172.21.15.12:0/1457120306 shutdown_connections 2024-05-17T22:38:09.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.455+0000 7f76670e8b80 1 -- 172.21.15.12:0/1457120306 wait complete. 2024-05-17T22:38:09.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.455+0000 7f76670e8b80 20 client.15135 trim_cache size 0 max 16384 2024-05-17T22:38:09.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.476+0000 7f76670e8b80 1 Processor -- start 2024-05-17T22:38:09.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.476+0000 7f76670e8b80 1 -- start start 2024-05-17T22:38:09.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.476+0000 7f76670e8b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55693736f9a0 0x5569374edbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:09.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.476+0000 7f76670e8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5569374ee120 0x5569374e2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:09.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.476+0000 7f76670e8b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569374f3a50 0x5569374f60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:09.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.476+0000 7f76670e8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5569375066b0 con 0x5569374ee120 2024-05-17T22:38:09.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.477+0000 7f76670e8b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556937506d80 con 0x5569374f3a50 2024-05-17T22:38:09.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.477+0000 7f76670e8b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556937507450 con 0x55693736f9a0 2024-05-17T22:38:09.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.477+0000 7f7655ca8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5569374ee120 0x5569374e2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:09.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.477+0000 7f7655ca8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5569374ee120 0x5569374e2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57644/0 (socket says 172.21.15.12:57644) 2024-05-17T22:38:09.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.477+0000 7f7655ca8700 1 -- 172.21.15.12:0/3020420621 learned_addr learned my addr 172.21.15.12:0/3020420621 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:09.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.477+0000 7f76564a9700 1 --2- 172.21.15.12:0/3020420621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55693736f9a0 0x5569374edbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:09.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.477+0000 7f7655ca8700 1 -- 172.21.15.12:0/3020420621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55693736f9a0 msgr2=0x5569374edbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:09.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.477+0000 7f7655ca8700 1 --2- 172.21.15.12:0/3020420621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55693736f9a0 0x5569374edbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:09.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.477+0000 7f7655ca8700 1 -- 172.21.15.12:0/3020420621 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569374f3a50 msgr2=0x5569374f60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:09.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.477+0000 7f7655ca8700 1 --2- 172.21.15.12:0/3020420621 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569374f3a50 0x5569374f60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:09.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.477+0000 7f7655ca8700 1 -- 172.21.15.12:0/3020420621 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569374f3080 con 0x5569374ee120 2024-05-17T22:38:09.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.477+0000 7f76564a9700 1 --2- 172.21.15.12:0/3020420621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55693736f9a0 0x5569374edbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:38:09.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.477+0000 7f7655ca8700 1 --2- 172.21.15.12:0/3020420621 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5569374ee120 0x5569374e2e60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f76501022e0 tx=0x7f764000c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c29aace5e732c2f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:09.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.477+0000 7f763bfff700 1 -- 172.21.15.12:0/3020420621 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7640013bd0 con 0x5569374ee120 2024-05-17T22:38:09.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.477+0000 7f763bfff700 1 -- 172.21.15.12:0/3020420621 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7640013d70 con 0x5569374ee120 2024-05-17T22:38:09.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.477+0000 7f763bfff700 1 -- 172.21.15.12:0/3020420621 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76400118e0 con 0x5569374ee120 2024-05-17T22:38:09.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.478+0000 7f76670e8b80 1 -- 172.21.15.12:0/3020420621 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5569374ee120 msgr2=0x5569374e2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:09.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.478+0000 7f76670e8b80 1 --2- 172.21.15.12:0/3020420621 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5569374ee120 0x5569374e2e60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f76501022e0 tx=0x7f764000c6e0 comp rx=0 tx=0).stop 2024-05-17T22:38:09.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.478+0000 7f76670e8b80 1 -- 172.21.15.12:0/3020420621 shutdown_connections 2024-05-17T22:38:09.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.478+0000 7f76670e8b80 1 --2- 172.21.15.12:0/3020420621 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569374f3a50 0x5569374f60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:09.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.478+0000 7f76670e8b80 1 --2- 172.21.15.12:0/3020420621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55693736f9a0 0x5569374edbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:09.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.478+0000 7f76670e8b80 1 --2- 172.21.15.12:0/3020420621 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5569374ee120 0x5569374e2e60 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:09.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.478+0000 7f76670e8b80 1 -- 172.21.15.12:0/3020420621 >> 172.21.15.12:0/3020420621 conn(0x5569373b3eb0 msgr2=0x5569374f59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:09.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.478+0000 7f76670e8b80 1 -- 172.21.15.12:0/3020420621 shutdown_connections 2024-05-17T22:38:09.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.478+0000 7f76670e8b80 1 -- 172.21.15.12:0/3020420621 wait complete. 2024-05-17T22:38:09.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.479+0000 7f76670e8b80 1 -- start start 2024-05-17T22:38:09.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.479+0000 7f76670e8b80 1 Processor -- start 2024-05-17T22:38:09.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.479+0000 7f76670e8b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55693736f9a0 0x5569374f6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:09.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.479+0000 7f76670e8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5569374ee120 0x5569374f6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:09.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.479+0000 7f76670e8b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569374f3a50 0x5569375107e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:09.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.479+0000 7f76670e8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5569375081f0 con 0x5569374ee120 2024-05-17T22:38:09.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.479+0000 7f76670e8b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556937507b20 con 0x5569374f3a50 2024-05-17T22:38:09.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.479+0000 7f76670e8b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5569375088c0 con 0x55693736f9a0 2024-05-17T22:38:09.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.479+0000 7f76670e8b80 10 client.0 _ll_register_callbacks cb 0x5569373a6ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:38:09.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.479+0000 7f7655ca8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5569374ee120 0x5569374f6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:09.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.479+0000 7f7655ca8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5569374ee120 0x5569374f6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57656/0 (socket says 172.21.15.12:57656) 2024-05-17T22:38:09.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.479+0000 7f7655ca8700 1 -- 172.21.15.12:0/1600404068 learned_addr learned my addr 172.21.15.12:0/1600404068 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:09.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.479+0000 7f76564a9700 1 --2- 172.21.15.12:0/1600404068 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55693736f9a0 0x5569374f6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:09.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.479+0000 7f7656caa700 1 --2- 172.21.15.12:0/1600404068 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569374f3a50 0x5569375107e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:09.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.479+0000 7f7655ca8700 1 -- 172.21.15.12:0/1600404068 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55693736f9a0 msgr2=0x5569374f6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:09.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.479+0000 7f7655ca8700 1 --2- 172.21.15.12:0/1600404068 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55693736f9a0 0x5569374f6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:09.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.479+0000 7f7655ca8700 1 -- 172.21.15.12:0/1600404068 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569374f3a50 msgr2=0x5569375107e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:09.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.479+0000 7f7655ca8700 1 --2- 172.21.15.12:0/1600404068 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569374f3a50 0x5569375107e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:09.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.480+0000 7f7655ca8700 1 -- 172.21.15.12:0/1600404068 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556937510d20 con 0x5569374ee120 2024-05-17T22:38:09.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.480+0000 7f7656caa700 1 --2- 172.21.15.12:0/1600404068 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569374f3a50 0x5569375107e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:38:09.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.480+0000 7f7655ca8700 1 --2- 172.21.15.12:0/1600404068 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5569374ee120 0x5569374f6ff0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7640020000 tx=0x7f7640011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de3209e0558830cd server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:09.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.480+0000 7f7639ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:38:09.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.480+0000 7f7639ffb700 1 -- 172.21.15.12:0/1600404068 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7640030060 con 0x5569374ee120 2024-05-17T22:38:09.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.480+0000 7f76670e8b80 1 -- 172.21.15.12:0/1600404068 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7640013d70 con 0x5569374ee120 2024-05-17T22:38:09.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.480+0000 7f7639ffb700 1 -- 172.21.15.12:0/1600404068 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f764001a530 con 0x5569374ee120 2024-05-17T22:38:09.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.480+0000 7f7639ffb700 1 -- 172.21.15.12:0/1600404068 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f764001a770 con 0x5569374ee120 2024-05-17T22:38:09.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.480+0000 7f76670e8b80 10 client.15141 fetch_fsmap learned FSMap version 26 2024-05-17T22:38:09.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.480+0000 7f76670e8b80 1 -- 172.21.15.12:0/1600404068 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556937511f10 con 0x5569374ee120 2024-05-17T22:38:09.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.480+0000 7f7639ffb700 1 -- 172.21.15.12:0/1600404068 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7640013d70 con 0x5569374ee120 2024-05-17T22:38:09.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.481+0000 7f7639ffb700 1 -- 172.21.15.12:0/1600404068 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f7640023660 con 0x5569374ee120 2024-05-17T22:38:09.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.481+0000 7f76670e8b80 10 client.15141 fetch_fsmap finished waiting for FSMap version 26 2024-05-17T22:38:09.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.481+0000 7f76670e8b80 10 client.15141 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-17T22:38:09.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.481+0000 7f76670e8b80 20 client.15141 populate_metadata read hostname 'smithi012' 2024-05-17T22:38:09.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.481+0000 7f76670e8b80 1 --2- 172.21.15.12:0/1600404068 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x556937573d50 0x5569375751c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:09.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.481+0000 7f76670e8b80 4 client.15141 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-17T22:38:09.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.481+0000 7f76670e8b80 1 -- 172.21.15.12:0/1600404068 --> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] -- command(tid 0: {"prefix": "status"}) -- 0x556937417970 con 0x556937573d50 2024-05-17T22:38:09.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.481+0000 7f76564a9700 1 --2- 172.21.15.12:0/1600404068 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x556937573d50 0x5569375751c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:09.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.482+0000 7f7639ffb700 10 client.15141 ms_handle_connect on v2:172.21.15.96:6836/4222574338 2024-05-17T22:38:09.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.482+0000 7f76564a9700 1 --2- 172.21.15.12:0/1600404068 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x556937573d50 0x5569375751c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d309bf147d9c2263 server_cookie=74258d6aca3a6150 in_seq=0 out_seq=0 2024-05-17T22:38:09.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.482+0000 7f7639ffb700 1 -- 172.21.15.12:0/1600404068 <== mds.1 v2:172.21.15.96:6836/4222574338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x556937417970 con 0x556937573d50 2024-05-17T22:38:09.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.482+0000 7f7639ffb700 10 client.15141 handle_command_reply: tid=0 2024-05-17T22:38:09.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.482+0000 7f76670e8b80 1 client.15141 shutdown 2024-05-17T22:38:09.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.482+0000 7f76670e8b80 1 -- 172.21.15.12:0/1600404068 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5569374ee120 msgr2=0x5569374f6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:09.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.482+0000 7f76670e8b80 1 --2- 172.21.15.12:0/1600404068 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5569374ee120 0x5569374f6ff0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7640020000 tx=0x7f7640011440 comp rx=0 tx=0).stop 2024-05-17T22:38:09.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.482+0000 7f76670e8b80 1 -- 172.21.15.12:0/1600404068 shutdown_connections 2024-05-17T22:38:09.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.482+0000 7f76670e8b80 1 --2- 172.21.15.12:0/1600404068 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569374f3a50 0x5569375107e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:09.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.482+0000 7f76670e8b80 1 --2- 172.21.15.12:0/1600404068 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55693736f9a0 0x5569374f6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:09.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.482+0000 7f76670e8b80 1 --2- 172.21.15.12:0/1600404068 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x556937573d50 0x5569375751c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:09.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.482+0000 7f76670e8b80 1 --2- 172.21.15.12:0/1600404068 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5569374ee120 0x5569374f6ff0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:09.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.482+0000 7f76670e8b80 1 -- 172.21.15.12:0/1600404068 >> 172.21.15.12:0/1600404068 conn(0x5569373b3eb0 msgr2=0x5569374f59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:09.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.482+0000 7f7639ffb700 0 client.15141 ms_handle_reset on v2:172.21.15.96:6836/4222574338 2024-05-17T22:38:09.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.482+0000 7f76670e8b80 1 -- 172.21.15.12:0/1600404068 shutdown_connections 2024-05-17T22:38:09.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.482+0000 7f76670e8b80 1 -- 172.21.15.12:0/1600404068 wait complete. 2024-05-17T22:38:09.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.482+0000 7f76670e8b80 20 client.15141 trim_cache size 0 max 16384 2024-05-17T22:38:09.496 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:38:09.496 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "4d593780-149c-11ef-bc9a-c7b262605968", 2024-05-17T22:38:09.496 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 1, 2024-05-17T22:38:09.496 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24461, 2024-05-17T22:38:09.496 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-05-17T22:38:09.496 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-05-17T22:38:09.496 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-05-17T22:38:09.496 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 142.474156182, 2024-05-17T22:38:09.497 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 26, 2024-05-17T22:38:09.497 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 89, 2024-05-17T22:38:09.497 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 87, 2024-05-17T22:38:09.497 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 234.56597509700001 2024-05-17T22:38:09.497 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:38:09.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.483+0000 7f765870d700 1 -- 172.21.15.12:0/2926864560 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f763c0b2900 msgr2=0x7f763c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:09.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.483+0000 7f765870d700 1 --2- 172.21.15.12:0/2926864560 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f763c0b2900 0x7f763c0b4e10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f764c0098e0 tx=0x7f764c005960 comp rx=0 tx=0).stop 2024-05-17T22:38:09.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.483+0000 7f765870d700 1 -- 172.21.15.12:0/2926864560 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f765010e220 msgr2=0x7f765019efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:09.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.483+0000 7f765870d700 1 --2- 172.21.15.12:0/2926864560 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f765010e220 0x7f765019efb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7648007ee0 tx=0x7f764800d3b0 comp rx=0 tx=0).stop 2024-05-17T22:38:09.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.483+0000 7f765870d700 1 -- 172.21.15.12:0/2926864560 shutdown_connections 2024-05-17T22:38:09.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.483+0000 7f765870d700 1 --2- 172.21.15.12:0/2926864560 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f763c0b2900 0x7f763c0b4e10 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:09.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.483+0000 7f765870d700 1 --2- 172.21.15.12:0/2926864560 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f765010e220 0x7f765019efb0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:09.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.483+0000 7f765870d700 1 --2- 172.21.15.12:0/2926864560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7650104dc0 0x7f765019a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:09.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.483+0000 7f765870d700 1 --2- 172.21.15.12:0/2926864560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7650105710 0x7f765019ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:09.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.483+0000 7f765870d700 1 -- 172.21.15.12:0/2926864560 >> 172.21.15.12:0/2926864560 conn(0x7f7650079af0 msgr2=0x7f76500ffba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:09.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.483+0000 7f765870d700 1 -- 172.21.15.12:0/2926864560 shutdown_connections 2024-05-17T22:38:09.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:09.483+0000 7f765870d700 1 -- 172.21.15.12:0/2926864560 wait complete. 2024-05-17T22:38:10.160 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.b status 2024-05-17T22:38:10.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:09 smithi096 ceph-mon[36065]: pgmap v406: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:38:10.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:09 smithi012 ceph-mon[31632]: pgmap v406: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:38:10.411 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:38:10.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:09 smithi151 ceph-mon[36101]: pgmap v406: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:38:12.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.038+0000 7f5940ca3700 1 -- 172.21.15.12:0/1233672702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f593c105760 msgr2=0x7f593c10dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:12.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.038+0000 7f5940ca3700 1 --2- 172.21.15.12:0/1233672702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f593c105760 0x7f593c10dcb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5930009990 tx=0x7f5930009ca0 comp rx=0 tx=0).stop 2024-05-17T22:38:12.042 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:11 smithi012 ceph-mon[31632]: pgmap v407: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T22:38:12.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.038+0000 7f5940ca3700 1 -- 172.21.15.12:0/1233672702 shutdown_connections 2024-05-17T22:38:12.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.038+0000 7f5940ca3700 1 --2- 172.21.15.12:0/1233672702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f593c10e240 0x7f593c110750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:12.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.038+0000 7f5940ca3700 1 --2- 172.21.15.12:0/1233672702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f593c104e10 0x7f593c105220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:12.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.038+0000 7f5940ca3700 1 --2- 172.21.15.12:0/1233672702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f593c105760 0x7f593c10dcb0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:12.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.038+0000 7f5940ca3700 1 -- 172.21.15.12:0/1233672702 >> 172.21.15.12:0/1233672702 conn(0x7f593c079ae0 msgr2=0x7f593c079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:12.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.039+0000 7f5940ca3700 1 -- 172.21.15.12:0/1233672702 shutdown_connections 2024-05-17T22:38:12.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.039+0000 7f5940ca3700 1 -- 172.21.15.12:0/1233672702 wait complete. 2024-05-17T22:38:12.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.040+0000 7f5940ca3700 1 Processor -- start 2024-05-17T22:38:12.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.040+0000 7f5940ca3700 1 -- start start 2024-05-17T22:38:12.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.040+0000 7f5940ca3700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f593c104e10 0x7f593c19a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:12.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.040+0000 7f5940ca3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f593c105760 0x7f593c19ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:12.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.040+0000 7f5940ca3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f593c10e240 0x7f593c19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:12.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.040+0000 7f5940ca3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f593c113e40 con 0x7f593c10e240 2024-05-17T22:38:12.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.040+0000 7f5940ca3700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f593c113cc0 con 0x7f593c105760 2024-05-17T22:38:12.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.040+0000 7f5940ca3700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f593c113fc0 con 0x7f593c104e10 2024-05-17T22:38:12.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.040+0000 7f593ad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f593c10e240 0x7f593c19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:12.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.040+0000 7f593ad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f593c10e240 0x7f593c19f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36298/0 (socket says 172.21.15.12:36298) 2024-05-17T22:38:12.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.040+0000 7f593ad9d700 1 -- 172.21.15.12:0/357059248 learned_addr learned my addr 172.21.15.12:0/357059248 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:12.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.040+0000 7f5939d9b700 1 --2- 172.21.15.12:0/357059248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f593c105760 0x7f593c19ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:12.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.040+0000 7f593a59c700 1 --2- 172.21.15.12:0/357059248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f593c104e10 0x7f593c19a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:12.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.041+0000 7f593ad9d700 1 -- 172.21.15.12:0/357059248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f593c104e10 msgr2=0x7f593c19a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:12.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.041+0000 7f593ad9d700 1 --2- 172.21.15.12:0/357059248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f593c104e10 0x7f593c19a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:12.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.041+0000 7f593ad9d700 1 -- 172.21.15.12:0/357059248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f593c105760 msgr2=0x7f593c19ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:12.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.041+0000 7f593ad9d700 1 --2- 172.21.15.12:0/357059248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f593c105760 0x7f593c19ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:12.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.041+0000 7f593ad9d700 1 -- 172.21.15.12:0/357059248 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f593c19f6b0 con 0x7f593c10e240 2024-05-17T22:38:12.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.041+0000 7f593ad9d700 1 --2- 172.21.15.12:0/357059248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f593c10e240 0x7f593c19f010 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f593400b550 tx=0x7f593400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:12.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.041+0000 7f592b7fe700 1 -- 172.21.15.12:0/357059248 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5934015020 con 0x7f593c10e240 2024-05-17T22:38:12.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.041+0000 7f5940ca3700 1 -- 172.21.15.12:0/357059248 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f593c19f9a0 con 0x7f593c10e240 2024-05-17T22:38:12.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.041+0000 7f592b7fe700 1 -- 172.21.15.12:0/357059248 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5934011ce0 con 0x7f593c10e240 2024-05-17T22:38:12.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.041+0000 7f592b7fe700 1 -- 172.21.15.12:0/357059248 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5934010c90 con 0x7f593c10e240 2024-05-17T22:38:12.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.041+0000 7f5940ca3700 1 -- 172.21.15.12:0/357059248 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f593c1a72c0 con 0x7f593c10e240 2024-05-17T22:38:12.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.044+0000 7f592b7fe700 1 -- 172.21.15.12:0/357059248 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5934018020 con 0x7f593c10e240 2024-05-17T22:38:12.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.045+0000 7f592b7fe700 1 --2- 172.21.15.12:0/357059248 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f59240b2880 0x7f59240b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:12.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.045+0000 7f594f67eb80 1 Processor -- start 2024-05-17T22:38:12.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.045+0000 7f593a59c700 1 --2- 172.21.15.12:0/357059248 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f59240b2880 0x7f59240b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:12.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.045+0000 7f592b7fe700 1 -- 172.21.15.12:0/357059248 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f59340e1c30 con 0x7f593c10e240 2024-05-17T22:38:12.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.045+0000 7f594f67eb80 1 -- start start 2024-05-17T22:38:12.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.045+0000 7f594f67eb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5be4d09a0 0x55e5be653e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:12.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.045+0000 7f594f67eb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5be6543b0 0x55e5be64e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:12.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.045+0000 7f594f67eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e5be64eab0 0x55e5be64eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:12.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.045+0000 7f594f67eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e5be57d080 con 0x55e5be64eab0 2024-05-17T22:38:12.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.045+0000 7f594f67eb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e5be56cc20 con 0x55e5be6543b0 2024-05-17T22:38:12.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.045+0000 7f594f67eb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e5be570c60 con 0x55e5be4d09a0 2024-05-17T22:38:12.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.045+0000 7f593a59c700 1 --2- 172.21.15.12:0/357059248 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f59240b2880 0x7f59240b4d90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f592c0060b0 tx=0x7f592c00c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:12.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.046+0000 7f593a59c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5be4d09a0 0x55e5be653e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:12.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.046+0000 7f593ad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e5be64eab0 0x55e5be64eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:12.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.046+0000 7f593a59c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5be4d09a0 0x55e5be653e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:59204/0 (socket says 172.21.15.12:59204) 2024-05-17T22:38:12.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.046+0000 7f593a59c700 1 -- 172.21.15.12:0/1700587040 learned_addr learned my addr 172.21.15.12:0/1700587040 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:12.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.046+0000 7f5939d9b700 1 --2- 172.21.15.12:0/1700587040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5be6543b0 0x55e5be64e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:12.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.046+0000 7f593a59c700 1 -- 172.21.15.12:0/1700587040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5be6543b0 msgr2=0x55e5be64e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:12.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.046+0000 7f593a59c700 1 --2- 172.21.15.12:0/1700587040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5be6543b0 0x55e5be64e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:12.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.046+0000 7f593a59c700 1 -- 172.21.15.12:0/1700587040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e5be64eab0 msgr2=0x55e5be64eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:12.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.046+0000 7f593a59c700 1 --2- 172.21.15.12:0/1700587040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e5be64eab0 0x55e5be64eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:12.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.046+0000 7f593a59c700 1 -- 172.21.15.12:0/1700587040 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5be506570 con 0x55e5be4d09a0 2024-05-17T22:38:12.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.046+0000 7f593ad9d700 1 --2- 172.21.15.12:0/1700587040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e5be64eab0 0x55e5be64eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:38:12.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.046+0000 7f593a59c700 1 --2- 172.21.15.12:0/1700587040 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5be4d09a0 0x55e5be653e70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f592c00a480 tx=0x7f592c012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a36af70cf974f45 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:12.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.046+0000 7f5928ff9700 1 -- 172.21.15.12:0/1700587040 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c026070 con 0x55e5be4d09a0 2024-05-17T22:38:12.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.046+0000 7f5928ff9700 1 -- 172.21.15.12:0/1700587040 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f592c0218b0 con 0x55e5be4d09a0 2024-05-17T22:38:12.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.046+0000 7f5928ff9700 1 -- 172.21.15.12:0/1700587040 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c014860 con 0x55e5be4d09a0 2024-05-17T22:38:12.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.047+0000 7f594f67eb80 1 -- 172.21.15.12:0/1700587040 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5be4d09a0 msgr2=0x55e5be653e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:12.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.047+0000 7f594f67eb80 1 --2- 172.21.15.12:0/1700587040 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5be4d09a0 0x55e5be653e70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f592c00a480 tx=0x7f592c012000 comp rx=0 tx=0).stop 2024-05-17T22:38:12.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.047+0000 7f594f67eb80 1 -- 172.21.15.12:0/1700587040 shutdown_connections 2024-05-17T22:38:12.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.047+0000 7f594f67eb80 1 --2- 172.21.15.12:0/1700587040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e5be64eab0 0x55e5be64eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:12.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.047+0000 7f594f67eb80 1 --2- 172.21.15.12:0/1700587040 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5be4d09a0 0x55e5be653e70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:12.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.047+0000 7f594f67eb80 1 --2- 172.21.15.12:0/1700587040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5be6543b0 0x55e5be64e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:12.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.047+0000 7f594f67eb80 1 -- 172.21.15.12:0/1700587040 >> 172.21.15.12:0/1700587040 conn(0x55e5be530780 msgr2=0x55e5be649170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:12.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.047+0000 7f594f67eb80 1 -- 172.21.15.12:0/1700587040 shutdown_connections 2024-05-17T22:38:12.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.048+0000 7f594f67eb80 1 -- 172.21.15.12:0/1700587040 wait complete. 2024-05-17T22:38:12.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.048+0000 7f594f67eb80 1 -- start start 2024-05-17T22:38:12.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.048+0000 7f594f67eb80 1 Processor -- start 2024-05-17T22:38:12.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.048+0000 7f594f67eb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5be4d09a0 0x55e5be6579a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:12.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.048+0000 7f594f67eb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5be64eab0 0x55e5be657ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:12.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.048+0000 7f594f67eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e5be6543b0 0x55e5be658420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:12.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.048+0000 7f594f67eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e5be474100 con 0x55e5be6543b0 2024-05-17T22:38:12.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.048+0000 7f594f67eb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e5be50e8f0 con 0x55e5be64eab0 2024-05-17T22:38:12.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.048+0000 7f594f67eb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e5be65ad80 con 0x55e5be4d09a0 2024-05-17T22:38:12.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.049+0000 7f594f67eb80 10 client.0 _ll_register_callbacks cb 0x55e5be503b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:38:12.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.049+0000 7f5939d9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5be64eab0 0x55e5be657ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:12.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.049+0000 7f593ad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e5be6543b0 0x55e5be658420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:12.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.049+0000 7f593ad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e5be6543b0 0x55e5be658420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36302/0 (socket says 172.21.15.12:36302) 2024-05-17T22:38:12.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.049+0000 7f5939d9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5be64eab0 0x55e5be657ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:41382/0 (socket says 172.21.15.12:41382) 2024-05-17T22:38:12.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.049+0000 7f593ad9d700 1 -- 172.21.15.12:0/2031128754 learned_addr learned my addr 172.21.15.12:0/2031128754 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:12.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.049+0000 7f593a59c700 1 --2- 172.21.15.12:0/2031128754 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5be4d09a0 0x55e5be6579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:12.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.049+0000 7f593ad9d700 1 -- 172.21.15.12:0/2031128754 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5be4d09a0 msgr2=0x55e5be6579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:12.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.049+0000 7f593ad9d700 1 --2- 172.21.15.12:0/2031128754 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5be4d09a0 0x55e5be6579a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:12.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.049+0000 7f593ad9d700 1 -- 172.21.15.12:0/2031128754 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5be64eab0 msgr2=0x55e5be657ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:12.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.049+0000 7f593ad9d700 1 --2- 172.21.15.12:0/2031128754 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5be64eab0 0x55e5be657ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:12.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.049+0000 7f593ad9d700 1 -- 172.21.15.12:0/2031128754 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5be4bd430 con 0x55e5be6543b0 2024-05-17T22:38:12.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.049+0000 7f593ad9d700 1 --2- 172.21.15.12:0/2031128754 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e5be6543b0 0x55e5be658420 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f593400bbb0 tx=0x7f59340a8db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b8fb6eae2597d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:12.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.049+0000 7f5921ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:38:12.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.049+0000 7f5921ffb700 1 -- 172.21.15.12:0/2031128754 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5934094070 con 0x55e5be6543b0 2024-05-17T22:38:12.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.049+0000 7f5921ffb700 1 -- 172.21.15.12:0/2031128754 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f59340a9120 con 0x55e5be6543b0 2024-05-17T22:38:12.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.049+0000 7f594f67eb80 1 -- 172.21.15.12:0/2031128754 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f592c0218b0 con 0x55e5be6543b0 2024-05-17T22:38:12.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.050+0000 7f5921ffb700 1 -- 172.21.15.12:0/2031128754 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f59340c8050 con 0x55e5be6543b0 2024-05-17T22:38:12.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.050+0000 7f594f67eb80 10 client.15153 fetch_fsmap learned FSMap version 26 2024-05-17T22:38:12.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.050+0000 7f594f67eb80 1 -- 172.21.15.12:0/2031128754 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e5be665800 con 0x55e5be6543b0 2024-05-17T22:38:12.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.050+0000 7f5921ffb700 1 -- 172.21.15.12:0/2031128754 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f592c0218b0 con 0x55e5be6543b0 2024-05-17T22:38:12.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.050+0000 7f5921ffb700 1 -- 172.21.15.12:0/2031128754 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f59340b1890 con 0x55e5be6543b0 2024-05-17T22:38:12.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.050+0000 7f594f67eb80 10 client.15153 fetch_fsmap finished waiting for FSMap version 26 2024-05-17T22:38:12.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.050+0000 7f594f67eb80 10 client.15153 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-17T22:38:12.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.050+0000 7f594f67eb80 20 client.15153 populate_metadata read hostname 'smithi012' 2024-05-17T22:38:12.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.050+0000 7f594f67eb80 1 --2- 172.21.15.12:0/2031128754 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55e5be667e60 0x55e5be6692d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:12.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.050+0000 7f594f67eb80 4 client.15153 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:38:12.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.050+0000 7f594f67eb80 1 -- 172.21.15.12:0/2031128754 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5be57efd0 con 0x55e5be667e60 2024-05-17T22:38:12.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.050+0000 7f593a59c700 1 --2- 172.21.15.12:0/2031128754 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55e5be667e60 0x55e5be6692d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:12.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.051+0000 7f5921ffb700 10 client.15153 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:38:12.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.051+0000 7f593a59c700 1 --2- 172.21.15.12:0/2031128754 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55e5be667e60 0x55e5be6692d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e598ef0aa08b362e server_cookie=3b68feaa66ab9d06 in_seq=0 out_seq=0 2024-05-17T22:38:12.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.052+0000 7f5921ffb700 1 -- 172.21.15.12:0/2031128754 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e5be57efd0 con 0x55e5be667e60 2024-05-17T22:38:12.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.052+0000 7f5921ffb700 10 client.15153 handle_command_reply: tid=0 2024-05-17T22:38:12.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.052+0000 7f594f67eb80 1 client.15153 shutdown 2024-05-17T22:38:12.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.052+0000 7f594f67eb80 1 -- 172.21.15.12:0/2031128754 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e5be6543b0 msgr2=0x55e5be658420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:12.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.052+0000 7f594f67eb80 1 --2- 172.21.15.12:0/2031128754 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e5be6543b0 0x55e5be658420 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f593400bbb0 tx=0x7f59340a8db0 comp rx=0 tx=0).stop 2024-05-17T22:38:12.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.052+0000 7f594f67eb80 1 -- 172.21.15.12:0/2031128754 shutdown_connections 2024-05-17T22:38:12.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.052+0000 7f594f67eb80 1 --2- 172.21.15.12:0/2031128754 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55e5be667e60 0x55e5be6692d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:12.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.052+0000 7f594f67eb80 1 --2- 172.21.15.12:0/2031128754 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e5be6543b0 0x55e5be658420 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:12.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.052+0000 7f594f67eb80 1 --2- 172.21.15.12:0/2031128754 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5be4d09a0 0x55e5be6579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:12.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.052+0000 7f594f67eb80 1 --2- 172.21.15.12:0/2031128754 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5be64eab0 0x55e5be657ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:12.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.052+0000 7f594f67eb80 1 -- 172.21.15.12:0/2031128754 >> 172.21.15.12:0/2031128754 conn(0x55e5be530780 msgr2=0x55e5be649110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:12.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.052+0000 7f5921ffb700 0 client.15153 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:38:12.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.052+0000 7f594f67eb80 1 -- 172.21.15.12:0/2031128754 shutdown_connections 2024-05-17T22:38:12.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.052+0000 7f594f67eb80 1 -- 172.21.15.12:0/2031128754 wait complete. 2024-05-17T22:38:12.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.052+0000 7f594f67eb80 20 client.15153 trim_cache size 0 max 16384 2024-05-17T22:38:12.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.074+0000 7f594f67eb80 1 Processor -- start 2024-05-17T22:38:12.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.074+0000 7f594f67eb80 1 -- start start 2024-05-17T22:38:12.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.074+0000 7f594f67eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e5be4d09a0 0x55e5be664080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:12.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.075+0000 7f594f67eb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5be64eab0 0x55e5be658270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:12.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.075+0000 7f594f67eb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5be6543b0 0x55e5be652cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:12.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.075+0000 7f594f67eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e5be667610 con 0x55e5be4d09a0 2024-05-17T22:38:12.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.075+0000 7f594f67eb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e5be667ce0 con 0x55e5be64eab0 2024-05-17T22:38:12.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.075+0000 7f594f67eb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e5be6683b0 con 0x55e5be6543b0 2024-05-17T22:38:12.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.075+0000 7f593a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e5be4d09a0 0x55e5be664080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:12.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.075+0000 7f593a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e5be4d09a0 0x55e5be664080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36314/0 (socket says 172.21.15.12:36314) 2024-05-17T22:38:12.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.075+0000 7f593a59c700 1 -- 172.21.15.12:0/588642110 learned_addr learned my addr 172.21.15.12:0/588642110 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:12.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.075+0000 7f593ad9d700 1 --2- 172.21.15.12:0/588642110 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5be6543b0 0x55e5be652cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:12.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.075+0000 7f5939d9b700 1 --2- 172.21.15.12:0/588642110 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5be64eab0 0x55e5be658270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:12.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.075+0000 7f593a59c700 1 -- 172.21.15.12:0/588642110 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5be6543b0 msgr2=0x55e5be652cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:12.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.075+0000 7f593a59c700 1 --2- 172.21.15.12:0/588642110 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5be6543b0 0x55e5be652cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:12.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.075+0000 7f593a59c700 1 -- 172.21.15.12:0/588642110 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5be64eab0 msgr2=0x55e5be658270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:12.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.075+0000 7f593a59c700 1 --2- 172.21.15.12:0/588642110 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5be64eab0 0x55e5be658270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:12.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.075+0000 7f593a59c700 1 -- 172.21.15.12:0/588642110 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5be6496b0 con 0x55e5be4d09a0 2024-05-17T22:38:12.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.075+0000 7f593a59c700 1 --2- 172.21.15.12:0/588642110 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e5be4d09a0 0x55e5be664080 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f592c00a420 tx=0x7f592c021f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b4245f28b2e044 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:12.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.075+0000 7f5923fff700 1 -- 172.21.15.12:0/588642110 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c026040 con 0x55e5be4d09a0 2024-05-17T22:38:12.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.075+0000 7f5923fff700 1 -- 172.21.15.12:0/588642110 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f592c037030 con 0x55e5be4d09a0 2024-05-17T22:38:12.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.075+0000 7f5923fff700 1 -- 172.21.15.12:0/588642110 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c0147b0 con 0x55e5be4d09a0 2024-05-17T22:38:12.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.076+0000 7f594f67eb80 1 -- 172.21.15.12:0/588642110 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e5be4d09a0 msgr2=0x55e5be664080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:12.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.076+0000 7f594f67eb80 1 --2- 172.21.15.12:0/588642110 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e5be4d09a0 0x55e5be664080 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f592c00a420 tx=0x7f592c021f90 comp rx=0 tx=0).stop 2024-05-17T22:38:12.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.076+0000 7f594f67eb80 1 -- 172.21.15.12:0/588642110 shutdown_connections 2024-05-17T22:38:12.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.076+0000 7f594f67eb80 1 --2- 172.21.15.12:0/588642110 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5be6543b0 0x55e5be652cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:12.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.076+0000 7f594f67eb80 1 --2- 172.21.15.12:0/588642110 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e5be4d09a0 0x55e5be664080 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:12.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.076+0000 7f594f67eb80 1 --2- 172.21.15.12:0/588642110 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5be64eab0 0x55e5be658270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:12.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.076+0000 7f594f67eb80 1 -- 172.21.15.12:0/588642110 >> 172.21.15.12:0/588642110 conn(0x55e5be530780 msgr2=0x55e5be656270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:12.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.076+0000 7f594f67eb80 1 -- 172.21.15.12:0/588642110 shutdown_connections 2024-05-17T22:38:12.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.076+0000 7f594f67eb80 1 -- 172.21.15.12:0/588642110 wait complete. 2024-05-17T22:38:12.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.077+0000 7f594f67eb80 1 -- start start 2024-05-17T22:38:12.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.077+0000 7f594f67eb80 1 Processor -- start 2024-05-17T22:38:12.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.077+0000 7f594f67eb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5be4d09a0 0x55e5be663770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:12.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.077+0000 7f594f67eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e5be64eab0 0x55e5be663cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:12.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.077+0000 7f594f67eb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5be6543b0 0x55e5be664230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:12.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.077+0000 7f594f67eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e5be669150 con 0x55e5be64eab0 2024-05-17T22:38:12.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.077+0000 7f594f67eb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e5be668a80 con 0x55e5be6543b0 2024-05-17T22:38:12.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.077+0000 7f594f67eb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e5be669820 con 0x55e5be4d09a0 2024-05-17T22:38:12.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.077+0000 7f594f67eb80 10 client.0 _ll_register_callbacks cb 0x55e5be503b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:38:12.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.077+0000 7f5939d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e5be64eab0 0x55e5be663cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:12.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.077+0000 7f5939d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e5be64eab0 0x55e5be663cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36324/0 (socket says 172.21.15.12:36324) 2024-05-17T22:38:12.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.077+0000 7f5939d9b700 1 -- 172.21.15.12:0/1921733591 learned_addr learned my addr 172.21.15.12:0/1921733591 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:12.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.077+0000 7f593a59c700 1 --2- 172.21.15.12:0/1921733591 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5be4d09a0 0x55e5be663770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:12.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.078+0000 7f5939d9b700 1 -- 172.21.15.12:0/1921733591 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5be4d09a0 msgr2=0x55e5be663770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:12.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.078+0000 7f5939d9b700 1 --2- 172.21.15.12:0/1921733591 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5be4d09a0 0x55e5be663770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:12.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.078+0000 7f5939d9b700 1 -- 172.21.15.12:0/1921733591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5be6543b0 msgr2=0x55e5be664230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:12.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.078+0000 7f593ad9d700 1 --2- 172.21.15.12:0/1921733591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5be6543b0 0x55e5be664230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:12.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.078+0000 7f5939d9b700 1 --2- 172.21.15.12:0/1921733591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5be6543b0 0x55e5be664230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:12.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.078+0000 7f5939d9b700 1 -- 172.21.15.12:0/1921733591 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5be670f20 con 0x55e5be64eab0 2024-05-17T22:38:12.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.078+0000 7f5939d9b700 1 --2- 172.21.15.12:0/1921733591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e5be64eab0 0x55e5be663cb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5930000c00 tx=0x7f5930003f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c14a237ccfb01a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:12.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.078+0000 7f5921ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:38:12.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.078+0000 7f5921ffb700 1 -- 172.21.15.12:0/1921733591 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5930005670 con 0x55e5be64eab0 2024-05-17T22:38:12.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.078+0000 7f5921ffb700 1 -- 172.21.15.12:0/1921733591 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f593000b7d0 con 0x55e5be64eab0 2024-05-17T22:38:12.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.078+0000 7f594f67eb80 1 -- 172.21.15.12:0/1921733591 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f592c037030 con 0x55e5be64eab0 2024-05-17T22:38:12.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.078+0000 7f5921ffb700 1 -- 172.21.15.12:0/1921733591 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5930033050 con 0x55e5be64eab0 2024-05-17T22:38:12.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.078+0000 7f594f67eb80 10 client.15159 fetch_fsmap learned FSMap version 26 2024-05-17T22:38:12.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.078+0000 7f594f67eb80 1 -- 172.21.15.12:0/1921733591 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e5be6d37b0 con 0x55e5be64eab0 2024-05-17T22:38:12.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.079+0000 7f5921ffb700 1 -- 172.21.15.12:0/1921733591 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f592c037030 con 0x55e5be64eab0 2024-05-17T22:38:12.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.079+0000 7f5921ffb700 1 -- 172.21.15.12:0/1921733591 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f593000fb10 con 0x55e5be64eab0 2024-05-17T22:38:12.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.079+0000 7f594f67eb80 10 client.15159 fetch_fsmap finished waiting for FSMap version 26 2024-05-17T22:38:12.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.079+0000 7f594f67eb80 10 client.15159 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-17T22:38:12.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.079+0000 7f594f67eb80 20 client.15159 populate_metadata read hostname 'smithi012' 2024-05-17T22:38:12.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.079+0000 7f594f67eb80 1 --2- 172.21.15.12:0/1921733591 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55e5be6d5cb0 0x55e5be6d7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:12.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.079+0000 7f594f67eb80 4 client.15159 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-17T22:38:12.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.079+0000 7f594f67eb80 1 -- 172.21.15.12:0/1921733591 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "status"}) -- 0x55e5be5787b0 con 0x55e5be6d5cb0 2024-05-17T22:38:12.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.079+0000 7f593a59c700 1 --2- 172.21.15.12:0/1921733591 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55e5be6d5cb0 0x55e5be6d7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:12.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.080+0000 7f593a59c700 1 --2- 172.21.15.12:0/1921733591 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55e5be6d5cb0 0x55e5be6d7120 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76d0d3f134d39a6e server_cookie=b09798b11ef7787e in_seq=0 out_seq=0 2024-05-17T22:38:12.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.080+0000 7f5921ffb700 10 client.15159 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:38:12.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.080+0000 7f5921ffb700 1 -- 172.21.15.12:0/1921733591 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55e5be5787b0 con 0x55e5be6d5cb0 2024-05-17T22:38:12.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.080+0000 7f5921ffb700 10 client.15159 handle_command_reply: tid=0 2024-05-17T22:38:12.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.080+0000 7f594f67eb80 1 client.15159 shutdown 2024-05-17T22:38:12.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.080+0000 7f594f67eb80 1 -- 172.21.15.12:0/1921733591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e5be64eab0 msgr2=0x55e5be663cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:12.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.080+0000 7f594f67eb80 1 --2- 172.21.15.12:0/1921733591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e5be64eab0 0x55e5be663cb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5930000c00 tx=0x7f5930003f00 comp rx=0 tx=0).stop 2024-05-17T22:38:12.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.080+0000 7f594f67eb80 1 -- 172.21.15.12:0/1921733591 shutdown_connections 2024-05-17T22:38:12.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.080+0000 7f594f67eb80 1 --2- 172.21.15.12:0/1921733591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5be6543b0 0x55e5be664230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:12.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.080+0000 7f594f67eb80 1 --2- 172.21.15.12:0/1921733591 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5be4d09a0 0x55e5be663770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:12.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.080+0000 7f594f67eb80 1 --2- 172.21.15.12:0/1921733591 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55e5be6d5cb0 0x55e5be6d7120 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:12.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.080+0000 7f594f67eb80 1 --2- 172.21.15.12:0/1921733591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e5be64eab0 0x55e5be663cb0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:12.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.080+0000 7f5921ffb700 0 client.15159 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:38:12.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.080+0000 7f594f67eb80 1 -- 172.21.15.12:0/1921733591 >> 172.21.15.12:0/1921733591 conn(0x55e5be530780 msgr2=0x55e5be6517a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:12.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.080+0000 7f594f67eb80 1 -- 172.21.15.12:0/1921733591 shutdown_connections 2024-05-17T22:38:12.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.080+0000 7f594f67eb80 1 -- 172.21.15.12:0/1921733591 wait complete. 2024-05-17T22:38:12.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.081+0000 7f594f67eb80 20 client.15159 trim_cache size 0 max 16384 2024-05-17T22:38:12.088 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:38:12.088 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "4d593780-149c-11ef-bc9a-c7b262605968", 2024-05-17T22:38:12.088 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 0, 2024-05-17T22:38:12.088 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24482, 2024-05-17T22:38:12.089 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-05-17T22:38:12.089 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-05-17T22:38:12.089 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-05-17T22:38:12.089 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 208.654160381, 2024-05-17T22:38:12.089 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 26, 2024-05-17T22:38:12.089 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 89, 2024-05-17T22:38:12.089 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 87, 2024-05-17T22:38:12.089 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 223.95068696199999 2024-05-17T22:38:12.089 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:38:12.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.081+0000 7f5940ca3700 1 -- 172.21.15.12:0/357059248 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f59240b2880 msgr2=0x7f59240b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:12.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.081+0000 7f5940ca3700 1 --2- 172.21.15.12:0/357059248 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f59240b2880 0x7f59240b4d90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f592c0060b0 tx=0x7f592c00c040 comp rx=0 tx=0).stop 2024-05-17T22:38:12.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.081+0000 7f5940ca3700 1 -- 172.21.15.12:0/357059248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f593c10e240 msgr2=0x7f593c19f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:12.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.081+0000 7f5940ca3700 1 --2- 172.21.15.12:0/357059248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f593c10e240 0x7f593c19f010 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f593400b550 tx=0x7f593400b860 comp rx=0 tx=0).stop 2024-05-17T22:38:12.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.081+0000 7f5940ca3700 1 -- 172.21.15.12:0/357059248 shutdown_connections 2024-05-17T22:38:12.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.081+0000 7f5940ca3700 1 --2- 172.21.15.12:0/357059248 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f59240b2880 0x7f59240b4d90 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:12.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.081+0000 7f5940ca3700 1 --2- 172.21.15.12:0/357059248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f593c10e240 0x7f593c19f010 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:12.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.081+0000 7f5940ca3700 1 --2- 172.21.15.12:0/357059248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f593c104e10 0x7f593c19a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:12.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.081+0000 7f5940ca3700 1 --2- 172.21.15.12:0/357059248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f593c105760 0x7f593c19ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:12.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.081+0000 7f5940ca3700 1 -- 172.21.15.12:0/357059248 >> 172.21.15.12:0/357059248 conn(0x7f593c079ae0 msgr2=0x7f593c0ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:12.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.081+0000 7f5940ca3700 1 -- 172.21.15.12:0/357059248 shutdown_connections 2024-05-17T22:38:12.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:12.081+0000 7f5940ca3700 1 -- 172.21.15.12:0/357059248 wait complete. 2024-05-17T22:38:12.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:11 smithi096 ceph-mon[36065]: pgmap v407: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T22:38:12.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:11 smithi151 ceph-mon[36101]: pgmap v407: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T22:38:12.800 INFO:tasks.cephadm:Setup complete, yielding 2024-05-17T22:38:12.800 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-17T22:38:12.810 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi012.front.sepia.ceph.com 2024-05-17T22:38:12.810 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch status' 2024-05-17T22:38:13.065 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:38:14.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:14 smithi096 ceph-mon[36065]: pgmap v408: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T22:38:14.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:14 smithi012 ceph-mon[31632]: pgmap v408: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T22:38:14.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:14 smithi151 ceph-mon[36101]: pgmap v408: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T22:38:14.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:14.781+0000 7f5cd45c0700 1 -- 172.21.15.12:0/1636004089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ccc106dd0 msgr2=0x7f5ccc1092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:14.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:14.781+0000 7f5cd45c0700 1 --2- 172.21.15.12:0/1636004089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ccc106dd0 0x7f5ccc1092b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc009990 tx=0x7f5cbc009ca0 comp rx=0 tx=0).stop 2024-05-17T22:38:14.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:14.781+0000 7f5cd45c0700 1 -- 172.21.15.12:0/1636004089 shutdown_connections 2024-05-17T22:38:14.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:14.781+0000 7f5cd45c0700 1 --2- 172.21.15.12:0/1636004089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ccc109940 0x7f5ccc10be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:14.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:14.781+0000 7f5cd45c0700 1 --2- 172.21.15.12:0/1636004089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ccc06c340 0x7f5ccc106800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:14.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:14.781+0000 7f5cd45c0700 1 --2- 172.21.15.12:0/1636004089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ccc106dd0 0x7f5ccc1092b0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:14.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:14.781+0000 7f5cd45c0700 1 -- 172.21.15.12:0/1636004089 >> 172.21.15.12:0/1636004089 conn(0x7f5ccc0fe3c0 msgr2=0x7f5ccc100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:14.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:14.781+0000 7f5cd45c0700 1 -- 172.21.15.12:0/1636004089 shutdown_connections 2024-05-17T22:38:14.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:14.781+0000 7f5cd45c0700 1 -- 172.21.15.12:0/1636004089 wait complete. 2024-05-17T22:38:14.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:14.782+0000 7f5cd45c0700 1 Processor -- start 2024-05-17T22:38:14.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:14.782+0000 7f5cd45c0700 1 -- start start 2024-05-17T22:38:14.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:14.782+0000 7f5cd45c0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ccc06c340 0x7f5ccc19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:14.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:14.782+0000 7f5cd45c0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ccc106dd0 0x7f5ccc19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:14.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:14.782+0000 7f5cd45c0700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ccc109940 0x7f5ccc1a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:14.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:14.782+0000 7f5cd45c0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc10f810 con 0x7f5ccc06c340 2024-05-17T22:38:14.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:14.782+0000 7f5cd45c0700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc10f690 con 0x7f5ccc106dd0 2024-05-17T22:38:14.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:14.782+0000 7f5cd45c0700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc10f990 con 0x7f5ccc109940 2024-05-17T22:38:14.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:14.783+0000 7f5cd235c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ccc06c340 0x7f5ccc19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:14.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:14.783+0000 7f5cd235c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ccc06c340 0x7f5ccc19a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36338/0 (socket says 172.21.15.12:36338) 2024-05-17T22:38:14.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:14.783+0000 7f5cd235c700 1 -- 172.21.15.12:0/2631061738 learned_addr learned my addr 172.21.15.12:0/2631061738 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:14.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:14.783+0000 7f5cd1b5b700 1 --2- 172.21.15.12:0/2631061738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ccc106dd0 0x7f5ccc19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:14.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:14.783+0000 7f5cd2b5d700 1 --2- 172.21.15.12:0/2631061738 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ccc109940 0x7f5ccc1a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:14.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:14.783+0000 7f5cd235c700 1 -- 172.21.15.12:0/2631061738 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ccc109940 msgr2=0x7f5ccc1a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:14.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:14.783+0000 7f5cd235c700 1 --2- 172.21.15.12:0/2631061738 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ccc109940 0x7f5ccc1a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:14.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:14.783+0000 7f5cd235c700 1 -- 172.21.15.12:0/2631061738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ccc106dd0 msgr2=0x7f5ccc19a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:14.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:14.783+0000 7f5cd235c700 1 --2- 172.21.15.12:0/2631061738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ccc106dd0 0x7f5ccc19a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:14.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:14.783+0000 7f5cd235c700 1 -- 172.21.15.12:0/2631061738 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ccc1a1fc0 con 0x7f5ccc06c340 2024-05-17T22:38:14.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:14.783+0000 7f5cd235c700 1 --2- 172.21.15.12:0/2631061738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ccc06c340 0x7f5ccc19a460 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5cc800b530 tx=0x7f5cc800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:14.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:14.783+0000 7f5cc37fe700 1 -- 172.21.15.12:0/2631061738 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc8015020 con 0x7f5ccc06c340 2024-05-17T22:38:14.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:14.783+0000 7f5cc37fe700 1 -- 172.21.15.12:0/2631061738 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cc8011ce0 con 0x7f5ccc06c340 2024-05-17T22:38:14.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:14.783+0000 7f5cc37fe700 1 -- 172.21.15.12:0/2631061738 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc8010dd0 con 0x7f5ccc06c340 2024-05-17T22:38:14.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:14.783+0000 7f5cd45c0700 1 -- 172.21.15.12:0/2631061738 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ccc1a22b0 con 0x7f5ccc06c340 2024-05-17T22:38:14.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:14.783+0000 7f5cd45c0700 1 -- 172.21.15.12:0/2631061738 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ccc1a2870 con 0x7f5ccc06c340 2024-05-17T22:38:14.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:14.784+0000 7f5cd45c0700 1 -- 172.21.15.12:0/2631061738 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ccc107320 con 0x7f5ccc06c340 2024-05-17T22:38:14.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:14.786+0000 7f5cc37fe700 1 -- 172.21.15.12:0/2631061738 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5cc8018020 con 0x7f5ccc06c340 2024-05-17T22:38:14.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:14.787+0000 7f5cc37fe700 1 --2- 172.21.15.12:0/2631061738 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5cb80b2900 0x7f5cb80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:14.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:14.787+0000 7f5cd1b5b700 1 --2- 172.21.15.12:0/2631061738 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5cb80b2900 0x7f5cb80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:14.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:14.787+0000 7f5cc37fe700 1 -- 172.21.15.12:0/2631061738 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f5cc80e2180 con 0x7f5ccc06c340 2024-05-17T22:38:14.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:14.787+0000 7f5cd1b5b700 1 --2- 172.21.15.12:0/2631061738 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5cb80b2900 0x7f5cb80b4e10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc0038a0 tx=0x7f5cbc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:14.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:14.790+0000 7f5cc37fe700 1 -- 172.21.15.12:0/2631061738 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5cc801f020 con 0x7f5ccc06c340 2024-05-17T22:38:15.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:15.023+0000 7f5cd45c0700 1 -- 172.21.15.12:0/2631061738 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f5ccc064700 con 0x7f5cb80b2900 2024-05-17T22:38:15.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:15.025+0000 7f5cc37fe700 1 -- 172.21.15.12:0/2631061738 <== mgr.14152 v2:172.21.15.12:6800/1388514047 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f5ccc064700 con 0x7f5cb80b2900 2024-05-17T22:38:15.029 INFO:teuthology.orchestra.run.smithi012.stdout:Backend: cephadm 2024-05-17T22:38:15.029 INFO:teuthology.orchestra.run.smithi012.stdout:Available: Yes 2024-05-17T22:38:15.029 INFO:teuthology.orchestra.run.smithi012.stdout:Paused: No 2024-05-17T22:38:15.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:15.027+0000 7f5cd45c0700 1 -- 172.21.15.12:0/2631061738 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5cb80b2900 msgr2=0x7f5cb80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:15.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:15.027+0000 7f5cd45c0700 1 --2- 172.21.15.12:0/2631061738 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5cb80b2900 0x7f5cb80b4e10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc0038a0 tx=0x7f5cbc01a040 comp rx=0 tx=0).stop 2024-05-17T22:38:15.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:15.027+0000 7f5cd45c0700 1 -- 172.21.15.12:0/2631061738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ccc06c340 msgr2=0x7f5ccc19a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:15.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:15.027+0000 7f5cd45c0700 1 --2- 172.21.15.12:0/2631061738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ccc06c340 0x7f5ccc19a460 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5cc800b530 tx=0x7f5cc800b840 comp rx=0 tx=0).stop 2024-05-17T22:38:15.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:15.027+0000 7f5cd45c0700 1 -- 172.21.15.12:0/2631061738 shutdown_connections 2024-05-17T22:38:15.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:15.027+0000 7f5cd45c0700 1 --2- 172.21.15.12:0/2631061738 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5cb80b2900 0x7f5cb80b4e10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:15.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:15.027+0000 7f5cd45c0700 1 --2- 172.21.15.12:0/2631061738 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ccc109940 0x7f5ccc1a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:15.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:15.027+0000 7f5cd45c0700 1 --2- 172.21.15.12:0/2631061738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ccc06c340 0x7f5ccc19a460 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:15.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:15.027+0000 7f5cd45c0700 1 --2- 172.21.15.12:0/2631061738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ccc106dd0 0x7f5ccc19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:15.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:15.027+0000 7f5cd45c0700 1 -- 172.21.15.12:0/2631061738 >> 172.21.15.12:0/2631061738 conn(0x7f5ccc0fe3c0 msgr2=0x7f5ccc104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:15.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:15.027+0000 7f5cd45c0700 1 -- 172.21.15.12:0/2631061738 shutdown_connections 2024-05-17T22:38:15.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:15.027+0000 7f5cd45c0700 1 -- 172.21.15.12:0/2631061738 wait complete. 2024-05-17T22:38:15.722 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-17T22:38:15.979 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:38:16.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:16 smithi096 ceph-mon[36065]: pgmap v409: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T22:38:16.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:16 smithi012 ceph-mon[31632]: pgmap v409: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T22:38:16.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:16 smithi151 ceph-mon[36101]: pgmap v409: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T22:38:17.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:17 smithi012 ceph-mon[31632]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:38:17.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:17 smithi096 ceph-mon[36065]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:38:17.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:17 smithi151 ceph-mon[36101]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:38:17.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.693+0000 7f8c0bc6f700 1 -- 172.21.15.12:0/252004711 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c04104660 msgr2=0x7f8c0410af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:17.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.693+0000 7f8c0bc6f700 1 --2- 172.21.15.12:0/252004711 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c04104660 0x7f8c0410af10 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc0099c0 tx=0x7f8bfc009cd0 comp rx=0 tx=0).stop 2024-05-17T22:38:17.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.693+0000 7f8c0bc6f700 1 -- 172.21.15.12:0/252004711 shutdown_connections 2024-05-17T22:38:17.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.693+0000 7f8c0bc6f700 1 --2- 172.21.15.12:0/252004711 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c04104660 0x7f8c0410af10 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:17.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.693+0000 7f8c0bc6f700 1 --2- 172.21.15.12:0/252004711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c04102b90 0x7f8c04102fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:17.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.693+0000 7f8c0bc6f700 1 --2- 172.21.15.12:0/252004711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c04103cc0 0x7f8c041040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:17.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.693+0000 7f8c0bc6f700 1 -- 172.21.15.12:0/252004711 >> 172.21.15.12:0/252004711 conn(0x7f8c040fe3c0 msgr2=0x7f8c04100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:17.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.694+0000 7f8c0bc6f700 1 -- 172.21.15.12:0/252004711 shutdown_connections 2024-05-17T22:38:17.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.694+0000 7f8c0bc6f700 1 -- 172.21.15.12:0/252004711 wait complete. 2024-05-17T22:38:17.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.694+0000 7f8c0bc6f700 1 Processor -- start 2024-05-17T22:38:17.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.694+0000 7f8c0bc6f700 1 -- start start 2024-05-17T22:38:17.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.695+0000 7f8c0bc6f700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c04102b90 0x7f8c041a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:17.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.695+0000 7f8c0bc6f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c04103cc0 0x7f8c041a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:17.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.695+0000 7f8c0bc6f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c04104660 0x7f8c041a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:17.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.695+0000 7f8c0bc6f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0410e610 con 0x7f8c04104660 2024-05-17T22:38:17.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.695+0000 7f8c0bc6f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0410e490 con 0x7f8c04103cc0 2024-05-17T22:38:17.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.695+0000 7f8c0bc6f700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0410e790 con 0x7f8c04102b90 2024-05-17T22:38:17.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.695+0000 7f8c0a20c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c04104660 0x7f8c041a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:17.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.695+0000 7f8c0a20c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c04104660 0x7f8c041a7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36352/0 (socket says 172.21.15.12:36352) 2024-05-17T22:38:17.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.695+0000 7f8c0a20c700 1 -- 172.21.15.12:0/582712665 learned_addr learned my addr 172.21.15.12:0/582712665 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:17.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.695+0000 7f8c0920a700 1 --2- 172.21.15.12:0/582712665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c04103cc0 0x7f8c041a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:17.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.695+0000 7f8c09a0b700 1 --2- 172.21.15.12:0/582712665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c04102b90 0x7f8c041a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:17.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.695+0000 7f8c0a20c700 1 -- 172.21.15.12:0/582712665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c04102b90 msgr2=0x7f8c041a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:17.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.695+0000 7f8c0a20c700 1 --2- 172.21.15.12:0/582712665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c04102b90 0x7f8c041a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:17.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.695+0000 7f8c0a20c700 1 -- 172.21.15.12:0/582712665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c04103cc0 msgr2=0x7f8c041a10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:17.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.695+0000 7f8c0a20c700 1 --2- 172.21.15.12:0/582712665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c04103cc0 0x7f8c041a10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:17.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.695+0000 7f8c0a20c700 1 -- 172.21.15.12:0/582712665 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c041a8310 con 0x7f8c04104660 2024-05-17T22:38:17.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.695+0000 7f8c09a0b700 1 --2- 172.21.15.12:0/582712665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c04102b90 0x7f8c041a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:38:17.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.696+0000 7f8c0a20c700 1 --2- 172.21.15.12:0/582712665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c04104660 0x7f8c041a7cd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc009f50 tx=0x7f8bfc0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:17.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.696+0000 7f8bfaffd700 1 -- 172.21.15.12:0/582712665 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc027070 con 0x7f8c04104660 2024-05-17T22:38:17.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.696+0000 7f8c0bc6f700 1 -- 172.21.15.12:0/582712665 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c041a85a0 con 0x7f8c04104660 2024-05-17T22:38:17.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.696+0000 7f8bfaffd700 1 -- 172.21.15.12:0/582712665 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bfc0040e0 con 0x7f8c04104660 2024-05-17T22:38:17.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.696+0000 7f8c0bc6f700 1 -- 172.21.15.12:0/582712665 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c041a8a50 con 0x7f8c04104660 2024-05-17T22:38:17.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.696+0000 7f8bfaffd700 1 -- 172.21.15.12:0/582712665 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc022db0 con 0x7f8c04104660 2024-05-17T22:38:17.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.697+0000 7f8c0bc6f700 1 -- 172.21.15.12:0/582712665 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c04102fa0 con 0x7f8c04104660 2024-05-17T22:38:17.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.699+0000 7f8bfaffd700 1 -- 172.21.15.12:0/582712665 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8bfc02a070 con 0x7f8c04104660 2024-05-17T22:38:17.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.700+0000 7f8bfaffd700 1 --2- 172.21.15.12:0/582712665 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8bf00b2880 0x7f8bf00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:17.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.700+0000 7f8c09a0b700 1 --2- 172.21.15.12:0/582712665 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8bf00b2880 0x7f8bf00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:17.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.700+0000 7f8bfaffd700 1 -- 172.21.15.12:0/582712665 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f8bfc0e3ff0 con 0x7f8c04104660 2024-05-17T22:38:17.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.700+0000 7f8c09a0b700 1 --2- 172.21.15.12:0/582712665 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8bf00b2880 0x7f8bf00b4d90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8c041a1c30 tx=0x7f8c00009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.703+0000 7f8bfaffd700 1 -- 172.21.15.12:0/582712665 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8bfc013070 con 0x7f8c04104660 2024-05-17T22:38:17.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.928+0000 7f8c0bc6f700 1 -- 172.21.15.12:0/582712665 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f8c04100440 con 0x7f8bf00b2880 2024-05-17T22:38:17.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.942+0000 7f8bfaffd700 1 -- 172.21.15.12:0/582712665 <== mgr.14152 v2:172.21.15.12:6800/1388514047 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f8c04100440 con 0x7f8bf00b2880 2024-05-17T22:38:17.946 INFO:teuthology.orchestra.run.smithi012.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-17T22:38:17.946 INFO:teuthology.orchestra.run.smithi012.stdout:mds.a smithi012 running (3m) 90s ago 3m 84.4M - 19.0.0-3794-g931809a9 8880f7613d02 c6d5b4f6405f 2024-05-17T22:38:17.946 INFO:teuthology.orchestra.run.smithi012.stdout:mds.b smithi096 running (3m) 92s ago 3m 95.7M - 19.0.0-3794-g931809a9 8880f7613d02 78c4d4ccf405 2024-05-17T22:38:17.946 INFO:teuthology.orchestra.run.smithi012.stdout:mds.c smithi151 running (4m) 92s ago 4m 85.2M - 19.0.0-3794-g931809a9 8880f7613d02 25c9781f4db4 2024-05-17T22:38:17.946 INFO:teuthology.orchestra.run.smithi012.stdout:mds.d smithi012 running (4m) 90s ago 4m 82.1M - 19.0.0-3794-g931809a9 8880f7613d02 16550dd7efce 2024-05-17T22:38:17.946 INFO:teuthology.orchestra.run.smithi012.stdout:mds.e smithi096 running (4m) 92s ago 4m 84.6M - 19.0.0-3794-g931809a9 8880f7613d02 49d01cd75e8b 2024-05-17T22:38:17.946 INFO:teuthology.orchestra.run.smithi012.stdout:mds.f smithi151 running (4m) 92s ago 4m 83.6M - 19.0.0-3794-g931809a9 8880f7613d02 3033908f4219 2024-05-17T22:38:17.946 INFO:teuthology.orchestra.run.smithi012.stdout:mds.g smithi012 running (3m) 90s ago 3m 81.9M - 19.0.0-3794-g931809a9 8880f7613d02 54adbc3cf4cf 2024-05-17T22:38:17.946 INFO:teuthology.orchestra.run.smithi012.stdout:mds.h smithi096 running (4m) 92s ago 4m 87.5M - 19.0.0-3794-g931809a9 8880f7613d02 fe0a8da3710d 2024-05-17T22:38:17.947 INFO:teuthology.orchestra.run.smithi012.stdout:mds.i smithi151 running (4m) 92s ago 3m 86.1M - 19.0.0-3794-g931809a9 8880f7613d02 b5eb2658cde1 2024-05-17T22:38:17.947 INFO:teuthology.orchestra.run.smithi012.stdout:mds.j smithi012 running (4m) 90s ago 4m 81.2M - 19.0.0-3794-g931809a9 8880f7613d02 8b1e97fd39c3 2024-05-17T22:38:17.947 INFO:teuthology.orchestra.run.smithi012.stdout:mds.k smithi096 running (4m) 92s ago 4m 83.6M - 19.0.0-3794-g931809a9 8880f7613d02 74fca3e35371 2024-05-17T22:38:17.947 INFO:teuthology.orchestra.run.smithi012.stdout:mds.l smithi151 running (4m) 92s ago 4m 81.7M - 19.0.0-3794-g931809a9 8880f7613d02 c7816c3d6909 2024-05-17T22:38:17.947 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.x smithi012 *:9283,8765 running (11m) 90s ago 11m 599M - 19.0.0-3794-g931809a9 8880f7613d02 16e3da619b4f 2024-05-17T22:38:17.947 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.y smithi096 *:8443,8765 running (9m) 92s ago 9m 535M - 19.0.0-3794-g931809a9 8880f7613d02 b2c251c0b105 2024-05-17T22:38:17.947 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.z smithi151 *:8443,8765 running (9m) 92s ago 9m 534M - 19.0.0-3794-g931809a9 8880f7613d02 8bc7ce2a5c73 2024-05-17T22:38:17.947 INFO:teuthology.orchestra.run.smithi012.stdout:mon.a smithi012 running (11m) 90s ago 12m 122M 2048M 19.0.0-3794-g931809a9 8880f7613d02 094715d271e6 2024-05-17T22:38:17.947 INFO:teuthology.orchestra.run.smithi012.stdout:mon.b smithi096 running (10m) 92s ago 10m 110M 2048M 19.0.0-3794-g931809a9 8880f7613d02 10fbdf09832d 2024-05-17T22:38:17.947 INFO:teuthology.orchestra.run.smithi012.stdout:mon.c smithi151 running (10m) 92s ago 10m 109M 2048M 19.0.0-3794-g931809a9 8880f7613d02 4e7aba1032cc 2024-05-17T22:38:17.947 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0 smithi012 running (9m) 90s ago 9m 112M 4273M 19.0.0-3794-g931809a9 8880f7613d02 d0519659339b 2024-05-17T22:38:17.947 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 smithi096 running (9m) 92s ago 9m 123M 4272M 19.0.0-3794-g931809a9 8880f7613d02 3e5db6f9040a 2024-05-17T22:38:17.947 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2 smithi151 running (8m) 92s ago 8m 121M 4272M 19.0.0-3794-g931809a9 8880f7613d02 5accbd07f069 2024-05-17T22:38:17.947 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3 smithi012 running (8m) 90s ago 8m 123M 4273M 19.0.0-3794-g931809a9 8880f7613d02 e4851f49c8b5 2024-05-17T22:38:17.947 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4 smithi096 running (7m) 92s ago 7m 119M 4272M 19.0.0-3794-g931809a9 8880f7613d02 1062eb66a2c3 2024-05-17T22:38:17.948 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5 smithi151 running (7m) 92s ago 7m 119M 4272M 19.0.0-3794-g931809a9 8880f7613d02 9940a8bbe7b8 2024-05-17T22:38:17.948 INFO:teuthology.orchestra.run.smithi012.stdout:osd.6 smithi012 running (7m) 90s ago 7m 112M 4273M 19.0.0-3794-g931809a9 8880f7613d02 4292790a6524 2024-05-17T22:38:17.948 INFO:teuthology.orchestra.run.smithi012.stdout:osd.7 smithi096 running (6m) 92s ago 6m 112M 4272M 19.0.0-3794-g931809a9 8880f7613d02 bd9d07228d66 2024-05-17T22:38:17.948 INFO:teuthology.orchestra.run.smithi012.stdout:osd.8 smithi151 running (6m) 92s ago 6m 120M 4272M 19.0.0-3794-g931809a9 8880f7613d02 d7128d3bc674 2024-05-17T22:38:17.948 INFO:teuthology.orchestra.run.smithi012.stdout:osd.9 smithi012 running (5m) 90s ago 5m 111M 4273M 19.0.0-3794-g931809a9 8880f7613d02 0926f45ab2b0 2024-05-17T22:38:17.948 INFO:teuthology.orchestra.run.smithi012.stdout:osd.10 smithi096 running (5m) 92s ago 5m 111M 4272M 19.0.0-3794-g931809a9 8880f7613d02 6943c096ae47 2024-05-17T22:38:17.948 INFO:teuthology.orchestra.run.smithi012.stdout:osd.11 smithi151 running (4m) 92s ago 4m 120M 4272M 19.0.0-3794-g931809a9 8880f7613d02 8c0fa222680b 2024-05-17T22:38:17.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.944+0000 7f8c0bc6f700 1 -- 172.21.15.12:0/582712665 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8bf00b2880 msgr2=0x7f8bf00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:17.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.944+0000 7f8c0bc6f700 1 --2- 172.21.15.12:0/582712665 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8bf00b2880 0x7f8bf00b4d90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8c041a1c30 tx=0x7f8c00009040 comp rx=0 tx=0).stop 2024-05-17T22:38:17.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.944+0000 7f8c0bc6f700 1 -- 172.21.15.12:0/582712665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c04104660 msgr2=0x7f8c041a7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:17.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.944+0000 7f8c0bc6f700 1 --2- 172.21.15.12:0/582712665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c04104660 0x7f8c041a7cd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc009f50 tx=0x7f8bfc0057c0 comp rx=0 tx=0).stop 2024-05-17T22:38:17.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.944+0000 7f8c0bc6f700 1 -- 172.21.15.12:0/582712665 shutdown_connections 2024-05-17T22:38:17.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.944+0000 7f8c0bc6f700 1 --2- 172.21.15.12:0/582712665 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8bf00b2880 0x7f8bf00b4d90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:17.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.944+0000 7f8c0bc6f700 1 --2- 172.21.15.12:0/582712665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c04104660 0x7f8c041a7cd0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:17.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.944+0000 7f8c0bc6f700 1 --2- 172.21.15.12:0/582712665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c04102b90 0x7f8c041a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:17.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.944+0000 7f8c0bc6f700 1 --2- 172.21.15.12:0/582712665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c04103cc0 0x7f8c041a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:17.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.944+0000 7f8c0bc6f700 1 -- 172.21.15.12:0/582712665 >> 172.21.15.12:0/582712665 conn(0x7f8c040fe3c0 msgr2=0x7f8c040ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:17.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.944+0000 7f8c0bc6f700 1 -- 172.21.15.12:0/582712665 shutdown_connections 2024-05-17T22:38:17.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:17.944+0000 7f8c0bc6f700 1 -- 172.21.15.12:0/582712665 wait complete. 2024-05-17T22:38:18.256 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:18 smithi012 ceph-mon[31632]: pgmap v410: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T22:38:18.256 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:18 smithi012 ceph-mon[31632]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:38:18.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:18 smithi096 ceph-mon[36065]: pgmap v410: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T22:38:18.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:18 smithi096 ceph-mon[36065]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:38:18.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:18 smithi151 ceph-mon[36101]: pgmap v410: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T22:38:18.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:18 smithi151 ceph-mon[36101]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:38:18.598 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-17T22:38:18.859 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:38:19.218 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:19 smithi012 ceph-mon[31632]: pgmap v411: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:38:19.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:19 smithi151 ceph-mon[36101]: pgmap v411: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:38:19.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:19 smithi096 ceph-mon[36065]: pgmap v411: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:38:20.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.602+0000 7effa70ef700 1 -- 172.21.15.12:0/3078913002 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effa0106dd0 msgr2=0x7effa01092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:20.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.602+0000 7effa70ef700 1 --2- 172.21.15.12:0/3078913002 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effa0106dd0 0x7effa01092b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7eff8c009990 tx=0x7eff8c009ca0 comp rx=0 tx=0).stop 2024-05-17T22:38:20.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.602+0000 7effa70ef700 1 -- 172.21.15.12:0/3078913002 shutdown_connections 2024-05-17T22:38:20.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.602+0000 7effa70ef700 1 --2- 172.21.15.12:0/3078913002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7effa0109940 0x7effa010be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:20.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.602+0000 7effa70ef700 1 --2- 172.21.15.12:0/3078913002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7effa006cd00 0x7effa0106800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:20.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.602+0000 7effa70ef700 1 --2- 172.21.15.12:0/3078913002 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effa0106dd0 0x7effa01092b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:20.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.602+0000 7effa70ef700 1 -- 172.21.15.12:0/3078913002 >> 172.21.15.12:0/3078913002 conn(0x7effa00fe3c0 msgr2=0x7effa0100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:20.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.602+0000 7effa70ef700 1 -- 172.21.15.12:0/3078913002 shutdown_connections 2024-05-17T22:38:20.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.602+0000 7effa70ef700 1 -- 172.21.15.12:0/3078913002 wait complete. 2024-05-17T22:38:20.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.603+0000 7effa70ef700 1 Processor -- start 2024-05-17T22:38:20.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.603+0000 7effa70ef700 1 -- start start 2024-05-17T22:38:20.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.603+0000 7effa70ef700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7effa006cd00 0x7effa019a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:20.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.603+0000 7effa70ef700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7effa0106dd0 0x7effa019a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:20.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.603+0000 7effa70ef700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effa0109940 0x7effa01a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:20.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.603+0000 7effa70ef700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7effa010f790 con 0x7effa0109940 2024-05-17T22:38:20.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.603+0000 7effa70ef700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7effa010f610 con 0x7effa0106dd0 2024-05-17T22:38:20.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.603+0000 7effa70ef700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7effa010f910 con 0x7effa006cd00 2024-05-17T22:38:20.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.604+0000 7effa568c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effa0109940 0x7effa01a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:20.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.604+0000 7effa568c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effa0109940 0x7effa01a17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56924/0 (socket says 172.21.15.12:56924) 2024-05-17T22:38:20.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.604+0000 7effa568c700 1 -- 172.21.15.12:0/3430497175 learned_addr learned my addr 172.21.15.12:0/3430497175 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:20.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.604+0000 7effa4e8b700 1 --2- 172.21.15.12:0/3430497175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7effa006cd00 0x7effa019a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.604+0000 7eff9ffff700 1 --2- 172.21.15.12:0/3430497175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7effa0106dd0 0x7effa019a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.604+0000 7effa568c700 1 -- 172.21.15.12:0/3430497175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7effa006cd00 msgr2=0x7effa019a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.604+0000 7effa568c700 1 --2- 172.21.15.12:0/3430497175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7effa006cd00 0x7effa019a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.604+0000 7effa568c700 1 -- 172.21.15.12:0/3430497175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7effa0106dd0 msgr2=0x7effa019a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.604+0000 7effa568c700 1 --2- 172.21.15.12:0/3430497175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7effa0106dd0 0x7effa019a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.604+0000 7effa568c700 1 -- 172.21.15.12:0/3430497175 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa01a1e40 con 0x7effa0109940 2024-05-17T22:38:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.604+0000 7effa568c700 1 --2- 172.21.15.12:0/3430497175 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effa0109940 0x7effa01a17c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7eff9800b970 tx=0x7eff9800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.604+0000 7eff9dffb700 1 -- 172.21.15.12:0/3430497175 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9800d860 con 0x7effa0109940 2024-05-17T22:38:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.604+0000 7eff9dffb700 1 -- 172.21.15.12:0/3430497175 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff9800e430 con 0x7effa0109940 2024-05-17T22:38:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.604+0000 7effa70ef700 1 -- 172.21.15.12:0/3430497175 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effa01a2130 con 0x7effa0109940 2024-05-17T22:38:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.604+0000 7eff9dffb700 1 -- 172.21.15.12:0/3430497175 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff98017660 con 0x7effa0109940 2024-05-17T22:38:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.605+0000 7effa70ef700 1 -- 172.21.15.12:0/3430497175 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effa01a26a0 con 0x7effa0109940 2024-05-17T22:38:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.606+0000 7effa70ef700 1 -- 172.21.15.12:0/3430497175 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effa01072a0 con 0x7effa0109940 2024-05-17T22:38:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.607+0000 7eff9dffb700 1 -- 172.21.15.12:0/3430497175 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7eff9800da00 con 0x7effa0109940 2024-05-17T22:38:20.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.608+0000 7eff9dffb700 1 --2- 172.21.15.12:0/3430497175 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7eff900b2880 0x7eff900b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:20.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.608+0000 7effa4e8b700 1 --2- 172.21.15.12:0/3430497175 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7eff900b2880 0x7eff900b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:20.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.608+0000 7eff9dffb700 1 -- 172.21.15.12:0/3430497175 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7eff980e23c0 con 0x7effa0109940 2024-05-17T22:38:20.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.609+0000 7effa4e8b700 1 --2- 172.21.15.12:0/3430497175 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7eff900b2880 0x7eff900b4d90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7eff94005950 tx=0x7eff9400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:20.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.611+0000 7eff9dffb700 1 -- 172.21.15.12:0/3430497175 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7eff98053df0 con 0x7effa0109940 2024-05-17T22:38:20.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.837+0000 7effa70ef700 1 -- 172.21.15.12:0/3430497175 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7effa0064700 con 0x7eff900b2880 2024-05-17T22:38:20.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.844+0000 7eff9dffb700 1 -- 172.21.15.12:0/3430497175 <== mgr.14152 v2:172.21.15.12:6800/1388514047 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7effa0064700 con 0x7eff900b2880 2024-05-17T22:38:20.848 INFO:teuthology.orchestra.run.smithi012.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-17T22:38:20.848 INFO:teuthology.orchestra.run.smithi012.stdout:mds.all 12/12 95s ago 4m smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l;count:12 2024-05-17T22:38:20.848 INFO:teuthology.orchestra.run.smithi012.stdout:mgr 3/3 95s ago 9m smithi012=x;smithi096=y;smithi151=z;count:3 2024-05-17T22:38:20.848 INFO:teuthology.orchestra.run.smithi012.stdout:mon 3/3 95s ago 10m smithi012:172.21.15.12=a;smithi096:172.21.15.96=b;smithi151:172.21.15.151=c;count:3 2024-05-17T22:38:20.848 INFO:teuthology.orchestra.run.smithi012.stdout:osd 12 95s ago - 2024-05-17T22:38:20.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.846+0000 7effa70ef700 1 -- 172.21.15.12:0/3430497175 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7eff900b2880 msgr2=0x7eff900b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:20.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.846+0000 7effa70ef700 1 --2- 172.21.15.12:0/3430497175 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7eff900b2880 0x7eff900b4d90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7eff94005950 tx=0x7eff9400b410 comp rx=0 tx=0).stop 2024-05-17T22:38:20.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.846+0000 7effa70ef700 1 -- 172.21.15.12:0/3430497175 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effa0109940 msgr2=0x7effa01a17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:20.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.846+0000 7effa70ef700 1 --2- 172.21.15.12:0/3430497175 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effa0109940 0x7effa01a17c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7eff9800b970 tx=0x7eff9800bc80 comp rx=0 tx=0).stop 2024-05-17T22:38:20.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.846+0000 7effa70ef700 1 -- 172.21.15.12:0/3430497175 shutdown_connections 2024-05-17T22:38:20.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.846+0000 7effa70ef700 1 --2- 172.21.15.12:0/3430497175 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7eff900b2880 0x7eff900b4d90 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:20.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.846+0000 7effa70ef700 1 --2- 172.21.15.12:0/3430497175 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effa0109940 0x7effa01a17c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:20.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.846+0000 7effa70ef700 1 --2- 172.21.15.12:0/3430497175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7effa006cd00 0x7effa019a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:20.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.846+0000 7effa70ef700 1 --2- 172.21.15.12:0/3430497175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7effa0106dd0 0x7effa019a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:20.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.846+0000 7effa70ef700 1 -- 172.21.15.12:0/3430497175 >> 172.21.15.12:0/3430497175 conn(0x7effa00fe3c0 msgr2=0x7effa01051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:20.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.846+0000 7effa70ef700 1 -- 172.21.15.12:0/3430497175 shutdown_connections 2024-05-17T22:38:20.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:20.846+0000 7effa70ef700 1 -- 172.21.15.12:0/3430497175 wait complete. 2024-05-17T22:38:21.522 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-17T22:38:21.777 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:38:21.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:21 smithi012 ceph-mon[31632]: pgmap v412: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T22:38:21.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:21 smithi012 ceph-mon[31632]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:38:21.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:21 smithi151 ceph-mon[36101]: pgmap v412: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T22:38:21.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:21 smithi151 ceph-mon[36101]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:38:22.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:21 smithi096 ceph-mon[36065]: pgmap v412: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T22:38:22.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:21 smithi096 ceph-mon[36065]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:38:23.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.457+0000 7fa55bce9700 1 -- 172.21.15.12:0/1718120296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa554104800 msgr2=0x7fa554106c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:23.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.457+0000 7fa55bce9700 1 --2- 172.21.15.12:0/1718120296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa554104800 0x7fa554106c20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa550009990 tx=0x7fa550009ca0 comp rx=0 tx=0).stop 2024-05-17T22:38:23.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.457+0000 7fa55bce9700 1 -- 172.21.15.12:0/1718120296 shutdown_connections 2024-05-17T22:38:23.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.457+0000 7fa55bce9700 1 --2- 172.21.15.12:0/1718120296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa554109ac0 0x7fa55410bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:23.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.457+0000 7fa55bce9700 1 --2- 172.21.15.12:0/1718120296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa554104800 0x7fa554106c20 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:23.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.457+0000 7fa55bce9700 1 --2- 172.21.15.12:0/1718120296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa554107160 0x7fa554109580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:23.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.457+0000 7fa55bce9700 1 -- 172.21.15.12:0/1718120296 >> 172.21.15.12:0/1718120296 conn(0x7fa5540fe450 msgr2=0x7fa5541008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:23.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.457+0000 7fa55bce9700 1 -- 172.21.15.12:0/1718120296 shutdown_connections 2024-05-17T22:38:23.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.457+0000 7fa55bce9700 1 -- 172.21.15.12:0/1718120296 wait complete. 2024-05-17T22:38:23.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.458+0000 7fa55bce9700 1 Processor -- start 2024-05-17T22:38:23.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.458+0000 7fa55bce9700 1 -- start start 2024-05-17T22:38:23.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.458+0000 7fa55bce9700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa554104800 0x7fa554104060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:23.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.458+0000 7fa55bce9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa554107160 0x7fa554102710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:23.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.458+0000 7fa55bce9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa554109ac0 0x7fa554102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:23.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.458+0000 7fa55bce9700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa55410f710 con 0x7fa554107160 2024-05-17T22:38:23.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.458+0000 7fa55bce9700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa55410f590 con 0x7fa554109ac0 2024-05-17T22:38:23.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.458+0000 7fa55bce9700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa55410f890 con 0x7fa554104800 2024-05-17T22:38:23.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.458+0000 7fa559284700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa554107160 0x7fa554102710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:23.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.458+0000 7fa559284700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa554107160 0x7fa554102710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56948/0 (socket says 172.21.15.12:56948) 2024-05-17T22:38:23.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.458+0000 7fa559284700 1 -- 172.21.15.12:0/2781579583 learned_addr learned my addr 172.21.15.12:0/2781579583 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:23.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.458+0000 7fa559a85700 1 --2- 172.21.15.12:0/2781579583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa554104800 0x7fa554104060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:23.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.459+0000 7fa55a286700 1 --2- 172.21.15.12:0/2781579583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa554109ac0 0x7fa554102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:23.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.459+0000 7fa559284700 1 -- 172.21.15.12:0/2781579583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa554104800 msgr2=0x7fa554104060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:23.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.459+0000 7fa559284700 1 --2- 172.21.15.12:0/2781579583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa554104800 0x7fa554104060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:23.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.459+0000 7fa559284700 1 -- 172.21.15.12:0/2781579583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa554109ac0 msgr2=0x7fa554102c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:23.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.459+0000 7fa559284700 1 --2- 172.21.15.12:0/2781579583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa554109ac0 0x7fa554102c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:23.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.459+0000 7fa559284700 1 -- 172.21.15.12:0/2781579583 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5541033b0 con 0x7fa554107160 2024-05-17T22:38:23.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.459+0000 7fa559284700 1 --2- 172.21.15.12:0/2781579583 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa554107160 0x7fa554102710 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa54400c920 tx=0x7fa54400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:23.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.459+0000 7fa54affd700 1 -- 172.21.15.12:0/2781579583 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa544004a30 con 0x7fa554107160 2024-05-17T22:38:23.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.459+0000 7fa54affd700 1 -- 172.21.15.12:0/2781579583 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5440054d0 con 0x7fa554107160 2024-05-17T22:38:23.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.459+0000 7fa54affd700 1 -- 172.21.15.12:0/2781579583 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa544007c70 con 0x7fa554107160 2024-05-17T22:38:23.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.459+0000 7fa55bce9700 1 -- 172.21.15.12:0/2781579583 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5541a43c0 con 0x7fa554107160 2024-05-17T22:38:23.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.459+0000 7fa55bce9700 1 -- 172.21.15.12:0/2781579583 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5541a48a0 con 0x7fa554107160 2024-05-17T22:38:23.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.460+0000 7fa55bce9700 1 -- 172.21.15.12:0/2781579583 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa554107570 con 0x7fa554107160 2024-05-17T22:38:23.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.464+0000 7fa54affd700 1 -- 172.21.15.12:0/2781579583 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa544004c60 con 0x7fa554107160 2024-05-17T22:38:23.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.464+0000 7fa54affd700 1 --2- 172.21.15.12:0/2781579583 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa5400b2880 0x7fa5400b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:23.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.464+0000 7fa559a85700 1 --2- 172.21.15.12:0/2781579583 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa5400b2880 0x7fa5400b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:23.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.464+0000 7fa54affd700 1 -- 172.21.15.12:0/2781579583 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fa5440e2520 con 0x7fa554107160 2024-05-17T22:38:23.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.464+0000 7fa559a85700 1 --2- 172.21.15.12:0/2781579583 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa5400b2880 0x7fa5400b4d90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa55000b5c0 tx=0x7fa55001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:23.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.467+0000 7fa54affd700 1 -- 172.21.15.12:0/2781579583 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa544014030 con 0x7fa554107160 2024-05-17T22:38:23.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.691+0000 7fa55bce9700 1 -- 172.21.15.12:0/2781579583 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fa554064690 con 0x7fa5400b2880 2024-05-17T22:38:23.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.693+0000 7fa54affd700 1 -- 172.21.15.12:0/2781579583 <== mgr.14152 v2:172.21.15.12:6800/1388514047 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fa554064690 con 0x7fa5400b2880 2024-05-17T22:38:23.697 INFO:teuthology.orchestra.run.smithi012.stdout:HOST ADDR LABELS STATUS 2024-05-17T22:38:23.697 INFO:teuthology.orchestra.run.smithi012.stdout:smithi012 172.21.15.12 2024-05-17T22:38:23.697 INFO:teuthology.orchestra.run.smithi012.stdout:smithi096 172.21.15.96 2024-05-17T22:38:23.697 INFO:teuthology.orchestra.run.smithi012.stdout:smithi151 172.21.15.151 2024-05-17T22:38:23.697 INFO:teuthology.orchestra.run.smithi012.stdout:3 hosts in cluster 2024-05-17T22:38:23.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.695+0000 7fa55bce9700 1 -- 172.21.15.12:0/2781579583 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa5400b2880 msgr2=0x7fa5400b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:23.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.695+0000 7fa55bce9700 1 --2- 172.21.15.12:0/2781579583 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa5400b2880 0x7fa5400b4d90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa55000b5c0 tx=0x7fa55001a040 comp rx=0 tx=0).stop 2024-05-17T22:38:23.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.695+0000 7fa55bce9700 1 -- 172.21.15.12:0/2781579583 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa554107160 msgr2=0x7fa554102710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:23.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.695+0000 7fa55bce9700 1 --2- 172.21.15.12:0/2781579583 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa554107160 0x7fa554102710 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa54400c920 tx=0x7fa54400cc30 comp rx=0 tx=0).stop 2024-05-17T22:38:23.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.695+0000 7fa55bce9700 1 -- 172.21.15.12:0/2781579583 shutdown_connections 2024-05-17T22:38:23.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.695+0000 7fa55bce9700 1 --2- 172.21.15.12:0/2781579583 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa5400b2880 0x7fa5400b4d90 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:23.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.695+0000 7fa55bce9700 1 --2- 172.21.15.12:0/2781579583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa554109ac0 0x7fa554102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:23.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.695+0000 7fa55bce9700 1 --2- 172.21.15.12:0/2781579583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa554104800 0x7fa554104060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:23.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.695+0000 7fa55bce9700 1 --2- 172.21.15.12:0/2781579583 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa554107160 0x7fa554102710 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:23.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.695+0000 7fa55bce9700 1 -- 172.21.15.12:0/2781579583 >> 172.21.15.12:0/2781579583 conn(0x7fa5540fe450 msgr2=0x7fa5541052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:23.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.695+0000 7fa55bce9700 1 -- 172.21.15.12:0/2781579583 shutdown_connections 2024-05-17T22:38:23.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:23.695+0000 7fa55bce9700 1 -- 172.21.15.12:0/2781579583 wait complete. 2024-05-17T22:38:23.711 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:23 smithi012 ceph-mon[31632]: pgmap v413: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:38:23.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:23 smithi151 ceph-mon[36101]: pgmap v413: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:38:24.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:23 smithi096 ceph-mon[36065]: pgmap v413: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:38:24.341 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-17T22:38:24.599 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:38:24.875 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:24 smithi012 ceph-mon[31632]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:38:24.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:24 smithi151 ceph-mon[36101]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:38:25.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:24 smithi096 ceph-mon[36065]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:38:25.795 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:25 smithi012 ceph-mon[31632]: pgmap v414: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T22:38:25.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:25 smithi151 ceph-mon[36101]: pgmap v414: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T22:38:26.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:25 smithi096 ceph-mon[36065]: pgmap v414: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T22:38:26.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.403+0000 7fc34ae69700 1 -- 172.21.15.12:0/2705201041 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc34406cce0 msgr2=0x7fc344106820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:26.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.403+0000 7fc34ae69700 1 --2- 172.21.15.12:0/2705201041 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc34406cce0 0x7fc344106820 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc338009990 tx=0x7fc338009ca0 comp rx=0 tx=0).stop 2024-05-17T22:38:26.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.403+0000 7fc34ae69700 1 -- 172.21.15.12:0/2705201041 shutdown_connections 2024-05-17T22:38:26.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.403+0000 7fc34ae69700 1 --2- 172.21.15.12:0/2705201041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc344109950 0x7fc34410be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:26.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.403+0000 7fc34ae69700 1 --2- 172.21.15.12:0/2705201041 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc34406cce0 0x7fc344106820 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:26.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.403+0000 7fc34ae69700 1 --2- 172.21.15.12:0/2705201041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc344106df0 0x7fc344109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:26.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.403+0000 7fc34ae69700 1 -- 172.21.15.12:0/2705201041 >> 172.21.15.12:0/2705201041 conn(0x7fc3440fe3c0 msgr2=0x7fc344100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:26.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.403+0000 7fc34ae69700 1 -- 172.21.15.12:0/2705201041 shutdown_connections 2024-05-17T22:38:26.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.403+0000 7fc34ae69700 1 -- 172.21.15.12:0/2705201041 wait complete. 2024-05-17T22:38:26.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.404+0000 7fc34ae69700 1 Processor -- start 2024-05-17T22:38:26.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.404+0000 7fc34ae69700 1 -- start start 2024-05-17T22:38:26.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.404+0000 7fc34ae69700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc34406cce0 0x7fc34419a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:26.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.404+0000 7fc34ae69700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc344106df0 0x7fc34419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:26.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.404+0000 7fc34ae69700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc344109950 0x7fc3441a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:26.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.404+0000 7fc34ae69700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc34410f750 con 0x7fc344109950 2024-05-17T22:38:26.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.404+0000 7fc34ae69700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc34410f5d0 con 0x7fc34406cce0 2024-05-17T22:38:26.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.404+0000 7fc34ae69700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc34410f8d0 con 0x7fc344106df0 2024-05-17T22:38:26.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.404+0000 7fc349406700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc344109950 0x7fc3441a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:26.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.404+0000 7fc348c05700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc34406cce0 0x7fc34419a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:26.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.404+0000 7fc349406700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc344109950 0x7fc3441a17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56958/0 (socket says 172.21.15.12:56958) 2024-05-17T22:38:26.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.404+0000 7fc349406700 1 -- 172.21.15.12:0/1403407105 learned_addr learned my addr 172.21.15.12:0/1403407105 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:26.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.404+0000 7fc343fff700 1 --2- 172.21.15.12:0/1403407105 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc344106df0 0x7fc34419a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:26.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.405+0000 7fc349406700 1 -- 172.21.15.12:0/1403407105 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc344106df0 msgr2=0x7fc34419a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:26.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.405+0000 7fc349406700 1 --2- 172.21.15.12:0/1403407105 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc344106df0 0x7fc34419a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:26.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.405+0000 7fc349406700 1 -- 172.21.15.12:0/1403407105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc34406cce0 msgr2=0x7fc34419a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:26.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.405+0000 7fc349406700 1 --2- 172.21.15.12:0/1403407105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc34406cce0 0x7fc34419a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:26.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.405+0000 7fc349406700 1 -- 172.21.15.12:0/1403407105 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3441a1e70 con 0x7fc344109950 2024-05-17T22:38:26.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.405+0000 7fc348c05700 1 --2- 172.21.15.12:0/1403407105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc34406cce0 0x7fc34419a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:38:26.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.405+0000 7fc349406700 1 --2- 172.21.15.12:0/1403407105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc344109950 0x7fc3441a17f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc33c00ada0 tx=0x7fc33c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:26.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.405+0000 7fc341ffb700 1 -- 172.21.15.12:0/1403407105 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33c017020 con 0x7fc344109950 2024-05-17T22:38:26.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.405+0000 7fc34ae69700 1 -- 172.21.15.12:0/1403407105 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3441a2160 con 0x7fc344109950 2024-05-17T22:38:26.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.405+0000 7fc341ffb700 1 -- 172.21.15.12:0/1403407105 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc33c00cd80 con 0x7fc344109950 2024-05-17T22:38:26.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.405+0000 7fc341ffb700 1 -- 172.21.15.12:0/1403407105 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33c005260 con 0x7fc344109950 2024-05-17T22:38:26.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.405+0000 7fc34ae69700 1 -- 172.21.15.12:0/1403407105 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3441a2720 con 0x7fc344109950 2024-05-17T22:38:26.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.407+0000 7fc34ae69700 1 -- 172.21.15.12:0/1403407105 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc344107280 con 0x7fc344109950 2024-05-17T22:38:26.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.408+0000 7fc341ffb700 1 -- 172.21.15.12:0/1403407105 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc33c00c8d0 con 0x7fc344109950 2024-05-17T22:38:26.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.409+0000 7fc341ffb700 1 --2- 172.21.15.12:0/1403407105 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fc3340b2900 0x7fc3340b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:26.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.409+0000 7fc348c05700 1 --2- 172.21.15.12:0/1403407105 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fc3340b2900 0x7fc3340b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:26.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.409+0000 7fc341ffb700 1 -- 172.21.15.12:0/1403407105 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fc33c0e1c50 con 0x7fc344109950 2024-05-17T22:38:26.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.409+0000 7fc348c05700 1 --2- 172.21.15.12:0/1403407105 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fc3340b2900 0x7fc3340b4e10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc3380038a0 tx=0x7fc33801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:26.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.413+0000 7fc341ffb700 1 -- 172.21.15.12:0/1403407105 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc33c0e7020 con 0x7fc344109950 2024-05-17T22:38:26.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.638+0000 7fc34ae69700 1 -- 172.21.15.12:0/1403407105 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fc344063c70 con 0x7fc3340b2900 2024-05-17T22:38:26.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.640+0000 7fc341ffb700 1 -- 172.21.15.12:0/1403407105 <== mgr.14152 v2:172.21.15.12:6800/1388514047 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fc344063c70 con 0x7fc3340b2900 2024-05-17T22:38:26.643 INFO:teuthology.orchestra.run.smithi012.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-17T22:38:26.643 INFO:teuthology.orchestra.run.smithi012.stdout:smithi012 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5331005J400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-05-17T22:38:26.643 INFO:teuthology.orchestra.run.smithi012.stdout:smithi096 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NH400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-05-17T22:38:26.643 INFO:teuthology.orchestra.run.smithi012.stdout:smithi151 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300JJ400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-05-17T22:38:26.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.642+0000 7fc34ae69700 1 -- 172.21.15.12:0/1403407105 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fc3340b2900 msgr2=0x7fc3340b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:26.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.642+0000 7fc34ae69700 1 --2- 172.21.15.12:0/1403407105 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fc3340b2900 0x7fc3340b4e10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc3380038a0 tx=0x7fc33801a040 comp rx=0 tx=0).stop 2024-05-17T22:38:26.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.642+0000 7fc34ae69700 1 -- 172.21.15.12:0/1403407105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc344109950 msgr2=0x7fc3441a17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:26.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.642+0000 7fc34ae69700 1 --2- 172.21.15.12:0/1403407105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc344109950 0x7fc3441a17f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc33c00ada0 tx=0x7fc33c00c330 comp rx=0 tx=0).stop 2024-05-17T22:38:26.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.642+0000 7fc34ae69700 1 -- 172.21.15.12:0/1403407105 shutdown_connections 2024-05-17T22:38:26.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.642+0000 7fc34ae69700 1 --2- 172.21.15.12:0/1403407105 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fc3340b2900 0x7fc3340b4e10 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:26.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.642+0000 7fc34ae69700 1 --2- 172.21.15.12:0/1403407105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc344109950 0x7fc3441a17f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:26.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.642+0000 7fc34ae69700 1 --2- 172.21.15.12:0/1403407105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc34406cce0 0x7fc34419a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:26.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.642+0000 7fc34ae69700 1 --2- 172.21.15.12:0/1403407105 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc344106df0 0x7fc34419a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:26.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.642+0000 7fc34ae69700 1 -- 172.21.15.12:0/1403407105 >> 172.21.15.12:0/1403407105 conn(0x7fc3440fe3c0 msgr2=0x7fc3441051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:26.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.642+0000 7fc34ae69700 1 -- 172.21.15.12:0/1403407105 shutdown_connections 2024-05-17T22:38:26.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:26.642+0000 7fc34ae69700 1 -- 172.21.15.12:0/1403407105 wait complete. 2024-05-17T22:38:27.358 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-17T22:38:27.367 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi012.front.sepia.ceph.com 2024-05-17T22:38:27.368 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-17T22:38:27.630 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:38:27.874 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:27 smithi012 ceph-mon[31632]: pgmap v415: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T22:38:27.874 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:27 smithi012 ceph-mon[31632]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:38:27.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:27 smithi151 ceph-mon[36101]: pgmap v415: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T22:38:27.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:27 smithi151 ceph-mon[36101]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:38:28.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:27 smithi096 ceph-mon[36065]: pgmap v415: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T22:38:28.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:27 smithi096 ceph-mon[36065]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T22:38:29.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.219+0000 7f4bdb8b3700 1 -- 172.21.15.12:0/2390658789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4bd4109aa0 msgr2=0x7f4bd410bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:29.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.219+0000 7f4bdb8b3700 1 --2- 172.21.15.12:0/2390658789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4bd4109aa0 0x7f4bd410bec0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc00b0a0 tx=0x7f4bcc00b3b0 comp rx=0 tx=0).stop 2024-05-17T22:38:29.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.219+0000 7f4bdb8b3700 1 -- 172.21.15.12:0/2390658789 shutdown_connections 2024-05-17T22:38:29.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.219+0000 7f4bdb8b3700 1 --2- 172.21.15.12:0/2390658789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4bd4109aa0 0x7f4bd410bec0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:29.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.219+0000 7f4bdb8b3700 1 --2- 172.21.15.12:0/2390658789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4bd4101fc0 0x7f4bd41023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:29.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.219+0000 7f4bdb8b3700 1 --2- 172.21.15.12:0/2390658789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4bd4107140 0x7f4bd4109560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:29.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.219+0000 7f4bdb8b3700 1 -- 172.21.15.12:0/2390658789 >> 172.21.15.12:0/2390658789 conn(0x7f4bd40fdb50 msgr2=0x7f4bd40fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:29.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.219+0000 7f4bdb8b3700 1 -- 172.21.15.12:0/2390658789 shutdown_connections 2024-05-17T22:38:29.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.219+0000 7f4bdb8b3700 1 -- 172.21.15.12:0/2390658789 wait complete. 2024-05-17T22:38:29.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.220+0000 7f4bdb8b3700 1 Processor -- start 2024-05-17T22:38:29.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.220+0000 7f4bdb8b3700 1 -- start start 2024-05-17T22:38:29.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.220+0000 7f4bdb8b3700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4bd4101fc0 0x7f4bd419a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:29.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.220+0000 7f4bdb8b3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4bd4107140 0x7f4bd419a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:29.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.220+0000 7f4bdb8b3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4bd4109aa0 0x7f4bd41a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:29.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.220+0000 7f4bdb8b3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd410f6e0 con 0x7f4bd4109aa0 2024-05-17T22:38:29.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.220+0000 7f4bdb8b3700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd410f560 con 0x7f4bd4107140 2024-05-17T22:38:29.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.220+0000 7f4bdb8b3700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd410f860 con 0x7f4bd4101fc0 2024-05-17T22:38:29.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.220+0000 7f4bd964f700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4bd4101fc0 0x7f4bd419a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:29.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.220+0000 7f4bd964f700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4bd4101fc0 0x7f4bd419a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:46606/0 (socket says 172.21.15.12:46606) 2024-05-17T22:38:29.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.220+0000 7f4bd964f700 1 -- 172.21.15.12:0/3673920154 learned_addr learned my addr 172.21.15.12:0/3673920154 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:29.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.220+0000 7f4bd9e50700 1 --2- 172.21.15.12:0/3673920154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4bd4109aa0 0x7f4bd41a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:29.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.220+0000 7f4bd8e4e700 1 --2- 172.21.15.12:0/3673920154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4bd4107140 0x7f4bd419a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:29.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.221+0000 7f4bd964f700 1 -- 172.21.15.12:0/3673920154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4bd4107140 msgr2=0x7f4bd419a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:29.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.221+0000 7f4bd964f700 1 --2- 172.21.15.12:0/3673920154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4bd4107140 0x7f4bd419a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:29.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.221+0000 7f4bd964f700 1 -- 172.21.15.12:0/3673920154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4bd4109aa0 msgr2=0x7f4bd41a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:29.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.221+0000 7f4bd964f700 1 --2- 172.21.15.12:0/3673920154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4bd4109aa0 0x7f4bd41a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:29.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.221+0000 7f4bd964f700 1 -- 172.21.15.12:0/3673920154 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd41a1d60 con 0x7f4bd4101fc0 2024-05-17T22:38:29.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.221+0000 7f4bd8e4e700 1 --2- 172.21.15.12:0/3673920154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4bd4107140 0x7f4bd419a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:38:29.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.221+0000 7f4bd964f700 1 --2- 172.21.15.12:0/3673920154 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4bd4101fc0 0x7f4bd419a200 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4bd000e970 tx=0x7f4bd000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:29.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.221+0000 7f4bca7fc700 1 -- 172.21.15.12:0/3673920154 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd000cbd0 con 0x7f4bd4101fc0 2024-05-17T22:38:29.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.221+0000 7f4bdb8b3700 1 -- 172.21.15.12:0/3673920154 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bd41a1ff0 con 0x7f4bd4101fc0 2024-05-17T22:38:29.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.221+0000 7f4bca7fc700 1 -- 172.21.15.12:0/3673920154 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bd000cd70 con 0x7f4bd4101fc0 2024-05-17T22:38:29.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.221+0000 7f4bca7fc700 1 -- 172.21.15.12:0/3673920154 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0005680 con 0x7f4bd4101fc0 2024-05-17T22:38:29.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.221+0000 7f4bdb8b3700 1 -- 172.21.15.12:0/3673920154 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bd41a24d0 con 0x7f4bd4101fc0 2024-05-17T22:38:29.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.222+0000 7f4bdb8b3700 1 -- 172.21.15.12:0/3673920154 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bd41023d0 con 0x7f4bd4101fc0 2024-05-17T22:38:29.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.224+0000 7f4bca7fc700 1 -- 172.21.15.12:0/3673920154 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4bd0004750 con 0x7f4bd4101fc0 2024-05-17T22:38:29.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.225+0000 7f4bca7fc700 1 --2- 172.21.15.12:0/3673920154 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f4bc00b2880 0x7f4bc00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:29.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.225+0000 7f4bca7fc700 1 -- 172.21.15.12:0/3673920154 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f4bd00e2a40 con 0x7f4bd4101fc0 2024-05-17T22:38:29.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.225+0000 7f4bd8e4e700 1 --2- 172.21.15.12:0/3673920154 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f4bc00b2880 0x7f4bc00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:29.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.225+0000 7f4bd8e4e700 1 --2- 172.21.15.12:0/3673920154 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f4bc00b2880 0x7f4bc00b4d90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4005950 tx=0x7f4bc40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:29.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.228+0000 7f4bca7fc700 1 -- 172.21.15.12:0/3673920154 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4bd0054df0 con 0x7f4bd4101fc0 2024-05-17T22:38:29.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.496+0000 7f4bdb8b3700 1 -- 172.21.15.12:0/3673920154 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f4bd4064720 con 0x7f4bd4101fc0 2024-05-17T22:38:29.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.497+0000 7f4bca7fc700 1 -- 172.21.15.12:0/3673920154 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) ==== 76+0+3050 (secure 0 0 0) 0x7f4bd00aa370 con 0x7f4bd4101fc0 2024-05-17T22:38:29.501 INFO:teuthology.orchestra.run.smithi012.stdout:e26 2024-05-17T22:38:29.501 INFO:teuthology.orchestra.run.smithi012.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-17T22:38:29.501 INFO:teuthology.orchestra.run.smithi012.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-17T22:38:29.501 INFO:teuthology.orchestra.run.smithi012.stdout:legacy client fscid: 1 2024-05-17T22:38:29.501 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:38:29.501 INFO:teuthology.orchestra.run.smithi012.stdout:Filesystem 'cephfs' (1) 2024-05-17T22:38:29.501 INFO:teuthology.orchestra.run.smithi012.stdout:fs_name cephfs 2024-05-17T22:38:29.501 INFO:teuthology.orchestra.run.smithi012.stdout:epoch 26 2024-05-17T22:38:29.502 INFO:teuthology.orchestra.run.smithi012.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-17T22:38:29.502 INFO:teuthology.orchestra.run.smithi012.stdout:created 2024-05-17T22:34:42.642111+0000 2024-05-17T22:38:29.502 INFO:teuthology.orchestra.run.smithi012.stdout:modified 2024-05-17T22:35:52.999468+0000 2024-05-17T22:38:29.502 INFO:teuthology.orchestra.run.smithi012.stdout:tableserver 0 2024-05-17T22:38:29.502 INFO:teuthology.orchestra.run.smithi012.stdout:root 0 2024-05-17T22:38:29.502 INFO:teuthology.orchestra.run.smithi012.stdout:session_timeout 300 2024-05-17T22:38:29.502 INFO:teuthology.orchestra.run.smithi012.stdout:session_autoclose 300 2024-05-17T22:38:29.502 INFO:teuthology.orchestra.run.smithi012.stdout:max_file_size 1099511627776 2024-05-17T22:38:29.502 INFO:teuthology.orchestra.run.smithi012.stdout:max_xattr_size 65536 2024-05-17T22:38:29.505 INFO:teuthology.orchestra.run.smithi012.stdout:required_client_features {} 2024-05-17T22:38:29.505 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure 0 2024-05-17T22:38:29.505 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure_osd_epoch 0 2024-05-17T22:38:29.505 INFO:teuthology.orchestra.run.smithi012.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-17T22:38:29.505 INFO:teuthology.orchestra.run.smithi012.stdout:max_mds 3 2024-05-17T22:38:29.505 INFO:teuthology.orchestra.run.smithi012.stdout:in 0,1,2 2024-05-17T22:38:29.506 INFO:teuthology.orchestra.run.smithi012.stdout:up {0=24482,1=24461,2=24457} 2024-05-17T22:38:29.506 INFO:teuthology.orchestra.run.smithi012.stdout:failed 2024-05-17T22:38:29.506 INFO:teuthology.orchestra.run.smithi012.stdout:damaged 2024-05-17T22:38:29.506 INFO:teuthology.orchestra.run.smithi012.stdout:stopped 2024-05-17T22:38:29.506 INFO:teuthology.orchestra.run.smithi012.stdout:data_pools [3,4] 2024-05-17T22:38:29.506 INFO:teuthology.orchestra.run.smithi012.stdout:metadata_pool 2 2024-05-17T22:38:29.506 INFO:teuthology.orchestra.run.smithi012.stdout:inline_data disabled 2024-05-17T22:38:29.506 INFO:teuthology.orchestra.run.smithi012.stdout:balancer 2024-05-17T22:38:29.506 INFO:teuthology.orchestra.run.smithi012.stdout:bal_rank_mask -1 2024-05-17T22:38:29.506 INFO:teuthology.orchestra.run.smithi012.stdout:standby_count_wanted 0 2024-05-17T22:38:29.506 INFO:teuthology.orchestra.run.smithi012.stdout:qdb_cluster leader: 24482 members: 24457,24482,24461 2024-05-17T22:38:29.506 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.b{0:24482} state up:active seq 5 addr [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T22:38:29.506 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.c{0:24442} state up:standby-replay seq 1 addr [v2:172.21.15.151:6836/3403324038,v1:172.21.15.151:6837/3403324038] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T22:38:29.506 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.h{1:24461} state up:active seq 25 addr [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T22:38:29.507 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.e{1:24437} state up:standby-replay seq 1 addr [v2:172.21.15.96:6834/1758442232,v1:172.21.15.96:6835/1758442232] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T22:38:29.507 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.i{2:24457} state up:active seq 24 addr [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T22:38:29.507 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.f{2:24436} state up:standby-replay seq 1 addr [v2:172.21.15.151:6834/542323754,v1:172.21.15.151:6835/542323754] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T22:38:29.507 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:38:29.507 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:38:29.507 INFO:teuthology.orchestra.run.smithi012.stdout:Standby daemons: 2024-05-17T22:38:29.507 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:38:29.507 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.j{-1:14568} state up:standby seq 1 addr [v2:172.21.15.12:6834/1407219319,v1:172.21.15.12:6835/1407219319] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T22:38:29.507 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.d{-1:14580} state up:standby seq 1 addr [v2:172.21.15.12:6836/630977886,v1:172.21.15.12:6837/630977886] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T22:38:29.507 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.g{-1:14631} state up:standby seq 1 addr [v2:172.21.15.12:6838/3731914742,v1:172.21.15.12:6839/3731914742] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T22:38:29.507 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.a{-1:14643} state up:standby seq 1 addr [v2:172.21.15.12:6840/3642142501,v1:172.21.15.12:6841/3642142501] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T22:38:29.507 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.l{-1:24397} state up:standby seq 1 addr [v2:172.21.15.151:6832/3554053059,v1:172.21.15.151:6833/3554053059] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T22:38:29.507 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.k{-1:24419} state up:standby seq 1 addr [v2:172.21.15.96:6832/4228325569,v1:172.21.15.96:6833/4228325569] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T22:38:29.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.499+0000 7f4bdb8b3700 1 -- 172.21.15.12:0/3673920154 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f4bc00b2880 msgr2=0x7f4bc00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:29.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.500+0000 7f4bdb8b3700 1 --2- 172.21.15.12:0/3673920154 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f4bc00b2880 0x7f4bc00b4d90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4005950 tx=0x7f4bc40058e0 comp rx=0 tx=0).stop 2024-05-17T22:38:29.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.500+0000 7f4bdb8b3700 1 -- 172.21.15.12:0/3673920154 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4bd4101fc0 msgr2=0x7f4bd419a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:29.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.500+0000 7f4bdb8b3700 1 --2- 172.21.15.12:0/3673920154 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4bd4101fc0 0x7f4bd419a200 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4bd000e970 tx=0x7f4bd000ec80 comp rx=0 tx=0).stop 2024-05-17T22:38:29.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.500+0000 7f4bdb8b3700 1 -- 172.21.15.12:0/3673920154 shutdown_connections 2024-05-17T22:38:29.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.500+0000 7f4bdb8b3700 1 --2- 172.21.15.12:0/3673920154 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f4bc00b2880 0x7f4bc00b4d90 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:29.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.500+0000 7f4bdb8b3700 1 --2- 172.21.15.12:0/3673920154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4bd4109aa0 0x7f4bd41a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:29.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.500+0000 7f4bdb8b3700 1 --2- 172.21.15.12:0/3673920154 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4bd4101fc0 0x7f4bd419a200 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:29.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.500+0000 7f4bdb8b3700 1 --2- 172.21.15.12:0/3673920154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4bd4107140 0x7f4bd419a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:29.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.500+0000 7f4bdb8b3700 1 -- 172.21.15.12:0/3673920154 >> 172.21.15.12:0/3673920154 conn(0x7f4bd40fdb50 msgr2=0x7f4bd4107f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:29.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.500+0000 7f4bdb8b3700 1 -- 172.21.15.12:0/3673920154 shutdown_connections 2024-05-17T22:38:29.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:29.500+0000 7f4bdb8b3700 1 -- 172.21.15.12:0/3673920154 wait complete. 2024-05-17T22:38:29.510 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 26 2024-05-17T22:38:29.885 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:29 smithi012 ceph-mon[31632]: pgmap v416: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:38:29.885 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:29 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/3673920154' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T22:38:29.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:29 smithi151 ceph-mon[36101]: pgmap v416: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:38:29.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:29 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/3673920154' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T22:38:30.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:29 smithi096 ceph-mon[36065]: pgmap v416: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:38:30.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:29 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/3673920154' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T22:38:30.218 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-17T22:38:30.469 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:38:31.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:31 smithi012 ceph-mon[31632]: pgmap v417: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T22:38:31.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:31 smithi151 ceph-mon[36101]: pgmap v417: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T22:38:32.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:31 smithi096 ceph-mon[36065]: pgmap v417: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T22:38:32.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.130+0000 7faea9e48700 1 -- 172.21.15.12:0/1727829098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faea4102b90 msgr2=0x7faea4102fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:32.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.130+0000 7faea9e48700 1 --2- 172.21.15.12:0/1727829098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faea4102b90 0x7faea4102fa0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fae940099c0 tx=0x7fae94009cd0 comp rx=0 tx=0).stop 2024-05-17T22:38:32.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.130+0000 7faea9e48700 1 -- 172.21.15.12:0/1727829098 shutdown_connections 2024-05-17T22:38:32.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.130+0000 7faea9e48700 1 --2- 172.21.15.12:0/1727829098 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faea4104660 0x7faea410af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:32.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.130+0000 7faea9e48700 1 --2- 172.21.15.12:0/1727829098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faea4102b90 0x7faea4102fa0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:32.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.130+0000 7faea9e48700 1 --2- 172.21.15.12:0/1727829098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faea4103cc0 0x7faea41040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:32.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.130+0000 7faea9e48700 1 -- 172.21.15.12:0/1727829098 >> 172.21.15.12:0/1727829098 conn(0x7faea40fe3c0 msgr2=0x7faea4100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:32.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.130+0000 7faea9e48700 1 -- 172.21.15.12:0/1727829098 shutdown_connections 2024-05-17T22:38:32.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.130+0000 7faea9e48700 1 -- 172.21.15.12:0/1727829098 wait complete. 2024-05-17T22:38:32.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.131+0000 7faea9e48700 1 Processor -- start 2024-05-17T22:38:32.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.131+0000 7faea9e48700 1 -- start start 2024-05-17T22:38:32.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.131+0000 7faea9e48700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faea4102b90 0x7faea406caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:32.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.131+0000 7faea9e48700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faea4103cc0 0x7faea419f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:32.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.131+0000 7faea9e48700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faea4104660 0x7faea41a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:32.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.131+0000 7faea9e48700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7faea410e5e0 con 0x7faea4102b90 2024-05-17T22:38:32.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.131+0000 7faea9e48700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faea410e460 con 0x7faea4103cc0 2024-05-17T22:38:32.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.131+0000 7faea9e48700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faea410e760 con 0x7faea4104660 2024-05-17T22:38:32.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.131+0000 7faea37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faea4102b90 0x7faea406caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:32.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.131+0000 7faea37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faea4102b90 0x7faea406caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42330/0 (socket says 172.21.15.12:42330) 2024-05-17T22:38:32.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.131+0000 7faea37fe700 1 -- 172.21.15.12:0/3728440537 learned_addr learned my addr 172.21.15.12:0/3728440537 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:32.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.132+0000 7faea2ffd700 1 --2- 172.21.15.12:0/3728440537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faea4103cc0 0x7faea419f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:32.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.132+0000 7faea3fff700 1 --2- 172.21.15.12:0/3728440537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faea4104660 0x7faea41a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:32.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.132+0000 7faea37fe700 1 -- 172.21.15.12:0/3728440537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faea4104660 msgr2=0x7faea41a1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:32.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.132+0000 7faea37fe700 1 --2- 172.21.15.12:0/3728440537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faea4104660 0x7faea41a1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:32.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.132+0000 7faea37fe700 1 -- 172.21.15.12:0/3728440537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faea4103cc0 msgr2=0x7faea419f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:32.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.132+0000 7faea37fe700 1 --2- 172.21.15.12:0/3728440537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faea4103cc0 0x7faea419f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:32.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.132+0000 7faea37fe700 1 -- 172.21.15.12:0/3728440537 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faea41a1d30 con 0x7faea4102b90 2024-05-17T22:38:32.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.132+0000 7faea3fff700 1 --2- 172.21.15.12:0/3728440537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faea4104660 0x7faea41a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:38:32.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.132+0000 7faea2ffd700 1 --2- 172.21.15.12:0/3728440537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faea4103cc0 0x7faea419f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:38:32.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.132+0000 7faea37fe700 1 --2- 172.21.15.12:0/3728440537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faea4102b90 0x7faea406caa0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fae94009f50 tx=0x7fae940047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:32.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.132+0000 7faea0ff9700 1 -- 172.21.15.12:0/3728440537 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae94027070 con 0x7faea4102b90 2024-05-17T22:38:32.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.132+0000 7faea0ff9700 1 -- 172.21.15.12:0/3728440537 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae94022470 con 0x7faea4102b90 2024-05-17T22:38:32.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.132+0000 7faea9e48700 1 -- 172.21.15.12:0/3728440537 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faea41a1fc0 con 0x7faea4102b90 2024-05-17T22:38:32.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.132+0000 7faea0ff9700 1 -- 172.21.15.12:0/3728440537 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9400f6c0 con 0x7faea4102b90 2024-05-17T22:38:32.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.133+0000 7faea9e48700 1 -- 172.21.15.12:0/3728440537 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faea41a2410 con 0x7faea4102b90 2024-05-17T22:38:32.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.134+0000 7faea9e48700 1 -- 172.21.15.12:0/3728440537 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faea4102fa0 con 0x7faea4102b90 2024-05-17T22:38:32.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.135+0000 7faea0ff9700 1 -- 172.21.15.12:0/3728440537 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fae9402a070 con 0x7faea4102b90 2024-05-17T22:38:32.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.136+0000 7faea0ff9700 1 --2- 172.21.15.12:0/3728440537 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fae8c0b2880 0x7fae8c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:32.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.136+0000 7faea2ffd700 1 --2- 172.21.15.12:0/3728440537 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fae8c0b2880 0x7fae8c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:32.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.136+0000 7faea0ff9700 1 -- 172.21.15.12:0/3728440537 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fae940e3840 con 0x7faea4102b90 2024-05-17T22:38:32.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.137+0000 7faea2ffd700 1 --2- 172.21.15.12:0/3728440537 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fae8c0b2880 0x7fae8c0b4d90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fae98005950 tx=0x7fae9800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:32.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.139+0000 7faea0ff9700 1 -- 172.21.15.12:0/3728440537 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fae94013070 con 0x7faea4102b90 2024-05-17T22:38:32.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.361+0000 7faea9e48700 1 -- 172.21.15.12:0/3728440537 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7faea40647e0 con 0x7faea4102b90 2024-05-17T22:38:32.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.362+0000 7faea0ff9700 1 -- 172.21.15.12:0/3728440537 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v89) ==== 56+0+5644 (secure 0 0 0) 0x7fae940ab260 con 0x7faea4102b90 2024-05-17T22:38:32.365 INFO:teuthology.orchestra.run.smithi012.stdout:epoch 89 2024-05-17T22:38:32.365 INFO:teuthology.orchestra.run.smithi012.stdout:fsid 4d593780-149c-11ef-bc9a-c7b262605968 2024-05-17T22:38:32.365 INFO:teuthology.orchestra.run.smithi012.stdout:created 2024-05-17T22:26:16.955626+0000 2024-05-17T22:38:32.366 INFO:teuthology.orchestra.run.smithi012.stdout:modified 2024-05-17T22:36:15.618992+0000 2024-05-17T22:38:32.366 INFO:teuthology.orchestra.run.smithi012.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-17T22:38:32.366 INFO:teuthology.orchestra.run.smithi012.stdout:crush_version 27 2024-05-17T22:38:32.366 INFO:teuthology.orchestra.run.smithi012.stdout:full_ratio 0.95 2024-05-17T22:38:32.366 INFO:teuthology.orchestra.run.smithi012.stdout:backfillfull_ratio 0.9 2024-05-17T22:38:32.366 INFO:teuthology.orchestra.run.smithi012.stdout:nearfull_ratio 0.85 2024-05-17T22:38:32.366 INFO:teuthology.orchestra.run.smithi012.stdout:require_min_compat_client luminous 2024-05-17T22:38:32.366 INFO:teuthology.orchestra.run.smithi012.stdout:min_compat_client luminous 2024-05-17T22:38:32.366 INFO:teuthology.orchestra.run.smithi012.stdout:require_osd_release squid 2024-05-17T22:38:32.366 INFO:teuthology.orchestra.run.smithi012.stdout:stretch_mode_enabled false 2024-05-17T22:38:32.366 INFO:teuthology.orchestra.run.smithi012.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-17T22:38:32.366 INFO:teuthology.orchestra.run.smithi012.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-17T22:38:32.367 INFO:teuthology.orchestra.run.smithi012.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-05-17T22:38:32.367 INFO:teuthology.orchestra.run.smithi012.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-05-17T22:38:32.367 INFO:teuthology.orchestra.run.smithi012.stdout:max_osd 12 2024-05-17T22:38:32.367 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0 up in weight 1 up_from 8 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.12:6802/2015605593,v1:172.21.15.12:6803/2015605593] [v2:172.21.15.12:6804/2015605593,v1:172.21.15.12:6805/2015605593] exists,up 5d42798f-dc5f-4e66-bce2-ec33dc789e4d 2024-05-17T22:38:32.367 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 up in weight 1 up_from 13 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6800/1238316358,v1:172.21.15.96:6801/1238316358] [v2:172.21.15.96:6802/1238316358,v1:172.21.15.96:6803/1238316358] exists,up 58afecc7-119b-4c16-a401-640fb77d4714 2024-05-17T22:38:32.367 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2 up in weight 1 up_from 18 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6800/148729815,v1:172.21.15.151:6801/148729815] [v2:172.21.15.151:6802/148729815,v1:172.21.15.151:6803/148729815] exists,up 4bef4049-482f-49fb-82fb-38052400eb2a 2024-05-17T22:38:32.367 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3 up in weight 1 up_from 25 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.12:6810/3563309320,v1:172.21.15.12:6811/3563309320] [v2:172.21.15.12:6812/3563309320,v1:172.21.15.12:6813/3563309320] exists,up 6fed8e36-82a2-49e4-82a3-16e6d9de87f5 2024-05-17T22:38:32.367 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4 up in weight 1 up_from 31 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6808/2219103166,v1:172.21.15.96:6809/2219103166] [v2:172.21.15.96:6810/2219103166,v1:172.21.15.96:6811/2219103166] exists,up c0e8689a-2816-4089-ade3-931960bae4f3 2024-05-17T22:38:32.367 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5 up in weight 1 up_from 36 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6808/2589835078,v1:172.21.15.151:6809/2589835078] [v2:172.21.15.151:6810/2589835078,v1:172.21.15.151:6811/2589835078] exists,up e76ba432-94ff-4998-aaba-2645d9e6055d 2024-05-17T22:38:32.367 INFO:teuthology.orchestra.run.smithi012.stdout:osd.6 up in weight 1 up_from 42 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.12:6818/3112661724,v1:172.21.15.12:6819/3112661724] [v2:172.21.15.12:6820/3112661724,v1:172.21.15.12:6821/3112661724] exists,up b7ece35c-efd1-40df-95da-05a0cc147372 2024-05-17T22:38:32.367 INFO:teuthology.orchestra.run.smithi012.stdout:osd.7 up in weight 1 up_from 47 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6816/2449989110,v1:172.21.15.96:6817/2449989110] [v2:172.21.15.96:6818/2449989110,v1:172.21.15.96:6819/2449989110] exists,up 6a65bf30-daf5-4fba-b67e-8e513d88a971 2024-05-17T22:38:32.367 INFO:teuthology.orchestra.run.smithi012.stdout:osd.8 up in weight 1 up_from 53 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6816/2545224628,v1:172.21.15.151:6817/2545224628] [v2:172.21.15.151:6818/2545224628,v1:172.21.15.151:6819/2545224628] exists,up ad62ea44-e416-4a3e-bbbe-93f2a710dbc1 2024-05-17T22:38:32.367 INFO:teuthology.orchestra.run.smithi012.stdout:osd.9 up in weight 1 up_from 59 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.12:6826/2100661273,v1:172.21.15.12:6827/2100661273] [v2:172.21.15.12:6828/2100661273,v1:172.21.15.12:6829/2100661273] exists,up 1275de08-2a3c-48f3-920b-dd68b4024d42 2024-05-17T22:38:32.367 INFO:teuthology.orchestra.run.smithi012.stdout:osd.10 up in weight 1 up_from 64 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6824/304016309,v1:172.21.15.96:6825/304016309] [v2:172.21.15.96:6826/304016309,v1:172.21.15.96:6827/304016309] exists,up 347e4e40-9a9d-43fe-8f52-fdd3199a2a5c 2024-05-17T22:38:32.368 INFO:teuthology.orchestra.run.smithi012.stdout:osd.11 up in weight 1 up_from 70 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6824/3822307314,v1:172.21.15.151:6825/3822307314] [v2:172.21.15.151:6826/3822307314,v1:172.21.15.151:6827/3822307314] exists,up ee8d3502-8b3c-4e6f-824f-3a97e19111ae 2024-05-17T22:38:32.368 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 2.4 [2,11] 2024-05-17T22:38:32.368 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 2.f [8,11] 2024-05-17T22:38:32.368 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 2.16 [2,4] 2024-05-17T22:38:32.368 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 3.1 [4,9] 2024-05-17T22:38:32.368 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 3.18 [3,9] 2024-05-17T22:38:32.368 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 3.28 [5,6] 2024-05-17T22:38:32.368 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 3.3a [4,8] 2024-05-17T22:38:32.368 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 3.3c [4,9] 2024-05-17T22:38:32.368 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 4.3 [5,10] 2024-05-17T22:38:32.368 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 4.e [9,11] 2024-05-17T22:38:32.368 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 4.17 [9,8] 2024-05-17T22:38:32.368 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 4.1c [9,8] 2024-05-17T22:38:32.369 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:0/2682700253 expires 2024-05-18T22:27:14.372972+0000 2024-05-17T22:38:32.369 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:6801/2250046521 expires 2024-05-18T22:27:14.372972+0000 2024-05-17T22:38:32.369 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:0/4241094239 expires 2024-05-18T22:26:45.466719+0000 2024-05-17T22:38:32.369 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:0/4172568611 expires 2024-05-18T22:26:45.466719+0000 2024-05-17T22:38:32.369 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:0/3512529242 expires 2024-05-18T22:27:14.372972+0000 2024-05-17T22:38:32.369 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:6800/1039789215 expires 2024-05-18T22:26:45.466719+0000 2024-05-17T22:38:32.369 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:6800/2250046521 expires 2024-05-18T22:27:14.372972+0000 2024-05-17T22:38:32.369 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:6801/1039789215 expires 2024-05-18T22:26:45.466719+0000 2024-05-17T22:38:32.369 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:0/2043758453 expires 2024-05-18T22:27:14.372972+0000 2024-05-17T22:38:32.369 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:0/2694593732 expires 2024-05-18T22:26:45.466719+0000 2024-05-17T22:38:32.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.364+0000 7faea9e48700 1 -- 172.21.15.12:0/3728440537 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fae8c0b2880 msgr2=0x7fae8c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:32.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.364+0000 7faea9e48700 1 --2- 172.21.15.12:0/3728440537 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fae8c0b2880 0x7fae8c0b4d90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fae98005950 tx=0x7fae9800a300 comp rx=0 tx=0).stop 2024-05-17T22:38:32.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.364+0000 7faea9e48700 1 -- 172.21.15.12:0/3728440537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faea4102b90 msgr2=0x7faea406caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:32.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.364+0000 7faea9e48700 1 --2- 172.21.15.12:0/3728440537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faea4102b90 0x7faea406caa0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fae94009f50 tx=0x7fae940047b0 comp rx=0 tx=0).stop 2024-05-17T22:38:32.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.365+0000 7faea9e48700 1 -- 172.21.15.12:0/3728440537 shutdown_connections 2024-05-17T22:38:32.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.365+0000 7faea9e48700 1 --2- 172.21.15.12:0/3728440537 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fae8c0b2880 0x7fae8c0b4d90 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:32.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.365+0000 7faea9e48700 1 --2- 172.21.15.12:0/3728440537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faea4104660 0x7faea41a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:32.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.365+0000 7faea9e48700 1 --2- 172.21.15.12:0/3728440537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faea4102b90 0x7faea406caa0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:32.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.365+0000 7faea9e48700 1 --2- 172.21.15.12:0/3728440537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faea4103cc0 0x7faea419f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:32.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.365+0000 7faea9e48700 1 -- 172.21.15.12:0/3728440537 >> 172.21.15.12:0/3728440537 conn(0x7faea40fe3c0 msgr2=0x7faea4106ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:32.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.365+0000 7faea9e48700 1 -- 172.21.15.12:0/3728440537 shutdown_connections 2024-05-17T22:38:32.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:32.365+0000 7faea9e48700 1 -- 172.21.15.12:0/3728440537 wait complete. 2024-05-17T22:38:32.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:32 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/3728440537' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-17T22:38:33.025 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:32 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/3728440537' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-17T22:38:33.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:32 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/3728440537' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-17T22:38:33.059 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-17T22:38:33.071 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T22:38:33.330 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:38:33.712 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:33 smithi012 ceph-mon[31632]: pgmap v418: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:38:33.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:33 smithi151 ceph-mon[36101]: pgmap v418: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:38:34.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:33 smithi096 ceph-mon[36065]: pgmap v418: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:38:35.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.141+0000 7f0820f41700 1 -- 172.21.15.12:0/1573423179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f081c1088b0 msgr2=0x7f081c108cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:35.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.141+0000 7f0820f41700 1 --2- 172.21.15.12:0/1573423179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f081c1088b0 0x7f081c108cc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0810009990 tx=0x7f0810009ca0 comp rx=0 tx=0).stop 2024-05-17T22:38:35.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.141+0000 7f0820f41700 1 -- 172.21.15.12:0/1573423179 shutdown_connections 2024-05-17T22:38:35.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.141+0000 7f0820f41700 1 --2- 172.21.15.12:0/1573423179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f081c109200 0x7f081c110ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:35.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.141+0000 7f0820f41700 1 --2- 172.21.15.12:0/1573423179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f081c1023a0 0x7f081c1027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:35.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.141+0000 7f0820f41700 1 --2- 172.21.15.12:0/1573423179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f081c1088b0 0x7f081c108cc0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:35.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.141+0000 7f0820f41700 1 -- 172.21.15.12:0/1573423179 >> 172.21.15.12:0/1573423179 conn(0x7f081c0780d0 msgr2=0x7f081c078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:35.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.141+0000 7f0820f41700 1 -- 172.21.15.12:0/1573423179 shutdown_connections 2024-05-17T22:38:35.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.141+0000 7f0820f41700 1 -- 172.21.15.12:0/1573423179 wait complete. 2024-05-17T22:38:35.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.142+0000 7f0820f41700 1 Processor -- start 2024-05-17T22:38:35.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.142+0000 7f0820f41700 1 -- start start 2024-05-17T22:38:35.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.142+0000 7f0820f41700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f081c1023a0 0x7f081c1a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:35.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.142+0000 7f0820f41700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f081c1088b0 0x7f081c1a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:35.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.142+0000 7f0820f41700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f081c109200 0x7f081c19a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:35.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.142+0000 7f0820f41700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f081c113ec0 con 0x7f081c109200 2024-05-17T22:38:35.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.142+0000 7f0820f41700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f081c113d40 con 0x7f081c1088b0 2024-05-17T22:38:35.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.142+0000 7f0820f41700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f081c114040 con 0x7f081c1023a0 2024-05-17T22:38:35.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.143+0000 7f081ad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f081c109200 0x7f081c19a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:35.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.143+0000 7f081ad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f081c109200 0x7f081c19a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42348/0 (socket says 172.21.15.12:42348) 2024-05-17T22:38:35.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.143+0000 7f081ad9d700 1 -- 172.21.15.12:0/3829711974 learned_addr learned my addr 172.21.15.12:0/3829711974 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:35.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.143+0000 7f0819d9b700 1 --2- 172.21.15.12:0/3829711974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f081c1088b0 0x7f081c1a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:35.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.143+0000 7f081a59c700 1 --2- 172.21.15.12:0/3829711974 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f081c1023a0 0x7f081c1a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:35.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.143+0000 7f081ad9d700 1 -- 172.21.15.12:0/3829711974 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f081c1023a0 msgr2=0x7f081c1a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:35.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.143+0000 7f081ad9d700 1 --2- 172.21.15.12:0/3829711974 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f081c1023a0 0x7f081c1a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:35.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.143+0000 7f081ad9d700 1 -- 172.21.15.12:0/3829711974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f081c1088b0 msgr2=0x7f081c1a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:35.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.143+0000 7f081ad9d700 1 --2- 172.21.15.12:0/3829711974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f081c1088b0 0x7f081c1a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:35.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.143+0000 7f081ad9d700 1 -- 172.21.15.12:0/3829711974 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f081c19ae90 con 0x7f081c109200 2024-05-17T22:38:35.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.143+0000 7f081a59c700 1 --2- 172.21.15.12:0/3829711974 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f081c1023a0 0x7f081c1a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:38:35.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.143+0000 7f081ad9d700 1 --2- 172.21.15.12:0/3829711974 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f081c109200 0x7f081c19a850 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0814007d70 tx=0x7f081400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:35.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.143+0000 7f080b7fe700 1 -- 172.21.15.12:0/3829711974 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0814016020 con 0x7f081c109200 2024-05-17T22:38:35.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.143+0000 7f0820f41700 1 -- 172.21.15.12:0/3829711974 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f081c19b180 con 0x7f081c109200 2024-05-17T22:38:35.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.143+0000 7f080b7fe700 1 -- 172.21.15.12:0/3829711974 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f081400ad40 con 0x7f081c109200 2024-05-17T22:38:35.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.143+0000 7f080b7fe700 1 -- 172.21.15.12:0/3829711974 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08140115e0 con 0x7f081c109200 2024-05-17T22:38:35.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.144+0000 7f0820f41700 1 -- 172.21.15.12:0/3829711974 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f081c19b670 con 0x7f081c109200 2024-05-17T22:38:35.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.144+0000 7f0820f41700 1 -- 172.21.15.12:0/3829711974 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f081c1027b0 con 0x7f081c109200 2024-05-17T22:38:35.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.146+0000 7f080b7fe700 1 -- 172.21.15.12:0/3829711974 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0814019020 con 0x7f081c109200 2024-05-17T22:38:35.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.147+0000 7f080b7fe700 1 --2- 172.21.15.12:0/3829711974 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f08040b2880 0x7f08040b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:35.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.147+0000 7f081a59c700 1 --2- 172.21.15.12:0/3829711974 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f08040b2880 0x7f08040b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:35.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.147+0000 7f080b7fe700 1 -- 172.21.15.12:0/3829711974 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f08140e2a40 con 0x7f081c109200 2024-05-17T22:38:35.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.148+0000 7f081a59c700 1 --2- 172.21.15.12:0/3829711974 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f08040b2880 0x7f08040b4d90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f081c108fa0 tx=0x7f080c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:35.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.150+0000 7f080b7fe700 1 -- 172.21.15.12:0/3829711974 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0814020020 con 0x7f081c109200 2024-05-17T22:38:35.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.411+0000 7f0820f41700 1 -- 172.21.15.12:0/3829711974 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f081c02ea80 con 0x7f081c109200 2024-05-17T22:38:35.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.411+0000 7f080b7fe700 1 -- 172.21.15.12:0/3829711974 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11870 (secure 0 0 0) 0x7f08140aa3d0 con 0x7f081c109200 2024-05-17T22:38:35.414 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:38:35.415 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1407219319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1407219319},{"type":"v1","addr":"172.21.15.12:6835","nonce":1407219319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/630977886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":630977886},{"type":"v1","addr":"172.21.15.12:6837","nonce":630977886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3731914742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3731914742},{"type":"v1","addr":"172.21.15.12:6839","nonce":3731914742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/3642142501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3642142501},{"type":"v1","addr":"172.21.15.12:6841","nonce":3642142501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3554053059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3554053059},{"type":"v1","addr":"172.21.15.151:6833","nonce":3554053059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/4228325569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":4228325569},{"type":"v1","addr":"172.21.15.96:6833","nonce":4228325569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T22:34:42.642111+0000","modified":"2024-05-17T22:35:52.999468+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24461,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/542323754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":542323754},{"type":"v1","addr":"172.21.15.151:6835","nonce":542323754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1758442232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1758442232},{"type":"v1","addr":"172.21.15.96:6835","nonce":1758442232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3403324038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3403324038},{"type":"v1","addr":"172.21.15.151:6837","nonce":3403324038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.151:6839/2338353849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2338353849},{"type":"v1","addr":"172.21.15.151:6839","nonce":2338353849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.96:6837/4222574338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":4222574338},{"type":"v1","addr":"172.21.15.96:6837","nonce":4222574338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/2986295519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2986295519},{"type":"v1","addr":"172.21.15.96:6839","nonce":2986295519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24457,24482,24461]},"id":1}]} 2024-05-17T22:38:35.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.413+0000 7f0820f41700 1 -- 172.21.15.12:0/3829711974 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f08040b2880 msgr2=0x7f08040b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:35.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.413+0000 7f0820f41700 1 --2- 172.21.15.12:0/3829711974 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f08040b2880 0x7f08040b4d90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f081c108fa0 tx=0x7f080c009260 comp rx=0 tx=0).stop 2024-05-17T22:38:35.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.413+0000 7f0820f41700 1 -- 172.21.15.12:0/3829711974 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f081c109200 msgr2=0x7f081c19a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:35.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.413+0000 7f0820f41700 1 --2- 172.21.15.12:0/3829711974 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f081c109200 0x7f081c19a850 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0814007d70 tx=0x7f081400a340 comp rx=0 tx=0).stop 2024-05-17T22:38:35.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.414+0000 7f0820f41700 1 -- 172.21.15.12:0/3829711974 shutdown_connections 2024-05-17T22:38:35.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.414+0000 7f0820f41700 1 --2- 172.21.15.12:0/3829711974 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f08040b2880 0x7f08040b4d90 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:35.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.414+0000 7f0820f41700 1 --2- 172.21.15.12:0/3829711974 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f081c109200 0x7f081c19a850 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:35.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.414+0000 7f0820f41700 1 --2- 172.21.15.12:0/3829711974 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f081c1023a0 0x7f081c1a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:35.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.414+0000 7f0820f41700 1 --2- 172.21.15.12:0/3829711974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f081c1088b0 0x7f081c1a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:35.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.414+0000 7f0820f41700 1 -- 172.21.15.12:0/3829711974 >> 172.21.15.12:0/3829711974 conn(0x7f081c0780d0 msgr2=0x7f081c106440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:35.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.414+0000 7f0820f41700 1 -- 172.21.15.12:0/3829711974 shutdown_connections 2024-05-17T22:38:35.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:35.414+0000 7f0820f41700 1 -- 172.21.15.12:0/3829711974 wait complete. 2024-05-17T22:38:35.418 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 26 2024-05-17T22:38:35.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:35 smithi012 ceph-mon[31632]: pgmap v419: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T22:38:35.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:35 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/3829711974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:38:35.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:35 smithi151 ceph-mon[36101]: pgmap v419: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T22:38:35.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:35 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/3829711974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:38:36.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:35 smithi096 ceph-mon[36065]: pgmap v419: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T22:38:36.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:35 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/3829711974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:38:36.219 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T22:38:36.475 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:38:37.965 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:37 smithi012 ceph-mon[31632]: pgmap v420: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T22:38:37.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:37 smithi151 ceph-mon[36101]: pgmap v420: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T22:38:38.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:37 smithi096 ceph-mon[36065]: pgmap v420: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T22:38:38.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.208+0000 7feaed912700 1 -- 172.21.15.12:0/972577108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feae8104da0 msgr2=0x7feae81051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:38.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.208+0000 7feaed912700 1 --2- 172.21.15.12:0/972577108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feae8104da0 0x7feae81051b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fead800b0a0 tx=0x7fead800b3b0 comp rx=0 tx=0).stop 2024-05-17T22:38:38.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.208+0000 7feaed912700 1 -- 172.21.15.12:0/972577108 shutdown_connections 2024-05-17T22:38:38.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.208+0000 7feaed912700 1 --2- 172.21.15.12:0/972577108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feae810e200 0x7feae8110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:38.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.208+0000 7feaed912700 1 --2- 172.21.15.12:0/972577108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feae8104da0 0x7feae81051b0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:38.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.208+0000 7feaed912700 1 --2- 172.21.15.12:0/972577108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feae81056f0 0x7feae810dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:38.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.208+0000 7feaed912700 1 -- 172.21.15.12:0/972577108 >> 172.21.15.12:0/972577108 conn(0x7feae8079bd0 msgr2=0x7feae8079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:38.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.208+0000 7feaed912700 1 -- 172.21.15.12:0/972577108 shutdown_connections 2024-05-17T22:38:38.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.208+0000 7feaed912700 1 -- 172.21.15.12:0/972577108 wait complete. 2024-05-17T22:38:38.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.208+0000 7feaed912700 1 Processor -- start 2024-05-17T22:38:38.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.209+0000 7feaed912700 1 -- start start 2024-05-17T22:38:38.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.209+0000 7feaed912700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feae8104da0 0x7feae819a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:38.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.209+0000 7feaed912700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feae81056f0 0x7feae819ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:38.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.209+0000 7feaed912700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feae810e200 0x7feae819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:38.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.209+0000 7feaed912700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7feae8113d80 con 0x7feae81056f0 2024-05-17T22:38:38.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.209+0000 7feaed912700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7feae8113c00 con 0x7feae8104da0 2024-05-17T22:38:38.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.209+0000 7feaed912700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7feae8113f00 con 0x7feae810e200 2024-05-17T22:38:38.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.209+0000 7feae67fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feae81056f0 0x7feae819ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:38.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.209+0000 7feae67fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feae81056f0 0x7feae819ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42362/0 (socket says 172.21.15.12:42362) 2024-05-17T22:38:38.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.209+0000 7feae67fc700 1 -- 172.21.15.12:0/3871641410 learned_addr learned my addr 172.21.15.12:0/3871641410 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:38.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.209+0000 7feae6ffd700 1 --2- 172.21.15.12:0/3871641410 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feae8104da0 0x7feae819a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:38.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.209+0000 7feae77fe700 1 --2- 172.21.15.12:0/3871641410 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feae810e200 0x7feae819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:38.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.209+0000 7feae67fc700 1 -- 172.21.15.12:0/3871641410 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feae810e200 msgr2=0x7feae819efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:38.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.209+0000 7feae67fc700 1 --2- 172.21.15.12:0/3871641410 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feae810e200 0x7feae819efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:38.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.209+0000 7feae67fc700 1 -- 172.21.15.12:0/3871641410 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feae8104da0 msgr2=0x7feae819a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:38.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.209+0000 7feae67fc700 1 --2- 172.21.15.12:0/3871641410 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feae8104da0 0x7feae819a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:38.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.210+0000 7feae67fc700 1 -- 172.21.15.12:0/3871641410 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feae819f650 con 0x7feae81056f0 2024-05-17T22:38:38.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.210+0000 7feae6ffd700 1 --2- 172.21.15.12:0/3871641410 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feae8104da0 0x7feae819a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:38:38.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.210+0000 7feae67fc700 1 --2- 172.21.15.12:0/3871641410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feae81056f0 0x7feae819ad20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7feadc00c920 tx=0x7feadc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:38.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.210+0000 7feaec910700 1 -- 172.21.15.12:0/3871641410 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc007b00 con 0x7feae81056f0 2024-05-17T22:38:38.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.210+0000 7feaed912700 1 -- 172.21.15.12:0/3871641410 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feae819f940 con 0x7feae81056f0 2024-05-17T22:38:38.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.210+0000 7feaec910700 1 -- 172.21.15.12:0/3871641410 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feadc004ce0 con 0x7feae81056f0 2024-05-17T22:38:38.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.210+0000 7feaec910700 1 -- 172.21.15.12:0/3871641410 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc005650 con 0x7feae81056f0 2024-05-17T22:38:38.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.210+0000 7feaed912700 1 -- 172.21.15.12:0/3871641410 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feae81a7260 con 0x7feae81056f0 2024-05-17T22:38:38.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.211+0000 7feaed912700 1 -- 172.21.15.12:0/3871641410 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feae81051b0 con 0x7feae81056f0 2024-05-17T22:38:38.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.213+0000 7feaec910700 1 -- 172.21.15.12:0/3871641410 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7feadc007ca0 con 0x7feae81056f0 2024-05-17T22:38:38.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.214+0000 7feaec910700 1 --2- 172.21.15.12:0/3871641410 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fead00b2880 0x7fead00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:38.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.214+0000 7feae6ffd700 1 --2- 172.21.15.12:0/3871641410 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fead00b2880 0x7fead00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:38.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.214+0000 7feaec910700 1 -- 172.21.15.12:0/3871641410 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7feadc0e2590 con 0x7feae81056f0 2024-05-17T22:38:38.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.214+0000 7feae6ffd700 1 --2- 172.21.15.12:0/3871641410 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fead00b2880 0x7fead00b4d90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fead8009fd0 tx=0x7fead80098a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:38.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.217+0000 7feaec910700 1 -- 172.21.15.12:0/3871641410 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7feadc014030 con 0x7feae81056f0 2024-05-17T22:38:38.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.482+0000 7feaed912700 1 -- 172.21.15.12:0/3871641410 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feae802ea80 con 0x7feae81056f0 2024-05-17T22:38:38.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.483+0000 7feaec910700 1 -- 172.21.15.12:0/3871641410 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11870 (secure 0 0 0) 0x7feadc0a9f20 con 0x7feae81056f0 2024-05-17T22:38:38.486 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:38:38.486 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1407219319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1407219319},{"type":"v1","addr":"172.21.15.12:6835","nonce":1407219319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/630977886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":630977886},{"type":"v1","addr":"172.21.15.12:6837","nonce":630977886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3731914742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3731914742},{"type":"v1","addr":"172.21.15.12:6839","nonce":3731914742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/3642142501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3642142501},{"type":"v1","addr":"172.21.15.12:6841","nonce":3642142501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3554053059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3554053059},{"type":"v1","addr":"172.21.15.151:6833","nonce":3554053059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/4228325569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":4228325569},{"type":"v1","addr":"172.21.15.96:6833","nonce":4228325569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T22:34:42.642111+0000","modified":"2024-05-17T22:35:52.999468+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24461,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/542323754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":542323754},{"type":"v1","addr":"172.21.15.151:6835","nonce":542323754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1758442232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1758442232},{"type":"v1","addr":"172.21.15.96:6835","nonce":1758442232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3403324038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3403324038},{"type":"v1","addr":"172.21.15.151:6837","nonce":3403324038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.151:6839/2338353849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2338353849},{"type":"v1","addr":"172.21.15.151:6839","nonce":2338353849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.96:6837/4222574338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":4222574338},{"type":"v1","addr":"172.21.15.96:6837","nonce":4222574338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/2986295519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2986295519},{"type":"v1","addr":"172.21.15.96:6839","nonce":2986295519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24457,24482,24461]},"id":1}]} 2024-05-17T22:38:38.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.485+0000 7feaed912700 1 -- 172.21.15.12:0/3871641410 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fead00b2880 msgr2=0x7fead00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:38.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.485+0000 7feaed912700 1 --2- 172.21.15.12:0/3871641410 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fead00b2880 0x7fead00b4d90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fead8009fd0 tx=0x7fead80098a0 comp rx=0 tx=0).stop 2024-05-17T22:38:38.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.485+0000 7feaed912700 1 -- 172.21.15.12:0/3871641410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feae81056f0 msgr2=0x7feae819ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:38.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.485+0000 7feaed912700 1 --2- 172.21.15.12:0/3871641410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feae81056f0 0x7feae819ad20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7feadc00c920 tx=0x7feadc00cce0 comp rx=0 tx=0).stop 2024-05-17T22:38:38.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.485+0000 7feaed912700 1 -- 172.21.15.12:0/3871641410 shutdown_connections 2024-05-17T22:38:38.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.485+0000 7feaed912700 1 --2- 172.21.15.12:0/3871641410 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fead00b2880 0x7fead00b4d90 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:38.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.485+0000 7feaed912700 1 --2- 172.21.15.12:0/3871641410 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feae810e200 0x7feae819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:38.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.485+0000 7feaed912700 1 --2- 172.21.15.12:0/3871641410 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feae8104da0 0x7feae819a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:38.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.485+0000 7feaed912700 1 --2- 172.21.15.12:0/3871641410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feae81056f0 0x7feae819ad20 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:38.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.485+0000 7feaed912700 1 -- 172.21.15.12:0/3871641410 >> 172.21.15.12:0/3871641410 conn(0x7feae8079bd0 msgr2=0x7feae80ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:38.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.485+0000 7feaed912700 1 -- 172.21.15.12:0/3871641410 shutdown_connections 2024-05-17T22:38:38.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:38.485+0000 7feaed912700 1 -- 172.21.15.12:0/3871641410 wait complete. 2024-05-17T22:38:38.490 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 26 2024-05-17T22:38:38.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:38 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/3871641410' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:38:38.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:38 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/3871641410' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:38:39.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:38 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/3871641410' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:38:39.257 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-17T22:38:39.516 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:38:39.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:39 smithi012 ceph-mon[31632]: pgmap v421: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:38:39.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:39 smithi151 ceph-mon[36101]: pgmap v421: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:38:40.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:39 smithi096 ceph-mon[36065]: pgmap v421: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:38:41.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.340+0000 7f276ef22700 1 -- 172.21.15.12:0/3109680802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2768103f10 msgr2=0x7f2768104320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:41.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.340+0000 7f276ef22700 1 --2- 172.21.15.12:0/3109680802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2768103f10 0x7f2768104320 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f275c009990 tx=0x7f275c009ca0 comp rx=0 tx=0).stop 2024-05-17T22:38:41.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.340+0000 7f276ef22700 1 -- 172.21.15.12:0/3109680802 shutdown_connections 2024-05-17T22:38:41.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.340+0000 7f276ef22700 1 --2- 172.21.15.12:0/3109680802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2768109190 0x7f2768110af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:41.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.340+0000 7f276ef22700 1 --2- 172.21.15.12:0/3109680802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2768103f10 0x7f2768104320 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:41.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.340+0000 7f276ef22700 1 --2- 172.21.15.12:0/3109680802 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2768108840 0x7f2768108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:41.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.340+0000 7f276ef22700 1 -- 172.21.15.12:0/3109680802 >> 172.21.15.12:0/3109680802 conn(0x7f27680ff9b0 msgr2=0x7f2768101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:41.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.340+0000 7f276ef22700 1 -- 172.21.15.12:0/3109680802 shutdown_connections 2024-05-17T22:38:41.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.340+0000 7f276ef22700 1 -- 172.21.15.12:0/3109680802 wait complete. 2024-05-17T22:38:41.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.341+0000 7f276ef22700 1 Processor -- start 2024-05-17T22:38:41.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.341+0000 7f276ef22700 1 -- start start 2024-05-17T22:38:41.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.341+0000 7f276ef22700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2768103f10 0x7f27681a0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:41.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.341+0000 7f276ef22700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2768108840 0x7f27681a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:41.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.341+0000 7f276ef22700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2768109190 0x7f276819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:41.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.341+0000 7f276ef22700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2768113e40 con 0x7f2768103f10 2024-05-17T22:38:41.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.341+0000 7f276ef22700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2768113cc0 con 0x7f2768109190 2024-05-17T22:38:41.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.341+0000 7f276ef22700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2768113fc0 con 0x7f2768108840 2024-05-17T22:38:41.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.341+0000 7f276ccbe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2768103f10 0x7f27681a0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:41.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.341+0000 7f276ccbe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2768103f10 0x7f27681a0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47038/0 (socket says 172.21.15.12:47038) 2024-05-17T22:38:41.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.341+0000 7f276ccbe700 1 -- 172.21.15.12:0/1691200102 learned_addr learned my addr 172.21.15.12:0/1691200102 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:41.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.342+0000 7f276d4bf700 1 --2- 172.21.15.12:0/1691200102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2768109190 0x7f276819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:41.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.342+0000 7f276ccbe700 1 -- 172.21.15.12:0/1691200102 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2768108840 msgr2=0x7f27681a0a40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-17T22:38:41.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.342+0000 7f276ccbe700 1 --2- 172.21.15.12:0/1691200102 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2768108840 0x7f27681a0a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:41.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.342+0000 7f276ccbe700 1 -- 172.21.15.12:0/1691200102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2768109190 msgr2=0x7f276819a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:41.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.342+0000 7f276ccbe700 1 --2- 172.21.15.12:0/1691200102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2768109190 0x7f276819a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:41.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.342+0000 7f276ccbe700 1 -- 172.21.15.12:0/1691200102 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f276819adc0 con 0x7f2768103f10 2024-05-17T22:38:41.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.342+0000 7f276ccbe700 1 --2- 172.21.15.12:0/1691200102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2768103f10 0x7f27681a0500 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f275c000c00 tx=0x7f275c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:41.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.342+0000 7f2765ffb700 1 -- 172.21.15.12:0/1691200102 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275c027070 con 0x7f2768103f10 2024-05-17T22:38:41.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.342+0000 7f276ef22700 1 -- 172.21.15.12:0/1691200102 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f276819b050 con 0x7f2768103f10 2024-05-17T22:38:41.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.342+0000 7f2765ffb700 1 -- 172.21.15.12:0/1691200102 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f275c003f20 con 0x7f2768103f10 2024-05-17T22:38:41.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.342+0000 7f2765ffb700 1 -- 172.21.15.12:0/1691200102 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275c022cf0 con 0x7f2768103f10 2024-05-17T22:38:41.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.342+0000 7f276ef22700 1 -- 172.21.15.12:0/1691200102 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27681a72b0 con 0x7f2768103f10 2024-05-17T22:38:41.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.344+0000 7f276ef22700 1 -- 172.21.15.12:0/1691200102 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f276819b530 con 0x7f2768103f10 2024-05-17T22:38:41.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.345+0000 7f2765ffb700 1 -- 172.21.15.12:0/1691200102 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f275c02a070 con 0x7f2768103f10 2024-05-17T22:38:41.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.346+0000 7f2765ffb700 1 --2- 172.21.15.12:0/1691200102 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f27580b2880 0x7f27580b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:41.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.346+0000 7f2767fff700 1 --2- 172.21.15.12:0/1691200102 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f27580b2880 0x7f27580b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:41.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.346+0000 7f2765ffb700 1 -- 172.21.15.12:0/1691200102 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f275c0e37c0 con 0x7f2768103f10 2024-05-17T22:38:41.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.347+0000 7f2767fff700 1 --2- 172.21.15.12:0/1691200102 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f27580b2880 0x7f27580b4d90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f276819be30 tx=0x7f2754006cb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:41.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.349+0000 7f2765ffb700 1 -- 172.21.15.12:0/1691200102 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f275c013070 con 0x7f2768103f10 2024-05-17T22:38:41.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.563+0000 7f276ef22700 1 -- 172.21.15.12:0/1691200102 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2768104320 con 0x7f2768103f10 2024-05-17T22:38:41.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.565+0000 7f2765ffb700 1 -- 172.21.15.12:0/1691200102 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24715 (secure 0 0 0) 0x7f275c0ab180 con 0x7f2768103f10 2024-05-17T22:38:41.568 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:38:41.568 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":89,"fsid":"4d593780-149c-11ef-bc9a-c7b262605968","created":"2024-05-17T22:26:16.955626+0000","modified":"2024-05-17T22:36:15.618992+0000","last_up_change":"2024-05-17T22:33:36.551093+0000","last_in_change":"2024-05-17T22:33:15.032677+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T22:29:42.475513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T22:34:35.504697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T22:34:39.377097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-17T22:34:50.467684+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5d42798f-dc5f-4e66-bce2-ec33dc789e4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6803","nonce":2015605593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6805","nonce":2015605593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6809","nonce":2015605593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6807","nonce":2015605593}]},"public_addr":"172.21.15.12:6803/2015605593","cluster_addr":"172.21.15.12:6805/2015605593","heartbeat_back_addr":"172.21.15.12:6809/2015605593","heartbeat_front_addr":"172.21.15.12:6807/2015605593","state":["exists","up"]},{"osd":1,"uuid":"58afecc7-119b-4c16-a401-640fb77d4714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6801","nonce":1238316358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6803","nonce":1238316358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6807","nonce":1238316358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6805","nonce":1238316358}]},"public_addr":"172.21.15.96:6801/1238316358","cluster_addr":"172.21.15.96:6803/1238316358","heartbeat_back_addr":"172.21.15.96:6807/1238316358","heartbeat_front_addr":"172.21.15.96:6805/1238316358","state":["exists","up"]},{"osd":2,"uuid":"4bef4049-482f-49fb-82fb-38052400eb2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6801","nonce":148729815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6803","nonce":148729815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6807","nonce":148729815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6805","nonce":148729815}]},"public_addr":"172.21.15.151:6801/148729815","cluster_addr":"172.21.15.151:6803/148729815","heartbeat_back_addr":"172.21.15.151:6807/148729815","heartbeat_front_addr":"172.21.15.151:6805/148729815","state":["exists","up"]},{"osd":3,"uuid":"6fed8e36-82a2-49e4-82a3-16e6d9de87f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6811","nonce":3563309320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6813","nonce":3563309320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6817","nonce":3563309320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6815","nonce":3563309320}]},"public_addr":"172.21.15.12:6811/3563309320","cluster_addr":"172.21.15.12:6813/3563309320","heartbeat_back_addr":"172.21.15.12:6817/3563309320","heartbeat_front_addr":"172.21.15.12:6815/3563309320","state":["exists","up"]},{"osd":4,"uuid":"c0e8689a-2816-4089-ade3-931960bae4f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6809","nonce":2219103166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6811","nonce":2219103166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6815","nonce":2219103166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6813","nonce":2219103166}]},"public_addr":"172.21.15.96:6809/2219103166","cluster_addr":"172.21.15.96:6811/2219103166","heartbeat_back_addr":"172.21.15.96:6815/2219103166","heartbeat_front_addr":"172.21.15.96:6813/2219103166","state":["exists","up"]},{"osd":5,"uuid":"e76ba432-94ff-4998-aaba-2645d9e6055d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6809","nonce":2589835078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6811","nonce":2589835078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6815","nonce":2589835078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6813","nonce":2589835078}]},"public_addr":"172.21.15.151:6809/2589835078","cluster_addr":"172.21.15.151:6811/2589835078","heartbeat_back_addr":"172.21.15.151:6815/2589835078","heartbeat_front_addr":"172.21.15.151:6813/2589835078","state":["exists","up"]},{"osd":6,"uuid":"b7ece35c-efd1-40df-95da-05a0cc147372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6819","nonce":3112661724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6821","nonce":3112661724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6825","nonce":3112661724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6823","nonce":3112661724}]},"public_addr":"172.21.15.12:6819/3112661724","cluster_addr":"172.21.15.12:6821/3112661724","heartbeat_back_addr":"172.21.15.12:6825/3112661724","heartbeat_front_addr":"172.21.15.12:6823/3112661724","state":["exists","up"]},{"osd":7,"uuid":"6a65bf30-daf5-4fba-b67e-8e513d88a971","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6817","nonce":2449989110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6819","nonce":2449989110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6823","nonce":2449989110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6821","nonce":2449989110}]},"public_addr":"172.21.15.96:6817/2449989110","cluster_addr":"172.21.15.96:6819/2449989110","heartbeat_back_addr":"172.21.15.96:6823/2449989110","heartbeat_front_addr":"172.21.15.96:6821/2449989110","state":["exists","up"]},{"osd":8,"uuid":"ad62ea44-e416-4a3e-bbbe-93f2a710dbc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6817","nonce":2545224628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6819","nonce":2545224628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6823","nonce":2545224628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6821","nonce":2545224628}]},"public_addr":"172.21.15.151:6817/2545224628","cluster_addr":"172.21.15.151:6819/2545224628","heartbeat_back_addr":"172.21.15.151:6823/2545224628","heartbeat_front_addr":"172.21.15.151:6821/2545224628","state":["exists","up"]},{"osd":9,"uuid":"1275de08-2a3c-48f3-920b-dd68b4024d42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6827","nonce":2100661273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6829","nonce":2100661273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6833","nonce":2100661273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6831","nonce":2100661273}]},"public_addr":"172.21.15.12:6827/2100661273","cluster_addr":"172.21.15.12:6829/2100661273","heartbeat_back_addr":"172.21.15.12:6833/2100661273","heartbeat_front_addr":"172.21.15.12:6831/2100661273","state":["exists","up"]},{"osd":10,"uuid":"347e4e40-9a9d-43fe-8f52-fdd3199a2a5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6825","nonce":304016309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6827","nonce":304016309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6831","nonce":304016309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6829","nonce":304016309}]},"public_addr":"172.21.15.96:6825/304016309","cluster_addr":"172.21.15.96:6827/304016309","heartbeat_back_addr":"172.21.15.96:6831/304016309","heartbeat_front_addr":"172.21.15.96:6829/304016309","state":["exists","up"]},{"osd":11,"uuid":"ee8d3502-8b3c-4e6f-824f-3a97e19111ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6825","nonce":3822307314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6827","nonce":3822307314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6831","nonce":3822307314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6829","nonce":3822307314}]},"public_addr":"172.21.15.151:6825/3822307314","cluster_addr":"172.21.15.151:6827/3822307314","heartbeat_back_addr":"172.21.15.151:6831/3822307314","heartbeat_front_addr":"172.21.15.151:6829/3822307314","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:28:51.542055+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:29:15.566424+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:29:39.500981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:04.797907+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:29.655650+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:55.211880+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:31:21.559462+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:31:47.154852+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:32:13.114892+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:32:40.529708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:33:06.930942+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:33:34.061783+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":10}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/2682700253":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6801/2250046521":"2024-05-18T22:27:14.372972+0000","172.21.15.12:0/4241094239":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/4172568611":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/3512529242":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6800/1039789215":"2024-05-18T22:26:45.466719+0000","172.21.15.12:6800/2250046521":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6801/1039789215":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/2043758453":"2024-05-18T22:27:14.372972+0000","172.21.15.12:0/2694593732":"2024-05-18T22:26:45.466719+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T22:38:41.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.567+0000 7f276ef22700 1 -- 172.21.15.12:0/1691200102 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f27580b2880 msgr2=0x7f27580b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:41.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.567+0000 7f276ef22700 1 --2- 172.21.15.12:0/1691200102 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f27580b2880 0x7f27580b4d90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f276819be30 tx=0x7f2754006cb0 comp rx=0 tx=0).stop 2024-05-17T22:38:41.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.567+0000 7f276ef22700 1 -- 172.21.15.12:0/1691200102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2768103f10 msgr2=0x7f27681a0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:41.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.567+0000 7f276ef22700 1 --2- 172.21.15.12:0/1691200102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2768103f10 0x7f27681a0500 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f275c000c00 tx=0x7f275c0047b0 comp rx=0 tx=0).stop 2024-05-17T22:38:41.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.567+0000 7f276ef22700 1 -- 172.21.15.12:0/1691200102 shutdown_connections 2024-05-17T22:38:41.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.567+0000 7f276ef22700 1 --2- 172.21.15.12:0/1691200102 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f27580b2880 0x7f27580b4d90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:41.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.567+0000 7f276ef22700 1 --2- 172.21.15.12:0/1691200102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2768109190 0x7f276819a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:41.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.567+0000 7f276ef22700 1 --2- 172.21.15.12:0/1691200102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2768103f10 0x7f27681a0500 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:41.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.567+0000 7f276ef22700 1 --2- 172.21.15.12:0/1691200102 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2768108840 0x7f27681a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:41.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.567+0000 7f276ef22700 1 -- 172.21.15.12:0/1691200102 >> 172.21.15.12:0/1691200102 conn(0x7f27680ff9b0 msgr2=0x7f27681000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:41.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.568+0000 7f276ef22700 1 -- 172.21.15.12:0/1691200102 shutdown_connections 2024-05-17T22:38:41.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:41.568+0000 7f276ef22700 1 -- 172.21.15.12:0/1691200102 wait complete. 2024-05-17T22:38:41.968 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:41 smithi012 ceph-mon[31632]: pgmap v422: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T22:38:41.968 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:41 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/1691200102' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:38:41.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:41 smithi151 ceph-mon[36101]: pgmap v422: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T22:38:41.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:41 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/1691200102' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:38:42.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:41 smithi096 ceph-mon[36065]: pgmap v422: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T22:38:42.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:41 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/1691200102' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:38:42.280 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-17T22:34:42.642111+0000', 'modified': '2024-05-17T22:35:52.999468+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24482, 'mds_1': 24461, 'mds_2': 24457}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24436': {'gid': 24436, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.151:6835/542323754', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6834', 'nonce': 542323754}, {'type': 'v1', 'addr': '172.21.15.151:6835', 'nonce': 542323754}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24437': {'gid': 24437, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.96:6835/1758442232', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6834', 'nonce': 1758442232}, {'type': 'v1', 'addr': '172.21.15.96:6835', 'nonce': 1758442232}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.151:6837/3403324038', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6836', 'nonce': 3403324038}, {'type': 'v1', 'addr': '172.21.15.151:6837', 'nonce': 3403324038}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 24, 'addr': '172.21.15.151:6839/2338353849', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6838', 'nonce': 2338353849}, {'type': 'v1', 'addr': '172.21.15.151:6839', 'nonce': 2338353849}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24461': {'gid': 24461, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 25, 'addr': '172.21.15.96:6837/4222574338', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6836', 'nonce': 4222574338}, {'type': 'v1', 'addr': '172.21.15.96:6837', 'nonce': 4222574338}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.96:6839/2986295519', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6838', 'nonce': 2986295519}, {'type': 'v1', 'addr': '172.21.15.96:6839', 'nonce': 2986295519}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24482, 'qdb_cluster': [24457, 24482, 24461]} 2024-05-17T22:38:42.280 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-17T22:38:42.280 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.i status 2024-05-17T22:38:42.533 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:38:43.808 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:43 smithi012 ceph-mon[31632]: pgmap v423: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:38:43.993 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:43 smithi151 ceph-mon[36101]: pgmap v423: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:38:44.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:43 smithi096 ceph-mon[36065]: pgmap v423: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:38:44.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.351+0000 7f6a1ac7b700 1 -- 172.21.15.12:0/449031988 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a14102e70 msgr2=0x7f6a14103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:44.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.351+0000 7f6a1ac7b700 1 --2- 172.21.15.12:0/449031988 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a14102e70 0x7f6a14103280 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6a10009990 tx=0x7f6a10009ca0 comp rx=0 tx=0).stop 2024-05-17T22:38:44.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.351+0000 7f6a1ac7b700 1 -- 172.21.15.12:0/449031988 shutdown_connections 2024-05-17T22:38:44.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.351+0000 7f6a1ac7b700 1 --2- 172.21.15.12:0/449031988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a141080a0 0x7f6a1410fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:44.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.351+0000 7f6a1ac7b700 1 --2- 172.21.15.12:0/449031988 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a14102e70 0x7f6a14103280 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:44.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.351+0000 7f6a1ac7b700 1 --2- 172.21.15.12:0/449031988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a14107750 0x7f6a14107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:44.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.351+0000 7f6a1ac7b700 1 -- 172.21.15.12:0/449031988 >> 172.21.15.12:0/449031988 conn(0x7f6a14079ae0 msgr2=0x7f6a14079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:44.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.351+0000 7f6a1ac7b700 1 -- 172.21.15.12:0/449031988 shutdown_connections 2024-05-17T22:38:44.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.351+0000 7f6a1ac7b700 1 -- 172.21.15.12:0/449031988 wait complete. 2024-05-17T22:38:44.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.352+0000 7f6a1ac7b700 1 Processor -- start 2024-05-17T22:38:44.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.352+0000 7f6a1ac7b700 1 -- start start 2024-05-17T22:38:44.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.352+0000 7f6a1ac7b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a14102e70 0x7f6a140ffa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:44.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.352+0000 7f6a1ac7b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a14107750 0x7f6a140fff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:44.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.352+0000 7f6a1ac7b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a141080a0 0x7f6a141004a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:44.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.352+0000 7f6a1ac7b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6a14112cd0 con 0x7f6a14102e70 2024-05-17T22:38:44.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.352+0000 7f6a1ac7b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6a14112b50 con 0x7f6a141080a0 2024-05-17T22:38:44.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.352+0000 7f6a1ac7b700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6a14112e50 con 0x7f6a14107750 2024-05-17T22:38:44.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.352+0000 7f6a18a17700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a14102e70 0x7f6a140ffa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:44.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.352+0000 7f6a18a17700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a14102e70 0x7f6a140ffa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47046/0 (socket says 172.21.15.12:47046) 2024-05-17T22:38:44.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.352+0000 7f6a18a17700 1 -- 172.21.15.12:0/2604921241 learned_addr learned my addr 172.21.15.12:0/2604921241 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:44.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.352+0000 7f6a19218700 1 --2- 172.21.15.12:0/2604921241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a141080a0 0x7f6a141004a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:44.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.352+0000 7f6a0bfff700 1 --2- 172.21.15.12:0/2604921241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a14107750 0x7f6a140fff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:44.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.353+0000 7f6a18a17700 1 -- 172.21.15.12:0/2604921241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a14107750 msgr2=0x7f6a140fff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:44.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.353+0000 7f6a18a17700 1 --2- 172.21.15.12:0/2604921241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a14107750 0x7f6a140fff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:44.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.353+0000 7f6a18a17700 1 -- 172.21.15.12:0/2604921241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a141080a0 msgr2=0x7f6a141004a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:44.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.353+0000 7f6a18a17700 1 --2- 172.21.15.12:0/2604921241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a141080a0 0x7f6a141004a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:44.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.353+0000 7f6a18a17700 1 -- 172.21.15.12:0/2604921241 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a141a8b60 con 0x7f6a14102e70 2024-05-17T22:38:44.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.353+0000 7f6a19218700 1 --2- 172.21.15.12:0/2604921241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a141080a0 0x7f6a141004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:38:44.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.353+0000 7f6a18a17700 1 --2- 172.21.15.12:0/2604921241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a14102e70 0x7f6a140ffa20 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6a100057c0 tx=0x7f6a100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:44.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.353+0000 7f6a09ffb700 1 -- 172.21.15.12:0/2604921241 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a10027070 con 0x7f6a14102e70 2024-05-17T22:38:44.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.353+0000 7f6a09ffb700 1 -- 172.21.15.12:0/2604921241 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a10004020 con 0x7f6a14102e70 2024-05-17T22:38:44.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.353+0000 7f6a1ac7b700 1 -- 172.21.15.12:0/2604921241 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a141a8df0 con 0x7f6a14102e70 2024-05-17T22:38:44.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.353+0000 7f6a09ffb700 1 -- 172.21.15.12:0/2604921241 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a10022bf0 con 0x7f6a14102e70 2024-05-17T22:38:44.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.353+0000 7f6a1ac7b700 1 -- 172.21.15.12:0/2604921241 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a141a91f0 con 0x7f6a14102e70 2024-05-17T22:38:44.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.356+0000 7f6a09ffb700 1 -- 172.21.15.12:0/2604921241 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6a1002a070 con 0x7f6a14102e70 2024-05-17T22:38:44.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.357+0000 7f6a09ffb700 1 --2- 172.21.15.12:0/2604921241 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f69fc0b2900 0x7f69fc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:44.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.357+0000 7f6a0bfff700 1 --2- 172.21.15.12:0/2604921241 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f69fc0b2900 0x7f69fc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:44.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.357+0000 7f6a09ffb700 1 -- 172.21.15.12:0/2604921241 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f6a100e3510 con 0x7f6a14102e70 2024-05-17T22:38:44.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.357+0000 7f6a29656b80 1 Processor -- start 2024-05-17T22:38:44.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.357+0000 7f6a29656b80 1 -- start start 2024-05-17T22:38:44.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.358+0000 7f6a29656b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583a68da9a0 0x5583a6a5e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:44.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.358+0000 7f6a29656b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583a6a5ea50 0x5583a6a58be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:44.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.358+0000 7f6a29656b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6a59120 0x5583a6a59530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:44.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.358+0000 7f6a0bfff700 1 --2- 172.21.15.12:0/2604921241 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f69fc0b2900 0x7f69fc0b4e10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6a04005fd0 tx=0x7f6a04005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:44.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.358+0000 7f6a29656b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5583a6987240 con 0x5583a6a59120 2024-05-17T22:38:44.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.358+0000 7f6a29656b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5583a6976f80 con 0x5583a6a5ea50 2024-05-17T22:38:44.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.358+0000 7f6a29656b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5583a697ae20 con 0x5583a68da9a0 2024-05-17T22:38:44.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.358+0000 7f6a19218700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6a59120 0x5583a6a59530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:44.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.358+0000 7f6a19218700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6a59120 0x5583a6a59530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47052/0 (socket says 172.21.15.12:47052) 2024-05-17T22:38:44.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.358+0000 7f6a19218700 1 -- 172.21.15.12:0/3437775618 learned_addr learned my addr 172.21.15.12:0/3437775618 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:44.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.358+0000 7f6a18a17700 1 --2- 172.21.15.12:0/3437775618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583a68da9a0 0x5583a6a5e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:44.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.358+0000 7f6a0bfff700 1 --2- 172.21.15.12:0/3437775618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583a6a5ea50 0x5583a6a58be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:44.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.358+0000 7f6a19218700 1 -- 172.21.15.12:0/3437775618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583a68da9a0 msgr2=0x5583a6a5e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:44.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.358+0000 7f6a19218700 1 --2- 172.21.15.12:0/3437775618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583a68da9a0 0x5583a6a5e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:44.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.358+0000 7f6a19218700 1 -- 172.21.15.12:0/3437775618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583a6a5ea50 msgr2=0x5583a6a58be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:44.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.358+0000 7f6a19218700 1 --2- 172.21.15.12:0/3437775618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583a6a5ea50 0x5583a6a58be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:44.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.358+0000 7f6a19218700 1 -- 172.21.15.12:0/3437775618 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583a6920540 con 0x5583a6a59120 2024-05-17T22:38:44.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.359+0000 7f6a19218700 1 --2- 172.21.15.12:0/3437775618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6a59120 0x5583a6a59530 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c009dd0 tx=0x7f6a0c0077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0fa77eab3c76231 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:44.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.359+0000 7f69faffd700 1 -- 172.21.15.12:0/3437775618 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c014610 con 0x5583a6a59120 2024-05-17T22:38:44.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.359+0000 7f69faffd700 1 -- 172.21.15.12:0/3437775618 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a0c019070 con 0x5583a6a59120 2024-05-17T22:38:44.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.359+0000 7f69faffd700 1 -- 172.21.15.12:0/3437775618 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c013a50 con 0x5583a6a59120 2024-05-17T22:38:44.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.360+0000 7f6a29656b80 1 -- 172.21.15.12:0/3437775618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6a59120 msgr2=0x5583a6a59530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:44.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.360+0000 7f6a29656b80 1 --2- 172.21.15.12:0/3437775618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6a59120 0x5583a6a59530 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c009dd0 tx=0x7f6a0c0077e0 comp rx=0 tx=0).stop 2024-05-17T22:38:44.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.360+0000 7f6a29656b80 1 -- 172.21.15.12:0/3437775618 shutdown_connections 2024-05-17T22:38:44.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.360+0000 7f6a29656b80 1 --2- 172.21.15.12:0/3437775618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6a59120 0x5583a6a59530 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:44.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.360+0000 7f6a29656b80 1 --2- 172.21.15.12:0/3437775618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583a68da9a0 0x5583a6a5e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:44.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.360+0000 7f6a29656b80 1 --2- 172.21.15.12:0/3437775618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583a6a5ea50 0x5583a6a58be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:44.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.360+0000 7f6a29656b80 1 -- 172.21.15.12:0/3437775618 >> 172.21.15.12:0/3437775618 conn(0x5583a691eeb0 msgr2=0x5583a6a53b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:44.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.360+0000 7f6a29656b80 1 -- 172.21.15.12:0/3437775618 shutdown_connections 2024-05-17T22:38:44.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.360+0000 7f6a29656b80 1 -- 172.21.15.12:0/3437775618 wait complete. 2024-05-17T22:38:44.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.360+0000 7f6a29656b80 1 -- start start 2024-05-17T22:38:44.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.360+0000 7f6a29656b80 1 Processor -- start 2024-05-17T22:38:44.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.361+0000 7f6a29656b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583a68da9a0 0x5583a6a62100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:44.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.361+0000 7f6a29656b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583a6a59120 0x5583a6a62640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:44.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.361+0000 7f6a29656b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6a5ea50 0x5583a6a62b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:44.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.361+0000 7f6a29656b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5583a687e100 con 0x5583a6a5ea50 2024-05-17T22:38:44.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.361+0000 7f6a29656b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5583a6917e30 con 0x5583a68da9a0 2024-05-17T22:38:44.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.361+0000 7f6a29656b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5583a6a65340 con 0x5583a6a59120 2024-05-17T22:38:44.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.361+0000 7f6a29656b80 10 client.0 _ll_register_callbacks cb 0x5583a6911ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:38:44.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.361+0000 7f6a19218700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6a5ea50 0x5583a6a62b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:44.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.361+0000 7f6a19218700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6a5ea50 0x5583a6a62b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47056/0 (socket says 172.21.15.12:47056) 2024-05-17T22:38:44.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.361+0000 7f6a19218700 1 -- 172.21.15.12:0/2494218354 learned_addr learned my addr 172.21.15.12:0/2494218354 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:44.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.361+0000 7f6a18a17700 1 --2- 172.21.15.12:0/2494218354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583a68da9a0 0x5583a6a62100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:44.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.361+0000 7f6a0bfff700 1 --2- 172.21.15.12:0/2494218354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583a6a59120 0x5583a6a62640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:44.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.361+0000 7f6a19218700 1 -- 172.21.15.12:0/2494218354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583a6a59120 msgr2=0x5583a6a62640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:44.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.361+0000 7f6a19218700 1 --2- 172.21.15.12:0/2494218354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583a6a59120 0x5583a6a62640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:44.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.361+0000 7f6a19218700 1 -- 172.21.15.12:0/2494218354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583a68da9a0 msgr2=0x5583a6a62100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:44.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.361+0000 7f6a19218700 1 --2- 172.21.15.12:0/2494218354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583a68da9a0 0x5583a6a62100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:44.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.361+0000 7f6a19218700 1 -- 172.21.15.12:0/2494218354 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583a68c7430 con 0x5583a6a5ea50 2024-05-17T22:38:44.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.362+0000 7f6a19218700 1 --2- 172.21.15.12:0/2494218354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6a5ea50 0x5583a6a62b80 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c0133b0 tx=0x7f6a0c013e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9254a60b9d72c6d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:44.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.362+0000 7f69f3fff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:38:44.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.362+0000 7f69f3fff700 1 -- 172.21.15.12:0/2494218354 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c022070 con 0x5583a6a5ea50 2024-05-17T22:38:44.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.362+0000 7f69f3fff700 1 -- 172.21.15.12:0/2494218354 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f6a0c027460 con 0x5583a6a5ea50 2024-05-17T22:38:44.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.362+0000 7f6a29656b80 1 -- 172.21.15.12:0/2494218354 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a0c019070 con 0x5583a6a5ea50 2024-05-17T22:38:44.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.362+0000 7f69f3fff700 1 -- 172.21.15.12:0/2494218354 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6a0c030050 con 0x5583a6a5ea50 2024-05-17T22:38:44.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.362+0000 7f6a29656b80 10 client.15231 fetch_fsmap learned FSMap version 26 2024-05-17T22:38:44.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.362+0000 7f6a29656b80 1 -- 172.21.15.12:0/2494218354 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5583a6a6fd00 con 0x5583a6a5ea50 2024-05-17T22:38:44.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.362+0000 7f69f3fff700 1 -- 172.21.15.12:0/2494218354 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a0c019070 con 0x5583a6a5ea50 2024-05-17T22:38:44.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.362+0000 7f69f3fff700 1 -- 172.21.15.12:0/2494218354 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f6a0c005050 con 0x5583a6a5ea50 2024-05-17T22:38:44.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.363+0000 7f6a29656b80 10 client.15231 fetch_fsmap finished waiting for FSMap version 26 2024-05-17T22:38:44.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.363+0000 7f6a29656b80 10 client.15231 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-17T22:38:44.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.363+0000 7f6a29656b80 20 client.15231 populate_metadata read hostname 'smithi012' 2024-05-17T22:38:44.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.363+0000 7f6a29656b80 1 --2- 172.21.15.12:0/2494218354 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x5583a6a72000 0x5583a6a73470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:44.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.363+0000 7f6a29656b80 4 client.15231 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:38:44.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.363+0000 7f6a29656b80 1 -- 172.21.15.12:0/2494218354 --> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583a6989190 con 0x5583a6a72000 2024-05-17T22:38:44.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.363+0000 7f6a18a17700 1 --2- 172.21.15.12:0/2494218354 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x5583a6a72000 0x5583a6a73470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:44.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.363+0000 7f6a18a17700 1 --2- 172.21.15.12:0/2494218354 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x5583a6a72000 0x5583a6a73470 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=96f5599aafc89e32 server_cookie=f4369e378ac3ef2e in_seq=0 out_seq=0 2024-05-17T22:38:44.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.363+0000 7f69f3fff700 10 client.15231 ms_handle_connect on v2:172.21.15.151:6838/2338353849 2024-05-17T22:38:44.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.364+0000 7f69f3fff700 1 -- 172.21.15.12:0/2494218354 <== mds.2 v2:172.21.15.151:6838/2338353849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5583a6989190 con 0x5583a6a72000 2024-05-17T22:38:44.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.364+0000 7f69f3fff700 10 client.15231 handle_command_reply: tid=0 2024-05-17T22:38:44.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.364+0000 7f6a29656b80 1 client.15231 shutdown 2024-05-17T22:38:44.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.364+0000 7f6a29656b80 1 -- 172.21.15.12:0/2494218354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6a5ea50 msgr2=0x5583a6a62b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:44.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.364+0000 7f6a29656b80 1 --2- 172.21.15.12:0/2494218354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6a5ea50 0x5583a6a62b80 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c0133b0 tx=0x7f6a0c013e70 comp rx=0 tx=0).stop 2024-05-17T22:38:44.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.364+0000 7f6a29656b80 1 -- 172.21.15.12:0/2494218354 shutdown_connections 2024-05-17T22:38:44.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.364+0000 7f6a29656b80 1 --2- 172.21.15.12:0/2494218354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6a5ea50 0x5583a6a62b80 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:44.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.364+0000 7f6a29656b80 1 --2- 172.21.15.12:0/2494218354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583a68da9a0 0x5583a6a62100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:44.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.364+0000 7f6a29656b80 1 --2- 172.21.15.12:0/2494218354 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x5583a6a72000 0x5583a6a73470 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:44.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.364+0000 7f6a29656b80 1 --2- 172.21.15.12:0/2494218354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583a6a59120 0x5583a6a62640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:44.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.364+0000 7f6a29656b80 1 -- 172.21.15.12:0/2494218354 >> 172.21.15.12:0/2494218354 conn(0x5583a691eeb0 msgr2=0x5583a6a4de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:44.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.364+0000 7f69f3fff700 0 client.15231 ms_handle_reset on v2:172.21.15.151:6838/2338353849 2024-05-17T22:38:44.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.365+0000 7f6a29656b80 1 -- 172.21.15.12:0/2494218354 shutdown_connections 2024-05-17T22:38:44.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.365+0000 7f6a29656b80 1 -- 172.21.15.12:0/2494218354 wait complete. 2024-05-17T22:38:44.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.365+0000 7f6a29656b80 20 client.15231 trim_cache size 0 max 16384 2024-05-17T22:38:44.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.386+0000 7f6a29656b80 1 Processor -- start 2024-05-17T22:38:44.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.386+0000 7f6a29656b80 1 -- start start 2024-05-17T22:38:44.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.386+0000 7f6a29656b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583a68da9a0 0x5583a6a58be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:44.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.386+0000 7f6a29656b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583a6a59120 0x5583a6a6e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:44.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.386+0000 7f6a29656b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6a5ea50 0x5583a6a711c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:44.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.386+0000 7f6a29656b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5583a6a717b0 con 0x5583a6a5ea50 2024-05-17T22:38:44.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.386+0000 7f6a29656b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5583a6a71e80 con 0x5583a68da9a0 2024-05-17T22:38:44.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.386+0000 7f6a29656b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5583a6a72550 con 0x5583a6a59120 2024-05-17T22:38:44.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.386+0000 7f6a19218700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6a5ea50 0x5583a6a711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:44.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.386+0000 7f6a19218700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6a5ea50 0x5583a6a711c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47066/0 (socket says 172.21.15.12:47066) 2024-05-17T22:38:44.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.387+0000 7f6a19218700 1 -- 172.21.15.12:0/1475915908 learned_addr learned my addr 172.21.15.12:0/1475915908 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:44.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.387+0000 7f6a0bfff700 1 --2- 172.21.15.12:0/1475915908 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583a6a59120 0x5583a6a6e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:44.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.387+0000 7f6a18a17700 1 --2- 172.21.15.12:0/1475915908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583a68da9a0 0x5583a6a58be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:44.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.387+0000 7f6a19218700 1 -- 172.21.15.12:0/1475915908 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583a6a59120 msgr2=0x5583a6a6e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:44.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.387+0000 7f6a19218700 1 --2- 172.21.15.12:0/1475915908 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583a6a59120 0x5583a6a6e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:44.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.387+0000 7f6a19218700 1 -- 172.21.15.12:0/1475915908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583a68da9a0 msgr2=0x5583a6a58be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:44.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.387+0000 7f6a19218700 1 --2- 172.21.15.12:0/1475915908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583a68da9a0 0x5583a6a58be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:44.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.387+0000 7f6a19218700 1 -- 172.21.15.12:0/1475915908 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583a6a60670 con 0x5583a6a5ea50 2024-05-17T22:38:44.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.387+0000 7f6a19218700 1 --2- 172.21.15.12:0/1475915908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6a5ea50 0x5583a6a711c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c009cf0 tx=0x7f6a0c0077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42bf5b6c3719fc48 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:44.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.387+0000 7f69fa7fc700 1 -- 172.21.15.12:0/1475915908 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c022030 con 0x5583a6a5ea50 2024-05-17T22:38:44.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.387+0000 7f69fa7fc700 1 -- 172.21.15.12:0/1475915908 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a0c019040 con 0x5583a6a5ea50 2024-05-17T22:38:44.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.387+0000 7f69fa7fc700 1 -- 172.21.15.12:0/1475915908 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c0051f0 con 0x5583a6a5ea50 2024-05-17T22:38:44.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.388+0000 7f6a29656b80 1 -- 172.21.15.12:0/1475915908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6a5ea50 msgr2=0x5583a6a711c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:44.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.388+0000 7f6a29656b80 1 --2- 172.21.15.12:0/1475915908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6a5ea50 0x5583a6a711c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c009cf0 tx=0x7f6a0c0077e0 comp rx=0 tx=0).stop 2024-05-17T22:38:44.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.388+0000 7f6a29656b80 1 -- 172.21.15.12:0/1475915908 shutdown_connections 2024-05-17T22:38:44.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.388+0000 7f6a29656b80 1 --2- 172.21.15.12:0/1475915908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a6a5ea50 0x5583a6a711c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:44.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.388+0000 7f6a29656b80 1 --2- 172.21.15.12:0/1475915908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583a68da9a0 0x5583a6a58be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:44.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.388+0000 7f6a29656b80 1 --2- 172.21.15.12:0/1475915908 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583a6a59120 0x5583a6a6e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:44.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.388+0000 7f6a29656b80 1 -- 172.21.15.12:0/1475915908 >> 172.21.15.12:0/1475915908 conn(0x5583a691eeb0 msgr2=0x5583a6a5bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:44.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.388+0000 7f6a29656b80 1 -- 172.21.15.12:0/1475915908 shutdown_connections 2024-05-17T22:38:44.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.388+0000 7f6a29656b80 1 -- 172.21.15.12:0/1475915908 wait complete. 2024-05-17T22:38:44.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.388+0000 7f6a29656b80 1 -- start start 2024-05-17T22:38:44.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.389+0000 7f6a29656b80 1 Processor -- start 2024-05-17T22:38:44.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.389+0000 7f6a29656b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a68da9a0 0x5583a6a618a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:44.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.389+0000 7f6a29656b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583a6a59120 0x5583a6a61de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:44.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.389+0000 7f6a29656b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583a6a5ea50 0x5583a6a7b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:44.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.389+0000 7f6a29656b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5583a6a732f0 con 0x5583a68da9a0 2024-05-17T22:38:44.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.389+0000 7f6a29656b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5583a6a72c20 con 0x5583a6a5ea50 2024-05-17T22:38:44.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.389+0000 7f6a29656b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5583a6a739c0 con 0x5583a6a59120 2024-05-17T22:38:44.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.389+0000 7f6a29656b80 10 client.0 _ll_register_callbacks cb 0x5583a6911ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:38:44.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.389+0000 7f6a18a17700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a68da9a0 0x5583a6a618a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:44.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.389+0000 7f6a18a17700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a68da9a0 0x5583a6a618a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47068/0 (socket says 172.21.15.12:47068) 2024-05-17T22:38:44.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.389+0000 7f6a18a17700 1 -- 172.21.15.12:0/4097136305 learned_addr learned my addr 172.21.15.12:0/4097136305 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:44.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.389+0000 7f6a19218700 1 --2- 172.21.15.12:0/4097136305 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583a6a5ea50 0x5583a6a7b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:44.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.389+0000 7f6a0bfff700 1 --2- 172.21.15.12:0/4097136305 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583a6a59120 0x5583a6a61de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:44.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.390+0000 7f6a18a17700 1 -- 172.21.15.12:0/4097136305 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583a6a59120 msgr2=0x5583a6a61de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:44.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.390+0000 7f6a18a17700 1 --2- 172.21.15.12:0/4097136305 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583a6a59120 0x5583a6a61de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:44.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.390+0000 7f6a18a17700 1 -- 172.21.15.12:0/4097136305 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583a6a5ea50 msgr2=0x5583a6a7b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:44.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.390+0000 7f6a18a17700 1 --2- 172.21.15.12:0/4097136305 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583a6a5ea50 0x5583a6a7b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:44.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.390+0000 7f6a18a17700 1 -- 172.21.15.12:0/4097136305 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583a6a7be50 con 0x5583a68da9a0 2024-05-17T22:38:44.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.390+0000 7f6a19218700 1 --2- 172.21.15.12:0/4097136305 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583a6a5ea50 0x5583a6a7b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:38:44.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.390+0000 7f6a18a17700 1 --2- 172.21.15.12:0/4097136305 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a68da9a0 0x5583a6a618a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6a100b3070 tx=0x7f6a1000b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e6e4889db24b2f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:44.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.390+0000 7f69f3fff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:38:44.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.390+0000 7f69f3fff700 1 -- 172.21.15.12:0/4097136305 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a100aeb80 con 0x5583a68da9a0 2024-05-17T22:38:44.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.390+0000 7f6a29656b80 1 -- 172.21.15.12:0/4097136305 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a0c019040 con 0x5583a68da9a0 2024-05-17T22:38:44.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.390+0000 7f69f3fff700 1 -- 172.21.15.12:0/4097136305 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f6a100e0d30 con 0x5583a68da9a0 2024-05-17T22:38:44.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.390+0000 7f69f3fff700 1 -- 172.21.15.12:0/4097136305 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6a100b35a0 con 0x5583a68da9a0 2024-05-17T22:38:44.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.390+0000 7f6a29656b80 10 client.15237 fetch_fsmap learned FSMap version 26 2024-05-17T22:38:44.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.390+0000 7f6a29656b80 1 -- 172.21.15.12:0/4097136305 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5583a6a7d070 con 0x5583a68da9a0 2024-05-17T22:38:44.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.390+0000 7f69f3fff700 1 -- 172.21.15.12:0/4097136305 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a0c019040 con 0x5583a68da9a0 2024-05-17T22:38:44.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.391+0000 7f69f3fff700 1 -- 172.21.15.12:0/4097136305 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f6a100abb00 con 0x5583a68da9a0 2024-05-17T22:38:44.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.391+0000 7f6a29656b80 10 client.15237 fetch_fsmap finished waiting for FSMap version 26 2024-05-17T22:38:44.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.391+0000 7f6a29656b80 10 client.15237 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-17T22:38:44.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.391+0000 7f6a29656b80 20 client.15237 populate_metadata read hostname 'smithi012' 2024-05-17T22:38:44.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.391+0000 7f6a29656b80 1 --2- 172.21.15.12:0/4097136305 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x5583a6adf3e0 0x5583a6ae0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:44.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.391+0000 7f6a29656b80 4 client.15237 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-17T22:38:44.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.391+0000 7f6a29656b80 1 -- 172.21.15.12:0/4097136305 --> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] -- command(tid 0: {"prefix": "status"}) -- 0x5583a6982970 con 0x5583a6adf3e0 2024-05-17T22:38:44.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.391+0000 7f6a0bfff700 1 --2- 172.21.15.12:0/4097136305 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x5583a6adf3e0 0x5583a6ae0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:44.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.391+0000 7f6a0bfff700 1 --2- 172.21.15.12:0/4097136305 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x5583a6adf3e0 0x5583a6ae0850 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3eefa664d36c8c0d server_cookie=ea2d5118e19dc39 in_seq=0 out_seq=0 2024-05-17T22:38:44.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.391+0000 7f69f3fff700 10 client.15237 ms_handle_connect on v2:172.21.15.151:6838/2338353849 2024-05-17T22:38:44.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.392+0000 7f69f3fff700 1 -- 172.21.15.12:0/4097136305 <== mds.2 v2:172.21.15.151:6838/2338353849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5583a6982970 con 0x5583a6adf3e0 2024-05-17T22:38:44.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.392+0000 7f69f3fff700 10 client.15237 handle_command_reply: tid=0 2024-05-17T22:38:44.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.392+0000 7f6a29656b80 1 client.15237 shutdown 2024-05-17T22:38:44.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.392+0000 7f6a29656b80 1 -- 172.21.15.12:0/4097136305 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a68da9a0 msgr2=0x5583a6a618a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:44.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.392+0000 7f6a29656b80 1 --2- 172.21.15.12:0/4097136305 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a68da9a0 0x5583a6a618a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6a100b3070 tx=0x7f6a1000b540 comp rx=0 tx=0).stop 2024-05-17T22:38:44.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.392+0000 7f6a29656b80 1 -- 172.21.15.12:0/4097136305 shutdown_connections 2024-05-17T22:38:44.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.392+0000 7f6a29656b80 1 --2- 172.21.15.12:0/4097136305 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583a6a5ea50 0x5583a6a7b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:44.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.392+0000 7f6a29656b80 1 --2- 172.21.15.12:0/4097136305 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583a68da9a0 0x5583a6a618a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:44.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.392+0000 7f6a29656b80 1 --2- 172.21.15.12:0/4097136305 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x5583a6adf3e0 0x5583a6ae0850 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:44.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.392+0000 7f6a29656b80 1 --2- 172.21.15.12:0/4097136305 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583a6a59120 0x5583a6a61de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:44.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.392+0000 7f69f3fff700 0 client.15237 ms_handle_reset on v2:172.21.15.151:6838/2338353849 2024-05-17T22:38:44.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.392+0000 7f6a29656b80 1 -- 172.21.15.12:0/4097136305 >> 172.21.15.12:0/4097136305 conn(0x5583a691eeb0 msgr2=0x5583a6a6e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:44.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.392+0000 7f6a29656b80 1 -- 172.21.15.12:0/4097136305 shutdown_connections 2024-05-17T22:38:44.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.392+0000 7f6a29656b80 1 -- 172.21.15.12:0/4097136305 wait complete. 2024-05-17T22:38:44.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.392+0000 7f6a29656b80 20 client.15237 trim_cache size 0 max 16384 2024-05-17T22:38:44.401 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:38:44.401 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "4d593780-149c-11ef-bc9a-c7b262605968", 2024-05-17T22:38:44.402 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 2, 2024-05-17T22:38:44.402 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24457, 2024-05-17T22:38:44.402 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-05-17T22:38:44.402 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-05-17T22:38:44.402 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-05-17T22:38:44.402 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 176.37493879499999, 2024-05-17T22:38:44.402 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 26, 2024-05-17T22:38:44.402 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 89, 2024-05-17T22:38:44.402 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 87, 2024-05-17T22:38:44.402 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 266.47804737299998 2024-05-17T22:38:44.402 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:38:44.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.393+0000 7f6a1ac7b700 1 -- 172.21.15.12:0/2604921241 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f69fc0b2900 msgr2=0x7f69fc0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:44.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.393+0000 7f6a1ac7b700 1 --2- 172.21.15.12:0/2604921241 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f69fc0b2900 0x7f69fc0b4e10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6a04005fd0 tx=0x7f6a04005f60 comp rx=0 tx=0).stop 2024-05-17T22:38:44.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.393+0000 7f6a1ac7b700 1 -- 172.21.15.12:0/2604921241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a14102e70 msgr2=0x7f6a140ffa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:44.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.393+0000 7f6a1ac7b700 1 --2- 172.21.15.12:0/2604921241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a14102e70 0x7f6a140ffa20 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6a100057c0 tx=0x7f6a100057f0 comp rx=0 tx=0).stop 2024-05-17T22:38:44.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.393+0000 7f6a1ac7b700 1 -- 172.21.15.12:0/2604921241 shutdown_connections 2024-05-17T22:38:44.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.393+0000 7f6a1ac7b700 1 --2- 172.21.15.12:0/2604921241 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f69fc0b2900 0x7f69fc0b4e10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:44.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.393+0000 7f6a1ac7b700 1 --2- 172.21.15.12:0/2604921241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a141080a0 0x7f6a141004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:44.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.393+0000 7f6a1ac7b700 1 --2- 172.21.15.12:0/2604921241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a14102e70 0x7f6a140ffa20 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:44.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.393+0000 7f6a1ac7b700 1 --2- 172.21.15.12:0/2604921241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a14107750 0x7f6a140fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:44.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.393+0000 7f6a1ac7b700 1 -- 172.21.15.12:0/2604921241 >> 172.21.15.12:0/2604921241 conn(0x7f6a14079ae0 msgr2=0x7f6a14106600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:44.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.393+0000 7f6a1ac7b700 1 -- 172.21.15.12:0/2604921241 shutdown_connections 2024-05-17T22:38:44.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:44.393+0000 7f6a1ac7b700 1 -- 172.21.15.12:0/2604921241 wait complete. 2024-05-17T22:38:45.201 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.h status 2024-05-17T22:38:45.461 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:38:45.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:45 smithi012 ceph-mon[31632]: pgmap v424: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T22:38:45.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:45 smithi151 ceph-mon[36101]: pgmap v424: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T22:38:46.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:45 smithi096 ceph-mon[36065]: pgmap v424: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T22:38:47.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.407+0000 7f84a0271700 1 -- 172.21.15.12:0/3464076115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8498107750 msgr2=0x7f8498107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:47.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.407+0000 7f84a0271700 1 --2- 172.21.15.12:0/3464076115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8498107750 0x7f8498107b60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8488009990 tx=0x7f8488009ca0 comp rx=0 tx=0).stop 2024-05-17T22:38:47.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.407+0000 7f84a0271700 1 -- 172.21.15.12:0/3464076115 shutdown_connections 2024-05-17T22:38:47.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.407+0000 7f84a0271700 1 --2- 172.21.15.12:0/3464076115 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84981080a0 0x7f849810fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:47.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.407+0000 7f84a0271700 1 --2- 172.21.15.12:0/3464076115 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8498102e70 0x7f8498103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:47.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.407+0000 7f84a0271700 1 --2- 172.21.15.12:0/3464076115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8498107750 0x7f8498107b60 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:47.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.407+0000 7f84a0271700 1 -- 172.21.15.12:0/3464076115 >> 172.21.15.12:0/3464076115 conn(0x7f8498079ae0 msgr2=0x7f8498079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:47.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.407+0000 7f84a0271700 1 -- 172.21.15.12:0/3464076115 shutdown_connections 2024-05-17T22:38:47.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.407+0000 7f84a0271700 1 -- 172.21.15.12:0/3464076115 wait complete. 2024-05-17T22:38:47.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.408+0000 7f84a0271700 1 Processor -- start 2024-05-17T22:38:47.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.408+0000 7f84a0271700 1 -- start start 2024-05-17T22:38:47.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.408+0000 7f84a0271700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8498102e70 0x7f84981a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:47.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.408+0000 7f84a0271700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8498107750 0x7f84981a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:47.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.408+0000 7f84a0271700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84981080a0 0x7f849819a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:47.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.408+0000 7f84a0271700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8498112d00 con 0x7f8498107750 2024-05-17T22:38:47.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.408+0000 7f84a0271700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8498112b80 con 0x7f84981080a0 2024-05-17T22:38:47.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.408+0000 7f84a0271700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8498112e80 con 0x7f8498102e70 2024-05-17T22:38:47.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.408+0000 7f849d80c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8498107750 0x7f84981a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:47.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.408+0000 7f849d80c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8498107750 0x7f84981a0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47086/0 (socket says 172.21.15.12:47086) 2024-05-17T22:38:47.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.408+0000 7f849d80c700 1 -- 172.21.15.12:0/1965871189 learned_addr learned my addr 172.21.15.12:0/1965871189 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:47.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.408+0000 7f849e00d700 1 --2- 172.21.15.12:0/1965871189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8498102e70 0x7f84981a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:47.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.408+0000 7f849d80c700 1 -- 172.21.15.12:0/1965871189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8498102e70 msgr2=0x7f84981a04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:47.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.408+0000 7f849d80c700 1 --2- 172.21.15.12:0/1965871189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8498102e70 0x7f84981a04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:47.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.408+0000 7f849e80e700 1 --2- 172.21.15.12:0/1965871189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84981080a0 0x7f849819a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:47.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.408+0000 7f849d80c700 1 -- 172.21.15.12:0/1965871189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84981080a0 msgr2=0x7f849819a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:47.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.408+0000 7f849d80c700 1 --2- 172.21.15.12:0/1965871189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84981080a0 0x7f849819a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:47.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.408+0000 7f849d80c700 1 -- 172.21.15.12:0/1965871189 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f849819ad90 con 0x7f8498107750 2024-05-17T22:38:47.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.409+0000 7f849e00d700 1 --2- 172.21.15.12:0/1965871189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8498102e70 0x7f84981a04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:38:47.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.409+0000 7f849e80e700 1 --2- 172.21.15.12:0/1965871189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84981080a0 0x7f849819a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:38:47.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.409+0000 7f849d80c700 1 --2- 172.21.15.12:0/1965871189 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8498107750 0x7f84981a0a10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8488000c00 tx=0x7f84880047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:47.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.409+0000 7f848f7fe700 1 -- 172.21.15.12:0/1965871189 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8488026070 con 0x7f8498107750 2024-05-17T22:38:47.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.409+0000 7f84a0271700 1 -- 172.21.15.12:0/1965871189 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f849819b020 con 0x7f8498107750 2024-05-17T22:38:47.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.409+0000 7f848f7fe700 1 -- 172.21.15.12:0/1965871189 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8488021410 con 0x7f8498107750 2024-05-17T22:38:47.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.409+0000 7f848f7fe700 1 -- 172.21.15.12:0/1965871189 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8488017650 con 0x7f8498107750 2024-05-17T22:38:47.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.409+0000 7f84a0271700 1 -- 172.21.15.12:0/1965871189 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84981a7280 con 0x7f8498107750 2024-05-17T22:38:47.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.412+0000 7f848f7fe700 1 -- 172.21.15.12:0/1965871189 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8488029070 con 0x7f8498107750 2024-05-17T22:38:47.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.413+0000 7f848f7fe700 1 --2- 172.21.15.12:0/1965871189 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f84840b2880 0x7f84840b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:47.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.413+0000 7f849e00d700 1 --2- 172.21.15.12:0/1965871189 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f84840b2880 0x7f84840b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:47.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.413+0000 7f84aec4cb80 1 Processor -- start 2024-05-17T22:38:47.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.413+0000 7f848f7fe700 1 -- 172.21.15.12:0/1965871189 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f84880e3530 con 0x7f8498107750 2024-05-17T22:38:47.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.413+0000 7f84aec4cb80 1 -- start start 2024-05-17T22:38:47.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.413+0000 7f849e00d700 1 --2- 172.21.15.12:0/1965871189 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f84840b2880 0x7f84840b4d90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8494005fd0 tx=0x7f8494005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:47.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.414+0000 7f84aec4cb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55931de8a9a0 0x55931e00d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:47.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.414+0000 7f84aec4cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55931e00d7f0 0x55931e011460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:47.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.414+0000 7f84aec4cb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55931e0119a0 0x55931e011db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:47.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.414+0000 7f84aec4cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55931df37240 con 0x55931e00d7f0 2024-05-17T22:38:47.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.414+0000 7f84aec4cb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55931df26f80 con 0x55931e0119a0 2024-05-17T22:38:47.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.414+0000 7f84aec4cb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55931df2ae20 con 0x55931de8a9a0 2024-05-17T22:38:47.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.414+0000 7f849d80c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55931e00d7f0 0x55931e011460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:47.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.414+0000 7f849d80c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55931e00d7f0 0x55931e011460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47102/0 (socket says 172.21.15.12:47102) 2024-05-17T22:38:47.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.414+0000 7f849d80c700 1 -- 172.21.15.12:0/3297263242 learned_addr learned my addr 172.21.15.12:0/3297263242 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:47.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.414+0000 7f849e00d700 1 --2- 172.21.15.12:0/3297263242 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55931de8a9a0 0x55931e00d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:47.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.414+0000 7f849e80e700 1 --2- 172.21.15.12:0/3297263242 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55931e0119a0 0x55931e011db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:47.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.414+0000 7f849d80c700 1 -- 172.21.15.12:0/3297263242 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55931de8a9a0 msgr2=0x55931e00d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:47.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.414+0000 7f849d80c700 1 --2- 172.21.15.12:0/3297263242 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55931de8a9a0 0x55931e00d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:47.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.414+0000 7f849d80c700 1 -- 172.21.15.12:0/3297263242 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55931e0119a0 msgr2=0x55931e011db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:47.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.414+0000 7f849d80c700 1 --2- 172.21.15.12:0/3297263242 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55931e0119a0 0x55931e011db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:47.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.414+0000 7f849d80c700 1 -- 172.21.15.12:0/3297263242 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55931ded0540 con 0x55931e00d7f0 2024-05-17T22:38:47.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.414+0000 7f849e80e700 1 --2- 172.21.15.12:0/3297263242 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55931e0119a0 0x55931e011db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:38:47.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.414+0000 7f849e00d700 1 --2- 172.21.15.12:0/3297263242 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55931de8a9a0 0x55931e00d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:38:47.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.414+0000 7f849d80c700 1 --2- 172.21.15.12:0/3297263242 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55931e00d7f0 0x55931e011460 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f84880e0cf0 tx=0x7f84880a9fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d40aefa6e3f2058 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:47.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.415+0000 7f848cd46700 1 -- 172.21.15.12:0/3297263242 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84880aa850 con 0x55931e00d7f0 2024-05-17T22:38:47.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.415+0000 7f848cd46700 1 -- 172.21.15.12:0/3297263242 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84880aec00 con 0x55931e00d7f0 2024-05-17T22:38:47.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.415+0000 7f848cd46700 1 -- 172.21.15.12:0/3297263242 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84880aaab0 con 0x55931e00d7f0 2024-05-17T22:38:47.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.416+0000 7f84aec4cb80 1 -- 172.21.15.12:0/3297263242 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55931e00d7f0 msgr2=0x55931e011460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:47.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.416+0000 7f84aec4cb80 1 --2- 172.21.15.12:0/3297263242 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55931e00d7f0 0x55931e011460 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f84880e0cf0 tx=0x7f84880a9fc0 comp rx=0 tx=0).stop 2024-05-17T22:38:47.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.416+0000 7f84aec4cb80 1 -- 172.21.15.12:0/3297263242 shutdown_connections 2024-05-17T22:38:47.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.416+0000 7f84aec4cb80 1 --2- 172.21.15.12:0/3297263242 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55931e0119a0 0x55931e011db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:47.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.416+0000 7f84aec4cb80 1 --2- 172.21.15.12:0/3297263242 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55931de8a9a0 0x55931e00d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:47.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.416+0000 7f84aec4cb80 1 --2- 172.21.15.12:0/3297263242 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55931e00d7f0 0x55931e011460 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:47.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.416+0000 7f84aec4cb80 1 -- 172.21.15.12:0/3297263242 >> 172.21.15.12:0/3297263242 conn(0x55931deceeb0 msgr2=0x55931e003b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:47.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.416+0000 7f84aec4cb80 1 -- 172.21.15.12:0/3297263242 shutdown_connections 2024-05-17T22:38:47.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.416+0000 7f84aec4cb80 1 -- 172.21.15.12:0/3297263242 wait complete. 2024-05-17T22:38:47.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.416+0000 7f84aec4cb80 1 -- start start 2024-05-17T22:38:47.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.416+0000 7f84aec4cb80 1 Processor -- start 2024-05-17T22:38:47.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.417+0000 7f84aec4cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55931de8a9a0 0x55931e0168f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:47.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.417+0000 7f84aec4cb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55931e00d7f0 0x55931e016e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:47.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.417+0000 7f84aec4cb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55931e0119a0 0x55931e01b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:47.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.417+0000 7f84aec4cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55931de2e100 con 0x55931de8a9a0 2024-05-17T22:38:47.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.417+0000 7f84aec4cb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55931dec7e30 con 0x55931e0119a0 2024-05-17T22:38:47.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.417+0000 7f84aec4cb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55931df37240 con 0x55931e00d7f0 2024-05-17T22:38:47.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.417+0000 7f84aec4cb80 10 client.0 _ll_register_callbacks cb 0x55931dec1ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:38:47.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.417+0000 7f849e00d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55931de8a9a0 0x55931e0168f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:47.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.417+0000 7f849e00d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55931de8a9a0 0x55931e0168f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47114/0 (socket says 172.21.15.12:47114) 2024-05-17T22:38:47.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.417+0000 7f849e00d700 1 -- 172.21.15.12:0/142665608 learned_addr learned my addr 172.21.15.12:0/142665608 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:47.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.417+0000 7f849d80c700 1 --2- 172.21.15.12:0/142665608 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55931e00d7f0 0x55931e016e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:47.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.417+0000 7f849e80e700 1 --2- 172.21.15.12:0/142665608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55931e0119a0 0x55931e01b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:47.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.417+0000 7f849e00d700 1 -- 172.21.15.12:0/142665608 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55931e00d7f0 msgr2=0x55931e016e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:47.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.417+0000 7f849e00d700 1 --2- 172.21.15.12:0/142665608 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55931e00d7f0 0x55931e016e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:47.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.417+0000 7f849e00d700 1 -- 172.21.15.12:0/142665608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55931e0119a0 msgr2=0x55931e01b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:47.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.417+0000 7f849e00d700 1 --2- 172.21.15.12:0/142665608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55931e0119a0 0x55931e01b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:47.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.418+0000 7f849e00d700 1 -- 172.21.15.12:0/142665608 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55931de77430 con 0x55931de8a9a0 2024-05-17T22:38:47.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.418+0000 7f849e00d700 1 --2- 172.21.15.12:0/142665608 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55931de8a9a0 0x55931e0168f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f84940117a0 tx=0x7f84940047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18af1388399ad9ee server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:47.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.418+0000 7f8481ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:38:47.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.418+0000 7f8481ffb700 1 -- 172.21.15.12:0/142665608 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8494023070 con 0x55931de8a9a0 2024-05-17T22:38:47.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.418+0000 7f8481ffb700 1 -- 172.21.15.12:0/142665608 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f849401f830 con 0x55931de8a9a0 2024-05-17T22:38:47.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.418+0000 7f84aec4cb80 1 -- 172.21.15.12:0/142665608 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84880aec00 con 0x55931de8a9a0 2024-05-17T22:38:47.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.418+0000 7f8481ffb700 1 -- 172.21.15.12:0/142665608 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f849402f050 con 0x55931de8a9a0 2024-05-17T22:38:47.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.418+0000 7f84aec4cb80 10 client.15249 fetch_fsmap learned FSMap version 26 2024-05-17T22:38:47.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.418+0000 7f84aec4cb80 1 -- 172.21.15.12:0/142665608 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55931e023fc0 con 0x55931de8a9a0 2024-05-17T22:38:47.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.418+0000 7f8481ffb700 1 -- 172.21.15.12:0/142665608 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84880aec00 con 0x55931de8a9a0 2024-05-17T22:38:47.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.419+0000 7f8481ffb700 1 -- 172.21.15.12:0/142665608 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f849400f040 con 0x55931de8a9a0 2024-05-17T22:38:47.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.419+0000 7f84aec4cb80 10 client.15249 fetch_fsmap finished waiting for FSMap version 26 2024-05-17T22:38:47.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.419+0000 7f84aec4cb80 10 client.15249 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-17T22:38:47.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.419+0000 7f84aec4cb80 20 client.15249 populate_metadata read hostname 'smithi012' 2024-05-17T22:38:47.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.419+0000 7f84aec4cb80 1 --2- 172.21.15.12:0/142665608 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55931e026630 0x55931e027aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:47.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.419+0000 7f84aec4cb80 4 client.15249 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:38:47.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.419+0000 7f84aec4cb80 1 -- 172.21.15.12:0/142665608 --> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55931df39190 con 0x55931e026630 2024-05-17T22:38:47.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.419+0000 7f849d80c700 1 --2- 172.21.15.12:0/142665608 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55931e026630 0x55931e027aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:47.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.420+0000 7f8481ffb700 10 client.15249 ms_handle_connect on v2:172.21.15.96:6836/4222574338 2024-05-17T22:38:47.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.420+0000 7f849d80c700 1 --2- 172.21.15.12:0/142665608 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55931e026630 0x55931e027aa0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=efefbcaf08dddfd6 server_cookie=7901a81dd383b70a in_seq=0 out_seq=0 2024-05-17T22:38:47.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.420+0000 7f8481ffb700 1 -- 172.21.15.12:0/142665608 <== mds.1 v2:172.21.15.96:6836/4222574338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55931df39190 con 0x55931e026630 2024-05-17T22:38:47.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.420+0000 7f8481ffb700 10 client.15249 handle_command_reply: tid=0 2024-05-17T22:38:47.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.420+0000 7f84aec4cb80 1 client.15249 shutdown 2024-05-17T22:38:47.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.420+0000 7f84aec4cb80 1 -- 172.21.15.12:0/142665608 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55931de8a9a0 msgr2=0x55931e0168f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:47.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.420+0000 7f84aec4cb80 1 --2- 172.21.15.12:0/142665608 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55931de8a9a0 0x55931e0168f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f84940117a0 tx=0x7f84940047c0 comp rx=0 tx=0).stop 2024-05-17T22:38:47.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.421+0000 7f84aec4cb80 1 -- 172.21.15.12:0/142665608 shutdown_connections 2024-05-17T22:38:47.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.421+0000 7f84aec4cb80 1 --2- 172.21.15.12:0/142665608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55931e0119a0 0x55931e01b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:47.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.421+0000 7f84aec4cb80 1 --2- 172.21.15.12:0/142665608 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55931e026630 0x55931e027aa0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:47.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.421+0000 7f84aec4cb80 1 --2- 172.21.15.12:0/142665608 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55931de8a9a0 0x55931e0168f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:47.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.421+0000 7f8481ffb700 0 client.15249 ms_handle_reset on v2:172.21.15.96:6836/4222574338 2024-05-17T22:38:47.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.421+0000 7f84aec4cb80 1 --2- 172.21.15.12:0/142665608 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55931e00d7f0 0x55931e016e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:47.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.421+0000 7f84aec4cb80 1 -- 172.21.15.12:0/142665608 >> 172.21.15.12:0/142665608 conn(0x55931deceeb0 msgr2=0x55931e007bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:47.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.421+0000 7f84aec4cb80 1 -- 172.21.15.12:0/142665608 shutdown_connections 2024-05-17T22:38:47.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.421+0000 7f84aec4cb80 1 -- 172.21.15.12:0/142665608 wait complete. 2024-05-17T22:38:47.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.421+0000 7f84aec4cb80 20 client.15249 trim_cache size 0 max 16384 2024-05-17T22:38:47.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.442+0000 7f84aec4cb80 1 Processor -- start 2024-05-17T22:38:47.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.442+0000 7f84aec4cb80 1 -- start start 2024-05-17T22:38:47.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.443+0000 7f84aec4cb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55931de8a9a0 0x55931e015980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:47.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.443+0000 7f84aec4cb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55931e00d7f0 0x55931dffde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:47.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.443+0000 7f84aec4cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55931e0119a0 0x55931e01aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:47.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.443+0000 7f84aec4cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55931e025de0 con 0x55931e0119a0 2024-05-17T22:38:47.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.443+0000 7f84aec4cb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55931e0264b0 con 0x55931de8a9a0 2024-05-17T22:38:47.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.443+0000 7f84aec4cb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55931e026b80 con 0x55931e00d7f0 2024-05-17T22:38:47.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.443+0000 7f849e80e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55931e0119a0 0x55931e01aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:47.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.443+0000 7f849e80e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55931e0119a0 0x55931e01aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47118/0 (socket says 172.21.15.12:47118) 2024-05-17T22:38:47.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.443+0000 7f849e80e700 1 -- 172.21.15.12:0/4125238137 learned_addr learned my addr 172.21.15.12:0/4125238137 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:47.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.443+0000 7f849e00d700 1 --2- 172.21.15.12:0/4125238137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55931de8a9a0 0x55931e015980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:47.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.443+0000 7f849d80c700 1 --2- 172.21.15.12:0/4125238137 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55931e00d7f0 0x55931dffde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:47.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.443+0000 7f849e80e700 1 -- 172.21.15.12:0/4125238137 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55931e00d7f0 msgr2=0x55931dffde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:47.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.443+0000 7f849e80e700 1 --2- 172.21.15.12:0/4125238137 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55931e00d7f0 0x55931dffde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:47.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.443+0000 7f849e80e700 1 -- 172.21.15.12:0/4125238137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55931de8a9a0 msgr2=0x55931e015980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:47.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.443+0000 7f849e80e700 1 --2- 172.21.15.12:0/4125238137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55931de8a9a0 0x55931e015980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:47.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.443+0000 7f849e80e700 1 -- 172.21.15.12:0/4125238137 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55931e0174c0 con 0x55931e0119a0 2024-05-17T22:38:47.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.443+0000 7f849d80c700 1 --2- 172.21.15.12:0/4125238137 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55931e00d7f0 0x55931dffde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:38:47.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.443+0000 7f849e80e700 1 --2- 172.21.15.12:0/4125238137 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55931e0119a0 0x55931e01aeb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8490007580 tx=0x7f849000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83307f72bd3ec0b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:47.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.443+0000 7f8483fff700 1 -- 172.21.15.12:0/4125238137 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849001c070 con 0x55931e0119a0 2024-05-17T22:38:47.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.443+0000 7f8483fff700 1 -- 172.21.15.12:0/4125238137 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8490015c00 con 0x55931e0119a0 2024-05-17T22:38:47.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.443+0000 7f8483fff700 1 -- 172.21.15.12:0/4125238137 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8490017cb0 con 0x55931e0119a0 2024-05-17T22:38:47.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.444+0000 7f84aec4cb80 1 -- 172.21.15.12:0/4125238137 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55931e0119a0 msgr2=0x55931e01aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:47.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.444+0000 7f84aec4cb80 1 --2- 172.21.15.12:0/4125238137 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55931e0119a0 0x55931e01aeb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8490007580 tx=0x7f849000b040 comp rx=0 tx=0).stop 2024-05-17T22:38:47.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.444+0000 7f84aec4cb80 1 -- 172.21.15.12:0/4125238137 shutdown_connections 2024-05-17T22:38:47.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.444+0000 7f84aec4cb80 1 --2- 172.21.15.12:0/4125238137 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55931e0119a0 0x55931e01aeb0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:47.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.444+0000 7f84aec4cb80 1 --2- 172.21.15.12:0/4125238137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55931de8a9a0 0x55931e015980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:47.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.444+0000 7f84aec4cb80 1 --2- 172.21.15.12:0/4125238137 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55931e00d7f0 0x55931dffde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:47.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.444+0000 7f84aec4cb80 1 -- 172.21.15.12:0/4125238137 >> 172.21.15.12:0/4125238137 conn(0x55931deceeb0 msgr2=0x55931e010c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:47.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.444+0000 7f84aec4cb80 1 -- 172.21.15.12:0/4125238137 shutdown_connections 2024-05-17T22:38:47.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.444+0000 7f84aec4cb80 1 -- 172.21.15.12:0/4125238137 wait complete. 2024-05-17T22:38:47.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.445+0000 7f84aec4cb80 1 -- start start 2024-05-17T22:38:47.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.445+0000 7f84aec4cb80 1 Processor -- start 2024-05-17T22:38:47.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.445+0000 7f84aec4cb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55931de8a9a0 0x55931e01e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:47.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.445+0000 7f84aec4cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55931e00d7f0 0x55931e01ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:47.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.445+0000 7f84aec4cb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55931e0119a0 0x55931e01f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:47.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.445+0000 7f84aec4cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55931e027920 con 0x55931e00d7f0 2024-05-17T22:38:47.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.445+0000 7f84aec4cb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55931e027250 con 0x55931de8a9a0 2024-05-17T22:38:47.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.445+0000 7f84aec4cb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55931e027ff0 con 0x55931e0119a0 2024-05-17T22:38:47.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.445+0000 7f84aec4cb80 10 client.0 _ll_register_callbacks cb 0x55931dec1ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:38:47.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.445+0000 7f849d80c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55931e00d7f0 0x55931e01ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:47.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.445+0000 7f849d80c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55931e00d7f0 0x55931e01ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47130/0 (socket says 172.21.15.12:47130) 2024-05-17T22:38:47.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.445+0000 7f849d80c700 1 -- 172.21.15.12:0/2898858902 learned_addr learned my addr 172.21.15.12:0/2898858902 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:47.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.445+0000 7f849e00d700 1 --2- 172.21.15.12:0/2898858902 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55931de8a9a0 0x55931e01e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:47.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.445+0000 7f849e80e700 1 --2- 172.21.15.12:0/2898858902 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55931e0119a0 0x55931e01f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:47.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.446+0000 7f849d80c700 1 -- 172.21.15.12:0/2898858902 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55931e0119a0 msgr2=0x55931e01f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:47.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.446+0000 7f849d80c700 1 --2- 172.21.15.12:0/2898858902 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55931e0119a0 0x55931e01f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:47.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.446+0000 7f849d80c700 1 -- 172.21.15.12:0/2898858902 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55931de8a9a0 msgr2=0x55931e01e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:47.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.446+0000 7f849d80c700 1 --2- 172.21.15.12:0/2898858902 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55931de8a9a0 0x55931e01e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:47.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.446+0000 7f849d80c700 1 -- 172.21.15.12:0/2898858902 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55931e090cb0 con 0x55931e00d7f0 2024-05-17T22:38:47.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.446+0000 7f849d80c700 1 --2- 172.21.15.12:0/2898858902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55931e00d7f0 0x55931e01ecf0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f84880a68d0 tx=0x7f84880b3040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c16ca262814bfd server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:47.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.446+0000 7f8481ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:38:47.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.446+0000 7f8481ffb700 1 -- 172.21.15.12:0/2898858902 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84880c3020 con 0x55931e00d7f0 2024-05-17T22:38:47.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.446+0000 7f84aec4cb80 1 -- 172.21.15.12:0/2898858902 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8490015c00 con 0x55931e00d7f0 2024-05-17T22:38:47.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.446+0000 7f8481ffb700 1 -- 172.21.15.12:0/2898858902 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f84880e3260 con 0x55931e00d7f0 2024-05-17T22:38:47.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.446+0000 7f8481ffb700 1 -- 172.21.15.12:0/2898858902 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f84880aec00 con 0x55931e00d7f0 2024-05-17T22:38:47.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.446+0000 7f84aec4cb80 10 client.15255 fetch_fsmap learned FSMap version 26 2024-05-17T22:38:47.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.446+0000 7f84aec4cb80 1 -- 172.21.15.12:0/2898858902 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55931e091e80 con 0x55931e00d7f0 2024-05-17T22:38:47.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.446+0000 7f8481ffb700 1 -- 172.21.15.12:0/2898858902 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8490015c00 con 0x55931e00d7f0 2024-05-17T22:38:47.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.446+0000 7f8481ffb700 1 -- 172.21.15.12:0/2898858902 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f84880700d0 con 0x55931e00d7f0 2024-05-17T22:38:47.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.447+0000 7f84aec4cb80 10 client.15255 fetch_fsmap finished waiting for FSMap version 26 2024-05-17T22:38:47.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.447+0000 7f84aec4cb80 10 client.15255 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-17T22:38:47.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.447+0000 7f84aec4cb80 20 client.15255 populate_metadata read hostname 'smithi012' 2024-05-17T22:38:47.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.447+0000 7f84aec4cb80 1 --2- 172.21.15.12:0/2898858902 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55931e094170 0x55931e0955e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:47.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.447+0000 7f84aec4cb80 4 client.15255 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-17T22:38:47.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.447+0000 7f84aec4cb80 1 -- 172.21.15.12:0/2898858902 --> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] -- command(tid 0: {"prefix": "status"}) -- 0x55931df32970 con 0x55931e094170 2024-05-17T22:38:47.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.447+0000 7f849e00d700 1 --2- 172.21.15.12:0/2898858902 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55931e094170 0x55931e0955e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:47.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.447+0000 7f849e00d700 1 --2- 172.21.15.12:0/2898858902 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55931e094170 0x55931e0955e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cbca5250709e5e4c server_cookie=ea07bb641b241919 in_seq=0 out_seq=0 2024-05-17T22:38:47.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.447+0000 7f8481ffb700 10 client.15255 ms_handle_connect on v2:172.21.15.96:6836/4222574338 2024-05-17T22:38:47.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.448+0000 7f8481ffb700 1 -- 172.21.15.12:0/2898858902 <== mds.1 v2:172.21.15.96:6836/4222574338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55931df32970 con 0x55931e094170 2024-05-17T22:38:47.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.448+0000 7f8481ffb700 10 client.15255 handle_command_reply: tid=0 2024-05-17T22:38:47.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.448+0000 7f84aec4cb80 1 client.15255 shutdown 2024-05-17T22:38:47.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.448+0000 7f84aec4cb80 1 -- 172.21.15.12:0/2898858902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55931e00d7f0 msgr2=0x55931e01ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:47.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.448+0000 7f84aec4cb80 1 --2- 172.21.15.12:0/2898858902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55931e00d7f0 0x55931e01ecf0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f84880a68d0 tx=0x7f84880b3040 comp rx=0 tx=0).stop 2024-05-17T22:38:47.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.448+0000 7f84aec4cb80 1 -- 172.21.15.12:0/2898858902 shutdown_connections 2024-05-17T22:38:47.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.448+0000 7f84aec4cb80 1 --2- 172.21.15.12:0/2898858902 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55931e0119a0 0x55931e01f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:47.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.448+0000 7f84aec4cb80 1 --2- 172.21.15.12:0/2898858902 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55931de8a9a0 0x55931e01e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:47.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.448+0000 7f84aec4cb80 1 --2- 172.21.15.12:0/2898858902 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55931e094170 0x55931e0955e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:47.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.448+0000 7f84aec4cb80 1 --2- 172.21.15.12:0/2898858902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55931e00d7f0 0x55931e01ecf0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:47.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.448+0000 7f84aec4cb80 1 -- 172.21.15.12:0/2898858902 >> 172.21.15.12:0/2898858902 conn(0x55931deceeb0 msgr2=0x55931e007bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:47.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.448+0000 7f8481ffb700 0 client.15255 ms_handle_reset on v2:172.21.15.96:6836/4222574338 2024-05-17T22:38:47.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.448+0000 7f84aec4cb80 1 -- 172.21.15.12:0/2898858902 shutdown_connections 2024-05-17T22:38:47.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.448+0000 7f84aec4cb80 1 -- 172.21.15.12:0/2898858902 wait complete. 2024-05-17T22:38:47.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.448+0000 7f84aec4cb80 20 client.15255 trim_cache size 0 max 16384 2024-05-17T22:38:47.457 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:38:47.457 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "4d593780-149c-11ef-bc9a-c7b262605968", 2024-05-17T22:38:47.457 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 1, 2024-05-17T22:38:47.457 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24461, 2024-05-17T22:38:47.457 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-05-17T22:38:47.457 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-05-17T22:38:47.457 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-05-17T22:38:47.458 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 180.43991524899999, 2024-05-17T22:38:47.458 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 26, 2024-05-17T22:38:47.458 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 89, 2024-05-17T22:38:47.458 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 87, 2024-05-17T22:38:47.458 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 272.531733567 2024-05-17T22:38:47.458 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:38:47.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.448+0000 7f84a0271700 1 -- 172.21.15.12:0/1965871189 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f84840b2880 msgr2=0x7f84840b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:47.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.448+0000 7f84a0271700 1 --2- 172.21.15.12:0/1965871189 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f84840b2880 0x7f84840b4d90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8494005fd0 tx=0x7f8494005ee0 comp rx=0 tx=0).stop 2024-05-17T22:38:47.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.448+0000 7f84a0271700 1 -- 172.21.15.12:0/1965871189 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8498107750 msgr2=0x7f84981a0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:47.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.448+0000 7f84a0271700 1 --2- 172.21.15.12:0/1965871189 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8498107750 0x7f84981a0a10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8488000c00 tx=0x7f84880047b0 comp rx=0 tx=0).stop 2024-05-17T22:38:47.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.449+0000 7f84a0271700 1 -- 172.21.15.12:0/1965871189 shutdown_connections 2024-05-17T22:38:47.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.449+0000 7f84a0271700 1 --2- 172.21.15.12:0/1965871189 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f84840b2880 0x7f84840b4d90 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:47.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.449+0000 7f84a0271700 1 --2- 172.21.15.12:0/1965871189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84981080a0 0x7f849819a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:47.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.449+0000 7f84a0271700 1 --2- 172.21.15.12:0/1965871189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8498102e70 0x7f84981a04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:47.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.449+0000 7f84a0271700 1 --2- 172.21.15.12:0/1965871189 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8498107750 0x7f84981a0a10 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:47.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.449+0000 7f84a0271700 1 -- 172.21.15.12:0/1965871189 >> 172.21.15.12:0/1965871189 conn(0x7f8498079ae0 msgr2=0x7f849806c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:47.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.449+0000 7f84a0271700 1 -- 172.21.15.12:0/1965871189 shutdown_connections 2024-05-17T22:38:47.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:47.449+0000 7f84a0271700 1 -- 172.21.15.12:0/1965871189 wait complete. 2024-05-17T22:38:47.759 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:47 smithi012 ceph-mon[31632]: pgmap v425: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T22:38:47.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:47 smithi151 ceph-mon[36101]: pgmap v425: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T22:38:48.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:47 smithi096 ceph-mon[36065]: pgmap v425: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T22:38:48.181 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.b status 2024-05-17T22:38:48.440 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:38:49.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:49 smithi012 ceph-mon[31632]: pgmap v426: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:38:49.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:49 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:38:49.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:49 smithi151 ceph-mon[36101]: pgmap v426: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:38:49.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:49 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:38:50.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:49 smithi096 ceph-mon[36065]: pgmap v426: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:38:50.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:49 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:38:50.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.158+0000 7fe16df98700 1 -- 172.21.15.12:0/4054902967 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1681034f0 msgr2=0x7fe168103900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:50.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.158+0000 7fe16df98700 1 --2- 172.21.15.12:0/4054902967 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1681034f0 0x7fe168103900 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe1580099c0 tx=0x7fe158009cd0 comp rx=0 tx=0).stop 2024-05-17T22:38:50.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.158+0000 7fe16df98700 1 -- 172.21.15.12:0/4054902967 shutdown_connections 2024-05-17T22:38:50.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.158+0000 7fe16df98700 1 --2- 172.21.15.12:0/4054902967 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe1681080f0 0x7fe16810fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:50.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.158+0000 7fe16df98700 1 --2- 172.21.15.12:0/4054902967 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1681034f0 0x7fe168103900 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:50.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.158+0000 7fe16df98700 1 --2- 172.21.15.12:0/4054902967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1681077a0 0x7fe168107bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:50.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.158+0000 7fe16df98700 1 -- 172.21.15.12:0/4054902967 >> 172.21.15.12:0/4054902967 conn(0x7fe168078040 msgr2=0x7fe1680784a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:50.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.158+0000 7fe16df98700 1 -- 172.21.15.12:0/4054902967 shutdown_connections 2024-05-17T22:38:50.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.158+0000 7fe16df98700 1 -- 172.21.15.12:0/4054902967 wait complete. 2024-05-17T22:38:50.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.159+0000 7fe16df98700 1 Processor -- start 2024-05-17T22:38:50.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.159+0000 7fe16df98700 1 -- start start 2024-05-17T22:38:50.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.159+0000 7fe16df98700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1681034f0 0x7fe1681a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:50.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.159+0000 7fe16df98700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe1681077a0 0x7fe1681a2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:50.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.159+0000 7fe16df98700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1681080f0 0x7fe16819c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:50.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.159+0000 7fe16df98700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe168112cd0 con 0x7fe1681034f0 2024-05-17T22:38:50.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.159+0000 7fe16df98700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe168112b50 con 0x7fe1681080f0 2024-05-17T22:38:50.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.159+0000 7fe16df98700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe168112e50 con 0x7fe1681077a0 2024-05-17T22:38:50.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.160+0000 7fe1677fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1681034f0 0x7fe1681a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:50.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.160+0000 7fe1677fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1681034f0 0x7fe1681a2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33572/0 (socket says 172.21.15.12:33572) 2024-05-17T22:38:50.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.160+0000 7fe1677fe700 1 -- 172.21.15.12:0/3745726860 learned_addr learned my addr 172.21.15.12:0/3745726860 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:50.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.160+0000 7fe167fff700 1 --2- 172.21.15.12:0/3745726860 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1681080f0 0x7fe16819c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:50.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.160+0000 7fe15ffff700 1 --2- 172.21.15.12:0/3745726860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe1681077a0 0x7fe1681a2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:50.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.160+0000 7fe1677fe700 1 -- 172.21.15.12:0/3745726860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe1681077a0 msgr2=0x7fe1681a2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:50.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.160+0000 7fe1677fe700 1 --2- 172.21.15.12:0/3745726860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe1681077a0 0x7fe1681a2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:50.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.160+0000 7fe1677fe700 1 -- 172.21.15.12:0/3745726860 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1681080f0 msgr2=0x7fe16819c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:50.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.160+0000 7fe1677fe700 1 --2- 172.21.15.12:0/3745726860 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1681080f0 0x7fe16819c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:50.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.160+0000 7fe1677fe700 1 -- 172.21.15.12:0/3745726860 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe16819cef0 con 0x7fe1681034f0 2024-05-17T22:38:50.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.160+0000 7fe1677fe700 1 --2- 172.21.15.12:0/3745726860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1681034f0 0x7fe1681a2580 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe1580045e0 tx=0x7fe158004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:50.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.160+0000 7fe1657fa700 1 -- 172.21.15.12:0/3745726860 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe158026070 con 0x7fe1681034f0 2024-05-17T22:38:50.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.160+0000 7fe1657fa700 1 -- 172.21.15.12:0/3745726860 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe15800f460 con 0x7fe1681034f0 2024-05-17T22:38:50.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.160+0000 7fe1657fa700 1 -- 172.21.15.12:0/3745726860 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe158009e60 con 0x7fe1681034f0 2024-05-17T22:38:50.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.160+0000 7fe16df98700 1 -- 172.21.15.12:0/3745726860 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe16819d180 con 0x7fe1681034f0 2024-05-17T22:38:50.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.161+0000 7fe16df98700 1 -- 172.21.15.12:0/3745726860 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe16819d610 con 0x7fe1681034f0 2024-05-17T22:38:50.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.163+0000 7fe1657fa700 1 -- 172.21.15.12:0/3745726860 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe158029070 con 0x7fe1681034f0 2024-05-17T22:38:50.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.164+0000 7fe1657fa700 1 --2- 172.21.15.12:0/3745726860 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe1540b2900 0x7fe1540b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:50.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.164+0000 7fe1657fa700 1 -- 172.21.15.12:0/3745726860 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe1580e2cb0 con 0x7fe1681034f0 2024-05-17T22:38:50.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.164+0000 7fe15ffff700 1 --2- 172.21.15.12:0/3745726860 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe1540b2900 0x7fe1540b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:50.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.164+0000 7fe17c973b80 1 Processor -- start 2024-05-17T22:38:50.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.164+0000 7fe17c973b80 1 -- start start 2024-05-17T22:38:50.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.165+0000 7fe17c973b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a01a26c9a0 0x55a01a3efe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:50.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.165+0000 7fe17c973b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a01a3f03b0 0x55a01a3ea570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:50.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.165+0000 7fe17c973b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a01a3eaab0 0x55a01a3eaec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:50.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.165+0000 7fe17c973b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a01a319080 con 0x55a01a26c9a0 2024-05-17T22:38:50.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.165+0000 7fe17c973b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a01a308c20 con 0x55a01a3eaab0 2024-05-17T22:38:50.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.165+0000 7fe17c973b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a01a30cc60 con 0x55a01a3f03b0 2024-05-17T22:38:50.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.165+0000 7fe1677fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a01a26c9a0 0x55a01a3efe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:50.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.165+0000 7fe1677fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a01a26c9a0 0x55a01a3efe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33574/0 (socket says 172.21.15.12:33574) 2024-05-17T22:38:50.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.165+0000 7fe15ffff700 1 --2- 172.21.15.12:0/3745726860 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe1540b2900 0x7fe1540b4e10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe168105640 tx=0x7fe150009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:50.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.165+0000 7fe1677fe700 1 -- 172.21.15.12:0/769748289 learned_addr learned my addr 172.21.15.12:0/769748289 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:50.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.165+0000 7fe15ffff700 1 --2- 172.21.15.12:0/769748289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a01a3f03b0 0x55a01a3ea570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:50.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.165+0000 7fe167fff700 1 --2- 172.21.15.12:0/769748289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a01a3eaab0 0x55a01a3eaec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:50.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.165+0000 7fe1677fe700 1 -- 172.21.15.12:0/769748289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a01a3f03b0 msgr2=0x55a01a3ea570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:50.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.165+0000 7fe1677fe700 1 --2- 172.21.15.12:0/769748289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a01a3f03b0 0x55a01a3ea570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:50.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.165+0000 7fe1677fe700 1 -- 172.21.15.12:0/769748289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a01a3eaab0 msgr2=0x55a01a3eaec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:50.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.165+0000 7fe1677fe700 1 --2- 172.21.15.12:0/769748289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a01a3eaab0 0x55a01a3eaec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:50.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.165+0000 7fe1677fe700 1 -- 172.21.15.12:0/769748289 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a01a2a2570 con 0x55a01a26c9a0 2024-05-17T22:38:50.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.165+0000 7fe167fff700 1 --2- 172.21.15.12:0/769748289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a01a3eaab0 0x55a01a3eaec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:38:50.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.166+0000 7fe1677fe700 1 --2- 172.21.15.12:0/769748289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a01a26c9a0 0x55a01a3efe70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe1580a6050 tx=0x7fe1580a99a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f80bb11faf2b85b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:50.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.166+0000 7fe15dffb700 1 -- 172.21.15.12:0/769748289 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1580aa6d0 con 0x55a01a26c9a0 2024-05-17T22:38:50.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.166+0000 7fe15dffb700 1 -- 172.21.15.12:0/769748289 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1580a9fd0 con 0x55a01a26c9a0 2024-05-17T22:38:50.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.166+0000 7fe15dffb700 1 -- 172.21.15.12:0/769748289 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1580b2910 con 0x55a01a26c9a0 2024-05-17T22:38:50.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.167+0000 7fe17c973b80 1 -- 172.21.15.12:0/769748289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a01a26c9a0 msgr2=0x55a01a3efe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:50.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.167+0000 7fe17c973b80 1 --2- 172.21.15.12:0/769748289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a01a26c9a0 0x55a01a3efe70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe1580a6050 tx=0x7fe1580a99a0 comp rx=0 tx=0).stop 2024-05-17T22:38:50.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.167+0000 7fe17c973b80 1 -- 172.21.15.12:0/769748289 shutdown_connections 2024-05-17T22:38:50.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.167+0000 7fe17c973b80 1 --2- 172.21.15.12:0/769748289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a01a3eaab0 0x55a01a3eaec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:50.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.167+0000 7fe17c973b80 1 --2- 172.21.15.12:0/769748289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a01a26c9a0 0x55a01a3efe70 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:50.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.167+0000 7fe17c973b80 1 --2- 172.21.15.12:0/769748289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a01a3f03b0 0x55a01a3ea570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:50.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.167+0000 7fe17c973b80 1 -- 172.21.15.12:0/769748289 >> 172.21.15.12:0/769748289 conn(0x55a01a2cc780 msgr2=0x55a01a3e5170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:50.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.167+0000 7fe17c973b80 1 -- 172.21.15.12:0/769748289 shutdown_connections 2024-05-17T22:38:50.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.167+0000 7fe17c973b80 1 -- 172.21.15.12:0/769748289 wait complete. 2024-05-17T22:38:50.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.167+0000 7fe17c973b80 1 -- start start 2024-05-17T22:38:50.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.168+0000 7fe17c973b80 1 Processor -- start 2024-05-17T22:38:50.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.168+0000 7fe17c973b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a01a26c9a0 0x55a01a3f3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:50.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.168+0000 7fe17c973b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a01a3eaab0 0x55a01a3f3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:50.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.168+0000 7fe17c973b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a01a3f03b0 0x55a01a3f4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:50.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.168+0000 7fe17c973b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a01a210100 con 0x55a01a26c9a0 2024-05-17T22:38:50.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.168+0000 7fe17c973b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a01a2aa8f0 con 0x55a01a3eaab0 2024-05-17T22:38:50.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.168+0000 7fe17c973b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a01a3f6bd0 con 0x55a01a3f03b0 2024-05-17T22:38:50.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.168+0000 7fe17c973b80 10 client.0 _ll_register_callbacks cb 0x55a01a29fb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:38:50.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.168+0000 7fe1677fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a01a26c9a0 0x55a01a3f3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:50.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.168+0000 7fe1677fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a01a26c9a0 0x55a01a3f3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33588/0 (socket says 172.21.15.12:33588) 2024-05-17T22:38:50.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.168+0000 7fe167fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a01a3f03b0 0x55a01a3f4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:50.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.168+0000 7fe1677fe700 1 -- 172.21.15.12:0/779251669 learned_addr learned my addr 172.21.15.12:0/779251669 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:50.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.168+0000 7fe15ffff700 1 --2- 172.21.15.12:0/779251669 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a01a3eaab0 0x55a01a3f3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:50.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.169+0000 7fe167fff700 1 -- 172.21.15.12:0/779251669 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a01a3eaab0 msgr2=0x55a01a3f3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:50.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.169+0000 7fe167fff700 1 --2- 172.21.15.12:0/779251669 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a01a3eaab0 0x55a01a3f3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:50.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.169+0000 7fe167fff700 1 -- 172.21.15.12:0/779251669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a01a26c9a0 msgr2=0x55a01a3f3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:50.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.169+0000 7fe167fff700 1 --2- 172.21.15.12:0/779251669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a01a26c9a0 0x55a01a3f3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:50.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.169+0000 7fe167fff700 1 -- 172.21.15.12:0/779251669 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a01a259430 con 0x55a01a3f03b0 2024-05-17T22:38:50.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.169+0000 7fe1677fe700 1 --2- 172.21.15.12:0/779251669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a01a26c9a0 0x55a01a3f3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:38:50.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.169+0000 7fe15ffff700 1 --2- 172.21.15.12:0/779251669 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a01a3eaab0 0x55a01a3f3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:38:50.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.169+0000 7fe167fff700 1 --2- 172.21.15.12:0/779251669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a01a3f03b0 0x55a01a3f4510 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe160004040 tx=0x7fe160004350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b182f3535260d3e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:50.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.169+0000 7fe146ffd700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:38:50.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.169+0000 7fe146ffd700 1 -- 172.21.15.12:0/779251669 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16001a020 con 0x55a01a3f03b0 2024-05-17T22:38:50.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.169+0000 7fe146ffd700 1 -- 172.21.15.12:0/779251669 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe1600079d0 con 0x55a01a3f03b0 2024-05-17T22:38:50.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.169+0000 7fe17c973b80 1 -- 172.21.15.12:0/779251669 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1580a9fd0 con 0x55a01a3f03b0 2024-05-17T22:38:50.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.169+0000 7fe146ffd700 1 -- 172.21.15.12:0/779251669 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe160025050 con 0x55a01a3f03b0 2024-05-17T22:38:50.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.169+0000 7fe17c973b80 10 client.24916 fetch_fsmap learned FSMap version 26 2024-05-17T22:38:50.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.169+0000 7fe17c973b80 1 -- 172.21.15.12:0/779251669 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a01a401620 con 0x55a01a3f03b0 2024-05-17T22:38:50.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.170+0000 7fe146ffd700 1 -- 172.21.15.12:0/779251669 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1580a9fd0 con 0x55a01a3f03b0 2024-05-17T22:38:50.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.170+0000 7fe146ffd700 1 -- 172.21.15.12:0/779251669 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fe1600153c0 con 0x55a01a3f03b0 2024-05-17T22:38:50.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.170+0000 7fe17c973b80 10 client.24916 fetch_fsmap finished waiting for FSMap version 26 2024-05-17T22:38:50.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.170+0000 7fe17c973b80 10 client.24916 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-17T22:38:50.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.170+0000 7fe17c973b80 20 client.24916 populate_metadata read hostname 'smithi012' 2024-05-17T22:38:50.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.170+0000 7fe17c973b80 1 --2- 172.21.15.12:0/779251669 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55a01a403cc0 0x55a01a405130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:50.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.170+0000 7fe17c973b80 4 client.24916 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:38:50.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.170+0000 7fe17c973b80 1 -- 172.21.15.12:0/779251669 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a01a31afd0 con 0x55a01a403cc0 2024-05-17T22:38:50.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.170+0000 7fe1677fe700 1 --2- 172.21.15.12:0/779251669 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55a01a403cc0 0x55a01a405130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:50.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.171+0000 7fe1677fe700 1 --2- 172.21.15.12:0/779251669 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55a01a403cc0 0x55a01a405130 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b502b5d6b5fcc62 server_cookie=99ec2724564d677b in_seq=0 out_seq=0 2024-05-17T22:38:50.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.171+0000 7fe146ffd700 10 client.24916 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:38:50.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.171+0000 7fe146ffd700 1 -- 172.21.15.12:0/779251669 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a01a31afd0 con 0x55a01a403cc0 2024-05-17T22:38:50.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.171+0000 7fe146ffd700 10 client.24916 handle_command_reply: tid=0 2024-05-17T22:38:50.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.171+0000 7fe17c973b80 1 client.24916 shutdown 2024-05-17T22:38:50.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.172+0000 7fe17c973b80 1 -- 172.21.15.12:0/779251669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a01a3f03b0 msgr2=0x55a01a3f4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:50.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.172+0000 7fe17c973b80 1 --2- 172.21.15.12:0/779251669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a01a3f03b0 0x55a01a3f4510 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe160004040 tx=0x7fe160004350 comp rx=0 tx=0).stop 2024-05-17T22:38:50.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.172+0000 7fe17c973b80 1 -- 172.21.15.12:0/779251669 shutdown_connections 2024-05-17T22:38:50.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.172+0000 7fe17c973b80 1 --2- 172.21.15.12:0/779251669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a01a3f03b0 0x55a01a3f4510 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:50.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.172+0000 7fe17c973b80 1 --2- 172.21.15.12:0/779251669 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55a01a403cc0 0x55a01a405130 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:50.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.172+0000 7fe17c973b80 1 --2- 172.21.15.12:0/779251669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a01a26c9a0 0x55a01a3f3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:50.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.172+0000 7fe17c973b80 1 --2- 172.21.15.12:0/779251669 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a01a3eaab0 0x55a01a3f3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:50.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.172+0000 7fe17c973b80 1 -- 172.21.15.12:0/779251669 >> 172.21.15.12:0/779251669 conn(0x55a01a2cc780 msgr2=0x55a01a3e5110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:50.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.172+0000 7fe146ffd700 0 client.24916 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:38:50.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.172+0000 7fe17c973b80 1 -- 172.21.15.12:0/779251669 shutdown_connections 2024-05-17T22:38:50.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.172+0000 7fe17c973b80 1 -- 172.21.15.12:0/779251669 wait complete. 2024-05-17T22:38:50.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.172+0000 7fe17c973b80 20 client.24916 trim_cache size 0 max 16384 2024-05-17T22:38:50.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.194+0000 7fe17c973b80 1 Processor -- start 2024-05-17T22:38:50.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.194+0000 7fe17c973b80 1 -- start start 2024-05-17T22:38:50.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.194+0000 7fe17c973b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a01a26c9a0 0x55a01a3ea570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:50.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.194+0000 7fe17c973b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a01a3eaab0 0x55a01a3f4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:50.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.194+0000 7fe17c973b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a01a3f03b0 0x55a01a3ed7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:50.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.194+0000 7fe17c973b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a01a403470 con 0x55a01a3eaab0 2024-05-17T22:38:50.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.194+0000 7fe17c973b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a01a403b40 con 0x55a01a3f03b0 2024-05-17T22:38:50.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.194+0000 7fe17c973b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a01a404210 con 0x55a01a26c9a0 2024-05-17T22:38:50.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.194+0000 7fe15ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a01a3eaab0 0x55a01a3f4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:50.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.194+0000 7fe15ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a01a3eaab0 0x55a01a3f4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33602/0 (socket says 172.21.15.12:33602) 2024-05-17T22:38:50.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.194+0000 7fe15ffff700 1 -- 172.21.15.12:0/3967020511 learned_addr learned my addr 172.21.15.12:0/3967020511 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:50.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.194+0000 7fe1677fe700 1 --2- 172.21.15.12:0/3967020511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a01a26c9a0 0x55a01a3ea570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:50.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.194+0000 7fe167fff700 1 --2- 172.21.15.12:0/3967020511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a01a3f03b0 0x55a01a3ed7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:50.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.194+0000 7fe1677fe700 1 -- 172.21.15.12:0/3967020511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a01a3f03b0 msgr2=0x55a01a3ed7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:50.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.194+0000 7fe1677fe700 1 --2- 172.21.15.12:0/3967020511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a01a3f03b0 0x55a01a3ed7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:50.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.194+0000 7fe1677fe700 1 -- 172.21.15.12:0/3967020511 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a01a3eaab0 msgr2=0x55a01a3f4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:50.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.194+0000 7fe1677fe700 1 --2- 172.21.15.12:0/3967020511 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a01a3eaab0 0x55a01a3f4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:50.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.195+0000 7fe1677fe700 1 -- 172.21.15.12:0/3967020511 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a01a3e1bd0 con 0x55a01a26c9a0 2024-05-17T22:38:50.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.195+0000 7fe15ffff700 1 --2- 172.21.15.12:0/3967020511 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a01a3eaab0 0x55a01a3f4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:38:50.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.195+0000 7fe1677fe700 1 --2- 172.21.15.12:0/3967020511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a01a26c9a0 0x55a01a3ea570 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x55a01a3f3300 tx=0x7fe15800fc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5dda587bdc666eb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:50.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.195+0000 7fe15d7fa700 1 -- 172.21.15.12:0/3967020511 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1580c4070 con 0x55a01a26c9a0 2024-05-17T22:38:50.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.195+0000 7fe15d7fa700 1 -- 172.21.15.12:0/3967020511 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1580aa230 con 0x55a01a26c9a0 2024-05-17T22:38:50.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.195+0000 7fe15d7fa700 1 -- 172.21.15.12:0/3967020511 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1580a5ba0 con 0x55a01a26c9a0 2024-05-17T22:38:50.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.196+0000 7fe17c973b80 1 -- 172.21.15.12:0/3967020511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a01a26c9a0 msgr2=0x55a01a3ea570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:50.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.196+0000 7fe17c973b80 1 --2- 172.21.15.12:0/3967020511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a01a26c9a0 0x55a01a3ea570 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x55a01a3f3300 tx=0x7fe15800fc20 comp rx=0 tx=0).stop 2024-05-17T22:38:50.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.196+0000 7fe17c973b80 1 -- 172.21.15.12:0/3967020511 shutdown_connections 2024-05-17T22:38:50.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.196+0000 7fe17c973b80 1 --2- 172.21.15.12:0/3967020511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a01a3f03b0 0x55a01a3ed7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:50.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.196+0000 7fe17c973b80 1 --2- 172.21.15.12:0/3967020511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a01a26c9a0 0x55a01a3ea570 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:50.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.196+0000 7fe17c973b80 1 --2- 172.21.15.12:0/3967020511 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a01a3eaab0 0x55a01a3f4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:50.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.196+0000 7fe17c973b80 1 -- 172.21.15.12:0/3967020511 >> 172.21.15.12:0/3967020511 conn(0x55a01a2cc780 msgr2=0x55a01a3f2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:50.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.196+0000 7fe17c973b80 1 -- 172.21.15.12:0/3967020511 shutdown_connections 2024-05-17T22:38:50.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.196+0000 7fe17c973b80 1 -- 172.21.15.12:0/3967020511 wait complete. 2024-05-17T22:38:50.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.196+0000 7fe17c973b80 1 -- start start 2024-05-17T22:38:50.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.197+0000 7fe17c973b80 1 Processor -- start 2024-05-17T22:38:50.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.197+0000 7fe17c973b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a01a26c9a0 0x55a01a40ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:50.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.197+0000 7fe17c973b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a01a3eaab0 0x55a01a40c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:50.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.197+0000 7fe17c973b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a01a3f03b0 0x55a01a40cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:50.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.197+0000 7fe17c973b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a01a404fb0 con 0x55a01a3eaab0 2024-05-17T22:38:50.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.197+0000 7fe17c973b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a01a4048e0 con 0x55a01a3f03b0 2024-05-17T22:38:50.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.197+0000 7fe17c973b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a01a405680 con 0x55a01a26c9a0 2024-05-17T22:38:50.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.197+0000 7fe17c973b80 10 client.0 _ll_register_callbacks cb 0x55a01a29fb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:38:50.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.197+0000 7fe15ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a01a3eaab0 0x55a01a40c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:50.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.197+0000 7fe15ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a01a3eaab0 0x55a01a40c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33604/0 (socket says 172.21.15.12:33604) 2024-05-17T22:38:50.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.197+0000 7fe15ffff700 1 -- 172.21.15.12:0/302245353 learned_addr learned my addr 172.21.15.12:0/302245353 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:50.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.197+0000 7fe167fff700 1 --2- 172.21.15.12:0/302245353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a01a3f03b0 0x55a01a40cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:50.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.197+0000 7fe1677fe700 1 --2- 172.21.15.12:0/302245353 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a01a26c9a0 0x55a01a40ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:50.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.197+0000 7fe15ffff700 1 -- 172.21.15.12:0/302245353 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a01a26c9a0 msgr2=0x55a01a40ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:50.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.197+0000 7fe15ffff700 1 --2- 172.21.15.12:0/302245353 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a01a26c9a0 0x55a01a40ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:50.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.197+0000 7fe15ffff700 1 -- 172.21.15.12:0/302245353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a01a3f03b0 msgr2=0x55a01a40cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:38:50.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.197+0000 7fe15ffff700 1 --2- 172.21.15.12:0/302245353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a01a3f03b0 0x55a01a40cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:50.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.198+0000 7fe15ffff700 1 -- 172.21.15.12:0/302245353 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a01a3f4880 con 0x55a01a3eaab0 2024-05-17T22:38:50.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.198+0000 7fe1677fe700 1 --2- 172.21.15.12:0/302245353 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a01a26c9a0 0x55a01a40ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:38:50.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.198+0000 7fe15ffff700 1 --2- 172.21.15.12:0/302245353 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a01a3eaab0 0x55a01a40c590 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe15001dc00 tx=0x7fe1500047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a465b9fe9397b4e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:50.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.198+0000 7fe146ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:38:50.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.198+0000 7fe146ffd700 1 -- 172.21.15.12:0/302245353 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe150004020 con 0x55a01a3eaab0 2024-05-17T22:38:50.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.198+0000 7fe146ffd700 1 -- 172.21.15.12:0/302245353 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe1500041c0 con 0x55a01a3eaab0 2024-05-17T22:38:50.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.198+0000 7fe17c973b80 1 -- 172.21.15.12:0/302245353 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1580aa230 con 0x55a01a3eaab0 2024-05-17T22:38:50.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.198+0000 7fe146ffd700 1 -- 172.21.15.12:0/302245353 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe1500043c0 con 0x55a01a3eaab0 2024-05-17T22:38:50.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.198+0000 7fe17c973b80 10 client.15273 fetch_fsmap learned FSMap version 26 2024-05-17T22:38:50.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.198+0000 7fe17c973b80 1 -- 172.21.15.12:0/302245353 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a01a3f4000 con 0x55a01a3eaab0 2024-05-17T22:38:50.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.198+0000 7fe146ffd700 1 -- 172.21.15.12:0/302245353 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1580aa230 con 0x55a01a3eaab0 2024-05-17T22:38:50.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.199+0000 7fe146ffd700 1 -- 172.21.15.12:0/302245353 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fe150027070 con 0x55a01a3eaab0 2024-05-17T22:38:50.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.199+0000 7fe17c973b80 10 client.15273 fetch_fsmap finished waiting for FSMap version 26 2024-05-17T22:38:50.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.199+0000 7fe17c973b80 10 client.15273 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-17T22:38:50.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.199+0000 7fe17c973b80 20 client.15273 populate_metadata read hostname 'smithi012' 2024-05-17T22:38:50.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.199+0000 7fe17c973b80 1 --2- 172.21.15.12:0/302245353 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55a01a4712f0 0x55a01a472760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:50.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.199+0000 7fe17c973b80 4 client.15273 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-17T22:38:50.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.199+0000 7fe17c973b80 1 -- 172.21.15.12:0/302245353 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "status"}) -- 0x55a01a3147b0 con 0x55a01a4712f0 2024-05-17T22:38:50.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.199+0000 7fe1677fe700 1 --2- 172.21.15.12:0/302245353 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55a01a4712f0 0x55a01a472760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:50.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.199+0000 7fe1677fe700 1 --2- 172.21.15.12:0/302245353 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55a01a4712f0 0x55a01a472760 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d841751c01dc9c68 server_cookie=6a92b82d62aa3b37 in_seq=0 out_seq=0 2024-05-17T22:38:50.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.199+0000 7fe146ffd700 10 client.15273 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:38:50.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.200+0000 7fe146ffd700 1 -- 172.21.15.12:0/302245353 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a01a3147b0 con 0x55a01a4712f0 2024-05-17T22:38:50.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.200+0000 7fe146ffd700 10 client.15273 handle_command_reply: tid=0 2024-05-17T22:38:50.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.200+0000 7fe17c973b80 1 client.15273 shutdown 2024-05-17T22:38:50.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.200+0000 7fe17c973b80 1 -- 172.21.15.12:0/302245353 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a01a3eaab0 msgr2=0x55a01a40c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:50.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.200+0000 7fe17c973b80 1 --2- 172.21.15.12:0/302245353 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a01a3eaab0 0x55a01a40c590 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe15001dc00 tx=0x7fe1500047c0 comp rx=0 tx=0).stop 2024-05-17T22:38:50.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.200+0000 7fe17c973b80 1 -- 172.21.15.12:0/302245353 shutdown_connections 2024-05-17T22:38:50.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.200+0000 7fe17c973b80 1 --2- 172.21.15.12:0/302245353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a01a3f03b0 0x55a01a40cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:50.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.200+0000 7fe17c973b80 1 --2- 172.21.15.12:0/302245353 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a01a26c9a0 0x55a01a40ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:50.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.200+0000 7fe17c973b80 1 --2- 172.21.15.12:0/302245353 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55a01a4712f0 0x55a01a472760 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:50.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.200+0000 7fe17c973b80 1 --2- 172.21.15.12:0/302245353 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a01a3eaab0 0x55a01a40c590 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:50.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.200+0000 7fe17c973b80 1 -- 172.21.15.12:0/302245353 >> 172.21.15.12:0/302245353 conn(0x55a01a2cc780 msgr2=0x55a01a3ed7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:50.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.200+0000 7fe146ffd700 0 client.15273 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:38:50.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.200+0000 7fe17c973b80 1 -- 172.21.15.12:0/302245353 shutdown_connections 2024-05-17T22:38:50.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.200+0000 7fe17c973b80 1 -- 172.21.15.12:0/302245353 wait complete. 2024-05-17T22:38:50.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.200+0000 7fe17c973b80 20 client.15273 trim_cache size 0 max 16384 2024-05-17T22:38:50.213 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:38:50.213 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "4d593780-149c-11ef-bc9a-c7b262605968", 2024-05-17T22:38:50.213 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 0, 2024-05-17T22:38:50.213 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24482, 2024-05-17T22:38:50.213 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-05-17T22:38:50.213 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-05-17T22:38:50.213 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-05-17T22:38:50.214 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 246.77365698200001, 2024-05-17T22:38:50.214 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 26, 2024-05-17T22:38:50.214 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 89, 2024-05-17T22:38:50.214 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 87, 2024-05-17T22:38:50.214 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 262.07018247100001 2024-05-17T22:38:50.214 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:38:50.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.201+0000 7fe16df98700 1 -- 172.21.15.12:0/3745726860 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe1540b2900 msgr2=0x7fe1540b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:50.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.201+0000 7fe16df98700 1 --2- 172.21.15.12:0/3745726860 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe1540b2900 0x7fe1540b4e10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe168105640 tx=0x7fe150009260 comp rx=0 tx=0).stop 2024-05-17T22:38:50.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.201+0000 7fe16df98700 1 -- 172.21.15.12:0/3745726860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1681034f0 msgr2=0x7fe1681a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:50.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.201+0000 7fe16df98700 1 --2- 172.21.15.12:0/3745726860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1681034f0 0x7fe1681a2580 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe1580045e0 tx=0x7fe158004610 comp rx=0 tx=0).stop 2024-05-17T22:38:50.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.201+0000 7fe16df98700 1 -- 172.21.15.12:0/3745726860 shutdown_connections 2024-05-17T22:38:50.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.201+0000 7fe16df98700 1 --2- 172.21.15.12:0/3745726860 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe1540b2900 0x7fe1540b4e10 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:50.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.201+0000 7fe16df98700 1 --2- 172.21.15.12:0/3745726860 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1681080f0 0x7fe16819c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:50.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.201+0000 7fe16df98700 1 --2- 172.21.15.12:0/3745726860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1681034f0 0x7fe1681a2580 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:50.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.201+0000 7fe16df98700 1 --2- 172.21.15.12:0/3745726860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe1681077a0 0x7fe1681a2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:50.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.201+0000 7fe16df98700 1 -- 172.21.15.12:0/3745726860 >> 172.21.15.12:0/3745726860 conn(0x7fe168078040 msgr2=0x7fe1681060d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:50.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.201+0000 7fe16df98700 1 -- 172.21.15.12:0/3745726860 shutdown_connections 2024-05-17T22:38:50.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:50.201+0000 7fe16df98700 1 -- 172.21.15.12:0/3745726860 wait complete. 2024-05-17T22:38:50.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:50 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:38:50.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:50 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:38:50.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:50 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:38:51.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:50 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:38:51.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:50 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:38:51.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:50 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:38:51.056 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:50 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:38:51.056 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:50 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:38:51.056 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:50 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:38:51.084 INFO:teuthology.run_tasks:Running task sequential... 2024-05-17T22:38:51.096 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-17T22:38:51.096 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-17T22:38:51.096 INFO:teuthology.task.print:Enabling mgr modules 2024-05-17T22:38:51.096 INFO:teuthology.run_tasks:Running task kclient... 2024-05-17T22:38:51.159 INFO:tasks.kclient:Mounting kernel clients... 2024-05-17T22:38:51.159 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-17T22:38:51.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:38:51.159 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2024-05-17T22:38:51.176 INFO:teuthology.orchestra.run.smithi012.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-05-17T22:38:51.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:38:51.177 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-17T22:38:51.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:38:51.247 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link delete ceph-brx 2024-05-17T22:38:51.355 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'wsync', 'context=system_u:object_r:postgresql_db_t:s0'], 'syntax': 'v2', 'mount_subvol_num': 1, 'mountpoint': '/tmp/cephfs'} 2024-05-17T22:38:51.355 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-17T22:38:51.356 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2024-05-17T22:38:51.356 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2024-05-17T22:38:51.356 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs ls 2024-05-17T22:38:51.608 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:38:51.960 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:51 smithi012 ceph-mon[31632]: pgmap v427: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T22:38:51.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:51 smithi151 ceph-mon[36101]: pgmap v427: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T22:38:52.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:51 smithi096 ceph-mon[36065]: pgmap v427: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T22:38:53.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.284+0000 7fdcf108f700 1 -- 172.21.15.12:0/2476310046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcec109190 msgr2=0x7fdcec110af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:53.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.284+0000 7fdcf108f700 1 --2- 172.21.15.12:0/2476310046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcec109190 0x7fdcec110af0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fdce400b0a0 tx=0x7fdce400b3b0 comp rx=0 tx=0).stop 2024-05-17T22:38:53.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.285+0000 7fdcf108f700 1 -- 172.21.15.12:0/2476310046 shutdown_connections 2024-05-17T22:38:53.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.285+0000 7fdcf108f700 1 --2- 172.21.15.12:0/2476310046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcec109190 0x7fdcec110af0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:53.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.285+0000 7fdcf108f700 1 --2- 172.21.15.12:0/2476310046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcec103f10 0x7fdcec104320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:53.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.285+0000 7fdcf108f700 1 --2- 172.21.15.12:0/2476310046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcec108840 0x7fdcec108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:53.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.285+0000 7fdcf108f700 1 -- 172.21.15.12:0/2476310046 >> 172.21.15.12:0/2476310046 conn(0x7fdcec0ff9b0 msgr2=0x7fdcec101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:53.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.285+0000 7fdcf108f700 1 -- 172.21.15.12:0/2476310046 shutdown_connections 2024-05-17T22:38:53.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.285+0000 7fdcf108f700 1 -- 172.21.15.12:0/2476310046 wait complete. 2024-05-17T22:38:53.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.286+0000 7fdcf108f700 1 Processor -- start 2024-05-17T22:38:53.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.286+0000 7fdcf108f700 1 -- start start 2024-05-17T22:38:53.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.286+0000 7fdcf108f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcec103f10 0x7fdcec19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:53.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.286+0000 7fdcf108f700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcec108840 0x7fdcec19acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:53.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.286+0000 7fdcf108f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcec109190 0x7fdcec19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:53.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.286+0000 7fdcf108f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec113dd0 con 0x7fdcec103f10 2024-05-17T22:38:53.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.286+0000 7fdcf108f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec113c50 con 0x7fdcec109190 2024-05-17T22:38:53.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.286+0000 7fdcf108f700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec113f50 con 0x7fdcec108840 2024-05-17T22:38:53.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.286+0000 7fdcead9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcec103f10 0x7fdcec19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:53.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.286+0000 7fdcead9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcec103f10 0x7fdcec19a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33628/0 (socket says 172.21.15.12:33628) 2024-05-17T22:38:53.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.286+0000 7fdcead9d700 1 -- 172.21.15.12:0/3450092104 learned_addr learned my addr 172.21.15.12:0/3450092104 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:53.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.287+0000 7fdceb59e700 1 --2- 172.21.15.12:0/3450092104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcec109190 0x7fdcec19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:53.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.287+0000 7fdce259c700 1 --2- 172.21.15.12:0/3450092104 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcec108840 0x7fdcec19acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:53.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.287+0000 7fdcead9d700 1 -- 172.21.15.12:0/3450092104 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcec108840 msgr2=0x7fdcec19acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:53.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.287+0000 7fdcead9d700 1 --2- 172.21.15.12:0/3450092104 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcec108840 0x7fdcec19acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:53.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.287+0000 7fdcead9d700 1 -- 172.21.15.12:0/3450092104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcec109190 msgr2=0x7fdcec19ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:53.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.287+0000 7fdcead9d700 1 --2- 172.21.15.12:0/3450092104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcec109190 0x7fdcec19ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:53.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.287+0000 7fdcead9d700 1 -- 172.21.15.12:0/3450092104 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcec19f620 con 0x7fdcec103f10 2024-05-17T22:38:53.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.287+0000 7fdcead9d700 1 --2- 172.21.15.12:0/3450092104 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcec103f10 0x7fdcec19a750 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc00d950 tx=0x7fdcdc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:53.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.287+0000 7fdce8d99700 1 -- 172.21.15.12:0/3450092104 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc016070 con 0x7fdcec103f10 2024-05-17T22:38:53.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.287+0000 7fdcf108f700 1 -- 172.21.15.12:0/3450092104 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcec19f910 con 0x7fdcec103f10 2024-05-17T22:38:53.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.287+0000 7fdce8d99700 1 -- 172.21.15.12:0/3450092104 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcdc00fd40 con 0x7fdcec103f10 2024-05-17T22:38:53.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.287+0000 7fdce8d99700 1 -- 172.21.15.12:0/3450092104 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc00e3b0 con 0x7fdcec103f10 2024-05-17T22:38:53.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.287+0000 7fdcf108f700 1 -- 172.21.15.12:0/3450092104 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcec1a7230 con 0x7fdcec103f10 2024-05-17T22:38:53.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.288+0000 7fdcf108f700 1 -- 172.21.15.12:0/3450092104 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcec104320 con 0x7fdcec103f10 2024-05-17T22:38:53.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.290+0000 7fdce8d99700 1 -- 172.21.15.12:0/3450092104 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fdcdc019070 con 0x7fdcec103f10 2024-05-17T22:38:53.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.291+0000 7fdce8d99700 1 --2- 172.21.15.12:0/3450092104 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fdcd80b2880 0x7fdcd80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:53.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.291+0000 7fdce259c700 1 --2- 172.21.15.12:0/3450092104 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fdcd80b2880 0x7fdcd80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:53.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.291+0000 7fdce8d99700 1 -- 172.21.15.12:0/3450092104 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fdcdc0e1ff0 con 0x7fdcec103f10 2024-05-17T22:38:53.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.291+0000 7fdce259c700 1 --2- 172.21.15.12:0/3450092104 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fdcd80b2880 0x7fdcd80b4d90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdcec108770 tx=0x7fdcd400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:53.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.294+0000 7fdce8d99700 1 -- 172.21.15.12:0/3450092104 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdcdc0a9980 con 0x7fdcec103f10 2024-05-17T22:38:53.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.565+0000 7fdcf108f700 1 -- 172.21.15.12:0/3450092104 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fdcec02ea80 con 0x7fdcec103f10 2024-05-17T22:38:53.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.565+0000 7fdce8d99700 1 -- 172.21.15.12:0/3450092104 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+88 (secure 0 0 0) 0x7fdcdc0a9280 con 0x7fdcec103f10 2024-05-17T22:38:53.569 INFO:teuthology.orchestra.run.smithi012.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-17T22:38:53.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.568+0000 7fdcf108f700 1 -- 172.21.15.12:0/3450092104 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fdcd80b2880 msgr2=0x7fdcd80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:53.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.568+0000 7fdcf108f700 1 --2- 172.21.15.12:0/3450092104 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fdcd80b2880 0x7fdcd80b4d90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdcec108770 tx=0x7fdcd400a300 comp rx=0 tx=0).stop 2024-05-17T22:38:53.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.568+0000 7fdcf108f700 1 -- 172.21.15.12:0/3450092104 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcec103f10 msgr2=0x7fdcec19a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:53.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.568+0000 7fdcf108f700 1 --2- 172.21.15.12:0/3450092104 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcec103f10 0x7fdcec19a750 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc00d950 tx=0x7fdcdc00dc60 comp rx=0 tx=0).stop 2024-05-17T22:38:53.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.568+0000 7fdcf108f700 1 -- 172.21.15.12:0/3450092104 shutdown_connections 2024-05-17T22:38:53.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.568+0000 7fdcf108f700 1 --2- 172.21.15.12:0/3450092104 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fdcd80b2880 0x7fdcd80b4d90 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:53.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.568+0000 7fdcf108f700 1 --2- 172.21.15.12:0/3450092104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcec109190 0x7fdcec19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:53.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.568+0000 7fdcf108f700 1 --2- 172.21.15.12:0/3450092104 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcec103f10 0x7fdcec19a750 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:53.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.568+0000 7fdcf108f700 1 --2- 172.21.15.12:0/3450092104 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcec108840 0x7fdcec19acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:53.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.568+0000 7fdcf108f700 1 -- 172.21.15.12:0/3450092104 >> 172.21.15.12:0/3450092104 conn(0x7fdcec0ff9b0 msgr2=0x7fdcec101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:53.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.568+0000 7fdcf108f700 1 -- 172.21.15.12:0/3450092104 shutdown_connections 2024-05-17T22:38:53.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:53.568+0000 7fdcf108f700 1 -- 172.21.15.12:0/3450092104 wait complete. 2024-05-17T22:38:53.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:53 smithi012 ceph-mon[31632]: pgmap v428: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:38:53.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:53 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/3450092104' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-17T22:38:53.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:53 smithi151 ceph-mon[36101]: pgmap v428: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:38:53.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:53 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/3450092104' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-17T22:38:54.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:53 smithi096 ceph-mon[36065]: pgmap v428: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T22:38:54.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:53 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/3450092104' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-17T22:38:54.264 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-17T22:38:54.264 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-17T22:38:54.264 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi012.front.sepia.ceph.com 2024-05-17T22:38:54.264 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-17T22:38:54.264 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2024-05-17T22:38:54.265 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-17T22:38:54.265 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-17T22:38:54.265 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-17T22:38:54.265 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2024-05-17T22:38:54.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:38:54.266 DEBUG:teuthology.orchestra.run.smithi012:> ip addr 2024-05-17T22:38:54.285 INFO:teuthology.orchestra.run.smithi012.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-17T22:38:54.285 INFO:teuthology.orchestra.run.smithi012.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-17T22:38:54.285 INFO:teuthology.orchestra.run.smithi012.stdout: inet 127.0.0.1/8 scope host lo 2024-05-17T22:38:54.285 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:38:54.285 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 ::1/128 scope host 2024-05-17T22:38:54.285 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:38:54.285 INFO:teuthology.orchestra.run.smithi012.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-17T22:38:54.285 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 0c:c4:7a:6b:fd:a4 brd ff:ff:ff:ff:ff:ff 2024-05-17T22:38:54.285 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp7s0f0 2024-05-17T22:38:54.285 INFO:teuthology.orchestra.run.smithi012.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-17T22:38:54.285 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether ac:1f:6b:f8:25:ea brd ff:ff:ff:ff:ff:ff 2024-05-17T22:38:54.286 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp1s0f0 2024-05-17T22:38:54.286 INFO:teuthology.orchestra.run.smithi012.stdout: inet 172.21.15.12/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-17T22:38:54.286 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft 41971sec preferred_lft 41971sec 2024-05-17T22:38:54.286 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 fe80::ae1f:6bff:fef8:25ea/64 scope link noprefixroute 2024-05-17T22:38:54.286 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:38:54.286 INFO:teuthology.orchestra.run.smithi012.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-17T22:38:54.286 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 0c:c4:7a:6b:fd:a5 brd ff:ff:ff:ff:ff:ff 2024-05-17T22:38:54.286 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp7s0f1 2024-05-17T22:38:54.286 INFO:teuthology.orchestra.run.smithi012.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-17T22:38:54.286 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether ac:1f:6b:f8:25:eb brd ff:ff:ff:ff:ff:ff 2024-05-17T22:38:54.286 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp1s0f1 2024-05-17T22:38:54.287 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-17T22:38:54.287 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T22:38:54.287 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-05-17T22:38:54.287 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link add name ceph-brx type bridge 2024-05-17T22:38:54.287 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip addr flush dev ceph-brx 2024-05-17T22:38:54.287 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set ceph-brx up 2024-05-17T22:38:54.287 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-17T22:38:54.287 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-05-17T22:38:54.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T22:38:54.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:38:54.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:38:54.439 DEBUG:teuthology.orchestra.run.smithi012:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-17T22:38:54.464 INFO:teuthology.orchestra.run.smithi012.stdout:1 2024-05-17T22:38:54.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:38:54.466 DEBUG:teuthology.orchestra.run.smithi012:> route 2024-05-17T22:38:54.527 INFO:teuthology.orchestra.run.smithi012.stdout:Kernel IP routing table 2024-05-17T22:38:54.527 INFO:teuthology.orchestra.run.smithi012.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-17T22:38:54.527 INFO:teuthology.orchestra.run.smithi012.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-17T22:38:54.527 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-17T22:38:54.527 INFO:teuthology.orchestra.run.smithi012.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-17T22:38:54.527 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T22:38:54.527 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-05-17T22:38:54.528 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-17T22:38:54.528 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-17T22:38:54.528 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-17T22:38:54.528 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-05-17T22:38:54.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T22:38:54.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:38:54.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:38:54.659 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2024-05-17T22:38:54.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:38:54.675 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list-id 2024-05-17T22:38:54.731 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T22:38:54.731 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-05-17T22:38:54.731 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns add ceph-ns--tmp-cephfs 2024-05-17T22:38:54.731 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns set ceph-ns--tmp-cephfs 0 2024-05-17T22:38:54.731 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-05-17T22:38:54.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T22:38:54.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:38:54.853 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2024-05-17T22:38:54.854 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T22:38:54.854 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-05-17T22:38:54.854 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2024-05-17T22:38:54.854 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-17T22:38:54.854 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2024-05-17T22:38:54.854 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2024-05-17T22:38:54.854 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2024-05-17T22:38:54.854 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-05-17T22:38:54.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T22:38:54.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:38:54.973 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T22:38:54.973 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-05-17T22:38:54.973 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set brx.0 up 2024-05-17T22:38:54.973 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set dev brx.0 master ceph-brx 2024-05-17T22:38:54.973 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-05-17T22:38:55.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T22:38:55.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:38:55.053 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T22:38:55.054 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -v /tmp/cephfs 2024-05-17T22:38:55.069 INFO:teuthology.orchestra.run.smithi012.stdout:mkdir: created directory '/tmp/cephfs' 2024-05-17T22:38:55.069 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T22:38:55.069 DEBUG:teuthology.orchestra.run.smithi012:> chmod 0000 /tmp/cephfs 2024-05-17T22:38:55.126 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-05-17T22:38:55.126 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-05-17T22:38:55.126 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/682b7b25-6ede-403c-acf6-bc1f5a3307a1 2024-05-17T22:38:55.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:38:55.126 DEBUG:teuthology.orchestra.run.smithi012:> sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/682b7b25-6ede-403c-acf6-bc1f5a3307a1 /tmp/cephfs -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,context=system_u:object_r:postgresql_db_t:s0,nofallback 2024-05-17T22:38:55.220 INFO:teuthology.orchestra.run.smithi012.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2024-05-17T22:38:55.221 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-05-17T22:38:55.376 INFO:teuthology.orchestra.run.smithi012.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2024-05-17T22:38:55.376 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-05-17T22:38:55.376 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: resolved to: "172.21.15.12:6789,172.21.15.96:6789,172.21.15.151:6789" 2024-05-17T22:38:55.376 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: trying mount with new device syntax: 0@4d593780-149c-11ef-bc9a-c7b262605968.cephfs=/volumes/qa/sv_1/682b7b25-6ede-403c-acf6-bc1f5a3307a1 2024-05-17T22:38:55.376 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync,context="system_u:object_r:postgresql_db_t:s0",name=0,key=0,mon_addr=172.21.15.12:6789/172.21.15.96:6789/172.21.15.151:6789" will pass to kernel 2024-05-17T22:38:55.379 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-17T22:38:55.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:38:55.379 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 1777 /tmp/cephfs 2024-05-17T22:38:55.406 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-17T22:38:55.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-17T22:38:55.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:38:55.507 INFO:teuthology.orchestra.run.smithi012.stdout:client15282 2024-05-17T22:38:55.507 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-17T22:38:55.507 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:38:55.524 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-17T22:38:55.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-17T22:38:55.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:38:55.623 INFO:teuthology.orchestra.run.smithi012.stdout:client15282 2024-05-17T22:38:55.624 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-17T22:38:55.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-17T22:38:55.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:38:55.682 INFO:teuthology.orchestra.run.smithi012.stdout:client15282 2024-05-17T22:38:55.683 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/cephfs && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/4d593780-149c-11ef-bc9a-c7b262605968.client15282/status') 2024-05-17T22:38:55.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T22:38:55.743 INFO:teuthology.orchestra.run.smithi012.stdout:instance: client.15282 (3)192.168.0.1:0/562237573 2024-05-17T22:38:55.743 INFO:teuthology.orchestra.run.smithi012.stdout:blocklisted: false 2024-05-17T22:38:55.744 INFO:teuthology.orchestra.run.smithi012.stderr:0+1 records in 2024-05-17T22:38:55.744 INFO:teuthology.orchestra.run.smithi012.stderr:0+1 records out 2024-05-17T22:38:55.744 INFO:teuthology.orchestra.run.smithi012.stderr:69 bytes copied, 2.4153e-05 s, 2.9 MB/s 2024-05-17T22:38:55.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:38:55.751 INFO:teuthology.run_tasks:Running task exec... 2024-05-17T22:38:55.761 INFO:teuthology.task.exec:Executing custom commands... 2024-05-17T22:38:55.761 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi012.front.sepia.ceph.com 2024-05-17T22:38:55.761 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-05-17T22:38:55.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55.875+0000 7ff9d417a640 1 -- 172.21.15.12:0/4283295848 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9cc102d20 msgr2=0x7ff9cc103130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:55.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55.875+0000 7ff9d417a640 1 --2- 172.21.15.12:0/4283295848 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9cc102d20 0x7ff9cc103130 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4009ca0 tx=0x7ff9c402a8c0 comp rx=0 tx=0).stop 2024-05-17T22:38:55.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55.875+0000 7ff9d417a640 1 -- 172.21.15.12:0/4283295848 shutdown_connections 2024-05-17T22:38:55.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55.875+0000 7ff9d417a640 1 --2- 172.21.15.12:0/4283295848 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9cc102d20 0x7ff9cc103130 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:55.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55.875+0000 7ff9d417a640 1 --2- 172.21.15.12:0/4283295848 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9cc1023d0 0x7ff9cc1027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:55.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55.875+0000 7ff9d417a640 1 --2- 172.21.15.12:0/4283295848 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9cc101a80 0x7ff9cc101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:55.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55.875+0000 7ff9d417a640 1 -- 172.21.15.12:0/4283295848 >> 172.21.15.12:0/4283295848 conn(0x7ff9cc0fd880 msgr2=0x7ff9cc0ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:55.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55.875+0000 7ff9d417a640 1 -- 172.21.15.12:0/4283295848 shutdown_connections 2024-05-17T22:38:55.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55.875+0000 7ff9d417a640 1 -- 172.21.15.12:0/4283295848 wait complete. 2024-05-17T22:38:55.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55.875+0000 7ff9d417a640 1 Processor -- start 2024-05-17T22:38:55.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55.875+0000 7ff9d417a640 1 -- start start 2024-05-17T22:38:55.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55.876+0000 7ff9d417a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9cc101a80 0x7ff9cc19ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:55.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55.876+0000 7ff9d417a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9cc1023d0 0x7ff9cc19bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:55.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55.876+0000 7ff9d417a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9cc102d20 0x7ff9cc1a2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:55.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55.876+0000 7ff9d417a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc110ec0 con 0x7ff9cc1023d0 2024-05-17T22:38:55.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55.876+0000 7ff9d417a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc110d40 con 0x7ff9cc102d20 2024-05-17T22:38:55.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55.876+0000 7ff9d417a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc111040 con 0x7ff9cc101a80 2024-05-17T22:38:55.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55.876+0000 7ff9d1715640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9cc1023d0 0x7ff9cc19bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:55.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55.876+0000 7ff9d1715640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9cc1023d0 0x7ff9cc19bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33638/0 (socket says 172.21.15.12:33638) 2024-05-17T22:38:55.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55.876+0000 7ff9d1715640 1 -- 172.21.15.12:0/563213630 learned_addr learned my addr 172.21.15.12:0/563213630 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:55.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55.876+0000 7ff9d1f16640 1 --2- 172.21.15.12:0/563213630 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9cc101a80 0x7ff9cc19ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:55.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55.876+0000 7ff9d2717640 1 --2- 172.21.15.12:0/563213630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9cc102d20 0x7ff9cc1a2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:55.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55.876+0000 7ff9d1715640 1 -- 172.21.15.12:0/563213630 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9cc101a80 msgr2=0x7ff9cc19ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:55.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55.876+0000 7ff9d1715640 1 --2- 172.21.15.12:0/563213630 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9cc101a80 0x7ff9cc19ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:55.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55.876+0000 7ff9d1715640 1 -- 172.21.15.12:0/563213630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9cc102d20 msgr2=0x7ff9cc1a2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:55.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55.876+0000 7ff9d1715640 1 --2- 172.21.15.12:0/563213630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9cc102d20 0x7ff9cc1a2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:55.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55.876+0000 7ff9d1715640 1 -- 172.21.15.12:0/563213630 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9cc1a34c0 con 0x7ff9cc1023d0 2024-05-17T22:38:55.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55.876+0000 7ff9d1f16640 1 --2- 172.21.15.12:0/563213630 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9cc101a80 0x7ff9cc19ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:38:55.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55.876+0000 7ff9d2717640 1 --2- 172.21.15.12:0/563213630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9cc102d20 0x7ff9cc1a2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:38:55.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55.876+0000 7ff9d1715640 1 --2- 172.21.15.12:0/563213630 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9cc1023d0 0x7ff9cc19bf70 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc00c920 tx=0x7ff9bc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:55.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55.877+0000 7ff9c2ffd640 1 -- 172.21.15.12:0/563213630 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc007c70 con 0x7ff9cc1023d0 2024-05-17T22:38:55.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55.877+0000 7ff9c2ffd640 1 -- 172.21.15.12:0/563213630 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9bc007e10 con 0x7ff9cc1023d0 2024-05-17T22:38:55.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55.877+0000 7ff9d417a640 1 -- 172.21.15.12:0/563213630 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9cc1a37b0 con 0x7ff9cc1023d0 2024-05-17T22:38:55.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55.877+0000 7ff9c2ffd640 1 -- 172.21.15.12:0/563213630 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc018820 con 0x7ff9cc1023d0 2024-05-17T22:38:55.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55.877+0000 7ff9d417a640 1 -- 172.21.15.12:0/563213630 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9cc1a3d50 con 0x7ff9cc1023d0 2024-05-17T22:38:55.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55.878+0000 7ff9d417a640 1 -- 172.21.15.12:0/563213630 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9cc10cee0 con 0x7ff9cc1023d0 2024-05-17T22:38:55.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55.879+0000 7ff9c2ffd640 1 -- 172.21.15.12:0/563213630 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff9bc00f450 con 0x7ff9cc1023d0 2024-05-17T22:38:55.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55.880+0000 7ff9c2ffd640 1 --2- 172.21.15.12:0/563213630 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff9b80b2860 0x7ff9b80b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:55.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55.880+0000 7ff9d1f16640 1 --2- 172.21.15.12:0/563213630 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff9b80b2860 0x7ff9b80b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:55.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55.880+0000 7ff9c2ffd640 1 -- 172.21.15.12:0/563213630 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7ff9bc0e29f0 con 0x7ff9cc1023d0 2024-05-17T22:38:55.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55.881+0000 7ff9d1f16640 1 --2- 172.21.15.12:0/563213630 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff9b80b2860 0x7ff9b80b4d70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc06bdc0 tx=0x7ff9c8005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:55.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:55.883+0000 7ff9c2ffd640 1 -- 172.21.15.12:0/563213630 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff9bc014030 con 0x7ff9cc1023d0 2024-05-17T22:38:55.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:55 smithi151 ceph-mon[36101]: pgmap v429: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T22:38:56.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:55 smithi096 ceph-mon[36065]: pgmap v429: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T22:38:56.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:55 smithi012 ceph-mon[31632]: pgmap v429: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T22:38:56.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.117+0000 7ff9d417a640 1 -- 172.21.15.12:0/563213630 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7ff9cc02dca0 con 0x7ff9cc1023d0 2024-05-17T22:38:56.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.664+0000 7ff9c2ffd640 1 -- 172.21.15.12:0/563213630 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v27) ==== 117+0+0 (secure 0 0 0) 0x7ff9bc0aa490 con 0x7ff9cc1023d0 2024-05-17T22:38:56.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.666+0000 7ff9d417a640 1 -- 172.21.15.12:0/563213630 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff9b80b2860 msgr2=0x7ff9b80b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:56.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.666+0000 7ff9d417a640 1 --2- 172.21.15.12:0/563213630 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff9b80b2860 0x7ff9b80b4d70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc06bdc0 tx=0x7ff9c8005e70 comp rx=0 tx=0).stop 2024-05-17T22:38:56.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.667+0000 7ff9d417a640 1 -- 172.21.15.12:0/563213630 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9cc1023d0 msgr2=0x7ff9cc19bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:56.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.667+0000 7ff9d417a640 1 --2- 172.21.15.12:0/563213630 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9cc1023d0 0x7ff9cc19bf70 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc00c920 tx=0x7ff9bc00cde0 comp rx=0 tx=0).stop 2024-05-17T22:38:56.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.667+0000 7ff9d417a640 1 -- 172.21.15.12:0/563213630 shutdown_connections 2024-05-17T22:38:56.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.667+0000 7ff9d417a640 1 --2- 172.21.15.12:0/563213630 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff9b80b2860 0x7ff9b80b4d70 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:56.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.667+0000 7ff9d417a640 1 --2- 172.21.15.12:0/563213630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9cc102d20 0x7ff9cc1a2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:56.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.667+0000 7ff9d417a640 1 --2- 172.21.15.12:0/563213630 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9cc1023d0 0x7ff9cc19bf70 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:56.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.667+0000 7ff9d417a640 1 --2- 172.21.15.12:0/563213630 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9cc101a80 0x7ff9cc19ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:56.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.667+0000 7ff9d417a640 1 -- 172.21.15.12:0/563213630 >> 172.21.15.12:0/563213630 conn(0x7ff9cc0fd880 msgr2=0x7ff9cc10a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:56.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.667+0000 7ff9d417a640 1 -- 172.21.15.12:0/563213630 shutdown_connections 2024-05-17T22:38:56.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.667+0000 7ff9d417a640 1 -- 172.21.15.12:0/563213630 wait complete. 2024-05-17T22:38:56.684 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-17T22:38:56.696 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-05-17T22:38:56.732 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T22:38:56.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.856+0000 7f7183438640 1 -- 172.21.15.12:0/1400077711 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f717c1065b0 msgr2=0x7f717c1089d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:56.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.856+0000 7f7183438640 1 --2- 172.21.15.12:0/1400077711 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f717c1065b0 0x7f717c1089d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f716c009900 tx=0x7f716c02a410 comp rx=0 tx=0).stop 2024-05-17T22:38:56.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.856+0000 7f7183438640 1 -- 172.21.15.12:0/1400077711 shutdown_connections 2024-05-17T22:38:56.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.856+0000 7f7183438640 1 --2- 172.21.15.12:0/1400077711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f717c108f10 0x7f717c10b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:56.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.856+0000 7f7183438640 1 --2- 172.21.15.12:0/1400077711 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f717c1065b0 0x7f717c1089d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:56.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.856+0000 7f7183438640 1 --2- 172.21.15.12:0/1400077711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f717c103c50 0x7f717c106070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:56.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.856+0000 7f7183438640 1 -- 172.21.15.12:0/1400077711 >> 172.21.15.12:0/1400077711 conn(0x7f717c0fd860 msgr2=0x7f717c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:56.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.856+0000 7f7183438640 1 -- 172.21.15.12:0/1400077711 shutdown_connections 2024-05-17T22:38:56.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.856+0000 7f7183438640 1 -- 172.21.15.12:0/1400077711 wait complete. 2024-05-17T22:38:56.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.857+0000 7f7183438640 1 Processor -- start 2024-05-17T22:38:56.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.857+0000 7f7183438640 1 -- start start 2024-05-17T22:38:56.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.857+0000 7f7183438640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f717c103c50 0x7f717c199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:56.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.857+0000 7f7183438640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f717c1065b0 0x7f717c199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:56.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.857+0000 7f7183438640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f717c108f10 0x7f717c1a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:56.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.857+0000 7f7183438640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f717c10ec80 con 0x7f717c103c50 2024-05-17T22:38:56.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.857+0000 7f7183438640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f717c10eb00 con 0x7f717c1065b0 2024-05-17T22:38:56.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.857+0000 7f7183438640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f717c10ee00 con 0x7f717c108f10 2024-05-17T22:38:56.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.857+0000 7f71811d4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f717c103c50 0x7f717c199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:56.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.857+0000 7f71811d4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f717c103c50 0x7f717c199860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33660/0 (socket says 172.21.15.12:33660) 2024-05-17T22:38:56.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.857+0000 7f71811d4640 1 -- 172.21.15.12:0/400052529 learned_addr learned my addr 172.21.15.12:0/400052529 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:56.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.857+0000 7f71819d5640 1 --2- 172.21.15.12:0/400052529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f717c108f10 0x7f717c1a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:56.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.857+0000 7f71811d4640 1 -- 172.21.15.12:0/400052529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f717c108f10 msgr2=0x7f717c1a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:56.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.857+0000 7f71809d3640 1 --2- 172.21.15.12:0/400052529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f717c1065b0 0x7f717c199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:56.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.857+0000 7f71811d4640 1 --2- 172.21.15.12:0/400052529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f717c108f10 0x7f717c1a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:56.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.857+0000 7f71811d4640 1 -- 172.21.15.12:0/400052529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f717c1065b0 msgr2=0x7f717c199da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:56.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.857+0000 7f71811d4640 1 --2- 172.21.15.12:0/400052529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f717c1065b0 0x7f717c199da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:56.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.857+0000 7f71811d4640 1 -- 172.21.15.12:0/400052529 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f717c1a13a0 con 0x7f717c103c50 2024-05-17T22:38:56.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.857+0000 7f71809d3640 1 --2- 172.21.15.12:0/400052529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f717c1065b0 0x7f717c199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:38:56.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.857+0000 7f71811d4640 1 --2- 172.21.15.12:0/400052529 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f717c103c50 0x7f717c199860 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f717800e9c0 tx=0x7f717800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:56.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.858+0000 7f71727fc640 1 -- 172.21.15.12:0/400052529 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717800cd80 con 0x7f717c103c50 2024-05-17T22:38:56.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.858+0000 7f71727fc640 1 -- 172.21.15.12:0/400052529 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71780035f0 con 0x7f717c103c50 2024-05-17T22:38:56.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.858+0000 7f71727fc640 1 -- 172.21.15.12:0/400052529 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7178010690 con 0x7f717c103c50 2024-05-17T22:38:56.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.858+0000 7f7183438640 1 -- 172.21.15.12:0/400052529 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f717c1a1690 con 0x7f717c103c50 2024-05-17T22:38:56.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.858+0000 7f7183438640 1 -- 172.21.15.12:0/400052529 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f717c1a1c30 con 0x7f717c103c50 2024-05-17T22:38:56.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.859+0000 7f7183438640 1 -- 172.21.15.12:0/400052529 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f717c0ff380 con 0x7f717c103c50 2024-05-17T22:38:56.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.860+0000 7f71727fc640 1 -- 172.21.15.12:0/400052529 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f71780037a0 con 0x7f717c103c50 2024-05-17T22:38:56.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.861+0000 7f71727fc640 1 --2- 172.21.15.12:0/400052529 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f71680b2870 0x7f71680b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:56.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.861+0000 7f71809d3640 1 --2- 172.21.15.12:0/400052529 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f71680b2870 0x7f71680b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:56.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.861+0000 7f71727fc640 1 -- 172.21.15.12:0/400052529 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f71780e29e0 con 0x7f717c103c50 2024-05-17T22:38:56.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.862+0000 7f71809d3640 1 --2- 172.21.15.12:0/400052529 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f71680b2870 0x7f71680b4d80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f716c006010 tx=0x7f716c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:56.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:56.866+0000 7f71727fc640 1 -- 172.21.15.12:0/400052529 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f71780aa370 con 0x7f717c103c50 2024-05-17T22:38:56.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:56 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/563213630' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-17T22:38:57.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:56 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/563213630' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-17T22:38:57.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.059+0000 7f7183438640 1 -- 172.21.15.12:0/400052529 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f717c063970 con 0x7f717c103c50 2024-05-17T22:38:57.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.061+0000 7f71727fc640 1 -- 172.21.15.12:0/400052529 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24715 (secure 0 0 0) 0x7f71780a9c70 con 0x7f717c103c50 2024-05-17T22:38:57.064 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:38:57.064 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":89,"fsid":"4d593780-149c-11ef-bc9a-c7b262605968","created":"2024-05-17T22:26:16.955626+0000","modified":"2024-05-17T22:36:15.618992+0000","last_up_change":"2024-05-17T22:33:36.551093+0000","last_in_change":"2024-05-17T22:33:15.032677+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T22:29:42.475513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T22:34:35.504697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T22:34:39.377097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-17T22:34:50.467684+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5d42798f-dc5f-4e66-bce2-ec33dc789e4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6803","nonce":2015605593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6805","nonce":2015605593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6809","nonce":2015605593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6807","nonce":2015605593}]},"public_addr":"172.21.15.12:6803/2015605593","cluster_addr":"172.21.15.12:6805/2015605593","heartbeat_back_addr":"172.21.15.12:6809/2015605593","heartbeat_front_addr":"172.21.15.12:6807/2015605593","state":["exists","up"]},{"osd":1,"uuid":"58afecc7-119b-4c16-a401-640fb77d4714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6801","nonce":1238316358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6803","nonce":1238316358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6807","nonce":1238316358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6805","nonce":1238316358}]},"public_addr":"172.21.15.96:6801/1238316358","cluster_addr":"172.21.15.96:6803/1238316358","heartbeat_back_addr":"172.21.15.96:6807/1238316358","heartbeat_front_addr":"172.21.15.96:6805/1238316358","state":["exists","up"]},{"osd":2,"uuid":"4bef4049-482f-49fb-82fb-38052400eb2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6801","nonce":148729815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6803","nonce":148729815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6807","nonce":148729815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6805","nonce":148729815}]},"public_addr":"172.21.15.151:6801/148729815","cluster_addr":"172.21.15.151:6803/148729815","heartbeat_back_addr":"172.21.15.151:6807/148729815","heartbeat_front_addr":"172.21.15.151:6805/148729815","state":["exists","up"]},{"osd":3,"uuid":"6fed8e36-82a2-49e4-82a3-16e6d9de87f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6811","nonce":3563309320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6813","nonce":3563309320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6817","nonce":3563309320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6815","nonce":3563309320}]},"public_addr":"172.21.15.12:6811/3563309320","cluster_addr":"172.21.15.12:6813/3563309320","heartbeat_back_addr":"172.21.15.12:6817/3563309320","heartbeat_front_addr":"172.21.15.12:6815/3563309320","state":["exists","up"]},{"osd":4,"uuid":"c0e8689a-2816-4089-ade3-931960bae4f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6809","nonce":2219103166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6811","nonce":2219103166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6815","nonce":2219103166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6813","nonce":2219103166}]},"public_addr":"172.21.15.96:6809/2219103166","cluster_addr":"172.21.15.96:6811/2219103166","heartbeat_back_addr":"172.21.15.96:6815/2219103166","heartbeat_front_addr":"172.21.15.96:6813/2219103166","state":["exists","up"]},{"osd":5,"uuid":"e76ba432-94ff-4998-aaba-2645d9e6055d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6809","nonce":2589835078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6811","nonce":2589835078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6815","nonce":2589835078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6813","nonce":2589835078}]},"public_addr":"172.21.15.151:6809/2589835078","cluster_addr":"172.21.15.151:6811/2589835078","heartbeat_back_addr":"172.21.15.151:6815/2589835078","heartbeat_front_addr":"172.21.15.151:6813/2589835078","state":["exists","up"]},{"osd":6,"uuid":"b7ece35c-efd1-40df-95da-05a0cc147372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6819","nonce":3112661724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6821","nonce":3112661724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6825","nonce":3112661724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6823","nonce":3112661724}]},"public_addr":"172.21.15.12:6819/3112661724","cluster_addr":"172.21.15.12:6821/3112661724","heartbeat_back_addr":"172.21.15.12:6825/3112661724","heartbeat_front_addr":"172.21.15.12:6823/3112661724","state":["exists","up"]},{"osd":7,"uuid":"6a65bf30-daf5-4fba-b67e-8e513d88a971","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6817","nonce":2449989110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6819","nonce":2449989110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6823","nonce":2449989110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6821","nonce":2449989110}]},"public_addr":"172.21.15.96:6817/2449989110","cluster_addr":"172.21.15.96:6819/2449989110","heartbeat_back_addr":"172.21.15.96:6823/2449989110","heartbeat_front_addr":"172.21.15.96:6821/2449989110","state":["exists","up"]},{"osd":8,"uuid":"ad62ea44-e416-4a3e-bbbe-93f2a710dbc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6817","nonce":2545224628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6819","nonce":2545224628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6823","nonce":2545224628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6821","nonce":2545224628}]},"public_addr":"172.21.15.151:6817/2545224628","cluster_addr":"172.21.15.151:6819/2545224628","heartbeat_back_addr":"172.21.15.151:6823/2545224628","heartbeat_front_addr":"172.21.15.151:6821/2545224628","state":["exists","up"]},{"osd":9,"uuid":"1275de08-2a3c-48f3-920b-dd68b4024d42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6827","nonce":2100661273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6829","nonce":2100661273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6833","nonce":2100661273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6831","nonce":2100661273}]},"public_addr":"172.21.15.12:6827/2100661273","cluster_addr":"172.21.15.12:6829/2100661273","heartbeat_back_addr":"172.21.15.12:6833/2100661273","heartbeat_front_addr":"172.21.15.12:6831/2100661273","state":["exists","up"]},{"osd":10,"uuid":"347e4e40-9a9d-43fe-8f52-fdd3199a2a5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6825","nonce":304016309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6827","nonce":304016309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6831","nonce":304016309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6829","nonce":304016309}]},"public_addr":"172.21.15.96:6825/304016309","cluster_addr":"172.21.15.96:6827/304016309","heartbeat_back_addr":"172.21.15.96:6831/304016309","heartbeat_front_addr":"172.21.15.96:6829/304016309","state":["exists","up"]},{"osd":11,"uuid":"ee8d3502-8b3c-4e6f-824f-3a97e19111ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6825","nonce":3822307314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6827","nonce":3822307314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6831","nonce":3822307314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6829","nonce":3822307314}]},"public_addr":"172.21.15.151:6825/3822307314","cluster_addr":"172.21.15.151:6827/3822307314","heartbeat_back_addr":"172.21.15.151:6831/3822307314","heartbeat_front_addr":"172.21.15.151:6829/3822307314","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:28:51.542055+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:29:15.566424+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:29:39.500981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:04.797907+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:29.655650+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:55.211880+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:31:21.559462+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:31:47.154852+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:32:13.114892+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:32:40.529708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:33:06.930942+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:33:34.061783+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":10}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/2682700253":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6801/2250046521":"2024-05-18T22:27:14.372972+0000","172.21.15.12:0/4241094239":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/4172568611":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/3512529242":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6800/1039789215":"2024-05-18T22:26:45.466719+0000","172.21.15.12:6800/2250046521":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6801/1039789215":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/2043758453":"2024-05-18T22:27:14.372972+0000","172.21.15.12:0/2694593732":"2024-05-18T22:26:45.466719+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T22:38:57.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.063+0000 7f7183438640 1 -- 172.21.15.12:0/400052529 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f71680b2870 msgr2=0x7f71680b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:57.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.063+0000 7f7183438640 1 --2- 172.21.15.12:0/400052529 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f71680b2870 0x7f71680b4d80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f716c006010 tx=0x7f716c000bc0 comp rx=0 tx=0).stop 2024-05-17T22:38:57.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.063+0000 7f7183438640 1 -- 172.21.15.12:0/400052529 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f717c103c50 msgr2=0x7f717c199860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:57.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.063+0000 7f7183438640 1 --2- 172.21.15.12:0/400052529 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f717c103c50 0x7f717c199860 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f717800e9c0 tx=0x7f717800ee80 comp rx=0 tx=0).stop 2024-05-17T22:38:57.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.063+0000 7f7183438640 1 -- 172.21.15.12:0/400052529 shutdown_connections 2024-05-17T22:38:57.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.063+0000 7f7183438640 1 --2- 172.21.15.12:0/400052529 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f71680b2870 0x7f71680b4d80 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:57.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.063+0000 7f7183438640 1 --2- 172.21.15.12:0/400052529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f717c108f10 0x7f717c1a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:57.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.063+0000 7f7183438640 1 --2- 172.21.15.12:0/400052529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f717c1065b0 0x7f717c199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:57.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.063+0000 7f7183438640 1 --2- 172.21.15.12:0/400052529 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f717c103c50 0x7f717c199860 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:57.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.063+0000 7f7183438640 1 -- 172.21.15.12:0/400052529 >> 172.21.15.12:0/400052529 conn(0x7f717c0fd860 msgr2=0x7f717c104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:57.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.063+0000 7f7183438640 1 -- 172.21.15.12:0/400052529 shutdown_connections 2024-05-17T22:38:57.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.063+0000 7f7183438640 1 -- 172.21.15.12:0/400052529 wait complete. 2024-05-17T22:38:57.076 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:56 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/563213630' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-17T22:38:57.078 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-17T22:29:42.475513+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-17T22:34:35.504697+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-17T22:34:39.377097+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-17T22:34:50.467684+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-17T22:38:57.079 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-17T22:38:57.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.194+0000 7fb17b7ac640 1 -- 172.21.15.12:0/974131063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb174103c30 msgr2=0x7fb174106050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:57.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.194+0000 7fb17b7ac640 1 --2- 172.21.15.12:0/974131063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb174103c30 0x7fb174106050 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb170009900 tx=0x7fb17002a410 comp rx=0 tx=0).stop 2024-05-17T22:38:57.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.194+0000 7fb17b7ac640 1 -- 172.21.15.12:0/974131063 shutdown_connections 2024-05-17T22:38:57.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.194+0000 7fb17b7ac640 1 --2- 172.21.15.12:0/974131063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb174108ef0 0x7fb17410b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:57.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.194+0000 7fb17b7ac640 1 --2- 172.21.15.12:0/974131063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb174106590 0x7fb1741089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:57.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.194+0000 7fb17b7ac640 1 --2- 172.21.15.12:0/974131063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb174103c30 0x7fb174106050 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:57.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.194+0000 7fb17b7ac640 1 -- 172.21.15.12:0/974131063 >> 172.21.15.12:0/974131063 conn(0x7fb1740fd860 msgr2=0x7fb1740ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:57.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.194+0000 7fb17b7ac640 1 -- 172.21.15.12:0/974131063 shutdown_connections 2024-05-17T22:38:57.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.194+0000 7fb17b7ac640 1 -- 172.21.15.12:0/974131063 wait complete. 2024-05-17T22:38:57.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.195+0000 7fb17b7ac640 1 Processor -- start 2024-05-17T22:38:57.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.195+0000 7fb17b7ac640 1 -- start start 2024-05-17T22:38:57.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.195+0000 7fb17b7ac640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb174103c30 0x7fb174199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:57.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.195+0000 7fb17b7ac640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb174106590 0x7fb174199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:57.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.195+0000 7fb17b7ac640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb174108ef0 0x7fb1741a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:57.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.195+0000 7fb17b7ac640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb17410ec60 con 0x7fb174106590 2024-05-17T22:38:57.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.195+0000 7fb17b7ac640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb17410eae0 con 0x7fb174103c30 2024-05-17T22:38:57.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.195+0000 7fb17b7ac640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb17410ede0 con 0x7fb174108ef0 2024-05-17T22:38:57.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.195+0000 7fb178d47640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb174106590 0x7fb174199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:57.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.195+0000 7fb178d47640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb174106590 0x7fb174199dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33692/0 (socket says 172.21.15.12:33692) 2024-05-17T22:38:57.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.195+0000 7fb178d47640 1 -- 172.21.15.12:0/4070417874 learned_addr learned my addr 172.21.15.12:0/4070417874 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:57.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.195+0000 7fb179548640 1 --2- 172.21.15.12:0/4070417874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb174103c30 0x7fb174199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:57.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.196+0000 7fb179d49640 1 --2- 172.21.15.12:0/4070417874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb174108ef0 0x7fb1741a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:57.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.196+0000 7fb178d47640 1 -- 172.21.15.12:0/4070417874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb174108ef0 msgr2=0x7fb1741a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:57.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.196+0000 7fb178d47640 1 --2- 172.21.15.12:0/4070417874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb174108ef0 0x7fb1741a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:57.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.196+0000 7fb178d47640 1 -- 172.21.15.12:0/4070417874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb174103c30 msgr2=0x7fb174199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:57.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.196+0000 7fb178d47640 1 --2- 172.21.15.12:0/4070417874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb174103c30 0x7fb174199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:57.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.196+0000 7fb178d47640 1 -- 172.21.15.12:0/4070417874 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1741a13c0 con 0x7fb174106590 2024-05-17T22:38:57.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.196+0000 7fb178d47640 1 --2- 172.21.15.12:0/4070417874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb174106590 0x7fb174199dc0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb16400d950 tx=0x7fb16400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:57.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.196+0000 7fb16a7fc640 1 -- 172.21.15.12:0/4070417874 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb164017070 con 0x7fb174106590 2024-05-17T22:38:57.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.196+0000 7fb16a7fc640 1 -- 172.21.15.12:0/4070417874 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb164009c30 con 0x7fb174106590 2024-05-17T22:38:57.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.196+0000 7fb16a7fc640 1 -- 172.21.15.12:0/4070417874 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16400f620 con 0x7fb174106590 2024-05-17T22:38:57.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.196+0000 7fb17b7ac640 1 -- 172.21.15.12:0/4070417874 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1741a16b0 con 0x7fb174106590 2024-05-17T22:38:57.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.196+0000 7fb17b7ac640 1 -- 172.21.15.12:0/4070417874 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1741a1c00 con 0x7fb174106590 2024-05-17T22:38:57.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.197+0000 7fb17b7ac640 1 -- 172.21.15.12:0/4070417874 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1740ff380 con 0x7fb174106590 2024-05-17T22:38:57.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.200+0000 7fb16a7fc640 1 -- 172.21.15.12:0/4070417874 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb164010460 con 0x7fb174106590 2024-05-17T22:38:57.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.200+0000 7fb16a7fc640 1 --2- 172.21.15.12:0/4070417874 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb1600b27f0 0x7fb1600b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:57.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.201+0000 7fb179548640 1 --2- 172.21.15.12:0/4070417874 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb1600b27f0 0x7fb1600b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:57.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.201+0000 7fb16a7fc640 1 -- 172.21.15.12:0/4070417874 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fb1640e21b0 con 0x7fb174106590 2024-05-17T22:38:57.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.201+0000 7fb179548640 1 --2- 172.21.15.12:0/4070417874 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb1600b27f0 0x7fb1600b4d00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb170009230 tx=0x7fb170031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:57.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.204+0000 7fb16a7fc640 1 -- 172.21.15.12:0/4070417874 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb1640a9b40 con 0x7fb174106590 2024-05-17T22:38:57.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.397+0000 7fb17b7ac640 1 -- 172.21.15.12:0/4070417874 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb17402de30 con 0x7fb174106590 2024-05-17T22:38:57.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.398+0000 7fb16a7fc640 1 -- 172.21.15.12:0/4070417874 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7fb164009de0 con 0x7fb174106590 2024-05-17T22:38:57.401 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-05-17T22:38:57.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.400+0000 7fb17b7ac640 1 -- 172.21.15.12:0/4070417874 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb1600b27f0 msgr2=0x7fb1600b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:57.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.400+0000 7fb17b7ac640 1 --2- 172.21.15.12:0/4070417874 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb1600b27f0 0x7fb1600b4d00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb170009230 tx=0x7fb170031400 comp rx=0 tx=0).stop 2024-05-17T22:38:57.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.400+0000 7fb17b7ac640 1 -- 172.21.15.12:0/4070417874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb174106590 msgr2=0x7fb174199dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:57.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.400+0000 7fb17b7ac640 1 --2- 172.21.15.12:0/4070417874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb174106590 0x7fb174199dc0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb16400d950 tx=0x7fb16400de10 comp rx=0 tx=0).stop 2024-05-17T22:38:57.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.400+0000 7fb17b7ac640 1 -- 172.21.15.12:0/4070417874 shutdown_connections 2024-05-17T22:38:57.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.400+0000 7fb17b7ac640 1 --2- 172.21.15.12:0/4070417874 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb1600b27f0 0x7fb1600b4d00 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:57.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.400+0000 7fb17b7ac640 1 --2- 172.21.15.12:0/4070417874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb174108ef0 0x7fb1741a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:57.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.400+0000 7fb17b7ac640 1 --2- 172.21.15.12:0/4070417874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb174106590 0x7fb174199dc0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:57.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.400+0000 7fb17b7ac640 1 --2- 172.21.15.12:0/4070417874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb174103c30 0x7fb174199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:57.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.400+0000 7fb17b7ac640 1 -- 172.21.15.12:0/4070417874 >> 172.21.15.12:0/4070417874 conn(0x7fb1740fd860 msgr2=0x7fb1741048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:57.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.400+0000 7fb17b7ac640 1 -- 172.21.15.12:0/4070417874 shutdown_connections 2024-05-17T22:38:57.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.400+0000 7fb17b7ac640 1 -- 172.21.15.12:0/4070417874 wait complete. 2024-05-17T22:38:57.414 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-17T22:38:57.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.529+0000 7fb1b1bf0640 1 -- 172.21.15.12:0/445766610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1ac1062d0 msgr2=0x7fb1ac108750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:57.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.529+0000 7fb1b1bf0640 1 --2- 172.21.15.12:0/445766610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1ac1062d0 0x7fb1ac108750 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0009900 tx=0x7fb1a002a3f0 comp rx=0 tx=0).stop 2024-05-17T22:38:57.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.530+0000 7fb1b1bf0640 1 -- 172.21.15.12:0/445766610 shutdown_connections 2024-05-17T22:38:57.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.530+0000 7fb1b1bf0640 1 --2- 172.21.15.12:0/445766610 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1ac108e10 0x7fb1ac10b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:57.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.530+0000 7fb1b1bf0640 1 --2- 172.21.15.12:0/445766610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1ac1062d0 0x7fb1ac108750 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:57.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.530+0000 7fb1b1bf0640 1 --2- 172.21.15.12:0/445766610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1ac06bf30 0x7fb1ac105d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:57.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.530+0000 7fb1b1bf0640 1 -- 172.21.15.12:0/445766610 >> 172.21.15.12:0/445766610 conn(0x7fb1ac0fd860 msgr2=0x7fb1ac0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:57.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.530+0000 7fb1b1bf0640 1 -- 172.21.15.12:0/445766610 shutdown_connections 2024-05-17T22:38:57.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.530+0000 7fb1b1bf0640 1 -- 172.21.15.12:0/445766610 wait complete. 2024-05-17T22:38:57.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.530+0000 7fb1b1bf0640 1 Processor -- start 2024-05-17T22:38:57.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.530+0000 7fb1b1bf0640 1 -- start start 2024-05-17T22:38:57.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.531+0000 7fb1b1bf0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1ac06bf30 0x7fb1ac199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:57.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.531+0000 7fb1b1bf0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1ac1062d0 0x7fb1ac199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:57.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.531+0000 7fb1b1bf0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1ac108e10 0x7fb1ac1a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:57.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.531+0000 7fb1b1bf0640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ac10ec90 con 0x7fb1ac108e10 2024-05-17T22:38:57.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.531+0000 7fb1b1bf0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ac10eb10 con 0x7fb1ac06bf30 2024-05-17T22:38:57.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.531+0000 7fb1b1bf0640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ac10ee10 con 0x7fb1ac1062d0 2024-05-17T22:38:57.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.531+0000 7fb1abfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1ac108e10 0x7fb1ac1a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:57.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.531+0000 7fb1abfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1ac108e10 0x7fb1ac1a0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33708/0 (socket says 172.21.15.12:33708) 2024-05-17T22:38:57.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.531+0000 7fb1abfff640 1 -- 172.21.15.12:0/895037897 learned_addr learned my addr 172.21.15.12:0/895037897 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:57.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.531+0000 7fb1aaffd640 1 --2- 172.21.15.12:0/895037897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1ac1062d0 0x7fb1ac199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:57.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.531+0000 7fb1ab7fe640 1 --2- 172.21.15.12:0/895037897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1ac06bf30 0x7fb1ac199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:57.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.531+0000 7fb1abfff640 1 -- 172.21.15.12:0/895037897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1ac1062d0 msgr2=0x7fb1ac199dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:57.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.531+0000 7fb1abfff640 1 --2- 172.21.15.12:0/895037897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1ac1062d0 0x7fb1ac199dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:57.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.531+0000 7fb1abfff640 1 -- 172.21.15.12:0/895037897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1ac06bf30 msgr2=0x7fb1ac199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:57.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.531+0000 7fb1abfff640 1 --2- 172.21.15.12:0/895037897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1ac06bf30 0x7fb1ac199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:57.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.531+0000 7fb1abfff640 1 -- 172.21.15.12:0/895037897 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1ac1a1320 con 0x7fb1ac108e10 2024-05-17T22:38:57.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.531+0000 7fb1abfff640 1 --2- 172.21.15.12:0/895037897 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1ac108e10 0x7fb1ac1a0d60 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb1a400ccb0 tx=0x7fb1a400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:57.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.532+0000 7fb1a8ff9640 1 -- 172.21.15.12:0/895037897 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a4004b90 con 0x7fb1ac108e10 2024-05-17T22:38:57.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.532+0000 7fb1b1bf0640 1 -- 172.21.15.12:0/895037897 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1ac1a1610 con 0x7fb1ac108e10 2024-05-17T22:38:57.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.532+0000 7fb1a8ff9640 1 -- 172.21.15.12:0/895037897 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1a4004d30 con 0x7fb1ac108e10 2024-05-17T22:38:57.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.532+0000 7fb1a8ff9640 1 -- 172.21.15.12:0/895037897 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a4005320 con 0x7fb1ac108e10 2024-05-17T22:38:57.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.532+0000 7fb1b1bf0640 1 -- 172.21.15.12:0/895037897 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1ac1a1b60 con 0x7fb1ac108e10 2024-05-17T22:38:57.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.533+0000 7fb1b1bf0640 1 -- 172.21.15.12:0/895037897 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1ac0ff380 con 0x7fb1ac108e10 2024-05-17T22:38:57.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.534+0000 7fb1a8ff9640 1 -- 172.21.15.12:0/895037897 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb1a400f450 con 0x7fb1ac108e10 2024-05-17T22:38:57.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.535+0000 7fb1a8ff9640 1 --2- 172.21.15.12:0/895037897 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb1940b27f0 0x7fb1940b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:57.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.535+0000 7fb1ab7fe640 1 --2- 172.21.15.12:0/895037897 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb1940b27f0 0x7fb1940b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:57.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.535+0000 7fb1a8ff9640 1 -- 172.21.15.12:0/895037897 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fb1a40e2b50 con 0x7fb1ac108e10 2024-05-17T22:38:57.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.536+0000 7fb1ab7fe640 1 --2- 172.21.15.12:0/895037897 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb1940b27f0 0x7fb1940b4d00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb19c0058c0 tx=0x7fb19c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:57.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.538+0000 7fb1a8ff9640 1 -- 172.21.15.12:0/895037897 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb1a4014070 con 0x7fb1ac108e10 2024-05-17T22:38:57.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.730+0000 7fb1b1bf0640 1 -- 172.21.15.12:0/895037897 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb1ac063940 con 0x7fb1ac108e10 2024-05-17T22:38:57.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.731+0000 7fb1a8ff9640 1 -- 172.21.15.12:0/895037897 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7fb1a40aa480 con 0x7fb1ac108e10 2024-05-17T22:38:57.734 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-05-17T22:38:57.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.733+0000 7fb1b1bf0640 1 -- 172.21.15.12:0/895037897 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb1940b27f0 msgr2=0x7fb1940b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:57.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.733+0000 7fb1b1bf0640 1 --2- 172.21.15.12:0/895037897 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb1940b27f0 0x7fb1940b4d00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb19c0058c0 tx=0x7fb19c005850 comp rx=0 tx=0).stop 2024-05-17T22:38:57.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.733+0000 7fb1b1bf0640 1 -- 172.21.15.12:0/895037897 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1ac108e10 msgr2=0x7fb1ac1a0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:57.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.733+0000 7fb1b1bf0640 1 --2- 172.21.15.12:0/895037897 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1ac108e10 0x7fb1ac1a0d60 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb1a400ccb0 tx=0x7fb1a400a620 comp rx=0 tx=0).stop 2024-05-17T22:38:57.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.733+0000 7fb1b1bf0640 1 -- 172.21.15.12:0/895037897 shutdown_connections 2024-05-17T22:38:57.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.733+0000 7fb1b1bf0640 1 --2- 172.21.15.12:0/895037897 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb1940b27f0 0x7fb1940b4d00 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:57.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.733+0000 7fb1b1bf0640 1 --2- 172.21.15.12:0/895037897 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1ac108e10 0x7fb1ac1a0d60 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:57.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.733+0000 7fb1b1bf0640 1 --2- 172.21.15.12:0/895037897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1ac1062d0 0x7fb1ac199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:57.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.733+0000 7fb1b1bf0640 1 --2- 172.21.15.12:0/895037897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1ac06bf30 0x7fb1ac199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:57.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.733+0000 7fb1b1bf0640 1 -- 172.21.15.12:0/895037897 >> 172.21.15.12:0/895037897 conn(0x7fb1ac0fd860 msgr2=0x7fb1ac104510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:57.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.733+0000 7fb1b1bf0640 1 -- 172.21.15.12:0/895037897 shutdown_connections 2024-05-17T22:38:57.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.733+0000 7fb1b1bf0640 1 -- 172.21.15.12:0/895037897 wait complete. 2024-05-17T22:38:57.748 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-17T22:38:57.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.864+0000 7f8564f19640 1 -- 172.21.15.12:0/2289302499 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85601065b0 msgr2=0x7f85601089d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:57.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.864+0000 7f8564f19640 1 --2- 172.21.15.12:0/2289302499 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85601065b0 0x7f85601089d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8548009900 tx=0x7f854802a3f0 comp rx=0 tx=0).stop 2024-05-17T22:38:57.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.864+0000 7f8564f19640 1 -- 172.21.15.12:0/2289302499 shutdown_connections 2024-05-17T22:38:57.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.864+0000 7f8564f19640 1 --2- 172.21.15.12:0/2289302499 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8560108f10 0x7f856010b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:57.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.864+0000 7f8564f19640 1 --2- 172.21.15.12:0/2289302499 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85601065b0 0x7f85601089d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:57.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.864+0000 7f8564f19640 1 --2- 172.21.15.12:0/2289302499 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8560103c50 0x7f8560106070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:57.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.864+0000 7f8564f19640 1 -- 172.21.15.12:0/2289302499 >> 172.21.15.12:0/2289302499 conn(0x7f85600fd860 msgr2=0x7f85600ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:57.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.864+0000 7f8564f19640 1 -- 172.21.15.12:0/2289302499 shutdown_connections 2024-05-17T22:38:57.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.864+0000 7f8564f19640 1 -- 172.21.15.12:0/2289302499 wait complete. 2024-05-17T22:38:57.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.865+0000 7f8564f19640 1 Processor -- start 2024-05-17T22:38:57.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.865+0000 7f8564f19640 1 -- start start 2024-05-17T22:38:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.865+0000 7f8564f19640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8560103c50 0x7f85601034c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.865+0000 7f8564f19640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85601065b0 0x7f8560101b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.865+0000 7f8564f19640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8560108f10 0x7f85601020a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.865+0000 7f8564f19640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f856010ec80 con 0x7f8560108f10 2024-05-17T22:38:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.865+0000 7f8564f19640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f856010eb00 con 0x7f85601065b0 2024-05-17T22:38:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.865+0000 7f8564f19640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f856010ee00 con 0x7f8560103c50 2024-05-17T22:38:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.865+0000 7f855ed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8560108f10 0x7f85601020a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.865+0000 7f855ed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8560108f10 0x7f85601020a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33728/0 (socket says 172.21.15.12:33728) 2024-05-17T22:38:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.865+0000 7f855ed9d640 1 -- 172.21.15.12:0/53064119 learned_addr learned my addr 172.21.15.12:0/53064119 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.865+0000 7f855e59c640 1 --2- 172.21.15.12:0/53064119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8560103c50 0x7f85601034c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.865+0000 7f8557fff640 1 --2- 172.21.15.12:0/53064119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85601065b0 0x7f8560101b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.865+0000 7f855ed9d640 1 -- 172.21.15.12:0/53064119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8560103c50 msgr2=0x7f85601034c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:57.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.865+0000 7f855ed9d640 1 --2- 172.21.15.12:0/53064119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8560103c50 0x7f85601034c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:57.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.865+0000 7f855ed9d640 1 -- 172.21.15.12:0/53064119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85601065b0 msgr2=0x7f8560101b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:57.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.865+0000 7f855ed9d640 1 --2- 172.21.15.12:0/53064119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85601065b0 0x7f8560101b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:57.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.865+0000 7f855ed9d640 1 -- 172.21.15.12:0/53064119 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8560102740 con 0x7f8560108f10 2024-05-17T22:38:57.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.866+0000 7f855ed9d640 1 --2- 172.21.15.12:0/53064119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8560108f10 0x7f85601020a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f855800cce0 tx=0x7f855800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:57.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.866+0000 7f85577fe640 1 -- 172.21.15.12:0/53064119 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855800f450 con 0x7f8560108f10 2024-05-17T22:38:57.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.866+0000 7f8564f19640 1 -- 172.21.15.12:0/53064119 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85601a3970 con 0x7f8560108f10 2024-05-17T22:38:57.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.866+0000 7f85577fe640 1 -- 172.21.15.12:0/53064119 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f855800faa0 con 0x7f8560108f10 2024-05-17T22:38:57.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.866+0000 7f85577fe640 1 -- 172.21.15.12:0/53064119 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8558018b70 con 0x7f8560108f10 2024-05-17T22:38:57.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.866+0000 7f8564f19640 1 -- 172.21.15.12:0/53064119 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85601a3d80 con 0x7f8560108f10 2024-05-17T22:38:57.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.867+0000 7f8564f19640 1 -- 172.21.15.12:0/53064119 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85600ff380 con 0x7f8560108f10 2024-05-17T22:38:57.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.868+0000 7f85577fe640 1 -- 172.21.15.12:0/53064119 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8558018d10 con 0x7f8560108f10 2024-05-17T22:38:57.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.869+0000 7f85577fe640 1 --2- 172.21.15.12:0/53064119 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f854c0b27f0 0x7f854c0b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:57.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.869+0000 7f85577fe640 1 -- 172.21.15.12:0/53064119 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f85580e2ab0 con 0x7f8560108f10 2024-05-17T22:38:57.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.869+0000 7f855e59c640 1 --2- 172.21.15.12:0/53064119 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f854c0b27f0 0x7f854c0b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:57.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.870+0000 7f855e59c640 1 --2- 172.21.15.12:0/53064119 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f854c0b27f0 0x7f854c0b4d00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8550005fd0 tx=0x7f855000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:57.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:57.872+0000 7f85577fe640 1 -- 172.21.15.12:0/53064119 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8558014030 con 0x7f8560108f10 2024-05-17T22:38:57.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:57 smithi151 ceph-mon[36101]: pgmap v430: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T22:38:57.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:57 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/563213630' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-17T22:38:57.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:57 smithi151 ceph-mon[36101]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:38:57.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:57 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/400052529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:38:57.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:57 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/4070417874' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-17T22:38:58.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:57 smithi096 ceph-mon[36065]: pgmap v430: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T22:38:58.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:57 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/563213630' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-17T22:38:58.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:57 smithi096 ceph-mon[36065]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:38:58.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:57 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/400052529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:38:58.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:57 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/4070417874' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-17T22:38:58.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.065+0000 7f8564f19640 1 -- 172.21.15.12:0/53064119 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f856002dca0 con 0x7f8560108f10 2024-05-17T22:38:58.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.065+0000 7f85577fe640 1 -- 172.21.15.12:0/53064119 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f85580aa4d0 con 0x7f8560108f10 2024-05-17T22:38:58.069 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-05-17T22:38:58.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.067+0000 7f8564f19640 1 -- 172.21.15.12:0/53064119 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f854c0b27f0 msgr2=0x7f854c0b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:58.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.067+0000 7f8564f19640 1 --2- 172.21.15.12:0/53064119 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f854c0b27f0 0x7f854c0b4d00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8550005fd0 tx=0x7f855000b390 comp rx=0 tx=0).stop 2024-05-17T22:38:58.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.067+0000 7f8564f19640 1 -- 172.21.15.12:0/53064119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8560108f10 msgr2=0x7f85601020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:58.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.067+0000 7f8564f19640 1 --2- 172.21.15.12:0/53064119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8560108f10 0x7f85601020a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f855800cce0 tx=0x7f855800a620 comp rx=0 tx=0).stop 2024-05-17T22:38:58.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.067+0000 7f8564f19640 1 -- 172.21.15.12:0/53064119 shutdown_connections 2024-05-17T22:38:58.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.067+0000 7f8564f19640 1 --2- 172.21.15.12:0/53064119 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f854c0b27f0 0x7f854c0b4d00 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:58.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.068+0000 7f8564f19640 1 --2- 172.21.15.12:0/53064119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8560108f10 0x7f85601020a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:58.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.068+0000 7f8564f19640 1 --2- 172.21.15.12:0/53064119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85601065b0 0x7f8560101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:58.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.068+0000 7f8564f19640 1 --2- 172.21.15.12:0/53064119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8560103c50 0x7f85601034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:58.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.068+0000 7f8564f19640 1 -- 172.21.15.12:0/53064119 >> 172.21.15.12:0/53064119 conn(0x7f85600fd860 msgr2=0x7f8560104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:58.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.068+0000 7f8564f19640 1 -- 172.21.15.12:0/53064119 shutdown_connections 2024-05-17T22:38:58.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.068+0000 7f8564f19640 1 -- 172.21.15.12:0/53064119 wait complete. 2024-05-17T22:38:58.081 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:57 smithi012 ceph-mon[31632]: pgmap v430: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T22:38:58.081 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:57 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/563213630' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-17T22:38:58.082 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:57 smithi012 ceph-mon[31632]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:38:58.082 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:57 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/400052529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:38:58.082 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:57 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/4070417874' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-17T22:38:58.083 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-17T22:38:58.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.198+0000 7f12d4ff5640 1 -- 172.21.15.12:0/2642708838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12d01026c0 msgr2=0x7f12d010a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:58.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.198+0000 7f12d4ff5640 1 --2- 172.21.15.12:0/2642708838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12d01026c0 0x7f12d010a3d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f12c8009900 tx=0x7f12c802a410 comp rx=0 tx=0).stop 2024-05-17T22:38:58.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.198+0000 7f12d4ff5640 1 -- 172.21.15.12:0/2642708838 shutdown_connections 2024-05-17T22:38:58.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.198+0000 7f12d4ff5640 1 --2- 172.21.15.12:0/2642708838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12d01026c0 0x7f12d010a3d0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:58.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.198+0000 7f12d4ff5640 1 --2- 172.21.15.12:0/2642708838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12d0101c40 0x7f12d0102050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:58.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.198+0000 7f12d4ff5640 1 --2- 172.21.15.12:0/2642708838 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12d01036a0 0x7f12d0103ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:58.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.198+0000 7f12d4ff5640 1 -- 172.21.15.12:0/2642708838 >> 172.21.15.12:0/2642708838 conn(0x7f12d00fd880 msgr2=0x7f12d00ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:58.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.198+0000 7f12d4ff5640 1 -- 172.21.15.12:0/2642708838 shutdown_connections 2024-05-17T22:38:58.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.198+0000 7f12d4ff5640 1 -- 172.21.15.12:0/2642708838 wait complete. 2024-05-17T22:38:58.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.199+0000 7f12d4ff5640 1 Processor -- start 2024-05-17T22:38:58.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.199+0000 7f12d4ff5640 1 -- start start 2024-05-17T22:38:58.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.199+0000 7f12d4ff5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12d0101c40 0x7f12d0198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:58.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.199+0000 7f12d4ff5640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12d01026c0 0x7f12d0198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:58.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.199+0000 7f12d4ff5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12d01036a0 0x7f12d019fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:58.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.199+0000 7f12d4ff5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f12d010dca0 con 0x7f12d0101c40 2024-05-17T22:38:58.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.200+0000 7f12d4ff5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f12d010db20 con 0x7f12d01036a0 2024-05-17T22:38:58.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.200+0000 7f12d4ff5640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f12d010de20 con 0x7f12d01026c0 2024-05-17T22:38:58.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.200+0000 7f12ce59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12d0101c40 0x7f12d0198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:58.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.200+0000 7f12ce59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12d0101c40 0x7f12d0198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33738/0 (socket says 172.21.15.12:33738) 2024-05-17T22:38:58.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.200+0000 7f12ce59c640 1 -- 172.21.15.12:0/3158991272 learned_addr learned my addr 172.21.15.12:0/3158991272 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:38:58.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.200+0000 7f12ced9d640 1 --2- 172.21.15.12:0/3158991272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12d01036a0 0x7f12d019fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:58.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.200+0000 7f12cdd9b640 1 --2- 172.21.15.12:0/3158991272 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12d01026c0 0x7f12d0198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:58.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.200+0000 7f12ce59c640 1 -- 172.21.15.12:0/3158991272 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12d01026c0 msgr2=0x7f12d0198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:58.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.200+0000 7f12ce59c640 1 --2- 172.21.15.12:0/3158991272 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12d01026c0 0x7f12d0198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:58.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.200+0000 7f12ce59c640 1 -- 172.21.15.12:0/3158991272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12d01036a0 msgr2=0x7f12d019fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:58.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.200+0000 7f12ce59c640 1 --2- 172.21.15.12:0/3158991272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12d01036a0 0x7f12d019fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:58.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.200+0000 7f12ce59c640 1 -- 172.21.15.12:0/3158991272 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12d01a03a0 con 0x7f12d0101c40 2024-05-17T22:38:58.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.200+0000 7f12ced9d640 1 --2- 172.21.15.12:0/3158991272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12d01036a0 0x7f12d019fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:38:58.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.200+0000 7f12ce59c640 1 --2- 172.21.15.12:0/3158991272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12d0101c40 0x7f12d0198860 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f12c000d740 tx=0x7f12c000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:58.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.200+0000 7f12bf7fe640 1 -- 172.21.15.12:0/3158991272 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c0017070 con 0x7f12d0101c40 2024-05-17T22:38:58.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.200+0000 7f12d4ff5640 1 -- 172.21.15.12:0/3158991272 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12d01a0690 con 0x7f12d0101c40 2024-05-17T22:38:58.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.200+0000 7f12bf7fe640 1 -- 172.21.15.12:0/3158991272 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12c000de50 con 0x7f12d0101c40 2024-05-17T22:38:58.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.200+0000 7f12bf7fe640 1 -- 172.21.15.12:0/3158991272 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c0004ce0 con 0x7f12d0101c40 2024-05-17T22:38:58.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.200+0000 7f12d4ff5640 1 -- 172.21.15.12:0/3158991272 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12d01a0be0 con 0x7f12d0101c40 2024-05-17T22:38:58.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.202+0000 7f12d4ff5640 1 -- 172.21.15.12:0/3158991272 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12d0102050 con 0x7f12d0101c40 2024-05-17T22:38:58.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.203+0000 7f12bf7fe640 1 -- 172.21.15.12:0/3158991272 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f12c0004e80 con 0x7f12d0101c40 2024-05-17T22:38:58.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.204+0000 7f12bf7fe640 1 --2- 172.21.15.12:0/3158991272 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f12b80b27f0 0x7f12b80b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:38:58.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.204+0000 7f12bf7fe640 1 -- 172.21.15.12:0/3158991272 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f12c00e23c0 con 0x7f12d0101c40 2024-05-17T22:38:58.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.204+0000 7f12cdd9b640 1 --2- 172.21.15.12:0/3158991272 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f12b80b27f0 0x7f12b80b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:38:58.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.205+0000 7f12cdd9b640 1 --2- 172.21.15.12:0/3158991272 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f12b80b27f0 0x7f12b80b4d00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f12d0199900 tx=0x7f12c4009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:38:58.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.208+0000 7f12bf7fe640 1 -- 172.21.15.12:0/3158991272 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f12c00a9cf0 con 0x7f12d0101c40 2024-05-17T22:38:58.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.402+0000 7f12d4ff5640 1 -- 172.21.15.12:0/3158991272 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f12d01091e0 con 0x7f12d0101c40 2024-05-17T22:38:58.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.402+0000 7f12bf7fe640 1 -- 172.21.15.12:0/3158991272 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) ==== 103+0+11 (secure 0 0 0) 0x7f12c00a95f0 con 0x7f12d0101c40 2024-05-17T22:38:58.406 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-05-17T22:38:58.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.404+0000 7f12d4ff5640 1 -- 172.21.15.12:0/3158991272 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f12b80b27f0 msgr2=0x7f12b80b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:58.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.404+0000 7f12d4ff5640 1 --2- 172.21.15.12:0/3158991272 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f12b80b27f0 0x7f12b80b4d00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f12d0199900 tx=0x7f12c4009680 comp rx=0 tx=0).stop 2024-05-17T22:38:58.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.404+0000 7f12d4ff5640 1 -- 172.21.15.12:0/3158991272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12d0101c40 msgr2=0x7f12d0198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:38:58.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.404+0000 7f12d4ff5640 1 --2- 172.21.15.12:0/3158991272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12d0101c40 0x7f12d0198860 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f12c000d740 tx=0x7f12c000dc00 comp rx=0 tx=0).stop 2024-05-17T22:38:58.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.404+0000 7f12d4ff5640 1 -- 172.21.15.12:0/3158991272 shutdown_connections 2024-05-17T22:38:58.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.404+0000 7f12d4ff5640 1 --2- 172.21.15.12:0/3158991272 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f12b80b27f0 0x7f12b80b4d00 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:58.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.404+0000 7f12d4ff5640 1 --2- 172.21.15.12:0/3158991272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12d01036a0 0x7f12d019fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:58.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.404+0000 7f12d4ff5640 1 --2- 172.21.15.12:0/3158991272 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12d01026c0 0x7f12d0198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:58.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.404+0000 7f12d4ff5640 1 --2- 172.21.15.12:0/3158991272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12d0101c40 0x7f12d0198860 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:38:58.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.404+0000 7f12d4ff5640 1 -- 172.21.15.12:0/3158991272 >> 172.21.15.12:0/3158991272 conn(0x7f12d00fd880 msgr2=0x7f12d01063b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:38:58.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.404+0000 7f12d4ff5640 1 -- 172.21.15.12:0/3158991272 shutdown_connections 2024-05-17T22:38:58.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:38:58.404+0000 7f12d4ff5640 1 -- 172.21.15.12:0/3158991272 wait complete. 2024-05-17T22:38:58.419 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-05-17T22:38:58.419 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T22:38:58.672 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:38:58.965 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:58 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/895037897' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-17T22:38:58.965 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:58 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/53064119' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-17T22:38:58.966 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:58 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/3158991272' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-17T22:38:58.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:58 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/895037897' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-17T22:38:58.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:58 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/53064119' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-17T22:38:58.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:58 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/3158991272' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-17T22:38:59.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:58 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/895037897' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-17T22:38:59.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:58 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/53064119' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-17T22:38:59.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:58 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/3158991272' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-17T22:38:59.971 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:38:59 smithi012 ceph-mon[31632]: pgmap v431: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-17T22:38:59.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:38:59 smithi151 ceph-mon[36101]: pgmap v431: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-17T22:39:00.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:38:59 smithi096 ceph-mon[36065]: pgmap v431: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-17T22:39:00.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.553+0000 7f35b93ad700 1 -- 172.21.15.12:0/2184732700 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f35b41080a0 msgr2=0x7f35b410fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:00.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.553+0000 7f35b93ad700 1 --2- 172.21.15.12:0/2184732700 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f35b41080a0 0x7f35b410fa00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f35ac009990 tx=0x7f35ac009ca0 comp rx=0 tx=0).stop 2024-05-17T22:39:00.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.553+0000 7f35b93ad700 1 -- 172.21.15.12:0/2184732700 shutdown_connections 2024-05-17T22:39:00.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.553+0000 7f35b93ad700 1 --2- 172.21.15.12:0/2184732700 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f35b41080a0 0x7f35b410fa00 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:00.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.553+0000 7f35b93ad700 1 --2- 172.21.15.12:0/2184732700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35b4102e70 0x7f35b4103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:00.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.553+0000 7f35b93ad700 1 --2- 172.21.15.12:0/2184732700 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f35b4107750 0x7f35b4107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:00.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.553+0000 7f35b93ad700 1 -- 172.21.15.12:0/2184732700 >> 172.21.15.12:0/2184732700 conn(0x7f35b4079ae0 msgr2=0x7f35b4079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:39:00.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.553+0000 7f35b93ad700 1 -- 172.21.15.12:0/2184732700 shutdown_connections 2024-05-17T22:39:00.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.553+0000 7f35b93ad700 1 -- 172.21.15.12:0/2184732700 wait complete. 2024-05-17T22:39:00.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.553+0000 7f35b93ad700 1 Processor -- start 2024-05-17T22:39:00.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.554+0000 7f35b93ad700 1 -- start start 2024-05-17T22:39:00.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.554+0000 7f35b93ad700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f35b4102e70 0x7f35b4199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:00.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.554+0000 7f35b93ad700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35b4107750 0x7f35b4199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:00.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.554+0000 7f35b93ad700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f35b41080a0 0x7f35b419de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:00.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.554+0000 7f35b93ad700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f35b4112ce0 con 0x7f35b41080a0 2024-05-17T22:39:00.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.554+0000 7f35b93ad700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f35b4112b60 con 0x7f35b4107750 2024-05-17T22:39:00.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.554+0000 7f35b93ad700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f35b4112e60 con 0x7f35b4102e70 2024-05-17T22:39:00.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.554+0000 7f35b37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f35b41080a0 0x7f35b419de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:00.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.554+0000 7f35b37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f35b41080a0 0x7f35b419de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60932/0 (socket says 172.21.15.12:60932) 2024-05-17T22:39:00.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.554+0000 7f35b37fe700 1 -- 172.21.15.12:0/2442701764 learned_addr learned my addr 172.21.15.12:0/2442701764 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:39:00.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.554+0000 7f35b2ffd700 1 --2- 172.21.15.12:0/2442701764 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f35b4102e70 0x7f35b4199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:00.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.554+0000 7f35b27fc700 1 --2- 172.21.15.12:0/2442701764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35b4107750 0x7f35b4199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:00.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.555+0000 7f35b37fe700 1 -- 172.21.15.12:0/2442701764 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f35b4102e70 msgr2=0x7f35b4199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:00.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.555+0000 7f35b37fe700 1 --2- 172.21.15.12:0/2442701764 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f35b4102e70 0x7f35b4199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:00.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.555+0000 7f35b37fe700 1 -- 172.21.15.12:0/2442701764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35b4107750 msgr2=0x7f35b4199c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:00.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.555+0000 7f35b37fe700 1 --2- 172.21.15.12:0/2442701764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35b4107750 0x7f35b4199c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:00.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.555+0000 7f35b37fe700 1 -- 172.21.15.12:0/2442701764 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35b419e530 con 0x7f35b41080a0 2024-05-17T22:39:00.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.555+0000 7f35b37fe700 1 --2- 172.21.15.12:0/2442701764 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f35b41080a0 0x7f35b419de90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f35ac0047b0 tx=0x7f35ac005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:39:00.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.555+0000 7f359bfff700 1 -- 172.21.15.12:0/2442701764 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ac028070 con 0x7f35b41080a0 2024-05-17T22:39:00.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.555+0000 7f359bfff700 1 -- 172.21.15.12:0/2442701764 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35ac019cc0 con 0x7f35b41080a0 2024-05-17T22:39:00.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.555+0000 7f35b93ad700 1 -- 172.21.15.12:0/2442701764 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35b419e7c0 con 0x7f35b41080a0 2024-05-17T22:39:00.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.555+0000 7f359bfff700 1 -- 172.21.15.12:0/2442701764 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ac005290 con 0x7f35b41080a0 2024-05-17T22:39:00.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.555+0000 7f35b93ad700 1 -- 172.21.15.12:0/2442701764 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35b419eca0 con 0x7f35b41080a0 2024-05-17T22:39:00.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.556+0000 7f35b93ad700 1 -- 172.21.15.12:0/2442701764 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35b4103280 con 0x7f35b41080a0 2024-05-17T22:39:00.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.558+0000 7f359bfff700 1 -- 172.21.15.12:0/2442701764 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f35ac02b070 con 0x7f35b41080a0 2024-05-17T22:39:00.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.559+0000 7f359bfff700 1 --2- 172.21.15.12:0/2442701764 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f359c0b2880 0x7f359c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:00.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.559+0000 7f35b2ffd700 1 --2- 172.21.15.12:0/2442701764 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f359c0b2880 0x7f359c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:00.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.559+0000 7f359bfff700 1 -- 172.21.15.12:0/2442701764 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f35ac0e5450 con 0x7f35b41080a0 2024-05-17T22:39:00.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.560+0000 7f35b2ffd700 1 --2- 172.21.15.12:0/2442701764 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f359c0b2880 0x7f359c0b4d90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f35a4009550 tx=0x7f35a40091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:39:00.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.563+0000 7f359bfff700 1 -- 172.21.15.12:0/2442701764 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f35ac032080 con 0x7f35b41080a0 2024-05-17T22:39:00.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.826+0000 7f35b93ad700 1 -- 172.21.15.12:0/2442701764 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35b410af50 con 0x7f35b41080a0 2024-05-17T22:39:00.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.827+0000 7f359bfff700 1 -- 172.21.15.12:0/2442701764 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11870 (secure 0 0 0) 0x7f35ac013070 con 0x7f35b41080a0 2024-05-17T22:39:00.830 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:39:00.831 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1407219319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1407219319},{"type":"v1","addr":"172.21.15.12:6835","nonce":1407219319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/630977886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":630977886},{"type":"v1","addr":"172.21.15.12:6837","nonce":630977886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3731914742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3731914742},{"type":"v1","addr":"172.21.15.12:6839","nonce":3731914742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/3642142501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3642142501},{"type":"v1","addr":"172.21.15.12:6841","nonce":3642142501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3554053059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3554053059},{"type":"v1","addr":"172.21.15.151:6833","nonce":3554053059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/4228325569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":4228325569},{"type":"v1","addr":"172.21.15.96:6833","nonce":4228325569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T22:34:42.642111+0000","modified":"2024-05-17T22:38:56.118632+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24461,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/542323754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":542323754},{"type":"v1","addr":"172.21.15.151:6835","nonce":542323754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1758442232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1758442232},{"type":"v1","addr":"172.21.15.96:6835","nonce":1758442232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3403324038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3403324038},{"type":"v1","addr":"172.21.15.151:6837","nonce":3403324038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.151:6839/2338353849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2338353849},{"type":"v1","addr":"172.21.15.151:6839","nonce":2338353849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.96:6837/4222574338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":4222574338},{"type":"v1","addr":"172.21.15.96:6837","nonce":4222574338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/2986295519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2986295519},{"type":"v1","addr":"172.21.15.96:6839","nonce":2986295519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24457,24461,24482]},"id":1}]} 2024-05-17T22:39:00.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.830+0000 7f35b93ad700 1 -- 172.21.15.12:0/2442701764 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f359c0b2880 msgr2=0x7f359c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:00.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.830+0000 7f35b93ad700 1 --2- 172.21.15.12:0/2442701764 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f359c0b2880 0x7f359c0b4d90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f35a4009550 tx=0x7f35a40091b0 comp rx=0 tx=0).stop 2024-05-17T22:39:00.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.830+0000 7f35b93ad700 1 -- 172.21.15.12:0/2442701764 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f35b41080a0 msgr2=0x7f35b419de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:00.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.830+0000 7f35b93ad700 1 --2- 172.21.15.12:0/2442701764 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f35b41080a0 0x7f35b419de90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f35ac0047b0 tx=0x7f35ac005e80 comp rx=0 tx=0).stop 2024-05-17T22:39:00.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.830+0000 7f35b93ad700 1 -- 172.21.15.12:0/2442701764 shutdown_connections 2024-05-17T22:39:00.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.830+0000 7f35b93ad700 1 --2- 172.21.15.12:0/2442701764 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f359c0b2880 0x7f359c0b4d90 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:00.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.830+0000 7f35b93ad700 1 --2- 172.21.15.12:0/2442701764 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f35b41080a0 0x7f35b419de90 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:00.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.830+0000 7f35b93ad700 1 --2- 172.21.15.12:0/2442701764 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f35b4102e70 0x7f35b4199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:00.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.830+0000 7f35b93ad700 1 --2- 172.21.15.12:0/2442701764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35b4107750 0x7f35b4199c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:00.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.830+0000 7f35b93ad700 1 -- 172.21.15.12:0/2442701764 >> 172.21.15.12:0/2442701764 conn(0x7f35b4079ae0 msgr2=0x7f35b41052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:39:00.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.830+0000 7f35b93ad700 1 -- 172.21.15.12:0/2442701764 shutdown_connections 2024-05-17T22:39:00.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:00.830+0000 7f35b93ad700 1 -- 172.21.15.12:0/2442701764 wait complete. 2024-05-17T22:39:00.834 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 27 2024-05-17T22:39:01.533 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-05-17T22:39:01.534 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-05-17T22:39:01.534 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-17T22:39:01.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.653+0000 7f168a688640 1 -- 172.21.15.12:0/1379432009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1684102060 msgr2=0x7f1684102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:01.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.653+0000 7f168a688640 1 --2- 172.21.15.12:0/1379432009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1684102060 0x7f1684102470 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1674009900 tx=0x7f167402a410 comp rx=0 tx=0).stop 2024-05-17T22:39:01.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.654+0000 7f168a688640 1 -- 172.21.15.12:0/1379432009 shutdown_connections 2024-05-17T22:39:01.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.654+0000 7f168a688640 1 --2- 172.21.15.12:0/1379432009 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1684103ad0 0x7f168410a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:01.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.654+0000 7f168a688640 1 --2- 172.21.15.12:0/1379432009 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1684103180 0x7f1684103590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:01.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.654+0000 7f168a688640 1 --2- 172.21.15.12:0/1379432009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1684102060 0x7f1684102470 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:01.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.654+0000 7f168a688640 1 -- 172.21.15.12:0/1379432009 >> 172.21.15.12:0/1379432009 conn(0x7f16840fd880 msgr2=0x7f16840ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:39:01.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.654+0000 7f168a688640 1 -- 172.21.15.12:0/1379432009 shutdown_connections 2024-05-17T22:39:01.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.654+0000 7f168a688640 1 -- 172.21.15.12:0/1379432009 wait complete. 2024-05-17T22:39:01.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.655+0000 7f168a688640 1 Processor -- start 2024-05-17T22:39:01.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.655+0000 7f168a688640 1 -- start start 2024-05-17T22:39:01.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.655+0000 7f168a688640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1684102060 0x7f1684199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:01.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.655+0000 7f168a688640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1684103180 0x7f1684199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:01.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.655+0000 7f168a688640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1684103ad0 0x7f16841a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:01.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.655+0000 7f168a688640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f168410eca0 con 0x7f1684102060 2024-05-17T22:39:01.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.655+0000 7f168a688640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f168410eb20 con 0x7f1684103ad0 2024-05-17T22:39:01.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.655+0000 7f168a688640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f168410ee20 con 0x7f1684103180 2024-05-17T22:39:01.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.655+0000 7f1683fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1684102060 0x7f1684199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:01.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.655+0000 7f1683fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1684102060 0x7f1684199860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60954/0 (socket says 172.21.15.12:60954) 2024-05-17T22:39:01.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.655+0000 7f1683fff640 1 -- 172.21.15.12:0/2304153408 learned_addr learned my addr 172.21.15.12:0/2304153408 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:39:01.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.655+0000 7f1688c25640 1 --2- 172.21.15.12:0/2304153408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1684103ad0 0x7f16841a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:01.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.655+0000 7f167bfff640 1 --2- 172.21.15.12:0/2304153408 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1684103180 0x7f1684199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:01.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.655+0000 7f1683fff640 1 -- 172.21.15.12:0/2304153408 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1684103180 msgr2=0x7f1684199da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:01.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.655+0000 7f1683fff640 1 --2- 172.21.15.12:0/2304153408 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1684103180 0x7f1684199da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:01.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.656+0000 7f1683fff640 1 -- 172.21.15.12:0/2304153408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1684103ad0 msgr2=0x7f16841a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:01.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.656+0000 7f1683fff640 1 --2- 172.21.15.12:0/2304153408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1684103ad0 0x7f16841a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:01.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.656+0000 7f1683fff640 1 -- 172.21.15.12:0/2304153408 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16841a13a0 con 0x7f1684102060 2024-05-17T22:39:01.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.656+0000 7f1688c25640 1 --2- 172.21.15.12:0/2304153408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1684103ad0 0x7f16841a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:39:01.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.656+0000 7f1683fff640 1 --2- 172.21.15.12:0/2304153408 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1684102060 0x7f1684199860 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1674006010 tx=0x7f1674004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:39:01.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.656+0000 7f1681ffb640 1 -- 172.21.15.12:0/2304153408 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167403f030 con 0x7f1684102060 2024-05-17T22:39:01.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.656+0000 7f1681ffb640 1 -- 172.21.15.12:0/2304153408 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1674003aa0 con 0x7f1684102060 2024-05-17T22:39:01.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.656+0000 7f168a688640 1 -- 172.21.15.12:0/2304153408 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16841a1630 con 0x7f1684102060 2024-05-17T22:39:01.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.656+0000 7f1681ffb640 1 -- 172.21.15.12:0/2304153408 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1674035070 con 0x7f1684102060 2024-05-17T22:39:01.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.656+0000 7f168a688640 1 -- 172.21.15.12:0/2304153408 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16841a1b20 con 0x7f1684102060 2024-05-17T22:39:01.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.657+0000 7f168a688640 1 -- 172.21.15.12:0/2304153408 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16841091e0 con 0x7f1684102060 2024-05-17T22:39:01.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.658+0000 7f1681ffb640 1 -- 172.21.15.12:0/2304153408 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1674042030 con 0x7f1684102060 2024-05-17T22:39:01.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.660+0000 7f1681ffb640 1 --2- 172.21.15.12:0/2304153408 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f16700b27f0 0x7f16700b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:01.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.660+0000 7f167bfff640 1 --2- 172.21.15.12:0/2304153408 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f16700b27f0 0x7f16700b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:01.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.660+0000 7f1681ffb640 1 -- 172.21.15.12:0/2304153408 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f1674103170 con 0x7f1684102060 2024-05-17T22:39:01.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.660+0000 7f167bfff640 1 --2- 172.21.15.12:0/2304153408 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f16700b27f0 0x7f16700b4d00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f168419a900 tx=0x7f166c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:39:01.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.663+0000 7f1681ffb640 1 -- 172.21.15.12:0/2304153408 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f16740cab00 con 0x7f1684102060 2024-05-17T22:39:01.825 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:01 smithi012 ceph-mon[31632]: pgmap v432: 129 pgs: 129 active+clean; 648 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-17T22:39:01.825 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:01 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/2442701764' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:39:01.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.854+0000 7f168a688640 1 -- 172.21.15.12:0/2304153408 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1684063180 con 0x7f16700b27f0 2024-05-17T22:39:01.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.865+0000 7f1681ffb640 1 -- 172.21.15.12:0/2304153408 <== mgr.14152 v2:172.21.15.12:6800/1388514047 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340471 (secure 0 0 0) 0x7f1684063180 con 0x7f16700b27f0 2024-05-17T22:39:01.872 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:39:01.874 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-05-17T22:39:01.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.868+0000 7f168a688640 1 -- 172.21.15.12:0/2304153408 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f16700b27f0 msgr2=0x7f16700b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:01.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.868+0000 7f168a688640 1 --2- 172.21.15.12:0/2304153408 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f16700b27f0 0x7f16700b4d00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f168419a900 tx=0x7f166c005f90 comp rx=0 tx=0).stop 2024-05-17T22:39:01.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.868+0000 7f168a688640 1 -- 172.21.15.12:0/2304153408 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1684102060 msgr2=0x7f1684199860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:01.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.868+0000 7f168a688640 1 --2- 172.21.15.12:0/2304153408 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1684102060 0x7f1684199860 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1674006010 tx=0x7f1674004270 comp rx=0 tx=0).stop 2024-05-17T22:39:01.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.868+0000 7f168a688640 1 -- 172.21.15.12:0/2304153408 shutdown_connections 2024-05-17T22:39:01.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.868+0000 7f168a688640 1 --2- 172.21.15.12:0/2304153408 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f16700b27f0 0x7f16700b4d00 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:01.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.868+0000 7f168a688640 1 --2- 172.21.15.12:0/2304153408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1684103ad0 0x7f16841a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:01.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.868+0000 7f168a688640 1 --2- 172.21.15.12:0/2304153408 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1684103180 0x7f1684199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:01.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.868+0000 7f168a688640 1 --2- 172.21.15.12:0/2304153408 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1684102060 0x7f1684199860 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:01.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.868+0000 7f168a688640 1 -- 172.21.15.12:0/2304153408 >> 172.21.15.12:0/2304153408 conn(0x7f16840fd880 msgr2=0x7f16841063b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:39:01.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.868+0000 7f168a688640 1 -- 172.21.15.12:0/2304153408 shutdown_connections 2024-05-17T22:39:01.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:01.868+0000 7f168a688640 1 -- 172.21.15.12:0/2304153408 wait complete. 2024-05-17T22:39:01.884 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2024-05-17T22:39:00.554581+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663586,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1246,"num_read_kb":729,"num_write":310,"num_write_kb":2288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":752,"ondisk_log_size":752,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":388212,"kb_used_data":14100,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553612,"statfs":{"total":1151940427776,"available":1151542898688,"internally_reserved":0,"allocated":14438400,"data_stored":8136110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5989,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":83,"num_read_kb":53,"num_write":7,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003416"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.522500+0000","last_change":"2024-05-17T22:34:40.422850+0000","last_active":"2024-05-17T22:37:14.522500+0000","last_peered":"2024-05-17T22:37:14.522500+0000","last_clean":"2024-05-17T22:37:14.522500+0000","last_became_active":"2024-05-17T22:34:40.422474+0000","last_became_peered":"2024-05-17T22:34:40.422474+0000","last_unstale":"2024-05-17T22:37:14.522500+0000","last_undegraded":"2024-05-17T22:37:14.522500+0000","last_fullsized":"2024-05-17T22:37:14.522500+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:23:51.537128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.520370+0000","last_change":"2024-05-17T22:34:40.427907+0000","last_active":"2024-05-17T22:37:14.520370+0000","last_peered":"2024-05-17T22:37:14.520370+0000","last_clean":"2024-05-17T22:37:14.520370+0000","last_became_active":"2024-05-17T22:34:40.427511+0000","last_became_peered":"2024-05-17T22:34:40.427511+0000","last_unstale":"2024-05-17T22:37:14.520370+0000","last_undegraded":"2024-05-17T22:37:14.520370+0000","last_fullsized":"2024-05-17T22:37:14.520370+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:01:46.707820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.525760+0000","last_change":"2024-05-17T22:34:40.436370+0000","last_active":"2024-05-17T22:37:14.525760+0000","last_peered":"2024-05-17T22:37:14.525760+0000","last_clean":"2024-05-17T22:37:14.525760+0000","last_became_active":"2024-05-17T22:34:40.434089+0000","last_became_peered":"2024-05-17T22:34:40.434089+0000","last_unstale":"2024-05-17T22:37:14.525760+0000","last_undegraded":"2024-05-17T22:37:14.525760+0000","last_fullsized":"2024-05-17T22:37:14.525760+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:58:16.775471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.509106+0000","last_change":"2024-05-17T22:35:15.556681+0000","last_active":"2024-05-17T22:37:14.509106+0000","last_peered":"2024-05-17T22:37:14.509106+0000","last_clean":"2024-05-17T22:37:14.509106+0000","last_became_active":"2024-05-17T22:35:15.556427+0000","last_became_peered":"2024-05-17T22:35:15.556427+0000","last_unstale":"2024-05-17T22:37:14.509106+0000","last_undegraded":"2024-05-17T22:37:14.509106+0000","last_fullsized":"2024-05-17T22:37:14.509106+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:28:01.726262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.495402+0000","last_change":"2024-05-17T22:34:40.434394+0000","last_active":"2024-05-17T22:37:14.495402+0000","last_peered":"2024-05-17T22:37:14.495402+0000","last_clean":"2024-05-17T22:37:14.495402+0000","last_became_active":"2024-05-17T22:34:40.433489+0000","last_became_peered":"2024-05-17T22:34:40.433489+0000","last_unstale":"2024-05-17T22:37:14.495402+0000","last_undegraded":"2024-05-17T22:37:14.495402+0000","last_fullsized":"2024-05-17T22:37:14.495402+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:26:24.068814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.515238+0000","last_change":"2024-05-17T22:35:15.553601+0000","last_active":"2024-05-17T22:37:14.515238+0000","last_peered":"2024-05-17T22:37:14.515238+0000","last_clean":"2024-05-17T22:37:14.515238+0000","last_became_active":"2024-05-17T22:35:15.553116+0000","last_became_peered":"2024-05-17T22:35:15.553116+0000","last_unstale":"2024-05-17T22:37:14.515238+0000","last_undegraded":"2024-05-17T22:37:14.515238+0000","last_fullsized":"2024-05-17T22:37:14.515238+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:09:38.401107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.518979+0000","last_change":"2024-05-17T22:34:40.430479+0000","last_active":"2024-05-17T22:37:14.518979+0000","last_peered":"2024-05-17T22:37:14.518979+0000","last_clean":"2024-05-17T22:37:14.518979+0000","last_became_active":"2024-05-17T22:34:40.430025+0000","last_became_peered":"2024-05-17T22:34:40.430025+0000","last_unstale":"2024-05-17T22:37:14.518979+0000","last_undegraded":"2024-05-17T22:37:14.518979+0000","last_fullsized":"2024-05-17T22:37:14.518979+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:14:19.725096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.519153+0000","last_change":"2024-05-17T22:34:40.431711+0000","last_active":"2024-05-17T22:37:14.519153+0000","last_peered":"2024-05-17T22:37:14.519153+0000","last_clean":"2024-05-17T22:37:14.519153+0000","last_became_active":"2024-05-17T22:34:40.431368+0000","last_became_peered":"2024-05-17T22:34:40.431368+0000","last_unstale":"2024-05-17T22:37:14.519153+0000","last_undegraded":"2024-05-17T22:37:14.519153+0000","last_fullsized":"2024-05-17T22:37:14.519153+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:06:34.904652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.521909+0000","last_change":"2024-05-17T22:34:40.435593+0000","last_active":"2024-05-17T22:37:14.521909+0000","last_peered":"2024-05-17T22:37:14.521909+0000","last_clean":"2024-05-17T22:37:14.521909+0000","last_became_active":"2024-05-17T22:34:40.435241+0000","last_became_peered":"2024-05-17T22:34:40.435241+0000","last_unstale":"2024-05-17T22:37:14.521909+0000","last_undegraded":"2024-05-17T22:37:14.521909+0000","last_fullsized":"2024-05-17T22:37:14.521909+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:08:57.239998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.526427+0000","last_change":"2024-05-17T22:34:40.436248+0000","last_active":"2024-05-17T22:37:14.526427+0000","last_peered":"2024-05-17T22:37:14.526427+0000","last_clean":"2024-05-17T22:37:14.526427+0000","last_became_active":"2024-05-17T22:34:40.435824+0000","last_became_peered":"2024-05-17T22:34:40.435824+0000","last_unstale":"2024-05-17T22:37:14.526427+0000","last_undegraded":"2024-05-17T22:37:14.526427+0000","last_fullsized":"2024-05-17T22:37:14.526427+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:39:07.297470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.524640+0000","last_change":"2024-05-17T22:34:40.435843+0000","last_active":"2024-05-17T22:37:14.524640+0000","last_peered":"2024-05-17T22:37:14.524640+0000","last_clean":"2024-05-17T22:37:14.524640+0000","last_became_active":"2024-05-17T22:34:40.435500+0000","last_became_peered":"2024-05-17T22:34:40.435500+0000","last_unstale":"2024-05-17T22:37:14.524640+0000","last_undegraded":"2024-05-17T22:37:14.524640+0000","last_fullsized":"2024-05-17T22:37:14.524640+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:11:28.115850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.501798+0000","last_change":"2024-05-17T22:34:40.432044+0000","last_active":"2024-05-17T22:37:14.501798+0000","last_peered":"2024-05-17T22:37:14.501798+0000","last_clean":"2024-05-17T22:37:14.501798+0000","last_became_active":"2024-05-17T22:34:40.429141+0000","last_became_peered":"2024-05-17T22:34:40.429141+0000","last_unstale":"2024-05-17T22:37:14.501798+0000","last_undegraded":"2024-05-17T22:37:14.501798+0000","last_fullsized":"2024-05-17T22:37:14.501798+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:04:53.428681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.526534+0000","last_change":"2024-05-17T22:34:40.438474+0000","last_active":"2024-05-17T22:37:14.526534+0000","last_peered":"2024-05-17T22:37:14.526534+0000","last_clean":"2024-05-17T22:37:14.526534+0000","last_became_active":"2024-05-17T22:34:40.438109+0000","last_became_peered":"2024-05-17T22:34:40.438109+0000","last_unstale":"2024-05-17T22:37:14.526534+0000","last_undegraded":"2024-05-17T22:37:14.526534+0000","last_fullsized":"2024-05-17T22:37:14.526534+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:26:00.665246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.516486+0000","last_change":"2024-05-17T22:34:40.431563+0000","last_active":"2024-05-17T22:37:14.516486+0000","last_peered":"2024-05-17T22:37:14.516486+0000","last_clean":"2024-05-17T22:37:14.516486+0000","last_became_active":"2024-05-17T22:34:40.430992+0000","last_became_peered":"2024-05-17T22:34:40.430992+0000","last_unstale":"2024-05-17T22:37:14.516486+0000","last_undegraded":"2024-05-17T22:37:14.516486+0000","last_fullsized":"2024-05-17T22:37:14.516486+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:51:26.280440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.516587+0000","last_change":"2024-05-17T22:34:40.433629+0000","last_active":"2024-05-17T22:37:14.516587+0000","last_peered":"2024-05-17T22:37:14.516587+0000","last_clean":"2024-05-17T22:37:14.516587+0000","last_became_active":"2024-05-17T22:34:40.433270+0000","last_became_peered":"2024-05-17T22:34:40.433270+0000","last_unstale":"2024-05-17T22:37:14.516587+0000","last_undegraded":"2024-05-17T22:37:14.516587+0000","last_fullsized":"2024-05-17T22:37:14.516587+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:57:51.164078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.525064+0000","last_change":"2024-05-17T22:34:40.434218+0000","last_active":"2024-05-17T22:37:14.525064+0000","last_peered":"2024-05-17T22:37:14.525064+0000","last_clean":"2024-05-17T22:37:14.525064+0000","last_became_active":"2024-05-17T22:34:40.433849+0000","last_became_peered":"2024-05-17T22:34:40.433849+0000","last_unstale":"2024-05-17T22:37:14.525064+0000","last_undegraded":"2024-05-17T22:37:14.525064+0000","last_fullsized":"2024-05-17T22:37:14.525064+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:41:55.984109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.522605+0000","last_change":"2024-05-17T22:34:40.427892+0000","last_active":"2024-05-17T22:37:14.522605+0000","last_peered":"2024-05-17T22:37:14.522605+0000","last_clean":"2024-05-17T22:37:14.522605+0000","last_became_active":"2024-05-17T22:34:40.427303+0000","last_became_peered":"2024-05-17T22:34:40.427303+0000","last_unstale":"2024-05-17T22:37:14.522605+0000","last_undegraded":"2024-05-17T22:37:14.522605+0000","last_fullsized":"2024-05-17T22:37:14.522605+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:59:38.632123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.524943+0000","last_change":"2024-05-17T22:34:40.424784+0000","last_active":"2024-05-17T22:37:14.524943+0000","last_peered":"2024-05-17T22:37:14.524943+0000","last_clean":"2024-05-17T22:37:14.524943+0000","last_became_active":"2024-05-17T22:34:40.424439+0000","last_became_peered":"2024-05-17T22:34:40.424439+0000","last_unstale":"2024-05-17T22:37:14.524943+0000","last_undegraded":"2024-05-17T22:37:14.524943+0000","last_fullsized":"2024-05-17T22:37:14.524943+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:02:53.681600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.513305+0000","last_change":"2024-05-17T22:34:40.435023+0000","last_active":"2024-05-17T22:37:14.513305+0000","last_peered":"2024-05-17T22:37:14.513305+0000","last_clean":"2024-05-17T22:37:14.513305+0000","last_became_active":"2024-05-17T22:34:40.434441+0000","last_became_peered":"2024-05-17T22:34:40.434441+0000","last_unstale":"2024-05-17T22:37:14.513305+0000","last_undegraded":"2024-05-17T22:37:14.513305+0000","last_fullsized":"2024-05-17T22:37:14.513305+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:16:07.671897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624113+0000","last_change":"2024-05-17T22:34:52.493192+0000","last_active":"2024-05-17T22:36:15.624113+0000","last_peered":"2024-05-17T22:36:15.624113+0000","last_clean":"2024-05-17T22:36:15.624113+0000","last_became_active":"2024-05-17T22:34:52.492786+0000","last_became_peered":"2024-05-17T22:34:52.492786+0000","last_unstale":"2024-05-17T22:36:15.624113+0000","last_undegraded":"2024-05-17T22:36:15.624113+0000","last_fullsized":"2024-05-17T22:36:15.624113+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:43:40.152240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":56,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:43.067607+0000","last_change":"2024-05-17T22:34:37.402360+0000","last_active":"2024-05-17T22:36:43.067607+0000","last_peered":"2024-05-17T22:36:43.067607+0000","last_clean":"2024-05-17T22:36:43.067607+0000","last_became_active":"2024-05-17T22:34:37.401882+0000","last_became_peered":"2024-05-17T22:34:37.401882+0000","last_unstale":"2024-05-17T22:36:43.067607+0000","last_undegraded":"2024-05-17T22:36:43.067607+0000","last_fullsized":"2024-05-17T22:36:43.067607+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:53:13.300626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.506469+0000","last_change":"2024-05-17T22:34:40.436259+0000","last_active":"2024-05-17T22:37:14.506469+0000","last_peered":"2024-05-17T22:37:14.506469+0000","last_clean":"2024-05-17T22:37:14.506469+0000","last_became_active":"2024-05-17T22:34:40.435899+0000","last_became_peered":"2024-05-17T22:34:40.435899+0000","last_unstale":"2024-05-17T22:37:14.506469+0000","last_undegraded":"2024-05-17T22:37:14.506469+0000","last_fullsized":"2024-05-17T22:37:14.506469+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:26:29.327513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003713+0000","last_change":"2024-05-17T22:34:52.495236+0000","last_active":"2024-05-17T22:36:16.003713+0000","last_peered":"2024-05-17T22:36:16.003713+0000","last_clean":"2024-05-17T22:36:16.003713+0000","last_became_active":"2024-05-17T22:34:52.494783+0000","last_became_peered":"2024-05-17T22:34:52.494783+0000","last_unstale":"2024-05-17T22:36:16.003713+0000","last_undegraded":"2024-05-17T22:36:16.003713+0000","last_fullsized":"2024-05-17T22:36:16.003713+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:08:15.010706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:37.768914+0000","last_change":"2024-05-17T22:34:37.441129+0000","last_active":"2024-05-17T22:36:37.768914+0000","last_peered":"2024-05-17T22:36:37.768914+0000","last_clean":"2024-05-17T22:36:37.768914+0000","last_became_active":"2024-05-17T22:34:37.440738+0000","last_became_peered":"2024-05-17T22:34:37.440738+0000","last_unstale":"2024-05-17T22:36:37.768914+0000","last_undegraded":"2024-05-17T22:36:37.768914+0000","last_fullsized":"2024-05-17T22:36:37.768914+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:03:16.945076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.525530+0000","last_change":"2024-05-17T22:34:40.434146+0000","last_active":"2024-05-17T22:37:14.525530+0000","last_peered":"2024-05-17T22:37:14.525530+0000","last_clean":"2024-05-17T22:37:14.525530+0000","last_became_active":"2024-05-17T22:34:40.433802+0000","last_became_peered":"2024-05-17T22:34:40.433802+0000","last_unstale":"2024-05-17T22:37:14.525530+0000","last_undegraded":"2024-05-17T22:37:14.525530+0000","last_fullsized":"2024-05-17T22:37:14.525530+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:08:39.905409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004211+0000","last_change":"2024-05-17T22:34:52.501134+0000","last_active":"2024-05-17T22:36:16.004211+0000","last_peered":"2024-05-17T22:36:16.004211+0000","last_clean":"2024-05-17T22:36:16.004211+0000","last_became_active":"2024-05-17T22:34:52.500547+0000","last_became_peered":"2024-05-17T22:34:52.500547+0000","last_unstale":"2024-05-17T22:36:16.004211+0000","last_undegraded":"2024-05-17T22:36:16.004211+0000","last_fullsized":"2024-05-17T22:36:16.004211+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:13:04.200865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.626422+0000","last_change":"2024-05-17T22:34:37.404019+0000","last_active":"2024-05-17T22:36:15.626422+0000","last_peered":"2024-05-17T22:36:15.626422+0000","last_clean":"2024-05-17T22:36:15.626422+0000","last_became_active":"2024-05-17T22:34:37.403607+0000","last_became_peered":"2024-05-17T22:34:37.403607+0000","last_unstale":"2024-05-17T22:36:15.626422+0000","last_undegraded":"2024-05-17T22:36:15.626422+0000","last_fullsized":"2024-05-17T22:36:15.626422+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:30:21.405195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.525402+0000","last_change":"2024-05-17T22:34:40.426889+0000","last_active":"2024-05-17T22:37:14.525402+0000","last_peered":"2024-05-17T22:37:14.525402+0000","last_clean":"2024-05-17T22:37:14.525402+0000","last_became_active":"2024-05-17T22:34:40.426512+0000","last_became_peered":"2024-05-17T22:34:40.426512+0000","last_unstale":"2024-05-17T22:37:14.525402+0000","last_undegraded":"2024-05-17T22:37:14.525402+0000","last_fullsized":"2024-05-17T22:37:14.525402+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:30:51.363177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004528+0000","last_change":"2024-05-17T22:34:52.487514+0000","last_active":"2024-05-17T22:36:16.004528+0000","last_peered":"2024-05-17T22:36:16.004528+0000","last_clean":"2024-05-17T22:36:16.004528+0000","last_became_active":"2024-05-17T22:34:52.487068+0000","last_became_peered":"2024-05-17T22:34:52.487068+0000","last_unstale":"2024-05-17T22:36:16.004528+0000","last_undegraded":"2024-05-17T22:36:16.004528+0000","last_fullsized":"2024-05-17T22:36:16.004528+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:52:13.573986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.626123+0000","last_change":"2024-05-17T22:34:37.400518+0000","last_active":"2024-05-17T22:36:15.626123+0000","last_peered":"2024-05-17T22:36:15.626123+0000","last_clean":"2024-05-17T22:36:15.626123+0000","last_became_active":"2024-05-17T22:34:37.400145+0000","last_became_peered":"2024-05-17T22:34:37.400145+0000","last_unstale":"2024-05-17T22:36:15.626123+0000","last_undegraded":"2024-05-17T22:36:15.626123+0000","last_fullsized":"2024-05-17T22:36:15.626123+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:56:53.593624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.525513+0000","last_change":"2024-05-17T22:34:40.434148+0000","last_active":"2024-05-17T22:37:14.525513+0000","last_peered":"2024-05-17T22:37:14.525513+0000","last_clean":"2024-05-17T22:37:14.525513+0000","last_became_active":"2024-05-17T22:34:40.433613+0000","last_became_peered":"2024-05-17T22:34:40.433613+0000","last_unstale":"2024-05-17T22:37:14.525513+0000","last_undegraded":"2024-05-17T22:37:14.525513+0000","last_fullsized":"2024-05-17T22:37:14.525513+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:09:39.049852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624340+0000","last_change":"2024-05-17T22:34:52.490417+0000","last_active":"2024-05-17T22:36:15.624340+0000","last_peered":"2024-05-17T22:36:15.624340+0000","last_clean":"2024-05-17T22:36:15.624340+0000","last_became_active":"2024-05-17T22:34:52.489953+0000","last_became_peered":"2024-05-17T22:34:52.489953+0000","last_unstale":"2024-05-17T22:36:15.624340+0000","last_undegraded":"2024-05-17T22:36:15.624340+0000","last_fullsized":"2024-05-17T22:36:15.624340+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:23:56.491899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'8","reported_seq":246,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:38:56.337995+0000","last_change":"2024-05-17T22:34:37.405304+0000","last_active":"2024-05-17T22:38:56.337995+0000","last_peered":"2024-05-17T22:38:56.337995+0000","last_clean":"2024-05-17T22:38:56.337995+0000","last_became_active":"2024-05-17T22:34:37.404552+0000","last_became_peered":"2024-05-17T22:34:37.404552+0000","last_unstale":"2024-05-17T22:38:56.337995+0000","last_undegraded":"2024-05-17T22:38:56.337995+0000","last_fullsized":"2024-05-17T22:38:56.337995+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:57:06.556886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1462,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":193,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.512663+0000","last_change":"2024-05-17T22:34:40.430537+0000","last_active":"2024-05-17T22:37:14.512663+0000","last_peered":"2024-05-17T22:37:14.512663+0000","last_clean":"2024-05-17T22:37:14.512663+0000","last_became_active":"2024-05-17T22:34:40.430193+0000","last_became_peered":"2024-05-17T22:34:40.430193+0000","last_unstale":"2024-05-17T22:37:14.512663+0000","last_undegraded":"2024-05-17T22:37:14.512663+0000","last_fullsized":"2024-05-17T22:37:14.512663+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:58:42.091445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624546+0000","last_change":"2024-05-17T22:35:15.559077+0000","last_active":"2024-05-17T22:36:15.624546+0000","last_peered":"2024-05-17T22:36:15.624546+0000","last_clean":"2024-05-17T22:36:15.624546+0000","last_became_active":"2024-05-17T22:35:15.558382+0000","last_became_peered":"2024-05-17T22:35:15.558382+0000","last_unstale":"2024-05-17T22:36:15.624546+0000","last_undegraded":"2024-05-17T22:36:15.624546+0000","last_fullsized":"2024-05-17T22:36:15.624546+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:20:16.009620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.626030+0000","last_change":"2024-05-17T22:34:37.405332+0000","last_active":"2024-05-17T22:36:15.626030+0000","last_peered":"2024-05-17T22:36:15.626030+0000","last_clean":"2024-05-17T22:36:15.626030+0000","last_became_active":"2024-05-17T22:34:37.404970+0000","last_became_peered":"2024-05-17T22:34:37.404970+0000","last_unstale":"2024-05-17T22:36:15.626030+0000","last_undegraded":"2024-05-17T22:36:15.626030+0000","last_fullsized":"2024-05-17T22:36:15.626030+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:37:39.933686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.517661+0000","last_change":"2024-05-17T22:34:40.432690+0000","last_active":"2024-05-17T22:37:14.517661+0000","last_peered":"2024-05-17T22:37:14.517661+0000","last_clean":"2024-05-17T22:37:14.517661+0000","last_became_active":"2024-05-17T22:34:40.432274+0000","last_became_peered":"2024-05-17T22:34:40.432274+0000","last_unstale":"2024-05-17T22:37:14.517661+0000","last_undegraded":"2024-05-17T22:37:14.517661+0000","last_fullsized":"2024-05-17T22:37:14.517661+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:12:47.926865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628628+0000","last_change":"2024-05-17T22:34:52.494645+0000","last_active":"2024-05-17T22:36:15.628628+0000","last_peered":"2024-05-17T22:36:15.628628+0000","last_clean":"2024-05-17T22:36:15.628628+0000","last_became_active":"2024-05-17T22:34:52.494255+0000","last_became_peered":"2024-05-17T22:34:52.494255+0000","last_unstale":"2024-05-17T22:36:15.628628+0000","last_undegraded":"2024-05-17T22:36:15.628628+0000","last_fullsized":"2024-05-17T22:36:15.628628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:41:26.633571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685426+0000","last_change":"2024-05-17T22:34:37.398582+0000","last_active":"2024-05-17T22:36:15.685426+0000","last_peered":"2024-05-17T22:36:15.685426+0000","last_clean":"2024-05-17T22:36:15.685426+0000","last_became_active":"2024-05-17T22:34:37.398156+0000","last_became_peered":"2024-05-17T22:34:37.398156+0000","last_unstale":"2024-05-17T22:36:15.685426+0000","last_undegraded":"2024-05-17T22:36:15.685426+0000","last_fullsized":"2024-05-17T22:36:15.685426+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:46:00.276214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.518565+0000","last_change":"2024-05-17T22:35:15.551402+0000","last_active":"2024-05-17T22:37:14.518565+0000","last_peered":"2024-05-17T22:37:14.518565+0000","last_clean":"2024-05-17T22:37:14.518565+0000","last_became_active":"2024-05-17T22:35:15.551021+0000","last_became_peered":"2024-05-17T22:35:15.551021+0000","last_unstale":"2024-05-17T22:37:14.518565+0000","last_undegraded":"2024-05-17T22:37:14.518565+0000","last_fullsized":"2024-05-17T22:37:14.518565+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:35:18.573620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:54.120653+0000","last_change":"2024-05-17T22:34:52.487413+0000","last_active":"2024-05-17T22:36:54.120653+0000","last_peered":"2024-05-17T22:36:54.120653+0000","last_clean":"2024-05-17T22:36:54.120653+0000","last_became_active":"2024-05-17T22:34:52.487032+0000","last_became_peered":"2024-05-17T22:34:52.487032+0000","last_unstale":"2024-05-17T22:36:54.120653+0000","last_undegraded":"2024-05-17T22:36:54.120653+0000","last_fullsized":"2024-05-17T22:36:54.120653+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:34:49.122008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003315+0000","last_change":"2024-05-17T22:34:37.404894+0000","last_active":"2024-05-17T22:36:16.003315+0000","last_peered":"2024-05-17T22:36:16.003315+0000","last_clean":"2024-05-17T22:36:16.003315+0000","last_became_active":"2024-05-17T22:34:37.404567+0000","last_became_peered":"2024-05-17T22:34:37.404567+0000","last_unstale":"2024-05-17T22:36:16.003315+0000","last_undegraded":"2024-05-17T22:36:16.003315+0000","last_fullsized":"2024-05-17T22:36:16.003315+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:13:09.761556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.525152+0000","last_change":"2024-05-17T22:34:40.418267+0000","last_active":"2024-05-17T22:37:14.525152+0000","last_peered":"2024-05-17T22:37:14.525152+0000","last_clean":"2024-05-17T22:37:14.525152+0000","last_became_active":"2024-05-17T22:34:40.417928+0000","last_became_peered":"2024-05-17T22:34:40.417928+0000","last_unstale":"2024-05-17T22:37:14.525152+0000","last_undegraded":"2024-05-17T22:37:14.525152+0000","last_fullsized":"2024-05-17T22:37:14.525152+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:17:37.055004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984205+0000","last_change":"2024-05-17T22:34:52.492126+0000","last_active":"2024-05-17T22:36:15.984205+0000","last_peered":"2024-05-17T22:36:15.984205+0000","last_clean":"2024-05-17T22:36:15.984205+0000","last_became_active":"2024-05-17T22:34:52.491695+0000","last_became_peered":"2024-05-17T22:34:52.491695+0000","last_unstale":"2024-05-17T22:36:15.984205+0000","last_undegraded":"2024-05-17T22:36:15.984205+0000","last_fullsized":"2024-05-17T22:36:15.984205+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:15:17.710279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'5","reported_seq":240,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:38:58.340899+0000","last_change":"2024-05-17T22:34:37.406989+0000","last_active":"2024-05-17T22:38:58.340899+0000","last_peered":"2024-05-17T22:38:58.340899+0000","last_clean":"2024-05-17T22:38:58.340899+0000","last_became_active":"2024-05-17T22:34:37.406612+0000","last_became_peered":"2024-05-17T22:34:37.406612+0000","last_unstale":"2024-05-17T22:38:58.340899+0000","last_undegraded":"2024-05-17T22:38:58.340899+0000","last_fullsized":"2024-05-17T22:38:58.340899+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:47:52.480804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":190,"num_read_kb":190,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.524255+0000","last_change":"2024-05-17T22:34:40.427221+0000","last_active":"2024-05-17T22:37:14.524255+0000","last_peered":"2024-05-17T22:37:14.524255+0000","last_clean":"2024-05-17T22:37:14.524255+0000","last_became_active":"2024-05-17T22:34:40.426703+0000","last_became_peered":"2024-05-17T22:34:40.426703+0000","last_unstale":"2024-05-17T22:37:14.524255+0000","last_undegraded":"2024-05-17T22:37:14.524255+0000","last_fullsized":"2024-05-17T22:37:14.524255+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:49:15.629048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.622725+0000","last_change":"2024-05-17T22:34:52.499086+0000","last_active":"2024-05-17T22:36:15.622725+0000","last_peered":"2024-05-17T22:36:15.622725+0000","last_clean":"2024-05-17T22:36:15.622725+0000","last_became_active":"2024-05-17T22:34:52.498666+0000","last_became_peered":"2024-05-17T22:34:52.498666+0000","last_unstale":"2024-05-17T22:36:15.622725+0000","last_undegraded":"2024-05-17T22:36:15.622725+0000","last_fullsized":"2024-05-17T22:36:15.622725+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:46:25.882964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.005213+0000","last_change":"2024-05-17T22:35:15.836294+0000","last_active":"2024-05-17T22:36:16.005213+0000","last_peered":"2024-05-17T22:36:16.005213+0000","last_clean":"2024-05-17T22:36:16.005213+0000","last_became_active":"2024-05-17T22:35:15.835759+0000","last_became_peered":"2024-05-17T22:35:15.835759+0000","last_unstale":"2024-05-17T22:36:16.005213+0000","last_undegraded":"2024-05-17T22:36:16.005213+0000","last_fullsized":"2024-05-17T22:36:16.005213+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:23:32.106692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.519861+0000","last_change":"2024-05-17T22:34:40.433331+0000","last_active":"2024-05-17T22:37:14.519861+0000","last_peered":"2024-05-17T22:37:14.519861+0000","last_clean":"2024-05-17T22:37:14.519861+0000","last_became_active":"2024-05-17T22:34:40.432989+0000","last_became_peered":"2024-05-17T22:34:40.432989+0000","last_unstale":"2024-05-17T22:37:14.519861+0000","last_undegraded":"2024-05-17T22:37:14.519861+0000","last_fullsized":"2024-05-17T22:37:14.519861+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:22:49.146148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004235+0000","last_change":"2024-05-17T22:34:52.494418+0000","last_active":"2024-05-17T22:36:16.004235+0000","last_peered":"2024-05-17T22:36:16.004235+0000","last_clean":"2024-05-17T22:36:16.004235+0000","last_became_active":"2024-05-17T22:34:52.493953+0000","last_became_peered":"2024-05-17T22:34:52.493953+0000","last_unstale":"2024-05-17T22:36:16.004235+0000","last_undegraded":"2024-05-17T22:36:16.004235+0000","last_fullsized":"2024-05-17T22:36:16.004235+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:50:40.211770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004010+0000","last_change":"2024-05-17T22:34:37.442986+0000","last_active":"2024-05-17T22:36:16.004010+0000","last_peered":"2024-05-17T22:36:16.004010+0000","last_clean":"2024-05-17T22:36:16.004010+0000","last_became_active":"2024-05-17T22:34:37.442399+0000","last_became_peered":"2024-05-17T22:34:37.442399+0000","last_unstale":"2024-05-17T22:36:16.004010+0000","last_undegraded":"2024-05-17T22:36:16.004010+0000","last_fullsized":"2024-05-17T22:36:16.004010+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:34:05.785811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.525214+0000","last_change":"2024-05-17T22:34:40.426793+0000","last_active":"2024-05-17T22:37:14.525214+0000","last_peered":"2024-05-17T22:37:14.525214+0000","last_clean":"2024-05-17T22:37:14.525214+0000","last_became_active":"2024-05-17T22:34:40.426420+0000","last_became_peered":"2024-05-17T22:34:40.426420+0000","last_unstale":"2024-05-17T22:37:14.525214+0000","last_undegraded":"2024-05-17T22:37:14.525214+0000","last_fullsized":"2024-05-17T22:37:14.525214+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:57:07.099254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.622953+0000","last_change":"2024-05-17T22:34:52.498762+0000","last_active":"2024-05-17T22:36:15.622953+0000","last_peered":"2024-05-17T22:36:15.622953+0000","last_clean":"2024-05-17T22:36:15.622953+0000","last_became_active":"2024-05-17T22:34:52.498306+0000","last_became_peered":"2024-05-17T22:34:52.498306+0000","last_unstale":"2024-05-17T22:36:15.622953+0000","last_undegraded":"2024-05-17T22:36:15.622953+0000","last_fullsized":"2024-05-17T22:36:15.622953+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:18:32.717563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'39","reported_seq":476,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:38:59.343737+0000","last_change":"2024-05-17T22:34:37.407885+0000","last_active":"2024-05-17T22:38:59.343737+0000","last_peered":"2024-05-17T22:38:59.343737+0000","last_clean":"2024-05-17T22:38:59.343737+0000","last_became_active":"2024-05-17T22:34:37.407463+0000","last_became_peered":"2024-05-17T22:34:37.407463+0000","last_unstale":"2024-05-17T22:38:59.343737+0000","last_undegraded":"2024-05-17T22:38:59.343737+0000","last_fullsized":"2024-05-17T22:38:59.343737+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:26:20.288557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68002,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":392,"num_read_kb":263,"num_write":40,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.501714+0000","last_change":"2024-05-17T22:34:40.435800+0000","last_active":"2024-05-17T22:37:14.501714+0000","last_peered":"2024-05-17T22:37:14.501714+0000","last_clean":"2024-05-17T22:37:14.501714+0000","last_became_active":"2024-05-17T22:34:40.435245+0000","last_became_peered":"2024-05-17T22:34:40.435245+0000","last_unstale":"2024-05-17T22:37:14.501714+0000","last_undegraded":"2024-05-17T22:37:14.501714+0000","last_fullsized":"2024-05-17T22:37:14.501714+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:09:58.699026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004448+0000","last_change":"2024-05-17T22:34:52.487336+0000","last_active":"2024-05-17T22:36:16.004448+0000","last_peered":"2024-05-17T22:36:16.004448+0000","last_clean":"2024-05-17T22:36:16.004448+0000","last_became_active":"2024-05-17T22:34:52.486883+0000","last_became_peered":"2024-05-17T22:34:52.486883+0000","last_unstale":"2024-05-17T22:36:16.004448+0000","last_undegraded":"2024-05-17T22:36:16.004448+0000","last_fullsized":"2024-05-17T22:36:16.004448+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:44:41.264072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624096+0000","last_change":"2024-05-17T22:34:37.405044+0000","last_active":"2024-05-17T22:36:15.624096+0000","last_peered":"2024-05-17T22:36:15.624096+0000","last_clean":"2024-05-17T22:36:15.624096+0000","last_became_active":"2024-05-17T22:34:37.404610+0000","last_became_peered":"2024-05-17T22:34:37.404610+0000","last_unstale":"2024-05-17T22:36:15.624096+0000","last_undegraded":"2024-05-17T22:36:15.624096+0000","last_fullsized":"2024-05-17T22:36:15.624096+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:20:37.939069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.521845+0000","last_change":"2024-05-17T22:34:40.430655+0000","last_active":"2024-05-17T22:37:14.521845+0000","last_peered":"2024-05-17T22:37:14.521845+0000","last_clean":"2024-05-17T22:37:14.521845+0000","last_became_active":"2024-05-17T22:34:40.430307+0000","last_became_peered":"2024-05-17T22:34:40.430307+0000","last_unstale":"2024-05-17T22:37:14.521845+0000","last_undegraded":"2024-05-17T22:37:14.521845+0000","last_fullsized":"2024-05-17T22:37:14.521845+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:57:19.036117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628175+0000","last_change":"2024-05-17T22:34:52.494709+0000","last_active":"2024-05-17T22:36:15.628175+0000","last_peered":"2024-05-17T22:36:15.628175+0000","last_clean":"2024-05-17T22:36:15.628175+0000","last_became_active":"2024-05-17T22:34:52.494195+0000","last_became_peered":"2024-05-17T22:34:52.494195+0000","last_unstale":"2024-05-17T22:36:15.628175+0000","last_undegraded":"2024-05-17T22:36:15.628175+0000","last_fullsized":"2024-05-17T22:36:15.628175+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:35:41.188459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003957+0000","last_change":"2024-05-17T22:34:37.402494+0000","last_active":"2024-05-17T22:36:16.003957+0000","last_peered":"2024-05-17T22:36:16.003957+0000","last_clean":"2024-05-17T22:36:16.003957+0000","last_became_active":"2024-05-17T22:34:37.402102+0000","last_became_peered":"2024-05-17T22:34:37.402102+0000","last_unstale":"2024-05-17T22:36:16.003957+0000","last_undegraded":"2024-05-17T22:36:16.003957+0000","last_fullsized":"2024-05-17T22:36:16.003957+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:22:30.915780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.522351+0000","last_change":"2024-05-17T22:34:40.440731+0000","last_active":"2024-05-17T22:37:14.522351+0000","last_peered":"2024-05-17T22:37:14.522351+0000","last_clean":"2024-05-17T22:37:14.522351+0000","last_became_active":"2024-05-17T22:34:40.440374+0000","last_became_peered":"2024-05-17T22:34:40.440374+0000","last_unstale":"2024-05-17T22:37:14.522351+0000","last_undegraded":"2024-05-17T22:37:14.522351+0000","last_fullsized":"2024-05-17T22:37:14.522351+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:26:40.029155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.006411+0000","last_change":"2024-05-17T22:36:16.005856+0000","last_active":"2024-05-17T22:36:16.006411+0000","last_peered":"2024-05-17T22:36:16.006411+0000","last_clean":"2024-05-17T22:36:16.006411+0000","last_became_active":"2024-05-17T22:36:16.005303+0000","last_became_peered":"2024-05-17T22:36:16.005303+0000","last_unstale":"2024-05-17T22:36:16.006411+0000","last_undegraded":"2024-05-17T22:36:16.006411+0000","last_fullsized":"2024-05-17T22:36:16.006411+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:41:15.708879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":241,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:38:56.372568+0000","last_change":"2024-05-17T22:34:37.403123+0000","last_active":"2024-05-17T22:38:56.372568+0000","last_peered":"2024-05-17T22:38:56.372568+0000","last_clean":"2024-05-17T22:38:56.372568+0000","last_became_active":"2024-05-17T22:34:37.402702+0000","last_became_peered":"2024-05-17T22:34:37.402702+0000","last_unstale":"2024-05-17T22:38:56.372568+0000","last_undegraded":"2024-05-17T22:38:56.372568+0000","last_fullsized":"2024-05-17T22:38:56.372568+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:50:02.031651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":193,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.512621+0000","last_change":"2024-05-17T22:34:40.431895+0000","last_active":"2024-05-17T22:37:14.512621+0000","last_peered":"2024-05-17T22:37:14.512621+0000","last_clean":"2024-05-17T22:37:14.512621+0000","last_became_active":"2024-05-17T22:34:40.431527+0000","last_became_peered":"2024-05-17T22:34:40.431527+0000","last_unstale":"2024-05-17T22:37:14.512621+0000","last_undegraded":"2024-05-17T22:37:14.512621+0000","last_fullsized":"2024-05-17T22:37:14.512621+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:30:09.404255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.686140+0000","last_change":"2024-05-17T22:34:52.495583+0000","last_active":"2024-05-17T22:36:15.686140+0000","last_peered":"2024-05-17T22:36:15.686140+0000","last_clean":"2024-05-17T22:36:15.686140+0000","last_became_active":"2024-05-17T22:34:52.495030+0000","last_became_peered":"2024-05-17T22:34:52.495030+0000","last_unstale":"2024-05-17T22:36:15.686140+0000","last_undegraded":"2024-05-17T22:36:15.686140+0000","last_fullsized":"2024-05-17T22:36:15.686140+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:04:54.046891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":61,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625592+0000","last_change":"2024-05-17T22:34:37.442154+0000","last_active":"2024-05-17T22:36:15.625592+0000","last_peered":"2024-05-17T22:36:15.625592+0000","last_clean":"2024-05-17T22:36:15.625592+0000","last_became_active":"2024-05-17T22:34:37.441691+0000","last_became_peered":"2024-05-17T22:34:37.441691+0000","last_unstale":"2024-05-17T22:36:15.625592+0000","last_undegraded":"2024-05-17T22:36:15.625592+0000","last_fullsized":"2024-05-17T22:36:15.625592+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:47:29.862031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.517818+0000","last_change":"2024-05-17T22:34:40.430894+0000","last_active":"2024-05-17T22:37:14.517818+0000","last_peered":"2024-05-17T22:37:14.517818+0000","last_clean":"2024-05-17T22:37:14.517818+0000","last_became_active":"2024-05-17T22:34:40.430328+0000","last_became_peered":"2024-05-17T22:34:40.430328+0000","last_unstale":"2024-05-17T22:37:14.517818+0000","last_undegraded":"2024-05-17T22:37:14.517818+0000","last_fullsized":"2024-05-17T22:37:14.517818+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:36:46.619111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003488+0000","last_change":"2024-05-17T22:34:52.487555+0000","last_active":"2024-05-17T22:36:16.003488+0000","last_peered":"2024-05-17T22:36:16.003488+0000","last_clean":"2024-05-17T22:36:16.003488+0000","last_became_active":"2024-05-17T22:34:52.487076+0000","last_became_peered":"2024-05-17T22:34:52.487076+0000","last_unstale":"2024-05-17T22:36:16.003488+0000","last_undegraded":"2024-05-17T22:36:16.003488+0000","last_fullsized":"2024-05-17T22:36:16.003488+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:05:46.720665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628417+0000","last_change":"2024-05-17T22:35:15.558860+0000","last_active":"2024-05-17T22:36:15.628417+0000","last_peered":"2024-05-17T22:36:15.628417+0000","last_clean":"2024-05-17T22:36:15.628417+0000","last_became_active":"2024-05-17T22:35:15.553683+0000","last_became_peered":"2024-05-17T22:35:15.553683+0000","last_unstale":"2024-05-17T22:36:15.628417+0000","last_undegraded":"2024-05-17T22:36:15.628417+0000","last_fullsized":"2024-05-17T22:36:15.628417+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:27:24.978282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.506967+0000","last_change":"2024-05-17T22:34:40.433499+0000","last_active":"2024-05-17T22:37:14.506967+0000","last_peered":"2024-05-17T22:37:14.506967+0000","last_clean":"2024-05-17T22:37:14.506967+0000","last_became_active":"2024-05-17T22:34:40.431370+0000","last_became_peered":"2024-05-17T22:34:40.431370+0000","last_unstale":"2024-05-17T22:37:14.506967+0000","last_undegraded":"2024-05-17T22:37:14.506967+0000","last_fullsized":"2024-05-17T22:37:14.506967+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:25:08.383021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625953+0000","last_change":"2024-05-17T22:34:52.489627+0000","last_active":"2024-05-17T22:36:15.625953+0000","last_peered":"2024-05-17T22:36:15.625953+0000","last_clean":"2024-05-17T22:36:15.625953+0000","last_became_active":"2024-05-17T22:34:52.489173+0000","last_became_peered":"2024-05-17T22:34:52.489173+0000","last_unstale":"2024-05-17T22:36:15.625953+0000","last_undegraded":"2024-05-17T22:36:15.625953+0000","last_fullsized":"2024-05-17T22:36:15.625953+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:52:26.908817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:43.067577+0000","last_change":"2024-05-17T22:35:15.553325+0000","last_active":"2024-05-17T22:36:43.067577+0000","last_peered":"2024-05-17T22:36:43.067577+0000","last_clean":"2024-05-17T22:36:43.067577+0000","last_became_active":"2024-05-17T22:35:15.552867+0000","last_became_peered":"2024-05-17T22:35:15.552867+0000","last_unstale":"2024-05-17T22:36:43.067577+0000","last_undegraded":"2024-05-17T22:36:43.067577+0000","last_fullsized":"2024-05-17T22:36:43.067577+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:28:36.618331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.526055+0000","last_change":"2024-05-17T22:34:40.423221+0000","last_active":"2024-05-17T22:37:14.526055+0000","last_peered":"2024-05-17T22:37:14.526055+0000","last_clean":"2024-05-17T22:37:14.526055+0000","last_became_active":"2024-05-17T22:34:40.422838+0000","last_became_peered":"2024-05-17T22:34:40.422838+0000","last_unstale":"2024-05-17T22:37:14.526055+0000","last_undegraded":"2024-05-17T22:37:14.526055+0000","last_fullsized":"2024-05-17T22:37:14.526055+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:39:51.873212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685905+0000","last_change":"2024-05-17T22:34:52.484802+0000","last_active":"2024-05-17T22:36:15.685905+0000","last_peered":"2024-05-17T22:36:15.685905+0000","last_clean":"2024-05-17T22:36:15.685905+0000","last_became_active":"2024-05-17T22:34:52.483766+0000","last_became_peered":"2024-05-17T22:34:52.483766+0000","last_unstale":"2024-05-17T22:36:15.685905+0000","last_undegraded":"2024-05-17T22:36:15.685905+0000","last_fullsized":"2024-05-17T22:36:15.685905+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:32:01.427078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:43.941225+0000","last_change":"2024-05-17T22:34:37.404797+0000","last_active":"2024-05-17T22:36:43.941225+0000","last_peered":"2024-05-17T22:36:43.941225+0000","last_clean":"2024-05-17T22:36:43.941225+0000","last_became_active":"2024-05-17T22:34:37.404393+0000","last_became_peered":"2024-05-17T22:34:37.404393+0000","last_unstale":"2024-05-17T22:36:43.941225+0000","last_undegraded":"2024-05-17T22:36:43.941225+0000","last_fullsized":"2024-05-17T22:36:43.941225+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:26:36.887919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.515960+0000","last_change":"2024-05-17T22:34:40.428210+0000","last_active":"2024-05-17T22:37:14.515960+0000","last_peered":"2024-05-17T22:37:14.515960+0000","last_clean":"2024-05-17T22:37:14.515960+0000","last_became_active":"2024-05-17T22:34:40.427845+0000","last_became_peered":"2024-05-17T22:34:40.427845+0000","last_unstale":"2024-05-17T22:37:14.515960+0000","last_undegraded":"2024-05-17T22:37:14.515960+0000","last_fullsized":"2024-05-17T22:37:14.515960+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:45:30.437139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.686040+0000","last_change":"2024-05-17T22:34:52.490502+0000","last_active":"2024-05-17T22:36:15.686040+0000","last_peered":"2024-05-17T22:36:15.686040+0000","last_clean":"2024-05-17T22:36:15.686040+0000","last_became_active":"2024-05-17T22:34:52.490008+0000","last_became_peered":"2024-05-17T22:34:52.490008+0000","last_unstale":"2024-05-17T22:36:15.686040+0000","last_undegraded":"2024-05-17T22:36:15.686040+0000","last_fullsized":"2024-05-17T22:36:15.686040+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:58:29.122668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.518044+0000","last_change":"2024-05-17T22:34:40.432248+0000","last_active":"2024-05-17T22:37:14.518044+0000","last_peered":"2024-05-17T22:37:14.518044+0000","last_clean":"2024-05-17T22:37:14.518044+0000","last_became_active":"2024-05-17T22:34:40.431863+0000","last_became_peered":"2024-05-17T22:34:40.431863+0000","last_unstale":"2024-05-17T22:37:14.518044+0000","last_undegraded":"2024-05-17T22:37:14.518044+0000","last_fullsized":"2024-05-17T22:37:14.518044+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:20:34.730542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":240,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:38:57.373331+0000","last_change":"2024-05-17T22:34:37.441186+0000","last_active":"2024-05-17T22:38:57.373331+0000","last_peered":"2024-05-17T22:38:57.373331+0000","last_clean":"2024-05-17T22:38:57.373331+0000","last_became_active":"2024-05-17T22:34:37.440781+0000","last_became_peered":"2024-05-17T22:34:37.440781+0000","last_unstale":"2024-05-17T22:38:57.373331+0000","last_undegraded":"2024-05-17T22:38:57.373331+0000","last_fullsized":"2024-05-17T22:38:57.373331+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:44:34.435147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":189,"num_read_kb":189,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.005134+0000","last_change":"2024-05-17T22:34:52.491771+0000","last_active":"2024-05-17T22:36:16.005134+0000","last_peered":"2024-05-17T22:36:16.005134+0000","last_clean":"2024-05-17T22:36:16.005134+0000","last_became_active":"2024-05-17T22:34:52.491172+0000","last_became_peered":"2024-05-17T22:34:52.491172+0000","last_unstale":"2024-05-17T22:36:16.005134+0000","last_undegraded":"2024-05-17T22:36:16.005134+0000","last_fullsized":"2024-05-17T22:36:16.005134+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:15:21.010592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.514209+0000","last_change":"2024-05-17T22:34:40.430860+0000","last_active":"2024-05-17T22:37:14.514209+0000","last_peered":"2024-05-17T22:37:14.514209+0000","last_clean":"2024-05-17T22:37:14.514209+0000","last_became_active":"2024-05-17T22:34:40.430542+0000","last_became_peered":"2024-05-17T22:34:40.430542+0000","last_unstale":"2024-05-17T22:37:14.514209+0000","last_undegraded":"2024-05-17T22:37:14.514209+0000","last_fullsized":"2024-05-17T22:37:14.514209+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:16:26.257380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:42.729213+0000","last_change":"2024-05-17T22:34:37.403026+0000","last_active":"2024-05-17T22:36:42.729213+0000","last_peered":"2024-05-17T22:36:42.729213+0000","last_clean":"2024-05-17T22:36:42.729213+0000","last_became_active":"2024-05-17T22:34:37.402560+0000","last_became_peered":"2024-05-17T22:34:37.402560+0000","last_unstale":"2024-05-17T22:36:42.729213+0000","last_undegraded":"2024-05-17T22:36:42.729213+0000","last_fullsized":"2024-05-17T22:36:42.729213+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:23:03.784411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":120,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003564+0000","last_change":"2024-05-17T22:33:11.055878+0000","last_active":"2024-05-17T22:36:16.003564+0000","last_peered":"2024-05-17T22:36:16.003564+0000","last_clean":"2024-05-17T22:36:16.003564+0000","last_became_active":"2024-05-17T22:33:11.050409+0000","last_became_peered":"2024-05-17T22:33:11.050409+0000","last_unstale":"2024-05-17T22:36:16.003564+0000","last_undegraded":"2024-05-17T22:36:16.003564+0000","last_fullsized":"2024-05-17T22:36:16.003564+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:29:43.341170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:29:43.341170+0000","last_clean_scrub_stamp":"2024-05-17T22:29:43.341170+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:30:54.153865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003960+0000","last_change":"2024-05-17T22:34:52.497064+0000","last_active":"2024-05-17T22:36:16.003960+0000","last_peered":"2024-05-17T22:36:16.003960+0000","last_clean":"2024-05-17T22:36:16.003960+0000","last_became_active":"2024-05-17T22:34:52.496611+0000","last_became_peered":"2024-05-17T22:34:52.496611+0000","last_unstale":"2024-05-17T22:36:16.003960+0000","last_undegraded":"2024-05-17T22:36:16.003960+0000","last_fullsized":"2024-05-17T22:36:16.003960+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:20:29.597589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.519893+0000","last_change":"2024-05-17T22:35:15.555413+0000","last_active":"2024-05-17T22:37:14.519893+0000","last_peered":"2024-05-17T22:37:14.519893+0000","last_clean":"2024-05-17T22:37:14.519893+0000","last_became_active":"2024-05-17T22:35:15.555005+0000","last_became_peered":"2024-05-17T22:35:15.555005+0000","last_unstale":"2024-05-17T22:37:14.519893+0000","last_undegraded":"2024-05-17T22:37:14.519893+0000","last_fullsized":"2024-05-17T22:37:14.519893+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:51:49.860673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:38.940738+0000","last_change":"2024-05-17T22:34:37.406546+0000","last_active":"2024-05-17T22:36:38.940738+0000","last_peered":"2024-05-17T22:36:38.940738+0000","last_clean":"2024-05-17T22:36:38.940738+0000","last_became_active":"2024-05-17T22:34:37.406118+0000","last_became_peered":"2024-05-17T22:34:37.406118+0000","last_unstale":"2024-05-17T22:36:38.940738+0000","last_undegraded":"2024-05-17T22:36:38.940738+0000","last_fullsized":"2024-05-17T22:36:38.940738+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:47:16.350091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.006276+0000","last_change":"2024-05-17T22:36:16.005647+0000","last_active":"2024-05-17T22:36:16.006276+0000","last_peered":"2024-05-17T22:36:16.006276+0000","last_clean":"2024-05-17T22:36:16.006276+0000","last_became_active":"2024-05-17T22:36:16.005128+0000","last_became_peered":"2024-05-17T22:36:16.005128+0000","last_unstale":"2024-05-17T22:36:16.006276+0000","last_undegraded":"2024-05-17T22:36:16.006276+0000","last_fullsized":"2024-05-17T22:36:16.006276+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:16:24.481236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,7,6],"acting":[11,10,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:45.570153+0000","last_change":"2024-05-17T22:34:37.442676+0000","last_active":"2024-05-17T22:36:45.570153+0000","last_peered":"2024-05-17T22:36:45.570153+0000","last_clean":"2024-05-17T22:36:45.570153+0000","last_became_active":"2024-05-17T22:34:37.442275+0000","last_became_peered":"2024-05-17T22:34:37.442275+0000","last_unstale":"2024-05-17T22:36:45.570153+0000","last_undegraded":"2024-05-17T22:36:45.570153+0000","last_fullsized":"2024-05-17T22:36:45.570153+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:54:18.542987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.519824+0000","last_change":"2024-05-17T22:34:40.424266+0000","last_active":"2024-05-17T22:37:14.519824+0000","last_peered":"2024-05-17T22:37:14.519824+0000","last_clean":"2024-05-17T22:37:14.519824+0000","last_became_active":"2024-05-17T22:34:40.423905+0000","last_became_peered":"2024-05-17T22:34:40.423905+0000","last_unstale":"2024-05-17T22:37:14.519824+0000","last_undegraded":"2024-05-17T22:37:14.519824+0000","last_fullsized":"2024-05-17T22:37:14.519824+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:03:43.397871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:52.770677+0000","last_change":"2024-05-17T22:34:52.493386+0000","last_active":"2024-05-17T22:36:52.770677+0000","last_peered":"2024-05-17T22:36:52.770677+0000","last_clean":"2024-05-17T22:36:52.770677+0000","last_became_active":"2024-05-17T22:34:52.492997+0000","last_became_peered":"2024-05-17T22:34:52.492997+0000","last_unstale":"2024-05-17T22:36:52.770677+0000","last_undegraded":"2024-05-17T22:36:52.770677+0000","last_fullsized":"2024-05-17T22:36:52.770677+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:14:40.818271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'13","reported_seq":59,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:38:55.469975+0000","last_change":"2024-05-17T22:34:37.399397+0000","last_active":"2024-05-17T22:38:55.469975+0000","last_peered":"2024-05-17T22:38:55.469975+0000","last_clean":"2024-05-17T22:38:55.469975+0000","last_became_active":"2024-05-17T22:34:37.399062+0000","last_became_peered":"2024-05-17T22:34:37.399062+0000","last_unstale":"2024-05-17T22:38:55.469975+0000","last_undegraded":"2024-05-17T22:38:55.469975+0000","last_fullsized":"2024-05-17T22:38:55.469975+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:02:05.230580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.518907+0000","last_change":"2024-05-17T22:34:40.426995+0000","last_active":"2024-05-17T22:37:14.518907+0000","last_peered":"2024-05-17T22:37:14.518907+0000","last_clean":"2024-05-17T22:37:14.518907+0000","last_became_active":"2024-05-17T22:34:40.426226+0000","last_became_peered":"2024-05-17T22:34:40.426226+0000","last_unstale":"2024-05-17T22:37:14.518907+0000","last_undegraded":"2024-05-17T22:37:14.518907+0000","last_fullsized":"2024-05-17T22:37:14.518907+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:49:53.331076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623704+0000","last_change":"2024-05-17T22:34:52.487098+0000","last_active":"2024-05-17T22:36:15.623704+0000","last_peered":"2024-05-17T22:36:15.623704+0000","last_clean":"2024-05-17T22:36:15.623704+0000","last_became_active":"2024-05-17T22:34:52.486605+0000","last_became_peered":"2024-05-17T22:34:52.486605+0000","last_unstale":"2024-05-17T22:36:15.623704+0000","last_undegraded":"2024-05-17T22:36:15.623704+0000","last_fullsized":"2024-05-17T22:36:15.623704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:33:28.234208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:47.729708+0000","last_change":"2024-05-17T22:34:37.800830+0000","last_active":"2024-05-17T22:36:47.729708+0000","last_peered":"2024-05-17T22:36:47.729708+0000","last_clean":"2024-05-17T22:36:47.729708+0000","last_became_active":"2024-05-17T22:34:37.800405+0000","last_became_peered":"2024-05-17T22:34:37.800405+0000","last_unstale":"2024-05-17T22:36:47.729708+0000","last_undegraded":"2024-05-17T22:36:47.729708+0000","last_fullsized":"2024-05-17T22:36:47.729708+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:19:10.087442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.520738+0000","last_change":"2024-05-17T22:34:40.426307+0000","last_active":"2024-05-17T22:37:14.520738+0000","last_peered":"2024-05-17T22:37:14.520738+0000","last_clean":"2024-05-17T22:37:14.520738+0000","last_became_active":"2024-05-17T22:34:40.425969+0000","last_became_peered":"2024-05-17T22:34:40.425969+0000","last_unstale":"2024-05-17T22:37:14.520738+0000","last_undegraded":"2024-05-17T22:37:14.520738+0000","last_fullsized":"2024-05-17T22:37:14.520738+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:25:49.680886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004511+0000","last_change":"2024-05-17T22:35:15.556857+0000","last_active":"2024-05-17T22:36:16.004511+0000","last_peered":"2024-05-17T22:36:16.004511+0000","last_clean":"2024-05-17T22:36:16.004511+0000","last_became_active":"2024-05-17T22:35:15.556298+0000","last_became_peered":"2024-05-17T22:35:15.556298+0000","last_unstale":"2024-05-17T22:36:16.004511+0000","last_undegraded":"2024-05-17T22:36:16.004511+0000","last_fullsized":"2024-05-17T22:36:16.004511+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:23:20.111544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:47.852816+0000","last_change":"2024-05-17T22:34:37.442971+0000","last_active":"2024-05-17T22:36:47.852816+0000","last_peered":"2024-05-17T22:36:47.852816+0000","last_clean":"2024-05-17T22:36:47.852816+0000","last_became_active":"2024-05-17T22:34:37.442536+0000","last_became_peered":"2024-05-17T22:34:37.442536+0000","last_unstale":"2024-05-17T22:36:47.852816+0000","last_undegraded":"2024-05-17T22:36:47.852816+0000","last_fullsized":"2024-05-17T22:36:47.852816+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:03:45.224071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.467874+0000","last_change":"2024-05-17T22:34:40.432056+0000","last_active":"2024-05-17T22:37:14.467874+0000","last_peered":"2024-05-17T22:37:14.467874+0000","last_clean":"2024-05-17T22:37:14.467874+0000","last_became_active":"2024-05-17T22:34:40.431626+0000","last_became_peered":"2024-05-17T22:34:40.431626+0000","last_unstale":"2024-05-17T22:37:14.467874+0000","last_undegraded":"2024-05-17T22:37:14.467874+0000","last_fullsized":"2024-05-17T22:37:14.467874+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:12:24.650746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623664+0000","last_change":"2024-05-17T22:34:52.500451+0000","last_active":"2024-05-17T22:36:15.623664+0000","last_peered":"2024-05-17T22:36:15.623664+0000","last_clean":"2024-05-17T22:36:15.623664+0000","last_became_active":"2024-05-17T22:34:52.500031+0000","last_became_peered":"2024-05-17T22:34:52.500031+0000","last_unstale":"2024-05-17T22:36:15.623664+0000","last_undegraded":"2024-05-17T22:36:15.623664+0000","last_fullsized":"2024-05-17T22:36:15.623664+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:54:36.825116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:47.770054+0000","last_change":"2024-05-17T22:34:37.402335+0000","last_active":"2024-05-17T22:36:47.770054+0000","last_peered":"2024-05-17T22:36:47.770054+0000","last_clean":"2024-05-17T22:36:47.770054+0000","last_became_active":"2024-05-17T22:34:37.401982+0000","last_became_peered":"2024-05-17T22:34:37.401982+0000","last_unstale":"2024-05-17T22:36:47.770054+0000","last_undegraded":"2024-05-17T22:36:47.770054+0000","last_fullsized":"2024-05-17T22:36:47.770054+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:58:06.194335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.517775+0000","last_change":"2024-05-17T22:34:40.425524+0000","last_active":"2024-05-17T22:37:14.517775+0000","last_peered":"2024-05-17T22:37:14.517775+0000","last_clean":"2024-05-17T22:37:14.517775+0000","last_became_active":"2024-05-17T22:34:40.425135+0000","last_became_peered":"2024-05-17T22:34:40.425135+0000","last_unstale":"2024-05-17T22:37:14.517775+0000","last_undegraded":"2024-05-17T22:37:14.517775+0000","last_fullsized":"2024-05-17T22:37:14.517775+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:41:14.176477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004843+0000","last_change":"2024-05-17T22:34:52.496290+0000","last_active":"2024-05-17T22:36:16.004843+0000","last_peered":"2024-05-17T22:36:16.004843+0000","last_clean":"2024-05-17T22:36:16.004843+0000","last_became_active":"2024-05-17T22:34:52.495817+0000","last_became_peered":"2024-05-17T22:34:52.495817+0000","last_unstale":"2024-05-17T22:36:16.004843+0000","last_undegraded":"2024-05-17T22:36:16.004843+0000","last_fullsized":"2024-05-17T22:36:16.004843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:47:22.056372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'1","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:47.385104+0000","last_change":"2024-05-17T22:34:37.404234+0000","last_active":"2024-05-17T22:36:47.385104+0000","last_peered":"2024-05-17T22:36:47.385104+0000","last_clean":"2024-05-17T22:36:47.385104+0000","last_became_active":"2024-05-17T22:34:37.403561+0000","last_became_peered":"2024-05-17T22:34:37.403561+0000","last_unstale":"2024-05-17T22:36:47.385104+0000","last_undegraded":"2024-05-17T22:36:47.385104+0000","last_fullsized":"2024-05-17T22:36:47.385104+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:30:28.432641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.517802+0000","last_change":"2024-05-17T22:34:40.426522+0000","last_active":"2024-05-17T22:37:14.517802+0000","last_peered":"2024-05-17T22:37:14.517802+0000","last_clean":"2024-05-17T22:37:14.517802+0000","last_became_active":"2024-05-17T22:34:40.426139+0000","last_became_peered":"2024-05-17T22:34:40.426139+0000","last_unstale":"2024-05-17T22:37:14.517802+0000","last_undegraded":"2024-05-17T22:37:14.517802+0000","last_fullsized":"2024-05-17T22:37:14.517802+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:30:19.503513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004076+0000","last_change":"2024-05-17T22:34:52.487131+0000","last_active":"2024-05-17T22:36:16.004076+0000","last_peered":"2024-05-17T22:36:16.004076+0000","last_clean":"2024-05-17T22:36:16.004076+0000","last_became_active":"2024-05-17T22:34:52.486654+0000","last_became_peered":"2024-05-17T22:34:52.486654+0000","last_unstale":"2024-05-17T22:36:16.004076+0000","last_undegraded":"2024-05-17T22:36:16.004076+0000","last_fullsized":"2024-05-17T22:36:16.004076+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:28:25.041746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625893+0000","last_change":"2024-05-17T22:34:37.405020+0000","last_active":"2024-05-17T22:36:15.625893+0000","last_peered":"2024-05-17T22:36:15.625893+0000","last_clean":"2024-05-17T22:36:15.625893+0000","last_became_active":"2024-05-17T22:34:37.404633+0000","last_became_peered":"2024-05-17T22:34:37.404633+0000","last_unstale":"2024-05-17T22:36:15.625893+0000","last_undegraded":"2024-05-17T22:36:15.625893+0000","last_fullsized":"2024-05-17T22:36:15.625893+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:06:00.001190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.525054+0000","last_change":"2024-05-17T22:34:40.435374+0000","last_active":"2024-05-17T22:37:14.525054+0000","last_peered":"2024-05-17T22:37:14.525054+0000","last_clean":"2024-05-17T22:37:14.525054+0000","last_became_active":"2024-05-17T22:34:40.434522+0000","last_became_peered":"2024-05-17T22:34:40.434522+0000","last_unstale":"2024-05-17T22:37:14.525054+0000","last_undegraded":"2024-05-17T22:37:14.525054+0000","last_fullsized":"2024-05-17T22:37:14.525054+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:36:41.244970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.805274+0000","last_change":"2024-05-17T22:34:52.491708+0000","last_active":"2024-05-17T22:36:15.805274+0000","last_peered":"2024-05-17T22:36:15.805274+0000","last_clean":"2024-05-17T22:36:15.805274+0000","last_became_active":"2024-05-17T22:34:52.491293+0000","last_became_peered":"2024-05-17T22:34:52.491293+0000","last_unstale":"2024-05-17T22:36:15.805274+0000","last_undegraded":"2024-05-17T22:36:15.805274+0000","last_fullsized":"2024-05-17T22:36:15.805274+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:23:01.634026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624679+0000","last_change":"2024-05-17T22:34:37.800525+0000","last_active":"2024-05-17T22:36:15.624679+0000","last_peered":"2024-05-17T22:36:15.624679+0000","last_clean":"2024-05-17T22:36:15.624679+0000","last_became_active":"2024-05-17T22:34:37.800093+0000","last_became_peered":"2024-05-17T22:34:37.800093+0000","last_unstale":"2024-05-17T22:36:15.624679+0000","last_undegraded":"2024-05-17T22:36:15.624679+0000","last_fullsized":"2024-05-17T22:36:15.624679+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:25:27.249889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.524773+0000","last_change":"2024-05-17T22:34:40.434891+0000","last_active":"2024-05-17T22:37:14.524773+0000","last_peered":"2024-05-17T22:37:14.524773+0000","last_clean":"2024-05-17T22:37:14.524773+0000","last_became_active":"2024-05-17T22:34:40.434543+0000","last_became_peered":"2024-05-17T22:34:40.434543+0000","last_unstale":"2024-05-17T22:37:14.524773+0000","last_undegraded":"2024-05-17T22:37:14.524773+0000","last_fullsized":"2024-05-17T22:37:14.524773+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:17:47.770225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968412+0000","last_change":"2024-05-17T22:34:52.491544+0000","last_active":"2024-05-17T22:36:15.968412+0000","last_peered":"2024-05-17T22:36:15.968412+0000","last_clean":"2024-05-17T22:36:15.968412+0000","last_became_active":"2024-05-17T22:34:52.491138+0000","last_became_peered":"2024-05-17T22:34:52.491138+0000","last_unstale":"2024-05-17T22:36:15.968412+0000","last_undegraded":"2024-05-17T22:36:15.968412+0000","last_fullsized":"2024-05-17T22:36:15.968412+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:34:20.411299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004302+0000","last_change":"2024-05-17T22:34:37.403220+0000","last_active":"2024-05-17T22:36:16.004302+0000","last_peered":"2024-05-17T22:36:16.004302+0000","last_clean":"2024-05-17T22:36:16.004302+0000","last_became_active":"2024-05-17T22:34:37.402856+0000","last_became_peered":"2024-05-17T22:34:37.402856+0000","last_unstale":"2024-05-17T22:36:16.004302+0000","last_undegraded":"2024-05-17T22:36:16.004302+0000","last_fullsized":"2024-05-17T22:36:16.004302+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:04:19.469294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.511989+0000","last_change":"2024-05-17T22:34:40.434487+0000","last_active":"2024-05-17T22:37:14.511989+0000","last_peered":"2024-05-17T22:37:14.511989+0000","last_clean":"2024-05-17T22:37:14.511989+0000","last_became_active":"2024-05-17T22:34:40.434160+0000","last_became_peered":"2024-05-17T22:34:40.434160+0000","last_unstale":"2024-05-17T22:37:14.511989+0000","last_undegraded":"2024-05-17T22:37:14.511989+0000","last_fullsized":"2024-05-17T22:37:14.511989+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:24:38.782999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003180+0000","last_change":"2024-05-17T22:34:52.487887+0000","last_active":"2024-05-17T22:36:16.003180+0000","last_peered":"2024-05-17T22:36:16.003180+0000","last_clean":"2024-05-17T22:36:16.003180+0000","last_became_active":"2024-05-17T22:34:52.486837+0000","last_became_peered":"2024-05-17T22:34:52.486837+0000","last_unstale":"2024-05-17T22:36:16.003180+0000","last_undegraded":"2024-05-17T22:36:16.003180+0000","last_fullsized":"2024-05-17T22:36:16.003180+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:12:22.206536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.686208+0000","last_change":"2024-05-17T22:34:37.399498+0000","last_active":"2024-05-17T22:36:15.686208+0000","last_peered":"2024-05-17T22:36:15.686208+0000","last_clean":"2024-05-17T22:36:15.686208+0000","last_became_active":"2024-05-17T22:34:37.399144+0000","last_became_peered":"2024-05-17T22:34:37.399144+0000","last_unstale":"2024-05-17T22:36:15.686208+0000","last_undegraded":"2024-05-17T22:36:15.686208+0000","last_fullsized":"2024-05-17T22:36:15.686208+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:13:37.346006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.514102+0000","last_change":"2024-05-17T22:34:40.433839+0000","last_active":"2024-05-17T22:37:14.514102+0000","last_peered":"2024-05-17T22:37:14.514102+0000","last_clean":"2024-05-17T22:37:14.514102+0000","last_became_active":"2024-05-17T22:34:40.433489+0000","last_became_peered":"2024-05-17T22:34:40.433489+0000","last_unstale":"2024-05-17T22:37:14.514102+0000","last_undegraded":"2024-05-17T22:37:14.514102+0000","last_fullsized":"2024-05-17T22:37:14.514102+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:13:56.014094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.512654+0000","last_change":"2024-05-17T22:34:40.434953+0000","last_active":"2024-05-17T22:37:14.512654+0000","last_peered":"2024-05-17T22:37:14.512654+0000","last_clean":"2024-05-17T22:37:14.512654+0000","last_became_active":"2024-05-17T22:34:40.434593+0000","last_became_peered":"2024-05-17T22:34:40.434593+0000","last_unstale":"2024-05-17T22:37:14.512654+0000","last_undegraded":"2024-05-17T22:37:14.512654+0000","last_fullsized":"2024-05-17T22:37:14.512654+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:20:27.075382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.513666+0000","last_change":"2024-05-17T22:34:40.432787+0000","last_active":"2024-05-17T22:37:14.513666+0000","last_peered":"2024-05-17T22:37:14.513666+0000","last_clean":"2024-05-17T22:37:14.513666+0000","last_became_active":"2024-05-17T22:34:40.432402+0000","last_became_peered":"2024-05-17T22:34:40.432402+0000","last_unstale":"2024-05-17T22:37:14.513666+0000","last_undegraded":"2024-05-17T22:37:14.513666+0000","last_fullsized":"2024-05-17T22:37:14.513666+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:48:13.034276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.515714+0000","last_change":"2024-05-17T22:34:40.426423+0000","last_active":"2024-05-17T22:37:14.515714+0000","last_peered":"2024-05-17T22:37:14.515714+0000","last_clean":"2024-05-17T22:37:14.515714+0000","last_became_active":"2024-05-17T22:34:40.426049+0000","last_became_peered":"2024-05-17T22:34:40.426049+0000","last_unstale":"2024-05-17T22:37:14.515714+0000","last_undegraded":"2024-05-17T22:37:14.515714+0000","last_fullsized":"2024-05-17T22:37:14.515714+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:56:55.631322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.520890+0000","last_change":"2024-05-17T22:34:40.435706+0000","last_active":"2024-05-17T22:37:14.520890+0000","last_peered":"2024-05-17T22:37:14.520890+0000","last_clean":"2024-05-17T22:37:14.520890+0000","last_became_active":"2024-05-17T22:34:40.435330+0000","last_became_peered":"2024-05-17T22:34:40.435330+0000","last_unstale":"2024-05-17T22:37:14.520890+0000","last_undegraded":"2024-05-17T22:37:14.520890+0000","last_fullsized":"2024-05-17T22:37:14.520890+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:25:36.529454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.517285+0000","last_change":"2024-05-17T22:34:40.428997+0000","last_active":"2024-05-17T22:37:14.517285+0000","last_peered":"2024-05-17T22:37:14.517285+0000","last_clean":"2024-05-17T22:37:14.517285+0000","last_became_active":"2024-05-17T22:34:40.428594+0000","last_became_peered":"2024-05-17T22:34:40.428594+0000","last_unstale":"2024-05-17T22:37:14.517285+0000","last_undegraded":"2024-05-17T22:37:14.517285+0000","last_fullsized":"2024-05-17T22:37:14.517285+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:59:01.777751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.509268+0000","last_change":"2024-05-17T22:34:40.434241+0000","last_active":"2024-05-17T22:37:14.509268+0000","last_peered":"2024-05-17T22:37:14.509268+0000","last_clean":"2024-05-17T22:37:14.509268+0000","last_became_active":"2024-05-17T22:34:40.431652+0000","last_became_peered":"2024-05-17T22:34:40.431652+0000","last_unstale":"2024-05-17T22:37:14.509268+0000","last_undegraded":"2024-05-17T22:37:14.509268+0000","last_fullsized":"2024-05-17T22:37:14.509268+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:33:27.035284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.505573+0000","last_change":"2024-05-17T22:34:40.431485+0000","last_active":"2024-05-17T22:37:14.505573+0000","last_peered":"2024-05-17T22:37:14.505573+0000","last_clean":"2024-05-17T22:37:14.505573+0000","last_became_active":"2024-05-17T22:34:40.431120+0000","last_became_peered":"2024-05-17T22:34:40.431120+0000","last_unstale":"2024-05-17T22:37:14.505573+0000","last_undegraded":"2024-05-17T22:37:14.505573+0000","last_fullsized":"2024-05-17T22:37:14.505573+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:46:55.213602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.501567+0000","last_change":"2024-05-17T22:34:40.423117+0000","last_active":"2024-05-17T22:37:14.501567+0000","last_peered":"2024-05-17T22:37:14.501567+0000","last_clean":"2024-05-17T22:37:14.501567+0000","last_became_active":"2024-05-17T22:34:40.422694+0000","last_became_peered":"2024-05-17T22:34:40.422694+0000","last_unstale":"2024-05-17T22:37:14.501567+0000","last_undegraded":"2024-05-17T22:37:14.501567+0000","last_fullsized":"2024-05-17T22:37:14.501567+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:19:55.589532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.518389+0000","last_change":"2024-05-17T22:35:15.711796+0000","last_active":"2024-05-17T22:37:14.518389+0000","last_peered":"2024-05-17T22:37:14.518389+0000","last_clean":"2024-05-17T22:37:14.518389+0000","last_became_active":"2024-05-17T22:35:15.711327+0000","last_became_peered":"2024-05-17T22:35:15.711327+0000","last_unstale":"2024-05-17T22:37:14.518389+0000","last_undegraded":"2024-05-17T22:37:14.518389+0000","last_fullsized":"2024-05-17T22:37:14.518389+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:48:48.913312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.512607+0000","last_change":"2024-05-17T22:34:40.430393+0000","last_active":"2024-05-17T22:37:14.512607+0000","last_peered":"2024-05-17T22:37:14.512607+0000","last_clean":"2024-05-17T22:37:14.512607+0000","last_became_active":"2024-05-17T22:34:40.430030+0000","last_became_peered":"2024-05-17T22:34:40.430030+0000","last_unstale":"2024-05-17T22:37:14.512607+0000","last_undegraded":"2024-05-17T22:37:14.512607+0000","last_fullsized":"2024-05-17T22:37:14.512607+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:39:37.204246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.526052+0000","last_change":"2024-05-17T22:34:40.432443+0000","last_active":"2024-05-17T22:37:14.526052+0000","last_peered":"2024-05-17T22:37:14.526052+0000","last_clean":"2024-05-17T22:37:14.526052+0000","last_became_active":"2024-05-17T22:34:40.431991+0000","last_became_peered":"2024-05-17T22:34:40.431991+0000","last_unstale":"2024-05-17T22:37:14.526052+0000","last_undegraded":"2024-05-17T22:37:14.526052+0000","last_fullsized":"2024-05-17T22:37:14.526052+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:37:45.593575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.523169+0000","last_change":"2024-05-17T22:34:40.434388+0000","last_active":"2024-05-17T22:37:14.523169+0000","last_peered":"2024-05-17T22:37:14.523169+0000","last_clean":"2024-05-17T22:37:14.523169+0000","last_became_active":"2024-05-17T22:34:40.434044+0000","last_became_peered":"2024-05-17T22:34:40.434044+0000","last_unstale":"2024-05-17T22:37:14.523169+0000","last_undegraded":"2024-05-17T22:37:14.523169+0000","last_fullsized":"2024-05-17T22:37:14.523169+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:20:43.296190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.507137+0000","last_change":"2024-05-17T22:34:40.427137+0000","last_active":"2024-05-17T22:37:14.507137+0000","last_peered":"2024-05-17T22:37:14.507137+0000","last_clean":"2024-05-17T22:37:14.507137+0000","last_became_active":"2024-05-17T22:34:40.426703+0000","last_became_peered":"2024-05-17T22:34:40.426703+0000","last_unstale":"2024-05-17T22:37:14.507137+0000","last_undegraded":"2024-05-17T22:37:14.507137+0000","last_fullsized":"2024-05-17T22:37:14.507137+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:55:52.992817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72984,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1170,"num_read_kb":665,"num_write":185,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":434176,"data_stored":208206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710788,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":515036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907018,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28696,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716456,"statfs":{"total":95995035648,"available":95965650944,"internally_reserved":0,"allocated":1720320,"data_stored":1170816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070541,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":519346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266772,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":515148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863463001,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1634304,"data_stored":1112060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626525,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":510426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574968,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1089536,"data_stored":572965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738493,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":511814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411443,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1114112,"data_stored":575155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182510,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":510620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986280,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":515608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822756,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36940,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708212,"statfs":{"total":95995035648,"available":95957209088,"internally_reserved":0,"allocated":1642496,"data_stored":1107116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T22:39:01.890 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-17T22:39:01.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:01 smithi151 ceph-mon[36101]: pgmap v432: 129 pgs: 129 active+clean; 648 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-17T22:39:01.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:01 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/2442701764' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:39:02.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.039+0000 7f4561be0640 1 -- 172.21.15.12:0/2708012755 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f455c1014a0 msgr2=0x7f455c1018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:02.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.039+0000 7f4561be0640 1 --2- 172.21.15.12:0/2708012755 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f455c1014a0 0x7f455c1018d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f454c009900 tx=0x7f454c02a410 comp rx=0 tx=0).stop 2024-05-17T22:39:02.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.039+0000 7f4561be0640 1 -- 172.21.15.12:0/2708012755 shutdown_connections 2024-05-17T22:39:02.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.039+0000 7f4561be0640 1 --2- 172.21.15.12:0/2708012755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f455c108ee0 0x7f455c10b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:02.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.039+0000 7f4561be0640 1 --2- 172.21.15.12:0/2708012755 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f455c106580 0x7f455c1089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:02.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.039+0000 7f4561be0640 1 --2- 172.21.15.12:0/2708012755 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f455c1014a0 0x7f455c1018d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:02.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.039+0000 7f4561be0640 1 -- 172.21.15.12:0/2708012755 >> 172.21.15.12:0/2708012755 conn(0x7f455c0fcfe0 msgr2=0x7f455c0ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:39:02.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.039+0000 7f4561be0640 1 -- 172.21.15.12:0/2708012755 shutdown_connections 2024-05-17T22:39:02.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.040+0000 7f4561be0640 1 -- 172.21.15.12:0/2708012755 wait complete. 2024-05-17T22:39:02.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.040+0000 7f4561be0640 1 Processor -- start 2024-05-17T22:39:02.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.040+0000 7f4561be0640 1 -- start start 2024-05-17T22:39:02.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.041+0000 7f4561be0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f455c1014a0 0x7f455c199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:02.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.041+0000 7f4561be0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f455c106580 0x7f455c199d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:02.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.041+0000 7f4561be0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f455c108ee0 0x7f455c1a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:02.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.041+0000 7f4561be0640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f455c10ec50 con 0x7f455c106580 2024-05-17T22:39:02.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.041+0000 7f4561be0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f455c10ead0 con 0x7f455c108ee0 2024-05-17T22:39:02.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.041+0000 7f4561be0640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f455c10edd0 con 0x7f455c1014a0 2024-05-17T22:39:02.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.041+0000 7f4552ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f455c106580 0x7f455c199d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:02.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.041+0000 7f4552ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f455c106580 0x7f455c199d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60970/0 (socket says 172.21.15.12:60970) 2024-05-17T22:39:02.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.041+0000 7f4552ffd640 1 -- 172.21.15.12:0/1442164270 learned_addr learned my addr 172.21.15.12:0/1442164270 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:39:02.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.041+0000 7f455bfff640 1 --2- 172.21.15.12:0/1442164270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f455c108ee0 0x7f455c1a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:02.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.041+0000 7f455b7fe640 1 --2- 172.21.15.12:0/1442164270 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f455c1014a0 0x7f455c199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:02.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.041+0000 7f4552ffd640 1 -- 172.21.15.12:0/1442164270 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f455c1014a0 msgr2=0x7f455c199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:02.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.041+0000 7f4552ffd640 1 --2- 172.21.15.12:0/1442164270 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f455c1014a0 0x7f455c199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:02.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.041+0000 7f4552ffd640 1 -- 172.21.15.12:0/1442164270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f455c108ee0 msgr2=0x7f455c1a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:02.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.041+0000 7f4552ffd640 1 --2- 172.21.15.12:0/1442164270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f455c108ee0 0x7f455c1a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:02.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.041+0000 7f4552ffd640 1 -- 172.21.15.12:0/1442164270 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f455c1a1380 con 0x7f455c106580 2024-05-17T22:39:02.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.041+0000 7f4552ffd640 1 --2- 172.21.15.12:0/1442164270 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f455c106580 0x7f455c199d80 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f454400d770 tx=0x7f454400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:39:02.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.042+0000 7f45597fa640 1 -- 172.21.15.12:0/1442164270 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4544017070 con 0x7f455c106580 2024-05-17T22:39:02.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.042+0000 7f4561be0640 1 -- 172.21.15.12:0/1442164270 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f455c1a1670 con 0x7f455c106580 2024-05-17T22:39:02.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.042+0000 7f45597fa640 1 -- 172.21.15.12:0/1442164270 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45440054d0 con 0x7f455c106580 2024-05-17T22:39:02.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.042+0000 7f45597fa640 1 -- 172.21.15.12:0/1442164270 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4544004c30 con 0x7f455c106580 2024-05-17T22:39:02.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.042+0000 7f4561be0640 1 -- 172.21.15.12:0/1442164270 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f455c1a1b70 con 0x7f455c106580 2024-05-17T22:39:02.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.043+0000 7f4561be0640 1 -- 172.21.15.12:0/1442164270 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f455c0feb00 con 0x7f455c106580 2024-05-17T22:39:02.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.044+0000 7f45597fa640 1 -- 172.21.15.12:0/1442164270 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4544004dd0 con 0x7f455c106580 2024-05-17T22:39:02.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.045+0000 7f45597fa640 1 --2- 172.21.15.12:0/1442164270 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f45480b27f0 0x7f45480b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:02.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.045+0000 7f455b7fe640 1 --2- 172.21.15.12:0/1442164270 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f45480b27f0 0x7f45480b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:02.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.045+0000 7f45597fa640 1 -- 172.21.15.12:0/1442164270 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f45440e2430 con 0x7f455c106580 2024-05-17T22:39:02.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:01 smithi096 ceph-mon[36065]: pgmap v432: 129 pgs: 129 active+clean; 648 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-17T22:39:02.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:01 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/2442701764' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:39:02.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.046+0000 7f455b7fe640 1 --2- 172.21.15.12:0/1442164270 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f45480b27f0 0x7f45480b4d00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f454c000c00 tx=0x7f454c031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:39:02.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.048+0000 7f45597fa640 1 -- 172.21.15.12:0/1442164270 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f45440a9710 con 0x7f455c106580 2024-05-17T22:39:02.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.238+0000 7f4561be0640 1 -- 172.21.15.12:0/1442164270 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f455c063180 con 0x7f45480b27f0 2024-05-17T22:39:02.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.244+0000 7f45597fa640 1 -- 172.21.15.12:0/1442164270 <== mgr.14152 v2:172.21.15.12:6800/1388514047 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340471 (secure 0 0 0) 0x7f455c063180 con 0x7f45480b27f0 2024-05-17T22:39:02.250 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:39:02.252 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-05-17T22:39:02.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.247+0000 7f4561be0640 1 -- 172.21.15.12:0/1442164270 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f45480b27f0 msgr2=0x7f45480b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:02.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.247+0000 7f4561be0640 1 --2- 172.21.15.12:0/1442164270 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f45480b27f0 0x7f45480b4d00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f454c000c00 tx=0x7f454c031400 comp rx=0 tx=0).stop 2024-05-17T22:39:02.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.247+0000 7f4561be0640 1 -- 172.21.15.12:0/1442164270 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f455c106580 msgr2=0x7f455c199d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:02.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.247+0000 7f4561be0640 1 --2- 172.21.15.12:0/1442164270 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f455c106580 0x7f455c199d80 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f454400d770 tx=0x7f454400dc30 comp rx=0 tx=0).stop 2024-05-17T22:39:02.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.247+0000 7f4561be0640 1 -- 172.21.15.12:0/1442164270 shutdown_connections 2024-05-17T22:39:02.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.247+0000 7f4561be0640 1 --2- 172.21.15.12:0/1442164270 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f45480b27f0 0x7f45480b4d00 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:02.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.247+0000 7f4561be0640 1 --2- 172.21.15.12:0/1442164270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f455c108ee0 0x7f455c1a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:02.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.247+0000 7f4561be0640 1 --2- 172.21.15.12:0/1442164270 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f455c106580 0x7f455c199d80 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:02.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.247+0000 7f4561be0640 1 --2- 172.21.15.12:0/1442164270 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f455c1014a0 0x7f455c199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:02.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.247+0000 7f4561be0640 1 -- 172.21.15.12:0/1442164270 >> 172.21.15.12:0/1442164270 conn(0x7f455c0fcfe0 msgr2=0x7f455c107430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:39:02.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.247+0000 7f4561be0640 1 -- 172.21.15.12:0/1442164270 shutdown_connections 2024-05-17T22:39:02.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.247+0000 7f4561be0640 1 -- 172.21.15.12:0/1442164270 wait complete. 2024-05-17T22:39:02.263 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2024-05-17T22:39:00.554581+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663586,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1246,"num_read_kb":729,"num_write":310,"num_write_kb":2288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":752,"ondisk_log_size":752,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":388212,"kb_used_data":14100,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553612,"statfs":{"total":1151940427776,"available":1151542898688,"internally_reserved":0,"allocated":14438400,"data_stored":8136110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5989,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":83,"num_read_kb":53,"num_write":7,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003416"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.522500+0000","last_change":"2024-05-17T22:34:40.422850+0000","last_active":"2024-05-17T22:37:14.522500+0000","last_peered":"2024-05-17T22:37:14.522500+0000","last_clean":"2024-05-17T22:37:14.522500+0000","last_became_active":"2024-05-17T22:34:40.422474+0000","last_became_peered":"2024-05-17T22:34:40.422474+0000","last_unstale":"2024-05-17T22:37:14.522500+0000","last_undegraded":"2024-05-17T22:37:14.522500+0000","last_fullsized":"2024-05-17T22:37:14.522500+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:23:51.537128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.520370+0000","last_change":"2024-05-17T22:34:40.427907+0000","last_active":"2024-05-17T22:37:14.520370+0000","last_peered":"2024-05-17T22:37:14.520370+0000","last_clean":"2024-05-17T22:37:14.520370+0000","last_became_active":"2024-05-17T22:34:40.427511+0000","last_became_peered":"2024-05-17T22:34:40.427511+0000","last_unstale":"2024-05-17T22:37:14.520370+0000","last_undegraded":"2024-05-17T22:37:14.520370+0000","last_fullsized":"2024-05-17T22:37:14.520370+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:01:46.707820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.525760+0000","last_change":"2024-05-17T22:34:40.436370+0000","last_active":"2024-05-17T22:37:14.525760+0000","last_peered":"2024-05-17T22:37:14.525760+0000","last_clean":"2024-05-17T22:37:14.525760+0000","last_became_active":"2024-05-17T22:34:40.434089+0000","last_became_peered":"2024-05-17T22:34:40.434089+0000","last_unstale":"2024-05-17T22:37:14.525760+0000","last_undegraded":"2024-05-17T22:37:14.525760+0000","last_fullsized":"2024-05-17T22:37:14.525760+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:58:16.775471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.509106+0000","last_change":"2024-05-17T22:35:15.556681+0000","last_active":"2024-05-17T22:37:14.509106+0000","last_peered":"2024-05-17T22:37:14.509106+0000","last_clean":"2024-05-17T22:37:14.509106+0000","last_became_active":"2024-05-17T22:35:15.556427+0000","last_became_peered":"2024-05-17T22:35:15.556427+0000","last_unstale":"2024-05-17T22:37:14.509106+0000","last_undegraded":"2024-05-17T22:37:14.509106+0000","last_fullsized":"2024-05-17T22:37:14.509106+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:28:01.726262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.495402+0000","last_change":"2024-05-17T22:34:40.434394+0000","last_active":"2024-05-17T22:37:14.495402+0000","last_peered":"2024-05-17T22:37:14.495402+0000","last_clean":"2024-05-17T22:37:14.495402+0000","last_became_active":"2024-05-17T22:34:40.433489+0000","last_became_peered":"2024-05-17T22:34:40.433489+0000","last_unstale":"2024-05-17T22:37:14.495402+0000","last_undegraded":"2024-05-17T22:37:14.495402+0000","last_fullsized":"2024-05-17T22:37:14.495402+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:26:24.068814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.515238+0000","last_change":"2024-05-17T22:35:15.553601+0000","last_active":"2024-05-17T22:37:14.515238+0000","last_peered":"2024-05-17T22:37:14.515238+0000","last_clean":"2024-05-17T22:37:14.515238+0000","last_became_active":"2024-05-17T22:35:15.553116+0000","last_became_peered":"2024-05-17T22:35:15.553116+0000","last_unstale":"2024-05-17T22:37:14.515238+0000","last_undegraded":"2024-05-17T22:37:14.515238+0000","last_fullsized":"2024-05-17T22:37:14.515238+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:09:38.401107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.518979+0000","last_change":"2024-05-17T22:34:40.430479+0000","last_active":"2024-05-17T22:37:14.518979+0000","last_peered":"2024-05-17T22:37:14.518979+0000","last_clean":"2024-05-17T22:37:14.518979+0000","last_became_active":"2024-05-17T22:34:40.430025+0000","last_became_peered":"2024-05-17T22:34:40.430025+0000","last_unstale":"2024-05-17T22:37:14.518979+0000","last_undegraded":"2024-05-17T22:37:14.518979+0000","last_fullsized":"2024-05-17T22:37:14.518979+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:14:19.725096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.519153+0000","last_change":"2024-05-17T22:34:40.431711+0000","last_active":"2024-05-17T22:37:14.519153+0000","last_peered":"2024-05-17T22:37:14.519153+0000","last_clean":"2024-05-17T22:37:14.519153+0000","last_became_active":"2024-05-17T22:34:40.431368+0000","last_became_peered":"2024-05-17T22:34:40.431368+0000","last_unstale":"2024-05-17T22:37:14.519153+0000","last_undegraded":"2024-05-17T22:37:14.519153+0000","last_fullsized":"2024-05-17T22:37:14.519153+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:06:34.904652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.521909+0000","last_change":"2024-05-17T22:34:40.435593+0000","last_active":"2024-05-17T22:37:14.521909+0000","last_peered":"2024-05-17T22:37:14.521909+0000","last_clean":"2024-05-17T22:37:14.521909+0000","last_became_active":"2024-05-17T22:34:40.435241+0000","last_became_peered":"2024-05-17T22:34:40.435241+0000","last_unstale":"2024-05-17T22:37:14.521909+0000","last_undegraded":"2024-05-17T22:37:14.521909+0000","last_fullsized":"2024-05-17T22:37:14.521909+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:08:57.239998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.526427+0000","last_change":"2024-05-17T22:34:40.436248+0000","last_active":"2024-05-17T22:37:14.526427+0000","last_peered":"2024-05-17T22:37:14.526427+0000","last_clean":"2024-05-17T22:37:14.526427+0000","last_became_active":"2024-05-17T22:34:40.435824+0000","last_became_peered":"2024-05-17T22:34:40.435824+0000","last_unstale":"2024-05-17T22:37:14.526427+0000","last_undegraded":"2024-05-17T22:37:14.526427+0000","last_fullsized":"2024-05-17T22:37:14.526427+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:39:07.297470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.524640+0000","last_change":"2024-05-17T22:34:40.435843+0000","last_active":"2024-05-17T22:37:14.524640+0000","last_peered":"2024-05-17T22:37:14.524640+0000","last_clean":"2024-05-17T22:37:14.524640+0000","last_became_active":"2024-05-17T22:34:40.435500+0000","last_became_peered":"2024-05-17T22:34:40.435500+0000","last_unstale":"2024-05-17T22:37:14.524640+0000","last_undegraded":"2024-05-17T22:37:14.524640+0000","last_fullsized":"2024-05-17T22:37:14.524640+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:11:28.115850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.501798+0000","last_change":"2024-05-17T22:34:40.432044+0000","last_active":"2024-05-17T22:37:14.501798+0000","last_peered":"2024-05-17T22:37:14.501798+0000","last_clean":"2024-05-17T22:37:14.501798+0000","last_became_active":"2024-05-17T22:34:40.429141+0000","last_became_peered":"2024-05-17T22:34:40.429141+0000","last_unstale":"2024-05-17T22:37:14.501798+0000","last_undegraded":"2024-05-17T22:37:14.501798+0000","last_fullsized":"2024-05-17T22:37:14.501798+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:04:53.428681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.526534+0000","last_change":"2024-05-17T22:34:40.438474+0000","last_active":"2024-05-17T22:37:14.526534+0000","last_peered":"2024-05-17T22:37:14.526534+0000","last_clean":"2024-05-17T22:37:14.526534+0000","last_became_active":"2024-05-17T22:34:40.438109+0000","last_became_peered":"2024-05-17T22:34:40.438109+0000","last_unstale":"2024-05-17T22:37:14.526534+0000","last_undegraded":"2024-05-17T22:37:14.526534+0000","last_fullsized":"2024-05-17T22:37:14.526534+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:26:00.665246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.516486+0000","last_change":"2024-05-17T22:34:40.431563+0000","last_active":"2024-05-17T22:37:14.516486+0000","last_peered":"2024-05-17T22:37:14.516486+0000","last_clean":"2024-05-17T22:37:14.516486+0000","last_became_active":"2024-05-17T22:34:40.430992+0000","last_became_peered":"2024-05-17T22:34:40.430992+0000","last_unstale":"2024-05-17T22:37:14.516486+0000","last_undegraded":"2024-05-17T22:37:14.516486+0000","last_fullsized":"2024-05-17T22:37:14.516486+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:51:26.280440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.516587+0000","last_change":"2024-05-17T22:34:40.433629+0000","last_active":"2024-05-17T22:37:14.516587+0000","last_peered":"2024-05-17T22:37:14.516587+0000","last_clean":"2024-05-17T22:37:14.516587+0000","last_became_active":"2024-05-17T22:34:40.433270+0000","last_became_peered":"2024-05-17T22:34:40.433270+0000","last_unstale":"2024-05-17T22:37:14.516587+0000","last_undegraded":"2024-05-17T22:37:14.516587+0000","last_fullsized":"2024-05-17T22:37:14.516587+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:57:51.164078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.525064+0000","last_change":"2024-05-17T22:34:40.434218+0000","last_active":"2024-05-17T22:37:14.525064+0000","last_peered":"2024-05-17T22:37:14.525064+0000","last_clean":"2024-05-17T22:37:14.525064+0000","last_became_active":"2024-05-17T22:34:40.433849+0000","last_became_peered":"2024-05-17T22:34:40.433849+0000","last_unstale":"2024-05-17T22:37:14.525064+0000","last_undegraded":"2024-05-17T22:37:14.525064+0000","last_fullsized":"2024-05-17T22:37:14.525064+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:41:55.984109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.522605+0000","last_change":"2024-05-17T22:34:40.427892+0000","last_active":"2024-05-17T22:37:14.522605+0000","last_peered":"2024-05-17T22:37:14.522605+0000","last_clean":"2024-05-17T22:37:14.522605+0000","last_became_active":"2024-05-17T22:34:40.427303+0000","last_became_peered":"2024-05-17T22:34:40.427303+0000","last_unstale":"2024-05-17T22:37:14.522605+0000","last_undegraded":"2024-05-17T22:37:14.522605+0000","last_fullsized":"2024-05-17T22:37:14.522605+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:59:38.632123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.524943+0000","last_change":"2024-05-17T22:34:40.424784+0000","last_active":"2024-05-17T22:37:14.524943+0000","last_peered":"2024-05-17T22:37:14.524943+0000","last_clean":"2024-05-17T22:37:14.524943+0000","last_became_active":"2024-05-17T22:34:40.424439+0000","last_became_peered":"2024-05-17T22:34:40.424439+0000","last_unstale":"2024-05-17T22:37:14.524943+0000","last_undegraded":"2024-05-17T22:37:14.524943+0000","last_fullsized":"2024-05-17T22:37:14.524943+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:02:53.681600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.513305+0000","last_change":"2024-05-17T22:34:40.435023+0000","last_active":"2024-05-17T22:37:14.513305+0000","last_peered":"2024-05-17T22:37:14.513305+0000","last_clean":"2024-05-17T22:37:14.513305+0000","last_became_active":"2024-05-17T22:34:40.434441+0000","last_became_peered":"2024-05-17T22:34:40.434441+0000","last_unstale":"2024-05-17T22:37:14.513305+0000","last_undegraded":"2024-05-17T22:37:14.513305+0000","last_fullsized":"2024-05-17T22:37:14.513305+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:16:07.671897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624113+0000","last_change":"2024-05-17T22:34:52.493192+0000","last_active":"2024-05-17T22:36:15.624113+0000","last_peered":"2024-05-17T22:36:15.624113+0000","last_clean":"2024-05-17T22:36:15.624113+0000","last_became_active":"2024-05-17T22:34:52.492786+0000","last_became_peered":"2024-05-17T22:34:52.492786+0000","last_unstale":"2024-05-17T22:36:15.624113+0000","last_undegraded":"2024-05-17T22:36:15.624113+0000","last_fullsized":"2024-05-17T22:36:15.624113+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:43:40.152240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":56,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:43.067607+0000","last_change":"2024-05-17T22:34:37.402360+0000","last_active":"2024-05-17T22:36:43.067607+0000","last_peered":"2024-05-17T22:36:43.067607+0000","last_clean":"2024-05-17T22:36:43.067607+0000","last_became_active":"2024-05-17T22:34:37.401882+0000","last_became_peered":"2024-05-17T22:34:37.401882+0000","last_unstale":"2024-05-17T22:36:43.067607+0000","last_undegraded":"2024-05-17T22:36:43.067607+0000","last_fullsized":"2024-05-17T22:36:43.067607+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:53:13.300626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.506469+0000","last_change":"2024-05-17T22:34:40.436259+0000","last_active":"2024-05-17T22:37:14.506469+0000","last_peered":"2024-05-17T22:37:14.506469+0000","last_clean":"2024-05-17T22:37:14.506469+0000","last_became_active":"2024-05-17T22:34:40.435899+0000","last_became_peered":"2024-05-17T22:34:40.435899+0000","last_unstale":"2024-05-17T22:37:14.506469+0000","last_undegraded":"2024-05-17T22:37:14.506469+0000","last_fullsized":"2024-05-17T22:37:14.506469+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:26:29.327513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003713+0000","last_change":"2024-05-17T22:34:52.495236+0000","last_active":"2024-05-17T22:36:16.003713+0000","last_peered":"2024-05-17T22:36:16.003713+0000","last_clean":"2024-05-17T22:36:16.003713+0000","last_became_active":"2024-05-17T22:34:52.494783+0000","last_became_peered":"2024-05-17T22:34:52.494783+0000","last_unstale":"2024-05-17T22:36:16.003713+0000","last_undegraded":"2024-05-17T22:36:16.003713+0000","last_fullsized":"2024-05-17T22:36:16.003713+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:08:15.010706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:37.768914+0000","last_change":"2024-05-17T22:34:37.441129+0000","last_active":"2024-05-17T22:36:37.768914+0000","last_peered":"2024-05-17T22:36:37.768914+0000","last_clean":"2024-05-17T22:36:37.768914+0000","last_became_active":"2024-05-17T22:34:37.440738+0000","last_became_peered":"2024-05-17T22:34:37.440738+0000","last_unstale":"2024-05-17T22:36:37.768914+0000","last_undegraded":"2024-05-17T22:36:37.768914+0000","last_fullsized":"2024-05-17T22:36:37.768914+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:03:16.945076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.525530+0000","last_change":"2024-05-17T22:34:40.434146+0000","last_active":"2024-05-17T22:37:14.525530+0000","last_peered":"2024-05-17T22:37:14.525530+0000","last_clean":"2024-05-17T22:37:14.525530+0000","last_became_active":"2024-05-17T22:34:40.433802+0000","last_became_peered":"2024-05-17T22:34:40.433802+0000","last_unstale":"2024-05-17T22:37:14.525530+0000","last_undegraded":"2024-05-17T22:37:14.525530+0000","last_fullsized":"2024-05-17T22:37:14.525530+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:08:39.905409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004211+0000","last_change":"2024-05-17T22:34:52.501134+0000","last_active":"2024-05-17T22:36:16.004211+0000","last_peered":"2024-05-17T22:36:16.004211+0000","last_clean":"2024-05-17T22:36:16.004211+0000","last_became_active":"2024-05-17T22:34:52.500547+0000","last_became_peered":"2024-05-17T22:34:52.500547+0000","last_unstale":"2024-05-17T22:36:16.004211+0000","last_undegraded":"2024-05-17T22:36:16.004211+0000","last_fullsized":"2024-05-17T22:36:16.004211+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:13:04.200865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.626422+0000","last_change":"2024-05-17T22:34:37.404019+0000","last_active":"2024-05-17T22:36:15.626422+0000","last_peered":"2024-05-17T22:36:15.626422+0000","last_clean":"2024-05-17T22:36:15.626422+0000","last_became_active":"2024-05-17T22:34:37.403607+0000","last_became_peered":"2024-05-17T22:34:37.403607+0000","last_unstale":"2024-05-17T22:36:15.626422+0000","last_undegraded":"2024-05-17T22:36:15.626422+0000","last_fullsized":"2024-05-17T22:36:15.626422+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:30:21.405195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.525402+0000","last_change":"2024-05-17T22:34:40.426889+0000","last_active":"2024-05-17T22:37:14.525402+0000","last_peered":"2024-05-17T22:37:14.525402+0000","last_clean":"2024-05-17T22:37:14.525402+0000","last_became_active":"2024-05-17T22:34:40.426512+0000","last_became_peered":"2024-05-17T22:34:40.426512+0000","last_unstale":"2024-05-17T22:37:14.525402+0000","last_undegraded":"2024-05-17T22:37:14.525402+0000","last_fullsized":"2024-05-17T22:37:14.525402+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:30:51.363177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004528+0000","last_change":"2024-05-17T22:34:52.487514+0000","last_active":"2024-05-17T22:36:16.004528+0000","last_peered":"2024-05-17T22:36:16.004528+0000","last_clean":"2024-05-17T22:36:16.004528+0000","last_became_active":"2024-05-17T22:34:52.487068+0000","last_became_peered":"2024-05-17T22:34:52.487068+0000","last_unstale":"2024-05-17T22:36:16.004528+0000","last_undegraded":"2024-05-17T22:36:16.004528+0000","last_fullsized":"2024-05-17T22:36:16.004528+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:52:13.573986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.626123+0000","last_change":"2024-05-17T22:34:37.400518+0000","last_active":"2024-05-17T22:36:15.626123+0000","last_peered":"2024-05-17T22:36:15.626123+0000","last_clean":"2024-05-17T22:36:15.626123+0000","last_became_active":"2024-05-17T22:34:37.400145+0000","last_became_peered":"2024-05-17T22:34:37.400145+0000","last_unstale":"2024-05-17T22:36:15.626123+0000","last_undegraded":"2024-05-17T22:36:15.626123+0000","last_fullsized":"2024-05-17T22:36:15.626123+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:56:53.593624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.525513+0000","last_change":"2024-05-17T22:34:40.434148+0000","last_active":"2024-05-17T22:37:14.525513+0000","last_peered":"2024-05-17T22:37:14.525513+0000","last_clean":"2024-05-17T22:37:14.525513+0000","last_became_active":"2024-05-17T22:34:40.433613+0000","last_became_peered":"2024-05-17T22:34:40.433613+0000","last_unstale":"2024-05-17T22:37:14.525513+0000","last_undegraded":"2024-05-17T22:37:14.525513+0000","last_fullsized":"2024-05-17T22:37:14.525513+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:09:39.049852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624340+0000","last_change":"2024-05-17T22:34:52.490417+0000","last_active":"2024-05-17T22:36:15.624340+0000","last_peered":"2024-05-17T22:36:15.624340+0000","last_clean":"2024-05-17T22:36:15.624340+0000","last_became_active":"2024-05-17T22:34:52.489953+0000","last_became_peered":"2024-05-17T22:34:52.489953+0000","last_unstale":"2024-05-17T22:36:15.624340+0000","last_undegraded":"2024-05-17T22:36:15.624340+0000","last_fullsized":"2024-05-17T22:36:15.624340+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:23:56.491899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'8","reported_seq":246,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:38:56.337995+0000","last_change":"2024-05-17T22:34:37.405304+0000","last_active":"2024-05-17T22:38:56.337995+0000","last_peered":"2024-05-17T22:38:56.337995+0000","last_clean":"2024-05-17T22:38:56.337995+0000","last_became_active":"2024-05-17T22:34:37.404552+0000","last_became_peered":"2024-05-17T22:34:37.404552+0000","last_unstale":"2024-05-17T22:38:56.337995+0000","last_undegraded":"2024-05-17T22:38:56.337995+0000","last_fullsized":"2024-05-17T22:38:56.337995+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:57:06.556886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1462,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":193,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.512663+0000","last_change":"2024-05-17T22:34:40.430537+0000","last_active":"2024-05-17T22:37:14.512663+0000","last_peered":"2024-05-17T22:37:14.512663+0000","last_clean":"2024-05-17T22:37:14.512663+0000","last_became_active":"2024-05-17T22:34:40.430193+0000","last_became_peered":"2024-05-17T22:34:40.430193+0000","last_unstale":"2024-05-17T22:37:14.512663+0000","last_undegraded":"2024-05-17T22:37:14.512663+0000","last_fullsized":"2024-05-17T22:37:14.512663+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:58:42.091445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624546+0000","last_change":"2024-05-17T22:35:15.559077+0000","last_active":"2024-05-17T22:36:15.624546+0000","last_peered":"2024-05-17T22:36:15.624546+0000","last_clean":"2024-05-17T22:36:15.624546+0000","last_became_active":"2024-05-17T22:35:15.558382+0000","last_became_peered":"2024-05-17T22:35:15.558382+0000","last_unstale":"2024-05-17T22:36:15.624546+0000","last_undegraded":"2024-05-17T22:36:15.624546+0000","last_fullsized":"2024-05-17T22:36:15.624546+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:20:16.009620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.626030+0000","last_change":"2024-05-17T22:34:37.405332+0000","last_active":"2024-05-17T22:36:15.626030+0000","last_peered":"2024-05-17T22:36:15.626030+0000","last_clean":"2024-05-17T22:36:15.626030+0000","last_became_active":"2024-05-17T22:34:37.404970+0000","last_became_peered":"2024-05-17T22:34:37.404970+0000","last_unstale":"2024-05-17T22:36:15.626030+0000","last_undegraded":"2024-05-17T22:36:15.626030+0000","last_fullsized":"2024-05-17T22:36:15.626030+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:37:39.933686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.517661+0000","last_change":"2024-05-17T22:34:40.432690+0000","last_active":"2024-05-17T22:37:14.517661+0000","last_peered":"2024-05-17T22:37:14.517661+0000","last_clean":"2024-05-17T22:37:14.517661+0000","last_became_active":"2024-05-17T22:34:40.432274+0000","last_became_peered":"2024-05-17T22:34:40.432274+0000","last_unstale":"2024-05-17T22:37:14.517661+0000","last_undegraded":"2024-05-17T22:37:14.517661+0000","last_fullsized":"2024-05-17T22:37:14.517661+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:12:47.926865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628628+0000","last_change":"2024-05-17T22:34:52.494645+0000","last_active":"2024-05-17T22:36:15.628628+0000","last_peered":"2024-05-17T22:36:15.628628+0000","last_clean":"2024-05-17T22:36:15.628628+0000","last_became_active":"2024-05-17T22:34:52.494255+0000","last_became_peered":"2024-05-17T22:34:52.494255+0000","last_unstale":"2024-05-17T22:36:15.628628+0000","last_undegraded":"2024-05-17T22:36:15.628628+0000","last_fullsized":"2024-05-17T22:36:15.628628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:41:26.633571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685426+0000","last_change":"2024-05-17T22:34:37.398582+0000","last_active":"2024-05-17T22:36:15.685426+0000","last_peered":"2024-05-17T22:36:15.685426+0000","last_clean":"2024-05-17T22:36:15.685426+0000","last_became_active":"2024-05-17T22:34:37.398156+0000","last_became_peered":"2024-05-17T22:34:37.398156+0000","last_unstale":"2024-05-17T22:36:15.685426+0000","last_undegraded":"2024-05-17T22:36:15.685426+0000","last_fullsized":"2024-05-17T22:36:15.685426+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:46:00.276214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.518565+0000","last_change":"2024-05-17T22:35:15.551402+0000","last_active":"2024-05-17T22:37:14.518565+0000","last_peered":"2024-05-17T22:37:14.518565+0000","last_clean":"2024-05-17T22:37:14.518565+0000","last_became_active":"2024-05-17T22:35:15.551021+0000","last_became_peered":"2024-05-17T22:35:15.551021+0000","last_unstale":"2024-05-17T22:37:14.518565+0000","last_undegraded":"2024-05-17T22:37:14.518565+0000","last_fullsized":"2024-05-17T22:37:14.518565+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:35:18.573620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:54.120653+0000","last_change":"2024-05-17T22:34:52.487413+0000","last_active":"2024-05-17T22:36:54.120653+0000","last_peered":"2024-05-17T22:36:54.120653+0000","last_clean":"2024-05-17T22:36:54.120653+0000","last_became_active":"2024-05-17T22:34:52.487032+0000","last_became_peered":"2024-05-17T22:34:52.487032+0000","last_unstale":"2024-05-17T22:36:54.120653+0000","last_undegraded":"2024-05-17T22:36:54.120653+0000","last_fullsized":"2024-05-17T22:36:54.120653+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:34:49.122008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003315+0000","last_change":"2024-05-17T22:34:37.404894+0000","last_active":"2024-05-17T22:36:16.003315+0000","last_peered":"2024-05-17T22:36:16.003315+0000","last_clean":"2024-05-17T22:36:16.003315+0000","last_became_active":"2024-05-17T22:34:37.404567+0000","last_became_peered":"2024-05-17T22:34:37.404567+0000","last_unstale":"2024-05-17T22:36:16.003315+0000","last_undegraded":"2024-05-17T22:36:16.003315+0000","last_fullsized":"2024-05-17T22:36:16.003315+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:13:09.761556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.525152+0000","last_change":"2024-05-17T22:34:40.418267+0000","last_active":"2024-05-17T22:37:14.525152+0000","last_peered":"2024-05-17T22:37:14.525152+0000","last_clean":"2024-05-17T22:37:14.525152+0000","last_became_active":"2024-05-17T22:34:40.417928+0000","last_became_peered":"2024-05-17T22:34:40.417928+0000","last_unstale":"2024-05-17T22:37:14.525152+0000","last_undegraded":"2024-05-17T22:37:14.525152+0000","last_fullsized":"2024-05-17T22:37:14.525152+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:17:37.055004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984205+0000","last_change":"2024-05-17T22:34:52.492126+0000","last_active":"2024-05-17T22:36:15.984205+0000","last_peered":"2024-05-17T22:36:15.984205+0000","last_clean":"2024-05-17T22:36:15.984205+0000","last_became_active":"2024-05-17T22:34:52.491695+0000","last_became_peered":"2024-05-17T22:34:52.491695+0000","last_unstale":"2024-05-17T22:36:15.984205+0000","last_undegraded":"2024-05-17T22:36:15.984205+0000","last_fullsized":"2024-05-17T22:36:15.984205+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:15:17.710279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'5","reported_seq":240,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:38:58.340899+0000","last_change":"2024-05-17T22:34:37.406989+0000","last_active":"2024-05-17T22:38:58.340899+0000","last_peered":"2024-05-17T22:38:58.340899+0000","last_clean":"2024-05-17T22:38:58.340899+0000","last_became_active":"2024-05-17T22:34:37.406612+0000","last_became_peered":"2024-05-17T22:34:37.406612+0000","last_unstale":"2024-05-17T22:38:58.340899+0000","last_undegraded":"2024-05-17T22:38:58.340899+0000","last_fullsized":"2024-05-17T22:38:58.340899+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:47:52.480804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":190,"num_read_kb":190,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.524255+0000","last_change":"2024-05-17T22:34:40.427221+0000","last_active":"2024-05-17T22:37:14.524255+0000","last_peered":"2024-05-17T22:37:14.524255+0000","last_clean":"2024-05-17T22:37:14.524255+0000","last_became_active":"2024-05-17T22:34:40.426703+0000","last_became_peered":"2024-05-17T22:34:40.426703+0000","last_unstale":"2024-05-17T22:37:14.524255+0000","last_undegraded":"2024-05-17T22:37:14.524255+0000","last_fullsized":"2024-05-17T22:37:14.524255+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:49:15.629048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.622725+0000","last_change":"2024-05-17T22:34:52.499086+0000","last_active":"2024-05-17T22:36:15.622725+0000","last_peered":"2024-05-17T22:36:15.622725+0000","last_clean":"2024-05-17T22:36:15.622725+0000","last_became_active":"2024-05-17T22:34:52.498666+0000","last_became_peered":"2024-05-17T22:34:52.498666+0000","last_unstale":"2024-05-17T22:36:15.622725+0000","last_undegraded":"2024-05-17T22:36:15.622725+0000","last_fullsized":"2024-05-17T22:36:15.622725+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:46:25.882964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.005213+0000","last_change":"2024-05-17T22:35:15.836294+0000","last_active":"2024-05-17T22:36:16.005213+0000","last_peered":"2024-05-17T22:36:16.005213+0000","last_clean":"2024-05-17T22:36:16.005213+0000","last_became_active":"2024-05-17T22:35:15.835759+0000","last_became_peered":"2024-05-17T22:35:15.835759+0000","last_unstale":"2024-05-17T22:36:16.005213+0000","last_undegraded":"2024-05-17T22:36:16.005213+0000","last_fullsized":"2024-05-17T22:36:16.005213+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:23:32.106692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.519861+0000","last_change":"2024-05-17T22:34:40.433331+0000","last_active":"2024-05-17T22:37:14.519861+0000","last_peered":"2024-05-17T22:37:14.519861+0000","last_clean":"2024-05-17T22:37:14.519861+0000","last_became_active":"2024-05-17T22:34:40.432989+0000","last_became_peered":"2024-05-17T22:34:40.432989+0000","last_unstale":"2024-05-17T22:37:14.519861+0000","last_undegraded":"2024-05-17T22:37:14.519861+0000","last_fullsized":"2024-05-17T22:37:14.519861+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:22:49.146148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004235+0000","last_change":"2024-05-17T22:34:52.494418+0000","last_active":"2024-05-17T22:36:16.004235+0000","last_peered":"2024-05-17T22:36:16.004235+0000","last_clean":"2024-05-17T22:36:16.004235+0000","last_became_active":"2024-05-17T22:34:52.493953+0000","last_became_peered":"2024-05-17T22:34:52.493953+0000","last_unstale":"2024-05-17T22:36:16.004235+0000","last_undegraded":"2024-05-17T22:36:16.004235+0000","last_fullsized":"2024-05-17T22:36:16.004235+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:50:40.211770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004010+0000","last_change":"2024-05-17T22:34:37.442986+0000","last_active":"2024-05-17T22:36:16.004010+0000","last_peered":"2024-05-17T22:36:16.004010+0000","last_clean":"2024-05-17T22:36:16.004010+0000","last_became_active":"2024-05-17T22:34:37.442399+0000","last_became_peered":"2024-05-17T22:34:37.442399+0000","last_unstale":"2024-05-17T22:36:16.004010+0000","last_undegraded":"2024-05-17T22:36:16.004010+0000","last_fullsized":"2024-05-17T22:36:16.004010+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:34:05.785811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.525214+0000","last_change":"2024-05-17T22:34:40.426793+0000","last_active":"2024-05-17T22:37:14.525214+0000","last_peered":"2024-05-17T22:37:14.525214+0000","last_clean":"2024-05-17T22:37:14.525214+0000","last_became_active":"2024-05-17T22:34:40.426420+0000","last_became_peered":"2024-05-17T22:34:40.426420+0000","last_unstale":"2024-05-17T22:37:14.525214+0000","last_undegraded":"2024-05-17T22:37:14.525214+0000","last_fullsized":"2024-05-17T22:37:14.525214+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:57:07.099254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.622953+0000","last_change":"2024-05-17T22:34:52.498762+0000","last_active":"2024-05-17T22:36:15.622953+0000","last_peered":"2024-05-17T22:36:15.622953+0000","last_clean":"2024-05-17T22:36:15.622953+0000","last_became_active":"2024-05-17T22:34:52.498306+0000","last_became_peered":"2024-05-17T22:34:52.498306+0000","last_unstale":"2024-05-17T22:36:15.622953+0000","last_undegraded":"2024-05-17T22:36:15.622953+0000","last_fullsized":"2024-05-17T22:36:15.622953+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:18:32.717563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'39","reported_seq":476,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:38:59.343737+0000","last_change":"2024-05-17T22:34:37.407885+0000","last_active":"2024-05-17T22:38:59.343737+0000","last_peered":"2024-05-17T22:38:59.343737+0000","last_clean":"2024-05-17T22:38:59.343737+0000","last_became_active":"2024-05-17T22:34:37.407463+0000","last_became_peered":"2024-05-17T22:34:37.407463+0000","last_unstale":"2024-05-17T22:38:59.343737+0000","last_undegraded":"2024-05-17T22:38:59.343737+0000","last_fullsized":"2024-05-17T22:38:59.343737+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:26:20.288557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68002,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":392,"num_read_kb":263,"num_write":40,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.501714+0000","last_change":"2024-05-17T22:34:40.435800+0000","last_active":"2024-05-17T22:37:14.501714+0000","last_peered":"2024-05-17T22:37:14.501714+0000","last_clean":"2024-05-17T22:37:14.501714+0000","last_became_active":"2024-05-17T22:34:40.435245+0000","last_became_peered":"2024-05-17T22:34:40.435245+0000","last_unstale":"2024-05-17T22:37:14.501714+0000","last_undegraded":"2024-05-17T22:37:14.501714+0000","last_fullsized":"2024-05-17T22:37:14.501714+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:09:58.699026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004448+0000","last_change":"2024-05-17T22:34:52.487336+0000","last_active":"2024-05-17T22:36:16.004448+0000","last_peered":"2024-05-17T22:36:16.004448+0000","last_clean":"2024-05-17T22:36:16.004448+0000","last_became_active":"2024-05-17T22:34:52.486883+0000","last_became_peered":"2024-05-17T22:34:52.486883+0000","last_unstale":"2024-05-17T22:36:16.004448+0000","last_undegraded":"2024-05-17T22:36:16.004448+0000","last_fullsized":"2024-05-17T22:36:16.004448+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:44:41.264072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624096+0000","last_change":"2024-05-17T22:34:37.405044+0000","last_active":"2024-05-17T22:36:15.624096+0000","last_peered":"2024-05-17T22:36:15.624096+0000","last_clean":"2024-05-17T22:36:15.624096+0000","last_became_active":"2024-05-17T22:34:37.404610+0000","last_became_peered":"2024-05-17T22:34:37.404610+0000","last_unstale":"2024-05-17T22:36:15.624096+0000","last_undegraded":"2024-05-17T22:36:15.624096+0000","last_fullsized":"2024-05-17T22:36:15.624096+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:20:37.939069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.521845+0000","last_change":"2024-05-17T22:34:40.430655+0000","last_active":"2024-05-17T22:37:14.521845+0000","last_peered":"2024-05-17T22:37:14.521845+0000","last_clean":"2024-05-17T22:37:14.521845+0000","last_became_active":"2024-05-17T22:34:40.430307+0000","last_became_peered":"2024-05-17T22:34:40.430307+0000","last_unstale":"2024-05-17T22:37:14.521845+0000","last_undegraded":"2024-05-17T22:37:14.521845+0000","last_fullsized":"2024-05-17T22:37:14.521845+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:57:19.036117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628175+0000","last_change":"2024-05-17T22:34:52.494709+0000","last_active":"2024-05-17T22:36:15.628175+0000","last_peered":"2024-05-17T22:36:15.628175+0000","last_clean":"2024-05-17T22:36:15.628175+0000","last_became_active":"2024-05-17T22:34:52.494195+0000","last_became_peered":"2024-05-17T22:34:52.494195+0000","last_unstale":"2024-05-17T22:36:15.628175+0000","last_undegraded":"2024-05-17T22:36:15.628175+0000","last_fullsized":"2024-05-17T22:36:15.628175+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:35:41.188459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003957+0000","last_change":"2024-05-17T22:34:37.402494+0000","last_active":"2024-05-17T22:36:16.003957+0000","last_peered":"2024-05-17T22:36:16.003957+0000","last_clean":"2024-05-17T22:36:16.003957+0000","last_became_active":"2024-05-17T22:34:37.402102+0000","last_became_peered":"2024-05-17T22:34:37.402102+0000","last_unstale":"2024-05-17T22:36:16.003957+0000","last_undegraded":"2024-05-17T22:36:16.003957+0000","last_fullsized":"2024-05-17T22:36:16.003957+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:22:30.915780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.522351+0000","last_change":"2024-05-17T22:34:40.440731+0000","last_active":"2024-05-17T22:37:14.522351+0000","last_peered":"2024-05-17T22:37:14.522351+0000","last_clean":"2024-05-17T22:37:14.522351+0000","last_became_active":"2024-05-17T22:34:40.440374+0000","last_became_peered":"2024-05-17T22:34:40.440374+0000","last_unstale":"2024-05-17T22:37:14.522351+0000","last_undegraded":"2024-05-17T22:37:14.522351+0000","last_fullsized":"2024-05-17T22:37:14.522351+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:26:40.029155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.006411+0000","last_change":"2024-05-17T22:36:16.005856+0000","last_active":"2024-05-17T22:36:16.006411+0000","last_peered":"2024-05-17T22:36:16.006411+0000","last_clean":"2024-05-17T22:36:16.006411+0000","last_became_active":"2024-05-17T22:36:16.005303+0000","last_became_peered":"2024-05-17T22:36:16.005303+0000","last_unstale":"2024-05-17T22:36:16.006411+0000","last_undegraded":"2024-05-17T22:36:16.006411+0000","last_fullsized":"2024-05-17T22:36:16.006411+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:41:15.708879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":241,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:38:56.372568+0000","last_change":"2024-05-17T22:34:37.403123+0000","last_active":"2024-05-17T22:38:56.372568+0000","last_peered":"2024-05-17T22:38:56.372568+0000","last_clean":"2024-05-17T22:38:56.372568+0000","last_became_active":"2024-05-17T22:34:37.402702+0000","last_became_peered":"2024-05-17T22:34:37.402702+0000","last_unstale":"2024-05-17T22:38:56.372568+0000","last_undegraded":"2024-05-17T22:38:56.372568+0000","last_fullsized":"2024-05-17T22:38:56.372568+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:50:02.031651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":193,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.512621+0000","last_change":"2024-05-17T22:34:40.431895+0000","last_active":"2024-05-17T22:37:14.512621+0000","last_peered":"2024-05-17T22:37:14.512621+0000","last_clean":"2024-05-17T22:37:14.512621+0000","last_became_active":"2024-05-17T22:34:40.431527+0000","last_became_peered":"2024-05-17T22:34:40.431527+0000","last_unstale":"2024-05-17T22:37:14.512621+0000","last_undegraded":"2024-05-17T22:37:14.512621+0000","last_fullsized":"2024-05-17T22:37:14.512621+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:30:09.404255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.686140+0000","last_change":"2024-05-17T22:34:52.495583+0000","last_active":"2024-05-17T22:36:15.686140+0000","last_peered":"2024-05-17T22:36:15.686140+0000","last_clean":"2024-05-17T22:36:15.686140+0000","last_became_active":"2024-05-17T22:34:52.495030+0000","last_became_peered":"2024-05-17T22:34:52.495030+0000","last_unstale":"2024-05-17T22:36:15.686140+0000","last_undegraded":"2024-05-17T22:36:15.686140+0000","last_fullsized":"2024-05-17T22:36:15.686140+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:04:54.046891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":61,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625592+0000","last_change":"2024-05-17T22:34:37.442154+0000","last_active":"2024-05-17T22:36:15.625592+0000","last_peered":"2024-05-17T22:36:15.625592+0000","last_clean":"2024-05-17T22:36:15.625592+0000","last_became_active":"2024-05-17T22:34:37.441691+0000","last_became_peered":"2024-05-17T22:34:37.441691+0000","last_unstale":"2024-05-17T22:36:15.625592+0000","last_undegraded":"2024-05-17T22:36:15.625592+0000","last_fullsized":"2024-05-17T22:36:15.625592+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:47:29.862031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.517818+0000","last_change":"2024-05-17T22:34:40.430894+0000","last_active":"2024-05-17T22:37:14.517818+0000","last_peered":"2024-05-17T22:37:14.517818+0000","last_clean":"2024-05-17T22:37:14.517818+0000","last_became_active":"2024-05-17T22:34:40.430328+0000","last_became_peered":"2024-05-17T22:34:40.430328+0000","last_unstale":"2024-05-17T22:37:14.517818+0000","last_undegraded":"2024-05-17T22:37:14.517818+0000","last_fullsized":"2024-05-17T22:37:14.517818+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:36:46.619111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003488+0000","last_change":"2024-05-17T22:34:52.487555+0000","last_active":"2024-05-17T22:36:16.003488+0000","last_peered":"2024-05-17T22:36:16.003488+0000","last_clean":"2024-05-17T22:36:16.003488+0000","last_became_active":"2024-05-17T22:34:52.487076+0000","last_became_peered":"2024-05-17T22:34:52.487076+0000","last_unstale":"2024-05-17T22:36:16.003488+0000","last_undegraded":"2024-05-17T22:36:16.003488+0000","last_fullsized":"2024-05-17T22:36:16.003488+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:05:46.720665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628417+0000","last_change":"2024-05-17T22:35:15.558860+0000","last_active":"2024-05-17T22:36:15.628417+0000","last_peered":"2024-05-17T22:36:15.628417+0000","last_clean":"2024-05-17T22:36:15.628417+0000","last_became_active":"2024-05-17T22:35:15.553683+0000","last_became_peered":"2024-05-17T22:35:15.553683+0000","last_unstale":"2024-05-17T22:36:15.628417+0000","last_undegraded":"2024-05-17T22:36:15.628417+0000","last_fullsized":"2024-05-17T22:36:15.628417+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:27:24.978282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.506967+0000","last_change":"2024-05-17T22:34:40.433499+0000","last_active":"2024-05-17T22:37:14.506967+0000","last_peered":"2024-05-17T22:37:14.506967+0000","last_clean":"2024-05-17T22:37:14.506967+0000","last_became_active":"2024-05-17T22:34:40.431370+0000","last_became_peered":"2024-05-17T22:34:40.431370+0000","last_unstale":"2024-05-17T22:37:14.506967+0000","last_undegraded":"2024-05-17T22:37:14.506967+0000","last_fullsized":"2024-05-17T22:37:14.506967+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:25:08.383021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625953+0000","last_change":"2024-05-17T22:34:52.489627+0000","last_active":"2024-05-17T22:36:15.625953+0000","last_peered":"2024-05-17T22:36:15.625953+0000","last_clean":"2024-05-17T22:36:15.625953+0000","last_became_active":"2024-05-17T22:34:52.489173+0000","last_became_peered":"2024-05-17T22:34:52.489173+0000","last_unstale":"2024-05-17T22:36:15.625953+0000","last_undegraded":"2024-05-17T22:36:15.625953+0000","last_fullsized":"2024-05-17T22:36:15.625953+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:52:26.908817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:43.067577+0000","last_change":"2024-05-17T22:35:15.553325+0000","last_active":"2024-05-17T22:36:43.067577+0000","last_peered":"2024-05-17T22:36:43.067577+0000","last_clean":"2024-05-17T22:36:43.067577+0000","last_became_active":"2024-05-17T22:35:15.552867+0000","last_became_peered":"2024-05-17T22:35:15.552867+0000","last_unstale":"2024-05-17T22:36:43.067577+0000","last_undegraded":"2024-05-17T22:36:43.067577+0000","last_fullsized":"2024-05-17T22:36:43.067577+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:28:36.618331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.526055+0000","last_change":"2024-05-17T22:34:40.423221+0000","last_active":"2024-05-17T22:37:14.526055+0000","last_peered":"2024-05-17T22:37:14.526055+0000","last_clean":"2024-05-17T22:37:14.526055+0000","last_became_active":"2024-05-17T22:34:40.422838+0000","last_became_peered":"2024-05-17T22:34:40.422838+0000","last_unstale":"2024-05-17T22:37:14.526055+0000","last_undegraded":"2024-05-17T22:37:14.526055+0000","last_fullsized":"2024-05-17T22:37:14.526055+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:39:51.873212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685905+0000","last_change":"2024-05-17T22:34:52.484802+0000","last_active":"2024-05-17T22:36:15.685905+0000","last_peered":"2024-05-17T22:36:15.685905+0000","last_clean":"2024-05-17T22:36:15.685905+0000","last_became_active":"2024-05-17T22:34:52.483766+0000","last_became_peered":"2024-05-17T22:34:52.483766+0000","last_unstale":"2024-05-17T22:36:15.685905+0000","last_undegraded":"2024-05-17T22:36:15.685905+0000","last_fullsized":"2024-05-17T22:36:15.685905+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:32:01.427078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:43.941225+0000","last_change":"2024-05-17T22:34:37.404797+0000","last_active":"2024-05-17T22:36:43.941225+0000","last_peered":"2024-05-17T22:36:43.941225+0000","last_clean":"2024-05-17T22:36:43.941225+0000","last_became_active":"2024-05-17T22:34:37.404393+0000","last_became_peered":"2024-05-17T22:34:37.404393+0000","last_unstale":"2024-05-17T22:36:43.941225+0000","last_undegraded":"2024-05-17T22:36:43.941225+0000","last_fullsized":"2024-05-17T22:36:43.941225+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:26:36.887919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.515960+0000","last_change":"2024-05-17T22:34:40.428210+0000","last_active":"2024-05-17T22:37:14.515960+0000","last_peered":"2024-05-17T22:37:14.515960+0000","last_clean":"2024-05-17T22:37:14.515960+0000","last_became_active":"2024-05-17T22:34:40.427845+0000","last_became_peered":"2024-05-17T22:34:40.427845+0000","last_unstale":"2024-05-17T22:37:14.515960+0000","last_undegraded":"2024-05-17T22:37:14.515960+0000","last_fullsized":"2024-05-17T22:37:14.515960+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:45:30.437139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.686040+0000","last_change":"2024-05-17T22:34:52.490502+0000","last_active":"2024-05-17T22:36:15.686040+0000","last_peered":"2024-05-17T22:36:15.686040+0000","last_clean":"2024-05-17T22:36:15.686040+0000","last_became_active":"2024-05-17T22:34:52.490008+0000","last_became_peered":"2024-05-17T22:34:52.490008+0000","last_unstale":"2024-05-17T22:36:15.686040+0000","last_undegraded":"2024-05-17T22:36:15.686040+0000","last_fullsized":"2024-05-17T22:36:15.686040+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:58:29.122668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.518044+0000","last_change":"2024-05-17T22:34:40.432248+0000","last_active":"2024-05-17T22:37:14.518044+0000","last_peered":"2024-05-17T22:37:14.518044+0000","last_clean":"2024-05-17T22:37:14.518044+0000","last_became_active":"2024-05-17T22:34:40.431863+0000","last_became_peered":"2024-05-17T22:34:40.431863+0000","last_unstale":"2024-05-17T22:37:14.518044+0000","last_undegraded":"2024-05-17T22:37:14.518044+0000","last_fullsized":"2024-05-17T22:37:14.518044+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:20:34.730542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":240,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:38:57.373331+0000","last_change":"2024-05-17T22:34:37.441186+0000","last_active":"2024-05-17T22:38:57.373331+0000","last_peered":"2024-05-17T22:38:57.373331+0000","last_clean":"2024-05-17T22:38:57.373331+0000","last_became_active":"2024-05-17T22:34:37.440781+0000","last_became_peered":"2024-05-17T22:34:37.440781+0000","last_unstale":"2024-05-17T22:38:57.373331+0000","last_undegraded":"2024-05-17T22:38:57.373331+0000","last_fullsized":"2024-05-17T22:38:57.373331+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:44:34.435147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":189,"num_read_kb":189,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.005134+0000","last_change":"2024-05-17T22:34:52.491771+0000","last_active":"2024-05-17T22:36:16.005134+0000","last_peered":"2024-05-17T22:36:16.005134+0000","last_clean":"2024-05-17T22:36:16.005134+0000","last_became_active":"2024-05-17T22:34:52.491172+0000","last_became_peered":"2024-05-17T22:34:52.491172+0000","last_unstale":"2024-05-17T22:36:16.005134+0000","last_undegraded":"2024-05-17T22:36:16.005134+0000","last_fullsized":"2024-05-17T22:36:16.005134+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:15:21.010592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.514209+0000","last_change":"2024-05-17T22:34:40.430860+0000","last_active":"2024-05-17T22:37:14.514209+0000","last_peered":"2024-05-17T22:37:14.514209+0000","last_clean":"2024-05-17T22:37:14.514209+0000","last_became_active":"2024-05-17T22:34:40.430542+0000","last_became_peered":"2024-05-17T22:34:40.430542+0000","last_unstale":"2024-05-17T22:37:14.514209+0000","last_undegraded":"2024-05-17T22:37:14.514209+0000","last_fullsized":"2024-05-17T22:37:14.514209+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:16:26.257380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:42.729213+0000","last_change":"2024-05-17T22:34:37.403026+0000","last_active":"2024-05-17T22:36:42.729213+0000","last_peered":"2024-05-17T22:36:42.729213+0000","last_clean":"2024-05-17T22:36:42.729213+0000","last_became_active":"2024-05-17T22:34:37.402560+0000","last_became_peered":"2024-05-17T22:34:37.402560+0000","last_unstale":"2024-05-17T22:36:42.729213+0000","last_undegraded":"2024-05-17T22:36:42.729213+0000","last_fullsized":"2024-05-17T22:36:42.729213+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:23:03.784411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":120,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003564+0000","last_change":"2024-05-17T22:33:11.055878+0000","last_active":"2024-05-17T22:36:16.003564+0000","last_peered":"2024-05-17T22:36:16.003564+0000","last_clean":"2024-05-17T22:36:16.003564+0000","last_became_active":"2024-05-17T22:33:11.050409+0000","last_became_peered":"2024-05-17T22:33:11.050409+0000","last_unstale":"2024-05-17T22:36:16.003564+0000","last_undegraded":"2024-05-17T22:36:16.003564+0000","last_fullsized":"2024-05-17T22:36:16.003564+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:29:43.341170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:29:43.341170+0000","last_clean_scrub_stamp":"2024-05-17T22:29:43.341170+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:30:54.153865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003960+0000","last_change":"2024-05-17T22:34:52.497064+0000","last_active":"2024-05-17T22:36:16.003960+0000","last_peered":"2024-05-17T22:36:16.003960+0000","last_clean":"2024-05-17T22:36:16.003960+0000","last_became_active":"2024-05-17T22:34:52.496611+0000","last_became_peered":"2024-05-17T22:34:52.496611+0000","last_unstale":"2024-05-17T22:36:16.003960+0000","last_undegraded":"2024-05-17T22:36:16.003960+0000","last_fullsized":"2024-05-17T22:36:16.003960+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:20:29.597589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.519893+0000","last_change":"2024-05-17T22:35:15.555413+0000","last_active":"2024-05-17T22:37:14.519893+0000","last_peered":"2024-05-17T22:37:14.519893+0000","last_clean":"2024-05-17T22:37:14.519893+0000","last_became_active":"2024-05-17T22:35:15.555005+0000","last_became_peered":"2024-05-17T22:35:15.555005+0000","last_unstale":"2024-05-17T22:37:14.519893+0000","last_undegraded":"2024-05-17T22:37:14.519893+0000","last_fullsized":"2024-05-17T22:37:14.519893+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:51:49.860673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:38.940738+0000","last_change":"2024-05-17T22:34:37.406546+0000","last_active":"2024-05-17T22:36:38.940738+0000","last_peered":"2024-05-17T22:36:38.940738+0000","last_clean":"2024-05-17T22:36:38.940738+0000","last_became_active":"2024-05-17T22:34:37.406118+0000","last_became_peered":"2024-05-17T22:34:37.406118+0000","last_unstale":"2024-05-17T22:36:38.940738+0000","last_undegraded":"2024-05-17T22:36:38.940738+0000","last_fullsized":"2024-05-17T22:36:38.940738+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:47:16.350091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.006276+0000","last_change":"2024-05-17T22:36:16.005647+0000","last_active":"2024-05-17T22:36:16.006276+0000","last_peered":"2024-05-17T22:36:16.006276+0000","last_clean":"2024-05-17T22:36:16.006276+0000","last_became_active":"2024-05-17T22:36:16.005128+0000","last_became_peered":"2024-05-17T22:36:16.005128+0000","last_unstale":"2024-05-17T22:36:16.006276+0000","last_undegraded":"2024-05-17T22:36:16.006276+0000","last_fullsized":"2024-05-17T22:36:16.006276+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:16:24.481236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,7,6],"acting":[11,10,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:45.570153+0000","last_change":"2024-05-17T22:34:37.442676+0000","last_active":"2024-05-17T22:36:45.570153+0000","last_peered":"2024-05-17T22:36:45.570153+0000","last_clean":"2024-05-17T22:36:45.570153+0000","last_became_active":"2024-05-17T22:34:37.442275+0000","last_became_peered":"2024-05-17T22:34:37.442275+0000","last_unstale":"2024-05-17T22:36:45.570153+0000","last_undegraded":"2024-05-17T22:36:45.570153+0000","last_fullsized":"2024-05-17T22:36:45.570153+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:54:18.542987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.519824+0000","last_change":"2024-05-17T22:34:40.424266+0000","last_active":"2024-05-17T22:37:14.519824+0000","last_peered":"2024-05-17T22:37:14.519824+0000","last_clean":"2024-05-17T22:37:14.519824+0000","last_became_active":"2024-05-17T22:34:40.423905+0000","last_became_peered":"2024-05-17T22:34:40.423905+0000","last_unstale":"2024-05-17T22:37:14.519824+0000","last_undegraded":"2024-05-17T22:37:14.519824+0000","last_fullsized":"2024-05-17T22:37:14.519824+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:03:43.397871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:52.770677+0000","last_change":"2024-05-17T22:34:52.493386+0000","last_active":"2024-05-17T22:36:52.770677+0000","last_peered":"2024-05-17T22:36:52.770677+0000","last_clean":"2024-05-17T22:36:52.770677+0000","last_became_active":"2024-05-17T22:34:52.492997+0000","last_became_peered":"2024-05-17T22:34:52.492997+0000","last_unstale":"2024-05-17T22:36:52.770677+0000","last_undegraded":"2024-05-17T22:36:52.770677+0000","last_fullsized":"2024-05-17T22:36:52.770677+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:14:40.818271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'13","reported_seq":59,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:38:55.469975+0000","last_change":"2024-05-17T22:34:37.399397+0000","last_active":"2024-05-17T22:38:55.469975+0000","last_peered":"2024-05-17T22:38:55.469975+0000","last_clean":"2024-05-17T22:38:55.469975+0000","last_became_active":"2024-05-17T22:34:37.399062+0000","last_became_peered":"2024-05-17T22:34:37.399062+0000","last_unstale":"2024-05-17T22:38:55.469975+0000","last_undegraded":"2024-05-17T22:38:55.469975+0000","last_fullsized":"2024-05-17T22:38:55.469975+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:02:05.230580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.518907+0000","last_change":"2024-05-17T22:34:40.426995+0000","last_active":"2024-05-17T22:37:14.518907+0000","last_peered":"2024-05-17T22:37:14.518907+0000","last_clean":"2024-05-17T22:37:14.518907+0000","last_became_active":"2024-05-17T22:34:40.426226+0000","last_became_peered":"2024-05-17T22:34:40.426226+0000","last_unstale":"2024-05-17T22:37:14.518907+0000","last_undegraded":"2024-05-17T22:37:14.518907+0000","last_fullsized":"2024-05-17T22:37:14.518907+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:49:53.331076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623704+0000","last_change":"2024-05-17T22:34:52.487098+0000","last_active":"2024-05-17T22:36:15.623704+0000","last_peered":"2024-05-17T22:36:15.623704+0000","last_clean":"2024-05-17T22:36:15.623704+0000","last_became_active":"2024-05-17T22:34:52.486605+0000","last_became_peered":"2024-05-17T22:34:52.486605+0000","last_unstale":"2024-05-17T22:36:15.623704+0000","last_undegraded":"2024-05-17T22:36:15.623704+0000","last_fullsized":"2024-05-17T22:36:15.623704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:33:28.234208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:47.729708+0000","last_change":"2024-05-17T22:34:37.800830+0000","last_active":"2024-05-17T22:36:47.729708+0000","last_peered":"2024-05-17T22:36:47.729708+0000","last_clean":"2024-05-17T22:36:47.729708+0000","last_became_active":"2024-05-17T22:34:37.800405+0000","last_became_peered":"2024-05-17T22:34:37.800405+0000","last_unstale":"2024-05-17T22:36:47.729708+0000","last_undegraded":"2024-05-17T22:36:47.729708+0000","last_fullsized":"2024-05-17T22:36:47.729708+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:19:10.087442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.520738+0000","last_change":"2024-05-17T22:34:40.426307+0000","last_active":"2024-05-17T22:37:14.520738+0000","last_peered":"2024-05-17T22:37:14.520738+0000","last_clean":"2024-05-17T22:37:14.520738+0000","last_became_active":"2024-05-17T22:34:40.425969+0000","last_became_peered":"2024-05-17T22:34:40.425969+0000","last_unstale":"2024-05-17T22:37:14.520738+0000","last_undegraded":"2024-05-17T22:37:14.520738+0000","last_fullsized":"2024-05-17T22:37:14.520738+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:25:49.680886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004511+0000","last_change":"2024-05-17T22:35:15.556857+0000","last_active":"2024-05-17T22:36:16.004511+0000","last_peered":"2024-05-17T22:36:16.004511+0000","last_clean":"2024-05-17T22:36:16.004511+0000","last_became_active":"2024-05-17T22:35:15.556298+0000","last_became_peered":"2024-05-17T22:35:15.556298+0000","last_unstale":"2024-05-17T22:36:16.004511+0000","last_undegraded":"2024-05-17T22:36:16.004511+0000","last_fullsized":"2024-05-17T22:36:16.004511+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:23:20.111544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:47.852816+0000","last_change":"2024-05-17T22:34:37.442971+0000","last_active":"2024-05-17T22:36:47.852816+0000","last_peered":"2024-05-17T22:36:47.852816+0000","last_clean":"2024-05-17T22:36:47.852816+0000","last_became_active":"2024-05-17T22:34:37.442536+0000","last_became_peered":"2024-05-17T22:34:37.442536+0000","last_unstale":"2024-05-17T22:36:47.852816+0000","last_undegraded":"2024-05-17T22:36:47.852816+0000","last_fullsized":"2024-05-17T22:36:47.852816+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:03:45.224071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.467874+0000","last_change":"2024-05-17T22:34:40.432056+0000","last_active":"2024-05-17T22:37:14.467874+0000","last_peered":"2024-05-17T22:37:14.467874+0000","last_clean":"2024-05-17T22:37:14.467874+0000","last_became_active":"2024-05-17T22:34:40.431626+0000","last_became_peered":"2024-05-17T22:34:40.431626+0000","last_unstale":"2024-05-17T22:37:14.467874+0000","last_undegraded":"2024-05-17T22:37:14.467874+0000","last_fullsized":"2024-05-17T22:37:14.467874+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:12:24.650746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623664+0000","last_change":"2024-05-17T22:34:52.500451+0000","last_active":"2024-05-17T22:36:15.623664+0000","last_peered":"2024-05-17T22:36:15.623664+0000","last_clean":"2024-05-17T22:36:15.623664+0000","last_became_active":"2024-05-17T22:34:52.500031+0000","last_became_peered":"2024-05-17T22:34:52.500031+0000","last_unstale":"2024-05-17T22:36:15.623664+0000","last_undegraded":"2024-05-17T22:36:15.623664+0000","last_fullsized":"2024-05-17T22:36:15.623664+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:54:36.825116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:47.770054+0000","last_change":"2024-05-17T22:34:37.402335+0000","last_active":"2024-05-17T22:36:47.770054+0000","last_peered":"2024-05-17T22:36:47.770054+0000","last_clean":"2024-05-17T22:36:47.770054+0000","last_became_active":"2024-05-17T22:34:37.401982+0000","last_became_peered":"2024-05-17T22:34:37.401982+0000","last_unstale":"2024-05-17T22:36:47.770054+0000","last_undegraded":"2024-05-17T22:36:47.770054+0000","last_fullsized":"2024-05-17T22:36:47.770054+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:58:06.194335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.517775+0000","last_change":"2024-05-17T22:34:40.425524+0000","last_active":"2024-05-17T22:37:14.517775+0000","last_peered":"2024-05-17T22:37:14.517775+0000","last_clean":"2024-05-17T22:37:14.517775+0000","last_became_active":"2024-05-17T22:34:40.425135+0000","last_became_peered":"2024-05-17T22:34:40.425135+0000","last_unstale":"2024-05-17T22:37:14.517775+0000","last_undegraded":"2024-05-17T22:37:14.517775+0000","last_fullsized":"2024-05-17T22:37:14.517775+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:41:14.176477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004843+0000","last_change":"2024-05-17T22:34:52.496290+0000","last_active":"2024-05-17T22:36:16.004843+0000","last_peered":"2024-05-17T22:36:16.004843+0000","last_clean":"2024-05-17T22:36:16.004843+0000","last_became_active":"2024-05-17T22:34:52.495817+0000","last_became_peered":"2024-05-17T22:34:52.495817+0000","last_unstale":"2024-05-17T22:36:16.004843+0000","last_undegraded":"2024-05-17T22:36:16.004843+0000","last_fullsized":"2024-05-17T22:36:16.004843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:47:22.056372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'1","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:47.385104+0000","last_change":"2024-05-17T22:34:37.404234+0000","last_active":"2024-05-17T22:36:47.385104+0000","last_peered":"2024-05-17T22:36:47.385104+0000","last_clean":"2024-05-17T22:36:47.385104+0000","last_became_active":"2024-05-17T22:34:37.403561+0000","last_became_peered":"2024-05-17T22:34:37.403561+0000","last_unstale":"2024-05-17T22:36:47.385104+0000","last_undegraded":"2024-05-17T22:36:47.385104+0000","last_fullsized":"2024-05-17T22:36:47.385104+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:30:28.432641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.517802+0000","last_change":"2024-05-17T22:34:40.426522+0000","last_active":"2024-05-17T22:37:14.517802+0000","last_peered":"2024-05-17T22:37:14.517802+0000","last_clean":"2024-05-17T22:37:14.517802+0000","last_became_active":"2024-05-17T22:34:40.426139+0000","last_became_peered":"2024-05-17T22:34:40.426139+0000","last_unstale":"2024-05-17T22:37:14.517802+0000","last_undegraded":"2024-05-17T22:37:14.517802+0000","last_fullsized":"2024-05-17T22:37:14.517802+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:30:19.503513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004076+0000","last_change":"2024-05-17T22:34:52.487131+0000","last_active":"2024-05-17T22:36:16.004076+0000","last_peered":"2024-05-17T22:36:16.004076+0000","last_clean":"2024-05-17T22:36:16.004076+0000","last_became_active":"2024-05-17T22:34:52.486654+0000","last_became_peered":"2024-05-17T22:34:52.486654+0000","last_unstale":"2024-05-17T22:36:16.004076+0000","last_undegraded":"2024-05-17T22:36:16.004076+0000","last_fullsized":"2024-05-17T22:36:16.004076+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:28:25.041746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625893+0000","last_change":"2024-05-17T22:34:37.405020+0000","last_active":"2024-05-17T22:36:15.625893+0000","last_peered":"2024-05-17T22:36:15.625893+0000","last_clean":"2024-05-17T22:36:15.625893+0000","last_became_active":"2024-05-17T22:34:37.404633+0000","last_became_peered":"2024-05-17T22:34:37.404633+0000","last_unstale":"2024-05-17T22:36:15.625893+0000","last_undegraded":"2024-05-17T22:36:15.625893+0000","last_fullsized":"2024-05-17T22:36:15.625893+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:06:00.001190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.525054+0000","last_change":"2024-05-17T22:34:40.435374+0000","last_active":"2024-05-17T22:37:14.525054+0000","last_peered":"2024-05-17T22:37:14.525054+0000","last_clean":"2024-05-17T22:37:14.525054+0000","last_became_active":"2024-05-17T22:34:40.434522+0000","last_became_peered":"2024-05-17T22:34:40.434522+0000","last_unstale":"2024-05-17T22:37:14.525054+0000","last_undegraded":"2024-05-17T22:37:14.525054+0000","last_fullsized":"2024-05-17T22:37:14.525054+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:36:41.244970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.805274+0000","last_change":"2024-05-17T22:34:52.491708+0000","last_active":"2024-05-17T22:36:15.805274+0000","last_peered":"2024-05-17T22:36:15.805274+0000","last_clean":"2024-05-17T22:36:15.805274+0000","last_became_active":"2024-05-17T22:34:52.491293+0000","last_became_peered":"2024-05-17T22:34:52.491293+0000","last_unstale":"2024-05-17T22:36:15.805274+0000","last_undegraded":"2024-05-17T22:36:15.805274+0000","last_fullsized":"2024-05-17T22:36:15.805274+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:23:01.634026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624679+0000","last_change":"2024-05-17T22:34:37.800525+0000","last_active":"2024-05-17T22:36:15.624679+0000","last_peered":"2024-05-17T22:36:15.624679+0000","last_clean":"2024-05-17T22:36:15.624679+0000","last_became_active":"2024-05-17T22:34:37.800093+0000","last_became_peered":"2024-05-17T22:34:37.800093+0000","last_unstale":"2024-05-17T22:36:15.624679+0000","last_undegraded":"2024-05-17T22:36:15.624679+0000","last_fullsized":"2024-05-17T22:36:15.624679+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:25:27.249889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.524773+0000","last_change":"2024-05-17T22:34:40.434891+0000","last_active":"2024-05-17T22:37:14.524773+0000","last_peered":"2024-05-17T22:37:14.524773+0000","last_clean":"2024-05-17T22:37:14.524773+0000","last_became_active":"2024-05-17T22:34:40.434543+0000","last_became_peered":"2024-05-17T22:34:40.434543+0000","last_unstale":"2024-05-17T22:37:14.524773+0000","last_undegraded":"2024-05-17T22:37:14.524773+0000","last_fullsized":"2024-05-17T22:37:14.524773+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:17:47.770225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968412+0000","last_change":"2024-05-17T22:34:52.491544+0000","last_active":"2024-05-17T22:36:15.968412+0000","last_peered":"2024-05-17T22:36:15.968412+0000","last_clean":"2024-05-17T22:36:15.968412+0000","last_became_active":"2024-05-17T22:34:52.491138+0000","last_became_peered":"2024-05-17T22:34:52.491138+0000","last_unstale":"2024-05-17T22:36:15.968412+0000","last_undegraded":"2024-05-17T22:36:15.968412+0000","last_fullsized":"2024-05-17T22:36:15.968412+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:34:20.411299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004302+0000","last_change":"2024-05-17T22:34:37.403220+0000","last_active":"2024-05-17T22:36:16.004302+0000","last_peered":"2024-05-17T22:36:16.004302+0000","last_clean":"2024-05-17T22:36:16.004302+0000","last_became_active":"2024-05-17T22:34:37.402856+0000","last_became_peered":"2024-05-17T22:34:37.402856+0000","last_unstale":"2024-05-17T22:36:16.004302+0000","last_undegraded":"2024-05-17T22:36:16.004302+0000","last_fullsized":"2024-05-17T22:36:16.004302+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:04:19.469294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.511989+0000","last_change":"2024-05-17T22:34:40.434487+0000","last_active":"2024-05-17T22:37:14.511989+0000","last_peered":"2024-05-17T22:37:14.511989+0000","last_clean":"2024-05-17T22:37:14.511989+0000","last_became_active":"2024-05-17T22:34:40.434160+0000","last_became_peered":"2024-05-17T22:34:40.434160+0000","last_unstale":"2024-05-17T22:37:14.511989+0000","last_undegraded":"2024-05-17T22:37:14.511989+0000","last_fullsized":"2024-05-17T22:37:14.511989+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:24:38.782999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003180+0000","last_change":"2024-05-17T22:34:52.487887+0000","last_active":"2024-05-17T22:36:16.003180+0000","last_peered":"2024-05-17T22:36:16.003180+0000","last_clean":"2024-05-17T22:36:16.003180+0000","last_became_active":"2024-05-17T22:34:52.486837+0000","last_became_peered":"2024-05-17T22:34:52.486837+0000","last_unstale":"2024-05-17T22:36:16.003180+0000","last_undegraded":"2024-05-17T22:36:16.003180+0000","last_fullsized":"2024-05-17T22:36:16.003180+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:12:22.206536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.686208+0000","last_change":"2024-05-17T22:34:37.399498+0000","last_active":"2024-05-17T22:36:15.686208+0000","last_peered":"2024-05-17T22:36:15.686208+0000","last_clean":"2024-05-17T22:36:15.686208+0000","last_became_active":"2024-05-17T22:34:37.399144+0000","last_became_peered":"2024-05-17T22:34:37.399144+0000","last_unstale":"2024-05-17T22:36:15.686208+0000","last_undegraded":"2024-05-17T22:36:15.686208+0000","last_fullsized":"2024-05-17T22:36:15.686208+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:13:37.346006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.514102+0000","last_change":"2024-05-17T22:34:40.433839+0000","last_active":"2024-05-17T22:37:14.514102+0000","last_peered":"2024-05-17T22:37:14.514102+0000","last_clean":"2024-05-17T22:37:14.514102+0000","last_became_active":"2024-05-17T22:34:40.433489+0000","last_became_peered":"2024-05-17T22:34:40.433489+0000","last_unstale":"2024-05-17T22:37:14.514102+0000","last_undegraded":"2024-05-17T22:37:14.514102+0000","last_fullsized":"2024-05-17T22:37:14.514102+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:13:56.014094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.512654+0000","last_change":"2024-05-17T22:34:40.434953+0000","last_active":"2024-05-17T22:37:14.512654+0000","last_peered":"2024-05-17T22:37:14.512654+0000","last_clean":"2024-05-17T22:37:14.512654+0000","last_became_active":"2024-05-17T22:34:40.434593+0000","last_became_peered":"2024-05-17T22:34:40.434593+0000","last_unstale":"2024-05-17T22:37:14.512654+0000","last_undegraded":"2024-05-17T22:37:14.512654+0000","last_fullsized":"2024-05-17T22:37:14.512654+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:20:27.075382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.513666+0000","last_change":"2024-05-17T22:34:40.432787+0000","last_active":"2024-05-17T22:37:14.513666+0000","last_peered":"2024-05-17T22:37:14.513666+0000","last_clean":"2024-05-17T22:37:14.513666+0000","last_became_active":"2024-05-17T22:34:40.432402+0000","last_became_peered":"2024-05-17T22:34:40.432402+0000","last_unstale":"2024-05-17T22:37:14.513666+0000","last_undegraded":"2024-05-17T22:37:14.513666+0000","last_fullsized":"2024-05-17T22:37:14.513666+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:48:13.034276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.515714+0000","last_change":"2024-05-17T22:34:40.426423+0000","last_active":"2024-05-17T22:37:14.515714+0000","last_peered":"2024-05-17T22:37:14.515714+0000","last_clean":"2024-05-17T22:37:14.515714+0000","last_became_active":"2024-05-17T22:34:40.426049+0000","last_became_peered":"2024-05-17T22:34:40.426049+0000","last_unstale":"2024-05-17T22:37:14.515714+0000","last_undegraded":"2024-05-17T22:37:14.515714+0000","last_fullsized":"2024-05-17T22:37:14.515714+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:56:55.631322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.520890+0000","last_change":"2024-05-17T22:34:40.435706+0000","last_active":"2024-05-17T22:37:14.520890+0000","last_peered":"2024-05-17T22:37:14.520890+0000","last_clean":"2024-05-17T22:37:14.520890+0000","last_became_active":"2024-05-17T22:34:40.435330+0000","last_became_peered":"2024-05-17T22:34:40.435330+0000","last_unstale":"2024-05-17T22:37:14.520890+0000","last_undegraded":"2024-05-17T22:37:14.520890+0000","last_fullsized":"2024-05-17T22:37:14.520890+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:25:36.529454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.517285+0000","last_change":"2024-05-17T22:34:40.428997+0000","last_active":"2024-05-17T22:37:14.517285+0000","last_peered":"2024-05-17T22:37:14.517285+0000","last_clean":"2024-05-17T22:37:14.517285+0000","last_became_active":"2024-05-17T22:34:40.428594+0000","last_became_peered":"2024-05-17T22:34:40.428594+0000","last_unstale":"2024-05-17T22:37:14.517285+0000","last_undegraded":"2024-05-17T22:37:14.517285+0000","last_fullsized":"2024-05-17T22:37:14.517285+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:59:01.777751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.509268+0000","last_change":"2024-05-17T22:34:40.434241+0000","last_active":"2024-05-17T22:37:14.509268+0000","last_peered":"2024-05-17T22:37:14.509268+0000","last_clean":"2024-05-17T22:37:14.509268+0000","last_became_active":"2024-05-17T22:34:40.431652+0000","last_became_peered":"2024-05-17T22:34:40.431652+0000","last_unstale":"2024-05-17T22:37:14.509268+0000","last_undegraded":"2024-05-17T22:37:14.509268+0000","last_fullsized":"2024-05-17T22:37:14.509268+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:33:27.035284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.505573+0000","last_change":"2024-05-17T22:34:40.431485+0000","last_active":"2024-05-17T22:37:14.505573+0000","last_peered":"2024-05-17T22:37:14.505573+0000","last_clean":"2024-05-17T22:37:14.505573+0000","last_became_active":"2024-05-17T22:34:40.431120+0000","last_became_peered":"2024-05-17T22:34:40.431120+0000","last_unstale":"2024-05-17T22:37:14.505573+0000","last_undegraded":"2024-05-17T22:37:14.505573+0000","last_fullsized":"2024-05-17T22:37:14.505573+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:46:55.213602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.501567+0000","last_change":"2024-05-17T22:34:40.423117+0000","last_active":"2024-05-17T22:37:14.501567+0000","last_peered":"2024-05-17T22:37:14.501567+0000","last_clean":"2024-05-17T22:37:14.501567+0000","last_became_active":"2024-05-17T22:34:40.422694+0000","last_became_peered":"2024-05-17T22:34:40.422694+0000","last_unstale":"2024-05-17T22:37:14.501567+0000","last_undegraded":"2024-05-17T22:37:14.501567+0000","last_fullsized":"2024-05-17T22:37:14.501567+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:19:55.589532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.518389+0000","last_change":"2024-05-17T22:35:15.711796+0000","last_active":"2024-05-17T22:37:14.518389+0000","last_peered":"2024-05-17T22:37:14.518389+0000","last_clean":"2024-05-17T22:37:14.518389+0000","last_became_active":"2024-05-17T22:35:15.711327+0000","last_became_peered":"2024-05-17T22:35:15.711327+0000","last_unstale":"2024-05-17T22:37:14.518389+0000","last_undegraded":"2024-05-17T22:37:14.518389+0000","last_fullsized":"2024-05-17T22:37:14.518389+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:48:48.913312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.512607+0000","last_change":"2024-05-17T22:34:40.430393+0000","last_active":"2024-05-17T22:37:14.512607+0000","last_peered":"2024-05-17T22:37:14.512607+0000","last_clean":"2024-05-17T22:37:14.512607+0000","last_became_active":"2024-05-17T22:34:40.430030+0000","last_became_peered":"2024-05-17T22:34:40.430030+0000","last_unstale":"2024-05-17T22:37:14.512607+0000","last_undegraded":"2024-05-17T22:37:14.512607+0000","last_fullsized":"2024-05-17T22:37:14.512607+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:39:37.204246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.526052+0000","last_change":"2024-05-17T22:34:40.432443+0000","last_active":"2024-05-17T22:37:14.526052+0000","last_peered":"2024-05-17T22:37:14.526052+0000","last_clean":"2024-05-17T22:37:14.526052+0000","last_became_active":"2024-05-17T22:34:40.431991+0000","last_became_peered":"2024-05-17T22:34:40.431991+0000","last_unstale":"2024-05-17T22:37:14.526052+0000","last_undegraded":"2024-05-17T22:37:14.526052+0000","last_fullsized":"2024-05-17T22:37:14.526052+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:37:45.593575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.523169+0000","last_change":"2024-05-17T22:34:40.434388+0000","last_active":"2024-05-17T22:37:14.523169+0000","last_peered":"2024-05-17T22:37:14.523169+0000","last_clean":"2024-05-17T22:37:14.523169+0000","last_became_active":"2024-05-17T22:34:40.434044+0000","last_became_peered":"2024-05-17T22:34:40.434044+0000","last_unstale":"2024-05-17T22:37:14.523169+0000","last_undegraded":"2024-05-17T22:37:14.523169+0000","last_fullsized":"2024-05-17T22:37:14.523169+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:20:43.296190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.507137+0000","last_change":"2024-05-17T22:34:40.427137+0000","last_active":"2024-05-17T22:37:14.507137+0000","last_peered":"2024-05-17T22:37:14.507137+0000","last_clean":"2024-05-17T22:37:14.507137+0000","last_became_active":"2024-05-17T22:34:40.426703+0000","last_became_peered":"2024-05-17T22:34:40.426703+0000","last_unstale":"2024-05-17T22:37:14.507137+0000","last_undegraded":"2024-05-17T22:37:14.507137+0000","last_fullsized":"2024-05-17T22:37:14.507137+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:55:52.992817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72984,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1170,"num_read_kb":665,"num_write":185,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":434176,"data_stored":208206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710788,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":515036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907018,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28696,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716456,"statfs":{"total":95995035648,"available":95965650944,"internally_reserved":0,"allocated":1720320,"data_stored":1170816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070541,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":519346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266772,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":515148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863463001,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1634304,"data_stored":1112060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626525,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":510426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574968,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1089536,"data_stored":572965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738493,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":511814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411443,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1114112,"data_stored":575155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182510,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":510620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986280,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":515608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822756,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36940,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708212,"statfs":{"total":95995035648,"available":95957209088,"internally_reserved":0,"allocated":1642496,"data_stored":1107116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T22:39:02.268 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-05-17T22:39:02.268 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-17T22:39:02.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.415+0000 7f3195144640 1 -- 172.21.15.12:0/2579542250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3190102040 msgr2=0x7f3190102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:02.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.415+0000 7f3195144640 1 --2- 172.21.15.12:0/2579542250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3190102040 0x7f3190102450 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3180009900 tx=0x7f318002a410 comp rx=0 tx=0).stop 2024-05-17T22:39:02.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.415+0000 7f3195144640 1 -- 172.21.15.12:0/2579542250 shutdown_connections 2024-05-17T22:39:02.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.415+0000 7f3195144640 1 --2- 172.21.15.12:0/2579542250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3190103ab0 0x7f319010a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:02.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.415+0000 7f3195144640 1 --2- 172.21.15.12:0/2579542250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3190103160 0x7f3190103570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:02.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.415+0000 7f3195144640 1 --2- 172.21.15.12:0/2579542250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3190102040 0x7f3190102450 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:02.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.415+0000 7f3195144640 1 -- 172.21.15.12:0/2579542250 >> 172.21.15.12:0/2579542250 conn(0x7f31900fd860 msgr2=0x7f31900ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:39:02.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.415+0000 7f3195144640 1 -- 172.21.15.12:0/2579542250 shutdown_connections 2024-05-17T22:39:02.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.415+0000 7f3195144640 1 -- 172.21.15.12:0/2579542250 wait complete. 2024-05-17T22:39:02.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.416+0000 7f3195144640 1 Processor -- start 2024-05-17T22:39:02.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.416+0000 7f3195144640 1 -- start start 2024-05-17T22:39:02.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.416+0000 7f3195144640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3190102040 0x7f319006aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:02.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.416+0000 7f3195144640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3190103160 0x7f319019e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:02.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.416+0000 7f3195144640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3190103ab0 0x7f31901a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:02.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.416+0000 7f3195144640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f319010db70 con 0x7f3190103160 2024-05-17T22:39:02.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.416+0000 7f3195144640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f319010d9f0 con 0x7f3190102040 2024-05-17T22:39:02.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.416+0000 7f3195144640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f319010dcf0 con 0x7f3190103ab0 2024-05-17T22:39:02.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.416+0000 7f318e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3190103160 0x7f319019e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:02.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.416+0000 7f318e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3190103160 0x7f319019e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60988/0 (socket says 172.21.15.12:60988) 2024-05-17T22:39:02.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.416+0000 7f318e59c640 1 -- 172.21.15.12:0/3993647611 learned_addr learned my addr 172.21.15.12:0/3993647611 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:39:02.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.416+0000 7f318ed9d640 1 --2- 172.21.15.12:0/3993647611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3190102040 0x7f319006aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:02.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.417+0000 7f318e59c640 1 -- 172.21.15.12:0/3993647611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3190103ab0 msgr2=0x7f31901a0ba0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-17T22:39:02.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.417+0000 7f318e59c640 1 --2- 172.21.15.12:0/3993647611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3190103ab0 0x7f31901a0ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:02.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.417+0000 7f318e59c640 1 -- 172.21.15.12:0/3993647611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3190102040 msgr2=0x7f319006aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:02.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.417+0000 7f318e59c640 1 --2- 172.21.15.12:0/3993647611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3190102040 0x7f319006aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:02.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.417+0000 7f318e59c640 1 -- 172.21.15.12:0/3993647611 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31901a13c0 con 0x7f3190103160 2024-05-17T22:39:02.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.417+0000 7f318e59c640 1 --2- 172.21.15.12:0/3993647611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3190103160 0x7f319019e650 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f318400d950 tx=0x7f318400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:39:02.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.417+0000 7f3177fff640 1 -- 172.21.15.12:0/3993647611 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3184017070 con 0x7f3190103160 2024-05-17T22:39:02.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.417+0000 7f3195144640 1 -- 172.21.15.12:0/3993647611 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31901a16b0 con 0x7f3190103160 2024-05-17T22:39:02.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.417+0000 7f3177fff640 1 -- 172.21.15.12:0/3993647611 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3184009c30 con 0x7f3190103160 2024-05-17T22:39:02.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.417+0000 7f3177fff640 1 -- 172.21.15.12:0/3993647611 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318400f620 con 0x7f3190103160 2024-05-17T22:39:02.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.417+0000 7f3195144640 1 -- 172.21.15.12:0/3993647611 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31901a1c50 con 0x7f3190103160 2024-05-17T22:39:02.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.418+0000 7f3195144640 1 -- 172.21.15.12:0/3993647611 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3190109290 con 0x7f3190103160 2024-05-17T22:39:02.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.420+0000 7f3177fff640 1 -- 172.21.15.12:0/3993647611 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3184010460 con 0x7f3190103160 2024-05-17T22:39:02.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.420+0000 7f3177fff640 1 --2- 172.21.15.12:0/3993647611 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f31780b2870 0x7f31780b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:02.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.421+0000 7f318ed9d640 1 --2- 172.21.15.12:0/3993647611 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f31780b2870 0x7f31780b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:02.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.421+0000 7f3177fff640 1 -- 172.21.15.12:0/3993647611 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f31840e2170 con 0x7f3190103160 2024-05-17T22:39:02.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.421+0000 7f318ed9d640 1 --2- 172.21.15.12:0/3993647611 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f31780b2870 0x7f31780b4d80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3180009230 tx=0x7f3180031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:39:02.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.425+0000 7f3177fff640 1 -- 172.21.15.12:0/3993647611 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f31840a9b00 con 0x7f3190103160 2024-05-17T22:39:02.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.614+0000 7f3195144640 1 -- 172.21.15.12:0/3993647611 --> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3190063180 con 0x7f31780b2870 2024-05-17T22:39:02.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.620+0000 7f3177fff640 1 -- 172.21.15.12:0/3993647611 <== mgr.14152 v2:172.21.15.12:6800/1388514047 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340472 (secure 0 0 0) 0x7f3190063180 con 0x7f31780b2870 2024-05-17T22:39:02.626 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:39:02.629 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-05-17T22:39:02.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.623+0000 7f3195144640 1 -- 172.21.15.12:0/3993647611 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f31780b2870 msgr2=0x7f31780b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:02.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.623+0000 7f3195144640 1 --2- 172.21.15.12:0/3993647611 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f31780b2870 0x7f31780b4d80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3180009230 tx=0x7f3180031400 comp rx=0 tx=0).stop 2024-05-17T22:39:02.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.623+0000 7f3195144640 1 -- 172.21.15.12:0/3993647611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3190103160 msgr2=0x7f319019e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:02.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.623+0000 7f3195144640 1 --2- 172.21.15.12:0/3993647611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3190103160 0x7f319019e650 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f318400d950 tx=0x7f318400de10 comp rx=0 tx=0).stop 2024-05-17T22:39:02.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.624+0000 7f3195144640 1 -- 172.21.15.12:0/3993647611 shutdown_connections 2024-05-17T22:39:02.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.624+0000 7f3195144640 1 --2- 172.21.15.12:0/3993647611 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f31780b2870 0x7f31780b4d80 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:02.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.624+0000 7f3195144640 1 --2- 172.21.15.12:0/3993647611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3190103ab0 0x7f31901a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:02.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.624+0000 7f3195144640 1 --2- 172.21.15.12:0/3993647611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3190103160 0x7f319019e650 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:02.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.624+0000 7f3195144640 1 --2- 172.21.15.12:0/3993647611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3190102040 0x7f319006aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:02.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.624+0000 7f3195144640 1 -- 172.21.15.12:0/3993647611 >> 172.21.15.12:0/3993647611 conn(0x7f31900fd860 msgr2=0x7f3190106460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:39:02.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.624+0000 7f3195144640 1 -- 172.21.15.12:0/3993647611 shutdown_connections 2024-05-17T22:39:02.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:02.624+0000 7f3195144640 1 -- 172.21.15.12:0/3993647611 wait complete. 2024-05-17T22:39:02.638 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":433,"stamp":"2024-05-17T22:39:02.555236+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663586,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1251,"num_read_kb":729,"num_write":310,"num_write_kb":2288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":752,"ondisk_log_size":752,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":388216,"kb_used_data":14104,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553608,"statfs":{"total":1151940427776,"available":1151542894592,"internally_reserved":0,"allocated":14442496,"data_stored":8141483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5989,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":68,"num_read_kb":38,"num_write":7,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003425"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.522500+0000","last_change":"2024-05-17T22:34:40.422850+0000","last_active":"2024-05-17T22:37:14.522500+0000","last_peered":"2024-05-17T22:37:14.522500+0000","last_clean":"2024-05-17T22:37:14.522500+0000","last_became_active":"2024-05-17T22:34:40.422474+0000","last_became_peered":"2024-05-17T22:34:40.422474+0000","last_unstale":"2024-05-17T22:37:14.522500+0000","last_undegraded":"2024-05-17T22:37:14.522500+0000","last_fullsized":"2024-05-17T22:37:14.522500+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:23:51.537128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.520370+0000","last_change":"2024-05-17T22:34:40.427907+0000","last_active":"2024-05-17T22:37:14.520370+0000","last_peered":"2024-05-17T22:37:14.520370+0000","last_clean":"2024-05-17T22:37:14.520370+0000","last_became_active":"2024-05-17T22:34:40.427511+0000","last_became_peered":"2024-05-17T22:34:40.427511+0000","last_unstale":"2024-05-17T22:37:14.520370+0000","last_undegraded":"2024-05-17T22:37:14.520370+0000","last_fullsized":"2024-05-17T22:37:14.520370+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:01:46.707820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.525760+0000","last_change":"2024-05-17T22:34:40.436370+0000","last_active":"2024-05-17T22:37:14.525760+0000","last_peered":"2024-05-17T22:37:14.525760+0000","last_clean":"2024-05-17T22:37:14.525760+0000","last_became_active":"2024-05-17T22:34:40.434089+0000","last_became_peered":"2024-05-17T22:34:40.434089+0000","last_unstale":"2024-05-17T22:37:14.525760+0000","last_undegraded":"2024-05-17T22:37:14.525760+0000","last_fullsized":"2024-05-17T22:37:14.525760+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:58:16.775471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.509106+0000","last_change":"2024-05-17T22:35:15.556681+0000","last_active":"2024-05-17T22:37:14.509106+0000","last_peered":"2024-05-17T22:37:14.509106+0000","last_clean":"2024-05-17T22:37:14.509106+0000","last_became_active":"2024-05-17T22:35:15.556427+0000","last_became_peered":"2024-05-17T22:35:15.556427+0000","last_unstale":"2024-05-17T22:37:14.509106+0000","last_undegraded":"2024-05-17T22:37:14.509106+0000","last_fullsized":"2024-05-17T22:37:14.509106+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:28:01.726262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.495402+0000","last_change":"2024-05-17T22:34:40.434394+0000","last_active":"2024-05-17T22:37:14.495402+0000","last_peered":"2024-05-17T22:37:14.495402+0000","last_clean":"2024-05-17T22:37:14.495402+0000","last_became_active":"2024-05-17T22:34:40.433489+0000","last_became_peered":"2024-05-17T22:34:40.433489+0000","last_unstale":"2024-05-17T22:37:14.495402+0000","last_undegraded":"2024-05-17T22:37:14.495402+0000","last_fullsized":"2024-05-17T22:37:14.495402+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:26:24.068814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.515238+0000","last_change":"2024-05-17T22:35:15.553601+0000","last_active":"2024-05-17T22:37:14.515238+0000","last_peered":"2024-05-17T22:37:14.515238+0000","last_clean":"2024-05-17T22:37:14.515238+0000","last_became_active":"2024-05-17T22:35:15.553116+0000","last_became_peered":"2024-05-17T22:35:15.553116+0000","last_unstale":"2024-05-17T22:37:14.515238+0000","last_undegraded":"2024-05-17T22:37:14.515238+0000","last_fullsized":"2024-05-17T22:37:14.515238+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:09:38.401107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.518979+0000","last_change":"2024-05-17T22:34:40.430479+0000","last_active":"2024-05-17T22:37:14.518979+0000","last_peered":"2024-05-17T22:37:14.518979+0000","last_clean":"2024-05-17T22:37:14.518979+0000","last_became_active":"2024-05-17T22:34:40.430025+0000","last_became_peered":"2024-05-17T22:34:40.430025+0000","last_unstale":"2024-05-17T22:37:14.518979+0000","last_undegraded":"2024-05-17T22:37:14.518979+0000","last_fullsized":"2024-05-17T22:37:14.518979+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:14:19.725096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.519153+0000","last_change":"2024-05-17T22:34:40.431711+0000","last_active":"2024-05-17T22:37:14.519153+0000","last_peered":"2024-05-17T22:37:14.519153+0000","last_clean":"2024-05-17T22:37:14.519153+0000","last_became_active":"2024-05-17T22:34:40.431368+0000","last_became_peered":"2024-05-17T22:34:40.431368+0000","last_unstale":"2024-05-17T22:37:14.519153+0000","last_undegraded":"2024-05-17T22:37:14.519153+0000","last_fullsized":"2024-05-17T22:37:14.519153+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:06:34.904652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.521909+0000","last_change":"2024-05-17T22:34:40.435593+0000","last_active":"2024-05-17T22:37:14.521909+0000","last_peered":"2024-05-17T22:37:14.521909+0000","last_clean":"2024-05-17T22:37:14.521909+0000","last_became_active":"2024-05-17T22:34:40.435241+0000","last_became_peered":"2024-05-17T22:34:40.435241+0000","last_unstale":"2024-05-17T22:37:14.521909+0000","last_undegraded":"2024-05-17T22:37:14.521909+0000","last_fullsized":"2024-05-17T22:37:14.521909+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:08:57.239998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.526427+0000","last_change":"2024-05-17T22:34:40.436248+0000","last_active":"2024-05-17T22:37:14.526427+0000","last_peered":"2024-05-17T22:37:14.526427+0000","last_clean":"2024-05-17T22:37:14.526427+0000","last_became_active":"2024-05-17T22:34:40.435824+0000","last_became_peered":"2024-05-17T22:34:40.435824+0000","last_unstale":"2024-05-17T22:37:14.526427+0000","last_undegraded":"2024-05-17T22:37:14.526427+0000","last_fullsized":"2024-05-17T22:37:14.526427+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:39:07.297470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.524640+0000","last_change":"2024-05-17T22:34:40.435843+0000","last_active":"2024-05-17T22:37:14.524640+0000","last_peered":"2024-05-17T22:37:14.524640+0000","last_clean":"2024-05-17T22:37:14.524640+0000","last_became_active":"2024-05-17T22:34:40.435500+0000","last_became_peered":"2024-05-17T22:34:40.435500+0000","last_unstale":"2024-05-17T22:37:14.524640+0000","last_undegraded":"2024-05-17T22:37:14.524640+0000","last_fullsized":"2024-05-17T22:37:14.524640+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:11:28.115850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.501798+0000","last_change":"2024-05-17T22:34:40.432044+0000","last_active":"2024-05-17T22:37:14.501798+0000","last_peered":"2024-05-17T22:37:14.501798+0000","last_clean":"2024-05-17T22:37:14.501798+0000","last_became_active":"2024-05-17T22:34:40.429141+0000","last_became_peered":"2024-05-17T22:34:40.429141+0000","last_unstale":"2024-05-17T22:37:14.501798+0000","last_undegraded":"2024-05-17T22:37:14.501798+0000","last_fullsized":"2024-05-17T22:37:14.501798+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:04:53.428681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.526534+0000","last_change":"2024-05-17T22:34:40.438474+0000","last_active":"2024-05-17T22:37:14.526534+0000","last_peered":"2024-05-17T22:37:14.526534+0000","last_clean":"2024-05-17T22:37:14.526534+0000","last_became_active":"2024-05-17T22:34:40.438109+0000","last_became_peered":"2024-05-17T22:34:40.438109+0000","last_unstale":"2024-05-17T22:37:14.526534+0000","last_undegraded":"2024-05-17T22:37:14.526534+0000","last_fullsized":"2024-05-17T22:37:14.526534+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:26:00.665246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.516486+0000","last_change":"2024-05-17T22:34:40.431563+0000","last_active":"2024-05-17T22:37:14.516486+0000","last_peered":"2024-05-17T22:37:14.516486+0000","last_clean":"2024-05-17T22:37:14.516486+0000","last_became_active":"2024-05-17T22:34:40.430992+0000","last_became_peered":"2024-05-17T22:34:40.430992+0000","last_unstale":"2024-05-17T22:37:14.516486+0000","last_undegraded":"2024-05-17T22:37:14.516486+0000","last_fullsized":"2024-05-17T22:37:14.516486+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:51:26.280440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.516587+0000","last_change":"2024-05-17T22:34:40.433629+0000","last_active":"2024-05-17T22:37:14.516587+0000","last_peered":"2024-05-17T22:37:14.516587+0000","last_clean":"2024-05-17T22:37:14.516587+0000","last_became_active":"2024-05-17T22:34:40.433270+0000","last_became_peered":"2024-05-17T22:34:40.433270+0000","last_unstale":"2024-05-17T22:37:14.516587+0000","last_undegraded":"2024-05-17T22:37:14.516587+0000","last_fullsized":"2024-05-17T22:37:14.516587+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:57:51.164078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.525064+0000","last_change":"2024-05-17T22:34:40.434218+0000","last_active":"2024-05-17T22:37:14.525064+0000","last_peered":"2024-05-17T22:37:14.525064+0000","last_clean":"2024-05-17T22:37:14.525064+0000","last_became_active":"2024-05-17T22:34:40.433849+0000","last_became_peered":"2024-05-17T22:34:40.433849+0000","last_unstale":"2024-05-17T22:37:14.525064+0000","last_undegraded":"2024-05-17T22:37:14.525064+0000","last_fullsized":"2024-05-17T22:37:14.525064+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:41:55.984109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.522605+0000","last_change":"2024-05-17T22:34:40.427892+0000","last_active":"2024-05-17T22:37:14.522605+0000","last_peered":"2024-05-17T22:37:14.522605+0000","last_clean":"2024-05-17T22:37:14.522605+0000","last_became_active":"2024-05-17T22:34:40.427303+0000","last_became_peered":"2024-05-17T22:34:40.427303+0000","last_unstale":"2024-05-17T22:37:14.522605+0000","last_undegraded":"2024-05-17T22:37:14.522605+0000","last_fullsized":"2024-05-17T22:37:14.522605+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:59:38.632123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.524943+0000","last_change":"2024-05-17T22:34:40.424784+0000","last_active":"2024-05-17T22:37:14.524943+0000","last_peered":"2024-05-17T22:37:14.524943+0000","last_clean":"2024-05-17T22:37:14.524943+0000","last_became_active":"2024-05-17T22:34:40.424439+0000","last_became_peered":"2024-05-17T22:34:40.424439+0000","last_unstale":"2024-05-17T22:37:14.524943+0000","last_undegraded":"2024-05-17T22:37:14.524943+0000","last_fullsized":"2024-05-17T22:37:14.524943+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:02:53.681600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.513305+0000","last_change":"2024-05-17T22:34:40.435023+0000","last_active":"2024-05-17T22:37:14.513305+0000","last_peered":"2024-05-17T22:37:14.513305+0000","last_clean":"2024-05-17T22:37:14.513305+0000","last_became_active":"2024-05-17T22:34:40.434441+0000","last_became_peered":"2024-05-17T22:34:40.434441+0000","last_unstale":"2024-05-17T22:37:14.513305+0000","last_undegraded":"2024-05-17T22:37:14.513305+0000","last_fullsized":"2024-05-17T22:37:14.513305+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:16:07.671897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624113+0000","last_change":"2024-05-17T22:34:52.493192+0000","last_active":"2024-05-17T22:36:15.624113+0000","last_peered":"2024-05-17T22:36:15.624113+0000","last_clean":"2024-05-17T22:36:15.624113+0000","last_became_active":"2024-05-17T22:34:52.492786+0000","last_became_peered":"2024-05-17T22:34:52.492786+0000","last_unstale":"2024-05-17T22:36:15.624113+0000","last_undegraded":"2024-05-17T22:36:15.624113+0000","last_fullsized":"2024-05-17T22:36:15.624113+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:43:40.152240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":56,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:43.067607+0000","last_change":"2024-05-17T22:34:37.402360+0000","last_active":"2024-05-17T22:36:43.067607+0000","last_peered":"2024-05-17T22:36:43.067607+0000","last_clean":"2024-05-17T22:36:43.067607+0000","last_became_active":"2024-05-17T22:34:37.401882+0000","last_became_peered":"2024-05-17T22:34:37.401882+0000","last_unstale":"2024-05-17T22:36:43.067607+0000","last_undegraded":"2024-05-17T22:36:43.067607+0000","last_fullsized":"2024-05-17T22:36:43.067607+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:53:13.300626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.506469+0000","last_change":"2024-05-17T22:34:40.436259+0000","last_active":"2024-05-17T22:37:14.506469+0000","last_peered":"2024-05-17T22:37:14.506469+0000","last_clean":"2024-05-17T22:37:14.506469+0000","last_became_active":"2024-05-17T22:34:40.435899+0000","last_became_peered":"2024-05-17T22:34:40.435899+0000","last_unstale":"2024-05-17T22:37:14.506469+0000","last_undegraded":"2024-05-17T22:37:14.506469+0000","last_fullsized":"2024-05-17T22:37:14.506469+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:26:29.327513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003713+0000","last_change":"2024-05-17T22:34:52.495236+0000","last_active":"2024-05-17T22:36:16.003713+0000","last_peered":"2024-05-17T22:36:16.003713+0000","last_clean":"2024-05-17T22:36:16.003713+0000","last_became_active":"2024-05-17T22:34:52.494783+0000","last_became_peered":"2024-05-17T22:34:52.494783+0000","last_unstale":"2024-05-17T22:36:16.003713+0000","last_undegraded":"2024-05-17T22:36:16.003713+0000","last_fullsized":"2024-05-17T22:36:16.003713+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:08:15.010706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:37.768914+0000","last_change":"2024-05-17T22:34:37.441129+0000","last_active":"2024-05-17T22:36:37.768914+0000","last_peered":"2024-05-17T22:36:37.768914+0000","last_clean":"2024-05-17T22:36:37.768914+0000","last_became_active":"2024-05-17T22:34:37.440738+0000","last_became_peered":"2024-05-17T22:34:37.440738+0000","last_unstale":"2024-05-17T22:36:37.768914+0000","last_undegraded":"2024-05-17T22:36:37.768914+0000","last_fullsized":"2024-05-17T22:36:37.768914+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:03:16.945076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.525530+0000","last_change":"2024-05-17T22:34:40.434146+0000","last_active":"2024-05-17T22:37:14.525530+0000","last_peered":"2024-05-17T22:37:14.525530+0000","last_clean":"2024-05-17T22:37:14.525530+0000","last_became_active":"2024-05-17T22:34:40.433802+0000","last_became_peered":"2024-05-17T22:34:40.433802+0000","last_unstale":"2024-05-17T22:37:14.525530+0000","last_undegraded":"2024-05-17T22:37:14.525530+0000","last_fullsized":"2024-05-17T22:37:14.525530+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:08:39.905409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004211+0000","last_change":"2024-05-17T22:34:52.501134+0000","last_active":"2024-05-17T22:36:16.004211+0000","last_peered":"2024-05-17T22:36:16.004211+0000","last_clean":"2024-05-17T22:36:16.004211+0000","last_became_active":"2024-05-17T22:34:52.500547+0000","last_became_peered":"2024-05-17T22:34:52.500547+0000","last_unstale":"2024-05-17T22:36:16.004211+0000","last_undegraded":"2024-05-17T22:36:16.004211+0000","last_fullsized":"2024-05-17T22:36:16.004211+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:13:04.200865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.626422+0000","last_change":"2024-05-17T22:34:37.404019+0000","last_active":"2024-05-17T22:36:15.626422+0000","last_peered":"2024-05-17T22:36:15.626422+0000","last_clean":"2024-05-17T22:36:15.626422+0000","last_became_active":"2024-05-17T22:34:37.403607+0000","last_became_peered":"2024-05-17T22:34:37.403607+0000","last_unstale":"2024-05-17T22:36:15.626422+0000","last_undegraded":"2024-05-17T22:36:15.626422+0000","last_fullsized":"2024-05-17T22:36:15.626422+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:30:21.405195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.525402+0000","last_change":"2024-05-17T22:34:40.426889+0000","last_active":"2024-05-17T22:37:14.525402+0000","last_peered":"2024-05-17T22:37:14.525402+0000","last_clean":"2024-05-17T22:37:14.525402+0000","last_became_active":"2024-05-17T22:34:40.426512+0000","last_became_peered":"2024-05-17T22:34:40.426512+0000","last_unstale":"2024-05-17T22:37:14.525402+0000","last_undegraded":"2024-05-17T22:37:14.525402+0000","last_fullsized":"2024-05-17T22:37:14.525402+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:30:51.363177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004528+0000","last_change":"2024-05-17T22:34:52.487514+0000","last_active":"2024-05-17T22:36:16.004528+0000","last_peered":"2024-05-17T22:36:16.004528+0000","last_clean":"2024-05-17T22:36:16.004528+0000","last_became_active":"2024-05-17T22:34:52.487068+0000","last_became_peered":"2024-05-17T22:34:52.487068+0000","last_unstale":"2024-05-17T22:36:16.004528+0000","last_undegraded":"2024-05-17T22:36:16.004528+0000","last_fullsized":"2024-05-17T22:36:16.004528+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:52:13.573986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.626123+0000","last_change":"2024-05-17T22:34:37.400518+0000","last_active":"2024-05-17T22:36:15.626123+0000","last_peered":"2024-05-17T22:36:15.626123+0000","last_clean":"2024-05-17T22:36:15.626123+0000","last_became_active":"2024-05-17T22:34:37.400145+0000","last_became_peered":"2024-05-17T22:34:37.400145+0000","last_unstale":"2024-05-17T22:36:15.626123+0000","last_undegraded":"2024-05-17T22:36:15.626123+0000","last_fullsized":"2024-05-17T22:36:15.626123+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:56:53.593624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.525513+0000","last_change":"2024-05-17T22:34:40.434148+0000","last_active":"2024-05-17T22:37:14.525513+0000","last_peered":"2024-05-17T22:37:14.525513+0000","last_clean":"2024-05-17T22:37:14.525513+0000","last_became_active":"2024-05-17T22:34:40.433613+0000","last_became_peered":"2024-05-17T22:34:40.433613+0000","last_unstale":"2024-05-17T22:37:14.525513+0000","last_undegraded":"2024-05-17T22:37:14.525513+0000","last_fullsized":"2024-05-17T22:37:14.525513+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:09:39.049852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624340+0000","last_change":"2024-05-17T22:34:52.490417+0000","last_active":"2024-05-17T22:36:15.624340+0000","last_peered":"2024-05-17T22:36:15.624340+0000","last_clean":"2024-05-17T22:36:15.624340+0000","last_became_active":"2024-05-17T22:34:52.489953+0000","last_became_peered":"2024-05-17T22:34:52.489953+0000","last_unstale":"2024-05-17T22:36:15.624340+0000","last_undegraded":"2024-05-17T22:36:15.624340+0000","last_fullsized":"2024-05-17T22:36:15.624340+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:23:56.491899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'8","reported_seq":251,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:39:01.346668+0000","last_change":"2024-05-17T22:34:37.405304+0000","last_active":"2024-05-17T22:39:01.346668+0000","last_peered":"2024-05-17T22:39:01.346668+0000","last_clean":"2024-05-17T22:39:01.346668+0000","last_became_active":"2024-05-17T22:34:37.404552+0000","last_became_peered":"2024-05-17T22:34:37.404552+0000","last_unstale":"2024-05-17T22:39:01.346668+0000","last_undegraded":"2024-05-17T22:39:01.346668+0000","last_fullsized":"2024-05-17T22:39:01.346668+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:57:06.556886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1462,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":198,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.512663+0000","last_change":"2024-05-17T22:34:40.430537+0000","last_active":"2024-05-17T22:37:14.512663+0000","last_peered":"2024-05-17T22:37:14.512663+0000","last_clean":"2024-05-17T22:37:14.512663+0000","last_became_active":"2024-05-17T22:34:40.430193+0000","last_became_peered":"2024-05-17T22:34:40.430193+0000","last_unstale":"2024-05-17T22:37:14.512663+0000","last_undegraded":"2024-05-17T22:37:14.512663+0000","last_fullsized":"2024-05-17T22:37:14.512663+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:58:42.091445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624546+0000","last_change":"2024-05-17T22:35:15.559077+0000","last_active":"2024-05-17T22:36:15.624546+0000","last_peered":"2024-05-17T22:36:15.624546+0000","last_clean":"2024-05-17T22:36:15.624546+0000","last_became_active":"2024-05-17T22:35:15.558382+0000","last_became_peered":"2024-05-17T22:35:15.558382+0000","last_unstale":"2024-05-17T22:36:15.624546+0000","last_undegraded":"2024-05-17T22:36:15.624546+0000","last_fullsized":"2024-05-17T22:36:15.624546+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:20:16.009620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.626030+0000","last_change":"2024-05-17T22:34:37.405332+0000","last_active":"2024-05-17T22:36:15.626030+0000","last_peered":"2024-05-17T22:36:15.626030+0000","last_clean":"2024-05-17T22:36:15.626030+0000","last_became_active":"2024-05-17T22:34:37.404970+0000","last_became_peered":"2024-05-17T22:34:37.404970+0000","last_unstale":"2024-05-17T22:36:15.626030+0000","last_undegraded":"2024-05-17T22:36:15.626030+0000","last_fullsized":"2024-05-17T22:36:15.626030+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:37:39.933686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.517661+0000","last_change":"2024-05-17T22:34:40.432690+0000","last_active":"2024-05-17T22:37:14.517661+0000","last_peered":"2024-05-17T22:37:14.517661+0000","last_clean":"2024-05-17T22:37:14.517661+0000","last_became_active":"2024-05-17T22:34:40.432274+0000","last_became_peered":"2024-05-17T22:34:40.432274+0000","last_unstale":"2024-05-17T22:37:14.517661+0000","last_undegraded":"2024-05-17T22:37:14.517661+0000","last_fullsized":"2024-05-17T22:37:14.517661+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:12:47.926865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:56.856503+0000","last_change":"2024-05-17T22:34:52.494645+0000","last_active":"2024-05-17T22:36:56.856503+0000","last_peered":"2024-05-17T22:36:56.856503+0000","last_clean":"2024-05-17T22:36:56.856503+0000","last_became_active":"2024-05-17T22:34:52.494255+0000","last_became_peered":"2024-05-17T22:34:52.494255+0000","last_unstale":"2024-05-17T22:36:56.856503+0000","last_undegraded":"2024-05-17T22:36:56.856503+0000","last_fullsized":"2024-05-17T22:36:56.856503+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:41:26.633571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685426+0000","last_change":"2024-05-17T22:34:37.398582+0000","last_active":"2024-05-17T22:36:15.685426+0000","last_peered":"2024-05-17T22:36:15.685426+0000","last_clean":"2024-05-17T22:36:15.685426+0000","last_became_active":"2024-05-17T22:34:37.398156+0000","last_became_peered":"2024-05-17T22:34:37.398156+0000","last_unstale":"2024-05-17T22:36:15.685426+0000","last_undegraded":"2024-05-17T22:36:15.685426+0000","last_fullsized":"2024-05-17T22:36:15.685426+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:46:00.276214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.518565+0000","last_change":"2024-05-17T22:35:15.551402+0000","last_active":"2024-05-17T22:37:14.518565+0000","last_peered":"2024-05-17T22:37:14.518565+0000","last_clean":"2024-05-17T22:37:14.518565+0000","last_became_active":"2024-05-17T22:35:15.551021+0000","last_became_peered":"2024-05-17T22:35:15.551021+0000","last_unstale":"2024-05-17T22:37:14.518565+0000","last_undegraded":"2024-05-17T22:37:14.518565+0000","last_fullsized":"2024-05-17T22:37:14.518565+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:35:18.573620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:54.120653+0000","last_change":"2024-05-17T22:34:52.487413+0000","last_active":"2024-05-17T22:36:54.120653+0000","last_peered":"2024-05-17T22:36:54.120653+0000","last_clean":"2024-05-17T22:36:54.120653+0000","last_became_active":"2024-05-17T22:34:52.487032+0000","last_became_peered":"2024-05-17T22:34:52.487032+0000","last_unstale":"2024-05-17T22:36:54.120653+0000","last_undegraded":"2024-05-17T22:36:54.120653+0000","last_fullsized":"2024-05-17T22:36:54.120653+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:34:49.122008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003315+0000","last_change":"2024-05-17T22:34:37.404894+0000","last_active":"2024-05-17T22:36:16.003315+0000","last_peered":"2024-05-17T22:36:16.003315+0000","last_clean":"2024-05-17T22:36:16.003315+0000","last_became_active":"2024-05-17T22:34:37.404567+0000","last_became_peered":"2024-05-17T22:34:37.404567+0000","last_unstale":"2024-05-17T22:36:16.003315+0000","last_undegraded":"2024-05-17T22:36:16.003315+0000","last_fullsized":"2024-05-17T22:36:16.003315+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:13:09.761556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.525152+0000","last_change":"2024-05-17T22:34:40.418267+0000","last_active":"2024-05-17T22:37:14.525152+0000","last_peered":"2024-05-17T22:37:14.525152+0000","last_clean":"2024-05-17T22:37:14.525152+0000","last_became_active":"2024-05-17T22:34:40.417928+0000","last_became_peered":"2024-05-17T22:34:40.417928+0000","last_unstale":"2024-05-17T22:37:14.525152+0000","last_undegraded":"2024-05-17T22:37:14.525152+0000","last_fullsized":"2024-05-17T22:37:14.525152+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:17:37.055004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.984205+0000","last_change":"2024-05-17T22:34:52.492126+0000","last_active":"2024-05-17T22:36:15.984205+0000","last_peered":"2024-05-17T22:36:15.984205+0000","last_clean":"2024-05-17T22:36:15.984205+0000","last_became_active":"2024-05-17T22:34:52.491695+0000","last_became_peered":"2024-05-17T22:34:52.491695+0000","last_unstale":"2024-05-17T22:36:15.984205+0000","last_undegraded":"2024-05-17T22:36:15.984205+0000","last_fullsized":"2024-05-17T22:36:15.984205+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:15:17.710279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'5","reported_seq":240,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:38:58.340899+0000","last_change":"2024-05-17T22:34:37.406989+0000","last_active":"2024-05-17T22:38:58.340899+0000","last_peered":"2024-05-17T22:38:58.340899+0000","last_clean":"2024-05-17T22:38:58.340899+0000","last_became_active":"2024-05-17T22:34:37.406612+0000","last_became_peered":"2024-05-17T22:34:37.406612+0000","last_unstale":"2024-05-17T22:38:58.340899+0000","last_undegraded":"2024-05-17T22:38:58.340899+0000","last_fullsized":"2024-05-17T22:38:58.340899+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:47:52.480804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":190,"num_read_kb":190,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.524255+0000","last_change":"2024-05-17T22:34:40.427221+0000","last_active":"2024-05-17T22:37:14.524255+0000","last_peered":"2024-05-17T22:37:14.524255+0000","last_clean":"2024-05-17T22:37:14.524255+0000","last_became_active":"2024-05-17T22:34:40.426703+0000","last_became_peered":"2024-05-17T22:34:40.426703+0000","last_unstale":"2024-05-17T22:37:14.524255+0000","last_undegraded":"2024-05-17T22:37:14.524255+0000","last_fullsized":"2024-05-17T22:37:14.524255+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:49:15.629048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.622725+0000","last_change":"2024-05-17T22:34:52.499086+0000","last_active":"2024-05-17T22:36:15.622725+0000","last_peered":"2024-05-17T22:36:15.622725+0000","last_clean":"2024-05-17T22:36:15.622725+0000","last_became_active":"2024-05-17T22:34:52.498666+0000","last_became_peered":"2024-05-17T22:34:52.498666+0000","last_unstale":"2024-05-17T22:36:15.622725+0000","last_undegraded":"2024-05-17T22:36:15.622725+0000","last_fullsized":"2024-05-17T22:36:15.622725+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:46:25.882964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.005213+0000","last_change":"2024-05-17T22:35:15.836294+0000","last_active":"2024-05-17T22:36:16.005213+0000","last_peered":"2024-05-17T22:36:16.005213+0000","last_clean":"2024-05-17T22:36:16.005213+0000","last_became_active":"2024-05-17T22:35:15.835759+0000","last_became_peered":"2024-05-17T22:35:15.835759+0000","last_unstale":"2024-05-17T22:36:16.005213+0000","last_undegraded":"2024-05-17T22:36:16.005213+0000","last_fullsized":"2024-05-17T22:36:16.005213+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:23:32.106692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.519861+0000","last_change":"2024-05-17T22:34:40.433331+0000","last_active":"2024-05-17T22:37:14.519861+0000","last_peered":"2024-05-17T22:37:14.519861+0000","last_clean":"2024-05-17T22:37:14.519861+0000","last_became_active":"2024-05-17T22:34:40.432989+0000","last_became_peered":"2024-05-17T22:34:40.432989+0000","last_unstale":"2024-05-17T22:37:14.519861+0000","last_undegraded":"2024-05-17T22:37:14.519861+0000","last_fullsized":"2024-05-17T22:37:14.519861+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:22:49.146148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004235+0000","last_change":"2024-05-17T22:34:52.494418+0000","last_active":"2024-05-17T22:36:16.004235+0000","last_peered":"2024-05-17T22:36:16.004235+0000","last_clean":"2024-05-17T22:36:16.004235+0000","last_became_active":"2024-05-17T22:34:52.493953+0000","last_became_peered":"2024-05-17T22:34:52.493953+0000","last_unstale":"2024-05-17T22:36:16.004235+0000","last_undegraded":"2024-05-17T22:36:16.004235+0000","last_fullsized":"2024-05-17T22:36:16.004235+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:50:40.211770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004010+0000","last_change":"2024-05-17T22:34:37.442986+0000","last_active":"2024-05-17T22:36:16.004010+0000","last_peered":"2024-05-17T22:36:16.004010+0000","last_clean":"2024-05-17T22:36:16.004010+0000","last_became_active":"2024-05-17T22:34:37.442399+0000","last_became_peered":"2024-05-17T22:34:37.442399+0000","last_unstale":"2024-05-17T22:36:16.004010+0000","last_undegraded":"2024-05-17T22:36:16.004010+0000","last_fullsized":"2024-05-17T22:36:16.004010+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:34:05.785811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.525214+0000","last_change":"2024-05-17T22:34:40.426793+0000","last_active":"2024-05-17T22:37:14.525214+0000","last_peered":"2024-05-17T22:37:14.525214+0000","last_clean":"2024-05-17T22:37:14.525214+0000","last_became_active":"2024-05-17T22:34:40.426420+0000","last_became_peered":"2024-05-17T22:34:40.426420+0000","last_unstale":"2024-05-17T22:37:14.525214+0000","last_undegraded":"2024-05-17T22:37:14.525214+0000","last_fullsized":"2024-05-17T22:37:14.525214+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:57:07.099254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.622953+0000","last_change":"2024-05-17T22:34:52.498762+0000","last_active":"2024-05-17T22:36:15.622953+0000","last_peered":"2024-05-17T22:36:15.622953+0000","last_clean":"2024-05-17T22:36:15.622953+0000","last_became_active":"2024-05-17T22:34:52.498306+0000","last_became_peered":"2024-05-17T22:34:52.498306+0000","last_unstale":"2024-05-17T22:36:15.622953+0000","last_undegraded":"2024-05-17T22:36:15.622953+0000","last_fullsized":"2024-05-17T22:36:15.622953+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:18:32.717563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'39","reported_seq":476,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:38:59.343737+0000","last_change":"2024-05-17T22:34:37.407885+0000","last_active":"2024-05-17T22:38:59.343737+0000","last_peered":"2024-05-17T22:38:59.343737+0000","last_clean":"2024-05-17T22:38:59.343737+0000","last_became_active":"2024-05-17T22:34:37.407463+0000","last_became_peered":"2024-05-17T22:34:37.407463+0000","last_unstale":"2024-05-17T22:38:59.343737+0000","last_undegraded":"2024-05-17T22:38:59.343737+0000","last_fullsized":"2024-05-17T22:38:59.343737+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:26:20.288557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68002,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":392,"num_read_kb":263,"num_write":40,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.501714+0000","last_change":"2024-05-17T22:34:40.435800+0000","last_active":"2024-05-17T22:37:14.501714+0000","last_peered":"2024-05-17T22:37:14.501714+0000","last_clean":"2024-05-17T22:37:14.501714+0000","last_became_active":"2024-05-17T22:34:40.435245+0000","last_became_peered":"2024-05-17T22:34:40.435245+0000","last_unstale":"2024-05-17T22:37:14.501714+0000","last_undegraded":"2024-05-17T22:37:14.501714+0000","last_fullsized":"2024-05-17T22:37:14.501714+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:09:58.699026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004448+0000","last_change":"2024-05-17T22:34:52.487336+0000","last_active":"2024-05-17T22:36:16.004448+0000","last_peered":"2024-05-17T22:36:16.004448+0000","last_clean":"2024-05-17T22:36:16.004448+0000","last_became_active":"2024-05-17T22:34:52.486883+0000","last_became_peered":"2024-05-17T22:34:52.486883+0000","last_unstale":"2024-05-17T22:36:16.004448+0000","last_undegraded":"2024-05-17T22:36:16.004448+0000","last_fullsized":"2024-05-17T22:36:16.004448+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:44:41.264072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624096+0000","last_change":"2024-05-17T22:34:37.405044+0000","last_active":"2024-05-17T22:36:15.624096+0000","last_peered":"2024-05-17T22:36:15.624096+0000","last_clean":"2024-05-17T22:36:15.624096+0000","last_became_active":"2024-05-17T22:34:37.404610+0000","last_became_peered":"2024-05-17T22:34:37.404610+0000","last_unstale":"2024-05-17T22:36:15.624096+0000","last_undegraded":"2024-05-17T22:36:15.624096+0000","last_fullsized":"2024-05-17T22:36:15.624096+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:20:37.939069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.521845+0000","last_change":"2024-05-17T22:34:40.430655+0000","last_active":"2024-05-17T22:37:14.521845+0000","last_peered":"2024-05-17T22:37:14.521845+0000","last_clean":"2024-05-17T22:37:14.521845+0000","last_became_active":"2024-05-17T22:34:40.430307+0000","last_became_peered":"2024-05-17T22:34:40.430307+0000","last_unstale":"2024-05-17T22:37:14.521845+0000","last_undegraded":"2024-05-17T22:37:14.521845+0000","last_fullsized":"2024-05-17T22:37:14.521845+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:57:19.036117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628175+0000","last_change":"2024-05-17T22:34:52.494709+0000","last_active":"2024-05-17T22:36:15.628175+0000","last_peered":"2024-05-17T22:36:15.628175+0000","last_clean":"2024-05-17T22:36:15.628175+0000","last_became_active":"2024-05-17T22:34:52.494195+0000","last_became_peered":"2024-05-17T22:34:52.494195+0000","last_unstale":"2024-05-17T22:36:15.628175+0000","last_undegraded":"2024-05-17T22:36:15.628175+0000","last_fullsized":"2024-05-17T22:36:15.628175+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:35:41.188459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003957+0000","last_change":"2024-05-17T22:34:37.402494+0000","last_active":"2024-05-17T22:36:16.003957+0000","last_peered":"2024-05-17T22:36:16.003957+0000","last_clean":"2024-05-17T22:36:16.003957+0000","last_became_active":"2024-05-17T22:34:37.402102+0000","last_became_peered":"2024-05-17T22:34:37.402102+0000","last_unstale":"2024-05-17T22:36:16.003957+0000","last_undegraded":"2024-05-17T22:36:16.003957+0000","last_fullsized":"2024-05-17T22:36:16.003957+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:22:30.915780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.522351+0000","last_change":"2024-05-17T22:34:40.440731+0000","last_active":"2024-05-17T22:37:14.522351+0000","last_peered":"2024-05-17T22:37:14.522351+0000","last_clean":"2024-05-17T22:37:14.522351+0000","last_became_active":"2024-05-17T22:34:40.440374+0000","last_became_peered":"2024-05-17T22:34:40.440374+0000","last_unstale":"2024-05-17T22:37:14.522351+0000","last_undegraded":"2024-05-17T22:37:14.522351+0000","last_fullsized":"2024-05-17T22:37:14.522351+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:26:40.029155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.006411+0000","last_change":"2024-05-17T22:36:16.005856+0000","last_active":"2024-05-17T22:36:16.006411+0000","last_peered":"2024-05-17T22:36:16.006411+0000","last_clean":"2024-05-17T22:36:16.006411+0000","last_became_active":"2024-05-17T22:36:16.005303+0000","last_became_peered":"2024-05-17T22:36:16.005303+0000","last_unstale":"2024-05-17T22:36:16.006411+0000","last_undegraded":"2024-05-17T22:36:16.006411+0000","last_fullsized":"2024-05-17T22:36:16.006411+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:41:15.708879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":241,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:38:56.372568+0000","last_change":"2024-05-17T22:34:37.403123+0000","last_active":"2024-05-17T22:38:56.372568+0000","last_peered":"2024-05-17T22:38:56.372568+0000","last_clean":"2024-05-17T22:38:56.372568+0000","last_became_active":"2024-05-17T22:34:37.402702+0000","last_became_peered":"2024-05-17T22:34:37.402702+0000","last_unstale":"2024-05-17T22:38:56.372568+0000","last_undegraded":"2024-05-17T22:38:56.372568+0000","last_fullsized":"2024-05-17T22:38:56.372568+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:50:02.031651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":193,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.512621+0000","last_change":"2024-05-17T22:34:40.431895+0000","last_active":"2024-05-17T22:37:14.512621+0000","last_peered":"2024-05-17T22:37:14.512621+0000","last_clean":"2024-05-17T22:37:14.512621+0000","last_became_active":"2024-05-17T22:34:40.431527+0000","last_became_peered":"2024-05-17T22:34:40.431527+0000","last_unstale":"2024-05-17T22:37:14.512621+0000","last_undegraded":"2024-05-17T22:37:14.512621+0000","last_fullsized":"2024-05-17T22:37:14.512621+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:30:09.404255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.686140+0000","last_change":"2024-05-17T22:34:52.495583+0000","last_active":"2024-05-17T22:36:15.686140+0000","last_peered":"2024-05-17T22:36:15.686140+0000","last_clean":"2024-05-17T22:36:15.686140+0000","last_became_active":"2024-05-17T22:34:52.495030+0000","last_became_peered":"2024-05-17T22:34:52.495030+0000","last_unstale":"2024-05-17T22:36:15.686140+0000","last_undegraded":"2024-05-17T22:36:15.686140+0000","last_fullsized":"2024-05-17T22:36:15.686140+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:04:54.046891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":61,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625592+0000","last_change":"2024-05-17T22:34:37.442154+0000","last_active":"2024-05-17T22:36:15.625592+0000","last_peered":"2024-05-17T22:36:15.625592+0000","last_clean":"2024-05-17T22:36:15.625592+0000","last_became_active":"2024-05-17T22:34:37.441691+0000","last_became_peered":"2024-05-17T22:34:37.441691+0000","last_unstale":"2024-05-17T22:36:15.625592+0000","last_undegraded":"2024-05-17T22:36:15.625592+0000","last_fullsized":"2024-05-17T22:36:15.625592+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:47:29.862031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.517818+0000","last_change":"2024-05-17T22:34:40.430894+0000","last_active":"2024-05-17T22:37:14.517818+0000","last_peered":"2024-05-17T22:37:14.517818+0000","last_clean":"2024-05-17T22:37:14.517818+0000","last_became_active":"2024-05-17T22:34:40.430328+0000","last_became_peered":"2024-05-17T22:34:40.430328+0000","last_unstale":"2024-05-17T22:37:14.517818+0000","last_undegraded":"2024-05-17T22:37:14.517818+0000","last_fullsized":"2024-05-17T22:37:14.517818+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:36:46.619111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003488+0000","last_change":"2024-05-17T22:34:52.487555+0000","last_active":"2024-05-17T22:36:16.003488+0000","last_peered":"2024-05-17T22:36:16.003488+0000","last_clean":"2024-05-17T22:36:16.003488+0000","last_became_active":"2024-05-17T22:34:52.487076+0000","last_became_peered":"2024-05-17T22:34:52.487076+0000","last_unstale":"2024-05-17T22:36:16.003488+0000","last_undegraded":"2024-05-17T22:36:16.003488+0000","last_fullsized":"2024-05-17T22:36:16.003488+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:05:46.720665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.628417+0000","last_change":"2024-05-17T22:35:15.558860+0000","last_active":"2024-05-17T22:36:15.628417+0000","last_peered":"2024-05-17T22:36:15.628417+0000","last_clean":"2024-05-17T22:36:15.628417+0000","last_became_active":"2024-05-17T22:35:15.553683+0000","last_became_peered":"2024-05-17T22:35:15.553683+0000","last_unstale":"2024-05-17T22:36:15.628417+0000","last_undegraded":"2024-05-17T22:36:15.628417+0000","last_fullsized":"2024-05-17T22:36:15.628417+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:27:24.978282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.506967+0000","last_change":"2024-05-17T22:34:40.433499+0000","last_active":"2024-05-17T22:37:14.506967+0000","last_peered":"2024-05-17T22:37:14.506967+0000","last_clean":"2024-05-17T22:37:14.506967+0000","last_became_active":"2024-05-17T22:34:40.431370+0000","last_became_peered":"2024-05-17T22:34:40.431370+0000","last_unstale":"2024-05-17T22:37:14.506967+0000","last_undegraded":"2024-05-17T22:37:14.506967+0000","last_fullsized":"2024-05-17T22:37:14.506967+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:25:08.383021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625953+0000","last_change":"2024-05-17T22:34:52.489627+0000","last_active":"2024-05-17T22:36:15.625953+0000","last_peered":"2024-05-17T22:36:15.625953+0000","last_clean":"2024-05-17T22:36:15.625953+0000","last_became_active":"2024-05-17T22:34:52.489173+0000","last_became_peered":"2024-05-17T22:34:52.489173+0000","last_unstale":"2024-05-17T22:36:15.625953+0000","last_undegraded":"2024-05-17T22:36:15.625953+0000","last_fullsized":"2024-05-17T22:36:15.625953+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:52:26.908817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:43.067577+0000","last_change":"2024-05-17T22:35:15.553325+0000","last_active":"2024-05-17T22:36:43.067577+0000","last_peered":"2024-05-17T22:36:43.067577+0000","last_clean":"2024-05-17T22:36:43.067577+0000","last_became_active":"2024-05-17T22:35:15.552867+0000","last_became_peered":"2024-05-17T22:35:15.552867+0000","last_unstale":"2024-05-17T22:36:43.067577+0000","last_undegraded":"2024-05-17T22:36:43.067577+0000","last_fullsized":"2024-05-17T22:36:43.067577+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:28:36.618331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.526055+0000","last_change":"2024-05-17T22:34:40.423221+0000","last_active":"2024-05-17T22:37:14.526055+0000","last_peered":"2024-05-17T22:37:14.526055+0000","last_clean":"2024-05-17T22:37:14.526055+0000","last_became_active":"2024-05-17T22:34:40.422838+0000","last_became_peered":"2024-05-17T22:34:40.422838+0000","last_unstale":"2024-05-17T22:37:14.526055+0000","last_undegraded":"2024-05-17T22:37:14.526055+0000","last_fullsized":"2024-05-17T22:37:14.526055+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:39:51.873212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.685905+0000","last_change":"2024-05-17T22:34:52.484802+0000","last_active":"2024-05-17T22:36:15.685905+0000","last_peered":"2024-05-17T22:36:15.685905+0000","last_clean":"2024-05-17T22:36:15.685905+0000","last_became_active":"2024-05-17T22:34:52.483766+0000","last_became_peered":"2024-05-17T22:34:52.483766+0000","last_unstale":"2024-05-17T22:36:15.685905+0000","last_undegraded":"2024-05-17T22:36:15.685905+0000","last_fullsized":"2024-05-17T22:36:15.685905+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:32:01.427078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:43.941225+0000","last_change":"2024-05-17T22:34:37.404797+0000","last_active":"2024-05-17T22:36:43.941225+0000","last_peered":"2024-05-17T22:36:43.941225+0000","last_clean":"2024-05-17T22:36:43.941225+0000","last_became_active":"2024-05-17T22:34:37.404393+0000","last_became_peered":"2024-05-17T22:34:37.404393+0000","last_unstale":"2024-05-17T22:36:43.941225+0000","last_undegraded":"2024-05-17T22:36:43.941225+0000","last_fullsized":"2024-05-17T22:36:43.941225+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:26:36.887919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.515960+0000","last_change":"2024-05-17T22:34:40.428210+0000","last_active":"2024-05-17T22:37:14.515960+0000","last_peered":"2024-05-17T22:37:14.515960+0000","last_clean":"2024-05-17T22:37:14.515960+0000","last_became_active":"2024-05-17T22:34:40.427845+0000","last_became_peered":"2024-05-17T22:34:40.427845+0000","last_unstale":"2024-05-17T22:37:14.515960+0000","last_undegraded":"2024-05-17T22:37:14.515960+0000","last_fullsized":"2024-05-17T22:37:14.515960+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:45:30.437139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.686040+0000","last_change":"2024-05-17T22:34:52.490502+0000","last_active":"2024-05-17T22:36:15.686040+0000","last_peered":"2024-05-17T22:36:15.686040+0000","last_clean":"2024-05-17T22:36:15.686040+0000","last_became_active":"2024-05-17T22:34:52.490008+0000","last_became_peered":"2024-05-17T22:34:52.490008+0000","last_unstale":"2024-05-17T22:36:15.686040+0000","last_undegraded":"2024-05-17T22:36:15.686040+0000","last_fullsized":"2024-05-17T22:36:15.686040+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:58:29.122668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.518044+0000","last_change":"2024-05-17T22:34:40.432248+0000","last_active":"2024-05-17T22:37:14.518044+0000","last_peered":"2024-05-17T22:37:14.518044+0000","last_clean":"2024-05-17T22:37:14.518044+0000","last_became_active":"2024-05-17T22:34:40.431863+0000","last_became_peered":"2024-05-17T22:34:40.431863+0000","last_unstale":"2024-05-17T22:37:14.518044+0000","last_undegraded":"2024-05-17T22:37:14.518044+0000","last_fullsized":"2024-05-17T22:37:14.518044+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:20:34.730542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":240,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:38:57.373331+0000","last_change":"2024-05-17T22:34:37.441186+0000","last_active":"2024-05-17T22:38:57.373331+0000","last_peered":"2024-05-17T22:38:57.373331+0000","last_clean":"2024-05-17T22:38:57.373331+0000","last_became_active":"2024-05-17T22:34:37.440781+0000","last_became_peered":"2024-05-17T22:34:37.440781+0000","last_unstale":"2024-05-17T22:38:57.373331+0000","last_undegraded":"2024-05-17T22:38:57.373331+0000","last_fullsized":"2024-05-17T22:38:57.373331+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:44:34.435147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":189,"num_read_kb":189,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.005134+0000","last_change":"2024-05-17T22:34:52.491771+0000","last_active":"2024-05-17T22:36:16.005134+0000","last_peered":"2024-05-17T22:36:16.005134+0000","last_clean":"2024-05-17T22:36:16.005134+0000","last_became_active":"2024-05-17T22:34:52.491172+0000","last_became_peered":"2024-05-17T22:34:52.491172+0000","last_unstale":"2024-05-17T22:36:16.005134+0000","last_undegraded":"2024-05-17T22:36:16.005134+0000","last_fullsized":"2024-05-17T22:36:16.005134+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:15:21.010592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.514209+0000","last_change":"2024-05-17T22:34:40.430860+0000","last_active":"2024-05-17T22:37:14.514209+0000","last_peered":"2024-05-17T22:37:14.514209+0000","last_clean":"2024-05-17T22:37:14.514209+0000","last_became_active":"2024-05-17T22:34:40.430542+0000","last_became_peered":"2024-05-17T22:34:40.430542+0000","last_unstale":"2024-05-17T22:37:14.514209+0000","last_undegraded":"2024-05-17T22:37:14.514209+0000","last_fullsized":"2024-05-17T22:37:14.514209+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:16:26.257380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:42.729213+0000","last_change":"2024-05-17T22:34:37.403026+0000","last_active":"2024-05-17T22:36:42.729213+0000","last_peered":"2024-05-17T22:36:42.729213+0000","last_clean":"2024-05-17T22:36:42.729213+0000","last_became_active":"2024-05-17T22:34:37.402560+0000","last_became_peered":"2024-05-17T22:34:37.402560+0000","last_unstale":"2024-05-17T22:36:42.729213+0000","last_undegraded":"2024-05-17T22:36:42.729213+0000","last_fullsized":"2024-05-17T22:36:42.729213+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:23:03.784411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":120,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003564+0000","last_change":"2024-05-17T22:33:11.055878+0000","last_active":"2024-05-17T22:36:16.003564+0000","last_peered":"2024-05-17T22:36:16.003564+0000","last_clean":"2024-05-17T22:36:16.003564+0000","last_became_active":"2024-05-17T22:33:11.050409+0000","last_became_peered":"2024-05-17T22:33:11.050409+0000","last_unstale":"2024-05-17T22:36:16.003564+0000","last_undegraded":"2024-05-17T22:36:16.003564+0000","last_fullsized":"2024-05-17T22:36:16.003564+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:29:43.341170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:29:43.341170+0000","last_clean_scrub_stamp":"2024-05-17T22:29:43.341170+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:30:54.153865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003960+0000","last_change":"2024-05-17T22:34:52.497064+0000","last_active":"2024-05-17T22:36:16.003960+0000","last_peered":"2024-05-17T22:36:16.003960+0000","last_clean":"2024-05-17T22:36:16.003960+0000","last_became_active":"2024-05-17T22:34:52.496611+0000","last_became_peered":"2024-05-17T22:34:52.496611+0000","last_unstale":"2024-05-17T22:36:16.003960+0000","last_undegraded":"2024-05-17T22:36:16.003960+0000","last_fullsized":"2024-05-17T22:36:16.003960+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:20:29.597589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.519893+0000","last_change":"2024-05-17T22:35:15.555413+0000","last_active":"2024-05-17T22:37:14.519893+0000","last_peered":"2024-05-17T22:37:14.519893+0000","last_clean":"2024-05-17T22:37:14.519893+0000","last_became_active":"2024-05-17T22:35:15.555005+0000","last_became_peered":"2024-05-17T22:35:15.555005+0000","last_unstale":"2024-05-17T22:37:14.519893+0000","last_undegraded":"2024-05-17T22:37:14.519893+0000","last_fullsized":"2024-05-17T22:37:14.519893+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:51:49.860673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:38.940738+0000","last_change":"2024-05-17T22:34:37.406546+0000","last_active":"2024-05-17T22:36:38.940738+0000","last_peered":"2024-05-17T22:36:38.940738+0000","last_clean":"2024-05-17T22:36:38.940738+0000","last_became_active":"2024-05-17T22:34:37.406118+0000","last_became_peered":"2024-05-17T22:34:37.406118+0000","last_unstale":"2024-05-17T22:36:38.940738+0000","last_undegraded":"2024-05-17T22:36:38.940738+0000","last_fullsized":"2024-05-17T22:36:38.940738+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:47:16.350091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:56.856451+0000","last_change":"2024-05-17T22:36:16.005647+0000","last_active":"2024-05-17T22:36:56.856451+0000","last_peered":"2024-05-17T22:36:56.856451+0000","last_clean":"2024-05-17T22:36:56.856451+0000","last_became_active":"2024-05-17T22:36:16.005128+0000","last_became_peered":"2024-05-17T22:36:16.005128+0000","last_unstale":"2024-05-17T22:36:56.856451+0000","last_undegraded":"2024-05-17T22:36:56.856451+0000","last_fullsized":"2024-05-17T22:36:56.856451+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:16:24.481236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,7,6],"acting":[11,10,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:45.570153+0000","last_change":"2024-05-17T22:34:37.442676+0000","last_active":"2024-05-17T22:36:45.570153+0000","last_peered":"2024-05-17T22:36:45.570153+0000","last_clean":"2024-05-17T22:36:45.570153+0000","last_became_active":"2024-05-17T22:34:37.442275+0000","last_became_peered":"2024-05-17T22:34:37.442275+0000","last_unstale":"2024-05-17T22:36:45.570153+0000","last_undegraded":"2024-05-17T22:36:45.570153+0000","last_fullsized":"2024-05-17T22:36:45.570153+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:54:18.542987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.519824+0000","last_change":"2024-05-17T22:34:40.424266+0000","last_active":"2024-05-17T22:37:14.519824+0000","last_peered":"2024-05-17T22:37:14.519824+0000","last_clean":"2024-05-17T22:37:14.519824+0000","last_became_active":"2024-05-17T22:34:40.423905+0000","last_became_peered":"2024-05-17T22:34:40.423905+0000","last_unstale":"2024-05-17T22:37:14.519824+0000","last_undegraded":"2024-05-17T22:37:14.519824+0000","last_fullsized":"2024-05-17T22:37:14.519824+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:03:43.397871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:52.770677+0000","last_change":"2024-05-17T22:34:52.493386+0000","last_active":"2024-05-17T22:36:52.770677+0000","last_peered":"2024-05-17T22:36:52.770677+0000","last_clean":"2024-05-17T22:36:52.770677+0000","last_became_active":"2024-05-17T22:34:52.492997+0000","last_became_peered":"2024-05-17T22:34:52.492997+0000","last_unstale":"2024-05-17T22:36:52.770677+0000","last_undegraded":"2024-05-17T22:36:52.770677+0000","last_fullsized":"2024-05-17T22:36:52.770677+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:14:40.818271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'13","reported_seq":59,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:38:55.469975+0000","last_change":"2024-05-17T22:34:37.399397+0000","last_active":"2024-05-17T22:38:55.469975+0000","last_peered":"2024-05-17T22:38:55.469975+0000","last_clean":"2024-05-17T22:38:55.469975+0000","last_became_active":"2024-05-17T22:34:37.399062+0000","last_became_peered":"2024-05-17T22:34:37.399062+0000","last_unstale":"2024-05-17T22:38:55.469975+0000","last_undegraded":"2024-05-17T22:38:55.469975+0000","last_fullsized":"2024-05-17T22:38:55.469975+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:02:05.230580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.518907+0000","last_change":"2024-05-17T22:34:40.426995+0000","last_active":"2024-05-17T22:37:14.518907+0000","last_peered":"2024-05-17T22:37:14.518907+0000","last_clean":"2024-05-17T22:37:14.518907+0000","last_became_active":"2024-05-17T22:34:40.426226+0000","last_became_peered":"2024-05-17T22:34:40.426226+0000","last_unstale":"2024-05-17T22:37:14.518907+0000","last_undegraded":"2024-05-17T22:37:14.518907+0000","last_fullsized":"2024-05-17T22:37:14.518907+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:49:53.331076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:57.386510+0000","last_change":"2024-05-17T22:34:52.487098+0000","last_active":"2024-05-17T22:36:57.386510+0000","last_peered":"2024-05-17T22:36:57.386510+0000","last_clean":"2024-05-17T22:36:57.386510+0000","last_became_active":"2024-05-17T22:34:52.486605+0000","last_became_peered":"2024-05-17T22:34:52.486605+0000","last_unstale":"2024-05-17T22:36:57.386510+0000","last_undegraded":"2024-05-17T22:36:57.386510+0000","last_fullsized":"2024-05-17T22:36:57.386510+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:33:28.234208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:47.729708+0000","last_change":"2024-05-17T22:34:37.800830+0000","last_active":"2024-05-17T22:36:47.729708+0000","last_peered":"2024-05-17T22:36:47.729708+0000","last_clean":"2024-05-17T22:36:47.729708+0000","last_became_active":"2024-05-17T22:34:37.800405+0000","last_became_peered":"2024-05-17T22:34:37.800405+0000","last_unstale":"2024-05-17T22:36:47.729708+0000","last_undegraded":"2024-05-17T22:36:47.729708+0000","last_fullsized":"2024-05-17T22:36:47.729708+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:19:10.087442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.520738+0000","last_change":"2024-05-17T22:34:40.426307+0000","last_active":"2024-05-17T22:37:14.520738+0000","last_peered":"2024-05-17T22:37:14.520738+0000","last_clean":"2024-05-17T22:37:14.520738+0000","last_became_active":"2024-05-17T22:34:40.425969+0000","last_became_peered":"2024-05-17T22:34:40.425969+0000","last_unstale":"2024-05-17T22:37:14.520738+0000","last_undegraded":"2024-05-17T22:37:14.520738+0000","last_fullsized":"2024-05-17T22:37:14.520738+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:25:49.680886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004511+0000","last_change":"2024-05-17T22:35:15.556857+0000","last_active":"2024-05-17T22:36:16.004511+0000","last_peered":"2024-05-17T22:36:16.004511+0000","last_clean":"2024-05-17T22:36:16.004511+0000","last_became_active":"2024-05-17T22:35:15.556298+0000","last_became_peered":"2024-05-17T22:35:15.556298+0000","last_unstale":"2024-05-17T22:36:16.004511+0000","last_undegraded":"2024-05-17T22:36:16.004511+0000","last_fullsized":"2024-05-17T22:36:16.004511+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:23:20.111544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:47.852816+0000","last_change":"2024-05-17T22:34:37.442971+0000","last_active":"2024-05-17T22:36:47.852816+0000","last_peered":"2024-05-17T22:36:47.852816+0000","last_clean":"2024-05-17T22:36:47.852816+0000","last_became_active":"2024-05-17T22:34:37.442536+0000","last_became_peered":"2024-05-17T22:34:37.442536+0000","last_unstale":"2024-05-17T22:36:47.852816+0000","last_undegraded":"2024-05-17T22:36:47.852816+0000","last_fullsized":"2024-05-17T22:36:47.852816+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:03:45.224071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.500911+0000","last_change":"2024-05-17T22:34:40.432056+0000","last_active":"2024-05-17T22:37:14.500911+0000","last_peered":"2024-05-17T22:37:14.500911+0000","last_clean":"2024-05-17T22:37:14.500911+0000","last_became_active":"2024-05-17T22:34:40.431626+0000","last_became_peered":"2024-05-17T22:34:40.431626+0000","last_unstale":"2024-05-17T22:37:14.500911+0000","last_undegraded":"2024-05-17T22:37:14.500911+0000","last_fullsized":"2024-05-17T22:37:14.500911+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:12:24.650746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.623664+0000","last_change":"2024-05-17T22:34:52.500451+0000","last_active":"2024-05-17T22:36:15.623664+0000","last_peered":"2024-05-17T22:36:15.623664+0000","last_clean":"2024-05-17T22:36:15.623664+0000","last_became_active":"2024-05-17T22:34:52.500031+0000","last_became_peered":"2024-05-17T22:34:52.500031+0000","last_unstale":"2024-05-17T22:36:15.623664+0000","last_undegraded":"2024-05-17T22:36:15.623664+0000","last_fullsized":"2024-05-17T22:36:15.623664+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:54:36.825116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:47.770054+0000","last_change":"2024-05-17T22:34:37.402335+0000","last_active":"2024-05-17T22:36:47.770054+0000","last_peered":"2024-05-17T22:36:47.770054+0000","last_clean":"2024-05-17T22:36:47.770054+0000","last_became_active":"2024-05-17T22:34:37.401982+0000","last_became_peered":"2024-05-17T22:34:37.401982+0000","last_unstale":"2024-05-17T22:36:47.770054+0000","last_undegraded":"2024-05-17T22:36:47.770054+0000","last_fullsized":"2024-05-17T22:36:47.770054+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:58:06.194335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.519857+0000","last_change":"2024-05-17T22:34:40.425524+0000","last_active":"2024-05-17T22:37:14.519857+0000","last_peered":"2024-05-17T22:37:14.519857+0000","last_clean":"2024-05-17T22:37:14.519857+0000","last_became_active":"2024-05-17T22:34:40.425135+0000","last_became_peered":"2024-05-17T22:34:40.425135+0000","last_unstale":"2024-05-17T22:37:14.519857+0000","last_undegraded":"2024-05-17T22:37:14.519857+0000","last_fullsized":"2024-05-17T22:37:14.519857+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:41:14.176477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004843+0000","last_change":"2024-05-17T22:34:52.496290+0000","last_active":"2024-05-17T22:36:16.004843+0000","last_peered":"2024-05-17T22:36:16.004843+0000","last_clean":"2024-05-17T22:36:16.004843+0000","last_became_active":"2024-05-17T22:34:52.495817+0000","last_became_peered":"2024-05-17T22:34:52.495817+0000","last_unstale":"2024-05-17T22:36:16.004843+0000","last_undegraded":"2024-05-17T22:36:16.004843+0000","last_fullsized":"2024-05-17T22:36:16.004843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:47:22.056372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'1","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:47.385104+0000","last_change":"2024-05-17T22:34:37.404234+0000","last_active":"2024-05-17T22:36:47.385104+0000","last_peered":"2024-05-17T22:36:47.385104+0000","last_clean":"2024-05-17T22:36:47.385104+0000","last_became_active":"2024-05-17T22:34:37.403561+0000","last_became_peered":"2024-05-17T22:34:37.403561+0000","last_unstale":"2024-05-17T22:36:47.385104+0000","last_undegraded":"2024-05-17T22:36:47.385104+0000","last_fullsized":"2024-05-17T22:36:47.385104+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:30:28.432641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.517802+0000","last_change":"2024-05-17T22:34:40.426522+0000","last_active":"2024-05-17T22:37:14.517802+0000","last_peered":"2024-05-17T22:37:14.517802+0000","last_clean":"2024-05-17T22:37:14.517802+0000","last_became_active":"2024-05-17T22:34:40.426139+0000","last_became_peered":"2024-05-17T22:34:40.426139+0000","last_unstale":"2024-05-17T22:37:14.517802+0000","last_undegraded":"2024-05-17T22:37:14.517802+0000","last_fullsized":"2024-05-17T22:37:14.517802+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:30:19.503513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004076+0000","last_change":"2024-05-17T22:34:52.487131+0000","last_active":"2024-05-17T22:36:16.004076+0000","last_peered":"2024-05-17T22:36:16.004076+0000","last_clean":"2024-05-17T22:36:16.004076+0000","last_became_active":"2024-05-17T22:34:52.486654+0000","last_became_peered":"2024-05-17T22:34:52.486654+0000","last_unstale":"2024-05-17T22:36:16.004076+0000","last_undegraded":"2024-05-17T22:36:16.004076+0000","last_fullsized":"2024-05-17T22:36:16.004076+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:28:25.041746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.625893+0000","last_change":"2024-05-17T22:34:37.405020+0000","last_active":"2024-05-17T22:36:15.625893+0000","last_peered":"2024-05-17T22:36:15.625893+0000","last_clean":"2024-05-17T22:36:15.625893+0000","last_became_active":"2024-05-17T22:34:37.404633+0000","last_became_peered":"2024-05-17T22:34:37.404633+0000","last_unstale":"2024-05-17T22:36:15.625893+0000","last_undegraded":"2024-05-17T22:36:15.625893+0000","last_fullsized":"2024-05-17T22:36:15.625893+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:06:00.001190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.525054+0000","last_change":"2024-05-17T22:34:40.435374+0000","last_active":"2024-05-17T22:37:14.525054+0000","last_peered":"2024-05-17T22:37:14.525054+0000","last_clean":"2024-05-17T22:37:14.525054+0000","last_became_active":"2024-05-17T22:34:40.434522+0000","last_became_peered":"2024-05-17T22:34:40.434522+0000","last_unstale":"2024-05-17T22:37:14.525054+0000","last_undegraded":"2024-05-17T22:37:14.525054+0000","last_fullsized":"2024-05-17T22:37:14.525054+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:36:41.244970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.805274+0000","last_change":"2024-05-17T22:34:52.491708+0000","last_active":"2024-05-17T22:36:15.805274+0000","last_peered":"2024-05-17T22:36:15.805274+0000","last_clean":"2024-05-17T22:36:15.805274+0000","last_became_active":"2024-05-17T22:34:52.491293+0000","last_became_peered":"2024-05-17T22:34:52.491293+0000","last_unstale":"2024-05-17T22:36:15.805274+0000","last_undegraded":"2024-05-17T22:36:15.805274+0000","last_fullsized":"2024-05-17T22:36:15.805274+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:23:01.634026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.624679+0000","last_change":"2024-05-17T22:34:37.800525+0000","last_active":"2024-05-17T22:36:15.624679+0000","last_peered":"2024-05-17T22:36:15.624679+0000","last_clean":"2024-05-17T22:36:15.624679+0000","last_became_active":"2024-05-17T22:34:37.800093+0000","last_became_peered":"2024-05-17T22:34:37.800093+0000","last_unstale":"2024-05-17T22:36:15.624679+0000","last_undegraded":"2024-05-17T22:36:15.624679+0000","last_fullsized":"2024-05-17T22:36:15.624679+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:25:27.249889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.524773+0000","last_change":"2024-05-17T22:34:40.434891+0000","last_active":"2024-05-17T22:37:14.524773+0000","last_peered":"2024-05-17T22:37:14.524773+0000","last_clean":"2024-05-17T22:37:14.524773+0000","last_became_active":"2024-05-17T22:34:40.434543+0000","last_became_peered":"2024-05-17T22:34:40.434543+0000","last_unstale":"2024-05-17T22:37:14.524773+0000","last_undegraded":"2024-05-17T22:37:14.524773+0000","last_fullsized":"2024-05-17T22:37:14.524773+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:17:47.770225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.968412+0000","last_change":"2024-05-17T22:34:52.491544+0000","last_active":"2024-05-17T22:36:15.968412+0000","last_peered":"2024-05-17T22:36:15.968412+0000","last_clean":"2024-05-17T22:36:15.968412+0000","last_became_active":"2024-05-17T22:34:52.491138+0000","last_became_peered":"2024-05-17T22:34:52.491138+0000","last_unstale":"2024-05-17T22:36:15.968412+0000","last_undegraded":"2024-05-17T22:36:15.968412+0000","last_fullsized":"2024-05-17T22:36:15.968412+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:34:20.411299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.004302+0000","last_change":"2024-05-17T22:34:37.403220+0000","last_active":"2024-05-17T22:36:16.004302+0000","last_peered":"2024-05-17T22:36:16.004302+0000","last_clean":"2024-05-17T22:36:16.004302+0000","last_became_active":"2024-05-17T22:34:37.402856+0000","last_became_peered":"2024-05-17T22:34:37.402856+0000","last_unstale":"2024-05-17T22:36:16.004302+0000","last_undegraded":"2024-05-17T22:36:16.004302+0000","last_fullsized":"2024-05-17T22:36:16.004302+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:04:19.469294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.511989+0000","last_change":"2024-05-17T22:34:40.434487+0000","last_active":"2024-05-17T22:37:14.511989+0000","last_peered":"2024-05-17T22:37:14.511989+0000","last_clean":"2024-05-17T22:37:14.511989+0000","last_became_active":"2024-05-17T22:34:40.434160+0000","last_became_peered":"2024-05-17T22:34:40.434160+0000","last_unstale":"2024-05-17T22:37:14.511989+0000","last_undegraded":"2024-05-17T22:37:14.511989+0000","last_fullsized":"2024-05-17T22:37:14.511989+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:24:38.782999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:16.003180+0000","last_change":"2024-05-17T22:34:52.487887+0000","last_active":"2024-05-17T22:36:16.003180+0000","last_peered":"2024-05-17T22:36:16.003180+0000","last_clean":"2024-05-17T22:36:16.003180+0000","last_became_active":"2024-05-17T22:34:52.486837+0000","last_became_peered":"2024-05-17T22:34:52.486837+0000","last_unstale":"2024-05-17T22:36:16.003180+0000","last_undegraded":"2024-05-17T22:36:16.003180+0000","last_fullsized":"2024-05-17T22:36:16.003180+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:51.460688+0000","last_clean_scrub_stamp":"2024-05-17T22:34:51.460688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:12:22.206536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:36:15.686208+0000","last_change":"2024-05-17T22:34:37.399498+0000","last_active":"2024-05-17T22:36:15.686208+0000","last_peered":"2024-05-17T22:36:15.686208+0000","last_clean":"2024-05-17T22:36:15.686208+0000","last_became_active":"2024-05-17T22:34:37.399144+0000","last_became_peered":"2024-05-17T22:34:37.399144+0000","last_unstale":"2024-05-17T22:36:15.686208+0000","last_undegraded":"2024-05-17T22:36:15.686208+0000","last_fullsized":"2024-05-17T22:36:15.686208+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:36.370397+0000","last_clean_scrub_stamp":"2024-05-17T22:34:36.370397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:13:37.346006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.514102+0000","last_change":"2024-05-17T22:34:40.433839+0000","last_active":"2024-05-17T22:37:14.514102+0000","last_peered":"2024-05-17T22:37:14.514102+0000","last_clean":"2024-05-17T22:37:14.514102+0000","last_became_active":"2024-05-17T22:34:40.433489+0000","last_became_peered":"2024-05-17T22:34:40.433489+0000","last_unstale":"2024-05-17T22:37:14.514102+0000","last_undegraded":"2024-05-17T22:37:14.514102+0000","last_fullsized":"2024-05-17T22:37:14.514102+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:13:56.014094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.512654+0000","last_change":"2024-05-17T22:34:40.434953+0000","last_active":"2024-05-17T22:37:14.512654+0000","last_peered":"2024-05-17T22:37:14.512654+0000","last_clean":"2024-05-17T22:37:14.512654+0000","last_became_active":"2024-05-17T22:34:40.434593+0000","last_became_peered":"2024-05-17T22:34:40.434593+0000","last_unstale":"2024-05-17T22:37:14.512654+0000","last_undegraded":"2024-05-17T22:37:14.512654+0000","last_fullsized":"2024-05-17T22:37:14.512654+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:20:27.075382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.513666+0000","last_change":"2024-05-17T22:34:40.432787+0000","last_active":"2024-05-17T22:37:14.513666+0000","last_peered":"2024-05-17T22:37:14.513666+0000","last_clean":"2024-05-17T22:37:14.513666+0000","last_became_active":"2024-05-17T22:34:40.432402+0000","last_became_peered":"2024-05-17T22:34:40.432402+0000","last_unstale":"2024-05-17T22:37:14.513666+0000","last_undegraded":"2024-05-17T22:37:14.513666+0000","last_fullsized":"2024-05-17T22:37:14.513666+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:48:13.034276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.515714+0000","last_change":"2024-05-17T22:34:40.426423+0000","last_active":"2024-05-17T22:37:14.515714+0000","last_peered":"2024-05-17T22:37:14.515714+0000","last_clean":"2024-05-17T22:37:14.515714+0000","last_became_active":"2024-05-17T22:34:40.426049+0000","last_became_peered":"2024-05-17T22:34:40.426049+0000","last_unstale":"2024-05-17T22:37:14.515714+0000","last_undegraded":"2024-05-17T22:37:14.515714+0000","last_fullsized":"2024-05-17T22:37:14.515714+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:56:55.631322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.520890+0000","last_change":"2024-05-17T22:34:40.435706+0000","last_active":"2024-05-17T22:37:14.520890+0000","last_peered":"2024-05-17T22:37:14.520890+0000","last_clean":"2024-05-17T22:37:14.520890+0000","last_became_active":"2024-05-17T22:34:40.435330+0000","last_became_peered":"2024-05-17T22:34:40.435330+0000","last_unstale":"2024-05-17T22:37:14.520890+0000","last_undegraded":"2024-05-17T22:37:14.520890+0000","last_fullsized":"2024-05-17T22:37:14.520890+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:25:36.529454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.517285+0000","last_change":"2024-05-17T22:34:40.428997+0000","last_active":"2024-05-17T22:37:14.517285+0000","last_peered":"2024-05-17T22:37:14.517285+0000","last_clean":"2024-05-17T22:37:14.517285+0000","last_became_active":"2024-05-17T22:34:40.428594+0000","last_became_peered":"2024-05-17T22:34:40.428594+0000","last_unstale":"2024-05-17T22:37:14.517285+0000","last_undegraded":"2024-05-17T22:37:14.517285+0000","last_fullsized":"2024-05-17T22:37:14.517285+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:59:01.777751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.510746+0000","last_change":"2024-05-17T22:34:40.434241+0000","last_active":"2024-05-17T22:37:14.510746+0000","last_peered":"2024-05-17T22:37:14.510746+0000","last_clean":"2024-05-17T22:37:14.510746+0000","last_became_active":"2024-05-17T22:34:40.431652+0000","last_became_peered":"2024-05-17T22:34:40.431652+0000","last_unstale":"2024-05-17T22:37:14.510746+0000","last_undegraded":"2024-05-17T22:37:14.510746+0000","last_fullsized":"2024-05-17T22:37:14.510746+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:33:27.035284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.513962+0000","last_change":"2024-05-17T22:34:40.431485+0000","last_active":"2024-05-17T22:37:14.513962+0000","last_peered":"2024-05-17T22:37:14.513962+0000","last_clean":"2024-05-17T22:37:14.513962+0000","last_became_active":"2024-05-17T22:34:40.431120+0000","last_became_peered":"2024-05-17T22:34:40.431120+0000","last_unstale":"2024-05-17T22:37:14.513962+0000","last_undegraded":"2024-05-17T22:37:14.513962+0000","last_fullsized":"2024-05-17T22:37:14.513962+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:46:55.213602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.501567+0000","last_change":"2024-05-17T22:34:40.423117+0000","last_active":"2024-05-17T22:37:14.501567+0000","last_peered":"2024-05-17T22:37:14.501567+0000","last_clean":"2024-05-17T22:37:14.501567+0000","last_became_active":"2024-05-17T22:34:40.422694+0000","last_became_peered":"2024-05-17T22:34:40.422694+0000","last_unstale":"2024-05-17T22:37:14.501567+0000","last_undegraded":"2024-05-17T22:37:14.501567+0000","last_fullsized":"2024-05-17T22:37:14.501567+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:19:55.589532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.518389+0000","last_change":"2024-05-17T22:35:15.711796+0000","last_active":"2024-05-17T22:37:14.518389+0000","last_peered":"2024-05-17T22:37:14.518389+0000","last_clean":"2024-05-17T22:37:14.518389+0000","last_became_active":"2024-05-17T22:35:15.711327+0000","last_became_peered":"2024-05-17T22:35:15.711327+0000","last_unstale":"2024-05-17T22:37:14.518389+0000","last_undegraded":"2024-05-17T22:37:14.518389+0000","last_fullsized":"2024-05-17T22:37:14.518389+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:48:48.913312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.512607+0000","last_change":"2024-05-17T22:34:40.430393+0000","last_active":"2024-05-17T22:37:14.512607+0000","last_peered":"2024-05-17T22:37:14.512607+0000","last_clean":"2024-05-17T22:37:14.512607+0000","last_became_active":"2024-05-17T22:34:40.430030+0000","last_became_peered":"2024-05-17T22:34:40.430030+0000","last_unstale":"2024-05-17T22:37:14.512607+0000","last_undegraded":"2024-05-17T22:37:14.512607+0000","last_fullsized":"2024-05-17T22:37:14.512607+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:39:37.204246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.526052+0000","last_change":"2024-05-17T22:34:40.432443+0000","last_active":"2024-05-17T22:37:14.526052+0000","last_peered":"2024-05-17T22:37:14.526052+0000","last_clean":"2024-05-17T22:37:14.526052+0000","last_became_active":"2024-05-17T22:34:40.431991+0000","last_became_peered":"2024-05-17T22:34:40.431991+0000","last_unstale":"2024-05-17T22:37:14.526052+0000","last_undegraded":"2024-05-17T22:37:14.526052+0000","last_fullsized":"2024-05-17T22:37:14.526052+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:37:45.593575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.523169+0000","last_change":"2024-05-17T22:34:40.434388+0000","last_active":"2024-05-17T22:37:14.523169+0000","last_peered":"2024-05-17T22:37:14.523169+0000","last_clean":"2024-05-17T22:37:14.523169+0000","last_became_active":"2024-05-17T22:34:40.434044+0000","last_became_peered":"2024-05-17T22:34:40.434044+0000","last_unstale":"2024-05-17T22:37:14.523169+0000","last_undegraded":"2024-05-17T22:37:14.523169+0000","last_fullsized":"2024-05-17T22:37:14.523169+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:20:43.296190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T22:37:14.507137+0000","last_change":"2024-05-17T22:34:40.427137+0000","last_active":"2024-05-17T22:37:14.507137+0000","last_peered":"2024-05-17T22:37:14.507137+0000","last_clean":"2024-05-17T22:37:14.507137+0000","last_became_active":"2024-05-17T22:34:40.426703+0000","last_became_peered":"2024-05-17T22:34:40.426703+0000","last_unstale":"2024-05-17T22:37:14.507137+0000","last_undegraded":"2024-05-17T22:37:14.507137+0000","last_fullsized":"2024-05-17T22:37:14.507137+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:34:39.393951+0000","last_clean_scrub_stamp":"2024-05-17T22:34:39.393951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:55:52.992817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72984,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1175,"num_read_kb":665,"num_write":185,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":438272,"data_stored":213579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710789,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":515036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907018,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28696,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716456,"statfs":{"total":95995035648,"available":95965650944,"internally_reserved":0,"allocated":1720320,"data_stored":1170816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070541,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":519346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266773,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":515148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863463001,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1634304,"data_stored":1112060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626525,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":510426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574968,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1089536,"data_stored":572965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738493,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":511814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411444,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1118208,"data_stored":580528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182510,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":510620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986280,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":515608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822757,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36940,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708212,"statfs":{"total":95995035648,"available":95957209088,"internally_reserved":0,"allocated":1642496,"data_stored":1107116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":70192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T22:39:02.644 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T22:39:02.931 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:39:02.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:02 smithi151 ceph-mon[36101]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T22:39:03.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:02 smithi096 ceph-mon[36065]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T22:39:03.053 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:02 smithi012 ceph-mon[31632]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T22:39:03.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:03 smithi151 ceph-mon[36101]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T22:39:03.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:03 smithi151 ceph-mon[36101]: pgmap v433: 129 pgs: 129 active+clean; 648 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-05-17T22:39:03.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:03 smithi151 ceph-mon[36101]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T22:39:04.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:03 smithi096 ceph-mon[36065]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T22:39:04.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:03 smithi096 ceph-mon[36065]: pgmap v433: 129 pgs: 129 active+clean; 648 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-05-17T22:39:04.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:03 smithi096 ceph-mon[36065]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T22:39:04.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:03 smithi012 ceph-mon[31632]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T22:39:04.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:03 smithi012 ceph-mon[31632]: pgmap v433: 129 pgs: 129 active+clean; 648 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-05-17T22:39:04.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:03 smithi012 ceph-mon[31632]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T22:39:04.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:04.802+0000 7f17b87ea700 1 -- 172.21.15.12:0/1678180244 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17b0105790 msgr2=0x7f17b010dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:04.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:04.802+0000 7f17b87ea700 1 --2- 172.21.15.12:0/1678180244 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17b0105790 0x7f17b010dce0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f17a0009990 tx=0x7f17a0009ca0 comp rx=0 tx=0).stop 2024-05-17T22:39:04.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:04.802+0000 7f17b87ea700 1 -- 172.21.15.12:0/1678180244 shutdown_connections 2024-05-17T22:39:04.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:04.802+0000 7f17b87ea700 1 --2- 172.21.15.12:0/1678180244 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17b010e270 0x7f17b0110740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:04.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:04.802+0000 7f17b87ea700 1 --2- 172.21.15.12:0/1678180244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17b0104e40 0x7f17b0105250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:04.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:04.802+0000 7f17b87ea700 1 --2- 172.21.15.12:0/1678180244 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17b0105790 0x7f17b010dce0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:04.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:04.802+0000 7f17b87ea700 1 -- 172.21.15.12:0/1678180244 >> 172.21.15.12:0/1678180244 conn(0x7f17b0079bd0 msgr2=0x7f17b0079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:39:04.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:04.802+0000 7f17b87ea700 1 -- 172.21.15.12:0/1678180244 shutdown_connections 2024-05-17T22:39:04.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:04.802+0000 7f17b87ea700 1 -- 172.21.15.12:0/1678180244 wait complete. 2024-05-17T22:39:04.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:04.803+0000 7f17b87ea700 1 Processor -- start 2024-05-17T22:39:04.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:04.803+0000 7f17b87ea700 1 -- start start 2024-05-17T22:39:04.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:04.803+0000 7f17b87ea700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17b0104e40 0x7f17b019a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:04.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:04.803+0000 7f17b87ea700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17b0105790 0x7f17b019ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:04.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:04.803+0000 7f17b87ea700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17b010e270 0x7f17b019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:04.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:04.803+0000 7f17b87ea700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f17b0113db0 con 0x7f17b0105790 2024-05-17T22:39:04.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:04.803+0000 7f17b87ea700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f17b0113c30 con 0x7f17b0104e40 2024-05-17T22:39:04.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:04.803+0000 7f17b87ea700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f17b0113f30 con 0x7f17b010e270 2024-05-17T22:39:04.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:04.804+0000 7f17b6586700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17b0104e40 0x7f17b019a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:04.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:04.804+0000 7f17b5d85700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17b0105790 0x7f17b019ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:04.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:04.804+0000 7f17b6586700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17b0104e40 0x7f17b019a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:56150/0 (socket says 172.21.15.12:56150) 2024-05-17T22:39:04.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:04.804+0000 7f17b6586700 1 -- 172.21.15.12:0/313726507 learned_addr learned my addr 172.21.15.12:0/313726507 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:39:04.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:04.804+0000 7f17b6d87700 1 --2- 172.21.15.12:0/313726507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17b010e270 0x7f17b019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:04.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:04.804+0000 7f17b5d85700 1 -- 172.21.15.12:0/313726507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17b010e270 msgr2=0x7f17b019efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:04.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:04.804+0000 7f17b5d85700 1 --2- 172.21.15.12:0/313726507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17b010e270 0x7f17b019efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:04.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:04.804+0000 7f17b5d85700 1 -- 172.21.15.12:0/313726507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17b0104e40 msgr2=0x7f17b019a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:04.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:04.804+0000 7f17b5d85700 1 --2- 172.21.15.12:0/313726507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17b0104e40 0x7f17b019a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:04.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:04.804+0000 7f17b5d85700 1 -- 172.21.15.12:0/313726507 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17b019f640 con 0x7f17b0105790 2024-05-17T22:39:04.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:04.804+0000 7f17b6586700 1 --2- 172.21.15.12:0/313726507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17b0104e40 0x7f17b019a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:39:04.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:04.804+0000 7f17b5d85700 1 --2- 172.21.15.12:0/313726507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17b0105790 0x7f17b019ad10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f17a00059d0 tx=0x7f17a00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:39:04.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:04.804+0000 7f17a77fe700 1 -- 172.21.15.12:0/313726507 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a0027070 con 0x7f17b0105790 2024-05-17T22:39:04.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:04.804+0000 7f17a77fe700 1 -- 172.21.15.12:0/313726507 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17a0003f20 con 0x7f17b0105790 2024-05-17T22:39:04.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:04.804+0000 7f17b87ea700 1 -- 172.21.15.12:0/313726507 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17b019f8d0 con 0x7f17b0105790 2024-05-17T22:39:04.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:04.804+0000 7f17a77fe700 1 -- 172.21.15.12:0/313726507 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a0022ad0 con 0x7f17b0105790 2024-05-17T22:39:04.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:04.805+0000 7f17b87ea700 1 -- 172.21.15.12:0/313726507 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17b019fdd0 con 0x7f17b0105790 2024-05-17T22:39:04.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:04.805+0000 7f17b87ea700 1 -- 172.21.15.12:0/313726507 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17b0105250 con 0x7f17b0105790 2024-05-17T22:39:04.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:04.808+0000 7f17a77fe700 1 -- 172.21.15.12:0/313726507 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f17a002a070 con 0x7f17b0105790 2024-05-17T22:39:04.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:04.809+0000 7f17a77fe700 1 --2- 172.21.15.12:0/313726507 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f179c0b2900 0x7f179c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:04.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:04.809+0000 7f17b6586700 1 --2- 172.21.15.12:0/313726507 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f179c0b2900 0x7f179c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:04.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:04.809+0000 7f17a77fe700 1 -- 172.21.15.12:0/313726507 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f17a00e39a0 con 0x7f17b0105790 2024-05-17T22:39:04.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:04.810+0000 7f17b6586700 1 --2- 172.21.15.12:0/313726507 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f179c0b2900 0x7f179c0b4e10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f17ac005950 tx=0x7f17ac00b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:39:04.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:04.812+0000 7f17a77fe700 1 -- 172.21.15.12:0/313726507 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f17b019fdd0 con 0x7f17b0105790 2024-05-17T22:39:05.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:05.072+0000 7f17b87ea700 1 -- 172.21.15.12:0/313726507 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17b019bad0 con 0x7f17b0105790 2024-05-17T22:39:05.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:05.073+0000 7f17a77fe700 1 -- 172.21.15.12:0/313726507 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11870 (secure 0 0 0) 0x7f17a0013070 con 0x7f17b0105790 2024-05-17T22:39:05.076 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:39:05.076 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1407219319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1407219319},{"type":"v1","addr":"172.21.15.12:6835","nonce":1407219319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/630977886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":630977886},{"type":"v1","addr":"172.21.15.12:6837","nonce":630977886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3731914742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3731914742},{"type":"v1","addr":"172.21.15.12:6839","nonce":3731914742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/3642142501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3642142501},{"type":"v1","addr":"172.21.15.12:6841","nonce":3642142501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3554053059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3554053059},{"type":"v1","addr":"172.21.15.151:6833","nonce":3554053059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/4228325569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":4228325569},{"type":"v1","addr":"172.21.15.96:6833","nonce":4228325569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T22:34:42.642111+0000","modified":"2024-05-17T22:38:56.118632+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24461,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/542323754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":542323754},{"type":"v1","addr":"172.21.15.151:6835","nonce":542323754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1758442232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1758442232},{"type":"v1","addr":"172.21.15.96:6835","nonce":1758442232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3403324038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3403324038},{"type":"v1","addr":"172.21.15.151:6837","nonce":3403324038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.151:6839/2338353849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2338353849},{"type":"v1","addr":"172.21.15.151:6839","nonce":2338353849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.96:6837/4222574338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":4222574338},{"type":"v1","addr":"172.21.15.96:6837","nonce":4222574338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/2986295519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2986295519},{"type":"v1","addr":"172.21.15.96:6839","nonce":2986295519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24457,24461,24482]},"id":1}]} 2024-05-17T22:39:05.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:05.075+0000 7f17b87ea700 1 -- 172.21.15.12:0/313726507 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f179c0b2900 msgr2=0x7f179c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:05.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:05.075+0000 7f17b87ea700 1 --2- 172.21.15.12:0/313726507 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f179c0b2900 0x7f179c0b4e10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f17ac005950 tx=0x7f17ac00b500 comp rx=0 tx=0).stop 2024-05-17T22:39:05.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:05.075+0000 7f17b87ea700 1 -- 172.21.15.12:0/313726507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17b0105790 msgr2=0x7f17b019ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:05.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:05.075+0000 7f17b87ea700 1 --2- 172.21.15.12:0/313726507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17b0105790 0x7f17b019ad10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f17a00059d0 tx=0x7f17a00047b0 comp rx=0 tx=0).stop 2024-05-17T22:39:05.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:05.075+0000 7f17b87ea700 1 -- 172.21.15.12:0/313726507 shutdown_connections 2024-05-17T22:39:05.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:05.075+0000 7f17b87ea700 1 --2- 172.21.15.12:0/313726507 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f179c0b2900 0x7f179c0b4e10 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:05.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:05.075+0000 7f17b87ea700 1 --2- 172.21.15.12:0/313726507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17b010e270 0x7f17b019efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:05.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:05.075+0000 7f17b87ea700 1 --2- 172.21.15.12:0/313726507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17b0104e40 0x7f17b019a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:05.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:05.075+0000 7f17b87ea700 1 --2- 172.21.15.12:0/313726507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17b0105790 0x7f17b019ad10 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:05.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:05.075+0000 7f17b87ea700 1 -- 172.21.15.12:0/313726507 >> 172.21.15.12:0/313726507 conn(0x7f17b0079bd0 msgr2=0x7f17b00ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:39:05.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:05.075+0000 7f17b87ea700 1 -- 172.21.15.12:0/313726507 shutdown_connections 2024-05-17T22:39:05.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:05.075+0000 7f17b87ea700 1 -- 172.21.15.12:0/313726507 wait complete. 2024-05-17T22:39:05.079 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 27 2024-05-17T22:39:05.750 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:05 smithi012 ceph-mon[31632]: pgmap v434: 129 pgs: 129 active+clean; 648 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 938 B/s wr, 6 op/s 2024-05-17T22:39:05.750 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:05 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/313726507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:39:05.783 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-17T22:39:05.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:05 smithi151 ceph-mon[36101]: pgmap v434: 129 pgs: 129 active+clean; 648 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 938 B/s wr, 6 op/s 2024-05-17T22:39:05.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:05 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/313726507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:39:06.038 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:39:06.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:05 smithi096 ceph-mon[36065]: pgmap v434: 129 pgs: 129 active+clean; 648 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 938 B/s wr, 6 op/s 2024-05-17T22:39:06.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:05 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/313726507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:39:07.796 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:07 smithi012 ceph-mon[31632]: pgmap v435: 129 pgs: 129 active+clean; 648 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-17T22:39:07.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:07.813+0000 7f2fe9ea9700 1 -- 172.21.15.12:0/48385088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fe4108840 msgr2=0x7f2fe4108c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:07.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:07.813+0000 7f2fe9ea9700 1 --2- 172.21.15.12:0/48385088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fe4108840 0x7f2fe4108c50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8009990 tx=0x7f2fd8009ca0 comp rx=0 tx=0).stop 2024-05-17T22:39:07.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:07.813+0000 7f2fe9ea9700 1 -- 172.21.15.12:0/48385088 shutdown_connections 2024-05-17T22:39:07.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:07.813+0000 7f2fe9ea9700 1 --2- 172.21.15.12:0/48385088 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2fe4109190 0x7f2fe4110af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:07.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:07.813+0000 7f2fe9ea9700 1 --2- 172.21.15.12:0/48385088 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fe4103f10 0x7f2fe4104320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:07.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:07.813+0000 7f2fe9ea9700 1 --2- 172.21.15.12:0/48385088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fe4108840 0x7f2fe4108c50 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:07.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:07.813+0000 7f2fe9ea9700 1 -- 172.21.15.12:0/48385088 >> 172.21.15.12:0/48385088 conn(0x7f2fe40ff9b0 msgr2=0x7f2fe4101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:39:07.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:07.813+0000 7f2fe9ea9700 1 -- 172.21.15.12:0/48385088 shutdown_connections 2024-05-17T22:39:07.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:07.813+0000 7f2fe9ea9700 1 -- 172.21.15.12:0/48385088 wait complete. 2024-05-17T22:39:07.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:07.813+0000 7f2fe9ea9700 1 Processor -- start 2024-05-17T22:39:07.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:07.814+0000 7f2fe9ea9700 1 -- start start 2024-05-17T22:39:07.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:07.814+0000 7f2fe9ea9700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2fe4103f10 0x7f2fe419a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:07.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:07.814+0000 7f2fe9ea9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fe4108840 0x7f2fe419ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:07.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:07.814+0000 7f2fe9ea9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fe4109190 0x7f2fe419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:07.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:07.814+0000 7f2fe9ea9700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe4113dc0 con 0x7f2fe4109190 2024-05-17T22:39:07.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:07.814+0000 7f2fe9ea9700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe4113c40 con 0x7f2fe4108840 2024-05-17T22:39:07.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:07.814+0000 7f2fe9ea9700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe4113f40 con 0x7f2fe4103f10 2024-05-17T22:39:07.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:07.814+0000 7f2fe3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fe4109190 0x7f2fe419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:07.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:07.814+0000 7f2fe3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fe4109190 0x7f2fe419ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32804/0 (socket says 172.21.15.12:32804) 2024-05-17T22:39:07.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:07.814+0000 7f2fe3fff700 1 -- 172.21.15.12:0/921701112 learned_addr learned my addr 172.21.15.12:0/921701112 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:39:07.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:07.814+0000 7f2fe37fe700 1 --2- 172.21.15.12:0/921701112 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2fe4103f10 0x7f2fe419a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:07.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:07.814+0000 7f2fe2ffd700 1 --2- 172.21.15.12:0/921701112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fe4108840 0x7f2fe419ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:07.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:07.814+0000 7f2fe3fff700 1 -- 172.21.15.12:0/921701112 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2fe4103f10 msgr2=0x7f2fe419a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:07.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:07.814+0000 7f2fe3fff700 1 --2- 172.21.15.12:0/921701112 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2fe4103f10 0x7f2fe419a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:07.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:07.814+0000 7f2fe3fff700 1 -- 172.21.15.12:0/921701112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fe4108840 msgr2=0x7f2fe419ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:07.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:07.814+0000 7f2fe3fff700 1 --2- 172.21.15.12:0/921701112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fe4108840 0x7f2fe419ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:07.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:07.814+0000 7f2fe3fff700 1 -- 172.21.15.12:0/921701112 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fe419f580 con 0x7f2fe4109190 2024-05-17T22:39:07.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:07.815+0000 7f2fe3fff700 1 --2- 172.21.15.12:0/921701112 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fe4109190 0x7f2fe419ef90 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc007ee0 tx=0x7f2fdc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:39:07.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:07.815+0000 7f2fe0ff9700 1 -- 172.21.15.12:0/921701112 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc004bb0 con 0x7f2fe4109190 2024-05-17T22:39:07.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:07.815+0000 7f2fe0ff9700 1 -- 172.21.15.12:0/921701112 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fdc004d50 con 0x7f2fe4109190 2024-05-17T22:39:07.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:07.815+0000 7f2fe9ea9700 1 -- 172.21.15.12:0/921701112 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fe419f870 con 0x7f2fe4109190 2024-05-17T22:39:07.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:07.815+0000 7f2fe0ff9700 1 -- 172.21.15.12:0/921701112 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc005920 con 0x7f2fe4109190 2024-05-17T22:39:07.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:07.815+0000 7f2fe9ea9700 1 -- 172.21.15.12:0/921701112 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fe419fd30 con 0x7f2fe4109190 2024-05-17T22:39:07.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:07.816+0000 7f2fe9ea9700 1 -- 172.21.15.12:0/921701112 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fe4104320 con 0x7f2fe4109190 2024-05-17T22:39:07.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:07.817+0000 7f2fe0ff9700 1 -- 172.21.15.12:0/921701112 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2fdc01a020 con 0x7f2fe4109190 2024-05-17T22:39:07.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:07.819+0000 7f2fe0ff9700 1 --2- 172.21.15.12:0/921701112 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f2fcc0b2880 0x7f2fcc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:07.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:07.819+0000 7f2fe37fe700 1 --2- 172.21.15.12:0/921701112 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f2fcc0b2880 0x7f2fcc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:07.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:07.819+0000 7f2fe0ff9700 1 -- 172.21.15.12:0/921701112 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f2fdc0e2200 con 0x7f2fe4109190 2024-05-17T22:39:07.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:07.819+0000 7f2fe37fe700 1 --2- 172.21.15.12:0/921701112 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f2fcc0b2880 0x7f2fcc0b4d90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2fd400ac50 tx=0x7f2fd400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:39:07.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:07.822+0000 7f2fe0ff9700 1 -- 172.21.15.12:0/921701112 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2fdc010070 con 0x7f2fe4109190 2024-05-17T22:39:07.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:07 smithi151 ceph-mon[36101]: pgmap v435: 129 pgs: 129 active+clean; 648 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-17T22:39:08.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:08.040+0000 7f2fe9ea9700 1 -- 172.21.15.12:0/921701112 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2fe419bac0 con 0x7f2fe4109190 2024-05-17T22:39:08.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:08.042+0000 7f2fe0ff9700 1 -- 172.21.15.12:0/921701112 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24715 (secure 0 0 0) 0x7f2fdc0a9b90 con 0x7f2fe4109190 2024-05-17T22:39:08.045 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:39:08.045 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":89,"fsid":"4d593780-149c-11ef-bc9a-c7b262605968","created":"2024-05-17T22:26:16.955626+0000","modified":"2024-05-17T22:36:15.618992+0000","last_up_change":"2024-05-17T22:33:36.551093+0000","last_in_change":"2024-05-17T22:33:15.032677+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T22:29:42.475513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T22:34:35.504697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T22:34:39.377097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-17T22:34:50.467684+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5d42798f-dc5f-4e66-bce2-ec33dc789e4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6803","nonce":2015605593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6805","nonce":2015605593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6809","nonce":2015605593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6807","nonce":2015605593}]},"public_addr":"172.21.15.12:6803/2015605593","cluster_addr":"172.21.15.12:6805/2015605593","heartbeat_back_addr":"172.21.15.12:6809/2015605593","heartbeat_front_addr":"172.21.15.12:6807/2015605593","state":["exists","up"]},{"osd":1,"uuid":"58afecc7-119b-4c16-a401-640fb77d4714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6801","nonce":1238316358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6803","nonce":1238316358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6807","nonce":1238316358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6805","nonce":1238316358}]},"public_addr":"172.21.15.96:6801/1238316358","cluster_addr":"172.21.15.96:6803/1238316358","heartbeat_back_addr":"172.21.15.96:6807/1238316358","heartbeat_front_addr":"172.21.15.96:6805/1238316358","state":["exists","up"]},{"osd":2,"uuid":"4bef4049-482f-49fb-82fb-38052400eb2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6801","nonce":148729815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6803","nonce":148729815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6807","nonce":148729815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6805","nonce":148729815}]},"public_addr":"172.21.15.151:6801/148729815","cluster_addr":"172.21.15.151:6803/148729815","heartbeat_back_addr":"172.21.15.151:6807/148729815","heartbeat_front_addr":"172.21.15.151:6805/148729815","state":["exists","up"]},{"osd":3,"uuid":"6fed8e36-82a2-49e4-82a3-16e6d9de87f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6811","nonce":3563309320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6813","nonce":3563309320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6817","nonce":3563309320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6815","nonce":3563309320}]},"public_addr":"172.21.15.12:6811/3563309320","cluster_addr":"172.21.15.12:6813/3563309320","heartbeat_back_addr":"172.21.15.12:6817/3563309320","heartbeat_front_addr":"172.21.15.12:6815/3563309320","state":["exists","up"]},{"osd":4,"uuid":"c0e8689a-2816-4089-ade3-931960bae4f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6809","nonce":2219103166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6811","nonce":2219103166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6815","nonce":2219103166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6813","nonce":2219103166}]},"public_addr":"172.21.15.96:6809/2219103166","cluster_addr":"172.21.15.96:6811/2219103166","heartbeat_back_addr":"172.21.15.96:6815/2219103166","heartbeat_front_addr":"172.21.15.96:6813/2219103166","state":["exists","up"]},{"osd":5,"uuid":"e76ba432-94ff-4998-aaba-2645d9e6055d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6809","nonce":2589835078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6811","nonce":2589835078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6815","nonce":2589835078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6813","nonce":2589835078}]},"public_addr":"172.21.15.151:6809/2589835078","cluster_addr":"172.21.15.151:6811/2589835078","heartbeat_back_addr":"172.21.15.151:6815/2589835078","heartbeat_front_addr":"172.21.15.151:6813/2589835078","state":["exists","up"]},{"osd":6,"uuid":"b7ece35c-efd1-40df-95da-05a0cc147372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6819","nonce":3112661724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6821","nonce":3112661724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6825","nonce":3112661724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6823","nonce":3112661724}]},"public_addr":"172.21.15.12:6819/3112661724","cluster_addr":"172.21.15.12:6821/3112661724","heartbeat_back_addr":"172.21.15.12:6825/3112661724","heartbeat_front_addr":"172.21.15.12:6823/3112661724","state":["exists","up"]},{"osd":7,"uuid":"6a65bf30-daf5-4fba-b67e-8e513d88a971","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6817","nonce":2449989110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6819","nonce":2449989110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6823","nonce":2449989110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6821","nonce":2449989110}]},"public_addr":"172.21.15.96:6817/2449989110","cluster_addr":"172.21.15.96:6819/2449989110","heartbeat_back_addr":"172.21.15.96:6823/2449989110","heartbeat_front_addr":"172.21.15.96:6821/2449989110","state":["exists","up"]},{"osd":8,"uuid":"ad62ea44-e416-4a3e-bbbe-93f2a710dbc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6817","nonce":2545224628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6819","nonce":2545224628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6823","nonce":2545224628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6821","nonce":2545224628}]},"public_addr":"172.21.15.151:6817/2545224628","cluster_addr":"172.21.15.151:6819/2545224628","heartbeat_back_addr":"172.21.15.151:6823/2545224628","heartbeat_front_addr":"172.21.15.151:6821/2545224628","state":["exists","up"]},{"osd":9,"uuid":"1275de08-2a3c-48f3-920b-dd68b4024d42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6827","nonce":2100661273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6829","nonce":2100661273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6833","nonce":2100661273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6831","nonce":2100661273}]},"public_addr":"172.21.15.12:6827/2100661273","cluster_addr":"172.21.15.12:6829/2100661273","heartbeat_back_addr":"172.21.15.12:6833/2100661273","heartbeat_front_addr":"172.21.15.12:6831/2100661273","state":["exists","up"]},{"osd":10,"uuid":"347e4e40-9a9d-43fe-8f52-fdd3199a2a5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6825","nonce":304016309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6827","nonce":304016309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6831","nonce":304016309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6829","nonce":304016309}]},"public_addr":"172.21.15.96:6825/304016309","cluster_addr":"172.21.15.96:6827/304016309","heartbeat_back_addr":"172.21.15.96:6831/304016309","heartbeat_front_addr":"172.21.15.96:6829/304016309","state":["exists","up"]},{"osd":11,"uuid":"ee8d3502-8b3c-4e6f-824f-3a97e19111ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6825","nonce":3822307314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6827","nonce":3822307314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6831","nonce":3822307314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6829","nonce":3822307314}]},"public_addr":"172.21.15.151:6825/3822307314","cluster_addr":"172.21.15.151:6827/3822307314","heartbeat_back_addr":"172.21.15.151:6831/3822307314","heartbeat_front_addr":"172.21.15.151:6829/3822307314","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:28:51.542055+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:29:15.566424+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:29:39.500981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:04.797907+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:29.655650+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:55.211880+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:31:21.559462+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:31:47.154852+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:32:13.114892+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:32:40.529708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:33:06.930942+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:33:34.061783+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":10}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/2682700253":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6801/2250046521":"2024-05-18T22:27:14.372972+0000","172.21.15.12:0/4241094239":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/4172568611":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/3512529242":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6800/1039789215":"2024-05-18T22:26:45.466719+0000","172.21.15.12:6800/2250046521":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6801/1039789215":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/2043758453":"2024-05-18T22:27:14.372972+0000","172.21.15.12:0/2694593732":"2024-05-18T22:26:45.466719+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T22:39:08.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:08.044+0000 7f2fe9ea9700 1 -- 172.21.15.12:0/921701112 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f2fcc0b2880 msgr2=0x7f2fcc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:08.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:08.044+0000 7f2fe9ea9700 1 --2- 172.21.15.12:0/921701112 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f2fcc0b2880 0x7f2fcc0b4d90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2fd400ac50 tx=0x7f2fd400a380 comp rx=0 tx=0).stop 2024-05-17T22:39:08.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:08.044+0000 7f2fe9ea9700 1 -- 172.21.15.12:0/921701112 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fe4109190 msgr2=0x7f2fe419ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:08.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:08.044+0000 7f2fe9ea9700 1 --2- 172.21.15.12:0/921701112 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fe4109190 0x7f2fe419ef90 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc007ee0 tx=0x7f2fdc00d3b0 comp rx=0 tx=0).stop 2024-05-17T22:39:08.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:08.044+0000 7f2fe9ea9700 1 -- 172.21.15.12:0/921701112 shutdown_connections 2024-05-17T22:39:08.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:08.044+0000 7f2fe9ea9700 1 --2- 172.21.15.12:0/921701112 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f2fcc0b2880 0x7f2fcc0b4d90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:08.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:08.044+0000 7f2fe9ea9700 1 --2- 172.21.15.12:0/921701112 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fe4109190 0x7f2fe419ef90 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:08.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:08.044+0000 7f2fe9ea9700 1 --2- 172.21.15.12:0/921701112 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2fe4103f10 0x7f2fe419a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:08.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:08.044+0000 7f2fe9ea9700 1 --2- 172.21.15.12:0/921701112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fe4108840 0x7f2fe419ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:08.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:08.044+0000 7f2fe9ea9700 1 -- 172.21.15.12:0/921701112 >> 172.21.15.12:0/921701112 conn(0x7f2fe40ff9b0 msgr2=0x7f2fe4101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:39:08.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:08.045+0000 7f2fe9ea9700 1 -- 172.21.15.12:0/921701112 shutdown_connections 2024-05-17T22:39:08.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:08.045+0000 7f2fe9ea9700 1 -- 172.21.15.12:0/921701112 wait complete. 2024-05-17T22:39:08.053 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:07 smithi096 ceph-mon[36065]: pgmap v435: 129 pgs: 129 active+clean; 648 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-17T22:39:08.754 DEBUG:tasks.fwd_scrub:Yielding 2024-05-17T22:39:08.754 INFO:teuthology.run_tasks:Running task exec... 2024-05-17T22:39:08.767 INFO:teuthology.task.exec:Executing custom commands... 2024-05-17T22:39:08.768 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi012.front.sepia.ceph.com 2024-05-17T22:39:08.768 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2024-05-17T22:39:08.768 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-05-17T22:39:08.768 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-17T22:39:08.769 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-17T22:39:08.811 INFO:teuthology.orchestra.run.smithi012.stdout:total 0 2024-05-17T22:39:08.823 INFO:teuthology.orchestra.run.smithi012.stdout:Filesystem Size Used Avail Use% Mounted on 2024-05-17T22:39:08.824 INFO:teuthology.orchestra.run.smithi012.stdout:0@4d593780-149c-11ef-bc9a-c7b262605968.cephfs=/volumes/qa/sv_1/682b7b25-6ede-403c-acf6-bc1f5a3307a1 1.1T 380M 1.1T 1% /tmp/cephfs 2024-05-17T22:39:08.826 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2024-05-17T22:39:08.923 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2024-05-17T22:39:08.956 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:08 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/921701112' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:39:08.966 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2024-05-17T22:39:08.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:08 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/921701112' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:39:09.038 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:39:09.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:08 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/921701112' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:39:09.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:09 smithi012 ceph-mon[31632]: pgmap v436: 129 pgs: 129 active+clean; 648 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-05-17T22:39:09.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:09 smithi151 ceph-mon[36101]: pgmap v436: 129 pgs: 129 active+clean; 648 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-05-17T22:39:10.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:09 smithi096 ceph-mon[36065]: pgmap v436: 129 pgs: 129 active+clean; 648 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-05-17T22:39:10.817 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2024-05-17T22:39:10.895 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2024-05-17T22:39:10.898 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2024-05-17T22:39:10.979 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-05-17T22:39:11.057 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/: 2024-05-17T22:39:11.057 INFO:teuthology.orchestra.run.smithi012.stdout:total 0 2024-05-17T22:39:11.057 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 17 22:39 . 2024-05-17T22:39:11.057 INFO:teuthology.orchestra.run.smithi012.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 May 17 22:39 .. 2024-05-17T22:39:11.057 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 data 2024-05-17T22:39:11.057 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:39:11.057 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data: 2024-05-17T22:39:11.057 INFO:teuthology.orchestra.run.smithi012.stdout:total 0 2024-05-17T22:39:11.057 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 . 2024-05-17T22:39:11.057 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 17 22:39 .. 2024-05-17T22:39:11.059 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2024-05-17T22:39:11.147 INFO:teuthology.orchestra.run.smithi012.stdout:[Service] 2024-05-17T22:39:11.147 INFO:teuthology.orchestra.run.smithi012.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2024-05-17T22:39:11.147 INFO:teuthology.orchestra.run.smithi012.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2024-05-17T22:39:11.149 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2024-05-17T22:39:11.260 INFO:teuthology.orchestra.run.smithi012.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2024-05-17T22:39:11.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.699+0000 7f4dcf790700 1 -- 172.21.15.12:0/2891744731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dc8102e70 msgr2=0x7f4dc8103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:11.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.699+0000 7f4dcf790700 1 --2- 172.21.15.12:0/2891744731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dc8102e70 0x7f4dc8103280 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4009990 tx=0x7f4dc4009ca0 comp rx=0 tx=0).stop 2024-05-17T22:39:11.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.699+0000 7f4dcf790700 1 -- 172.21.15.12:0/2891744731 shutdown_connections 2024-05-17T22:39:11.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.699+0000 7f4dcf790700 1 --2- 172.21.15.12:0/2891744731 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4dc81080a0 0x7f4dc810fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:11.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.699+0000 7f4dcf790700 1 --2- 172.21.15.12:0/2891744731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dc8102e70 0x7f4dc8103280 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:11.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.699+0000 7f4dcf790700 1 --2- 172.21.15.12:0/2891744731 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4dc8107750 0x7f4dc8107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:11.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.699+0000 7f4dcf790700 1 -- 172.21.15.12:0/2891744731 >> 172.21.15.12:0/2891744731 conn(0x7f4dc8079ae0 msgr2=0x7f4dc8079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:39:11.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.699+0000 7f4dcf790700 1 -- 172.21.15.12:0/2891744731 shutdown_connections 2024-05-17T22:39:11.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.699+0000 7f4dcf790700 1 -- 172.21.15.12:0/2891744731 wait complete. 2024-05-17T22:39:11.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.700+0000 7f4dcf790700 1 Processor -- start 2024-05-17T22:39:11.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.700+0000 7f4dcf790700 1 -- start start 2024-05-17T22:39:11.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.700+0000 7f4dcf790700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dc8102e70 0x7f4dc80ffa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:11.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.700+0000 7f4dcf790700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4dc8107750 0x7f4dc80fff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:11.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.700+0000 7f4dcf790700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4dc81080a0 0x7f4dc81004a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:11.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.700+0000 7f4dcf790700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc8112cd0 con 0x7f4dc8102e70 2024-05-17T22:39:11.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.700+0000 7f4dcf790700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc8112b50 con 0x7f4dc81080a0 2024-05-17T22:39:11.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.700+0000 7f4dcf790700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc8112e50 con 0x7f4dc8107750 2024-05-17T22:39:11.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.701+0000 7f4dcd52c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dc8102e70 0x7f4dc80ffa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:11.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.701+0000 7f4dcd52c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dc8102e70 0x7f4dc80ffa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55474/0 (socket says 172.21.15.12:55474) 2024-05-17T22:39:11.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.701+0000 7f4dcd52c700 1 -- 172.21.15.12:0/179013238 learned_addr learned my addr 172.21.15.12:0/179013238 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:39:11.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.701+0000 7f4dcdd2d700 1 --2- 172.21.15.12:0/179013238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4dc81080a0 0x7f4dc81004a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:11.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.701+0000 7f4dccd2b700 1 --2- 172.21.15.12:0/179013238 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4dc8107750 0x7f4dc80fff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:11.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.701+0000 7f4dcd52c700 1 -- 172.21.15.12:0/179013238 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4dc8107750 msgr2=0x7f4dc80fff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:11.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.701+0000 7f4dcd52c700 1 --2- 172.21.15.12:0/179013238 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4dc8107750 0x7f4dc80fff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:11.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.701+0000 7f4dcd52c700 1 -- 172.21.15.12:0/179013238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4dc81080a0 msgr2=0x7f4dc81004a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:11.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.701+0000 7f4dcd52c700 1 --2- 172.21.15.12:0/179013238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4dc81080a0 0x7f4dc81004a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:11.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.701+0000 7f4dcd52c700 1 -- 172.21.15.12:0/179013238 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dc81a8ab0 con 0x7f4dc8102e70 2024-05-17T22:39:11.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.701+0000 7f4dccd2b700 1 --2- 172.21.15.12:0/179013238 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4dc8107750 0x7f4dc80fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:39:11.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.701+0000 7f4dcd52c700 1 --2- 172.21.15.12:0/179013238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dc8102e70 0x7f4dc80ffa20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4dc40057c0 tx=0x7f4dc4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:39:11.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.701+0000 7f4dbe7fc700 1 -- 172.21.15.12:0/179013238 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc4026070 con 0x7f4dc8102e70 2024-05-17T22:39:11.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.701+0000 7f4dcf790700 1 -- 172.21.15.12:0/179013238 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dc81a8d40 con 0x7f4dc8102e70 2024-05-17T22:39:11.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.701+0000 7f4dbe7fc700 1 -- 172.21.15.12:0/179013238 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dc4021410 con 0x7f4dc8102e70 2024-05-17T22:39:11.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.701+0000 7f4dbe7fc700 1 -- 172.21.15.12:0/179013238 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc4017650 con 0x7f4dc8102e70 2024-05-17T22:39:11.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.701+0000 7f4dcf790700 1 -- 172.21.15.12:0/179013238 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dc81a9170 con 0x7f4dc8102e70 2024-05-17T22:39:11.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.704+0000 7f4dbe7fc700 1 -- 172.21.15.12:0/179013238 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4dc4029070 con 0x7f4dc8102e70 2024-05-17T22:39:11.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.705+0000 7f4dbe7fc700 1 --2- 172.21.15.12:0/179013238 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f4db40b2880 0x7f4db40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:11.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.705+0000 7f4dccd2b700 1 --2- 172.21.15.12:0/179013238 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f4db40b2880 0x7f4db40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:11.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.705+0000 7f4dbe7fc700 1 -- 172.21.15.12:0/179013238 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f4dc40e3350 con 0x7f4dc8102e70 2024-05-17T22:39:11.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.706+0000 7f4dde16bb80 1 Processor -- start 2024-05-17T22:39:11.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.706+0000 7f4dde16bb80 1 -- start start 2024-05-17T22:39:11.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.706+0000 7f4dccd2b700 1 --2- 172.21.15.12:0/179013238 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f4db40b2880 0x7f4db40b4d90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4db8005fd0 tx=0x7f4db8005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:39:11.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.706+0000 7f4dde16bb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a8198fda70 0x55a819a815e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:11.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.706+0000 7f4dde16bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a819a81b20 0x55a819a7bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:11.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.706+0000 7f4dde16bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a819a7c1f0 0x55a819a7c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:11.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.706+0000 7f4dde16bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a8199aa310 con 0x55a819a7c1f0 2024-05-17T22:39:11.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.706+0000 7f4dde16bb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a81999a050 con 0x55a8198fda70 2024-05-17T22:39:11.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.706+0000 7f4dde16bb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a81999def0 con 0x55a819a81b20 2024-05-17T22:39:11.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.706+0000 7f4dcdd2d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a819a7c1f0 0x55a819a7c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:11.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.706+0000 7f4dcdd2d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a819a7c1f0 0x55a819a7c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55482/0 (socket says 172.21.15.12:55482) 2024-05-17T22:39:11.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.706+0000 7f4dcdd2d700 1 -- 172.21.15.12:0/711554223 learned_addr learned my addr 172.21.15.12:0/711554223 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:39:11.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.706+0000 7f4dcd52c700 1 --2- 172.21.15.12:0/711554223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a8198fda70 0x55a819a815e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:11.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.706+0000 7f4dccd2b700 1 --2- 172.21.15.12:0/711554223 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a819a81b20 0x55a819a7bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:11.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.707+0000 7f4dcdd2d700 1 -- 172.21.15.12:0/711554223 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a819a81b20 msgr2=0x55a819a7bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:39:11.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.707+0000 7f4dcdd2d700 1 --2- 172.21.15.12:0/711554223 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a819a81b20 0x55a819a7bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:11.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.707+0000 7f4dcdd2d700 1 -- 172.21.15.12:0/711554223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a8198fda70 msgr2=0x55a819a815e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:39:11.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.707+0000 7f4dcdd2d700 1 --2- 172.21.15.12:0/711554223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a8198fda70 0x55a819a815e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:11.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.707+0000 7f4dcdd2d700 1 -- 172.21.15.12:0/711554223 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a819943610 con 0x55a819a7c1f0 2024-05-17T22:39:11.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.707+0000 7f4dcd52c700 1 --2- 172.21.15.12:0/711554223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a8198fda70 0x55a819a815e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:39:11.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.707+0000 7f4dcdd2d700 1 --2- 172.21.15.12:0/711554223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a819a7c1f0 0x55a819a7c600 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0011680 tx=0x7f4dc0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d525c6c9f1be4d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:39:11.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.708+0000 7f4db37fe700 1 -- 172.21.15.12:0/711554223 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0019030 con 0x55a819a7c1f0 2024-05-17T22:39:11.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.708+0000 7f4db37fe700 1 -- 172.21.15.12:0/711554223 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dc00092e0 con 0x55a819a7c1f0 2024-05-17T22:39:11.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.708+0000 7f4db37fe700 1 -- 172.21.15.12:0/711554223 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0007500 con 0x55a819a7c1f0 2024-05-17T22:39:11.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.709+0000 7f4dde16bb80 1 -- 172.21.15.12:0/711554223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a819a7c1f0 msgr2=0x55a819a7c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:11.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.709+0000 7f4dde16bb80 1 --2- 172.21.15.12:0/711554223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a819a7c1f0 0x55a819a7c600 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0011680 tx=0x7f4dc0009920 comp rx=0 tx=0).stop 2024-05-17T22:39:11.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.709+0000 7f4dde16bb80 1 -- 172.21.15.12:0/711554223 shutdown_connections 2024-05-17T22:39:11.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.709+0000 7f4dde16bb80 1 --2- 172.21.15.12:0/711554223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a819a7c1f0 0x55a819a7c600 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:11.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.709+0000 7f4dde16bb80 1 --2- 172.21.15.12:0/711554223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a8198fda70 0x55a819a815e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:11.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.709+0000 7f4dde16bb80 1 --2- 172.21.15.12:0/711554223 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a819a81b20 0x55a819a7bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:11.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.709+0000 7f4dde16bb80 1 -- 172.21.15.12:0/711554223 >> 172.21.15.12:0/711554223 conn(0x55a819941f80 msgr2=0x55a819a76c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:39:11.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.709+0000 7f4dde16bb80 1 -- 172.21.15.12:0/711554223 shutdown_connections 2024-05-17T22:39:11.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.709+0000 7f4dde16bb80 1 -- 172.21.15.12:0/711554223 wait complete. 2024-05-17T22:39:11.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.710+0000 7f4dde16bb80 1 -- start start 2024-05-17T22:39:11.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.710+0000 7f4dde16bb80 1 Processor -- start 2024-05-17T22:39:11.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.710+0000 7f4dde16bb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a8198fda70 0x55a819a852e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:11.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.710+0000 7f4dde16bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a819a7c1f0 0x55a819a85820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:11.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.711+0000 7f4dde16bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a819a81b20 0x55a819a916b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:11.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.711+0000 7f4dde16bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a8198a11d0 con 0x55a819a81b20 2024-05-17T22:39:11.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.711+0000 7f4dde16bb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a81993af00 con 0x55a8198fda70 2024-05-17T22:39:11.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.711+0000 7f4dde16bb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a819a88410 con 0x55a819a7c1f0 2024-05-17T22:39:11.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.711+0000 7f4dde16bb80 10 client.0 _ll_register_callbacks cb 0x55a819934f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:39:11.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.711+0000 7f4dcdd2d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a819a81b20 0x55a819a916b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:11.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.711+0000 7f4dcd52c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a8198fda70 0x55a819a852e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:11.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.711+0000 7f4dcd52c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a8198fda70 0x55a819a852e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:32842/0 (socket says 172.21.15.12:32842) 2024-05-17T22:39:11.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.711+0000 7f4dcd52c700 1 -- 172.21.15.12:0/2375723231 learned_addr learned my addr 172.21.15.12:0/2375723231 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:39:11.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.711+0000 7f4dcdd2d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a819a81b20 0x55a819a916b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55498/0 (socket says 172.21.15.12:55498) 2024-05-17T22:39:11.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.711+0000 7f4dccd2b700 1 --2- 172.21.15.12:0/2375723231 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a819a7c1f0 0x55a819a85820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:11.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.711+0000 7f4dcd52c700 1 -- 172.21.15.12:0/2375723231 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a819a7c1f0 msgr2=0x55a819a85820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:39:11.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.711+0000 7f4dcd52c700 1 --2- 172.21.15.12:0/2375723231 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a819a7c1f0 0x55a819a85820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:11.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.711+0000 7f4dcd52c700 1 -- 172.21.15.12:0/2375723231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a819a81b20 msgr2=0x55a819a916b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:39:11.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.711+0000 7f4dcd52c700 1 --2- 172.21.15.12:0/2375723231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a819a81b20 0x55a819a916b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:11.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.711+0000 7f4dcd52c700 1 -- 172.21.15.12:0/2375723231 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8198ea500 con 0x55a8198fda70 2024-05-17T22:39:11.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.711+0000 7f4dcd52c700 1 --2- 172.21.15.12:0/2375723231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a8198fda70 0x55a819a852e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4dc40e0b50 tx=0x7f4dc40a9e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52b2d19ce8f549cf server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:39:11.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.711+0000 7f4db0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:39:11.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.711+0000 7f4db0ff9700 1 -- 172.21.15.12:0/2375723231 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc40e3170 con 0x55a8198fda70 2024-05-17T22:39:11.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.712+0000 7f4db0ff9700 1 -- 172.21.15.12:0/2375723231 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f4dc40aab70 con 0x55a8198fda70 2024-05-17T22:39:11.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.712+0000 7f4dde16bb80 1 -- 172.21.15.12:0/2375723231 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dc00092e0 con 0x55a8198fda70 2024-05-17T22:39:11.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.712+0000 7f4db0ff9700 1 -- 172.21.15.12:0/2375723231 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4dc406d050 con 0x55a8198fda70 2024-05-17T22:39:11.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.712+0000 7f4dde16bb80 10 client.25031 fetch_fsmap learned FSMap version 27 2024-05-17T22:39:11.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.712+0000 7f4dde16bb80 1 -- 172.21.15.12:0/2375723231 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a819a92ea0 con 0x55a8198fda70 2024-05-17T22:39:11.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.712+0000 7f4db0ff9700 1 -- 172.21.15.12:0/2375723231 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dc00092e0 con 0x55a8198fda70 2024-05-17T22:39:11.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.712+0000 7f4db0ff9700 1 -- 172.21.15.12:0/2375723231 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4dc4070020 con 0x55a8198fda70 2024-05-17T22:39:11.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.712+0000 7f4dde16bb80 10 client.25031 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T22:39:11.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.712+0000 7f4dde16bb80 10 client.25031 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:39:11.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.712+0000 7f4dde16bb80 20 client.25031 populate_metadata read hostname 'smithi012' 2024-05-17T22:39:11.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.712+0000 7f4dde16bb80 1 --2- 172.21.15.12:0/2375723231 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55a819a95410 0x55a819a97870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:11.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.712+0000 7f4dde16bb80 4 client.25031 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:39:11.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.712+0000 7f4dde16bb80 1 -- 172.21.15.12:0/2375723231 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8199ac260 con 0x55a819a95410 2024-05-17T22:39:11.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.713+0000 7f4dccd2b700 1 --2- 172.21.15.12:0/2375723231 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55a819a95410 0x55a819a97870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:11.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.713+0000 7f4dccd2b700 1 --2- 172.21.15.12:0/2375723231 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55a819a95410 0x55a819a97870 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cc376ae363db43a server_cookie=28f38284f22e76c0 in_seq=0 out_seq=0 2024-05-17T22:39:11.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.713+0000 7f4db0ff9700 10 client.25031 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:39:11.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.714+0000 7f4db0ff9700 1 -- 172.21.15.12:0/2375723231 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a8199ac260 con 0x55a819a95410 2024-05-17T22:39:11.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.714+0000 7f4db0ff9700 10 client.25031 handle_command_reply: tid=0 2024-05-17T22:39:11.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.714+0000 7f4dde16bb80 1 client.25031 shutdown 2024-05-17T22:39:11.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.714+0000 7f4dde16bb80 1 -- 172.21.15.12:0/2375723231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a8198fda70 msgr2=0x55a819a852e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:11.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.714+0000 7f4dde16bb80 1 --2- 172.21.15.12:0/2375723231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a8198fda70 0x55a819a852e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4dc40e0b50 tx=0x7f4dc40a9e70 comp rx=0 tx=0).stop 2024-05-17T22:39:11.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.714+0000 7f4dde16bb80 1 -- 172.21.15.12:0/2375723231 shutdown_connections 2024-05-17T22:39:11.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.714+0000 7f4dde16bb80 1 --2- 172.21.15.12:0/2375723231 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55a819a95410 0x55a819a97870 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:11.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.714+0000 7f4dde16bb80 1 --2- 172.21.15.12:0/2375723231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a819a81b20 0x55a819a916b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:11.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.714+0000 7f4dde16bb80 1 --2- 172.21.15.12:0/2375723231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a8198fda70 0x55a819a852e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:11.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.714+0000 7f4db0ff9700 0 client.25031 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:39:11.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.714+0000 7f4dde16bb80 1 --2- 172.21.15.12:0/2375723231 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a819a7c1f0 0x55a819a85820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:11.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.714+0000 7f4dde16bb80 1 -- 172.21.15.12:0/2375723231 >> 172.21.15.12:0/2375723231 conn(0x55a819941f80 msgr2=0x55a819a70f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:39:11.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.714+0000 7f4dde16bb80 1 -- 172.21.15.12:0/2375723231 shutdown_connections 2024-05-17T22:39:11.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.714+0000 7f4dde16bb80 1 -- 172.21.15.12:0/2375723231 wait complete. 2024-05-17T22:39:11.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.714+0000 7f4dde16bb80 20 client.25031 trim_cache size 0 max 16384 2024-05-17T22:39:11.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.735+0000 7f4dde16bb80 1 Processor -- start 2024-05-17T22:39:11.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.735+0000 7f4dde16bb80 1 -- start start 2024-05-17T22:39:11.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.735+0000 7f4dde16bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8198fda70 0x55a819a7bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:11.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.735+0000 7f4dde16bb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a819a7c1f0 0x55a819a70f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:11.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.735+0000 7f4dde16bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a819a81b20 0x55a819a9d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:11.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.735+0000 7f4dde16bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc81b4760 con 0x55a819a81b20 2024-05-17T22:39:11.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.735+0000 7f4dde16bb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc81b1ff0 con 0x55a819a7c1f0 2024-05-17T22:39:11.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.735+0000 7f4dde16bb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc81afe40 con 0x55a8198fda70 2024-05-17T22:39:11.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.736+0000 7f4dccd2b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a819a7c1f0 0x55a819a70f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:11.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.736+0000 7f4dccd2b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a819a7c1f0 0x55a819a70f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:32850/0 (socket says 172.21.15.12:32850) 2024-05-17T22:39:11.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.736+0000 7f4dccd2b700 1 -- 172.21.15.12:0/661661972 learned_addr learned my addr 172.21.15.12:0/661661972 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:39:11.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.736+0000 7f4dcdd2d700 1 --2- 172.21.15.12:0/661661972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a819a81b20 0x55a819a9d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:11.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.736+0000 7f4dcd52c700 1 --2- 172.21.15.12:0/661661972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8198fda70 0x55a819a7bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:11.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.736+0000 7f4dccd2b700 1 -- 172.21.15.12:0/661661972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8198fda70 msgr2=0x55a819a7bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:39:11.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.736+0000 7f4dccd2b700 1 --2- 172.21.15.12:0/661661972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8198fda70 0x55a819a7bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:11.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.736+0000 7f4dccd2b700 1 -- 172.21.15.12:0/661661972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a819a81b20 msgr2=0x55a819a9d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:39:11.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.736+0000 7f4dccd2b700 1 --2- 172.21.15.12:0/661661972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a819a81b20 0x55a819a9d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:11.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.736+0000 7f4dccd2b700 1 -- 172.21.15.12:0/661661972 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a819a83830 con 0x55a819a7c1f0 2024-05-17T22:39:11.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.736+0000 7f4dccd2b700 1 --2- 172.21.15.12:0/661661972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a819a7c1f0 0x55a819a70f30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4db8005ea0 tx=0x7f4db8014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcc32d5ff0a60c90 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:39:11.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.736+0000 7f4db2ffd700 1 -- 172.21.15.12:0/661661972 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db8027530 con 0x55a819a7c1f0 2024-05-17T22:39:11.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.736+0000 7f4db2ffd700 1 -- 172.21.15.12:0/661661972 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4db802d070 con 0x55a819a7c1f0 2024-05-17T22:39:11.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.737+0000 7f4db2ffd700 1 -- 172.21.15.12:0/661661972 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db801ecd0 con 0x55a819a7c1f0 2024-05-17T22:39:11.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.737+0000 7f4dde16bb80 1 -- 172.21.15.12:0/661661972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a819a7c1f0 msgr2=0x55a819a70f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:11.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.737+0000 7f4dde16bb80 1 --2- 172.21.15.12:0/661661972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a819a7c1f0 0x55a819a70f30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4db8005ea0 tx=0x7f4db8014000 comp rx=0 tx=0).stop 2024-05-17T22:39:11.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.737+0000 7f4dde16bb80 1 -- 172.21.15.12:0/661661972 shutdown_connections 2024-05-17T22:39:11.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.737+0000 7f4dde16bb80 1 --2- 172.21.15.12:0/661661972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a819a81b20 0x55a819a9d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:11.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.737+0000 7f4dde16bb80 1 --2- 172.21.15.12:0/661661972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8198fda70 0x55a819a7bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:11.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.737+0000 7f4dde16bb80 1 --2- 172.21.15.12:0/661661972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a819a7c1f0 0x55a819a70f30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:11.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.737+0000 7f4dde16bb80 1 -- 172.21.15.12:0/661661972 >> 172.21.15.12:0/661661972 conn(0x55a819941f80 msgr2=0x55a819a7f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:39:11.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.738+0000 7f4dde16bb80 1 -- 172.21.15.12:0/661661972 shutdown_connections 2024-05-17T22:39:11.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.738+0000 7f4dde16bb80 1 -- 172.21.15.12:0/661661972 wait complete. 2024-05-17T22:39:11.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.738+0000 7f4dde16bb80 1 -- start start 2024-05-17T22:39:11.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.738+0000 7f4dde16bb80 1 Processor -- start 2024-05-17T22:39:11.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.738+0000 7f4dde16bb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a8198fda70 0x55a819a892a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:11.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.739+0000 7f4dde16bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a819a7c1f0 0x55a819a897e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:11.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.739+0000 7f4dde16bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a819a81b20 0x55a819afebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:11.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.739+0000 7f4dde16bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc81b04b0 con 0x55a819a81b20 2024-05-17T22:39:11.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.739+0000 7f4dde16bb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc81b1920 con 0x55a8198fda70 2024-05-17T22:39:11.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.739+0000 7f4dde16bb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc81b0b80 con 0x55a819a7c1f0 2024-05-17T22:39:11.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.739+0000 7f4dde16bb80 10 client.0 _ll_register_callbacks cb 0x55a819934f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:39:11.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.739+0000 7f4dcdd2d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a819a81b20 0x55a819afebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:11.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.739+0000 7f4dcd52c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a8198fda70 0x55a819a892a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:11.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.739+0000 7f4dcdd2d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a819a81b20 0x55a819afebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55518/0 (socket says 172.21.15.12:55518) 2024-05-17T22:39:11.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.739+0000 7f4dcd52c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a8198fda70 0x55a819a892a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:32852/0 (socket says 172.21.15.12:32852) 2024-05-17T22:39:11.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.739+0000 7f4dcdd2d700 1 -- 172.21.15.12:0/1678592911 learned_addr learned my addr 172.21.15.12:0/1678592911 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:39:11.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.739+0000 7f4dccd2b700 1 --2- 172.21.15.12:0/1678592911 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a819a7c1f0 0x55a819a897e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:11.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.739+0000 7f4dcdd2d700 1 -- 172.21.15.12:0/1678592911 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a819a7c1f0 msgr2=0x55a819a897e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:39:11.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.739+0000 7f4dcdd2d700 1 --2- 172.21.15.12:0/1678592911 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a819a7c1f0 0x55a819a897e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:11.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.739+0000 7f4dcdd2d700 1 -- 172.21.15.12:0/1678592911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a8198fda70 msgr2=0x55a819a892a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:39:11.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.739+0000 7f4dcdd2d700 1 --2- 172.21.15.12:0/1678592911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a8198fda70 0x55a819a892a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:11.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.739+0000 7f4dcdd2d700 1 -- 172.21.15.12:0/1678592911 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a819a8a0c0 con 0x55a819a81b20 2024-05-17T22:39:11.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.739+0000 7f4dcd52c700 1 --2- 172.21.15.12:0/1678592911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a8198fda70 0x55a819a892a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:39:11.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.739+0000 7f4dcdd2d700 1 --2- 172.21.15.12:0/1678592911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a819a81b20 0x55a819afebf0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4dc000afe0 tx=0x7f4dc0004da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d67fa893578c0a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:39:11.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.739+0000 7f4db0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:39:11.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.739+0000 7f4db0ff9700 1 -- 172.21.15.12:0/1678592911 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0007700 con 0x55a819a81b20 2024-05-17T22:39:11.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.740+0000 7f4db0ff9700 1 -- 172.21.15.12:0/1678592911 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f4dc00078a0 con 0x55a819a81b20 2024-05-17T22:39:11.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.740+0000 7f4dde16bb80 1 -- 172.21.15.12:0/1678592911 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4db802d070 con 0x55a819a81b20 2024-05-17T22:39:11.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.740+0000 7f4db0ff9700 1 -- 172.21.15.12:0/1678592911 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4dc0030050 con 0x55a819a81b20 2024-05-17T22:39:11.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.740+0000 7f4dde16bb80 10 client.15372 fetch_fsmap learned FSMap version 27 2024-05-17T22:39:11.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.740+0000 7f4dde16bb80 1 -- 172.21.15.12:0/1678592911 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a819b001e0 con 0x55a819a81b20 2024-05-17T22:39:11.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.740+0000 7f4db0ff9700 1 -- 172.21.15.12:0/1678592911 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4db802d070 con 0x55a819a81b20 2024-05-17T22:39:11.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.740+0000 7f4db0ff9700 1 -- 172.21.15.12:0/1678592911 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4dc0003f60 con 0x55a819a81b20 2024-05-17T22:39:11.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.740+0000 7f4dde16bb80 10 client.15372 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T22:39:11.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.740+0000 7f4dde16bb80 10 client.15372 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:39:11.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.740+0000 7f4dde16bb80 20 client.15372 populate_metadata read hostname 'smithi012' 2024-05-17T22:39:11.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.740+0000 7f4dde16bb80 1 --2- 172.21.15.12:0/1678592911 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55a819b00640 0x55a819b02a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:11.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.740+0000 7f4dde16bb80 4 client.15372 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-17T22:39:11.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.740+0000 7f4dde16bb80 1 -- 172.21.15.12:0/1678592911 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a8199a5a40 con 0x55a819b00640 2024-05-17T22:39:11.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.740+0000 7f4dcd52c700 1 --2- 172.21.15.12:0/1678592911 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55a819b00640 0x55a819b02a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:11.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.741+0000 7f4db0ff9700 10 client.15372 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:39:11.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.741+0000 7f4dcd52c700 1 --2- 172.21.15.12:0/1678592911 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55a819b00640 0x55a819b02a60 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=820ab5ca12213b53 server_cookie=476d574e9949b80e in_seq=0 out_seq=0 2024-05-17T22:39:11.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.742+0000 7f4db0ff9700 1 -- 172.21.15.12:0/1678592911 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a8199a5a40 con 0x55a819b00640 2024-05-17T22:39:11.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.742+0000 7f4db0ff9700 10 client.15372 handle_command_reply: tid=0 2024-05-17T22:39:11.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.742+0000 7f4dde16bb80 1 client.15372 shutdown 2024-05-17T22:39:11.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.743+0000 7f4dde16bb80 1 -- 172.21.15.12:0/1678592911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a819a81b20 msgr2=0x55a819afebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:11.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.743+0000 7f4dde16bb80 1 --2- 172.21.15.12:0/1678592911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a819a81b20 0x55a819afebf0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4dc000afe0 tx=0x7f4dc0004da0 comp rx=0 tx=0).stop 2024-05-17T22:39:11.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.743+0000 7f4dde16bb80 1 -- 172.21.15.12:0/1678592911 shutdown_connections 2024-05-17T22:39:11.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.743+0000 7f4dde16bb80 1 --2- 172.21.15.12:0/1678592911 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55a819b00640 0x55a819b02a60 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:11.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.743+0000 7f4dde16bb80 1 --2- 172.21.15.12:0/1678592911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a819a81b20 0x55a819afebf0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:11.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.743+0000 7f4dde16bb80 1 --2- 172.21.15.12:0/1678592911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a8198fda70 0x55a819a892a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:11.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.743+0000 7f4dde16bb80 1 --2- 172.21.15.12:0/1678592911 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a819a7c1f0 0x55a819a897e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:11.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.743+0000 7f4db0ff9700 0 client.15372 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:39:11.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.743+0000 7f4dde16bb80 1 -- 172.21.15.12:0/1678592911 >> 172.21.15.12:0/1678592911 conn(0x55a819941f80 msgr2=0x55a819a7f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:39:11.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.743+0000 7f4dde16bb80 1 -- 172.21.15.12:0/1678592911 shutdown_connections 2024-05-17T22:39:11.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.743+0000 7f4dde16bb80 1 -- 172.21.15.12:0/1678592911 wait complete. 2024-05-17T22:39:11.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.743+0000 7f4dde16bb80 20 client.15372 trim_cache size 0 max 16384 2024-05-17T22:39:11.749 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:39:11.749 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-17T22:39:11.749 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "65198cc9-f62d-4163-858c-7f47eaf98eef", 2024-05-17T22:39:11.749 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-17T22:39:11.749 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:39:11.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.744+0000 7f4dcf790700 1 -- 172.21.15.12:0/179013238 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f4db40b2880 msgr2=0x7f4db40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:11.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.744+0000 7f4dcf790700 1 --2- 172.21.15.12:0/179013238 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f4db40b2880 0x7f4db40b4d90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4db8005fd0 tx=0x7f4db8005e30 comp rx=0 tx=0).stop 2024-05-17T22:39:11.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.744+0000 7f4dcf790700 1 -- 172.21.15.12:0/179013238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dc8102e70 msgr2=0x7f4dc80ffa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:11.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.744+0000 7f4dcf790700 1 --2- 172.21.15.12:0/179013238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dc8102e70 0x7f4dc80ffa20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4dc40057c0 tx=0x7f4dc4009580 comp rx=0 tx=0).stop 2024-05-17T22:39:11.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.744+0000 7f4dcf790700 1 -- 172.21.15.12:0/179013238 shutdown_connections 2024-05-17T22:39:11.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.744+0000 7f4dcf790700 1 --2- 172.21.15.12:0/179013238 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f4db40b2880 0x7f4db40b4d90 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:11.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.744+0000 7f4dcf790700 1 --2- 172.21.15.12:0/179013238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4dc81080a0 0x7f4dc81004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:11.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.744+0000 7f4dcf790700 1 --2- 172.21.15.12:0/179013238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dc8102e70 0x7f4dc80ffa20 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:11.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.744+0000 7f4dcf790700 1 --2- 172.21.15.12:0/179013238 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4dc8107750 0x7f4dc80fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:11.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.744+0000 7f4dcf790700 1 -- 172.21.15.12:0/179013238 >> 172.21.15.12:0/179013238 conn(0x7f4dc8079ae0 msgr2=0x7f4dc81065e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:39:11.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.744+0000 7f4dcf790700 1 -- 172.21.15.12:0/179013238 shutdown_connections 2024-05-17T22:39:11.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:11.744+0000 7f4dcf790700 1 -- 172.21.15.12:0/179013238 wait complete. 2024-05-17T22:39:11.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:11 smithi012 ceph-mon[31632]: pgmap v437: 129 pgs: 129 active+clean; 655 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-05-17T22:39:11.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:11 smithi151 ceph-mon[36101]: pgmap v437: 129 pgs: 129 active+clean; 655 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-05-17T22:39:12.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:11 smithi096 ceph-mon[36065]: pgmap v437: 129 pgs: 129 active+clean; 655 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-05-17T22:39:12.788 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-17T22:39:12.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:12 smithi151 ceph-mon[36101]: scrub queued for path: / 2024-05-17T22:39:12.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:12 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:39:12.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:12 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:39:12.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:12 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:39:13.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:12 smithi096 ceph-mon[36065]: scrub queued for path: / 2024-05-17T22:39:13.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:12 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:39:13.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:12 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:39:13.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:12 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:39:13.053 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:39:13.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:12 smithi012 ceph-mon[31632]: scrub queued for path: / 2024-05-17T22:39:13.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:12 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:39:13.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:12 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:39:13.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:12 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:39:13.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:13 smithi151 ceph-mon[36101]: pgmap v438: 129 pgs: 129 active+clean; 655 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-05-17T22:39:14.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:13 smithi096 ceph-mon[36065]: pgmap v438: 129 pgs: 129 active+clean; 655 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-05-17T22:39:14.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:13 smithi012 ceph-mon[31632]: pgmap v438: 129 pgs: 129 active+clean; 655 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-05-17T22:39:14.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.885+0000 7f98d43d3700 1 -- 172.21.15.12:0/3577522065 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98cc108080 msgr2=0x7f98cc10f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:14.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.885+0000 7f98d43d3700 1 --2- 172.21.15.12:0/3577522065 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98cc108080 0x7f98cc10f9e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f98c400b0a0 tx=0x7f98c400b3b0 comp rx=0 tx=0).stop 2024-05-17T22:39:14.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.885+0000 7f98d43d3700 1 -- 172.21.15.12:0/3577522065 shutdown_connections 2024-05-17T22:39:14.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.885+0000 7f98d43d3700 1 --2- 172.21.15.12:0/3577522065 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98cc108080 0x7f98cc10f9e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:14.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.885+0000 7f98d43d3700 1 --2- 172.21.15.12:0/3577522065 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98cc102e00 0x7f98cc103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:14.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.885+0000 7f98d43d3700 1 --2- 172.21.15.12:0/3577522065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98cc107730 0x7f98cc107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:14.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.885+0000 7f98d43d3700 1 -- 172.21.15.12:0/3577522065 >> 172.21.15.12:0/3577522065 conn(0x7f98cc079bd0 msgr2=0x7f98cc079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:39:14.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.885+0000 7f98d43d3700 1 -- 172.21.15.12:0/3577522065 shutdown_connections 2024-05-17T22:39:14.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.885+0000 7f98d43d3700 1 -- 172.21.15.12:0/3577522065 wait complete. 2024-05-17T22:39:14.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.886+0000 7f98d43d3700 1 Processor -- start 2024-05-17T22:39:14.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.886+0000 7f98d43d3700 1 -- start start 2024-05-17T22:39:14.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.886+0000 7f98d43d3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98cc102e00 0x7f98cc19b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:14.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.886+0000 7f98d43d3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98cc107730 0x7f98cc19bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:14.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.886+0000 7f98d43d3700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98cc108080 0x7f98cc1a6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:14.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.886+0000 7f98d43d3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc112cc0 con 0x7f98cc102e00 2024-05-17T22:39:14.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.886+0000 7f98d43d3700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc112b40 con 0x7f98cc107730 2024-05-17T22:39:14.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.886+0000 7f98d43d3700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc112e40 con 0x7f98cc108080 2024-05-17T22:39:14.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.886+0000 7f98d216f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98cc102e00 0x7f98cc19b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:14.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.886+0000 7f98d196e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98cc107730 0x7f98cc19bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:14.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.886+0000 7f98d216f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98cc102e00 0x7f98cc19b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55538/0 (socket says 172.21.15.12:55538) 2024-05-17T22:39:14.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.886+0000 7f98d216f700 1 -- 172.21.15.12:0/3305671029 learned_addr learned my addr 172.21.15.12:0/3305671029 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:39:14.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.886+0000 7f98d2970700 1 --2- 172.21.15.12:0/3305671029 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98cc108080 0x7f98cc1a6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:14.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.887+0000 7f98d196e700 1 -- 172.21.15.12:0/3305671029 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98cc108080 msgr2=0x7f98cc1a6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:14.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.887+0000 7f98d196e700 1 --2- 172.21.15.12:0/3305671029 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98cc108080 0x7f98cc1a6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:14.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.887+0000 7f98d196e700 1 -- 172.21.15.12:0/3305671029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98cc102e00 msgr2=0x7f98cc19b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:14.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.887+0000 7f98d196e700 1 --2- 172.21.15.12:0/3305671029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98cc102e00 0x7f98cc19b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:14.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.887+0000 7f98d196e700 1 -- 172.21.15.12:0/3305671029 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98cc19c360 con 0x7f98cc107730 2024-05-17T22:39:14.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.887+0000 7f98d2970700 1 --2- 172.21.15.12:0/3305671029 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98cc108080 0x7f98cc1a6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:39:14.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.887+0000 7f98d216f700 1 --2- 172.21.15.12:0/3305671029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98cc102e00 0x7f98cc19b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:39:14.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.887+0000 7f98d196e700 1 --2- 172.21.15.12:0/3305671029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98cc107730 0x7f98cc19bad0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f98bc00e9c0 tx=0x7f98bc00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:39:14.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.887+0000 7f98c37fe700 1 -- 172.21.15.12:0/3305671029 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98bc00cbd0 con 0x7f98cc107730 2024-05-17T22:39:14.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.887+0000 7f98c37fe700 1 -- 172.21.15.12:0/3305671029 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98bc00cd70 con 0x7f98cc107730 2024-05-17T22:39:14.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.887+0000 7f98d43d3700 1 -- 172.21.15.12:0/3305671029 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98cc1a6c10 con 0x7f98cc107730 2024-05-17T22:39:14.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.887+0000 7f98c37fe700 1 -- 172.21.15.12:0/3305671029 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98bc018680 con 0x7f98cc107730 2024-05-17T22:39:14.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.887+0000 7f98d43d3700 1 -- 172.21.15.12:0/3305671029 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98cc1a71a0 con 0x7f98cc107730 2024-05-17T22:39:14.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.890+0000 7f98c37fe700 1 -- 172.21.15.12:0/3305671029 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f98bc010430 con 0x7f98cc107730 2024-05-17T22:39:14.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.891+0000 7f98c37fe700 1 --2- 172.21.15.12:0/3305671029 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f98b80b2900 0x7f98b80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:14.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.891+0000 7f98d216f700 1 --2- 172.21.15.12:0/3305671029 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f98b80b2900 0x7f98b80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:14.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.891+0000 7f98c37fe700 1 -- 172.21.15.12:0/3305671029 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f98bc0e2690 con 0x7f98cc107730 2024-05-17T22:39:14.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.892+0000 7f98e2daeb80 1 Processor -- start 2024-05-17T22:39:14.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.892+0000 7f98d216f700 1 --2- 172.21.15.12:0/3305671029 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f98b80b2900 0x7f98b80b4e10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f98c8011550 tx=0x7f98c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:39:14.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.892+0000 7f98e2daeb80 1 -- start start 2024-05-17T22:39:14.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.892+0000 7f98e2daeb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5645b58ae9f0 0x5645b5a32560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.892+0000 7f98e2daeb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645b5a32aa0 0x5645b5a2cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.892+0000 7f98e2daeb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645b5a2d170 0x5645b5a2d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.892+0000 7f98e2daeb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5645b595b290 con 0x5645b58ae9f0 2024-05-17T22:39:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.892+0000 7f98e2daeb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5645b594afd0 con 0x5645b5a2d170 2024-05-17T22:39:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.892+0000 7f98e2daeb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5645b594ee70 con 0x5645b5a32aa0 2024-05-17T22:39:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.892+0000 7f98d216f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5645b58ae9f0 0x5645b5a32560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.892+0000 7f98d2970700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645b5a2d170 0x5645b5a2d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.892+0000 7f98d216f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5645b58ae9f0 0x5645b5a32560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55552/0 (socket says 172.21.15.12:55552) 2024-05-17T22:39:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.892+0000 7f98d216f700 1 -- 172.21.15.12:0/1640182047 learned_addr learned my addr 172.21.15.12:0/1640182047 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:39:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.893+0000 7f98d196e700 1 --2- 172.21.15.12:0/1640182047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645b5a32aa0 0x5645b5a2cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.893+0000 7f98d216f700 1 -- 172.21.15.12:0/1640182047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645b5a32aa0 msgr2=0x5645b5a2cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:39:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.893+0000 7f98d216f700 1 --2- 172.21.15.12:0/1640182047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645b5a32aa0 0x5645b5a2cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.893+0000 7f98d216f700 1 -- 172.21.15.12:0/1640182047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645b5a2d170 msgr2=0x5645b5a2d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:39:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.893+0000 7f98d216f700 1 --2- 172.21.15.12:0/1640182047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645b5a2d170 0x5645b5a2d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:14.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.893+0000 7f98d216f700 1 -- 172.21.15.12:0/1640182047 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645b58f4590 con 0x5645b58ae9f0 2024-05-17T22:39:14.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.893+0000 7f98d2970700 1 --2- 172.21.15.12:0/1640182047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645b5a2d170 0x5645b5a2d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:39:14.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.893+0000 7f98d216f700 1 --2- 172.21.15.12:0/1640182047 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5645b58ae9f0 0x5645b5a32560 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f98c8009880 tx=0x7f98c8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c7e07493a2916a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:39:14.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.893+0000 7f98c0d46700 1 -- 172.21.15.12:0/1640182047 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c8021a20 con 0x5645b58ae9f0 2024-05-17T22:39:14.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.893+0000 7f98c0d46700 1 -- 172.21.15.12:0/1640182047 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98c800ee40 con 0x5645b58ae9f0 2024-05-17T22:39:14.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.893+0000 7f98c0d46700 1 -- 172.21.15.12:0/1640182047 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c80293d0 con 0x5645b58ae9f0 2024-05-17T22:39:14.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.894+0000 7f98e2daeb80 1 -- 172.21.15.12:0/1640182047 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5645b58ae9f0 msgr2=0x5645b5a32560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:14.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.894+0000 7f98e2daeb80 1 --2- 172.21.15.12:0/1640182047 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5645b58ae9f0 0x5645b5a32560 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f98c8009880 tx=0x7f98c8014000 comp rx=0 tx=0).stop 2024-05-17T22:39:14.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.894+0000 7f98e2daeb80 1 -- 172.21.15.12:0/1640182047 shutdown_connections 2024-05-17T22:39:14.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.894+0000 7f98e2daeb80 1 --2- 172.21.15.12:0/1640182047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645b5a2d170 0x5645b5a2d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:14.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.894+0000 7f98e2daeb80 1 --2- 172.21.15.12:0/1640182047 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5645b58ae9f0 0x5645b5a32560 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:14.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.894+0000 7f98e2daeb80 1 --2- 172.21.15.12:0/1640182047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645b5a32aa0 0x5645b5a2cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:14.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.894+0000 7f98e2daeb80 1 -- 172.21.15.12:0/1640182047 >> 172.21.15.12:0/1640182047 conn(0x5645b58f2f00 msgr2=0x5645b5a27b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:39:14.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.894+0000 7f98e2daeb80 1 -- 172.21.15.12:0/1640182047 shutdown_connections 2024-05-17T22:39:14.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.894+0000 7f98e2daeb80 1 -- 172.21.15.12:0/1640182047 wait complete. 2024-05-17T22:39:14.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.895+0000 7f98e2daeb80 1 -- start start 2024-05-17T22:39:14.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.895+0000 7f98e2daeb80 1 Processor -- start 2024-05-17T22:39:14.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.895+0000 7f98e2daeb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645b58ae9f0 0x5645b5a361b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:14.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.895+0000 7f98e2daeb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645b5a2d170 0x5645b5a366f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:14.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.895+0000 7f98e2daeb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5645b5a32aa0 0x5645b5a36c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:14.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.895+0000 7f98e2daeb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5645b5852150 con 0x5645b5a32aa0 2024-05-17T22:39:14.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.895+0000 7f98e2daeb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5645b58ebe80 con 0x5645b5a2d170 2024-05-17T22:39:14.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.895+0000 7f98e2daeb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5645b5a39390 con 0x5645b58ae9f0 2024-05-17T22:39:14.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.895+0000 7f98e2daeb80 10 client.0 _ll_register_callbacks cb 0x5645b58e5f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:39:14.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.896+0000 7f98d196e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645b5a2d170 0x5645b5a366f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:14.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.896+0000 7f98d2970700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5645b5a32aa0 0x5645b5a36c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:14.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.896+0000 7f98d196e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645b5a2d170 0x5645b5a366f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:32894/0 (socket says 172.21.15.12:32894) 2024-05-17T22:39:14.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.896+0000 7f98d196e700 1 -- 172.21.15.12:0/1274945863 learned_addr learned my addr 172.21.15.12:0/1274945863 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:39:14.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.895+0000 7f98d216f700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645b58ae9f0 0x5645b5a361b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:14.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.896+0000 7f98d2970700 1 -- 172.21.15.12:0/1274945863 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645b58ae9f0 msgr2=0x5645b5a361b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:39:14.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.896+0000 7f98d2970700 1 --2- 172.21.15.12:0/1274945863 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645b58ae9f0 0x5645b5a361b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:14.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.896+0000 7f98d2970700 1 -- 172.21.15.12:0/1274945863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645b5a2d170 msgr2=0x5645b5a366f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:39:14.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.896+0000 7f98d2970700 1 --2- 172.21.15.12:0/1274945863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645b5a2d170 0x5645b5a366f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:14.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.896+0000 7f98d2970700 1 -- 172.21.15.12:0/1274945863 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645b589b480 con 0x5645b5a32aa0 2024-05-17T22:39:14.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.896+0000 7f98d196e700 1 --2- 172.21.15.12:0/1274945863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645b5a2d170 0x5645b5a366f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:39:14.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.896+0000 7f98d216f700 1 --2- 172.21.15.12:0/1274945863 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645b58ae9f0 0x5645b5a361b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:39:14.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.896+0000 7f98d2970700 1 --2- 172.21.15.12:0/1274945863 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5645b5a32aa0 0x5645b5a36c30 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f98c4009fd0 tx=0x7f98c400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=972bf977b360eb4c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:39:14.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.896+0000 7f98b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:39:14.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.896+0000 7f98b5ffb700 1 -- 172.21.15.12:0/1274945863 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c402a060 con 0x5645b5a32aa0 2024-05-17T22:39:14.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.896+0000 7f98b5ffb700 1 -- 172.21.15.12:0/1274945863 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f98c4004c40 con 0x5645b5a32aa0 2024-05-17T22:39:14.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.896+0000 7f98e2daeb80 1 -- 172.21.15.12:0/1274945863 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5645b595d1e0 con 0x5645b5a32aa0 2024-05-17T22:39:14.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.897+0000 7f98b5ffb700 1 -- 172.21.15.12:0/1274945863 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f98c4031050 con 0x5645b5a32aa0 2024-05-17T22:39:14.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.897+0000 7f98e2daeb80 10 client.15384 fetch_fsmap learned FSMap version 27 2024-05-17T22:39:14.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.897+0000 7f98e2daeb80 1 -- 172.21.15.12:0/1274945863 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5645b5a43d80 con 0x5645b5a32aa0 2024-05-17T22:39:14.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.897+0000 7f98b5ffb700 1 -- 172.21.15.12:0/1274945863 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5645b595d1e0 con 0x5645b5a32aa0 2024-05-17T22:39:14.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.897+0000 7f98b5ffb700 1 -- 172.21.15.12:0/1274945863 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f98c4011070 con 0x5645b5a32aa0 2024-05-17T22:39:14.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.897+0000 7f98e2daeb80 10 client.15384 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T22:39:14.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.897+0000 7f98e2daeb80 10 client.15384 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:39:14.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.897+0000 7f98e2daeb80 20 client.15384 populate_metadata read hostname 'smithi012' 2024-05-17T22:39:14.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.897+0000 7f98e2daeb80 1 --2- 172.21.15.12:0/1274945863 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5645b5a463c0 0x5645b5a487e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:14.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.897+0000 7f98e2daeb80 4 client.15384 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:39:14.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.897+0000 7f98e2daeb80 1 -- 172.21.15.12:0/1274945863 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645b59569c0 con 0x5645b5a463c0 2024-05-17T22:39:14.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.897+0000 7f98d216f700 1 --2- 172.21.15.12:0/1274945863 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5645b5a463c0 0x5645b5a487e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:14.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.898+0000 7f98d216f700 1 --2- 172.21.15.12:0/1274945863 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5645b5a463c0 0x5645b5a487e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dea3cdfcadc576ff server_cookie=52dd197d97f5f25f in_seq=0 out_seq=0 2024-05-17T22:39:14.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.898+0000 7f98b5ffb700 10 client.15384 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:39:14.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.899+0000 7f98b5ffb700 1 -- 172.21.15.12:0/1274945863 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5645b59569c0 con 0x5645b5a463c0 2024-05-17T22:39:14.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.899+0000 7f98b5ffb700 10 client.15384 handle_command_reply: tid=0 2024-05-17T22:39:14.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.899+0000 7f98e2daeb80 1 client.15384 shutdown 2024-05-17T22:39:14.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.899+0000 7f98e2daeb80 1 -- 172.21.15.12:0/1274945863 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5645b5a32aa0 msgr2=0x5645b5a36c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:14.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.899+0000 7f98e2daeb80 1 --2- 172.21.15.12:0/1274945863 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5645b5a32aa0 0x5645b5a36c30 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f98c4009fd0 tx=0x7f98c400be20 comp rx=0 tx=0).stop 2024-05-17T22:39:14.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.899+0000 7f98e2daeb80 1 -- 172.21.15.12:0/1274945863 shutdown_connections 2024-05-17T22:39:14.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.899+0000 7f98e2daeb80 1 --2- 172.21.15.12:0/1274945863 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5645b5a463c0 0x5645b5a487e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:14.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.899+0000 7f98e2daeb80 1 --2- 172.21.15.12:0/1274945863 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5645b5a32aa0 0x5645b5a36c30 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:14.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.899+0000 7f98e2daeb80 1 --2- 172.21.15.12:0/1274945863 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645b58ae9f0 0x5645b5a361b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:14.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.899+0000 7f98e2daeb80 1 --2- 172.21.15.12:0/1274945863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645b5a2d170 0x5645b5a366f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:14.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.899+0000 7f98e2daeb80 1 -- 172.21.15.12:0/1274945863 >> 172.21.15.12:0/1274945863 conn(0x5645b58f2f00 msgr2=0x5645b5a21ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:39:14.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.899+0000 7f98b5ffb700 0 client.15384 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:39:14.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.899+0000 7f98e2daeb80 1 -- 172.21.15.12:0/1274945863 shutdown_connections 2024-05-17T22:39:14.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.899+0000 7f98e2daeb80 1 -- 172.21.15.12:0/1274945863 wait complete. 2024-05-17T22:39:14.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.899+0000 7f98e2daeb80 20 client.15384 trim_cache size 0 max 16384 2024-05-17T22:39:14.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.922+0000 7f98e2daeb80 1 Processor -- start 2024-05-17T22:39:14.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.922+0000 7f98e2daeb80 1 -- start start 2024-05-17T22:39:14.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.922+0000 7f98e2daeb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645b58ae9f0 0x5645b5a2cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:14.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.922+0000 7f98e2daeb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645b5a2d170 0x5645b5a421d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:14.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.922+0000 7f98e2daeb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5645b5a32aa0 0x5645b5a21eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:14.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.922+0000 7f98e2daeb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc1b0330 con 0x5645b5a32aa0 2024-05-17T22:39:14.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.922+0000 7f98e2daeb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc1adbc0 con 0x5645b5a2d170 2024-05-17T22:39:14.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.922+0000 7f98e2daeb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc1aba10 con 0x5645b58ae9f0 2024-05-17T22:39:14.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.922+0000 7f98d2970700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5645b5a32aa0 0x5645b5a21eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:14.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.922+0000 7f98d2970700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5645b5a32aa0 0x5645b5a21eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55574/0 (socket says 172.21.15.12:55574) 2024-05-17T22:39:14.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.922+0000 7f98d2970700 1 -- 172.21.15.12:0/3946504518 learned_addr learned my addr 172.21.15.12:0/3946504518 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:39:14.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.922+0000 7f98d196e700 1 --2- 172.21.15.12:0/3946504518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645b5a2d170 0x5645b5a421d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:14.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.922+0000 7f98d216f700 1 --2- 172.21.15.12:0/3946504518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645b58ae9f0 0x5645b5a2cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:14.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.922+0000 7f98d2970700 1 -- 172.21.15.12:0/3946504518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645b58ae9f0 msgr2=0x5645b5a2cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:39:14.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.923+0000 7f98d2970700 1 --2- 172.21.15.12:0/3946504518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645b58ae9f0 0x5645b5a2cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:14.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.923+0000 7f98d2970700 1 -- 172.21.15.12:0/3946504518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645b5a2d170 msgr2=0x5645b5a421d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:39:14.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.923+0000 7f98d2970700 1 --2- 172.21.15.12:0/3946504518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645b5a2d170 0x5645b5a421d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:14.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.923+0000 7f98d2970700 1 -- 172.21.15.12:0/3946504518 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645b5a28b10 con 0x5645b5a32aa0 2024-05-17T22:39:14.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.923+0000 7f98d216f700 1 --2- 172.21.15.12:0/3946504518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645b58ae9f0 0x5645b5a2cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:39:14.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.923+0000 7f98d2970700 1 --2- 172.21.15.12:0/3946504518 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5645b5a32aa0 0x5645b5a21eb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f98c40053d0 tx=0x7f98c400b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48ccca7fd6d318a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:39:14.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.923+0000 7f98b7fff700 1 -- 172.21.15.12:0/3946504518 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c402a070 con 0x5645b5a32aa0 2024-05-17T22:39:14.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.923+0000 7f98b7fff700 1 -- 172.21.15.12:0/3946504518 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98c4005450 con 0x5645b5a32aa0 2024-05-17T22:39:14.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.923+0000 7f98b7fff700 1 -- 172.21.15.12:0/3946504518 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c4007b00 con 0x5645b5a32aa0 2024-05-17T22:39:14.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.924+0000 7f98e2daeb80 1 -- 172.21.15.12:0/3946504518 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5645b5a32aa0 msgr2=0x5645b5a21eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:14.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.924+0000 7f98e2daeb80 1 --2- 172.21.15.12:0/3946504518 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5645b5a32aa0 0x5645b5a21eb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f98c40053d0 tx=0x7f98c400b3b0 comp rx=0 tx=0).stop 2024-05-17T22:39:14.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.924+0000 7f98e2daeb80 1 -- 172.21.15.12:0/3946504518 shutdown_connections 2024-05-17T22:39:14.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.924+0000 7f98e2daeb80 1 --2- 172.21.15.12:0/3946504518 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5645b5a32aa0 0x5645b5a21eb0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:14.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.924+0000 7f98e2daeb80 1 --2- 172.21.15.12:0/3946504518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645b58ae9f0 0x5645b5a2cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:14.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.924+0000 7f98e2daeb80 1 --2- 172.21.15.12:0/3946504518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645b5a2d170 0x5645b5a421d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:14.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.924+0000 7f98e2daeb80 1 -- 172.21.15.12:0/3946504518 >> 172.21.15.12:0/3946504518 conn(0x5645b58f2f00 msgr2=0x5645b5a31290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:39:14.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.924+0000 7f98e2daeb80 1 -- 172.21.15.12:0/3946504518 shutdown_connections 2024-05-17T22:39:14.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.924+0000 7f98e2daeb80 1 -- 172.21.15.12:0/3946504518 wait complete. 2024-05-17T22:39:14.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.924+0000 7f98e2daeb80 1 -- start start 2024-05-17T22:39:14.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.924+0000 7f98e2daeb80 1 Processor -- start 2024-05-17T22:39:14.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.925+0000 7f98e2daeb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5645b58ae9f0 0x5645b5aa8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:14.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.925+0000 7f98e2daeb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645b5a2d170 0x5645b5aa93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:14.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.925+0000 7f98e2daeb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645b5a32aa0 0x5645b5a3ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:14.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.925+0000 7f98e2daeb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc1ac080 con 0x5645b58ae9f0 2024-05-17T22:39:14.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.925+0000 7f98e2daeb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc1ad4f0 con 0x5645b5a2d170 2024-05-17T22:39:14.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.925+0000 7f98e2daeb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc1ac750 con 0x5645b5a32aa0 2024-05-17T22:39:14.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.925+0000 7f98e2daeb80 10 client.0 _ll_register_callbacks cb 0x5645b58e5f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:39:14.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.925+0000 7f98d216f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5645b58ae9f0 0x5645b5aa8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:14.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.925+0000 7f98d216f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5645b58ae9f0 0x5645b5aa8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55586/0 (socket says 172.21.15.12:55586) 2024-05-17T22:39:14.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.925+0000 7f98d196e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645b5a2d170 0x5645b5aa93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:14.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.925+0000 7f98d216f700 1 -- 172.21.15.12:0/1193755602 learned_addr learned my addr 172.21.15.12:0/1193755602 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:39:14.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.925+0000 7f98d2970700 1 --2- 172.21.15.12:0/1193755602 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645b5a32aa0 0x5645b5a3ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:14.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.925+0000 7f98d216f700 1 -- 172.21.15.12:0/1193755602 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645b5a32aa0 msgr2=0x5645b5a3ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:39:14.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.925+0000 7f98d216f700 1 --2- 172.21.15.12:0/1193755602 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645b5a32aa0 0x5645b5a3ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:14.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.925+0000 7f98d216f700 1 -- 172.21.15.12:0/1193755602 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645b5a2d170 msgr2=0x5645b5aa93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:39:14.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.925+0000 7f98d216f700 1 --2- 172.21.15.12:0/1193755602 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645b5a2d170 0x5645b5aa93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:14.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.925+0000 7f98d216f700 1 -- 172.21.15.12:0/1193755602 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645b5a3c100 con 0x5645b58ae9f0 2024-05-17T22:39:14.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.925+0000 7f98d216f700 1 --2- 172.21.15.12:0/1193755602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5645b58ae9f0 0x5645b5aa8e70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f98c800e6a0 tx=0x7f98c800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a3266d5fd32f109 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:39:14.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.925+0000 7f98b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:39:14.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.925+0000 7f98b5ffb700 1 -- 172.21.15.12:0/1193755602 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c803d060 con 0x5645b58ae9f0 2024-05-17T22:39:14.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.925+0000 7f98b5ffb700 1 -- 172.21.15.12:0/1193755602 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f98c801ccd0 con 0x5645b58ae9f0 2024-05-17T22:39:14.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.925+0000 7f98e2daeb80 1 -- 172.21.15.12:0/1193755602 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98c4005450 con 0x5645b58ae9f0 2024-05-17T22:39:14.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.926+0000 7f98b5ffb700 1 -- 172.21.15.12:0/1193755602 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f98c8044050 con 0x5645b58ae9f0 2024-05-17T22:39:14.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.926+0000 7f98e2daeb80 10 client.15390 fetch_fsmap learned FSMap version 27 2024-05-17T22:39:14.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.926+0000 7f98e2daeb80 1 -- 172.21.15.12:0/1193755602 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5645b5ab0530 con 0x5645b58ae9f0 2024-05-17T22:39:14.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.926+0000 7f98b5ffb700 1 -- 172.21.15.12:0/1193755602 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98c4005450 con 0x5645b58ae9f0 2024-05-17T22:39:14.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.926+0000 7f98b5ffb700 1 -- 172.21.15.12:0/1193755602 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f98c8005040 con 0x5645b58ae9f0 2024-05-17T22:39:14.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.926+0000 7f98e2daeb80 10 client.15390 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T22:39:14.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.926+0000 7f98e2daeb80 10 client.15390 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:39:14.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.926+0000 7f98e2daeb80 20 client.15390 populate_metadata read hostname 'smithi012' 2024-05-17T22:39:14.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.926+0000 7f98e2daeb80 1 --2- 172.21.15.12:0/1193755602 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5645b5ab0910 0x5645b5ab2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:14.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.926+0000 7f98e2daeb80 4 client.15390 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:39:14.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.926+0000 7f98e2daeb80 1 -- 172.21.15.12:0/1193755602 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5645b5827360 con 0x5645b5ab0910 2024-05-17T22:39:14.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.926+0000 7f98d196e700 1 --2- 172.21.15.12:0/1193755602 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5645b5ab0910 0x5645b5ab2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:14.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.927+0000 7f98b5ffb700 10 client.15390 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:39:14.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.927+0000 7f98d196e700 1 --2- 172.21.15.12:0/1193755602 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5645b5ab0910 0x5645b5ab2d30 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5d3570134a0b93a server_cookie=d34d822cbe08113e in_seq=0 out_seq=0 2024-05-17T22:39:14.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.928+0000 7f98b5ffb700 1 -- 172.21.15.12:0/1193755602 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5645b5827360 con 0x5645b5ab0910 2024-05-17T22:39:14.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.928+0000 7f98b5ffb700 10 client.15390 handle_command_reply: tid=0 2024-05-17T22:39:14.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.928+0000 7f98e2daeb80 1 client.15390 shutdown 2024-05-17T22:39:14.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.928+0000 7f98e2daeb80 1 -- 172.21.15.12:0/1193755602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5645b58ae9f0 msgr2=0x5645b5aa8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:14.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.928+0000 7f98e2daeb80 1 --2- 172.21.15.12:0/1193755602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5645b58ae9f0 0x5645b5aa8e70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f98c800e6a0 tx=0x7f98c800ed60 comp rx=0 tx=0).stop 2024-05-17T22:39:14.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.928+0000 7f98e2daeb80 1 -- 172.21.15.12:0/1193755602 shutdown_connections 2024-05-17T22:39:14.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.928+0000 7f98e2daeb80 1 --2- 172.21.15.12:0/1193755602 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645b5a32aa0 0x5645b5a3ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:14.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.928+0000 7f98e2daeb80 1 --2- 172.21.15.12:0/1193755602 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5645b5ab0910 0x5645b5ab2d30 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:14.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.928+0000 7f98e2daeb80 1 --2- 172.21.15.12:0/1193755602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5645b58ae9f0 0x5645b5aa8e70 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:14.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.928+0000 7f98e2daeb80 1 --2- 172.21.15.12:0/1193755602 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645b5a2d170 0x5645b5aa93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:14.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.928+0000 7f98e2daeb80 1 -- 172.21.15.12:0/1193755602 >> 172.21.15.12:0/1193755602 conn(0x5645b58f2f00 msgr2=0x5645b5a31290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:39:14.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.928+0000 7f98b5ffb700 0 client.15390 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:39:14.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.928+0000 7f98e2daeb80 1 -- 172.21.15.12:0/1193755602 shutdown_connections 2024-05-17T22:39:14.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.928+0000 7f98e2daeb80 1 -- 172.21.15.12:0/1193755602 wait complete. 2024-05-17T22:39:14.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.928+0000 7f98e2daeb80 20 client.15390 trim_cache size 0 max 16384 2024-05-17T22:39:14.936 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:39:14.936 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-17T22:39:14.936 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-17T22:39:14.936 INFO:teuthology.orchestra.run.smithi012.stdout: "65198cc9-f62d-4163-858c-7f47eaf98eef": { 2024-05-17T22:39:14.936 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-17T22:39:14.936 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "65198cc9-f62d-4163-858c-7f47eaf98eef", 2024-05-17T22:39:14.936 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-17T22:39:14.936 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:39:14.936 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:39:14.937 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:39:14.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.929+0000 7f98d43d3700 1 -- 172.21.15.12:0/3305671029 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f98b80b2900 msgr2=0x7f98b80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:14.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.929+0000 7f98d43d3700 1 --2- 172.21.15.12:0/3305671029 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f98b80b2900 0x7f98b80b4e10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f98c8011550 tx=0x7f98c8009260 comp rx=0 tx=0).stop 2024-05-17T22:39:14.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.929+0000 7f98d43d3700 1 -- 172.21.15.12:0/3305671029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98cc107730 msgr2=0x7f98cc19bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:14.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.929+0000 7f98d43d3700 1 --2- 172.21.15.12:0/3305671029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98cc107730 0x7f98cc19bad0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f98bc00e9c0 tx=0x7f98bc00ecd0 comp rx=0 tx=0).stop 2024-05-17T22:39:14.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.929+0000 7f98d43d3700 1 -- 172.21.15.12:0/3305671029 shutdown_connections 2024-05-17T22:39:14.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.929+0000 7f98d43d3700 1 --2- 172.21.15.12:0/3305671029 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f98b80b2900 0x7f98b80b4e10 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:14.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.929+0000 7f98d43d3700 1 --2- 172.21.15.12:0/3305671029 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98cc108080 0x7f98cc1a6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:14.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.929+0000 7f98d43d3700 1 --2- 172.21.15.12:0/3305671029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98cc102e00 0x7f98cc19b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:14.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.929+0000 7f98d43d3700 1 --2- 172.21.15.12:0/3305671029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98cc107730 0x7f98cc19bad0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:14.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.929+0000 7f98d43d3700 1 -- 172.21.15.12:0/3305671029 >> 172.21.15.12:0/3305671029 conn(0x7f98cc079bd0 msgr2=0x7f98cc1057d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:39:14.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.929+0000 7f98d43d3700 1 -- 172.21.15.12:0/3305671029 shutdown_connections 2024-05-17T22:39:14.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:14.929+0000 7f98d43d3700 1 -- 172.21.15.12:0/3305671029 wait complete. 2024-05-17T22:39:15.135 INFO:teuthology.orchestra.run.smithi012.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2024-05-17T22:39:15.138 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-05-17T22:39:15.181 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/: 2024-05-17T22:39:15.181 INFO:teuthology.orchestra.run.smithi012.stdout:total 0 2024-05-17T22:39:15.181 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 17 22:39 . 2024-05-17T22:39:15.181 INFO:teuthology.orchestra.run.smithi012.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 May 17 22:39 .. 2024-05-17T22:39:15.181 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 17 22:39 data 2024-05-17T22:39:15.181 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:39:15.181 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data: 2024-05-17T22:39:15.181 INFO:teuthology.orchestra.run.smithi012.stdout:total 35 2024-05-17T22:39:15.181 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 17 22:39 . 2024-05-17T22:39:15.181 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 17 22:39 .. 2024-05-17T22:39:15.182 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 17 22:39 base 2024-05-17T22:39:15.182 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 May 17 22:39 global 2024-05-17T22:39:15.182 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 log 2024-05-17T22:39:15.182 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 pg_commit_ts 2024-05-17T22:39:15.182 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 pg_dynshmem 2024-05-17T22:39:15.182 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 May 17 22:39 pg_hba.conf 2024-05-17T22:39:15.182 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 May 17 22:39 pg_ident.conf 2024-05-17T22:39:15.182 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 17 22:39 pg_logical 2024-05-17T22:39:15.182 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 17 22:39 pg_multixact 2024-05-17T22:39:15.182 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 pg_notify 2024-05-17T22:39:15.182 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 pg_replslot 2024-05-17T22:39:15.182 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 pg_serial 2024-05-17T22:39:15.182 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 pg_snapshots 2024-05-17T22:39:15.182 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 pg_stat 2024-05-17T22:39:15.183 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 pg_stat_tmp 2024-05-17T22:39:15.183 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 17 22:39 pg_subtrans 2024-05-17T22:39:15.183 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 pg_tblspc 2024-05-17T22:39:15.183 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 pg_twophase 2024-05-17T22:39:15.183 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 17 22:39 PG_VERSION 2024-05-17T22:39:15.183 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 17 22:39 pg_wal 2024-05-17T22:39:15.183 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 17 22:39 pg_xact 2024-05-17T22:39:15.183 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 May 17 22:39 postgresql.auto.conf 2024-05-17T22:39:15.183 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 May 17 22:39 postgresql.conf 2024-05-17T22:39:15.183 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:39:15.183 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/base: 2024-05-17T22:39:15.183 INFO:teuthology.orchestra.run.smithi012.stdout:total 0 2024-05-17T22:39:15.183 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 17 22:39 . 2024-05-17T22:39:15.183 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 17 22:39 .. 2024-05-17T22:39:15.184 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 May 17 22:39 1 2024-05-17T22:39:15.184 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 May 17 22:39 13493 2024-05-17T22:39:15.184 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 May 17 22:39 13494 2024-05-17T22:39:15.184 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:39:15.184 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/base/1: 2024-05-17T22:39:15.184 INFO:teuthology.orchestra.run.smithi012.stdout:total 7777 2024-05-17T22:39:15.184 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 May 17 22:39 . 2024-05-17T22:39:15.184 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 17 22:39 .. 2024-05-17T22:39:15.184 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 112 2024-05-17T22:39:15.184 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 113 2024-05-17T22:39:15.184 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 17 22:39 1247 2024-05-17T22:39:15.184 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 1247_fsm 2024-05-17T22:39:15.184 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1247_vm 2024-05-17T22:39:15.184 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 17 22:39 1249 2024-05-17T22:39:15.185 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 1249_fsm 2024-05-17T22:39:15.185 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1249_vm 2024-05-17T22:39:15.185 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 17 22:39 1255 2024-05-17T22:39:15.185 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 1255_fsm 2024-05-17T22:39:15.185 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1255_vm 2024-05-17T22:39:15.185 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 17 22:39 1259 2024-05-17T22:39:15.185 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 1259_fsm 2024-05-17T22:39:15.185 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1259_vm 2024-05-17T22:39:15.185 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 17 22:39 13344 2024-05-17T22:39:15.185 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 13344_fsm 2024-05-17T22:39:15.185 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13344_vm 2024-05-17T22:39:15.185 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 13346 2024-05-17T22:39:15.185 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13348 2024-05-17T22:39:15.186 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13349 2024-05-17T22:39:15.186 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 13349_fsm 2024-05-17T22:39:15.186 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13349_vm 2024-05-17T22:39:15.186 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 13351 2024-05-17T22:39:15.186 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13353 2024-05-17T22:39:15.186 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13354 2024-05-17T22:39:15.186 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 13354_fsm 2024-05-17T22:39:15.186 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13354_vm 2024-05-17T22:39:15.186 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 13356 2024-05-17T22:39:15.186 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13358 2024-05-17T22:39:15.186 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13359 2024-05-17T22:39:15.186 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 13359_fsm 2024-05-17T22:39:15.186 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13359_vm 2024-05-17T22:39:15.186 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 13361 2024-05-17T22:39:15.186 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13363 2024-05-17T22:39:15.187 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 1417 2024-05-17T22:39:15.187 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 1418 2024-05-17T22:39:15.187 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 174 2024-05-17T22:39:15.187 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 175 2024-05-17T22:39:15.187 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2187 2024-05-17T22:39:15.187 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2224 2024-05-17T22:39:15.187 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2328 2024-05-17T22:39:15.187 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2336 2024-05-17T22:39:15.187 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2337 2024-05-17T22:39:15.187 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2579 2024-05-17T22:39:15.187 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2600 2024-05-17T22:39:15.187 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2600_fsm 2024-05-17T22:39:15.187 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2600_vm 2024-05-17T22:39:15.187 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2601 2024-05-17T22:39:15.188 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2601_fsm 2024-05-17T22:39:15.188 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2601_vm 2024-05-17T22:39:15.188 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 17 22:39 2602 2024-05-17T22:39:15.188 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2602_fsm 2024-05-17T22:39:15.188 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2602_vm 2024-05-17T22:39:15.188 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 2603 2024-05-17T22:39:15.188 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2603_fsm 2024-05-17T22:39:15.188 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2603_vm 2024-05-17T22:39:15.188 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2604 2024-05-17T22:39:15.188 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2605 2024-05-17T22:39:15.188 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2605_fsm 2024-05-17T22:39:15.188 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2605_vm 2024-05-17T22:39:15.188 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2606 2024-05-17T22:39:15.188 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2606_fsm 2024-05-17T22:39:15.188 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2606_vm 2024-05-17T22:39:15.188 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2607 2024-05-17T22:39:15.188 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2607_fsm 2024-05-17T22:39:15.189 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2607_vm 2024-05-17T22:39:15.189 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 17 22:39 2608 2024-05-17T22:39:15.189 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2608_fsm 2024-05-17T22:39:15.189 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2608_vm 2024-05-17T22:39:15.189 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 17 22:39 2609 2024-05-17T22:39:15.189 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2609_fsm 2024-05-17T22:39:15.189 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2609_vm 2024-05-17T22:39:15.189 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 2610 2024-05-17T22:39:15.189 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2610_fsm 2024-05-17T22:39:15.189 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2610_vm 2024-05-17T22:39:15.189 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2611 2024-05-17T22:39:15.189 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2612 2024-05-17T22:39:15.189 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2612_fsm 2024-05-17T22:39:15.189 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2612_vm 2024-05-17T22:39:15.189 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2613 2024-05-17T22:39:15.190 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2615 2024-05-17T22:39:15.190 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2615_fsm 2024-05-17T22:39:15.190 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2615_vm 2024-05-17T22:39:15.190 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2616 2024-05-17T22:39:15.190 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2616_fsm 2024-05-17T22:39:15.190 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2616_vm 2024-05-17T22:39:15.190 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 17 22:39 2617 2024-05-17T22:39:15.190 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2617_fsm 2024-05-17T22:39:15.190 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2617_vm 2024-05-17T22:39:15.190 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 17 22:39 2618 2024-05-17T22:39:15.190 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2618_fsm 2024-05-17T22:39:15.190 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2618_vm 2024-05-17T22:39:15.190 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 17 22:39 2619 2024-05-17T22:39:15.190 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2619_fsm 2024-05-17T22:39:15.190 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2619_vm 2024-05-17T22:39:15.191 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2620 2024-05-17T22:39:15.191 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2650 2024-05-17T22:39:15.191 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2651 2024-05-17T22:39:15.191 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2652 2024-05-17T22:39:15.191 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 22:39 2653 2024-05-17T22:39:15.191 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 22:39 2654 2024-05-17T22:39:15.191 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 2655 2024-05-17T22:39:15.191 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2656 2024-05-17T22:39:15.191 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2657 2024-05-17T22:39:15.191 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 17 22:39 2658 2024-05-17T22:39:15.191 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 17 22:39 2659 2024-05-17T22:39:15.191 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2660 2024-05-17T22:39:15.191 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2661 2024-05-17T22:39:15.191 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 2662 2024-05-17T22:39:15.191 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 17 22:39 2663 2024-05-17T22:39:15.192 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2664 2024-05-17T22:39:15.192 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2665 2024-05-17T22:39:15.192 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2666 2024-05-17T22:39:15.192 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2667 2024-05-17T22:39:15.192 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2668 2024-05-17T22:39:15.192 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2669 2024-05-17T22:39:15.192 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2670 2024-05-17T22:39:15.192 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 17 22:39 2673 2024-05-17T22:39:15.192 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 17 22:39 2674 2024-05-17T22:39:15.192 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 17 22:39 2675 2024-05-17T22:39:15.192 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2678 2024-05-17T22:39:15.192 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2679 2024-05-17T22:39:15.192 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2680 2024-05-17T22:39:15.192 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2681 2024-05-17T22:39:15.193 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2682 2024-05-17T22:39:15.193 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2683 2024-05-17T22:39:15.193 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2684 2024-05-17T22:39:15.193 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2685 2024-05-17T22:39:15.193 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2686 2024-05-17T22:39:15.193 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2687 2024-05-17T22:39:15.193 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 22:39 2688 2024-05-17T22:39:15.193 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 17 22:39 2689 2024-05-17T22:39:15.193 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 17 22:39 2690 2024-05-17T22:39:15.193 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 17 22:39 2691 2024-05-17T22:39:15.193 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2692 2024-05-17T22:39:15.193 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2693 2024-05-17T22:39:15.193 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2696 2024-05-17T22:39:15.193 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2699 2024-05-17T22:39:15.194 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2701 2024-05-17T22:39:15.194 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2702 2024-05-17T22:39:15.194 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 2703 2024-05-17T22:39:15.194 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 22:39 2704 2024-05-17T22:39:15.194 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2753 2024-05-17T22:39:15.194 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2753_fsm 2024-05-17T22:39:15.194 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2753_vm 2024-05-17T22:39:15.194 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2754 2024-05-17T22:39:15.194 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2755 2024-05-17T22:39:15.194 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 2756 2024-05-17T22:39:15.194 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 2757 2024-05-17T22:39:15.194 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2830 2024-05-17T22:39:15.194 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2831 2024-05-17T22:39:15.194 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2832 2024-05-17T22:39:15.195 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2833 2024-05-17T22:39:15.195 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2834 2024-05-17T22:39:15.195 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2835 2024-05-17T22:39:15.195 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2836 2024-05-17T22:39:15.195 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2837 2024-05-17T22:39:15.195 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 17 22:39 2838 2024-05-17T22:39:15.195 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2838_fsm 2024-05-17T22:39:15.195 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2838_vm 2024-05-17T22:39:15.195 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2839 2024-05-17T22:39:15.195 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2840 2024-05-17T22:39:15.195 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2840_fsm 2024-05-17T22:39:15.195 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2840_vm 2024-05-17T22:39:15.195 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2841 2024-05-17T22:39:15.196 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2995 2024-05-17T22:39:15.196 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2996 2024-05-17T22:39:15.196 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3079 2024-05-17T22:39:15.196 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3079_fsm 2024-05-17T22:39:15.196 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3079_vm 2024-05-17T22:39:15.196 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3080 2024-05-17T22:39:15.196 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3081 2024-05-17T22:39:15.196 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 22:39 3085 2024-05-17T22:39:15.196 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3118 2024-05-17T22:39:15.196 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3119 2024-05-17T22:39:15.196 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 17 22:39 3164 2024-05-17T22:39:15.196 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3256 2024-05-17T22:39:15.196 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3257 2024-05-17T22:39:15.197 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3258 2024-05-17T22:39:15.197 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3350 2024-05-17T22:39:15.197 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3351 2024-05-17T22:39:15.197 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3379 2024-05-17T22:39:15.197 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3380 2024-05-17T22:39:15.197 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3381 2024-05-17T22:39:15.197 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3394 2024-05-17T22:39:15.197 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3394_fsm 2024-05-17T22:39:15.197 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3394_vm 2024-05-17T22:39:15.197 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3395 2024-05-17T22:39:15.197 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3429 2024-05-17T22:39:15.197 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3430 2024-05-17T22:39:15.197 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3431 2024-05-17T22:39:15.197 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3433 2024-05-17T22:39:15.197 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3439 2024-05-17T22:39:15.197 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3440 2024-05-17T22:39:15.198 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 22:39 3455 2024-05-17T22:39:15.198 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 17 22:39 3456 2024-05-17T22:39:15.198 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3456_fsm 2024-05-17T22:39:15.198 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3456_vm 2024-05-17T22:39:15.198 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3466 2024-05-17T22:39:15.198 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3467 2024-05-17T22:39:15.198 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3468 2024-05-17T22:39:15.198 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3501 2024-05-17T22:39:15.198 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3502 2024-05-17T22:39:15.198 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3503 2024-05-17T22:39:15.198 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3534 2024-05-17T22:39:15.198 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3541 2024-05-17T22:39:15.198 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3541_fsm 2024-05-17T22:39:15.198 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3541_vm 2024-05-17T22:39:15.199 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3542 2024-05-17T22:39:15.199 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3574 2024-05-17T22:39:15.199 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3575 2024-05-17T22:39:15.199 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3576 2024-05-17T22:39:15.199 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3596 2024-05-17T22:39:15.199 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3597 2024-05-17T22:39:15.199 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3598 2024-05-17T22:39:15.199 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3599 2024-05-17T22:39:15.199 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3600 2024-05-17T22:39:15.199 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3600_fsm 2024-05-17T22:39:15.199 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3600_vm 2024-05-17T22:39:15.199 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3601 2024-05-17T22:39:15.199 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3601_fsm 2024-05-17T22:39:15.199 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3601_vm 2024-05-17T22:39:15.199 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3602 2024-05-17T22:39:15.200 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3602_fsm 2024-05-17T22:39:15.200 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3602_vm 2024-05-17T22:39:15.200 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3603 2024-05-17T22:39:15.200 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3603_fsm 2024-05-17T22:39:15.200 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3603_vm 2024-05-17T22:39:15.200 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3604 2024-05-17T22:39:15.200 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3605 2024-05-17T22:39:15.200 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3606 2024-05-17T22:39:15.200 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3607 2024-05-17T22:39:15.200 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3608 2024-05-17T22:39:15.200 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 3609 2024-05-17T22:39:15.200 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3712 2024-05-17T22:39:15.200 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3764 2024-05-17T22:39:15.200 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3764_fsm 2024-05-17T22:39:15.200 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3764_vm 2024-05-17T22:39:15.200 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3766 2024-05-17T22:39:15.201 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3767 2024-05-17T22:39:15.201 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3997 2024-05-17T22:39:15.201 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4143 2024-05-17T22:39:15.201 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4144 2024-05-17T22:39:15.201 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4145 2024-05-17T22:39:15.201 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4146 2024-05-17T22:39:15.201 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4147 2024-05-17T22:39:15.201 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4148 2024-05-17T22:39:15.201 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4149 2024-05-17T22:39:15.201 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4150 2024-05-17T22:39:15.201 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4151 2024-05-17T22:39:15.201 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4152 2024-05-17T22:39:15.201 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4153 2024-05-17T22:39:15.201 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4154 2024-05-17T22:39:15.201 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4155 2024-05-17T22:39:15.202 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4156 2024-05-17T22:39:15.202 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4157 2024-05-17T22:39:15.202 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4158 2024-05-17T22:39:15.202 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4159 2024-05-17T22:39:15.202 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4160 2024-05-17T22:39:15.202 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4161 2024-05-17T22:39:15.202 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4162 2024-05-17T22:39:15.202 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4163 2024-05-17T22:39:15.202 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4164 2024-05-17T22:39:15.202 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4165 2024-05-17T22:39:15.208 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4166 2024-05-17T22:39:15.208 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4167 2024-05-17T22:39:15.208 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4168 2024-05-17T22:39:15.209 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4169 2024-05-17T22:39:15.209 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4170 2024-05-17T22:39:15.209 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4171 2024-05-17T22:39:15.209 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4172 2024-05-17T22:39:15.209 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4173 2024-05-17T22:39:15.209 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4174 2024-05-17T22:39:15.209 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 5002 2024-05-17T22:39:15.209 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 548 2024-05-17T22:39:15.209 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 549 2024-05-17T22:39:15.209 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 6102 2024-05-17T22:39:15.209 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 6104 2024-05-17T22:39:15.209 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 6106 2024-05-17T22:39:15.209 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 6110 2024-05-17T22:39:15.209 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 6111 2024-05-17T22:39:15.209 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 6112 2024-05-17T22:39:15.210 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 6113 2024-05-17T22:39:15.210 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 6117 2024-05-17T22:39:15.210 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 826 2024-05-17T22:39:15.210 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 827 2024-05-17T22:39:15.210 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 828 2024-05-17T22:39:15.210 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 17 22:39 pg_filenode.map 2024-05-17T22:39:15.210 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 17 22:39 PG_VERSION 2024-05-17T22:39:15.210 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:39:15.210 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-05-17T22:39:15.210 INFO:teuthology.orchestra.run.smithi012.stdout:total 7777 2024-05-17T22:39:15.210 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 May 17 22:39 . 2024-05-17T22:39:15.210 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 17 22:39 .. 2024-05-17T22:39:15.210 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 112 2024-05-17T22:39:15.210 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 113 2024-05-17T22:39:15.211 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 17 22:39 1247 2024-05-17T22:39:15.211 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 1247_fsm 2024-05-17T22:39:15.211 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1247_vm 2024-05-17T22:39:15.211 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 17 22:39 1249 2024-05-17T22:39:15.211 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 1249_fsm 2024-05-17T22:39:15.211 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1249_vm 2024-05-17T22:39:15.211 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 17 22:39 1255 2024-05-17T22:39:15.211 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 1255_fsm 2024-05-17T22:39:15.211 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1255_vm 2024-05-17T22:39:15.211 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 17 22:39 1259 2024-05-17T22:39:15.211 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 1259_fsm 2024-05-17T22:39:15.211 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1259_vm 2024-05-17T22:39:15.211 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 17 22:39 13344 2024-05-17T22:39:15.211 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 13344_fsm 2024-05-17T22:39:15.212 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13344_vm 2024-05-17T22:39:15.212 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 13346 2024-05-17T22:39:15.212 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13348 2024-05-17T22:39:15.212 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13349 2024-05-17T22:39:15.212 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 13349_fsm 2024-05-17T22:39:15.212 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13349_vm 2024-05-17T22:39:15.212 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 13351 2024-05-17T22:39:15.212 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13353 2024-05-17T22:39:15.212 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13354 2024-05-17T22:39:15.212 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 13354_fsm 2024-05-17T22:39:15.212 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13354_vm 2024-05-17T22:39:15.212 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 13356 2024-05-17T22:39:15.212 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13358 2024-05-17T22:39:15.212 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13359 2024-05-17T22:39:15.213 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 13359_fsm 2024-05-17T22:39:15.213 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13359_vm 2024-05-17T22:39:15.213 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 13361 2024-05-17T22:39:15.213 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13363 2024-05-17T22:39:15.213 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 1417 2024-05-17T22:39:15.213 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 1418 2024-05-17T22:39:15.213 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 174 2024-05-17T22:39:15.213 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 175 2024-05-17T22:39:15.213 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2187 2024-05-17T22:39:15.213 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2224 2024-05-17T22:39:15.213 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2328 2024-05-17T22:39:15.213 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2336 2024-05-17T22:39:15.213 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2337 2024-05-17T22:39:15.213 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2579 2024-05-17T22:39:15.214 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2600 2024-05-17T22:39:15.214 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2600_fsm 2024-05-17T22:39:15.214 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2600_vm 2024-05-17T22:39:15.214 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2601 2024-05-17T22:39:15.214 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2601_fsm 2024-05-17T22:39:15.214 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2601_vm 2024-05-17T22:39:15.214 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 17 22:39 2602 2024-05-17T22:39:15.214 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2602_fsm 2024-05-17T22:39:15.214 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2602_vm 2024-05-17T22:39:15.214 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 2603 2024-05-17T22:39:15.214 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2603_fsm 2024-05-17T22:39:15.214 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2603_vm 2024-05-17T22:39:15.214 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2604 2024-05-17T22:39:15.215 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2605 2024-05-17T22:39:15.215 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2605_fsm 2024-05-17T22:39:15.215 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2605_vm 2024-05-17T22:39:15.215 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2606 2024-05-17T22:39:15.215 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2606_fsm 2024-05-17T22:39:15.215 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2606_vm 2024-05-17T22:39:15.215 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2607 2024-05-17T22:39:15.215 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2607_fsm 2024-05-17T22:39:15.215 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2607_vm 2024-05-17T22:39:15.215 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 17 22:39 2608 2024-05-17T22:39:15.215 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2608_fsm 2024-05-17T22:39:15.215 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2608_vm 2024-05-17T22:39:15.215 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 17 22:39 2609 2024-05-17T22:39:15.216 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2609_fsm 2024-05-17T22:39:15.216 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2609_vm 2024-05-17T22:39:15.216 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 2610 2024-05-17T22:39:15.216 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2610_fsm 2024-05-17T22:39:15.216 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2610_vm 2024-05-17T22:39:15.216 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2611 2024-05-17T22:39:15.216 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2612 2024-05-17T22:39:15.216 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2612_fsm 2024-05-17T22:39:15.216 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2612_vm 2024-05-17T22:39:15.216 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2613 2024-05-17T22:39:15.216 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2615 2024-05-17T22:39:15.216 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2615_fsm 2024-05-17T22:39:15.217 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2615_vm 2024-05-17T22:39:15.217 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2616 2024-05-17T22:39:15.217 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2616_fsm 2024-05-17T22:39:15.217 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2616_vm 2024-05-17T22:39:15.217 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 17 22:39 2617 2024-05-17T22:39:15.217 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2617_fsm 2024-05-17T22:39:15.217 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2617_vm 2024-05-17T22:39:15.217 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 17 22:39 2618 2024-05-17T22:39:15.217 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2618_fsm 2024-05-17T22:39:15.217 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2618_vm 2024-05-17T22:39:15.217 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 17 22:39 2619 2024-05-17T22:39:15.217 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2619_fsm 2024-05-17T22:39:15.217 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2619_vm 2024-05-17T22:39:15.217 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2620 2024-05-17T22:39:15.218 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2650 2024-05-17T22:39:15.218 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2651 2024-05-17T22:39:15.218 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2652 2024-05-17T22:39:15.218 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 22:39 2653 2024-05-17T22:39:15.218 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 22:39 2654 2024-05-17T22:39:15.218 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 2655 2024-05-17T22:39:15.218 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2656 2024-05-17T22:39:15.218 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2657 2024-05-17T22:39:15.218 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 17 22:39 2658 2024-05-17T22:39:15.218 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 17 22:39 2659 2024-05-17T22:39:15.218 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2660 2024-05-17T22:39:15.218 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2661 2024-05-17T22:39:15.218 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 2662 2024-05-17T22:39:15.219 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 17 22:39 2663 2024-05-17T22:39:15.219 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2664 2024-05-17T22:39:15.219 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2665 2024-05-17T22:39:15.219 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2666 2024-05-17T22:39:15.219 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2667 2024-05-17T22:39:15.219 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2668 2024-05-17T22:39:15.219 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2669 2024-05-17T22:39:15.219 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2670 2024-05-17T22:39:15.219 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 17 22:39 2673 2024-05-17T22:39:15.219 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 17 22:39 2674 2024-05-17T22:39:15.219 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 17 22:39 2675 2024-05-17T22:39:15.219 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2678 2024-05-17T22:39:15.219 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2679 2024-05-17T22:39:15.219 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2680 2024-05-17T22:39:15.220 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2681 2024-05-17T22:39:15.220 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2682 2024-05-17T22:39:15.220 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2683 2024-05-17T22:39:15.220 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2684 2024-05-17T22:39:15.220 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2685 2024-05-17T22:39:15.220 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2686 2024-05-17T22:39:15.220 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2687 2024-05-17T22:39:15.220 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 22:39 2688 2024-05-17T22:39:15.220 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 17 22:39 2689 2024-05-17T22:39:15.220 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 17 22:39 2690 2024-05-17T22:39:15.220 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 17 22:39 2691 2024-05-17T22:39:15.220 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2692 2024-05-17T22:39:15.220 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2693 2024-05-17T22:39:15.220 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2696 2024-05-17T22:39:15.221 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2699 2024-05-17T22:39:15.221 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2701 2024-05-17T22:39:15.221 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2702 2024-05-17T22:39:15.221 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 2703 2024-05-17T22:39:15.221 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 22:39 2704 2024-05-17T22:39:15.221 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2753 2024-05-17T22:39:15.221 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2753_fsm 2024-05-17T22:39:15.221 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2753_vm 2024-05-17T22:39:15.221 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2754 2024-05-17T22:39:15.221 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2755 2024-05-17T22:39:15.221 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 2756 2024-05-17T22:39:15.221 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 2757 2024-05-17T22:39:15.221 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2830 2024-05-17T22:39:15.221 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2831 2024-05-17T22:39:15.221 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2832 2024-05-17T22:39:15.222 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2833 2024-05-17T22:39:15.222 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2834 2024-05-17T22:39:15.222 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2835 2024-05-17T22:39:15.222 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2836 2024-05-17T22:39:15.222 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2837 2024-05-17T22:39:15.222 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 17 22:39 2838 2024-05-17T22:39:15.222 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2838_fsm 2024-05-17T22:39:15.222 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2838_vm 2024-05-17T22:39:15.222 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2839 2024-05-17T22:39:15.222 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2840 2024-05-17T22:39:15.222 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2840_fsm 2024-05-17T22:39:15.222 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2840_vm 2024-05-17T22:39:15.222 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2841 2024-05-17T22:39:15.222 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2995 2024-05-17T22:39:15.223 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2996 2024-05-17T22:39:15.223 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3079 2024-05-17T22:39:15.223 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3079_fsm 2024-05-17T22:39:15.223 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3079_vm 2024-05-17T22:39:15.223 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3080 2024-05-17T22:39:15.223 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3081 2024-05-17T22:39:15.223 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 22:39 3085 2024-05-17T22:39:15.223 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3118 2024-05-17T22:39:15.223 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3119 2024-05-17T22:39:15.223 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 17 22:39 3164 2024-05-17T22:39:15.223 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3256 2024-05-17T22:39:15.223 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3257 2024-05-17T22:39:15.223 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3258 2024-05-17T22:39:15.224 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3350 2024-05-17T22:39:15.224 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3351 2024-05-17T22:39:15.224 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3379 2024-05-17T22:39:15.224 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3380 2024-05-17T22:39:15.224 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3381 2024-05-17T22:39:15.224 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3394 2024-05-17T22:39:15.224 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3394_fsm 2024-05-17T22:39:15.224 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3394_vm 2024-05-17T22:39:15.224 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3395 2024-05-17T22:39:15.224 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3429 2024-05-17T22:39:15.224 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3430 2024-05-17T22:39:15.224 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3431 2024-05-17T22:39:15.224 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3433 2024-05-17T22:39:15.224 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3439 2024-05-17T22:39:15.225 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3440 2024-05-17T22:39:15.225 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 22:39 3455 2024-05-17T22:39:15.225 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 17 22:39 3456 2024-05-17T22:39:15.225 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3456_fsm 2024-05-17T22:39:15.225 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3456_vm 2024-05-17T22:39:15.225 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3466 2024-05-17T22:39:15.225 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3467 2024-05-17T22:39:15.225 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3468 2024-05-17T22:39:15.225 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3501 2024-05-17T22:39:15.225 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3502 2024-05-17T22:39:15.225 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3503 2024-05-17T22:39:15.225 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3534 2024-05-17T22:39:15.225 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3541 2024-05-17T22:39:15.225 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3541_fsm 2024-05-17T22:39:15.226 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3541_vm 2024-05-17T22:39:15.226 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3542 2024-05-17T22:39:15.226 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3574 2024-05-17T22:39:15.226 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3575 2024-05-17T22:39:15.226 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3576 2024-05-17T22:39:15.226 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3596 2024-05-17T22:39:15.226 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3597 2024-05-17T22:39:15.226 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3598 2024-05-17T22:39:15.226 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3599 2024-05-17T22:39:15.226 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3600 2024-05-17T22:39:15.226 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3600_fsm 2024-05-17T22:39:15.226 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3600_vm 2024-05-17T22:39:15.226 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3601 2024-05-17T22:39:15.226 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3601_fsm 2024-05-17T22:39:15.227 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3601_vm 2024-05-17T22:39:15.227 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3602 2024-05-17T22:39:15.227 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3602_fsm 2024-05-17T22:39:15.227 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3602_vm 2024-05-17T22:39:15.227 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3603 2024-05-17T22:39:15.227 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3603_fsm 2024-05-17T22:39:15.227 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3603_vm 2024-05-17T22:39:15.227 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3604 2024-05-17T22:39:15.227 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3605 2024-05-17T22:39:15.227 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3606 2024-05-17T22:39:15.227 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3607 2024-05-17T22:39:15.227 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3608 2024-05-17T22:39:15.227 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 3609 2024-05-17T22:39:15.228 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3712 2024-05-17T22:39:15.228 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3764 2024-05-17T22:39:15.228 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3764_fsm 2024-05-17T22:39:15.228 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3764_vm 2024-05-17T22:39:15.228 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3766 2024-05-17T22:39:15.228 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3767 2024-05-17T22:39:15.228 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3997 2024-05-17T22:39:15.228 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4143 2024-05-17T22:39:15.228 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4144 2024-05-17T22:39:15.228 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4145 2024-05-17T22:39:15.228 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4146 2024-05-17T22:39:15.228 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4147 2024-05-17T22:39:15.228 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4148 2024-05-17T22:39:15.228 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4149 2024-05-17T22:39:15.229 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4150 2024-05-17T22:39:15.229 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4151 2024-05-17T22:39:15.229 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4152 2024-05-17T22:39:15.229 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4153 2024-05-17T22:39:15.229 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4154 2024-05-17T22:39:15.229 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4155 2024-05-17T22:39:15.229 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4156 2024-05-17T22:39:15.229 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4157 2024-05-17T22:39:15.229 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4158 2024-05-17T22:39:15.229 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4159 2024-05-17T22:39:15.229 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4160 2024-05-17T22:39:15.229 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4161 2024-05-17T22:39:15.229 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4162 2024-05-17T22:39:15.230 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4163 2024-05-17T22:39:15.230 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4164 2024-05-17T22:39:15.230 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4165 2024-05-17T22:39:15.230 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4166 2024-05-17T22:39:15.230 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4167 2024-05-17T22:39:15.230 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4168 2024-05-17T22:39:15.230 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4169 2024-05-17T22:39:15.230 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4170 2024-05-17T22:39:15.230 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4171 2024-05-17T22:39:15.230 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4172 2024-05-17T22:39:15.230 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4173 2024-05-17T22:39:15.230 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4174 2024-05-17T22:39:15.230 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 5002 2024-05-17T22:39:15.231 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 548 2024-05-17T22:39:15.231 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 549 2024-05-17T22:39:15.231 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 6102 2024-05-17T22:39:15.231 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 6104 2024-05-17T22:39:15.231 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 6106 2024-05-17T22:39:15.231 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 6110 2024-05-17T22:39:15.231 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 6111 2024-05-17T22:39:15.231 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 6112 2024-05-17T22:39:15.231 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 6113 2024-05-17T22:39:15.231 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 6117 2024-05-17T22:39:15.231 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 826 2024-05-17T22:39:15.231 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 827 2024-05-17T22:39:15.231 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 828 2024-05-17T22:39:15.232 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 17 22:39 pg_filenode.map 2024-05-17T22:39:15.232 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 17 22:39 PG_VERSION 2024-05-17T22:39:15.232 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:39:15.232 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-05-17T22:39:15.232 INFO:teuthology.orchestra.run.smithi012.stdout:total 7777 2024-05-17T22:39:15.232 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 May 17 22:39 . 2024-05-17T22:39:15.232 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 17 22:39 .. 2024-05-17T22:39:15.232 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 112 2024-05-17T22:39:15.232 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 113 2024-05-17T22:39:15.232 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 17 22:39 1247 2024-05-17T22:39:15.232 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 1247_fsm 2024-05-17T22:39:15.232 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1247_vm 2024-05-17T22:39:15.232 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 17 22:39 1249 2024-05-17T22:39:15.232 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 1249_fsm 2024-05-17T22:39:15.233 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1249_vm 2024-05-17T22:39:15.233 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 17 22:39 1255 2024-05-17T22:39:15.233 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 1255_fsm 2024-05-17T22:39:15.233 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1255_vm 2024-05-17T22:39:15.233 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 17 22:39 1259 2024-05-17T22:39:15.233 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 1259_fsm 2024-05-17T22:39:15.233 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1259_vm 2024-05-17T22:39:15.233 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 17 22:39 13344 2024-05-17T22:39:15.233 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 13344_fsm 2024-05-17T22:39:15.233 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13344_vm 2024-05-17T22:39:15.233 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 13346 2024-05-17T22:39:15.233 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13348 2024-05-17T22:39:15.233 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13349 2024-05-17T22:39:15.233 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 13349_fsm 2024-05-17T22:39:15.233 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13349_vm 2024-05-17T22:39:15.234 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 13351 2024-05-17T22:39:15.234 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13353 2024-05-17T22:39:15.234 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13354 2024-05-17T22:39:15.234 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 13354_fsm 2024-05-17T22:39:15.234 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13354_vm 2024-05-17T22:39:15.234 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 13356 2024-05-17T22:39:15.234 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13358 2024-05-17T22:39:15.234 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13359 2024-05-17T22:39:15.234 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 13359_fsm 2024-05-17T22:39:15.234 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13359_vm 2024-05-17T22:39:15.234 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 13361 2024-05-17T22:39:15.234 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13363 2024-05-17T22:39:15.234 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 1417 2024-05-17T22:39:15.234 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 1418 2024-05-17T22:39:15.235 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 174 2024-05-17T22:39:15.235 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 175 2024-05-17T22:39:15.235 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2187 2024-05-17T22:39:15.235 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2224 2024-05-17T22:39:15.235 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2328 2024-05-17T22:39:15.235 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2336 2024-05-17T22:39:15.235 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2337 2024-05-17T22:39:15.235 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2579 2024-05-17T22:39:15.235 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2600 2024-05-17T22:39:15.235 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2600_fsm 2024-05-17T22:39:15.235 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2600_vm 2024-05-17T22:39:15.235 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2601 2024-05-17T22:39:15.235 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2601_fsm 2024-05-17T22:39:15.235 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2601_vm 2024-05-17T22:39:15.236 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 17 22:39 2602 2024-05-17T22:39:15.236 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2602_fsm 2024-05-17T22:39:15.236 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2602_vm 2024-05-17T22:39:15.236 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 2603 2024-05-17T22:39:15.236 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2603_fsm 2024-05-17T22:39:15.236 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2603_vm 2024-05-17T22:39:15.236 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2604 2024-05-17T22:39:15.236 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2605 2024-05-17T22:39:15.236 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2605_fsm 2024-05-17T22:39:15.236 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2605_vm 2024-05-17T22:39:15.236 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2606 2024-05-17T22:39:15.236 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2606_fsm 2024-05-17T22:39:15.236 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2606_vm 2024-05-17T22:39:15.236 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2607 2024-05-17T22:39:15.236 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2607_fsm 2024-05-17T22:39:15.237 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2607_vm 2024-05-17T22:39:15.237 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 17 22:39 2608 2024-05-17T22:39:15.237 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2608_fsm 2024-05-17T22:39:15.237 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2608_vm 2024-05-17T22:39:15.237 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 17 22:39 2609 2024-05-17T22:39:15.237 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2609_fsm 2024-05-17T22:39:15.237 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2609_vm 2024-05-17T22:39:15.237 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 2610 2024-05-17T22:39:15.237 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2610_fsm 2024-05-17T22:39:15.237 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2610_vm 2024-05-17T22:39:15.237 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2611 2024-05-17T22:39:15.237 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2612 2024-05-17T22:39:15.237 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2612_fsm 2024-05-17T22:39:15.237 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2612_vm 2024-05-17T22:39:15.238 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2613 2024-05-17T22:39:15.238 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2615 2024-05-17T22:39:15.238 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2615_fsm 2024-05-17T22:39:15.238 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2615_vm 2024-05-17T22:39:15.238 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2616 2024-05-17T22:39:15.238 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2616_fsm 2024-05-17T22:39:15.238 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2616_vm 2024-05-17T22:39:15.238 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 17 22:39 2617 2024-05-17T22:39:15.238 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2617_fsm 2024-05-17T22:39:15.238 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2617_vm 2024-05-17T22:39:15.238 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 17 22:39 2618 2024-05-17T22:39:15.238 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2618_fsm 2024-05-17T22:39:15.238 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2618_vm 2024-05-17T22:39:15.238 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 17 22:39 2619 2024-05-17T22:39:15.238 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2619_fsm 2024-05-17T22:39:15.239 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2619_vm 2024-05-17T22:39:15.239 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2620 2024-05-17T22:39:15.239 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2650 2024-05-17T22:39:15.239 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2651 2024-05-17T22:39:15.239 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2652 2024-05-17T22:39:15.239 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 22:39 2653 2024-05-17T22:39:15.239 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 22:39 2654 2024-05-17T22:39:15.239 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 2655 2024-05-17T22:39:15.239 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2656 2024-05-17T22:39:15.239 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2657 2024-05-17T22:39:15.239 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 17 22:39 2658 2024-05-17T22:39:15.239 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 17 22:39 2659 2024-05-17T22:39:15.239 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2660 2024-05-17T22:39:15.239 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2661 2024-05-17T22:39:15.240 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 2662 2024-05-17T22:39:15.240 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 17 22:39 2663 2024-05-17T22:39:15.240 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2664 2024-05-17T22:39:15.240 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2665 2024-05-17T22:39:15.240 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2666 2024-05-17T22:39:15.240 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2667 2024-05-17T22:39:15.240 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2668 2024-05-17T22:39:15.240 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2669 2024-05-17T22:39:15.240 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2670 2024-05-17T22:39:15.240 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 17 22:39 2673 2024-05-17T22:39:15.240 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 17 22:39 2674 2024-05-17T22:39:15.240 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 17 22:39 2675 2024-05-17T22:39:15.240 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2678 2024-05-17T22:39:15.240 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2679 2024-05-17T22:39:15.241 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2680 2024-05-17T22:39:15.241 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2681 2024-05-17T22:39:15.241 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2682 2024-05-17T22:39:15.241 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2683 2024-05-17T22:39:15.241 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2684 2024-05-17T22:39:15.241 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2685 2024-05-17T22:39:15.241 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2686 2024-05-17T22:39:15.241 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2687 2024-05-17T22:39:15.241 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 22:39 2688 2024-05-17T22:39:15.241 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 17 22:39 2689 2024-05-17T22:39:15.241 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 17 22:39 2690 2024-05-17T22:39:15.241 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 17 22:39 2691 2024-05-17T22:39:15.241 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2692 2024-05-17T22:39:15.241 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2693 2024-05-17T22:39:15.242 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2696 2024-05-17T22:39:15.242 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2699 2024-05-17T22:39:15.242 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2701 2024-05-17T22:39:15.242 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2702 2024-05-17T22:39:15.242 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 2703 2024-05-17T22:39:15.242 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 22:39 2704 2024-05-17T22:39:15.242 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2753 2024-05-17T22:39:15.242 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2753_fsm 2024-05-17T22:39:15.242 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2753_vm 2024-05-17T22:39:15.242 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2754 2024-05-17T22:39:15.242 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2755 2024-05-17T22:39:15.242 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 2756 2024-05-17T22:39:15.242 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 2757 2024-05-17T22:39:15.243 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2830 2024-05-17T22:39:15.243 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2831 2024-05-17T22:39:15.243 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2832 2024-05-17T22:39:15.243 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2833 2024-05-17T22:39:15.243 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2834 2024-05-17T22:39:15.243 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2835 2024-05-17T22:39:15.243 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2836 2024-05-17T22:39:15.243 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2837 2024-05-17T22:39:15.243 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 17 22:39 2838 2024-05-17T22:39:15.243 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2838_fsm 2024-05-17T22:39:15.243 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2838_vm 2024-05-17T22:39:15.243 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2839 2024-05-17T22:39:15.243 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2840 2024-05-17T22:39:15.243 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2840_fsm 2024-05-17T22:39:15.243 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2840_vm 2024-05-17T22:39:15.244 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2841 2024-05-17T22:39:15.244 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2995 2024-05-17T22:39:15.244 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2996 2024-05-17T22:39:15.244 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3079 2024-05-17T22:39:15.244 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3079_fsm 2024-05-17T22:39:15.244 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3079_vm 2024-05-17T22:39:15.244 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3080 2024-05-17T22:39:15.244 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3081 2024-05-17T22:39:15.244 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 22:39 3085 2024-05-17T22:39:15.244 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3118 2024-05-17T22:39:15.244 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3119 2024-05-17T22:39:15.244 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 17 22:39 3164 2024-05-17T22:39:15.244 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3256 2024-05-17T22:39:15.244 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3257 2024-05-17T22:39:15.245 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3258 2024-05-17T22:39:15.245 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3350 2024-05-17T22:39:15.245 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3351 2024-05-17T22:39:15.245 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3379 2024-05-17T22:39:15.245 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3380 2024-05-17T22:39:15.245 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3381 2024-05-17T22:39:15.245 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3394 2024-05-17T22:39:15.245 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3394_fsm 2024-05-17T22:39:15.245 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3394_vm 2024-05-17T22:39:15.245 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3395 2024-05-17T22:39:15.245 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3429 2024-05-17T22:39:15.245 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3430 2024-05-17T22:39:15.245 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3431 2024-05-17T22:39:15.245 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3433 2024-05-17T22:39:15.246 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3439 2024-05-17T22:39:15.246 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3440 2024-05-17T22:39:15.246 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 22:39 3455 2024-05-17T22:39:15.246 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 17 22:39 3456 2024-05-17T22:39:15.246 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3456_fsm 2024-05-17T22:39:15.246 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3456_vm 2024-05-17T22:39:15.246 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3466 2024-05-17T22:39:15.246 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3467 2024-05-17T22:39:15.246 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3468 2024-05-17T22:39:15.246 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3501 2024-05-17T22:39:15.246 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3502 2024-05-17T22:39:15.246 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3503 2024-05-17T22:39:15.246 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3534 2024-05-17T22:39:15.247 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3541 2024-05-17T22:39:15.247 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3541_fsm 2024-05-17T22:39:15.247 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3541_vm 2024-05-17T22:39:15.247 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3542 2024-05-17T22:39:15.247 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3574 2024-05-17T22:39:15.247 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3575 2024-05-17T22:39:15.247 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3576 2024-05-17T22:39:15.247 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3596 2024-05-17T22:39:15.247 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3597 2024-05-17T22:39:15.247 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3598 2024-05-17T22:39:15.247 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3599 2024-05-17T22:39:15.247 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3600 2024-05-17T22:39:15.247 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3600_fsm 2024-05-17T22:39:15.248 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3600_vm 2024-05-17T22:39:15.248 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3601 2024-05-17T22:39:15.248 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3601_fsm 2024-05-17T22:39:15.248 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3601_vm 2024-05-17T22:39:15.248 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3602 2024-05-17T22:39:15.248 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3602_fsm 2024-05-17T22:39:15.248 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3602_vm 2024-05-17T22:39:15.248 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3603 2024-05-17T22:39:15.248 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3603_fsm 2024-05-17T22:39:15.248 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3603_vm 2024-05-17T22:39:15.248 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3604 2024-05-17T22:39:15.248 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3605 2024-05-17T22:39:15.248 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3606 2024-05-17T22:39:15.248 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3607 2024-05-17T22:39:15.249 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3608 2024-05-17T22:39:15.249 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 3609 2024-05-17T22:39:15.249 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3712 2024-05-17T22:39:15.249 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3764 2024-05-17T22:39:15.249 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3764_fsm 2024-05-17T22:39:15.249 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3764_vm 2024-05-17T22:39:15.249 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3766 2024-05-17T22:39:15.249 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3767 2024-05-17T22:39:15.249 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3997 2024-05-17T22:39:15.249 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4143 2024-05-17T22:39:15.249 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4144 2024-05-17T22:39:15.249 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4145 2024-05-17T22:39:15.249 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4146 2024-05-17T22:39:15.250 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4147 2024-05-17T22:39:15.250 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4148 2024-05-17T22:39:15.250 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4149 2024-05-17T22:39:15.250 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4150 2024-05-17T22:39:15.250 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4151 2024-05-17T22:39:15.250 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4152 2024-05-17T22:39:15.250 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4153 2024-05-17T22:39:15.250 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4154 2024-05-17T22:39:15.250 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4155 2024-05-17T22:39:15.250 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4156 2024-05-17T22:39:15.250 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4157 2024-05-17T22:39:15.250 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4158 2024-05-17T22:39:15.250 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4159 2024-05-17T22:39:15.251 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4160 2024-05-17T22:39:15.251 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4161 2024-05-17T22:39:15.251 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4162 2024-05-17T22:39:15.251 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4163 2024-05-17T22:39:15.251 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4164 2024-05-17T22:39:15.251 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4165 2024-05-17T22:39:15.251 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4166 2024-05-17T22:39:15.251 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4167 2024-05-17T22:39:15.251 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4168 2024-05-17T22:39:15.251 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4169 2024-05-17T22:39:15.251 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4170 2024-05-17T22:39:15.251 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4171 2024-05-17T22:39:15.251 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4172 2024-05-17T22:39:15.251 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4173 2024-05-17T22:39:15.252 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4174 2024-05-17T22:39:15.252 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 5002 2024-05-17T22:39:15.252 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 548 2024-05-17T22:39:15.252 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 549 2024-05-17T22:39:15.252 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 6102 2024-05-17T22:39:15.252 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 6104 2024-05-17T22:39:15.252 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 6106 2024-05-17T22:39:15.252 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 6110 2024-05-17T22:39:15.252 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 6111 2024-05-17T22:39:15.252 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 6112 2024-05-17T22:39:15.252 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 6113 2024-05-17T22:39:15.252 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 6117 2024-05-17T22:39:15.252 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 826 2024-05-17T22:39:15.252 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 827 2024-05-17T22:39:15.252 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 828 2024-05-17T22:39:15.253 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 17 22:39 pg_filenode.map 2024-05-17T22:39:15.253 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 17 22:39 PG_VERSION 2024-05-17T22:39:15.253 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:39:15.253 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/global: 2024-05-17T22:39:15.253 INFO:teuthology.orchestra.run.smithi012.stdout:total 537 2024-05-17T22:39:15.253 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 May 17 22:39 . 2024-05-17T22:39:15.253 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 17 22:39 .. 2024-05-17T22:39:15.253 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1213 2024-05-17T22:39:15.253 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 1213_fsm 2024-05-17T22:39:15.253 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1213_vm 2024-05-17T22:39:15.253 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1214 2024-05-17T22:39:15.253 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 1214_fsm 2024-05-17T22:39:15.253 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1214_vm 2024-05-17T22:39:15.253 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 1232 2024-05-17T22:39:15.254 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 1233 2024-05-17T22:39:15.254 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1260 2024-05-17T22:39:15.254 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 1260_fsm 2024-05-17T22:39:15.254 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1260_vm 2024-05-17T22:39:15.254 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1261 2024-05-17T22:39:15.254 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 1261_fsm 2024-05-17T22:39:15.254 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1261_vm 2024-05-17T22:39:15.254 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1262 2024-05-17T22:39:15.254 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 1262_fsm 2024-05-17T22:39:15.254 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1262_vm 2024-05-17T22:39:15.254 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2396 2024-05-17T22:39:15.254 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2396_fsm 2024-05-17T22:39:15.254 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2396_vm 2024-05-17T22:39:15.254 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2397 2024-05-17T22:39:15.255 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2671 2024-05-17T22:39:15.255 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2672 2024-05-17T22:39:15.255 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2676 2024-05-17T22:39:15.255 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2677 2024-05-17T22:39:15.255 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2694 2024-05-17T22:39:15.255 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2695 2024-05-17T22:39:15.255 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2697 2024-05-17T22:39:15.255 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2698 2024-05-17T22:39:15.255 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2846 2024-05-17T22:39:15.255 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2847 2024-05-17T22:39:15.255 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2964 2024-05-17T22:39:15.255 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2965 2024-05-17T22:39:15.255 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2966 2024-05-17T22:39:15.255 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2967 2024-05-17T22:39:15.256 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3592 2024-05-17T22:39:15.256 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3593 2024-05-17T22:39:15.256 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4060 2024-05-17T22:39:15.256 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4061 2024-05-17T22:39:15.256 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4175 2024-05-17T22:39:15.256 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4176 2024-05-17T22:39:15.256 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4177 2024-05-17T22:39:15.256 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4178 2024-05-17T22:39:15.256 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4181 2024-05-17T22:39:15.256 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4182 2024-05-17T22:39:15.256 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4183 2024-05-17T22:39:15.256 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4184 2024-05-17T22:39:15.256 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4185 2024-05-17T22:39:15.256 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4186 2024-05-17T22:39:15.257 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 6000 2024-05-17T22:39:15.257 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 6001 2024-05-17T22:39:15.257 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 6002 2024-05-17T22:39:15.257 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 6100 2024-05-17T22:39:15.257 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 6114 2024-05-17T22:39:15.257 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 6115 2024-05-17T22:39:15.257 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 pg_control 2024-05-17T22:39:15.257 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 17 22:39 pg_filenode.map 2024-05-17T22:39:15.257 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:39:15.257 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/log: 2024-05-17T22:39:15.257 INFO:teuthology.orchestra.run.smithi012.stdout:total 0 2024-05-17T22:39:15.257 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 . 2024-05-17T22:39:15.257 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 17 22:39 .. 2024-05-17T22:39:15.257 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:39:15.258 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-05-17T22:39:15.258 INFO:teuthology.orchestra.run.smithi012.stdout:total 0 2024-05-17T22:39:15.258 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 . 2024-05-17T22:39:15.258 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 17 22:39 .. 2024-05-17T22:39:15.258 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:39:15.258 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-05-17T22:39:15.258 INFO:teuthology.orchestra.run.smithi012.stdout:total 0 2024-05-17T22:39:15.258 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 . 2024-05-17T22:39:15.258 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 17 22:39 .. 2024-05-17T22:39:15.258 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:39:15.258 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-05-17T22:39:15.258 INFO:teuthology.orchestra.run.smithi012.stdout:total 1 2024-05-17T22:39:15.258 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 17 22:39 . 2024-05-17T22:39:15.259 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 17 22:39 .. 2024-05-17T22:39:15.259 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 mappings 2024-05-17T22:39:15.259 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 17 22:39 replorigin_checkpoint 2024-05-17T22:39:15.259 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 snapshots 2024-05-17T22:39:15.259 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:39:15.259 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-05-17T22:39:15.259 INFO:teuthology.orchestra.run.smithi012.stdout:total 0 2024-05-17T22:39:15.259 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 . 2024-05-17T22:39:15.259 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 17 22:39 .. 2024-05-17T22:39:15.259 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:39:15.259 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-05-17T22:39:15.259 INFO:teuthology.orchestra.run.smithi012.stdout:total 0 2024-05-17T22:39:15.259 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 . 2024-05-17T22:39:15.259 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 17 22:39 .. 2024-05-17T22:39:15.260 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:39:15.260 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-05-17T22:39:15.260 INFO:teuthology.orchestra.run.smithi012.stdout:total 0 2024-05-17T22:39:15.260 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 17 22:39 . 2024-05-17T22:39:15.260 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 17 22:39 .. 2024-05-17T22:39:15.260 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 17 22:39 members 2024-05-17T22:39:15.260 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 17 22:39 offsets 2024-05-17T22:39:15.260 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:39:15.260 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-05-17T22:39:15.260 INFO:teuthology.orchestra.run.smithi012.stdout:total 8 2024-05-17T22:39:15.260 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 17 22:39 . 2024-05-17T22:39:15.260 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 17 22:39 .. 2024-05-17T22:39:15.260 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 0000 2024-05-17T22:39:15.260 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:39:15.261 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-05-17T22:39:15.261 INFO:teuthology.orchestra.run.smithi012.stdout:total 8 2024-05-17T22:39:15.261 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 17 22:39 . 2024-05-17T22:39:15.261 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 17 22:39 .. 2024-05-17T22:39:15.261 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 0000 2024-05-17T22:39:15.261 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:39:15.261 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-05-17T22:39:15.261 INFO:teuthology.orchestra.run.smithi012.stdout:total 0 2024-05-17T22:39:15.261 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 . 2024-05-17T22:39:15.261 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 17 22:39 .. 2024-05-17T22:39:15.261 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:39:15.261 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-05-17T22:39:15.261 INFO:teuthology.orchestra.run.smithi012.stdout:total 0 2024-05-17T22:39:15.261 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 . 2024-05-17T22:39:15.262 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 17 22:39 .. 2024-05-17T22:39:15.262 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:39:15.262 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-05-17T22:39:15.262 INFO:teuthology.orchestra.run.smithi012.stdout:total 0 2024-05-17T22:39:15.262 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 . 2024-05-17T22:39:15.262 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 17 22:39 .. 2024-05-17T22:39:15.262 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:39:15.262 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-05-17T22:39:15.262 INFO:teuthology.orchestra.run.smithi012.stdout:total 0 2024-05-17T22:39:15.262 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 . 2024-05-17T22:39:15.262 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 17 22:39 .. 2024-05-17T22:39:15.262 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:39:15.262 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-05-17T22:39:15.262 INFO:teuthology.orchestra.run.smithi012.stdout:total 0 2024-05-17T22:39:15.263 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 . 2024-05-17T22:39:15.263 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 17 22:39 .. 2024-05-17T22:39:15.263 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:39:15.263 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-05-17T22:39:15.263 INFO:teuthology.orchestra.run.smithi012.stdout:total 0 2024-05-17T22:39:15.263 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 . 2024-05-17T22:39:15.263 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 17 22:39 .. 2024-05-17T22:39:15.263 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:39:15.263 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-05-17T22:39:15.263 INFO:teuthology.orchestra.run.smithi012.stdout:total 8 2024-05-17T22:39:15.263 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 17 22:39 . 2024-05-17T22:39:15.263 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 17 22:39 .. 2024-05-17T22:39:15.263 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 0000 2024-05-17T22:39:15.263 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:39:15.263 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-05-17T22:39:15.264 INFO:teuthology.orchestra.run.smithi012.stdout:total 0 2024-05-17T22:39:15.264 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 . 2024-05-17T22:39:15.264 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 17 22:39 .. 2024-05-17T22:39:15.264 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:39:15.264 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-05-17T22:39:15.264 INFO:teuthology.orchestra.run.smithi012.stdout:total 0 2024-05-17T22:39:15.264 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 . 2024-05-17T22:39:15.264 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 17 22:39 .. 2024-05-17T22:39:15.264 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:39:15.264 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-05-17T22:39:15.264 INFO:teuthology.orchestra.run.smithi012.stdout:total 16384 2024-05-17T22:39:15.264 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 17 22:39 . 2024-05-17T22:39:15.264 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 17 22:39 .. 2024-05-17T22:39:15.264 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:39 000000010000000000000001 2024-05-17T22:39:15.265 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 archive_status 2024-05-17T22:39:15.265 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:39:15.265 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-05-17T22:39:15.265 INFO:teuthology.orchestra.run.smithi012.stdout:total 0 2024-05-17T22:39:15.265 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 . 2024-05-17T22:39:15.265 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 17 22:39 .. 2024-05-17T22:39:15.265 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:39:15.265 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-05-17T22:39:15.265 INFO:teuthology.orchestra.run.smithi012.stdout:total 8 2024-05-17T22:39:15.265 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 17 22:39 . 2024-05-17T22:39:15.265 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 17 22:39 .. 2024-05-17T22:39:15.265 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 0000 2024-05-17T22:39:15.266 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2024-05-17T22:39:15.418 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2024-05-17T22:39:15.467 INFO:teuthology.orchestra.run.smithi012.stderr:dropping old tables... 2024-05-17T22:39:15.467 INFO:teuthology.orchestra.run.smithi012.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2024-05-17T22:39:15.467 INFO:teuthology.orchestra.run.smithi012.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2024-05-17T22:39:15.467 INFO:teuthology.orchestra.run.smithi012.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2024-05-17T22:39:15.467 INFO:teuthology.orchestra.run.smithi012.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2024-05-17T22:39:15.468 INFO:teuthology.orchestra.run.smithi012.stderr:creating tables... 2024-05-17T22:39:15.481 INFO:teuthology.orchestra.run.smithi012.stderr:generating data (client-side)... 2024-05-17T22:39:15.558 INFO:teuthology.orchestra.run.smithi012.stderr:100000 of 3200000 tuples (3%) done (elapsed 0.06 s, remaining 1.86 s) 2024-05-17T22:39:15.707 INFO:teuthology.orchestra.run.smithi012.stderr:200000 of 3200000 tuples (6%) done (elapsed 0.21 s, remaining 3.15 s) 2024-05-17T22:39:15.727 INFO:tasks.cephfs.filesystem:scrub status for tag:65198cc9-f62d-4163-858c-7f47eaf98eef - {'path': '/', 'tag': '65198cc9-f62d-4163-858c-7f47eaf98eef', 'options': 'recursive,force'} 2024-05-17T22:39:15.805 INFO:teuthology.orchestra.run.smithi012.stderr:300000 of 3200000 tuples (9%) done (elapsed 0.31 s, remaining 2.97 s) 2024-05-17T22:39:15.945 INFO:teuthology.orchestra.run.smithi012.stderr:400000 of 3200000 tuples (12%) done (elapsed 0.45 s, remaining 3.14 s) 2024-05-17T22:39:15.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:15 smithi151 ceph-mon[36101]: pgmap v439: 129 pgs: 129 active+clean; 655 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 16 KiB/s wr, 8 op/s 2024-05-17T22:39:16.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:15 smithi096 ceph-mon[36065]: pgmap v439: 129 pgs: 129 active+clean; 655 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 16 KiB/s wr, 8 op/s 2024-05-17T22:39:16.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:15 smithi012 ceph-mon[31632]: pgmap v439: 129 pgs: 129 active+clean; 655 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 16 KiB/s wr, 8 op/s 2024-05-17T22:39:16.099 INFO:teuthology.orchestra.run.smithi012.stderr:500000 of 3200000 tuples (15%) done (elapsed 0.60 s, remaining 3.25 s) 2024-05-17T22:39:16.243 INFO:teuthology.orchestra.run.smithi012.stderr:600000 of 3200000 tuples (18%) done (elapsed 0.75 s, remaining 3.23 s) 2024-05-17T22:39:16.321 INFO:teuthology.orchestra.run.smithi012.stderr:700000 of 3200000 tuples (21%) done (elapsed 0.82 s, remaining 2.94 s) 2024-05-17T22:39:16.458 INFO:teuthology.orchestra.run.smithi012.stderr:800000 of 3200000 tuples (25%) done (elapsed 0.96 s, remaining 2.88 s) 2024-05-17T22:39:16.546 INFO:teuthology.orchestra.run.smithi012.stderr:900000 of 3200000 tuples (28%) done (elapsed 1.05 s, remaining 2.68 s) 2024-05-17T22:39:16.679 INFO:teuthology.orchestra.run.smithi012.stderr:1000000 of 3200000 tuples (31%) done (elapsed 1.18 s, remaining 2.60 s) 2024-05-17T22:39:16.846 INFO:teuthology.orchestra.run.smithi012.stderr:1100000 of 3200000 tuples (34%) done (elapsed 1.35 s, remaining 2.57 s) 2024-05-17T22:39:16.935 INFO:teuthology.orchestra.run.smithi012.stderr:1200000 of 3200000 tuples (37%) done (elapsed 1.44 s, remaining 2.40 s) 2024-05-17T22:39:17.064 INFO:teuthology.orchestra.run.smithi012.stderr:1300000 of 3200000 tuples (40%) done (elapsed 1.57 s, remaining 2.29 s) 2024-05-17T22:39:17.158 INFO:teuthology.orchestra.run.smithi012.stderr:1400000 of 3200000 tuples (43%) done (elapsed 1.66 s, remaining 2.14 s) 2024-05-17T22:39:17.320 INFO:teuthology.orchestra.run.smithi012.stderr:1500000 of 3200000 tuples (46%) done (elapsed 1.82 s, remaining 2.07 s) 2024-05-17T22:39:17.448 INFO:teuthology.orchestra.run.smithi012.stderr:1600000 of 3200000 tuples (50%) done (elapsed 1.95 s, remaining 1.95 s) 2024-05-17T22:39:17.548 INFO:teuthology.orchestra.run.smithi012.stderr:1700000 of 3200000 tuples (53%) done (elapsed 2.05 s, remaining 1.81 s) 2024-05-17T22:39:17.712 INFO:teuthology.orchestra.run.smithi012.stderr:1800000 of 3200000 tuples (56%) done (elapsed 2.21 s, remaining 1.72 s) 2024-05-17T22:39:17.855 INFO:teuthology.orchestra.run.smithi012.stderr:1900000 of 3200000 tuples (59%) done (elapsed 2.36 s, remaining 1.61 s) 2024-05-17T22:39:17.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:17 smithi151 ceph-mon[36101]: pgmap v440: 129 pgs: 129 active+clean; 4.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 311 KiB/s wr, 15 op/s 2024-05-17T22:39:18.005 INFO:teuthology.orchestra.run.smithi012.stderr:2000000 of 3200000 tuples (62%) done (elapsed 2.51 s, remaining 1.50 s) 2024-05-17T22:39:18.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:17 smithi096 ceph-mon[36065]: pgmap v440: 129 pgs: 129 active+clean; 4.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 311 KiB/s wr, 15 op/s 2024-05-17T22:39:18.074 INFO:teuthology.orchestra.run.smithi012.stderr:2100000 of 3200000 tuples (65%) done (elapsed 2.58 s, remaining 1.35 s) 2024-05-17T22:39:18.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:17 smithi012 ceph-mon[31632]: pgmap v440: 129 pgs: 129 active+clean; 4.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 311 KiB/s wr, 15 op/s 2024-05-17T22:39:18.225 INFO:teuthology.orchestra.run.smithi012.stderr:2200000 of 3200000 tuples (68%) done (elapsed 2.73 s, remaining 1.24 s) 2024-05-17T22:39:18.308 INFO:teuthology.orchestra.run.smithi012.stderr:2300000 of 3200000 tuples (71%) done (elapsed 2.81 s, remaining 1.10 s) 2024-05-17T22:39:18.453 INFO:teuthology.orchestra.run.smithi012.stderr:2400000 of 3200000 tuples (75%) done (elapsed 2.96 s, remaining 0.99 s) 2024-05-17T22:39:18.624 INFO:teuthology.orchestra.run.smithi012.stderr:2500000 of 3200000 tuples (78%) done (elapsed 3.13 s, remaining 0.88 s) 2024-05-17T22:39:18.708 INFO:teuthology.orchestra.run.smithi012.stderr:2600000 of 3200000 tuples (81%) done (elapsed 3.21 s, remaining 0.74 s) 2024-05-17T22:39:18.842 INFO:teuthology.orchestra.run.smithi012.stderr:2700000 of 3200000 tuples (84%) done (elapsed 3.34 s, remaining 0.62 s) 2024-05-17T22:39:18.937 INFO:teuthology.orchestra.run.smithi012.stderr:2800000 of 3200000 tuples (87%) done (elapsed 3.44 s, remaining 0.49 s) 2024-05-17T22:39:19.094 INFO:teuthology.orchestra.run.smithi012.stderr:2900000 of 3200000 tuples (90%) done (elapsed 3.60 s, remaining 0.37 s) 2024-05-17T22:39:19.222 INFO:teuthology.orchestra.run.smithi012.stderr:3000000 of 3200000 tuples (93%) done (elapsed 3.72 s, remaining 0.25 s) 2024-05-17T22:39:19.306 INFO:teuthology.orchestra.run.smithi012.stderr:3100000 of 3200000 tuples (96%) done (elapsed 3.81 s, remaining 0.12 s) 2024-05-17T22:39:19.473 INFO:teuthology.orchestra.run.smithi012.stderr:3200000 of 3200000 tuples (100%) done (elapsed 3.98 s, remaining 0.00 s) 2024-05-17T22:39:19.509 INFO:teuthology.orchestra.run.smithi012.stderr:vacuuming... 2024-05-17T22:39:19.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:19 smithi151 ceph-mon[36101]: pgmap v441: 129 pgs: 129 active+clean; 133 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 20 MiB/s wr, 74 op/s 2024-05-17T22:39:20.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:19 smithi096 ceph-mon[36065]: pgmap v441: 129 pgs: 129 active+clean; 133 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 20 MiB/s wr, 74 op/s 2024-05-17T22:39:20.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:19 smithi012 ceph-mon[31632]: pgmap v441: 129 pgs: 129 active+clean; 133 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 20 MiB/s wr, 74 op/s 2024-05-17T22:39:20.130 INFO:teuthology.orchestra.run.smithi012.stderr:creating primary keys... 2024-05-17T22:39:21.297 INFO:teuthology.orchestra.run.smithi012.stderr:done in 5.83 s (drop tables 0.00 s, create tables 0.01 s, client-side generate 4.03 s, vacuum 0.62 s, primary keys 1.17 s). 2024-05-17T22:39:21.300 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2024-05-17T22:39:21.369 INFO:teuthology.orchestra.run.smithi012.stderr:starting vacuum...end. 2024-05-17T22:39:21.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:21 smithi151 ceph-mon[36101]: pgmap v442: 129 pgs: 129 active+clean; 216 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 33 MiB/s wr, 109 op/s 2024-05-17T22:39:22.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:21 smithi096 ceph-mon[36065]: pgmap v442: 129 pgs: 129 active+clean; 216 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 33 MiB/s wr, 109 op/s 2024-05-17T22:39:22.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:21 smithi012 ceph-mon[31632]: pgmap v442: 129 pgs: 129 active+clean; 216 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 33 MiB/s wr, 109 op/s 2024-05-17T22:39:23.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:23 smithi151 ceph-mon[36101]: pgmap v443: 129 pgs: 129 active+clean; 357 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 54 MiB/s wr, 128 op/s 2024-05-17T22:39:23.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:23 smithi151 ceph-mon[36101]: scrub summary: idle 2024-05-17T22:39:24.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:23 smithi096 ceph-mon[36065]: pgmap v443: 129 pgs: 129 active+clean; 357 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 54 MiB/s wr, 128 op/s 2024-05-17T22:39:24.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:23 smithi096 ceph-mon[36065]: scrub summary: idle 2024-05-17T22:39:24.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:23 smithi012 ceph-mon[31632]: pgmap v443: 129 pgs: 129 active+clean; 357 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 54 MiB/s wr, 128 op/s 2024-05-17T22:39:24.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:23 smithi012 ceph-mon[31632]: scrub summary: idle 2024-05-17T22:39:25.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:25 smithi151 ceph-mon[36101]: pgmap v444: 129 pgs: 129 active+clean; 528 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 76 MiB/s wr, 176 op/s 2024-05-17T22:39:26.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:25 smithi096 ceph-mon[36065]: pgmap v444: 129 pgs: 129 active+clean; 528 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 76 MiB/s wr, 176 op/s 2024-05-17T22:39:26.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:25 smithi012 ceph-mon[31632]: pgmap v444: 129 pgs: 129 active+clean; 528 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 76 MiB/s wr, 176 op/s 2024-05-17T22:39:26.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 5.0 s, 3908.0 tps, lat 25.191 ms stddev 28.138 2024-05-17T22:39:27.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:27 smithi151 ceph-mon[36101]: pgmap v445: 129 pgs: 129 active+clean; 553 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 76 MiB/s wr, 180 op/s 2024-05-17T22:39:28.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:27 smithi096 ceph-mon[36065]: pgmap v445: 129 pgs: 129 active+clean; 553 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 76 MiB/s wr, 180 op/s 2024-05-17T22:39:28.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:27 smithi012 ceph-mon[31632]: pgmap v445: 129 pgs: 129 active+clean; 553 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 76 MiB/s wr, 180 op/s 2024-05-17T22:39:29.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:29 smithi151 ceph-mon[36101]: pgmap v446: 129 pgs: 129 active+clean; 833 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 78 MiB/s wr, 395 op/s 2024-05-17T22:39:30.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:29 smithi096 ceph-mon[36065]: pgmap v446: 129 pgs: 129 active+clean; 833 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 78 MiB/s wr, 395 op/s 2024-05-17T22:39:30.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:29 smithi012 ceph-mon[31632]: pgmap v446: 129 pgs: 129 active+clean; 833 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 78 MiB/s wr, 395 op/s 2024-05-17T22:39:31.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 10.0 s, 3326.6 tps, lat 29.795 ms stddev 31.194 2024-05-17T22:39:32.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:31 smithi096 ceph-mon[36065]: pgmap v447: 129 pgs: 129 active+clean; 891 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 59 MiB/s wr, 413 op/s 2024-05-17T22:39:32.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:31 smithi012 ceph-mon[31632]: pgmap v447: 129 pgs: 129 active+clean; 891 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 59 MiB/s wr, 413 op/s 2024-05-17T22:39:32.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:31 smithi151 ceph-mon[36101]: pgmap v447: 129 pgs: 129 active+clean; 891 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 59 MiB/s wr, 413 op/s 2024-05-17T22:39:34.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:33 smithi096 ceph-mon[36065]: pgmap v448: 129 pgs: 129 active+clean; 909 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 48 MiB/s wr, 458 op/s 2024-05-17T22:39:34.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:33 smithi012 ceph-mon[31632]: pgmap v448: 129 pgs: 129 active+clean; 909 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 48 MiB/s wr, 458 op/s 2024-05-17T22:39:34.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:33 smithi151 ceph-mon[36101]: pgmap v448: 129 pgs: 129 active+clean; 909 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 48 MiB/s wr, 458 op/s 2024-05-17T22:39:36.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:35 smithi096 ceph-mon[36065]: pgmap v449: 129 pgs: 129 active+clean; 930 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 29 MiB/s wr, 629 op/s 2024-05-17T22:39:36.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:35 smithi012 ceph-mon[31632]: pgmap v449: 129 pgs: 129 active+clean; 930 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 29 MiB/s wr, 629 op/s 2024-05-17T22:39:36.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:35 smithi151 ceph-mon[36101]: pgmap v449: 129 pgs: 129 active+clean; 930 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 29 MiB/s wr, 629 op/s 2024-05-17T22:39:36.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 15.0 s, 3409.6 tps, lat 29.549 ms stddev 32.269 2024-05-17T22:39:38.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:37 smithi096 ceph-mon[36065]: pgmap v450: 129 pgs: 129 active+clean; 930 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.2 MiB/s wr, 594 op/s 2024-05-17T22:39:38.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:37 smithi012 ceph-mon[31632]: pgmap v450: 129 pgs: 129 active+clean; 930 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.2 MiB/s wr, 594 op/s 2024-05-17T22:39:38.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:37 smithi151 ceph-mon[36101]: pgmap v450: 129 pgs: 129 active+clean; 930 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.2 MiB/s wr, 594 op/s 2024-05-17T22:39:40.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:39 smithi096 ceph-mon[36065]: pgmap v451: 129 pgs: 129 active+clean; 936 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 862 op/s 2024-05-17T22:39:40.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:39 smithi096 ceph-mon[36065]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:39:40.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:39 smithi012 ceph-mon[31632]: pgmap v451: 129 pgs: 129 active+clean; 936 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 862 op/s 2024-05-17T22:39:40.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:39 smithi012 ceph-mon[31632]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:39:40.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:39 smithi151 ceph-mon[36101]: pgmap v451: 129 pgs: 129 active+clean; 936 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 862 op/s 2024-05-17T22:39:40.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:39 smithi151 ceph-mon[36101]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:39:41.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 20.0 s, 3056.2 tps, lat 32.694 ms stddev 37.400 2024-05-17T22:39:42.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:41 smithi096 ceph-mon[36065]: pgmap v452: 129 pgs: 129 active+clean; 936 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.1 MiB/s wr, 675 op/s 2024-05-17T22:39:42.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:41 smithi012 ceph-mon[31632]: pgmap v452: 129 pgs: 129 active+clean; 936 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.1 MiB/s wr, 675 op/s 2024-05-17T22:39:42.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:41 smithi151 ceph-mon[36101]: pgmap v452: 129 pgs: 129 active+clean; 936 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.1 MiB/s wr, 675 op/s 2024-05-17T22:39:44.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:43 smithi096 ceph-mon[36065]: pgmap v453: 129 pgs: 129 active+clean; 952 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.6 MiB/s wr, 699 op/s 2024-05-17T22:39:44.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:43 smithi012 ceph-mon[31632]: pgmap v453: 129 pgs: 129 active+clean; 952 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.6 MiB/s wr, 699 op/s 2024-05-17T22:39:44.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:43 smithi151 ceph-mon[36101]: pgmap v453: 129 pgs: 129 active+clean; 952 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.6 MiB/s wr, 699 op/s 2024-05-17T22:39:45.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:44 smithi096 ceph-mon[36065]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:39:45.093 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:44 smithi012 ceph-mon[31632]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:39:45.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:44 smithi151 ceph-mon[36101]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:39:45.730 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-17T22:39:46.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:45 smithi096 ceph-mon[36065]: pgmap v454: 129 pgs: 129 active+clean; 975 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 14 MiB/s wr, 840 op/s 2024-05-17T22:39:46.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:45 smithi012 ceph-mon[31632]: pgmap v454: 129 pgs: 129 active+clean; 975 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 14 MiB/s wr, 840 op/s 2024-05-17T22:39:46.161 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:39:46.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:45 smithi151 ceph-mon[36101]: pgmap v454: 129 pgs: 129 active+clean; 975 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 14 MiB/s wr, 840 op/s 2024-05-17T22:39:46.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 25.0 s, 2959.2 tps, lat 33.509 ms stddev 53.399 2024-05-17T22:39:47.935 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:47 smithi012 ceph-mon[31632]: pgmap v455: 129 pgs: 129 active+clean; 979 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 11 MiB/s wr, 668 op/s 2024-05-17T22:39:48.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:47 smithi096 ceph-mon[36065]: pgmap v455: 129 pgs: 129 active+clean; 979 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 11 MiB/s wr, 668 op/s 2024-05-17T22:39:48.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:47 smithi151 ceph-mon[36101]: pgmap v455: 129 pgs: 129 active+clean; 979 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 11 MiB/s wr, 668 op/s 2024-05-17T22:39:48.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.265+0000 7f17da75c700 1 -- 172.21.15.12:0/954682888 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17d4107730 msgr2=0x7f17d4107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:48.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.265+0000 7f17da75c700 1 --2- 172.21.15.12:0/954682888 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17d4107730 0x7f17d4107b40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f17c80098f0 tx=0x7f17c8009c00 comp rx=0 tx=0).stop 2024-05-17T22:39:48.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.266+0000 7f17da75c700 1 -- 172.21.15.12:0/954682888 shutdown_connections 2024-05-17T22:39:48.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.266+0000 7f17da75c700 1 --2- 172.21.15.12:0/954682888 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17d4108080 0x7f17d410f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:48.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.266+0000 7f17da75c700 1 --2- 172.21.15.12:0/954682888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17d4102e00 0x7f17d4103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:48.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.266+0000 7f17da75c700 1 --2- 172.21.15.12:0/954682888 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17d4107730 0x7f17d4107b40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:48.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.266+0000 7f17da75c700 1 -- 172.21.15.12:0/954682888 >> 172.21.15.12:0/954682888 conn(0x7f17d4079bd0 msgr2=0x7f17d4079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:39:48.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.267+0000 7f17da75c700 1 -- 172.21.15.12:0/954682888 shutdown_connections 2024-05-17T22:39:48.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.267+0000 7f17da75c700 1 -- 172.21.15.12:0/954682888 wait complete. 2024-05-17T22:39:48.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.268+0000 7f17da75c700 1 Processor -- start 2024-05-17T22:39:48.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.268+0000 7f17da75c700 1 -- start start 2024-05-17T22:39:48.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.269+0000 7f17da75c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17d4102e00 0x7f17d41a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:48.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.269+0000 7f17da75c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17d4107730 0x7f17d41a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:48.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.269+0000 7f17da75c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17d4108080 0x7f17d41a5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:48.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.269+0000 7f17da75c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f17d4112c90 con 0x7f17d4107730 2024-05-17T22:39:48.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.269+0000 7f17da75c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f17d4112b10 con 0x7f17d4102e00 2024-05-17T22:39:48.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.269+0000 7f17da75c700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f17d4112e10 con 0x7f17d4108080 2024-05-17T22:39:48.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.269+0000 7f17d37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17d4107730 0x7f17d41a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:48.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.269+0000 7f17d37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17d4107730 0x7f17d41a13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36628/0 (socket says 172.21.15.12:36628) 2024-05-17T22:39:48.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.269+0000 7f17d37fe700 1 -- 172.21.15.12:0/2975746703 learned_addr learned my addr 172.21.15.12:0/2975746703 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:39:48.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.269+0000 7f17d3fff700 1 --2- 172.21.15.12:0/2975746703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17d4102e00 0x7f17d41a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:48.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.269+0000 7f17d8cf9700 1 --2- 172.21.15.12:0/2975746703 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17d4108080 0x7f17d41a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:48.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.270+0000 7f17d37fe700 1 -- 172.21.15.12:0/2975746703 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17d4108080 msgr2=0x7f17d41a5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:48.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.270+0000 7f17d37fe700 1 --2- 172.21.15.12:0/2975746703 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17d4108080 0x7f17d41a5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:48.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.270+0000 7f17d37fe700 1 -- 172.21.15.12:0/2975746703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17d4102e00 msgr2=0x7f17d41a0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:48.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.270+0000 7f17d37fe700 1 --2- 172.21.15.12:0/2975746703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17d4102e00 0x7f17d41a0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:48.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.270+0000 7f17d37fe700 1 -- 172.21.15.12:0/2975746703 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d41a5cf0 con 0x7f17d4107730 2024-05-17T22:39:48.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.270+0000 7f17d8cf9700 1 --2- 172.21.15.12:0/2975746703 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17d4108080 0x7f17d41a5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:39:48.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.270+0000 7f17d3fff700 1 --2- 172.21.15.12:0/2975746703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17d4102e00 0x7f17d41a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:39:48.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.270+0000 7f17d37fe700 1 --2- 172.21.15.12:0/2975746703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17d4107730 0x7f17d41a13c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f17c8009230 tx=0x7f17c800fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:39:48.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.271+0000 7f17d17fa700 1 -- 172.21.15.12:0/2975746703 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c80051c0 con 0x7f17d4107730 2024-05-17T22:39:48.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.271+0000 7f17da75c700 1 -- 172.21.15.12:0/2975746703 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17d41a5f80 con 0x7f17d4107730 2024-05-17T22:39:48.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.271+0000 7f17da75c700 1 -- 172.21.15.12:0/2975746703 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17d41ad900 con 0x7f17d4107730 2024-05-17T22:39:48.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.271+0000 7f17d17fa700 1 -- 172.21.15.12:0/2975746703 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17c800b560 con 0x7f17d4107730 2024-05-17T22:39:48.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.271+0000 7f17d17fa700 1 -- 172.21.15.12:0/2975746703 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c8003ae0 con 0x7f17d4107730 2024-05-17T22:39:48.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.275+0000 7f17e9137b80 1 Processor -- start 2024-05-17T22:39:48.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.275+0000 7f17d17fa700 1 -- 172.21.15.12:0/2975746703 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f17c8003cc0 con 0x7f17d4107730 2024-05-17T22:39:48.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.276+0000 7f17e9137b80 1 -- start start 2024-05-17T22:39:48.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.276+0000 7f17e9137b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e0f3049f0 0x555e0f487ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:48.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.277+0000 7f17e9137b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e0f488400 0x555e0f4825c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:48.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.277+0000 7f17d8cf9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e0f3049f0 0x555e0f487ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:48.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.277+0000 7f17d8cf9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e0f3049f0 0x555e0f487ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36634/0 (socket says 172.21.15.12:36634) 2024-05-17T22:39:48.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.277+0000 7f17d8cf9700 1 -- 172.21.15.12:0/66780403 learned_addr learned my addr 172.21.15.12:0/66780403 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:39:48.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.277+0000 7f17e9137b80 1 --2- 172.21.15.12:0/66780403 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e0f482b00 0x555e0f482f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:48.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.277+0000 7f17e9137b80 1 -- 172.21.15.12:0/66780403 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x555e0f3b10d0 con 0x555e0f3049f0 2024-05-17T22:39:48.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.277+0000 7f17e9137b80 1 -- 172.21.15.12:0/66780403 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555e0f3a0c70 con 0x555e0f488400 2024-05-17T22:39:48.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.277+0000 7f17e9137b80 1 -- 172.21.15.12:0/66780403 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555e0f3a4cb0 con 0x555e0f482b00 2024-05-17T22:39:48.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.278+0000 7f17d3fff700 1 --2- 172.21.15.12:0/66780403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e0f488400 0x555e0f4825c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:48.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.278+0000 7f17d17fa700 1 --2- 172.21.15.12:0/2975746703 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f17bc0b2880 0x7f17bc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:48.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.278+0000 7f17d37fe700 1 --2- 172.21.15.12:0/66780403 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e0f482b00 0x555e0f482f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:48.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.278+0000 7f17d8cf9700 1 -- 172.21.15.12:0/66780403 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e0f482b00 msgr2=0x555e0f482f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:39:48.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.278+0000 7f17d8cf9700 1 --2- 172.21.15.12:0/66780403 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e0f482b00 0x555e0f482f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:48.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.278+0000 7f17d8cf9700 1 -- 172.21.15.12:0/66780403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e0f488400 msgr2=0x555e0f4825c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:39:48.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.278+0000 7f17d8cf9700 1 --2- 172.21.15.12:0/66780403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e0f488400 0x555e0f4825c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:48.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.278+0000 7f17d8cf9700 1 -- 172.21.15.12:0/66780403 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e0f33a5c0 con 0x555e0f3049f0 2024-05-17T22:39:48.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.278+0000 7f17d8cf9700 1 --2- 172.21.15.12:0/2975746703 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f17bc0b2880 0x7f17bc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:48.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.278+0000 7f17d17fa700 1 -- 172.21.15.12:0/2975746703 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f17c80e3600 con 0x7f17d4107730 2024-05-17T22:39:48.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.279+0000 7f17d3fff700 1 --2- 172.21.15.12:0/66780403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e0f488400 0x555e0f4825c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:39:48.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.279+0000 7f17d8cf9700 1 --2- 172.21.15.12:0/66780403 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e0f3049f0 0x555e0f487ec0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f17cc009590 tx=0x7f17cc0078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef84e17ebbd0b1e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:39:48.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.281+0000 7f17ba7fc700 1 -- 172.21.15.12:0/66780403 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc0146a0 con 0x555e0f3049f0 2024-05-17T22:39:48.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.281+0000 7f17ba7fc700 1 -- 172.21.15.12:0/66780403 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17cc014cf0 con 0x555e0f3049f0 2024-05-17T22:39:48.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.281+0000 7f17ba7fc700 1 -- 172.21.15.12:0/66780403 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc023bb0 con 0x555e0f3049f0 2024-05-17T22:39:48.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.282+0000 7f17e9137b80 1 -- 172.21.15.12:0/66780403 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e0f3049f0 msgr2=0x555e0f487ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:48.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.282+0000 7f17d8cf9700 1 --2- 172.21.15.12:0/2975746703 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f17bc0b2880 0x7f17bc0b4d90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f17cc020000 tx=0x7f17cc021000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:39:48.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.282+0000 7f17e9137b80 1 --2- 172.21.15.12:0/66780403 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e0f3049f0 0x555e0f487ec0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f17cc009590 tx=0x7f17cc0078a0 comp rx=0 tx=0).stop 2024-05-17T22:39:48.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.282+0000 7f17e9137b80 1 -- 172.21.15.12:0/66780403 shutdown_connections 2024-05-17T22:39:48.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.282+0000 7f17e9137b80 1 --2- 172.21.15.12:0/66780403 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e0f482b00 0x555e0f482f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:48.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.282+0000 7f17e9137b80 1 --2- 172.21.15.12:0/66780403 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e0f3049f0 0x555e0f487ec0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:48.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.282+0000 7f17e9137b80 1 --2- 172.21.15.12:0/66780403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e0f488400 0x555e0f4825c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:48.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.282+0000 7f17e9137b80 1 -- 172.21.15.12:0/66780403 >> 172.21.15.12:0/66780403 conn(0x555e0f3647d0 msgr2=0x555e0f47d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:39:48.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.282+0000 7f17e9137b80 1 -- 172.21.15.12:0/66780403 shutdown_connections 2024-05-17T22:39:48.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.282+0000 7f17e9137b80 1 -- 172.21.15.12:0/66780403 wait complete. 2024-05-17T22:39:48.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.283+0000 7f17e9137b80 1 -- start start 2024-05-17T22:39:48.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.283+0000 7f17e9137b80 1 Processor -- start 2024-05-17T22:39:48.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.283+0000 7f17e9137b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e0f3049f0 0x555e0f48baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:48.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.283+0000 7f17e9137b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e0f482b00 0x555e0f48bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:48.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.283+0000 7f17e9137b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e0f488400 0x555e0f48c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:48.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.283+0000 7f17e9137b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x555e0f2a8150 con 0x555e0f482b00 2024-05-17T22:39:48.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.283+0000 7f17e9137b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555e0f342940 con 0x555e0f3049f0 2024-05-17T22:39:48.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.283+0000 7f17e9137b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555e0f48ee20 con 0x555e0f488400 2024-05-17T22:39:48.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.283+0000 7f17e9137b80 10 client.0 _ll_register_callbacks cb 0x555e0f337b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:39:48.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.284+0000 7f17d3fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e0f3049f0 0x555e0f48baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:48.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.284+0000 7f17d8cf9700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e0f488400 0x555e0f48c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:48.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.284+0000 7f17d3fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e0f3049f0 0x555e0f48baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:49382/0 (socket says 172.21.15.12:49382) 2024-05-17T22:39:48.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.284+0000 7f17d3fff700 1 -- 172.21.15.12:0/364049679 learned_addr learned my addr 172.21.15.12:0/364049679 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:39:48.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.285+0000 7f17d8cf9700 1 -- 172.21.15.12:0/364049679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e0f3049f0 msgr2=0x555e0f48baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:39:48.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.285+0000 7f17d8cf9700 1 --2- 172.21.15.12:0/364049679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e0f3049f0 0x555e0f48baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:48.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.285+0000 7f17d8cf9700 1 -- 172.21.15.12:0/364049679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e0f482b00 msgr2=0x555e0f48bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:39:48.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.285+0000 7f17d8cf9700 1 --2- 172.21.15.12:0/364049679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e0f482b00 0x555e0f48bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:48.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.285+0000 7f17d8cf9700 1 -- 172.21.15.12:0/364049679 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e0f2f1480 con 0x555e0f488400 2024-05-17T22:39:48.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.285+0000 7f17d8cf9700 1 --2- 172.21.15.12:0/364049679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e0f488400 0x555e0f48c520 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f17cc009560 tx=0x7f17cc023810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=477342e9a29023e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:39:48.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.285+0000 7f17b37fe700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:39:48.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.285+0000 7f17b37fe700 1 -- 172.21.15.12:0/364049679 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc033060 con 0x555e0f488400 2024-05-17T22:39:48.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.286+0000 7f17e9137b80 1 -- 172.21.15.12:0/364049679 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17cc014cf0 con 0x555e0f488400 2024-05-17T22:39:48.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.286+0000 7f17b37fe700 1 -- 172.21.15.12:0/364049679 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f17cc014700 con 0x555e0f488400 2024-05-17T22:39:48.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.286+0000 7f17b37fe700 1 -- 172.21.15.12:0/364049679 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f17cc023e10 con 0x555e0f488400 2024-05-17T22:39:48.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.286+0000 7f17e9137b80 10 client.25021 fetch_fsmap learned FSMap version 29 2024-05-17T22:39:48.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.286+0000 7f17e9137b80 1 -- 172.21.15.12:0/364049679 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555e0f4998f0 con 0x555e0f488400 2024-05-17T22:39:48.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.286+0000 7f17b37fe700 1 -- 172.21.15.12:0/364049679 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17cc014cf0 con 0x555e0f488400 2024-05-17T22:39:48.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.287+0000 7f17b37fe700 1 -- 172.21.15.12:0/364049679 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f17cc005500 con 0x555e0f488400 2024-05-17T22:39:48.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.287+0000 7f17e9137b80 10 client.25021 fetch_fsmap finished waiting for FSMap version 29 2024-05-17T22:39:48.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.287+0000 7f17e9137b80 10 client.25021 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:39:48.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.287+0000 7f17e9137b80 20 client.25021 populate_metadata read hostname 'smithi012' 2024-05-17T22:39:48.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.287+0000 7f17e9137b80 1 --2- 172.21.15.12:0/364049679 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555e0f49bdc0 0x555e0f49e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:48.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.287+0000 7f17e9137b80 4 client.25021 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:39:48.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.287+0000 7f17e9137b80 1 -- 172.21.15.12:0/364049679 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e0f3b3020 con 0x555e0f49bdc0 2024-05-17T22:39:48.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.287+0000 7f17d3fff700 1 --2- 172.21.15.12:0/364049679 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555e0f49bdc0 0x555e0f49e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:48.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.288+0000 7f17d3fff700 1 --2- 172.21.15.12:0/364049679 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555e0f49bdc0 0x555e0f49e1e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d49d55a9f865189 server_cookie=61944c2c17da350e in_seq=0 out_seq=0 2024-05-17T22:39:48.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.288+0000 7f17b37fe700 10 client.25021 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:39:48.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.289+0000 7f17b37fe700 1 -- 172.21.15.12:0/364049679 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555e0f3b3020 con 0x555e0f49bdc0 2024-05-17T22:39:48.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.289+0000 7f17b37fe700 10 client.25021 handle_command_reply: tid=0 2024-05-17T22:39:48.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.289+0000 7f17e9137b80 1 client.25021 shutdown 2024-05-17T22:39:48.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.289+0000 7f17e9137b80 1 -- 172.21.15.12:0/364049679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e0f488400 msgr2=0x555e0f48c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:48.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.289+0000 7f17e9137b80 1 --2- 172.21.15.12:0/364049679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e0f488400 0x555e0f48c520 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f17cc009560 tx=0x7f17cc023810 comp rx=0 tx=0).stop 2024-05-17T22:39:48.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.289+0000 7f17e9137b80 1 -- 172.21.15.12:0/364049679 shutdown_connections 2024-05-17T22:39:48.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.289+0000 7f17e9137b80 1 --2- 172.21.15.12:0/364049679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e0f488400 0x555e0f48c520 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:48.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.289+0000 7f17e9137b80 1 --2- 172.21.15.12:0/364049679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e0f3049f0 0x555e0f48baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:48.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.289+0000 7f17e9137b80 1 --2- 172.21.15.12:0/364049679 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555e0f49bdc0 0x555e0f49e1e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:48.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.289+0000 7f17e9137b80 1 --2- 172.21.15.12:0/364049679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e0f482b00 0x555e0f48bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:48.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.289+0000 7f17e9137b80 1 -- 172.21.15.12:0/364049679 >> 172.21.15.12:0/364049679 conn(0x555e0f3647d0 msgr2=0x555e0f47d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:39:48.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.289+0000 7f17b37fe700 0 client.25021 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:39:48.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.289+0000 7f17e9137b80 1 -- 172.21.15.12:0/364049679 shutdown_connections 2024-05-17T22:39:48.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.290+0000 7f17e9137b80 1 -- 172.21.15.12:0/364049679 wait complete. 2024-05-17T22:39:48.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.290+0000 7f17e9137b80 20 client.25021 trim_cache size 0 max 16384 2024-05-17T22:39:48.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.327+0000 7f17e9137b80 1 Processor -- start 2024-05-17T22:39:48.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.328+0000 7f17e9137b80 1 -- start start 2024-05-17T22:39:48.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.328+0000 7f17e9137b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e0f3049f0 0x555e0f497d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:48.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.328+0000 7f17e9137b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e0f482b00 0x555e0f487510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:48.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.328+0000 7f17e9137b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e0f488400 0x555e0f48bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:48.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.328+0000 7f17e9137b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f17d41b8b20 con 0x555e0f488400 2024-05-17T22:39:48.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.328+0000 7f17e9137b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f17d41b63b0 con 0x555e0f3049f0 2024-05-17T22:39:48.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.328+0000 7f17e9137b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f17d41b4200 con 0x555e0f482b00 2024-05-17T22:39:48.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.328+0000 7f17d3fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e0f3049f0 0x555e0f497d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:48.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.329+0000 7f17d3fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e0f3049f0 0x555e0f497d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:49390/0 (socket says 172.21.15.12:49390) 2024-05-17T22:39:48.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.329+0000 7f17d3fff700 1 -- 172.21.15.12:0/1494825835 learned_addr learned my addr 172.21.15.12:0/1494825835 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:39:48.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.329+0000 7f17d37fe700 1 --2- 172.21.15.12:0/1494825835 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e0f482b00 0x555e0f487510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:48.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.329+0000 7f17d3fff700 1 -- 172.21.15.12:0/1494825835 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e0f482b00 msgr2=0x555e0f487510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:39:48.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.329+0000 7f17d3fff700 1 --2- 172.21.15.12:0/1494825835 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e0f482b00 0x555e0f487510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:48.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.329+0000 7f17d3fff700 1 -- 172.21.15.12:0/1494825835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e0f488400 msgr2=0x555e0f48bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:39:48.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.329+0000 7f17d3fff700 1 --2- 172.21.15.12:0/1494825835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e0f488400 0x555e0f48bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:48.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.329+0000 7f17d3fff700 1 -- 172.21.15.12:0/1494825835 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e0f479c20 con 0x555e0f3049f0 2024-05-17T22:39:48.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.329+0000 7f17d3fff700 1 --2- 172.21.15.12:0/1494825835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e0f3049f0 0x555e0f497d30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009a90 tx=0x7f17c4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7076bf7d46524e62 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:39:48.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.330+0000 7f17b9ffb700 1 -- 172.21.15.12:0/1494825835 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c4004250 con 0x555e0f3049f0 2024-05-17T22:39:48.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.330+0000 7f17b9ffb700 1 -- 172.21.15.12:0/1494825835 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17c40043f0 con 0x555e0f3049f0 2024-05-17T22:39:48.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.330+0000 7f17b9ffb700 1 -- 172.21.15.12:0/1494825835 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c4021620 con 0x555e0f3049f0 2024-05-17T22:39:48.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.331+0000 7f17e9137b80 1 -- 172.21.15.12:0/1494825835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e0f3049f0 msgr2=0x555e0f497d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:48.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.331+0000 7f17e9137b80 1 --2- 172.21.15.12:0/1494825835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e0f3049f0 0x555e0f497d30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009a90 tx=0x7f17c4009260 comp rx=0 tx=0).stop 2024-05-17T22:39:48.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.331+0000 7f17e9137b80 1 -- 172.21.15.12:0/1494825835 shutdown_connections 2024-05-17T22:39:48.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.331+0000 7f17e9137b80 1 --2- 172.21.15.12:0/1494825835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e0f488400 0x555e0f48bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.331+0000 7f17e9137b80 1 --2- 172.21.15.12:0/1494825835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e0f3049f0 0x555e0f497d30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.331+0000 7f17e9137b80 1 --2- 172.21.15.12:0/1494825835 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e0f482b00 0x555e0f487510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.331+0000 7f17e9137b80 1 -- 172.21.15.12:0/1494825835 >> 172.21.15.12:0/1494825835 conn(0x555e0f3647d0 msgr2=0x555e0f486e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:39:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.331+0000 7f17e9137b80 1 -- 172.21.15.12:0/1494825835 shutdown_connections 2024-05-17T22:39:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.331+0000 7f17e9137b80 1 -- 172.21.15.12:0/1494825835 wait complete. 2024-05-17T22:39:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.332+0000 7f17e9137b80 1 -- start start 2024-05-17T22:39:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.332+0000 7f17e9137b80 1 Processor -- start 2024-05-17T22:39:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.333+0000 7f17e9137b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e0f3049f0 0x555e0f494600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.333+0000 7f17e9137b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e0f482b00 0x555e0f4a2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.333+0000 7f17e9137b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e0f488400 0x555e0f4a3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.333+0000 7f17e9137b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f17d41b4870 con 0x555e0f3049f0 2024-05-17T22:39:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.333+0000 7f17e9137b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f17d41b5ce0 con 0x555e0f482b00 2024-05-17T22:39:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.333+0000 7f17e9137b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f17d41b4f40 con 0x555e0f488400 2024-05-17T22:39:48.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.333+0000 7f17e9137b80 10 client.0 _ll_register_callbacks cb 0x555e0f337b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:39:48.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.333+0000 7f17d3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e0f3049f0 0x555e0f494600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:48.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.333+0000 7f17d37fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e0f482b00 0x555e0f4a2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:48.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.333+0000 7f17d3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e0f3049f0 0x555e0f494600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36672/0 (socket says 172.21.15.12:36672) 2024-05-17T22:39:48.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.333+0000 7f17d3fff700 1 -- 172.21.15.12:0/3997439686 learned_addr learned my addr 172.21.15.12:0/3997439686 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:39:48.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.333+0000 7f17d8cf9700 1 --2- 172.21.15.12:0/3997439686 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e0f488400 0x555e0f4a3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:48.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.334+0000 7f17d37fe700 1 -- 172.21.15.12:0/3997439686 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e0f488400 msgr2=0x555e0f4a3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:39:48.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.334+0000 7f17d37fe700 1 --2- 172.21.15.12:0/3997439686 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e0f488400 0x555e0f4a3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:48.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.334+0000 7f17d37fe700 1 -- 172.21.15.12:0/3997439686 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e0f3049f0 msgr2=0x555e0f494600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:39:48.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.334+0000 7f17d37fe700 1 --2- 172.21.15.12:0/3997439686 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e0f3049f0 0x555e0f494600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:48.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.334+0000 7f17d37fe700 1 -- 172.21.15.12:0/3997439686 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e0f4a3ad0 con 0x555e0f482b00 2024-05-17T22:39:48.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.334+0000 7f17d8cf9700 1 --2- 172.21.15.12:0/3997439686 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e0f488400 0x555e0f4a3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:39:48.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.334+0000 7f17d37fe700 1 --2- 172.21.15.12:0/3997439686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e0f482b00 0x555e0f4a2ed0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f17c80a7b30 tx=0x7f17c80aa290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=242d16a707a0a9ae server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:39:48.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.334+0000 7f17b37fe700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:39:48.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.334+0000 7f17b37fe700 1 -- 172.21.15.12:0/3997439686 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c80c4060 con 0x555e0f482b00 2024-05-17T22:39:48.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.334+0000 7f17b37fe700 1 -- 172.21.15.12:0/3997439686 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f17c80afe00 con 0x555e0f482b00 2024-05-17T22:39:48.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.334+0000 7f17e9137b80 1 -- 172.21.15.12:0/3997439686 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17c40043f0 con 0x555e0f482b00 2024-05-17T22:39:48.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.335+0000 7f17b37fe700 1 -- 172.21.15.12:0/3997439686 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f17c80aafc0 con 0x555e0f482b00 2024-05-17T22:39:48.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.335+0000 7f17e9137b80 10 client.25073 fetch_fsmap learned FSMap version 29 2024-05-17T22:39:48.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.335+0000 7f17e9137b80 1 -- 172.21.15.12:0/3997439686 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555e0f506380 con 0x555e0f482b00 2024-05-17T22:39:48.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.336+0000 7f17b37fe700 1 -- 172.21.15.12:0/3997439686 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17c40043f0 con 0x555e0f482b00 2024-05-17T22:39:48.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.336+0000 7f17b37fe700 1 -- 172.21.15.12:0/3997439686 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f17c80a6b50 con 0x555e0f482b00 2024-05-17T22:39:48.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.336+0000 7f17e9137b80 10 client.25073 fetch_fsmap finished waiting for FSMap version 29 2024-05-17T22:39:48.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.336+0000 7f17e9137b80 10 client.25073 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:39:48.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.336+0000 7f17e9137b80 20 client.25073 populate_metadata read hostname 'smithi012' 2024-05-17T22:39:48.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.336+0000 7f17e9137b80 1 --2- 172.21.15.12:0/3997439686 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555e0f506980 0x555e0f508da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:48.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.336+0000 7f17e9137b80 4 client.25073 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:39:48.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.336+0000 7f17e9137b80 1 -- 172.21.15.12:0/3997439686 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555e0f3ac800 con 0x555e0f506980 2024-05-17T22:39:48.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.337+0000 7f17d3fff700 1 --2- 172.21.15.12:0/3997439686 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555e0f506980 0x555e0f508da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:48.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.338+0000 7f17d3fff700 1 --2- 172.21.15.12:0/3997439686 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555e0f506980 0x555e0f508da0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91789baa44879f8c server_cookie=48dc9054ed9e88e8 in_seq=0 out_seq=0 2024-05-17T22:39:48.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.338+0000 7f17b37fe700 10 client.25073 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:39:48.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.338+0000 7f17b37fe700 1 -- 172.21.15.12:0/3997439686 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555e0f3ac800 con 0x555e0f506980 2024-05-17T22:39:48.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.338+0000 7f17b37fe700 10 client.25073 handle_command_reply: tid=0 2024-05-17T22:39:48.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.338+0000 7f17e9137b80 1 client.25073 shutdown 2024-05-17T22:39:48.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.339+0000 7f17e9137b80 1 -- 172.21.15.12:0/3997439686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e0f482b00 msgr2=0x555e0f4a2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:48.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.339+0000 7f17e9137b80 1 --2- 172.21.15.12:0/3997439686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e0f482b00 0x555e0f4a2ed0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f17c80a7b30 tx=0x7f17c80aa290 comp rx=0 tx=0).stop 2024-05-17T22:39:48.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.339+0000 7f17e9137b80 1 -- 172.21.15.12:0/3997439686 shutdown_connections 2024-05-17T22:39:48.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.339+0000 7f17e9137b80 1 --2- 172.21.15.12:0/3997439686 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e0f488400 0x555e0f4a3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:48.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.339+0000 7f17e9137b80 1 --2- 172.21.15.12:0/3997439686 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555e0f506980 0x555e0f508da0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:48.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.339+0000 7f17e9137b80 1 --2- 172.21.15.12:0/3997439686 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e0f3049f0 0x555e0f494600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:48.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.339+0000 7f17e9137b80 1 --2- 172.21.15.12:0/3997439686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e0f482b00 0x555e0f4a2ed0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:48.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.339+0000 7f17b37fe700 0 client.25073 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:39:48.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.339+0000 7f17e9137b80 1 -- 172.21.15.12:0/3997439686 >> 172.21.15.12:0/3997439686 conn(0x555e0f3647d0 msgr2=0x555e0f477110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:39:48.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.339+0000 7f17e9137b80 1 -- 172.21.15.12:0/3997439686 shutdown_connections 2024-05-17T22:39:48.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.339+0000 7f17e9137b80 1 -- 172.21.15.12:0/3997439686 wait complete. 2024-05-17T22:39:48.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.339+0000 7f17e9137b80 20 client.25073 trim_cache size 0 max 16384 2024-05-17T22:39:48.343 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:39:48.344 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-17T22:39:48.344 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-17T22:39:48.344 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:39:48.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.340+0000 7f17da75c700 1 -- 172.21.15.12:0/2975746703 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f17bc0b2880 msgr2=0x7f17bc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:48.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.340+0000 7f17da75c700 1 --2- 172.21.15.12:0/2975746703 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f17bc0b2880 0x7f17bc0b4d90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f17cc020000 tx=0x7f17cc021000 comp rx=0 tx=0).stop 2024-05-17T22:39:48.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.340+0000 7f17da75c700 1 -- 172.21.15.12:0/2975746703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17d4107730 msgr2=0x7f17d41a13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:48.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.340+0000 7f17da75c700 1 --2- 172.21.15.12:0/2975746703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17d4107730 0x7f17d41a13c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f17c8009230 tx=0x7f17c800fa40 comp rx=0 tx=0).stop 2024-05-17T22:39:48.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.341+0000 7f17da75c700 1 -- 172.21.15.12:0/2975746703 shutdown_connections 2024-05-17T22:39:48.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.341+0000 7f17da75c700 1 --2- 172.21.15.12:0/2975746703 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f17bc0b2880 0x7f17bc0b4d90 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:48.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.341+0000 7f17da75c700 1 --2- 172.21.15.12:0/2975746703 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17d4108080 0x7f17d41a5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:48.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.341+0000 7f17da75c700 1 --2- 172.21.15.12:0/2975746703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17d4102e00 0x7f17d41a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:48.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.341+0000 7f17da75c700 1 --2- 172.21.15.12:0/2975746703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17d4107730 0x7f17d41a13c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:48.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.341+0000 7f17da75c700 1 -- 172.21.15.12:0/2975746703 >> 172.21.15.12:0/2975746703 conn(0x7f17d4079bd0 msgr2=0x7f17d41048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:39:48.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.341+0000 7f17da75c700 1 -- 172.21.15.12:0/2975746703 shutdown_connections 2024-05-17T22:39:48.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:48.342+0000 7f17da75c700 1 -- 172.21.15.12:0/2975746703 wait complete. 2024-05-17T22:39:49.488 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-17T22:39:49.489 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T22:39:49.943 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:39:50.078 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:49 smithi012 ceph-mon[31632]: pgmap v456: 129 pgs: 129 active+clean; 1012 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 25 MiB/s wr, 1.07k op/s 2024-05-17T22:39:50.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:49 smithi151 ceph-mon[36101]: pgmap v456: 129 pgs: 129 active+clean; 1012 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 25 MiB/s wr, 1.07k op/s 2024-05-17T22:39:50.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:49 smithi096 ceph-mon[36065]: pgmap v456: 129 pgs: 129 active+clean; 1012 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 25 MiB/s wr, 1.07k op/s 2024-05-17T22:39:51.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:50 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:39:51.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:50 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:39:51.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:50 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:39:51.370 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 30.0 s, 2629.6 tps, lat 38.372 ms stddev 46.758 2024-05-17T22:39:52.082 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:51 smithi012 ceph-mon[31632]: pgmap v457: 129 pgs: 129 active+clean; 1009 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 25 MiB/s wr, 861 op/s 2024-05-17T22:39:52.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:51 smithi151 ceph-mon[36101]: pgmap v457: 129 pgs: 129 active+clean; 1009 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 25 MiB/s wr, 861 op/s 2024-05-17T22:39:52.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:51 smithi096 ceph-mon[36065]: pgmap v457: 129 pgs: 129 active+clean; 1009 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 25 MiB/s wr, 861 op/s 2024-05-17T22:39:52.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.408+0000 7f39c17b6700 1 -- 172.21.15.12:0/2630038814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39bc10aa50 msgr2=0x7f39bc10ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:52.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.408+0000 7f39c17b6700 1 --2- 172.21.15.12:0/2630038814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39bc10aa50 0x7f39bc10ae60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f39b40098f0 tx=0x7f39b4009c00 comp rx=0 tx=0).stop 2024-05-17T22:39:52.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.408+0000 7f39c17b6700 1 -- 172.21.15.12:0/2630038814 shutdown_connections 2024-05-17T22:39:52.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.408+0000 7f39c17b6700 1 --2- 172.21.15.12:0/2630038814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39bc10b3a0 0x7f39bc112d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:52.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.408+0000 7f39c17b6700 1 --2- 172.21.15.12:0/2630038814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39bc106120 0x7f39bc106530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:52.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.408+0000 7f39c17b6700 1 --2- 172.21.15.12:0/2630038814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39bc10aa50 0x7f39bc10ae60 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:52.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.408+0000 7f39c17b6700 1 -- 172.21.15.12:0/2630038814 >> 172.21.15.12:0/2630038814 conn(0x7f39bc101b80 msgr2=0x7f39bc103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:39:52.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.408+0000 7f39c17b6700 1 -- 172.21.15.12:0/2630038814 shutdown_connections 2024-05-17T22:39:52.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.408+0000 7f39c17b6700 1 -- 172.21.15.12:0/2630038814 wait complete. 2024-05-17T22:39:52.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.409+0000 7f39c17b6700 1 Processor -- start 2024-05-17T22:39:52.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.409+0000 7f39c17b6700 1 -- start start 2024-05-17T22:39:52.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.410+0000 7f39c17b6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39bc106120 0x7f39bc1a2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:52.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.410+0000 7f39c17b6700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39bc10aa50 0x7f39bc1a2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:52.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.410+0000 7f39c17b6700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39bc10b3a0 0x7f39bc19c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:52.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.410+0000 7f39c17b6700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc116010 con 0x7f39bc106120 2024-05-17T22:39:52.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.410+0000 7f39c17b6700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc115e90 con 0x7f39bc10b3a0 2024-05-17T22:39:52.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.410+0000 7f39c17b6700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc116190 con 0x7f39bc10aa50 2024-05-17T22:39:52.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.410+0000 7f39ba7fc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39bc10aa50 0x7f39bc1a2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:52.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.410+0000 7f39ba7fc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39bc10aa50 0x7f39bc1a2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:44436/0 (socket says 172.21.15.12:44436) 2024-05-17T22:39:52.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.410+0000 7f39ba7fc700 1 -- 172.21.15.12:0/3993113893 learned_addr learned my addr 172.21.15.12:0/3993113893 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:39:52.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.411+0000 7f39ba7fc700 1 -- 172.21.15.12:0/3993113893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39bc10b3a0 msgr2=0x7f39bc19c8a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-17T22:39:52.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.411+0000 7f39ba7fc700 1 --2- 172.21.15.12:0/3993113893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39bc10b3a0 0x7f39bc19c8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:52.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.411+0000 7f39ba7fc700 1 -- 172.21.15.12:0/3993113893 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39bc106120 msgr2=0x7f39bc1a2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:52.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.411+0000 7f39ba7fc700 1 --2- 172.21.15.12:0/3993113893 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39bc106120 0x7f39bc1a2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:52.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.411+0000 7f39ba7fc700 1 -- 172.21.15.12:0/3993113893 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39bc19cf50 con 0x7f39bc10aa50 2024-05-17T22:39:52.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.411+0000 7f39ba7fc700 1 --2- 172.21.15.12:0/3993113893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39bc10aa50 0x7f39bc1a2bc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f39b4009fd0 tx=0x7f39b4005e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:39:52.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.411+0000 7f39b3fff700 1 -- 172.21.15.12:0/3993113893 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b4005270 con 0x7f39bc10aa50 2024-05-17T22:39:52.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.412+0000 7f39b3fff700 1 -- 172.21.15.12:0/3993113893 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39b400ba10 con 0x7f39bc10aa50 2024-05-17T22:39:52.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.412+0000 7f39b3fff700 1 -- 172.21.15.12:0/3993113893 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b4003bd0 con 0x7f39bc10aa50 2024-05-17T22:39:52.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.412+0000 7f39c17b6700 1 -- 172.21.15.12:0/3993113893 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39bc19d1e0 con 0x7f39bc10aa50 2024-05-17T22:39:52.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.412+0000 7f39c17b6700 1 -- 172.21.15.12:0/3993113893 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39bc19d5b0 con 0x7f39bc10aa50 2024-05-17T22:39:52.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.415+0000 7f39c17b6700 1 -- 172.21.15.12:0/3993113893 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39a0005330 con 0x7f39bc10aa50 2024-05-17T22:39:52.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.416+0000 7f39b3fff700 1 -- 172.21.15.12:0/3993113893 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f39b402c440 con 0x7f39bc10aa50 2024-05-17T22:39:52.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.417+0000 7f39b3fff700 1 --2- 172.21.15.12:0/3993113893 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f399c0b29d0 0x7f399c0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:52.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.417+0000 7f39b3fff700 1 -- 172.21.15.12:0/3993113893 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f39b40e3540 con 0x7f39bc10aa50 2024-05-17T22:39:52.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.418+0000 7f39baffd700 1 --2- 172.21.15.12:0/3993113893 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f399c0b29d0 0x7f399c0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:52.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.418+0000 7f39baffd700 1 --2- 172.21.15.12:0/3993113893 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f399c0b29d0 0x7f399c0b4ee0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f39a4005950 tx=0x7f39a40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:39:52.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.430+0000 7f39b3fff700 1 -- 172.21.15.12:0/3993113893 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f39b4013030 con 0x7f39bc10aa50 2024-05-17T22:39:52.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.929+0000 7f39c17b6700 1 -- 172.21.15.12:0/3993113893 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39a0005a40 con 0x7f39bc10aa50 2024-05-17T22:39:52.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.930+0000 7f39b3fff700 1 -- 172.21.15.12:0/3993113893 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11873 (secure 0 0 0) 0x7f39b40aaf60 con 0x7f39bc10aa50 2024-05-17T22:39:52.933 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:39:52.933 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1407219319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1407219319},{"type":"v1","addr":"172.21.15.12:6835","nonce":1407219319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/630977886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":630977886},{"type":"v1","addr":"172.21.15.12:6837","nonce":630977886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3731914742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3731914742},{"type":"v1","addr":"172.21.15.12:6839","nonce":3731914742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/3642142501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3642142501},{"type":"v1","addr":"172.21.15.12:6841","nonce":3642142501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3554053059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3554053059},{"type":"v1","addr":"172.21.15.151:6833","nonce":3554053059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/4228325569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":4228325569},{"type":"v1","addr":"172.21.15.96:6833","nonce":4228325569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T22:34:42.642111+0000","modified":"2024-05-17T22:39:43.140707+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24461,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/542323754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":542323754},{"type":"v1","addr":"172.21.15.151:6835","nonce":542323754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1758442232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1758442232},{"type":"v1","addr":"172.21.15.96:6835","nonce":1758442232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3403324038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3403324038},{"type":"v1","addr":"172.21.15.151:6837","nonce":3403324038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.151:6839/2338353849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2338353849},{"type":"v1","addr":"172.21.15.151:6839","nonce":2338353849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.96:6837/4222574338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":4222574338},{"type":"v1","addr":"172.21.15.96:6837","nonce":4222574338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/2986295519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2986295519},{"type":"v1","addr":"172.21.15.96:6839","nonce":2986295519}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24457,24461,24482]},"id":1}]} 2024-05-17T22:39:52.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.934+0000 7f39c17b6700 1 -- 172.21.15.12:0/3993113893 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f399c0b29d0 msgr2=0x7f399c0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:52.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.934+0000 7f39c17b6700 1 --2- 172.21.15.12:0/3993113893 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f399c0b29d0 0x7f399c0b4ee0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f39a4005950 tx=0x7f39a40058e0 comp rx=0 tx=0).stop 2024-05-17T22:39:52.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.934+0000 7f39c17b6700 1 -- 172.21.15.12:0/3993113893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39bc10aa50 msgr2=0x7f39bc1a2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:52.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.934+0000 7f39c17b6700 1 --2- 172.21.15.12:0/3993113893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39bc10aa50 0x7f39bc1a2bc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f39b4009fd0 tx=0x7f39b4005e40 comp rx=0 tx=0).stop 2024-05-17T22:39:52.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.934+0000 7f39c17b6700 1 -- 172.21.15.12:0/3993113893 shutdown_connections 2024-05-17T22:39:52.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.934+0000 7f39c17b6700 1 --2- 172.21.15.12:0/3993113893 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f399c0b29d0 0x7f399c0b4ee0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:52.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.934+0000 7f39c17b6700 1 --2- 172.21.15.12:0/3993113893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39bc10b3a0 0x7f39bc19c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:52.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.934+0000 7f39c17b6700 1 --2- 172.21.15.12:0/3993113893 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39bc106120 0x7f39bc1a2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:52.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.934+0000 7f39c17b6700 1 --2- 172.21.15.12:0/3993113893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39bc10aa50 0x7f39bc1a2bc0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:52.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.934+0000 7f39c17b6700 1 -- 172.21.15.12:0/3993113893 >> 172.21.15.12:0/3993113893 conn(0x7f39bc101b80 msgr2=0x7f39bc102210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:39:52.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.934+0000 7f39c17b6700 1 -- 172.21.15.12:0/3993113893 shutdown_connections 2024-05-17T22:39:52.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:52.934+0000 7f39c17b6700 1 -- 172.21.15.12:0/3993113893 wait complete. 2024-05-17T22:39:52.939 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 29 2024-05-17T22:39:53.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:52 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:39:53.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:52 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:39:53.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:52 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:39:53.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:52 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:39:53.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:52 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:39:53.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:52 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:39:53.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:52 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:39:53.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:52 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:39:53.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:52 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:39:54.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:53 smithi012 ceph-mon[31632]: pgmap v458: 129 pgs: 129 active+clean; 991 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 26 MiB/s wr, 970 op/s 2024-05-17T22:39:54.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:53 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/3993113893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:39:54.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:53 smithi151 ceph-mon[36101]: pgmap v458: 129 pgs: 129 active+clean; 991 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 26 MiB/s wr, 970 op/s 2024-05-17T22:39:54.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:53 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/3993113893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:39:54.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:53 smithi096 ceph-mon[36065]: pgmap v458: 129 pgs: 129 active+clean; 991 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 26 MiB/s wr, 970 op/s 2024-05-17T22:39:54.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:53 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/3993113893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:39:56.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:55 smithi012 ceph-mon[31632]: pgmap v459: 129 pgs: 129 active+clean; 977 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 27 MiB/s wr, 1.05k op/s 2024-05-17T22:39:56.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:55 smithi151 ceph-mon[36101]: pgmap v459: 129 pgs: 129 active+clean; 977 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 27 MiB/s wr, 1.05k op/s 2024-05-17T22:39:56.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:55 smithi096 ceph-mon[36065]: pgmap v459: 129 pgs: 129 active+clean; 977 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 27 MiB/s wr, 1.05k op/s 2024-05-17T22:39:56.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 35.0 s, 2863.8 tps, lat 34.837 ms stddev 44.427 2024-05-17T22:39:57.365 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-17T22:39:57.807 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:39:58.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:57 smithi012 ceph-mon[31632]: pgmap v460: 129 pgs: 129 active+clean; 981 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 23 MiB/s wr, 878 op/s 2024-05-17T22:39:58.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:57 smithi151 ceph-mon[36101]: pgmap v460: 129 pgs: 129 active+clean; 981 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 23 MiB/s wr, 878 op/s 2024-05-17T22:39:58.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:57 smithi096 ceph-mon[36065]: pgmap v460: 129 pgs: 129 active+clean; 981 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 23 MiB/s wr, 878 op/s 2024-05-17T22:39:59.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.725+0000 7fe490350700 1 -- 172.21.15.12:0/2001341014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe488102e00 msgr2=0x7fe488103210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:59.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.725+0000 7fe490350700 1 --2- 172.21.15.12:0/2001341014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe488102e00 0x7fe488103210 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe484009990 tx=0x7fe484009ca0 comp rx=0 tx=0).stop 2024-05-17T22:39:59.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.725+0000 7fe490350700 1 -- 172.21.15.12:0/2001341014 shutdown_connections 2024-05-17T22:39:59.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.725+0000 7fe490350700 1 --2- 172.21.15.12:0/2001341014 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe488108080 0x7fe48810f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:59.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.725+0000 7fe490350700 1 --2- 172.21.15.12:0/2001341014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe488102e00 0x7fe488103210 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:59.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.725+0000 7fe490350700 1 --2- 172.21.15.12:0/2001341014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe488107730 0x7fe488107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:59.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.725+0000 7fe490350700 1 -- 172.21.15.12:0/2001341014 >> 172.21.15.12:0/2001341014 conn(0x7fe488079bd0 msgr2=0x7fe488079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:39:59.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.726+0000 7fe490350700 1 -- 172.21.15.12:0/2001341014 shutdown_connections 2024-05-17T22:39:59.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.726+0000 7fe490350700 1 -- 172.21.15.12:0/2001341014 wait complete. 2024-05-17T22:39:59.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.727+0000 7fe490350700 1 Processor -- start 2024-05-17T22:39:59.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.727+0000 7fe490350700 1 -- start start 2024-05-17T22:39:59.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.727+0000 7fe490350700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe488102e00 0x7fe4881a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:59.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.727+0000 7fe490350700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe488107730 0x7fe4881a6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:59.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.727+0000 7fe490350700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe488108080 0x7fe4881a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:59.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.727+0000 7fe490350700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe488111cc0 con 0x7fe488102e00 2024-05-17T22:39:59.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.727+0000 7fe490350700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe488111b40 con 0x7fe488107730 2024-05-17T22:39:59.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.727+0000 7fe490350700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe488111e40 con 0x7fe488108080 2024-05-17T22:39:59.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.727+0000 7fe48e0ec700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe488102e00 0x7fe4881a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:59.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.728+0000 7fe48e0ec700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe488102e00 0x7fe4881a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51052/0 (socket says 172.21.15.12:51052) 2024-05-17T22:39:59.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.728+0000 7fe48e0ec700 1 -- 172.21.15.12:0/2945326811 learned_addr learned my addr 172.21.15.12:0/2945326811 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:39:59.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.728+0000 7fe48d8eb700 1 --2- 172.21.15.12:0/2945326811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe488107730 0x7fe4881a6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:59.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.728+0000 7fe48e8ed700 1 --2- 172.21.15.12:0/2945326811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe488108080 0x7fe4881a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:59.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.728+0000 7fe48e0ec700 1 -- 172.21.15.12:0/2945326811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe488108080 msgr2=0x7fe4881a0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:59.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.728+0000 7fe48e0ec700 1 --2- 172.21.15.12:0/2945326811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe488108080 0x7fe4881a0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:59.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.728+0000 7fe48e0ec700 1 -- 172.21.15.12:0/2945326811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe488107730 msgr2=0x7fe4881a6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:59.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.728+0000 7fe48e0ec700 1 --2- 172.21.15.12:0/2945326811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe488107730 0x7fe4881a6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:59.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.728+0000 7fe48e0ec700 1 -- 172.21.15.12:0/2945326811 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4881a10c0 con 0x7fe488102e00 2024-05-17T22:39:59.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.728+0000 7fe48d8eb700 1 --2- 172.21.15.12:0/2945326811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe488107730 0x7fe4881a6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:39:59.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.728+0000 7fe48e0ec700 1 --2- 172.21.15.12:0/2945326811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe488102e00 0x7fe4881a67f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe484005d10 tx=0x7fe48400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:39:59.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.729+0000 7fe47f7fe700 1 -- 172.21.15.12:0/2945326811 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe484026070 con 0x7fe488102e00 2024-05-17T22:39:59.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.729+0000 7fe47f7fe700 1 -- 172.21.15.12:0/2945326811 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe48400fda0 con 0x7fe488102e00 2024-05-17T22:39:59.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.730+0000 7fe47f7fe700 1 -- 172.21.15.12:0/2945326811 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe484017650 con 0x7fe488102e00 2024-05-17T22:39:59.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.730+0000 7fe490350700 1 -- 172.21.15.12:0/2945326811 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4881a1350 con 0x7fe488102e00 2024-05-17T22:39:59.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.730+0000 7fe490350700 1 -- 172.21.15.12:0/2945326811 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4881a1720 con 0x7fe488102e00 2024-05-17T22:39:59.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.738+0000 7fe47f7fe700 1 -- 172.21.15.12:0/2945326811 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe484009d10 con 0x7fe488102e00 2024-05-17T22:39:59.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.739+0000 7fe47f7fe700 1 --2- 172.21.15.12:0/2945326811 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe4740b2a10 0x7fe4740b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:59.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.740+0000 7fe47f7fe700 1 -- 172.21.15.12:0/2945326811 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe4840e2ce0 con 0x7fe488102e00 2024-05-17T22:39:59.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.740+0000 7fe48d8eb700 1 --2- 172.21.15.12:0/2945326811 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe4740b2a10 0x7fe4740b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:59.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.740+0000 7fe49ed2bb80 1 Processor -- start 2024-05-17T22:39:59.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.742+0000 7fe48d8eb700 1 --2- 172.21.15.12:0/2945326811 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe4740b2a10 0x7fe4740b4f20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe488106cb0 tx=0x7fe478009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:39:59.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.742+0000 7fe49ed2bb80 1 -- start start 2024-05-17T22:39:59.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.743+0000 7fe49ed2bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3ed2d39f0 0x55d3ed456ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:59.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.743+0000 7fe49ed2bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3ed457400 0x55d3ed4515c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:59.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.743+0000 7fe49ed2bb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3ed451b00 0x55d3ed451f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:59.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.743+0000 7fe49ed2bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55d3ed3800d0 con 0x55d3ed2d39f0 2024-05-17T22:39:59.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.743+0000 7fe49ed2bb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d3ed36fc70 con 0x55d3ed451b00 2024-05-17T22:39:59.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.743+0000 7fe49ed2bb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d3ed373cb0 con 0x55d3ed457400 2024-05-17T22:39:59.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.743+0000 7fe48d8eb700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3ed457400 0x55d3ed4515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:59.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.743+0000 7fe48d8eb700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3ed457400 0x55d3ed4515c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:45180/0 (socket says 172.21.15.12:45180) 2024-05-17T22:39:59.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.743+0000 7fe48d8eb700 1 -- 172.21.15.12:0/2591177813 learned_addr learned my addr 172.21.15.12:0/2591177813 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:39:59.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.744+0000 7fe48e0ec700 1 --2- 172.21.15.12:0/2591177813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3ed2d39f0 0x55d3ed456ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:59.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.744+0000 7fe48e8ed700 1 --2- 172.21.15.12:0/2591177813 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3ed451b00 0x55d3ed451f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:59.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.744+0000 7fe48e0ec700 1 -- 172.21.15.12:0/2591177813 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3ed457400 msgr2=0x55d3ed4515c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:39:59.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.744+0000 7fe48e0ec700 1 --2- 172.21.15.12:0/2591177813 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3ed457400 0x55d3ed4515c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:59.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.744+0000 7fe48e0ec700 1 -- 172.21.15.12:0/2591177813 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3ed451b00 msgr2=0x55d3ed451f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:39:59.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.745+0000 7fe48e0ec700 1 --2- 172.21.15.12:0/2591177813 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3ed451b00 0x55d3ed451f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:59.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.745+0000 7fe48e0ec700 1 -- 172.21.15.12:0/2591177813 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3ed3095c0 con 0x55d3ed2d39f0 2024-05-17T22:39:59.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.745+0000 7fe48e8ed700 1 --2- 172.21.15.12:0/2591177813 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3ed451b00 0x55d3ed451f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:39:59.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.745+0000 7fe48e0ec700 1 --2- 172.21.15.12:0/2591177813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3ed2d39f0 0x55d3ed456ec0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe4840a9ca0 tx=0x7fe48400bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=141325e299ec3137 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:39:59.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.746+0000 7fe47cd46700 1 -- 172.21.15.12:0/2591177813 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4840aa6a0 con 0x55d3ed2d39f0 2024-05-17T22:39:59.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.746+0000 7fe47cd46700 1 -- 172.21.15.12:0/2591177813 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4840a9fa0 con 0x55d3ed2d39f0 2024-05-17T22:39:59.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.746+0000 7fe47cd46700 1 -- 172.21.15.12:0/2591177813 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4840a5910 con 0x55d3ed2d39f0 2024-05-17T22:39:59.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.747+0000 7fe49ed2bb80 1 -- 172.21.15.12:0/2591177813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3ed2d39f0 msgr2=0x55d3ed456ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:59.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.747+0000 7fe49ed2bb80 1 --2- 172.21.15.12:0/2591177813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3ed2d39f0 0x55d3ed456ec0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe4840a9ca0 tx=0x7fe48400bfd0 comp rx=0 tx=0).stop 2024-05-17T22:39:59.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.747+0000 7fe49ed2bb80 1 -- 172.21.15.12:0/2591177813 shutdown_connections 2024-05-17T22:39:59.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.747+0000 7fe49ed2bb80 1 --2- 172.21.15.12:0/2591177813 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3ed451b00 0x55d3ed451f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:59.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.747+0000 7fe49ed2bb80 1 --2- 172.21.15.12:0/2591177813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3ed2d39f0 0x55d3ed456ec0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:59.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.747+0000 7fe49ed2bb80 1 --2- 172.21.15.12:0/2591177813 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3ed457400 0x55d3ed4515c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:59.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.747+0000 7fe49ed2bb80 1 -- 172.21.15.12:0/2591177813 >> 172.21.15.12:0/2591177813 conn(0x55d3ed3337d0 msgr2=0x55d3ed44c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:39:59.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.753+0000 7fe49ed2bb80 1 -- 172.21.15.12:0/2591177813 shutdown_connections 2024-05-17T22:39:59.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.753+0000 7fe49ed2bb80 1 -- 172.21.15.12:0/2591177813 wait complete. 2024-05-17T22:39:59.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.754+0000 7fe49ed2bb80 1 -- start start 2024-05-17T22:39:59.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.754+0000 7fe49ed2bb80 1 Processor -- start 2024-05-17T22:39:59.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.755+0000 7fe49ed2bb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3ed2d39f0 0x55d3ed45ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:59.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.755+0000 7fe49ed2bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3ed451b00 0x55d3ed45b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:59.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.755+0000 7fe49ed2bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3ed457400 0x55d3ed45b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:59.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.756+0000 7fe49ed2bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55d3ed277150 con 0x55d3ed457400 2024-05-17T22:39:59.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.756+0000 7fe49ed2bb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d3ed311940 con 0x55d3ed2d39f0 2024-05-17T22:39:59.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.756+0000 7fe48d8eb700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3ed451b00 0x55d3ed45b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:59.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.756+0000 7fe48d8eb700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3ed451b00 0x55d3ed45b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:45196/0 (socket says 172.21.15.12:45196) 2024-05-17T22:39:59.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.756+0000 7fe48d8eb700 1 -- 172.21.15.12:0/4035364433 learned_addr learned my addr 172.21.15.12:0/4035364433 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:39:59.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.756+0000 7fe48e0ec700 1 --2- 172.21.15.12:0/4035364433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3ed2d39f0 0x55d3ed45ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:59.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.756+0000 7fe49ed2bb80 1 -- 172.21.15.12:0/4035364433 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d3ed45dc20 con 0x55d3ed451b00 2024-05-17T22:39:59.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.757+0000 7fe49ed2bb80 10 client.0 _ll_register_callbacks cb 0x55d3ed306b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:39:59.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.759+0000 7fe48e8ed700 1 --2- 172.21.15.12:0/4035364433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3ed457400 0x55d3ed45b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:59.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.759+0000 7fe48d8eb700 1 -- 172.21.15.12:0/4035364433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3ed2d39f0 msgr2=0x55d3ed45ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:39:59.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.759+0000 7fe48d8eb700 1 --2- 172.21.15.12:0/4035364433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3ed2d39f0 0x55d3ed45ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:59.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.759+0000 7fe48d8eb700 1 -- 172.21.15.12:0/4035364433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3ed457400 msgr2=0x55d3ed45b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:39:59.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.759+0000 7fe48d8eb700 1 --2- 172.21.15.12:0/4035364433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3ed457400 0x55d3ed45b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:59.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.759+0000 7fe48d8eb700 1 -- 172.21.15.12:0/4035364433 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3ed2c0480 con 0x55d3ed451b00 2024-05-17T22:39:59.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.760+0000 7fe48d8eb700 1 --2- 172.21.15.12:0/4035364433 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3ed451b00 0x55d3ed45b080 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe478009ee0 tx=0x7fe4780047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a7c0603beef3cfb server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:39:59.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.760+0000 7fe471ffb700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:39:59.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.763+0000 7fe471ffb700 1 -- 172.21.15.12:0/4035364433 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47801fa90 con 0x55d3ed451b00 2024-05-17T22:39:59.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.763+0000 7fe471ffb700 1 -- 172.21.15.12:0/4035364433 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe47801fc30 con 0x55d3ed451b00 2024-05-17T22:39:59.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.763+0000 7fe49ed2bb80 1 -- 172.21.15.12:0/4035364433 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4840a9fa0 con 0x55d3ed451b00 2024-05-17T22:39:59.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.764+0000 7fe471ffb700 1 -- 172.21.15.12:0/4035364433 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe47801fe30 con 0x55d3ed451b00 2024-05-17T22:39:59.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.765+0000 7fe49ed2bb80 10 client.25045 fetch_fsmap learned FSMap version 29 2024-05-17T22:39:59.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.765+0000 7fe49ed2bb80 1 -- 172.21.15.12:0/4035364433 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d3ed468490 con 0x55d3ed451b00 2024-05-17T22:39:59.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.767+0000 7fe471ffb700 1 -- 172.21.15.12:0/4035364433 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe478032920 con 0x55d3ed451b00 2024-05-17T22:39:59.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.767+0000 7fe471ffb700 1 -- 172.21.15.12:0/4035364433 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fe478026070 con 0x55d3ed451b00 2024-05-17T22:39:59.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.768+0000 7fe49ed2bb80 10 client.25045 fetch_fsmap finished waiting for FSMap version 29 2024-05-17T22:39:59.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.768+0000 7fe49ed2bb80 10 client.25045 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-17T22:39:59.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.768+0000 7fe49ed2bb80 20 client.25045 populate_metadata read hostname 'smithi012' 2024-05-17T22:39:59.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.768+0000 7fe49ed2bb80 1 --2- 172.21.15.12:0/4035364433 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55d3ed46aa20 0x55d3ed46ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:59.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.768+0000 7fe49ed2bb80 4 client.25045 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:39:59.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.768+0000 7fe49ed2bb80 1 -- 172.21.15.12:0/4035364433 --> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe4840a9fa0 con 0x55d3ed46aa20 2024-05-17T22:39:59.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.769+0000 7fe48e0ec700 1 --2- 172.21.15.12:0/4035364433 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55d3ed46aa20 0x55d3ed46ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:59.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.770+0000 7fe48e0ec700 1 --2- 172.21.15.12:0/4035364433 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55d3ed46aa20 0x55d3ed46ce40 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a61fcc4c4abea98f server_cookie=cf55bc604e313a8f in_seq=0 out_seq=0 2024-05-17T22:39:59.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.770+0000 7fe471ffb700 10 client.25045 ms_handle_connect on v2:172.21.15.151:6838/2338353849 2024-05-17T22:39:59.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.771+0000 7fe471ffb700 1 -- 172.21.15.12:0/4035364433 <== mds.2 v2:172.21.15.151:6838/2338353849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fe4840a9fa0 con 0x55d3ed46aa20 2024-05-17T22:39:59.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.771+0000 7fe471ffb700 10 client.25045 handle_command_reply: tid=0 2024-05-17T22:39:59.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.772+0000 7fe49ed2bb80 1 client.25045 shutdown 2024-05-17T22:39:59.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.773+0000 7fe49ed2bb80 1 -- 172.21.15.12:0/4035364433 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3ed451b00 msgr2=0x55d3ed45b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:59.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.773+0000 7fe49ed2bb80 1 --2- 172.21.15.12:0/4035364433 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3ed451b00 0x55d3ed45b080 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe478009ee0 tx=0x7fe4780047c0 comp rx=0 tx=0).stop 2024-05-17T22:39:59.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.773+0000 7fe49ed2bb80 1 -- 172.21.15.12:0/4035364433 shutdown_connections 2024-05-17T22:39:59.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.773+0000 7fe49ed2bb80 1 --2- 172.21.15.12:0/4035364433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3ed457400 0x55d3ed45b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:59.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.773+0000 7fe49ed2bb80 1 --2- 172.21.15.12:0/4035364433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3ed2d39f0 0x55d3ed45ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:59.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.773+0000 7fe49ed2bb80 1 --2- 172.21.15.12:0/4035364433 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55d3ed46aa20 0x55d3ed46ce40 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:59.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.773+0000 7fe49ed2bb80 1 --2- 172.21.15.12:0/4035364433 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3ed451b00 0x55d3ed45b080 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:59.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.773+0000 7fe49ed2bb80 1 -- 172.21.15.12:0/4035364433 >> 172.21.15.12:0/4035364433 conn(0x55d3ed3337d0 msgr2=0x55d3ed44c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:39:59.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.773+0000 7fe471ffb700 0 client.25045 ms_handle_reset on v2:172.21.15.151:6838/2338353849 2024-05-17T22:39:59.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.773+0000 7fe49ed2bb80 1 -- 172.21.15.12:0/4035364433 shutdown_connections 2024-05-17T22:39:59.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.773+0000 7fe49ed2bb80 1 -- 172.21.15.12:0/4035364433 wait complete. 2024-05-17T22:39:59.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.773+0000 7fe49ed2bb80 20 client.25045 trim_cache size 0 max 16384 2024-05-17T22:39:59.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.802+0000 7fe49ed2bb80 1 Processor -- start 2024-05-17T22:39:59.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.803+0000 7fe49ed2bb80 1 -- start start 2024-05-17T22:39:59.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.803+0000 7fe49ed2bb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3ed2d39f0 0x55d3ed4515c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:59.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.803+0000 7fe49ed2bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3ed451b00 0x55d3ed4551b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:59.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.803+0000 7fe49ed2bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3ed457400 0x55d3ed45b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:59.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.803+0000 7fe49ed2bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe4881b8b60 con 0x55d3ed451b00 2024-05-17T22:39:59.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.803+0000 7fe49ed2bb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe4881b63f0 con 0x55d3ed2d39f0 2024-05-17T22:39:59.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.803+0000 7fe49ed2bb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe4881b4240 con 0x55d3ed457400 2024-05-17T22:39:59.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.804+0000 7fe48e0ec700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3ed2d39f0 0x55d3ed4515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:59.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.804+0000 7fe48e0ec700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3ed2d39f0 0x55d3ed4515c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:46390/0 (socket says 172.21.15.12:46390) 2024-05-17T22:39:59.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.804+0000 7fe48e0ec700 1 -- 172.21.15.12:0/653965917 learned_addr learned my addr 172.21.15.12:0/653965917 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:39:59.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.804+0000 7fe48d8eb700 1 --2- 172.21.15.12:0/653965917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3ed451b00 0x55d3ed4551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:59.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.805+0000 7fe48e0ec700 1 -- 172.21.15.12:0/653965917 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3ed457400 msgr2=0x55d3ed45b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:39:59.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.805+0000 7fe48e0ec700 1 --2- 172.21.15.12:0/653965917 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3ed457400 0x55d3ed45b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:59.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.805+0000 7fe48e0ec700 1 -- 172.21.15.12:0/653965917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3ed451b00 msgr2=0x55d3ed4551b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:39:59.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.805+0000 7fe48e0ec700 1 --2- 172.21.15.12:0/653965917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3ed451b00 0x55d3ed4551b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:59.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.805+0000 7fe48e0ec700 1 -- 172.21.15.12:0/653965917 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3ed45b950 con 0x55d3ed2d39f0 2024-05-17T22:39:59.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.805+0000 7fe48e0ec700 1 --2- 172.21.15.12:0/653965917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3ed2d39f0 0x55d3ed4515c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe4840a6020 tx=0x7fe48400b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65482253306c1520 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:39:59.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.805+0000 7fe473fff700 1 -- 172.21.15.12:0/653965917 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4840b7070 con 0x55d3ed2d39f0 2024-05-17T22:39:59.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.805+0000 7fe473fff700 1 -- 172.21.15.12:0/653965917 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4840b2d70 con 0x55d3ed2d39f0 2024-05-17T22:39:59.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.805+0000 7fe473fff700 1 -- 172.21.15.12:0/653965917 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4840a5910 con 0x55d3ed2d39f0 2024-05-17T22:39:59.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.806+0000 7fe49ed2bb80 1 -- 172.21.15.12:0/653965917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3ed2d39f0 msgr2=0x55d3ed4515c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:59.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.806+0000 7fe49ed2bb80 1 --2- 172.21.15.12:0/653965917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3ed2d39f0 0x55d3ed4515c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe4840a6020 tx=0x7fe48400b040 comp rx=0 tx=0).stop 2024-05-17T22:39:59.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.806+0000 7fe49ed2bb80 1 -- 172.21.15.12:0/653965917 shutdown_connections 2024-05-17T22:39:59.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.806+0000 7fe49ed2bb80 1 --2- 172.21.15.12:0/653965917 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3ed457400 0x55d3ed45b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:59.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.806+0000 7fe49ed2bb80 1 --2- 172.21.15.12:0/653965917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3ed2d39f0 0x55d3ed4515c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:59.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.806+0000 7fe49ed2bb80 1 --2- 172.21.15.12:0/653965917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3ed451b00 0x55d3ed4551b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:59.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.806+0000 7fe49ed2bb80 1 -- 172.21.15.12:0/653965917 >> 172.21.15.12:0/653965917 conn(0x55d3ed3337d0 msgr2=0x55d3ed459410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:39:59.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.807+0000 7fe49ed2bb80 1 -- 172.21.15.12:0/653965917 shutdown_connections 2024-05-17T22:39:59.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.807+0000 7fe49ed2bb80 1 -- 172.21.15.12:0/653965917 wait complete. 2024-05-17T22:39:59.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.807+0000 7fe49ed2bb80 1 -- start start 2024-05-17T22:39:59.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.807+0000 7fe49ed2bb80 1 Processor -- start 2024-05-17T22:39:59.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.808+0000 7fe49ed2bb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3ed2d39f0 0x55d3ed4633d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:59.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.808+0000 7fe49ed2bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3ed451b00 0x55d3ed45e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:59.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.808+0000 7fe49ed2bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3ed457400 0x55d3ed45ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:59.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.808+0000 7fe49ed2bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe4881b48b0 con 0x55d3ed457400 2024-05-17T22:39:59.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.808+0000 7fe49ed2bb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe4881b5d20 con 0x55d3ed2d39f0 2024-05-17T22:39:59.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.808+0000 7fe49ed2bb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe4881b4f80 con 0x55d3ed451b00 2024-05-17T22:39:59.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.808+0000 7fe48e0ec700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3ed2d39f0 0x55d3ed4633d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:59.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.808+0000 7fe49ed2bb80 10 client.0 _ll_register_callbacks cb 0x55d3ed452430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:39:59.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.808+0000 7fe48e0ec700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3ed2d39f0 0x55d3ed4633d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:46392/0 (socket says 172.21.15.12:46392) 2024-05-17T22:39:59.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.808+0000 7fe48e0ec700 1 -- 172.21.15.12:0/279426510 learned_addr learned my addr 172.21.15.12:0/279426510 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:39:59.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.808+0000 7fe48d8eb700 1 --2- 172.21.15.12:0/279426510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3ed451b00 0x55d3ed45e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:59.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.808+0000 7fe48e8ed700 1 --2- 172.21.15.12:0/279426510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3ed457400 0x55d3ed45ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:59.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.809+0000 7fe48d8eb700 1 -- 172.21.15.12:0/279426510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3ed2d39f0 msgr2=0x55d3ed4633d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:39:59.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.809+0000 7fe48d8eb700 1 --2- 172.21.15.12:0/279426510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3ed2d39f0 0x55d3ed4633d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:59.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.809+0000 7fe48d8eb700 1 -- 172.21.15.12:0/279426510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3ed457400 msgr2=0x55d3ed45ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:39:59.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.809+0000 7fe48d8eb700 1 --2- 172.21.15.12:0/279426510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3ed457400 0x55d3ed45ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:59.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.809+0000 7fe48d8eb700 1 -- 172.21.15.12:0/279426510 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3ed45f480 con 0x55d3ed451b00 2024-05-17T22:39:59.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.809+0000 7fe48d8eb700 1 --2- 172.21.15.12:0/279426510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3ed451b00 0x55d3ed45e6e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe478009f10 tx=0x7fe478028670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e798505be32a0e4a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:39:59.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.809+0000 7fe471ffb700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:39:59.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.810+0000 7fe471ffb700 1 -- 172.21.15.12:0/279426510 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47801fc20 con 0x55d3ed451b00 2024-05-17T22:39:59.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.810+0000 7fe49ed2bb80 1 -- 172.21.15.12:0/279426510 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4840b2d70 con 0x55d3ed451b00 2024-05-17T22:39:59.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.810+0000 7fe471ffb700 1 -- 172.21.15.12:0/279426510 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe478032470 con 0x55d3ed451b00 2024-05-17T22:39:59.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.810+0000 7fe471ffb700 1 -- 172.21.15.12:0/279426510 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe478044050 con 0x55d3ed451b00 2024-05-17T22:39:59.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.812+0000 7fe49ed2bb80 10 client.25048 fetch_fsmap learned FSMap version 29 2024-05-17T22:39:59.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.812+0000 7fe49ed2bb80 1 -- 172.21.15.12:0/279426510 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d3ed4e61b0 con 0x55d3ed451b00 2024-05-17T22:39:59.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.812+0000 7fe471ffb700 1 -- 172.21.15.12:0/279426510 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe478032b20 con 0x55d3ed451b00 2024-05-17T22:39:59.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.813+0000 7fe471ffb700 1 -- 172.21.15.12:0/279426510 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fe478026040 con 0x55d3ed451b00 2024-05-17T22:39:59.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.813+0000 7fe49ed2bb80 10 client.25048 fetch_fsmap finished waiting for FSMap version 29 2024-05-17T22:39:59.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.813+0000 7fe49ed2bb80 10 client.25048 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-17T22:39:59.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.813+0000 7fe49ed2bb80 20 client.25048 populate_metadata read hostname 'smithi012' 2024-05-17T22:39:59.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.813+0000 7fe49ed2bb80 1 --2- 172.21.15.12:0/279426510 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55d3ed4e68f0 0x55d3ed4e8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:39:59.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.813+0000 7fe49ed2bb80 4 client.25048 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:39:59.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.813+0000 7fe49ed2bb80 1 -- 172.21.15.12:0/279426510 --> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fe4840b2d70 con 0x55d3ed4e68f0 2024-05-17T22:39:59.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.813+0000 7fe48e0ec700 1 --2- 172.21.15.12:0/279426510 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55d3ed4e68f0 0x55d3ed4e8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:39:59.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.814+0000 7fe471ffb700 10 client.25048 ms_handle_connect on v2:172.21.15.151:6838/2338353849 2024-05-17T22:39:59.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.814+0000 7fe48e0ec700 1 --2- 172.21.15.12:0/279426510 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55d3ed4e68f0 0x55d3ed4e8d10 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fb044454be371330 server_cookie=337935e3679db65c in_seq=0 out_seq=0 2024-05-17T22:39:59.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.814+0000 7fe471ffb700 1 -- 172.21.15.12:0/279426510 <== mds.2 v2:172.21.15.151:6838/2338353849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fe4840b2d70 con 0x55d3ed4e68f0 2024-05-17T22:39:59.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.814+0000 7fe471ffb700 10 client.25048 handle_command_reply: tid=0 2024-05-17T22:39:59.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.814+0000 7fe49ed2bb80 1 client.25048 shutdown 2024-05-17T22:39:59.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.814+0000 7fe49ed2bb80 1 -- 172.21.15.12:0/279426510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3ed451b00 msgr2=0x55d3ed45e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:59.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.814+0000 7fe49ed2bb80 1 --2- 172.21.15.12:0/279426510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3ed451b00 0x55d3ed45e6e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe478009f10 tx=0x7fe478028670 comp rx=0 tx=0).stop 2024-05-17T22:39:59.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.815+0000 7fe49ed2bb80 1 -- 172.21.15.12:0/279426510 shutdown_connections 2024-05-17T22:39:59.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.815+0000 7fe49ed2bb80 1 --2- 172.21.15.12:0/279426510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3ed457400 0x55d3ed45ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:59.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.815+0000 7fe49ed2bb80 1 --2- 172.21.15.12:0/279426510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3ed2d39f0 0x55d3ed4633d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:59.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.815+0000 7fe49ed2bb80 1 --2- 172.21.15.12:0/279426510 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55d3ed4e68f0 0x55d3ed4e8d10 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:59.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.815+0000 7fe49ed2bb80 1 --2- 172.21.15.12:0/279426510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3ed451b00 0x55d3ed45e6e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:59.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.815+0000 7fe49ed2bb80 1 -- 172.21.15.12:0/279426510 >> 172.21.15.12:0/279426510 conn(0x55d3ed3337d0 msgr2=0x55d3ed454f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:39:59.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.815+0000 7fe471ffb700 0 client.25048 ms_handle_reset on v2:172.21.15.151:6838/2338353849 2024-05-17T22:39:59.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.815+0000 7fe49ed2bb80 1 -- 172.21.15.12:0/279426510 shutdown_connections 2024-05-17T22:39:59.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.815+0000 7fe49ed2bb80 1 -- 172.21.15.12:0/279426510 wait complete. 2024-05-17T22:39:59.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.815+0000 7fe49ed2bb80 20 client.25048 trim_cache size 0 max 16384 2024-05-17T22:39:59.819 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:39:59.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.816+0000 7fe490350700 1 -- 172.21.15.12:0/2945326811 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe4740b2a10 msgr2=0x7fe4740b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:59.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.816+0000 7fe490350700 1 --2- 172.21.15.12:0/2945326811 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe4740b2a10 0x7fe4740b4f20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe488106cb0 tx=0x7fe478009260 comp rx=0 tx=0).stop 2024-05-17T22:39:59.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.816+0000 7fe490350700 1 -- 172.21.15.12:0/2945326811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe488102e00 msgr2=0x7fe4881a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:39:59.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.816+0000 7fe490350700 1 --2- 172.21.15.12:0/2945326811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe488102e00 0x7fe4881a67f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe484005d10 tx=0x7fe48400f690 comp rx=0 tx=0).stop 2024-05-17T22:39:59.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.816+0000 7fe490350700 1 -- 172.21.15.12:0/2945326811 shutdown_connections 2024-05-17T22:39:59.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.816+0000 7fe490350700 1 --2- 172.21.15.12:0/2945326811 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe4740b2a10 0x7fe4740b4f20 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:59.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.816+0000 7fe490350700 1 --2- 172.21.15.12:0/2945326811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe488108080 0x7fe4881a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:59.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.816+0000 7fe490350700 1 --2- 172.21.15.12:0/2945326811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe488102e00 0x7fe4881a67f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:59.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.816+0000 7fe490350700 1 --2- 172.21.15.12:0/2945326811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe488107730 0x7fe4881a6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:39:59.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.816+0000 7fe490350700 1 -- 172.21.15.12:0/2945326811 >> 172.21.15.12:0/2945326811 conn(0x7fe488079bd0 msgr2=0x7fe48810dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:39:59.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.816+0000 7fe490350700 1 -- 172.21.15.12:0/2945326811 shutdown_connections 2024-05-17T22:39:59.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:39:59.816+0000 7fe490350700 1 -- 172.21.15.12:0/2945326811 wait complete. 2024-05-17T22:40:00.068 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:39:59 smithi012 ceph-mon[31632]: pgmap v461: 129 pgs: 129 active+clean; 981 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 25 MiB/s wr, 1.08k op/s 2024-05-17T22:40:00.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:39:59 smithi151 ceph-mon[36101]: pgmap v461: 129 pgs: 129 active+clean; 981 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 25 MiB/s wr, 1.08k op/s 2024-05-17T22:40:00.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:39:59 smithi096 ceph-mon[36065]: pgmap v461: 129 pgs: 129 active+clean; 981 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 25 MiB/s wr, 1.08k op/s 2024-05-17T22:40:00.667 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-17T22:40:01.094 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:00 smithi012 ceph-mon[31632]: overall HEALTH_OK 2024-05-17T22:40:01.120 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:40:01.200 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:00 smithi096 ceph-mon[36065]: overall HEALTH_OK 2024-05-17T22:40:01.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:00 smithi151 ceph-mon[36101]: overall HEALTH_OK 2024-05-17T22:40:01.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 40.0 s, 2620.0 tps, lat 38.217 ms stddev 41.795 2024-05-17T22:40:02.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:01 smithi151 ceph-mon[36101]: pgmap v462: 129 pgs: 129 active+clean; 982 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 782 op/s 2024-05-17T22:40:02.287 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:01 smithi012 ceph-mon[31632]: pgmap v462: 129 pgs: 129 active+clean; 982 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 782 op/s 2024-05-17T22:40:02.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:01 smithi096 ceph-mon[36065]: pgmap v462: 129 pgs: 129 active+clean; 982 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 782 op/s 2024-05-17T22:40:03.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.103+0000 7f5c76afc700 1 -- 172.21.15.12:0/2037256130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c7010a830 msgr2=0x7f5c7010ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:03.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.103+0000 7f5c76afc700 1 --2- 172.21.15.12:0/2037256130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c7010a830 0x7f5c7010ac40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f5c58009990 tx=0x7f5c58009ca0 comp rx=0 tx=0).stop 2024-05-17T22:40:03.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.104+0000 7f5c76afc700 1 -- 172.21.15.12:0/2037256130 shutdown_connections 2024-05-17T22:40:03.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.104+0000 7f5c76afc700 1 --2- 172.21.15.12:0/2037256130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c7011d550 0x7f5c7011f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:03.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.105+0000 7f5c76afc700 1 --2- 172.21.15.12:0/2037256130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c7010a830 0x7f5c7010ac40 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:03.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.105+0000 7f5c76afc700 1 --2- 172.21.15.12:0/2037256130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c7010b180 0x7f5c7011d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:03.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.105+0000 7f5c76afc700 1 -- 172.21.15.12:0/2037256130 >> 172.21.15.12:0/2037256130 conn(0x7f5c7006efc0 msgr2=0x7f5c7006f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:40:03.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.107+0000 7f5c76afc700 1 -- 172.21.15.12:0/2037256130 shutdown_connections 2024-05-17T22:40:03.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.108+0000 7f5c76afc700 1 -- 172.21.15.12:0/2037256130 wait complete. 2024-05-17T22:40:03.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.109+0000 7f5c76afc700 1 Processor -- start 2024-05-17T22:40:03.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.110+0000 7f5c76afc700 1 -- start start 2024-05-17T22:40:03.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.111+0000 7f5c76afc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c7010a830 0x7f5c701a9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:03.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.111+0000 7f5c76afc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c7010b180 0x7f5c701a9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:03.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.111+0000 7f5c76afc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c7011d550 0x7f5c701ade70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:03.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.111+0000 7f5c76afc700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5c70122c40 con 0x7f5c7011d550 2024-05-17T22:40:03.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.111+0000 7f5c76afc700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5c70122ac0 con 0x7f5c7010b180 2024-05-17T22:40:03.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.111+0000 7f5c76afc700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5c70122dc0 con 0x7f5c7010a830 2024-05-17T22:40:03.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.114+0000 7f5c6ffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c7010b180 0x7f5c701a9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:03.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.114+0000 7f5c6ffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c7010b180 0x7f5c701a9be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:46404/0 (socket says 172.21.15.12:46404) 2024-05-17T22:40:03.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.114+0000 7f5c6ffff700 1 -- 172.21.15.12:0/506034200 learned_addr learned my addr 172.21.15.12:0/506034200 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:40:03.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.116+0000 7f5c75099700 1 --2- 172.21.15.12:0/506034200 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c7011d550 0x7f5c701ade70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:03.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.116+0000 7f5c6ffff700 1 -- 172.21.15.12:0/506034200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c7010a830 msgr2=0x7f5c701a9640 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-17T22:40:03.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.116+0000 7f5c6ffff700 1 --2- 172.21.15.12:0/506034200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c7010a830 0x7f5c701a9640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:03.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.116+0000 7f5c6ffff700 1 -- 172.21.15.12:0/506034200 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c7011d550 msgr2=0x7f5c701ade70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:03.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.116+0000 7f5c6ffff700 1 --2- 172.21.15.12:0/506034200 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c7011d550 0x7f5c701ade70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:03.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.116+0000 7f5c6ffff700 1 -- 172.21.15.12:0/506034200 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c701ae510 con 0x7f5c7010b180 2024-05-17T22:40:03.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.118+0000 7f5c6ffff700 1 --2- 172.21.15.12:0/506034200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c7010b180 0x7f5c701a9be0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5c6000ef10 tx=0x7f5c6000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:40:03.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.122+0000 7f5c6dffb700 1 -- 172.21.15.12:0/506034200 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6000cad0 con 0x7f5c7010b180 2024-05-17T22:40:03.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.122+0000 7f5c6dffb700 1 -- 172.21.15.12:0/506034200 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c60019070 con 0x7f5c7010b180 2024-05-17T22:40:03.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.122+0000 7f5c6dffb700 1 -- 172.21.15.12:0/506034200 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c600146c0 con 0x7f5c7010b180 2024-05-17T22:40:03.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.122+0000 7f5c76afc700 1 -- 172.21.15.12:0/506034200 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c701ae7a0 con 0x7f5c7010b180 2024-05-17T22:40:03.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.122+0000 7f5c76afc700 1 -- 172.21.15.12:0/506034200 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c701aec90 con 0x7f5c7010b180 2024-05-17T22:40:03.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.129+0000 7f5c854d7b80 1 Processor -- start 2024-05-17T22:40:03.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.131+0000 7f5c854d7b80 1 -- start start 2024-05-17T22:40:03.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.131+0000 7f5c854d7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562a064fe9f0 0x562a06682560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:03.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.131+0000 7f5c854d7b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a06682aa0 0x562a0667cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:03.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.131+0000 7f5c854d7b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562a0667d170 0x562a0667d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:03.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.131+0000 7f5c854d7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x562a065ab290 con 0x562a064fe9f0 2024-05-17T22:40:03.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.131+0000 7f5c854d7b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562a0659afd0 con 0x562a06682aa0 2024-05-17T22:40:03.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.131+0000 7f5c854d7b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562a0659ee70 con 0x562a0667d170 2024-05-17T22:40:03.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.133+0000 7f5c74898700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a06682aa0 0x562a0667cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:03.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.133+0000 7f5c74898700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a06682aa0 0x562a0667cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:46416/0 (socket says 172.21.15.12:46416) 2024-05-17T22:40:03.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.133+0000 7f5c74898700 1 -- 172.21.15.12:0/1305619583 learned_addr learned my addr 172.21.15.12:0/1305619583 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:40:03.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.133+0000 7f5c75099700 1 --2- 172.21.15.12:0/1305619583 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562a064fe9f0 0x562a06682560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:03.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.133+0000 7f5c74898700 1 -- 172.21.15.12:0/1305619583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562a0667d170 msgr2=0x562a0667d580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T22:40:03.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.133+0000 7f5c74898700 1 --2- 172.21.15.12:0/1305619583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562a0667d170 0x562a0667d580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:03.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.133+0000 7f5c74898700 1 -- 172.21.15.12:0/1305619583 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562a064fe9f0 msgr2=0x562a06682560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:40:03.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.133+0000 7f5c74898700 1 --2- 172.21.15.12:0/1305619583 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562a064fe9f0 0x562a06682560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:03.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.133+0000 7f5c74898700 1 -- 172.21.15.12:0/1305619583 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a06544590 con 0x562a06682aa0 2024-05-17T22:40:03.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.134+0000 7f5c74898700 1 --2- 172.21.15.12:0/1305619583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a06682aa0 0x562a0667cc30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5c58005910 tx=0x7f5c58005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7aea8a758cdf50e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:40:03.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.134+0000 7f5c6dffb700 1 -- 172.21.15.12:0/506034200 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5c60012050 con 0x7f5c7010b180 2024-05-17T22:40:03.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.134+0000 7f5c66ffd700 1 -- 172.21.15.12:0/1305619583 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c58009510 con 0x562a06682aa0 2024-05-17T22:40:03.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.134+0000 7f5c66ffd700 1 -- 172.21.15.12:0/1305619583 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c58004830 con 0x562a06682aa0 2024-05-17T22:40:03.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.134+0000 7f5c66ffd700 1 -- 172.21.15.12:0/1305619583 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c58005380 con 0x562a06682aa0 2024-05-17T22:40:03.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.135+0000 7f5c6dffb700 1 --2- 172.21.15.12:0/506034200 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5c5c0b29d0 0x7f5c5c0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:03.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.136+0000 7f5c6dffb700 1 -- 172.21.15.12:0/506034200 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f5c600e24e0 con 0x7f5c7010b180 2024-05-17T22:40:03.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.136+0000 7f5c854d7b80 1 -- 172.21.15.12:0/1305619583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a06682aa0 msgr2=0x562a0667cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:03.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.136+0000 7f5c854d7b80 1 --2- 172.21.15.12:0/1305619583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a06682aa0 0x562a0667cc30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5c58005910 tx=0x7f5c58005e80 comp rx=0 tx=0).stop 2024-05-17T22:40:03.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.136+0000 7f5c854d7b80 1 -- 172.21.15.12:0/1305619583 shutdown_connections 2024-05-17T22:40:03.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.136+0000 7f5c854d7b80 1 --2- 172.21.15.12:0/1305619583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562a0667d170 0x562a0667d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:03.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.136+0000 7f5c854d7b80 1 --2- 172.21.15.12:0/1305619583 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562a064fe9f0 0x562a06682560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:03.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.136+0000 7f5c854d7b80 1 --2- 172.21.15.12:0/1305619583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a06682aa0 0x562a0667cc30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:03.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.136+0000 7f5c854d7b80 1 -- 172.21.15.12:0/1305619583 >> 172.21.15.12:0/1305619583 conn(0x562a06542f00 msgr2=0x562a06677b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:40:03.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.136+0000 7f5c75099700 1 --2- 172.21.15.12:0/506034200 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5c5c0b29d0 0x7f5c5c0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:03.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.137+0000 7f5c854d7b80 1 -- 172.21.15.12:0/1305619583 shutdown_connections 2024-05-17T22:40:03.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.137+0000 7f5c75099700 1 --2- 172.21.15.12:0/506034200 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5c5c0b29d0 0x7f5c5c0b4ee0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x562a06678ae0 tx=0x7f5c68009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:40:03.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.137+0000 7f5c854d7b80 1 -- 172.21.15.12:0/1305619583 wait complete. 2024-05-17T22:40:03.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.138+0000 7f5c854d7b80 1 -- start start 2024-05-17T22:40:03.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.138+0000 7f5c854d7b80 1 Processor -- start 2024-05-17T22:40:03.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.139+0000 7f5c854d7b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a064fe9f0 0x562a06686200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:03.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.139+0000 7f5c854d7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562a0667d170 0x562a06686740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:03.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.139+0000 7f5c854d7b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562a06682aa0 0x562a06686c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:03.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.139+0000 7f5c854d7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x562a064a2150 con 0x562a0667d170 2024-05-17T22:40:03.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.139+0000 7f5c854d7b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562a0653be80 con 0x562a064fe9f0 2024-05-17T22:40:03.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.139+0000 7f5c854d7b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562a06689330 con 0x562a06682aa0 2024-05-17T22:40:03.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.139+0000 7f5c854d7b80 10 client.0 _ll_register_callbacks cb 0x562a06535f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:40:03.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.139+0000 7f5c74898700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a064fe9f0 0x562a06686200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:03.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.139+0000 7f5c75099700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562a06682aa0 0x562a06686c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:03.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.139+0000 7f5c74898700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a064fe9f0 0x562a06686200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:46426/0 (socket says 172.21.15.12:46426) 2024-05-17T22:40:03.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.139+0000 7f5c74898700 1 -- 172.21.15.12:0/2187139089 learned_addr learned my addr 172.21.15.12:0/2187139089 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:40:03.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.140+0000 7f5c6ffff700 1 --2- 172.21.15.12:0/2187139089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562a0667d170 0x562a06686740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:03.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.140+0000 7f5c74898700 1 -- 172.21.15.12:0/2187139089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562a06682aa0 msgr2=0x562a06686c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:40:03.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.140+0000 7f5c74898700 1 --2- 172.21.15.12:0/2187139089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562a06682aa0 0x562a06686c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:03.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.140+0000 7f5c74898700 1 -- 172.21.15.12:0/2187139089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562a0667d170 msgr2=0x562a06686740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:40:03.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.140+0000 7f5c74898700 1 --2- 172.21.15.12:0/2187139089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562a0667d170 0x562a06686740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:03.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.140+0000 7f5c74898700 1 -- 172.21.15.12:0/2187139089 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a064eb480 con 0x562a064fe9f0 2024-05-17T22:40:03.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.140+0000 7f5c74898700 1 --2- 172.21.15.12:0/2187139089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a064fe9f0 0x562a06686200 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5c580058e0 tx=0x7f5c58004c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40dc48d778f45c46 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:40:03.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.140+0000 7f5c4ffff700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:40:03.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.140+0000 7f5c4ffff700 1 -- 172.21.15.12:0/2187139089 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5802f020 con 0x562a064fe9f0 2024-05-17T22:40:03.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.141+0000 7f5c854d7b80 1 -- 172.21.15.12:0/2187139089 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c58004830 con 0x562a064fe9f0 2024-05-17T22:40:03.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.141+0000 7f5c4ffff700 1 -- 172.21.15.12:0/2187139089 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f5c58005230 con 0x562a064fe9f0 2024-05-17T22:40:03.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.141+0000 7f5c4ffff700 1 -- 172.21.15.12:0/2187139089 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5c5803a050 con 0x562a064fe9f0 2024-05-17T22:40:03.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.141+0000 7f5c854d7b80 10 client.25103 fetch_fsmap learned FSMap version 29 2024-05-17T22:40:03.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.141+0000 7f5c854d7b80 1 -- 172.21.15.12:0/2187139089 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562a06693dd0 con 0x562a064fe9f0 2024-05-17T22:40:03.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.141+0000 7f5c4ffff700 1 -- 172.21.15.12:0/2187139089 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c58004830 con 0x562a064fe9f0 2024-05-17T22:40:03.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.141+0000 7f5c4ffff700 1 -- 172.21.15.12:0/2187139089 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f5c58021a40 con 0x562a064fe9f0 2024-05-17T22:40:03.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.142+0000 7f5c854d7b80 10 client.25103 fetch_fsmap finished waiting for FSMap version 29 2024-05-17T22:40:03.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.142+0000 7f5c854d7b80 10 client.25103 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-17T22:40:03.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.142+0000 7f5c854d7b80 20 client.25103 populate_metadata read hostname 'smithi012' 2024-05-17T22:40:03.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.142+0000 7f5c854d7b80 1 --2- 172.21.15.12:0/2187139089 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x562a066962a0 0x562a066986c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:03.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.142+0000 7f5c854d7b80 4 client.25103 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:40:03.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.142+0000 7f5c854d7b80 1 -- 172.21.15.12:0/2187139089 --> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a065ad1e0 con 0x562a066962a0 2024-05-17T22:40:03.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.142+0000 7f5c6ffff700 1 --2- 172.21.15.12:0/2187139089 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x562a066962a0 0x562a066986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:03.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.143+0000 7f5c6ffff700 1 --2- 172.21.15.12:0/2187139089 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x562a066962a0 0x562a066986c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=286c77bcb0a0bec0 server_cookie=e9bbe9b238eb6557 in_seq=0 out_seq=0 2024-05-17T22:40:03.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.143+0000 7f5c4ffff700 10 client.25103 ms_handle_connect on v2:172.21.15.96:6836/4222574338 2024-05-17T22:40:03.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.144+0000 7f5c4ffff700 1 -- 172.21.15.12:0/2187139089 <== mds.1 v2:172.21.15.96:6836/4222574338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562a065ad1e0 con 0x562a066962a0 2024-05-17T22:40:03.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.144+0000 7f5c4ffff700 10 client.25103 handle_command_reply: tid=0 2024-05-17T22:40:03.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.144+0000 7f5c854d7b80 1 client.25103 shutdown 2024-05-17T22:40:03.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.144+0000 7f5c854d7b80 1 -- 172.21.15.12:0/2187139089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a064fe9f0 msgr2=0x562a06686200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:03.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.144+0000 7f5c854d7b80 1 --2- 172.21.15.12:0/2187139089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a064fe9f0 0x562a06686200 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5c580058e0 tx=0x7f5c58004c90 comp rx=0 tx=0).stop 2024-05-17T22:40:03.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.144+0000 7f5c854d7b80 1 -- 172.21.15.12:0/2187139089 shutdown_connections 2024-05-17T22:40:03.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.144+0000 7f5c854d7b80 1 --2- 172.21.15.12:0/2187139089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562a06682aa0 0x562a06686c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:03.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.144+0000 7f5c854d7b80 1 --2- 172.21.15.12:0/2187139089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a064fe9f0 0x562a06686200 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:03.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.144+0000 7f5c854d7b80 1 --2- 172.21.15.12:0/2187139089 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x562a066962a0 0x562a066986c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:03.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.144+0000 7f5c854d7b80 1 --2- 172.21.15.12:0/2187139089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562a0667d170 0x562a06686740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:03.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.144+0000 7f5c854d7b80 1 -- 172.21.15.12:0/2187139089 >> 172.21.15.12:0/2187139089 conn(0x562a06542f00 msgr2=0x562a06671ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:40:03.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.145+0000 7f5c4ffff700 0 client.25103 ms_handle_reset on v2:172.21.15.96:6836/4222574338 2024-05-17T22:40:03.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.147+0000 7f5c854d7b80 1 -- 172.21.15.12:0/2187139089 shutdown_connections 2024-05-17T22:40:03.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.147+0000 7f5c854d7b80 1 -- 172.21.15.12:0/2187139089 wait complete. 2024-05-17T22:40:03.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.148+0000 7f5c854d7b80 20 client.25103 trim_cache size 0 max 16384 2024-05-17T22:40:03.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.173+0000 7f5c854d7b80 1 Processor -- start 2024-05-17T22:40:03.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.173+0000 7f5c854d7b80 1 -- start start 2024-05-17T22:40:03.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.173+0000 7f5c854d7b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a064fe9f0 0x562a0667cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:03.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.174+0000 7f5c854d7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562a0667d170 0x562a06681850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:03.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.174+0000 7f5c74898700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a064fe9f0 0x562a0667cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:03.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.174+0000 7f5c74898700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a064fe9f0 0x562a0667cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:46434/0 (socket says 172.21.15.12:46434) 2024-05-17T22:40:03.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.174+0000 7f5c74898700 1 -- 172.21.15.12:0/4098627092 learned_addr learned my addr 172.21.15.12:0/4098627092 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:40:03.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.174+0000 7f5c854d7b80 1 --2- 172.21.15.12:0/4098627092 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562a06682aa0 0x562a06686ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:03.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.175+0000 7f5c854d7b80 1 -- 172.21.15.12:0/4098627092 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5c701d2530 con 0x562a0667d170 2024-05-17T22:40:03.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.175+0000 7f5c6ffff700 1 --2- 172.21.15.12:0/4098627092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562a0667d170 0x562a06681850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:03.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.175+0000 7f5c854d7b80 1 -- 172.21.15.12:0/4098627092 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5c70084720 con 0x562a064fe9f0 2024-05-17T22:40:03.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.175+0000 7f5c854d7b80 1 -- 172.21.15.12:0/4098627092 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5c70082570 con 0x562a06682aa0 2024-05-17T22:40:03.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.175+0000 7f5c75099700 1 --2- 172.21.15.12:0/4098627092 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562a06682aa0 0x562a06686ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:03.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.176+0000 7f5c74898700 1 -- 172.21.15.12:0/4098627092 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562a06682aa0 msgr2=0x562a06686ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:40:03.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.176+0000 7f5c74898700 1 --2- 172.21.15.12:0/4098627092 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562a06682aa0 0x562a06686ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:03.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.176+0000 7f5c74898700 1 -- 172.21.15.12:0/4098627092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562a0667d170 msgr2=0x562a06681850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:40:03.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.176+0000 7f5c74898700 1 --2- 172.21.15.12:0/4098627092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562a0667d170 0x562a06681850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:03.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.176+0000 7f5c74898700 1 -- 172.21.15.12:0/4098627092 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a06684750 con 0x562a064fe9f0 2024-05-17T22:40:03.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.176+0000 7f5c74898700 1 --2- 172.21.15.12:0/4098627092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a064fe9f0 0x562a0667cc30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5c580098e0 tx=0x7f5c58004bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d544f863f10e6428 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:40:03.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.176+0000 7f5c667fc700 1 -- 172.21.15.12:0/4098627092 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5802f040 con 0x562a064fe9f0 2024-05-17T22:40:03.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.176+0000 7f5c667fc700 1 -- 172.21.15.12:0/4098627092 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c58004d80 con 0x562a064fe9f0 2024-05-17T22:40:03.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.176+0000 7f5c667fc700 1 -- 172.21.15.12:0/4098627092 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5802abc0 con 0x562a064fe9f0 2024-05-17T22:40:03.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.177+0000 7f5c854d7b80 1 -- 172.21.15.12:0/4098627092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a064fe9f0 msgr2=0x562a0667cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:03.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.177+0000 7f5c854d7b80 1 --2- 172.21.15.12:0/4098627092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a064fe9f0 0x562a0667cc30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5c580098e0 tx=0x7f5c58004bf0 comp rx=0 tx=0).stop 2024-05-17T22:40:03.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.178+0000 7f5c854d7b80 1 -- 172.21.15.12:0/4098627092 shutdown_connections 2024-05-17T22:40:03.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.178+0000 7f5c854d7b80 1 --2- 172.21.15.12:0/4098627092 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562a06682aa0 0x562a06686ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:03.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.178+0000 7f5c854d7b80 1 --2- 172.21.15.12:0/4098627092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a064fe9f0 0x562a0667cc30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:03.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.178+0000 7f5c854d7b80 1 --2- 172.21.15.12:0/4098627092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562a0667d170 0x562a06681850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:03.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.178+0000 7f5c854d7b80 1 -- 172.21.15.12:0/4098627092 >> 172.21.15.12:0/4098627092 conn(0x562a06542f00 msgr2=0x562a066884b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:40:03.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.178+0000 7f5c854d7b80 1 -- 172.21.15.12:0/4098627092 shutdown_connections 2024-05-17T22:40:03.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.178+0000 7f5c854d7b80 1 -- 172.21.15.12:0/4098627092 wait complete. 2024-05-17T22:40:03.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.179+0000 7f5c854d7b80 1 -- start start 2024-05-17T22:40:03.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.179+0000 7f5c854d7b80 1 Processor -- start 2024-05-17T22:40:03.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.180+0000 7f5c854d7b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562a064fe9f0 0x562a06698020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:03.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.180+0000 7f5c854d7b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a0667d170 0x562a06698560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:03.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.180+0000 7f5c854d7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562a06682aa0 0x562a06698aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:03.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.180+0000 7f5c74898700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562a064fe9f0 0x562a06698020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:03.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.180+0000 7f5c75099700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562a06682aa0 0x562a06698aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:03.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.180+0000 7f5c75099700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562a06682aa0 0x562a06698aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51166/0 (socket says 172.21.15.12:51166) 2024-05-17T22:40:03.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.180+0000 7f5c74898700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562a064fe9f0 0x562a06698020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:45250/0 (socket says 172.21.15.12:45250) 2024-05-17T22:40:03.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.180+0000 7f5c75099700 1 -- 172.21.15.12:0/2402421821 learned_addr learned my addr 172.21.15.12:0/2402421821 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:40:03.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.180+0000 7f5c6ffff700 1 --2- 172.21.15.12:0/2402421821 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a0667d170 0x562a06698560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:03.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.180+0000 7f5c854d7b80 1 -- 172.21.15.12:0/2402421821 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5c70082be0 con 0x562a06682aa0 2024-05-17T22:40:03.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.180+0000 7f5c854d7b80 1 -- 172.21.15.12:0/2402421821 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5c70084050 con 0x562a0667d170 2024-05-17T22:40:03.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.180+0000 7f5c854d7b80 1 -- 172.21.15.12:0/2402421821 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5c700832b0 con 0x562a064fe9f0 2024-05-17T22:40:03.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.180+0000 7f5c854d7b80 10 client.0 _ll_register_callbacks cb 0x562a06535f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:40:03.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.180+0000 7f5c74898700 1 -- 172.21.15.12:0/2402421821 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a0667d170 msgr2=0x562a06698560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:40:03.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.180+0000 7f5c74898700 1 --2- 172.21.15.12:0/2402421821 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a0667d170 0x562a06698560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:03.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.180+0000 7f5c74898700 1 -- 172.21.15.12:0/2402421821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562a06682aa0 msgr2=0x562a06698aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:40:03.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.180+0000 7f5c74898700 1 --2- 172.21.15.12:0/2402421821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562a06682aa0 0x562a06698aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:03.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.180+0000 7f5c74898700 1 -- 172.21.15.12:0/2402421821 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a066fe940 con 0x562a064fe9f0 2024-05-17T22:40:03.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.182+0000 7f5c74898700 1 --2- 172.21.15.12:0/2402421821 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562a064fe9f0 0x562a06698020 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5c5800b2e0 tx=0x7f5c58005460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53df1e1d6ae929a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:40:03.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.182+0000 7f5c4ffff700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:40:03.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.182+0000 7f5c4ffff700 1 -- 172.21.15.12:0/2402421821 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5802f070 con 0x562a064fe9f0 2024-05-17T22:40:03.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.182+0000 7f5c854d7b80 1 -- 172.21.15.12:0/2402421821 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c58004d80 con 0x562a064fe9f0 2024-05-17T22:40:03.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.182+0000 7f5c4ffff700 1 -- 172.21.15.12:0/2402421821 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f5c5802ab20 con 0x562a064fe9f0 2024-05-17T22:40:03.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.184+0000 7f5c4ffff700 1 -- 172.21.15.12:0/2402421821 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5c5802ad20 con 0x562a064fe9f0 2024-05-17T22:40:03.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.185+0000 7f5c854d7b80 10 client.25060 fetch_fsmap learned FSMap version 29 2024-05-17T22:40:03.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.185+0000 7f5c854d7b80 1 -- 172.21.15.12:0/2402421821 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562a066ffb60 con 0x562a064fe9f0 2024-05-17T22:40:03.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.185+0000 7f5c4ffff700 1 -- 172.21.15.12:0/2402421821 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c58004d80 con 0x562a064fe9f0 2024-05-17T22:40:03.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.185+0000 7f5c4ffff700 1 -- 172.21.15.12:0/2402421821 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f5c5803d070 con 0x562a064fe9f0 2024-05-17T22:40:03.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.186+0000 7f5c854d7b80 10 client.25060 fetch_fsmap finished waiting for FSMap version 29 2024-05-17T22:40:03.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.186+0000 7f5c854d7b80 10 client.25060 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-17T22:40:03.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.186+0000 7f5c854d7b80 20 client.25060 populate_metadata read hostname 'smithi012' 2024-05-17T22:40:03.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.186+0000 7f5c854d7b80 1 --2- 172.21.15.12:0/2402421821 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x562a066fff70 0x562a06702390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:03.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.186+0000 7f5c854d7b80 4 client.25060 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:40:03.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.186+0000 7f5c854d7b80 1 -- 172.21.15.12:0/2402421821 --> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562a065a69c0 con 0x562a066fff70 2024-05-17T22:40:03.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.186+0000 7f5c6ffff700 1 --2- 172.21.15.12:0/2402421821 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x562a066fff70 0x562a06702390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:03.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.186+0000 7f5c4ffff700 10 client.25060 ms_handle_connect on v2:172.21.15.96:6836/4222574338 2024-05-17T22:40:03.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.186+0000 7f5c6ffff700 1 --2- 172.21.15.12:0/2402421821 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x562a066fff70 0x562a06702390 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2bb54e62e98601c0 server_cookie=74193c1bf90f736d in_seq=0 out_seq=0 2024-05-17T22:40:03.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.187+0000 7f5c4ffff700 1 -- 172.21.15.12:0/2402421821 <== mds.1 v2:172.21.15.96:6836/4222574338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562a065a69c0 con 0x562a066fff70 2024-05-17T22:40:03.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.187+0000 7f5c4ffff700 10 client.25060 handle_command_reply: tid=0 2024-05-17T22:40:03.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.187+0000 7f5c854d7b80 1 client.25060 shutdown 2024-05-17T22:40:03.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.187+0000 7f5c854d7b80 1 -- 172.21.15.12:0/2402421821 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562a064fe9f0 msgr2=0x562a06698020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:03.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.187+0000 7f5c854d7b80 1 --2- 172.21.15.12:0/2402421821 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562a064fe9f0 0x562a06698020 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5c5800b2e0 tx=0x7f5c58005460 comp rx=0 tx=0).stop 2024-05-17T22:40:03.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.188+0000 7f5c854d7b80 1 -- 172.21.15.12:0/2402421821 shutdown_connections 2024-05-17T22:40:03.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.188+0000 7f5c854d7b80 1 --2- 172.21.15.12:0/2402421821 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x562a066fff70 0x562a06702390 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:03.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.188+0000 7f5c854d7b80 1 --2- 172.21.15.12:0/2402421821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562a06682aa0 0x562a06698aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:03.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.188+0000 7f5c854d7b80 1 --2- 172.21.15.12:0/2402421821 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562a064fe9f0 0x562a06698020 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:03.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.188+0000 7f5c854d7b80 1 --2- 172.21.15.12:0/2402421821 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a0667d170 0x562a06698560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:03.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.188+0000 7f5c854d7b80 1 -- 172.21.15.12:0/2402421821 >> 172.21.15.12:0/2402421821 conn(0x562a06542f00 msgr2=0x562a066803e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:40:03.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.188+0000 7f5c4ffff700 0 client.25060 ms_handle_reset on v2:172.21.15.96:6836/4222574338 2024-05-17T22:40:03.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.190+0000 7f5c854d7b80 1 -- 172.21.15.12:0/2402421821 shutdown_connections 2024-05-17T22:40:03.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.190+0000 7f5c854d7b80 1 -- 172.21.15.12:0/2402421821 wait complete. 2024-05-17T22:40:03.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.190+0000 7f5c854d7b80 20 client.25060 trim_cache size 0 max 16384 2024-05-17T22:40:03.194 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:40:03.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.192+0000 7f5c76afc700 1 -- 172.21.15.12:0/506034200 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5c5c0b29d0 msgr2=0x7f5c5c0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:03.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.192+0000 7f5c76afc700 1 --2- 172.21.15.12:0/506034200 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5c5c0b29d0 0x7f5c5c0b4ee0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x562a06678ae0 tx=0x7f5c68009260 comp rx=0 tx=0).stop 2024-05-17T22:40:03.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.192+0000 7f5c76afc700 1 -- 172.21.15.12:0/506034200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c7010b180 msgr2=0x7f5c701a9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:03.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.192+0000 7f5c76afc700 1 --2- 172.21.15.12:0/506034200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c7010b180 0x7f5c701a9be0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5c6000ef10 tx=0x7f5c6000c3a0 comp rx=0 tx=0).stop 2024-05-17T22:40:03.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.192+0000 7f5c76afc700 1 -- 172.21.15.12:0/506034200 shutdown_connections 2024-05-17T22:40:03.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.192+0000 7f5c76afc700 1 --2- 172.21.15.12:0/506034200 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5c5c0b29d0 0x7f5c5c0b4ee0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:03.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.192+0000 7f5c76afc700 1 --2- 172.21.15.12:0/506034200 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c7011d550 0x7f5c701ade70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:03.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.192+0000 7f5c76afc700 1 --2- 172.21.15.12:0/506034200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c7010a830 0x7f5c701a9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:03.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.192+0000 7f5c76afc700 1 --2- 172.21.15.12:0/506034200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c7010b180 0x7f5c701a9be0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:03.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.192+0000 7f5c76afc700 1 -- 172.21.15.12:0/506034200 >> 172.21.15.12:0/506034200 conn(0x7f5c7006efc0 msgr2=0x7f5c7011b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:40:03.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.192+0000 7f5c76afc700 1 -- 172.21.15.12:0/506034200 shutdown_connections 2024-05-17T22:40:03.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:03.192+0000 7f5c76afc700 1 -- 172.21.15.12:0/506034200 wait complete. 2024-05-17T22:40:03.977 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-17T22:40:04.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:03 smithi151 ceph-mon[36101]: pgmap v463: 129 pgs: 129 active+clean; 982 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 825 op/s 2024-05-17T22:40:04.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:03 smithi096 ceph-mon[36065]: pgmap v463: 129 pgs: 129 active+clean; 982 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 825 op/s 2024-05-17T22:40:04.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:03 smithi012 ceph-mon[31632]: pgmap v463: 129 pgs: 129 active+clean; 982 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 825 op/s 2024-05-17T22:40:04.414 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:40:06.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:05 smithi151 ceph-mon[36101]: pgmap v464: 129 pgs: 129 active+clean; 983 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 11 MiB/s wr, 859 op/s 2024-05-17T22:40:06.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:05 smithi096 ceph-mon[36065]: pgmap v464: 129 pgs: 129 active+clean; 983 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 11 MiB/s wr, 859 op/s 2024-05-17T22:40:06.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:05 smithi012 ceph-mon[31632]: pgmap v464: 129 pgs: 129 active+clean; 983 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 11 MiB/s wr, 859 op/s 2024-05-17T22:40:06.370 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 45.0 s, 2391.0 tps, lat 41.677 ms stddev 47.436 2024-05-17T22:40:08.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:07 smithi151 ceph-mon[36101]: pgmap v465: 129 pgs: 129 active+clean; 983 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 8.5 MiB/s wr, 697 op/s 2024-05-17T22:40:08.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:07 smithi096 ceph-mon[36065]: pgmap v465: 129 pgs: 129 active+clean; 983 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 8.5 MiB/s wr, 697 op/s 2024-05-17T22:40:08.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:07 smithi012 ceph-mon[31632]: pgmap v465: 129 pgs: 129 active+clean; 983 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 8.5 MiB/s wr, 697 op/s 2024-05-17T22:40:09.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.698+0000 7f4558ce7700 1 -- 172.21.15.12:0/2675208517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f455410f7c0 msgr2=0x7f4554117120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:09.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.698+0000 7f4558ce7700 1 --2- 172.21.15.12:0/2675208517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f455410f7c0 0x7f4554117120 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f454c009f90 tx=0x7f454c00b330 comp rx=0 tx=0).stop 2024-05-17T22:40:09.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.699+0000 7f4558ce7700 1 -- 172.21.15.12:0/2675208517 shutdown_connections 2024-05-17T22:40:09.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.699+0000 7f4558ce7700 1 --2- 172.21.15.12:0/2675208517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f455410f7c0 0x7f4554117120 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:09.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.699+0000 7f4558ce7700 1 --2- 172.21.15.12:0/2675208517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f455410a540 0x7f455410a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:09.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.699+0000 7f4558ce7700 1 --2- 172.21.15.12:0/2675208517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f455410ee70 0x7f455410f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:09.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.699+0000 7f4558ce7700 1 -- 172.21.15.12:0/2675208517 >> 172.21.15.12:0/2675208517 conn(0x7f4554079c00 msgr2=0x7f4554108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:40:09.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.701+0000 7f4558ce7700 1 -- 172.21.15.12:0/2675208517 shutdown_connections 2024-05-17T22:40:09.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.702+0000 7f4558ce7700 1 -- 172.21.15.12:0/2675208517 wait complete. 2024-05-17T22:40:09.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.703+0000 7f4558ce7700 1 Processor -- start 2024-05-17T22:40:09.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.704+0000 7f4558ce7700 1 -- start start 2024-05-17T22:40:09.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.704+0000 7f4558ce7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f455410a540 0x7f4554084260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:09.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.704+0000 7f4558ce7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f455410ee70 0x7f455407d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:09.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.704+0000 7f4558ce7700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f455410f7c0 0x7f455407d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:09.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.704+0000 7f4558ce7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f455411a4c0 con 0x7f455410a540 2024-05-17T22:40:09.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.704+0000 7f4558ce7700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f455411a340 con 0x7f455410ee70 2024-05-17T22:40:09.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.704+0000 7f4558ce7700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f455411a640 con 0x7f455410f7c0 2024-05-17T22:40:09.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.704+0000 7f455259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f455410a540 0x7f4554084260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:09.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.704+0000 7f455259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f455410a540 0x7f4554084260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59944/0 (socket says 172.21.15.12:59944) 2024-05-17T22:40:09.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.704+0000 7f455259c700 1 -- 172.21.15.12:0/295160400 learned_addr learned my addr 172.21.15.12:0/295160400 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:40:09.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.704+0000 7f454bfff700 1 --2- 172.21.15.12:0/295160400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f455410ee70 0x7f455407d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:09.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.705+0000 7f4552d9d700 1 --2- 172.21.15.12:0/295160400 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f455410f7c0 0x7f455407d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:09.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.705+0000 7f454bfff700 1 -- 172.21.15.12:0/295160400 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f455410f7c0 msgr2=0x7f455407d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:09.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.705+0000 7f454bfff700 1 --2- 172.21.15.12:0/295160400 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f455410f7c0 0x7f455407d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:09.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.705+0000 7f454bfff700 1 -- 172.21.15.12:0/295160400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f455410a540 msgr2=0x7f4554084260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:09.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.705+0000 7f454bfff700 1 --2- 172.21.15.12:0/295160400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f455410a540 0x7f4554084260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:09.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.705+0000 7f454bfff700 1 -- 172.21.15.12:0/295160400 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4554081d90 con 0x7f455410ee70 2024-05-17T22:40:09.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.706+0000 7f455259c700 1 --2- 172.21.15.12:0/295160400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f455410a540 0x7f4554084260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:40:09.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.706+0000 7f454bfff700 1 --2- 172.21.15.12:0/295160400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f455410ee70 0x7f455407d390 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f455406c110 tx=0x7f454400b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:40:09.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.707+0000 7f454b7fe700 1 -- 172.21.15.12:0/295160400 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4544015020 con 0x7f455410ee70 2024-05-17T22:40:09.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.707+0000 7f4558ce7700 1 -- 172.21.15.12:0/295160400 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4554082080 con 0x7f455410ee70 2024-05-17T22:40:09.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.707+0000 7f4558ce7700 1 -- 172.21.15.12:0/295160400 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45540825c0 con 0x7f455410ee70 2024-05-17T22:40:09.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.710+0000 7f454b7fe700 1 -- 172.21.15.12:0/295160400 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4544011ce0 con 0x7f455410ee70 2024-05-17T22:40:09.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.710+0000 7f454b7fe700 1 -- 172.21.15.12:0/295160400 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4544010c80 con 0x7f455410ee70 2024-05-17T22:40:09.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.714+0000 7f45676c2b80 1 Processor -- start 2024-05-17T22:40:09.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.715+0000 7f45676c2b80 1 -- start start 2024-05-17T22:40:09.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.715+0000 7f45676c2b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6cc9489f0 0x55c6ccacc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:09.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.715+0000 7f45676c2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6ccaccaa0 0x55c6ccac6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:09.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.715+0000 7f45676c2b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6ccac7170 0x55c6ccac7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:09.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.715+0000 7f45676c2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c6cc9f5290 con 0x55c6ccaccaa0 2024-05-17T22:40:09.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.715+0000 7f45676c2b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c6cc9e4fd0 con 0x55c6cc9489f0 2024-05-17T22:40:09.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.715+0000 7f45676c2b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c6cc9e8e70 con 0x55c6ccac7170 2024-05-17T22:40:09.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.716+0000 7f455259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6ccaccaa0 0x55c6ccac6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:09.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.716+0000 7f455259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6ccaccaa0 0x55c6ccac6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59950/0 (socket says 172.21.15.12:59950) 2024-05-17T22:40:09.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.716+0000 7f455259c700 1 -- 172.21.15.12:0/56595469 learned_addr learned my addr 172.21.15.12:0/56595469 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:40:09.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.716+0000 7f4552d9d700 1 --2- 172.21.15.12:0/56595469 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6cc9489f0 0x55c6ccacc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:09.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.716+0000 7f454b7fe700 1 -- 172.21.15.12:0/295160400 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4544018020 con 0x7f455410ee70 2024-05-17T22:40:09.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.716+0000 7f455259c700 1 -- 172.21.15.12:0/56595469 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6ccac7170 msgr2=0x55c6ccac7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:40:09.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.716+0000 7f455259c700 1 --2- 172.21.15.12:0/56595469 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6ccac7170 0x55c6ccac7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:09.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.716+0000 7f455259c700 1 -- 172.21.15.12:0/56595469 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6cc9489f0 msgr2=0x55c6ccacc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:40:09.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.716+0000 7f455259c700 1 --2- 172.21.15.12:0/56595469 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6cc9489f0 0x55c6ccacc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:09.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.716+0000 7f455259c700 1 -- 172.21.15.12:0/56595469 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6cc98e590 con 0x55c6ccaccaa0 2024-05-17T22:40:09.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.717+0000 7f455259c700 1 --2- 172.21.15.12:0/56595469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6ccaccaa0 0x55c6ccac6c30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f453c009e20 tx=0x7f453c0116c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375c6aefcb707576 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:40:09.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.717+0000 7f4548ff9700 1 -- 172.21.15.12:0/56595469 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453c004830 con 0x55c6ccaccaa0 2024-05-17T22:40:09.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.717+0000 7f4548ff9700 1 -- 172.21.15.12:0/56595469 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f453c005020 con 0x55c6ccaccaa0 2024-05-17T22:40:09.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.719+0000 7f45676c2b80 1 -- 172.21.15.12:0/56595469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6ccaccaa0 msgr2=0x55c6ccac6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:09.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.719+0000 7f45676c2b80 1 --2- 172.21.15.12:0/56595469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6ccaccaa0 0x55c6ccac6c30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f453c009e20 tx=0x7f453c0116c0 comp rx=0 tx=0).stop 2024-05-17T22:40:09.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.719+0000 7f454b7fe700 1 --2- 172.21.15.12:0/295160400 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f45400b2a90 0x7f45400b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:09.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.720+0000 7f454b7fe700 1 -- 172.21.15.12:0/295160400 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f45440e1b30 con 0x7f455410ee70 2024-05-17T22:40:09.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.721+0000 7f4552d9d700 1 --2- 172.21.15.12:0/295160400 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f45400b2a90 0x7f45400b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:09.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.722+0000 7f45676c2b80 1 -- 172.21.15.12:0/56595469 shutdown_connections 2024-05-17T22:40:09.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.722+0000 7f45676c2b80 1 --2- 172.21.15.12:0/56595469 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6ccac7170 0x55c6ccac7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:09.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.722+0000 7f45676c2b80 1 --2- 172.21.15.12:0/56595469 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6cc9489f0 0x55c6ccacc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:09.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.722+0000 7f45676c2b80 1 --2- 172.21.15.12:0/56595469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6ccaccaa0 0x55c6ccac6c30 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:09.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.722+0000 7f45676c2b80 1 -- 172.21.15.12:0/56595469 >> 172.21.15.12:0/56595469 conn(0x55c6cc98cf00 msgr2=0x55c6ccac1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:40:09.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.724+0000 7f4552d9d700 1 --2- 172.21.15.12:0/295160400 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f45400b2a90 0x7f45400b4fa0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55c6ccac2ae0 tx=0x7f454c009b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:40:09.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.724+0000 7f45676c2b80 1 -- 172.21.15.12:0/56595469 shutdown_connections 2024-05-17T22:40:09.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.725+0000 7f45676c2b80 1 -- 172.21.15.12:0/56595469 wait complete. 2024-05-17T22:40:09.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.725+0000 7f45676c2b80 1 -- start start 2024-05-17T22:40:09.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.726+0000 7f45676c2b80 1 Processor -- start 2024-05-17T22:40:09.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.726+0000 7f45676c2b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6cc9489f0 0x55c6ccad01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:09.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.726+0000 7f45676c2b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6ccac7170 0x55c6ccad0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:09.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.726+0000 7f45676c2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6ccaccaa0 0x55c6ccad0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:09.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.726+0000 7f45676c2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c6cc8ec150 con 0x55c6ccaccaa0 2024-05-17T22:40:09.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.726+0000 7f45676c2b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c6cc985e80 con 0x55c6cc9489f0 2024-05-17T22:40:09.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.726+0000 7f45676c2b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c6ccad33e0 con 0x55c6ccac7170 2024-05-17T22:40:09.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.726+0000 7f45676c2b80 10 client.0 _ll_register_callbacks cb 0x55c6cc97ff10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:40:09.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.726+0000 7f454bfff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6ccac7170 0x55c6ccad0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:09.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.727+0000 7f455259c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6cc9489f0 0x55c6ccad01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:09.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.727+0000 7f4552d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6ccaccaa0 0x55c6ccad0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:09.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.727+0000 7f4552d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6ccaccaa0 0x55c6ccad0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59962/0 (socket says 172.21.15.12:59962) 2024-05-17T22:40:09.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.727+0000 7f4552d9d700 1 -- 172.21.15.12:0/3135238564 learned_addr learned my addr 172.21.15.12:0/3135238564 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:40:09.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.728+0000 7f4552d9d700 1 -- 172.21.15.12:0/3135238564 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6ccac7170 msgr2=0x55c6ccad0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:40:09.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.728+0000 7f4552d9d700 1 --2- 172.21.15.12:0/3135238564 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6ccac7170 0x55c6ccad0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:09.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.728+0000 7f4552d9d700 1 -- 172.21.15.12:0/3135238564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6cc9489f0 msgr2=0x55c6ccad01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:40:09.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.728+0000 7f4552d9d700 1 --2- 172.21.15.12:0/3135238564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6cc9489f0 0x55c6ccad01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:09.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.728+0000 7f4552d9d700 1 -- 172.21.15.12:0/3135238564 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6cc935480 con 0x55c6ccaccaa0 2024-05-17T22:40:09.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.728+0000 7f4552d9d700 1 --2- 172.21.15.12:0/3135238564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6ccaccaa0 0x55c6ccad0c50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f454c00cd80 tx=0x7f454c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8389627b531dbb05 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:40:09.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.728+0000 7f4539ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:40:09.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.728+0000 7f4539ffb700 1 -- 172.21.15.12:0/3135238564 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c02a030 con 0x55c6ccaccaa0 2024-05-17T22:40:09.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.729+0000 7f45676c2b80 1 -- 172.21.15.12:0/3135238564 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f453c005020 con 0x55c6ccaccaa0 2024-05-17T22:40:09.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.729+0000 7f4539ffb700 1 -- 172.21.15.12:0/3135238564 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f454c007ba0 con 0x55c6ccaccaa0 2024-05-17T22:40:09.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.729+0000 7f4539ffb700 1 -- 172.21.15.12:0/3135238564 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f454c037050 con 0x55c6ccaccaa0 2024-05-17T22:40:09.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.731+0000 7f45676c2b80 10 client.15450 fetch_fsmap learned FSMap version 29 2024-05-17T22:40:09.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.731+0000 7f45676c2b80 1 -- 172.21.15.12:0/3135238564 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c6ccaddf00 con 0x55c6ccaccaa0 2024-05-17T22:40:09.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.731+0000 7f4539ffb700 1 -- 172.21.15.12:0/3135238564 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f453c005020 con 0x55c6ccaccaa0 2024-05-17T22:40:09.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.732+0000 7f4539ffb700 1 -- 172.21.15.12:0/3135238564 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f454c011070 con 0x55c6ccaccaa0 2024-05-17T22:40:09.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.732+0000 7f45676c2b80 10 client.15450 fetch_fsmap finished waiting for FSMap version 29 2024-05-17T22:40:09.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.732+0000 7f45676c2b80 10 client.15450 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:40:09.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.732+0000 7f45676c2b80 20 client.15450 populate_metadata read hostname 'smithi012' 2024-05-17T22:40:09.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.732+0000 7f45676c2b80 1 --2- 172.21.15.12:0/3135238564 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55c6ccae0530 0x55c6ccae29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:09.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.732+0000 7f45676c2b80 4 client.15450 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:40:09.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.732+0000 7f45676c2b80 1 -- 172.21.15.12:0/3135238564 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6cc9f71e0 con 0x55c6ccae0530 2024-05-17T22:40:09.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.733+0000 7f455259c700 1 --2- 172.21.15.12:0/3135238564 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55c6ccae0530 0x55c6ccae29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:09.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.735+0000 7f455259c700 1 --2- 172.21.15.12:0/3135238564 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55c6ccae0530 0x55c6ccae29a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14c5360eebabe60 server_cookie=f92fc7b37b046620 in_seq=0 out_seq=0 2024-05-17T22:40:09.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.735+0000 7f4539ffb700 10 client.15450 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:40:09.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.736+0000 7f4539ffb700 1 -- 172.21.15.12:0/3135238564 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c6cc9f71e0 con 0x55c6ccae0530 2024-05-17T22:40:09.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.736+0000 7f4539ffb700 10 client.15450 handle_command_reply: tid=0 2024-05-17T22:40:09.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.736+0000 7f45676c2b80 1 client.15450 shutdown 2024-05-17T22:40:09.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.737+0000 7f45676c2b80 1 -- 172.21.15.12:0/3135238564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6ccaccaa0 msgr2=0x55c6ccad0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:09.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.737+0000 7f45676c2b80 1 --2- 172.21.15.12:0/3135238564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6ccaccaa0 0x55c6ccad0c50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f454c00cd80 tx=0x7f454c009920 comp rx=0 tx=0).stop 2024-05-17T22:40:09.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.738+0000 7f45676c2b80 1 -- 172.21.15.12:0/3135238564 shutdown_connections 2024-05-17T22:40:09.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.738+0000 7f45676c2b80 1 --2- 172.21.15.12:0/3135238564 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55c6ccae0530 0x55c6ccae29a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:09.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.738+0000 7f45676c2b80 1 --2- 172.21.15.12:0/3135238564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6ccaccaa0 0x55c6ccad0c50 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:09.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.738+0000 7f45676c2b80 1 --2- 172.21.15.12:0/3135238564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6cc9489f0 0x55c6ccad01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:09.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.738+0000 7f45676c2b80 1 --2- 172.21.15.12:0/3135238564 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6ccac7170 0x55c6ccad0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:09.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.738+0000 7f45676c2b80 1 -- 172.21.15.12:0/3135238564 >> 172.21.15.12:0/3135238564 conn(0x55c6cc98cf00 msgr2=0x55c6ccabbed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:40:09.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.738+0000 7f4539ffb700 0 client.15450 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:40:09.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.738+0000 7f45676c2b80 1 -- 172.21.15.12:0/3135238564 shutdown_connections 2024-05-17T22:40:09.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.738+0000 7f45676c2b80 1 -- 172.21.15.12:0/3135238564 wait complete. 2024-05-17T22:40:09.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.740+0000 7f45676c2b80 20 client.15450 trim_cache size 0 max 16384 2024-05-17T22:40:09.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.771+0000 7f45676c2b80 1 Processor -- start 2024-05-17T22:40:09.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.771+0000 7f45676c2b80 1 -- start start 2024-05-17T22:40:09.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.772+0000 7f45676c2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6cc9489f0 0x55c6ccac6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:09.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.772+0000 7f45676c2b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6ccac7170 0x55c6ccadc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:09.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.772+0000 7f45676c2b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6ccaccaa0 0x55c6ccabbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:09.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.772+0000 7f455259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6cc9489f0 0x55c6ccac6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:09.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.772+0000 7f455259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6cc9489f0 0x55c6ccac6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59970/0 (socket says 172.21.15.12:59970) 2024-05-17T22:40:09.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.772+0000 7f455259c700 1 -- 172.21.15.12:0/3717377507 learned_addr learned my addr 172.21.15.12:0/3717377507 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:40:09.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.772+0000 7f45676c2b80 1 -- 172.21.15.12:0/3717377507 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f45340113d0 con 0x55c6cc9489f0 2024-05-17T22:40:09.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.772+0000 7f45676c2b80 1 -- 172.21.15.12:0/3717377507 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f453400ec60 con 0x55c6ccaccaa0 2024-05-17T22:40:09.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.772+0000 7f45676c2b80 1 -- 172.21.15.12:0/3717377507 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f453400cab0 con 0x55c6ccac7170 2024-05-17T22:40:09.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.772+0000 7f4552d9d700 1 --2- 172.21.15.12:0/3717377507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6ccaccaa0 0x55c6ccabbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:09.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.772+0000 7f454bfff700 1 --2- 172.21.15.12:0/3717377507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6ccac7170 0x55c6ccadc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:09.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.773+0000 7f4552d9d700 1 -- 172.21.15.12:0/3717377507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6ccac7170 msgr2=0x55c6ccadc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:40:09.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.773+0000 7f4552d9d700 1 --2- 172.21.15.12:0/3717377507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6ccac7170 0x55c6ccadc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:09.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.773+0000 7f4552d9d700 1 -- 172.21.15.12:0/3717377507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6cc9489f0 msgr2=0x55c6ccac6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:40:09.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.773+0000 7f4552d9d700 1 --2- 172.21.15.12:0/3717377507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6cc9489f0 0x55c6ccac6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:09.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.773+0000 7f4552d9d700 1 -- 172.21.15.12:0/3717377507 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6ccace720 con 0x55c6ccaccaa0 2024-05-17T22:40:09.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.773+0000 7f455259c700 1 --2- 172.21.15.12:0/3717377507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6cc9489f0 0x55c6ccac6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:40:09.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.773+0000 7f4552d9d700 1 --2- 172.21.15.12:0/3717377507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6ccaccaa0 0x55c6ccabbeb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f454c009c10 tx=0x7f454c00cf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a3a583b50060b62 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:40:09.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.773+0000 7f453bfff700 1 -- 172.21.15.12:0/3717377507 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c02a070 con 0x55c6ccaccaa0 2024-05-17T22:40:09.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.774+0000 7f453bfff700 1 -- 172.21.15.12:0/3717377507 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f454c031110 con 0x55c6ccaccaa0 2024-05-17T22:40:09.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.774+0000 7f453bfff700 1 -- 172.21.15.12:0/3717377507 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c020070 con 0x55c6ccaccaa0 2024-05-17T22:40:09.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.775+0000 7f45676c2b80 1 -- 172.21.15.12:0/3717377507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6ccaccaa0 msgr2=0x55c6ccabbeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:09.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.775+0000 7f45676c2b80 1 --2- 172.21.15.12:0/3717377507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6ccaccaa0 0x55c6ccabbeb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f454c009c10 tx=0x7f454c00cf10 comp rx=0 tx=0).stop 2024-05-17T22:40:09.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.775+0000 7f45676c2b80 1 -- 172.21.15.12:0/3717377507 shutdown_connections 2024-05-17T22:40:09.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.775+0000 7f45676c2b80 1 --2- 172.21.15.12:0/3717377507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6ccaccaa0 0x55c6ccabbeb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:09.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.775+0000 7f45676c2b80 1 --2- 172.21.15.12:0/3717377507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6cc9489f0 0x55c6ccac6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:09.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.775+0000 7f45676c2b80 1 --2- 172.21.15.12:0/3717377507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6ccac7170 0x55c6ccadc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:09.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.775+0000 7f45676c2b80 1 -- 172.21.15.12:0/3717377507 >> 172.21.15.12:0/3717377507 conn(0x55c6cc98cf00 msgr2=0x55c6ccac9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:40:09.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.776+0000 7f45676c2b80 1 -- 172.21.15.12:0/3717377507 shutdown_connections 2024-05-17T22:40:09.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.776+0000 7f45676c2b80 1 -- 172.21.15.12:0/3717377507 wait complete. 2024-05-17T22:40:09.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.776+0000 7f45676c2b80 1 -- start start 2024-05-17T22:40:09.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.777+0000 7f45676c2b80 1 Processor -- start 2024-05-17T22:40:09.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.777+0000 7f45676c2b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6cc9489f0 0x55c6ccad8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:09.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.777+0000 7f45676c2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6ccac7170 0x55c6ccad8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:09.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.777+0000 7f45676c2b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6ccaccaa0 0x55c6ccae7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:09.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.777+0000 7f45676c2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f453400d120 con 0x55c6ccac7170 2024-05-17T22:40:09.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.777+0000 7f45676c2b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f453400e590 con 0x55c6ccaccaa0 2024-05-17T22:40:09.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.777+0000 7f45676c2b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f453400d7f0 con 0x55c6cc9489f0 2024-05-17T22:40:09.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.778+0000 7f454bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6ccac7170 0x55c6ccad8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:09.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.778+0000 7f4552d9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6ccaccaa0 0x55c6ccae7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:09.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.778+0000 7f455259c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6cc9489f0 0x55c6ccad8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:09.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.778+0000 7f45676c2b80 10 client.0 _ll_register_callbacks cb 0x55c6cc97ff10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:40:09.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.778+0000 7f4552d9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6ccaccaa0 0x55c6ccae7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:52700/0 (socket says 172.21.15.12:52700) 2024-05-17T22:40:09.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.778+0000 7f454bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6ccac7170 0x55c6ccad8c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59982/0 (socket says 172.21.15.12:59982) 2024-05-17T22:40:09.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.778+0000 7f4552d9d700 1 -- 172.21.15.12:0/1830647807 learned_addr learned my addr 172.21.15.12:0/1830647807 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:40:09.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.778+0000 7f455259c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6cc9489f0 0x55c6ccad8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:40726/0 (socket says 172.21.15.12:40726) 2024-05-17T22:40:09.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.778+0000 7f455259c700 1 -- 172.21.15.12:0/1830647807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6ccaccaa0 msgr2=0x55c6ccae7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:40:09.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.778+0000 7f455259c700 1 --2- 172.21.15.12:0/1830647807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6ccaccaa0 0x55c6ccae7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:09.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.778+0000 7f455259c700 1 -- 172.21.15.12:0/1830647807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6ccac7170 msgr2=0x55c6ccad8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:40:09.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.778+0000 7f455259c700 1 --2- 172.21.15.12:0/1830647807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6ccac7170 0x55c6ccad8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:09.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.778+0000 7f455259c700 1 -- 172.21.15.12:0/1830647807 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6ccae79d0 con 0x55c6cc9489f0 2024-05-17T22:40:09.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.779+0000 7f455259c700 1 --2- 172.21.15.12:0/1830647807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6cc9489f0 0x55c6ccad8740 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f453c000c00 tx=0x7f453c011c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=963d60c45644ebf4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:40:09.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.779+0000 7f4539ffb700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:40:09.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.779+0000 7f454bfff700 1 --2- 172.21.15.12:0/1830647807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6ccac7170 0x55c6ccad8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:40:09.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.779+0000 7f4539ffb700 1 -- 172.21.15.12:0/1830647807 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453c031060 con 0x55c6cc9489f0 2024-05-17T22:40:09.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.779+0000 7f4539ffb700 1 -- 172.21.15.12:0/1830647807 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f453c0053b0 con 0x55c6cc9489f0 2024-05-17T22:40:09.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.779+0000 7f45676c2b80 1 -- 172.21.15.12:0/1830647807 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f454c031110 con 0x55c6cc9489f0 2024-05-17T22:40:09.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.780+0000 7f4539ffb700 1 -- 172.21.15.12:0/1830647807 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f453c0055b0 con 0x55c6cc9489f0 2024-05-17T22:40:09.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.780+0000 7f45676c2b80 10 client.25075 fetch_fsmap learned FSMap version 29 2024-05-17T22:40:09.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.780+0000 7f45676c2b80 1 -- 172.21.15.12:0/1830647807 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c6ccb4a750 con 0x55c6cc9489f0 2024-05-17T22:40:09.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.781+0000 7f4539ffb700 1 -- 172.21.15.12:0/1830647807 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f454c031110 con 0x55c6cc9489f0 2024-05-17T22:40:09.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.781+0000 7f4539ffb700 1 -- 172.21.15.12:0/1830647807 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f453c02b3c0 con 0x55c6cc9489f0 2024-05-17T22:40:09.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.781+0000 7f45676c2b80 10 client.25075 fetch_fsmap finished waiting for FSMap version 29 2024-05-17T22:40:09.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.781+0000 7f45676c2b80 10 client.25075 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:40:09.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.781+0000 7f45676c2b80 20 client.25075 populate_metadata read hostname 'smithi012' 2024-05-17T22:40:09.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.781+0000 7f45676c2b80 1 --2- 172.21.15.12:0/1830647807 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55c6ccb4ab80 0x55c6ccb4cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:09.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.781+0000 7f45676c2b80 4 client.25075 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:40:09.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.781+0000 7f45676c2b80 1 -- 172.21.15.12:0/1830647807 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c6cc9f09c0 con 0x55c6ccb4ab80 2024-05-17T22:40:09.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.782+0000 7f454bfff700 1 --2- 172.21.15.12:0/1830647807 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55c6ccb4ab80 0x55c6ccb4cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:09.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.783+0000 7f4539ffb700 10 client.25075 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:40:09.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.783+0000 7f454bfff700 1 --2- 172.21.15.12:0/1830647807 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55c6ccb4ab80 0x55c6ccb4cfa0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7707c9d8e48c2611 server_cookie=753900680db1168c in_seq=0 out_seq=0 2024-05-17T22:40:09.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.783+0000 7f4539ffb700 1 -- 172.21.15.12:0/1830647807 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c6cc9f09c0 con 0x55c6ccb4ab80 2024-05-17T22:40:09.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.783+0000 7f4539ffb700 10 client.25075 handle_command_reply: tid=0 2024-05-17T22:40:09.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.783+0000 7f45676c2b80 1 client.25075 shutdown 2024-05-17T22:40:09.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.783+0000 7f45676c2b80 1 -- 172.21.15.12:0/1830647807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6cc9489f0 msgr2=0x55c6ccad8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:09.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.783+0000 7f45676c2b80 1 --2- 172.21.15.12:0/1830647807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6cc9489f0 0x55c6ccad8740 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f453c000c00 tx=0x7f453c011c70 comp rx=0 tx=0).stop 2024-05-17T22:40:09.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.783+0000 7f45676c2b80 1 -- 172.21.15.12:0/1830647807 shutdown_connections 2024-05-17T22:40:09.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.783+0000 7f45676c2b80 1 --2- 172.21.15.12:0/1830647807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6ccaccaa0 0x55c6ccae7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:09.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.783+0000 7f45676c2b80 1 --2- 172.21.15.12:0/1830647807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6cc9489f0 0x55c6ccad8740 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:09.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.784+0000 7f45676c2b80 1 --2- 172.21.15.12:0/1830647807 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55c6ccb4ab80 0x55c6ccb4cfa0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:09.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.784+0000 7f45676c2b80 1 --2- 172.21.15.12:0/1830647807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6ccac7170 0x55c6ccad8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:09.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.784+0000 7f45676c2b80 1 -- 172.21.15.12:0/1830647807 >> 172.21.15.12:0/1830647807 conn(0x55c6cc98cf00 msgr2=0x55c6ccac9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:40:09.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.784+0000 7f4539ffb700 0 client.25075 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:40:09.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.784+0000 7f45676c2b80 1 -- 172.21.15.12:0/1830647807 shutdown_connections 2024-05-17T22:40:09.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.785+0000 7f45676c2b80 1 -- 172.21.15.12:0/1830647807 wait complete. 2024-05-17T22:40:09.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.785+0000 7f45676c2b80 20 client.25075 trim_cache size 0 max 16384 2024-05-17T22:40:09.788 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:40:09.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.786+0000 7f4558ce7700 1 -- 172.21.15.12:0/295160400 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f45400b2a90 msgr2=0x7f45400b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:09.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.786+0000 7f4558ce7700 1 --2- 172.21.15.12:0/295160400 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f45400b2a90 0x7f45400b4fa0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55c6ccac2ae0 tx=0x7f454c009b40 comp rx=0 tx=0).stop 2024-05-17T22:40:09.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.786+0000 7f4558ce7700 1 -- 172.21.15.12:0/295160400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f455410ee70 msgr2=0x7f455407d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:09.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.786+0000 7f4558ce7700 1 --2- 172.21.15.12:0/295160400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f455410ee70 0x7f455407d390 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f455406c110 tx=0x7f454400b730 comp rx=0 tx=0).stop 2024-05-17T22:40:09.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.786+0000 7f4558ce7700 1 -- 172.21.15.12:0/295160400 shutdown_connections 2024-05-17T22:40:09.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.786+0000 7f4558ce7700 1 --2- 172.21.15.12:0/295160400 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f45400b2a90 0x7f45400b4fa0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:09.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.786+0000 7f4558ce7700 1 --2- 172.21.15.12:0/295160400 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f455410f7c0 0x7f455407d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:09.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.786+0000 7f4558ce7700 1 --2- 172.21.15.12:0/295160400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f455410a540 0x7f4554084260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:09.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.786+0000 7f4558ce7700 1 --2- 172.21.15.12:0/295160400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f455410ee70 0x7f455407d390 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:09.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.786+0000 7f4558ce7700 1 -- 172.21.15.12:0/295160400 >> 172.21.15.12:0/295160400 conn(0x7f4554079c00 msgr2=0x7f455407a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:40:09.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.787+0000 7f4558ce7700 1 -- 172.21.15.12:0/295160400 shutdown_connections 2024-05-17T22:40:09.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:09.787+0000 7f4558ce7700 1 -- 172.21.15.12:0/295160400 wait complete. 2024-05-17T22:40:10.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:09 smithi012 ceph-mon[31632]: pgmap v466: 129 pgs: 129 active+clean; 999 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 11 MiB/s wr, 859 op/s 2024-05-17T22:40:10.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:09 smithi151 ceph-mon[36101]: pgmap v466: 129 pgs: 129 active+clean; 999 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 11 MiB/s wr, 859 op/s 2024-05-17T22:40:10.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:09 smithi096 ceph-mon[36065]: pgmap v466: 129 pgs: 129 active+clean; 999 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 11 MiB/s wr, 859 op/s 2024-05-17T22:40:11.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 50.0 s, 3496.8 tps, lat 28.677 ms stddev 41.778 2024-05-17T22:40:11.599 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-17T22:40:11.599 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-17T22:40:12.063 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:40:12.207 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:11 smithi012 ceph-mon[31632]: pgmap v467: 129 pgs: 129 active+clean; 999 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.3 MiB/s wr, 671 op/s 2024-05-17T22:40:12.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:11 smithi151 ceph-mon[36101]: pgmap v467: 129 pgs: 129 active+clean; 999 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.3 MiB/s wr, 671 op/s 2024-05-17T22:40:12.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:11 smithi096 ceph-mon[36065]: pgmap v467: 129 pgs: 129 active+clean; 999 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.3 MiB/s wr, 671 op/s 2024-05-17T22:40:14.145 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:13 smithi012 ceph-mon[31632]: pgmap v468: 129 pgs: 129 active+clean; 999 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.2 MiB/s wr, 771 op/s 2024-05-17T22:40:14.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:13 smithi151 ceph-mon[36101]: pgmap v468: 129 pgs: 129 active+clean; 999 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.2 MiB/s wr, 771 op/s 2024-05-17T22:40:14.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:13 smithi096 ceph-mon[36065]: pgmap v468: 129 pgs: 129 active+clean; 999 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.2 MiB/s wr, 771 op/s 2024-05-17T22:40:16.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:15 smithi012 ceph-mon[31632]: pgmap v469: 129 pgs: 129 active+clean; 1001 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.9 MiB/s wr, 783 op/s 2024-05-17T22:40:16.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:15 smithi151 ceph-mon[36101]: pgmap v469: 129 pgs: 129 active+clean; 1001 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.9 MiB/s wr, 783 op/s 2024-05-17T22:40:16.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:15 smithi096 ceph-mon[36065]: pgmap v469: 129 pgs: 129 active+clean; 1001 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.9 MiB/s wr, 783 op/s 2024-05-17T22:40:16.371 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 55.0 s, 3475.9 tps, lat 27.603 ms stddev 32.873 2024-05-17T22:40:17.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.122+0000 7ffa0e677700 1 -- 172.21.15.12:0/1542697349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa081032d0 msgr2=0x7ffa081036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:17.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.122+0000 7ffa0e677700 1 --2- 172.21.15.12:0/1542697349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa081032d0 0x7ffa081036e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc009990 tx=0x7ff9fc009ca0 comp rx=0 tx=0).stop 2024-05-17T22:40:17.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.123+0000 7ffa0e677700 1 -- 172.21.15.12:0/1542697349 shutdown_connections 2024-05-17T22:40:17.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.123+0000 7ffa0e677700 1 --2- 172.21.15.12:0/1542697349 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa08103c20 0x7ffa0810f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:17.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.123+0000 7ffa0e677700 1 --2- 172.21.15.12:0/1542697349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa08107b30 0x7ffa08107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:17.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.123+0000 7ffa0e677700 1 --2- 172.21.15.12:0/1542697349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa081032d0 0x7ffa081036e0 secure :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc009990 tx=0x7ff9fc009ca0 comp rx=0 tx=0).stop 2024-05-17T22:40:17.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.123+0000 7ffa0e677700 1 -- 172.21.15.12:0/1542697349 >> 172.21.15.12:0/1542697349 conn(0x7ffa080fe030 msgr2=0x7ffa08100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:40:17.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.125+0000 7ffa0e677700 1 -- 172.21.15.12:0/1542697349 shutdown_connections 2024-05-17T22:40:17.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.127+0000 7ffa0e677700 1 -- 172.21.15.12:0/1542697349 wait complete. 2024-05-17T22:40:17.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.128+0000 7ffa0e677700 1 Processor -- start 2024-05-17T22:40:17.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.130+0000 7ffa0e677700 1 -- start start 2024-05-17T22:40:17.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.130+0000 7ffa0e677700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa08103c20 0x7ffa0806e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:17.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.130+0000 7ffa0e677700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa08107b30 0x7ffa0806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:17.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.130+0000 7ffa0e677700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa0806f070 0x7ffa08072ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:17.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.130+0000 7ffa0e677700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ffa08111c20 con 0x7ffa08103c20 2024-05-17T22:40:17.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.130+0000 7ffa0e677700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffa08111aa0 con 0x7ffa0806f070 2024-05-17T22:40:17.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.130+0000 7ffa0e677700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffa08111da0 con 0x7ffa08107b30 2024-05-17T22:40:17.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.131+0000 7ffa0cc14700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa0806f070 0x7ffa08072ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:17.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.131+0000 7ffa07fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa08103c20 0x7ffa0806e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:17.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.131+0000 7ffa07fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa08103c20 0x7ffa0806e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59992/0 (socket says 172.21.15.12:59992) 2024-05-17T22:40:17.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.131+0000 7ffa07fff700 1 -- 172.21.15.12:0/4046966005 learned_addr learned my addr 172.21.15.12:0/4046966005 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:40:17.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.132+0000 7ffa07fff700 1 -- 172.21.15.12:0/4046966005 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa08107b30 msgr2=0x7ffa0806eb30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-17T22:40:17.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.132+0000 7ffa07fff700 1 --2- 172.21.15.12:0/4046966005 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa08107b30 0x7ffa0806eb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:17.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.132+0000 7ffa07fff700 1 -- 172.21.15.12:0/4046966005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa0806f070 msgr2=0x7ffa08072ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:17.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.132+0000 7ffa07fff700 1 --2- 172.21.15.12:0/4046966005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa0806f070 0x7ffa08072ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:17.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.132+0000 7ffa07fff700 1 -- 172.21.15.12:0/4046966005 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa08073530 con 0x7ffa08103c20 2024-05-17T22:40:17.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.133+0000 7ffa07fff700 1 --2- 172.21.15.12:0/4046966005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa08103c20 0x7ffa0806e5f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff9f800b550 tx=0x7ff9f800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:40:17.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.133+0000 7ffa057fa700 1 -- 172.21.15.12:0/4046966005 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f8015020 con 0x7ffa08103c20 2024-05-17T22:40:17.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.133+0000 7ffa0e677700 1 -- 172.21.15.12:0/4046966005 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa08073820 con 0x7ffa08103c20 2024-05-17T22:40:17.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.133+0000 7ffa057fa700 1 -- 172.21.15.12:0/4046966005 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9f8011ce0 con 0x7ffa08103c20 2024-05-17T22:40:17.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.133+0000 7ffa057fa700 1 -- 172.21.15.12:0/4046966005 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f8010de0 con 0x7ffa08103c20 2024-05-17T22:40:17.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.133+0000 7ffa0e677700 1 -- 172.21.15.12:0/4046966005 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa08073d60 con 0x7ffa08103c20 2024-05-17T22:40:17.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.138+0000 7ffa057fa700 1 -- 172.21.15.12:0/4046966005 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff9f8018020 con 0x7ffa08103c20 2024-05-17T22:40:17.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.139+0000 7ffa057fa700 1 --2- 172.21.15.12:0/4046966005 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff9f00b28e0 0x7ff9f00b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:17.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.139+0000 7ffa057fa700 1 -- 172.21.15.12:0/4046966005 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7ff9f80e1e10 con 0x7ffa08103c20 2024-05-17T22:40:17.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.140+0000 7ffa077fe700 1 --2- 172.21.15.12:0/4046966005 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff9f00b28e0 0x7ff9f00b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:17.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.143+0000 7ffa077fe700 1 --2- 172.21.15.12:0/4046966005 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff9f00b28e0 0x7ff9f00b4df0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc000c00 tx=0x7ff9fc00b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:40:17.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.145+0000 7ffa1d052b80 1 Processor -- start 2024-05-17T22:40:17.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.145+0000 7ffa1d052b80 1 -- start start 2024-05-17T22:40:17.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.146+0000 7ffa1d052b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b1eebfa70 0x562b1f042f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:17.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.146+0000 7ffa1d052b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562b1f043480 0x562b1f03d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:17.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.146+0000 7ffa1d052b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b1f03db80 0x562b1f03df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:17.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.146+0000 7ffa1d052b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x562b1ef6c150 con 0x562b1eebfa70 2024-05-17T22:40:17.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.146+0000 7ffa1d052b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562b1ef5bcf0 con 0x562b1f03db80 2024-05-17T22:40:17.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.146+0000 7ffa1d052b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562b1ef5fd30 con 0x562b1f043480 2024-05-17T22:40:17.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.147+0000 7ffa0cc14700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b1f03db80 0x562b1f03df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:17.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.147+0000 7ffa0cc14700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b1f03db80 0x562b1f03df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:52734/0 (socket says 172.21.15.12:52734) 2024-05-17T22:40:17.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.147+0000 7ffa0cc14700 1 -- 172.21.15.12:0/2557943712 learned_addr learned my addr 172.21.15.12:0/2557943712 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:40:17.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.147+0000 7ffa077fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562b1f043480 0x562b1f03d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:17.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.147+0000 7ffa0cc14700 1 -- 172.21.15.12:0/2557943712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562b1f043480 msgr2=0x562b1f03d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:40:17.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.147+0000 7ffa0cc14700 1 --2- 172.21.15.12:0/2557943712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562b1f043480 0x562b1f03d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:17.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.147+0000 7ffa0cc14700 1 -- 172.21.15.12:0/2557943712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b1eebfa70 msgr2=0x562b1f042f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:40:17.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.147+0000 7ffa0cc14700 1 --2- 172.21.15.12:0/2557943712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b1eebfa70 0x562b1f042f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:17.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.147+0000 7ffa0cc14700 1 -- 172.21.15.12:0/2557943712 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b1eef5640 con 0x562b1f03db80 2024-05-17T22:40:17.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.147+0000 7ffa077fe700 1 --2- 172.21.15.12:0/2557943712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562b1f043480 0x562b1f03d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:40:17.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.148+0000 7ffa0cc14700 1 --2- 172.21.15.12:0/2557943712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b1f03db80 0x562b1f03df90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ffa00009370 tx=0x7ffa00009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b6affb48dc9a23a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:40:17.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.148+0000 7ff9ee7fc700 1 -- 172.21.15.12:0/2557943712 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0001a070 con 0x562b1f03db80 2024-05-17T22:40:17.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.148+0000 7ff9ee7fc700 1 -- 172.21.15.12:0/2557943712 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa00003e10 con 0x562b1f03db80 2024-05-17T22:40:17.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.148+0000 7ff9ee7fc700 1 -- 172.21.15.12:0/2557943712 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa00015a10 con 0x562b1f03db80 2024-05-17T22:40:17.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.150+0000 7ffa1d052b80 1 -- 172.21.15.12:0/2557943712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b1f03db80 msgr2=0x562b1f03df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:17.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.150+0000 7ffa1d052b80 1 --2- 172.21.15.12:0/2557943712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b1f03db80 0x562b1f03df90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ffa00009370 tx=0x7ffa00009920 comp rx=0 tx=0).stop 2024-05-17T22:40:17.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.150+0000 7ffa1d052b80 1 -- 172.21.15.12:0/2557943712 shutdown_connections 2024-05-17T22:40:17.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.150+0000 7ffa1d052b80 1 --2- 172.21.15.12:0/2557943712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b1f03db80 0x562b1f03df90 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:17.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.150+0000 7ffa1d052b80 1 --2- 172.21.15.12:0/2557943712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b1eebfa70 0x562b1f042f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:17.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.150+0000 7ffa1d052b80 1 --2- 172.21.15.12:0/2557943712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562b1f043480 0x562b1f03d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:17.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.150+0000 7ffa1d052b80 1 -- 172.21.15.12:0/2557943712 >> 172.21.15.12:0/2557943712 conn(0x562b1ef1f850 msgr2=0x562b1f038240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:40:17.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.150+0000 7ffa1d052b80 1 -- 172.21.15.12:0/2557943712 shutdown_connections 2024-05-17T22:40:17.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.150+0000 7ffa1d052b80 1 -- 172.21.15.12:0/2557943712 wait complete. 2024-05-17T22:40:17.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.151+0000 7ffa1d052b80 1 -- start start 2024-05-17T22:40:17.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.152+0000 7ffa1d052b80 1 Processor -- start 2024-05-17T22:40:17.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.152+0000 7ffa1d052b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562b1eebfa70 0x562b1f046c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:17.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.152+0000 7ffa1d052b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b1f03db80 0x562b1f047180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:17.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.153+0000 7ffa1d052b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b1f043480 0x562b1f053180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:17.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.153+0000 7ffa1d052b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x562b1ee631d0 con 0x562b1f03db80 2024-05-17T22:40:17.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.153+0000 7ffa07fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562b1eebfa70 0x562b1f046c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:17.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.153+0000 7ffa077fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b1f03db80 0x562b1f047180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:17.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.153+0000 7ffa07fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562b1eebfa70 0x562b1f046c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:40754/0 (socket says 172.21.15.12:40754) 2024-05-17T22:40:17.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.153+0000 7ffa07fff700 1 -- 172.21.15.12:0/3935338821 learned_addr learned my addr 172.21.15.12:0/3935338821 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:40:17.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.153+0000 7ffa0cc14700 1 --2- 172.21.15.12:0/3935338821 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b1f043480 0x562b1f053180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:17.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.153+0000 7ffa1d052b80 1 -- 172.21.15.12:0/3935338821 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562b1eefd9c0 con 0x562b1f043480 2024-05-17T22:40:17.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.153+0000 7ffa1d052b80 1 -- 172.21.15.12:0/3935338821 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562b1f049e50 con 0x562b1eebfa70 2024-05-17T22:40:17.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.154+0000 7ffa077fe700 1 -- 172.21.15.12:0/3935338821 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562b1eebfa70 msgr2=0x562b1f046c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:40:17.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.154+0000 7ffa077fe700 1 --2- 172.21.15.12:0/3935338821 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562b1eebfa70 0x562b1f046c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:17.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.154+0000 7ffa077fe700 1 -- 172.21.15.12:0/3935338821 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b1f043480 msgr2=0x562b1f053180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:40:17.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.154+0000 7ffa077fe700 1 --2- 172.21.15.12:0/3935338821 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b1f043480 0x562b1f053180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:17.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.154+0000 7ffa077fe700 1 -- 172.21.15.12:0/3935338821 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b1eeac500 con 0x562b1f03db80 2024-05-17T22:40:17.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.154+0000 7ffa077fe700 1 --2- 172.21.15.12:0/3935338821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b1f03db80 0x562b1f047180 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc008000 tx=0x7ff9fc009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad6118e0ce0907a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:40:17.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.154+0000 7ffa1d052b80 10 client.0 _ll_register_callbacks cb 0x562b1eef2bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:40:17.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.154+0000 7ff9e77fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:40:17.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.154+0000 7ffa1d052b80 1 -- 172.21.15.12:0/3935338821 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa00003e10 con 0x562b1f03db80 2024-05-17T22:40:17.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.154+0000 7ff9e77fe700 1 -- 172.21.15.12:0/3935338821 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc004230 con 0x562b1f03db80 2024-05-17T22:40:17.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.155+0000 7ff9e77fe700 1 -- 172.21.15.12:0/3935338821 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7ff9fc01e030 con 0x562b1f03db80 2024-05-17T22:40:17.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.155+0000 7ff9e77fe700 1 -- 172.21.15.12:0/3935338821 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff9fc0095d0 con 0x562b1f03db80 2024-05-17T22:40:17.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.155+0000 7ffa1d052b80 10 client.15462 fetch_fsmap learned FSMap version 29 2024-05-17T22:40:17.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.155+0000 7ffa1d052b80 1 -- 172.21.15.12:0/3935338821 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({fsmap=29}) -- 0x562b1f054910 con 0x562b1f03db80 2024-05-17T22:40:17.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.156+0000 7ff9e77fe700 1 -- 172.21.15.12:0/3935338821 <== mon.0 v2:172.21.15.12:3300/0 4 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff9fc003c70 con 0x562b1f03db80 2024-05-17T22:40:17.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.156+0000 7ffa1d052b80 10 client.15462 fetch_fsmap finished waiting for FSMap version 29 2024-05-17T22:40:17.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.156+0000 7ffa1d052b80 10 client.15462 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:40:17.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.156+0000 7ffa1d052b80 20 client.15462 populate_metadata read hostname 'smithi012' 2024-05-17T22:40:17.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.156+0000 7ffa1d052b80 1 --2- 172.21.15.12:0/3935338821 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x562b1f056e30 0x562b1f0592b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:17.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.156+0000 7ffa1d052b80 4 client.15462 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:40:17.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.156+0000 7ffa1d052b80 1 -- 172.21.15.12:0/3935338821 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ffa00003e10 con 0x562b1f056e30 2024-05-17T22:40:17.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.156+0000 7ffa07fff700 1 --2- 172.21.15.12:0/3935338821 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x562b1f056e30 0x562b1f0592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:17.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.157+0000 7ffa07fff700 1 --2- 172.21.15.12:0/3935338821 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x562b1f056e30 0x562b1f0592b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89c36267cdd7c145 server_cookie=7e4d5e5d19a8e954 in_seq=0 out_seq=0 2024-05-17T22:40:17.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.157+0000 7ff9e77fe700 10 client.15462 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:40:17.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.157+0000 7ff9e77fe700 1 -- 172.21.15.12:0/3935338821 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7ffa00003e10 con 0x562b1f056e30 2024-05-17T22:40:17.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.157+0000 7ff9e77fe700 10 client.15462 handle_command_reply: tid=0 2024-05-17T22:40:17.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.159+0000 7ffa1d052b80 1 client.15462 shutdown 2024-05-17T22:40:17.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.160+0000 7ffa1d052b80 1 -- 172.21.15.12:0/3935338821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b1f03db80 msgr2=0x562b1f047180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:17.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.160+0000 7ffa1d052b80 1 --2- 172.21.15.12:0/3935338821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b1f03db80 0x562b1f047180 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc008000 tx=0x7ff9fc009ca0 comp rx=0 tx=0).stop 2024-05-17T22:40:17.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.160+0000 7ffa1d052b80 1 -- 172.21.15.12:0/3935338821 shutdown_connections 2024-05-17T22:40:17.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.160+0000 7ffa1d052b80 1 --2- 172.21.15.12:0/3935338821 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b1f043480 0x562b1f053180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:17.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.160+0000 7ffa1d052b80 1 --2- 172.21.15.12:0/3935338821 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562b1eebfa70 0x562b1f046c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:17.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.160+0000 7ffa1d052b80 1 --2- 172.21.15.12:0/3935338821 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x562b1f056e30 0x562b1f0592b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:17.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.160+0000 7ffa1d052b80 1 --2- 172.21.15.12:0/3935338821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b1f03db80 0x562b1f047180 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:17.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.160+0000 7ffa1d052b80 1 -- 172.21.15.12:0/3935338821 >> 172.21.15.12:0/3935338821 conn(0x562b1ef1f850 msgr2=0x562b1f0381e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:40:17.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.160+0000 7ff9e77fe700 0 client.15462 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:40:17.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.160+0000 7ffa1d052b80 1 -- 172.21.15.12:0/3935338821 shutdown_connections 2024-05-17T22:40:17.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.160+0000 7ffa1d052b80 1 -- 172.21.15.12:0/3935338821 wait complete. 2024-05-17T22:40:17.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.160+0000 7ffa1d052b80 20 client.15462 trim_cache size 0 max 16384 2024-05-17T22:40:17.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.196+0000 7ffa1d052b80 1 Processor -- start 2024-05-17T22:40:17.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.196+0000 7ffa1d052b80 1 -- start start 2024-05-17T22:40:17.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.196+0000 7ffa1d052b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562b1ef1f850 0x562b1f045b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:17.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.196+0000 7ffa1d052b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b1eebfa70 0x562b1f047190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:17.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.196+0000 7ffa1d052b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b1f03db80 0x562b1f040f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:17.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.196+0000 7ffa1d052b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ffa081c1b80 con 0x562b1f03db80 2024-05-17T22:40:17.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.196+0000 7ffa1d052b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffa081bf410 con 0x562b1eebfa70 2024-05-17T22:40:17.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.196+0000 7ffa1d052b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffa081bd260 con 0x562b1ef1f850 2024-05-17T22:40:17.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.197+0000 7ffa07fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562b1ef1f850 0x562b1f045b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:17.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.197+0000 7ffa07fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562b1ef1f850 0x562b1f045b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:40770/0 (socket says 172.21.15.12:40770) 2024-05-17T22:40:17.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.197+0000 7ffa0cc14700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b1f03db80 0x562b1f040f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:17.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.197+0000 7ffa07fff700 1 -- 172.21.15.12:0/975488782 learned_addr learned my addr 172.21.15.12:0/975488782 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:40:17.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.197+0000 7ffa077fe700 1 --2- 172.21.15.12:0/975488782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b1eebfa70 0x562b1f047190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:17.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.197+0000 7ffa07fff700 1 -- 172.21.15.12:0/975488782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b1eebfa70 msgr2=0x562b1f047190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:40:17.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.197+0000 7ffa07fff700 1 --2- 172.21.15.12:0/975488782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b1eebfa70 0x562b1f047190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:17.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.197+0000 7ffa07fff700 1 -- 172.21.15.12:0/975488782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b1f03db80 msgr2=0x562b1f040f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:40:17.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.197+0000 7ffa07fff700 1 --2- 172.21.15.12:0/975488782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b1f03db80 0x562b1f040f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:17.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.197+0000 7ffa07fff700 1 -- 172.21.15.12:0/975488782 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b1f049900 con 0x562b1ef1f850 2024-05-17T22:40:17.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.197+0000 7ffa0cc14700 1 --2- 172.21.15.12:0/975488782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b1f03db80 0x562b1f040f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:40:17.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.197+0000 7ffa07fff700 1 --2- 172.21.15.12:0/975488782 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562b1ef1f850 0x562b1f045b80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff9f80a8d40 tx=0x7ff9f8006db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=678e3196b27c0b78 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:40:17.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.197+0000 7ff9edffb700 1 -- 172.21.15.12:0/975488782 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f80a9040 con 0x562b1ef1f850 2024-05-17T22:40:17.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.198+0000 7ff9edffb700 1 -- 172.21.15.12:0/975488782 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9f80c3070 con 0x562b1ef1f850 2024-05-17T22:40:17.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.198+0000 7ff9edffb700 1 -- 172.21.15.12:0/975488782 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f80a92a0 con 0x562b1ef1f850 2024-05-17T22:40:17.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.199+0000 7ffa1d052b80 1 -- 172.21.15.12:0/975488782 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562b1ef1f850 msgr2=0x562b1f045b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:17.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.199+0000 7ffa1d052b80 1 --2- 172.21.15.12:0/975488782 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562b1ef1f850 0x562b1f045b80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff9f80a8d40 tx=0x7ff9f8006db0 comp rx=0 tx=0).stop 2024-05-17T22:40:17.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.199+0000 7ffa1d052b80 1 -- 172.21.15.12:0/975488782 shutdown_connections 2024-05-17T22:40:17.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.199+0000 7ffa1d052b80 1 --2- 172.21.15.12:0/975488782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b1f03db80 0x562b1f040f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:17.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.199+0000 7ffa1d052b80 1 --2- 172.21.15.12:0/975488782 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562b1ef1f850 0x562b1f045b80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:17.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.199+0000 7ffa1d052b80 1 --2- 172.21.15.12:0/975488782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b1eebfa70 0x562b1f047190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:17.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.199+0000 7ffa1d052b80 1 -- 172.21.15.12:0/975488782 >> 172.21.15.12:0/975488782 conn(0x562b1f038540 msgr2=0x562b1f040870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:40:17.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.199+0000 7ffa1d052b80 1 -- 172.21.15.12:0/975488782 shutdown_connections 2024-05-17T22:40:17.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.199+0000 7ffa1d052b80 1 -- 172.21.15.12:0/975488782 wait complete. 2024-05-17T22:40:17.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.200+0000 7ffa1d052b80 1 -- start start 2024-05-17T22:40:17.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.200+0000 7ffa1d052b80 1 Processor -- start 2024-05-17T22:40:17.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.200+0000 7ffa1d052b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b1eebfa70 0x562b1f04f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:17.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.201+0000 7ffa1d052b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b1ef1f850 0x562b1f04c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:17.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.201+0000 7ffa1d052b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562b1f03db80 0x562b1f04c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:17.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.201+0000 7ffa1d052b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ffa081bd8d0 con 0x562b1eebfa70 2024-05-17T22:40:17.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.201+0000 7ffa1d052b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffa081bed40 con 0x562b1ef1f850 2024-05-17T22:40:17.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.201+0000 7ffa1d052b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffa081bdfa0 con 0x562b1f03db80 2024-05-17T22:40:17.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.201+0000 7ffa1d052b80 10 client.0 _ll_register_callbacks cb 0x562b1eef2bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:40:17.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.201+0000 7ffa07fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b1eebfa70 0x562b1f04f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:17.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.201+0000 7ffa07fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b1eebfa70 0x562b1f04f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60028/0 (socket says 172.21.15.12:60028) 2024-05-17T22:40:17.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.201+0000 7ffa07fff700 1 -- 172.21.15.12:0/4052556340 learned_addr learned my addr 172.21.15.12:0/4052556340 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:40:17.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.201+0000 7ffa0cc14700 1 --2- 172.21.15.12:0/4052556340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562b1f03db80 0x562b1f04c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:17.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.201+0000 7ffa077fe700 1 --2- 172.21.15.12:0/4052556340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b1ef1f850 0x562b1f04c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:17.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.201+0000 7ffa07fff700 1 -- 172.21.15.12:0/4052556340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562b1f03db80 msgr2=0x562b1f04c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:40:17.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.201+0000 7ffa07fff700 1 --2- 172.21.15.12:0/4052556340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562b1f03db80 0x562b1f04c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:17.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.201+0000 7ffa07fff700 1 -- 172.21.15.12:0/4052556340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b1ef1f850 msgr2=0x562b1f04c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:40:17.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.201+0000 7ffa07fff700 1 --2- 172.21.15.12:0/4052556340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b1ef1f850 0x562b1f04c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:17.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.201+0000 7ffa07fff700 1 -- 172.21.15.12:0/4052556340 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b1f04ce50 con 0x562b1eebfa70 2024-05-17T22:40:17.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.201+0000 7ffa0cc14700 1 --2- 172.21.15.12:0/4052556340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562b1f03db80 0x562b1f04c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:40:17.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.201+0000 7ffa07fff700 1 --2- 172.21.15.12:0/4052556340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b1eebfa70 0x562b1f04f760 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8000c00 tx=0x7ff9f80ad3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7995b0083e5b00eb server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:40:17.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.201+0000 7ff9e77fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:40:17.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.202+0000 7ff9e77fe700 1 -- 172.21.15.12:0/4052556340 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f80a92a0 con 0x562b1eebfa70 2024-05-17T22:40:17.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.202+0000 7ffa1d052b80 1 -- 172.21.15.12:0/4052556340 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9f80c3070 con 0x562b1eebfa70 2024-05-17T22:40:17.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.202+0000 7ff9e77fe700 1 -- 172.21.15.12:0/4052556340 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7ff9f80a9440 con 0x562b1eebfa70 2024-05-17T22:40:17.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.202+0000 7ff9e77fe700 1 -- 172.21.15.12:0/4052556340 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff9f80b1a10 con 0x562b1eebfa70 2024-05-17T22:40:17.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.202+0000 7ffa1d052b80 10 client.15468 fetch_fsmap learned FSMap version 29 2024-05-17T22:40:17.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.202+0000 7ffa1d052b80 1 -- 172.21.15.12:0/4052556340 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562b1f0c2680 con 0x562b1eebfa70 2024-05-17T22:40:17.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.202+0000 7ff9e77fe700 1 -- 172.21.15.12:0/4052556340 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9f80c3070 con 0x562b1eebfa70 2024-05-17T22:40:17.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.202+0000 7ff9e77fe700 1 -- 172.21.15.12:0/4052556340 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff9f80cb040 con 0x562b1eebfa70 2024-05-17T22:40:17.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.202+0000 7ffa1d052b80 10 client.15468 fetch_fsmap finished waiting for FSMap version 29 2024-05-17T22:40:17.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.202+0000 7ffa1d052b80 10 client.15468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:40:17.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.202+0000 7ffa1d052b80 20 client.15468 populate_metadata read hostname 'smithi012' 2024-05-17T22:40:17.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.203+0000 7ffa1d052b80 1 --2- 172.21.15.12:0/4052556340 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x562b1f043480 0x562b1f0c4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:17.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.203+0000 7ffa1d052b80 4 client.15468 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-17T22:40:17.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.203+0000 7ffa1d052b80 1 -- 172.21.15.12:0/4052556340 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562b1ef6e0a0 con 0x562b1f043480 2024-05-17T22:40:17.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.203+0000 7ffa077fe700 1 --2- 172.21.15.12:0/4052556340 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x562b1f043480 0x562b1f0c4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:17.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.203+0000 7ff9e77fe700 10 client.15468 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:40:17.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.204+0000 7ffa077fe700 1 --2- 172.21.15.12:0/4052556340 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x562b1f043480 0x562b1f0c4d40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d13cfdf7319a3583 server_cookie=a10cc34e95a45294 in_seq=0 out_seq=0 2024-05-17T22:40:17.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.205+0000 7ff9e77fe700 1 -- 172.21.15.12:0/4052556340 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562b1ef6e0a0 con 0x562b1f043480 2024-05-17T22:40:17.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.205+0000 7ff9e77fe700 10 client.15468 handle_command_reply: tid=0 2024-05-17T22:40:17.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.205+0000 7ffa1d052b80 1 client.15468 shutdown 2024-05-17T22:40:17.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.206+0000 7ffa1d052b80 1 -- 172.21.15.12:0/4052556340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b1eebfa70 msgr2=0x562b1f04f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:17.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.206+0000 7ffa1d052b80 1 --2- 172.21.15.12:0/4052556340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b1eebfa70 0x562b1f04f760 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8000c00 tx=0x7ff9f80ad3f0 comp rx=0 tx=0).stop 2024-05-17T22:40:17.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.206+0000 7ffa1d052b80 1 -- 172.21.15.12:0/4052556340 shutdown_connections 2024-05-17T22:40:17.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.206+0000 7ffa1d052b80 1 --2- 172.21.15.12:0/4052556340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562b1f03db80 0x562b1f04c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:17.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.206+0000 7ffa1d052b80 1 --2- 172.21.15.12:0/4052556340 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x562b1f043480 0x562b1f0c4d40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:17.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.206+0000 7ffa1d052b80 1 --2- 172.21.15.12:0/4052556340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b1eebfa70 0x562b1f04f760 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:17.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.206+0000 7ffa1d052b80 1 --2- 172.21.15.12:0/4052556340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b1ef1f850 0x562b1f04c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:17.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.206+0000 7ffa1d052b80 1 -- 172.21.15.12:0/4052556340 >> 172.21.15.12:0/4052556340 conn(0x562b1f038540 msgr2=0x562b1f032190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:40:17.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.207+0000 7ff9e77fe700 0 client.15468 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:40:17.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.207+0000 7ffa1d052b80 1 -- 172.21.15.12:0/4052556340 shutdown_connections 2024-05-17T22:40:17.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.207+0000 7ffa1d052b80 1 -- 172.21.15.12:0/4052556340 wait complete. 2024-05-17T22:40:17.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.207+0000 7ffa1d052b80 20 client.15468 trim_cache size 0 max 16384 2024-05-17T22:40:17.220 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:40:17.220 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-17T22:40:17.220 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "564ffffc-9dd5-4021-95f6-d77b83a72b8e", 2024-05-17T22:40:17.220 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-17T22:40:17.220 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:40:17.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.208+0000 7ffa0e677700 1 -- 172.21.15.12:0/4046966005 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff9f00b28e0 msgr2=0x7ff9f00b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:17.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.208+0000 7ffa0e677700 1 --2- 172.21.15.12:0/4046966005 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff9f00b28e0 0x7ff9f00b4df0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc000c00 tx=0x7ff9fc00b280 comp rx=0 tx=0).stop 2024-05-17T22:40:17.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.208+0000 7ffa0e677700 1 -- 172.21.15.12:0/4046966005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa08103c20 msgr2=0x7ffa0806e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:17.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.208+0000 7ffa0e677700 1 --2- 172.21.15.12:0/4046966005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa08103c20 0x7ffa0806e5f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff9f800b550 tx=0x7ff9f800b910 comp rx=0 tx=0).stop 2024-05-17T22:40:17.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.208+0000 7ffa0e677700 1 -- 172.21.15.12:0/4046966005 shutdown_connections 2024-05-17T22:40:17.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.208+0000 7ffa0e677700 1 --2- 172.21.15.12:0/4046966005 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff9f00b28e0 0x7ff9f00b4df0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:17.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.208+0000 7ffa0e677700 1 --2- 172.21.15.12:0/4046966005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa0806f070 0x7ffa08072ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:17.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.208+0000 7ffa0e677700 1 --2- 172.21.15.12:0/4046966005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa08103c20 0x7ffa0806e5f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:17.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.208+0000 7ffa0e677700 1 --2- 172.21.15.12:0/4046966005 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa08107b30 0x7ffa0806eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:17.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.208+0000 7ffa0e677700 1 -- 172.21.15.12:0/4046966005 >> 172.21.15.12:0/4046966005 conn(0x7ffa080fe030 msgr2=0x7ffa08100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:40:17.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.209+0000 7ffa0e677700 1 -- 172.21.15.12:0/4046966005 shutdown_connections 2024-05-17T22:40:17.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:17.209+0000 7ffa0e677700 1 -- 172.21.15.12:0/4046966005 wait complete. 2024-05-17T22:40:18.018 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-17T22:40:18.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:17 smithi151 ceph-mon[36101]: pgmap v470: 129 pgs: 129 active+clean; 1001 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.3 MiB/s wr, 620 op/s 2024-05-17T22:40:18.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:17 smithi096 ceph-mon[36065]: pgmap v470: 129 pgs: 129 active+clean; 1001 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.3 MiB/s wr, 620 op/s 2024-05-17T22:40:18.344 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:17 smithi012 ceph-mon[31632]: pgmap v470: 129 pgs: 129 active+clean; 1001 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.3 MiB/s wr, 620 op/s 2024-05-17T22:40:18.443 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:40:19.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:18 smithi151 ceph-mon[36101]: scrub queued for path: / 2024-05-17T22:40:19.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:18 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:40:19.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:18 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:40:19.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:18 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:40:19.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:18 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:40:19.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:18 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:40:19.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:18 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:40:19.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:18 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:40:19.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:18 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:40:19.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:18 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:40:19.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:18 smithi096 ceph-mon[36065]: scrub queued for path: / 2024-05-17T22:40:19.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:18 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:40:19.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:18 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:40:19.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:18 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:40:19.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:18 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:40:19.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:18 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:40:19.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:18 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:40:19.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:18 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:40:19.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:18 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:40:19.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:18 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:40:19.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:18 smithi012 ceph-mon[31632]: scrub queued for path: / 2024-05-17T22:40:19.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:18 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:40:19.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:18 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:40:19.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:18 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:40:19.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:18 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:40:19.343 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:18 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:40:19.343 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:18 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:40:19.343 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:18 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:40:19.343 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:18 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:40:19.343 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:18 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:40:20.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:19 smithi012 ceph-mon[31632]: pgmap v471: 129 pgs: 129 active+clean; 1018 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 17 MiB/s wr, 1.04k op/s 2024-05-17T22:40:20.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:19 smithi151 ceph-mon[36101]: pgmap v471: 129 pgs: 129 active+clean; 1018 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 17 MiB/s wr, 1.04k op/s 2024-05-17T22:40:20.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:19 smithi096 ceph-mon[36065]: pgmap v471: 129 pgs: 129 active+clean; 1018 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 17 MiB/s wr, 1.04k op/s 2024-05-17T22:40:20.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.639+0000 7fa8568f5700 1 -- 172.21.15.12:0/607593715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa850107730 msgr2=0x7fa850107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:20.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.639+0000 7fa8568f5700 1 --2- 172.21.15.12:0/607593715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa850107730 0x7fa850107b40 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa84000faf0 tx=0x7fa84000fe00 comp rx=0 tx=0).stop 2024-05-17T22:40:20.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.640+0000 7fa8568f5700 1 -- 172.21.15.12:0/607593715 shutdown_connections 2024-05-17T22:40:20.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.640+0000 7fa8568f5700 1 --2- 172.21.15.12:0/607593715 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa850108080 0x7fa85010f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:20.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.640+0000 7fa8568f5700 1 --2- 172.21.15.12:0/607593715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa850102e00 0x7fa850103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:20.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.640+0000 7fa8568f5700 1 --2- 172.21.15.12:0/607593715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa850107730 0x7fa850107b40 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:20.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.640+0000 7fa8568f5700 1 -- 172.21.15.12:0/607593715 >> 172.21.15.12:0/607593715 conn(0x7fa850079bd0 msgr2=0x7fa850079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:40:20.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.643+0000 7fa8568f5700 1 -- 172.21.15.12:0/607593715 shutdown_connections 2024-05-17T22:40:20.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.645+0000 7fa8568f5700 1 -- 172.21.15.12:0/607593715 wait complete. 2024-05-17T22:40:20.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.646+0000 7fa8568f5700 1 Processor -- start 2024-05-17T22:40:20.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.647+0000 7fa8568f5700 1 -- start start 2024-05-17T22:40:20.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.648+0000 7fa8568f5700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa850102e00 0x7fa850075430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:20.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.648+0000 7fa8568f5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa850107730 0x7fa85006f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:20.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.648+0000 7fa8568f5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa850108080 0x7fa85006fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:20.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.648+0000 7fa8568f5700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa85006db10 con 0x7fa850108080 2024-05-17T22:40:20.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.648+0000 7fa8568f5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa85006d990 con 0x7fa850107730 2024-05-17T22:40:20.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.648+0000 7fa8568f5700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa85006dc90 con 0x7fa850102e00 2024-05-17T22:40:20.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.648+0000 7fa854e92700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa850108080 0x7fa85006fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:20.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.648+0000 7fa854e92700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa850108080 0x7fa85006fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49156/0 (socket says 172.21.15.12:49156) 2024-05-17T22:40:20.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.648+0000 7fa854e92700 1 -- 172.21.15.12:0/542497887 learned_addr learned my addr 172.21.15.12:0/542497887 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:40:20.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.649+0000 7fa854e92700 1 -- 172.21.15.12:0/542497887 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa850102e00 msgr2=0x7fa850075430 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-17T22:40:20.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.649+0000 7fa854e92700 1 --2- 172.21.15.12:0/542497887 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa850102e00 0x7fa850075430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:20.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.649+0000 7fa854e92700 1 -- 172.21.15.12:0/542497887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa850107730 msgr2=0x7fa85006f550 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-17T22:40:20.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.649+0000 7fa854e92700 1 --2- 172.21.15.12:0/542497887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa850107730 0x7fa85006f550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:20.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.649+0000 7fa854e92700 1 -- 172.21.15.12:0/542497887 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8501be160 con 0x7fa850108080 2024-05-17T22:40:20.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.650+0000 7fa854e92700 1 --2- 172.21.15.12:0/542497887 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa850108080 0x7fa85006fb80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa848007ee0 tx=0x7fa84800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:40:20.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.650+0000 7fa84d7fa700 1 -- 172.21.15.12:0/542497887 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84800dbd0 con 0x7fa850108080 2024-05-17T22:40:20.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.650+0000 7fa84d7fa700 1 -- 172.21.15.12:0/542497887 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa84800dd70 con 0x7fa850108080 2024-05-17T22:40:20.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.650+0000 7fa84d7fa700 1 -- 172.21.15.12:0/542497887 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa848005990 con 0x7fa850108080 2024-05-17T22:40:20.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.651+0000 7fa8568f5700 1 -- 172.21.15.12:0/542497887 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8501be330 con 0x7fa850108080 2024-05-17T22:40:20.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.651+0000 7fa8568f5700 1 -- 172.21.15.12:0/542497887 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8501be790 con 0x7fa850108080 2024-05-17T22:40:20.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.656+0000 7fa84d7fa700 1 -- 172.21.15.12:0/542497887 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa84801a020 con 0x7fa850108080 2024-05-17T22:40:20.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.661+0000 7fa84d7fa700 1 --2- 172.21.15.12:0/542497887 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa8380b2a90 0x7fa8380b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:20.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.661+0000 7fa84ffff700 1 --2- 172.21.15.12:0/542497887 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa8380b2a90 0x7fa8380b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:20.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.662+0000 7fa8652d0b80 1 Processor -- start 2024-05-17T22:40:20.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.662+0000 7fa84ffff700 1 --2- 172.21.15.12:0/542497887 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa8380b2a90 0x7fa8380b4fa0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa84400ba70 tx=0x7fa84400b450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:40:20.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.662+0000 7fa84d7fa700 1 -- 172.21.15.12:0/542497887 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fa8480e1d10 con 0x7fa850108080 2024-05-17T22:40:20.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.662+0000 7fa8652d0b80 1 -- start start 2024-05-17T22:40:20.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.663+0000 7fa8652d0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56110f57c9f0 0x56110f6ffec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:20.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.663+0000 7fa8652d0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56110f700400 0x56110f6fa5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:20.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.663+0000 7fa8652d0b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56110f6fab00 0x56110f6faf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:20.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.663+0000 7fa8652d0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56110f6290d0 con 0x56110f700400 2024-05-17T22:40:20.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.664+0000 7fa84ffff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56110f57c9f0 0x56110f6ffec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:20.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.664+0000 7fa84ffff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56110f57c9f0 0x56110f6ffec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:48544/0 (socket says 172.21.15.12:48544) 2024-05-17T22:40:20.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.664+0000 7fa84ffff700 1 -- 172.21.15.12:0/3040896270 learned_addr learned my addr 172.21.15.12:0/3040896270 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:40:20.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.664+0000 7fa8652d0b80 1 -- 172.21.15.12:0/3040896270 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56110f618c70 con 0x56110f6fab00 2024-05-17T22:40:20.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.664+0000 7fa8652d0b80 1 -- 172.21.15.12:0/3040896270 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56110f61ccb0 con 0x56110f57c9f0 2024-05-17T22:40:20.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.664+0000 7fa84f7fe700 1 --2- 172.21.15.12:0/3040896270 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56110f700400 0x56110f6fa5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:20.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.664+0000 7fa854e92700 1 --2- 172.21.15.12:0/3040896270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56110f6fab00 0x56110f6faf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:20.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.665+0000 7fa84ffff700 1 -- 172.21.15.12:0/3040896270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56110f6fab00 msgr2=0x56110f6faf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:40:20.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.665+0000 7fa84ffff700 1 --2- 172.21.15.12:0/3040896270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56110f6fab00 0x56110f6faf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:20.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.665+0000 7fa84ffff700 1 -- 172.21.15.12:0/3040896270 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56110f700400 msgr2=0x56110f6fa5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:40:20.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.665+0000 7fa84ffff700 1 --2- 172.21.15.12:0/3040896270 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56110f700400 0x56110f6fa5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:20.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.665+0000 7fa84ffff700 1 -- 172.21.15.12:0/3040896270 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56110f5b25c0 con 0x56110f57c9f0 2024-05-17T22:40:20.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.665+0000 7fa84ffff700 1 --2- 172.21.15.12:0/3040896270 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56110f57c9f0 0x56110f6ffec0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa84400ab70 tx=0x7fa844010000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e55b143967b0c0df server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:40:20.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.665+0000 7fa8367fc700 1 -- 172.21.15.12:0/3040896270 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa844027070 con 0x56110f57c9f0 2024-05-17T22:40:20.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.666+0000 7fa8367fc700 1 -- 172.21.15.12:0/3040896270 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa844017ce0 con 0x56110f57c9f0 2024-05-17T22:40:20.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.666+0000 7fa8367fc700 1 -- 172.21.15.12:0/3040896270 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa844022b50 con 0x56110f57c9f0 2024-05-17T22:40:20.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.667+0000 7fa8652d0b80 1 -- 172.21.15.12:0/3040896270 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56110f57c9f0 msgr2=0x56110f6ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:20.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.667+0000 7fa8652d0b80 1 --2- 172.21.15.12:0/3040896270 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56110f57c9f0 0x56110f6ffec0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa84400ab70 tx=0x7fa844010000 comp rx=0 tx=0).stop 2024-05-17T22:40:20.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.668+0000 7fa8652d0b80 1 -- 172.21.15.12:0/3040896270 shutdown_connections 2024-05-17T22:40:20.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.668+0000 7fa8652d0b80 1 --2- 172.21.15.12:0/3040896270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56110f6fab00 0x56110f6faf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:20.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.668+0000 7fa8652d0b80 1 --2- 172.21.15.12:0/3040896270 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56110f57c9f0 0x56110f6ffec0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:20.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.668+0000 7fa8652d0b80 1 --2- 172.21.15.12:0/3040896270 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56110f700400 0x56110f6fa5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:20.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.668+0000 7fa8652d0b80 1 -- 172.21.15.12:0/3040896270 >> 172.21.15.12:0/3040896270 conn(0x56110f5dc7d0 msgr2=0x56110f6f51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:40:20.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.668+0000 7fa8652d0b80 1 -- 172.21.15.12:0/3040896270 shutdown_connections 2024-05-17T22:40:20.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.669+0000 7fa8652d0b80 1 -- 172.21.15.12:0/3040896270 wait complete. 2024-05-17T22:40:20.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.669+0000 7fa8652d0b80 1 -- start start 2024-05-17T22:40:20.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.669+0000 7fa8652d0b80 1 Processor -- start 2024-05-17T22:40:20.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.670+0000 7fa8652d0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56110f57c9f0 0x56110f703b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:20.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.670+0000 7fa8652d0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56110f6fab00 0x56110f704040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:20.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.670+0000 7fa8652d0b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56110f700400 0x56110f710100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:20.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.670+0000 7fa8652d0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56110f520150 con 0x56110f6fab00 2024-05-17T22:40:20.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.671+0000 7fa8652d0b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56110f5ba940 con 0x56110f700400 2024-05-17T22:40:20.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.671+0000 7fa8652d0b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56110f706dd0 con 0x56110f57c9f0 2024-05-17T22:40:20.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.671+0000 7fa8652d0b80 10 client.0 _ll_register_callbacks cb 0x56110f5afb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:40:20.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.671+0000 7fa84f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56110f6fab00 0x56110f704040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:20.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.671+0000 7fa84f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56110f6fab00 0x56110f704040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49174/0 (socket says 172.21.15.12:49174) 2024-05-17T22:40:20.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.671+0000 7fa84f7fe700 1 -- 172.21.15.12:0/3558770741 learned_addr learned my addr 172.21.15.12:0/3558770741 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:40:20.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.672+0000 7fa84f7fe700 1 -- 172.21.15.12:0/3558770741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56110f57c9f0 msgr2=0x56110f703b00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T22:40:20.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.672+0000 7fa84f7fe700 1 --2- 172.21.15.12:0/3558770741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56110f57c9f0 0x56110f703b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:20.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.672+0000 7fa84f7fe700 1 -- 172.21.15.12:0/3558770741 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56110f700400 msgr2=0x56110f710100 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T22:40:20.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.672+0000 7fa84f7fe700 1 --2- 172.21.15.12:0/3558770741 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56110f700400 0x56110f710100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:20.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.672+0000 7fa84f7fe700 1 -- 172.21.15.12:0/3558770741 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56110f569480 con 0x56110f6fab00 2024-05-17T22:40:20.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.672+0000 7fa84f7fe700 1 --2- 172.21.15.12:0/3558770741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56110f6fab00 0x56110f704040 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa84000faf0 tx=0x7fa84000f320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2772b43e9964c154 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:40:20.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.672+0000 7fa82f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:40:20.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.673+0000 7fa82f7fe700 1 -- 172.21.15.12:0/3558770741 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa840025060 con 0x56110f6fab00 2024-05-17T22:40:20.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.673+0000 7fa82f7fe700 1 -- 172.21.15.12:0/3558770741 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fa8400052e0 con 0x56110f6fab00 2024-05-17T22:40:20.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.673+0000 7fa8652d0b80 1 -- 172.21.15.12:0/3558770741 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa844017ce0 con 0x56110f6fab00 2024-05-17T22:40:20.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.674+0000 7fa82f7fe700 1 -- 172.21.15.12:0/3558770741 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa84002c050 con 0x56110f6fab00 2024-05-17T22:40:20.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.674+0000 7fa8652d0b80 10 client.15480 fetch_fsmap learned FSMap version 29 2024-05-17T22:40:20.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.674+0000 7fa8652d0b80 1 -- 172.21.15.12:0/3558770741 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56110f711940 con 0x56110f6fab00 2024-05-17T22:40:20.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.678+0000 7fa82f7fe700 1 -- 172.21.15.12:0/3558770741 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa844017ce0 con 0x56110f6fab00 2024-05-17T22:40:20.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.679+0000 7fa82f7fe700 1 -- 172.21.15.12:0/3558770741 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa840019030 con 0x56110f6fab00 2024-05-17T22:40:20.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.679+0000 7fa8652d0b80 10 client.15480 fetch_fsmap finished waiting for FSMap version 29 2024-05-17T22:40:20.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.679+0000 7fa8652d0b80 10 client.15480 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:40:20.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.679+0000 7fa8652d0b80 20 client.15480 populate_metadata read hostname 'smithi012' 2024-05-17T22:40:20.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.679+0000 7fa8652d0b80 1 --2- 172.21.15.12:0/3558770741 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x56110f713f20 0x56110f716360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:20.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.679+0000 7fa8652d0b80 4 client.15480 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:40:20.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.679+0000 7fa8652d0b80 1 -- 172.21.15.12:0/3558770741 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56110f62b020 con 0x56110f713f20 2024-05-17T22:40:20.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.681+0000 7fa84ffff700 1 --2- 172.21.15.12:0/3558770741 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x56110f713f20 0x56110f716360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:20.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.683+0000 7fa82f7fe700 10 client.15480 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:40:20.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.683+0000 7fa84ffff700 1 --2- 172.21.15.12:0/3558770741 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x56110f713f20 0x56110f716360 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e0edad60a4c663 server_cookie=20b72b25b0074e32 in_seq=0 out_seq=0 2024-05-17T22:40:20.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.684+0000 7fa82f7fe700 1 -- 172.21.15.12:0/3558770741 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56110f62b020 con 0x56110f713f20 2024-05-17T22:40:20.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.684+0000 7fa82f7fe700 10 client.15480 handle_command_reply: tid=0 2024-05-17T22:40:20.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.684+0000 7fa8652d0b80 1 client.15480 shutdown 2024-05-17T22:40:20.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.685+0000 7fa8652d0b80 1 -- 172.21.15.12:0/3558770741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56110f6fab00 msgr2=0x56110f704040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:20.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.685+0000 7fa8652d0b80 1 --2- 172.21.15.12:0/3558770741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56110f6fab00 0x56110f704040 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa84000faf0 tx=0x7fa84000f320 comp rx=0 tx=0).stop 2024-05-17T22:40:20.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.685+0000 7fa8652d0b80 1 -- 172.21.15.12:0/3558770741 shutdown_connections 2024-05-17T22:40:20.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.685+0000 7fa8652d0b80 1 --2- 172.21.15.12:0/3558770741 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56110f700400 0x56110f710100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:20.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.685+0000 7fa8652d0b80 1 --2- 172.21.15.12:0/3558770741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56110f57c9f0 0x56110f703b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:20.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.685+0000 7fa8652d0b80 1 --2- 172.21.15.12:0/3558770741 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x56110f713f20 0x56110f716360 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:20.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.685+0000 7fa8652d0b80 1 --2- 172.21.15.12:0/3558770741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56110f6fab00 0x56110f704040 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:20.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.685+0000 7fa8652d0b80 1 -- 172.21.15.12:0/3558770741 >> 172.21.15.12:0/3558770741 conn(0x56110f5dc7d0 msgr2=0x56110f6f5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:40:20.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.686+0000 7fa82f7fe700 0 client.15480 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:40:20.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.686+0000 7fa8652d0b80 1 -- 172.21.15.12:0/3558770741 shutdown_connections 2024-05-17T22:40:20.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.686+0000 7fa8652d0b80 1 -- 172.21.15.12:0/3558770741 wait complete. 2024-05-17T22:40:20.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.687+0000 7fa8652d0b80 20 client.15480 trim_cache size 0 max 16384 2024-05-17T22:40:20.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.723+0000 7fa8652d0b80 1 Processor -- start 2024-05-17T22:40:20.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.724+0000 7fa8652d0b80 1 -- start start 2024-05-17T22:40:20.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.725+0000 7fa8652d0b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56110f57c9f0 0x56110f7165a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:20.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.725+0000 7fa8652d0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56110f6fab00 0x56110f702d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:20.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.725+0000 7fa8652d0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56110f700400 0x56110f6ef820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:20.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.725+0000 7fa8652d0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa8501c9a70 con 0x56110f700400 2024-05-17T22:40:20.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.725+0000 7fa8652d0b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa8501c7300 con 0x56110f57c9f0 2024-05-17T22:40:20.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.725+0000 7fa84f7fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56110f6fab00 0x56110f702d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:20.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.725+0000 7fa84ffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56110f57c9f0 0x56110f7165a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:20.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.725+0000 7fa84f7fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56110f6fab00 0x56110f702d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:48558/0 (socket says 172.21.15.12:48558) 2024-05-17T22:40:20.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.725+0000 7fa84f7fe700 1 -- 172.21.15.12:0/2752809883 learned_addr learned my addr 172.21.15.12:0/2752809883 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:40:20.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.725+0000 7fa8652d0b80 1 -- 172.21.15.12:0/2752809883 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa8501c5230 con 0x56110f6fab00 2024-05-17T22:40:20.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.726+0000 7fa854e92700 1 --2- 172.21.15.12:0/2752809883 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56110f700400 0x56110f6ef820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:20.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.727+0000 7fa84f7fe700 1 -- 172.21.15.12:0/2752809883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56110f57c9f0 msgr2=0x56110f7165a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:40:20.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.727+0000 7fa84f7fe700 1 --2- 172.21.15.12:0/2752809883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56110f57c9f0 0x56110f7165a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:20.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.727+0000 7fa84f7fe700 1 -- 172.21.15.12:0/2752809883 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56110f700400 msgr2=0x56110f6ef820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:40:20.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.727+0000 7fa84f7fe700 1 --2- 172.21.15.12:0/2752809883 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56110f700400 0x56110f6ef820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:20.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.727+0000 7fa84f7fe700 1 -- 172.21.15.12:0/2752809883 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56110f6ff930 con 0x56110f6fab00 2024-05-17T22:40:20.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.727+0000 7fa84f7fe700 1 --2- 172.21.15.12:0/2752809883 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56110f6fab00 0x56110f702d90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa840009350 tx=0x7fa840009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8ba39dff175c98c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:40:20.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.727+0000 7fa835ffb700 1 -- 172.21.15.12:0/2752809883 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa840025070 con 0x56110f6fab00 2024-05-17T22:40:20.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.728+0000 7fa835ffb700 1 -- 172.21.15.12:0/2752809883 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa84000ede0 con 0x56110f6fab00 2024-05-17T22:40:20.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.728+0000 7fa835ffb700 1 -- 172.21.15.12:0/2752809883 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa840003c20 con 0x56110f6fab00 2024-05-17T22:40:20.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.730+0000 7fa8652d0b80 1 -- 172.21.15.12:0/2752809883 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56110f6fab00 msgr2=0x56110f702d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:20.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.730+0000 7fa8652d0b80 1 --2- 172.21.15.12:0/2752809883 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56110f6fab00 0x56110f702d90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa840009350 tx=0x7fa840009f90 comp rx=0 tx=0).stop 2024-05-17T22:40:20.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.731+0000 7fa8652d0b80 1 -- 172.21.15.12:0/2752809883 shutdown_connections 2024-05-17T22:40:20.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.731+0000 7fa8652d0b80 1 --2- 172.21.15.12:0/2752809883 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56110f700400 0x56110f6ef820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:20.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.731+0000 7fa8652d0b80 1 --2- 172.21.15.12:0/2752809883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56110f57c9f0 0x56110f7165a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:20.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.731+0000 7fa8652d0b80 1 --2- 172.21.15.12:0/2752809883 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56110f6fab00 0x56110f702d90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:20.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.731+0000 7fa8652d0b80 1 -- 172.21.15.12:0/2752809883 >> 172.21.15.12:0/2752809883 conn(0x56110f5dc7d0 msgr2=0x56110f6ef110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:40:20.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.731+0000 7fa8652d0b80 1 -- 172.21.15.12:0/2752809883 shutdown_connections 2024-05-17T22:40:20.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.731+0000 7fa8652d0b80 1 -- 172.21.15.12:0/2752809883 wait complete. 2024-05-17T22:40:20.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.732+0000 7fa8652d0b80 1 -- start start 2024-05-17T22:40:20.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.732+0000 7fa8652d0b80 1 Processor -- start 2024-05-17T22:40:20.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.733+0000 7fa8652d0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56110f57c9f0 0x56110f70c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:20.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.733+0000 7fa8652d0b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56110f6fab00 0x56110f70c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:20.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.733+0000 7fa8652d0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56110f700400 0x56110f709030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:20.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.734+0000 7fa84f7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56110f6fab00 0x56110f70c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:20.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.734+0000 7fa84f7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56110f6fab00 0x56110f70c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:40022/0 (socket says 172.21.15.12:40022) 2024-05-17T22:40:20.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.734+0000 7fa84f7fe700 1 -- 172.21.15.12:0/588900529 learned_addr learned my addr 172.21.15.12:0/588900529 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:40:20.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.734+0000 7fa854e92700 1 --2- 172.21.15.12:0/588900529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56110f700400 0x56110f709030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:20.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.734+0000 7fa8652d0b80 1 -- 172.21.15.12:0/588900529 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa8501c57c0 con 0x56110f57c9f0 2024-05-17T22:40:20.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.734+0000 7fa8652d0b80 1 -- 172.21.15.12:0/588900529 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa8501c6c30 con 0x56110f6fab00 2024-05-17T22:40:20.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.734+0000 7fa8652d0b80 1 -- 172.21.15.12:0/588900529 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa8501c5e90 con 0x56110f700400 2024-05-17T22:40:20.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.735+0000 7fa84f7fe700 1 -- 172.21.15.12:0/588900529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56110f700400 msgr2=0x56110f709030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:40:20.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.735+0000 7fa84f7fe700 1 --2- 172.21.15.12:0/588900529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56110f700400 0x56110f709030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:20.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.735+0000 7fa84f7fe700 1 -- 172.21.15.12:0/588900529 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56110f57c9f0 msgr2=0x56110f70c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:40:20.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.735+0000 7fa84f7fe700 1 --2- 172.21.15.12:0/588900529 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56110f57c9f0 0x56110f70c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:20.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.735+0000 7fa84f7fe700 1 -- 172.21.15.12:0/588900529 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56110f709670 con 0x56110f6fab00 2024-05-17T22:40:20.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.735+0000 7fa84f7fe700 1 --2- 172.21.15.12:0/588900529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56110f6fab00 0x56110f70c5a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa840005a00 tx=0x7fa840003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d9c94bb6f5fb332 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:40:20.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.735+0000 7fa8652d0b80 10 client.0 _ll_register_callbacks cb 0x56110f5afb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:40:20.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.736+0000 7fa8652d0b80 1 -- 172.21.15.12:0/588900529 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa84000ede0 con 0x56110f6fab00 2024-05-17T22:40:20.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.736+0000 7fa82f7fe700 10 client.25151 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:40:20.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.736+0000 7fa82f7fe700 1 -- 172.21.15.12:0/588900529 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa840025050 con 0x56110f6fab00 2024-05-17T22:40:20.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.736+0000 7fa82f7fe700 1 -- 172.21.15.12:0/588900529 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fa840003e00 con 0x56110f6fab00 2024-05-17T22:40:20.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.736+0000 7fa82f7fe700 1 -- 172.21.15.12:0/588900529 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa840042050 con 0x56110f6fab00 2024-05-17T22:40:20.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.736+0000 7fa8652d0b80 10 client.25151 fetch_fsmap learned FSMap version 29 2024-05-17T22:40:20.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.736+0000 7fa8652d0b80 1 -- 172.21.15.12:0/588900529 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({fsmap=29}) -- 0x56110f77eda0 con 0x56110f6fab00 2024-05-17T22:40:20.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.737+0000 7fa82f7fe700 1 -- 172.21.15.12:0/588900529 <== mon.2 v2:172.21.15.96:3300/0 4 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa840019050 con 0x56110f6fab00 2024-05-17T22:40:20.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.737+0000 7fa8652d0b80 10 client.25151 fetch_fsmap finished waiting for FSMap version 29 2024-05-17T22:40:20.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.737+0000 7fa8652d0b80 10 client.25151 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:40:20.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.737+0000 7fa8652d0b80 20 client.25151 populate_metadata read hostname 'smithi012' 2024-05-17T22:40:20.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.737+0000 7fa8652d0b80 1 --2- 172.21.15.12:0/588900529 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x56110f77f350 0x56110f7817f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:20.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.737+0000 7fa8652d0b80 4 client.25151 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:40:20.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.737+0000 7fa8652d0b80 1 -- 172.21.15.12:0/588900529 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fa84000ede0 con 0x56110f77f350 2024-05-17T22:40:20.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.737+0000 7fa84ffff700 1 --2- 172.21.15.12:0/588900529 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x56110f77f350 0x56110f7817f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:20.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.739+0000 7fa84ffff700 1 --2- 172.21.15.12:0/588900529 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x56110f77f350 0x56110f7817f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ded7bf00d11f350 server_cookie=2d7f440bc69c3fca in_seq=0 out_seq=0 2024-05-17T22:40:20.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.739+0000 7fa82f7fe700 10 client.25151 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:40:20.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.739+0000 7fa82f7fe700 1 -- 172.21.15.12:0/588900529 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7fa84000ede0 con 0x56110f77f350 2024-05-17T22:40:20.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.739+0000 7fa82f7fe700 10 client.25151 handle_command_reply: tid=0 2024-05-17T22:40:20.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.739+0000 7fa8652d0b80 1 client.25151 shutdown 2024-05-17T22:40:20.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.740+0000 7fa8652d0b80 1 -- 172.21.15.12:0/588900529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56110f6fab00 msgr2=0x56110f70c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:20.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.740+0000 7fa8652d0b80 1 --2- 172.21.15.12:0/588900529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56110f6fab00 0x56110f70c5a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa840005a00 tx=0x7fa840003680 comp rx=0 tx=0).stop 2024-05-17T22:40:20.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.740+0000 7fa8652d0b80 1 -- 172.21.15.12:0/588900529 shutdown_connections 2024-05-17T22:40:20.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.740+0000 7fa8652d0b80 1 --2- 172.21.15.12:0/588900529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56110f700400 0x56110f709030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:20.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.740+0000 7fa8652d0b80 1 --2- 172.21.15.12:0/588900529 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x56110f77f350 0x56110f7817f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:20.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.740+0000 7fa8652d0b80 1 --2- 172.21.15.12:0/588900529 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56110f57c9f0 0x56110f70c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:20.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.740+0000 7fa8652d0b80 1 --2- 172.21.15.12:0/588900529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56110f6fab00 0x56110f70c5a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:20.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.740+0000 7fa8652d0b80 1 -- 172.21.15.12:0/588900529 >> 172.21.15.12:0/588900529 conn(0x56110f5dc7d0 msgr2=0x56110f702b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:40:20.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.740+0000 7fa82f7fe700 0 client.25151 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:40:20.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.740+0000 7fa8652d0b80 1 -- 172.21.15.12:0/588900529 shutdown_connections 2024-05-17T22:40:20.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.740+0000 7fa8652d0b80 1 -- 172.21.15.12:0/588900529 wait complete. 2024-05-17T22:40:20.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.741+0000 7fa8652d0b80 20 client.25151 trim_cache size 0 max 16384 2024-05-17T22:40:20.746 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:40:20.746 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-17T22:40:20.746 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-17T22:40:20.746 INFO:teuthology.orchestra.run.smithi012.stdout: "564ffffc-9dd5-4021-95f6-d77b83a72b8e": { 2024-05-17T22:40:20.746 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-17T22:40:20.746 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "564ffffc-9dd5-4021-95f6-d77b83a72b8e", 2024-05-17T22:40:20.746 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-17T22:40:20.746 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:40:20.747 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:40:20.747 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:40:20.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.743+0000 7fa8568f5700 1 -- 172.21.15.12:0/542497887 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa8380b2a90 msgr2=0x7fa8380b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:20.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.743+0000 7fa8568f5700 1 --2- 172.21.15.12:0/542497887 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa8380b2a90 0x7fa8380b4fa0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa84400ba70 tx=0x7fa84400b450 comp rx=0 tx=0).stop 2024-05-17T22:40:20.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.743+0000 7fa8568f5700 1 -- 172.21.15.12:0/542497887 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa850108080 msgr2=0x7fa85006fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:20.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.743+0000 7fa8568f5700 1 --2- 172.21.15.12:0/542497887 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa850108080 0x7fa85006fb80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa848007ee0 tx=0x7fa84800d3b0 comp rx=0 tx=0).stop 2024-05-17T22:40:20.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.744+0000 7fa8568f5700 1 -- 172.21.15.12:0/542497887 shutdown_connections 2024-05-17T22:40:20.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.744+0000 7fa8568f5700 1 --2- 172.21.15.12:0/542497887 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa8380b2a90 0x7fa8380b4fa0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:20.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.744+0000 7fa8568f5700 1 --2- 172.21.15.12:0/542497887 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa850108080 0x7fa85006fb80 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:20.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.744+0000 7fa8568f5700 1 --2- 172.21.15.12:0/542497887 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa850102e00 0x7fa850075430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:20.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.744+0000 7fa8568f5700 1 --2- 172.21.15.12:0/542497887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa850107730 0x7fa85006f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:20.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.744+0000 7fa8568f5700 1 -- 172.21.15.12:0/542497887 >> 172.21.15.12:0/542497887 conn(0x7fa850079bd0 msgr2=0x7fa8501064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:40:20.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.744+0000 7fa8568f5700 1 -- 172.21.15.12:0/542497887 shutdown_connections 2024-05-17T22:40:20.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:20.744+0000 7fa8568f5700 1 -- 172.21.15.12:0/542497887 wait complete. 2024-05-17T22:40:21.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 60.0 s, 2558.5 tps, lat 40.446 ms stddev 57.451 2024-05-17T22:40:21.422 INFO:tasks.cephfs.filesystem:scrub status for tag:564ffffc-9dd5-4021-95f6-d77b83a72b8e - {'path': '/', 'tag': '564ffffc-9dd5-4021-95f6-d77b83a72b8e', 'options': 'recursive,force'} 2024-05-17T22:40:22.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:21 smithi012 ceph-mon[31632]: pgmap v472: 129 pgs: 129 active+clean; 1019 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 16 MiB/s wr, 914 op/s 2024-05-17T22:40:22.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:21 smithi151 ceph-mon[36101]: pgmap v472: 129 pgs: 129 active+clean; 1019 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 16 MiB/s wr, 914 op/s 2024-05-17T22:40:22.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:21 smithi096 ceph-mon[36065]: pgmap v472: 129 pgs: 129 active+clean; 1019 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 16 MiB/s wr, 914 op/s 2024-05-17T22:40:24.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:23 smithi151 ceph-mon[36101]: pgmap v473: 129 pgs: 129 active+clean; 1020 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 17 MiB/s wr, 991 op/s 2024-05-17T22:40:24.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:23 smithi151 ceph-mon[36101]: scrub summary: idle 2024-05-17T22:40:24.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:23 smithi096 ceph-mon[36065]: pgmap v473: 129 pgs: 129 active+clean; 1020 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 17 MiB/s wr, 991 op/s 2024-05-17T22:40:24.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:23 smithi096 ceph-mon[36065]: scrub summary: idle 2024-05-17T22:40:24.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:23 smithi012 ceph-mon[31632]: pgmap v473: 129 pgs: 129 active+clean; 1020 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 17 MiB/s wr, 991 op/s 2024-05-17T22:40:24.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:23 smithi012 ceph-mon[31632]: scrub summary: idle 2024-05-17T22:40:25.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:24 smithi151 ceph-mon[36101]: scrub summary: idle 2024-05-17T22:40:25.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:24 smithi151 ceph-mon[36101]: scrub summary: idle 2024-05-17T22:40:25.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:24 smithi096 ceph-mon[36065]: scrub summary: idle 2024-05-17T22:40:25.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:24 smithi096 ceph-mon[36065]: scrub summary: idle 2024-05-17T22:40:25.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:24 smithi012 ceph-mon[31632]: scrub summary: idle 2024-05-17T22:40:25.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:24 smithi012 ceph-mon[31632]: scrub summary: idle 2024-05-17T22:40:26.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:25 smithi151 ceph-mon[36101]: pgmap v474: 129 pgs: 129 active+clean; 1020 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 17 MiB/s wr, 953 op/s 2024-05-17T22:40:26.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:25 smithi096 ceph-mon[36065]: pgmap v474: 129 pgs: 129 active+clean; 1020 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 17 MiB/s wr, 953 op/s 2024-05-17T22:40:26.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:25 smithi012 ceph-mon[31632]: pgmap v474: 129 pgs: 129 active+clean; 1020 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 17 MiB/s wr, 953 op/s 2024-05-17T22:40:26.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 65.0 s, 2153.6 tps, lat 46.711 ms stddev 52.308 2024-05-17T22:40:28.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:27 smithi151 ceph-mon[36101]: pgmap v475: 129 pgs: 129 active+clean; 1021 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 16 MiB/s wr, 850 op/s 2024-05-17T22:40:28.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:27 smithi096 ceph-mon[36065]: pgmap v475: 129 pgs: 129 active+clean; 1021 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 16 MiB/s wr, 850 op/s 2024-05-17T22:40:28.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:27 smithi012 ceph-mon[31632]: pgmap v475: 129 pgs: 129 active+clean; 1021 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 16 MiB/s wr, 850 op/s 2024-05-17T22:40:30.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:29 smithi151 ceph-mon[36101]: pgmap v476: 129 pgs: 129 active+clean; 1022 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 18 MiB/s wr, 1.06k op/s 2024-05-17T22:40:30.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:29 smithi096 ceph-mon[36065]: pgmap v476: 129 pgs: 129 active+clean; 1022 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 18 MiB/s wr, 1.06k op/s 2024-05-17T22:40:30.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:29 smithi012 ceph-mon[31632]: pgmap v476: 129 pgs: 129 active+clean; 1022 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 18 MiB/s wr, 1.06k op/s 2024-05-17T22:40:31.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 70.0 s, 2665.0 tps, lat 37.483 ms stddev 41.898 2024-05-17T22:40:32.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:31 smithi151 ceph-mon[36101]: pgmap v477: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 8.5 MiB/s wr, 688 op/s 2024-05-17T22:40:32.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:31 smithi096 ceph-mon[36065]: pgmap v477: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 8.5 MiB/s wr, 688 op/s 2024-05-17T22:40:32.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:31 smithi012 ceph-mon[31632]: pgmap v477: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 8.5 MiB/s wr, 688 op/s 2024-05-17T22:40:34.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:33 smithi151 ceph-mon[36101]: pgmap v478: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 7.4 MiB/s wr, 695 op/s 2024-05-17T22:40:34.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:33 smithi096 ceph-mon[36065]: pgmap v478: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 7.4 MiB/s wr, 695 op/s 2024-05-17T22:40:34.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:33 smithi012 ceph-mon[31632]: pgmap v478: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 7.4 MiB/s wr, 695 op/s 2024-05-17T22:40:36.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:35 smithi151 ceph-mon[36101]: pgmap v479: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 8.4 MiB/s wr, 738 op/s 2024-05-17T22:40:36.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:35 smithi096 ceph-mon[36065]: pgmap v479: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 8.4 MiB/s wr, 738 op/s 2024-05-17T22:40:36.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:35 smithi012 ceph-mon[31632]: pgmap v479: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 8.4 MiB/s wr, 738 op/s 2024-05-17T22:40:36.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 75.0 s, 2565.4 tps, lat 38.896 ms stddev 45.785 2024-05-17T22:40:38.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:37 smithi151 ceph-mon[36101]: pgmap v480: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 6.8 MiB/s wr, 577 op/s 2024-05-17T22:40:38.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:37 smithi096 ceph-mon[36065]: pgmap v480: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 6.8 MiB/s wr, 577 op/s 2024-05-17T22:40:38.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:37 smithi012 ceph-mon[31632]: pgmap v480: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 6.8 MiB/s wr, 577 op/s 2024-05-17T22:40:40.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:39 smithi151 ceph-mon[36101]: pgmap v481: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 11 MiB/s wr, 906 op/s 2024-05-17T22:40:40.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:39 smithi096 ceph-mon[36065]: pgmap v481: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 11 MiB/s wr, 906 op/s 2024-05-17T22:40:40.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:39 smithi012 ceph-mon[31632]: pgmap v481: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 11 MiB/s wr, 906 op/s 2024-05-17T22:40:41.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 80.0 s, 2546.2 tps, lat 39.362 ms stddev 46.884 2024-05-17T22:40:42.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:41 smithi151 ceph-mon[36101]: pgmap v482: 129 pgs: 129 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 777 op/s 2024-05-17T22:40:42.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:41 smithi096 ceph-mon[36065]: pgmap v482: 129 pgs: 129 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 777 op/s 2024-05-17T22:40:42.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:41 smithi012 ceph-mon[31632]: pgmap v482: 129 pgs: 129 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 777 op/s 2024-05-17T22:40:44.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:43 smithi151 ceph-mon[36101]: pgmap v483: 129 pgs: 129 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 796 op/s 2024-05-17T22:40:44.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:43 smithi096 ceph-mon[36065]: pgmap v483: 129 pgs: 129 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 796 op/s 2024-05-17T22:40:44.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:43 smithi012 ceph-mon[31632]: pgmap v483: 129 pgs: 129 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 796 op/s 2024-05-17T22:40:46.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:45 smithi151 ceph-mon[36101]: pgmap v484: 129 pgs: 129 active+clean; 1.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 12 MiB/s wr, 935 op/s 2024-05-17T22:40:46.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:45 smithi096 ceph-mon[36065]: pgmap v484: 129 pgs: 129 active+clean; 1.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 12 MiB/s wr, 935 op/s 2024-05-17T22:40:46.343 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:45 smithi012 ceph-mon[31632]: pgmap v484: 129 pgs: 129 active+clean; 1.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 12 MiB/s wr, 935 op/s 2024-05-17T22:40:46.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 85.0 s, 3183.2 tps, lat 31.328 ms stddev 38.676 2024-05-17T22:40:48.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:47 smithi151 ceph-mon[36101]: pgmap v485: 129 pgs: 129 active+clean; 1.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 11 MiB/s wr, 816 op/s 2024-05-17T22:40:48.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:47 smithi096 ceph-mon[36065]: pgmap v485: 129 pgs: 129 active+clean; 1.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 11 MiB/s wr, 816 op/s 2024-05-17T22:40:48.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:47 smithi012 ceph-mon[31632]: pgmap v485: 129 pgs: 129 active+clean; 1.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 11 MiB/s wr, 816 op/s 2024-05-17T22:40:50.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:49 smithi096 ceph-mon[36065]: pgmap v486: 129 pgs: 129 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 1.02k op/s 2024-05-17T22:40:50.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:49 smithi012 ceph-mon[31632]: pgmap v486: 129 pgs: 129 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 1.02k op/s 2024-05-17T22:40:50.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:49 smithi151 ceph-mon[36101]: pgmap v486: 129 pgs: 129 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 1.02k op/s 2024-05-17T22:40:51.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 90.0 s, 2758.8 tps, lat 36.337 ms stddev 41.727 2024-05-17T22:40:51.423 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-17T22:40:51.830 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:40:52.252 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:51 smithi012 ceph-mon[31632]: pgmap v487: 129 pgs: 129 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.7 MiB/s wr, 706 op/s 2024-05-17T22:40:52.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:51 smithi096 ceph-mon[36065]: pgmap v487: 129 pgs: 129 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.7 MiB/s wr, 706 op/s 2024-05-17T22:40:52.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:51 smithi151 ceph-mon[36101]: pgmap v487: 129 pgs: 129 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.7 MiB/s wr, 706 op/s 2024-05-17T22:40:53.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:53 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:40:53.343 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:53 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:40:53.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:53 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:40:53.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.822+0000 7f13ca963700 1 -- 172.21.15.12:0/2748354134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13c4103a50 msgr2=0x7f13c4103e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:53.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.822+0000 7f13ca963700 1 --2- 172.21.15.12:0/2748354134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13c4103a50 0x7f13c4103e60 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f13b4009990 tx=0x7f13b4009ca0 comp rx=0 tx=0).stop 2024-05-17T22:40:53.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.823+0000 7f13ca963700 1 -- 172.21.15.12:0/2748354134 shutdown_connections 2024-05-17T22:40:53.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.823+0000 7f13ca963700 1 --2- 172.21.15.12:0/2748354134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13c41043a0 0x7f13c4110920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:53.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.823+0000 7f13ca963700 1 --2- 172.21.15.12:0/2748354134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13c4109a80 0x7f13c4109e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:53.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.823+0000 7f13ca963700 1 --2- 172.21.15.12:0/2748354134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13c4103a50 0x7f13c4103e60 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:53.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.823+0000 7f13ca963700 1 -- 172.21.15.12:0/2748354134 >> 172.21.15.12:0/2748354134 conn(0x7f13c40ff780 msgr2=0x7f13c4101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:40:53.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.823+0000 7f13ca963700 1 -- 172.21.15.12:0/2748354134 shutdown_connections 2024-05-17T22:40:53.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.823+0000 7f13ca963700 1 -- 172.21.15.12:0/2748354134 wait complete. 2024-05-17T22:40:53.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.824+0000 7f13ca963700 1 Processor -- start 2024-05-17T22:40:53.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.824+0000 7f13ca963700 1 -- start start 2024-05-17T22:40:53.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.825+0000 7f13ca963700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13c4103a50 0x7f13c41983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:53.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.825+0000 7f13ca963700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13c41043a0 0x7f13c4198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:53.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.825+0000 7f13ca963700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13c4109a80 0x7f13c419cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:53.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.825+0000 7f13ca963700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f13c4111aa0 con 0x7f13c4103a50 2024-05-17T22:40:53.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.825+0000 7f13ca963700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f13c4111920 con 0x7f13c41043a0 2024-05-17T22:40:53.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.825+0000 7f13ca963700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f13c4111c20 con 0x7f13c4109a80 2024-05-17T22:40:53.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.825+0000 7f13c9961700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13c4103a50 0x7f13c41983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:53.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.825+0000 7f13c9961700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13c4103a50 0x7f13c41983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55022/0 (socket says 172.21.15.12:55022) 2024-05-17T22:40:53.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.825+0000 7f13c9961700 1 -- 172.21.15.12:0/574125271 learned_addr learned my addr 172.21.15.12:0/574125271 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:40:53.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.825+0000 7f13c9160700 1 --2- 172.21.15.12:0/574125271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13c41043a0 0x7f13c4198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:53.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.825+0000 7f13ca162700 1 --2- 172.21.15.12:0/574125271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13c4109a80 0x7f13c419cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:53.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.825+0000 7f13c9961700 1 -- 172.21.15.12:0/574125271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13c4109a80 msgr2=0x7f13c419cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:53.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.825+0000 7f13c9961700 1 --2- 172.21.15.12:0/574125271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13c4109a80 0x7f13c419cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:53.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.825+0000 7f13c9961700 1 -- 172.21.15.12:0/574125271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13c41043a0 msgr2=0x7f13c4198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:53.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.825+0000 7f13c9961700 1 --2- 172.21.15.12:0/574125271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13c41043a0 0x7f13c4198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:53.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.825+0000 7f13c9961700 1 -- 172.21.15.12:0/574125271 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c419d290 con 0x7f13c4103a50 2024-05-17T22:40:53.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.826+0000 7f13c9160700 1 --2- 172.21.15.12:0/574125271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13c41043a0 0x7f13c4198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:40:53.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.826+0000 7f13c9961700 1 --2- 172.21.15.12:0/574125271 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13c4103a50 0x7f13c41983c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f13c000e970 tx=0x7f13c000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:40:53.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.826+0000 7f13baffd700 1 -- 172.21.15.12:0/574125271 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c0004220 con 0x7f13c4103a50 2024-05-17T22:40:53.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.826+0000 7f13baffd700 1 -- 172.21.15.12:0/574125271 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13c00043c0 con 0x7f13c4103a50 2024-05-17T22:40:53.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.826+0000 7f13ca963700 1 -- 172.21.15.12:0/574125271 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13c419d580 con 0x7f13c4103a50 2024-05-17T22:40:53.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.826+0000 7f13baffd700 1 -- 172.21.15.12:0/574125271 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c0003ce0 con 0x7f13c4103a50 2024-05-17T22:40:53.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.826+0000 7f13ca963700 1 -- 172.21.15.12:0/574125271 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13c41a4ea0 con 0x7f13c4103a50 2024-05-17T22:40:53.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.831+0000 7f13baffd700 1 -- 172.21.15.12:0/574125271 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f13c000c7e0 con 0x7f13c4103a50 2024-05-17T22:40:53.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.832+0000 7f13baffd700 1 --2- 172.21.15.12:0/574125271 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f13b00b2900 0x7f13b00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:53.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.832+0000 7f13da5a0b80 1 Processor -- start 2024-05-17T22:40:53.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.832+0000 7f13baffd700 1 -- 172.21.15.12:0/574125271 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f13c00e2720 con 0x7f13c4103a50 2024-05-17T22:40:53.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.832+0000 7f13da5a0b80 1 -- start start 2024-05-17T22:40:53.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.832+0000 7f13c9160700 1 --2- 172.21.15.12:0/574125271 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f13b00b2900 0x7f13b00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:53.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.833+0000 7f13c9160700 1 --2- 172.21.15.12:0/574125271 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f13b00b2900 0x7f13b00b4e10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f13c4108550 tx=0x7f13b4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:40:53.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.833+0000 7f13da5a0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a32eeba9f0 0x55a32f03e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:53.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.833+0000 7f13da5a0b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a32f03ecb0 0x55a32f038c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:53.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.833+0000 7f13da5a0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a32f039170 0x55a32f039580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:53.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.833+0000 7f13da5a0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a32ef67290 con 0x55a32eeba9f0 2024-05-17T22:40:53.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.833+0000 7f13da5a0b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a32ef56fd0 con 0x55a32f03ecb0 2024-05-17T22:40:53.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.833+0000 7f13da5a0b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a32ef5ae70 con 0x55a32f039170 2024-05-17T22:40:53.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.833+0000 7f13c9160700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a32f03ecb0 0x55a32f038c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:53.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.833+0000 7f13c9160700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a32f03ecb0 0x55a32f038c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:59524/0 (socket says 172.21.15.12:59524) 2024-05-17T22:40:53.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.833+0000 7f13c9160700 1 -- 172.21.15.12:0/4134701996 learned_addr learned my addr 172.21.15.12:0/4134701996 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:40:53.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.834+0000 7f13c9961700 1 --2- 172.21.15.12:0/4134701996 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a32eeba9f0 0x55a32f03e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:53.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.834+0000 7f13ca162700 1 --2- 172.21.15.12:0/4134701996 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a32f039170 0x55a32f039580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:53.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.834+0000 7f13c9961700 1 -- 172.21.15.12:0/4134701996 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a32f039170 msgr2=0x55a32f039580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:40:53.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.834+0000 7f13c9961700 1 --2- 172.21.15.12:0/4134701996 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a32f039170 0x55a32f039580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:53.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.834+0000 7f13c9961700 1 -- 172.21.15.12:0/4134701996 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a32f03ecb0 msgr2=0x55a32f038c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:40:53.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.834+0000 7f13c9961700 1 --2- 172.21.15.12:0/4134701996 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a32f03ecb0 0x55a32f038c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:53.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.834+0000 7f13c9961700 1 -- 172.21.15.12:0/4134701996 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a32ef00590 con 0x55a32eeba9f0 2024-05-17T22:40:53.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.835+0000 7f13c9961700 1 --2- 172.21.15.12:0/4134701996 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a32eeba9f0 0x55a32f03e770 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f13c0005950 tx=0x7f13c00a90c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd1ff8fb07aa3f55 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:40:53.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.836+0000 7f13affff700 1 -- 172.21.15.12:0/4134701996 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c00a9460 con 0x55a32eeba9f0 2024-05-17T22:40:53.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.836+0000 7f13affff700 1 -- 172.21.15.12:0/4134701996 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13c00aa050 con 0x55a32eeba9f0 2024-05-17T22:40:53.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.837+0000 7f13da5a0b80 1 -- 172.21.15.12:0/4134701996 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a32eeba9f0 msgr2=0x55a32f03e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:53.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.837+0000 7f13da5a0b80 1 --2- 172.21.15.12:0/4134701996 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a32eeba9f0 0x55a32f03e770 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f13c0005950 tx=0x7f13c00a90c0 comp rx=0 tx=0).stop 2024-05-17T22:40:53.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.838+0000 7f13da5a0b80 1 -- 172.21.15.12:0/4134701996 shutdown_connections 2024-05-17T22:40:53.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.838+0000 7f13da5a0b80 1 --2- 172.21.15.12:0/4134701996 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a32f039170 0x55a32f039580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:53.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.838+0000 7f13da5a0b80 1 --2- 172.21.15.12:0/4134701996 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a32eeba9f0 0x55a32f03e770 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:53.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.838+0000 7f13da5a0b80 1 --2- 172.21.15.12:0/4134701996 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a32f03ecb0 0x55a32f038c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:53.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.838+0000 7f13da5a0b80 1 -- 172.21.15.12:0/4134701996 >> 172.21.15.12:0/4134701996 conn(0x55a32eefef00 msgr2=0x55a32f033b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:40:53.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.839+0000 7f13da5a0b80 1 -- 172.21.15.12:0/4134701996 shutdown_connections 2024-05-17T22:40:53.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.839+0000 7f13da5a0b80 1 -- 172.21.15.12:0/4134701996 wait complete. 2024-05-17T22:40:53.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.839+0000 7f13da5a0b80 1 -- start start 2024-05-17T22:40:53.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.840+0000 7f13da5a0b80 1 Processor -- start 2024-05-17T22:40:53.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.840+0000 7f13da5a0b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a32eeba9f0 0x55a32f044e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:53.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.840+0000 7f13da5a0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a32f039170 0x55a32f042170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:53.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.840+0000 7f13da5a0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a32f03ecb0 0x55a32f042770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:53.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.840+0000 7f13da5a0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a32ee5e150 con 0x55a32f03ecb0 2024-05-17T22:40:53.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.840+0000 7f13da5a0b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a32eef7e80 con 0x55a32eeba9f0 2024-05-17T22:40:53.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.840+0000 7f13da5a0b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a32f045640 con 0x55a32f039170 2024-05-17T22:40:53.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.840+0000 7f13c9961700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a32eeba9f0 0x55a32f044e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:53.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.840+0000 7f13da5a0b80 10 client.0 _ll_register_callbacks cb 0x55a32eef1f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:40:53.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.840+0000 7f13ca162700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a32f03ecb0 0x55a32f042770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:53.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.840+0000 7f13ca162700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a32f03ecb0 0x55a32f042770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55040/0 (socket says 172.21.15.12:55040) 2024-05-17T22:40:53.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.840+0000 7f13ca162700 1 -- 172.21.15.12:0/3303463587 learned_addr learned my addr 172.21.15.12:0/3303463587 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:40:53.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.840+0000 7f13c9160700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a32f039170 0x55a32f042170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:53.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.840+0000 7f13c9961700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a32eeba9f0 0x55a32f044e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:59528/0 (socket says 172.21.15.12:59528) 2024-05-17T22:40:53.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.841+0000 7f13c9160700 1 -- 172.21.15.12:0/3303463587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a32eeba9f0 msgr2=0x55a32f044e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:40:53.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.841+0000 7f13c9160700 1 --2- 172.21.15.12:0/3303463587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a32eeba9f0 0x55a32f044e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:53.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.841+0000 7f13c9160700 1 -- 172.21.15.12:0/3303463587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a32f03ecb0 msgr2=0x55a32f042770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:40:53.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.841+0000 7f13c9160700 1 --2- 172.21.15.12:0/3303463587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a32f03ecb0 0x55a32f042770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:53.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.841+0000 7f13c9160700 1 -- 172.21.15.12:0/3303463587 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a32eea7480 con 0x55a32f039170 2024-05-17T22:40:53.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.841+0000 7f13c9160700 1 --2- 172.21.15.12:0/3303463587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a32f039170 0x55a32f042170 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f13b400b2e0 tx=0x7f13b4009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cf390afa17252a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:40:53.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.841+0000 7f13ad7fa700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:40:53.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.841+0000 7f13ad7fa700 1 -- 172.21.15.12:0/3303463587 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b4034060 con 0x55a32f039170 2024-05-17T22:40:53.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.841+0000 7f13ad7fa700 1 -- 172.21.15.12:0/3303463587 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f13b400f870 con 0x55a32f039170 2024-05-17T22:40:53.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.841+0000 7f13da5a0b80 1 -- 172.21.15.12:0/3303463587 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13c00aa050 con 0x55a32f039170 2024-05-17T22:40:53.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.842+0000 7f13ad7fa700 1 -- 172.21.15.12:0/3303463587 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f13b403b050 con 0x55a32f039170 2024-05-17T22:40:53.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.842+0000 7f13da5a0b80 10 client.25114 fetch_fsmap learned FSMap version 29 2024-05-17T22:40:53.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.842+0000 7f13da5a0b80 1 -- 172.21.15.12:0/3303463587 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a32f04fc70 con 0x55a32f039170 2024-05-17T22:40:53.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.842+0000 7f13ad7fa700 1 -- 172.21.15.12:0/3303463587 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13c00aa050 con 0x55a32f039170 2024-05-17T22:40:53.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.843+0000 7f13ad7fa700 1 -- 172.21.15.12:0/3303463587 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f13b4022070 con 0x55a32f039170 2024-05-17T22:40:53.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.843+0000 7f13da5a0b80 10 client.25114 fetch_fsmap finished waiting for FSMap version 29 2024-05-17T22:40:53.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.843+0000 7f13da5a0b80 10 client.25114 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:40:53.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.843+0000 7f13da5a0b80 20 client.25114 populate_metadata read hostname 'smithi012' 2024-05-17T22:40:53.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.843+0000 7f13da5a0b80 1 --2- 172.21.15.12:0/3303463587 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55a32f0522b0 0x55a32f0546d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:53.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.843+0000 7f13da5a0b80 4 client.25114 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:40:53.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.843+0000 7f13da5a0b80 1 -- 172.21.15.12:0/3303463587 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a32ef691e0 con 0x55a32f0522b0 2024-05-17T22:40:53.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.843+0000 7f13c9961700 1 --2- 172.21.15.12:0/3303463587 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55a32f0522b0 0x55a32f0546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:53.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.844+0000 7f13ad7fa700 10 client.25114 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:40:53.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.844+0000 7f13c9961700 1 --2- 172.21.15.12:0/3303463587 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55a32f0522b0 0x55a32f0546d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4eebfa1c9134394a server_cookie=89e9510141e7464b in_seq=0 out_seq=0 2024-05-17T22:40:53.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.845+0000 7f13ad7fa700 1 -- 172.21.15.12:0/3303463587 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a32ef691e0 con 0x55a32f0522b0 2024-05-17T22:40:53.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.845+0000 7f13ad7fa700 10 client.25114 handle_command_reply: tid=0 2024-05-17T22:40:53.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.845+0000 7f13da5a0b80 1 client.25114 shutdown 2024-05-17T22:40:53.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.845+0000 7f13da5a0b80 1 -- 172.21.15.12:0/3303463587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a32f039170 msgr2=0x55a32f042170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:53.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.845+0000 7f13da5a0b80 1 --2- 172.21.15.12:0/3303463587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a32f039170 0x55a32f042170 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f13b400b2e0 tx=0x7f13b4009ca0 comp rx=0 tx=0).stop 2024-05-17T22:40:53.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.845+0000 7f13da5a0b80 1 -- 172.21.15.12:0/3303463587 shutdown_connections 2024-05-17T22:40:53.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.845+0000 7f13da5a0b80 1 --2- 172.21.15.12:0/3303463587 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55a32f0522b0 0x55a32f0546d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:53.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.845+0000 7f13da5a0b80 1 --2- 172.21.15.12:0/3303463587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a32f03ecb0 0x55a32f042770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:53.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.845+0000 7f13ad7fa700 0 client.25114 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:40:53.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.845+0000 7f13da5a0b80 1 --2- 172.21.15.12:0/3303463587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a32eeba9f0 0x55a32f044e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:53.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.845+0000 7f13da5a0b80 1 --2- 172.21.15.12:0/3303463587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a32f039170 0x55a32f042170 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:53.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.845+0000 7f13da5a0b80 1 -- 172.21.15.12:0/3303463587 >> 172.21.15.12:0/3303463587 conn(0x55a32eefef00 msgr2=0x55a32f033110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:40:53.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.846+0000 7f13da5a0b80 1 -- 172.21.15.12:0/3303463587 shutdown_connections 2024-05-17T22:40:53.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.846+0000 7f13da5a0b80 1 -- 172.21.15.12:0/3303463587 wait complete. 2024-05-17T22:40:53.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.846+0000 7f13da5a0b80 20 client.25114 trim_cache size 0 max 16384 2024-05-17T22:40:53.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.879+0000 7f13da5a0b80 1 Processor -- start 2024-05-17T22:40:53.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.879+0000 7f13da5a0b80 1 -- start start 2024-05-17T22:40:53.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.880+0000 7f13da5a0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a32eeba9f0 0x55a32f038c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:53.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.880+0000 7f13da5a0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a32f039170 0x55a32f044e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:53.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.880+0000 7f13da5a0b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a32f03ecb0 0x55a32f03c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:53.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.880+0000 7f13da5a0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f13c41b0330 con 0x55a32f039170 2024-05-17T22:40:53.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.880+0000 7f13da5a0b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f13c41adbc0 con 0x55a32f03ecb0 2024-05-17T22:40:53.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.880+0000 7f13da5a0b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f13c41aba10 con 0x55a32eeba9f0 2024-05-17T22:40:53.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.880+0000 7f13c9160700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a32f039170 0x55a32f044e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:53.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.880+0000 7f13c9160700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a32f039170 0x55a32f044e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55044/0 (socket says 172.21.15.12:55044) 2024-05-17T22:40:53.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.880+0000 7f13c9160700 1 -- 172.21.15.12:0/1509759681 learned_addr learned my addr 172.21.15.12:0/1509759681 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:40:53.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.880+0000 7f13ca162700 1 --2- 172.21.15.12:0/1509759681 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a32f03ecb0 0x55a32f03c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:53.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.880+0000 7f13c9961700 1 --2- 172.21.15.12:0/1509759681 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a32eeba9f0 0x55a32f038c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:53.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.880+0000 7f13c9160700 1 -- 172.21.15.12:0/1509759681 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a32eeba9f0 msgr2=0x55a32f038c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:40:53.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.881+0000 7f13c9160700 1 --2- 172.21.15.12:0/1509759681 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a32eeba9f0 0x55a32f038c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:53.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.881+0000 7f13c9160700 1 -- 172.21.15.12:0/1509759681 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a32f03ecb0 msgr2=0x55a32f03c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:40:53.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.881+0000 7f13c9160700 1 --2- 172.21.15.12:0/1509759681 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a32f03ecb0 0x55a32f03c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:53.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.881+0000 7f13c9160700 1 -- 172.21.15.12:0/1509759681 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a32f03eac0 con 0x55a32f039170 2024-05-17T22:40:53.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.881+0000 7f13c9961700 1 --2- 172.21.15.12:0/1509759681 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a32eeba9f0 0x55a32f038c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:40:53.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.881+0000 7f13c9160700 1 --2- 172.21.15.12:0/1509759681 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a32f039170 0x55a32f044e40 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f13b402fbc0 tx=0x7f13b4004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=837a06c5f2bf99a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:40:53.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.881+0000 7f13af7fe700 1 -- 172.21.15.12:0/1509759681 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b4044070 con 0x55a32f039170 2024-05-17T22:40:53.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.881+0000 7f13af7fe700 1 -- 172.21.15.12:0/1509759681 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13b400f670 con 0x55a32f039170 2024-05-17T22:40:53.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.881+0000 7f13af7fe700 1 -- 172.21.15.12:0/1509759681 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b4034040 con 0x55a32f039170 2024-05-17T22:40:53.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.882+0000 7f13da5a0b80 1 -- 172.21.15.12:0/1509759681 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a32f039170 msgr2=0x55a32f044e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:53.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.882+0000 7f13da5a0b80 1 --2- 172.21.15.12:0/1509759681 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a32f039170 0x55a32f044e40 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f13b402fbc0 tx=0x7f13b4004f80 comp rx=0 tx=0).stop 2024-05-17T22:40:53.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.882+0000 7f13da5a0b80 1 -- 172.21.15.12:0/1509759681 shutdown_connections 2024-05-17T22:40:53.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.882+0000 7f13da5a0b80 1 --2- 172.21.15.12:0/1509759681 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a32f03ecb0 0x55a32f03c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:53.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.882+0000 7f13da5a0b80 1 --2- 172.21.15.12:0/1509759681 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a32eeba9f0 0x55a32f038c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:53.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.882+0000 7f13da5a0b80 1 --2- 172.21.15.12:0/1509759681 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a32f039170 0x55a32f044e40 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:53.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.882+0000 7f13da5a0b80 1 -- 172.21.15.12:0/1509759681 >> 172.21.15.12:0/1509759681 conn(0x55a32eefef00 msgr2=0x55a32f033ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:40:53.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.883+0000 7f13da5a0b80 1 -- 172.21.15.12:0/1509759681 shutdown_connections 2024-05-17T22:40:53.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.883+0000 7f13da5a0b80 1 -- 172.21.15.12:0/1509759681 wait complete. 2024-05-17T22:40:53.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.883+0000 7f13da5a0b80 1 -- start start 2024-05-17T22:40:53.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.883+0000 7f13da5a0b80 1 Processor -- start 2024-05-17T22:40:53.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.885+0000 7f13da5a0b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a32eeba9f0 0x55a32f0579d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:53.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.885+0000 7f13da5a0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a32f039170 0x55a32f054d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:53.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.885+0000 7f13da5a0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a32f03ecb0 0x55a32f055290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:53.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.885+0000 7f13da5a0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f13c41ac080 con 0x55a32f03ecb0 2024-05-17T22:40:53.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.885+0000 7f13da5a0b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f13c41ad4f0 con 0x55a32eeba9f0 2024-05-17T22:40:53.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.885+0000 7f13da5a0b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f13c41ac750 con 0x55a32f039170 2024-05-17T22:40:53.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.885+0000 7f13da5a0b80 10 client.0 _ll_register_callbacks cb 0x55a32eef1f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:40:53.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.885+0000 7f13c9961700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a32eeba9f0 0x55a32f0579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:53.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.885+0000 7f13c9961700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a32eeba9f0 0x55a32f0579d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:59554/0 (socket says 172.21.15.12:59554) 2024-05-17T22:40:53.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.885+0000 7f13c9961700 1 -- 172.21.15.12:0/1429029370 learned_addr learned my addr 172.21.15.12:0/1429029370 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:40:53.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.885+0000 7f13ca162700 1 --2- 172.21.15.12:0/1429029370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a32f03ecb0 0x55a32f055290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:53.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.885+0000 7f13c9160700 1 --2- 172.21.15.12:0/1429029370 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a32f039170 0x55a32f054d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:53.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.885+0000 7f13ca162700 1 -- 172.21.15.12:0/1429029370 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a32f039170 msgr2=0x55a32f054d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:40:53.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.885+0000 7f13ca162700 1 --2- 172.21.15.12:0/1429029370 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a32f039170 0x55a32f054d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:53.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.885+0000 7f13ca162700 1 -- 172.21.15.12:0/1429029370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a32eeba9f0 msgr2=0x55a32f0579d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:40:53.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.885+0000 7f13ca162700 1 --2- 172.21.15.12:0/1429029370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a32eeba9f0 0x55a32f0579d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:53.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.885+0000 7f13ca162700 1 -- 172.21.15.12:0/1429029370 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a32f0557d0 con 0x55a32f03ecb0 2024-05-17T22:40:53.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.886+0000 7f13c9160700 1 --2- 172.21.15.12:0/1429029370 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a32f039170 0x55a32f054d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:40:53.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.886+0000 7f13ca162700 1 --2- 172.21.15.12:0/1429029370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a32f03ecb0 0x55a32f055290 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f13bc00b710 tx=0x7f13bc00f9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8ec0191190967f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:40:53.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.886+0000 7f13ad7fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:40:53.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.886+0000 7f13ad7fa700 1 -- 172.21.15.12:0/1429029370 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13bc003e10 con 0x55a32f03ecb0 2024-05-17T22:40:53.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.886+0000 7f13ad7fa700 1 -- 172.21.15.12:0/1429029370 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f13bc00d0c0 con 0x55a32f03ecb0 2024-05-17T22:40:53.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.886+0000 7f13da5a0b80 1 -- 172.21.15.12:0/1429029370 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13b400f670 con 0x55a32f03ecb0 2024-05-17T22:40:53.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.887+0000 7f13ad7fa700 1 -- 172.21.15.12:0/1429029370 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f13bc02b050 con 0x55a32f03ecb0 2024-05-17T22:40:53.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.887+0000 7f13da5a0b80 10 client.15501 fetch_fsmap learned FSMap version 29 2024-05-17T22:40:53.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.887+0000 7f13da5a0b80 1 -- 172.21.15.12:0/1429029370 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a32f0c4d50 con 0x55a32f03ecb0 2024-05-17T22:40:53.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.887+0000 7f13ad7fa700 1 -- 172.21.15.12:0/1429029370 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13bc01e030 con 0x55a32f03ecb0 2024-05-17T22:40:53.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.887+0000 7f13ad7fa700 1 -- 172.21.15.12:0/1429029370 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f13bc004660 con 0x55a32f03ecb0 2024-05-17T22:40:53.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.887+0000 7f13da5a0b80 10 client.15501 fetch_fsmap finished waiting for FSMap version 29 2024-05-17T22:40:53.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.887+0000 7f13da5a0b80 10 client.15501 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:40:53.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.887+0000 7f13da5a0b80 20 client.15501 populate_metadata read hostname 'smithi012' 2024-05-17T22:40:53.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.887+0000 7f13da5a0b80 1 --2- 172.21.15.12:0/1429029370 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55a32f0522b0 0x55a32f0c7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:53.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.887+0000 7f13da5a0b80 4 client.15501 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:40:53.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.887+0000 7f13da5a0b80 1 -- 172.21.15.12:0/1429029370 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f13b400f670 con 0x55a32f0522b0 2024-05-17T22:40:53.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.888+0000 7f13c9961700 1 --2- 172.21.15.12:0/1429029370 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55a32f0522b0 0x55a32f0c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:53.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.888+0000 7f13ad7fa700 10 client.15501 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:40:53.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.888+0000 7f13c9961700 1 --2- 172.21.15.12:0/1429029370 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55a32f0522b0 0x55a32f0c7530 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df583978b4ecfd90 server_cookie=2ba115608a813f72 in_seq=0 out_seq=0 2024-05-17T22:40:53.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.889+0000 7f13ad7fa700 1 -- 172.21.15.12:0/1429029370 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f13b400f670 con 0x55a32f0522b0 2024-05-17T22:40:53.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.889+0000 7f13ad7fa700 10 client.15501 handle_command_reply: tid=0 2024-05-17T22:40:53.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.889+0000 7f13da5a0b80 1 client.15501 shutdown 2024-05-17T22:40:53.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.889+0000 7f13da5a0b80 1 -- 172.21.15.12:0/1429029370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a32f03ecb0 msgr2=0x55a32f055290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:53.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.889+0000 7f13da5a0b80 1 --2- 172.21.15.12:0/1429029370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a32f03ecb0 0x55a32f055290 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f13bc00b710 tx=0x7f13bc00f9c0 comp rx=0 tx=0).stop 2024-05-17T22:40:53.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.889+0000 7f13da5a0b80 1 -- 172.21.15.12:0/1429029370 shutdown_connections 2024-05-17T22:40:53.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.889+0000 7f13da5a0b80 1 --2- 172.21.15.12:0/1429029370 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55a32f0522b0 0x55a32f0c7530 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:53.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.889+0000 7f13da5a0b80 1 --2- 172.21.15.12:0/1429029370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a32f03ecb0 0x55a32f055290 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:53.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.889+0000 7f13da5a0b80 1 --2- 172.21.15.12:0/1429029370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a32eeba9f0 0x55a32f0579d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:53.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.889+0000 7f13da5a0b80 1 --2- 172.21.15.12:0/1429029370 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a32f039170 0x55a32f054d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:53.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.889+0000 7f13da5a0b80 1 -- 172.21.15.12:0/1429029370 >> 172.21.15.12:0/1429029370 conn(0x55a32eefef00 msgr2=0x55a32f033ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:40:53.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.889+0000 7f13ad7fa700 0 client.15501 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:40:53.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.890+0000 7f13da5a0b80 1 -- 172.21.15.12:0/1429029370 shutdown_connections 2024-05-17T22:40:53.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.890+0000 7f13da5a0b80 1 -- 172.21.15.12:0/1429029370 wait complete. 2024-05-17T22:40:53.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.890+0000 7f13da5a0b80 20 client.15501 trim_cache size 0 max 16384 2024-05-17T22:40:53.895 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:40:53.895 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-17T22:40:53.895 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-17T22:40:53.895 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:40:53.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.891+0000 7f13ca963700 1 -- 172.21.15.12:0/574125271 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f13b00b2900 msgr2=0x7f13b00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:53.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.891+0000 7f13ca963700 1 --2- 172.21.15.12:0/574125271 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f13b00b2900 0x7f13b00b4e10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f13c4108550 tx=0x7f13b4005fd0 comp rx=0 tx=0).stop 2024-05-17T22:40:53.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.891+0000 7f13ca963700 1 -- 172.21.15.12:0/574125271 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13c4103a50 msgr2=0x7f13c41983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:53.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.891+0000 7f13ca963700 1 --2- 172.21.15.12:0/574125271 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13c4103a50 0x7f13c41983c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f13c000e970 tx=0x7f13c000ec80 comp rx=0 tx=0).stop 2024-05-17T22:40:53.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.893+0000 7f13ca963700 1 -- 172.21.15.12:0/574125271 shutdown_connections 2024-05-17T22:40:53.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.893+0000 7f13ca963700 1 --2- 172.21.15.12:0/574125271 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f13b00b2900 0x7f13b00b4e10 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:53.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.893+0000 7f13ca963700 1 --2- 172.21.15.12:0/574125271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13c4109a80 0x7f13c419cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:53.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.893+0000 7f13ca963700 1 --2- 172.21.15.12:0/574125271 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13c4103a50 0x7f13c41983c0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:53.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.893+0000 7f13ca963700 1 --2- 172.21.15.12:0/574125271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13c41043a0 0x7f13c4198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:53.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.893+0000 7f13ca963700 1 -- 172.21.15.12:0/574125271 >> 172.21.15.12:0/574125271 conn(0x7f13c40ff780 msgr2=0x7f13c4101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:40:53.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.894+0000 7f13ca963700 1 -- 172.21.15.12:0/574125271 shutdown_connections 2024-05-17T22:40:53.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:53.894+0000 7f13ca963700 1 -- 172.21.15.12:0/574125271 wait complete. 2024-05-17T22:40:54.261 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:54 smithi012 ceph-mon[31632]: pgmap v488: 129 pgs: 129 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 10 MiB/s wr, 797 op/s 2024-05-17T22:40:54.261 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:54 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:40:54.261 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:54 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:40:54.261 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:54 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:40:54.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:54 smithi096 ceph-mon[36065]: pgmap v488: 129 pgs: 129 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 10 MiB/s wr, 797 op/s 2024-05-17T22:40:54.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:54 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:40:54.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:54 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:40:54.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:54 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:40:54.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:54 smithi151 ceph-mon[36101]: pgmap v488: 129 pgs: 129 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 10 MiB/s wr, 797 op/s 2024-05-17T22:40:54.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:54 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:40:54.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:54 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:40:54.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:54 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:40:56.036 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-17T22:40:56.036 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T22:40:56.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:56 smithi096 ceph-mon[36065]: pgmap v489: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 MiB/s wr, 841 op/s 2024-05-17T22:40:56.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:56 smithi012 ceph-mon[31632]: pgmap v489: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 MiB/s wr, 841 op/s 2024-05-17T22:40:56.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 95.0 s, 1196.2 tps, lat 83.212 ms stddev 91.722 2024-05-17T22:40:56.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:56 smithi151 ceph-mon[36101]: pgmap v489: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 MiB/s wr, 841 op/s 2024-05-17T22:40:56.510 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:40:58.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:40:58 smithi096 ceph-mon[36065]: pgmap v490: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.8 MiB/s wr, 621 op/s 2024-05-17T22:40:58.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:40:58 smithi012 ceph-mon[31632]: pgmap v490: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.8 MiB/s wr, 621 op/s 2024-05-17T22:40:58.492 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:40:58 smithi151 ceph-mon[36101]: pgmap v490: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.8 MiB/s wr, 621 op/s 2024-05-17T22:40:59.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.247+0000 7f3c9e978700 1 -- 172.21.15.12:0/2920830330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c98075bd0 msgr2=0x7f3c98075fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:59.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.247+0000 7f3c9e978700 1 --2- 172.21.15.12:0/2920830330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c98075bd0 0x7f3c98075fe0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f3c90009f90 tx=0x7f3c9000b330 comp rx=0 tx=0).stop 2024-05-17T22:40:59.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.250+0000 7f3c9e978700 1 -- 172.21.15.12:0/2920830330 shutdown_connections 2024-05-17T22:40:59.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.250+0000 7f3c9e978700 1 --2- 172.21.15.12:0/2920830330 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3c98114c80 0x7f3c981170a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:59.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.250+0000 7f3c9e978700 1 --2- 172.21.15.12:0/2920830330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c98075bd0 0x7f3c98075fe0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:59.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.250+0000 7f3c9e978700 1 --2- 172.21.15.12:0/2920830330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c98076520 0x7f3c98114740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:59.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.250+0000 7f3c9e978700 1 -- 172.21.15.12:0/2920830330 >> 172.21.15.12:0/2920830330 conn(0x7f3c980ff9c0 msgr2=0x7f3c98101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:40:59.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.252+0000 7f3c9e978700 1 -- 172.21.15.12:0/2920830330 shutdown_connections 2024-05-17T22:40:59.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.252+0000 7f3c9e978700 1 -- 172.21.15.12:0/2920830330 wait complete. 2024-05-17T22:40:59.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.253+0000 7f3c9e978700 1 Processor -- start 2024-05-17T22:40:59.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.253+0000 7f3c9e978700 1 -- start start 2024-05-17T22:40:59.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.254+0000 7f3c9e978700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c98075bd0 0x7f3c98114520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:59.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.254+0000 7f3c9e978700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c98076520 0x7f3c98110660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:59.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.254+0000 7f3c9e978700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3c98114c80 0x7f3c98110ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:59.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.254+0000 7f3c9e978700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9810a6f0 con 0x7f3c98075bd0 2024-05-17T22:40:59.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.254+0000 7f3c9e978700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9810a570 con 0x7f3c98076520 2024-05-17T22:40:59.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.254+0000 7f3c9e978700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9810a870 con 0x7f3c98114c80 2024-05-17T22:40:59.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.254+0000 7f3c977fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c98076520 0x7f3c98110660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:59.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.254+0000 7f3c977fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c98076520 0x7f3c98110660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:59584/0 (socket says 172.21.15.12:59584) 2024-05-17T22:40:59.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.254+0000 7f3c977fe700 1 -- 172.21.15.12:0/3075271115 learned_addr learned my addr 172.21.15.12:0/3075271115 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:40:59.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.255+0000 7f3c977fe700 1 -- 172.21.15.12:0/3075271115 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3c98114c80 msgr2=0x7f3c98110ba0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-17T22:40:59.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.255+0000 7f3c977fe700 1 --2- 172.21.15.12:0/3075271115 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3c98114c80 0x7f3c98110ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:59.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.255+0000 7f3c977fe700 1 -- 172.21.15.12:0/3075271115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c98075bd0 msgr2=0x7f3c98114520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:59.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.255+0000 7f3c977fe700 1 --2- 172.21.15.12:0/3075271115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c98075bd0 0x7f3c98114520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:59.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.255+0000 7f3c977fe700 1 -- 172.21.15.12:0/3075271115 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c98111330 con 0x7f3c98076520 2024-05-17T22:40:59.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.255+0000 7f3c977fe700 1 --2- 172.21.15.12:0/3075271115 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c98076520 0x7f3c98110660 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c00d950 tx=0x7f3c8c00dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:40:59.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.255+0000 7f3c957fa700 1 -- 172.21.15.12:0/3075271115 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c016070 con 0x7f3c98076520 2024-05-17T22:40:59.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.256+0000 7f3c957fa700 1 -- 172.21.15.12:0/3075271115 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c8c010460 con 0x7f3c98076520 2024-05-17T22:40:59.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.256+0000 7f3c9e978700 1 -- 172.21.15.12:0/3075271115 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c981a4a50 con 0x7f3c98076520 2024-05-17T22:40:59.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.256+0000 7f3c9e978700 1 -- 172.21.15.12:0/3075271115 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c981a4f90 con 0x7f3c98076520 2024-05-17T22:40:59.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.257+0000 7f3c957fa700 1 -- 172.21.15.12:0/3075271115 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c00f620 con 0x7f3c98076520 2024-05-17T22:40:59.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.257+0000 7f3c9e978700 1 -- 172.21.15.12:0/3075271115 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c98077310 con 0x7f3c98076520 2024-05-17T22:40:59.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.259+0000 7f3c957fa700 1 -- 172.21.15.12:0/3075271115 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3c8c019070 con 0x7f3c98076520 2024-05-17T22:40:59.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.260+0000 7f3c957fa700 1 --2- 172.21.15.12:0/3075271115 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f3c800b2940 0x7f3c800b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:40:59.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.261+0000 7f3c957fa700 1 -- 172.21.15.12:0/3075271115 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f3c8c0e2610 con 0x7f3c98076520 2024-05-17T22:40:59.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.262+0000 7f3c97fff700 1 --2- 172.21.15.12:0/3075271115 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f3c800b2940 0x7f3c800b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:40:59.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.263+0000 7f3c97fff700 1 --2- 172.21.15.12:0/3075271115 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f3c800b2940 0x7f3c800b4e50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3c90009f90 tx=0x7f3c90000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:40:59.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.263+0000 7f3c957fa700 1 -- 172.21.15.12:0/3075271115 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3c8c0a9fa0 con 0x7f3c98076520 2024-05-17T22:40:59.715 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:40:59.716 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1407219319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1407219319},{"type":"v1","addr":"172.21.15.12:6835","nonce":1407219319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/630977886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":630977886},{"type":"v1","addr":"172.21.15.12:6837","nonce":630977886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3731914742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3731914742},{"type":"v1","addr":"172.21.15.12:6839","nonce":3731914742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/3642142501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3642142501},{"type":"v1","addr":"172.21.15.12:6841","nonce":3642142501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3554053059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3554053059},{"type":"v1","addr":"172.21.15.151:6833","nonce":3554053059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/4228325569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":4228325569},{"type":"v1","addr":"172.21.15.96:6833","nonce":4228325569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T22:34:42.642111+0000","modified":"2024-05-17T22:39:43.140707+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24461,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/542323754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":542323754},{"type":"v1","addr":"172.21.15.151:6835","nonce":542323754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1758442232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1758442232},{"type":"v1","addr":"172.21.15.96:6835","nonce":1758442232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3403324038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3403324038},{"type":"v1","addr":"172.21.15.151:6837","nonce":3403324038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.151:6839/2338353849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2338353849},{"type":"v1","addr":"172.21.15.151:6839","nonce":2338353849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.96:6837/4222574338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":4222574338},{"type":"v1","addr":"172.21.15.96:6837","nonce":4222574338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/2986295519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2986295519},{"type":"v1","addr":"172.21.15.96:6839","nonce":2986295519}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24457,24461,24482]},"id":1}]} 2024-05-17T22:40:59.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.711+0000 7f3c9e978700 1 -- 172.21.15.12:0/3075271115 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c98075fe0 con 0x7f3c98076520 2024-05-17T22:40:59.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.712+0000 7f3c957fa700 1 -- 172.21.15.12:0/3075271115 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11873 (secure 0 0 0) 0x7f3c8c0a98a0 con 0x7f3c98076520 2024-05-17T22:40:59.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.716+0000 7f3c9e978700 1 -- 172.21.15.12:0/3075271115 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f3c800b2940 msgr2=0x7f3c800b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:59.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.716+0000 7f3c9e978700 1 --2- 172.21.15.12:0/3075271115 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f3c800b2940 0x7f3c800b4e50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3c90009f90 tx=0x7f3c90000f40 comp rx=0 tx=0).stop 2024-05-17T22:40:59.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.716+0000 7f3c9e978700 1 -- 172.21.15.12:0/3075271115 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c98076520 msgr2=0x7f3c98110660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:40:59.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.716+0000 7f3c9e978700 1 --2- 172.21.15.12:0/3075271115 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c98076520 0x7f3c98110660 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c00d950 tx=0x7f3c8c00dd10 comp rx=0 tx=0).stop 2024-05-17T22:40:59.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.716+0000 7f3c9e978700 1 -- 172.21.15.12:0/3075271115 shutdown_connections 2024-05-17T22:40:59.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.716+0000 7f3c9e978700 1 --2- 172.21.15.12:0/3075271115 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f3c800b2940 0x7f3c800b4e50 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:59.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.716+0000 7f3c9e978700 1 --2- 172.21.15.12:0/3075271115 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3c98114c80 0x7f3c98110ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:59.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.716+0000 7f3c9e978700 1 --2- 172.21.15.12:0/3075271115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c98075bd0 0x7f3c98114520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:59.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.716+0000 7f3c9e978700 1 --2- 172.21.15.12:0/3075271115 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c98076520 0x7f3c98110660 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:40:59.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.716+0000 7f3c9e978700 1 -- 172.21.15.12:0/3075271115 >> 172.21.15.12:0/3075271115 conn(0x7f3c980ff9c0 msgr2=0x7f3c98100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:40:59.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.716+0000 7f3c9e978700 1 -- 172.21.15.12:0/3075271115 shutdown_connections 2024-05-17T22:40:59.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:40:59.716+0000 7f3c9e978700 1 -- 172.21.15.12:0/3075271115 wait complete. 2024-05-17T22:40:59.720 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 29 2024-05-17T22:41:00.114 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:00 smithi012 ceph-mon[31632]: pgmap v491: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 14 MiB/s wr, 952 op/s 2024-05-17T22:41:00.115 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:00 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/3075271115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:41:00.292 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:00 smithi151 ceph-mon[36101]: pgmap v491: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 14 MiB/s wr, 952 op/s 2024-05-17T22:41:00.292 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:00 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/3075271115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:41:00.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:00 smithi096 ceph-mon[36065]: pgmap v491: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 14 MiB/s wr, 952 op/s 2024-05-17T22:41:00.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:00 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/3075271115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:41:00.631 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-17T22:41:01.070 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:41:01.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 100.0 s, 1178.6 tps, lat 84.277 ms stddev 160.118 2024-05-17T22:41:02.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:02 smithi096 ceph-mon[36065]: pgmap v492: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 15 MiB/s wr, 878 op/s 2024-05-17T22:41:02.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:02 smithi012 ceph-mon[31632]: pgmap v492: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 15 MiB/s wr, 878 op/s 2024-05-17T22:41:02.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:02 smithi151 ceph-mon[36101]: pgmap v492: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 15 MiB/s wr, 878 op/s 2024-05-17T22:41:04.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:04 smithi096 ceph-mon[36065]: pgmap v493: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 18 MiB/s wr, 1.01k op/s 2024-05-17T22:41:04.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:04 smithi012 ceph-mon[31632]: pgmap v493: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 18 MiB/s wr, 1.01k op/s 2024-05-17T22:41:04.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:04 smithi151 ceph-mon[36101]: pgmap v493: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 18 MiB/s wr, 1.01k op/s 2024-05-17T22:41:06.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:06 smithi096 ceph-mon[36065]: pgmap v494: 129 pgs: 129 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 18 MiB/s wr, 953 op/s 2024-05-17T22:41:06.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:06 smithi012 ceph-mon[31632]: pgmap v494: 129 pgs: 129 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 18 MiB/s wr, 953 op/s 2024-05-17T22:41:06.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 105.0 s, 2173.0 tps, lat 46.259 ms stddev 60.652 2024-05-17T22:41:06.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:06 smithi151 ceph-mon[36101]: pgmap v494: 129 pgs: 129 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 18 MiB/s wr, 953 op/s 2024-05-17T22:41:07.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.035+0000 7f88295d9700 1 -- 172.21.15.12:0/3092585665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88241080a0 msgr2=0x7f882410fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:41:07.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.035+0000 7f88295d9700 1 --2- 172.21.15.12:0/3092585665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88241080a0 0x7f882410fa00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f881c00b0a0 tx=0x7f881c00b3b0 comp rx=0 tx=0).stop 2024-05-17T22:41:07.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.035+0000 7f88295d9700 1 -- 172.21.15.12:0/3092585665 shutdown_connections 2024-05-17T22:41:07.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.035+0000 7f88295d9700 1 --2- 172.21.15.12:0/3092585665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88241080a0 0x7f882410fa00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:07.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.035+0000 7f88295d9700 1 --2- 172.21.15.12:0/3092585665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8824102e70 0x7f8824103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:07.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.035+0000 7f88295d9700 1 --2- 172.21.15.12:0/3092585665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8824107750 0x7f8824107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:07.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.035+0000 7f88295d9700 1 -- 172.21.15.12:0/3092585665 >> 172.21.15.12:0/3092585665 conn(0x7f8824079ae0 msgr2=0x7f8824079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:41:07.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.036+0000 7f88295d9700 1 -- 172.21.15.12:0/3092585665 shutdown_connections 2024-05-17T22:41:07.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.036+0000 7f88295d9700 1 -- 172.21.15.12:0/3092585665 wait complete. 2024-05-17T22:41:07.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.037+0000 7f88295d9700 1 Processor -- start 2024-05-17T22:41:07.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.037+0000 7f88295d9700 1 -- start start 2024-05-17T22:41:07.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.038+0000 7f88295d9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8824102e70 0x7f8824199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:07.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.038+0000 7f88295d9700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8824107750 0x7f8824199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:07.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.038+0000 7f88295d9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88241080a0 0x7f8824199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:07.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.038+0000 7f88295d9700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8824112d50 con 0x7f88241080a0 2024-05-17T22:41:07.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.038+0000 7f88295d9700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8824112bd0 con 0x7f8824102e70 2024-05-17T22:41:07.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.038+0000 7f88295d9700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8824112ed0 con 0x7f8824107750 2024-05-17T22:41:07.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.038+0000 7f8822ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8824102e70 0x7f8824199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:07.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.038+0000 7f88227fc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8824107750 0x7f8824199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:07.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.038+0000 7f8822ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8824102e70 0x7f8824199410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:56416/0 (socket says 172.21.15.12:56416) 2024-05-17T22:41:07.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.038+0000 7f8822ffd700 1 -- 172.21.15.12:0/2806450711 learned_addr learned my addr 172.21.15.12:0/2806450711 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:41:07.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.038+0000 7f88237fe700 1 --2- 172.21.15.12:0/2806450711 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88241080a0 0x7f8824199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:07.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.038+0000 7f8822ffd700 1 -- 172.21.15.12:0/2806450711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8824107750 msgr2=0x7f8824199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:41:07.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.038+0000 7f8822ffd700 1 --2- 172.21.15.12:0/2806450711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8824107750 0x7f8824199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:07.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.038+0000 7f8822ffd700 1 -- 172.21.15.12:0/2806450711 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88241080a0 msgr2=0x7f8824199e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:41:07.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.038+0000 7f8822ffd700 1 --2- 172.21.15.12:0/2806450711 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88241080a0 0x7f8824199e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:07.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.038+0000 7f8822ffd700 1 -- 172.21.15.12:0/2806450711 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88241a8b10 con 0x7f8824102e70 2024-05-17T22:41:07.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.039+0000 7f8822ffd700 1 --2- 172.21.15.12:0/2806450711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8824102e70 0x7f8824199410 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f881400e9a0 tx=0x7f881400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:41:07.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.039+0000 7f880bfff700 1 -- 172.21.15.12:0/2806450711 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881400ca10 con 0x7f8824102e70 2024-05-17T22:41:07.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.039+0000 7f880bfff700 1 -- 172.21.15.12:0/2806450711 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f881400cbb0 con 0x7f8824102e70 2024-05-17T22:41:07.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.040+0000 7f880bfff700 1 -- 172.21.15.12:0/2806450711 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8814018680 con 0x7f8824102e70 2024-05-17T22:41:07.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.040+0000 7f88295d9700 1 -- 172.21.15.12:0/2806450711 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88241a8e00 con 0x7f8824102e70 2024-05-17T22:41:07.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.041+0000 7f88295d9700 1 -- 172.21.15.12:0/2806450711 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88241a92b0 con 0x7f8824102e70 2024-05-17T22:41:07.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.044+0000 7f880bfff700 1 -- 172.21.15.12:0/2806450711 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8814010430 con 0x7f8824102e70 2024-05-17T22:41:07.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.045+0000 7f8837fb4b80 1 Processor -- start 2024-05-17T22:41:07.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.045+0000 7f8837fb4b80 1 -- start start 2024-05-17T22:41:07.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.046+0000 7f8837fb4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c3d7889f0 0x558c3d914d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:07.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.046+0000 7f8837fb4b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c3d9152b0 0x558c3d90f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:07.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.046+0000 7f8837fb4b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c3d90f980 0x558c3d90fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:07.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.046+0000 7f880bfff700 1 --2- 172.21.15.12:0/2806450711 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f880c0b29d0 0x7f880c0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:07.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.046+0000 7f8837fb4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x558c3d835290 con 0x558c3d7889f0 2024-05-17T22:41:07.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.046+0000 7f8837fb4b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558c3d824fd0 con 0x558c3d9152b0 2024-05-17T22:41:07.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.046+0000 7f8837fb4b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558c3d828e70 con 0x558c3d90f980 2024-05-17T22:41:07.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.046+0000 7f88227fc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c3d90f980 0x558c3d90fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:07.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.046+0000 7f88227fc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c3d90f980 0x558c3d90fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:40364/0 (socket says 172.21.15.12:40364) 2024-05-17T22:41:07.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.046+0000 7f88227fc700 1 -- 172.21.15.12:0/2593072508 learned_addr learned my addr 172.21.15.12:0/2593072508 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:41:07.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.046+0000 7f8822ffd700 1 --2- 172.21.15.12:0/2593072508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c3d9152b0 0x558c3d90f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:07.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.046+0000 7f88237fe700 1 --2- 172.21.15.12:0/2593072508 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c3d7889f0 0x558c3d914d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:07.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.046+0000 7f880bfff700 1 -- 172.21.15.12:0/2806450711 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f88140e2690 con 0x7f8824102e70 2024-05-17T22:41:07.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.046+0000 7f88237fe700 1 --2- 172.21.15.12:0/2806450711 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f880c0b29d0 0x7f880c0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:07.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.046+0000 7f88227fc700 1 -- 172.21.15.12:0/2593072508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c3d9152b0 msgr2=0x558c3d90f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:41:07.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.046+0000 7f88227fc700 1 --2- 172.21.15.12:0/2593072508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c3d9152b0 0x558c3d90f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:07.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.046+0000 7f88227fc700 1 -- 172.21.15.12:0/2593072508 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c3d7889f0 msgr2=0x558c3d914d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:41:07.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.046+0000 7f88227fc700 1 --2- 172.21.15.12:0/2593072508 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c3d7889f0 0x558c3d914d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:07.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.046+0000 7f88227fc700 1 -- 172.21.15.12:0/2593072508 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c3d7ce590 con 0x558c3d90f980 2024-05-17T22:41:07.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.046+0000 7f88237fe700 1 --2- 172.21.15.12:0/2593072508 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c3d7889f0 0x558c3d914d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:41:07.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.046+0000 7f88227fc700 1 --2- 172.21.15.12:0/2593072508 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c3d90f980 0x558c3d90fd90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8818009a80 tx=0x7f8818009d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbc60a6e5e99015d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:41:07.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.047+0000 7f88097fa700 1 -- 172.21.15.12:0/2593072508 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8818017070 con 0x558c3d90f980 2024-05-17T22:41:07.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.047+0000 7f88097fa700 1 -- 172.21.15.12:0/2593072508 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f881800ed80 con 0x558c3d90f980 2024-05-17T22:41:07.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.047+0000 7f88097fa700 1 -- 172.21.15.12:0/2593072508 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8818013e00 con 0x558c3d90f980 2024-05-17T22:41:07.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.048+0000 7f8837fb4b80 1 -- 172.21.15.12:0/2593072508 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c3d90f980 msgr2=0x558c3d90fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:41:07.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.048+0000 7f8837fb4b80 1 --2- 172.21.15.12:0/2593072508 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c3d90f980 0x558c3d90fd90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8818009a80 tx=0x7f8818009d90 comp rx=0 tx=0).stop 2024-05-17T22:41:07.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.049+0000 7f88237fe700 1 --2- 172.21.15.12:0/2806450711 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f880c0b29d0 0x7f880c0b4ee0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f881c009e60 tx=0x7f881c00b720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:41:07.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.049+0000 7f8837fb4b80 1 -- 172.21.15.12:0/2593072508 shutdown_connections 2024-05-17T22:41:07.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.049+0000 7f8837fb4b80 1 --2- 172.21.15.12:0/2593072508 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c3d90f980 0x558c3d90fd90 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:07.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.049+0000 7f8837fb4b80 1 --2- 172.21.15.12:0/2593072508 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c3d7889f0 0x558c3d914d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:07.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.049+0000 7f8837fb4b80 1 --2- 172.21.15.12:0/2593072508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c3d9152b0 0x558c3d90f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:07.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.049+0000 7f8837fb4b80 1 -- 172.21.15.12:0/2593072508 >> 172.21.15.12:0/2593072508 conn(0x558c3d7ccf00 msgr2=0x558c3d901b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:41:07.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.049+0000 7f8837fb4b80 1 -- 172.21.15.12:0/2593072508 shutdown_connections 2024-05-17T22:41:07.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.049+0000 7f8837fb4b80 1 -- 172.21.15.12:0/2593072508 wait complete. 2024-05-17T22:41:07.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.050+0000 7f8837fb4b80 1 -- start start 2024-05-17T22:41:07.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.050+0000 7f8837fb4b80 1 Processor -- start 2024-05-17T22:41:07.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.050+0000 7f8837fb4b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c3d7889f0 0x558c3d9189c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:07.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.050+0000 7f8837fb4b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c3d90f980 0x558c3d918f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:07.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.050+0000 7f8837fb4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c3d9152b0 0x558c3d919440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:07.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.050+0000 7f8837fb4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x558c3d72c150 con 0x558c3d9152b0 2024-05-17T22:41:07.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.051+0000 7f8837fb4b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558c3d7c5e80 con 0x558c3d7889f0 2024-05-17T22:41:07.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.051+0000 7f8837fb4b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558c3d91bbf0 con 0x558c3d90f980 2024-05-17T22:41:07.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.051+0000 7f8822ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c3d7889f0 0x558c3d9189c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:07.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.051+0000 7f88227fc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c3d90f980 0x558c3d918f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:07.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.051+0000 7f8822ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c3d7889f0 0x558c3d9189c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:56428/0 (socket says 172.21.15.12:56428) 2024-05-17T22:41:07.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.051+0000 7f8822ffd700 1 -- 172.21.15.12:0/797172206 learned_addr learned my addr 172.21.15.12:0/797172206 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:41:07.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.051+0000 7f8837fb4b80 10 client.0 _ll_register_callbacks cb 0x558c3d7bff10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:41:07.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.051+0000 7f88237fe700 1 --2- 172.21.15.12:0/797172206 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c3d9152b0 0x558c3d919440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:07.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.051+0000 7f8822ffd700 1 -- 172.21.15.12:0/797172206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c3d90f980 msgr2=0x558c3d918f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:41:07.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.051+0000 7f8822ffd700 1 --2- 172.21.15.12:0/797172206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c3d90f980 0x558c3d918f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:07.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.051+0000 7f8822ffd700 1 -- 172.21.15.12:0/797172206 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c3d9152b0 msgr2=0x558c3d919440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:41:07.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.051+0000 7f8822ffd700 1 --2- 172.21.15.12:0/797172206 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c3d9152b0 0x558c3d919440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:07.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.051+0000 7f8822ffd700 1 -- 172.21.15.12:0/797172206 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c3d775480 con 0x558c3d7889f0 2024-05-17T22:41:07.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.051+0000 7f8822ffd700 1 --2- 172.21.15.12:0/797172206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c3d7889f0 0x558c3d9189c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f88140a6b30 tx=0x7f8814074250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=207e3e40fe59e66b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:41:07.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.051+0000 7f8806ff5700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:41:07.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.051+0000 7f8806ff5700 1 -- 172.21.15.12:0/797172206 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88140a9030 con 0x558c3d7889f0 2024-05-17T22:41:07.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.051+0000 7f8806ff5700 1 -- 172.21.15.12:0/797172206 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f88140cc0e0 con 0x558c3d7889f0 2024-05-17T22:41:07.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.051+0000 7f8837fb4b80 1 -- 172.21.15.12:0/797172206 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f881800ed80 con 0x558c3d7889f0 2024-05-17T22:41:07.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.052+0000 7f8806ff5700 1 -- 172.21.15.12:0/797172206 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f88140b2060 con 0x558c3d7889f0 2024-05-17T22:41:07.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.052+0000 7f8837fb4b80 10 client.25187 fetch_fsmap learned FSMap version 29 2024-05-17T22:41:07.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.052+0000 7f8837fb4b80 1 -- 172.21.15.12:0/797172206 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558c3d9263b0 con 0x558c3d7889f0 2024-05-17T22:41:07.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.052+0000 7f8806ff5700 1 -- 172.21.15.12:0/797172206 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88140a93d0 con 0x558c3d7889f0 2024-05-17T22:41:07.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.052+0000 7f8806ff5700 1 -- 172.21.15.12:0/797172206 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f88140b2290 con 0x558c3d7889f0 2024-05-17T22:41:07.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.052+0000 7f8837fb4b80 10 client.25187 fetch_fsmap finished waiting for FSMap version 29 2024-05-17T22:41:07.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.052+0000 7f8837fb4b80 10 client.25187 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-17T22:41:07.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.052+0000 7f8837fb4b80 20 client.25187 populate_metadata read hostname 'smithi012' 2024-05-17T22:41:07.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.052+0000 7f8837fb4b80 1 --2- 172.21.15.12:0/797172206 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x558c3d928d80 0x558c3d92b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:07.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.052+0000 7f8837fb4b80 4 client.25187 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:41:07.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.053+0000 7f8837fb4b80 1 -- 172.21.15.12:0/797172206 --> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f881800ed80 con 0x558c3d928d80 2024-05-17T22:41:07.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.053+0000 7f88227fc700 1 --2- 172.21.15.12:0/797172206 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x558c3d928d80 0x558c3d92b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:07.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.054+0000 7f88227fc700 1 --2- 172.21.15.12:0/797172206 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x558c3d928d80 0x558c3d92b230 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=68729c6fd72ffa1c server_cookie=2f3079e7f265a8c4 in_seq=0 out_seq=0 2024-05-17T22:41:07.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.054+0000 7f8806ff5700 10 client.25187 ms_handle_connect on v2:172.21.15.151:6838/2338353849 2024-05-17T22:41:07.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.055+0000 7f8806ff5700 1 -- 172.21.15.12:0/797172206 <== mds.2 v2:172.21.15.151:6838/2338353849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f881800ed80 con 0x558c3d928d80 2024-05-17T22:41:07.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.055+0000 7f8806ff5700 10 client.25187 handle_command_reply: tid=0 2024-05-17T22:41:07.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.055+0000 7f8837fb4b80 1 client.25187 shutdown 2024-05-17T22:41:07.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.055+0000 7f8837fb4b80 1 -- 172.21.15.12:0/797172206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c3d7889f0 msgr2=0x558c3d9189c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:41:07.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.055+0000 7f8837fb4b80 1 --2- 172.21.15.12:0/797172206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c3d7889f0 0x558c3d9189c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f88140a6b30 tx=0x7f8814074250 comp rx=0 tx=0).stop 2024-05-17T22:41:07.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.055+0000 7f8837fb4b80 1 -- 172.21.15.12:0/797172206 shutdown_connections 2024-05-17T22:41:07.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.055+0000 7f8837fb4b80 1 --2- 172.21.15.12:0/797172206 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c3d9152b0 0x558c3d919440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:07.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.055+0000 7f8837fb4b80 1 --2- 172.21.15.12:0/797172206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c3d7889f0 0x558c3d9189c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:07.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.055+0000 7f8837fb4b80 1 --2- 172.21.15.12:0/797172206 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x558c3d928d80 0x558c3d92b230 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:07.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.055+0000 7f8837fb4b80 1 --2- 172.21.15.12:0/797172206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c3d90f980 0x558c3d918f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:07.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.055+0000 7f8837fb4b80 1 -- 172.21.15.12:0/797172206 >> 172.21.15.12:0/797172206 conn(0x558c3d7ccf00 msgr2=0x558c3d8fbed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:41:07.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.055+0000 7f8806ff5700 0 client.25187 ms_handle_reset on v2:172.21.15.151:6838/2338353849 2024-05-17T22:41:07.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.056+0000 7f8837fb4b80 1 -- 172.21.15.12:0/797172206 shutdown_connections 2024-05-17T22:41:07.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.056+0000 7f8837fb4b80 1 -- 172.21.15.12:0/797172206 wait complete. 2024-05-17T22:41:07.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.056+0000 7f8837fb4b80 20 client.25187 trim_cache size 0 max 16384 2024-05-17T22:41:07.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.083+0000 7f8837fb4b80 1 Processor -- start 2024-05-17T22:41:07.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.083+0000 7f8837fb4b80 1 -- start start 2024-05-17T22:41:07.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.083+0000 7f8837fb4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c3d7889f0 0x558c3d90f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:07.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.083+0000 7f8837fb4b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c3d90f980 0x558c3d912ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:07.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.083+0000 7f8837fb4b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c3d9152b0 0x558c3d8fbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:07.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.083+0000 7f8837fb4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f88241b03b0 con 0x558c3d7889f0 2024-05-17T22:41:07.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.083+0000 7f8837fb4b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f88241adc40 con 0x558c3d90f980 2024-05-17T22:41:07.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.083+0000 7f8837fb4b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f88241aba90 con 0x558c3d9152b0 2024-05-17T22:41:07.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.084+0000 7f88227fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c3d90f980 0x558c3d912ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:07.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.084+0000 7f88227fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c3d90f980 0x558c3d912ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:56442/0 (socket says 172.21.15.12:56442) 2024-05-17T22:41:07.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.084+0000 7f88227fc700 1 -- 172.21.15.12:0/3446093425 learned_addr learned my addr 172.21.15.12:0/3446093425 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:41:07.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.084+0000 7f88237fe700 1 --2- 172.21.15.12:0/3446093425 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c3d9152b0 0x558c3d8fbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:07.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.084+0000 7f8822ffd700 1 --2- 172.21.15.12:0/3446093425 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c3d7889f0 0x558c3d90f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:07.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.084+0000 7f88227fc700 1 -- 172.21.15.12:0/3446093425 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c3d9152b0 msgr2=0x558c3d8fbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:41:07.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.084+0000 7f88227fc700 1 --2- 172.21.15.12:0/3446093425 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c3d9152b0 0x558c3d8fbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:07.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.084+0000 7f88227fc700 1 -- 172.21.15.12:0/3446093425 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c3d7889f0 msgr2=0x558c3d90f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:41:07.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.084+0000 7f88227fc700 1 --2- 172.21.15.12:0/3446093425 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c3d7889f0 0x558c3d90f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:07.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.084+0000 7f88227fc700 1 -- 172.21.15.12:0/3446093425 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c3d916f30 con 0x558c3d90f980 2024-05-17T22:41:07.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.085+0000 7f88227fc700 1 --2- 172.21.15.12:0/3446093425 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c3d90f980 0x558c3d912ee0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8818009a50 tx=0x7f8818009190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec4df727b4066c71 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:41:07.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.085+0000 7f8808ff9700 1 -- 172.21.15.12:0/3446093425 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881802d070 con 0x558c3d90f980 2024-05-17T22:41:07.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.085+0000 7f8808ff9700 1 -- 172.21.15.12:0/3446093425 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8818005a50 con 0x558c3d90f980 2024-05-17T22:41:07.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.085+0000 7f8808ff9700 1 -- 172.21.15.12:0/3446093425 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8818017040 con 0x558c3d90f980 2024-05-17T22:41:07.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.086+0000 7f8837fb4b80 1 -- 172.21.15.12:0/3446093425 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c3d90f980 msgr2=0x558c3d912ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:41:07.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.086+0000 7f8837fb4b80 1 --2- 172.21.15.12:0/3446093425 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c3d90f980 0x558c3d912ee0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8818009a50 tx=0x7f8818009190 comp rx=0 tx=0).stop 2024-05-17T22:41:07.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.086+0000 7f8837fb4b80 1 -- 172.21.15.12:0/3446093425 shutdown_connections 2024-05-17T22:41:07.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.086+0000 7f8837fb4b80 1 --2- 172.21.15.12:0/3446093425 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c3d9152b0 0x558c3d8fbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:07.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.086+0000 7f8837fb4b80 1 --2- 172.21.15.12:0/3446093425 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c3d7889f0 0x558c3d90f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:07.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.086+0000 7f8837fb4b80 1 --2- 172.21.15.12:0/3446093425 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c3d90f980 0x558c3d912ee0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:07.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.086+0000 7f8837fb4b80 1 -- 172.21.15.12:0/3446093425 >> 172.21.15.12:0/3446093425 conn(0x558c3d7ccf00 msgr2=0x558c3d9140a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:41:07.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.087+0000 7f8837fb4b80 1 -- 172.21.15.12:0/3446093425 shutdown_connections 2024-05-17T22:41:07.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.087+0000 7f8837fb4b80 1 -- 172.21.15.12:0/3446093425 wait complete. 2024-05-17T22:41:07.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.087+0000 7f8837fb4b80 1 -- start start 2024-05-17T22:41:07.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.087+0000 7f8837fb4b80 1 Processor -- start 2024-05-17T22:41:07.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.088+0000 7f8837fb4b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c3d7889f0 0x558c3d9209a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:07.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.088+0000 7f8837fb4b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c3d90f980 0x558c3d920ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:07.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.088+0000 7f8837fb4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c3d9152b0 0x558c3d923840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:07.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.088+0000 7f8837fb4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f88241ac100 con 0x558c3d9152b0 2024-05-17T22:41:07.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.088+0000 7f8837fb4b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f88241ad570 con 0x558c3d7889f0 2024-05-17T22:41:07.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.088+0000 7f8837fb4b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f88241ac7d0 con 0x558c3d90f980 2024-05-17T22:41:07.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.088+0000 7f8822ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c3d7889f0 0x558c3d9209a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:07.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.088+0000 7f8837fb4b80 10 client.0 _ll_register_callbacks cb 0x558c3d7bff10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:41:07.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.088+0000 7f88227fc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c3d90f980 0x558c3d920ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:07.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.088+0000 7f8822ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c3d7889f0 0x558c3d9209a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:56456/0 (socket says 172.21.15.12:56456) 2024-05-17T22:41:07.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.088+0000 7f8822ffd700 1 -- 172.21.15.12:0/2432746918 learned_addr learned my addr 172.21.15.12:0/2432746918 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:41:07.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.088+0000 7f8822ffd700 1 -- 172.21.15.12:0/2432746918 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c3d90f980 msgr2=0x558c3d920ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:41:07.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.088+0000 7f8822ffd700 1 --2- 172.21.15.12:0/2432746918 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c3d90f980 0x558c3d920ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:07.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.088+0000 7f8822ffd700 1 -- 172.21.15.12:0/2432746918 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c3d9152b0 msgr2=0x558c3d923840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T22:41:07.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.088+0000 7f8822ffd700 1 --2- 172.21.15.12:0/2432746918 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c3d9152b0 0x558c3d923840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:07.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.088+0000 7f8822ffd700 1 -- 172.21.15.12:0/2432746918 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c3d923d80 con 0x558c3d7889f0 2024-05-17T22:41:07.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.089+0000 7f88227fc700 1 --2- 172.21.15.12:0/2432746918 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c3d90f980 0x558c3d920ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:41:07.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.089+0000 7f8822ffd700 1 --2- 172.21.15.12:0/2432746918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c3d7889f0 0x558c3d9209a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f88140a5940 tx=0x7f88140b24c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=184e32a28a84250e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:41:07.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.089+0000 7f8806ff5700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:41:07.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.089+0000 7f8806ff5700 1 -- 172.21.15.12:0/2432746918 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88140a9030 con 0x558c3d7889f0 2024-05-17T22:41:07.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.089+0000 7f8837fb4b80 1 -- 172.21.15.12:0/2432746918 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8818005a50 con 0x558c3d7889f0 2024-05-17T22:41:07.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.089+0000 7f8806ff5700 1 -- 172.21.15.12:0/2432746918 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f881400cdd0 con 0x558c3d7889f0 2024-05-17T22:41:07.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.090+0000 7f8806ff5700 1 -- 172.21.15.12:0/2432746918 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f88140751f0 con 0x558c3d7889f0 2024-05-17T22:41:07.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.090+0000 7f8837fb4b80 10 client.25193 fetch_fsmap learned FSMap version 29 2024-05-17T22:41:07.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.090+0000 7f8837fb4b80 1 -- 172.21.15.12:0/2432746918 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558c3d992d40 con 0x558c3d7889f0 2024-05-17T22:41:07.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.090+0000 7f8806ff5700 1 -- 172.21.15.12:0/2432746918 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8818005a50 con 0x558c3d7889f0 2024-05-17T22:41:07.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.090+0000 7f8806ff5700 1 -- 172.21.15.12:0/2432746918 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f8814010720 con 0x558c3d7889f0 2024-05-17T22:41:07.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.090+0000 7f8837fb4b80 10 client.25193 fetch_fsmap finished waiting for FSMap version 29 2024-05-17T22:41:07.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.090+0000 7f8837fb4b80 10 client.25193 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-17T22:41:07.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.090+0000 7f8837fb4b80 20 client.25193 populate_metadata read hostname 'smithi012' 2024-05-17T22:41:07.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.090+0000 7f8837fb4b80 1 --2- 172.21.15.12:0/2432746918 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x558c3d9933e0 0x558c3d995800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:07.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.090+0000 7f8837fb4b80 4 client.25193 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:41:07.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.090+0000 7f8837fb4b80 1 -- 172.21.15.12:0/2432746918 --> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558c3d8371e0 con 0x558c3d9933e0 2024-05-17T22:41:07.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.090+0000 7f88227fc700 1 --2- 172.21.15.12:0/2432746918 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x558c3d9933e0 0x558c3d995800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:07.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.091+0000 7f8806ff5700 10 client.25193 ms_handle_connect on v2:172.21.15.151:6838/2338353849 2024-05-17T22:41:07.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.091+0000 7f88227fc700 1 --2- 172.21.15.12:0/2432746918 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x558c3d9933e0 0x558c3d995800 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4cb80ba55b2a63a2 server_cookie=fb36f4efbd3d7132 in_seq=0 out_seq=0 2024-05-17T22:41:07.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.091+0000 7f8806ff5700 1 -- 172.21.15.12:0/2432746918 <== mds.2 v2:172.21.15.151:6838/2338353849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558c3d8371e0 con 0x558c3d9933e0 2024-05-17T22:41:07.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.091+0000 7f8806ff5700 10 client.25193 handle_command_reply: tid=0 2024-05-17T22:41:07.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.091+0000 7f8837fb4b80 1 client.25193 shutdown 2024-05-17T22:41:07.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.091+0000 7f8837fb4b80 1 -- 172.21.15.12:0/2432746918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c3d7889f0 msgr2=0x558c3d9209a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:41:07.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.091+0000 7f8837fb4b80 1 --2- 172.21.15.12:0/2432746918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c3d7889f0 0x558c3d9209a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f88140a5940 tx=0x7f88140b24c0 comp rx=0 tx=0).stop 2024-05-17T22:41:07.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.092+0000 7f8837fb4b80 1 -- 172.21.15.12:0/2432746918 shutdown_connections 2024-05-17T22:41:07.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.092+0000 7f8837fb4b80 1 --2- 172.21.15.12:0/2432746918 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558c3d9152b0 0x558c3d923840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:07.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.092+0000 7f8837fb4b80 1 --2- 172.21.15.12:0/2432746918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c3d7889f0 0x558c3d9209a0 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f88140a5940 tx=0x7f88140b24c0 comp rx=0 tx=0).stop 2024-05-17T22:41:07.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.092+0000 7f8837fb4b80 1 --2- 172.21.15.12:0/2432746918 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x558c3d9933e0 0x558c3d995800 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:07.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.092+0000 7f8837fb4b80 1 --2- 172.21.15.12:0/2432746918 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c3d90f980 0x558c3d920ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:07.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.092+0000 7f8837fb4b80 1 -- 172.21.15.12:0/2432746918 >> 172.21.15.12:0/2432746918 conn(0x558c3d7ccf00 msgr2=0x558c3d9140a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:41:07.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.092+0000 7f8806ff5700 0 client.25193 ms_handle_reset on v2:172.21.15.151:6838/2338353849 2024-05-17T22:41:07.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.092+0000 7f8837fb4b80 1 -- 172.21.15.12:0/2432746918 shutdown_connections 2024-05-17T22:41:07.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.092+0000 7f8837fb4b80 1 -- 172.21.15.12:0/2432746918 wait complete. 2024-05-17T22:41:07.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.092+0000 7f8837fb4b80 20 client.25193 trim_cache size 0 max 16384 2024-05-17T22:41:07.101 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:41:07.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.094+0000 7f88295d9700 1 -- 172.21.15.12:0/2806450711 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f880c0b29d0 msgr2=0x7f880c0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:41:07.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.094+0000 7f88295d9700 1 --2- 172.21.15.12:0/2806450711 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f880c0b29d0 0x7f880c0b4ee0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f881c009e60 tx=0x7f881c00b720 comp rx=0 tx=0).stop 2024-05-17T22:41:07.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.094+0000 7f88295d9700 1 -- 172.21.15.12:0/2806450711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8824102e70 msgr2=0x7f8824199410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:41:07.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.094+0000 7f88295d9700 1 --2- 172.21.15.12:0/2806450711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8824102e70 0x7f8824199410 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f881400e9a0 tx=0x7f881400ecb0 comp rx=0 tx=0).stop 2024-05-17T22:41:07.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.094+0000 7f88295d9700 1 -- 172.21.15.12:0/2806450711 shutdown_connections 2024-05-17T22:41:07.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.094+0000 7f88295d9700 1 --2- 172.21.15.12:0/2806450711 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f880c0b29d0 0x7f880c0b4ee0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:07.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.094+0000 7f88295d9700 1 --2- 172.21.15.12:0/2806450711 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88241080a0 0x7f8824199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:07.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.094+0000 7f88295d9700 1 --2- 172.21.15.12:0/2806450711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8824102e70 0x7f8824199410 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:07.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.094+0000 7f88295d9700 1 --2- 172.21.15.12:0/2806450711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8824107750 0x7f8824199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:07.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.094+0000 7f88295d9700 1 -- 172.21.15.12:0/2806450711 >> 172.21.15.12:0/2806450711 conn(0x7f8824079ae0 msgr2=0x7f88241052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:41:07.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.094+0000 7f88295d9700 1 -- 172.21.15.12:0/2806450711 shutdown_connections 2024-05-17T22:41:07.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:07.094+0000 7f88295d9700 1 -- 172.21.15.12:0/2806450711 wait complete. 2024-05-17T22:41:08.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:08 smithi096 ceph-mon[36065]: pgmap v495: 129 pgs: 129 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 17 MiB/s wr, 837 op/s 2024-05-17T22:41:08.321 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:08 smithi012 ceph-mon[31632]: pgmap v495: 129 pgs: 129 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 17 MiB/s wr, 837 op/s 2024-05-17T22:41:08.360 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-17T22:41:08.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:08 smithi151 ceph-mon[36101]: pgmap v495: 129 pgs: 129 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 17 MiB/s wr, 837 op/s 2024-05-17T22:41:08.781 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:41:10.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:10 smithi012 ceph-mon[31632]: pgmap v496: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 19 MiB/s wr, 1.05k op/s 2024-05-17T22:41:10.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:10 smithi096 ceph-mon[36065]: pgmap v496: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 19 MiB/s wr, 1.05k op/s 2024-05-17T22:41:10.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:10 smithi151 ceph-mon[36101]: pgmap v496: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 19 MiB/s wr, 1.05k op/s 2024-05-17T22:41:10.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.588+0000 7f146272e700 1 -- 172.21.15.12:0/3504945247 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f145c11d550 msgr2=0x7f145c11f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:41:10.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.588+0000 7f146272e700 1 --2- 172.21.15.12:0/3504945247 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f145c11d550 0x7f145c11f970 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1454009f90 tx=0x7f145400b330 comp rx=0 tx=0).stop 2024-05-17T22:41:10.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.589+0000 7f146272e700 1 -- 172.21.15.12:0/3504945247 shutdown_connections 2024-05-17T22:41:10.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.589+0000 7f146272e700 1 --2- 172.21.15.12:0/3504945247 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f145c11d550 0x7f145c11f970 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:10.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.589+0000 7f146272e700 1 --2- 172.21.15.12:0/3504945247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f145c10a830 0x7f145c10ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:10.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.589+0000 7f146272e700 1 --2- 172.21.15.12:0/3504945247 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f145c10b180 0x7f145c11d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:10.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.589+0000 7f146272e700 1 -- 172.21.15.12:0/3504945247 >> 172.21.15.12:0/3504945247 conn(0x7f145c06efc0 msgr2=0x7f145c06f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:41:10.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.591+0000 7f146272e700 1 -- 172.21.15.12:0/3504945247 shutdown_connections 2024-05-17T22:41:10.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.593+0000 7f146272e700 1 -- 172.21.15.12:0/3504945247 wait complete. 2024-05-17T22:41:10.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.594+0000 7f146272e700 1 Processor -- start 2024-05-17T22:41:10.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.595+0000 7f146272e700 1 -- start start 2024-05-17T22:41:10.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.595+0000 7f146272e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f145c10a830 0x7f145c11b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:10.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.595+0000 7f146272e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f145c10b180 0x7f145c114ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:10.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.595+0000 7f146272e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f145c11d550 0x7f145c1150a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:10.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.595+0000 7f146272e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f145c122b90 con 0x7f145c10b180 2024-05-17T22:41:10.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.595+0000 7f146272e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f145c122a10 con 0x7f145c10a830 2024-05-17T22:41:10.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.595+0000 7f146272e700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f145c122d10 con 0x7f145c11d550 2024-05-17T22:41:10.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.597+0000 7f1460ccb700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f145c11d550 0x7f145c1150a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:10.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.597+0000 7f1460ccb700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f145c11d550 0x7f145c1150a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:40904/0 (socket says 172.21.15.12:40904) 2024-05-17T22:41:10.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.597+0000 7f1460ccb700 1 -- 172.21.15.12:0/743347303 learned_addr learned my addr 172.21.15.12:0/743347303 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:41:10.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.598+0000 7f145b7fe700 1 --2- 172.21.15.12:0/743347303 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f145c10b180 0x7f145c114ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:10.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.598+0000 7f145bfff700 1 --2- 172.21.15.12:0/743347303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f145c10a830 0x7f145c11b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:10.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.599+0000 7f145bfff700 1 -- 172.21.15.12:0/743347303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f145c11d550 msgr2=0x7f145c1150a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:41:10.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.599+0000 7f145bfff700 1 --2- 172.21.15.12:0/743347303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f145c11d550 0x7f145c1150a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:10.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.599+0000 7f145bfff700 1 -- 172.21.15.12:0/743347303 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f145c10b180 msgr2=0x7f145c114ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:41:10.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.599+0000 7f145bfff700 1 --2- 172.21.15.12:0/743347303 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f145c10b180 0x7f145c114ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:10.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.599+0000 7f145bfff700 1 -- 172.21.15.12:0/743347303 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f145c115900 con 0x7f145c10a830 2024-05-17T22:41:10.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.600+0000 7f145bfff700 1 --2- 172.21.15.12:0/743347303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f145c10a830 0x7f145c11b990 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f144c00e9c0 tx=0x7f144c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:41:10.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.601+0000 7f14597fa700 1 -- 172.21.15.12:0/743347303 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144c00c9c0 con 0x7f145c10a830 2024-05-17T22:41:10.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.601+0000 7f14597fa700 1 -- 172.21.15.12:0/743347303 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f144c00cb60 con 0x7f145c10a830 2024-05-17T22:41:10.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.601+0000 7f14597fa700 1 -- 172.21.15.12:0/743347303 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144c018680 con 0x7f145c10a830 2024-05-17T22:41:10.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.603+0000 7f146272e700 1 -- 172.21.15.12:0/743347303 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f145c1195e0 con 0x7f145c10a830 2024-05-17T22:41:10.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.603+0000 7f146272e700 1 -- 172.21.15.12:0/743347303 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f145c119ad0 con 0x7f145c10a830 2024-05-17T22:41:10.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.608+0000 7f14597fa700 1 -- 172.21.15.12:0/743347303 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f144c018820 con 0x7f145c10a830 2024-05-17T22:41:10.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.609+0000 7f1471109b80 1 Processor -- start 2024-05-17T22:41:10.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.609+0000 7f1471109b80 1 -- start start 2024-05-17T22:41:10.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.610+0000 7f1471109b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572ca1c69f0 0x5572ca349ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:10.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.610+0000 7f14597fa700 1 --2- 172.21.15.12:0/743347303 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f14440b2a90 0x7f14440b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:10.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.611+0000 7f1471109b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572ca34a400 0x5572ca3445c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:10.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.611+0000 7f14597fa700 1 -- 172.21.15.12:0/743347303 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f144c0e2650 con 0x7f145c10a830 2024-05-17T22:41:10.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.611+0000 7f1471109b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572ca344b00 0x5572ca344f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:10.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.611+0000 7f1471109b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5572ca2730d0 con 0x5572ca1c69f0 2024-05-17T22:41:10.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.611+0000 7f1471109b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5572ca262c70 con 0x5572ca344b00 2024-05-17T22:41:10.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.611+0000 7f1471109b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5572ca266cb0 con 0x5572ca34a400 2024-05-17T22:41:10.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.611+0000 7f1460ccb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572ca1c69f0 0x5572ca349ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:10.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.611+0000 7f1460ccb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572ca1c69f0 0x5572ca349ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40384/0 (socket says 172.21.15.12:40384) 2024-05-17T22:41:10.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.611+0000 7f1460ccb700 1 -- 172.21.15.12:0/1858612381 learned_addr learned my addr 172.21.15.12:0/1858612381 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:41:10.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.611+0000 7f145bfff700 1 --2- 172.21.15.12:0/743347303 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f14440b2a90 0x7f14440b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:10.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.611+0000 7f1460ccb700 1 --2- 172.21.15.12:0/1858612381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572ca344b00 0x5572ca344f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:10.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.612+0000 7f145b7fe700 1 --2- 172.21.15.12:0/1858612381 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572ca34a400 0x5572ca3445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:10.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.612+0000 7f1460ccb700 1 -- 172.21.15.12:0/1858612381 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572ca34a400 msgr2=0x5572ca3445c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:41:10.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.612+0000 7f1460ccb700 1 --2- 172.21.15.12:0/1858612381 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572ca34a400 0x5572ca3445c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:10.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.612+0000 7f1460ccb700 1 -- 172.21.15.12:0/1858612381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572ca344b00 msgr2=0x5572ca344f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:41:10.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.612+0000 7f1460ccb700 1 --2- 172.21.15.12:0/1858612381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572ca344b00 0x5572ca344f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:10.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.612+0000 7f1460ccb700 1 -- 172.21.15.12:0/1858612381 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572ca1fc5c0 con 0x5572ca1c69f0 2024-05-17T22:41:10.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.612+0000 7f145bfff700 1 --2- 172.21.15.12:0/743347303 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f14440b2a90 0x7f14440b4fa0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f144c0a7b60 tx=0x7f144c005ce0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:41:10.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.612+0000 7f1460ccb700 1 --2- 172.21.15.12:0/1858612381 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572ca1c69f0 0x5572ca349ec0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1454003fb0 tx=0x7f1454009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a482a7cfe9a6666a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:41:10.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.613+0000 7f14427fc700 1 -- 172.21.15.12:0/1858612381 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145400cc30 con 0x5572ca1c69f0 2024-05-17T22:41:10.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.613+0000 7f14427fc700 1 -- 172.21.15.12:0/1858612381 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f145400cdd0 con 0x5572ca1c69f0 2024-05-17T22:41:10.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.614+0000 7f14427fc700 1 -- 172.21.15.12:0/1858612381 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145401f770 con 0x5572ca1c69f0 2024-05-17T22:41:10.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.615+0000 7f1471109b80 1 -- 172.21.15.12:0/1858612381 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572ca1c69f0 msgr2=0x5572ca349ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:41:10.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.615+0000 7f1471109b80 1 --2- 172.21.15.12:0/1858612381 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572ca1c69f0 0x5572ca349ec0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1454003fb0 tx=0x7f1454009740 comp rx=0 tx=0).stop 2024-05-17T22:41:10.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.616+0000 7f1471109b80 1 -- 172.21.15.12:0/1858612381 shutdown_connections 2024-05-17T22:41:10.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.616+0000 7f1471109b80 1 --2- 172.21.15.12:0/1858612381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572ca344b00 0x5572ca344f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:10.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.616+0000 7f1471109b80 1 --2- 172.21.15.12:0/1858612381 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572ca1c69f0 0x5572ca349ec0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:10.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.616+0000 7f1471109b80 1 --2- 172.21.15.12:0/1858612381 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572ca34a400 0x5572ca3445c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:10.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.616+0000 7f1471109b80 1 -- 172.21.15.12:0/1858612381 >> 172.21.15.12:0/1858612381 conn(0x5572ca2267d0 msgr2=0x5572ca33f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:41:10.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.616+0000 7f1471109b80 1 -- 172.21.15.12:0/1858612381 shutdown_connections 2024-05-17T22:41:10.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.616+0000 7f1471109b80 1 -- 172.21.15.12:0/1858612381 wait complete. 2024-05-17T22:41:10.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.617+0000 7f1471109b80 1 -- start start 2024-05-17T22:41:10.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.617+0000 7f1471109b80 1 Processor -- start 2024-05-17T22:41:10.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.618+0000 7f1471109b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572ca1c69f0 0x5572ca34da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:10.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.618+0000 7f1471109b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572ca344b00 0x5572ca34df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:10.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.618+0000 7f1471109b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572ca34a400 0x5572ca34e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:10.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.618+0000 7f1471109b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5572ca16a150 con 0x5572ca344b00 2024-05-17T22:41:10.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.618+0000 7f1471109b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5572ca204940 con 0x5572ca34a400 2024-05-17T22:41:10.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.618+0000 7f1471109b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5572ca350dc0 con 0x5572ca1c69f0 2024-05-17T22:41:10.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.618+0000 7f1471109b80 10 client.0 _ll_register_callbacks cb 0x5572ca1f9b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:41:10.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.618+0000 7f1460ccb700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572ca1c69f0 0x5572ca34da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:10.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.618+0000 7f1460ccb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572ca34a400 0x5572ca34e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:10.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.618+0000 7f145b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572ca344b00 0x5572ca34df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:10.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.618+0000 7f145b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572ca344b00 0x5572ca34df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40396/0 (socket says 172.21.15.12:40396) 2024-05-17T22:41:10.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.618+0000 7f145b7fe700 1 -- 172.21.15.12:0/3681574856 learned_addr learned my addr 172.21.15.12:0/3681574856 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:41:10.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.620+0000 7f1460ccb700 1 -- 172.21.15.12:0/3681574856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572ca34a400 msgr2=0x5572ca34e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:41:10.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.620+0000 7f1460ccb700 1 --2- 172.21.15.12:0/3681574856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572ca34a400 0x5572ca34e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:10.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.620+0000 7f1460ccb700 1 -- 172.21.15.12:0/3681574856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572ca344b00 msgr2=0x5572ca34df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:41:10.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.620+0000 7f1460ccb700 1 --2- 172.21.15.12:0/3681574856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572ca344b00 0x5572ca34df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:10.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.620+0000 7f1460ccb700 1 -- 172.21.15.12:0/3681574856 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572ca1b3480 con 0x5572ca1c69f0 2024-05-17T22:41:10.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.621+0000 7f145b7fe700 1 --2- 172.21.15.12:0/3681574856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572ca344b00 0x5572ca34df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:41:10.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.621+0000 7f1460ccb700 1 --2- 172.21.15.12:0/3681574856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572ca1c69f0 0x5572ca34da50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x5572ca3404d0 tx=0x7f1454007f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d584965f9149c19c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:41:10.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.621+0000 7f143b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:41:10.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.622+0000 7f143b7fe700 1 -- 172.21.15.12:0/3681574856 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145401f510 con 0x5572ca1c69f0 2024-05-17T22:41:10.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.622+0000 7f143b7fe700 1 -- 172.21.15.12:0/3681574856 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f145401f6b0 con 0x5572ca1c69f0 2024-05-17T22:41:10.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.622+0000 7f1471109b80 1 -- 172.21.15.12:0/3681574856 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f145400cdd0 con 0x5572ca1c69f0 2024-05-17T22:41:10.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.623+0000 7f143b7fe700 1 -- 172.21.15.12:0/3681574856 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f145401f8b0 con 0x5572ca1c69f0 2024-05-17T22:41:10.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.623+0000 7f1471109b80 10 client.25144 fetch_fsmap learned FSMap version 29 2024-05-17T22:41:10.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.623+0000 7f1471109b80 1 -- 172.21.15.12:0/3681574856 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5572ca35b920 con 0x5572ca1c69f0 2024-05-17T22:41:10.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.623+0000 7f143b7fe700 1 -- 172.21.15.12:0/3681574856 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f145400cdd0 con 0x5572ca1c69f0 2024-05-17T22:41:10.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.624+0000 7f143b7fe700 1 -- 172.21.15.12:0/3681574856 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1454011070 con 0x5572ca1c69f0 2024-05-17T22:41:10.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.624+0000 7f1471109b80 10 client.25144 fetch_fsmap finished waiting for FSMap version 29 2024-05-17T22:41:10.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.624+0000 7f1471109b80 10 client.25144 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-17T22:41:10.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.624+0000 7f1471109b80 20 client.25144 populate_metadata read hostname 'smithi012' 2024-05-17T22:41:10.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.624+0000 7f1471109b80 1 --2- 172.21.15.12:0/3681574856 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5572ca35ddf0 0x5572ca360210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:10.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.624+0000 7f1471109b80 4 client.25144 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:41:10.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.624+0000 7f1471109b80 1 -- 172.21.15.12:0/3681574856 --> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572ca275020 con 0x5572ca35ddf0 2024-05-17T22:41:10.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.624+0000 7f145b7fe700 1 --2- 172.21.15.12:0/3681574856 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5572ca35ddf0 0x5572ca360210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:10.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.625+0000 7f143b7fe700 10 client.25144 ms_handle_connect on v2:172.21.15.96:6836/4222574338 2024-05-17T22:41:10.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.625+0000 7f145b7fe700 1 --2- 172.21.15.12:0/3681574856 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5572ca35ddf0 0x5572ca360210 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e2293efa7b2a1e08 server_cookie=987b5503b02a071c in_seq=0 out_seq=0 2024-05-17T22:41:10.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.625+0000 7f143b7fe700 1 -- 172.21.15.12:0/3681574856 <== mds.1 v2:172.21.15.96:6836/4222574338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5572ca275020 con 0x5572ca35ddf0 2024-05-17T22:41:10.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.625+0000 7f143b7fe700 10 client.25144 handle_command_reply: tid=0 2024-05-17T22:41:10.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.625+0000 7f1471109b80 1 client.25144 shutdown 2024-05-17T22:41:10.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.626+0000 7f1471109b80 1 -- 172.21.15.12:0/3681574856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572ca1c69f0 msgr2=0x5572ca34da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:41:10.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.626+0000 7f1471109b80 1 --2- 172.21.15.12:0/3681574856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572ca1c69f0 0x5572ca34da50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x5572ca3404d0 tx=0x7f1454007f60 comp rx=0 tx=0).stop 2024-05-17T22:41:10.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.626+0000 7f1471109b80 1 -- 172.21.15.12:0/3681574856 shutdown_connections 2024-05-17T22:41:10.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.626+0000 7f1471109b80 1 --2- 172.21.15.12:0/3681574856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572ca34a400 0x5572ca34e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:10.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.626+0000 7f1471109b80 1 --2- 172.21.15.12:0/3681574856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572ca1c69f0 0x5572ca34da50 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:10.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.626+0000 7f1471109b80 1 --2- 172.21.15.12:0/3681574856 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5572ca35ddf0 0x5572ca360210 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:10.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.626+0000 7f1471109b80 1 --2- 172.21.15.12:0/3681574856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572ca344b00 0x5572ca34df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:10.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.626+0000 7f143b7fe700 0 client.25144 ms_handle_reset on v2:172.21.15.96:6836/4222574338 2024-05-17T22:41:10.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.626+0000 7f1471109b80 1 -- 172.21.15.12:0/3681574856 >> 172.21.15.12:0/3681574856 conn(0x5572ca2267d0 msgr2=0x5572ca33f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:41:10.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.626+0000 7f1471109b80 1 -- 172.21.15.12:0/3681574856 shutdown_connections 2024-05-17T22:41:10.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.626+0000 7f1471109b80 1 -- 172.21.15.12:0/3681574856 wait complete. 2024-05-17T22:41:10.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.626+0000 7f1471109b80 20 client.25144 trim_cache size 0 max 16384 2024-05-17T22:41:10.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.652+0000 7f1471109b80 1 Processor -- start 2024-05-17T22:41:10.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.652+0000 7f1471109b80 1 -- start start 2024-05-17T22:41:10.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.652+0000 7f1471109b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572ca1c69f0 0x5572ca3445c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:10.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.652+0000 7f1471109b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572ca344b00 0x5572ca3494c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:10.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.652+0000 7f1471109b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572ca34a400 0x5572ca34e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:10.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.652+0000 7f1471109b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1448011470 con 0x5572ca344b00 2024-05-17T22:41:10.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.652+0000 7f1471109b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f144800ed00 con 0x5572ca34a400 2024-05-17T22:41:10.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.652+0000 7f1471109b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f144800cb50 con 0x5572ca1c69f0 2024-05-17T22:41:10.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.653+0000 7f1460ccb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572ca34a400 0x5572ca34e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:10.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.653+0000 7f1460ccb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572ca34a400 0x5572ca34e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:53606/0 (socket says 172.21.15.12:53606) 2024-05-17T22:41:10.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.653+0000 7f1460ccb700 1 -- 172.21.15.12:0/4265325787 learned_addr learned my addr 172.21.15.12:0/4265325787 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:41:10.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.653+0000 7f1460ccb700 1 --2- 172.21.15.12:0/4265325787 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572ca1c69f0 0x5572ca3445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:10.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.653+0000 7f145b7fe700 1 --2- 172.21.15.12:0/4265325787 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572ca344b00 0x5572ca3494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:10.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.653+0000 7f1460ccb700 1 -- 172.21.15.12:0/4265325787 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572ca1c69f0 msgr2=0x5572ca3445c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:41:10.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.653+0000 7f1460ccb700 1 --2- 172.21.15.12:0/4265325787 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572ca1c69f0 0x5572ca3445c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:10.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.653+0000 7f1460ccb700 1 -- 172.21.15.12:0/4265325787 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572ca344b00 msgr2=0x5572ca3494c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:41:10.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.653+0000 7f1460ccb700 1 --2- 172.21.15.12:0/4265325787 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572ca344b00 0x5572ca3494c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:10.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.653+0000 7f1460ccb700 1 -- 172.21.15.12:0/4265325787 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572ca349a50 con 0x5572ca34a400 2024-05-17T22:41:10.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.653+0000 7f1460ccb700 1 --2- 172.21.15.12:0/4265325787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572ca34a400 0x5572ca34e320 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f14540054b0 tx=0x7f145400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ec00d618a5f20a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:41:10.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.654+0000 7f1441ffb700 1 -- 172.21.15.12:0/4265325787 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145403e070 con 0x5572ca34a400 2024-05-17T22:41:10.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.654+0000 7f1441ffb700 1 -- 172.21.15.12:0/4265325787 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f145403f040 con 0x5572ca34a400 2024-05-17T22:41:10.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.654+0000 7f1441ffb700 1 -- 172.21.15.12:0/4265325787 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1454005c20 con 0x5572ca34a400 2024-05-17T22:41:10.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.655+0000 7f1471109b80 1 -- 172.21.15.12:0/4265325787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572ca34a400 msgr2=0x5572ca34e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:41:10.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.655+0000 7f1471109b80 1 --2- 172.21.15.12:0/4265325787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572ca34a400 0x5572ca34e320 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f14540054b0 tx=0x7f145400b330 comp rx=0 tx=0).stop 2024-05-17T22:41:10.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.655+0000 7f1471109b80 1 -- 172.21.15.12:0/4265325787 shutdown_connections 2024-05-17T22:41:10.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.655+0000 7f1471109b80 1 --2- 172.21.15.12:0/4265325787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572ca34a400 0x5572ca34e320 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:10.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.655+0000 7f1471109b80 1 --2- 172.21.15.12:0/4265325787 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572ca1c69f0 0x5572ca3445c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:10.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.655+0000 7f1471109b80 1 --2- 172.21.15.12:0/4265325787 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572ca344b00 0x5572ca3494c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:10.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.655+0000 7f1471109b80 1 -- 172.21.15.12:0/4265325787 >> 172.21.15.12:0/4265325787 conn(0x5572ca2267d0 msgr2=0x5572ca347e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:41:10.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.655+0000 7f1471109b80 1 -- 172.21.15.12:0/4265325787 shutdown_connections 2024-05-17T22:41:10.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.655+0000 7f1471109b80 1 -- 172.21.15.12:0/4265325787 wait complete. 2024-05-17T22:41:10.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.656+0000 7f1471109b80 1 -- start start 2024-05-17T22:41:10.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.656+0000 7f1471109b80 1 Processor -- start 2024-05-17T22:41:10.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.656+0000 7f1471109b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572ca1c69f0 0x5572ca3635a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:10.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.656+0000 7f1471109b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572ca344b00 0x5572ca3608d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:10.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.656+0000 7f1471109b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572ca34a400 0x5572ca360e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:10.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.656+0000 7f1471109b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f144800d1c0 con 0x5572ca1c69f0 2024-05-17T22:41:10.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.656+0000 7f1471109b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f144800e630 con 0x5572ca34a400 2024-05-17T22:41:10.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.656+0000 7f1471109b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f144800d890 con 0x5572ca344b00 2024-05-17T22:41:10.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.656+0000 7f1471109b80 10 client.0 _ll_register_callbacks cb 0x5572ca1f9b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:41:10.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.657+0000 7f145b7fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572ca344b00 0x5572ca3608d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:10.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.657+0000 7f1460ccb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572ca1c69f0 0x5572ca3635a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:10.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.657+0000 7f145b7fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572ca344b00 0x5572ca3608d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:40938/0 (socket says 172.21.15.12:40938) 2024-05-17T22:41:10.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.657+0000 7f145b7fe700 1 -- 172.21.15.12:0/2617436467 learned_addr learned my addr 172.21.15.12:0/2617436467 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:41:10.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.657+0000 7f1460ccb700 1 --2- 172.21.15.12:0/2617436467 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572ca34a400 0x5572ca360e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:10.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.657+0000 7f145b7fe700 1 -- 172.21.15.12:0/2617436467 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572ca34a400 msgr2=0x5572ca360e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:41:10.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.657+0000 7f145b7fe700 1 --2- 172.21.15.12:0/2617436467 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572ca34a400 0x5572ca360e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:10.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.657+0000 7f145b7fe700 1 -- 172.21.15.12:0/2617436467 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572ca1c69f0 msgr2=0x5572ca3635a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:41:10.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.657+0000 7f145b7fe700 1 --2- 172.21.15.12:0/2617436467 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572ca1c69f0 0x5572ca3635a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:10.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.657+0000 7f145b7fe700 1 -- 172.21.15.12:0/2617436467 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572ca361380 con 0x5572ca344b00 2024-05-17T22:41:10.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.657+0000 7f145b7fe700 1 --2- 172.21.15.12:0/2617436467 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572ca344b00 0x5572ca3608d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x5572ca273190 tx=0x7f1450003d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29fde1230d98ab7d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:41:10.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.657+0000 7f143b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:41:10.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.658+0000 7f143b7fe700 1 -- 172.21.15.12:0/2617436467 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1450028060 con 0x5572ca344b00 2024-05-17T22:41:10.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.658+0000 7f1471109b80 1 -- 172.21.15.12:0/2617436467 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f145403f040 con 0x5572ca344b00 2024-05-17T22:41:10.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.658+0000 7f143b7fe700 1 -- 172.21.15.12:0/2617436467 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f145001b780 con 0x5572ca344b00 2024-05-17T22:41:10.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.658+0000 7f143b7fe700 1 -- 172.21.15.12:0/2617436467 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f145002f050 con 0x5572ca344b00 2024-05-17T22:41:10.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.658+0000 7f1471109b80 10 client.25150 fetch_fsmap learned FSMap version 29 2024-05-17T22:41:10.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.658+0000 7f1471109b80 1 -- 172.21.15.12:0/2617436467 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5572ca3c80e0 con 0x5572ca344b00 2024-05-17T22:41:10.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.658+0000 7f143b7fe700 1 -- 172.21.15.12:0/2617436467 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f145403f040 con 0x5572ca344b00 2024-05-17T22:41:10.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.658+0000 7f143b7fe700 1 -- 172.21.15.12:0/2617436467 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1450022030 con 0x5572ca344b00 2024-05-17T22:41:10.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.658+0000 7f1471109b80 10 client.25150 fetch_fsmap finished waiting for FSMap version 29 2024-05-17T22:41:10.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.658+0000 7f1471109b80 10 client.25150 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-17T22:41:10.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.658+0000 7f1471109b80 20 client.25150 populate_metadata read hostname 'smithi012' 2024-05-17T22:41:10.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.658+0000 7f1471109b80 1 --2- 172.21.15.12:0/2617436467 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5572ca3c87d0 0x5572ca3cabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:10.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.658+0000 7f1471109b80 4 client.25150 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:41:10.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.658+0000 7f1471109b80 1 -- 172.21.15.12:0/2617436467 --> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5572ca26e800 con 0x5572ca3c87d0 2024-05-17T22:41:10.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.659+0000 7f1460ccb700 1 --2- 172.21.15.12:0/2617436467 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5572ca3c87d0 0x5572ca3cabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:10.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.659+0000 7f143b7fe700 10 client.25150 ms_handle_connect on v2:172.21.15.96:6836/4222574338 2024-05-17T22:41:10.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.659+0000 7f1460ccb700 1 --2- 172.21.15.12:0/2617436467 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5572ca3c87d0 0x5572ca3cabf0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=40d5b60999eec459 server_cookie=a93feb58a1eb0be1 in_seq=0 out_seq=0 2024-05-17T22:41:10.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.659+0000 7f143b7fe700 1 -- 172.21.15.12:0/2617436467 <== mds.1 v2:172.21.15.96:6836/4222574338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5572ca26e800 con 0x5572ca3c87d0 2024-05-17T22:41:10.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.659+0000 7f143b7fe700 10 client.25150 handle_command_reply: tid=0 2024-05-17T22:41:10.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.661+0000 7f1471109b80 1 client.25150 shutdown 2024-05-17T22:41:10.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.661+0000 7f1471109b80 1 -- 172.21.15.12:0/2617436467 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572ca344b00 msgr2=0x5572ca3608d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:41:10.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.661+0000 7f1471109b80 1 --2- 172.21.15.12:0/2617436467 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572ca344b00 0x5572ca3608d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x5572ca273190 tx=0x7f1450003d40 comp rx=0 tx=0).stop 2024-05-17T22:41:10.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.662+0000 7f1471109b80 1 -- 172.21.15.12:0/2617436467 shutdown_connections 2024-05-17T22:41:10.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.662+0000 7f1471109b80 1 --2- 172.21.15.12:0/2617436467 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572ca34a400 0x5572ca360e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:10.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.662+0000 7f1471109b80 1 --2- 172.21.15.12:0/2617436467 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5572ca3c87d0 0x5572ca3cabf0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:10.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.662+0000 7f1471109b80 1 --2- 172.21.15.12:0/2617436467 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572ca1c69f0 0x5572ca3635a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:10.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.662+0000 7f1471109b80 1 --2- 172.21.15.12:0/2617436467 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572ca344b00 0x5572ca3608d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:10.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.662+0000 7f1471109b80 1 -- 172.21.15.12:0/2617436467 >> 172.21.15.12:0/2617436467 conn(0x5572ca2267d0 msgr2=0x5572ca339110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:41:10.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.662+0000 7f143b7fe700 0 client.25150 ms_handle_reset on v2:172.21.15.96:6836/4222574338 2024-05-17T22:41:10.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.662+0000 7f1471109b80 1 -- 172.21.15.12:0/2617436467 shutdown_connections 2024-05-17T22:41:10.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.662+0000 7f1471109b80 1 -- 172.21.15.12:0/2617436467 wait complete. 2024-05-17T22:41:10.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.662+0000 7f1471109b80 20 client.25150 trim_cache size 0 max 16384 2024-05-17T22:41:10.667 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:41:10.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.663+0000 7f146272e700 1 -- 172.21.15.12:0/743347303 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f14440b2a90 msgr2=0x7f14440b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:41:10.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.663+0000 7f146272e700 1 --2- 172.21.15.12:0/743347303 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f14440b2a90 0x7f14440b4fa0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f144c0a7b60 tx=0x7f144c005ce0 comp rx=0 tx=0).stop 2024-05-17T22:41:10.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.663+0000 7f146272e700 1 -- 172.21.15.12:0/743347303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f145c10a830 msgr2=0x7f145c11b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:41:10.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.663+0000 7f146272e700 1 --2- 172.21.15.12:0/743347303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f145c10a830 0x7f145c11b990 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f144c00e9c0 tx=0x7f144c00ecd0 comp rx=0 tx=0).stop 2024-05-17T22:41:10.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.663+0000 7f146272e700 1 -- 172.21.15.12:0/743347303 shutdown_connections 2024-05-17T22:41:10.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.663+0000 7f146272e700 1 --2- 172.21.15.12:0/743347303 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f14440b2a90 0x7f14440b4fa0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:10.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.663+0000 7f146272e700 1 --2- 172.21.15.12:0/743347303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f145c11d550 0x7f145c1150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:10.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.663+0000 7f146272e700 1 --2- 172.21.15.12:0/743347303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f145c10a830 0x7f145c11b990 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:10.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.663+0000 7f146272e700 1 --2- 172.21.15.12:0/743347303 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f145c10b180 0x7f145c114ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:10.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.663+0000 7f146272e700 1 -- 172.21.15.12:0/743347303 >> 172.21.15.12:0/743347303 conn(0x7f145c06efc0 msgr2=0x7f145c072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:41:10.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.664+0000 7f146272e700 1 -- 172.21.15.12:0/743347303 shutdown_connections 2024-05-17T22:41:10.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:10.664+0000 7f146272e700 1 -- 172.21.15.12:0/743347303 wait complete. 2024-05-17T22:41:11.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 110.0 s, 2242.4 tps, lat 44.815 ms stddev 53.956 2024-05-17T22:41:11.479 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-17T22:41:11.879 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:41:12.271 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:12 smithi012 ceph-mon[31632]: pgmap v497: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 722 op/s 2024-05-17T22:41:12.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:12 smithi096 ceph-mon[36065]: pgmap v497: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 722 op/s 2024-05-17T22:41:12.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:12 smithi151 ceph-mon[36101]: pgmap v497: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 722 op/s 2024-05-17T22:41:14.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:14 smithi096 ceph-mon[36065]: pgmap v498: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 MiB/s wr, 679 op/s 2024-05-17T22:41:14.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:14 smithi012 ceph-mon[31632]: pgmap v498: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 MiB/s wr, 679 op/s 2024-05-17T22:41:14.492 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:14 smithi151 ceph-mon[36101]: pgmap v498: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 MiB/s wr, 679 op/s 2024-05-17T22:41:15.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.914+0000 7f97dd96f700 1 -- 172.21.15.12:0/1162149332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97d810a830 msgr2=0x7f97d810ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:41:15.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.914+0000 7f97dd96f700 1 --2- 172.21.15.12:0/1162149332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97d810a830 0x7f97d810ac40 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f97c8009990 tx=0x7f97c8009ca0 comp rx=0 tx=0).stop 2024-05-17T22:41:15.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.915+0000 7f97dd96f700 1 -- 172.21.15.12:0/1162149332 shutdown_connections 2024-05-17T22:41:15.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.915+0000 7f97dd96f700 1 --2- 172.21.15.12:0/1162149332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97d811d550 0x7f97d811f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:15.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.915+0000 7f97dd96f700 1 --2- 172.21.15.12:0/1162149332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97d810a830 0x7f97d810ac40 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:15.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.915+0000 7f97dd96f700 1 --2- 172.21.15.12:0/1162149332 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97d810b180 0x7f97d811d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:15.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.915+0000 7f97dd96f700 1 -- 172.21.15.12:0/1162149332 >> 172.21.15.12:0/1162149332 conn(0x7f97d806efc0 msgr2=0x7f97d806f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:41:15.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.916+0000 7f97dd96f700 1 -- 172.21.15.12:0/1162149332 shutdown_connections 2024-05-17T22:41:15.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.916+0000 7f97dd96f700 1 -- 172.21.15.12:0/1162149332 wait complete. 2024-05-17T22:41:15.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.917+0000 7f97dd96f700 1 Processor -- start 2024-05-17T22:41:15.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.917+0000 7f97dd96f700 1 -- start start 2024-05-17T22:41:15.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.917+0000 7f97dd96f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97d810a830 0x7f97d81a95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:15.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.917+0000 7f97dd96f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97d810b180 0x7f97d81a9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:15.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.917+0000 7f97dd96f700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97d811d550 0x7f97d81ade20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:15.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.917+0000 7f97dd96f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f97d8122bf0 con 0x7f97d810a830 2024-05-17T22:41:15.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.917+0000 7f97dd96f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f97d8122a70 con 0x7f97d810b180 2024-05-17T22:41:15.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.917+0000 7f97dd96f700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f97d8122d70 con 0x7f97d811d550 2024-05-17T22:41:15.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.918+0000 7f97d77fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97d811d550 0x7f97d81ade20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:15.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.918+0000 7f97d67fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97d810b180 0x7f97d81a9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:15.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.918+0000 7f97d77fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97d811d550 0x7f97d81ade20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:40952/0 (socket says 172.21.15.12:40952) 2024-05-17T22:41:15.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.918+0000 7f97d77fe700 1 -- 172.21.15.12:0/4007479642 learned_addr learned my addr 172.21.15.12:0/4007479642 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:41:15.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.918+0000 7f97d6ffd700 1 --2- 172.21.15.12:0/4007479642 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97d810a830 0x7f97d81a95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:15.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.918+0000 7f97d67fc700 1 -- 172.21.15.12:0/4007479642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97d811d550 msgr2=0x7f97d81ade20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:41:15.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.918+0000 7f97d67fc700 1 --2- 172.21.15.12:0/4007479642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97d811d550 0x7f97d81ade20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:15.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.918+0000 7f97d67fc700 1 -- 172.21.15.12:0/4007479642 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97d810a830 msgr2=0x7f97d81a95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:41:15.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.918+0000 7f97d67fc700 1 --2- 172.21.15.12:0/4007479642 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97d810a830 0x7f97d81a95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:15.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.918+0000 7f97d67fc700 1 -- 172.21.15.12:0/4007479642 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d81ae4c0 con 0x7f97d810b180 2024-05-17T22:41:15.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.918+0000 7f97d6ffd700 1 --2- 172.21.15.12:0/4007479642 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97d810a830 0x7f97d81a95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:41:15.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.919+0000 7f97d67fc700 1 --2- 172.21.15.12:0/4007479642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97d810b180 0x7f97d81a9b90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f97cc00e9f0 tx=0x7f97cc00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:41:15.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.919+0000 7f97dc96d700 1 -- 172.21.15.12:0/4007479642 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc00cbd0 con 0x7f97d810b180 2024-05-17T22:41:15.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.919+0000 7f97dc96d700 1 -- 172.21.15.12:0/4007479642 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97cc00cd70 con 0x7f97d810b180 2024-05-17T22:41:15.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.920+0000 7f97dd96f700 1 -- 172.21.15.12:0/4007479642 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97d81ae7b0 con 0x7f97d810b180 2024-05-17T22:41:15.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.920+0000 7f97dc96d700 1 -- 172.21.15.12:0/4007479642 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc018680 con 0x7f97d810b180 2024-05-17T22:41:15.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.920+0000 7f97dd96f700 1 -- 172.21.15.12:0/4007479642 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97d81b60d0 con 0x7f97d810b180 2024-05-17T22:41:15.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.924+0000 7f97dc96d700 1 -- 172.21.15.12:0/4007479642 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f97cc010430 con 0x7f97d810b180 2024-05-17T22:41:15.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.926+0000 7f97dc96d700 1 --2- 172.21.15.12:0/4007479642 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f97c00b2980 0x7f97c00b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:15.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.927+0000 7f97dc96d700 1 -- 172.21.15.12:0/4007479642 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f97cc0e26e0 con 0x7f97d810b180 2024-05-17T22:41:15.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.928+0000 7f97ec34ab80 1 Processor -- start 2024-05-17T22:41:15.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.928+0000 7f97d6ffd700 1 --2- 172.21.15.12:0/4007479642 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f97c00b2980 0x7f97c00b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:15.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.928+0000 7f97ec34ab80 1 -- start start 2024-05-17T22:41:15.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.929+0000 7f97ec34ab80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568b582f9f0 0x5568b59b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:15.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.929+0000 7f97d6ffd700 1 --2- 172.21.15.12:0/4007479642 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f97c00b2980 0x7f97c00b4e90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f97c8000c00 tx=0x7f97c8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:41:15.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.929+0000 7f97ec34ab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5568b59b3aa0 0x5568b59adc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:15.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.929+0000 7f97ec34ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5568b59ae170 0x5568b59ae580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:15.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.929+0000 7f97ec34ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5568b58dc290 con 0x5568b59ae170 2024-05-17T22:41:15.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.929+0000 7f97ec34ab80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5568b58cbfd0 con 0x5568b582f9f0 2024-05-17T22:41:15.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.929+0000 7f97ec34ab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5568b58cfe70 con 0x5568b59b3aa0 2024-05-17T22:41:15.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.929+0000 7f97d77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5568b59ae170 0x5568b59ae580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:15.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.929+0000 7f97d77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5568b59ae170 0x5568b59ae580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40436/0 (socket says 172.21.15.12:40436) 2024-05-17T22:41:15.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.929+0000 7f97d77fe700 1 -- 172.21.15.12:0/117075057 learned_addr learned my addr 172.21.15.12:0/117075057 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:41:15.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.930+0000 7f97d77fe700 1 -- 172.21.15.12:0/117075057 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5568b59b3aa0 msgr2=0x5568b59adc30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T22:41:15.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.930+0000 7f97d77fe700 1 --2- 172.21.15.12:0/117075057 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5568b59b3aa0 0x5568b59adc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:15.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.930+0000 7f97d77fe700 1 -- 172.21.15.12:0/117075057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568b582f9f0 msgr2=0x5568b59b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:41:15.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.930+0000 7f97d77fe700 1 --2- 172.21.15.12:0/117075057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568b582f9f0 0x5568b59b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:15.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.930+0000 7f97d77fe700 1 -- 172.21.15.12:0/117075057 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568b5875590 con 0x5568b59ae170 2024-05-17T22:41:15.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.930+0000 7f97d77fe700 1 --2- 172.21.15.12:0/117075057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5568b59ae170 0x5568b59ae580 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f97d0009f10 tx=0x7f97d00091b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1312e71e2fe85e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:41:15.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.931+0000 7f97bdffb700 1 -- 172.21.15.12:0/117075057 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d0007e40 con 0x5568b59ae170 2024-05-17T22:41:15.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.931+0000 7f97bdffb700 1 -- 172.21.15.12:0/117075057 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97d0004bb0 con 0x5568b59ae170 2024-05-17T22:41:15.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.931+0000 7f97bdffb700 1 -- 172.21.15.12:0/117075057 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d00158c0 con 0x5568b59ae170 2024-05-17T22:41:15.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.932+0000 7f97ec34ab80 1 -- 172.21.15.12:0/117075057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5568b59ae170 msgr2=0x5568b59ae580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:41:15.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.932+0000 7f97ec34ab80 1 --2- 172.21.15.12:0/117075057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5568b59ae170 0x5568b59ae580 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f97d0009f10 tx=0x7f97d00091b0 comp rx=0 tx=0).stop 2024-05-17T22:41:15.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.932+0000 7f97ec34ab80 1 -- 172.21.15.12:0/117075057 shutdown_connections 2024-05-17T22:41:15.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.932+0000 7f97ec34ab80 1 --2- 172.21.15.12:0/117075057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5568b59ae170 0x5568b59ae580 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:15.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.932+0000 7f97ec34ab80 1 --2- 172.21.15.12:0/117075057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568b582f9f0 0x5568b59b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:15.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.932+0000 7f97ec34ab80 1 --2- 172.21.15.12:0/117075057 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5568b59b3aa0 0x5568b59adc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:15.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.932+0000 7f97ec34ab80 1 -- 172.21.15.12:0/117075057 >> 172.21.15.12:0/117075057 conn(0x5568b5873f00 msgr2=0x5568b59a8b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:41:15.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.933+0000 7f97ec34ab80 1 -- 172.21.15.12:0/117075057 shutdown_connections 2024-05-17T22:41:15.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.933+0000 7f97ec34ab80 1 -- 172.21.15.12:0/117075057 wait complete. 2024-05-17T22:41:15.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.934+0000 7f97ec34ab80 1 -- start start 2024-05-17T22:41:15.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.934+0000 7f97ec34ab80 1 Processor -- start 2024-05-17T22:41:15.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.934+0000 7f97ec34ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5568b582f9f0 0x5568b59b72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:15.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.934+0000 7f97ec34ab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5568b59ae170 0x5568b59b7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:15.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.935+0000 7f97ec34ab80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568b59b3aa0 0x5568b59b7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:15.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.935+0000 7f97ec34ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5568b57d3150 con 0x5568b582f9f0 2024-05-17T22:41:15.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.935+0000 7f97ec34ab80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5568b586ce80 con 0x5568b59b3aa0 2024-05-17T22:41:15.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.935+0000 7f97ec34ab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5568b59ba390 con 0x5568b59ae170 2024-05-17T22:41:15.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.935+0000 7f97ec34ab80 10 client.0 _ll_register_callbacks cb 0x5568b5866f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:41:15.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.935+0000 7f97d77fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568b59b3aa0 0x5568b59b7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:15.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.935+0000 7f97d6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5568b582f9f0 0x5568b59b72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:15.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.935+0000 7f97d77fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568b59b3aa0 0x5568b59b7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:53658/0 (socket says 172.21.15.12:53658) 2024-05-17T22:41:15.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.935+0000 7f97d77fe700 1 -- 172.21.15.12:0/3456232474 learned_addr learned my addr 172.21.15.12:0/3456232474 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:41:15.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.935+0000 7f97d67fc700 1 --2- 172.21.15.12:0/3456232474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5568b59ae170 0x5568b59b7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:15.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.935+0000 7f97d77fe700 1 -- 172.21.15.12:0/3456232474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5568b59ae170 msgr2=0x5568b59b7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:41:15.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.935+0000 7f97d77fe700 1 --2- 172.21.15.12:0/3456232474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5568b59ae170 0x5568b59b7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:15.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.935+0000 7f97d77fe700 1 -- 172.21.15.12:0/3456232474 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5568b582f9f0 msgr2=0x5568b59b72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:41:15.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.935+0000 7f97d77fe700 1 --2- 172.21.15.12:0/3456232474 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5568b582f9f0 0x5568b59b72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:15.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.935+0000 7f97d77fe700 1 -- 172.21.15.12:0/3456232474 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568b581c480 con 0x5568b59b3aa0 2024-05-17T22:41:15.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.937+0000 7f97d77fe700 1 --2- 172.21.15.12:0/3456232474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568b59b3aa0 0x5568b59b7d40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f97d00158a0 tx=0x7f97d00158d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93bb97a3b2d5247d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:41:15.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.937+0000 7f97b6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:41:15.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.937+0000 7f97b6ffd700 1 -- 172.21.15.12:0/3456232474 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d0022020 con 0x5568b59b3aa0 2024-05-17T22:41:15.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.937+0000 7f97b6ffd700 1 -- 172.21.15.12:0/3456232474 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f97d0015c80 con 0x5568b59b3aa0 2024-05-17T22:41:15.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.937+0000 7f97ec34ab80 1 -- 172.21.15.12:0/3456232474 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97d0004bb0 con 0x5568b59b3aa0 2024-05-17T22:41:15.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.938+0000 7f97b6ffd700 1 -- 172.21.15.12:0/3456232474 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f97d0015480 con 0x5568b59b3aa0 2024-05-17T22:41:15.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.938+0000 7f97ec34ab80 10 client.25220 fetch_fsmap learned FSMap version 29 2024-05-17T22:41:15.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.938+0000 7f97ec34ab80 1 -- 172.21.15.12:0/3456232474 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5568b59c4d20 con 0x5568b59b3aa0 2024-05-17T22:41:15.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.938+0000 7f97b6ffd700 1 -- 172.21.15.12:0/3456232474 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97d0004bb0 con 0x5568b59b3aa0 2024-05-17T22:41:15.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.938+0000 7f97b6ffd700 1 -- 172.21.15.12:0/3456232474 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f97d002f020 con 0x5568b59b3aa0 2024-05-17T22:41:15.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.938+0000 7f97ec34ab80 10 client.25220 fetch_fsmap finished waiting for FSMap version 29 2024-05-17T22:41:15.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.938+0000 7f97ec34ab80 10 client.25220 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:41:15.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.938+0000 7f97ec34ab80 20 client.25220 populate_metadata read hostname 'smithi012' 2024-05-17T22:41:15.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.938+0000 7f97ec34ab80 1 --2- 172.21.15.12:0/3456232474 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5568b59c71f0 0x5568b59c9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:15.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.939+0000 7f97ec34ab80 4 client.25220 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:41:15.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.939+0000 7f97ec34ab80 1 -- 172.21.15.12:0/3456232474 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568b58de1e0 con 0x5568b59c71f0 2024-05-17T22:41:15.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.939+0000 7f97d6ffd700 1 --2- 172.21.15.12:0/3456232474 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5568b59c71f0 0x5568b59c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:15.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.940+0000 7f97d6ffd700 1 --2- 172.21.15.12:0/3456232474 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5568b59c71f0 0x5568b59c9610 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=406ebed64babf48c server_cookie=6f928ce2be765ba9 in_seq=0 out_seq=0 2024-05-17T22:41:15.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.940+0000 7f97b6ffd700 10 client.25220 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:41:15.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.940+0000 7f97b6ffd700 1 -- 172.21.15.12:0/3456232474 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5568b58de1e0 con 0x5568b59c71f0 2024-05-17T22:41:15.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.940+0000 7f97b6ffd700 10 client.25220 handle_command_reply: tid=0 2024-05-17T22:41:15.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.940+0000 7f97ec34ab80 1 client.25220 shutdown 2024-05-17T22:41:15.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.941+0000 7f97ec34ab80 1 -- 172.21.15.12:0/3456232474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568b59b3aa0 msgr2=0x5568b59b7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:41:15.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.941+0000 7f97ec34ab80 1 --2- 172.21.15.12:0/3456232474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568b59b3aa0 0x5568b59b7d40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f97d00158a0 tx=0x7f97d00158d0 comp rx=0 tx=0).stop 2024-05-17T22:41:15.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.941+0000 7f97ec34ab80 1 -- 172.21.15.12:0/3456232474 shutdown_connections 2024-05-17T22:41:15.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.941+0000 7f97ec34ab80 1 --2- 172.21.15.12:0/3456232474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568b59b3aa0 0x5568b59b7d40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:15.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.941+0000 7f97ec34ab80 1 --2- 172.21.15.12:0/3456232474 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5568b59c71f0 0x5568b59c9610 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:15.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.941+0000 7f97ec34ab80 1 --2- 172.21.15.12:0/3456232474 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5568b582f9f0 0x5568b59b72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:15.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.941+0000 7f97ec34ab80 1 --2- 172.21.15.12:0/3456232474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5568b59ae170 0x5568b59b7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:15.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.941+0000 7f97b6ffd700 0 client.25220 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:41:15.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.941+0000 7f97ec34ab80 1 -- 172.21.15.12:0/3456232474 >> 172.21.15.12:0/3456232474 conn(0x5568b5873f00 msgr2=0x5568b59a2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:41:15.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.941+0000 7f97ec34ab80 1 -- 172.21.15.12:0/3456232474 shutdown_connections 2024-05-17T22:41:15.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.941+0000 7f97ec34ab80 1 -- 172.21.15.12:0/3456232474 wait complete. 2024-05-17T22:41:15.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.942+0000 7f97ec34ab80 20 client.25220 trim_cache size 0 max 16384 2024-05-17T22:41:15.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.965+0000 7f97ec34ab80 1 Processor -- start 2024-05-17T22:41:15.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.965+0000 7f97ec34ab80 1 -- start start 2024-05-17T22:41:15.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.966+0000 7f97ec34ab80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568b582f9f0 0x5568b59adc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:15.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.966+0000 7f97ec34ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5568b59ae170 0x5568b59b15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:15.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.966+0000 7f97ec34ab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5568b59b3aa0 0x5568b59b98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:15.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.966+0000 7f97d6ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568b582f9f0 0x5568b59adc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:15.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.966+0000 7f97d6ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568b582f9f0 0x5568b59adc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:53666/0 (socket says 172.21.15.12:53666) 2024-05-17T22:41:15.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.966+0000 7f97d6ffd700 1 -- 172.21.15.12:0/65375719 learned_addr learned my addr 172.21.15.12:0/65375719 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:41:15.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.966+0000 7f97ec34ab80 1 -- 172.21.15.12:0/65375719 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f97d81c1620 con 0x5568b59ae170 2024-05-17T22:41:15.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.966+0000 7f97ec34ab80 1 -- 172.21.15.12:0/65375719 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f97d81beeb0 con 0x5568b582f9f0 2024-05-17T22:41:15.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.966+0000 7f97ec34ab80 1 -- 172.21.15.12:0/65375719 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f97d81bcd00 con 0x5568b59b3aa0 2024-05-17T22:41:15.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.966+0000 7f97d67fc700 1 --2- 172.21.15.12:0/65375719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5568b59ae170 0x5568b59b15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:15.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.966+0000 7f97d77fe700 1 --2- 172.21.15.12:0/65375719 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5568b59b3aa0 0x5568b59b98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:15.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.966+0000 7f97d77fe700 1 -- 172.21.15.12:0/65375719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568b582f9f0 msgr2=0x5568b59adc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:41:15.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.967+0000 7f97d77fe700 1 --2- 172.21.15.12:0/65375719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568b582f9f0 0x5568b59adc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:15.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.967+0000 7f97d77fe700 1 -- 172.21.15.12:0/65375719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5568b59ae170 msgr2=0x5568b59b15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:41:15.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.967+0000 7f97d77fe700 1 --2- 172.21.15.12:0/65375719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5568b59ae170 0x5568b59b15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:15.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.967+0000 7f97d77fe700 1 -- 172.21.15.12:0/65375719 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568b59aa710 con 0x5568b59b3aa0 2024-05-17T22:41:15.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.967+0000 7f97d77fe700 1 --2- 172.21.15.12:0/65375719 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5568b59b3aa0 0x5568b59b98b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f97d0015870 tx=0x7f97d00091f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9f9104f155ae5f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:41:15.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.967+0000 7f97bd7fa700 1 -- 172.21.15.12:0/65375719 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d0022040 con 0x5568b59b3aa0 2024-05-17T22:41:15.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.967+0000 7f97bd7fa700 1 -- 172.21.15.12:0/65375719 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97d00092e0 con 0x5568b59b3aa0 2024-05-17T22:41:15.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.967+0000 7f97bd7fa700 1 -- 172.21.15.12:0/65375719 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d002f940 con 0x5568b59b3aa0 2024-05-17T22:41:15.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.969+0000 7f97ec34ab80 1 -- 172.21.15.12:0/65375719 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5568b59b3aa0 msgr2=0x5568b59b98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:41:15.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.969+0000 7f97ec34ab80 1 --2- 172.21.15.12:0/65375719 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5568b59b3aa0 0x5568b59b98b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f97d0015870 tx=0x7f97d00091f0 comp rx=0 tx=0).stop 2024-05-17T22:41:15.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.969+0000 7f97ec34ab80 1 -- 172.21.15.12:0/65375719 shutdown_connections 2024-05-17T22:41:15.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.969+0000 7f97ec34ab80 1 --2- 172.21.15.12:0/65375719 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5568b59b3aa0 0x5568b59b98b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:15.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.969+0000 7f97ec34ab80 1 --2- 172.21.15.12:0/65375719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568b582f9f0 0x5568b59adc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:15.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.969+0000 7f97ec34ab80 1 --2- 172.21.15.12:0/65375719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5568b59ae170 0x5568b59b15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:15.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.969+0000 7f97ec34ab80 1 -- 172.21.15.12:0/65375719 >> 172.21.15.12:0/65375719 conn(0x5568b5873f00 msgr2=0x5568b59c24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:41:15.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.969+0000 7f97ec34ab80 1 -- 172.21.15.12:0/65375719 shutdown_connections 2024-05-17T22:41:15.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.969+0000 7f97ec34ab80 1 -- 172.21.15.12:0/65375719 wait complete. 2024-05-17T22:41:15.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.970+0000 7f97ec34ab80 1 -- start start 2024-05-17T22:41:15.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.970+0000 7f97ec34ab80 1 Processor -- start 2024-05-17T22:41:15.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.971+0000 7f97ec34ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5568b582f9f0 0x5568b59cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:15.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.971+0000 7f97ec34ab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5568b59ae170 0x5568b59ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:15.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.971+0000 7f97ec34ab80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568b59b3aa0 0x5568b59c99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:15.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.972+0000 7f97ec34ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f97d81bd370 con 0x5568b582f9f0 2024-05-17T22:41:15.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.972+0000 7f97ec34ab80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f97d81be7e0 con 0x5568b59b3aa0 2024-05-17T22:41:15.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.972+0000 7f97ec34ab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f97d81bda40 con 0x5568b59ae170 2024-05-17T22:41:15.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.972+0000 7f97ec34ab80 10 client.0 _ll_register_callbacks cb 0x5568b5866f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:41:15.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.972+0000 7f97d6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5568b582f9f0 0x5568b59cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:15.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.972+0000 7f97d6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5568b582f9f0 0x5568b59cc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40442/0 (socket says 172.21.15.12:40442) 2024-05-17T22:41:15.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.972+0000 7f97d6ffd700 1 -- 172.21.15.12:0/3522415309 learned_addr learned my addr 172.21.15.12:0/3522415309 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:41:15.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.972+0000 7f97d67fc700 1 --2- 172.21.15.12:0/3522415309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5568b59ae170 0x5568b59ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:15.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.973+0000 7f97d67fc700 1 -- 172.21.15.12:0/3522415309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568b59b3aa0 msgr2=0x5568b59c99d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T22:41:15.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.973+0000 7f97d67fc700 1 --2- 172.21.15.12:0/3522415309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568b59b3aa0 0x5568b59c99d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:15.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.973+0000 7f97d67fc700 1 -- 172.21.15.12:0/3522415309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5568b582f9f0 msgr2=0x5568b59cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:41:15.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.973+0000 7f97d67fc700 1 --2- 172.21.15.12:0/3522415309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5568b582f9f0 0x5568b59cc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:15.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.973+0000 7f97d67fc700 1 -- 172.21.15.12:0/3522415309 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568b59ca170 con 0x5568b59ae170 2024-05-17T22:41:15.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.973+0000 7f97d6ffd700 1 --2- 172.21.15.12:0/3522415309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5568b582f9f0 0x5568b59cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:41:15.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.973+0000 7f97d67fc700 1 --2- 172.21.15.12:0/3522415309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5568b59ae170 0x5568b59ccaf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x5568b59b3030 tx=0x7f97cc0e2610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32152822cda839ca server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:41:15.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.973+0000 7f97b6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:41:15.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.973+0000 7f97b6ffd700 1 -- 172.21.15.12:0/3522415309 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc0a9420 con 0x5568b59ae170 2024-05-17T22:41:15.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.973+0000 7f97ec34ab80 1 -- 172.21.15.12:0/3522415309 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97d00092e0 con 0x5568b59ae170 2024-05-17T22:41:15.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.973+0000 7f97b6ffd700 1 -- 172.21.15.12:0/3522415309 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f97cc0a9d80 con 0x5568b59ae170 2024-05-17T22:41:15.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.973+0000 7f97b6ffd700 1 -- 172.21.15.12:0/3522415309 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f97cc0aa010 con 0x5568b59ae170 2024-05-17T22:41:15.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.973+0000 7f97ec34ab80 10 client.25165 fetch_fsmap learned FSMap version 29 2024-05-17T22:41:15.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.973+0000 7f97ec34ab80 1 -- 172.21.15.12:0/3522415309 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5568b5a31210 con 0x5568b59ae170 2024-05-17T22:41:15.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.974+0000 7f97b6ffd700 1 -- 172.21.15.12:0/3522415309 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97d00092e0 con 0x5568b59ae170 2024-05-17T22:41:15.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.974+0000 7f97b6ffd700 1 -- 172.21.15.12:0/3522415309 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f97cc0cabc0 con 0x5568b59ae170 2024-05-17T22:41:15.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.974+0000 7f97ec34ab80 10 client.25165 fetch_fsmap finished waiting for FSMap version 29 2024-05-17T22:41:15.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.974+0000 7f97ec34ab80 10 client.25165 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:41:15.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.974+0000 7f97ec34ab80 20 client.25165 populate_metadata read hostname 'smithi012' 2024-05-17T22:41:15.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.974+0000 7f97ec34ab80 1 --2- 172.21.15.12:0/3522415309 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5568b5a31a40 0x5568b5a33e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:15.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.974+0000 7f97ec34ab80 4 client.25165 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:41:15.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.974+0000 7f97ec34ab80 1 -- 172.21.15.12:0/3522415309 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5568b58d79c0 con 0x5568b5a31a40 2024-05-17T22:41:15.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.974+0000 7f97d6ffd700 1 --2- 172.21.15.12:0/3522415309 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5568b5a31a40 0x5568b5a33e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:15.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.974+0000 7f97b6ffd700 10 client.25165 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:41:15.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.975+0000 7f97d6ffd700 1 --2- 172.21.15.12:0/3522415309 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5568b5a31a40 0x5568b5a33e60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14d26cc9327ee5c2 server_cookie=b5e44df367e4eaa7 in_seq=0 out_seq=0 2024-05-17T22:41:15.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.975+0000 7f97b6ffd700 1 -- 172.21.15.12:0/3522415309 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5568b58d79c0 con 0x5568b5a31a40 2024-05-17T22:41:15.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.975+0000 7f97b6ffd700 10 client.25165 handle_command_reply: tid=0 2024-05-17T22:41:15.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.975+0000 7f97ec34ab80 1 client.25165 shutdown 2024-05-17T22:41:15.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.976+0000 7f97ec34ab80 1 -- 172.21.15.12:0/3522415309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5568b59ae170 msgr2=0x5568b59ccaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:41:15.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.976+0000 7f97ec34ab80 1 --2- 172.21.15.12:0/3522415309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5568b59ae170 0x5568b59ccaf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x5568b59b3030 tx=0x7f97cc0e2610 comp rx=0 tx=0).stop 2024-05-17T22:41:15.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.976+0000 7f97ec34ab80 1 -- 172.21.15.12:0/3522415309 shutdown_connections 2024-05-17T22:41:15.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.976+0000 7f97ec34ab80 1 --2- 172.21.15.12:0/3522415309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568b59b3aa0 0x5568b59c99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:15.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.976+0000 7f97ec34ab80 1 --2- 172.21.15.12:0/3522415309 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5568b5a31a40 0x5568b5a33e60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:15.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.976+0000 7f97ec34ab80 1 --2- 172.21.15.12:0/3522415309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5568b582f9f0 0x5568b59cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:15.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.976+0000 7f97ec34ab80 1 --2- 172.21.15.12:0/3522415309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5568b59ae170 0x5568b59ccaf0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:15.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.976+0000 7f97b6ffd700 0 client.25165 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:41:15.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.976+0000 7f97ec34ab80 1 -- 172.21.15.12:0/3522415309 >> 172.21.15.12:0/3522415309 conn(0x5568b5873f00 msgr2=0x5568b59c2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:41:15.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.976+0000 7f97ec34ab80 1 -- 172.21.15.12:0/3522415309 shutdown_connections 2024-05-17T22:41:15.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.977+0000 7f97ec34ab80 1 -- 172.21.15.12:0/3522415309 wait complete. 2024-05-17T22:41:15.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.977+0000 7f97ec34ab80 20 client.25165 trim_cache size 0 max 16384 2024-05-17T22:41:15.982 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:41:15.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.977+0000 7f97dd96f700 1 -- 172.21.15.12:0/4007479642 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f97c00b2980 msgr2=0x7f97c00b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:41:15.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.978+0000 7f97dd96f700 1 --2- 172.21.15.12:0/4007479642 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f97c00b2980 0x7f97c00b4e90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f97c8000c00 tx=0x7f97c8005fd0 comp rx=0 tx=0).stop 2024-05-17T22:41:15.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.978+0000 7f97dd96f700 1 -- 172.21.15.12:0/4007479642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97d810b180 msgr2=0x7f97d81a9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:41:15.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.978+0000 7f97dd96f700 1 --2- 172.21.15.12:0/4007479642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97d810b180 0x7f97d81a9b90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f97cc00e9f0 tx=0x7f97cc00ed00 comp rx=0 tx=0).stop 2024-05-17T22:41:15.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.978+0000 7f97dd96f700 1 -- 172.21.15.12:0/4007479642 shutdown_connections 2024-05-17T22:41:15.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.978+0000 7f97dd96f700 1 --2- 172.21.15.12:0/4007479642 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f97c00b2980 0x7f97c00b4e90 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:15.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.978+0000 7f97dd96f700 1 --2- 172.21.15.12:0/4007479642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97d811d550 0x7f97d81ade20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:15.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.978+0000 7f97dd96f700 1 --2- 172.21.15.12:0/4007479642 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97d810a830 0x7f97d81a95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:15.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.978+0000 7f97dd96f700 1 --2- 172.21.15.12:0/4007479642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97d810b180 0x7f97d81a9b90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:15.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.978+0000 7f97dd96f700 1 -- 172.21.15.12:0/4007479642 >> 172.21.15.12:0/4007479642 conn(0x7f97d806efc0 msgr2=0x7f97d811b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:41:15.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.978+0000 7f97dd96f700 1 -- 172.21.15.12:0/4007479642 shutdown_connections 2024-05-17T22:41:15.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:15.978+0000 7f97dd96f700 1 -- 172.21.15.12:0/4007479642 wait complete. 2024-05-17T22:41:16.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:16 smithi096 ceph-mon[36065]: pgmap v499: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.7 MiB/s wr, 664 op/s 2024-05-17T22:41:16.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:16 smithi012 ceph-mon[31632]: pgmap v499: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.7 MiB/s wr, 664 op/s 2024-05-17T22:41:16.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 115.0 s, 1604.8 tps, lat 62.246 ms stddev 77.109 2024-05-17T22:41:16.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:16 smithi151 ceph-mon[36101]: pgmap v499: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.7 MiB/s wr, 664 op/s 2024-05-17T22:41:18.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:18 smithi012 ceph-mon[31632]: pgmap v500: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.4 MiB/s wr, 571 op/s 2024-05-17T22:41:18.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:18 smithi151 ceph-mon[36101]: pgmap v500: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.4 MiB/s wr, 571 op/s 2024-05-17T22:41:18.541 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-17T22:41:18.541 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-17T22:41:18.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:18 smithi096 ceph-mon[36065]: pgmap v500: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.4 MiB/s wr, 571 op/s 2024-05-17T22:41:18.935 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:41:20.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:20 smithi012 ceph-mon[31632]: pgmap v501: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 10 MiB/s wr, 883 op/s 2024-05-17T22:41:20.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:20 smithi151 ceph-mon[36101]: pgmap v501: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 10 MiB/s wr, 883 op/s 2024-05-17T22:41:20.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:20 smithi096 ceph-mon[36065]: pgmap v501: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 10 MiB/s wr, 883 op/s 2024-05-17T22:41:20.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.875+0000 7fec58dca700 1 -- 172.21.15.12:0/2968113179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec5406f2e0 msgr2=0x7fec5406f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:41:20.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.875+0000 7fec58dca700 1 --2- 172.21.15.12:0/2968113179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec5406f2e0 0x7fec5406f6f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fec4400b0a0 tx=0x7fec4400b3b0 comp rx=0 tx=0).stop 2024-05-17T22:41:20.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.880+0000 7fec58dca700 1 -- 172.21.15.12:0/2968113179 shutdown_connections 2024-05-17T22:41:20.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.880+0000 7fec58dca700 1 --2- 172.21.15.12:0/2968113179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec5411d4a0 0x7fec5411f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:20.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.881+0000 7fec58dca700 1 --2- 172.21.15.12:0/2968113179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec5406f2e0 0x7fec5406f6f0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:20.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.881+0000 7fec58dca700 1 --2- 172.21.15.12:0/2968113179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec5406fc30 0x7fec5411cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:20.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.881+0000 7fec58dca700 1 -- 172.21.15.12:0/2968113179 >> 172.21.15.12:0/2968113179 conn(0x7fec5410a830 msgr2=0x7fec5410cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:41:20.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.881+0000 7fec58dca700 1 -- 172.21.15.12:0/2968113179 shutdown_connections 2024-05-17T22:41:20.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.881+0000 7fec58dca700 1 -- 172.21.15.12:0/2968113179 wait complete. 2024-05-17T22:41:20.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.883+0000 7fec58dca700 1 Processor -- start 2024-05-17T22:41:20.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.883+0000 7fec58dca700 1 -- start start 2024-05-17T22:41:20.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.883+0000 7fec58dca700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec5406f2e0 0x7fec541a96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:20.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.883+0000 7fec58dca700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec5406fc30 0x7fec541a9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:20.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.883+0000 7fec58dca700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec5411d4a0 0x7fec541adf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:20.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.883+0000 7fec58dca700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fec54122d10 con 0x7fec5411d4a0 2024-05-17T22:41:20.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.883+0000 7fec58dca700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fec54122b90 con 0x7fec5406fc30 2024-05-17T22:41:20.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.883+0000 7fec58dca700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fec54122e90 con 0x7fec5406f2e0 2024-05-17T22:41:20.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.883+0000 7fec52d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec5411d4a0 0x7fec541adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:20.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.884+0000 7fec52d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec5411d4a0 0x7fec541adf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59100/0 (socket says 172.21.15.12:59100) 2024-05-17T22:41:20.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.884+0000 7fec52d9d700 1 -- 172.21.15.12:0/3951228568 learned_addr learned my addr 172.21.15.12:0/3951228568 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:41:20.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.884+0000 7fec51d9b700 1 --2- 172.21.15.12:0/3951228568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec5406fc30 0x7fec541a9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:20.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.884+0000 7fec52d9d700 1 -- 172.21.15.12:0/3951228568 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec5406f2e0 msgr2=0x7fec541a96d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-17T22:41:20.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.884+0000 7fec52d9d700 1 --2- 172.21.15.12:0/3951228568 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec5406f2e0 0x7fec541a96d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:20.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.884+0000 7fec52d9d700 1 -- 172.21.15.12:0/3951228568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec5406fc30 msgr2=0x7fec541a9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:41:20.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.884+0000 7fec52d9d700 1 --2- 172.21.15.12:0/3951228568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec5406fc30 0x7fec541a9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:20.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.884+0000 7fec52d9d700 1 -- 172.21.15.12:0/3951228568 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec541ae5a0 con 0x7fec5411d4a0 2024-05-17T22:41:20.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.884+0000 7fec51d9b700 1 --2- 172.21.15.12:0/3951228568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec5406fc30 0x7fec541a9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:41:20.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.885+0000 7fec52d9d700 1 --2- 172.21.15.12:0/3951228568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec5411d4a0 0x7fec541adf00 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fec4c00b4a0 tx=0x7fec4c00b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:41:20.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.885+0000 7fec437fe700 1 -- 172.21.15.12:0/3951228568 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4c015020 con 0x7fec5411d4a0 2024-05-17T22:41:20.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.885+0000 7fec437fe700 1 -- 172.21.15.12:0/3951228568 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec4c011ce0 con 0x7fec5411d4a0 2024-05-17T22:41:20.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.885+0000 7fec437fe700 1 -- 172.21.15.12:0/3951228568 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4c010d40 con 0x7fec5411d4a0 2024-05-17T22:41:20.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.885+0000 7fec58dca700 1 -- 172.21.15.12:0/3951228568 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec541ae890 con 0x7fec5411d4a0 2024-05-17T22:41:20.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.885+0000 7fec58dca700 1 -- 172.21.15.12:0/3951228568 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec541aed20 con 0x7fec5411d4a0 2024-05-17T22:41:20.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.891+0000 7fec437fe700 1 -- 172.21.15.12:0/3951228568 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fec4c018020 con 0x7fec5411d4a0 2024-05-17T22:41:20.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.892+0000 7fec437fe700 1 --2- 172.21.15.12:0/3951228568 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fec3c0b2a10 0x7fec3c0b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:20.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.893+0000 7fec5259c700 1 --2- 172.21.15.12:0/3951228568 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fec3c0b2a10 0x7fec3c0b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:20.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.893+0000 7fec437fe700 1 -- 172.21.15.12:0/3951228568 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fec4c0e1c50 con 0x7fec5411d4a0 2024-05-17T22:41:20.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.893+0000 7fec5259c700 1 --2- 172.21.15.12:0/3951228568 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fec3c0b2a10 0x7fec3c0b4f20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fec4400d010 tx=0x7fec44006000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:41:20.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.895+0000 7fec677a5b80 1 Processor -- start 2024-05-17T22:41:20.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.896+0000 7fec677a5b80 1 -- start start 2024-05-17T22:41:20.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.896+0000 7fec677a5b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b074250a70 0x55b0743d45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:20.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.897+0000 7fec677a5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0743d4b20 0x55b0743cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:20.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.897+0000 7fec677a5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b0743cf1f0 0x55b0743cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:20.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.897+0000 7fec677a5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b0742fd310 con 0x55b0743cf1f0 2024-05-17T22:41:20.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.897+0000 7fec52d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b0743cf1f0 0x55b0743cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:20.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.897+0000 7fec52d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b0743cf1f0 0x55b0743cf600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59110/0 (socket says 172.21.15.12:59110) 2024-05-17T22:41:20.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.897+0000 7fec52d9d700 1 -- 172.21.15.12:0/2214868854 learned_addr learned my addr 172.21.15.12:0/2214868854 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:41:20.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.897+0000 7fec677a5b80 1 -- 172.21.15.12:0/2214868854 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b0742ed050 con 0x55b074250a70 2024-05-17T22:41:20.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.897+0000 7fec677a5b80 1 -- 172.21.15.12:0/2214868854 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b0742f0ef0 con 0x55b0743d4b20 2024-05-17T22:41:20.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.897+0000 7fec51d9b700 1 --2- 172.21.15.12:0/2214868854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0743d4b20 0x55b0743cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:20.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.897+0000 7fec52d9d700 1 -- 172.21.15.12:0/2214868854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0743d4b20 msgr2=0x55b0743cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:41:20.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.897+0000 7fec52d9d700 1 --2- 172.21.15.12:0/2214868854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0743d4b20 0x55b0743cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:20.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.897+0000 7fec52d9d700 1 -- 172.21.15.12:0/2214868854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b074250a70 msgr2=0x55b0743d45e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T22:41:20.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.897+0000 7fec52d9d700 1 --2- 172.21.15.12:0/2214868854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b074250a70 0x55b0743d45e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:20.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.898+0000 7fec52d9d700 1 -- 172.21.15.12:0/2214868854 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b074296610 con 0x55b0743cf1f0 2024-05-17T22:41:20.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.898+0000 7fec52d9d700 1 --2- 172.21.15.12:0/2214868854 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b0743cf1f0 0x55b0743cf600 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fec4c010710 tx=0x7fec4c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93f9e62c2707700c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:41:20.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.898+0000 7fec40ff9700 1 -- 172.21.15.12:0/2214868854 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4c0a9580 con 0x55b0743cf1f0 2024-05-17T22:41:20.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.898+0000 7fec40ff9700 1 -- 172.21.15.12:0/2214868854 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec4c0a8e80 con 0x55b0743cf1f0 2024-05-17T22:41:20.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.898+0000 7fec40ff9700 1 -- 172.21.15.12:0/2214868854 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4c0b1ab0 con 0x55b0743cf1f0 2024-05-17T22:41:20.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.900+0000 7fec677a5b80 1 -- 172.21.15.12:0/2214868854 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b0743cf1f0 msgr2=0x55b0743cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:41:20.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.900+0000 7fec677a5b80 1 --2- 172.21.15.12:0/2214868854 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b0743cf1f0 0x55b0743cf600 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fec4c010710 tx=0x7fec4c000bc0 comp rx=0 tx=0).stop 2024-05-17T22:41:20.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.900+0000 7fec677a5b80 1 -- 172.21.15.12:0/2214868854 shutdown_connections 2024-05-17T22:41:20.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.900+0000 7fec677a5b80 1 --2- 172.21.15.12:0/2214868854 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b0743cf1f0 0x55b0743cf600 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:20.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.900+0000 7fec677a5b80 1 --2- 172.21.15.12:0/2214868854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b074250a70 0x55b0743d45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:20.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.900+0000 7fec677a5b80 1 --2- 172.21.15.12:0/2214868854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0743d4b20 0x55b0743cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:20.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.900+0000 7fec677a5b80 1 -- 172.21.15.12:0/2214868854 >> 172.21.15.12:0/2214868854 conn(0x55b074294f80 msgr2=0x55b0743c9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:41:20.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.900+0000 7fec677a5b80 1 -- 172.21.15.12:0/2214868854 shutdown_connections 2024-05-17T22:41:20.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.900+0000 7fec677a5b80 1 -- 172.21.15.12:0/2214868854 wait complete. 2024-05-17T22:41:20.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.901+0000 7fec677a5b80 1 -- start start 2024-05-17T22:41:20.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.901+0000 7fec677a5b80 1 Processor -- start 2024-05-17T22:41:20.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.901+0000 7fec677a5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b074250a70 0x55b0743d8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:20.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.902+0000 7fec677a5b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0743cf1f0 0x55b0743d8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:20.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.902+0000 7fec677a5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0743d4b20 0x55b0743d8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:20.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.902+0000 7fec677a5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b0741f41d0 con 0x55b074250a70 2024-05-17T22:41:20.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.902+0000 7fec52d9d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0743d4b20 0x55b0743d8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:20.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.902+0000 7fec52d9d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0743d4b20 0x55b0743d8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:53556/0 (socket says 172.21.15.12:53556) 2024-05-17T22:41:20.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.902+0000 7fec52d9d700 1 -- 172.21.15.12:0/1149340485 learned_addr learned my addr 172.21.15.12:0/1149340485 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:41:20.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.902+0000 7fec51d9b700 1 --2- 172.21.15.12:0/1149340485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0743cf1f0 0x55b0743d8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:20.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.902+0000 7fec5259c700 1 --2- 172.21.15.12:0/1149340485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b074250a70 0x55b0743d8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:20.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.902+0000 7fec677a5b80 1 -- 172.21.15.12:0/1149340485 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b07428df00 con 0x55b0743cf1f0 2024-05-17T22:41:20.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.902+0000 7fec677a5b80 1 -- 172.21.15.12:0/1149340485 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b0743db410 con 0x55b0743d4b20 2024-05-17T22:41:20.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.902+0000 7fec677a5b80 10 client.0 _ll_register_callbacks cb 0x55b074287f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:41:20.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.903+0000 7fec52d9d700 1 -- 172.21.15.12:0/1149340485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0743cf1f0 msgr2=0x55b0743d8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:41:20.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.903+0000 7fec52d9d700 1 --2- 172.21.15.12:0/1149340485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0743cf1f0 0x55b0743d8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:20.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.903+0000 7fec52d9d700 1 -- 172.21.15.12:0/1149340485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b074250a70 msgr2=0x55b0743d8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:41:20.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.903+0000 7fec52d9d700 1 --2- 172.21.15.12:0/1149340485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b074250a70 0x55b0743d8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:20.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.903+0000 7fec52d9d700 1 -- 172.21.15.12:0/1149340485 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b07423d500 con 0x55b0743d4b20 2024-05-17T22:41:20.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.903+0000 7fec51d9b700 1 --2- 172.21.15.12:0/1149340485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0743cf1f0 0x55b0743d8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:41:20.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.903+0000 7fec52d9d700 1 --2- 172.21.15.12:0/1149340485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0743d4b20 0x55b0743d8d90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fec4c0df360 tx=0x7fec4c0b1e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20c3df1933c8ed22 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:41:20.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.903+0000 7fec39ffb700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:41:20.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.903+0000 7fec39ffb700 1 -- 172.21.15.12:0/1149340485 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4c0b6060 con 0x55b0743d4b20 2024-05-17T22:41:20.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.903+0000 7fec39ffb700 1 -- 172.21.15.12:0/1149340485 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fec4c0b1850 con 0x55b0743d4b20 2024-05-17T22:41:20.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.904+0000 7fec677a5b80 1 -- 172.21.15.12:0/1149340485 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec4c0a8e80 con 0x55b0743d4b20 2024-05-17T22:41:20.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.904+0000 7fec39ffb700 1 -- 172.21.15.12:0/1149340485 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fec4c0bd050 con 0x55b0743d4b20 2024-05-17T22:41:20.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.904+0000 7fec677a5b80 10 client.25174 fetch_fsmap learned FSMap version 30 2024-05-17T22:41:20.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.904+0000 7fec677a5b80 1 -- 172.21.15.12:0/1149340485 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b0743e5cc0 con 0x55b0743d4b20 2024-05-17T22:41:20.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.904+0000 7fec39ffb700 1 -- 172.21.15.12:0/1149340485 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec4c0b1a70 con 0x55b0743d4b20 2024-05-17T22:41:20.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.904+0000 7fec39ffb700 1 -- 172.21.15.12:0/1149340485 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fec4c0a7b50 con 0x55b0743d4b20 2024-05-17T22:41:20.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.904+0000 7fec677a5b80 10 client.25174 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T22:41:20.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.904+0000 7fec677a5b80 10 client.25174 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:41:20.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.904+0000 7fec677a5b80 20 client.25174 populate_metadata read hostname 'smithi012' 2024-05-17T22:41:20.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.904+0000 7fec677a5b80 1 --2- 172.21.15.12:0/1149340485 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55b0743e8190 0x55b0743ea5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:20.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.904+0000 7fec677a5b80 4 client.25174 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:41:20.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.904+0000 7fec677a5b80 1 -- 172.21.15.12:0/1149340485 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fec4c0a8e80 con 0x55b0743e8190 2024-05-17T22:41:20.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.905+0000 7fec5259c700 1 --2- 172.21.15.12:0/1149340485 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55b0743e8190 0x55b0743ea5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:20.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.905+0000 7fec39ffb700 10 client.25174 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:41:20.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.905+0000 7fec5259c700 1 --2- 172.21.15.12:0/1149340485 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55b0743e8190 0x55b0743ea5b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b82175b259007d3 server_cookie=e8451d4581f8c6fa in_seq=0 out_seq=0 2024-05-17T22:41:20.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.906+0000 7fec39ffb700 1 -- 172.21.15.12:0/1149340485 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fec4c0a8e80 con 0x55b0743e8190 2024-05-17T22:41:20.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.906+0000 7fec39ffb700 10 client.25174 handle_command_reply: tid=0 2024-05-17T22:41:20.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.906+0000 7fec677a5b80 1 client.25174 shutdown 2024-05-17T22:41:20.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.906+0000 7fec677a5b80 1 -- 172.21.15.12:0/1149340485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0743d4b20 msgr2=0x55b0743d8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:41:20.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.906+0000 7fec677a5b80 1 --2- 172.21.15.12:0/1149340485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0743d4b20 0x55b0743d8d90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fec4c0df360 tx=0x7fec4c0b1e10 comp rx=0 tx=0).stop 2024-05-17T22:41:20.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.907+0000 7fec677a5b80 1 -- 172.21.15.12:0/1149340485 shutdown_connections 2024-05-17T22:41:20.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.907+0000 7fec677a5b80 1 --2- 172.21.15.12:0/1149340485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0743d4b20 0x55b0743d8d90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:20.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.907+0000 7fec677a5b80 1 --2- 172.21.15.12:0/1149340485 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55b0743e8190 0x55b0743ea5b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:20.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.907+0000 7fec677a5b80 1 --2- 172.21.15.12:0/1149340485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b074250a70 0x55b0743d8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:20.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.907+0000 7fec677a5b80 1 --2- 172.21.15.12:0/1149340485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0743cf1f0 0x55b0743d8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:20.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.907+0000 7fec677a5b80 1 -- 172.21.15.12:0/1149340485 >> 172.21.15.12:0/1149340485 conn(0x55b074294f80 msgr2=0x55b0743c3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:41:20.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.907+0000 7fec39ffb700 0 client.25174 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:41:20.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.907+0000 7fec677a5b80 1 -- 172.21.15.12:0/1149340485 shutdown_connections 2024-05-17T22:41:20.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.907+0000 7fec677a5b80 1 -- 172.21.15.12:0/1149340485 wait complete. 2024-05-17T22:41:20.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.907+0000 7fec677a5b80 20 client.25174 trim_cache size 0 max 16384 2024-05-17T22:41:20.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.940+0000 7fec677a5b80 1 Processor -- start 2024-05-17T22:41:20.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.940+0000 7fec677a5b80 1 -- start start 2024-05-17T22:41:20.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.941+0000 7fec677a5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b074250a70 0x55b0743c3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:20.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.941+0000 7fec677a5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0743cf1f0 0x55b0743e45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:20.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.941+0000 7fec5259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b074250a70 0x55b0743c3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:20.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.941+0000 7fec677a5b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0743d4b20 0x55b0743d8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:20.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.941+0000 7fec677a5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fec541c14c0 con 0x55b074250a70 2024-05-17T22:41:20.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.941+0000 7fec677a5b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fec541bed20 con 0x55b0743d4b20 2024-05-17T22:41:20.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.941+0000 7fec677a5b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fec541bcb70 con 0x55b0743cf1f0 2024-05-17T22:41:20.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.941+0000 7fec5259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b074250a70 0x55b0743c3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59124/0 (socket says 172.21.15.12:59124) 2024-05-17T22:41:20.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.941+0000 7fec5259c700 1 -- 172.21.15.12:0/2391091258 learned_addr learned my addr 172.21.15.12:0/2391091258 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:41:20.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.941+0000 7fec51d9b700 1 --2- 172.21.15.12:0/2391091258 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0743cf1f0 0x55b0743e45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:20.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.941+0000 7fec52d9d700 1 --2- 172.21.15.12:0/2391091258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0743d4b20 0x55b0743d8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:20.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.942+0000 7fec52d9d700 1 -- 172.21.15.12:0/2391091258 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0743cf1f0 msgr2=0x55b0743e45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:41:20.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.942+0000 7fec52d9d700 1 --2- 172.21.15.12:0/2391091258 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0743cf1f0 0x55b0743e45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:20.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.942+0000 7fec52d9d700 1 -- 172.21.15.12:0/2391091258 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b074250a70 msgr2=0x55b0743c3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:41:20.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.942+0000 7fec52d9d700 1 --2- 172.21.15.12:0/2391091258 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b074250a70 0x55b0743c3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:20.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.942+0000 7fec52d9d700 1 -- 172.21.15.12:0/2391091258 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0743cb190 con 0x55b0743d4b20 2024-05-17T22:41:20.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.942+0000 7fec52d9d700 1 --2- 172.21.15.12:0/2391091258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0743d4b20 0x55b0743d8be0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fec4c0a7fa0 tx=0x7fec4c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ecb0a490cf6600d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:41:20.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.942+0000 7fec3bfff700 1 -- 172.21.15.12:0/2391091258 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4c087020 con 0x55b0743d4b20 2024-05-17T22:41:20.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.942+0000 7fec3bfff700 1 -- 172.21.15.12:0/2391091258 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec4c0a47f0 con 0x55b0743d4b20 2024-05-17T22:41:20.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.942+0000 7fec3bfff700 1 -- 172.21.15.12:0/2391091258 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4c0b6040 con 0x55b0743d4b20 2024-05-17T22:41:20.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.943+0000 7fec677a5b80 1 -- 172.21.15.12:0/2391091258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0743d4b20 msgr2=0x55b0743d8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:41:20.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.943+0000 7fec677a5b80 1 --2- 172.21.15.12:0/2391091258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0743d4b20 0x55b0743d8be0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fec4c0a7fa0 tx=0x7fec4c000bc0 comp rx=0 tx=0).stop 2024-05-17T22:41:20.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.944+0000 7fec677a5b80 1 -- 172.21.15.12:0/2391091258 shutdown_connections 2024-05-17T22:41:20.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.944+0000 7fec677a5b80 1 --2- 172.21.15.12:0/2391091258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0743d4b20 0x55b0743d8be0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:20.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.944+0000 7fec677a5b80 1 --2- 172.21.15.12:0/2391091258 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b074250a70 0x55b0743c3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:20.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.944+0000 7fec677a5b80 1 --2- 172.21.15.12:0/2391091258 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0743cf1f0 0x55b0743e45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:20.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.944+0000 7fec677a5b80 1 -- 172.21.15.12:0/2391091258 >> 172.21.15.12:0/2391091258 conn(0x55b074294f80 msgr2=0x55b0743d52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:41:20.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.945+0000 7fec677a5b80 1 -- 172.21.15.12:0/2391091258 shutdown_connections 2024-05-17T22:41:20.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.945+0000 7fec677a5b80 1 -- 172.21.15.12:0/2391091258 wait complete. 2024-05-17T22:41:20.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.946+0000 7fec677a5b80 1 -- start start 2024-05-17T22:41:20.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.946+0000 7fec677a5b80 1 Processor -- start 2024-05-17T22:41:20.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.946+0000 7fec677a5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b074250a70 0x55b0743eb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:20.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.947+0000 7fec677a5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b0743cf1f0 0x55b0743eb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:20.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.947+0000 7fec677a5b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0743d4b20 0x55b0744516e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:20.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.947+0000 7fec677a5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fec541bd1e0 con 0x55b0743cf1f0 2024-05-17T22:41:20.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.947+0000 7fec5259c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b074250a70 0x55b0743eb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:20.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.947+0000 7fec5259c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b074250a70 0x55b0743eb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:53582/0 (socket says 172.21.15.12:53582) 2024-05-17T22:41:20.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.947+0000 7fec5259c700 1 -- 172.21.15.12:0/2382805055 learned_addr learned my addr 172.21.15.12:0/2382805055 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:41:20.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.947+0000 7fec52d9d700 1 --2- 172.21.15.12:0/2382805055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0743d4b20 0x55b0744516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:20.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.947+0000 7fec51d9b700 1 --2- 172.21.15.12:0/2382805055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b0743cf1f0 0x55b0743eb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:20.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.947+0000 7fec677a5b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fec541be650 con 0x55b0743d4b20 2024-05-17T22:41:20.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.947+0000 7fec677a5b80 1 -- 172.21.15.12:0/2382805055 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fec541bd8b0 con 0x55b074250a70 2024-05-17T22:41:20.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.947+0000 7fec677a5b80 10 client.0 _ll_register_callbacks cb 0x55b074287f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:41:20.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.948+0000 7fec5259c700 1 -- 172.21.15.12:0/2382805055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0743d4b20 msgr2=0x55b0744516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:41:20.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.948+0000 7fec5259c700 1 --2- 172.21.15.12:0/2382805055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0743d4b20 0x55b0744516e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:20.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.948+0000 7fec5259c700 1 -- 172.21.15.12:0/2382805055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b0743cf1f0 msgr2=0x55b0743eb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:41:20.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.948+0000 7fec5259c700 1 --2- 172.21.15.12:0/2382805055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b0743cf1f0 0x55b0743eb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:20.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.948+0000 7fec5259c700 1 -- 172.21.15.12:0/2382805055 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b074451c20 con 0x55b074250a70 2024-05-17T22:41:20.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.948+0000 7fec5259c700 1 --2- 172.21.15.12:0/2382805055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b074250a70 0x55b0743eb180 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fec44000f80 tx=0x7fec4400b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a34316fe062b5414 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:41:20.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.948+0000 7fec39ffb700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:41:20.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.949+0000 7fec39ffb700 1 -- 172.21.15.12:0/2382805055 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec44003e10 con 0x55b074250a70 2024-05-17T22:41:20.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.949+0000 7fec39ffb700 1 -- 172.21.15.12:0/2382805055 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fec440077d0 con 0x55b074250a70 2024-05-17T22:41:20.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.949+0000 7fec677a5b80 1 -- 172.21.15.12:0/2382805055 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec4c0a47f0 con 0x55b074250a70 2024-05-17T22:41:20.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.949+0000 7fec39ffb700 1 -- 172.21.15.12:0/2382805055 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fec440079d0 con 0x55b074250a70 2024-05-17T22:41:20.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.949+0000 7fec677a5b80 10 client.25180 fetch_fsmap learned FSMap version 30 2024-05-17T22:41:20.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.949+0000 7fec677a5b80 1 -- 172.21.15.12:0/2382805055 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b074452da0 con 0x55b074250a70 2024-05-17T22:41:20.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.949+0000 7fec39ffb700 1 -- 172.21.15.12:0/2382805055 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec4c0a47f0 con 0x55b074250a70 2024-05-17T22:41:20.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.949+0000 7fec39ffb700 1 -- 172.21.15.12:0/2382805055 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fec44029040 con 0x55b074250a70 2024-05-17T22:41:20.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.949+0000 7fec677a5b80 10 client.25180 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T22:41:20.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.949+0000 7fec677a5b80 10 client.25180 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:41:20.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.949+0000 7fec677a5b80 20 client.25180 populate_metadata read hostname 'smithi012' 2024-05-17T22:41:20.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.949+0000 7fec677a5b80 1 --2- 172.21.15.12:0/2382805055 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55b074453560 0x55b074455980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:20.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.949+0000 7fec677a5b80 4 client.25180 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-17T22:41:20.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.949+0000 7fec677a5b80 1 -- 172.21.15.12:0/2382805055 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b0742ff260 con 0x55b074453560 2024-05-17T22:41:20.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.950+0000 7fec51d9b700 1 --2- 172.21.15.12:0/2382805055 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55b074453560 0x55b074455980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:20.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.950+0000 7fec39ffb700 10 client.25180 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:41:20.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.950+0000 7fec51d9b700 1 --2- 172.21.15.12:0/2382805055 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55b074453560 0x55b074455980 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd88cbd069db476 server_cookie=92625afea500eac3 in_seq=0 out_seq=0 2024-05-17T22:41:20.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.951+0000 7fec39ffb700 1 -- 172.21.15.12:0/2382805055 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b0742ff260 con 0x55b074453560 2024-05-17T22:41:20.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.952+0000 7fec39ffb700 10 client.25180 handle_command_reply: tid=0 2024-05-17T22:41:20.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.952+0000 7fec677a5b80 1 client.25180 shutdown 2024-05-17T22:41:20.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.952+0000 7fec677a5b80 1 -- 172.21.15.12:0/2382805055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b074250a70 msgr2=0x55b0743eb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:41:20.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.952+0000 7fec677a5b80 1 --2- 172.21.15.12:0/2382805055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b074250a70 0x55b0743eb180 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fec44000f80 tx=0x7fec4400b3b0 comp rx=0 tx=0).stop 2024-05-17T22:41:20.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.952+0000 7fec677a5b80 1 -- 172.21.15.12:0/2382805055 shutdown_connections 2024-05-17T22:41:20.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.952+0000 7fec677a5b80 1 --2- 172.21.15.12:0/2382805055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0743d4b20 0x55b0744516e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:20.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.952+0000 7fec677a5b80 1 --2- 172.21.15.12:0/2382805055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b074250a70 0x55b0743eb180 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fec44000f80 tx=0x7fec4400b3b0 comp rx=0 tx=0).stop 2024-05-17T22:41:20.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.952+0000 7fec677a5b80 1 --2- 172.21.15.12:0/2382805055 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55b074453560 0x55b074455980 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:20.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.952+0000 7fec39ffb700 0 client.25180 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:41:20.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.952+0000 7fec677a5b80 1 --2- 172.21.15.12:0/2382805055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b0743cf1f0 0x55b0743eb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:20.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.952+0000 7fec677a5b80 1 -- 172.21.15.12:0/2382805055 >> 172.21.15.12:0/2382805055 conn(0x55b074294f80 msgr2=0x55b0743da2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:41:20.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.953+0000 7fec677a5b80 1 -- 172.21.15.12:0/2382805055 shutdown_connections 2024-05-17T22:41:20.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.953+0000 7fec677a5b80 1 -- 172.21.15.12:0/2382805055 wait complete. 2024-05-17T22:41:20.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.953+0000 7fec677a5b80 20 client.25180 trim_cache size 0 max 16384 2024-05-17T22:41:20.958 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:41:20.958 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-17T22:41:20.958 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "8176a2f5-e0bb-454d-9a04-f98da67e5337", 2024-05-17T22:41:20.958 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-17T22:41:20.958 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:41:20.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.954+0000 7fec58dca700 1 -- 172.21.15.12:0/3951228568 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fec3c0b2a10 msgr2=0x7fec3c0b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:41:20.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.954+0000 7fec58dca700 1 --2- 172.21.15.12:0/3951228568 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fec3c0b2a10 0x7fec3c0b4f20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fec4400d010 tx=0x7fec44006000 comp rx=0 tx=0).stop 2024-05-17T22:41:20.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.954+0000 7fec58dca700 1 -- 172.21.15.12:0/3951228568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec5411d4a0 msgr2=0x7fec541adf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:41:20.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.954+0000 7fec58dca700 1 --2- 172.21.15.12:0/3951228568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec5411d4a0 0x7fec541adf00 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fec4c00b4a0 tx=0x7fec4c00b7b0 comp rx=0 tx=0).stop 2024-05-17T22:41:20.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.954+0000 7fec58dca700 1 -- 172.21.15.12:0/3951228568 shutdown_connections 2024-05-17T22:41:20.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.954+0000 7fec58dca700 1 --2- 172.21.15.12:0/3951228568 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fec3c0b2a10 0x7fec3c0b4f20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:20.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.954+0000 7fec58dca700 1 --2- 172.21.15.12:0/3951228568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec5411d4a0 0x7fec541adf00 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:20.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.954+0000 7fec58dca700 1 --2- 172.21.15.12:0/3951228568 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec5406f2e0 0x7fec541a96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:20.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.954+0000 7fec58dca700 1 --2- 172.21.15.12:0/3951228568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec5406fc30 0x7fec541a9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:20.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.954+0000 7fec58dca700 1 -- 172.21.15.12:0/3951228568 >> 172.21.15.12:0/3951228568 conn(0x7fec5410a830 msgr2=0x7fec54072570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:41:20.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.954+0000 7fec58dca700 1 -- 172.21.15.12:0/3951228568 shutdown_connections 2024-05-17T22:41:20.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:20.955+0000 7fec58dca700 1 -- 172.21.15.12:0/3951228568 wait complete. 2024-05-17T22:41:21.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:21 smithi012 ceph-mon[31632]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:41:21.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:21 smithi012 ceph-mon[31632]: pgmap v502: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.8 MiB/s wr, 708 op/s 2024-05-17T22:41:21.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:21 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:41:21.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:21 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:41:21.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:21 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:41:21.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:21 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:41:21.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:21 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:41:21.343 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:21 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:41:21.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 120.0 s, 1533.5 tps, lat 64.695 ms stddev 95.873 2024-05-17T22:41:21.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:21 smithi151 ceph-mon[36101]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:41:21.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:21 smithi151 ceph-mon[36101]: pgmap v502: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.8 MiB/s wr, 708 op/s 2024-05-17T22:41:21.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:21 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:41:21.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:21 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:41:21.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:21 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:41:21.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:21 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:41:21.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:21 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:41:21.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:21 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:41:21.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:21 smithi096 ceph-mon[36065]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:41:21.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:21 smithi096 ceph-mon[36065]: pgmap v502: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.8 MiB/s wr, 708 op/s 2024-05-17T22:41:21.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:21 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:41:21.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:21 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:41:21.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:21 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:41:21.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:21 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:41:21.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:21 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:41:21.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:21 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:41:21.792 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-17T22:41:22.080 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:22 smithi012 ceph-mon[31632]: scrub queued for path: / 2024-05-17T22:41:22.081 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:22 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:41:22.081 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:22 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:41:22.081 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:22 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:41:22.081 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:22 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:41:22.081 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:22 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:41:22.203 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:41:22.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:22 smithi151 ceph-mon[36101]: scrub queued for path: / 2024-05-17T22:41:22.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:22 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:41:22.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:22 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:41:22.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:22 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:41:22.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:22 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:41:22.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:22 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:41:22.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:22 smithi096 ceph-mon[36065]: scrub queued for path: / 2024-05-17T22:41:22.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:22 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:41:22.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:22 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:41:22.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:22 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:41:22.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:22 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:41:22.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:22 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:41:23.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:23 smithi012 ceph-mon[31632]: pgmap v503: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 11 MiB/s wr, 786 op/s 2024-05-17T22:41:23.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:23 smithi151 ceph-mon[36101]: pgmap v503: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 11 MiB/s wr, 786 op/s 2024-05-17T22:41:23.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:23 smithi096 ceph-mon[36065]: pgmap v503: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 11 MiB/s wr, 786 op/s 2024-05-17T22:41:25.748 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:25 smithi012 ceph-mon[31632]: pgmap v504: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 13 MiB/s wr, 899 op/s 2024-05-17T22:41:25.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.940+0000 7f981bfff700 1 -- 172.21.15.12:0/829782976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f981c1032d0 msgr2=0x7f981c10f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:41:25.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.940+0000 7f981bfff700 1 --2- 172.21.15.12:0/829782976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f981c1032d0 0x7f981c10f850 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f980c01a9a0 tx=0x7f980c01acb0 comp rx=0 tx=0).stop 2024-05-17T22:41:25.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.941+0000 7f981bfff700 1 -- 172.21.15.12:0/829782976 shutdown_connections 2024-05-17T22:41:25.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.941+0000 7f981bfff700 1 --2- 172.21.15.12:0/829782976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f981c1032d0 0x7f981c10f850 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:25.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.941+0000 7f981bfff700 1 --2- 172.21.15.12:0/829782976 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f981c1089b0 0x7f981c108dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:25.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.941+0000 7f981bfff700 1 --2- 172.21.15.12:0/829782976 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f981c102980 0x7f981c102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:25.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.941+0000 7f981bfff700 1 -- 172.21.15.12:0/829782976 >> 172.21.15.12:0/829782976 conn(0x7f981c0fe6b0 msgr2=0x7f981c100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:41:25.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.942+0000 7f981bfff700 1 -- 172.21.15.12:0/829782976 shutdown_connections 2024-05-17T22:41:25.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.947+0000 7f981bfff700 1 -- 172.21.15.12:0/829782976 wait complete. 2024-05-17T22:41:25.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.948+0000 7f981bfff700 1 Processor -- start 2024-05-17T22:41:25.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.948+0000 7f981bfff700 1 -- start start 2024-05-17T22:41:25.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.949+0000 7f981bfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f981c102980 0x7f981c075230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:25.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.949+0000 7f981bfff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f981c1032d0 0x7f981c070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:25.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.949+0000 7f981bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f981c1089b0 0x7f981c0708a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:25.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.949+0000 7f981bfff700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f981c06e8e0 con 0x7f981c1089b0 2024-05-17T22:41:25.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.949+0000 7f981bfff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f981c06e760 con 0x7f981c102980 2024-05-17T22:41:25.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.949+0000 7f981bfff700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f981c06ea60 con 0x7f981c1032d0 2024-05-17T22:41:25.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.950+0000 7f981b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f981c1089b0 0x7f981c0708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:25.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.950+0000 7f981b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f981c1089b0 0x7f981c0708a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59148/0 (socket says 172.21.15.12:59148) 2024-05-17T22:41:25.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.950+0000 7f981b7fe700 1 -- 172.21.15.12:0/1284954781 learned_addr learned my addr 172.21.15.12:0/1284954781 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:41:25.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.950+0000 7f981a7fc700 1 --2- 172.21.15.12:0/1284954781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f981c1032d0 0x7f981c070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:25.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.952+0000 7f981b7fe700 1 -- 172.21.15.12:0/1284954781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f981c1032d0 msgr2=0x7f981c070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:41:25.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.952+0000 7f981b7fe700 1 --2- 172.21.15.12:0/1284954781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f981c1032d0 0x7f981c070360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:25.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.952+0000 7f981b7fe700 1 -- 172.21.15.12:0/1284954781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f981c102980 msgr2=0x7f981c075230 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-17T22:41:25.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.952+0000 7f981b7fe700 1 --2- 172.21.15.12:0/1284954781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f981c102980 0x7f981c075230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:25.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.952+0000 7f981b7fe700 1 -- 172.21.15.12:0/1284954781 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f981c070fa0 con 0x7f981c1089b0 2024-05-17T22:41:25.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.952+0000 7f981b7fe700 1 --2- 172.21.15.12:0/1284954781 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f981c1089b0 0x7f981c0708a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f980c003b50 tx=0x7f980c003b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:41:25.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.955+0000 7f9803fff700 1 -- 172.21.15.12:0/1284954781 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980c057070 con 0x7f981c1089b0 2024-05-17T22:41:25.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.955+0000 7f981bfff700 1 -- 172.21.15.12:0/1284954781 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f981c1174e0 con 0x7f981c1089b0 2024-05-17T22:41:25.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.955+0000 7f981bfff700 1 -- 172.21.15.12:0/1284954781 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f981c1179c0 con 0x7f981c1089b0 2024-05-17T22:41:25.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.955+0000 7f9803fff700 1 -- 172.21.15.12:0/1284954781 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f980c0040a0 con 0x7f981c1089b0 2024-05-17T22:41:25.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.955+0000 7f9803fff700 1 -- 172.21.15.12:0/1284954781 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980c04a660 con 0x7f981c1089b0 2024-05-17T22:41:25.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.962+0000 7f982fc79b80 1 Processor -- start 2024-05-17T22:41:25.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.962+0000 7f982fc79b80 1 -- start start 2024-05-17T22:41:25.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.962+0000 7f982fc79b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587dff5c9f0 0x5587e00e0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:25.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.962+0000 7f982fc79b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587e00e0cb0 0x5587e00dac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:25.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.962+0000 7f982fc79b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587e00db170 0x5587e00db580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:25.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.962+0000 7f982fc79b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5587e0009290 con 0x5587e00e0cb0 2024-05-17T22:41:25.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.962+0000 7f982fc79b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5587dfff8fd0 con 0x5587dff5c9f0 2024-05-17T22:41:25.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.962+0000 7f982fc79b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5587dfffce70 con 0x5587e00db170 2024-05-17T22:41:25.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.963+0000 7f9803fff700 1 -- 172.21.15.12:0/1284954781 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f980c05a070 con 0x7f981c1089b0 2024-05-17T22:41:25.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.964+0000 7f9803fff700 1 --2- 172.21.15.12:0/1284954781 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f98040b2a90 0x7f98040b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:25.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.964+0000 7f981a7fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587dff5c9f0 0x5587e00e0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:25.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.964+0000 7f981a7fc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587e00db170 0x5587e00db580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:25.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.964+0000 7f981a7fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587dff5c9f0 0x5587e00e0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:49134/0 (socket says 172.21.15.12:49134) 2024-05-17T22:41:25.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.964+0000 7f981a7fc700 1 -- 172.21.15.12:0/317654759 learned_addr learned my addr 172.21.15.12:0/317654759 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:41:25.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.964+0000 7f9803fff700 1 -- 172.21.15.12:0/1284954781 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f980c115c80 con 0x7f981c1089b0 2024-05-17T22:41:25.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.965+0000 7f981b7fe700 1 --2- 172.21.15.12:0/1284954781 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f98040b2a90 0x7f98040b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:25.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.966+0000 7f981b7fe700 1 --2- 172.21.15.12:0/1284954781 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f98040b2a90 0x7f98040b4fa0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f980c052c80 tx=0x7f980c01acb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:41:25.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.966+0000 7f981a7fc700 1 -- 172.21.15.12:0/317654759 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587e00db170 msgr2=0x5587e00db580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:41:25.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.966+0000 7f981a7fc700 1 --2- 172.21.15.12:0/317654759 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587e00db170 0x5587e00db580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:25.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.966+0000 7f981a7fc700 1 -- 172.21.15.12:0/317654759 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587e00e0cb0 msgr2=0x5587e00dac30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T22:41:25.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.966+0000 7f981a7fc700 1 --2- 172.21.15.12:0/317654759 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587e00e0cb0 0x5587e00dac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:25.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.966+0000 7f981a7fc700 1 -- 172.21.15.12:0/317654759 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587dffa2590 con 0x5587dff5c9f0 2024-05-17T22:41:25.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.967+0000 7f981a7fc700 1 --2- 172.21.15.12:0/317654759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587dff5c9f0 0x5587e00e0770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f981400ad20 tx=0x7f98140096d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88f2c176cf2de7c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:41:25.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.969+0000 7f98017fa700 1 -- 172.21.15.12:0/317654759 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9814014040 con 0x5587dff5c9f0 2024-05-17T22:41:25.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.969+0000 7f98017fa700 1 -- 172.21.15.12:0/317654759 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f981400ae10 con 0x5587dff5c9f0 2024-05-17T22:41:25.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.969+0000 7f98017fa700 1 -- 172.21.15.12:0/317654759 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981401aac0 con 0x5587dff5c9f0 2024-05-17T22:41:25.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.970+0000 7f982fc79b80 1 -- 172.21.15.12:0/317654759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587dff5c9f0 msgr2=0x5587e00e0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:41:25.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.970+0000 7f982fc79b80 1 --2- 172.21.15.12:0/317654759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587dff5c9f0 0x5587e00e0770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f981400ad20 tx=0x7f98140096d0 comp rx=0 tx=0).stop 2024-05-17T22:41:25.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.971+0000 7f982fc79b80 1 -- 172.21.15.12:0/317654759 shutdown_connections 2024-05-17T22:41:25.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.971+0000 7f982fc79b80 1 --2- 172.21.15.12:0/317654759 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587e00db170 0x5587e00db580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:25.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.971+0000 7f982fc79b80 1 --2- 172.21.15.12:0/317654759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587dff5c9f0 0x5587e00e0770 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f981400ad20 tx=0x7f98140096d0 comp rx=0 tx=0).stop 2024-05-17T22:41:25.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.971+0000 7f982fc79b80 1 --2- 172.21.15.12:0/317654759 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587e00e0cb0 0x5587e00dac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:25.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.971+0000 7f982fc79b80 1 -- 172.21.15.12:0/317654759 >> 172.21.15.12:0/317654759 conn(0x5587dffa0f00 msgr2=0x5587e00d5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:41:25.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.977+0000 7f982fc79b80 1 -- 172.21.15.12:0/317654759 shutdown_connections 2024-05-17T22:41:25.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.982+0000 7f982fc79b80 1 -- 172.21.15.12:0/317654759 wait complete. 2024-05-17T22:41:25.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.982+0000 7f982fc79b80 1 -- start start 2024-05-17T22:41:25.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.982+0000 7f982fc79b80 1 Processor -- start 2024-05-17T22:41:25.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.983+0000 7f982fc79b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587e00db170 0x5587e00e43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:25.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.983+0000 7f982fc79b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587e00e0cb0 0x5587e00e4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:25.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.983+0000 7f982fc79b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587e00e4e60 0x5587e00f08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:25.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.983+0000 7f982fc79b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5587dff00150 con 0x5587e00e4e60 2024-05-17T22:41:25.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.983+0000 7f982fc79b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5587dff99e80 con 0x5587e00e0cb0 2024-05-17T22:41:25.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.983+0000 7f982fc79b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5587e00e75a0 con 0x5587e00db170 2024-05-17T22:41:25.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.983+0000 7f982fc79b80 10 client.0 _ll_register_callbacks cb 0x5587dff93f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:41:25.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.985+0000 7f981a7fc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587e00db170 0x5587e00e43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:25.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.985+0000 7f981a7fc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587e00db170 0x5587e00e43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:53616/0 (socket says 172.21.15.12:53616) 2024-05-17T22:41:25.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.985+0000 7f981a7fc700 1 -- 172.21.15.12:0/2168464540 learned_addr learned my addr 172.21.15.12:0/2168464540 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:41:25.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.985+0000 7f981a7fc700 1 --2- 172.21.15.12:0/2168464540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587e00e4e60 0x5587e00f08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:25.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.986+0000 7f981a7fc700 1 -- 172.21.15.12:0/2168464540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587e00e0cb0 msgr2=0x5587e00e4920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T22:41:25.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.986+0000 7f981a7fc700 1 --2- 172.21.15.12:0/2168464540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587e00e0cb0 0x5587e00e4920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:25.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.986+0000 7f981a7fc700 1 -- 172.21.15.12:0/2168464540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587e00e4e60 msgr2=0x5587e00f08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:41:25.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.986+0000 7f981a7fc700 1 --2- 172.21.15.12:0/2168464540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587e00e4e60 0x5587e00f08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:25.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.986+0000 7f981a7fc700 1 -- 172.21.15.12:0/2168464540 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587dff49480 con 0x5587e00db170 2024-05-17T22:41:25.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.986+0000 7f981a7fc700 1 --2- 172.21.15.12:0/2168464540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587e00db170 0x5587e00e43e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x5587e00d6ae0 tx=0x7f981401a6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68e35255a4d9410c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:41:25.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.987+0000 7f97feff5700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:41:25.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.987+0000 7f97feff5700 1 -- 172.21.15.12:0/2168464540 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9814020040 con 0x5587e00db170 2024-05-17T22:41:25.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.987+0000 7f982fc79b80 1 -- 172.21.15.12:0/2168464540 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f981400ae10 con 0x5587e00db170 2024-05-17T22:41:25.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:25 smithi151 ceph-mon[36101]: pgmap v504: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 13 MiB/s wr, 899 op/s 2024-05-17T22:41:25.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.987+0000 7f97feff5700 1 -- 172.21.15.12:0/2168464540 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f981401a7f0 con 0x5587e00db170 2024-05-17T22:41:25.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.988+0000 7f97feff5700 1 -- 172.21.15.12:0/2168464540 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f981401aa30 con 0x5587e00db170 2024-05-17T22:41:25.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.988+0000 7f982fc79b80 10 client.25189 fetch_fsmap learned FSMap version 30 2024-05-17T22:41:25.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.988+0000 7f982fc79b80 1 -- 172.21.15.12:0/2168464540 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5587e00f2500 con 0x5587e00db170 2024-05-17T22:41:25.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.992+0000 7f97feff5700 1 -- 172.21.15.12:0/2168464540 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f981400ae10 con 0x5587e00db170 2024-05-17T22:41:25.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.992+0000 7f97feff5700 1 -- 172.21.15.12:0/2168464540 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f9814026070 con 0x5587e00db170 2024-05-17T22:41:25.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.993+0000 7f982fc79b80 10 client.25189 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T22:41:25.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.993+0000 7f982fc79b80 10 client.25189 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:41:25.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.993+0000 7f982fc79b80 20 client.25189 populate_metadata read hostname 'smithi012' 2024-05-17T22:41:25.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.993+0000 7f982fc79b80 1 --2- 172.21.15.12:0/2168464540 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5587e00f49d0 0x5587e00f6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:25.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.993+0000 7f982fc79b80 4 client.25189 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:41:25.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.993+0000 7f982fc79b80 1 -- 172.21.15.12:0/2168464540 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587e000b1e0 con 0x5587e00f49d0 2024-05-17T22:41:25.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.994+0000 7f981affd700 1 --2- 172.21.15.12:0/2168464540 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5587e00f49d0 0x5587e00f6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:25.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.995+0000 7f981affd700 1 --2- 172.21.15.12:0/2168464540 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5587e00f49d0 0x5587e00f6df0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f34c123fc157d233 server_cookie=a78e4f39c1b00c2e in_seq=0 out_seq=0 2024-05-17T22:41:25.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.995+0000 7f97feff5700 10 client.25189 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:41:25.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.996+0000 7f97feff5700 1 -- 172.21.15.12:0/2168464540 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5587e000b1e0 con 0x5587e00f49d0 2024-05-17T22:41:25.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.996+0000 7f97feff5700 10 client.25189 handle_command_reply: tid=0 2024-05-17T22:41:25.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.996+0000 7f982fc79b80 1 client.25189 shutdown 2024-05-17T22:41:26.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.998+0000 7f982fc79b80 1 -- 172.21.15.12:0/2168464540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587e00db170 msgr2=0x5587e00e43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:41:26.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.998+0000 7f982fc79b80 1 --2- 172.21.15.12:0/2168464540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587e00db170 0x5587e00e43e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x5587e00d6ae0 tx=0x7f981401a6a0 comp rx=0 tx=0).stop 2024-05-17T22:41:26.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.998+0000 7f982fc79b80 1 -- 172.21.15.12:0/2168464540 shutdown_connections 2024-05-17T22:41:26.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.998+0000 7f982fc79b80 1 --2- 172.21.15.12:0/2168464540 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5587e00f49d0 0x5587e00f6df0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:26.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.998+0000 7f982fc79b80 1 --2- 172.21.15.12:0/2168464540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587e00e4e60 0x5587e00f08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:26.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.998+0000 7f982fc79b80 1 --2- 172.21.15.12:0/2168464540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587e00db170 0x5587e00e43e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:26.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.998+0000 7f982fc79b80 1 --2- 172.21.15.12:0/2168464540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587e00e0cb0 0x5587e00e4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:26.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.998+0000 7f982fc79b80 1 -- 172.21.15.12:0/2168464540 >> 172.21.15.12:0/2168464540 conn(0x5587dffa0f00 msgr2=0x5587e00d5110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:41:26.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.999+0000 7f97feff5700 0 client.25189 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:41:26.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:25.999+0000 7f982fc79b80 1 -- 172.21.15.12:0/2168464540 shutdown_connections 2024-05-17T22:41:26.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.000+0000 7f982fc79b80 1 -- 172.21.15.12:0/2168464540 wait complete. 2024-05-17T22:41:26.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.000+0000 7f982fc79b80 20 client.25189 trim_cache size 0 max 16384 2024-05-17T22:41:26.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.036+0000 7f982fc79b80 1 Processor -- start 2024-05-17T22:41:26.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.036+0000 7f982fc79b80 1 -- start start 2024-05-17T22:41:26.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.037+0000 7f982fc79b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587e00db170 0x5587e00dac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:26.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.037+0000 7f982fc79b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587e00e0cb0 0x5587e00e3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:26.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.037+0000 7f982fc79b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587e00e4e60 0x5587e00fc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:26.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.037+0000 7f982fc79b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9808011560 con 0x5587e00e0cb0 2024-05-17T22:41:26.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.037+0000 7f982fc79b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f980800edc0 con 0x5587e00e4e60 2024-05-17T22:41:26.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.037+0000 7f982fc79b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f980800cc10 con 0x5587e00db170 2024-05-17T22:41:26.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.037+0000 7f981affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587e00e0cb0 0x5587e00e3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:26.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.037+0000 7f981affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587e00e0cb0 0x5587e00e3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59170/0 (socket says 172.21.15.12:59170) 2024-05-17T22:41:26.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.037+0000 7f981affd700 1 -- 172.21.15.12:0/2417996285 learned_addr learned my addr 172.21.15.12:0/2417996285 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:41:26.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.037+0000 7f981a7fc700 1 --2- 172.21.15.12:0/2417996285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587e00db170 0x5587e00dac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:26.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.037+0000 7f981a7fc700 1 --2- 172.21.15.12:0/2417996285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587e00e4e60 0x5587e00fc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:26.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.037+0000 7f981affd700 1 -- 172.21.15.12:0/2417996285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587e00db170 msgr2=0x5587e00dac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:41:26.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.037+0000 7f981affd700 1 --2- 172.21.15.12:0/2417996285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587e00db170 0x5587e00dac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:26.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.037+0000 7f981affd700 1 -- 172.21.15.12:0/2417996285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587e00e4e60 msgr2=0x5587e00fc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:41:26.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.037+0000 7f981affd700 1 --2- 172.21.15.12:0/2417996285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587e00e4e60 0x5587e00fc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:26.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.037+0000 7f981affd700 1 -- 172.21.15.12:0/2417996285 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587e00e02e0 con 0x5587e00e0cb0 2024-05-17T22:41:26.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.038+0000 7f981affd700 1 --2- 172.21.15.12:0/2417996285 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587e00e0cb0 0x5587e00e3320 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x5587e00d7320 tx=0x7f9810009a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97e420b124aeaf28 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:41:26.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.038+0000 7f9800ff9700 1 -- 172.21.15.12:0/2417996285 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98100155c0 con 0x5587e00e0cb0 2024-05-17T22:41:26.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.038+0000 7f9800ff9700 1 -- 172.21.15.12:0/2417996285 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9810015c10 con 0x5587e00e0cb0 2024-05-17T22:41:26.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.038+0000 7f9800ff9700 1 -- 172.21.15.12:0/2417996285 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98100155c0 con 0x5587e00e0cb0 2024-05-17T22:41:26.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.039+0000 7f982fc79b80 1 -- 172.21.15.12:0/2417996285 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587e00e0cb0 msgr2=0x5587e00e3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:41:26.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.039+0000 7f982fc79b80 1 --2- 172.21.15.12:0/2417996285 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587e00e0cb0 0x5587e00e3320 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x5587e00d7320 tx=0x7f9810009a80 comp rx=0 tx=0).stop 2024-05-17T22:41:26.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.040+0000 7f982fc79b80 1 -- 172.21.15.12:0/2417996285 shutdown_connections 2024-05-17T22:41:26.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.040+0000 7f982fc79b80 1 --2- 172.21.15.12:0/2417996285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587e00e4e60 0x5587e00fc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:26.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.040+0000 7f982fc79b80 1 --2- 172.21.15.12:0/2417996285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587e00db170 0x5587e00dac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:26.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.040+0000 7f982fc79b80 1 --2- 172.21.15.12:0/2417996285 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587e00e0cb0 0x5587e00e3320 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:26.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.040+0000 7f982fc79b80 1 -- 172.21.15.12:0/2417996285 >> 172.21.15.12:0/2417996285 conn(0x5587dffa0f00 msgr2=0x5587e00e4660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:41:26.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.040+0000 7f982fc79b80 1 -- 172.21.15.12:0/2417996285 shutdown_connections 2024-05-17T22:41:26.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.040+0000 7f982fc79b80 1 -- 172.21.15.12:0/2417996285 wait complete. 2024-05-17T22:41:26.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.040+0000 7f982fc79b80 1 -- start start 2024-05-17T22:41:26.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.040+0000 7f982fc79b80 1 Processor -- start 2024-05-17T22:41:26.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.041+0000 7f982fc79b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587e00db170 0x5587e00ec290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:26.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.041+0000 7f982fc79b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587e00e0cb0 0x5587e00ec7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:26.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.041+0000 7f982fc79b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587e00e4e60 0x5587e00e9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:26.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.041+0000 7f982fc79b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f980800d280 con 0x5587e00db170 2024-05-17T22:41:26.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.041+0000 7f981a7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587e00db170 0x5587e00ec290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:26.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.041+0000 7f981a7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587e00db170 0x5587e00ec290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59178/0 (socket says 172.21.15.12:59178) 2024-05-17T22:41:26.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.041+0000 7f981a7fc700 1 -- 172.21.15.12:0/1561225818 learned_addr learned my addr 172.21.15.12:0/1561225818 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:41:26.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.041+0000 7f981affd700 1 --2- 172.21.15.12:0/1561225818 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587e00e0cb0 0x5587e00ec7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:26.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.041+0000 7f982fc79b80 1 -- 172.21.15.12:0/1561225818 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f980800e6f0 con 0x5587e00e4e60 2024-05-17T22:41:26.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.041+0000 7f982fc79b80 1 -- 172.21.15.12:0/1561225818 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f980800d950 con 0x5587e00e0cb0 2024-05-17T22:41:26.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.042+0000 7f982fc79b80 10 client.0 _ll_register_callbacks cb 0x5587dff93f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:41:26.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.042+0000 7f981a7fc700 1 --2- 172.21.15.12:0/1561225818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587e00e4e60 0x5587e00e9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:26.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.042+0000 7f981a7fc700 1 -- 172.21.15.12:0/1561225818 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587e00e0cb0 msgr2=0x5587e00ec7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:41:26.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.042+0000 7f981a7fc700 1 --2- 172.21.15.12:0/1561225818 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587e00e0cb0 0x5587e00ec7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:26.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.042+0000 7f981a7fc700 1 -- 172.21.15.12:0/1561225818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587e00e4e60 msgr2=0x5587e00e9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:41:26.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.042+0000 7f981a7fc700 1 --2- 172.21.15.12:0/1561225818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587e00e4e60 0x5587e00e9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:26.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.042+0000 7f981a7fc700 1 -- 172.21.15.12:0/1561225818 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587e00ea040 con 0x5587e00db170 2024-05-17T22:41:26.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.042+0000 7f981affd700 1 --2- 172.21.15.12:0/1561225818 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587e00e0cb0 0x5587e00ec7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:41:26.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.042+0000 7f981a7fc700 1 --2- 172.21.15.12:0/1561225818 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587e00db170 0x5587e00ec290 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f9814004ba0 tx=0x7f981401a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4a6f77ce8ebebc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:41:26.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.042+0000 7f97feff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:41:26.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.042+0000 7f97feff5700 1 -- 172.21.15.12:0/1561225818 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9814037030 con 0x5587e00db170 2024-05-17T22:41:26.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.043+0000 7f97feff5700 1 -- 172.21.15.12:0/1561225818 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f98140159f0 con 0x5587e00db170 2024-05-17T22:41:26.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.043+0000 7f982fc79b80 1 -- 172.21.15.12:0/1561225818 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9810015c10 con 0x5587e00db170 2024-05-17T22:41:26.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.043+0000 7f97feff5700 1 -- 172.21.15.12:0/1561225818 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9814015bf0 con 0x5587e00db170 2024-05-17T22:41:26.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.043+0000 7f982fc79b80 10 client.15582 fetch_fsmap learned FSMap version 30 2024-05-17T22:41:26.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.043+0000 7f982fc79b80 1 -- 172.21.15.12:0/1561225818 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5587e015eb20 con 0x5587e00db170 2024-05-17T22:41:26.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.043+0000 7f97feff5700 1 -- 172.21.15.12:0/1561225818 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9810015c10 con 0x5587e00db170 2024-05-17T22:41:26.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.044+0000 7f97feff5700 1 -- 172.21.15.12:0/1561225818 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f9814026040 con 0x5587e00db170 2024-05-17T22:41:26.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.044+0000 7f982fc79b80 10 client.15582 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T22:41:26.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.044+0000 7f982fc79b80 10 client.15582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:41:26.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.044+0000 7f982fc79b80 20 client.15582 populate_metadata read hostname 'smithi012' 2024-05-17T22:41:26.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.044+0000 7f982fc79b80 1 --2- 172.21.15.12:0/1561225818 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5587e015f260 0x5587e0161680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:41:26.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.044+0000 7f982fc79b80 4 client.15582 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:41:26.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.044+0000 7f982fc79b80 1 -- 172.21.15.12:0/1561225818 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5587e00049c0 con 0x5587e015f260 2024-05-17T22:41:26.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.044+0000 7f981affd700 1 --2- 172.21.15.12:0/1561225818 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5587e015f260 0x5587e0161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:41:26.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:25 smithi096 ceph-mon[36065]: pgmap v504: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 13 MiB/s wr, 899 op/s 2024-05-17T22:41:26.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.045+0000 7f981affd700 1 --2- 172.21.15.12:0/1561225818 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5587e015f260 0x5587e0161680 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f93b8da384dade4 server_cookie=9870284475091935 in_seq=0 out_seq=0 2024-05-17T22:41:26.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.045+0000 7f97feff5700 10 client.15582 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:41:26.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.045+0000 7f97feff5700 1 -- 172.21.15.12:0/1561225818 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5587e00049c0 con 0x5587e015f260 2024-05-17T22:41:26.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.045+0000 7f97feff5700 10 client.15582 handle_command_reply: tid=0 2024-05-17T22:41:26.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.045+0000 7f982fc79b80 1 client.15582 shutdown 2024-05-17T22:41:26.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.046+0000 7f982fc79b80 1 -- 172.21.15.12:0/1561225818 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587e00db170 msgr2=0x5587e00ec290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:41:26.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.046+0000 7f982fc79b80 1 --2- 172.21.15.12:0/1561225818 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587e00db170 0x5587e00ec290 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f9814004ba0 tx=0x7f981401a6a0 comp rx=0 tx=0).stop 2024-05-17T22:41:26.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.046+0000 7f982fc79b80 1 -- 172.21.15.12:0/1561225818 shutdown_connections 2024-05-17T22:41:26.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.046+0000 7f982fc79b80 1 --2- 172.21.15.12:0/1561225818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587e00e4e60 0x5587e00e9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:26.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.046+0000 7f982fc79b80 1 --2- 172.21.15.12:0/1561225818 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5587e015f260 0x5587e0161680 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:26.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.046+0000 7f982fc79b80 1 --2- 172.21.15.12:0/1561225818 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587e00db170 0x5587e00ec290 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:26.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.046+0000 7f982fc79b80 1 --2- 172.21.15.12:0/1561225818 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587e00e0cb0 0x5587e00ec7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:26.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.046+0000 7f982fc79b80 1 -- 172.21.15.12:0/1561225818 >> 172.21.15.12:0/1561225818 conn(0x5587dffa0f00 msgr2=0x5587e00e4660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:41:26.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.046+0000 7f97feff5700 0 client.15582 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:41:26.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.047+0000 7f982fc79b80 1 -- 172.21.15.12:0/1561225818 shutdown_connections 2024-05-17T22:41:26.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.047+0000 7f982fc79b80 1 -- 172.21.15.12:0/1561225818 wait complete. 2024-05-17T22:41:26.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.047+0000 7f982fc79b80 20 client.15582 trim_cache size 0 max 16384 2024-05-17T22:41:26.053 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:41:26.053 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-17T22:41:26.053 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-17T22:41:26.053 INFO:teuthology.orchestra.run.smithi012.stdout: "8176a2f5-e0bb-454d-9a04-f98da67e5337": { 2024-05-17T22:41:26.053 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-17T22:41:26.053 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "8176a2f5-e0bb-454d-9a04-f98da67e5337", 2024-05-17T22:41:26.053 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-17T22:41:26.053 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:41:26.053 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:41:26.053 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:41:26.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.047+0000 7f981bfff700 1 -- 172.21.15.12:0/1284954781 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f98040b2a90 msgr2=0x7f98040b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:41:26.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.047+0000 7f981bfff700 1 --2- 172.21.15.12:0/1284954781 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f98040b2a90 0x7f98040b4fa0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f980c052c80 tx=0x7f980c01acb0 comp rx=0 tx=0).stop 2024-05-17T22:41:26.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.047+0000 7f981bfff700 1 -- 172.21.15.12:0/1284954781 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f981c1089b0 msgr2=0x7f981c0708a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:41:26.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.047+0000 7f981bfff700 1 --2- 172.21.15.12:0/1284954781 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f981c1089b0 0x7f981c0708a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f980c003b50 tx=0x7f980c003b80 comp rx=0 tx=0).stop 2024-05-17T22:41:26.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.048+0000 7f981bfff700 1 -- 172.21.15.12:0/1284954781 shutdown_connections 2024-05-17T22:41:26.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.048+0000 7f981bfff700 1 --2- 172.21.15.12:0/1284954781 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f98040b2a90 0x7f98040b4fa0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:26.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.048+0000 7f981bfff700 1 --2- 172.21.15.12:0/1284954781 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f981c1089b0 0x7f981c0708a0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:26.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.048+0000 7f981bfff700 1 --2- 172.21.15.12:0/1284954781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f981c102980 0x7f981c075230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:26.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.048+0000 7f981bfff700 1 --2- 172.21.15.12:0/1284954781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f981c1032d0 0x7f981c070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:41:26.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.048+0000 7f981bfff700 1 -- 172.21.15.12:0/1284954781 >> 172.21.15.12:0/1284954781 conn(0x7f981c0fe6b0 msgr2=0x7f981c100ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:41:26.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.048+0000 7f981bfff700 1 -- 172.21.15.12:0/1284954781 shutdown_connections 2024-05-17T22:41:26.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:41:26.048+0000 7f981bfff700 1 -- 172.21.15.12:0/1284954781 wait complete. 2024-05-17T22:41:26.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 125.0 s, 1922.9 tps, lat 52.031 ms stddev 63.153 2024-05-17T22:41:26.901 INFO:tasks.cephfs.filesystem:scrub status for tag:8176a2f5-e0bb-454d-9a04-f98da67e5337 - {'path': '/', 'tag': '8176a2f5-e0bb-454d-9a04-f98da67e5337', 'options': 'recursive,force'} 2024-05-17T22:41:27.979 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:27 smithi012 ceph-mon[31632]: pgmap v505: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 12 MiB/s wr, 763 op/s 2024-05-17T22:41:27.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:27 smithi151 ceph-mon[36101]: pgmap v505: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 12 MiB/s wr, 763 op/s 2024-05-17T22:41:28.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:27 smithi096 ceph-mon[36065]: pgmap v505: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 12 MiB/s wr, 763 op/s 2024-05-17T22:41:28.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:28 smithi151 ceph-mon[36101]: scrub summary: idle 2024-05-17T22:41:29.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:28 smithi096 ceph-mon[36065]: scrub summary: idle 2024-05-17T22:41:29.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:28 smithi012 ceph-mon[31632]: scrub summary: idle 2024-05-17T22:41:29.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:29 smithi151 ceph-mon[36101]: scrub summary: idle 2024-05-17T22:41:29.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:29 smithi151 ceph-mon[36101]: scrub summary: idle 2024-05-17T22:41:29.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:29 smithi151 ceph-mon[36101]: pgmap v506: 129 pgs: 129 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 13 MiB/s wr, 928 op/s 2024-05-17T22:41:30.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:29 smithi096 ceph-mon[36065]: scrub summary: idle 2024-05-17T22:41:30.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:29 smithi096 ceph-mon[36065]: scrub summary: idle 2024-05-17T22:41:30.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:29 smithi096 ceph-mon[36065]: pgmap v506: 129 pgs: 129 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 13 MiB/s wr, 928 op/s 2024-05-17T22:41:30.071 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:29 smithi012 ceph-mon[31632]: scrub summary: idle 2024-05-17T22:41:30.071 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:29 smithi012 ceph-mon[31632]: scrub summary: idle 2024-05-17T22:41:30.071 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:29 smithi012 ceph-mon[31632]: pgmap v506: 129 pgs: 129 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 13 MiB/s wr, 928 op/s 2024-05-17T22:41:31.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 130.0 s, 1640.6 tps, lat 61.071 ms stddev 68.796 2024-05-17T22:41:31.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:31 smithi151 ceph-mon[36101]: pgmap v507: 129 pgs: 129 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.9 MiB/s wr, 652 op/s 2024-05-17T22:41:32.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:31 smithi096 ceph-mon[36065]: pgmap v507: 129 pgs: 129 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.9 MiB/s wr, 652 op/s 2024-05-17T22:41:32.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:31 smithi012 ceph-mon[31632]: pgmap v507: 129 pgs: 129 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.9 MiB/s wr, 652 op/s 2024-05-17T22:41:33.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:33 smithi151 ceph-mon[36101]: pgmap v508: 129 pgs: 129 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.6 MiB/s wr, 670 op/s 2024-05-17T22:41:34.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:33 smithi096 ceph-mon[36065]: pgmap v508: 129 pgs: 129 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.6 MiB/s wr, 670 op/s 2024-05-17T22:41:34.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:33 smithi012 ceph-mon[31632]: pgmap v508: 129 pgs: 129 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.6 MiB/s wr, 670 op/s 2024-05-17T22:41:35.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:35 smithi151 ceph-mon[36101]: pgmap v509: 129 pgs: 129 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.9 MiB/s wr, 824 op/s 2024-05-17T22:41:36.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:35 smithi096 ceph-mon[36065]: pgmap v509: 129 pgs: 129 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.9 MiB/s wr, 824 op/s 2024-05-17T22:41:36.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:35 smithi012 ceph-mon[31632]: pgmap v509: 129 pgs: 129 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.9 MiB/s wr, 824 op/s 2024-05-17T22:41:36.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 135.0 s, 1461.2 tps, lat 68.242 ms stddev 118.154 2024-05-17T22:41:38.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:37 smithi096 ceph-mon[36065]: pgmap v510: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 7.6 MiB/s wr, 640 op/s 2024-05-17T22:41:38.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:37 smithi012 ceph-mon[31632]: pgmap v510: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 7.6 MiB/s wr, 640 op/s 2024-05-17T22:41:38.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:37 smithi151 ceph-mon[36101]: pgmap v510: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 7.6 MiB/s wr, 640 op/s 2024-05-17T22:41:40.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:39 smithi096 ceph-mon[36065]: pgmap v511: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 13 MiB/s wr, 945 op/s 2024-05-17T22:41:40.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:39 smithi012 ceph-mon[31632]: pgmap v511: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 13 MiB/s wr, 945 op/s 2024-05-17T22:41:40.243 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:39 smithi151 ceph-mon[36101]: pgmap v511: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 13 MiB/s wr, 945 op/s 2024-05-17T22:41:41.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 140.0 s, 2143.4 tps, lat 46.945 ms stddev 53.477 2024-05-17T22:41:42.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:41 smithi096 ceph-mon[36065]: pgmap v512: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 13 MiB/s wr, 817 op/s 2024-05-17T22:41:42.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:41 smithi012 ceph-mon[31632]: pgmap v512: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 13 MiB/s wr, 817 op/s 2024-05-17T22:41:42.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:41 smithi151 ceph-mon[36101]: pgmap v512: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 13 MiB/s wr, 817 op/s 2024-05-17T22:41:44.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:43 smithi096 ceph-mon[36065]: pgmap v513: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 14 MiB/s wr, 864 op/s 2024-05-17T22:41:44.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:43 smithi012 ceph-mon[31632]: pgmap v513: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 14 MiB/s wr, 864 op/s 2024-05-17T22:41:44.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:43 smithi151 ceph-mon[36101]: pgmap v513: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 14 MiB/s wr, 864 op/s 2024-05-17T22:41:46.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:45 smithi096 ceph-mon[36065]: pgmap v514: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 964 op/s 2024-05-17T22:41:46.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:45 smithi012 ceph-mon[31632]: pgmap v514: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 964 op/s 2024-05-17T22:41:46.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:45 smithi151 ceph-mon[36101]: pgmap v514: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 964 op/s 2024-05-17T22:41:46.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 145.0 s, 1792.2 tps, lat 55.327 ms stddev 61.737 2024-05-17T22:41:48.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:47 smithi096 ceph-mon[36065]: pgmap v515: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 733 op/s 2024-05-17T22:41:48.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:47 smithi012 ceph-mon[31632]: pgmap v515: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 733 op/s 2024-05-17T22:41:48.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:47 smithi151 ceph-mon[36101]: pgmap v515: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 733 op/s 2024-05-17T22:41:50.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:49 smithi096 ceph-mon[36065]: pgmap v516: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 837 op/s 2024-05-17T22:41:50.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:49 smithi012 ceph-mon[31632]: pgmap v516: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 837 op/s 2024-05-17T22:41:50.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:49 smithi151 ceph-mon[36101]: pgmap v516: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 837 op/s 2024-05-17T22:41:51.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 150.0 s, 1950.0 tps, lat 51.788 ms stddev 60.459 2024-05-17T22:41:52.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:51 smithi096 ceph-mon[36065]: pgmap v517: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.7 MiB/s wr, 591 op/s 2024-05-17T22:41:52.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:51 smithi012 ceph-mon[31632]: pgmap v517: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.7 MiB/s wr, 591 op/s 2024-05-17T22:41:52.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:51 smithi151 ceph-mon[36101]: pgmap v517: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.7 MiB/s wr, 591 op/s 2024-05-17T22:41:54.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:53 smithi096 ceph-mon[36065]: pgmap v518: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.5 MiB/s wr, 663 op/s 2024-05-17T22:41:54.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:53 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:41:54.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:53 smithi012 ceph-mon[31632]: pgmap v518: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.5 MiB/s wr, 663 op/s 2024-05-17T22:41:54.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:53 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:41:54.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:53 smithi151 ceph-mon[36101]: pgmap v518: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.5 MiB/s wr, 663 op/s 2024-05-17T22:41:54.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:53 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:41:55.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:54 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:41:55.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:54 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:41:55.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:54 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:41:55.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:54 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:41:55.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:54 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:41:55.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:54 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:41:56.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:55 smithi096 ceph-mon[36065]: pgmap v519: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.1 MiB/s wr, 714 op/s 2024-05-17T22:41:56.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:55 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:41:56.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:55 smithi012 ceph-mon[31632]: pgmap v519: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.1 MiB/s wr, 714 op/s 2024-05-17T22:41:56.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:55 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:41:56.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:55 smithi151 ceph-mon[36101]: pgmap v519: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.1 MiB/s wr, 714 op/s 2024-05-17T22:41:56.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:55 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:41:56.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 155.0 s, 1671.2 tps, lat 59.644 ms stddev 70.592 2024-05-17T22:41:56.902 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-17T22:41:57.276 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:41:58.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:57 smithi012 ceph-mon[31632]: pgmap v520: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5.2 MiB/s wr, 554 op/s 2024-05-17T22:41:58.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:57 smithi151 ceph-mon[36101]: pgmap v520: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5.2 MiB/s wr, 554 op/s 2024-05-17T22:41:58.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:57 smithi096 ceph-mon[36065]: pgmap v520: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5.2 MiB/s wr, 554 op/s 2024-05-17T22:42:00.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:41:59 smithi012 ceph-mon[31632]: pgmap v521: 129 pgs: 129 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 5.9 MiB/s wr, 648 op/s 2024-05-17T22:42:00.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:41:59 smithi151 ceph-mon[36101]: pgmap v521: 129 pgs: 129 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 5.9 MiB/s wr, 648 op/s 2024-05-17T22:42:00.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:41:59 smithi096 ceph-mon[36065]: pgmap v521: 129 pgs: 129 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 5.9 MiB/s wr, 648 op/s 2024-05-17T22:42:01.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.166+0000 7f18770d7700 1 -- 172.21.15.12:0/83605555 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1870102590 msgr2=0x7f18701029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:01.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.166+0000 7f18770d7700 1 --2- 172.21.15.12:0/83605555 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1870102590 0x7f18701029a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1864009f90 tx=0x7f186400b330 comp rx=0 tx=0).stop 2024-05-17T22:42:01.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.168+0000 7f18770d7700 1 -- 172.21.15.12:0/83605555 shutdown_connections 2024-05-17T22:42:01.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.169+0000 7f18770d7700 1 --2- 172.21.15.12:0/83605555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1870107810 0x7f187010f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:01.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.169+0000 7f18770d7700 1 --2- 172.21.15.12:0/83605555 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1870102590 0x7f18701029a0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:01.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.169+0000 7f18770d7700 1 --2- 172.21.15.12:0/83605555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1870106ec0 0x7f18701072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:01.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.169+0000 7f18770d7700 1 -- 172.21.15.12:0/83605555 >> 172.21.15.12:0/83605555 conn(0x7f1870078040 msgr2=0x7f18700784a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:42:01.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.170+0000 7f18770d7700 1 -- 172.21.15.12:0/83605555 shutdown_connections 2024-05-17T22:42:01.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.170+0000 7f18770d7700 1 -- 172.21.15.12:0/83605555 wait complete. 2024-05-17T22:42:01.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.171+0000 7f18770d7700 1 Processor -- start 2024-05-17T22:42:01.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.171+0000 7f18770d7700 1 -- start start 2024-05-17T22:42:01.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.171+0000 7f18770d7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1870102590 0x7f187010aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:01.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.171+0000 7f18770d7700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1870106ec0 0x7f187010b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:01.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.171+0000 7f18770d7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1870107810 0x7f18701c62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:01.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.171+0000 7f18770d7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f187006f1f0 con 0x7f1870102590 2024-05-17T22:42:01.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.171+0000 7f18770d7700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f187006f070 con 0x7f1870107810 2024-05-17T22:42:01.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.171+0000 7f18770d7700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f187006f370 con 0x7f1870106ec0 2024-05-17T22:42:01.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.172+0000 7f1875674700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1870107810 0x7f18701c62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:01.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.172+0000 7f1875674700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1870107810 0x7f18701c62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:46118/0 (socket says 172.21.15.12:46118) 2024-05-17T22:42:01.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.172+0000 7f1875674700 1 -- 172.21.15.12:0/1647155080 learned_addr learned my addr 172.21.15.12:0/1647155080 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:42:01.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.172+0000 7f186ffff700 1 --2- 172.21.15.12:0/1647155080 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1870106ec0 0x7f187010b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:01.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.172+0000 7f1875674700 1 -- 172.21.15.12:0/1647155080 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1870106ec0 msgr2=0x7f187010b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:01.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.172+0000 7f1875674700 1 --2- 172.21.15.12:0/1647155080 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1870106ec0 0x7f187010b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:01.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.172+0000 7f1875674700 1 -- 172.21.15.12:0/1647155080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1870102590 msgr2=0x7f187010aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:01.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.172+0000 7f1874e73700 1 --2- 172.21.15.12:0/1647155080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1870102590 0x7f187010aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:01.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.172+0000 7f1875674700 1 --2- 172.21.15.12:0/1647155080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1870102590 0x7f187010aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:01.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.173+0000 7f1875674700 1 -- 172.21.15.12:0/1647155080 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18701c67f0 con 0x7f1870107810 2024-05-17T22:42:01.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.173+0000 7f1875674700 1 --2- 172.21.15.12:0/1647155080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1870107810 0x7f18701c62b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f186800b580 tx=0x7f186800b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:42:01.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.173+0000 7f1874e73700 1 --2- 172.21.15.12:0/1647155080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1870102590 0x7f187010aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:42:01.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.173+0000 7f186dffb700 1 -- 172.21.15.12:0/1647155080 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1868011830 con 0x7f1870107810 2024-05-17T22:42:01.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.173+0000 7f18770d7700 1 -- 172.21.15.12:0/1647155080 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18701c6ae0 con 0x7f1870107810 2024-05-17T22:42:01.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.173+0000 7f186dffb700 1 -- 172.21.15.12:0/1647155080 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1868016070 con 0x7f1870107810 2024-05-17T22:42:01.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.173+0000 7f186dffb700 1 -- 172.21.15.12:0/1647155080 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1868010d10 con 0x7f1870107810 2024-05-17T22:42:01.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.173+0000 7f18770d7700 1 -- 172.21.15.12:0/1647155080 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18701c7020 con 0x7f1870107810 2024-05-17T22:42:01.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.180+0000 7f186dffb700 1 -- 172.21.15.12:0/1647155080 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1868010410 con 0x7f1870107810 2024-05-17T22:42:01.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.183+0000 7f186dffb700 1 --2- 172.21.15.12:0/1647155080 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f18600b29c0 0x7f18600b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:01.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.183+0000 7f186dffb700 1 -- 172.21.15.12:0/1647155080 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f18680e1bf0 con 0x7f1870107810 2024-05-17T22:42:01.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.183+0000 7f1874e73700 1 --2- 172.21.15.12:0/1647155080 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f18600b29c0 0x7f18600b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:01.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.184+0000 7f1885ab2b80 1 Processor -- start 2024-05-17T22:42:01.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.184+0000 7f1885ab2b80 1 -- start start 2024-05-17T22:42:01.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.184+0000 7f1874e73700 1 --2- 172.21.15.12:0/1647155080 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f18600b29c0 0x7f18600b4ed0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1864009660 tx=0x7f1864009d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:42:01.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.184+0000 7f1885ab2b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f328c6d9f0 0x55f328df0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:01.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.185+0000 7f1885ab2b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f328df1400 0x55f328deb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:01.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.185+0000 7f1885ab2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f328debb00 0x55f328debf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:01.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.185+0000 7f1885ab2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f328d1a0d0 con 0x55f328debb00 2024-05-17T22:42:01.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.185+0000 7f1885ab2b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f328d09c70 con 0x55f328df1400 2024-05-17T22:42:01.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.185+0000 7f1885ab2b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f328d0dcb0 con 0x55f328c6d9f0 2024-05-17T22:42:01.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.185+0000 7f1874e73700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f328c6d9f0 0x55f328df0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:01.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.185+0000 7f1874e73700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f328c6d9f0 0x55f328df0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:56268/0 (socket says 172.21.15.12:56268) 2024-05-17T22:42:01.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.185+0000 7f1874e73700 1 -- 172.21.15.12:0/777313565 learned_addr learned my addr 172.21.15.12:0/777313565 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:42:01.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.185+0000 7f186ffff700 1 --2- 172.21.15.12:0/777313565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f328df1400 0x55f328deb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:01.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.185+0000 7f1875674700 1 --2- 172.21.15.12:0/777313565 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f328debb00 0x55f328debf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:01.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.186+0000 7f1874e73700 1 -- 172.21.15.12:0/777313565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f328df1400 msgr2=0x55f328deb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:42:01.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.186+0000 7f1874e73700 1 --2- 172.21.15.12:0/777313565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f328df1400 0x55f328deb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:01.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.186+0000 7f1874e73700 1 -- 172.21.15.12:0/777313565 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f328debb00 msgr2=0x55f328debf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:42:01.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.186+0000 7f1874e73700 1 --2- 172.21.15.12:0/777313565 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f328debb00 0x55f328debf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:01.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.186+0000 7f1874e73700 1 -- 172.21.15.12:0/777313565 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f328ca35c0 con 0x55f328c6d9f0 2024-05-17T22:42:01.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.186+0000 7f1874e73700 1 --2- 172.21.15.12:0/777313565 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f328c6d9f0 0x55f328df0ec0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1864003de0 tx=0x7f186401b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed74ee6ef9d37ecb server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:42:01.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.187+0000 7f185affd700 1 -- 172.21.15.12:0/777313565 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1864021070 con 0x55f328c6d9f0 2024-05-17T22:42:01.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.187+0000 7f185affd700 1 -- 172.21.15.12:0/777313565 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f186402d070 con 0x55f328c6d9f0 2024-05-17T22:42:01.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.187+0000 7f185affd700 1 -- 172.21.15.12:0/777313565 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18640286d0 con 0x55f328c6d9f0 2024-05-17T22:42:01.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.188+0000 7f1885ab2b80 1 -- 172.21.15.12:0/777313565 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f328c6d9f0 msgr2=0x55f328df0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:01.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.188+0000 7f1885ab2b80 1 --2- 172.21.15.12:0/777313565 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f328c6d9f0 0x55f328df0ec0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1864003de0 tx=0x7f186401b000 comp rx=0 tx=0).stop 2024-05-17T22:42:01.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.189+0000 7f1885ab2b80 1 -- 172.21.15.12:0/777313565 shutdown_connections 2024-05-17T22:42:01.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.189+0000 7f1885ab2b80 1 --2- 172.21.15.12:0/777313565 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f328debb00 0x55f328debf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:01.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.189+0000 7f1885ab2b80 1 --2- 172.21.15.12:0/777313565 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f328c6d9f0 0x55f328df0ec0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:01.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.189+0000 7f1885ab2b80 1 --2- 172.21.15.12:0/777313565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f328df1400 0x55f328deb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:01.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.189+0000 7f1885ab2b80 1 -- 172.21.15.12:0/777313565 >> 172.21.15.12:0/777313565 conn(0x55f328ccd7d0 msgr2=0x55f328de61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:42:01.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.192+0000 7f1885ab2b80 1 -- 172.21.15.12:0/777313565 shutdown_connections 2024-05-17T22:42:01.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.193+0000 7f1885ab2b80 1 -- 172.21.15.12:0/777313565 wait complete. 2024-05-17T22:42:01.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.193+0000 7f1885ab2b80 1 -- start start 2024-05-17T22:42:01.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.193+0000 7f1885ab2b80 1 Processor -- start 2024-05-17T22:42:01.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.194+0000 7f1885ab2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f328c6d9f0 0x55f328df4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:01.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.195+0000 7f1885ab2b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f328debb00 0x55f328df5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:01.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.195+0000 7f1885ab2b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f328df1400 0x55f328df5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:01.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.195+0000 7f1885ab2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f328c11150 con 0x55f328c6d9f0 2024-05-17T22:42:01.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.195+0000 7f1885ab2b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f328cab940 con 0x55f328df1400 2024-05-17T22:42:01.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.195+0000 7f1885ab2b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f328df7dc0 con 0x55f328debb00 2024-05-17T22:42:01.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.195+0000 7f1875674700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f328df1400 0x55f328df5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:01.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.195+0000 7f186ffff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f328debb00 0x55f328df5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:01.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.195+0000 7f186ffff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f328debb00 0x55f328df5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:56272/0 (socket says 172.21.15.12:56272) 2024-05-17T22:42:01.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.195+0000 7f186ffff700 1 -- 172.21.15.12:0/1796939986 learned_addr learned my addr 172.21.15.12:0/1796939986 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:42:01.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.196+0000 7f186ffff700 1 -- 172.21.15.12:0/1796939986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f328df1400 msgr2=0x55f328df5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:42:01.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.196+0000 7f186ffff700 1 --2- 172.21.15.12:0/1796939986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f328df1400 0x55f328df5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:01.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.196+0000 7f186ffff700 1 -- 172.21.15.12:0/1796939986 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f328c6d9f0 msgr2=0x55f328df4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:42:01.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.196+0000 7f186ffff700 1 --2- 172.21.15.12:0/1796939986 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f328c6d9f0 0x55f328df4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:01.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.196+0000 7f186ffff700 1 -- 172.21.15.12:0/1796939986 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f328c5a480 con 0x55f328debb00 2024-05-17T22:42:01.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.196+0000 7f1885ab2b80 10 client.0 _ll_register_callbacks cb 0x55f328ca0b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:42:01.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.196+0000 7f186ffff700 1 --2- 172.21.15.12:0/1796939986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f328debb00 0x55f328df5040 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f185c00f7b0 tx=0x7f185c00fb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5106fd9f7fcf4851 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:42:01.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.196+0000 7f184ffff700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:42:01.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.196+0000 7f184ffff700 1 -- 172.21.15.12:0/1796939986 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c004220 con 0x55f328debb00 2024-05-17T22:42:01.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.196+0000 7f184ffff700 1 -- 172.21.15.12:0/1796939986 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f185c0043c0 con 0x55f328debb00 2024-05-17T22:42:01.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.196+0000 7f1885ab2b80 1 -- 172.21.15.12:0/1796939986 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f186402d070 con 0x55f328debb00 2024-05-17T22:42:01.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.197+0000 7f184ffff700 1 -- 172.21.15.12:0/1796939986 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f185c00d8b0 con 0x55f328debb00 2024-05-17T22:42:01.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.197+0000 7f1885ab2b80 10 client.25204 fetch_fsmap learned FSMap version 30 2024-05-17T22:42:01.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.197+0000 7f1885ab2b80 1 -- 172.21.15.12:0/1796939986 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55f328e02800 con 0x55f328debb00 2024-05-17T22:42:01.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.198+0000 7f184ffff700 1 -- 172.21.15.12:0/1796939986 <== mon.1 v2:172.21.15.151:3300/0 4 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f185c00da50 con 0x55f328debb00 2024-05-17T22:42:01.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.198+0000 7f1885ab2b80 10 client.25204 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T22:42:01.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.198+0000 7f1885ab2b80 10 client.25204 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:42:01.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.198+0000 7f1885ab2b80 20 client.25204 populate_metadata read hostname 'smithi012' 2024-05-17T22:42:01.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.198+0000 7f1885ab2b80 1 --2- 172.21.15.12:0/1796939986 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55f328e04da0 0x55f328e072b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:01.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.198+0000 7f1885ab2b80 4 client.25204 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:42:01.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.198+0000 7f1885ab2b80 1 -- 172.21.15.12:0/1796939986 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f186402d070 con 0x55f328e04da0 2024-05-17T22:42:01.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.198+0000 7f1874e73700 1 --2- 172.21.15.12:0/1796939986 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55f328e04da0 0x55f328e072b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:01.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.199+0000 7f184ffff700 10 client.25204 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:42:01.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.199+0000 7f1874e73700 1 --2- 172.21.15.12:0/1796939986 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55f328e04da0 0x55f328e072b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3108576c71268075 server_cookie=d0c9616ae7135095 in_seq=0 out_seq=0 2024-05-17T22:42:01.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.200+0000 7f184ffff700 1 -- 172.21.15.12:0/1796939986 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f186402d070 con 0x55f328e04da0 2024-05-17T22:42:01.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.200+0000 7f184ffff700 10 client.25204 handle_command_reply: tid=0 2024-05-17T22:42:01.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.200+0000 7f1885ab2b80 1 client.25204 shutdown 2024-05-17T22:42:01.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.200+0000 7f1885ab2b80 1 -- 172.21.15.12:0/1796939986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f328debb00 msgr2=0x55f328df5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:01.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.200+0000 7f1885ab2b80 1 --2- 172.21.15.12:0/1796939986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f328debb00 0x55f328df5040 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f185c00f7b0 tx=0x7f185c00fb70 comp rx=0 tx=0).stop 2024-05-17T22:42:01.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.200+0000 7f1885ab2b80 1 -- 172.21.15.12:0/1796939986 shutdown_connections 2024-05-17T22:42:01.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.200+0000 7f1885ab2b80 1 --2- 172.21.15.12:0/1796939986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f328df1400 0x55f328df5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:01.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.200+0000 7f1885ab2b80 1 --2- 172.21.15.12:0/1796939986 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55f328e04da0 0x55f328e072b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:01.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.200+0000 7f1885ab2b80 1 --2- 172.21.15.12:0/1796939986 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f328c6d9f0 0x55f328df4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:01.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.200+0000 7f1885ab2b80 1 --2- 172.21.15.12:0/1796939986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f328debb00 0x55f328df5040 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:01.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.200+0000 7f184ffff700 0 client.25204 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:42:01.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.200+0000 7f1885ab2b80 1 -- 172.21.15.12:0/1796939986 >> 172.21.15.12:0/1796939986 conn(0x55f328ccd7d0 msgr2=0x55f328de6160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:42:01.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.201+0000 7f1885ab2b80 1 -- 172.21.15.12:0/1796939986 shutdown_connections 2024-05-17T22:42:01.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.201+0000 7f1885ab2b80 1 -- 172.21.15.12:0/1796939986 wait complete. 2024-05-17T22:42:01.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.201+0000 7f1885ab2b80 20 client.25204 trim_cache size 0 max 16384 2024-05-17T22:42:01.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.232+0000 7f1885ab2b80 1 Processor -- start 2024-05-17T22:42:01.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.232+0000 7f1885ab2b80 1 -- start start 2024-05-17T22:42:01.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.233+0000 7f1885ab2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f328c6d9f0 0x55f328e00d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:01.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.233+0000 7f1885ab2b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f328debb00 0x55f328df53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:01.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.233+0000 7f1885ab2b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f328df1400 0x55f328df0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:01.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.233+0000 7f1885ab2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f18701d24f0 con 0x55f328c6d9f0 2024-05-17T22:42:01.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.233+0000 7f1885ab2b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f18701cfd50 con 0x55f328df1400 2024-05-17T22:42:01.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.233+0000 7f1885ab2b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f18701cdba0 con 0x55f328debb00 2024-05-17T22:42:01.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.233+0000 7f1874e73700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f328c6d9f0 0x55f328e00d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:01.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.234+0000 7f1874e73700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f328c6d9f0 0x55f328e00d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42262/0 (socket says 172.21.15.12:42262) 2024-05-17T22:42:01.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.234+0000 7f1874e73700 1 -- 172.21.15.12:0/1664321200 learned_addr learned my addr 172.21.15.12:0/1664321200 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:42:01.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.234+0000 7f1875674700 1 --2- 172.21.15.12:0/1664321200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f328df1400 0x55f328df0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:01.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.234+0000 7f186ffff700 1 --2- 172.21.15.12:0/1664321200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f328debb00 0x55f328df53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:01.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.234+0000 7f1875674700 1 -- 172.21.15.12:0/1664321200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f328debb00 msgr2=0x55f328df53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:42:01.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.234+0000 7f1875674700 1 --2- 172.21.15.12:0/1664321200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f328debb00 0x55f328df53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:01.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.234+0000 7f1875674700 1 -- 172.21.15.12:0/1664321200 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f328c6d9f0 msgr2=0x55f328e00d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:42:01.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.234+0000 7f1875674700 1 --2- 172.21.15.12:0/1664321200 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f328c6d9f0 0x55f328e00d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:01.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.234+0000 7f1875674700 1 -- 172.21.15.12:0/1664321200 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f328de7560 con 0x55f328df1400 2024-05-17T22:42:01.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.234+0000 7f186ffff700 1 --2- 172.21.15.12:0/1664321200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f328debb00 0x55f328df53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:42:01.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.235+0000 7f1875674700 1 --2- 172.21.15.12:0/1664321200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f328df1400 0x55f328df0400 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f186800bd40 tx=0x7f1868010380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd038745f141c1e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:42:01.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.235+0000 7f185a7fc700 1 -- 172.21.15.12:0/1664321200 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18680a8e20 con 0x55f328df1400 2024-05-17T22:42:01.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.235+0000 7f185a7fc700 1 -- 172.21.15.12:0/1664321200 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18680ad1d0 con 0x55f328df1400 2024-05-17T22:42:01.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.235+0000 7f185a7fc700 1 -- 172.21.15.12:0/1664321200 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18680a4af0 con 0x55f328df1400 2024-05-17T22:42:01.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.236+0000 7f1885ab2b80 1 -- 172.21.15.12:0/1664321200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f328df1400 msgr2=0x55f328df0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:01.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.236+0000 7f1885ab2b80 1 --2- 172.21.15.12:0/1664321200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f328df1400 0x55f328df0400 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f186800bd40 tx=0x7f1868010380 comp rx=0 tx=0).stop 2024-05-17T22:42:01.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.236+0000 7f1885ab2b80 1 -- 172.21.15.12:0/1664321200 shutdown_connections 2024-05-17T22:42:01.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.236+0000 7f1885ab2b80 1 --2- 172.21.15.12:0/1664321200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f328df1400 0x55f328df0400 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:01.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.236+0000 7f1885ab2b80 1 --2- 172.21.15.12:0/1664321200 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f328c6d9f0 0x55f328e00d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:01.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.236+0000 7f1885ab2b80 1 --2- 172.21.15.12:0/1664321200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f328debb00 0x55f328df53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:01.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.236+0000 7f1885ab2b80 1 -- 172.21.15.12:0/1664321200 >> 172.21.15.12:0/1664321200 conn(0x55f328ccd7d0 msgr2=0x55f328defcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:42:01.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.236+0000 7f1885ab2b80 1 -- 172.21.15.12:0/1664321200 shutdown_connections 2024-05-17T22:42:01.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.236+0000 7f1885ab2b80 1 -- 172.21.15.12:0/1664321200 wait complete. 2024-05-17T22:42:01.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.237+0000 7f1885ab2b80 1 -- start start 2024-05-17T22:42:01.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.237+0000 7f1885ab2b80 1 Processor -- start 2024-05-17T22:42:01.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.237+0000 7f1885ab2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f328c6d9f0 0x55f328dfd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:01.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.237+0000 7f1885ab2b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f328debb00 0x55f328dfd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:01.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.237+0000 7f1885ab2b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f328df1400 0x55f328dfa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:01.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.237+0000 7f1885ab2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f18701ce210 con 0x55f328c6d9f0 2024-05-17T22:42:01.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.237+0000 7f1885ab2b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f18701cf680 con 0x55f328df1400 2024-05-17T22:42:01.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.237+0000 7f1885ab2b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f18701ce8e0 con 0x55f328debb00 2024-05-17T22:42:01.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.238+0000 7f1885ab2b80 10 client.0 _ll_register_callbacks cb 0x55f328ca0b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:42:01.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.238+0000 7f1875674700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f328df1400 0x55f328dfa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:01.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.238+0000 7f186ffff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f328debb00 0x55f328dfd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:01.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.238+0000 7f186ffff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f328debb00 0x55f328dfd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:56292/0 (socket says 172.21.15.12:56292) 2024-05-17T22:42:01.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.238+0000 7f186ffff700 1 -- 172.21.15.12:0/3879255300 learned_addr learned my addr 172.21.15.12:0/3879255300 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:42:01.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.238+0000 7f1874e73700 1 --2- 172.21.15.12:0/3879255300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f328c6d9f0 0x55f328dfd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:01.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.238+0000 7f186ffff700 1 -- 172.21.15.12:0/3879255300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f328df1400 msgr2=0x55f328dfa0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:42:01.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.238+0000 7f186ffff700 1 --2- 172.21.15.12:0/3879255300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f328df1400 0x55f328dfa0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:01.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.238+0000 7f186ffff700 1 -- 172.21.15.12:0/3879255300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f328c6d9f0 msgr2=0x55f328dfd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:42:01.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.238+0000 7f186ffff700 1 --2- 172.21.15.12:0/3879255300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f328c6d9f0 0x55f328dfd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:01.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.238+0000 7f186ffff700 1 -- 172.21.15.12:0/3879255300 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f328dfa7d0 con 0x55f328debb00 2024-05-17T22:42:01.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.239+0000 7f186ffff700 1 --2- 172.21.15.12:0/3879255300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f328debb00 0x55f328dfd610 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x55f328dde6c0 tx=0x7f185c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ef88b78c337adcf server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:42:01.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.239+0000 7f184ffff700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:42:01.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.239+0000 7f184ffff700 1 -- 172.21.15.12:0/3879255300 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c020630 con 0x55f328debb00 2024-05-17T22:42:01.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.239+0000 7f1885ab2b80 1 -- 172.21.15.12:0/3879255300 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18680ad1d0 con 0x55f328debb00 2024-05-17T22:42:01.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.239+0000 7f184ffff700 1 -- 172.21.15.12:0/3879255300 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f185c0207d0 con 0x55f328debb00 2024-05-17T22:42:01.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.239+0000 7f184ffff700 1 -- 172.21.15.12:0/3879255300 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f185c0209d0 con 0x55f328debb00 2024-05-17T22:42:01.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.239+0000 7f1885ab2b80 10 client.25210 fetch_fsmap learned FSMap version 30 2024-05-17T22:42:01.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.239+0000 7f1885ab2b80 1 -- 172.21.15.12:0/3879255300 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f328e6ffc0 con 0x55f328debb00 2024-05-17T22:42:01.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.240+0000 7f184ffff700 1 -- 172.21.15.12:0/3879255300 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18680ad1d0 con 0x55f328debb00 2024-05-17T22:42:01.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.240+0000 7f184ffff700 1 -- 172.21.15.12:0/3879255300 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f185c003850 con 0x55f328debb00 2024-05-17T22:42:01.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.240+0000 7f1885ab2b80 10 client.25210 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T22:42:01.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.240+0000 7f1885ab2b80 10 client.25210 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:42:01.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.240+0000 7f1885ab2b80 20 client.25210 populate_metadata read hostname 'smithi012' 2024-05-17T22:42:01.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.240+0000 7f1885ab2b80 1 --2- 172.21.15.12:0/3879255300 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55f328e70640 0x55f328e72a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:01.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.240+0000 7f1885ab2b80 4 client.25210 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:42:01.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.240+0000 7f1885ab2b80 1 -- 172.21.15.12:0/3879255300 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f328d1c020 con 0x55f328e70640 2024-05-17T22:42:01.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.240+0000 7f1874e73700 1 --2- 172.21.15.12:0/3879255300 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55f328e70640 0x55f328e72a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:01.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.241+0000 7f1874e73700 1 --2- 172.21.15.12:0/3879255300 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55f328e70640 0x55f328e72a60 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=447b3ed4dcc833c4 server_cookie=50969f0cdeeefa05 in_seq=0 out_seq=0 2024-05-17T22:42:01.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.241+0000 7f184ffff700 10 client.25210 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:42:01.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.241+0000 7f184ffff700 1 -- 172.21.15.12:0/3879255300 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f328d1c020 con 0x55f328e70640 2024-05-17T22:42:01.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.241+0000 7f184ffff700 10 client.25210 handle_command_reply: tid=0 2024-05-17T22:42:01.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.242+0000 7f1885ab2b80 1 client.25210 shutdown 2024-05-17T22:42:01.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.242+0000 7f1885ab2b80 1 -- 172.21.15.12:0/3879255300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f328debb00 msgr2=0x55f328dfd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:01.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.242+0000 7f1885ab2b80 1 --2- 172.21.15.12:0/3879255300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f328debb00 0x55f328dfd610 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x55f328dde6c0 tx=0x7f185c004750 comp rx=0 tx=0).stop 2024-05-17T22:42:01.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.242+0000 7f1885ab2b80 1 -- 172.21.15.12:0/3879255300 shutdown_connections 2024-05-17T22:42:01.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.242+0000 7f1885ab2b80 1 --2- 172.21.15.12:0/3879255300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f328df1400 0x55f328dfa0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:01.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.242+0000 7f1885ab2b80 1 --2- 172.21.15.12:0/3879255300 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55f328e70640 0x55f328e72a60 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:01.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.242+0000 7f1885ab2b80 1 --2- 172.21.15.12:0/3879255300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f328c6d9f0 0x55f328dfd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:01.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.242+0000 7f1885ab2b80 1 --2- 172.21.15.12:0/3879255300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f328debb00 0x55f328dfd610 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:01.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.242+0000 7f184ffff700 0 client.25210 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:42:01.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.242+0000 7f1885ab2b80 1 -- 172.21.15.12:0/3879255300 >> 172.21.15.12:0/3879255300 conn(0x55f328ccd7d0 msgr2=0x55f328de0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:42:01.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.242+0000 7f1885ab2b80 1 -- 172.21.15.12:0/3879255300 shutdown_connections 2024-05-17T22:42:01.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.242+0000 7f1885ab2b80 1 -- 172.21.15.12:0/3879255300 wait complete. 2024-05-17T22:42:01.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.242+0000 7f1885ab2b80 20 client.25210 trim_cache size 0 max 16384 2024-05-17T22:42:01.248 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:42:01.248 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-17T22:42:01.248 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-17T22:42:01.248 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:42:01.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.243+0000 7f18770d7700 1 -- 172.21.15.12:0/1647155080 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f18600b29c0 msgr2=0x7f18600b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:01.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.243+0000 7f18770d7700 1 --2- 172.21.15.12:0/1647155080 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f18600b29c0 0x7f18600b4ed0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1864009660 tx=0x7f1864009d40 comp rx=0 tx=0).stop 2024-05-17T22:42:01.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.243+0000 7f18770d7700 1 -- 172.21.15.12:0/1647155080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1870107810 msgr2=0x7f18701c62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:01.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.243+0000 7f18770d7700 1 --2- 172.21.15.12:0/1647155080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1870107810 0x7f18701c62b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f186800b580 tx=0x7f186800b940 comp rx=0 tx=0).stop 2024-05-17T22:42:01.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.243+0000 7f18770d7700 1 -- 172.21.15.12:0/1647155080 shutdown_connections 2024-05-17T22:42:01.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.243+0000 7f18770d7700 1 --2- 172.21.15.12:0/1647155080 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f18600b29c0 0x7f18600b4ed0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:01.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.243+0000 7f18770d7700 1 --2- 172.21.15.12:0/1647155080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1870107810 0x7f18701c62b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:01.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.243+0000 7f18770d7700 1 --2- 172.21.15.12:0/1647155080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1870102590 0x7f187010aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:01.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.243+0000 7f18770d7700 1 --2- 172.21.15.12:0/1647155080 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1870106ec0 0x7f187010b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:01.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.243+0000 7f18770d7700 1 -- 172.21.15.12:0/1647155080 >> 172.21.15.12:0/1647155080 conn(0x7f1870078040 msgr2=0x7f1870104ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:42:01.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.244+0000 7f18770d7700 1 -- 172.21.15.12:0/1647155080 shutdown_connections 2024-05-17T22:42:01.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:01.244+0000 7f18770d7700 1 -- 172.21.15.12:0/1647155080 wait complete. 2024-05-17T22:42:01.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 160.0 s, 1074.2 tps, lat 91.980 ms stddev 159.100 2024-05-17T22:42:01.973 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:01 smithi012 ceph-mon[31632]: pgmap v522: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8.7 MiB/s wr, 772 op/s 2024-05-17T22:42:02.015 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-17T22:42:02.016 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T22:42:02.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:01 smithi151 ceph-mon[36101]: pgmap v522: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8.7 MiB/s wr, 772 op/s 2024-05-17T22:42:02.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:01 smithi096 ceph-mon[36065]: pgmap v522: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8.7 MiB/s wr, 772 op/s 2024-05-17T22:42:02.456 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:42:04.008 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:03 smithi012 ceph-mon[31632]: pgmap v523: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 11 MiB/s wr, 873 op/s 2024-05-17T22:42:04.100 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:03 smithi096 ceph-mon[36065]: pgmap v523: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 11 MiB/s wr, 873 op/s 2024-05-17T22:42:04.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:03 smithi151 ceph-mon[36101]: pgmap v523: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 11 MiB/s wr, 873 op/s 2024-05-17T22:42:04.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:04.702+0000 7fd503d11700 1 -- 172.21.15.12:0/4171541750 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4fc10a530 msgr2=0x7fd4fc10a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:04.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:04.702+0000 7fd503d11700 1 --2- 172.21.15.12:0/4171541750 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4fc10a530 0x7fd4fc10a940 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec0098f0 tx=0x7fd4ec009c00 comp rx=0 tx=0).stop 2024-05-17T22:42:04.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:04.704+0000 7fd503d11700 1 -- 172.21.15.12:0/4171541750 shutdown_connections 2024-05-17T22:42:04.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:04.704+0000 7fd503d11700 1 --2- 172.21.15.12:0/4171541750 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4fc10f7b0 0x7fd4fc117110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:04.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:04.704+0000 7fd503d11700 1 --2- 172.21.15.12:0/4171541750 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4fc10a530 0x7fd4fc10a940 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:04.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:04.704+0000 7fd503d11700 1 --2- 172.21.15.12:0/4171541750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4fc10ee60 0x7fd4fc10f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:04.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:04.704+0000 7fd503d11700 1 -- 172.21.15.12:0/4171541750 >> 172.21.15.12:0/4171541750 conn(0x7fd4fc078570 msgr2=0x7fd4fc108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:42:04.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:04.705+0000 7fd503d11700 1 -- 172.21.15.12:0/4171541750 shutdown_connections 2024-05-17T22:42:04.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:04.708+0000 7fd503d11700 1 -- 172.21.15.12:0/4171541750 wait complete. 2024-05-17T22:42:04.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:04.710+0000 7fd503d11700 1 Processor -- start 2024-05-17T22:42:04.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:04.710+0000 7fd503d11700 1 -- start start 2024-05-17T22:42:04.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:04.711+0000 7fd503d11700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4fc10a530 0x7fd4fc06e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:04.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:04.711+0000 7fd503d11700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4fc10ee60 0x7fd4fc06ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:04.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:04.711+0000 7fd503d11700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4fc10f7b0 0x7fd4fc06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:04.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:04.711+0000 7fd503d11700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd4fc11a440 con 0x7fd4fc10a530 2024-05-17T22:42:04.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:04.711+0000 7fd503d11700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd4fc11a2c0 con 0x7fd4fc10ee60 2024-05-17T22:42:04.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:04.711+0000 7fd503d11700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd4fc11a5c0 con 0x7fd4fc10f7b0 2024-05-17T22:42:04.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:04.711+0000 7fd5012ac700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4fc10ee60 0x7fd4fc06ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:04.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:04.711+0000 7fd5012ac700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4fc10ee60 0x7fd4fc06ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:46156/0 (socket says 172.21.15.12:46156) 2024-05-17T22:42:04.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:04.711+0000 7fd5012ac700 1 -- 172.21.15.12:0/2177419941 learned_addr learned my addr 172.21.15.12:0/2177419941 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:42:04.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:04.711+0000 7fd5022ae700 1 --2- 172.21.15.12:0/2177419941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4fc10f7b0 0x7fd4fc06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:04.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:04.712+0000 7fd5012ac700 1 -- 172.21.15.12:0/2177419941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4fc10f7b0 msgr2=0x7fd4fc06f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:04.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:04.712+0000 7fd5012ac700 1 --2- 172.21.15.12:0/2177419941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4fc10f7b0 0x7fd4fc06f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:04.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:04.712+0000 7fd5012ac700 1 -- 172.21.15.12:0/2177419941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4fc10a530 msgr2=0x7fd4fc06e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:04.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:04.712+0000 7fd5012ac700 1 --2- 172.21.15.12:0/2177419941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4fc10a530 0x7fd4fc06e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:04.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:04.712+0000 7fd5012ac700 1 -- 172.21.15.12:0/2177419941 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4fc073090 con 0x7fd4fc10ee60 2024-05-17T22:42:04.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:04.712+0000 7fd5012ac700 1 --2- 172.21.15.12:0/2177419941 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4fc10ee60 0x7fd4fc06ebd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd4f800e970 tx=0x7fd4f800ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:42:04.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:04.712+0000 7fd4f2ffd700 1 -- 172.21.15.12:0/2177419941 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f800cbd0 con 0x7fd4fc10ee60 2024-05-17T22:42:04.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:04.712+0000 7fd4f2ffd700 1 -- 172.21.15.12:0/2177419941 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4f800cd70 con 0x7fd4fc10ee60 2024-05-17T22:42:04.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:04.712+0000 7fd503d11700 1 -- 172.21.15.12:0/2177419941 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4fc073380 con 0x7fd4fc10ee60 2024-05-17T22:42:04.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:04.713+0000 7fd503d11700 1 -- 172.21.15.12:0/2177419941 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4fc073870 con 0x7fd4fc10ee60 2024-05-17T22:42:04.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:04.713+0000 7fd4f2ffd700 1 -- 172.21.15.12:0/2177419941 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8010430 con 0x7fd4fc10ee60 2024-05-17T22:42:04.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:04.717+0000 7fd4f2ffd700 1 -- 172.21.15.12:0/2177419941 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd4f80105d0 con 0x7fd4fc10ee60 2024-05-17T22:42:04.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:04.717+0000 7fd503d11700 1 -- 172.21.15.12:0/2177419941 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4e0005330 con 0x7fd4fc10ee60 2024-05-17T22:42:04.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:04.718+0000 7fd4f2ffd700 1 --2- 172.21.15.12:0/2177419941 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd4e80b2900 0x7fd4e80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:04.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:04.718+0000 7fd4f2ffd700 1 -- 172.21.15.12:0/2177419941 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fd4f80e2a60 con 0x7fd4fc10ee60 2024-05-17T22:42:04.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:04.718+0000 7fd501aad700 1 --2- 172.21.15.12:0/2177419941 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd4e80b2900 0x7fd4e80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:04.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:04.719+0000 7fd501aad700 1 --2- 172.21.15.12:0/2177419941 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd4e80b2900 0x7fd4e80b4e10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec009680 tx=0x7fd4ec009f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:42:04.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:04.723+0000 7fd4f2ffd700 1 -- 172.21.15.12:0/2177419941 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd4f80ab1a0 con 0x7fd4fc10ee60 2024-05-17T22:42:05.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:05.188+0000 7fd503d11700 1 -- 172.21.15.12:0/2177419941 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4e0005a40 con 0x7fd4fc10ee60 2024-05-17T22:42:05.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:05.189+0000 7fd4f2ffd700 1 -- 172.21.15.12:0/2177419941 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11874 (secure 0 0 0) 0x7fd4f80aaaa0 con 0x7fd4fc10ee60 2024-05-17T22:42:05.193 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:42:05.193 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1407219319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1407219319},{"type":"v1","addr":"172.21.15.12:6835","nonce":1407219319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/630977886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":630977886},{"type":"v1","addr":"172.21.15.12:6837","nonce":630977886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3731914742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3731914742},{"type":"v1","addr":"172.21.15.12:6839","nonce":3731914742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/3642142501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3642142501},{"type":"v1","addr":"172.21.15.12:6841","nonce":3642142501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3554053059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3554053059},{"type":"v1","addr":"172.21.15.151:6833","nonce":3554053059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/4228325569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":4228325569},{"type":"v1","addr":"172.21.15.96:6833","nonce":4228325569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T22:34:42.642111+0000","modified":"2024-05-17T22:41:19.927691+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24461,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/542323754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":542323754},{"type":"v1","addr":"172.21.15.151:6835","nonce":542323754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1758442232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1758442232},{"type":"v1","addr":"172.21.15.96:6835","nonce":1758442232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3403324038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3403324038},{"type":"v1","addr":"172.21.15.151:6837","nonce":3403324038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.151:6839/2338353849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2338353849},{"type":"v1","addr":"172.21.15.151:6839","nonce":2338353849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.96:6837/4222574338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":4222574338},{"type":"v1","addr":"172.21.15.96:6837","nonce":4222574338}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/2986295519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2986295519},{"type":"v1","addr":"172.21.15.96:6839","nonce":2986295519}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24457,24482,24461]},"id":1}]} 2024-05-17T22:42:05.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:05.194+0000 7fd503d11700 1 -- 172.21.15.12:0/2177419941 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd4e80b2900 msgr2=0x7fd4e80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:05.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:05.194+0000 7fd503d11700 1 --2- 172.21.15.12:0/2177419941 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd4e80b2900 0x7fd4e80b4e10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec009680 tx=0x7fd4ec009f50 comp rx=0 tx=0).stop 2024-05-17T22:42:05.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:05.194+0000 7fd503d11700 1 -- 172.21.15.12:0/2177419941 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4fc10ee60 msgr2=0x7fd4fc06ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:05.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:05.194+0000 7fd503d11700 1 --2- 172.21.15.12:0/2177419941 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4fc10ee60 0x7fd4fc06ebd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd4f800e970 tx=0x7fd4f800ed30 comp rx=0 tx=0).stop 2024-05-17T22:42:05.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:05.194+0000 7fd503d11700 1 -- 172.21.15.12:0/2177419941 shutdown_connections 2024-05-17T22:42:05.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:05.194+0000 7fd503d11700 1 --2- 172.21.15.12:0/2177419941 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd4e80b2900 0x7fd4e80b4e10 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:05.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:05.194+0000 7fd503d11700 1 --2- 172.21.15.12:0/2177419941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4fc10f7b0 0x7fd4fc06f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:05.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:05.194+0000 7fd503d11700 1 --2- 172.21.15.12:0/2177419941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4fc10a530 0x7fd4fc06e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:05.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:05.194+0000 7fd503d11700 1 --2- 172.21.15.12:0/2177419941 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4fc10ee60 0x7fd4fc06ebd0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:05.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:05.194+0000 7fd503d11700 1 -- 172.21.15.12:0/2177419941 >> 172.21.15.12:0/2177419941 conn(0x7fd4fc078570 msgr2=0x7fd4fc078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:42:05.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:05.194+0000 7fd503d11700 1 -- 172.21.15.12:0/2177419941 shutdown_connections 2024-05-17T22:42:05.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:05.194+0000 7fd503d11700 1 -- 172.21.15.12:0/2177419941 wait complete. 2024-05-17T22:42:05.198 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 30 2024-05-17T22:42:05.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:05 smithi012 ceph-mon[31632]: pgmap v524: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 16 MiB/s wr, 1.06k op/s 2024-05-17T22:42:05.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:05 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/2177419941' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:42:05.998 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-17T22:42:06.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:05 smithi151 ceph-mon[36101]: pgmap v524: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 16 MiB/s wr, 1.06k op/s 2024-05-17T22:42:06.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:05 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/2177419941' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:42:06.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:05 smithi096 ceph-mon[36065]: pgmap v524: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 16 MiB/s wr, 1.06k op/s 2024-05-17T22:42:06.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:05 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/2177419941' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:42:06.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 165.0 s, 1419.7 tps, lat 71.250 ms stddev 81.743 2024-05-17T22:42:06.424 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:42:07.936 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:07 smithi012 ceph-mon[31632]: pgmap v525: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 15 MiB/s wr, 969 op/s 2024-05-17T22:42:08.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:07 smithi151 ceph-mon[36101]: pgmap v525: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 15 MiB/s wr, 969 op/s 2024-05-17T22:42:08.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:07 smithi096 ceph-mon[36065]: pgmap v525: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 15 MiB/s wr, 969 op/s 2024-05-17T22:42:09.278 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:42:09.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.396+0000 7f15ef117700 1 -- 172.21.15.12:0/2928100672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15e8103f10 msgr2=0x7f15e8104320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:09.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.396+0000 7f15ef117700 1 --2- 172.21.15.12:0/2928100672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15e8103f10 0x7f15e8104320 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f15dc009990 tx=0x7f15dc009ca0 comp rx=0 tx=0).stop 2024-05-17T22:42:09.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.396+0000 7f15ef117700 1 -- 172.21.15.12:0/2928100672 shutdown_connections 2024-05-17T22:42:09.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.396+0000 7f15ef117700 1 --2- 172.21.15.12:0/2928100672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15e8109190 0x7f15e8110af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:09.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.396+0000 7f15ef117700 1 --2- 172.21.15.12:0/2928100672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15e8103f10 0x7f15e8104320 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:09.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.396+0000 7f15ef117700 1 --2- 172.21.15.12:0/2928100672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15e8108840 0x7f15e8108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:09.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.396+0000 7f15ef117700 1 -- 172.21.15.12:0/2928100672 >> 172.21.15.12:0/2928100672 conn(0x7f15e80ff9b0 msgr2=0x7f15e8101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:42:09.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.396+0000 7f15ef117700 1 -- 172.21.15.12:0/2928100672 shutdown_connections 2024-05-17T22:42:09.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.396+0000 7f15ef117700 1 -- 172.21.15.12:0/2928100672 wait complete. 2024-05-17T22:42:09.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.397+0000 7f15ef117700 1 Processor -- start 2024-05-17T22:42:09.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.397+0000 7f15ef117700 1 -- start start 2024-05-17T22:42:09.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.397+0000 7f15ef117700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15e8103f10 0x7f15e81a04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:09.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.397+0000 7f15ef117700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15e8108840 0x7f15e81a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:09.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.397+0000 7f15ef117700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15e8109190 0x7f15e819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:09.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.397+0000 7f15ef117700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f15e8113de0 con 0x7f15e8109190 2024-05-17T22:42:09.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.397+0000 7f15ef117700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f15e8113c60 con 0x7f15e8103f10 2024-05-17T22:42:09.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.397+0000 7f15ef117700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f15e8113f60 con 0x7f15e8108840 2024-05-17T22:42:09.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.397+0000 7f15ed6b4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15e8109190 0x7f15e819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:09.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.397+0000 7f15ed6b4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15e8109190 0x7f15e819a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42308/0 (socket says 172.21.15.12:42308) 2024-05-17T22:42:09.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.397+0000 7f15ed6b4700 1 -- 172.21.15.12:0/4068724902 learned_addr learned my addr 172.21.15.12:0/4068724902 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:42:09.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.397+0000 7f15eceb3700 1 --2- 172.21.15.12:0/4068724902 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15e8103f10 0x7f15e81a04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:09.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.397+0000 7f15e7fff700 1 --2- 172.21.15.12:0/4068724902 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15e8108840 0x7f15e81a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:09.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.398+0000 7f15ed6b4700 1 -- 172.21.15.12:0/4068724902 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15e8108840 msgr2=0x7f15e81a0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:09.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.398+0000 7f15ed6b4700 1 --2- 172.21.15.12:0/4068724902 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15e8108840 0x7f15e81a0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:09.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.398+0000 7f15ed6b4700 1 -- 172.21.15.12:0/4068724902 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15e8103f10 msgr2=0x7f15e81a04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:09.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.398+0000 7f15ed6b4700 1 --2- 172.21.15.12:0/4068724902 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15e8103f10 0x7f15e81a04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:09.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.398+0000 7f15ed6b4700 1 -- 172.21.15.12:0/4068724902 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e819ada0 con 0x7f15e8109190 2024-05-17T22:42:09.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.398+0000 7f15eceb3700 1 --2- 172.21.15.12:0/4068724902 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15e8103f10 0x7f15e81a04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:42:09.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.398+0000 7f15ed6b4700 1 --2- 172.21.15.12:0/4068724902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15e8109190 0x7f15e819a670 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f15e0007ee0 tx=0x7f15e000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:42:09.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.398+0000 7f15e5ffb700 1 -- 172.21.15.12:0/4068724902 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e000dd20 con 0x7f15e8109190 2024-05-17T22:42:09.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.398+0000 7f15e5ffb700 1 -- 172.21.15.12:0/4068724902 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15e0004020 con 0x7f15e8109190 2024-05-17T22:42:09.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.398+0000 7f15e5ffb700 1 -- 172.21.15.12:0/4068724902 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e0015660 con 0x7f15e8109190 2024-05-17T22:42:09.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.398+0000 7f15ef117700 1 -- 172.21.15.12:0/4068724902 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15e819b090 con 0x7f15e8109190 2024-05-17T22:42:09.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.398+0000 7f15ef117700 1 -- 172.21.15.12:0/4068724902 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15e81a7290 con 0x7f15e8109190 2024-05-17T22:42:09.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.401+0000 7f15e5ffb700 1 -- 172.21.15.12:0/4068724902 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f15e00041d0 con 0x7f15e8109190 2024-05-17T22:42:09.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.402+0000 7f15e5ffb700 1 --2- 172.21.15.12:0/4068724902 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f15d80b2880 0x7f15d80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:09.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.402+0000 7f15eceb3700 1 --2- 172.21.15.12:0/4068724902 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f15d80b2880 0x7f15d80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:09.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.402+0000 7f15e5ffb700 1 -- 172.21.15.12:0/4068724902 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f15e00e1ef0 con 0x7f15e8109190 2024-05-17T22:42:09.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.402+0000 7f15fdaf2b80 1 Processor -- start 2024-05-17T22:42:09.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.402+0000 7f15fdaf2b80 1 -- start start 2024-05-17T22:42:09.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.402+0000 7f15eceb3700 1 --2- 172.21.15.12:0/4068724902 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f15d80b2880 0x7f15d80b4d90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f15dc00b2e0 tx=0x7f15dc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:42:09.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.402+0000 7f15fdaf2b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b715fe09f0 0x55b716164560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:09.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.402+0000 7f15fdaf2b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b716164aa0 0x55b71615ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:09.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.402+0000 7f15fdaf2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71615f170 0x55b71615f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:09.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.402+0000 7f15fdaf2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b71608d290 con 0x55b71615f170 2024-05-17T22:42:09.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.402+0000 7f15fdaf2b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b71607cfd0 con 0x55b715fe09f0 2024-05-17T22:42:09.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.402+0000 7f15fdaf2b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b716080e70 con 0x55b716164aa0 2024-05-17T22:42:09.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.403+0000 7f15ed6b4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71615f170 0x55b71615f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:09.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.403+0000 7f15ed6b4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71615f170 0x55b71615f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42312/0 (socket says 172.21.15.12:42312) 2024-05-17T22:42:09.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.403+0000 7f15ed6b4700 1 -- 172.21.15.12:0/1127074823 learned_addr learned my addr 172.21.15.12:0/1127074823 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:42:09.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.403+0000 7f15e7fff700 1 --2- 172.21.15.12:0/1127074823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b716164aa0 0x55b71615ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:09.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.403+0000 7f15eceb3700 1 --2- 172.21.15.12:0/1127074823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b715fe09f0 0x55b716164560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:09.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.403+0000 7f15ed6b4700 1 -- 172.21.15.12:0/1127074823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b716164aa0 msgr2=0x55b71615ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:42:09.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.403+0000 7f15ed6b4700 1 --2- 172.21.15.12:0/1127074823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b716164aa0 0x55b71615ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:09.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.403+0000 7f15ed6b4700 1 -- 172.21.15.12:0/1127074823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b715fe09f0 msgr2=0x55b716164560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:42:09.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.403+0000 7f15ed6b4700 1 --2- 172.21.15.12:0/1127074823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b715fe09f0 0x55b716164560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:09.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.403+0000 7f15ed6b4700 1 -- 172.21.15.12:0/1127074823 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b716026590 con 0x55b71615f170 2024-05-17T22:42:09.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.403+0000 7f15ed6b4700 1 --2- 172.21.15.12:0/1127074823 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71615f170 0x55b71615f580 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f15e00a5200 tx=0x7f15e00a8c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a54211c09433257 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:42:09.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.403+0000 7f15d2ffd700 1 -- 172.21.15.12:0/1127074823 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e00ad530 con 0x55b71615f170 2024-05-17T22:42:09.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.403+0000 7f15d2ffd700 1 -- 172.21.15.12:0/1127074823 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15e00b1fe0 con 0x55b71615f170 2024-05-17T22:42:09.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.403+0000 7f15d2ffd700 1 -- 172.21.15.12:0/1127074823 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e0004bd0 con 0x55b71615f170 2024-05-17T22:42:09.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.404+0000 7f15fdaf2b80 1 -- 172.21.15.12:0/1127074823 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71615f170 msgr2=0x55b71615f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:09.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.404+0000 7f15fdaf2b80 1 --2- 172.21.15.12:0/1127074823 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71615f170 0x55b71615f580 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f15e00a5200 tx=0x7f15e00a8c90 comp rx=0 tx=0).stop 2024-05-17T22:42:09.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.404+0000 7f15fdaf2b80 1 -- 172.21.15.12:0/1127074823 shutdown_connections 2024-05-17T22:42:09.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.404+0000 7f15fdaf2b80 1 --2- 172.21.15.12:0/1127074823 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71615f170 0x55b71615f580 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:09.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.404+0000 7f15fdaf2b80 1 --2- 172.21.15.12:0/1127074823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b715fe09f0 0x55b716164560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:09.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.404+0000 7f15fdaf2b80 1 --2- 172.21.15.12:0/1127074823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b716164aa0 0x55b71615ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:09.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.404+0000 7f15fdaf2b80 1 -- 172.21.15.12:0/1127074823 >> 172.21.15.12:0/1127074823 conn(0x55b716024f00 msgr2=0x55b716159b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:42:09.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.404+0000 7f15fdaf2b80 1 -- 172.21.15.12:0/1127074823 shutdown_connections 2024-05-17T22:42:09.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.404+0000 7f15fdaf2b80 1 -- 172.21.15.12:0/1127074823 wait complete. 2024-05-17T22:42:09.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.405+0000 7f15fdaf2b80 1 -- start start 2024-05-17T22:42:09.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.405+0000 7f15fdaf2b80 1 Processor -- start 2024-05-17T22:42:09.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.405+0000 7f15fdaf2b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b715fe09f0 0x55b716168220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:09.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.405+0000 7f15fdaf2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71615f170 0x55b716168760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:09.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.405+0000 7f15fdaf2b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b716164aa0 0x55b716168cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:09.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.405+0000 7f15fdaf2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b715f84150 con 0x55b71615f170 2024-05-17T22:42:09.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.405+0000 7f15fdaf2b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b71601de80 con 0x55b715fe09f0 2024-05-17T22:42:09.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.405+0000 7f15fdaf2b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b71616b390 con 0x55b716164aa0 2024-05-17T22:42:09.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.406+0000 7f15fdaf2b80 10 client.0 _ll_register_callbacks cb 0x55b716017f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:42:09.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.406+0000 7f15e7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71615f170 0x55b716168760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:09.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.406+0000 7f15eceb3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b715fe09f0 0x55b716168220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:09.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.406+0000 7f15e7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71615f170 0x55b716168760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42326/0 (socket says 172.21.15.12:42326) 2024-05-17T22:42:09.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.406+0000 7f15e7fff700 1 -- 172.21.15.12:0/2180167762 learned_addr learned my addr 172.21.15.12:0/2180167762 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:42:09.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.406+0000 7f15ed6b4700 1 --2- 172.21.15.12:0/2180167762 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b716164aa0 0x55b716168cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:09.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.406+0000 7f15e7fff700 1 -- 172.21.15.12:0/2180167762 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b716164aa0 msgr2=0x55b716168cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:42:09.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.406+0000 7f15e7fff700 1 --2- 172.21.15.12:0/2180167762 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b716164aa0 0x55b716168cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:09.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.406+0000 7f15e7fff700 1 -- 172.21.15.12:0/2180167762 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b715fe09f0 msgr2=0x55b716168220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:42:09.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.406+0000 7f15e7fff700 1 --2- 172.21.15.12:0/2180167762 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b715fe09f0 0x55b716168220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:09.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.406+0000 7f15e7fff700 1 -- 172.21.15.12:0/2180167762 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b715fcd480 con 0x55b71615f170 2024-05-17T22:42:09.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.406+0000 7f15eceb3700 1 --2- 172.21.15.12:0/2180167762 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b715fe09f0 0x55b716168220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:42:09.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.406+0000 7f15e7fff700 1 --2- 172.21.15.12:0/2180167762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71615f170 0x55b716168760 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f15d4012740 tx=0x7f15d4012a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc4ae7334226a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:42:09.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.406+0000 7f15c7fff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:42:09.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.406+0000 7f15c7fff700 1 -- 172.21.15.12:0/2180167762 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d4004030 con 0x55b71615f170 2024-05-17T22:42:09.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.406+0000 7f15c7fff700 1 -- 172.21.15.12:0/2180167762 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f15d40041d0 con 0x55b71615f170 2024-05-17T22:42:09.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.406+0000 7f15fdaf2b80 1 -- 172.21.15.12:0/2180167762 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15e00b1fe0 con 0x55b71615f170 2024-05-17T22:42:09.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.406+0000 7f15c7fff700 1 -- 172.21.15.12:0/2180167762 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f15d4025050 con 0x55b71615f170 2024-05-17T22:42:09.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.406+0000 7f15fdaf2b80 10 client.15609 fetch_fsmap learned FSMap version 30 2024-05-17T22:42:09.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.406+0000 7f15fdaf2b80 1 -- 172.21.15.12:0/2180167762 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b716175e90 con 0x55b71615f170 2024-05-17T22:42:09.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.407+0000 7f15c7fff700 1 -- 172.21.15.12:0/2180167762 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15e00b1fe0 con 0x55b71615f170 2024-05-17T22:42:09.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.407+0000 7f15c7fff700 1 -- 172.21.15.12:0/2180167762 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f15d40104a0 con 0x55b71615f170 2024-05-17T22:42:09.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.407+0000 7f15fdaf2b80 10 client.15609 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T22:42:09.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.407+0000 7f15fdaf2b80 10 client.15609 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-17T22:42:09.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.407+0000 7f15fdaf2b80 20 client.15609 populate_metadata read hostname 'smithi012' 2024-05-17T22:42:09.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.407+0000 7f15fdaf2b80 1 --2- 172.21.15.12:0/2180167762 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55b7161784d0 0x55b71617a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:09.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.407+0000 7f15fdaf2b80 4 client.15609 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:42:09.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.407+0000 7f15fdaf2b80 1 -- 172.21.15.12:0/2180167762 --> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b71608f1e0 con 0x55b7161784d0 2024-05-17T22:42:09.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.407+0000 7f15eceb3700 1 --2- 172.21.15.12:0/2180167762 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55b7161784d0 0x55b71617a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:09.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.408+0000 7f15eceb3700 1 --2- 172.21.15.12:0/2180167762 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55b7161784d0 0x55b71617a930 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=174c3674bbe5f65f server_cookie=c77ac99a7eabb344 in_seq=0 out_seq=0 2024-05-17T22:42:09.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.408+0000 7f15c7fff700 10 client.15609 ms_handle_connect on v2:172.21.15.151:6838/2338353849 2024-05-17T22:42:09.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.409+0000 7f15c7fff700 1 -- 172.21.15.12:0/2180167762 <== mds.2 v2:172.21.15.151:6838/2338353849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b71608f1e0 con 0x55b7161784d0 2024-05-17T22:42:09.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.409+0000 7f15c7fff700 10 client.15609 handle_command_reply: tid=0 2024-05-17T22:42:09.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.409+0000 7f15fdaf2b80 1 client.15609 shutdown 2024-05-17T22:42:09.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.409+0000 7f15fdaf2b80 1 -- 172.21.15.12:0/2180167762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71615f170 msgr2=0x55b716168760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:09.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.409+0000 7f15fdaf2b80 1 --2- 172.21.15.12:0/2180167762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71615f170 0x55b716168760 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f15d4012740 tx=0x7f15d4012a50 comp rx=0 tx=0).stop 2024-05-17T22:42:09.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.409+0000 7f15fdaf2b80 1 -- 172.21.15.12:0/2180167762 shutdown_connections 2024-05-17T22:42:09.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.409+0000 7f15fdaf2b80 1 --2- 172.21.15.12:0/2180167762 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55b7161784d0 0x55b71617a930 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:09.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.409+0000 7f15fdaf2b80 1 --2- 172.21.15.12:0/2180167762 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b716164aa0 0x55b716168cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:09.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.409+0000 7f15fdaf2b80 1 --2- 172.21.15.12:0/2180167762 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b715fe09f0 0x55b716168220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:09.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.409+0000 7f15c7fff700 0 client.15609 ms_handle_reset on v2:172.21.15.151:6838/2338353849 2024-05-17T22:42:09.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.409+0000 7f15fdaf2b80 1 --2- 172.21.15.12:0/2180167762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71615f170 0x55b716168760 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:09.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.409+0000 7f15fdaf2b80 1 -- 172.21.15.12:0/2180167762 >> 172.21.15.12:0/2180167762 conn(0x55b716024f00 msgr2=0x55b716153ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:42:09.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.409+0000 7f15fdaf2b80 1 -- 172.21.15.12:0/2180167762 shutdown_connections 2024-05-17T22:42:09.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.409+0000 7f15fdaf2b80 1 -- 172.21.15.12:0/2180167762 wait complete. 2024-05-17T22:42:09.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.409+0000 7f15fdaf2b80 20 client.15609 trim_cache size 0 max 16384 2024-05-17T22:42:09.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.426+0000 7f15fdaf2b80 1 Processor -- start 2024-05-17T22:42:09.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.427+0000 7f15fdaf2b80 1 -- start start 2024-05-17T22:42:09.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.427+0000 7f15fdaf2b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b715fe09f0 0x55b71615ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:09.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.427+0000 7f15fdaf2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71615f170 0x55b716174190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:09.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.427+0000 7f15fdaf2b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b716164aa0 0x55b716168af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:09.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.427+0000 7f15fdaf2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f15e81b2690 con 0x55b71615f170 2024-05-17T22:42:09.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.427+0000 7f15fdaf2b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f15e81afef0 con 0x55b716164aa0 2024-05-17T22:42:09.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.427+0000 7f15fdaf2b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f15e81add40 con 0x55b715fe09f0 2024-05-17T22:42:09.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.431+0000 7f15ed6b4700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b716164aa0 0x55b716168af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:09.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.431+0000 7f15ed6b4700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b716164aa0 0x55b716168af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:46192/0 (socket says 172.21.15.12:46192) 2024-05-17T22:42:09.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.431+0000 7f15ed6b4700 1 -- 172.21.15.12:0/2052988499 learned_addr learned my addr 172.21.15.12:0/2052988499 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:42:09.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.432+0000 7f15e7fff700 1 --2- 172.21.15.12:0/2052988499 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71615f170 0x55b716174190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:09.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.432+0000 7f15ed6b4700 1 -- 172.21.15.12:0/2052988499 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b715fe09f0 msgr2=0x55b71615ec30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T22:42:09.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.432+0000 7f15ed6b4700 1 --2- 172.21.15.12:0/2052988499 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b715fe09f0 0x55b71615ec30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:09.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.432+0000 7f15ed6b4700 1 -- 172.21.15.12:0/2052988499 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71615f170 msgr2=0x55b716174190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:42:09.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.432+0000 7f15ed6b4700 1 --2- 172.21.15.12:0/2052988499 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71615f170 0x55b716174190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:09.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.432+0000 7f15ed6b4700 1 -- 172.21.15.12:0/2052988499 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7161640d0 con 0x55b716164aa0 2024-05-17T22:42:09.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.432+0000 7f15e7fff700 1 --2- 172.21.15.12:0/2052988499 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71615f170 0x55b716174190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:42:09.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.432+0000 7f15ed6b4700 1 --2- 172.21.15.12:0/2052988499 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b716164aa0 0x55b716168af0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f15e00a8e80 tx=0x7f15e00a8f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6859c5781a3b663 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:42:09.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.433+0000 7f15d27fc700 1 -- 172.21.15.12:0/2052988499 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e00ad530 con 0x55b716164aa0 2024-05-17T22:42:09.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.433+0000 7f15d27fc700 1 -- 172.21.15.12:0/2052988499 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15e006d070 con 0x55b716164aa0 2024-05-17T22:42:09.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.433+0000 7f15d27fc700 1 -- 172.21.15.12:0/2052988499 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e00a9640 con 0x55b716164aa0 2024-05-17T22:42:09.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.435+0000 7f15fdaf2b80 1 -- 172.21.15.12:0/2052988499 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b716164aa0 msgr2=0x55b716168af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:09.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.435+0000 7f15fdaf2b80 1 --2- 172.21.15.12:0/2052988499 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b716164aa0 0x55b716168af0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f15e00a8e80 tx=0x7f15e00a8f50 comp rx=0 tx=0).stop 2024-05-17T22:42:09.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.441+0000 7f15fdaf2b80 1 -- 172.21.15.12:0/2052988499 shutdown_connections 2024-05-17T22:42:09.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.441+0000 7f15fdaf2b80 1 --2- 172.21.15.12:0/2052988499 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b716164aa0 0x55b716168af0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:09.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.441+0000 7f15fdaf2b80 1 --2- 172.21.15.12:0/2052988499 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b715fe09f0 0x55b71615ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:09.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.441+0000 7f15fdaf2b80 1 --2- 172.21.15.12:0/2052988499 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71615f170 0x55b716174190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:09.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.441+0000 7f15fdaf2b80 1 -- 172.21.15.12:0/2052988499 >> 172.21.15.12:0/2052988499 conn(0x55b716024f00 msgr2=0x55b716161de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:42:09.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.446+0000 7f15fdaf2b80 1 -- 172.21.15.12:0/2052988499 shutdown_connections 2024-05-17T22:42:09.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.446+0000 7f15fdaf2b80 1 -- 172.21.15.12:0/2052988499 wait complete. 2024-05-17T22:42:09.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.447+0000 7f15fdaf2b80 1 -- start start 2024-05-17T22:42:09.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.447+0000 7f15fdaf2b80 1 Processor -- start 2024-05-17T22:42:09.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.451+0000 7f15fdaf2b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b715fe09f0 0x55b716170510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:09.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.451+0000 7f15fdaf2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71615f170 0x55b716170a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:09.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.451+0000 7f15fdaf2b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b716164aa0 0x55b71617f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:09.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.451+0000 7f15fdaf2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f15e81ae3b0 con 0x55b71615f170 2024-05-17T22:42:09.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.451+0000 7f15fdaf2b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f15e81af820 con 0x55b716164aa0 2024-05-17T22:42:09.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.451+0000 7f15fdaf2b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f15e81aea80 con 0x55b715fe09f0 2024-05-17T22:42:09.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.451+0000 7f15fdaf2b80 10 client.0 _ll_register_callbacks cb 0x55b716017f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:42:09.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.452+0000 7f15e7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71615f170 0x55b716170a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:09.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.452+0000 7f15e7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71615f170 0x55b716170a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42344/0 (socket says 172.21.15.12:42344) 2024-05-17T22:42:09.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.452+0000 7f15e7fff700 1 -- 172.21.15.12:0/1489364590 learned_addr learned my addr 172.21.15.12:0/1489364590 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:42:09.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.453+0000 7f15ed6b4700 1 --2- 172.21.15.12:0/1489364590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b716164aa0 0x55b71617f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:09.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.453+0000 7f15eceb3700 1 --2- 172.21.15.12:0/1489364590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b715fe09f0 0x55b716170510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:09.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.453+0000 7f15eceb3700 1 -- 172.21.15.12:0/1489364590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b716164aa0 msgr2=0x55b71617f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:42:09.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.453+0000 7f15eceb3700 1 --2- 172.21.15.12:0/1489364590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b716164aa0 0x55b71617f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:09.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.453+0000 7f15eceb3700 1 -- 172.21.15.12:0/1489364590 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71615f170 msgr2=0x55b716170a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:42:09.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.453+0000 7f15eceb3700 1 --2- 172.21.15.12:0/1489364590 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71615f170 0x55b716170a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:09.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.453+0000 7f15eceb3700 1 -- 172.21.15.12:0/1489364590 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b716169030 con 0x55b715fe09f0 2024-05-17T22:42:09.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.454+0000 7f15eceb3700 1 --2- 172.21.15.12:0/1489364590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b715fe09f0 0x55b716170510 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f15dc0059d0 tx=0x7f15dc009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b748ed674c95ae8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:42:09.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.454+0000 7f15c7fff700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:42:09.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.454+0000 7f15c7fff700 1 -- 172.21.15.12:0/1489364590 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc035630 con 0x55b715fe09f0 2024-05-17T22:42:09.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.454+0000 7f15fdaf2b80 1 -- 172.21.15.12:0/1489364590 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15e006d070 con 0x55b715fe09f0 2024-05-17T22:42:09.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.455+0000 7f15c7fff700 1 -- 172.21.15.12:0/1489364590 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f15dc0357d0 con 0x55b715fe09f0 2024-05-17T22:42:09.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.455+0000 7f15c7fff700 1 -- 172.21.15.12:0/1489364590 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f15dc043050 con 0x55b715fe09f0 2024-05-17T22:42:09.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.455+0000 7f15fdaf2b80 10 client.25231 fetch_fsmap learned FSMap version 30 2024-05-17T22:42:09.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.455+0000 7f15fdaf2b80 1 -- 172.21.15.12:0/1489364590 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b7161e1ff0 con 0x55b715fe09f0 2024-05-17T22:42:09.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.456+0000 7f15c7fff700 1 -- 172.21.15.12:0/1489364590 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15e006d070 con 0x55b715fe09f0 2024-05-17T22:42:09.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.456+0000 7f15c7fff700 1 -- 172.21.15.12:0/1489364590 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f15dc022070 con 0x55b715fe09f0 2024-05-17T22:42:09.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.456+0000 7f15fdaf2b80 10 client.25231 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T22:42:09.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.456+0000 7f15fdaf2b80 10 client.25231 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-17T22:42:09.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.456+0000 7f15fdaf2b80 20 client.25231 populate_metadata read hostname 'smithi012' 2024-05-17T22:42:09.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.456+0000 7f15fdaf2b80 1 --2- 172.21.15.12:0/1489364590 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55b7161e26f0 0x55b7161e2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:09.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.456+0000 7f15fdaf2b80 4 client.25231 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:42:09.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.456+0000 7f15fdaf2b80 1 -- 172.21.15.12:0/1489364590 --> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b7160889c0 con 0x55b7161e26f0 2024-05-17T22:42:09.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.457+0000 7f15e7fff700 1 --2- 172.21.15.12:0/1489364590 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55b7161e26f0 0x55b7161e2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:09.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.458+0000 7f15e7fff700 1 --2- 172.21.15.12:0/1489364590 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55b7161e26f0 0x55b7161e2b00 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=464236cfb1315525 server_cookie=32499d98b4ed4e76 in_seq=0 out_seq=0 2024-05-17T22:42:09.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.458+0000 7f15c7fff700 10 client.25231 ms_handle_connect on v2:172.21.15.151:6838/2338353849 2024-05-17T22:42:09.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.458+0000 7f15c7fff700 1 -- 172.21.15.12:0/1489364590 <== mds.2 v2:172.21.15.151:6838/2338353849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b7160889c0 con 0x55b7161e26f0 2024-05-17T22:42:09.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.458+0000 7f15c7fff700 10 client.25231 handle_command_reply: tid=0 2024-05-17T22:42:09.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.458+0000 7f15fdaf2b80 1 client.25231 shutdown 2024-05-17T22:42:09.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.461+0000 7f15fdaf2b80 1 -- 172.21.15.12:0/1489364590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b715fe09f0 msgr2=0x55b716170510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:09.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.461+0000 7f15fdaf2b80 1 --2- 172.21.15.12:0/1489364590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b715fe09f0 0x55b716170510 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f15dc0059d0 tx=0x7f15dc009ca0 comp rx=0 tx=0).stop 2024-05-17T22:42:09.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.466+0000 7f15fdaf2b80 1 -- 172.21.15.12:0/1489364590 shutdown_connections 2024-05-17T22:42:09.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.466+0000 7f15fdaf2b80 1 --2- 172.21.15.12:0/1489364590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b716164aa0 0x55b71617f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:09.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.466+0000 7f15fdaf2b80 1 --2- 172.21.15.12:0/1489364590 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55b7161e26f0 0x55b7161e2b00 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:09.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.466+0000 7f15fdaf2b80 1 --2- 172.21.15.12:0/1489364590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b715fe09f0 0x55b716170510 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:09.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.466+0000 7f15fdaf2b80 1 --2- 172.21.15.12:0/1489364590 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b71615f170 0x55b716170a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:09.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.466+0000 7f15fdaf2b80 1 -- 172.21.15.12:0/1489364590 >> 172.21.15.12:0/1489364590 conn(0x55b716024f00 msgr2=0x55b716163990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:42:09.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.466+0000 7f15c7fff700 0 client.25231 ms_handle_reset on v2:172.21.15.151:6838/2338353849 2024-05-17T22:42:09.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.466+0000 7f15fdaf2b80 1 -- 172.21.15.12:0/1489364590 shutdown_connections 2024-05-17T22:42:09.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.467+0000 7f15fdaf2b80 1 -- 172.21.15.12:0/1489364590 wait complete. 2024-05-17T22:42:09.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.467+0000 7f15fdaf2b80 20 client.25231 trim_cache size 0 max 16384 2024-05-17T22:42:09.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.474+0000 7f15ef117700 1 -- 172.21.15.12:0/4068724902 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f15d80b2880 msgr2=0x7f15d80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:09.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.474+0000 7f15ef117700 1 --2- 172.21.15.12:0/4068724902 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f15d80b2880 0x7f15d80b4d90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f15dc00b2e0 tx=0x7f15dc005960 comp rx=0 tx=0).stop 2024-05-17T22:42:09.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.474+0000 7f15ef117700 1 -- 172.21.15.12:0/4068724902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15e8109190 msgr2=0x7f15e819a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:09.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.474+0000 7f15ef117700 1 --2- 172.21.15.12:0/4068724902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15e8109190 0x7f15e819a670 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f15e0007ee0 tx=0x7f15e000d3b0 comp rx=0 tx=0).stop 2024-05-17T22:42:09.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.475+0000 7f15ef117700 1 -- 172.21.15.12:0/4068724902 shutdown_connections 2024-05-17T22:42:09.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.475+0000 7f15ef117700 1 --2- 172.21.15.12:0/4068724902 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f15d80b2880 0x7f15d80b4d90 secure :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f15dc00b2e0 tx=0x7f15dc005960 comp rx=0 tx=0).stop 2024-05-17T22:42:09.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.475+0000 7f15ef117700 1 --2- 172.21.15.12:0/4068724902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15e8109190 0x7f15e819a670 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:09.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.475+0000 7f15ef117700 1 --2- 172.21.15.12:0/4068724902 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15e8103f10 0x7f15e81a04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:09.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.475+0000 7f15ef117700 1 --2- 172.21.15.12:0/4068724902 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15e8108840 0x7f15e81a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:09.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.475+0000 7f15ef117700 1 -- 172.21.15.12:0/4068724902 >> 172.21.15.12:0/4068724902 conn(0x7f15e80ff9b0 msgr2=0x7f15e8101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:42:09.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.477+0000 7f15ef117700 1 -- 172.21.15.12:0/4068724902 shutdown_connections 2024-05-17T22:42:09.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:08.477+0000 7f15ef117700 1 -- 172.21.15.12:0/4068724902 wait complete. 2024-05-17T22:42:10.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:09 smithi012 ceph-mon[31632]: pgmap v526: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 16 MiB/s wr, 1.08k op/s 2024-05-17T22:42:10.174 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-17T22:42:10.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:09 smithi151 ceph-mon[36101]: pgmap v526: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 16 MiB/s wr, 1.08k op/s 2024-05-17T22:42:10.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:09 smithi096 ceph-mon[36065]: pgmap v526: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 16 MiB/s wr, 1.08k op/s 2024-05-17T22:42:10.590 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:42:11.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 170.0 s, 1368.7 tps, lat 72.385 ms stddev 81.825 2024-05-17T22:42:12.079 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:11 smithi012 ceph-mon[31632]: pgmap v527: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 16 MiB/s wr, 1.03k op/s 2024-05-17T22:42:12.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:11 smithi151 ceph-mon[36101]: pgmap v527: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 16 MiB/s wr, 1.03k op/s 2024-05-17T22:42:12.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:11 smithi096 ceph-mon[36065]: pgmap v527: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 16 MiB/s wr, 1.03k op/s 2024-05-17T22:42:12.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.853+0000 7f3d11f45700 1 -- 172.21.15.12:0/507393884 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d0c11d550 msgr2=0x7f3d0c11f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:12.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.853+0000 7f3d11f45700 1 --2- 172.21.15.12:0/507393884 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d0c11d550 0x7f3d0c11f970 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3d0400b0a0 tx=0x7f3d0400b3b0 comp rx=0 tx=0).stop 2024-05-17T22:42:12.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.857+0000 7f3d11f45700 1 -- 172.21.15.12:0/507393884 shutdown_connections 2024-05-17T22:42:12.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.857+0000 7f3d11f45700 1 --2- 172.21.15.12:0/507393884 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d0c11d550 0x7f3d0c11f970 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:12.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.857+0000 7f3d11f45700 1 --2- 172.21.15.12:0/507393884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d0c10a830 0x7f3d0c10ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:12.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.857+0000 7f3d11f45700 1 --2- 172.21.15.12:0/507393884 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d0c10b180 0x7f3d0c11d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:12.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.857+0000 7f3d11f45700 1 -- 172.21.15.12:0/507393884 >> 172.21.15.12:0/507393884 conn(0x7f3d0c06efc0 msgr2=0x7f3d0c06f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:42:12.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.859+0000 7f3d11f45700 1 -- 172.21.15.12:0/507393884 shutdown_connections 2024-05-17T22:42:12.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.861+0000 7f3d11f45700 1 -- 172.21.15.12:0/507393884 wait complete. 2024-05-17T22:42:12.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.862+0000 7f3d11f45700 1 Processor -- start 2024-05-17T22:42:12.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.863+0000 7f3d11f45700 1 -- start start 2024-05-17T22:42:12.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.863+0000 7f3d11f45700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d0c10a830 0x7f3d0c117170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:12.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.863+0000 7f3d11f45700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d0c10b180 0x7f3d0c1176b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:12.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.863+0000 7f3d11f45700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d0c11d550 0x7f3d0c117bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:12.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.863+0000 7f3d11f45700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c1130f0 con 0x7f3d0c10a830 2024-05-17T22:42:12.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.863+0000 7f3d11f45700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c112f70 con 0x7f3d0c11d550 2024-05-17T22:42:12.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.863+0000 7f3d11f45700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c113270 con 0x7f3d0c10b180 2024-05-17T22:42:12.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.863+0000 7f3d0bfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d0c11d550 0x7f3d0c117bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:12.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.863+0000 7f3d0bfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d0c11d550 0x7f3d0c117bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:52536/0 (socket says 172.21.15.12:52536) 2024-05-17T22:42:12.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.863+0000 7f3d0bfff700 1 -- 172.21.15.12:0/2284075449 learned_addr learned my addr 172.21.15.12:0/2284075449 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:42:12.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.866+0000 7f3d0affd700 1 --2- 172.21.15.12:0/2284075449 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d0c10b180 0x7f3d0c1176b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:12.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.866+0000 7f3d0bfff700 1 -- 172.21.15.12:0/2284075449 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d0c10b180 msgr2=0x7f3d0c1176b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:12.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.866+0000 7f3d0bfff700 1 --2- 172.21.15.12:0/2284075449 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d0c10b180 0x7f3d0c1176b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:12.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.866+0000 7f3d0bfff700 1 -- 172.21.15.12:0/2284075449 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d0c10a830 msgr2=0x7f3d0c117170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:12.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.866+0000 7f3d0bfff700 1 --2- 172.21.15.12:0/2284075449 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d0c10a830 0x7f3d0c117170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:12.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.866+0000 7f3d0bfff700 1 -- 172.21.15.12:0/2284075449 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d0c1be1d0 con 0x7f3d0c11d550 2024-05-17T22:42:12.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.868+0000 7f3d0bfff700 1 --2- 172.21.15.12:0/2284075449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d0c11d550 0x7f3d0c117bf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3d04009fd0 tx=0x7f3d0400cfc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:42:12.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.868+0000 7f3d08ff9700 1 -- 172.21.15.12:0/2284075449 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d04021070 con 0x7f3d0c11d550 2024-05-17T22:42:12.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.868+0000 7f3d08ff9700 1 -- 172.21.15.12:0/2284075449 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d04007cb0 con 0x7f3d0c11d550 2024-05-17T22:42:12.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.868+0000 7f3d08ff9700 1 -- 172.21.15.12:0/2284075449 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0401d450 con 0x7f3d0c11d550 2024-05-17T22:42:12.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.868+0000 7f3d11f45700 1 -- 172.21.15.12:0/2284075449 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d0c1be400 con 0x7f3d0c11d550 2024-05-17T22:42:12.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.868+0000 7f3d11f45700 1 -- 172.21.15.12:0/2284075449 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d0c1be8c0 con 0x7f3d0c11d550 2024-05-17T22:42:12.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.876+0000 7f3d08ff9700 1 -- 172.21.15.12:0/2284075449 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3d0401d5f0 con 0x7f3d0c11d550 2024-05-17T22:42:12.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.876+0000 7f3d20920b80 1 Processor -- start 2024-05-17T22:42:12.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.876+0000 7f3d20920b80 1 -- start start 2024-05-17T22:42:12.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.877+0000 7f3d20920b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653ec3359f0 0x5653ec4b9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:12.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.877+0000 7f3d20920b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653ec4b9aa0 0x5653ec4b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:12.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.877+0000 7f3d20920b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653ec4b4170 0x5653ec4b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:12.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.877+0000 7f3d20920b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5653ec3e2290 con 0x5653ec4b9aa0 2024-05-17T22:42:12.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.877+0000 7f3d20920b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5653ec3d1fd0 con 0x5653ec4b4170 2024-05-17T22:42:12.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.877+0000 7f3d20920b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5653ec3d5e70 con 0x5653ec3359f0 2024-05-17T22:42:12.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.878+0000 7f3d08ff9700 1 --2- 172.21.15.12:0/2284075449 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f3cf40b2a90 0x7f3cf40b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:12.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.878+0000 7f3d0affd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653ec4b4170 0x5653ec4b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:12.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.878+0000 7f3d0affd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653ec4b4170 0x5653ec4b4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:52550/0 (socket says 172.21.15.12:52550) 2024-05-17T22:42:12.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.878+0000 7f3d0affd700 1 -- 172.21.15.12:0/642012166 learned_addr learned my addr 172.21.15.12:0/642012166 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:42:12.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.878+0000 7f3d0affd700 1 --2- 172.21.15.12:0/642012166 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653ec3359f0 0x5653ec4b9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:12.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.878+0000 7f3d0affd700 1 -- 172.21.15.12:0/642012166 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653ec3359f0 msgr2=0x5653ec4b9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:42:12.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.878+0000 7f3d0affd700 1 --2- 172.21.15.12:0/642012166 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653ec3359f0 0x5653ec4b9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:12.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.878+0000 7f3d0affd700 1 -- 172.21.15.12:0/642012166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653ec4b9aa0 msgr2=0x5653ec4b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:42:12.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.878+0000 7f3d0affd700 1 --2- 172.21.15.12:0/642012166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653ec4b9aa0 0x5653ec4b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:12.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.879+0000 7f3d0affd700 1 -- 172.21.15.12:0/642012166 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653ec37b590 con 0x5653ec4b4170 2024-05-17T22:42:12.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.879+0000 7f3d0affd700 1 --2- 172.21.15.12:0/642012166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653ec4b4170 0x5653ec4b4580 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3d00016560 tx=0x7f3d0000a330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83e0811900f76f0b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:42:12.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.879+0000 7f3cf1ffb700 1 -- 172.21.15.12:0/642012166 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d00016b80 con 0x5653ec4b4170 2024-05-17T22:42:12.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.879+0000 7f3cf1ffb700 1 -- 172.21.15.12:0/642012166 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d00016d20 con 0x5653ec4b4170 2024-05-17T22:42:12.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.879+0000 7f3cf1ffb700 1 -- 172.21.15.12:0/642012166 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d00017640 con 0x5653ec4b4170 2024-05-17T22:42:12.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.880+0000 7f3d20920b80 1 -- 172.21.15.12:0/642012166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653ec4b4170 msgr2=0x5653ec4b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:12.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.880+0000 7f3d20920b80 1 --2- 172.21.15.12:0/642012166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653ec4b4170 0x5653ec4b4580 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3d00016560 tx=0x7f3d0000a330 comp rx=0 tx=0).stop 2024-05-17T22:42:12.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.881+0000 7f3d08ff9700 1 -- 172.21.15.12:0/2284075449 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f3d040e3640 con 0x7f3d0c11d550 2024-05-17T22:42:12.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.881+0000 7f3d0bfff700 1 --2- 172.21.15.12:0/2284075449 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f3cf40b2a90 0x7f3cf40b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:12.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.882+0000 7f3d20920b80 1 -- 172.21.15.12:0/642012166 shutdown_connections 2024-05-17T22:42:12.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.882+0000 7f3d20920b80 1 --2- 172.21.15.12:0/642012166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653ec4b4170 0x5653ec4b4580 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:12.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.882+0000 7f3d20920b80 1 --2- 172.21.15.12:0/642012166 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653ec3359f0 0x5653ec4b9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:12.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.882+0000 7f3d20920b80 1 --2- 172.21.15.12:0/642012166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653ec4b9aa0 0x5653ec4b3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:12.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.882+0000 7f3d20920b80 1 -- 172.21.15.12:0/642012166 >> 172.21.15.12:0/642012166 conn(0x5653ec379f00 msgr2=0x5653ec4aeb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:42:12.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.882+0000 7f3d0bfff700 1 --2- 172.21.15.12:0/2284075449 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f3cf40b2a90 0x7f3cf40b4fa0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3d040a79a0 tx=0x7f3d040a7040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:42:12.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.882+0000 7f3d20920b80 1 -- 172.21.15.12:0/642012166 shutdown_connections 2024-05-17T22:42:12.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.882+0000 7f3d20920b80 1 -- 172.21.15.12:0/642012166 wait complete. 2024-05-17T22:42:12.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.883+0000 7f3d20920b80 1 -- start start 2024-05-17T22:42:12.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.883+0000 7f3d20920b80 1 Processor -- start 2024-05-17T22:42:12.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.884+0000 7f3d20920b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653ec3359f0 0x5653ec4bd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:12.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.884+0000 7f3d20920b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653ec4b4170 0x5653ec4bd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:12.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.884+0000 7f3d20920b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653ec4b9aa0 0x5653ec4bdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:12.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.884+0000 7f3d20920b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5653ec2d9150 con 0x5653ec4b9aa0 2024-05-17T22:42:12.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.884+0000 7f3d20920b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5653ec372e80 con 0x5653ec3359f0 2024-05-17T22:42:12.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.884+0000 7f3d20920b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5653ec4c0390 con 0x5653ec4b4170 2024-05-17T22:42:12.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.884+0000 7f3d20920b80 10 client.0 _ll_register_callbacks cb 0x5653ec36cf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:42:12.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.885+0000 7f3d0affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653ec4b9aa0 0x5653ec4bdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:12.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.885+0000 7f3d0affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653ec4b9aa0 0x5653ec4bdcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45634/0 (socket says 172.21.15.12:45634) 2024-05-17T22:42:12.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.885+0000 7f3d0affd700 1 -- 172.21.15.12:0/69472206 learned_addr learned my addr 172.21.15.12:0/69472206 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:42:12.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.885+0000 7f3d0affd700 1 --2- 172.21.15.12:0/69472206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653ec3359f0 0x5653ec4bd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:12.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.885+0000 7f3d0affd700 1 -- 172.21.15.12:0/69472206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653ec4b4170 msgr2=0x5653ec4bd760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T22:42:12.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.886+0000 7f3d0affd700 1 --2- 172.21.15.12:0/69472206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653ec4b4170 0x5653ec4bd760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:12.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.886+0000 7f3d0affd700 1 -- 172.21.15.12:0/69472206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653ec3359f0 msgr2=0x5653ec4bd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:42:12.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.886+0000 7f3d0affd700 1 --2- 172.21.15.12:0/69472206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653ec3359f0 0x5653ec4bd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:12.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.886+0000 7f3d0affd700 1 -- 172.21.15.12:0/69472206 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653ec322480 con 0x5653ec4b9aa0 2024-05-17T22:42:12.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.886+0000 7f3d0affd700 1 --2- 172.21.15.12:0/69472206 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653ec4b9aa0 0x5653ec4bdcc0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x5653ec4bcab0 tx=0x7f3d0000a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f618e99462be50ff server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:42:12.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.886+0000 7f3ceaffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:42:12.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.886+0000 7f3ceaffd700 1 -- 172.21.15.12:0/69472206 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d00026440 con 0x5653ec4b9aa0 2024-05-17T22:42:12.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.886+0000 7f3ceaffd700 1 -- 172.21.15.12:0/69472206 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f3d000265e0 con 0x5653ec4b9aa0 2024-05-17T22:42:12.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.886+0000 7f3d20920b80 1 -- 172.21.15.12:0/69472206 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d00016d20 con 0x5653ec4b9aa0 2024-05-17T22:42:12.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.887+0000 7f3ceaffd700 1 -- 172.21.15.12:0/69472206 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3d00034050 con 0x5653ec4b9aa0 2024-05-17T22:42:12.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.887+0000 7f3d20920b80 10 client.15618 fetch_fsmap learned FSMap version 30 2024-05-17T22:42:12.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.887+0000 7f3d20920b80 1 -- 172.21.15.12:0/69472206 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5653ec4cae90 con 0x5653ec4b9aa0 2024-05-17T22:42:12.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.887+0000 7f3ceaffd700 1 -- 172.21.15.12:0/69472206 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d00016d20 con 0x5653ec4b9aa0 2024-05-17T22:42:12.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.887+0000 7f3ceaffd700 1 -- 172.21.15.12:0/69472206 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f3d00012050 con 0x5653ec4b9aa0 2024-05-17T22:42:12.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.887+0000 7f3d20920b80 10 client.15618 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T22:42:12.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.887+0000 7f3d20920b80 10 client.15618 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-17T22:42:12.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.887+0000 7f3d20920b80 20 client.15618 populate_metadata read hostname 'smithi012' 2024-05-17T22:42:12.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.887+0000 7f3d20920b80 1 --2- 172.21.15.12:0/69472206 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5653ec4cd360 0x5653ec4cf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:12.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.887+0000 7f3d20920b80 4 client.15618 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:42:12.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.887+0000 7f3d20920b80 1 -- 172.21.15.12:0/69472206 --> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653ec3e41e0 con 0x5653ec4cd360 2024-05-17T22:42:12.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.888+0000 7f3d0b7fe700 1 --2- 172.21.15.12:0/69472206 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5653ec4cd360 0x5653ec4cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:12.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.888+0000 7f3ceaffd700 10 client.15618 ms_handle_connect on v2:172.21.15.96:6836/4222574338 2024-05-17T22:42:12.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.888+0000 7f3d0b7fe700 1 --2- 172.21.15.12:0/69472206 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5653ec4cd360 0x5653ec4cf780 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=99b0a03acf0eeb3f server_cookie=d852f5d77bcbc52d in_seq=0 out_seq=0 2024-05-17T22:42:12.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.889+0000 7f3ceaffd700 1 -- 172.21.15.12:0/69472206 <== mds.1 v2:172.21.15.96:6836/4222574338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5653ec3e41e0 con 0x5653ec4cd360 2024-05-17T22:42:12.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.889+0000 7f3ceaffd700 10 client.15618 handle_command_reply: tid=0 2024-05-17T22:42:12.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.889+0000 7f3d20920b80 1 client.15618 shutdown 2024-05-17T22:42:12.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.889+0000 7f3d20920b80 1 -- 172.21.15.12:0/69472206 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653ec4b9aa0 msgr2=0x5653ec4bdcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:12.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.889+0000 7f3d20920b80 1 --2- 172.21.15.12:0/69472206 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653ec4b9aa0 0x5653ec4bdcc0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x5653ec4bcab0 tx=0x7f3d0000a290 comp rx=0 tx=0).stop 2024-05-17T22:42:12.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.889+0000 7f3d20920b80 1 -- 172.21.15.12:0/69472206 shutdown_connections 2024-05-17T22:42:12.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.890+0000 7f3d20920b80 1 --2- 172.21.15.12:0/69472206 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5653ec4cd360 0x5653ec4cf780 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:12.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.890+0000 7f3ceaffd700 0 client.15618 ms_handle_reset on v2:172.21.15.96:6836/4222574338 2024-05-17T22:42:12.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.890+0000 7f3d20920b80 1 --2- 172.21.15.12:0/69472206 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653ec4b9aa0 0x5653ec4bdcc0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:12.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.890+0000 7f3d20920b80 1 --2- 172.21.15.12:0/69472206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653ec3359f0 0x5653ec4bd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:12.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.890+0000 7f3d20920b80 1 --2- 172.21.15.12:0/69472206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653ec4b4170 0x5653ec4bd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:12.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.890+0000 7f3d20920b80 1 -- 172.21.15.12:0/69472206 >> 172.21.15.12:0/69472206 conn(0x5653ec379f00 msgr2=0x5653ec4a8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:42:12.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.890+0000 7f3d20920b80 1 -- 172.21.15.12:0/69472206 shutdown_connections 2024-05-17T22:42:12.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.890+0000 7f3d20920b80 1 -- 172.21.15.12:0/69472206 wait complete. 2024-05-17T22:42:12.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.890+0000 7f3d20920b80 20 client.15618 trim_cache size 0 max 16384 2024-05-17T22:42:12.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.913+0000 7f3d20920b80 1 Processor -- start 2024-05-17T22:42:12.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.913+0000 7f3d20920b80 1 -- start start 2024-05-17T22:42:12.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.913+0000 7f3d20920b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653ec3359f0 0x5653ec4b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:12.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.913+0000 7f3d20920b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653ec4b4170 0x5653ec4c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:12.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.913+0000 7f3d20920b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653ec4b9aa0 0x5653ec4bdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:12.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.914+0000 7f3d20920b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf80113b0 con 0x5653ec3359f0 2024-05-17T22:42:12.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.914+0000 7f3d20920b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf800ec10 con 0x5653ec4b4170 2024-05-17T22:42:12.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.914+0000 7f3d20920b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf800ca60 con 0x5653ec4b9aa0 2024-05-17T22:42:12.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.914+0000 7f3d0affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653ec3359f0 0x5653ec4b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:12.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.914+0000 7f3d0affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653ec3359f0 0x5653ec4b3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45648/0 (socket says 172.21.15.12:45648) 2024-05-17T22:42:12.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.914+0000 7f3d0affd700 1 -- 172.21.15.12:0/2238056995 learned_addr learned my addr 172.21.15.12:0/2238056995 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:42:12.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.914+0000 7f3d0affd700 1 --2- 172.21.15.12:0/2238056995 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653ec4b9aa0 0x5653ec4bdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:12.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.914+0000 7f3d0b7fe700 1 --2- 172.21.15.12:0/2238056995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653ec4b4170 0x5653ec4c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:12.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.914+0000 7f3d0affd700 1 -- 172.21.15.12:0/2238056995 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653ec4b9aa0 msgr2=0x5653ec4bdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:42:12.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.914+0000 7f3d0affd700 1 --2- 172.21.15.12:0/2238056995 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653ec4b9aa0 0x5653ec4bdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:12.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.914+0000 7f3d0affd700 1 -- 172.21.15.12:0/2238056995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653ec4b4170 msgr2=0x5653ec4c91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:42:12.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.914+0000 7f3d0affd700 1 --2- 172.21.15.12:0/2238056995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653ec4b4170 0x5653ec4c91b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:12.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.914+0000 7f3d0affd700 1 -- 172.21.15.12:0/2238056995 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653ec4b0710 con 0x5653ec3359f0 2024-05-17T22:42:12.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.914+0000 7f3d0affd700 1 --2- 172.21.15.12:0/2238056995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653ec3359f0 0x5653ec4b3c30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3d00017ff0 tx=0x7f3d00018710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e24531d4d3ac2c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:42:12.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.915+0000 7f3cf17fa700 1 -- 172.21.15.12:0/2238056995 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0003c070 con 0x5653ec3359f0 2024-05-17T22:42:12.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.915+0000 7f3cf17fa700 1 -- 172.21.15.12:0/2238056995 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d0002b070 con 0x5653ec3359f0 2024-05-17T22:42:12.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.915+0000 7f3cf17fa700 1 -- 172.21.15.12:0/2238056995 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d00025e90 con 0x5653ec3359f0 2024-05-17T22:42:12.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.916+0000 7f3d20920b80 1 -- 172.21.15.12:0/2238056995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653ec3359f0 msgr2=0x5653ec4b3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:12.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.916+0000 7f3d20920b80 1 --2- 172.21.15.12:0/2238056995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653ec3359f0 0x5653ec4b3c30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3d00017ff0 tx=0x7f3d00018710 comp rx=0 tx=0).stop 2024-05-17T22:42:12.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.916+0000 7f3d20920b80 1 -- 172.21.15.12:0/2238056995 shutdown_connections 2024-05-17T22:42:12.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.916+0000 7f3d20920b80 1 --2- 172.21.15.12:0/2238056995 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653ec4b9aa0 0x5653ec4bdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:12.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.916+0000 7f3d20920b80 1 --2- 172.21.15.12:0/2238056995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653ec3359f0 0x5653ec4b3c30 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:12.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.916+0000 7f3d20920b80 1 --2- 172.21.15.12:0/2238056995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653ec4b4170 0x5653ec4c91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:12.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.916+0000 7f3d20920b80 1 -- 172.21.15.12:0/2238056995 >> 172.21.15.12:0/2238056995 conn(0x5653ec379f00 msgr2=0x5653ec4b6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:42:12.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.916+0000 7f3d20920b80 1 -- 172.21.15.12:0/2238056995 shutdown_connections 2024-05-17T22:42:12.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.916+0000 7f3d20920b80 1 -- 172.21.15.12:0/2238056995 wait complete. 2024-05-17T22:42:12.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.917+0000 7f3d20920b80 1 -- start start 2024-05-17T22:42:12.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.917+0000 7f3d20920b80 1 Processor -- start 2024-05-17T22:42:12.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.918+0000 7f3d20920b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653ec3359f0 0x5653ec4c5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:12.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.918+0000 7f3d20920b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653ec4b4170 0x5653ec4c5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:12.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.918+0000 7f3d20920b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653ec4b9aa0 0x5653ec4d3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:12.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.918+0000 7f3d20920b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf800d0d0 con 0x5653ec4b9aa0 2024-05-17T22:42:12.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.918+0000 7f3d0b7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653ec4b4170 0x5653ec4c5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:12.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.918+0000 7f3d0affd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653ec3359f0 0x5653ec4c5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:12.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.918+0000 7f3d0b7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653ec4b4170 0x5653ec4c5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:52572/0 (socket says 172.21.15.12:52572) 2024-05-17T22:42:12.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.918+0000 7f3d0b7fe700 1 -- 172.21.15.12:0/1092179289 learned_addr learned my addr 172.21.15.12:0/1092179289 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:42:12.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.918+0000 7f3d20920b80 1 -- 172.21.15.12:0/1092179289 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf800e540 con 0x5653ec4b4170 2024-05-17T22:42:12.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.918+0000 7f3d20920b80 1 -- 172.21.15.12:0/1092179289 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf800d7a0 con 0x5653ec3359f0 2024-05-17T22:42:12.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.918+0000 7f3d0affd700 1 --2- 172.21.15.12:0/1092179289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653ec4b9aa0 0x5653ec4d3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:12.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.919+0000 7f3d0b7fe700 1 -- 172.21.15.12:0/1092179289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653ec3359f0 msgr2=0x5653ec4c5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:42:12.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.919+0000 7f3d0b7fe700 1 --2- 172.21.15.12:0/1092179289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653ec3359f0 0x5653ec4c5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:12.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.919+0000 7f3d0b7fe700 1 -- 172.21.15.12:0/1092179289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653ec4b9aa0 msgr2=0x5653ec4d3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:42:12.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.919+0000 7f3d0b7fe700 1 --2- 172.21.15.12:0/1092179289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653ec4b9aa0 0x5653ec4d3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:12.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.919+0000 7f3d0b7fe700 1 -- 172.21.15.12:0/1092179289 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653ec4be030 con 0x5653ec4b4170 2024-05-17T22:42:12.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.919+0000 7f3d20920b80 10 client.0 _ll_register_callbacks cb 0x5653ec36cf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:42:12.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.919+0000 7f3d20920b80 1 -- 172.21.15.12:0/1092179289 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d0002b070 con 0x5653ec4b4170 2024-05-17T22:42:12.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.919+0000 7f3d0b7fe700 1 --2- 172.21.15.12:0/1092179289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653ec4b4170 0x5653ec4c5c40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x5653ec4be9e0 tx=0x7f3cfc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b39047d4bc7987f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:42:12.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.919+0000 7f3ceaffd700 10 client.25295 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:42:12.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.919+0000 7f3ceaffd700 1 -- 172.21.15.12:0/1092179289 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc012880 con 0x5653ec4b4170 2024-05-17T22:42:12.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.919+0000 7f3ceaffd700 1 -- 172.21.15.12:0/1092179289 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f3cfc012a20 con 0x5653ec4b4170 2024-05-17T22:42:12.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.919+0000 7f3ceaffd700 1 -- 172.21.15.12:0/1092179289 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3cfc012c80 con 0x5653ec4b4170 2024-05-17T22:42:12.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.919+0000 7f3d20920b80 10 client.25295 fetch_fsmap learned FSMap version 30 2024-05-17T22:42:12.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.919+0000 7f3d20920b80 1 -- 172.21.15.12:0/1092179289 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({fsmap=30}) -- 0x5653ec536f60 con 0x5653ec4b4170 2024-05-17T22:42:12.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.920+0000 7f3ceaffd700 1 -- 172.21.15.12:0/1092179289 <== mon.2 v2:172.21.15.96:3300/0 4 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f3cfc01e430 con 0x5653ec4b4170 2024-05-17T22:42:12.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.920+0000 7f3d20920b80 10 client.25295 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T22:42:12.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.920+0000 7f3d20920b80 10 client.25295 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-17T22:42:12.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.920+0000 7f3d20920b80 20 client.25295 populate_metadata read hostname 'smithi012' 2024-05-17T22:42:12.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.920+0000 7f3d20920b80 1 --2- 172.21.15.12:0/1092179289 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5653ec4cd360 0x5653ec5395a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:12.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.920+0000 7f3d20920b80 4 client.25295 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:42:12.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.920+0000 7f3d20920b80 1 -- 172.21.15.12:0/1092179289 --> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5653ec3dd9c0 con 0x5653ec4cd360 2024-05-17T22:42:12.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.920+0000 7f3d0affd700 1 --2- 172.21.15.12:0/1092179289 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5653ec4cd360 0x5653ec5395a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:12.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.921+0000 7f3ceaffd700 10 client.25295 ms_handle_connect on v2:172.21.15.96:6836/4222574338 2024-05-17T22:42:12.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.921+0000 7f3d0affd700 1 --2- 172.21.15.12:0/1092179289 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5653ec4cd360 0x5653ec5395a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4e5d0068be18eeaf server_cookie=af96b1035e26b6d0 in_seq=0 out_seq=0 2024-05-17T22:42:12.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.921+0000 7f3ceaffd700 1 -- 172.21.15.12:0/1092179289 <== mds.1 v2:172.21.15.96:6836/4222574338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5653ec3dd9c0 con 0x5653ec4cd360 2024-05-17T22:42:12.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.921+0000 7f3ceaffd700 10 client.25295 handle_command_reply: tid=0 2024-05-17T22:42:12.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.921+0000 7f3d20920b80 1 client.25295 shutdown 2024-05-17T22:42:12.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.921+0000 7f3d20920b80 1 -- 172.21.15.12:0/1092179289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653ec4b4170 msgr2=0x5653ec4c5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:12.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.921+0000 7f3d20920b80 1 --2- 172.21.15.12:0/1092179289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653ec4b4170 0x5653ec4c5c40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x5653ec4be9e0 tx=0x7f3cfc009c00 comp rx=0 tx=0).stop 2024-05-17T22:42:12.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.922+0000 7f3d20920b80 1 -- 172.21.15.12:0/1092179289 shutdown_connections 2024-05-17T22:42:12.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.922+0000 7f3d20920b80 1 --2- 172.21.15.12:0/1092179289 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5653ec4cd360 0x5653ec5395a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:12.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.922+0000 7f3ceaffd700 0 client.25295 ms_handle_reset on v2:172.21.15.96:6836/4222574338 2024-05-17T22:42:12.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.922+0000 7f3d20920b80 1 --2- 172.21.15.12:0/1092179289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653ec4b9aa0 0x5653ec4d3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:12.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.922+0000 7f3d20920b80 1 --2- 172.21.15.12:0/1092179289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653ec3359f0 0x5653ec4c5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:12.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.922+0000 7f3d20920b80 1 --2- 172.21.15.12:0/1092179289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653ec4b4170 0x5653ec4c5c40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:12.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.922+0000 7f3d20920b80 1 -- 172.21.15.12:0/1092179289 >> 172.21.15.12:0/1092179289 conn(0x5653ec379f00 msgr2=0x5653ec4b6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:42:12.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.922+0000 7f3d20920b80 1 -- 172.21.15.12:0/1092179289 shutdown_connections 2024-05-17T22:42:12.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.922+0000 7f3d20920b80 1 -- 172.21.15.12:0/1092179289 wait complete. 2024-05-17T22:42:12.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.922+0000 7f3d20920b80 20 client.25295 trim_cache size 0 max 16384 2024-05-17T22:42:12.927 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:42:12.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.924+0000 7f3d11f45700 1 -- 172.21.15.12:0/2284075449 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f3cf40b2a90 msgr2=0x7f3cf40b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:12.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.924+0000 7f3d11f45700 1 --2- 172.21.15.12:0/2284075449 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f3cf40b2a90 0x7f3cf40b4fa0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3d040a79a0 tx=0x7f3d040a7040 comp rx=0 tx=0).stop 2024-05-17T22:42:12.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.924+0000 7f3d11f45700 1 -- 172.21.15.12:0/2284075449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d0c11d550 msgr2=0x7f3d0c117bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:12.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.924+0000 7f3d11f45700 1 --2- 172.21.15.12:0/2284075449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d0c11d550 0x7f3d0c117bf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3d04009fd0 tx=0x7f3d0400cfc0 comp rx=0 tx=0).stop 2024-05-17T22:42:12.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.925+0000 7f3d11f45700 1 -- 172.21.15.12:0/2284075449 shutdown_connections 2024-05-17T22:42:12.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.925+0000 7f3d11f45700 1 --2- 172.21.15.12:0/2284075449 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f3cf40b2a90 0x7f3cf40b4fa0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:12.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.925+0000 7f3d11f45700 1 --2- 172.21.15.12:0/2284075449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d0c11d550 0x7f3d0c117bf0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:12.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.925+0000 7f3d11f45700 1 --2- 172.21.15.12:0/2284075449 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d0c10a830 0x7f3d0c117170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:12.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.925+0000 7f3d11f45700 1 --2- 172.21.15.12:0/2284075449 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d0c10b180 0x7f3d0c1176b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:12.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.925+0000 7f3d11f45700 1 -- 172.21.15.12:0/2284075449 >> 172.21.15.12:0/2284075449 conn(0x7f3d0c06efc0 msgr2=0x7f3d0c11dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:42:12.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.927+0000 7f3d11f45700 1 -- 172.21.15.12:0/2284075449 shutdown_connections 2024-05-17T22:42:12.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:12.927+0000 7f3d11f45700 1 -- 172.21.15.12:0/2284075449 wait complete. 2024-05-17T22:42:13.994 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-17T22:42:14.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:13 smithi151 ceph-mon[36101]: pgmap v528: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 MiB/s wr, 877 op/s 2024-05-17T22:42:14.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:13 smithi096 ceph-mon[36065]: pgmap v528: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 MiB/s wr, 877 op/s 2024-05-17T22:42:14.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:13 smithi012 ceph-mon[31632]: pgmap v528: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 MiB/s wr, 877 op/s 2024-05-17T22:42:14.397 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:42:16.044 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:15 smithi012 ceph-mon[31632]: pgmap v529: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 10 MiB/s wr, 810 op/s 2024-05-17T22:42:16.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:15 smithi151 ceph-mon[36101]: pgmap v529: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 10 MiB/s wr, 810 op/s 2024-05-17T22:42:16.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:15 smithi096 ceph-mon[36065]: pgmap v529: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 10 MiB/s wr, 810 op/s 2024-05-17T22:42:16.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 175.0 s, 1348.0 tps, lat 74.438 ms stddev 83.756 2024-05-17T22:42:16.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.792+0000 7efee290f700 1 -- 172.21.15.12:0/2377795742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efedc1032a0 msgr2=0x7efedc10f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:16.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.792+0000 7efee290f700 1 --2- 172.21.15.12:0/2377795742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efedc1032a0 0x7efedc10f820 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7efed400b0a0 tx=0x7efed400b3b0 comp rx=0 tx=0).stop 2024-05-17T22:42:16.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.793+0000 7efee290f700 1 -- 172.21.15.12:0/2377795742 shutdown_connections 2024-05-17T22:42:16.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.793+0000 7efee290f700 1 --2- 172.21.15.12:0/2377795742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efedc1032a0 0x7efedc10f820 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:16.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.793+0000 7efee290f700 1 --2- 172.21.15.12:0/2377795742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efedc108980 0x7efedc108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:16.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.793+0000 7efee290f700 1 --2- 172.21.15.12:0/2377795742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efedc102950 0x7efedc102d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:16.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.793+0000 7efee290f700 1 -- 172.21.15.12:0/2377795742 >> 172.21.15.12:0/2377795742 conn(0x7efedc0fe660 msgr2=0x7efedc100ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:42:16.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.793+0000 7efee290f700 1 -- 172.21.15.12:0/2377795742 shutdown_connections 2024-05-17T22:42:16.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.793+0000 7efee290f700 1 -- 172.21.15.12:0/2377795742 wait complete. 2024-05-17T22:42:16.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.794+0000 7efee290f700 1 Processor -- start 2024-05-17T22:42:16.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.794+0000 7efee290f700 1 -- start start 2024-05-17T22:42:16.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.794+0000 7efee290f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efedc102950 0x7efedc19a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:16.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.794+0000 7efee290f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efedc1032a0 0x7efedc19a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:16.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.794+0000 7efee290f700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efedc108980 0x7efedc19ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:16.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.795+0000 7efee110c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efedc1032a0 0x7efedc19a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:16.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.795+0000 7efee110c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efedc1032a0 0x7efedc19a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45670/0 (socket says 172.21.15.12:45670) 2024-05-17T22:42:16.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.795+0000 7efee110c700 1 -- 172.21.15.12:0/460111627 learned_addr learned my addr 172.21.15.12:0/460111627 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:42:16.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.795+0000 7efee210e700 1 --2- 172.21.15.12:0/460111627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efedc108980 0x7efedc19ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:16.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.795+0000 7efee290f700 1 -- 172.21.15.12:0/460111627 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7efedc113c80 con 0x7efedc1032a0 2024-05-17T22:42:16.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.795+0000 7efee290f700 1 -- 172.21.15.12:0/460111627 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efedc113b00 con 0x7efedc102950 2024-05-17T22:42:16.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.795+0000 7efee290f700 1 -- 172.21.15.12:0/460111627 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efedc113e00 con 0x7efedc108980 2024-05-17T22:42:16.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.795+0000 7efee110c700 1 -- 172.21.15.12:0/460111627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efedc108980 msgr2=0x7efedc19ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:16.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.795+0000 7efee110c700 1 --2- 172.21.15.12:0/460111627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efedc108980 0x7efedc19ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:16.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.795+0000 7efee190d700 1 --2- 172.21.15.12:0/460111627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efedc102950 0x7efedc19a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:16.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.795+0000 7efee110c700 1 -- 172.21.15.12:0/460111627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efedc102950 msgr2=0x7efedc19a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:16.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.795+0000 7efee110c700 1 --2- 172.21.15.12:0/460111627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efedc102950 0x7efedc19a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:16.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.795+0000 7efee110c700 1 -- 172.21.15.12:0/460111627 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efedc19f2b0 con 0x7efedc1032a0 2024-05-17T22:42:16.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.795+0000 7efee190d700 1 --2- 172.21.15.12:0/460111627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efedc102950 0x7efedc19a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:42:16.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.795+0000 7efee110c700 1 --2- 172.21.15.12:0/460111627 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efedc1032a0 0x7efedc19a980 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7efecc00e970 tx=0x7efecc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:42:16.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.796+0000 7efed2ffd700 1 -- 172.21.15.12:0/460111627 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc00cbd0 con 0x7efedc1032a0 2024-05-17T22:42:16.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.796+0000 7efee290f700 1 -- 172.21.15.12:0/460111627 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efedc19f5a0 con 0x7efedc1032a0 2024-05-17T22:42:16.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.796+0000 7efee290f700 1 -- 172.21.15.12:0/460111627 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efedc06d250 con 0x7efedc1032a0 2024-05-17T22:42:16.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.796+0000 7efed2ffd700 1 -- 172.21.15.12:0/460111627 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efecc00cd70 con 0x7efedc1032a0 2024-05-17T22:42:16.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.796+0000 7efed2ffd700 1 -- 172.21.15.12:0/460111627 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc005680 con 0x7efedc1032a0 2024-05-17T22:42:16.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.802+0000 7efef254cb80 1 Processor -- start 2024-05-17T22:42:16.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.802+0000 7efef254cb80 1 -- start start 2024-05-17T22:42:16.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.802+0000 7efef254cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc3c689f0 0x555cc3df0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:16.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.803+0000 7efef254cb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cc3df0890 0x555cc3dea9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:16.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.803+0000 7efef254cb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cc3deaf10 0x555cc3deb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:16.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.803+0000 7efef254cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x555cc3d150d0 con 0x555cc3c689f0 2024-05-17T22:42:16.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.803+0000 7efef254cb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555cc3d04c70 con 0x555cc3deaf10 2024-05-17T22:42:16.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.803+0000 7efef254cb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555cc3d08cb0 con 0x555cc3df0890 2024-05-17T22:42:16.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.803+0000 7efed2ffd700 1 -- 172.21.15.12:0/460111627 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7efecc004750 con 0x7efedc1032a0 2024-05-17T22:42:16.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.803+0000 7efee210e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc3c689f0 0x555cc3df0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:16.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.803+0000 7efee210e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc3c689f0 0x555cc3df0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45680/0 (socket says 172.21.15.12:45680) 2024-05-17T22:42:16.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.803+0000 7efee210e700 1 -- 172.21.15.12:0/2508267401 learned_addr learned my addr 172.21.15.12:0/2508267401 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:42:16.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.803+0000 7efee110c700 1 --2- 172.21.15.12:0/2508267401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cc3deaf10 0x555cc3deb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:16.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.803+0000 7efee210e700 1 -- 172.21.15.12:0/2508267401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cc3df0890 msgr2=0x555cc3dea9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T22:42:16.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.803+0000 7efee210e700 1 --2- 172.21.15.12:0/2508267401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cc3df0890 0x555cc3dea9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:16.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.803+0000 7efee210e700 1 -- 172.21.15.12:0/2508267401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cc3deaf10 msgr2=0x555cc3deb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:42:16.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.803+0000 7efee210e700 1 --2- 172.21.15.12:0/2508267401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cc3deaf10 0x555cc3deb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:16.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.804+0000 7efee210e700 1 -- 172.21.15.12:0/2508267401 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cc3c9e5c0 con 0x555cc3c689f0 2024-05-17T22:42:16.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.804+0000 7efee110c700 1 --2- 172.21.15.12:0/2508267401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cc3deaf10 0x555cc3deb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:42:16.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.804+0000 7efee210e700 1 --2- 172.21.15.12:0/2508267401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc3c689f0 0x555cc3df0350 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7efed400b0a0 tx=0x7efed4009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=326053bddab3a94 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:42:16.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.804+0000 7efec7fff700 1 -- 172.21.15.12:0/2508267401 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed400bcb0 con 0x555cc3c689f0 2024-05-17T22:42:16.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.804+0000 7efed2ffd700 1 --2- 172.21.15.12:0/460111627 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7efec80b2880 0x7efec80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:16.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.804+0000 7efec7fff700 1 -- 172.21.15.12:0/2508267401 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efed40075c0 con 0x555cc3c689f0 2024-05-17T22:42:16.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.805+0000 7efed2ffd700 1 -- 172.21.15.12:0/460111627 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7efecc0e2830 con 0x7efedc1032a0 2024-05-17T22:42:16.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.806+0000 7efef254cb80 1 -- 172.21.15.12:0/2508267401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc3c689f0 msgr2=0x555cc3df0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:16.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.806+0000 7efef254cb80 1 --2- 172.21.15.12:0/2508267401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc3c689f0 0x555cc3df0350 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7efed400b0a0 tx=0x7efed4009c90 comp rx=0 tx=0).stop 2024-05-17T22:42:16.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.806+0000 7efec7fff700 1 -- 172.21.15.12:0/2508267401 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed40052a0 con 0x555cc3c689f0 2024-05-17T22:42:16.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.806+0000 7efef254cb80 1 -- 172.21.15.12:0/2508267401 shutdown_connections 2024-05-17T22:42:16.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.806+0000 7efef254cb80 1 --2- 172.21.15.12:0/2508267401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cc3deaf10 0x555cc3deb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:16.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.806+0000 7efef254cb80 1 --2- 172.21.15.12:0/2508267401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc3c689f0 0x555cc3df0350 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:16.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.806+0000 7efef254cb80 1 --2- 172.21.15.12:0/2508267401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cc3df0890 0x555cc3dea9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:16.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.806+0000 7efef254cb80 1 -- 172.21.15.12:0/2508267401 >> 172.21.15.12:0/2508267401 conn(0x555cc3cc87d0 msgr2=0x555cc3de11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:42:16.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.806+0000 7efee190d700 1 --2- 172.21.15.12:0/460111627 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7efec80b2880 0x7efec80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:16.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.806+0000 7efef254cb80 1 -- 172.21.15.12:0/2508267401 shutdown_connections 2024-05-17T22:42:16.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.806+0000 7efef254cb80 1 -- 172.21.15.12:0/2508267401 wait complete. 2024-05-17T22:42:16.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.807+0000 7efee190d700 1 --2- 172.21.15.12:0/460111627 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7efec80b2880 0x7efec80b4d90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efed8005e90 tx=0x7efed8005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:42:16.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.807+0000 7efef254cb80 1 -- start start 2024-05-17T22:42:16.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.807+0000 7efef254cb80 1 Processor -- start 2024-05-17T22:42:16.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.807+0000 7efef254cb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cc3c689f0 0x555cc3df3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:16.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.807+0000 7efef254cb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cc3deaf10 0x555cc3df44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:16.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.807+0000 7efef254cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc3df0890 0x555cc3df4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:16.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.808+0000 7efef254cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x555cc3c0c150 con 0x555cc3df0890 2024-05-17T22:42:16.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.808+0000 7efef254cb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555cc3ca6940 con 0x555cc3c689f0 2024-05-17T22:42:16.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.808+0000 7efef254cb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555cc3df73a0 con 0x555cc3deaf10 2024-05-17T22:42:16.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.808+0000 7efef254cb80 10 client.0 _ll_register_callbacks cb 0x555cc3c9bb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:42:16.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.808+0000 7efee110c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cc3deaf10 0x555cc3df44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:16.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.808+0000 7efee210e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc3df0890 0x555cc3df4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:16.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.808+0000 7efee190d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cc3c689f0 0x555cc3df3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:16.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.808+0000 7efee110c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cc3deaf10 0x555cc3df44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:51128/0 (socket says 172.21.15.12:51128) 2024-05-17T22:42:16.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.808+0000 7efee110c700 1 -- 172.21.15.12:0/3043172552 learned_addr learned my addr 172.21.15.12:0/3043172552 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:42:16.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.808+0000 7efee110c700 1 -- 172.21.15.12:0/3043172552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cc3c689f0 msgr2=0x555cc3df3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:42:16.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.808+0000 7efee110c700 1 --2- 172.21.15.12:0/3043172552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cc3c689f0 0x555cc3df3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:16.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.808+0000 7efee110c700 1 -- 172.21.15.12:0/3043172552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc3df0890 msgr2=0x555cc3df4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:42:16.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.808+0000 7efee110c700 1 --2- 172.21.15.12:0/3043172552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc3df0890 0x555cc3df4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:16.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.808+0000 7efee110c700 1 -- 172.21.15.12:0/3043172552 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cc3c55480 con 0x555cc3deaf10 2024-05-17T22:42:16.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.808+0000 7efee110c700 1 --2- 172.21.15.12:0/3043172552 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cc3deaf10 0x555cc3df44d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efecc009230 tx=0x7efecc0a9500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c458a939a03cb103 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:42:16.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.808+0000 7efec57fa700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:42:16.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.808+0000 7efec57fa700 1 -- 172.21.15.12:0/3043172552 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc0b2260 con 0x555cc3deaf10 2024-05-17T22:42:16.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.808+0000 7efef254cb80 1 -- 172.21.15.12:0/3043172552 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efed40075c0 con 0x555cc3deaf10 2024-05-17T22:42:16.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.808+0000 7efec57fa700 1 -- 172.21.15.12:0/3043172552 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7efecc0b26a0 con 0x555cc3deaf10 2024-05-17T22:42:16.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.809+0000 7efec57fa700 1 -- 172.21.15.12:0/3043172552 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efecc06e050 con 0x555cc3deaf10 2024-05-17T22:42:16.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.809+0000 7efef254cb80 10 client.25252 fetch_fsmap learned FSMap version 30 2024-05-17T22:42:16.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.809+0000 7efef254cb80 1 -- 172.21.15.12:0/3043172552 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555cc3e01bc0 con 0x555cc3deaf10 2024-05-17T22:42:16.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.809+0000 7efec57fa700 1 -- 172.21.15.12:0/3043172552 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efed40075c0 con 0x555cc3deaf10 2024-05-17T22:42:16.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.809+0000 7efec57fa700 1 -- 172.21.15.12:0/3043172552 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7efecc0e2ce0 con 0x555cc3deaf10 2024-05-17T22:42:16.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.809+0000 7efef254cb80 10 client.25252 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T22:42:16.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.809+0000 7efef254cb80 10 client.25252 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:42:16.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.809+0000 7efef254cb80 20 client.25252 populate_metadata read hostname 'smithi012' 2024-05-17T22:42:16.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.809+0000 7efef254cb80 1 --2- 172.21.15.12:0/3043172552 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555cc3e04250 0x555cc3e066e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:16.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.809+0000 7efef254cb80 4 client.25252 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:42:16.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.809+0000 7efef254cb80 1 -- 172.21.15.12:0/3043172552 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555cc3d17020 con 0x555cc3e04250 2024-05-17T22:42:16.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.809+0000 7efee190d700 1 --2- 172.21.15.12:0/3043172552 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555cc3e04250 0x555cc3e066e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:16.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.810+0000 7efec57fa700 10 client.25252 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:42:16.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.810+0000 7efee190d700 1 --2- 172.21.15.12:0/3043172552 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555cc3e04250 0x555cc3e066e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45b86df18793aa6c server_cookie=d328832644d605b7 in_seq=0 out_seq=0 2024-05-17T22:42:16.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.812+0000 7efec57fa700 1 -- 172.21.15.12:0/3043172552 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555cc3d17020 con 0x555cc3e04250 2024-05-17T22:42:16.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.812+0000 7efec57fa700 10 client.25252 handle_command_reply: tid=0 2024-05-17T22:42:16.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.812+0000 7efef254cb80 1 client.25252 shutdown 2024-05-17T22:42:16.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.812+0000 7efef254cb80 1 -- 172.21.15.12:0/3043172552 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cc3deaf10 msgr2=0x555cc3df44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:16.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.812+0000 7efef254cb80 1 --2- 172.21.15.12:0/3043172552 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cc3deaf10 0x555cc3df44d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efecc009230 tx=0x7efecc0a9500 comp rx=0 tx=0).stop 2024-05-17T22:42:16.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.813+0000 7efef254cb80 1 -- 172.21.15.12:0/3043172552 shutdown_connections 2024-05-17T22:42:16.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.813+0000 7efef254cb80 1 --2- 172.21.15.12:0/3043172552 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555cc3e04250 0x555cc3e066e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:16.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.813+0000 7efef254cb80 1 --2- 172.21.15.12:0/3043172552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc3df0890 0x555cc3df4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:16.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.813+0000 7efef254cb80 1 --2- 172.21.15.12:0/3043172552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cc3c689f0 0x555cc3df3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:16.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.813+0000 7efef254cb80 1 --2- 172.21.15.12:0/3043172552 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cc3deaf10 0x555cc3df44d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:16.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.813+0000 7efef254cb80 1 -- 172.21.15.12:0/3043172552 >> 172.21.15.12:0/3043172552 conn(0x555cc3cc87d0 msgr2=0x555cc3de1140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:42:16.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.813+0000 7efec57fa700 0 client.25252 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:42:16.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.813+0000 7efef254cb80 1 -- 172.21.15.12:0/3043172552 shutdown_connections 2024-05-17T22:42:16.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.813+0000 7efef254cb80 1 -- 172.21.15.12:0/3043172552 wait complete. 2024-05-17T22:42:16.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.814+0000 7efef254cb80 20 client.25252 trim_cache size 0 max 16384 2024-05-17T22:42:16.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.836+0000 7efef254cb80 1 Processor -- start 2024-05-17T22:42:16.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.837+0000 7efef254cb80 1 -- start start 2024-05-17T22:42:16.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.837+0000 7efef254cb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cc3c689f0 0x555cc3dee4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:16.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.837+0000 7efef254cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc3deaf10 0x555cc3e00390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:16.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.837+0000 7efef254cb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cc3df0890 0x555cc3df4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:16.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.837+0000 7efee110c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc3deaf10 0x555cc3e00390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:16.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.837+0000 7efee110c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc3deaf10 0x555cc3e00390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45696/0 (socket says 172.21.15.12:45696) 2024-05-17T22:42:16.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.837+0000 7efee110c700 1 -- 172.21.15.12:0/3348692275 learned_addr learned my addr 172.21.15.12:0/3348692275 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:42:16.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.837+0000 7efee190d700 1 --2- 172.21.15.12:0/3348692275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cc3c689f0 0x555cc3dee4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:16.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.837+0000 7efef254cb80 1 -- 172.21.15.12:0/3348692275 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7efedc1b8a00 con 0x555cc3deaf10 2024-05-17T22:42:16.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.837+0000 7efef254cb80 1 -- 172.21.15.12:0/3348692275 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efedc073df0 con 0x555cc3df0890 2024-05-17T22:42:16.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.837+0000 7efef254cb80 1 -- 172.21.15.12:0/3348692275 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efedc071c40 con 0x555cc3c689f0 2024-05-17T22:42:16.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.837+0000 7efee210e700 1 --2- 172.21.15.12:0/3348692275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cc3df0890 0x555cc3df4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:16.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.838+0000 7efee210e700 1 -- 172.21.15.12:0/3348692275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cc3c689f0 msgr2=0x555cc3dee4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:42:16.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.838+0000 7efee210e700 1 --2- 172.21.15.12:0/3348692275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cc3c689f0 0x555cc3dee4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:16.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.838+0000 7efee210e700 1 -- 172.21.15.12:0/3348692275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc3deaf10 msgr2=0x555cc3e00390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:42:16.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.838+0000 7efee210e700 1 --2- 172.21.15.12:0/3348692275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc3deaf10 0x555cc3e00390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:16.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.838+0000 7efee210e700 1 -- 172.21.15.12:0/3348692275 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cc3df6f80 con 0x555cc3df0890 2024-05-17T22:42:16.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.838+0000 7efee210e700 1 --2- 172.21.15.12:0/3348692275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cc3df0890 0x555cc3df4500 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efed4009610 tx=0x7efed4007eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=220bd7e9e6e98c37 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:42:16.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.838+0000 7efec77fe700 1 -- 172.21.15.12:0/3348692275 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed400bde0 con 0x555cc3df0890 2024-05-17T22:42:16.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.838+0000 7efec77fe700 1 -- 172.21.15.12:0/3348692275 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efed4003c10 con 0x555cc3df0890 2024-05-17T22:42:16.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.838+0000 7efec77fe700 1 -- 172.21.15.12:0/3348692275 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed4005520 con 0x555cc3df0890 2024-05-17T22:42:16.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.839+0000 7efef254cb80 1 -- 172.21.15.12:0/3348692275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cc3df0890 msgr2=0x555cc3df4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:16.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.839+0000 7efef254cb80 1 --2- 172.21.15.12:0/3348692275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cc3df0890 0x555cc3df4500 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efed4009610 tx=0x7efed4007eb0 comp rx=0 tx=0).stop 2024-05-17T22:42:16.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.839+0000 7efef254cb80 1 -- 172.21.15.12:0/3348692275 shutdown_connections 2024-05-17T22:42:16.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.839+0000 7efef254cb80 1 --2- 172.21.15.12:0/3348692275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cc3df0890 0x555cc3df4500 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:16.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.839+0000 7efef254cb80 1 --2- 172.21.15.12:0/3348692275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cc3c689f0 0x555cc3dee4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:16.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.839+0000 7efef254cb80 1 --2- 172.21.15.12:0/3348692275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc3deaf10 0x555cc3e00390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:16.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.839+0000 7efef254cb80 1 -- 172.21.15.12:0/3348692275 >> 172.21.15.12:0/3348692275 conn(0x555cc3cc87d0 msgr2=0x555cc3dedde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:42:16.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.839+0000 7efef254cb80 1 -- 172.21.15.12:0/3348692275 shutdown_connections 2024-05-17T22:42:16.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.839+0000 7efef254cb80 1 -- 172.21.15.12:0/3348692275 wait complete. 2024-05-17T22:42:16.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.840+0000 7efef254cb80 1 -- start start 2024-05-17T22:42:16.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.840+0000 7efef254cb80 1 Processor -- start 2024-05-17T22:42:16.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.841+0000 7efef254cb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc3c689f0 0x555cc3e0c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:16.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.841+0000 7efef254cb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cc3deaf10 0x555cc3de75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:16.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.841+0000 7efef254cb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cc3df0890 0x555cc3de7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:16.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.841+0000 7efef254cb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7efedc0722b0 con 0x555cc3c689f0 2024-05-17T22:42:16.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.841+0000 7efef254cb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efedc073720 con 0x555cc3df0890 2024-05-17T22:42:16.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.841+0000 7efef254cb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efedc072980 con 0x555cc3deaf10 2024-05-17T22:42:16.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.841+0000 7efef254cb80 10 client.0 _ll_register_callbacks cb 0x555cc3c9bb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:42:16.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.841+0000 7efee190d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc3c689f0 0x555cc3e0c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:16.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.841+0000 7efee190d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc3c689f0 0x555cc3e0c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45706/0 (socket says 172.21.15.12:45706) 2024-05-17T22:42:16.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.841+0000 7efee190d700 1 -- 172.21.15.12:0/727033252 learned_addr learned my addr 172.21.15.12:0/727033252 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:42:16.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.841+0000 7efee110c700 1 --2- 172.21.15.12:0/727033252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cc3deaf10 0x555cc3de75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:16.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.842+0000 7efee190d700 1 -- 172.21.15.12:0/727033252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cc3deaf10 msgr2=0x555cc3de75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:42:16.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.842+0000 7efee190d700 1 --2- 172.21.15.12:0/727033252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cc3deaf10 0x555cc3de75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:16.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.842+0000 7efee190d700 1 -- 172.21.15.12:0/727033252 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cc3df0890 msgr2=0x555cc3de7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:42:16.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.842+0000 7efee210e700 1 --2- 172.21.15.12:0/727033252 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cc3df0890 0x555cc3de7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:16.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.842+0000 7efee190d700 1 --2- 172.21.15.12:0/727033252 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cc3df0890 0x555cc3de7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:16.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.842+0000 7efee190d700 1 -- 172.21.15.12:0/727033252 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cc3de8350 con 0x555cc3c689f0 2024-05-17T22:42:16.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.842+0000 7efee110c700 1 --2- 172.21.15.12:0/727033252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cc3deaf10 0x555cc3de75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:42:16.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.842+0000 7efee190d700 1 --2- 172.21.15.12:0/727033252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc3c689f0 0x555cc3e0c590 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7efedc103a90 tx=0x7efed8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f35af81fe5ee53 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:42:16.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.842+0000 7efec57fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:42:16.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.843+0000 7efec57fa700 1 -- 172.21.15.12:0/727033252 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed8011e10 con 0x555cc3c689f0 2024-05-17T22:42:16.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.843+0000 7efef254cb80 1 -- 172.21.15.12:0/727033252 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efed4003c10 con 0x555cc3c689f0 2024-05-17T22:42:16.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.843+0000 7efec57fa700 1 -- 172.21.15.12:0/727033252 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7efed80261a0 con 0x555cc3c689f0 2024-05-17T22:42:16.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.843+0000 7efec57fa700 1 -- 172.21.15.12:0/727033252 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efed800ee60 con 0x555cc3c689f0 2024-05-17T22:42:16.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.843+0000 7efef254cb80 10 client.15642 fetch_fsmap learned FSMap version 30 2024-05-17T22:42:16.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.843+0000 7efef254cb80 1 -- 172.21.15.12:0/727033252 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555cc3dfc790 con 0x555cc3c689f0 2024-05-17T22:42:16.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.843+0000 7efec57fa700 1 -- 172.21.15.12:0/727033252 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efed8017040 con 0x555cc3c689f0 2024-05-17T22:42:16.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.844+0000 7efec57fa700 1 -- 172.21.15.12:0/727033252 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7efed8026ce0 con 0x555cc3c689f0 2024-05-17T22:42:16.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.844+0000 7efef254cb80 10 client.15642 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T22:42:16.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.844+0000 7efef254cb80 10 client.15642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:42:16.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.844+0000 7efef254cb80 20 client.15642 populate_metadata read hostname 'smithi012' 2024-05-17T22:42:16.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.844+0000 7efef254cb80 1 --2- 172.21.15.12:0/727033252 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555cc3dfcee0 0x555cc3e75250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:16.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.844+0000 7efef254cb80 4 client.15642 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:42:16.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.844+0000 7efef254cb80 1 -- 172.21.15.12:0/727033252 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7efed4003c10 con 0x555cc3dfcee0 2024-05-17T22:42:16.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.844+0000 7efee110c700 1 --2- 172.21.15.12:0/727033252 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555cc3dfcee0 0x555cc3e75250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:16.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.845+0000 7efec57fa700 10 client.15642 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:42:16.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.845+0000 7efee110c700 1 --2- 172.21.15.12:0/727033252 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555cc3dfcee0 0x555cc3e75250 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=864d3331d385761c server_cookie=e91e4d49e452f6a6 in_seq=0 out_seq=0 2024-05-17T22:42:16.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.845+0000 7efec57fa700 1 -- 172.21.15.12:0/727033252 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7efed4003c10 con 0x555cc3dfcee0 2024-05-17T22:42:16.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.845+0000 7efec57fa700 10 client.15642 handle_command_reply: tid=0 2024-05-17T22:42:16.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.845+0000 7efef254cb80 1 client.15642 shutdown 2024-05-17T22:42:16.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.845+0000 7efef254cb80 1 -- 172.21.15.12:0/727033252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc3c689f0 msgr2=0x555cc3e0c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:16.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.845+0000 7efef254cb80 1 --2- 172.21.15.12:0/727033252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc3c689f0 0x555cc3e0c590 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7efedc103a90 tx=0x7efed8011000 comp rx=0 tx=0).stop 2024-05-17T22:42:16.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.847+0000 7efef254cb80 1 -- 172.21.15.12:0/727033252 shutdown_connections 2024-05-17T22:42:16.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.847+0000 7efef254cb80 1 --2- 172.21.15.12:0/727033252 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cc3df0890 0x555cc3de7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:16.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.847+0000 7efef254cb80 1 --2- 172.21.15.12:0/727033252 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555cc3dfcee0 0x555cc3e75250 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:16.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.847+0000 7efef254cb80 1 --2- 172.21.15.12:0/727033252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555cc3c689f0 0x555cc3e0c590 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:16.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.847+0000 7efef254cb80 1 --2- 172.21.15.12:0/727033252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cc3deaf10 0x555cc3de75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:16.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.847+0000 7efef254cb80 1 -- 172.21.15.12:0/727033252 >> 172.21.15.12:0/727033252 conn(0x555cc3cc87d0 msgr2=0x555cc3dee570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:42:16.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.847+0000 7efec57fa700 0 client.15642 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:42:16.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.847+0000 7efef254cb80 1 -- 172.21.15.12:0/727033252 shutdown_connections 2024-05-17T22:42:16.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.847+0000 7efef254cb80 1 -- 172.21.15.12:0/727033252 wait complete. 2024-05-17T22:42:16.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.847+0000 7efef254cb80 20 client.15642 trim_cache size 0 max 16384 2024-05-17T22:42:16.851 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:42:16.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.849+0000 7efee290f700 1 -- 172.21.15.12:0/460111627 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7efec80b2880 msgr2=0x7efec80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:16.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.849+0000 7efee290f700 1 --2- 172.21.15.12:0/460111627 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7efec80b2880 0x7efec80b4d90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efed8005e90 tx=0x7efed8005d00 comp rx=0 tx=0).stop 2024-05-17T22:42:16.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.849+0000 7efee290f700 1 -- 172.21.15.12:0/460111627 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efedc1032a0 msgr2=0x7efedc19a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:16.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.849+0000 7efee290f700 1 --2- 172.21.15.12:0/460111627 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efedc1032a0 0x7efedc19a980 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7efecc00e970 tx=0x7efecc00ec80 comp rx=0 tx=0).stop 2024-05-17T22:42:16.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.849+0000 7efee290f700 1 -- 172.21.15.12:0/460111627 shutdown_connections 2024-05-17T22:42:16.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.849+0000 7efee290f700 1 --2- 172.21.15.12:0/460111627 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7efec80b2880 0x7efec80b4d90 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:16.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.849+0000 7efee290f700 1 --2- 172.21.15.12:0/460111627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efedc108980 0x7efedc19ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:16.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.849+0000 7efee290f700 1 --2- 172.21.15.12:0/460111627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efedc102950 0x7efedc19a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:16.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.849+0000 7efee290f700 1 --2- 172.21.15.12:0/460111627 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efedc1032a0 0x7efedc19a980 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:16.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.849+0000 7efee290f700 1 -- 172.21.15.12:0/460111627 >> 172.21.15.12:0/460111627 conn(0x7efedc0fe660 msgr2=0x7efedc106b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:42:16.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.849+0000 7efee290f700 1 -- 172.21.15.12:0/460111627 shutdown_connections 2024-05-17T22:42:16.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:16.849+0000 7efee290f700 1 -- 172.21.15.12:0/460111627 wait complete. 2024-05-17T22:42:18.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:17 smithi151 ceph-mon[36101]: pgmap v530: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 4.8 MiB/s wr, 539 op/s 2024-05-17T22:42:18.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:17 smithi096 ceph-mon[36065]: pgmap v530: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 4.8 MiB/s wr, 539 op/s 2024-05-17T22:42:18.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:17 smithi012 ceph-mon[31632]: pgmap v530: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 4.8 MiB/s wr, 539 op/s 2024-05-17T22:42:19.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:19 smithi012 ceph-mon[31632]: pgmap v531: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.7 MiB/s wr, 641 op/s 2024-05-17T22:42:19.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:19 smithi151 ceph-mon[36101]: pgmap v531: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.7 MiB/s wr, 641 op/s 2024-05-17T22:42:19.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:19 smithi096 ceph-mon[36065]: pgmap v531: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.7 MiB/s wr, 641 op/s 2024-05-17T22:42:21.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 180.0 s, 922.6 tps, lat 107.865 ms stddev 147.553 2024-05-17T22:42:21.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:21 smithi151 ceph-mon[36101]: pgmap v532: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.4 MiB/s wr, 604 op/s 2024-05-17T22:42:22.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:21 smithi096 ceph-mon[36065]: pgmap v532: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.4 MiB/s wr, 604 op/s 2024-05-17T22:42:22.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:21 smithi012 ceph-mon[31632]: pgmap v532: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.4 MiB/s wr, 604 op/s 2024-05-17T22:42:23.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:23 smithi151 ceph-mon[36101]: pgmap v533: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.6 MiB/s wr, 609 op/s 2024-05-17T22:42:24.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:23 smithi096 ceph-mon[36065]: pgmap v533: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.6 MiB/s wr, 609 op/s 2024-05-17T22:42:24.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:23 smithi012 ceph-mon[31632]: pgmap v533: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.6 MiB/s wr, 609 op/s 2024-05-17T22:42:24.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:24 smithi151 ceph-mon[36101]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:42:25.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:24 smithi096 ceph-mon[36065]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:42:25.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:24 smithi012 ceph-mon[31632]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:42:25.363 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-17T22:42:25.363 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-17T22:42:25.788 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:42:25.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:25 smithi151 ceph-mon[36101]: pgmap v534: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 MiB/s wr, 636 op/s 2024-05-17T22:42:26.045 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:25 smithi012 ceph-mon[31632]: pgmap v534: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 MiB/s wr, 636 op/s 2024-05-17T22:42:26.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:25 smithi096 ceph-mon[36065]: pgmap v534: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 MiB/s wr, 636 op/s 2024-05-17T22:42:26.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 185.0 s, 521.8 tps, lat 187.049 ms stddev 295.801 2024-05-17T22:42:27.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:27 smithi151 ceph-mon[36101]: pgmap v535: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.9 MiB/s wr, 742 op/s 2024-05-17T22:42:28.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:27 smithi096 ceph-mon[36065]: pgmap v535: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.9 MiB/s wr, 742 op/s 2024-05-17T22:42:28.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:27 smithi012 ceph-mon[31632]: pgmap v535: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.9 MiB/s wr, 742 op/s 2024-05-17T22:42:30.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:29 smithi096 ceph-mon[36065]: pgmap v536: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 MiB/s wr, 1.07k op/s 2024-05-17T22:42:30.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:29 smithi096 ceph-mon[36065]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:42:30.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:29 smithi012 ceph-mon[31632]: pgmap v536: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 MiB/s wr, 1.07k op/s 2024-05-17T22:42:30.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:29 smithi012 ceph-mon[31632]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:42:30.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:29 smithi151 ceph-mon[36101]: pgmap v536: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 MiB/s wr, 1.07k op/s 2024-05-17T22:42:30.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:29 smithi151 ceph-mon[36101]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:42:31.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 190.0 s, 596.8 tps, lat 170.704 ms stddev 241.151 2024-05-17T22:42:31.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:31 smithi012 ceph-mon[31632]: pgmap v537: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 17 MiB/s wr, 1.05k op/s 2024-05-17T22:42:32.042 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:31 smithi151 ceph-mon[36101]: pgmap v537: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 17 MiB/s wr, 1.05k op/s 2024-05-17T22:42:32.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:31 smithi096 ceph-mon[36065]: pgmap v537: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 17 MiB/s wr, 1.05k op/s 2024-05-17T22:42:32.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.081+0000 7f90637fe700 1 -- 172.21.15.12:0/2979350298 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905c018e60 con 0x7f9064107800 2024-05-17T22:42:32.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.081+0000 7f906aa15700 1 -- 172.21.15.12:0/2979350298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9064107800 msgr2=0x7f906410f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:32.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.081+0000 7f906aa15700 1 --2- 172.21.15.12:0/2979350298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9064107800 0x7f906410f160 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f905c01c1a0 tx=0x7f905c01c4b0 comp rx=0 tx=0).stop 2024-05-17T22:42:32.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.082+0000 7f906aa15700 1 -- 172.21.15.12:0/2979350298 shutdown_connections 2024-05-17T22:42:32.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.082+0000 7f906aa15700 1 --2- 172.21.15.12:0/2979350298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9064107800 0x7f906410f160 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:32.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.083+0000 7f906aa15700 1 --2- 172.21.15.12:0/2979350298 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9064102580 0x7f9064102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:32.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.083+0000 7f906aa15700 1 --2- 172.21.15.12:0/2979350298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9064106eb0 0x7f90641072c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:32.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.083+0000 7f906aa15700 1 -- 172.21.15.12:0/2979350298 >> 172.21.15.12:0/2979350298 conn(0x7f90640fe030 msgr2=0x7f9064100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:42:32.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.083+0000 7f906aa15700 1 -- 172.21.15.12:0/2979350298 shutdown_connections 2024-05-17T22:42:32.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.083+0000 7f906aa15700 1 -- 172.21.15.12:0/2979350298 wait complete. 2024-05-17T22:42:32.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.084+0000 7f906aa15700 1 Processor -- start 2024-05-17T22:42:32.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.084+0000 7f906aa15700 1 -- start start 2024-05-17T22:42:32.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.085+0000 7f906aa15700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9064102580 0x7f906410a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:32.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.085+0000 7f906aa15700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9064106eb0 0x7f906410ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:32.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.085+0000 7f906aa15700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9064107800 0x7f9064116d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:32.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.085+0000 7f906aa15700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9064073600 con 0x7f9064106eb0 2024-05-17T22:42:32.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.085+0000 7f906aa15700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9064073480 con 0x7f9064102580 2024-05-17T22:42:32.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.085+0000 7f906aa15700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9064073780 con 0x7f9064107800 2024-05-17T22:42:32.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.086+0000 7f9068fb2700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9064107800 0x7f9064116d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:32.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.086+0000 7f9068fb2700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9064107800 0x7f9064116d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:58462/0 (socket says 172.21.15.12:58462) 2024-05-17T22:42:32.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.086+0000 7f9068fb2700 1 -- 172.21.15.12:0/3404496663 learned_addr learned my addr 172.21.15.12:0/3404496663 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:42:32.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.086+0000 7f9063fff700 1 --2- 172.21.15.12:0/3404496663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9064102580 0x7f906410a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:32.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.086+0000 7f9063fff700 1 -- 172.21.15.12:0/3404496663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9064107800 msgr2=0x7f9064116d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:32.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.086+0000 7f9063fff700 1 --2- 172.21.15.12:0/3404496663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9064107800 0x7f9064116d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:32.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.086+0000 7f9063fff700 1 -- 172.21.15.12:0/3404496663 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9064106eb0 msgr2=0x7f906410ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:32.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.086+0000 7f9063fff700 1 --2- 172.21.15.12:0/3404496663 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9064106eb0 0x7f906410ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:32.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.086+0000 7f9063fff700 1 -- 172.21.15.12:0/3404496663 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f906410b4d0 con 0x7f9064102580 2024-05-17T22:42:32.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.087+0000 7f9063fff700 1 --2- 172.21.15.12:0/3404496663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9064102580 0x7f906410a6e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f905400db70 tx=0x7f905400de80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:42:32.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.087+0000 7f9061ffb700 1 -- 172.21.15.12:0/3404496663 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905401b070 con 0x7f9064102580 2024-05-17T22:42:32.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.087+0000 7f906aa15700 1 -- 172.21.15.12:0/3404496663 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90641173d0 con 0x7f9064102580 2024-05-17T22:42:32.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.088+0000 7f906aa15700 1 -- 172.21.15.12:0/3404496663 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9064117910 con 0x7f9064102580 2024-05-17T22:42:32.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.088+0000 7f9061ffb700 1 -- 172.21.15.12:0/3404496663 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f905400ac10 con 0x7f9064102580 2024-05-17T22:42:32.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.088+0000 7f9061ffb700 1 -- 172.21.15.12:0/3404496663 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9054016730 con 0x7f9064102580 2024-05-17T22:42:32.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.092+0000 7f9061ffb700 1 -- 172.21.15.12:0/3404496663 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f905401e070 con 0x7f9064102580 2024-05-17T22:42:32.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.094+0000 7f9061ffb700 1 --2- 172.21.15.12:0/3404496663 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f90500b2900 0x7f90500b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:32.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.094+0000 7f905b7fe700 1 --2- 172.21.15.12:0/3404496663 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f90500b2900 0x7f90500b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:32.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.095+0000 7f9061ffb700 1 -- 172.21.15.12:0/3404496663 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f90540e3250 con 0x7f9064102580 2024-05-17T22:42:32.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.095+0000 7f905b7fe700 1 --2- 172.21.15.12:0/3404496663 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f90500b2900 0x7f90500b4e10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9064107620 tx=0x7f904c006cd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:42:32.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.097+0000 7f90793f0b80 1 Processor -- start 2024-05-17T22:42:32.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.097+0000 7f90793f0b80 1 -- start start 2024-05-17T22:42:32.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.097+0000 7f90793f0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b22e6ca70 0x555b22ff05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:32.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.097+0000 7f90793f0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b22ff0b20 0x555b22feacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:32.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.098+0000 7f90793f0b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b22feb1f0 0x555b22feb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:32.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.098+0000 7f90793f0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x555b22f19310 con 0x555b22ff0b20 2024-05-17T22:42:32.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.098+0000 7f90793f0b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555b22f09050 con 0x555b22feb1f0 2024-05-17T22:42:32.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.098+0000 7f90793f0b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555b22f0cef0 con 0x555b22e6ca70 2024-05-17T22:42:32.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.098+0000 7f905b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b22ff0b20 0x555b22feacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:32.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.098+0000 7f905b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b22ff0b20 0x555b22feacb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36518/0 (socket says 172.21.15.12:36518) 2024-05-17T22:42:32.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.098+0000 7f905b7fe700 1 -- 172.21.15.12:0/2973391874 learned_addr learned my addr 172.21.15.12:0/2973391874 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:42:32.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.098+0000 7f9068fb2700 1 --2- 172.21.15.12:0/2973391874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b22feb1f0 0x555b22feb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:32.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.099+0000 7f9068fb2700 1 -- 172.21.15.12:0/2973391874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b22e6ca70 msgr2=0x555b22ff05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:42:32.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.099+0000 7f9068fb2700 1 --2- 172.21.15.12:0/2973391874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b22e6ca70 0x555b22ff05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:32.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.099+0000 7f9068fb2700 1 -- 172.21.15.12:0/2973391874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b22ff0b20 msgr2=0x555b22feacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:42:32.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.099+0000 7f9068fb2700 1 --2- 172.21.15.12:0/2973391874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b22ff0b20 0x555b22feacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:32.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.099+0000 7f9068fb2700 1 -- 172.21.15.12:0/2973391874 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b22eb2610 con 0x555b22feb1f0 2024-05-17T22:42:32.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.099+0000 7f9068fb2700 1 --2- 172.21.15.12:0/2973391874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b22feb1f0 0x555b22feb600 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f905c01afd0 tx=0x7f905c01ae30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80b98ecea9b942c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:42:32.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.104+0000 7f905a7fc700 1 -- 172.21.15.12:0/2973391874 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905c0187d0 con 0x555b22feb1f0 2024-05-17T22:42:32.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.104+0000 7f905a7fc700 1 -- 172.21.15.12:0/2973391874 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f905c01a480 con 0x555b22feb1f0 2024-05-17T22:42:32.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.104+0000 7f905a7fc700 1 -- 172.21.15.12:0/2973391874 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905c0077f0 con 0x555b22feb1f0 2024-05-17T22:42:32.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.106+0000 7f90793f0b80 1 -- 172.21.15.12:0/2973391874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b22feb1f0 msgr2=0x555b22feb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:32.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.106+0000 7f90793f0b80 1 --2- 172.21.15.12:0/2973391874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b22feb1f0 0x555b22feb600 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f905c01afd0 tx=0x7f905c01ae30 comp rx=0 tx=0).stop 2024-05-17T22:42:32.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.106+0000 7f90793f0b80 1 -- 172.21.15.12:0/2973391874 shutdown_connections 2024-05-17T22:42:32.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.106+0000 7f90793f0b80 1 --2- 172.21.15.12:0/2973391874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b22feb1f0 0x555b22feb600 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:32.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.106+0000 7f90793f0b80 1 --2- 172.21.15.12:0/2973391874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b22e6ca70 0x555b22ff05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:32.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.106+0000 7f90793f0b80 1 --2- 172.21.15.12:0/2973391874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b22ff0b20 0x555b22feacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:32.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.106+0000 7f90793f0b80 1 -- 172.21.15.12:0/2973391874 >> 172.21.15.12:0/2973391874 conn(0x555b22eb0f80 msgr2=0x555b22fe5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:42:32.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.106+0000 7f90793f0b80 1 -- 172.21.15.12:0/2973391874 shutdown_connections 2024-05-17T22:42:32.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.106+0000 7f90793f0b80 1 -- 172.21.15.12:0/2973391874 wait complete. 2024-05-17T22:42:32.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.107+0000 7f90793f0b80 1 -- start start 2024-05-17T22:42:32.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.107+0000 7f90793f0b80 1 Processor -- start 2024-05-17T22:42:32.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.107+0000 7f90793f0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b22e6ca70 0x555b22ff42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:32.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.107+0000 7f90793f0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b22feb1f0 0x555b22ff4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:32.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.107+0000 7f90793f0b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b22ff0b20 0x555b23000740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:32.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.108+0000 7f90793f0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x555b22e101d0 con 0x555b22e6ca70 2024-05-17T22:42:32.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.108+0000 7f90793f0b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555b22ea9f00 con 0x555b22ff0b20 2024-05-17T22:42:32.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.108+0000 7f90793f0b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555b22ff7410 con 0x555b22feb1f0 2024-05-17T22:42:32.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.108+0000 7f90793f0b80 10 client.0 _ll_register_callbacks cb 0x555b22ea3f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:42:32.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.108+0000 7f9063fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b22e6ca70 0x555b22ff42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:32.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.108+0000 7f9063fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b22e6ca70 0x555b22ff42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36530/0 (socket says 172.21.15.12:36530) 2024-05-17T22:42:32.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.108+0000 7f9063fff700 1 -- 172.21.15.12:0/1929728328 learned_addr learned my addr 172.21.15.12:0/1929728328 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:42:32.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.108+0000 7f905b7fe700 1 --2- 172.21.15.12:0/1929728328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b22feb1f0 0x555b22ff4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:32.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.108+0000 7f9068fb2700 1 --2- 172.21.15.12:0/1929728328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b22ff0b20 0x555b23000740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:32.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.108+0000 7f9063fff700 1 -- 172.21.15.12:0/1929728328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b22feb1f0 msgr2=0x555b22ff4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:42:32.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.109+0000 7f9063fff700 1 --2- 172.21.15.12:0/1929728328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b22feb1f0 0x555b22ff4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:32.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.109+0000 7f9063fff700 1 -- 172.21.15.12:0/1929728328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b22ff0b20 msgr2=0x555b23000740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:42:32.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.109+0000 7f9063fff700 1 --2- 172.21.15.12:0/1929728328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b22ff0b20 0x555b23000740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:32.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.109+0000 7f9063fff700 1 -- 172.21.15.12:0/1929728328 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b22e59500 con 0x555b22e6ca70 2024-05-17T22:42:32.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.110+0000 7f9063fff700 1 --2- 172.21.15.12:0/1929728328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b22e6ca70 0x555b22ff42e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f90540aa470 tx=0x7f90540aa0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b96d5fc894886c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:42:32.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.111+0000 7f90437fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:42:32.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.111+0000 7f90437fe700 1 -- 172.21.15.12:0/1929728328 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90540ae890 con 0x555b22e6ca70 2024-05-17T22:42:32.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.111+0000 7f90437fe700 1 -- 172.21.15.12:0/1929728328 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f90540b30c0 con 0x555b22e6ca70 2024-05-17T22:42:32.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.111+0000 7f90793f0b80 1 -- 172.21.15.12:0/1929728328 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f905c01a480 con 0x555b22e6ca70 2024-05-17T22:42:32.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.112+0000 7f90437fe700 1 -- 172.21.15.12:0/1929728328 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f90540b3340 con 0x555b22e6ca70 2024-05-17T22:42:32.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.114+0000 7f90793f0b80 10 client.15651 fetch_fsmap learned FSMap version 32 2024-05-17T22:42:32.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.114+0000 7f90793f0b80 1 -- 172.21.15.12:0/1929728328 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555b23023f40 con 0x555b22e6ca70 2024-05-17T22:42:32.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.115+0000 7f90437fe700 1 -- 172.21.15.12:0/1929728328 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f905c01a480 con 0x555b22e6ca70 2024-05-17T22:42:32.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.115+0000 7f90437fe700 1 -- 172.21.15.12:0/1929728328 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7f905400b040 con 0x555b22e6ca70 2024-05-17T22:42:32.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.116+0000 7f90793f0b80 10 client.15651 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T22:42:32.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.116+0000 7f90793f0b80 10 client.15651 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:42:32.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.116+0000 7f90793f0b80 20 client.15651 populate_metadata read hostname 'smithi012' 2024-05-17T22:42:32.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.116+0000 7f90793f0b80 1 --2- 172.21.15.12:0/1929728328 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555b23026580 0x555b230289a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:32.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.116+0000 7f90793f0b80 4 client.15651 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:42:32.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.116+0000 7f90793f0b80 1 -- 172.21.15.12:0/1929728328 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b22f1b260 con 0x555b23026580 2024-05-17T22:42:32.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.117+0000 7f905b7fe700 1 --2- 172.21.15.12:0/1929728328 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555b23026580 0x555b230289a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:32.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.120+0000 7f905b7fe700 1 --2- 172.21.15.12:0/1929728328 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555b23026580 0x555b230289a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecc345f3dd41d987 server_cookie=3a9ab07d1907fc31 in_seq=0 out_seq=0 2024-05-17T22:42:32.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.121+0000 7f90437fe700 10 client.15651 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:42:32.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.122+0000 7f90437fe700 1 -- 172.21.15.12:0/1929728328 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555b22f1b260 con 0x555b23026580 2024-05-17T22:42:32.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.122+0000 7f90437fe700 10 client.15651 handle_command_reply: tid=0 2024-05-17T22:42:32.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.122+0000 7f90793f0b80 1 client.15651 shutdown 2024-05-17T22:42:32.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.123+0000 7f90793f0b80 1 -- 172.21.15.12:0/1929728328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b22e6ca70 msgr2=0x555b22ff42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:32.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.123+0000 7f90793f0b80 1 --2- 172.21.15.12:0/1929728328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b22e6ca70 0x555b22ff42e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f90540aa470 tx=0x7f90540aa0a0 comp rx=0 tx=0).stop 2024-05-17T22:42:32.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.124+0000 7f90793f0b80 1 -- 172.21.15.12:0/1929728328 shutdown_connections 2024-05-17T22:42:32.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.124+0000 7f90793f0b80 1 --2- 172.21.15.12:0/1929728328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b22ff0b20 0x555b23000740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:32.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.124+0000 7f90793f0b80 1 --2- 172.21.15.12:0/1929728328 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555b23026580 0x555b230289a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:32.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.124+0000 7f90793f0b80 1 --2- 172.21.15.12:0/1929728328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b22e6ca70 0x555b22ff42e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:32.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.124+0000 7f90793f0b80 1 --2- 172.21.15.12:0/1929728328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b22feb1f0 0x555b22ff4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:32.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.124+0000 7f90793f0b80 1 -- 172.21.15.12:0/1929728328 >> 172.21.15.12:0/1929728328 conn(0x555b22eb0f80 msgr2=0x555b22fdff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:42:32.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.124+0000 7f90437fe700 0 client.15651 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:42:32.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.126+0000 7f90793f0b80 1 -- 172.21.15.12:0/1929728328 shutdown_connections 2024-05-17T22:42:32.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.128+0000 7f90793f0b80 1 -- 172.21.15.12:0/1929728328 wait complete. 2024-05-17T22:42:32.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.130+0000 7f90793f0b80 20 client.15651 trim_cache size 0 max 16384 2024-05-17T22:42:32.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.169+0000 7f90793f0b80 1 Processor -- start 2024-05-17T22:42:32.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.170+0000 7f90793f0b80 1 -- start start 2024-05-17T22:42:32.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.170+0000 7f90793f0b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b22e6ca70 0x555b22fee7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:32.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.170+0000 7f90793f0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b22feb1f0 0x555b22fdff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:32.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.170+0000 7f90793f0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b22ff0b20 0x555b23000250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:32.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.170+0000 7f90793f0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9044011490 con 0x555b22ff0b20 2024-05-17T22:42:32.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.170+0000 7f90793f0b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f904400ecf0 con 0x555b22e6ca70 2024-05-17T22:42:32.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.170+0000 7f90793f0b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f904400cb40 con 0x555b22feb1f0 2024-05-17T22:42:32.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.171+0000 7f905b7fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b22feb1f0 0x555b22fdff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:32.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.171+0000 7f9068fb2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b22ff0b20 0x555b23000250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:32.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.171+0000 7f9068fb2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b22ff0b20 0x555b23000250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36546/0 (socket says 172.21.15.12:36546) 2024-05-17T22:42:32.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.171+0000 7f9068fb2700 1 -- 172.21.15.12:0/3296803358 learned_addr learned my addr 172.21.15.12:0/3296803358 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:42:32.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.171+0000 7f9063fff700 1 --2- 172.21.15.12:0/3296803358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b22e6ca70 0x555b22fee7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:32.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.171+0000 7f9068fb2700 1 -- 172.21.15.12:0/3296803358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b22feb1f0 msgr2=0x555b22fdff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:42:32.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.171+0000 7f9068fb2700 1 --2- 172.21.15.12:0/3296803358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b22feb1f0 0x555b22fdff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:32.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.171+0000 7f9068fb2700 1 -- 172.21.15.12:0/3296803358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b22e6ca70 msgr2=0x555b22fee7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:42:32.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.171+0000 7f9068fb2700 1 --2- 172.21.15.12:0/3296803358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b22e6ca70 0x555b22fee7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:32.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.171+0000 7f9068fb2700 1 -- 172.21.15.12:0/3296803358 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b22ff4bb0 con 0x555b22ff0b20 2024-05-17T22:42:32.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.171+0000 7f905b7fe700 1 --2- 172.21.15.12:0/3296803358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b22feb1f0 0x555b22fdff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:42:32.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.171+0000 7f9063fff700 1 --2- 172.21.15.12:0/3296803358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b22e6ca70 0x555b22fee7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:42:32.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.172+0000 7f9068fb2700 1 --2- 172.21.15.12:0/3296803358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b22ff0b20 0x555b23000250 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f905c000f80 tx=0x7f905c01c050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81b1e401851afd83 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:42:32.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.172+0000 7f9059ffb700 1 -- 172.21.15.12:0/3296803358 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905c003c00 con 0x555b22ff0b20 2024-05-17T22:42:32.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.172+0000 7f9059ffb700 1 -- 172.21.15.12:0/3296803358 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f905c014030 con 0x555b22ff0b20 2024-05-17T22:42:32.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.172+0000 7f9059ffb700 1 -- 172.21.15.12:0/3296803358 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905c007820 con 0x555b22ff0b20 2024-05-17T22:42:32.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.173+0000 7f90793f0b80 1 -- 172.21.15.12:0/3296803358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b22ff0b20 msgr2=0x555b23000250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:32.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.173+0000 7f90793f0b80 1 --2- 172.21.15.12:0/3296803358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b22ff0b20 0x555b23000250 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f905c000f80 tx=0x7f905c01c050 comp rx=0 tx=0).stop 2024-05-17T22:42:32.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.173+0000 7f90793f0b80 1 -- 172.21.15.12:0/3296803358 shutdown_connections 2024-05-17T22:42:32.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.173+0000 7f90793f0b80 1 --2- 172.21.15.12:0/3296803358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b22ff0b20 0x555b23000250 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:32.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.173+0000 7f90793f0b80 1 --2- 172.21.15.12:0/3296803358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b22e6ca70 0x555b22fee7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:32.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.173+0000 7f90793f0b80 1 --2- 172.21.15.12:0/3296803358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b22feb1f0 0x555b22fdff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:32.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.173+0000 7f90793f0b80 1 -- 172.21.15.12:0/3296803358 >> 172.21.15.12:0/3296803358 conn(0x555b22eb0f80 msgr2=0x555b22ff12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:42:32.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.174+0000 7f90793f0b80 1 -- 172.21.15.12:0/3296803358 shutdown_connections 2024-05-17T22:42:32.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.178+0000 7f90793f0b80 1 -- 172.21.15.12:0/3296803358 wait complete. 2024-05-17T22:42:32.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.178+0000 7f90793f0b80 1 -- start start 2024-05-17T22:42:32.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.178+0000 7f90793f0b80 1 Processor -- start 2024-05-17T22:42:32.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.179+0000 7f90793f0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b22e6ca70 0x555b22ffcbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:32.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.179+0000 7f90793f0b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b22feb1f0 0x555b22ff95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:32.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.179+0000 7f90793f0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b22ff0b20 0x555b22ff9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:32.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.179+0000 7f90793f0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f904400d1b0 con 0x555b22e6ca70 2024-05-17T22:42:32.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.179+0000 7f90793f0b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f904400e620 con 0x555b22feb1f0 2024-05-17T22:42:32.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.179+0000 7f90793f0b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f904400d880 con 0x555b22ff0b20 2024-05-17T22:42:32.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.179+0000 7f90793f0b80 10 client.0 _ll_register_callbacks cb 0x555b22ea3f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:42:32.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.182+0000 7f9068fb2700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b22ff0b20 0x555b22ff9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:32.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.182+0000 7f9068fb2700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b22ff0b20 0x555b22ff9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:58492/0 (socket says 172.21.15.12:58492) 2024-05-17T22:42:32.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.182+0000 7f9068fb2700 1 -- 172.21.15.12:0/2723957403 learned_addr learned my addr 172.21.15.12:0/2723957403 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:42:32.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.182+0000 7f905b7fe700 1 --2- 172.21.15.12:0/2723957403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b22feb1f0 0x555b22ff95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:32.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.182+0000 7f9068fb2700 1 -- 172.21.15.12:0/2723957403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b22feb1f0 msgr2=0x555b22ff95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:42:32.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.182+0000 7f9068fb2700 1 --2- 172.21.15.12:0/2723957403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b22feb1f0 0x555b22ff95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:32.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.182+0000 7f9068fb2700 1 -- 172.21.15.12:0/2723957403 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b22e6ca70 msgr2=0x555b22ffcbc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T22:42:32.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.182+0000 7f9068fb2700 1 --2- 172.21.15.12:0/2723957403 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b22e6ca70 0x555b22ffcbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:32.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.182+0000 7f9068fb2700 1 -- 172.21.15.12:0/2723957403 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b22ffa320 con 0x555b22ff0b20 2024-05-17T22:42:32.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.188+0000 7f9068fb2700 1 --2- 172.21.15.12:0/2723957403 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b22ff0b20 0x555b22ff9b80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f905c01a5e0 tx=0x7f905c003ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=faa41cd26e33dfa server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:42:32.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.188+0000 7f90437fe700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:42:32.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.188+0000 7f90437fe700 1 -- 172.21.15.12:0/2723957403 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905c018ae0 con 0x555b22ff0b20 2024-05-17T22:42:32.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.188+0000 7f90793f0b80 1 -- 172.21.15.12:0/2723957403 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f905c014030 con 0x555b22ff0b20 2024-05-17T22:42:32.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.188+0000 7f90437fe700 1 -- 172.21.15.12:0/2723957403 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f905c018c80 con 0x555b22ff0b20 2024-05-17T22:42:32.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.193+0000 7f90437fe700 1 -- 172.21.15.12:0/2723957403 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f905c0050d0 con 0x555b22ff0b20 2024-05-17T22:42:32.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.194+0000 7f90793f0b80 10 client.25273 fetch_fsmap learned FSMap version 32 2024-05-17T22:42:32.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.194+0000 7f90793f0b80 1 -- 172.21.15.12:0/2723957403 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555b23091d40 con 0x555b22ff0b20 2024-05-17T22:42:32.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.198+0000 7f90437fe700 1 -- 172.21.15.12:0/2723957403 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f905c014030 con 0x555b22ff0b20 2024-05-17T22:42:32.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.198+0000 7f90437fe700 1 -- 172.21.15.12:0/2723957403 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7f905c007610 con 0x555b22ff0b20 2024-05-17T22:42:32.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.198+0000 7f90793f0b80 10 client.25273 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T22:42:32.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.198+0000 7f90793f0b80 10 client.25273 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:42:32.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.198+0000 7f90793f0b80 20 client.25273 populate_metadata read hostname 'smithi012' 2024-05-17T22:42:32.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.198+0000 7f90793f0b80 1 --2- 172.21.15.12:0/2723957403 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555b230920f0 0x555b23094510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:32.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.198+0000 7f90793f0b80 4 client.25273 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-17T22:42:32.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.198+0000 7f90793f0b80 1 -- 172.21.15.12:0/2723957403 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555b22f14a40 con 0x555b230920f0 2024-05-17T22:42:32.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.198+0000 7f9063fff700 1 --2- 172.21.15.12:0/2723957403 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555b230920f0 0x555b23094510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:32.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.199+0000 7f9063fff700 1 --2- 172.21.15.12:0/2723957403 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555b230920f0 0x555b23094510 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dea4ab2c9f4fa25 server_cookie=e26a35de4c47e68d in_seq=0 out_seq=0 2024-05-17T22:42:32.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.199+0000 7f90437fe700 10 client.25273 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:42:32.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.200+0000 7f90437fe700 1 -- 172.21.15.12:0/2723957403 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555b22f14a40 con 0x555b230920f0 2024-05-17T22:42:32.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.200+0000 7f90437fe700 10 client.25273 handle_command_reply: tid=0 2024-05-17T22:42:32.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.200+0000 7f90793f0b80 1 client.25273 shutdown 2024-05-17T22:42:32.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.200+0000 7f90793f0b80 1 -- 172.21.15.12:0/2723957403 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b22ff0b20 msgr2=0x555b22ff9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:32.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.200+0000 7f90793f0b80 1 --2- 172.21.15.12:0/2723957403 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b22ff0b20 0x555b22ff9b80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f905c01a5e0 tx=0x7f905c003ab0 comp rx=0 tx=0).stop 2024-05-17T22:42:32.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.205+0000 7f90793f0b80 1 -- 172.21.15.12:0/2723957403 shutdown_connections 2024-05-17T22:42:32.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.205+0000 7f90793f0b80 1 --2- 172.21.15.12:0/2723957403 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b22ff0b20 0x555b22ff9b80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:32.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.205+0000 7f90793f0b80 1 --2- 172.21.15.12:0/2723957403 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555b230920f0 0x555b23094510 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:32.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.205+0000 7f90793f0b80 1 --2- 172.21.15.12:0/2723957403 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b22e6ca70 0x555b22ffcbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:32.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.205+0000 7f90793f0b80 1 --2- 172.21.15.12:0/2723957403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b22feb1f0 0x555b22ff95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:32.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.205+0000 7f90437fe700 0 client.25273 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:42:32.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.205+0000 7f90793f0b80 1 -- 172.21.15.12:0/2723957403 >> 172.21.15.12:0/2723957403 conn(0x555b22eb0f80 msgr2=0x555b22ff4330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:42:32.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.206+0000 7f90793f0b80 1 -- 172.21.15.12:0/2723957403 shutdown_connections 2024-05-17T22:42:32.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.206+0000 7f90793f0b80 1 -- 172.21.15.12:0/2723957403 wait complete. 2024-05-17T22:42:32.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.206+0000 7f90793f0b80 20 client.25273 trim_cache size 0 max 16384 2024-05-17T22:42:32.212 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:42:32.212 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-17T22:42:32.212 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "9742bd65-6100-4d0d-a9cd-09f6f1512b65", 2024-05-17T22:42:32.212 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-17T22:42:32.212 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:42:32.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.210+0000 7f906aa15700 1 -- 172.21.15.12:0/3404496663 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f90500b2900 msgr2=0x7f90500b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:32.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.210+0000 7f906aa15700 1 --2- 172.21.15.12:0/3404496663 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f90500b2900 0x7f90500b4e10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9064107620 tx=0x7f904c006cd0 comp rx=0 tx=0).stop 2024-05-17T22:42:32.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.211+0000 7f906aa15700 1 -- 172.21.15.12:0/3404496663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9064102580 msgr2=0x7f906410a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:32.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.211+0000 7f906aa15700 1 --2- 172.21.15.12:0/3404496663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9064102580 0x7f906410a6e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f905400db70 tx=0x7f905400de80 comp rx=0 tx=0).stop 2024-05-17T22:42:32.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.215+0000 7f906aa15700 1 -- 172.21.15.12:0/3404496663 shutdown_connections 2024-05-17T22:42:32.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.215+0000 7f906aa15700 1 --2- 172.21.15.12:0/3404496663 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f90500b2900 0x7f90500b4e10 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:32.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.215+0000 7f906aa15700 1 --2- 172.21.15.12:0/3404496663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9064107800 0x7f9064116d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:32.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.215+0000 7f906aa15700 1 --2- 172.21.15.12:0/3404496663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9064102580 0x7f906410a6e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:32.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.215+0000 7f906aa15700 1 --2- 172.21.15.12:0/3404496663 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9064106eb0 0x7f906410ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:32.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.215+0000 7f906aa15700 1 -- 172.21.15.12:0/3404496663 >> 172.21.15.12:0/3404496663 conn(0x7f90640fe030 msgr2=0x7f90640fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:42:32.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.218+0000 7f906aa15700 1 -- 172.21.15.12:0/3404496663 shutdown_connections 2024-05-17T22:42:32.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:32.218+0000 7f906aa15700 1 -- 172.21.15.12:0/3404496663 wait complete. 2024-05-17T22:42:33.111 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-17T22:42:33.545 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:42:33.992 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:33 smithi012 ceph-mon[31632]: scrub queued for path: / 2024-05-17T22:42:33.992 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:33 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:42:33.992 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:33 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:42:33.992 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:33 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:42:33.992 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:33 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:42:33.992 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:33 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:42:33.992 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:33 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:42:33.992 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:33 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:42:33.992 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:33 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:42:33.992 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:33 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:42:33.992 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:33 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:42:33.992 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:33 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:42:33.993 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:33 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:42:33.993 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:33 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:42:33.993 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:33 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:42:33.993 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:33 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:42:33.993 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:33 smithi012 ceph-mon[31632]: pgmap v538: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 17 MiB/s wr, 1.03k op/s 2024-05-17T22:42:34.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:33 smithi096 ceph-mon[36065]: scrub queued for path: / 2024-05-17T22:42:34.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:33 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:42:34.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:33 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:42:34.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:33 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:42:34.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:33 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:42:34.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:33 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:42:34.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:33 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:42:34.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:33 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:42:34.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:33 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:42:34.049 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:33 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:42:34.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:33 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:42:34.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:33 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:42:34.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:33 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:42:34.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:33 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:42:34.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:33 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:42:34.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:33 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:42:34.050 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:33 smithi096 ceph-mon[36065]: pgmap v538: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 17 MiB/s wr, 1.03k op/s 2024-05-17T22:42:34.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:33 smithi151 ceph-mon[36101]: scrub queued for path: / 2024-05-17T22:42:34.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:33 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:42:34.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:33 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:42:34.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:33 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:42:34.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:33 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:42:34.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:33 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:42:34.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:33 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:42:34.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:33 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:42:34.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:33 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:42:34.242 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:33 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:42:34.242 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:33 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:42:34.242 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:33 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:42:34.242 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:33 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:42:34.242 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:33 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:42:34.242 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:33 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:42:34.242 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:33 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:42:34.242 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:33 smithi151 ceph-mon[36101]: pgmap v538: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 17 MiB/s wr, 1.03k op/s 2024-05-17T22:42:35.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.914+0000 7ff2fa142700 1 -- 172.21.15.12:0/2397832050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2f410aac0 msgr2=0x7ff2f410aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:35.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.914+0000 7ff2fa142700 1 --2- 172.21.15.12:0/2397832050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2f410aac0 0x7ff2f410aed0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff2f00099c0 tx=0x7ff2f0009cd0 comp rx=0 tx=0).stop 2024-05-17T22:42:35.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.914+0000 7ff2fa142700 1 -- 172.21.15.12:0/2397832050 shutdown_connections 2024-05-17T22:42:35.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.914+0000 7ff2fa142700 1 --2- 172.21.15.12:0/2397832050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2f410f9f0 0x7ff2f4117350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:35.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.914+0000 7ff2fa142700 1 --2- 172.21.15.12:0/2397832050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2f410aac0 0x7ff2f410aed0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:35.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.914+0000 7ff2fa142700 1 --2- 172.21.15.12:0/2397832050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2f410f0a0 0x7ff2f410f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:35.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.914+0000 7ff2fa142700 1 -- 172.21.15.12:0/2397832050 >> 172.21.15.12:0/2397832050 conn(0x7ff2f406f1f0 msgr2=0x7ff2f406f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:42:35.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.915+0000 7ff2fa142700 1 -- 172.21.15.12:0/2397832050 shutdown_connections 2024-05-17T22:42:35.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.915+0000 7ff2fa142700 1 -- 172.21.15.12:0/2397832050 wait complete. 2024-05-17T22:42:35.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.917+0000 7ff2fa142700 1 Processor -- start 2024-05-17T22:42:35.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.917+0000 7ff2fa142700 1 -- start start 2024-05-17T22:42:35.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.918+0000 7ff2fa142700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2f410aac0 0x7ff2f41a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:35.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.918+0000 7ff2fa142700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2f410f0a0 0x7ff2f41a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:35.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.918+0000 7ff2fa142700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2f410f9f0 0x7ff2f41a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:35.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.918+0000 7ff2fa142700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f411a340 con 0x7ff2f410f9f0 2024-05-17T22:42:35.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.918+0000 7ff2fa142700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f411a1c0 con 0x7ff2f410aac0 2024-05-17T22:42:35.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.918+0000 7ff2fa142700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f411a4c0 con 0x7ff2f410f0a0 2024-05-17T22:42:35.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.919+0000 7ff2f9941700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2f410f9f0 0x7ff2f41a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:35.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.919+0000 7ff2f9941700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2f410f9f0 0x7ff2f41a0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36570/0 (socket says 172.21.15.12:36570) 2024-05-17T22:42:35.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.919+0000 7ff2f9941700 1 -- 172.21.15.12:0/951359013 learned_addr learned my addr 172.21.15.12:0/951359013 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:42:35.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.919+0000 7ff2f9140700 1 --2- 172.21.15.12:0/951359013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2f410aac0 0x7ff2f41a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:35.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.920+0000 7ff2f9941700 1 -- 172.21.15.12:0/951359013 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2f410f0a0 msgr2=0x7ff2f41a6e10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-17T22:42:35.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.920+0000 7ff2f9941700 1 --2- 172.21.15.12:0/951359013 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2f410f0a0 0x7ff2f41a6e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:35.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.920+0000 7ff2f9941700 1 -- 172.21.15.12:0/951359013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2f410aac0 msgr2=0x7ff2f41a68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:35.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.920+0000 7ff2f9941700 1 --2- 172.21.15.12:0/951359013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2f410aac0 0x7ff2f41a68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:35.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.920+0000 7ff2f9941700 1 -- 172.21.15.12:0/951359013 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2f41a11d0 con 0x7ff2f410f9f0 2024-05-17T22:42:35.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.920+0000 7ff2f9941700 1 --2- 172.21.15.12:0/951359013 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2f410f9f0 0x7ff2f41a0af0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00b970 tx=0x7ff2ec00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:42:35.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.921+0000 7ff2ea7fc700 1 -- 172.21.15.12:0/951359013 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec019070 con 0x7ff2f410f9f0 2024-05-17T22:42:35.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.921+0000 7ff2ea7fc700 1 -- 172.21.15.12:0/951359013 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2ec014440 con 0x7ff2f410f9f0 2024-05-17T22:42:35.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.921+0000 7ff2ea7fc700 1 -- 172.21.15.12:0/951359013 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec00ce60 con 0x7ff2f410f9f0 2024-05-17T22:42:35.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.921+0000 7ff2fa142700 1 -- 172.21.15.12:0/951359013 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2f41a14c0 con 0x7ff2f410f9f0 2024-05-17T22:42:35.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.921+0000 7ff2fa142700 1 -- 172.21.15.12:0/951359013 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2f41ad760 con 0x7ff2f410f9f0 2024-05-17T22:42:35.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.929+0000 7ff2ea7fc700 1 -- 172.21.15.12:0/951359013 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff2ec01c070 con 0x7ff2f410f9f0 2024-05-17T22:42:35.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.929+0000 7ff309d7fb80 1 Processor -- start 2024-05-17T22:42:35.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.930+0000 7ff309d7fb80 1 -- start start 2024-05-17T22:42:35.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.930+0000 7ff309d7fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555dc09c19f0 0x555dc0b45770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:35.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.930+0000 7ff309d7fb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555dc0b45cb0 0x555dc0b3fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:35.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.930+0000 7ff309d7fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc0b40170 0x555dc0b40580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:35.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.930+0000 7ff309d7fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x555dc0a6e290 con 0x555dc0b40170 2024-05-17T22:42:35.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.930+0000 7ff309d7fb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555dc0a5dfd0 con 0x555dc0b45cb0 2024-05-17T22:42:35.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.930+0000 7ff309d7fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555dc0a61e70 con 0x555dc09c19f0 2024-05-17T22:42:35.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.930+0000 7ff2f893f700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555dc09c19f0 0x555dc0b45770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:35.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.930+0000 7ff2f893f700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555dc09c19f0 0x555dc0b45770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:58518/0 (socket says 172.21.15.12:58518) 2024-05-17T22:42:35.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.930+0000 7ff2f893f700 1 -- 172.21.15.12:0/3488019944 learned_addr learned my addr 172.21.15.12:0/3488019944 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:42:35.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.930+0000 7ff2f893f700 1 --2- 172.21.15.12:0/3488019944 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc0b40170 0x555dc0b40580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:35.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.930+0000 7ff2f9140700 1 --2- 172.21.15.12:0/3488019944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555dc0b45cb0 0x555dc0b3fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:35.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.931+0000 7ff2f9140700 1 -- 172.21.15.12:0/3488019944 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555dc09c19f0 msgr2=0x555dc0b45770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:42:35.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.931+0000 7ff2f9140700 1 --2- 172.21.15.12:0/3488019944 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555dc09c19f0 0x555dc0b45770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:35.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.931+0000 7ff2f9140700 1 -- 172.21.15.12:0/3488019944 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc0b40170 msgr2=0x555dc0b40580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:42:35.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.931+0000 7ff2f9140700 1 --2- 172.21.15.12:0/3488019944 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc0b40170 0x555dc0b40580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:35.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.931+0000 7ff2f9140700 1 -- 172.21.15.12:0/3488019944 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dc0a07590 con 0x555dc0b45cb0 2024-05-17T22:42:35.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.931+0000 7ff2f893f700 1 --2- 172.21.15.12:0/3488019944 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555dc09c19f0 0x555dc0b45770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:42:35.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.931+0000 7ff2f9140700 1 --2- 172.21.15.12:0/3488019944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555dc0b45cb0 0x555dc0b3fc30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0006010 tx=0x7ff2f0003810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f9c52ace1803826 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:42:35.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.931+0000 7ff2df7fe700 1 -- 172.21.15.12:0/3488019944 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0003ad0 con 0x555dc0b45cb0 2024-05-17T22:42:35.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.932+0000 7ff2df7fe700 1 -- 172.21.15.12:0/3488019944 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2f00055c0 con 0x555dc0b45cb0 2024-05-17T22:42:35.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.932+0000 7ff2df7fe700 1 -- 172.21.15.12:0/3488019944 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f000be40 con 0x555dc0b45cb0 2024-05-17T22:42:35.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.932+0000 7ff2ea7fc700 1 --2- 172.21.15.12:0/951359013 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff2e00b4ca0 0x7ff2e00b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:35.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.932+0000 7ff2f893f700 1 --2- 172.21.15.12:0/951359013 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff2e00b4ca0 0x7ff2e00b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:35.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.932+0000 7ff2ea7fc700 1 -- 172.21.15.12:0/951359013 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7ff2ec0e1990 con 0x7ff2f410f9f0 2024-05-17T22:42:35.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.933+0000 7ff309d7fb80 1 -- 172.21.15.12:0/3488019944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555dc0b45cb0 msgr2=0x555dc0b3fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:35.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.933+0000 7ff309d7fb80 1 --2- 172.21.15.12:0/3488019944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555dc0b45cb0 0x555dc0b3fc30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0006010 tx=0x7ff2f0003810 comp rx=0 tx=0).stop 2024-05-17T22:42:35.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.933+0000 7ff2f893f700 1 --2- 172.21.15.12:0/951359013 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff2e00b4ca0 0x7ff2e00b71b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x555dc0b3bae0 tx=0x7ff2e400f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:42:35.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.933+0000 7ff309d7fb80 1 -- 172.21.15.12:0/3488019944 shutdown_connections 2024-05-17T22:42:35.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.933+0000 7ff309d7fb80 1 --2- 172.21.15.12:0/3488019944 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc0b40170 0x555dc0b40580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:35.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.933+0000 7ff309d7fb80 1 --2- 172.21.15.12:0/3488019944 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555dc09c19f0 0x555dc0b45770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:35.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.933+0000 7ff309d7fb80 1 --2- 172.21.15.12:0/3488019944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555dc0b45cb0 0x555dc0b3fc30 secure :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0006010 tx=0x7ff2f0003810 comp rx=0 tx=0).stop 2024-05-17T22:42:35.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.933+0000 7ff309d7fb80 1 -- 172.21.15.12:0/3488019944 >> 172.21.15.12:0/3488019944 conn(0x555dc0a05f00 msgr2=0x555dc0b3ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:42:35.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.933+0000 7ff309d7fb80 1 -- 172.21.15.12:0/3488019944 shutdown_connections 2024-05-17T22:42:35.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.933+0000 7ff309d7fb80 1 -- 172.21.15.12:0/3488019944 wait complete. 2024-05-17T22:42:35.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.934+0000 7ff309d7fb80 1 -- start start 2024-05-17T22:42:35.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.934+0000 7ff309d7fb80 1 Processor -- start 2024-05-17T22:42:35.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.935+0000 7ff309d7fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555dc09c19f0 0x555dc0b49430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:35.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.935+0000 7ff309d7fb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555dc0b40170 0x555dc0b49970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:35.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.935+0000 7ff309d7fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc0b49eb0 0x555dc0b4f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:35.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.935+0000 7ff309d7fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x555dc0965150 con 0x555dc0b49eb0 2024-05-17T22:42:35.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.935+0000 7ff309d7fb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555dc09fee80 con 0x555dc0b40170 2024-05-17T22:42:35.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.935+0000 7ff309d7fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555dc0b4c5a0 con 0x555dc09c19f0 2024-05-17T22:42:35.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.935+0000 7ff309d7fb80 10 client.0 _ll_register_callbacks cb 0x555dc09f8f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:42:35.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.939+0000 7ff2f9941700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc0b49eb0 0x555dc0b4f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:35.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.939+0000 7ff2f9941700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc0b49eb0 0x555dc0b4f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36578/0 (socket says 172.21.15.12:36578) 2024-05-17T22:42:35.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.939+0000 7ff2f9941700 1 -- 172.21.15.12:0/1595557971 learned_addr learned my addr 172.21.15.12:0/1595557971 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:42:35.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.940+0000 7ff2f9941700 1 -- 172.21.15.12:0/1595557971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555dc09c19f0 msgr2=0x555dc0b49430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T22:42:35.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.940+0000 7ff2f9941700 1 --2- 172.21.15.12:0/1595557971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555dc09c19f0 0x555dc0b49430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:35.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.940+0000 7ff2f9941700 1 -- 172.21.15.12:0/1595557971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555dc0b40170 msgr2=0x555dc0b49970 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T22:42:35.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.940+0000 7ff2f9941700 1 --2- 172.21.15.12:0/1595557971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555dc0b40170 0x555dc0b49970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:35.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.940+0000 7ff2f9941700 1 -- 172.21.15.12:0/1595557971 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dc09ae480 con 0x555dc0b49eb0 2024-05-17T22:42:35.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.940+0000 7ff2f9941700 1 --2- 172.21.15.12:0/1595557971 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc0b49eb0 0x555dc0b4f870 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec0a4c40 tx=0x7ff2ec014b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6869692a9f4ad921 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:42:35.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.940+0000 7ff2dcff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:42:35.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.941+0000 7ff2dcff9700 1 -- 172.21.15.12:0/1595557971 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec0a8bc0 con 0x555dc0b49eb0 2024-05-17T22:42:35.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.941+0000 7ff309d7fb80 1 -- 172.21.15.12:0/1595557971 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2f00055c0 con 0x555dc0b49eb0 2024-05-17T22:42:35.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.942+0000 7ff2dcff9700 1 -- 172.21.15.12:0/1595557971 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7ff2ec0b17a0 con 0x555dc0b49eb0 2024-05-17T22:42:35.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.942+0000 7ff2dcff9700 1 -- 172.21.15.12:0/1595557971 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff2ec0acf70 con 0x555dc0b49eb0 2024-05-17T22:42:35.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.942+0000 7ff309d7fb80 10 client.15663 fetch_fsmap learned FSMap version 33 2024-05-17T22:42:35.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.942+0000 7ff309d7fb80 1 -- 172.21.15.12:0/1595557971 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555dc0b515e0 con 0x555dc0b49eb0 2024-05-17T22:42:35.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.943+0000 7ff2dcff9700 1 -- 172.21.15.12:0/1595557971 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2f00055c0 con 0x555dc0b49eb0 2024-05-17T22:42:35.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.943+0000 7ff2dcff9700 1 -- 172.21.15.12:0/1595557971 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7ff2ec0b1040 con 0x555dc0b49eb0 2024-05-17T22:42:35.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.945+0000 7ff309d7fb80 10 client.15663 fetch_fsmap finished waiting for FSMap version 33 2024-05-17T22:42:35.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.945+0000 7ff309d7fb80 10 client.15663 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:42:35.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.945+0000 7ff309d7fb80 20 client.15663 populate_metadata read hostname 'smithi012' 2024-05-17T22:42:35.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.945+0000 7ff309d7fb80 1 --2- 172.21.15.12:0/1595557971 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555dc0b53fe0 0x555dc0b564d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:35.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.945+0000 7ff309d7fb80 4 client.15663 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:42:35.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.945+0000 7ff309d7fb80 1 -- 172.21.15.12:0/1595557971 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555dc0a701e0 con 0x555dc0b53fe0 2024-05-17T22:42:35.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.946+0000 7ff2f9140700 1 --2- 172.21.15.12:0/1595557971 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555dc0b53fe0 0x555dc0b564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:35.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.946+0000 7ff2dcff9700 10 client.15663 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:42:35.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.946+0000 7ff2f9140700 1 --2- 172.21.15.12:0/1595557971 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555dc0b53fe0 0x555dc0b564d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62317b75347c3bc9 server_cookie=48f195d8b5846b80 in_seq=0 out_seq=0 2024-05-17T22:42:35.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.947+0000 7ff2dcff9700 1 -- 172.21.15.12:0/1595557971 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555dc0a701e0 con 0x555dc0b53fe0 2024-05-17T22:42:35.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.947+0000 7ff2dcff9700 10 client.15663 handle_command_reply: tid=0 2024-05-17T22:42:35.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.947+0000 7ff309d7fb80 1 client.15663 shutdown 2024-05-17T22:42:35.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.948+0000 7ff309d7fb80 1 -- 172.21.15.12:0/1595557971 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc0b49eb0 msgr2=0x555dc0b4f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:35.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.948+0000 7ff309d7fb80 1 --2- 172.21.15.12:0/1595557971 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc0b49eb0 0x555dc0b4f870 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec0a4c40 tx=0x7ff2ec014b40 comp rx=0 tx=0).stop 2024-05-17T22:42:35.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.948+0000 7ff309d7fb80 1 -- 172.21.15.12:0/1595557971 shutdown_connections 2024-05-17T22:42:35.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.948+0000 7ff309d7fb80 1 --2- 172.21.15.12:0/1595557971 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555dc0b53fe0 0x555dc0b564d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:35.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.948+0000 7ff309d7fb80 1 --2- 172.21.15.12:0/1595557971 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc0b49eb0 0x555dc0b4f870 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:35.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.948+0000 7ff309d7fb80 1 --2- 172.21.15.12:0/1595557971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555dc09c19f0 0x555dc0b49430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:35.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.948+0000 7ff309d7fb80 1 --2- 172.21.15.12:0/1595557971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555dc0b40170 0x555dc0b49970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:35.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.948+0000 7ff309d7fb80 1 -- 172.21.15.12:0/1595557971 >> 172.21.15.12:0/1595557971 conn(0x555dc0a05f00 msgr2=0x555dc0b3a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:42:35.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.948+0000 7ff2dcff9700 0 client.15663 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:42:35.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.949+0000 7ff309d7fb80 1 -- 172.21.15.12:0/1595557971 shutdown_connections 2024-05-17T22:42:35.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.949+0000 7ff309d7fb80 1 -- 172.21.15.12:0/1595557971 wait complete. 2024-05-17T22:42:35.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.949+0000 7ff309d7fb80 20 client.15663 trim_cache size 0 max 16384 2024-05-17T22:42:35.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.986+0000 7ff309d7fb80 1 Processor -- start 2024-05-17T22:42:35.989 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:35 smithi012 ceph-mon[31632]: pgmap v539: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 1.01k op/s 2024-05-17T22:42:35.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.986+0000 7ff309d7fb80 1 -- start start 2024-05-17T22:42:35.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.987+0000 7ff309d7fb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555dc09c19f0 0x555dc0b3fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:35.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.987+0000 7ff309d7fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc0b40170 0x555dc0b437c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:35.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.987+0000 7ff2f9140700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555dc09c19f0 0x555dc0b3fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:35.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.987+0000 7ff2f9140700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555dc09c19f0 0x555dc0b3fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:35116/0 (socket says 172.21.15.12:35116) 2024-05-17T22:42:35.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.988+0000 7ff309d7fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555dc0b49eb0 0x555dc0b3a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:35.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.988+0000 7ff309d7fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f41b8ac0 con 0x555dc0b40170 2024-05-17T22:42:35.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.988+0000 7ff2f9140700 1 -- 172.21.15.12:0/530638577 learned_addr learned my addr 172.21.15.12:0/530638577 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:42:35.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.988+0000 7ff309d7fb80 1 -- 172.21.15.12:0/530638577 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f41b62f0 con 0x555dc09c19f0 2024-05-17T22:42:35.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.988+0000 7ff309d7fb80 1 -- 172.21.15.12:0/530638577 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f41b4140 con 0x555dc0b49eb0 2024-05-17T22:42:35.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.989+0000 7ff2f893f700 1 --2- 172.21.15.12:0/530638577 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc0b40170 0x555dc0b437c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:35.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.989+0000 7ff2f9140700 1 -- 172.21.15.12:0/530638577 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555dc0b49eb0 msgr2=0x555dc0b3a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:42:35.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.989+0000 7ff2f9140700 1 --2- 172.21.15.12:0/530638577 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555dc0b49eb0 0x555dc0b3a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:35.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.989+0000 7ff2f9140700 1 -- 172.21.15.12:0/530638577 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc0b40170 msgr2=0x555dc0b437c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:42:35.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.989+0000 7ff2f9140700 1 --2- 172.21.15.12:0/530638577 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc0b40170 0x555dc0b437c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:35.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.989+0000 7ff2f9140700 1 -- 172.21.15.12:0/530638577 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dc0b4bc30 con 0x555dc09c19f0 2024-05-17T22:42:35.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.989+0000 7ff2f9140700 1 --2- 172.21.15.12:0/530638577 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555dc09c19f0 0x555dc0b3fc30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0000c00 tx=0x7ff2f0009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66194d8275fbe032 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:42:35.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.989+0000 7ff2deffd700 1 -- 172.21.15.12:0/530638577 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0033070 con 0x555dc09c19f0 2024-05-17T22:42:35.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.990+0000 7ff2deffd700 1 -- 172.21.15.12:0/530638577 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2f000be80 con 0x555dc09c19f0 2024-05-17T22:42:35.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.990+0000 7ff2deffd700 1 -- 172.21.15.12:0/530638577 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0027040 con 0x555dc09c19f0 2024-05-17T22:42:35.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.991+0000 7ff309d7fb80 1 -- 172.21.15.12:0/530638577 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555dc09c19f0 msgr2=0x555dc0b3fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:35.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.991+0000 7ff309d7fb80 1 --2- 172.21.15.12:0/530638577 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555dc09c19f0 0x555dc0b3fc30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0000c00 tx=0x7ff2f0009310 comp rx=0 tx=0).stop 2024-05-17T22:42:35.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.991+0000 7ff309d7fb80 1 -- 172.21.15.12:0/530638577 shutdown_connections 2024-05-17T22:42:35.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.991+0000 7ff309d7fb80 1 --2- 172.21.15.12:0/530638577 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555dc0b49eb0 0x555dc0b3a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:35.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.991+0000 7ff309d7fb80 1 --2- 172.21.15.12:0/530638577 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555dc09c19f0 0x555dc0b3fc30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:35.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.991+0000 7ff309d7fb80 1 --2- 172.21.15.12:0/530638577 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc0b40170 0x555dc0b437c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:35.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.991+0000 7ff309d7fb80 1 -- 172.21.15.12:0/530638577 >> 172.21.15.12:0/530638577 conn(0x555dc0a05f00 msgr2=0x555dc0b430b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:42:35.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.991+0000 7ff309d7fb80 1 -- 172.21.15.12:0/530638577 shutdown_connections 2024-05-17T22:42:35.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.991+0000 7ff309d7fb80 1 -- 172.21.15.12:0/530638577 wait complete. 2024-05-17T22:42:35.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.991+0000 7ff309d7fb80 1 -- start start 2024-05-17T22:42:35.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.992+0000 7ff309d7fb80 1 Processor -- start 2024-05-17T22:42:35.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.992+0000 7ff309d7fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc09c19f0 0x555dc0b4fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:35.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.992+0000 7ff309d7fb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555dc0b40170 0x555dc0b5bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:35.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.992+0000 7ff309d7fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555dc0b49eb0 0x555dc0b5c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:35.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.992+0000 7ff2f893f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555dc0b40170 0x555dc0b5bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:35.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.992+0000 7ff2f893f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555dc0b40170 0x555dc0b5bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:35130/0 (socket says 172.21.15.12:35130) 2024-05-17T22:42:35.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.993+0000 7ff2f893f700 1 -- 172.21.15.12:0/3503706340 learned_addr learned my addr 172.21.15.12:0/3503706340 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:42:35.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.993+0000 7ff309d7fb80 1 -- 172.21.15.12:0/3503706340 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f41b47b0 con 0x555dc09c19f0 2024-05-17T22:42:35.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.993+0000 7ff2f9140700 1 --2- 172.21.15.12:0/3503706340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc09c19f0 0x555dc0b4fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:35.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.993+0000 7ff309d7fb80 1 -- 172.21.15.12:0/3503706340 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f41b5c20 con 0x555dc0b40170 2024-05-17T22:42:35.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.993+0000 7ff2f9941700 1 --2- 172.21.15.12:0/3503706340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555dc0b49eb0 0x555dc0b5c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:35.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.993+0000 7ff309d7fb80 1 -- 172.21.15.12:0/3503706340 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f41b4e80 con 0x555dc0b49eb0 2024-05-17T22:42:35.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.993+0000 7ff309d7fb80 10 client.0 _ll_register_callbacks cb 0x555dc09f8f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:42:35.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.993+0000 7ff2f893f700 1 -- 172.21.15.12:0/3503706340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555dc0b49eb0 msgr2=0x555dc0b5c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:42:35.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.993+0000 7ff2f893f700 1 --2- 172.21.15.12:0/3503706340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555dc0b49eb0 0x555dc0b5c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:35.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.993+0000 7ff2f893f700 1 -- 172.21.15.12:0/3503706340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc09c19f0 msgr2=0x555dc0b4fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:42:35.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.993+0000 7ff2f893f700 1 --2- 172.21.15.12:0/3503706340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc09c19f0 0x555dc0b4fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:35.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.993+0000 7ff2f893f700 1 -- 172.21.15.12:0/3503706340 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dc0b4e5f0 con 0x555dc0b40170 2024-05-17T22:42:35.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.993+0000 7ff2f9941700 1 --2- 172.21.15.12:0/3503706340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555dc0b49eb0 0x555dc0b5c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:42:35.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.993+0000 7ff2f893f700 1 --2- 172.21.15.12:0/3503706340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555dc0b40170 0x555dc0b5bcb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x555dc0b4abf0 tx=0x7ff2e4006960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=711d9af7d8dd6ff6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:42:35.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.993+0000 7ff2dcff9700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:42:35.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.994+0000 7ff2dcff9700 1 -- 172.21.15.12:0/3503706340 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e402a060 con 0x555dc0b40170 2024-05-17T22:42:35.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.994+0000 7ff2dcff9700 1 -- 172.21.15.12:0/3503706340 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7ff2e4018830 con 0x555dc0b40170 2024-05-17T22:42:35.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.994+0000 7ff309d7fb80 1 -- 172.21.15.12:0/3503706340 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555dc0a699c0 con 0x555dc0b40170 2024-05-17T22:42:35.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.994+0000 7ff2dcff9700 1 -- 172.21.15.12:0/3503706340 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff2e4031050 con 0x555dc0b40170 2024-05-17T22:42:35.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.994+0000 7ff309d7fb80 10 client.25328 fetch_fsmap learned FSMap version 33 2024-05-17T22:42:35.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.994+0000 7ff309d7fb80 1 -- 172.21.15.12:0/3503706340 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555dc0bd53c0 con 0x555dc0b40170 2024-05-17T22:42:35.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.994+0000 7ff2dcff9700 1 -- 172.21.15.12:0/3503706340 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x555dc0a699c0 con 0x555dc0b40170 2024-05-17T22:42:35.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.994+0000 7ff2dcff9700 1 -- 172.21.15.12:0/3503706340 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7ff2e4025a80 con 0x555dc0b40170 2024-05-17T22:42:35.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.994+0000 7ff309d7fb80 10 client.25328 fetch_fsmap finished waiting for FSMap version 33 2024-05-17T22:42:35.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.994+0000 7ff309d7fb80 10 client.25328 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:42:36.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.994+0000 7ff309d7fb80 20 client.25328 populate_metadata read hostname 'smithi012' 2024-05-17T22:42:36.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.994+0000 7ff309d7fb80 1 --2- 172.21.15.12:0/3503706340 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555dc0bd5b30 0x555dc0bd7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:42:36.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.995+0000 7ff309d7fb80 4 client.25328 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:42:36.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.995+0000 7ff309d7fb80 1 -- 172.21.15.12:0/3503706340 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555dc093a360 con 0x555dc0bd5b30 2024-05-17T22:42:36.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.995+0000 7ff2f9140700 1 --2- 172.21.15.12:0/3503706340 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555dc0bd5b30 0x555dc0bd7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:42:36.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.995+0000 7ff2dcff9700 10 client.25328 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:42:36.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.995+0000 7ff2f9140700 1 --2- 172.21.15.12:0/3503706340 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555dc0bd5b30 0x555dc0bd7f50 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4340d28d1c7c4f57 server_cookie=abc6307a121b2ea7 in_seq=0 out_seq=0 2024-05-17T22:42:36.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.996+0000 7ff2dcff9700 1 -- 172.21.15.12:0/3503706340 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555dc093a360 con 0x555dc0bd5b30 2024-05-17T22:42:36.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.996+0000 7ff2dcff9700 10 client.25328 handle_command_reply: tid=0 2024-05-17T22:42:36.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.996+0000 7ff309d7fb80 1 client.25328 shutdown 2024-05-17T22:42:36.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.996+0000 7ff309d7fb80 1 -- 172.21.15.12:0/3503706340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555dc0b40170 msgr2=0x555dc0b5bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:36.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.996+0000 7ff309d7fb80 1 --2- 172.21.15.12:0/3503706340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555dc0b40170 0x555dc0b5bcb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x555dc0b4abf0 tx=0x7ff2e4006960 comp rx=0 tx=0).stop 2024-05-17T22:42:36.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.996+0000 7ff309d7fb80 1 -- 172.21.15.12:0/3503706340 shutdown_connections 2024-05-17T22:42:36.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.996+0000 7ff309d7fb80 1 --2- 172.21.15.12:0/3503706340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555dc0b49eb0 0x555dc0b5c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:36.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.996+0000 7ff309d7fb80 1 --2- 172.21.15.12:0/3503706340 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555dc0bd5b30 0x555dc0bd7f50 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:36.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.996+0000 7ff309d7fb80 1 --2- 172.21.15.12:0/3503706340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555dc09c19f0 0x555dc0b4fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:36.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.996+0000 7ff309d7fb80 1 --2- 172.21.15.12:0/3503706340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555dc0b40170 0x555dc0b5bcb0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:36.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.996+0000 7ff309d7fb80 1 -- 172.21.15.12:0/3503706340 >> 172.21.15.12:0/3503706340 conn(0x555dc0a05f00 msgr2=0x555dc0b4ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:42:36.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.996+0000 7ff2dcff9700 0 client.25328 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:42:36.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.996+0000 7ff309d7fb80 1 -- 172.21.15.12:0/3503706340 shutdown_connections 2024-05-17T22:42:36.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.996+0000 7ff309d7fb80 1 -- 172.21.15.12:0/3503706340 wait complete. 2024-05-17T22:42:36.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.997+0000 7ff309d7fb80 20 client.25328 trim_cache size 0 max 16384 2024-05-17T22:42:36.003 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:42:36.003 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-17T22:42:36.003 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-17T22:42:36.003 INFO:teuthology.orchestra.run.smithi012.stdout: "9742bd65-6100-4d0d-a9cd-09f6f1512b65": { 2024-05-17T22:42:36.003 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-17T22:42:36.003 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "9742bd65-6100-4d0d-a9cd-09f6f1512b65", 2024-05-17T22:42:36.003 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-17T22:42:36.003 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:42:36.003 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:42:36.003 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:42:36.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.999+0000 7ff2fa142700 1 -- 172.21.15.12:0/951359013 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff2e00b4ca0 msgr2=0x7ff2e00b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:36.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.999+0000 7ff2fa142700 1 --2- 172.21.15.12:0/951359013 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff2e00b4ca0 0x7ff2e00b71b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x555dc0b3bae0 tx=0x7ff2e400f040 comp rx=0 tx=0).stop 2024-05-17T22:42:36.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.999+0000 7ff2fa142700 1 -- 172.21.15.12:0/951359013 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2f410f9f0 msgr2=0x7ff2f41a0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:42:36.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.999+0000 7ff2fa142700 1 --2- 172.21.15.12:0/951359013 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2f410f9f0 0x7ff2f41a0af0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00b970 tx=0x7ff2ec00bd30 comp rx=0 tx=0).stop 2024-05-17T22:42:36.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.999+0000 7ff2fa142700 1 -- 172.21.15.12:0/951359013 shutdown_connections 2024-05-17T22:42:36.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.999+0000 7ff2fa142700 1 --2- 172.21.15.12:0/951359013 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff2e00b4ca0 0x7ff2e00b71b0 secure :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0x555dc0b3bae0 tx=0x7ff2e400f040 comp rx=0 tx=0).stop 2024-05-17T22:42:36.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.999+0000 7ff2fa142700 1 --2- 172.21.15.12:0/951359013 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2f410f9f0 0x7ff2f41a0af0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:36.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.999+0000 7ff2fa142700 1 --2- 172.21.15.12:0/951359013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2f410aac0 0x7ff2f41a68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:36.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.999+0000 7ff2fa142700 1 --2- 172.21.15.12:0/951359013 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2f410f0a0 0x7ff2f41a6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:42:36.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:35.999+0000 7ff2fa142700 1 -- 172.21.15.12:0/951359013 >> 172.21.15.12:0/951359013 conn(0x7ff2f406f1f0 msgr2=0x7ff2f4073110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:42:36.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:36.000+0000 7ff2fa142700 1 -- 172.21.15.12:0/951359013 shutdown_connections 2024-05-17T22:42:36.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:42:36.001+0000 7ff2fa142700 1 -- 172.21.15.12:0/951359013 wait complete. 2024-05-17T22:42:36.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:35 smithi096 ceph-mon[36065]: pgmap v539: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 1.01k op/s 2024-05-17T22:42:36.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:35 smithi151 ceph-mon[36101]: pgmap v539: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 1.01k op/s 2024-05-17T22:42:36.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 195.0 s, 669.4 tps, lat 147.315 ms stddev 191.738 2024-05-17T22:42:36.933 INFO:tasks.cephfs.filesystem:scrub status for tag:9742bd65-6100-4d0d-a9cd-09f6f1512b65 - {'path': '/', 'tag': '9742bd65-6100-4d0d-a9cd-09f6f1512b65', 'options': 'recursive,force'} 2024-05-17T22:42:37.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:36 smithi096 ceph-mon[36065]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:42:37.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:36 smithi012 ceph-mon[31632]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:42:37.242 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:36 smithi151 ceph-mon[36101]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:42:38.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:37 smithi096 ceph-mon[36065]: pgmap v540: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 988 op/s 2024-05-17T22:42:38.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:37 smithi012 ceph-mon[31632]: pgmap v540: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 988 op/s 2024-05-17T22:42:38.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:37 smithi151 ceph-mon[36101]: pgmap v540: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 988 op/s 2024-05-17T22:42:39.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:38 smithi096 ceph-mon[36065]: scrub summary: idle 2024-05-17T22:42:39.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:38 smithi012 ceph-mon[31632]: scrub summary: idle 2024-05-17T22:42:39.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:38 smithi151 ceph-mon[36101]: scrub summary: idle 2024-05-17T22:42:40.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:39 smithi012 ceph-mon[31632]: scrub summary: idle 2024-05-17T22:42:40.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:39 smithi012 ceph-mon[31632]: scrub summary: idle 2024-05-17T22:42:40.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:39 smithi012 ceph-mon[31632]: pgmap v541: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 MiB/s wr, 896 op/s 2024-05-17T22:42:40.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:39 smithi151 ceph-mon[36101]: scrub summary: idle 2024-05-17T22:42:40.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:39 smithi151 ceph-mon[36101]: scrub summary: idle 2024-05-17T22:42:40.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:39 smithi151 ceph-mon[36101]: pgmap v541: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 MiB/s wr, 896 op/s 2024-05-17T22:42:40.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:39 smithi096 ceph-mon[36065]: scrub summary: idle 2024-05-17T22:42:40.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:39 smithi096 ceph-mon[36065]: scrub summary: idle 2024-05-17T22:42:40.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:39 smithi096 ceph-mon[36065]: pgmap v541: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 MiB/s wr, 896 op/s 2024-05-17T22:42:41.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 200.0 s, 1286.0 tps, lat 79.987 ms stddev 90.588 2024-05-17T22:42:42.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:41 smithi012 ceph-mon[31632]: pgmap v542: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.4 MiB/s wr, 599 op/s 2024-05-17T22:42:42.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:41 smithi151 ceph-mon[36101]: pgmap v542: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.4 MiB/s wr, 599 op/s 2024-05-17T22:42:42.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:41 smithi096 ceph-mon[36065]: pgmap v542: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.4 MiB/s wr, 599 op/s 2024-05-17T22:42:44.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:43 smithi012 ceph-mon[31632]: pgmap v543: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.8 MiB/s wr, 537 op/s 2024-05-17T22:42:44.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:43 smithi151 ceph-mon[36101]: pgmap v543: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.8 MiB/s wr, 537 op/s 2024-05-17T22:42:44.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:43 smithi096 ceph-mon[36065]: pgmap v543: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.8 MiB/s wr, 537 op/s 2024-05-17T22:42:46.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:45 smithi012 ceph-mon[31632]: pgmap v544: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.9 MiB/s wr, 549 op/s 2024-05-17T22:42:46.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:45 smithi151 ceph-mon[36101]: pgmap v544: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.9 MiB/s wr, 549 op/s 2024-05-17T22:42:46.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:45 smithi096 ceph-mon[36065]: pgmap v544: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.9 MiB/s wr, 549 op/s 2024-05-17T22:42:46.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 205.0 s, 1054.8 tps, lat 93.881 ms stddev 101.255 2024-05-17T22:42:48.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:47 smithi012 ceph-mon[31632]: pgmap v545: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.5 MiB/s wr, 589 op/s 2024-05-17T22:42:48.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:47 smithi151 ceph-mon[36101]: pgmap v545: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.5 MiB/s wr, 589 op/s 2024-05-17T22:42:48.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:47 smithi096 ceph-mon[36065]: pgmap v545: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.5 MiB/s wr, 589 op/s 2024-05-17T22:42:50.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:49 smithi012 ceph-mon[31632]: pgmap v546: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.5 MiB/s wr, 603 op/s 2024-05-17T22:42:50.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:49 smithi151 ceph-mon[36101]: pgmap v546: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.5 MiB/s wr, 603 op/s 2024-05-17T22:42:50.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:49 smithi096 ceph-mon[36065]: pgmap v546: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.5 MiB/s wr, 603 op/s 2024-05-17T22:42:51.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 210.0 s, 887.8 tps, lat 113.301 ms stddev 123.516 2024-05-17T22:42:52.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:51 smithi012 ceph-mon[31632]: pgmap v547: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 6.1 MiB/s wr, 574 op/s 2024-05-17T22:42:52.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:51 smithi151 ceph-mon[36101]: pgmap v547: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 6.1 MiB/s wr, 574 op/s 2024-05-17T22:42:52.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:51 smithi096 ceph-mon[36065]: pgmap v547: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 6.1 MiB/s wr, 574 op/s 2024-05-17T22:42:54.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:53 smithi012 ceph-mon[31632]: pgmap v548: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 4.8 MiB/s wr, 550 op/s 2024-05-17T22:42:54.242 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:53 smithi151 ceph-mon[36101]: pgmap v548: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 4.8 MiB/s wr, 550 op/s 2024-05-17T22:42:54.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:53 smithi096 ceph-mon[36065]: pgmap v548: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 4.8 MiB/s wr, 550 op/s 2024-05-17T22:42:56.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:55 smithi012 ceph-mon[31632]: pgmap v549: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 5.0 MiB/s wr, 565 op/s 2024-05-17T22:42:56.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:55 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:42:56.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:55 smithi151 ceph-mon[36101]: pgmap v549: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 5.0 MiB/s wr, 565 op/s 2024-05-17T22:42:56.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:55 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:42:56.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:55 smithi096 ceph-mon[36065]: pgmap v549: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 5.0 MiB/s wr, 565 op/s 2024-05-17T22:42:56.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:55 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:42:56.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 215.0 s, 784.4 tps, lat 125.169 ms stddev 146.624 2024-05-17T22:42:57.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:56 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:42:57.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:56 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:42:57.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:56 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:42:57.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:56 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:42:57.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:56 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:42:57.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:56 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:42:57.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:56 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:42:57.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:56 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:42:57.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:56 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:42:57.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:56 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:42:57.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:56 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:42:57.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:56 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:42:57.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:56 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:42:57.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:56 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:42:57.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:56 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:42:57.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:56 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:42:57.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:56 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:42:57.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:56 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:42:57.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:56 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:42:57.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:56 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:42:57.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:56 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:42:58.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:57 smithi012 ceph-mon[31632]: Adjusting osd_memory_target on smithi012 to 177.2M 2024-05-17T22:42:58.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:57 smithi012 ceph-mon[31632]: Unable to set osd_memory_target on smithi012 to 185879142: error parsing value: Value '185879142' is below minimum 939524096 2024-05-17T22:42:58.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:57 smithi012 ceph-mon[31632]: pgmap v550: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6.2 MiB/s wr, 642 op/s 2024-05-17T22:42:58.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:57 smithi151 ceph-mon[36101]: Adjusting osd_memory_target on smithi012 to 177.2M 2024-05-17T22:42:58.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:57 smithi151 ceph-mon[36101]: Unable to set osd_memory_target on smithi012 to 185879142: error parsing value: Value '185879142' is below minimum 939524096 2024-05-17T22:42:58.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:57 smithi151 ceph-mon[36101]: pgmap v550: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6.2 MiB/s wr, 642 op/s 2024-05-17T22:42:58.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:57 smithi096 ceph-mon[36065]: Adjusting osd_memory_target on smithi012 to 177.2M 2024-05-17T22:42:58.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:57 smithi096 ceph-mon[36065]: Unable to set osd_memory_target on smithi012 to 185879142: error parsing value: Value '185879142' is below minimum 939524096 2024-05-17T22:42:58.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:57 smithi096 ceph-mon[36065]: pgmap v550: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6.2 MiB/s wr, 642 op/s 2024-05-17T22:43:00.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:42:59 smithi151 ceph-mon[36101]: pgmap v551: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9.0 MiB/s wr, 787 op/s 2024-05-17T22:43:00.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:42:59 smithi096 ceph-mon[36065]: pgmap v551: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9.0 MiB/s wr, 787 op/s 2024-05-17T22:43:00.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:42:59 smithi012 ceph-mon[31632]: pgmap v551: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9.0 MiB/s wr, 787 op/s 2024-05-17T22:43:01.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 220.0 s, 1144.2 tps, lat 88.436 ms stddev 98.874 2024-05-17T22:43:02.144 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:01 smithi012 ceph-mon[31632]: pgmap v552: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 9.6 MiB/s wr, 798 op/s 2024-05-17T22:43:02.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:01 smithi151 ceph-mon[36101]: pgmap v552: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 9.6 MiB/s wr, 798 op/s 2024-05-17T22:43:02.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:01 smithi096 ceph-mon[36065]: pgmap v552: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 9.6 MiB/s wr, 798 op/s 2024-05-17T22:43:04.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:03 smithi151 ceph-mon[36101]: pgmap v553: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9.2 MiB/s wr, 770 op/s 2024-05-17T22:43:04.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:03 smithi096 ceph-mon[36065]: pgmap v553: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9.2 MiB/s wr, 770 op/s 2024-05-17T22:43:04.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:03 smithi012 ceph-mon[31632]: pgmap v553: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9.2 MiB/s wr, 770 op/s 2024-05-17T22:43:06.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:05 smithi151 ceph-mon[36101]: pgmap v554: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 9.1 MiB/s wr, 770 op/s 2024-05-17T22:43:06.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:05 smithi096 ceph-mon[36065]: pgmap v554: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 9.1 MiB/s wr, 770 op/s 2024-05-17T22:43:06.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:05 smithi012 ceph-mon[31632]: pgmap v554: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 9.1 MiB/s wr, 770 op/s 2024-05-17T22:43:06.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 225.0 s, 1195.4 tps, lat 84.487 ms stddev 99.722 2024-05-17T22:43:06.935 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-17T22:43:07.327 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:43:08.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:07 smithi012 ceph-mon[31632]: pgmap v555: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.1 MiB/s wr, 781 op/s 2024-05-17T22:43:08.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:07 smithi151 ceph-mon[36101]: pgmap v555: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.1 MiB/s wr, 781 op/s 2024-05-17T22:43:08.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:07 smithi096 ceph-mon[36065]: pgmap v555: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.1 MiB/s wr, 781 op/s 2024-05-17T22:43:09.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.415+0000 7fc79923a700 1 -- 172.21.15.12:0/2991657307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc79410ee60 msgr2=0x7fc79410f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:09.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.415+0000 7fc79923a700 1 --2- 172.21.15.12:0/2991657307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc79410ee60 0x7fc79410f270 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc77c009990 tx=0x7fc77c009ca0 comp rx=0 tx=0).stop 2024-05-17T22:43:09.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.417+0000 7fc79923a700 1 -- 172.21.15.12:0/2991657307 shutdown_connections 2024-05-17T22:43:09.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.417+0000 7fc79923a700 1 --2- 172.21.15.12:0/2991657307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc79410f7b0 0x7fc794117110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:09.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.417+0000 7fc79923a700 1 --2- 172.21.15.12:0/2991657307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc79410a530 0x7fc79410a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:09.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.417+0000 7fc79923a700 1 --2- 172.21.15.12:0/2991657307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc79410ee60 0x7fc79410f270 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:09.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.417+0000 7fc79923a700 1 -- 172.21.15.12:0/2991657307 >> 172.21.15.12:0/2991657307 conn(0x7fc794078570 msgr2=0x7fc794108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:43:09.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.421+0000 7fc79923a700 1 -- 172.21.15.12:0/2991657307 shutdown_connections 2024-05-17T22:43:09.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.421+0000 7fc79923a700 1 -- 172.21.15.12:0/2991657307 wait complete. 2024-05-17T22:43:09.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.422+0000 7fc79923a700 1 Processor -- start 2024-05-17T22:43:09.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.424+0000 7fc79923a700 1 -- start start 2024-05-17T22:43:09.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.424+0000 7fc79923a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc79410a530 0x7fc79406f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:09.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.424+0000 7fc79923a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc79410ee60 0x7fc79406fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:09.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.424+0000 7fc79923a700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc79410f7b0 0x7fc794070130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:09.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.424+0000 7fc79923a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc79406db30 con 0x7fc79410a530 2024-05-17T22:43:09.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.424+0000 7fc79923a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc79406d9b0 con 0x7fc79410ee60 2024-05-17T22:43:09.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.424+0000 7fc79923a700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc79406dcb0 con 0x7fc79410f7b0 2024-05-17T22:43:09.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.425+0000 7fc792d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc79410a530 0x7fc79406f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:09.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.425+0000 7fc792d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc79410a530 0x7fc79406f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60344/0 (socket says 172.21.15.12:60344) 2024-05-17T22:43:09.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.425+0000 7fc792d9d700 1 -- 172.21.15.12:0/1662981111 learned_addr learned my addr 172.21.15.12:0/1662981111 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:43:09.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.427+0000 7fc792d9d700 1 -- 172.21.15.12:0/1662981111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc79410f7b0 msgr2=0x7fc794070130 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-17T22:43:09.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.427+0000 7fc792d9d700 1 --2- 172.21.15.12:0/1662981111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc79410f7b0 0x7fc794070130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:09.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.427+0000 7fc792d9d700 1 -- 172.21.15.12:0/1662981111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc79410ee60 msgr2=0x7fc79406fbf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-17T22:43:09.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.427+0000 7fc792d9d700 1 --2- 172.21.15.12:0/1662981111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc79410ee60 0x7fc79406fbf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:09.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.427+0000 7fc792d9d700 1 -- 172.21.15.12:0/1662981111 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7941b58e0 con 0x7fc79410a530 2024-05-17T22:43:09.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.428+0000 7fc792d9d700 1 --2- 172.21.15.12:0/1662981111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc79410a530 0x7fc79406f6b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc78c0060b0 tx=0x7fc78c007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:43:09.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.436+0000 7fc78bfff700 1 -- 172.21.15.12:0/1662981111 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c01a070 con 0x7fc79410a530 2024-05-17T22:43:09.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.436+0000 7fc78bfff700 1 -- 172.21.15.12:0/1662981111 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc78c00de20 con 0x7fc79410a530 2024-05-17T22:43:09.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.436+0000 7fc78bfff700 1 -- 172.21.15.12:0/1662981111 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c014720 con 0x7fc79410a530 2024-05-17T22:43:09.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.436+0000 7fc79923a700 1 -- 172.21.15.12:0/1662981111 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7941b5b70 con 0x7fc79410a530 2024-05-17T22:43:09.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.437+0000 7fc79923a700 1 -- 172.21.15.12:0/1662981111 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7941b60b0 con 0x7fc79410a530 2024-05-17T22:43:09.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.449+0000 7fc7a7c15b80 1 Processor -- start 2024-05-17T22:43:09.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.449+0000 7fc7a7c15b80 1 -- start start 2024-05-17T22:43:09.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.450+0000 7fc78bfff700 1 -- 172.21.15.12:0/1662981111 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc78c01d070 con 0x7fc79410a530 2024-05-17T22:43:09.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.450+0000 7fc7a7c15b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfb71189f0 0x55bfb729bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:09.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.450+0000 7fc7a7c15b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfb729c400 0x55bfb72965c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:09.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.450+0000 7fc7a7c15b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfb7296b00 0x55bfb7296f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:09.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.451+0000 7fc7a7c15b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55bfb71c50d0 con 0x55bfb7296b00 2024-05-17T22:43:09.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.451+0000 7fc7a7c15b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55bfb71b4c70 con 0x55bfb729c400 2024-05-17T22:43:09.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.451+0000 7fc7a7c15b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55bfb71b8cb0 con 0x55bfb71189f0 2024-05-17T22:43:09.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.451+0000 7fc78bfff700 1 --2- 172.21.15.12:0/1662981111 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fc7800b2a90 0x7fc7800b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:09.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.451+0000 7fc78bfff700 1 -- 172.21.15.12:0/1662981111 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fc78c0e1e30 con 0x7fc79410a530 2024-05-17T22:43:09.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.453+0000 7fc79259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfb7296b00 0x55bfb7296f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:09.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.453+0000 7fc79259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfb7296b00 0x55bfb7296f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60360/0 (socket says 172.21.15.12:60360) 2024-05-17T22:43:09.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.453+0000 7fc79259c700 1 -- 172.21.15.12:0/1498736702 learned_addr learned my addr 172.21.15.12:0/1498736702 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:43:09.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.454+0000 7fc792d9d700 1 --2- 172.21.15.12:0/1498736702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfb729c400 0x55bfb72965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:09.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.454+0000 7fc79359e700 1 --2- 172.21.15.12:0/1498736702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfb71189f0 0x55bfb729bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:09.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.454+0000 7fc79259c700 1 -- 172.21.15.12:0/1498736702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfb71189f0 msgr2=0x55bfb729bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:43:09.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.454+0000 7fc79259c700 1 --2- 172.21.15.12:0/1498736702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfb71189f0 0x55bfb729bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:09.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.454+0000 7fc79259c700 1 -- 172.21.15.12:0/1498736702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfb729c400 msgr2=0x55bfb72965c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:43:09.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.454+0000 7fc79259c700 1 --2- 172.21.15.12:0/1498736702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfb729c400 0x55bfb72965c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:09.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.454+0000 7fc79259c700 1 -- 172.21.15.12:0/1498736702 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfb714e5c0 con 0x55bfb7296b00 2024-05-17T22:43:09.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.455+0000 7fc79359e700 1 --2- 172.21.15.12:0/1662981111 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fc7800b2a90 0x7fc7800b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:09.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.456+0000 7fc79359e700 1 --2- 172.21.15.12:0/1662981111 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fc7800b2a90 0x7fc7800b4fa0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x55bfb72921d0 tx=0x7fc7840047c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:43:09.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.457+0000 7fc79259c700 1 --2- 172.21.15.12:0/1498736702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfb7296b00 0x55bfb7296f10 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc77c000c00 tx=0x7fc77c00b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bff9ca9250ec657 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:43:09.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.459+0000 7fc7897fa700 1 -- 172.21.15.12:0/1498736702 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77c003860 con 0x55bfb7296b00 2024-05-17T22:43:09.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.459+0000 7fc7897fa700 1 -- 172.21.15.12:0/1498736702 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc77c005070 con 0x55bfb7296b00 2024-05-17T22:43:09.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.459+0000 7fc7897fa700 1 -- 172.21.15.12:0/1498736702 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77c00ba30 con 0x55bfb7296b00 2024-05-17T22:43:09.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.460+0000 7fc7a7c15b80 1 -- 172.21.15.12:0/1498736702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfb7296b00 msgr2=0x55bfb7296f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:09.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.460+0000 7fc7a7c15b80 1 --2- 172.21.15.12:0/1498736702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfb7296b00 0x55bfb7296f10 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc77c000c00 tx=0x7fc77c00b540 comp rx=0 tx=0).stop 2024-05-17T22:43:09.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.461+0000 7fc7a7c15b80 1 -- 172.21.15.12:0/1498736702 shutdown_connections 2024-05-17T22:43:09.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.461+0000 7fc7a7c15b80 1 --2- 172.21.15.12:0/1498736702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfb7296b00 0x55bfb7296f10 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:09.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.461+0000 7fc7a7c15b80 1 --2- 172.21.15.12:0/1498736702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfb71189f0 0x55bfb729bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:09.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.461+0000 7fc7a7c15b80 1 --2- 172.21.15.12:0/1498736702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfb729c400 0x55bfb72965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:09.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.461+0000 7fc7a7c15b80 1 -- 172.21.15.12:0/1498736702 >> 172.21.15.12:0/1498736702 conn(0x55bfb71787d0 msgr2=0x55bfb72911c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:43:09.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.461+0000 7fc7a7c15b80 1 -- 172.21.15.12:0/1498736702 shutdown_connections 2024-05-17T22:43:09.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.461+0000 7fc7a7c15b80 1 -- 172.21.15.12:0/1498736702 wait complete. 2024-05-17T22:43:09.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.462+0000 7fc7a7c15b80 1 -- start start 2024-05-17T22:43:09.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.462+0000 7fc7a7c15b80 1 Processor -- start 2024-05-17T22:43:09.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.463+0000 7fc7a7c15b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfb71189f0 0x55bfb729fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:09.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.463+0000 7fc7a7c15b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfb7296b00 0x55bfb72a0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:09.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.463+0000 7fc7a7c15b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfb729c400 0x55bfb72a0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:09.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.464+0000 7fc7a7c15b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55bfb70bc150 con 0x55bfb7296b00 2024-05-17T22:43:09.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.464+0000 7fc7a7c15b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55bfb7156940 con 0x55bfb729c400 2024-05-17T22:43:09.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.464+0000 7fc7a7c15b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55bfb72a2d10 con 0x55bfb71189f0 2024-05-17T22:43:09.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.464+0000 7fc7a7c15b80 10 client.0 _ll_register_callbacks cb 0x55bfb714bb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:43:09.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.464+0000 7fc79359e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfb729c400 0x55bfb72a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:09.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.465+0000 7fc79359e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfb729c400 0x55bfb72a0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:53922/0 (socket says 172.21.15.12:53922) 2024-05-17T22:43:09.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.465+0000 7fc79359e700 1 -- 172.21.15.12:0/143021493 learned_addr learned my addr 172.21.15.12:0/143021493 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:43:09.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.467+0000 7fc792d9d700 1 --2- 172.21.15.12:0/143021493 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfb71189f0 0x55bfb729fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:09.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.467+0000 7fc79259c700 1 --2- 172.21.15.12:0/143021493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfb7296b00 0x55bfb72a0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:09.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.468+0000 7fc792d9d700 1 -- 172.21.15.12:0/143021493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfb729c400 msgr2=0x55bfb72a0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:43:09.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.468+0000 7fc792d9d700 1 --2- 172.21.15.12:0/143021493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfb729c400 0x55bfb72a0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:09.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.468+0000 7fc792d9d700 1 -- 172.21.15.12:0/143021493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfb7296b00 msgr2=0x55bfb72a0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:43:09.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.468+0000 7fc792d9d700 1 --2- 172.21.15.12:0/143021493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfb7296b00 0x55bfb72a0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:09.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.468+0000 7fc792d9d700 1 -- 172.21.15.12:0/143021493 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfb7105480 con 0x55bfb71189f0 2024-05-17T22:43:09.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.469+0000 7fc792d9d700 1 --2- 172.21.15.12:0/143021493 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfb71189f0 0x55bfb729fae0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc78c000f80 tx=0x7fc78c0a8c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6138df5534bab98a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:43:09.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.469+0000 7fc77a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:43:09.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.469+0000 7fc77a7fc700 1 -- 172.21.15.12:0/143021493 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c0ad410 con 0x55bfb71189f0 2024-05-17T22:43:09.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.470+0000 7fc77a7fc700 1 -- 172.21.15.12:0/143021493 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fc78c0b1a30 con 0x55bfb71189f0 2024-05-17T22:43:09.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.472+0000 7fc7a7c15b80 1 -- 172.21.15.12:0/143021493 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc77c005070 con 0x55bfb71189f0 2024-05-17T22:43:09.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.476+0000 7fc77a7fc700 1 -- 172.21.15.12:0/143021493 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc78c0b1ec0 con 0x55bfb71189f0 2024-05-17T22:43:09.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.477+0000 7fc7a7c15b80 10 client.25288 fetch_fsmap learned FSMap version 33 2024-05-17T22:43:09.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.477+0000 7fc7a7c15b80 1 -- 172.21.15.12:0/143021493 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55bfb72ad6e0 con 0x55bfb71189f0 2024-05-17T22:43:09.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.479+0000 7fc77a7fc700 1 -- 172.21.15.12:0/143021493 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc77c005070 con 0x55bfb71189f0 2024-05-17T22:43:09.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.479+0000 7fc77a7fc700 1 -- 172.21.15.12:0/143021493 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7fc78c004ad0 con 0x55bfb71189f0 2024-05-17T22:43:09.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.479+0000 7fc7a7c15b80 10 client.25288 fetch_fsmap finished waiting for FSMap version 33 2024-05-17T22:43:09.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.479+0000 7fc7a7c15b80 10 client.25288 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:43:09.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.479+0000 7fc7a7c15b80 20 client.25288 populate_metadata read hostname 'smithi012' 2024-05-17T22:43:09.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.479+0000 7fc7a7c15b80 1 --2- 172.21.15.12:0/143021493 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55bfb72afd20 0x55bfb72b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:09.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.479+0000 7fc7a7c15b80 4 client.25288 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:43:09.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.479+0000 7fc7a7c15b80 1 -- 172.21.15.12:0/143021493 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfb71c7020 con 0x55bfb72afd20 2024-05-17T22:43:09.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.479+0000 7fc79259c700 1 --2- 172.21.15.12:0/143021493 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55bfb72afd20 0x55bfb72b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:09.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.481+0000 7fc79259c700 1 --2- 172.21.15.12:0/143021493 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55bfb72afd20 0x55bfb72b2140 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a3db6feca7cd649 server_cookie=af70e476fb8db08c in_seq=0 out_seq=0 2024-05-17T22:43:09.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.481+0000 7fc77a7fc700 10 client.25288 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:43:09.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.484+0000 7fc77a7fc700 1 -- 172.21.15.12:0/143021493 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bfb71c7020 con 0x55bfb72afd20 2024-05-17T22:43:09.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.484+0000 7fc77a7fc700 10 client.25288 handle_command_reply: tid=0 2024-05-17T22:43:09.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.485+0000 7fc7a7c15b80 1 client.25288 shutdown 2024-05-17T22:43:09.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.485+0000 7fc7a7c15b80 1 -- 172.21.15.12:0/143021493 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfb71189f0 msgr2=0x55bfb729fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:09.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.485+0000 7fc7a7c15b80 1 --2- 172.21.15.12:0/143021493 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfb71189f0 0x55bfb729fae0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc78c000f80 tx=0x7fc78c0a8c40 comp rx=0 tx=0).stop 2024-05-17T22:43:09.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.485+0000 7fc7a7c15b80 1 -- 172.21.15.12:0/143021493 shutdown_connections 2024-05-17T22:43:09.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.485+0000 7fc7a7c15b80 1 --2- 172.21.15.12:0/143021493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfb729c400 0x55bfb72a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:09.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.485+0000 7fc7a7c15b80 1 --2- 172.21.15.12:0/143021493 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfb71189f0 0x55bfb729fae0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:09.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.485+0000 7fc7a7c15b80 1 --2- 172.21.15.12:0/143021493 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55bfb72afd20 0x55bfb72b2140 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:09.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.485+0000 7fc7a7c15b80 1 --2- 172.21.15.12:0/143021493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfb7296b00 0x55bfb72a0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:09.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.485+0000 7fc7a7c15b80 1 -- 172.21.15.12:0/143021493 >> 172.21.15.12:0/143021493 conn(0x55bfb71787d0 msgr2=0x55bfb7291160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:43:09.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.485+0000 7fc77a7fc700 0 client.25288 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:43:09.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.486+0000 7fc7a7c15b80 1 -- 172.21.15.12:0/143021493 shutdown_connections 2024-05-17T22:43:09.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.486+0000 7fc7a7c15b80 1 -- 172.21.15.12:0/143021493 wait complete. 2024-05-17T22:43:09.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.486+0000 7fc7a7c15b80 20 client.25288 trim_cache size 0 max 16384 2024-05-17T22:43:09.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.519+0000 7fc7a7c15b80 1 Processor -- start 2024-05-17T22:43:09.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.519+0000 7fc7a7c15b80 1 -- start start 2024-05-17T22:43:09.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.520+0000 7fc7a7c15b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfb71189f0 0x55bfb72965c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:09.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.520+0000 7fc7a7c15b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfb7296b00 0x55bfb729a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:09.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.520+0000 7fc7a7c15b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfb729c400 0x55bfb72a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:09.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.520+0000 7fc792d9d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfb71189f0 0x55bfb72965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:09.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.521+0000 7fc79259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfb7296b00 0x55bfb729a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:09.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.521+0000 7fc79259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfb7296b00 0x55bfb729a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60364/0 (socket says 172.21.15.12:60364) 2024-05-17T22:43:09.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.521+0000 7fc792d9d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfb71189f0 0x55bfb72965c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:51192/0 (socket says 172.21.15.12:51192) 2024-05-17T22:43:09.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.521+0000 7fc79259c700 1 -- 172.21.15.12:0/3844424825 learned_addr learned my addr 172.21.15.12:0/3844424825 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:43:09.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.521+0000 7fc7a7c15b80 1 -- 172.21.15.12:0/3844424825 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc7941c15b0 con 0x55bfb7296b00 2024-05-17T22:43:09.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.521+0000 7fc7a7c15b80 1 -- 172.21.15.12:0/3844424825 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc7941bede0 con 0x55bfb729c400 2024-05-17T22:43:09.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.521+0000 7fc79359e700 1 --2- 172.21.15.12:0/3844424825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfb729c400 0x55bfb72a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:09.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.521+0000 7fc7a7c15b80 1 -- 172.21.15.12:0/3844424825 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc7941bcc30 con 0x55bfb71189f0 2024-05-17T22:43:09.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.522+0000 7fc79259c700 1 -- 172.21.15.12:0/3844424825 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfb71189f0 msgr2=0x55bfb72965c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:43:09.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.522+0000 7fc79259c700 1 --2- 172.21.15.12:0/3844424825 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfb71189f0 0x55bfb72965c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:09.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.522+0000 7fc79259c700 1 -- 172.21.15.12:0/3844424825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfb729c400 msgr2=0x55bfb72a03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:43:09.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.522+0000 7fc79259c700 1 --2- 172.21.15.12:0/3844424825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfb729c400 0x55bfb72a03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:09.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.522+0000 7fc79259c700 1 -- 172.21.15.12:0/3844424825 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfb72b25e0 con 0x55bfb7296b00 2024-05-17T22:43:09.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.522+0000 7fc79259c700 1 --2- 172.21.15.12:0/3844424825 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfb7296b00 0x55bfb729a1b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc77c000c00 tx=0x7fc77c019040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b570349cdd6e39f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:43:09.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.522+0000 7fc788ff9700 1 -- 172.21.15.12:0/3844424825 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77c0053c0 con 0x55bfb7296b00 2024-05-17T22:43:09.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.522+0000 7fc788ff9700 1 -- 172.21.15.12:0/3844424825 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc77c003d10 con 0x55bfb7296b00 2024-05-17T22:43:09.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.522+0000 7fc788ff9700 1 -- 172.21.15.12:0/3844424825 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77c00bac0 con 0x55bfb7296b00 2024-05-17T22:43:09.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.523+0000 7fc7a7c15b80 1 -- 172.21.15.12:0/3844424825 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfb7296b00 msgr2=0x55bfb729a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:09.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.523+0000 7fc7a7c15b80 1 --2- 172.21.15.12:0/3844424825 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfb7296b00 0x55bfb729a1b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc77c000c00 tx=0x7fc77c019040 comp rx=0 tx=0).stop 2024-05-17T22:43:09.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.523+0000 7fc7a7c15b80 1 -- 172.21.15.12:0/3844424825 shutdown_connections 2024-05-17T22:43:09.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.523+0000 7fc7a7c15b80 1 --2- 172.21.15.12:0/3844424825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfb729c400 0x55bfb72a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:09.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.523+0000 7fc7a7c15b80 1 --2- 172.21.15.12:0/3844424825 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfb71189f0 0x55bfb72965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:09.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.523+0000 7fc7a7c15b80 1 --2- 172.21.15.12:0/3844424825 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfb7296b00 0x55bfb729a1b0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:09.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.523+0000 7fc7a7c15b80 1 -- 172.21.15.12:0/3844424825 >> 172.21.15.12:0/3844424825 conn(0x55bfb71787d0 msgr2=0x55bfb729ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:43:09.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.523+0000 7fc7a7c15b80 1 -- 172.21.15.12:0/3844424825 shutdown_connections 2024-05-17T22:43:09.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.523+0000 7fc7a7c15b80 1 -- 172.21.15.12:0/3844424825 wait complete. 2024-05-17T22:43:09.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.524+0000 7fc7a7c15b80 1 -- start start 2024-05-17T22:43:09.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.524+0000 7fc7a7c15b80 1 Processor -- start 2024-05-17T22:43:09.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.525+0000 7fc7a7c15b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfb71189f0 0x55bfb72b6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:09.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.525+0000 7fc7a7c15b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfb7296b00 0x55bfb72b74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:09.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.525+0000 7fc792d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfb71189f0 0x55bfb72b6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:09.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.525+0000 7fc7a7c15b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfb729c400 0x55bfb72b79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:09.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.525+0000 7fc7a7c15b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc7941bd2a0 con 0x55bfb71189f0 2024-05-17T22:43:09.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.525+0000 7fc7a7c15b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc7941be710 con 0x55bfb7296b00 2024-05-17T22:43:09.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.525+0000 7fc7a7c15b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc7941bd970 con 0x55bfb729c400 2024-05-17T22:43:09.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.525+0000 7fc792d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfb71189f0 0x55bfb72b6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60374/0 (socket says 172.21.15.12:60374) 2024-05-17T22:43:09.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.525+0000 7fc792d9d700 1 -- 172.21.15.12:0/3277678402 learned_addr learned my addr 172.21.15.12:0/3277678402 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:43:09.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.525+0000 7fc7a7c15b80 10 client.0 _ll_register_callbacks cb 0x55bfb714bb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:43:09.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.525+0000 7fc79359e700 1 --2- 172.21.15.12:0/3277678402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfb729c400 0x55bfb72b79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:09.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.525+0000 7fc79259c700 1 --2- 172.21.15.12:0/3277678402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfb7296b00 0x55bfb72b74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:09.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.525+0000 7fc792d9d700 1 -- 172.21.15.12:0/3277678402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfb729c400 msgr2=0x55bfb72b79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:43:09.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.525+0000 7fc792d9d700 1 --2- 172.21.15.12:0/3277678402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfb729c400 0x55bfb72b79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:09.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.525+0000 7fc792d9d700 1 -- 172.21.15.12:0/3277678402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfb7296b00 msgr2=0x55bfb72b74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:43:09.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.525+0000 7fc792d9d700 1 --2- 172.21.15.12:0/3277678402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfb7296b00 0x55bfb72b74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:09.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.525+0000 7fc792d9d700 1 -- 172.21.15.12:0/3277678402 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfb72a8a50 con 0x55bfb71189f0 2024-05-17T22:43:09.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.526+0000 7fc79259c700 1 --2- 172.21.15.12:0/3277678402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfb7296b00 0x55bfb72b74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:43:09.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.526+0000 7fc79359e700 1 --2- 172.21.15.12:0/3277678402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfb729c400 0x55bfb72b79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:43:09.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.527+0000 7fc792d9d700 1 --2- 172.21.15.12:0/3277678402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfb71189f0 0x55bfb72b6f60 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fc78c000f80 tx=0x7fc78c0a88e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19619536c590a296 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:43:09.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.527+0000 7fc77a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:43:09.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.527+0000 7fc77a7fc700 1 -- 172.21.15.12:0/3277678402 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c005c60 con 0x55bfb71189f0 2024-05-17T22:43:09.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.527+0000 7fc77a7fc700 1 -- 172.21.15.12:0/3277678402 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fc78c005e00 con 0x55bfb71189f0 2024-05-17T22:43:09.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.527+0000 7fc7a7c15b80 1 -- 172.21.15.12:0/3277678402 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc77c003d10 con 0x55bfb71189f0 2024-05-17T22:43:09.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.528+0000 7fc77a7fc700 1 -- 172.21.15.12:0/3277678402 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc78c0b1a30 con 0x55bfb71189f0 2024-05-17T22:43:09.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.528+0000 7fc7a7c15b80 10 client.15687 fetch_fsmap learned FSMap version 33 2024-05-17T22:43:09.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.528+0000 7fc7a7c15b80 1 -- 172.21.15.12:0/3277678402 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55bfb731a4e0 con 0x55bfb71189f0 2024-05-17T22:43:09.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.528+0000 7fc77a7fc700 1 -- 172.21.15.12:0/3277678402 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc77c003d10 con 0x55bfb71189f0 2024-05-17T22:43:09.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.529+0000 7fc77a7fc700 1 -- 172.21.15.12:0/3277678402 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7fc78c0a4af0 con 0x55bfb71189f0 2024-05-17T22:43:09.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.529+0000 7fc7a7c15b80 10 client.15687 fetch_fsmap finished waiting for FSMap version 33 2024-05-17T22:43:09.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.529+0000 7fc7a7c15b80 10 client.15687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:43:09.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.529+0000 7fc7a7c15b80 20 client.15687 populate_metadata read hostname 'smithi012' 2024-05-17T22:43:09.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.529+0000 7fc7a7c15b80 1 --2- 172.21.15.12:0/3277678402 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55bfb731aa90 0x55bfb731ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:09.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.529+0000 7fc7a7c15b80 4 client.15687 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:43:09.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.529+0000 7fc7a7c15b80 1 -- 172.21.15.12:0/3277678402 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bfb71c0800 con 0x55bfb731aa90 2024-05-17T22:43:09.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.529+0000 7fc79259c700 1 --2- 172.21.15.12:0/3277678402 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55bfb731aa90 0x55bfb731ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:09.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.529+0000 7fc77a7fc700 10 client.15687 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:43:09.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.529+0000 7fc79259c700 1 --2- 172.21.15.12:0/3277678402 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55bfb731aa90 0x55bfb731ceb0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6de6394294195ca server_cookie=8b1bbef768c2c4d in_seq=0 out_seq=0 2024-05-17T22:43:09.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.530+0000 7fc77a7fc700 1 -- 172.21.15.12:0/3277678402 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bfb71c0800 con 0x55bfb731aa90 2024-05-17T22:43:09.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.530+0000 7fc77a7fc700 10 client.15687 handle_command_reply: tid=0 2024-05-17T22:43:09.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.530+0000 7fc7a7c15b80 1 client.15687 shutdown 2024-05-17T22:43:09.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.530+0000 7fc7a7c15b80 1 -- 172.21.15.12:0/3277678402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfb71189f0 msgr2=0x55bfb72b6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:09.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.530+0000 7fc7a7c15b80 1 --2- 172.21.15.12:0/3277678402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfb71189f0 0x55bfb72b6f60 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fc78c000f80 tx=0x7fc78c0a88e0 comp rx=0 tx=0).stop 2024-05-17T22:43:09.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.530+0000 7fc7a7c15b80 1 -- 172.21.15.12:0/3277678402 shutdown_connections 2024-05-17T22:43:09.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.530+0000 7fc7a7c15b80 1 --2- 172.21.15.12:0/3277678402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfb729c400 0x55bfb72b79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:09.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.530+0000 7fc7a7c15b80 1 --2- 172.21.15.12:0/3277678402 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55bfb731aa90 0x55bfb731ceb0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:09.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.530+0000 7fc7a7c15b80 1 --2- 172.21.15.12:0/3277678402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bfb71189f0 0x55bfb72b6f60 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:09.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.530+0000 7fc7a7c15b80 1 --2- 172.21.15.12:0/3277678402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfb7296b00 0x55bfb72b74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:09.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.530+0000 7fc7a7c15b80 1 -- 172.21.15.12:0/3277678402 >> 172.21.15.12:0/3277678402 conn(0x55bfb71787d0 msgr2=0x55bfb729ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:43:09.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.530+0000 7fc77a7fc700 0 client.15687 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:43:09.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.530+0000 7fc7a7c15b80 1 -- 172.21.15.12:0/3277678402 shutdown_connections 2024-05-17T22:43:09.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.530+0000 7fc7a7c15b80 1 -- 172.21.15.12:0/3277678402 wait complete. 2024-05-17T22:43:09.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.530+0000 7fc7a7c15b80 20 client.15687 trim_cache size 0 max 16384 2024-05-17T22:43:09.536 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:43:09.536 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-17T22:43:09.536 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-17T22:43:09.537 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:43:09.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.531+0000 7fc79923a700 1 -- 172.21.15.12:0/1662981111 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fc7800b2a90 msgr2=0x7fc7800b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:09.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.531+0000 7fc79923a700 1 --2- 172.21.15.12:0/1662981111 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fc7800b2a90 0x7fc7800b4fa0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x55bfb72921d0 tx=0x7fc7840047c0 comp rx=0 tx=0).stop 2024-05-17T22:43:09.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.531+0000 7fc79923a700 1 -- 172.21.15.12:0/1662981111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc79410a530 msgr2=0x7fc79406f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:09.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.531+0000 7fc79923a700 1 --2- 172.21.15.12:0/1662981111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc79410a530 0x7fc79406f6b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc78c0060b0 tx=0x7fc78c007f90 comp rx=0 tx=0).stop 2024-05-17T22:43:09.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.531+0000 7fc79923a700 1 -- 172.21.15.12:0/1662981111 shutdown_connections 2024-05-17T22:43:09.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.531+0000 7fc79923a700 1 --2- 172.21.15.12:0/1662981111 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fc7800b2a90 0x7fc7800b4fa0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:09.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.531+0000 7fc79923a700 1 --2- 172.21.15.12:0/1662981111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc79410f7b0 0x7fc794070130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:09.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.531+0000 7fc79923a700 1 --2- 172.21.15.12:0/1662981111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc79410a530 0x7fc79406f6b0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:09.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.531+0000 7fc79923a700 1 --2- 172.21.15.12:0/1662981111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc79410ee60 0x7fc79406fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:09.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.531+0000 7fc79923a700 1 -- 172.21.15.12:0/1662981111 >> 172.21.15.12:0/1662981111 conn(0x7fc794078570 msgr2=0x7fc794078c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:43:09.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.531+0000 7fc79923a700 1 -- 172.21.15.12:0/1662981111 shutdown_connections 2024-05-17T22:43:09.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:09.531+0000 7fc79923a700 1 -- 172.21.15.12:0/1662981111 wait complete. 2024-05-17T22:43:10.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:09 smithi151 ceph-mon[36101]: pgmap v556: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.8 MiB/s wr, 788 op/s 2024-05-17T22:43:10.268 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:09 smithi012 ceph-mon[31632]: pgmap v556: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.8 MiB/s wr, 788 op/s 2024-05-17T22:43:10.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:09 smithi096 ceph-mon[36065]: pgmap v556: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.8 MiB/s wr, 788 op/s 2024-05-17T22:43:10.320 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-17T22:43:10.321 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T22:43:10.788 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:43:11.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 230.0 s, 1171.6 tps, lat 84.491 ms stddev 96.878 2024-05-17T22:43:12.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:11 smithi151 ceph-mon[36101]: pgmap v557: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.4 MiB/s wr, 594 op/s 2024-05-17T22:43:12.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:11 smithi096 ceph-mon[36065]: pgmap v557: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.4 MiB/s wr, 594 op/s 2024-05-17T22:43:12.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:11 smithi012 ceph-mon[31632]: pgmap v557: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.4 MiB/s wr, 594 op/s 2024-05-17T22:43:13.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.202+0000 7f041ca65700 1 -- 172.21.15.12:0/3755246234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f041811d550 msgr2=0x7f041811f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:13.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.202+0000 7f041ca65700 1 --2- 172.21.15.12:0/3755246234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f041811d550 0x7f041811f970 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f041001c0d0 tx=0x7f041001c3e0 comp rx=0 tx=0).stop 2024-05-17T22:43:13.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.204+0000 7f041ca65700 1 -- 172.21.15.12:0/3755246234 shutdown_connections 2024-05-17T22:43:13.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.204+0000 7f041ca65700 1 --2- 172.21.15.12:0/3755246234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f041811d550 0x7f041811f970 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:13.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.204+0000 7f041ca65700 1 --2- 172.21.15.12:0/3755246234 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f041810a830 0x7f041810ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:13.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.204+0000 7f041ca65700 1 --2- 172.21.15.12:0/3755246234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f041810b180 0x7f041811d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:13.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.204+0000 7f041ca65700 1 -- 172.21.15.12:0/3755246234 >> 172.21.15.12:0/3755246234 conn(0x7f041806efc0 msgr2=0x7f041806f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:43:13.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.209+0000 7f041ca65700 1 -- 172.21.15.12:0/3755246234 shutdown_connections 2024-05-17T22:43:13.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.211+0000 7f041ca65700 1 -- 172.21.15.12:0/3755246234 wait complete. 2024-05-17T22:43:13.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.212+0000 7f041ca65700 1 Processor -- start 2024-05-17T22:43:13.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.217+0000 7f041ca65700 1 -- start start 2024-05-17T22:43:13.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.217+0000 7f041ca65700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f041810a830 0x7f041811cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:13.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.217+0000 7f041ca65700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f041810b180 0x7f0418119100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:13.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.217+0000 7f041ca65700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f041811d550 0x7f0418119760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:13.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.217+0000 7f041ca65700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0418115080 con 0x7f041811d550 2024-05-17T22:43:13.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.217+0000 7f041ca65700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0418114f00 con 0x7f041810b180 2024-05-17T22:43:13.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.217+0000 7f041ca65700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0418115200 con 0x7f041810a830 2024-05-17T22:43:13.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.220+0000 7f041659c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f041810a830 0x7f041811cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:13.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.220+0000 7f041659c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f041810a830 0x7f041811cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:32856/0 (socket says 172.21.15.12:32856) 2024-05-17T22:43:13.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.220+0000 7f041659c700 1 -- 172.21.15.12:0/2639633882 learned_addr learned my addr 172.21.15.12:0/2639633882 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:43:13.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.221+0000 7f040dd9b700 1 --2- 172.21.15.12:0/2639633882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f041810b180 0x7f0418119100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:13.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.221+0000 7f0416d9d700 1 --2- 172.21.15.12:0/2639633882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f041811d550 0x7f0418119760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:13.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.222+0000 7f0416d9d700 1 -- 172.21.15.12:0/2639633882 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f041810a830 msgr2=0x7f041811cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:13.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.222+0000 7f0416d9d700 1 --2- 172.21.15.12:0/2639633882 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f041810a830 0x7f041811cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:13.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.222+0000 7f0416d9d700 1 -- 172.21.15.12:0/2639633882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f041810b180 msgr2=0x7f0418119100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:13.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.222+0000 7f0416d9d700 1 --2- 172.21.15.12:0/2639633882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f041810b180 0x7f0418119100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:13.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.222+0000 7f0416d9d700 1 -- 172.21.15.12:0/2639633882 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04181ad250 con 0x7f041811d550 2024-05-17T22:43:13.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.223+0000 7f0416d9d700 1 --2- 172.21.15.12:0/2639633882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f041811d550 0x7f0418119760 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f041001c0d0 tx=0x7f04100199c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:43:13.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.225+0000 7f040ffff700 1 -- 172.21.15.12:0/2639633882 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0410030030 con 0x7f041811d550 2024-05-17T22:43:13.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.226+0000 7f041ca65700 1 -- 172.21.15.12:0/2639633882 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04181ad420 con 0x7f041811d550 2024-05-17T22:43:13.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.226+0000 7f041ca65700 1 -- 172.21.15.12:0/2639633882 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04181ad870 con 0x7f041811d550 2024-05-17T22:43:13.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.226+0000 7f040ffff700 1 -- 172.21.15.12:0/2639633882 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0410019df0 con 0x7f041811d550 2024-05-17T22:43:13.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.226+0000 7f040ffff700 1 -- 172.21.15.12:0/2639633882 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041002c700 con 0x7f041811d550 2024-05-17T22:43:13.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.230+0000 7f040ffff700 1 -- 172.21.15.12:0/2639633882 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0410033030 con 0x7f041811d550 2024-05-17T22:43:13.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.236+0000 7f040ffff700 1 --2- 172.21.15.12:0/2639633882 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f04040b2940 0x7f04040b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:13.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.237+0000 7f040ffff700 1 -- 172.21.15.12:0/2639633882 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f04100f4160 con 0x7f041811d550 2024-05-17T22:43:13.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.237+0000 7f041659c700 1 --2- 172.21.15.12:0/2639633882 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f04040b2940 0x7f04040b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:13.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.237+0000 7f041ca65700 1 -- 172.21.15.12:0/2639633882 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03f8005330 con 0x7f041811d550 2024-05-17T22:43:13.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.241+0000 7f041659c700 1 --2- 172.21.15.12:0/2639633882 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f04040b2940 0x7f04040b4e50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0400009ad0 tx=0x7f0400009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:43:13.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.249+0000 7f040ffff700 1 -- 172.21.15.12:0/2639633882 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f041003a030 con 0x7f041811d550 2024-05-17T22:43:13.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.706+0000 7f041ca65700 1 -- 172.21.15.12:0/2639633882 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03f8005a40 con 0x7f041811d550 2024-05-17T22:43:13.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.707+0000 7f040ffff700 1 -- 172.21.15.12:0/2639633882 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11873 (secure 0 0 0) 0x7f04100bbb20 con 0x7f041811d550 2024-05-17T22:43:13.710 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:43:13.711 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1407219319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1407219319},{"type":"v1","addr":"172.21.15.12:6835","nonce":1407219319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/630977886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":630977886},{"type":"v1","addr":"172.21.15.12:6837","nonce":630977886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3731914742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3731914742},{"type":"v1","addr":"172.21.15.12:6839","nonce":3731914742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/3642142501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3642142501},{"type":"v1","addr":"172.21.15.12:6841","nonce":3642142501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3554053059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3554053059},{"type":"v1","addr":"172.21.15.151:6833","nonce":3554053059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/4228325569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":4228325569},{"type":"v1","addr":"172.21.15.96:6833","nonce":4228325569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T22:34:42.642111+0000","modified":"2024-05-17T22:42:34.928791+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24461,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/542323754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":542323754},{"type":"v1","addr":"172.21.15.151:6835","nonce":542323754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1758442232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1758442232},{"type":"v1","addr":"172.21.15.96:6835","nonce":1758442232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3403324038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3403324038},{"type":"v1","addr":"172.21.15.151:6837","nonce":3403324038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.151:6839/2338353849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2338353849},{"type":"v1","addr":"172.21.15.151:6839","nonce":2338353849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.96:6837/4222574338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":4222574338},{"type":"v1","addr":"172.21.15.96:6837","nonce":4222574338}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/2986295519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2986295519},{"type":"v1","addr":"172.21.15.96:6839","nonce":2986295519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24457,24461,24482]},"id":1}]} 2024-05-17T22:43:13.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.712+0000 7f041ca65700 1 -- 172.21.15.12:0/2639633882 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f04040b2940 msgr2=0x7f04040b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:13.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.712+0000 7f041ca65700 1 --2- 172.21.15.12:0/2639633882 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f04040b2940 0x7f04040b4e50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0400009ad0 tx=0x7f0400009210 comp rx=0 tx=0).stop 2024-05-17T22:43:13.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.712+0000 7f041ca65700 1 -- 172.21.15.12:0/2639633882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f041811d550 msgr2=0x7f0418119760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:13.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.712+0000 7f041ca65700 1 --2- 172.21.15.12:0/2639633882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f041811d550 0x7f0418119760 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f041001c0d0 tx=0x7f04100199c0 comp rx=0 tx=0).stop 2024-05-17T22:43:13.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.712+0000 7f041ca65700 1 -- 172.21.15.12:0/2639633882 shutdown_connections 2024-05-17T22:43:13.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.712+0000 7f041ca65700 1 --2- 172.21.15.12:0/2639633882 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f04040b2940 0x7f04040b4e50 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:13.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.712+0000 7f041ca65700 1 --2- 172.21.15.12:0/2639633882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f041811d550 0x7f0418119760 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:13.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.712+0000 7f041ca65700 1 --2- 172.21.15.12:0/2639633882 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f041810a830 0x7f041811cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:13.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.712+0000 7f041ca65700 1 --2- 172.21.15.12:0/2639633882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f041810b180 0x7f0418119100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:13.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.712+0000 7f041ca65700 1 -- 172.21.15.12:0/2639633882 >> 172.21.15.12:0/2639633882 conn(0x7f041806efc0 msgr2=0x7f041811dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:43:13.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.712+0000 7f041ca65700 1 -- 172.21.15.12:0/2639633882 shutdown_connections 2024-05-17T22:43:13.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:13.712+0000 7f041ca65700 1 -- 172.21.15.12:0/2639633882 wait complete. 2024-05-17T22:43:13.716 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 33 2024-05-17T22:43:14.055 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:13 smithi012 ceph-mon[31632]: pgmap v558: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.4 MiB/s wr, 524 op/s 2024-05-17T22:43:14.055 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:13 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/2639633882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:43:14.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:13 smithi151 ceph-mon[36101]: pgmap v558: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.4 MiB/s wr, 524 op/s 2024-05-17T22:43:14.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:13 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/2639633882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:43:14.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:13 smithi096 ceph-mon[36065]: pgmap v558: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.4 MiB/s wr, 524 op/s 2024-05-17T22:43:14.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:13 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/2639633882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:43:14.531 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-17T22:43:14.958 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:43:16.142 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:15 smithi012 ceph-mon[31632]: pgmap v559: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.5 MiB/s wr, 589 op/s 2024-05-17T22:43:16.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:15 smithi151 ceph-mon[36101]: pgmap v559: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.5 MiB/s wr, 589 op/s 2024-05-17T22:43:16.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:15 smithi096 ceph-mon[36065]: pgmap v559: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.5 MiB/s wr, 589 op/s 2024-05-17T22:43:16.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 235.0 s, 1319.4 tps, lat 76.550 ms stddev 98.344 2024-05-17T22:43:17.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.499+0000 7fe4a8d16700 1 -- 172.21.15.12:0/2330874232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4a410ee60 msgr2=0x7fe4a410f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:17.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.499+0000 7fe4a8d16700 1 --2- 172.21.15.12:0/2330874232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4a410ee60 0x7fe4a410f270 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe498009990 tx=0x7fe498009ca0 comp rx=0 tx=0).stop 2024-05-17T22:43:17.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.502+0000 7fe4a8d16700 1 -- 172.21.15.12:0/2330874232 shutdown_connections 2024-05-17T22:43:17.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.502+0000 7fe4a8d16700 1 --2- 172.21.15.12:0/2330874232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4a410f7b0 0x7fe4a4117110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:17.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.502+0000 7fe4a8d16700 1 --2- 172.21.15.12:0/2330874232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4a410a530 0x7fe4a410a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:17.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.502+0000 7fe4a8d16700 1 --2- 172.21.15.12:0/2330874232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4a410ee60 0x7fe4a410f270 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:17.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.502+0000 7fe4a8d16700 1 -- 172.21.15.12:0/2330874232 >> 172.21.15.12:0/2330874232 conn(0x7fe4a4078570 msgr2=0x7fe4a4108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:43:17.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.503+0000 7fe4a8d16700 1 -- 172.21.15.12:0/2330874232 shutdown_connections 2024-05-17T22:43:17.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.503+0000 7fe4a8d16700 1 -- 172.21.15.12:0/2330874232 wait complete. 2024-05-17T22:43:17.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.504+0000 7fe4a8d16700 1 Processor -- start 2024-05-17T22:43:17.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.505+0000 7fe4a8d16700 1 -- start start 2024-05-17T22:43:17.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.505+0000 7fe4a8d16700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4a410a530 0x7fe4a41a0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:17.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.505+0000 7fe4a8d16700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4a410ee60 0x7fe4a41a1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:17.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.505+0000 7fe4a8d16700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4a410f7b0 0x7fe4a41a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:17.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.505+0000 7fe4a8d16700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a411a470 con 0x7fe4a410f7b0 2024-05-17T22:43:17.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.505+0000 7fe4a8d16700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a411a2f0 con 0x7fe4a410ee60 2024-05-17T22:43:17.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.505+0000 7fe4a8d16700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a411a5f0 con 0x7fe4a410a530 2024-05-17T22:43:17.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.505+0000 7fe4a259c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4a410a530 0x7fe4a41a0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:17.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.505+0000 7fe4a259c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4a410a530 0x7fe4a41a0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:32874/0 (socket says 172.21.15.12:32874) 2024-05-17T22:43:17.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.505+0000 7fe4a259c700 1 -- 172.21.15.12:0/3720259439 learned_addr learned my addr 172.21.15.12:0/3720259439 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:43:17.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.506+0000 7fe4a1d9b700 1 --2- 172.21.15.12:0/3720259439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4a410ee60 0x7fe4a41a1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:17.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.506+0000 7fe4a2d9d700 1 --2- 172.21.15.12:0/3720259439 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4a410f7b0 0x7fe4a41a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:17.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.506+0000 7fe4a1d9b700 1 -- 172.21.15.12:0/3720259439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4a410a530 msgr2=0x7fe4a41a0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:17.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.506+0000 7fe4a1d9b700 1 --2- 172.21.15.12:0/3720259439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4a410a530 0x7fe4a41a0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:17.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.506+0000 7fe4a1d9b700 1 -- 172.21.15.12:0/3720259439 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4a410f7b0 msgr2=0x7fe4a41a5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:17.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.506+0000 7fe4a1d9b700 1 --2- 172.21.15.12:0/3720259439 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4a410f7b0 0x7fe4a41a5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:17.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.506+0000 7fe4a1d9b700 1 -- 172.21.15.12:0/3720259439 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4a41a5da0 con 0x7fe4a410ee60 2024-05-17T22:43:17.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.506+0000 7fe4a2d9d700 1 --2- 172.21.15.12:0/3720259439 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4a410f7b0 0x7fe4a41a5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:43:17.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.507+0000 7fe4a1d9b700 1 --2- 172.21.15.12:0/3720259439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4a410ee60 0x7fe4a41a1470 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe4980098e0 tx=0x7fe498005e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:43:17.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.507+0000 7fe4937fe700 1 -- 172.21.15.12:0/3720259439 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe498005230 con 0x7fe4a410ee60 2024-05-17T22:43:17.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.507+0000 7fe4a8d16700 1 -- 172.21.15.12:0/3720259439 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4a41a6030 con 0x7fe4a410ee60 2024-05-17T22:43:17.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.507+0000 7fe4937fe700 1 -- 172.21.15.12:0/3720259439 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe498004680 con 0x7fe4a410ee60 2024-05-17T22:43:17.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.507+0000 7fe4937fe700 1 -- 172.21.15.12:0/3720259439 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe498003cb0 con 0x7fe4a410ee60 2024-05-17T22:43:17.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.507+0000 7fe4a8d16700 1 -- 172.21.15.12:0/3720259439 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4a41a64d0 con 0x7fe4a410ee60 2024-05-17T22:43:17.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.514+0000 7fe4b76f1b80 1 Processor -- start 2024-05-17T22:43:17.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.515+0000 7fe4b76f1b80 1 -- start start 2024-05-17T22:43:17.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.515+0000 7fe4b76f1b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc278049f0 0x55bc27988560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:17.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.515+0000 7fe4b76f1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc27988aa0 0x55bc27982c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:17.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.515+0000 7fe4b76f1b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bc27983170 0x55bc27983580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:17.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.515+0000 7fe4b76f1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55bc278b1290 con 0x55bc27988aa0 2024-05-17T22:43:17.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.515+0000 7fe4b76f1b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55bc278a0fd0 con 0x55bc278049f0 2024-05-17T22:43:17.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.515+0000 7fe4b76f1b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55bc278a4e70 con 0x55bc27983170 2024-05-17T22:43:17.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.516+0000 7fe4a2d9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc278049f0 0x55bc27988560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:17.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.516+0000 7fe4a2d9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc278049f0 0x55bc27988560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:45510/0 (socket says 172.21.15.12:45510) 2024-05-17T22:43:17.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.516+0000 7fe4a2d9d700 1 -- 172.21.15.12:0/1203857841 learned_addr learned my addr 172.21.15.12:0/1203857841 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:43:17.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.518+0000 7fe4937fe700 1 -- 172.21.15.12:0/3720259439 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe49804a060 con 0x7fe4a410ee60 2024-05-17T22:43:17.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.518+0000 7fe4a259c700 1 --2- 172.21.15.12:0/1203857841 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc27988aa0 0x55bc27982c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:17.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.518+0000 7fe4a2d9d700 1 -- 172.21.15.12:0/1203857841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bc27983170 msgr2=0x55bc27983580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:43:17.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.518+0000 7fe4a2d9d700 1 --2- 172.21.15.12:0/1203857841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bc27983170 0x55bc27983580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:17.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.518+0000 7fe4a2d9d700 1 -- 172.21.15.12:0/1203857841 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc27988aa0 msgr2=0x55bc27982c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:43:17.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.518+0000 7fe4a2d9d700 1 --2- 172.21.15.12:0/1203857841 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc27988aa0 0x55bc27982c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:17.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.518+0000 7fe4a2d9d700 1 -- 172.21.15.12:0/1203857841 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc2784a590 con 0x55bc278049f0 2024-05-17T22:43:17.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.519+0000 7fe4937fe700 1 --2- 172.21.15.12:0/3720259439 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe48c0b29c0 0x7fe48c0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:17.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.519+0000 7fe4937fe700 1 -- 172.21.15.12:0/3720259439 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe498104b50 con 0x7fe4a410ee60 2024-05-17T22:43:17.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.520+0000 7fe4a2d9d700 1 --2- 172.21.15.12:0/1203857841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc278049f0 0x55bc27988560 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe49c009cd0 tx=0x7fe49c007500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e446a53a230cd84 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:43:17.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.522+0000 7fe4a259c700 1 --2- 172.21.15.12:0/3720259439 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe48c0b29c0 0x7fe48c0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:17.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.522+0000 7fe490ff9700 1 -- 172.21.15.12:0/1203857841 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49c019070 con 0x55bc278049f0 2024-05-17T22:43:17.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.522+0000 7fe490ff9700 1 -- 172.21.15.12:0/1203857841 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe49c0148b0 con 0x55bc278049f0 2024-05-17T22:43:17.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.522+0000 7fe490ff9700 1 -- 172.21.15.12:0/1203857841 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49c0138c0 con 0x55bc278049f0 2024-05-17T22:43:17.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.524+0000 7fe4b76f1b80 1 -- 172.21.15.12:0/1203857841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc278049f0 msgr2=0x55bc27988560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:17.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.524+0000 7fe4b76f1b80 1 --2- 172.21.15.12:0/1203857841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc278049f0 0x55bc27988560 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe49c009cd0 tx=0x7fe49c007500 comp rx=0 tx=0).stop 2024-05-17T22:43:17.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.524+0000 7fe4b76f1b80 1 -- 172.21.15.12:0/1203857841 shutdown_connections 2024-05-17T22:43:17.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.524+0000 7fe4b76f1b80 1 --2- 172.21.15.12:0/1203857841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bc27983170 0x55bc27983580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:17.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.524+0000 7fe4b76f1b80 1 --2- 172.21.15.12:0/1203857841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc278049f0 0x55bc27988560 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:17.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.524+0000 7fe4b76f1b80 1 --2- 172.21.15.12:0/1203857841 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc27988aa0 0x55bc27982c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:17.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.524+0000 7fe4b76f1b80 1 -- 172.21.15.12:0/1203857841 >> 172.21.15.12:0/1203857841 conn(0x55bc27848f00 msgr2=0x55bc2797db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:43:17.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.524+0000 7fe4a259c700 1 --2- 172.21.15.12:0/3720259439 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe48c0b29c0 0x7fe48c0b4ed0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x55bc2797f320 tx=0x7fe49400b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:43:17.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.525+0000 7fe4b76f1b80 1 -- 172.21.15.12:0/1203857841 shutdown_connections 2024-05-17T22:43:17.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.525+0000 7fe4b76f1b80 1 -- 172.21.15.12:0/1203857841 wait complete. 2024-05-17T22:43:17.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.526+0000 7fe4b76f1b80 1 -- start start 2024-05-17T22:43:17.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.526+0000 7fe4b76f1b80 1 Processor -- start 2024-05-17T22:43:17.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.526+0000 7fe4b76f1b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc278049f0 0x55bc2798c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:17.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.526+0000 7fe4b76f1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc27983170 0x55bc2798c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:17.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.526+0000 7fe4b76f1b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bc27988aa0 0x55bc279a9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:17.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.526+0000 7fe4b76f1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55bc277a8150 con 0x55bc27983170 2024-05-17T22:43:17.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.526+0000 7fe4b76f1b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55bc27841e80 con 0x55bc278049f0 2024-05-17T22:43:17.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.526+0000 7fe4b76f1b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55bc279a03a0 con 0x55bc27988aa0 2024-05-17T22:43:17.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.526+0000 7fe4b76f1b80 10 client.0 _ll_register_callbacks cb 0x55bc2783bf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:43:17.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.528+0000 7fe4a1d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc27983170 0x55bc2798c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:17.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.528+0000 7fe4a1d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc27983170 0x55bc2798c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46576/0 (socket says 172.21.15.12:46576) 2024-05-17T22:43:17.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.528+0000 7fe4a1d9b700 1 -- 172.21.15.12:0/2678658522 learned_addr learned my addr 172.21.15.12:0/2678658522 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:43:17.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.528+0000 7fe4a2d9d700 1 --2- 172.21.15.12:0/2678658522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bc27988aa0 0x55bc279a9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:17.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.529+0000 7fe4a259c700 1 --2- 172.21.15.12:0/2678658522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc278049f0 0x55bc2798c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:17.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.531+0000 7fe4a1d9b700 1 -- 172.21.15.12:0/2678658522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bc27988aa0 msgr2=0x55bc279a9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:43:17.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.531+0000 7fe4a1d9b700 1 --2- 172.21.15.12:0/2678658522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bc27988aa0 0x55bc279a9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:17.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.531+0000 7fe4a1d9b700 1 -- 172.21.15.12:0/2678658522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc278049f0 msgr2=0x55bc2798c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:43:17.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.531+0000 7fe4a1d9b700 1 --2- 172.21.15.12:0/2678658522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc278049f0 0x55bc2798c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:17.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.531+0000 7fe4a1d9b700 1 -- 172.21.15.12:0/2678658522 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc277f1480 con 0x55bc27983170 2024-05-17T22:43:17.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.532+0000 7fe4a1d9b700 1 --2- 172.21.15.12:0/2678658522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc27983170 0x55bc2798c880 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe4980c9080 tx=0x7fe4980cba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e7454ebb5176c3d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:43:17.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.532+0000 7fe489ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:43:17.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.533+0000 7fe489ffb700 1 -- 172.21.15.12:0/2678658522 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4980d01c0 con 0x55bc27983170 2024-05-17T22:43:17.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.533+0000 7fe4b76f1b80 1 -- 172.21.15.12:0/2678658522 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe49c0148b0 con 0x55bc27983170 2024-05-17T22:43:17.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.533+0000 7fe489ffb700 1 -- 172.21.15.12:0/2678658522 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe4980c8400 con 0x55bc27983170 2024-05-17T22:43:17.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.536+0000 7fe489ffb700 1 -- 172.21.15.12:0/2678658522 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe4980d4be0 con 0x55bc27983170 2024-05-17T22:43:17.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.538+0000 7fe4b76f1b80 10 client.15699 fetch_fsmap learned FSMap version 33 2024-05-17T22:43:17.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.538+0000 7fe4b76f1b80 1 -- 172.21.15.12:0/2678658522 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55bc279aaf20 con 0x55bc27983170 2024-05-17T22:43:17.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.539+0000 7fe489ffb700 1 -- 172.21.15.12:0/2678658522 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe49c0148b0 con 0x55bc27983170 2024-05-17T22:43:17.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.540+0000 7fe489ffb700 1 -- 172.21.15.12:0/2678658522 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7fe4980d2460 con 0x55bc27983170 2024-05-17T22:43:17.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.540+0000 7fe4b76f1b80 10 client.15699 fetch_fsmap finished waiting for FSMap version 33 2024-05-17T22:43:17.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.540+0000 7fe4b76f1b80 10 client.15699 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-17T22:43:17.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.540+0000 7fe4b76f1b80 20 client.15699 populate_metadata read hostname 'smithi012' 2024-05-17T22:43:17.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.540+0000 7fe4b76f1b80 1 --2- 172.21.15.12:0/2678658522 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55bc279ad490 0x55bc279af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:17.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.540+0000 7fe4b76f1b80 4 client.15699 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:43:17.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.540+0000 7fe4b76f1b80 1 -- 172.21.15.12:0/2678658522 --> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc278b31e0 con 0x55bc279ad490 2024-05-17T22:43:17.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.542+0000 7fe4a259c700 1 --2- 172.21.15.12:0/2678658522 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55bc279ad490 0x55bc279af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:17.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.546+0000 7fe4a259c700 1 --2- 172.21.15.12:0/2678658522 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55bc279ad490 0x55bc279af8e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=81838425bca5b3bb server_cookie=b50398732a0fb11b in_seq=0 out_seq=0 2024-05-17T22:43:17.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.546+0000 7fe489ffb700 10 client.15699 ms_handle_connect on v2:172.21.15.151:6838/2338353849 2024-05-17T22:43:17.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.551+0000 7fe489ffb700 1 -- 172.21.15.12:0/2678658522 <== mds.2 v2:172.21.15.151:6838/2338353849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bc278b31e0 con 0x55bc279ad490 2024-05-17T22:43:17.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.551+0000 7fe489ffb700 10 client.15699 handle_command_reply: tid=0 2024-05-17T22:43:17.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.551+0000 7fe4b76f1b80 1 client.15699 shutdown 2024-05-17T22:43:17.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.554+0000 7fe4b76f1b80 1 -- 172.21.15.12:0/2678658522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc27983170 msgr2=0x55bc2798c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:17.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.554+0000 7fe4b76f1b80 1 --2- 172.21.15.12:0/2678658522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc27983170 0x55bc2798c880 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe4980c9080 tx=0x7fe4980cba50 comp rx=0 tx=0).stop 2024-05-17T22:43:17.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.557+0000 7fe4b76f1b80 1 -- 172.21.15.12:0/2678658522 shutdown_connections 2024-05-17T22:43:17.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.557+0000 7fe4b76f1b80 1 --2- 172.21.15.12:0/2678658522 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55bc279ad490 0x55bc279af8e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:17.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.558+0000 7fe4b76f1b80 1 --2- 172.21.15.12:0/2678658522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bc27988aa0 0x55bc279a9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:17.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.558+0000 7fe4b76f1b80 1 --2- 172.21.15.12:0/2678658522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc278049f0 0x55bc2798c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:17.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.558+0000 7fe4b76f1b80 1 --2- 172.21.15.12:0/2678658522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc27983170 0x55bc2798c880 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:17.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.558+0000 7fe4b76f1b80 1 -- 172.21.15.12:0/2678658522 >> 172.21.15.12:0/2678658522 conn(0x55bc27848f00 msgr2=0x55bc27977ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:43:17.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.558+0000 7fe489ffb700 0 client.15699 ms_handle_reset on v2:172.21.15.151:6838/2338353849 2024-05-17T22:43:17.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.561+0000 7fe4b76f1b80 1 -- 172.21.15.12:0/2678658522 shutdown_connections 2024-05-17T22:43:17.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.565+0000 7fe4b76f1b80 1 -- 172.21.15.12:0/2678658522 wait complete. 2024-05-17T22:43:17.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.569+0000 7fe4b76f1b80 20 client.15699 trim_cache size 0 max 16384 2024-05-17T22:43:17.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.599+0000 7fe4b76f1b80 1 Processor -- start 2024-05-17T22:43:17.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.599+0000 7fe4b76f1b80 1 -- start start 2024-05-17T22:43:17.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.600+0000 7fe4b76f1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc278049f0 0x55bc27982c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:17.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.600+0000 7fe4b76f1b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bc27983170 0x55bc2798c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:17.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.600+0000 7fe4b76f1b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc27988aa0 0x55bc27977eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:17.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.600+0000 7fe4b76f1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a407ea10 con 0x55bc278049f0 2024-05-17T22:43:17.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.600+0000 7fe4b76f1b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a407c240 con 0x55bc27988aa0 2024-05-17T22:43:17.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.600+0000 7fe4b76f1b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a4073a60 con 0x55bc27983170 2024-05-17T22:43:17.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.600+0000 7fe4a1d9b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bc27983170 0x55bc2798c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:17.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.600+0000 7fe4a1d9b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bc27983170 0x55bc2798c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:32892/0 (socket says 172.21.15.12:32892) 2024-05-17T22:43:17.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.600+0000 7fe4a1d9b700 1 -- 172.21.15.12:0/2694282295 learned_addr learned my addr 172.21.15.12:0/2694282295 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:43:17.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.601+0000 7fe4a2d9d700 1 --2- 172.21.15.12:0/2694282295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc27988aa0 0x55bc27977eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:17.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.601+0000 7fe4a1d9b700 1 -- 172.21.15.12:0/2694282295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc27988aa0 msgr2=0x55bc27977eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:43:17.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.601+0000 7fe4a1d9b700 1 --2- 172.21.15.12:0/2694282295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc27988aa0 0x55bc27977eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:17.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.601+0000 7fe4a1d9b700 1 -- 172.21.15.12:0/2694282295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc278049f0 msgr2=0x55bc27982c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T22:43:17.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.601+0000 7fe4a1d9b700 1 --2- 172.21.15.12:0/2694282295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc278049f0 0x55bc27982c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:17.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.601+0000 7fe4a1d9b700 1 -- 172.21.15.12:0/2694282295 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc279880d0 con 0x55bc27983170 2024-05-17T22:43:17.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.602+0000 7fe4a1d9b700 1 --2- 172.21.15.12:0/2694282295 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bc27983170 0x55bc2798c890 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe4980cbd50 tx=0x7fe498000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12e8bc718fb4e25b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:43:17.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.603+0000 7fe48bfff700 1 -- 172.21.15.12:0/2694282295 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4980d01c0 con 0x55bc27983170 2024-05-17T22:43:17.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.603+0000 7fe48bfff700 1 -- 172.21.15.12:0/2694282295 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4980c8400 con 0x55bc27983170 2024-05-17T22:43:17.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.604+0000 7fe48bfff700 1 -- 172.21.15.12:0/2694282295 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4980d23d0 con 0x55bc27983170 2024-05-17T22:43:17.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.604+0000 7fe4b76f1b80 1 -- 172.21.15.12:0/2694282295 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bc27983170 msgr2=0x55bc2798c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:17.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.604+0000 7fe4b76f1b80 1 --2- 172.21.15.12:0/2694282295 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bc27983170 0x55bc2798c890 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe4980cbd50 tx=0x7fe498000bc0 comp rx=0 tx=0).stop 2024-05-17T22:43:17.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.604+0000 7fe4b76f1b80 1 -- 172.21.15.12:0/2694282295 shutdown_connections 2024-05-17T22:43:17.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.604+0000 7fe4b76f1b80 1 --2- 172.21.15.12:0/2694282295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc27988aa0 0x55bc27977eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:17.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.604+0000 7fe4b76f1b80 1 --2- 172.21.15.12:0/2694282295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc278049f0 0x55bc27982c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:17.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.604+0000 7fe4b76f1b80 1 --2- 172.21.15.12:0/2694282295 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bc27983170 0x55bc2798c890 secure :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe4980cbd50 tx=0x7fe498000bc0 comp rx=0 tx=0).stop 2024-05-17T22:43:17.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.604+0000 7fe4b76f1b80 1 -- 172.21.15.12:0/2694282295 >> 172.21.15.12:0/2694282295 conn(0x55bc27848f00 msgr2=0x55bc27986750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:43:17.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.606+0000 7fe4b76f1b80 1 -- 172.21.15.12:0/2694282295 shutdown_connections 2024-05-17T22:43:17.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.606+0000 7fe4b76f1b80 1 -- 172.21.15.12:0/2694282295 wait complete. 2024-05-17T22:43:17.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.607+0000 7fe4b76f1b80 1 -- start start 2024-05-17T22:43:17.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.607+0000 7fe4b76f1b80 1 Processor -- start 2024-05-17T22:43:17.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.607+0000 7fe4b76f1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc278049f0 0x55bc279a0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:17.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.607+0000 7fe4b76f1b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc27988aa0 0x55bc279a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:17.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.607+0000 7fe4b76f1b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bc279a19f0 0x55bc27a16c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:17.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.607+0000 7fe4b76f1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a40740d0 con 0x55bc278049f0 2024-05-17T22:43:17.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.607+0000 7fe4b76f1b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a4075540 con 0x55bc27988aa0 2024-05-17T22:43:17.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.607+0000 7fe4b76f1b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a40747a0 con 0x55bc279a19f0 2024-05-17T22:43:17.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.607+0000 7fe4b76f1b80 10 client.0 _ll_register_callbacks cb 0x55bc2783bf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:43:17.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.609+0000 7fe4a1d9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc27988aa0 0x55bc279a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:17.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.609+0000 7fe4a1d9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc27988aa0 0x55bc279a14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:45542/0 (socket says 172.21.15.12:45542) 2024-05-17T22:43:17.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.609+0000 7fe4a1d9b700 1 -- 172.21.15.12:0/41605458 learned_addr learned my addr 172.21.15.12:0/41605458 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:43:17.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.609+0000 7fe4a2d9d700 1 --2- 172.21.15.12:0/41605458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bc279a19f0 0x55bc27a16c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:17.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.610+0000 7fe4a1d9b700 1 -- 172.21.15.12:0/41605458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bc279a19f0 msgr2=0x55bc27a16c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:43:17.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.610+0000 7fe4a1d9b700 1 --2- 172.21.15.12:0/41605458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bc279a19f0 0x55bc27a16c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:17.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.610+0000 7fe4a1d9b700 1 -- 172.21.15.12:0/41605458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc278049f0 msgr2=0x55bc279a0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:43:17.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.610+0000 7fe4a1d9b700 1 --2- 172.21.15.12:0/41605458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc278049f0 0x55bc279a0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:17.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.610+0000 7fe4a1d9b700 1 -- 172.21.15.12:0/41605458 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc279a5ec0 con 0x55bc27988aa0 2024-05-17T22:43:17.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.610+0000 7fe4a1d9b700 1 --2- 172.21.15.12:0/41605458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc27988aa0 0x55bc279a14b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe4980c9080 tx=0x7fe498009380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d1c397edb970869 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:43:17.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.611+0000 7fe489ffb700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:43:17.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.611+0000 7fe489ffb700 1 -- 172.21.15.12:0/41605458 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4980d4e30 con 0x55bc27988aa0 2024-05-17T22:43:17.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.611+0000 7fe4b76f1b80 1 -- 172.21.15.12:0/41605458 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4980c8400 con 0x55bc27988aa0 2024-05-17T22:43:17.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.611+0000 7fe489ffb700 1 -- 172.21.15.12:0/41605458 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe498005590 con 0x55bc27988aa0 2024-05-17T22:43:17.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.612+0000 7fe489ffb700 1 -- 172.21.15.12:0/41605458 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe4980c78d0 con 0x55bc27988aa0 2024-05-17T22:43:17.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.612+0000 7fe4b76f1b80 10 client.25361 fetch_fsmap learned FSMap version 33 2024-05-17T22:43:17.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.612+0000 7fe4b76f1b80 1 -- 172.21.15.12:0/41605458 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55bc279a5ec0 con 0x55bc27988aa0 2024-05-17T22:43:17.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.613+0000 7fe489ffb700 1 -- 172.21.15.12:0/41605458 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4980c8400 con 0x55bc27988aa0 2024-05-17T22:43:17.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.614+0000 7fe489ffb700 1 -- 172.21.15.12:0/41605458 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7fe4980d4750 con 0x55bc27988aa0 2024-05-17T22:43:17.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.616+0000 7fe4b76f1b80 10 client.25361 fetch_fsmap finished waiting for FSMap version 33 2024-05-17T22:43:17.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.616+0000 7fe4b76f1b80 10 client.25361 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-17T22:43:17.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.616+0000 7fe4b76f1b80 20 client.25361 populate_metadata read hostname 'smithi012' 2024-05-17T22:43:17.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.616+0000 7fe4b76f1b80 1 --2- 172.21.15.12:0/41605458 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55bc27a182f0 0x55bc27a1a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:17.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.616+0000 7fe4b76f1b80 4 client.25361 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:43:17.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.616+0000 7fe4b76f1b80 1 -- 172.21.15.12:0/41605458 --> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bc278ac9c0 con 0x55bc27a182f0 2024-05-17T22:43:17.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.618+0000 7fe4a259c700 1 --2- 172.21.15.12:0/41605458 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55bc27a182f0 0x55bc27a1a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:17.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.625+0000 7fe489ffb700 10 client.25361 ms_handle_connect on v2:172.21.15.151:6838/2338353849 2024-05-17T22:43:17.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.625+0000 7fe4a259c700 1 --2- 172.21.15.12:0/41605458 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55bc27a182f0 0x55bc27a1a710 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=60720ec8be8a8aa1 server_cookie=e907b54a4e03068e in_seq=0 out_seq=0 2024-05-17T22:43:17.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.625+0000 7fe489ffb700 1 -- 172.21.15.12:0/41605458 <== mds.2 v2:172.21.15.151:6838/2338353849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bc278ac9c0 con 0x55bc27a182f0 2024-05-17T22:43:17.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.625+0000 7fe489ffb700 10 client.25361 handle_command_reply: tid=0 2024-05-17T22:43:17.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.625+0000 7fe4b76f1b80 1 client.25361 shutdown 2024-05-17T22:43:17.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.625+0000 7fe4b76f1b80 1 -- 172.21.15.12:0/41605458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc27988aa0 msgr2=0x55bc279a14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:17.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.625+0000 7fe4b76f1b80 1 --2- 172.21.15.12:0/41605458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc27988aa0 0x55bc279a14b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe4980c9080 tx=0x7fe498009380 comp rx=0 tx=0).stop 2024-05-17T22:43:17.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.625+0000 7fe4b76f1b80 1 -- 172.21.15.12:0/41605458 shutdown_connections 2024-05-17T22:43:17.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.625+0000 7fe4b76f1b80 1 --2- 172.21.15.12:0/41605458 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55bc27a182f0 0x55bc27a1a710 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:17.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.625+0000 7fe4b76f1b80 1 --2- 172.21.15.12:0/41605458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bc279a19f0 0x55bc27a16c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:17.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.625+0000 7fe4b76f1b80 1 --2- 172.21.15.12:0/41605458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc278049f0 0x55bc279a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:17.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.625+0000 7fe4b76f1b80 1 --2- 172.21.15.12:0/41605458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc27988aa0 0x55bc279a14b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:17.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.625+0000 7fe489ffb700 0 client.25361 ms_handle_reset on v2:172.21.15.151:6838/2338353849 2024-05-17T22:43:17.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.625+0000 7fe4b76f1b80 1 -- 172.21.15.12:0/41605458 >> 172.21.15.12:0/41605458 conn(0x55bc27848f00 msgr2=0x55bc27986520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:43:17.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.626+0000 7fe4b76f1b80 1 -- 172.21.15.12:0/41605458 shutdown_connections 2024-05-17T22:43:17.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.626+0000 7fe4b76f1b80 1 -- 172.21.15.12:0/41605458 wait complete. 2024-05-17T22:43:17.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.626+0000 7fe4b76f1b80 20 client.25361 trim_cache size 0 max 16384 2024-05-17T22:43:17.630 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:43:17.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.627+0000 7fe4a8d16700 1 -- 172.21.15.12:0/3720259439 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe48c0b29c0 msgr2=0x7fe48c0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:17.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.627+0000 7fe4a8d16700 1 --2- 172.21.15.12:0/3720259439 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe48c0b29c0 0x7fe48c0b4ed0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x55bc2797f320 tx=0x7fe49400b500 comp rx=0 tx=0).stop 2024-05-17T22:43:17.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.627+0000 7fe4a8d16700 1 -- 172.21.15.12:0/3720259439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4a410ee60 msgr2=0x7fe4a41a1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:17.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.627+0000 7fe4a8d16700 1 --2- 172.21.15.12:0/3720259439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4a410ee60 0x7fe4a41a1470 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe4980098e0 tx=0x7fe498005e10 comp rx=0 tx=0).stop 2024-05-17T22:43:17.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.627+0000 7fe4a8d16700 1 -- 172.21.15.12:0/3720259439 shutdown_connections 2024-05-17T22:43:17.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.627+0000 7fe4a8d16700 1 --2- 172.21.15.12:0/3720259439 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe48c0b29c0 0x7fe48c0b4ed0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:17.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.627+0000 7fe4a8d16700 1 --2- 172.21.15.12:0/3720259439 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4a410f7b0 0x7fe4a41a5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:17.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.627+0000 7fe4a8d16700 1 --2- 172.21.15.12:0/3720259439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4a410a530 0x7fe4a41a0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:17.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.627+0000 7fe4a8d16700 1 --2- 172.21.15.12:0/3720259439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4a410ee60 0x7fe4a41a1470 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:17.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.627+0000 7fe4a8d16700 1 -- 172.21.15.12:0/3720259439 >> 172.21.15.12:0/3720259439 conn(0x7fe4a4078570 msgr2=0x7fe4a4078c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:43:17.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.627+0000 7fe4a8d16700 1 -- 172.21.15.12:0/3720259439 shutdown_connections 2024-05-17T22:43:17.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:17.627+0000 7fe4a8d16700 1 -- 172.21.15.12:0/3720259439 wait complete. 2024-05-17T22:43:18.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:17 smithi012 ceph-mon[31632]: pgmap v560: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.5 MiB/s wr, 591 op/s 2024-05-17T22:43:18.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:17 smithi151 ceph-mon[36101]: pgmap v560: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.5 MiB/s wr, 591 op/s 2024-05-17T22:43:18.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:17 smithi096 ceph-mon[36065]: pgmap v560: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.5 MiB/s wr, 591 op/s 2024-05-17T22:43:18.578 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-17T22:43:19.078 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:43:20.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:19 smithi151 ceph-mon[36101]: pgmap v561: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 MiB/s wr, 634 op/s 2024-05-17T22:43:20.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:19 smithi096 ceph-mon[36065]: pgmap v561: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 MiB/s wr, 634 op/s 2024-05-17T22:43:20.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:19 smithi012 ceph-mon[31632]: pgmap v561: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 MiB/s wr, 634 op/s 2024-05-17T22:43:21.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.239+0000 7fd6ecee0700 1 -- 172.21.15.12:0/1033146949 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6e8076520 msgr2=0x7fd6e8114750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:21.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.239+0000 7fd6ecee0700 1 --2- 172.21.15.12:0/1033146949 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6e8076520 0x7fd6e8114750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0009f90 tx=0x7fd6e000b330 comp rx=0 tx=0).stop 2024-05-17T22:43:21.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.248+0000 7fd6ecee0700 1 -- 172.21.15.12:0/1033146949 shutdown_connections 2024-05-17T22:43:21.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.248+0000 7fd6ecee0700 1 --2- 172.21.15.12:0/1033146949 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6e8114c90 0x7fd6e81170b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:21.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.248+0000 7fd6ecee0700 1 --2- 172.21.15.12:0/1033146949 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6e8075bd0 0x7fd6e8075fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:21.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.248+0000 7fd6ecee0700 1 --2- 172.21.15.12:0/1033146949 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6e8076520 0x7fd6e8114750 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:21.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.248+0000 7fd6ecee0700 1 -- 172.21.15.12:0/1033146949 >> 172.21.15.12:0/1033146949 conn(0x7fd6e80ff990 msgr2=0x7fd6e8101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:43:21.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.250+0000 7fd6ecee0700 1 -- 172.21.15.12:0/1033146949 shutdown_connections 2024-05-17T22:43:21.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.251+0000 7fd6ecee0700 1 -- 172.21.15.12:0/1033146949 wait complete. 2024-05-17T22:43:21.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.252+0000 7fd6ecee0700 1 Processor -- start 2024-05-17T22:43:21.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.252+0000 7fd6ecee0700 1 -- start start 2024-05-17T22:43:21.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.252+0000 7fd6ecee0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6e8075bd0 0x7fd6e8114680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:21.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.252+0000 7fd6ecee0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6e8076520 0x7fd6e810d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:21.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.252+0000 7fd6ecee0700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6e8114c90 0x7fd6e810de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:21.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.252+0000 7fd6ecee0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e811a460 con 0x7fd6e8076520 2024-05-17T22:43:21.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.252+0000 7fd6ecee0700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e811a2e0 con 0x7fd6e8075bd0 2024-05-17T22:43:21.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.252+0000 7fd6ecee0700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e811a5e0 con 0x7fd6e8114c90 2024-05-17T22:43:21.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.253+0000 7fd6e5d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6e8076520 0x7fd6e810d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:21.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.253+0000 7fd6e5d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6e8076520 0x7fd6e810d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44648/0 (socket says 172.21.15.12:44648) 2024-05-17T22:43:21.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.253+0000 7fd6e5d9b700 1 -- 172.21.15.12:0/1541694287 learned_addr learned my addr 172.21.15.12:0/1541694287 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:43:21.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.255+0000 7fd6e5d9b700 1 -- 172.21.15.12:0/1541694287 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6e8114c90 msgr2=0x7fd6e810de00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-17T22:43:21.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.255+0000 7fd6e5d9b700 1 --2- 172.21.15.12:0/1541694287 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6e8114c90 0x7fd6e810de00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:21.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.255+0000 7fd6e5d9b700 1 -- 172.21.15.12:0/1541694287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6e8075bd0 msgr2=0x7fd6e8114680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-17T22:43:21.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.255+0000 7fd6e5d9b700 1 --2- 172.21.15.12:0/1541694287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6e8075bd0 0x7fd6e8114680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:21.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.255+0000 7fd6e5d9b700 1 -- 172.21.15.12:0/1541694287 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e81121b0 con 0x7fd6e8076520 2024-05-17T22:43:21.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.256+0000 7fd6e5d9b700 1 --2- 172.21.15.12:0/1541694287 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6e8076520 0x7fd6e810d7b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0009550 tx=0x7fd6e0004e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:43:21.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.256+0000 7fd6d77fe700 1 -- 172.21.15.12:0/1541694287 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0021070 con 0x7fd6e8076520 2024-05-17T22:43:21.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.256+0000 7fd6ecee0700 1 -- 172.21.15.12:0/1541694287 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6e8112440 con 0x7fd6e8076520 2024-05-17T22:43:21.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.257+0000 7fd6ecee0700 1 -- 172.21.15.12:0/1541694287 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6e81128c0 con 0x7fd6e8076520 2024-05-17T22:43:21.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.257+0000 7fd6d77fe700 1 -- 172.21.15.12:0/1541694287 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6e0007c40 con 0x7fd6e8076520 2024-05-17T22:43:21.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.257+0000 7fd6d77fe700 1 -- 172.21.15.12:0/1541694287 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0005370 con 0x7fd6e8076520 2024-05-17T22:43:21.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.265+0000 7fd6d77fe700 1 -- 172.21.15.12:0/1541694287 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd6e0024070 con 0x7fd6e8076520 2024-05-17T22:43:21.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.267+0000 7fd6d77fe700 1 --2- 172.21.15.12:0/1541694287 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd6d00b2a90 0x7fd6d00b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:21.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.267+0000 7fd6d77fe700 1 -- 172.21.15.12:0/1541694287 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fd6e00e45d0 con 0x7fd6e8076520 2024-05-17T22:43:21.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.268+0000 7fd6fb8bbb80 1 Processor -- start 2024-05-17T22:43:21.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.269+0000 7fd6fb8bbb80 1 -- start start 2024-05-17T22:43:21.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.269+0000 7fd6fb8bbb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625edf9d9f0 0x5625ee121560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:21.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.269+0000 7fd6e659c700 1 --2- 172.21.15.12:0/1541694287 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd6d00b2a90 0x7fd6d00b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:21.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.270+0000 7fd6fb8bbb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5625ee121aa0 0x5625ee11bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:21.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.270+0000 7fd6e659c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625edf9d9f0 0x5625ee121560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:21.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.270+0000 7fd6e659c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625edf9d9f0 0x5625ee121560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:51244/0 (socket says 172.21.15.12:51244) 2024-05-17T22:43:21.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.270+0000 7fd6e659c700 1 -- 172.21.15.12:0/131720610 learned_addr learned my addr 172.21.15.12:0/131720610 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:43:21.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.270+0000 7fd6fb8bbb80 1 --2- 172.21.15.12:0/131720610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625ee11c170 0x5625ee11c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:21.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.271+0000 7fd6fb8bbb80 1 -- 172.21.15.12:0/131720610 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5625ee04a290 con 0x5625ee11c170 2024-05-17T22:43:21.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.271+0000 7fd6fb8bbb80 1 -- 172.21.15.12:0/131720610 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5625ee039fd0 con 0x5625edf9d9f0 2024-05-17T22:43:21.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.271+0000 7fd6fb8bbb80 1 -- 172.21.15.12:0/131720610 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5625ee03de70 con 0x5625ee121aa0 2024-05-17T22:43:21.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.271+0000 7fd6e5d9b700 1 --2- 172.21.15.12:0/131720610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5625ee121aa0 0x5625ee11bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:21.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.272+0000 7fd6e659c700 1 -- 172.21.15.12:0/131720610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5625ee121aa0 msgr2=0x5625ee11bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:43:21.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.272+0000 7fd6e659c700 1 --2- 172.21.15.12:0/131720610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5625ee121aa0 0x5625ee11bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:21.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.272+0000 7fd6e659c700 1 -- 172.21.15.12:0/131720610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625ee11c170 msgr2=0x5625ee11c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:43:21.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.272+0000 7fd6e659c700 1 --2- 172.21.15.12:0/131720610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625ee11c170 0x5625ee11c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:21.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.272+0000 7fd6e659c700 1 -- 172.21.15.12:0/131720610 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625edfe3590 con 0x5625edf9d9f0 2024-05-17T22:43:21.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.272+0000 7fd6e659c700 1 --2- 172.21.15.12:0/131720610 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625edf9d9f0 0x5625ee121560 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8013ef0 tx=0x7fd6d8018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6da975ed6a5657e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:43:21.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.272+0000 7fd6e659c700 1 --2- 172.21.15.12:0/1541694287 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd6d00b2a90 0x7fd6d00b4fa0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd6d80137a0 tx=0x7fd6d800a960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:43:21.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.272+0000 7fd6d4ff9700 1 -- 172.21.15.12:0/131720610 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d8011c90 con 0x5625edf9d9f0 2024-05-17T22:43:21.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.273+0000 7fd6d4ff9700 1 -- 172.21.15.12:0/131720610 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6d80044d0 con 0x5625edf9d9f0 2024-05-17T22:43:21.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.273+0000 7fd6d4ff9700 1 -- 172.21.15.12:0/131720610 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d8003c00 con 0x5625edf9d9f0 2024-05-17T22:43:21.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.274+0000 7fd6fb8bbb80 1 -- 172.21.15.12:0/131720610 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625edf9d9f0 msgr2=0x5625ee121560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:21.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.274+0000 7fd6fb8bbb80 1 --2- 172.21.15.12:0/131720610 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625edf9d9f0 0x5625ee121560 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8013ef0 tx=0x7fd6d8018000 comp rx=0 tx=0).stop 2024-05-17T22:43:21.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.274+0000 7fd6fb8bbb80 1 -- 172.21.15.12:0/131720610 shutdown_connections 2024-05-17T22:43:21.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.274+0000 7fd6fb8bbb80 1 --2- 172.21.15.12:0/131720610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625ee11c170 0x5625ee11c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:21.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.274+0000 7fd6fb8bbb80 1 --2- 172.21.15.12:0/131720610 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625edf9d9f0 0x5625ee121560 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:21.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.274+0000 7fd6fb8bbb80 1 --2- 172.21.15.12:0/131720610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5625ee121aa0 0x5625ee11bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:21.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.274+0000 7fd6fb8bbb80 1 -- 172.21.15.12:0/131720610 >> 172.21.15.12:0/131720610 conn(0x5625edfe1f00 msgr2=0x5625ee116b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:43:21.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.276+0000 7fd6fb8bbb80 1 -- 172.21.15.12:0/131720610 shutdown_connections 2024-05-17T22:43:21.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.276+0000 7fd6fb8bbb80 1 -- 172.21.15.12:0/131720610 wait complete. 2024-05-17T22:43:21.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.277+0000 7fd6fb8bbb80 1 -- start start 2024-05-17T22:43:21.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.277+0000 7fd6fb8bbb80 1 Processor -- start 2024-05-17T22:43:21.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.278+0000 7fd6fb8bbb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5625edf9d9f0 0x5625ee1251d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:21.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.278+0000 7fd6fb8bbb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625ee11c170 0x5625ee125710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:21.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.278+0000 7fd6fb8bbb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625ee121aa0 0x5625ee125c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:21.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.278+0000 7fd6fb8bbb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5625edf41150 con 0x5625ee121aa0 2024-05-17T22:43:21.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.278+0000 7fd6fb8bbb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5625edfdae80 con 0x5625ee11c170 2024-05-17T22:43:21.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.278+0000 7fd6fb8bbb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5625ee128390 con 0x5625edf9d9f0 2024-05-17T22:43:21.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.279+0000 7fd6e6d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625ee121aa0 0x5625ee125c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:21.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.279+0000 7fd6e6d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625ee121aa0 0x5625ee125c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44674/0 (socket says 172.21.15.12:44674) 2024-05-17T22:43:21.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.279+0000 7fd6e6d9d700 1 -- 172.21.15.12:0/2487491337 learned_addr learned my addr 172.21.15.12:0/2487491337 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:43:21.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.279+0000 7fd6fb8bbb80 10 client.0 _ll_register_callbacks cb 0x5625edfd4f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:43:21.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.279+0000 7fd6e6d9d700 1 -- 172.21.15.12:0/2487491337 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5625edf9d9f0 msgr2=0x5625ee1251d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T22:43:21.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.279+0000 7fd6e6d9d700 1 --2- 172.21.15.12:0/2487491337 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5625edf9d9f0 0x5625ee1251d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:21.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.279+0000 7fd6e6d9d700 1 -- 172.21.15.12:0/2487491337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625ee11c170 msgr2=0x5625ee125710 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T22:43:21.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.279+0000 7fd6e6d9d700 1 --2- 172.21.15.12:0/2487491337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625ee11c170 0x5625ee125710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:21.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.279+0000 7fd6e6d9d700 1 -- 172.21.15.12:0/2487491337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625edf8a480 con 0x5625ee121aa0 2024-05-17T22:43:21.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.280+0000 7fd6e6d9d700 1 --2- 172.21.15.12:0/2487491337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625ee121aa0 0x5625ee125c50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc00bb90 tx=0x7fd6dc00bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a6d53af55aa438 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:43:21.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.281+0000 7fd6cdffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:43:21.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.281+0000 7fd6cdffb700 1 -- 172.21.15.12:0/2487491337 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc016070 con 0x5625ee121aa0 2024-05-17T22:43:21.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.281+0000 7fd6cdffb700 1 -- 172.21.15.12:0/2487491337 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fd6dc00ddb0 con 0x5625ee121aa0 2024-05-17T22:43:21.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.281+0000 7fd6fb8bbb80 1 -- 172.21.15.12:0/2487491337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6d80044d0 con 0x5625ee121aa0 2024-05-17T22:43:21.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.282+0000 7fd6cdffb700 1 -- 172.21.15.12:0/2487491337 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd6dc020050 con 0x5625ee121aa0 2024-05-17T22:43:21.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.282+0000 7fd6fb8bbb80 10 client.15708 fetch_fsmap learned FSMap version 33 2024-05-17T22:43:21.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.282+0000 7fd6fb8bbb80 1 -- 172.21.15.12:0/2487491337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5625ee132c30 con 0x5625ee121aa0 2024-05-17T22:43:21.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.283+0000 7fd6cdffb700 1 -- 172.21.15.12:0/2487491337 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6dc010910 con 0x5625ee121aa0 2024-05-17T22:43:21.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.284+0000 7fd6cdffb700 1 -- 172.21.15.12:0/2487491337 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7fd6dc00f710 con 0x5625ee121aa0 2024-05-17T22:43:21.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.284+0000 7fd6fb8bbb80 10 client.15708 fetch_fsmap finished waiting for FSMap version 33 2024-05-17T22:43:21.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.284+0000 7fd6fb8bbb80 10 client.15708 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-17T22:43:21.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.284+0000 7fd6fb8bbb80 20 client.15708 populate_metadata read hostname 'smithi012' 2024-05-17T22:43:21.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.285+0000 7fd6fb8bbb80 1 --2- 172.21.15.12:0/2487491337 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5625ee135270 0x5625ee137760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:21.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.285+0000 7fd6fb8bbb80 4 client.15708 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:43:21.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.285+0000 7fd6fb8bbb80 1 -- 172.21.15.12:0/2487491337 --> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd6d80044d0 con 0x5625ee135270 2024-05-17T22:43:21.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.286+0000 7fd6e659c700 1 --2- 172.21.15.12:0/2487491337 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5625ee135270 0x5625ee137760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:21.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.287+0000 7fd6cdffb700 10 client.15708 ms_handle_connect on v2:172.21.15.96:6836/4222574338 2024-05-17T22:43:21.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.287+0000 7fd6e659c700 1 --2- 172.21.15.12:0/2487491337 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5625ee135270 0x5625ee137760 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=314ead3f836648bb server_cookie=610e03c439fc0e74 in_seq=0 out_seq=0 2024-05-17T22:43:21.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.290+0000 7fd6cdffb700 1 -- 172.21.15.12:0/2487491337 <== mds.1 v2:172.21.15.96:6836/4222574338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fd6d80044d0 con 0x5625ee135270 2024-05-17T22:43:21.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.290+0000 7fd6cdffb700 10 client.15708 handle_command_reply: tid=0 2024-05-17T22:43:21.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.290+0000 7fd6fb8bbb80 1 client.15708 shutdown 2024-05-17T22:43:21.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.293+0000 7fd6fb8bbb80 1 -- 172.21.15.12:0/2487491337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625ee121aa0 msgr2=0x5625ee125c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:21.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.293+0000 7fd6fb8bbb80 1 --2- 172.21.15.12:0/2487491337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625ee121aa0 0x5625ee125c50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc00bb90 tx=0x7fd6dc00bf50 comp rx=0 tx=0).stop 2024-05-17T22:43:21.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.296+0000 7fd6fb8bbb80 1 -- 172.21.15.12:0/2487491337 shutdown_connections 2024-05-17T22:43:21.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.296+0000 7fd6fb8bbb80 1 --2- 172.21.15.12:0/2487491337 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5625ee135270 0x5625ee137760 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:21.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.296+0000 7fd6fb8bbb80 1 --2- 172.21.15.12:0/2487491337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625ee121aa0 0x5625ee125c50 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:21.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.296+0000 7fd6fb8bbb80 1 --2- 172.21.15.12:0/2487491337 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5625edf9d9f0 0x5625ee1251d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:21.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.296+0000 7fd6fb8bbb80 1 --2- 172.21.15.12:0/2487491337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625ee11c170 0x5625ee125710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:21.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.296+0000 7fd6fb8bbb80 1 -- 172.21.15.12:0/2487491337 >> 172.21.15.12:0/2487491337 conn(0x5625edfe1f00 msgr2=0x5625ee110ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:43:21.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.296+0000 7fd6cdffb700 0 client.15708 ms_handle_reset on v2:172.21.15.96:6836/4222574338 2024-05-17T22:43:21.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.298+0000 7fd6fb8bbb80 1 -- 172.21.15.12:0/2487491337 shutdown_connections 2024-05-17T22:43:21.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.299+0000 7fd6fb8bbb80 1 -- 172.21.15.12:0/2487491337 wait complete. 2024-05-17T22:43:21.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.299+0000 7fd6fb8bbb80 20 client.15708 trim_cache size 0 max 16384 2024-05-17T22:43:21.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.323+0000 7fd6fb8bbb80 1 Processor -- start 2024-05-17T22:43:21.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.323+0000 7fd6fb8bbb80 1 -- start start 2024-05-17T22:43:21.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.324+0000 7fd6fb8bbb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5625edf9d9f0 0x5625ee11bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:21.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.324+0000 7fd6fb8bbb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625ee11c170 0x5625ee11f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:21.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.324+0000 7fd6fb8bbb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625ee121aa0 0x5625ee116ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:21.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.324+0000 7fd6fb8bbb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c80114c0 con 0x5625ee121aa0 2024-05-17T22:43:21.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.324+0000 7fd6fb8bbb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c800ecf0 con 0x5625ee11c170 2024-05-17T22:43:21.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.324+0000 7fd6fb8bbb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c800cb40 con 0x5625edf9d9f0 2024-05-17T22:43:21.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.324+0000 7fd6e6d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625ee121aa0 0x5625ee116ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:21.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.324+0000 7fd6e659c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5625edf9d9f0 0x5625ee11bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:21.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.324+0000 7fd6e6d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625ee121aa0 0x5625ee116ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44686/0 (socket says 172.21.15.12:44686) 2024-05-17T22:43:21.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.324+0000 7fd6e6d9d700 1 -- 172.21.15.12:0/1326301038 learned_addr learned my addr 172.21.15.12:0/1326301038 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:43:21.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.324+0000 7fd6e5d9b700 1 --2- 172.21.15.12:0/1326301038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625ee11c170 0x5625ee11f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:21.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.325+0000 7fd6e659c700 1 -- 172.21.15.12:0/1326301038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625ee11c170 msgr2=0x5625ee11f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:43:21.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.325+0000 7fd6e659c700 1 --2- 172.21.15.12:0/1326301038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625ee11c170 0x5625ee11f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:21.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.325+0000 7fd6e659c700 1 -- 172.21.15.12:0/1326301038 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625ee121aa0 msgr2=0x5625ee116ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:43:21.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.325+0000 7fd6e659c700 1 --2- 172.21.15.12:0/1326301038 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625ee121aa0 0x5625ee116ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:21.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.325+0000 7fd6e659c700 1 -- 172.21.15.12:0/1326301038 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625ee127670 con 0x5625edf9d9f0 2024-05-17T22:43:21.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.325+0000 7fd6e6d9d700 1 --2- 172.21.15.12:0/1326301038 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625ee121aa0 0x5625ee116ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:43:21.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.325+0000 7fd6e659c700 1 --2- 172.21.15.12:0/1326301038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5625edf9d9f0 0x5625ee11bc30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd6d80113d0 tx=0x7fd6d8003920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed4f213a16e40e52 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:43:21.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.325+0000 7fd6cffff700 1 -- 172.21.15.12:0/1326301038 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d80036f0 con 0x5625edf9d9f0 2024-05-17T22:43:21.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.325+0000 7fd6cffff700 1 -- 172.21.15.12:0/1326301038 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6d8032070 con 0x5625edf9d9f0 2024-05-17T22:43:21.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.325+0000 7fd6cffff700 1 -- 172.21.15.12:0/1326301038 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d80042a0 con 0x5625edf9d9f0 2024-05-17T22:43:21.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.326+0000 7fd6fb8bbb80 1 -- 172.21.15.12:0/1326301038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5625edf9d9f0 msgr2=0x5625ee11bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:21.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.326+0000 7fd6fb8bbb80 1 --2- 172.21.15.12:0/1326301038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5625edf9d9f0 0x5625ee11bc30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd6d80113d0 tx=0x7fd6d8003920 comp rx=0 tx=0).stop 2024-05-17T22:43:21.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.326+0000 7fd6fb8bbb80 1 -- 172.21.15.12:0/1326301038 shutdown_connections 2024-05-17T22:43:21.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.326+0000 7fd6fb8bbb80 1 --2- 172.21.15.12:0/1326301038 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625ee121aa0 0x5625ee116ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:21.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.326+0000 7fd6fb8bbb80 1 --2- 172.21.15.12:0/1326301038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5625edf9d9f0 0x5625ee11bc30 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:21.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.326+0000 7fd6fb8bbb80 1 --2- 172.21.15.12:0/1326301038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625ee11c170 0x5625ee11f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:21.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.326+0000 7fd6fb8bbb80 1 -- 172.21.15.12:0/1326301038 >> 172.21.15.12:0/1326301038 conn(0x5625edfe1f00 msgr2=0x5625ee120810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:43:21.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.326+0000 7fd6fb8bbb80 1 -- 172.21.15.12:0/1326301038 shutdown_connections 2024-05-17T22:43:21.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.326+0000 7fd6fb8bbb80 1 -- 172.21.15.12:0/1326301038 wait complete. 2024-05-17T22:43:21.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.327+0000 7fd6fb8bbb80 1 -- start start 2024-05-17T22:43:21.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.327+0000 7fd6fb8bbb80 1 Processor -- start 2024-05-17T22:43:21.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.327+0000 7fd6fb8bbb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625edf9d9f0 0x5625ee128480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:21.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.327+0000 7fd6fb8bbb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625ee11c170 0x5625ee1289c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:21.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.327+0000 7fd6fb8bbb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5625ee121aa0 0x5625ee128f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:21.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.327+0000 7fd6e659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625edf9d9f0 0x5625ee128480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:21.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.327+0000 7fd6e659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625edf9d9f0 0x5625ee128480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44690/0 (socket says 172.21.15.12:44690) 2024-05-17T22:43:21.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.327+0000 7fd6e659c700 1 -- 172.21.15.12:0/1790869218 learned_addr learned my addr 172.21.15.12:0/1790869218 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:43:21.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.327+0000 7fd6fb8bbb80 1 -- 172.21.15.12:0/1790869218 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c800d1b0 con 0x5625edf9d9f0 2024-05-17T22:43:21.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.327+0000 7fd6fb8bbb80 1 -- 172.21.15.12:0/1790869218 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c800e620 con 0x5625ee11c170 2024-05-17T22:43:21.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.327+0000 7fd6fb8bbb80 1 -- 172.21.15.12:0/1790869218 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c800d880 con 0x5625ee121aa0 2024-05-17T22:43:21.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.327+0000 7fd6fb8bbb80 10 client.0 _ll_register_callbacks cb 0x5625edfd4f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:43:21.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.327+0000 7fd6e5d9b700 1 --2- 172.21.15.12:0/1790869218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625ee11c170 0x5625ee1289c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:21.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.328+0000 7fd6e6d9d700 1 --2- 172.21.15.12:0/1790869218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5625ee121aa0 0x5625ee128f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:21.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.328+0000 7fd6e659c700 1 -- 172.21.15.12:0/1790869218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5625ee121aa0 msgr2=0x5625ee128f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:43:21.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.328+0000 7fd6e659c700 1 --2- 172.21.15.12:0/1790869218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5625ee121aa0 0x5625ee128f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:21.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.328+0000 7fd6e659c700 1 -- 172.21.15.12:0/1790869218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625ee11c170 msgr2=0x5625ee1289c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:43:21.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.328+0000 7fd6e659c700 1 --2- 172.21.15.12:0/1790869218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625ee11c170 0x5625ee1289c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:21.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.328+0000 7fd6e659c700 1 -- 172.21.15.12:0/1790869218 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625ee19dac0 con 0x5625edf9d9f0 2024-05-17T22:43:21.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.328+0000 7fd6e6d9d700 1 --2- 172.21.15.12:0/1790869218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5625ee121aa0 0x5625ee128f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:43:21.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.328+0000 7fd6e659c700 1 --2- 172.21.15.12:0/1790869218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625edf9d9f0 0x5625ee128480 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8006010 tx=0x7fd6d8011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c160dbdc7683f53 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:43:21.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.328+0000 7fd6cdffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:43:21.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.329+0000 7fd6cdffb700 1 -- 172.21.15.12:0/1790869218 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d803a020 con 0x5625edf9d9f0 2024-05-17T22:43:21.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.329+0000 7fd6cdffb700 1 -- 172.21.15.12:0/1790869218 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fd6d8019040 con 0x5625edf9d9f0 2024-05-17T22:43:21.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.330+0000 7fd6fb8bbb80 1 -- 172.21.15.12:0/1790869218 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6d8032070 con 0x5625edf9d9f0 2024-05-17T22:43:21.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.330+0000 7fd6cdffb700 1 -- 172.21.15.12:0/1790869218 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd6d8003f00 con 0x5625edf9d9f0 2024-05-17T22:43:21.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.330+0000 7fd6fb8bbb80 10 client.15714 fetch_fsmap learned FSMap version 33 2024-05-17T22:43:21.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.330+0000 7fd6fb8bbb80 1 -- 172.21.15.12:0/1790869218 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5625ee19ec00 con 0x5625edf9d9f0 2024-05-17T22:43:21.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.331+0000 7fd6cdffb700 1 -- 172.21.15.12:0/1790869218 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6d8005040 con 0x5625edf9d9f0 2024-05-17T22:43:21.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.331+0000 7fd6cdffb700 1 -- 172.21.15.12:0/1790869218 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7fd6d800f040 con 0x5625edf9d9f0 2024-05-17T22:43:21.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.331+0000 7fd6fb8bbb80 10 client.15714 fetch_fsmap finished waiting for FSMap version 33 2024-05-17T22:43:21.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.331+0000 7fd6fb8bbb80 10 client.15714 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-17T22:43:21.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.331+0000 7fd6fb8bbb80 20 client.15714 populate_metadata read hostname 'smithi012' 2024-05-17T22:43:21.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.332+0000 7fd6fb8bbb80 1 --2- 172.21.15.12:0/1790869218 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5625ee19f070 0x5625ee1a1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:21.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.332+0000 7fd6fb8bbb80 4 client.15714 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:43:21.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.332+0000 7fd6fb8bbb80 1 -- 172.21.15.12:0/1790869218 --> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fd6d8032070 con 0x5625ee19f070 2024-05-17T22:43:21.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.332+0000 7fd6e5d9b700 1 --2- 172.21.15.12:0/1790869218 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5625ee19f070 0x5625ee1a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:21.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.332+0000 7fd6e5d9b700 1 --2- 172.21.15.12:0/1790869218 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5625ee19f070 0x5625ee1a1490 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b9172469d8558fb0 server_cookie=dba5d929dc14b5ef in_seq=0 out_seq=0 2024-05-17T22:43:21.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.332+0000 7fd6cdffb700 10 client.15714 ms_handle_connect on v2:172.21.15.96:6836/4222574338 2024-05-17T22:43:21.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.332+0000 7fd6cdffb700 1 -- 172.21.15.12:0/1790869218 <== mds.1 v2:172.21.15.96:6836/4222574338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fd6d8032070 con 0x5625ee19f070 2024-05-17T22:43:21.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.332+0000 7fd6cdffb700 10 client.15714 handle_command_reply: tid=0 2024-05-17T22:43:21.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.332+0000 7fd6fb8bbb80 1 client.15714 shutdown 2024-05-17T22:43:21.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.333+0000 7fd6fb8bbb80 1 -- 172.21.15.12:0/1790869218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625edf9d9f0 msgr2=0x5625ee128480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:21.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.333+0000 7fd6fb8bbb80 1 --2- 172.21.15.12:0/1790869218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625edf9d9f0 0x5625ee128480 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8006010 tx=0x7fd6d8011f40 comp rx=0 tx=0).stop 2024-05-17T22:43:21.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.333+0000 7fd6fb8bbb80 1 -- 172.21.15.12:0/1790869218 shutdown_connections 2024-05-17T22:43:21.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.333+0000 7fd6fb8bbb80 1 --2- 172.21.15.12:0/1790869218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5625ee121aa0 0x5625ee128f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:21.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.333+0000 7fd6fb8bbb80 1 --2- 172.21.15.12:0/1790869218 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5625ee19f070 0x5625ee1a1490 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:21.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.333+0000 7fd6cdffb700 0 client.15714 ms_handle_reset on v2:172.21.15.96:6836/4222574338 2024-05-17T22:43:21.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.333+0000 7fd6fb8bbb80 1 --2- 172.21.15.12:0/1790869218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625edf9d9f0 0x5625ee128480 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:21.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.333+0000 7fd6fb8bbb80 1 --2- 172.21.15.12:0/1790869218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625ee11c170 0x5625ee1289c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:21.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.333+0000 7fd6fb8bbb80 1 -- 172.21.15.12:0/1790869218 >> 172.21.15.12:0/1790869218 conn(0x5625edfe1f00 msgr2=0x5625ee125980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:43:21.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.333+0000 7fd6fb8bbb80 1 -- 172.21.15.12:0/1790869218 shutdown_connections 2024-05-17T22:43:21.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.333+0000 7fd6fb8bbb80 1 -- 172.21.15.12:0/1790869218 wait complete. 2024-05-17T22:43:21.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.333+0000 7fd6fb8bbb80 20 client.15714 trim_cache size 0 max 16384 2024-05-17T22:43:21.338 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:43:21.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.334+0000 7fd6ecee0700 1 -- 172.21.15.12:0/1541694287 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd6d00b2a90 msgr2=0x7fd6d00b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:21.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.334+0000 7fd6ecee0700 1 --2- 172.21.15.12:0/1541694287 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd6d00b2a90 0x7fd6d00b4fa0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd6d80137a0 tx=0x7fd6d800a960 comp rx=0 tx=0).stop 2024-05-17T22:43:21.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.334+0000 7fd6ecee0700 1 -- 172.21.15.12:0/1541694287 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6e8076520 msgr2=0x7fd6e810d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:21.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.334+0000 7fd6ecee0700 1 --2- 172.21.15.12:0/1541694287 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6e8076520 0x7fd6e810d7b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0009550 tx=0x7fd6e0004e70 comp rx=0 tx=0).stop 2024-05-17T22:43:21.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.334+0000 7fd6ecee0700 1 -- 172.21.15.12:0/1541694287 shutdown_connections 2024-05-17T22:43:21.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.334+0000 7fd6ecee0700 1 --2- 172.21.15.12:0/1541694287 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd6d00b2a90 0x7fd6d00b4fa0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:21.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.334+0000 7fd6ecee0700 1 --2- 172.21.15.12:0/1541694287 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6e8114c90 0x7fd6e810de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:21.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.334+0000 7fd6ecee0700 1 --2- 172.21.15.12:0/1541694287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6e8075bd0 0x7fd6e8114680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:21.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.334+0000 7fd6ecee0700 1 --2- 172.21.15.12:0/1541694287 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6e8076520 0x7fd6e810d7b0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:21.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.334+0000 7fd6ecee0700 1 -- 172.21.15.12:0/1541694287 >> 172.21.15.12:0/1541694287 conn(0x7fd6e80ff990 msgr2=0x7fd6e81001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:43:21.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.334+0000 7fd6ecee0700 1 -- 172.21.15.12:0/1541694287 shutdown_connections 2024-05-17T22:43:21.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:21.334+0000 7fd6ecee0700 1 -- 172.21.15.12:0/1541694287 wait complete. 2024-05-17T22:43:21.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 240.0 s, 2030.0 tps, lat 48.784 ms stddev 56.183 2024-05-17T22:43:22.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:21 smithi151 ceph-mon[36101]: pgmap v562: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.4 MiB/s wr, 571 op/s 2024-05-17T22:43:22.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:21 smithi096 ceph-mon[36065]: pgmap v562: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.4 MiB/s wr, 571 op/s 2024-05-17T22:43:22.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:21 smithi012 ceph-mon[31632]: pgmap v562: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.4 MiB/s wr, 571 op/s 2024-05-17T22:43:24.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:23 smithi151 ceph-mon[36101]: pgmap v563: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.4 MiB/s wr, 573 op/s 2024-05-17T22:43:24.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:23 smithi096 ceph-mon[36065]: pgmap v563: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.4 MiB/s wr, 573 op/s 2024-05-17T22:43:24.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:23 smithi012 ceph-mon[31632]: pgmap v563: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.4 MiB/s wr, 573 op/s 2024-05-17T22:43:26.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:25 smithi151 ceph-mon[36101]: pgmap v564: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.4 MiB/s wr, 646 op/s 2024-05-17T22:43:26.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:25 smithi096 ceph-mon[36065]: pgmap v564: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.4 MiB/s wr, 646 op/s 2024-05-17T22:43:26.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:25 smithi012 ceph-mon[31632]: pgmap v564: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.4 MiB/s wr, 646 op/s 2024-05-17T22:43:26.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 245.0 s, 964.0 tps, lat 94.966 ms stddev 155.312 2024-05-17T22:43:26.858 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-17T22:43:27.305 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:43:28.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:27 smithi012 ceph-mon[31632]: pgmap v565: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.6 MiB/s wr, 609 op/s 2024-05-17T22:43:28.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:27 smithi151 ceph-mon[36101]: pgmap v565: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.6 MiB/s wr, 609 op/s 2024-05-17T22:43:28.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:27 smithi096 ceph-mon[36065]: pgmap v565: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.6 MiB/s wr, 609 op/s 2024-05-17T22:43:30.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:29 smithi151 ceph-mon[36101]: pgmap v566: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 8.2 MiB/s wr, 781 op/s 2024-05-17T22:43:30.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:29 smithi096 ceph-mon[36065]: pgmap v566: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 8.2 MiB/s wr, 781 op/s 2024-05-17T22:43:30.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:29 smithi012 ceph-mon[31632]: pgmap v566: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 8.2 MiB/s wr, 781 op/s 2024-05-17T22:43:31.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 250.0 s, 1203.0 tps, lat 89.388 ms stddev 142.723 2024-05-17T22:43:31.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.367+0000 7f130ad08700 1 -- 172.21.15.12:0/3512548156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1304102e70 msgr2=0x7f1304103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:31.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.367+0000 7f130ad08700 1 --2- 172.21.15.12:0/3512548156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1304102e70 0x7f1304103280 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f12f801a9a0 tx=0x7f12f801acb0 comp rx=0 tx=0).stop 2024-05-17T22:43:31.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.369+0000 7f130ad08700 1 -- 172.21.15.12:0/3512548156 shutdown_connections 2024-05-17T22:43:31.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.369+0000 7f130ad08700 1 --2- 172.21.15.12:0/3512548156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13041080a0 0x7f130410fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:31.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.369+0000 7f130ad08700 1 --2- 172.21.15.12:0/3512548156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1304102e70 0x7f1304103280 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:31.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.369+0000 7f130ad08700 1 --2- 172.21.15.12:0/3512548156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1304107750 0x7f1304107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:31.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.369+0000 7f130ad08700 1 -- 172.21.15.12:0/3512548156 >> 172.21.15.12:0/3512548156 conn(0x7f1304079ae0 msgr2=0x7f1304079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:43:31.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.370+0000 7f130ad08700 1 -- 172.21.15.12:0/3512548156 shutdown_connections 2024-05-17T22:43:31.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.370+0000 7f130ad08700 1 -- 172.21.15.12:0/3512548156 wait complete. 2024-05-17T22:43:31.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.371+0000 7f130ad08700 1 Processor -- start 2024-05-17T22:43:31.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.371+0000 7f130ad08700 1 -- start start 2024-05-17T22:43:31.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.371+0000 7f130ad08700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1304102e70 0x7f130406f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:31.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.371+0000 7f130ad08700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1304107750 0x7f130406fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:31.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.371+0000 7f130ad08700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13041080a0 0x7f1304070170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:31.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.371+0000 7f130ad08700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f130406db50 con 0x7f13041080a0 2024-05-17T22:43:31.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.371+0000 7f130ad08700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f130406d9d0 con 0x7f1304107750 2024-05-17T22:43:31.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.371+0000 7f130ad08700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f130406dcd0 con 0x7f1304102e70 2024-05-17T22:43:31.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.372+0000 7f1308aa4700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1304102e70 0x7f130406f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:31.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.372+0000 7f1308aa4700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1304102e70 0x7f130406f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:37574/0 (socket says 172.21.15.12:37574) 2024-05-17T22:43:31.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.372+0000 7f1308aa4700 1 -- 172.21.15.12:0/3878076308 learned_addr learned my addr 172.21.15.12:0/3878076308 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:43:31.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.372+0000 7f1303fff700 1 --2- 172.21.15.12:0/3878076308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1304107750 0x7f130406fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:31.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.372+0000 7f1303fff700 1 -- 172.21.15.12:0/3878076308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1304102e70 msgr2=0x7f130406f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:31.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.372+0000 7f1303fff700 1 --2- 172.21.15.12:0/3878076308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1304102e70 0x7f130406f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:31.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.372+0000 7f1303fff700 1 -- 172.21.15.12:0/3878076308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13041080a0 msgr2=0x7f1304070170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:31.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.372+0000 7f1303fff700 1 --2- 172.21.15.12:0/3878076308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13041080a0 0x7f1304070170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:31.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.372+0000 7f1303fff700 1 -- 172.21.15.12:0/3878076308 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13041ad100 con 0x7f1304107750 2024-05-17T22:43:31.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.372+0000 7f1303fff700 1 --2- 172.21.15.12:0/3878076308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1304107750 0x7f130406fc30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f12f000e9c0 tx=0x7f12f000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:43:31.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.374+0000 7f1301ffb700 1 -- 172.21.15.12:0/3878076308 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f000cbd0 con 0x7f1304107750 2024-05-17T22:43:31.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.374+0000 7f1301ffb700 1 -- 172.21.15.12:0/3878076308 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12f000cd70 con 0x7f1304107750 2024-05-17T22:43:31.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.376+0000 7f1301ffb700 1 -- 172.21.15.12:0/3878076308 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f0018680 con 0x7f1304107750 2024-05-17T22:43:31.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.376+0000 7f130ad08700 1 -- 172.21.15.12:0/3878076308 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13041ad3f0 con 0x7f1304107750 2024-05-17T22:43:31.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.376+0000 7f130ad08700 1 -- 172.21.15.12:0/3878076308 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13041ad8b0 con 0x7f1304107750 2024-05-17T22:43:31.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.382+0000 7f1301ffb700 1 -- 172.21.15.12:0/3878076308 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f12f0010430 con 0x7f1304107750 2024-05-17T22:43:31.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.383+0000 7f13196e3b80 1 Processor -- start 2024-05-17T22:43:31.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.384+0000 7f13196e3b80 1 -- start start 2024-05-17T22:43:31.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.384+0000 7f1301ffb700 1 --2- 172.21.15.12:0/3878076308 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f12f40b2a50 0x7f12f40b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:31.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.384+0000 7f1301ffb700 1 -- 172.21.15.12:0/3878076308 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f12f00e27c0 con 0x7f1304107750 2024-05-17T22:43:31.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.384+0000 7f13196e3b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557efe7449f0 0x557efe8c8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:31.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.384+0000 7f13196e3b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557efe8c8aa0 0x557efe8c2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:31.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.384+0000 7f13196e3b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557efe8c3170 0x557efe8c3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:31.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.384+0000 7f13196e3b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x557efe7f1290 con 0x557efe8c3170 2024-05-17T22:43:31.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.384+0000 7f13196e3b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557efe7e0fd0 con 0x557efe7449f0 2024-05-17T22:43:31.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.384+0000 7f13196e3b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557efe7e4e70 con 0x557efe8c8aa0 2024-05-17T22:43:31.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.385+0000 7f13092a5700 1 --2- 172.21.15.12:0/3878076308 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f12f40b2a50 0x7f12f40b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:31.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.386+0000 7f13092a5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557efe8c3170 0x557efe8c3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:31.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.386+0000 7f13092a5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557efe8c3170 0x557efe8c3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41150/0 (socket says 172.21.15.12:41150) 2024-05-17T22:43:31.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.386+0000 7f13092a5700 1 -- 172.21.15.12:0/456577928 learned_addr learned my addr 172.21.15.12:0/456577928 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:43:31.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.387+0000 7f13092a5700 1 --2- 172.21.15.12:0/3878076308 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f12f40b2a50 0x7f12f40b4f60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f12fc010540 tx=0x7f12fc010900 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:43:31.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.387+0000 7f1308aa4700 1 --2- 172.21.15.12:0/456577928 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557efe7449f0 0x557efe8c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:31.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.387+0000 7f13092a5700 1 -- 172.21.15.12:0/456577928 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557efe8c8aa0 msgr2=0x557efe8c2c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T22:43:31.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.387+0000 7f13092a5700 1 --2- 172.21.15.12:0/456577928 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557efe8c8aa0 0x557efe8c2c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:31.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.387+0000 7f13092a5700 1 -- 172.21.15.12:0/456577928 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557efe7449f0 msgr2=0x557efe8c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:43:31.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.387+0000 7f13092a5700 1 --2- 172.21.15.12:0/456577928 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557efe7449f0 0x557efe8c8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:31.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.387+0000 7f13092a5700 1 -- 172.21.15.12:0/456577928 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557efe78a590 con 0x557efe8c3170 2024-05-17T22:43:31.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.388+0000 7f1308aa4700 1 --2- 172.21.15.12:0/456577928 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557efe7449f0 0x557efe8c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:43:31.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.389+0000 7f13092a5700 1 --2- 172.21.15.12:0/456577928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557efe8c3170 0x557efe8c3580 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f12fc01b750 tx=0x7f12fc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31459931db83944b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:43:31.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.390+0000 7f12eeffd700 1 -- 172.21.15.12:0/456577928 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12fc01bea0 con 0x557efe8c3170 2024-05-17T22:43:31.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.390+0000 7f12eeffd700 1 -- 172.21.15.12:0/456577928 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12fc022070 con 0x557efe8c3170 2024-05-17T22:43:31.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.390+0000 7f12eeffd700 1 -- 172.21.15.12:0/456577928 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12fc00e7c0 con 0x557efe8c3170 2024-05-17T22:43:31.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.392+0000 7f13196e3b80 1 -- 172.21.15.12:0/456577928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557efe8c3170 msgr2=0x557efe8c3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:31.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.392+0000 7f13196e3b80 1 --2- 172.21.15.12:0/456577928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557efe8c3170 0x557efe8c3580 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f12fc01b750 tx=0x7f12fc018000 comp rx=0 tx=0).stop 2024-05-17T22:43:31.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.392+0000 7f13196e3b80 1 -- 172.21.15.12:0/456577928 shutdown_connections 2024-05-17T22:43:31.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.392+0000 7f13196e3b80 1 --2- 172.21.15.12:0/456577928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557efe8c3170 0x557efe8c3580 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:31.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.392+0000 7f13196e3b80 1 --2- 172.21.15.12:0/456577928 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557efe7449f0 0x557efe8c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:31.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.392+0000 7f13196e3b80 1 --2- 172.21.15.12:0/456577928 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557efe8c8aa0 0x557efe8c2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:31.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.392+0000 7f13196e3b80 1 -- 172.21.15.12:0/456577928 >> 172.21.15.12:0/456577928 conn(0x557efe788f00 msgr2=0x557efe8bdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:43:31.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.393+0000 7f13196e3b80 1 -- 172.21.15.12:0/456577928 shutdown_connections 2024-05-17T22:43:31.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.393+0000 7f13196e3b80 1 -- 172.21.15.12:0/456577928 wait complete. 2024-05-17T22:43:31.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.394+0000 7f13196e3b80 1 -- start start 2024-05-17T22:43:31.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.394+0000 7f13196e3b80 1 Processor -- start 2024-05-17T22:43:31.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.395+0000 7f13196e3b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557efe7449f0 0x557efe8cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:31.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.395+0000 7f13196e3b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557efe8c3170 0x557efe8cc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:31.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.395+0000 7f13196e3b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557efe8c8aa0 0x557efe8ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:31.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.395+0000 7f13196e3b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x557efe6e8150 con 0x557efe7449f0 2024-05-17T22:43:31.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.395+0000 7f13196e3b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557efe781e80 con 0x557efe8c8aa0 2024-05-17T22:43:31.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.395+0000 7f13196e3b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557efe8cf390 con 0x557efe8c3170 2024-05-17T22:43:31.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.395+0000 7f13196e3b80 10 client.0 _ll_register_callbacks cb 0x557efe77bf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:43:31.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.396+0000 7f1308aa4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557efe7449f0 0x557efe8cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:31.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.396+0000 7f1308aa4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557efe7449f0 0x557efe8cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41162/0 (socket says 172.21.15.12:41162) 2024-05-17T22:43:31.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.396+0000 7f1308aa4700 1 -- 172.21.15.12:0/1798576532 learned_addr learned my addr 172.21.15.12:0/1798576532 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:43:31.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.396+0000 7f13092a5700 1 --2- 172.21.15.12:0/1798576532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557efe8c8aa0 0x557efe8ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:31.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.396+0000 7f1308aa4700 1 -- 172.21.15.12:0/1798576532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557efe8c3170 msgr2=0x557efe8cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:43:31.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.396+0000 7f1308aa4700 1 --2- 172.21.15.12:0/1798576532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557efe8c3170 0x557efe8cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:31.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.396+0000 7f1308aa4700 1 -- 172.21.15.12:0/1798576532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557efe8c8aa0 msgr2=0x557efe8ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:43:31.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.396+0000 7f1308aa4700 1 --2- 172.21.15.12:0/1798576532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557efe8c8aa0 0x557efe8ccd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:31.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.396+0000 7f1308aa4700 1 -- 172.21.15.12:0/1798576532 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557efe731480 con 0x557efe7449f0 2024-05-17T22:43:31.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.398+0000 7f1308aa4700 1 --2- 172.21.15.12:0/1798576532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557efe7449f0 0x557efe8cc240 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f12f801ca60 tx=0x7f12f801cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f4318015eb023d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:43:31.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.398+0000 7f12e3fff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:43:31.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.398+0000 7f12e3fff700 1 -- 172.21.15.12:0/1798576532 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f8036020 con 0x557efe7449f0 2024-05-17T22:43:31.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.398+0000 7f12e3fff700 1 -- 172.21.15.12:0/1798576532 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f12f8005250 con 0x557efe7449f0 2024-05-17T22:43:31.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.398+0000 7f13196e3b80 1 -- 172.21.15.12:0/1798576532 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12fc022070 con 0x557efe7449f0 2024-05-17T22:43:31.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.399+0000 7f12e3fff700 1 -- 172.21.15.12:0/1798576532 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f12f8005450 con 0x557efe7449f0 2024-05-17T22:43:31.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.400+0000 7f13196e3b80 10 client.15720 fetch_fsmap learned FSMap version 33 2024-05-17T22:43:31.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.400+0000 7f13196e3b80 1 -- 172.21.15.12:0/1798576532 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557efe8d9df0 con 0x557efe7449f0 2024-05-17T22:43:31.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.400+0000 7f12e3fff700 1 -- 172.21.15.12:0/1798576532 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12fc022070 con 0x557efe7449f0 2024-05-17T22:43:31.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.400+0000 7f12e3fff700 1 -- 172.21.15.12:0/1798576532 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f12f80324f0 con 0x557efe7449f0 2024-05-17T22:43:31.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.400+0000 7f13196e3b80 10 client.15720 fetch_fsmap finished waiting for FSMap version 33 2024-05-17T22:43:31.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.400+0000 7f13196e3b80 10 client.15720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:43:31.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.400+0000 7f13196e3b80 20 client.15720 populate_metadata read hostname 'smithi012' 2024-05-17T22:43:31.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.401+0000 7f13196e3b80 1 --2- 172.21.15.12:0/1798576532 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x557efe8dc4d0 0x557efe8de930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:31.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.401+0000 7f13196e3b80 4 client.15720 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:43:31.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.401+0000 7f13196e3b80 1 -- 172.21.15.12:0/1798576532 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557efe7f31e0 con 0x557efe8dc4d0 2024-05-17T22:43:31.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.401+0000 7f1303fff700 1 --2- 172.21.15.12:0/1798576532 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x557efe8dc4d0 0x557efe8de930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:31.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.402+0000 7f1303fff700 1 --2- 172.21.15.12:0/1798576532 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x557efe8dc4d0 0x557efe8de930 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc1d64ed6b27834d server_cookie=680f5538ed0c8076 in_seq=0 out_seq=0 2024-05-17T22:43:31.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.402+0000 7f12e3fff700 10 client.15720 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:43:31.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.402+0000 7f12e3fff700 1 -- 172.21.15.12:0/1798576532 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557efe7f31e0 con 0x557efe8dc4d0 2024-05-17T22:43:31.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.402+0000 7f12e3fff700 10 client.15720 handle_command_reply: tid=0 2024-05-17T22:43:31.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.402+0000 7f13196e3b80 1 client.15720 shutdown 2024-05-17T22:43:31.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.402+0000 7f13196e3b80 1 -- 172.21.15.12:0/1798576532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557efe7449f0 msgr2=0x557efe8cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:31.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.402+0000 7f13196e3b80 1 --2- 172.21.15.12:0/1798576532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557efe7449f0 0x557efe8cc240 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f12f801ca60 tx=0x7f12f801cb40 comp rx=0 tx=0).stop 2024-05-17T22:43:31.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.402+0000 7f13196e3b80 1 -- 172.21.15.12:0/1798576532 shutdown_connections 2024-05-17T22:43:31.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.402+0000 7f13196e3b80 1 --2- 172.21.15.12:0/1798576532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557efe8c8aa0 0x557efe8ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:31.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.403+0000 7f13196e3b80 1 --2- 172.21.15.12:0/1798576532 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x557efe8dc4d0 0x557efe8de930 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:31.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.403+0000 7f13196e3b80 1 --2- 172.21.15.12:0/1798576532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557efe7449f0 0x557efe8cc240 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:31.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.403+0000 7f13196e3b80 1 --2- 172.21.15.12:0/1798576532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557efe8c3170 0x557efe8cc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:31.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.403+0000 7f13196e3b80 1 -- 172.21.15.12:0/1798576532 >> 172.21.15.12:0/1798576532 conn(0x557efe788f00 msgr2=0x557efe8b7ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:43:31.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.403+0000 7f12e3fff700 0 client.15720 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:43:31.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.403+0000 7f13196e3b80 1 -- 172.21.15.12:0/1798576532 shutdown_connections 2024-05-17T22:43:31.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.403+0000 7f13196e3b80 1 -- 172.21.15.12:0/1798576532 wait complete. 2024-05-17T22:43:31.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.403+0000 7f13196e3b80 20 client.15720 trim_cache size 0 max 16384 2024-05-17T22:43:31.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.425+0000 7f13196e3b80 1 Processor -- start 2024-05-17T22:43:31.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.426+0000 7f13196e3b80 1 -- start start 2024-05-17T22:43:31.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.426+0000 7f13196e3b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557efe7449f0 0x557efe8c2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:31.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.426+0000 7f13196e3b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557efe8c3170 0x557efe8bf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:31.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.426+0000 7f13196e3b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557efe8c8aa0 0x557efe8b7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:31.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.426+0000 7f1303fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557efe8c3170 0x557efe8bf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:31.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.426+0000 7f1303fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557efe8c3170 0x557efe8bf350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41176/0 (socket says 172.21.15.12:41176) 2024-05-17T22:43:31.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.427+0000 7f1303fff700 1 -- 172.21.15.12:0/3050830295 learned_addr learned my addr 172.21.15.12:0/3050830295 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:43:31.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.427+0000 7f1308aa4700 1 --2- 172.21.15.12:0/3050830295 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557efe7449f0 0x557efe8c2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:31.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.427+0000 7f13196e3b80 1 -- 172.21.15.12:0/3050830295 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f13041b8d70 con 0x557efe8c3170 2024-05-17T22:43:31.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.427+0000 7f13092a5700 1 --2- 172.21.15.12:0/3050830295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557efe8c8aa0 0x557efe8b7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:31.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.427+0000 7f13196e3b80 1 -- 172.21.15.12:0/3050830295 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f13041b65a0 con 0x557efe8c8aa0 2024-05-17T22:43:31.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.427+0000 7f13196e3b80 1 -- 172.21.15.12:0/3050830295 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f13041b4410 con 0x557efe7449f0 2024-05-17T22:43:31.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.427+0000 7f1303fff700 1 -- 172.21.15.12:0/3050830295 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557efe7449f0 msgr2=0x557efe8c2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:43:31.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.427+0000 7f1303fff700 1 --2- 172.21.15.12:0/3050830295 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557efe7449f0 0x557efe8c2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:31.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.427+0000 7f1303fff700 1 -- 172.21.15.12:0/3050830295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557efe8c8aa0 msgr2=0x557efe8b7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:43:31.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.427+0000 7f1303fff700 1 --2- 172.21.15.12:0/3050830295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557efe8c8aa0 0x557efe8b7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:31.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.427+0000 7f1303fff700 1 -- 172.21.15.12:0/3050830295 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557efe8becc0 con 0x557efe8c3170 2024-05-17T22:43:31.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.428+0000 7f1303fff700 1 --2- 172.21.15.12:0/3050830295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557efe8c3170 0x557efe8bf350 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f12f00a6c60 tx=0x7f12f00a59d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc604a2a3a0468e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:43:31.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.429+0000 7f12ee7fc700 1 -- 172.21.15.12:0/3050830295 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f00a9500 con 0x557efe8c3170 2024-05-17T22:43:31.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.430+0000 7f12ee7fc700 1 -- 172.21.15.12:0/3050830295 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12f00aa0f0 con 0x557efe8c3170 2024-05-17T22:43:31.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.430+0000 7f12ee7fc700 1 -- 172.21.15.12:0/3050830295 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f0010720 con 0x557efe8c3170 2024-05-17T22:43:31.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.431+0000 7f13196e3b80 1 -- 172.21.15.12:0/3050830295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557efe8c3170 msgr2=0x557efe8bf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:31.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.431+0000 7f13196e3b80 1 --2- 172.21.15.12:0/3050830295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557efe8c3170 0x557efe8bf350 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f12f00a6c60 tx=0x7f12f00a59d0 comp rx=0 tx=0).stop 2024-05-17T22:43:31.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.432+0000 7f13196e3b80 1 -- 172.21.15.12:0/3050830295 shutdown_connections 2024-05-17T22:43:31.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.432+0000 7f13196e3b80 1 --2- 172.21.15.12:0/3050830295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557efe8c8aa0 0x557efe8b7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:31.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.432+0000 7f13196e3b80 1 --2- 172.21.15.12:0/3050830295 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557efe7449f0 0x557efe8c2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:31.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.432+0000 7f13196e3b80 1 --2- 172.21.15.12:0/3050830295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557efe8c3170 0x557efe8bf350 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:31.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.432+0000 7f13196e3b80 1 -- 172.21.15.12:0/3050830295 >> 172.21.15.12:0/3050830295 conn(0x557efe788f00 msgr2=0x557efe8cbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:43:31.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.432+0000 7f13196e3b80 1 -- 172.21.15.12:0/3050830295 shutdown_connections 2024-05-17T22:43:31.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.432+0000 7f13196e3b80 1 -- 172.21.15.12:0/3050830295 wait complete. 2024-05-17T22:43:31.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.433+0000 7f13196e3b80 1 -- start start 2024-05-17T22:43:31.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.433+0000 7f13196e3b80 1 Processor -- start 2024-05-17T22:43:31.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.434+0000 7f13196e3b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557efe7449f0 0x557efe8e4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:31.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.434+0000 7f13196e3b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557efe8c3170 0x557efe8d43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:31.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.434+0000 7f13196e3b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557efe8c8aa0 0x557efe8d48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:31.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.434+0000 7f13196e3b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f13041b4a60 con 0x557efe8c8aa0 2024-05-17T22:43:31.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.434+0000 7f13196e3b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f13041b5ed0 con 0x557efe7449f0 2024-05-17T22:43:31.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.434+0000 7f13196e3b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f13041b5130 con 0x557efe8c3170 2024-05-17T22:43:31.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.434+0000 7f13196e3b80 10 client.0 _ll_register_callbacks cb 0x557efe77bf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:43:31.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.434+0000 7f1303fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557efe8c3170 0x557efe8d43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:31.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.434+0000 7f1303fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557efe8c3170 0x557efe8d43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:37610/0 (socket says 172.21.15.12:37610) 2024-05-17T22:43:31.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.434+0000 7f13092a5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557efe8c8aa0 0x557efe8d48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:31.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.434+0000 7f1303fff700 1 -- 172.21.15.12:0/1464075829 learned_addr learned my addr 172.21.15.12:0/1464075829 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:43:31.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.434+0000 7f1308aa4700 1 --2- 172.21.15.12:0/1464075829 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557efe7449f0 0x557efe8e4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:31.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.434+0000 7f1303fff700 1 -- 172.21.15.12:0/1464075829 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557efe7449f0 msgr2=0x557efe8e4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:43:31.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.434+0000 7f1303fff700 1 --2- 172.21.15.12:0/1464075829 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557efe7449f0 0x557efe8e4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:31.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.434+0000 7f1303fff700 1 -- 172.21.15.12:0/1464075829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557efe8c8aa0 msgr2=0x557efe8d48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:43:31.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.434+0000 7f1303fff700 1 --2- 172.21.15.12:0/1464075829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557efe8c8aa0 0x557efe8d48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:31.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.435+0000 7f1303fff700 1 -- 172.21.15.12:0/1464075829 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557efe8d4f80 con 0x557efe8c3170 2024-05-17T22:43:31.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.435+0000 7f1303fff700 1 --2- 172.21.15.12:0/1464075829 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557efe8c3170 0x557efe8d43a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f12f00dfea0 tx=0x7f12f000c750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=631adb5e64063112 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:43:31.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.435+0000 7f12e3fff700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:43:31.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.436+0000 7f12e3fff700 1 -- 172.21.15.12:0/1464075829 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f00ca070 con 0x557efe8c3170 2024-05-17T22:43:31.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.436+0000 7f12e3fff700 1 -- 172.21.15.12:0/1464075829 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f12f00a5fe0 con 0x557efe8c3170 2024-05-17T22:43:31.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.436+0000 7f13196e3b80 1 -- 172.21.15.12:0/1464075829 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12f00aa0f0 con 0x557efe8c3170 2024-05-17T22:43:31.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.437+0000 7f12e3fff700 1 -- 172.21.15.12:0/1464075829 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f12f00adda0 con 0x557efe8c3170 2024-05-17T22:43:31.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.437+0000 7f13196e3b80 10 client.25336 fetch_fsmap learned FSMap version 33 2024-05-17T22:43:31.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.437+0000 7f13196e3b80 1 -- 172.21.15.12:0/1464075829 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557efe945d40 con 0x557efe8c3170 2024-05-17T22:43:31.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.437+0000 7f12e3fff700 1 -- 172.21.15.12:0/1464075829 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12f00a99f0 con 0x557efe8c3170 2024-05-17T22:43:31.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.437+0000 7f12e3fff700 1 -- 172.21.15.12:0/1464075829 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f12f00b23c0 con 0x557efe8c3170 2024-05-17T22:43:31.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.437+0000 7f13196e3b80 10 client.25336 fetch_fsmap finished waiting for FSMap version 33 2024-05-17T22:43:31.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.437+0000 7f13196e3b80 10 client.25336 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:43:31.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.437+0000 7f13196e3b80 20 client.25336 populate_metadata read hostname 'smithi012' 2024-05-17T22:43:31.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.437+0000 7f13196e3b80 1 --2- 172.21.15.12:0/1464075829 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x557efe946310 0x557efe948730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:31.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.437+0000 7f13196e3b80 4 client.25336 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:43:31.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.437+0000 7f13196e3b80 1 -- 172.21.15.12:0/1464075829 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f12f00aa0f0 con 0x557efe946310 2024-05-17T22:43:31.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.438+0000 7f1308aa4700 1 --2- 172.21.15.12:0/1464075829 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x557efe946310 0x557efe948730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:31.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.438+0000 7f12e3fff700 10 client.25336 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:43:31.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.438+0000 7f1308aa4700 1 --2- 172.21.15.12:0/1464075829 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x557efe946310 0x557efe948730 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ddc3a83339b4954 server_cookie=e245a7e58d10b313 in_seq=0 out_seq=0 2024-05-17T22:43:31.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.439+0000 7f12e3fff700 1 -- 172.21.15.12:0/1464075829 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f12f00aa0f0 con 0x557efe946310 2024-05-17T22:43:31.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.439+0000 7f12e3fff700 10 client.25336 handle_command_reply: tid=0 2024-05-17T22:43:31.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.439+0000 7f13196e3b80 1 client.25336 shutdown 2024-05-17T22:43:31.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.439+0000 7f13196e3b80 1 -- 172.21.15.12:0/1464075829 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557efe8c3170 msgr2=0x557efe8d43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:31.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.439+0000 7f13196e3b80 1 --2- 172.21.15.12:0/1464075829 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557efe8c3170 0x557efe8d43a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f12f00dfea0 tx=0x7f12f000c750 comp rx=0 tx=0).stop 2024-05-17T22:43:31.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.439+0000 7f13196e3b80 1 -- 172.21.15.12:0/1464075829 shutdown_connections 2024-05-17T22:43:31.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.439+0000 7f13196e3b80 1 --2- 172.21.15.12:0/1464075829 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x557efe946310 0x557efe948730 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:31.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.439+0000 7f12e3fff700 0 client.25336 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:43:31.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.439+0000 7f13196e3b80 1 --2- 172.21.15.12:0/1464075829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557efe8c8aa0 0x557efe8d48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:31.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.439+0000 7f13196e3b80 1 --2- 172.21.15.12:0/1464075829 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557efe7449f0 0x557efe8e4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:31.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.439+0000 7f13196e3b80 1 --2- 172.21.15.12:0/1464075829 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557efe8c3170 0x557efe8d43a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:31.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.439+0000 7f13196e3b80 1 -- 172.21.15.12:0/1464075829 >> 172.21.15.12:0/1464075829 conn(0x557efe788f00 msgr2=0x557efe8c79b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:43:31.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.440+0000 7f13196e3b80 1 -- 172.21.15.12:0/1464075829 shutdown_connections 2024-05-17T22:43:31.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.440+0000 7f13196e3b80 1 -- 172.21.15.12:0/1464075829 wait complete. 2024-05-17T22:43:31.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.440+0000 7f13196e3b80 20 client.25336 trim_cache size 0 max 16384 2024-05-17T22:43:31.444 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:43:31.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.441+0000 7f130ad08700 1 -- 172.21.15.12:0/3878076308 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f12f40b2a50 msgr2=0x7f12f40b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:31.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.441+0000 7f130ad08700 1 --2- 172.21.15.12:0/3878076308 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f12f40b2a50 0x7f12f40b4f60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f12fc010540 tx=0x7f12fc010900 comp rx=0 tx=0).stop 2024-05-17T22:43:31.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.441+0000 7f130ad08700 1 -- 172.21.15.12:0/3878076308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1304107750 msgr2=0x7f130406fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:31.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.441+0000 7f130ad08700 1 --2- 172.21.15.12:0/3878076308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1304107750 0x7f130406fc30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f12f000e9c0 tx=0x7f12f000ecd0 comp rx=0 tx=0).stop 2024-05-17T22:43:31.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.442+0000 7f130ad08700 1 -- 172.21.15.12:0/3878076308 shutdown_connections 2024-05-17T22:43:31.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.442+0000 7f130ad08700 1 --2- 172.21.15.12:0/3878076308 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f12f40b2a50 0x7f12f40b4f60 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:31.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.442+0000 7f130ad08700 1 --2- 172.21.15.12:0/3878076308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13041080a0 0x7f1304070170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:31.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.442+0000 7f130ad08700 1 --2- 172.21.15.12:0/3878076308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1304102e70 0x7f130406f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:31.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.442+0000 7f130ad08700 1 --2- 172.21.15.12:0/3878076308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1304107750 0x7f130406fc30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:31.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.442+0000 7f130ad08700 1 -- 172.21.15.12:0/3878076308 >> 172.21.15.12:0/3878076308 conn(0x7f1304079ae0 msgr2=0x7f1304105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:43:31.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.442+0000 7f130ad08700 1 -- 172.21.15.12:0/3878076308 shutdown_connections 2024-05-17T22:43:31.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:31.442+0000 7f130ad08700 1 -- 172.21.15.12:0/3878076308 wait complete. 2024-05-17T22:43:32.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:31 smithi151 ceph-mon[36101]: pgmap v567: 129 pgs: 129 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.4 MiB/s wr, 742 op/s 2024-05-17T22:43:32.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:31 smithi096 ceph-mon[36065]: pgmap v567: 129 pgs: 129 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.4 MiB/s wr, 742 op/s 2024-05-17T22:43:32.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:31 smithi012 ceph-mon[31632]: pgmap v567: 129 pgs: 129 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.4 MiB/s wr, 742 op/s 2024-05-17T22:43:33.896 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-17T22:43:33.896 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-17T22:43:34.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:33 smithi151 ceph-mon[36101]: pgmap v568: 129 pgs: 129 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 8.3 MiB/s wr, 746 op/s 2024-05-17T22:43:34.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:33 smithi096 ceph-mon[36065]: pgmap v568: 129 pgs: 129 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 8.3 MiB/s wr, 746 op/s 2024-05-17T22:43:34.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:33 smithi012 ceph-mon[31632]: pgmap v568: 129 pgs: 129 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 8.3 MiB/s wr, 746 op/s 2024-05-17T22:43:34.378 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:43:36.108 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:35 smithi012 ceph-mon[31632]: pgmap v569: 129 pgs: 129 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 8.8 MiB/s wr, 789 op/s 2024-05-17T22:43:36.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:35 smithi151 ceph-mon[36101]: pgmap v569: 129 pgs: 129 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 8.8 MiB/s wr, 789 op/s 2024-05-17T22:43:36.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:35 smithi096 ceph-mon[36065]: pgmap v569: 129 pgs: 129 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 8.8 MiB/s wr, 789 op/s 2024-05-17T22:43:36.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 255.0 s, 867.6 tps, lat 115.406 ms stddev 140.253 2024-05-17T22:43:36.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.433+0000 7f40fc7f8700 1 -- 172.21.15.12:0/4076430734 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40f410a830 msgr2=0x7f40f410ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:36.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.433+0000 7f40fc7f8700 1 --2- 172.21.15.12:0/4076430734 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40f410a830 0x7f40f410ac40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f40e4009990 tx=0x7f40e4009ca0 comp rx=0 tx=0).stop 2024-05-17T22:43:36.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.433+0000 7f40fc7f8700 1 -- 172.21.15.12:0/4076430734 shutdown_connections 2024-05-17T22:43:36.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.433+0000 7f40fc7f8700 1 --2- 172.21.15.12:0/4076430734 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40f411d550 0x7f40f411f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:36.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.433+0000 7f40fc7f8700 1 --2- 172.21.15.12:0/4076430734 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40f410a830 0x7f40f410ac40 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:36.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.433+0000 7f40fc7f8700 1 --2- 172.21.15.12:0/4076430734 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40f410b180 0x7f40f411d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:36.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.433+0000 7f40fc7f8700 1 -- 172.21.15.12:0/4076430734 >> 172.21.15.12:0/4076430734 conn(0x7f40f406efc0 msgr2=0x7f40f406f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:43:36.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.436+0000 7f40fc7f8700 1 -- 172.21.15.12:0/4076430734 shutdown_connections 2024-05-17T22:43:36.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.436+0000 7f40fc7f8700 1 -- 172.21.15.12:0/4076430734 wait complete. 2024-05-17T22:43:36.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.437+0000 7f40fc7f8700 1 Processor -- start 2024-05-17T22:43:36.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.438+0000 7f40fc7f8700 1 -- start start 2024-05-17T22:43:36.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.439+0000 7f40fc7f8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40f410a830 0x7f40f41a9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:36.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.439+0000 7f40fc7f8700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40f410b180 0x7f40f41a9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:36.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.439+0000 7f40fc7f8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40f411d550 0x7f40f41ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:36.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.439+0000 7f40fc7f8700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f40f4122c80 con 0x7f40f410a830 2024-05-17T22:43:36.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.439+0000 7f40fc7f8700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f40f4122b00 con 0x7f40f411d550 2024-05-17T22:43:36.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.439+0000 7f40fc7f8700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f40f4122e00 con 0x7f40f410b180 2024-05-17T22:43:36.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.439+0000 7f40f9d93700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40f410b180 0x7f40f41a9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:36.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.439+0000 7f40f9d93700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40f410b180 0x7f40f41a9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:37624/0 (socket says 172.21.15.12:37624) 2024-05-17T22:43:36.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.439+0000 7f40f9d93700 1 -- 172.21.15.12:0/2415840104 learned_addr learned my addr 172.21.15.12:0/2415840104 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:43:36.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.439+0000 7f40fad95700 1 --2- 172.21.15.12:0/2415840104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40f411d550 0x7f40f41ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:36.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.440+0000 7f40f9d93700 1 -- 172.21.15.12:0/2415840104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40f411d550 msgr2=0x7f40f41ade90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:36.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.440+0000 7f40f9d93700 1 --2- 172.21.15.12:0/2415840104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40f411d550 0x7f40f41ade90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:36.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.440+0000 7f40f9d93700 1 -- 172.21.15.12:0/2415840104 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40f410a830 msgr2=0x7f40f41a9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:36.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.440+0000 7f40f9d93700 1 --2- 172.21.15.12:0/2415840104 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40f410a830 0x7f40f41a9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:36.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.440+0000 7f40f9d93700 1 -- 172.21.15.12:0/2415840104 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f41ae530 con 0x7f40f410b180 2024-05-17T22:43:36.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.444+0000 7f40f9d93700 1 --2- 172.21.15.12:0/2415840104 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40f410b180 0x7f40f41a9c00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f40ec00ba50 tx=0x7f40ec00be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:43:36.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.446+0000 7f40eb7fe700 1 -- 172.21.15.12:0/2415840104 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec019070 con 0x7f40f410b180 2024-05-17T22:43:36.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.446+0000 7f40eb7fe700 1 -- 172.21.15.12:0/2415840104 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40ec014440 con 0x7f40f410b180 2024-05-17T22:43:36.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.447+0000 7f40eb7fe700 1 -- 172.21.15.12:0/2415840104 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec0075d0 con 0x7f40f410b180 2024-05-17T22:43:36.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.447+0000 7f40fc7f8700 1 -- 172.21.15.12:0/2415840104 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40f41ae820 con 0x7f40f410b180 2024-05-17T22:43:36.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.447+0000 7f40fc7f8700 1 -- 172.21.15.12:0/2415840104 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40f41b6140 con 0x7f40f410b180 2024-05-17T22:43:36.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.453+0000 7f40eb7fe700 1 -- 172.21.15.12:0/2415840104 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f40ec00cab0 con 0x7f40f410b180 2024-05-17T22:43:36.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.456+0000 7f40eb7fe700 1 --2- 172.21.15.12:0/2415840104 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f40e00b2920 0x7f40e00b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:36.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.456+0000 7f40eb7fe700 1 -- 172.21.15.12:0/2415840104 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f40ec0e1d50 con 0x7f40f410b180 2024-05-17T22:43:36.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.457+0000 7f410b1d3b80 1 Processor -- start 2024-05-17T22:43:36.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.457+0000 7f410b1d3b80 1 -- start start 2024-05-17T22:43:36.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.457+0000 7f410b1d3b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ecb1e18a70 0x55ecb1f9c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:36.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.458+0000 7f410b1d3b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecb1f9cb20 0x55ecb1f96cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:36.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.458+0000 7f410b1d3b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb1f971f0 0x55ecb1f97600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:36.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.458+0000 7f40f9d93700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecb1f9cb20 0x55ecb1f96cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:36.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.458+0000 7f40f9d93700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecb1f9cb20 0x55ecb1f96cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:60312/0 (socket says 172.21.15.12:60312) 2024-05-17T22:43:36.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.458+0000 7f40f9d93700 1 -- 172.21.15.12:0/2880751876 learned_addr learned my addr 172.21.15.12:0/2880751876 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:43:36.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.458+0000 7f410b1d3b80 1 -- 172.21.15.12:0/2880751876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ecb1ec5310 con 0x55ecb1f971f0 2024-05-17T22:43:36.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.458+0000 7f410b1d3b80 1 -- 172.21.15.12:0/2880751876 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ecb1eb5050 con 0x55ecb1f9cb20 2024-05-17T22:43:36.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.458+0000 7f40fad95700 1 --2- 172.21.15.12:0/2880751876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb1f971f0 0x55ecb1f97600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:36.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.458+0000 7f410b1d3b80 1 -- 172.21.15.12:0/2880751876 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ecb1eb8ef0 con 0x55ecb1e18a70 2024-05-17T22:43:36.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.459+0000 7f40fad95700 1 -- 172.21.15.12:0/2880751876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ecb1e18a70 msgr2=0x55ecb1f9c5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T22:43:36.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.459+0000 7f40fad95700 1 --2- 172.21.15.12:0/2880751876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ecb1e18a70 0x55ecb1f9c5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:36.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.459+0000 7f40fad95700 1 -- 172.21.15.12:0/2880751876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecb1f9cb20 msgr2=0x55ecb1f96cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:43:36.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.459+0000 7f40fad95700 1 --2- 172.21.15.12:0/2880751876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecb1f9cb20 0x55ecb1f96cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:36.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.459+0000 7f40fad95700 1 -- 172.21.15.12:0/2880751876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecb1e5e610 con 0x55ecb1f971f0 2024-05-17T22:43:36.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.459+0000 7f40f9d93700 1 --2- 172.21.15.12:0/2880751876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecb1f9cb20 0x55ecb1f96cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:43:36.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.459+0000 7f40fad95700 1 --2- 172.21.15.12:0/2880751876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb1f971f0 0x55ecb1f97600 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f40f0009b80 tx=0x7f40f0009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0ab4e763aaed91c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:43:36.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.460+0000 7f40e8ff9700 1 -- 172.21.15.12:0/2880751876 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f0022ba0 con 0x55ecb1f971f0 2024-05-17T22:43:36.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.460+0000 7f40e8ff9700 1 -- 172.21.15.12:0/2880751876 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40f0022d40 con 0x55ecb1f971f0 2024-05-17T22:43:36.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.460+0000 7f40e8ff9700 1 -- 172.21.15.12:0/2880751876 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f002a740 con 0x55ecb1f971f0 2024-05-17T22:43:36.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.461+0000 7f410b1d3b80 1 -- 172.21.15.12:0/2880751876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb1f971f0 msgr2=0x55ecb1f97600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:36.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.461+0000 7f410b1d3b80 1 --2- 172.21.15.12:0/2880751876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb1f971f0 0x55ecb1f97600 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f40f0009b80 tx=0x7f40f0009e90 comp rx=0 tx=0).stop 2024-05-17T22:43:36.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.462+0000 7f40fa594700 1 --2- 172.21.15.12:0/2415840104 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f40e00b2920 0x7f40e00b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:36.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.463+0000 7f410b1d3b80 1 -- 172.21.15.12:0/2880751876 shutdown_connections 2024-05-17T22:43:36.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.463+0000 7f410b1d3b80 1 --2- 172.21.15.12:0/2880751876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb1f971f0 0x55ecb1f97600 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:36.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.463+0000 7f410b1d3b80 1 --2- 172.21.15.12:0/2880751876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ecb1e18a70 0x55ecb1f9c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:36.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.463+0000 7f410b1d3b80 1 --2- 172.21.15.12:0/2880751876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecb1f9cb20 0x55ecb1f96cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:36.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.463+0000 7f410b1d3b80 1 -- 172.21.15.12:0/2880751876 >> 172.21.15.12:0/2880751876 conn(0x55ecb1e5cf80 msgr2=0x55ecb1f91c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:43:36.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.464+0000 7f410b1d3b80 1 -- 172.21.15.12:0/2880751876 shutdown_connections 2024-05-17T22:43:36.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.464+0000 7f40fa594700 1 --2- 172.21.15.12:0/2415840104 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f40e00b2920 0x7f40e00b4e30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f40e4009750 tx=0x7f40e400b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:43:36.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.464+0000 7f410b1d3b80 1 -- 172.21.15.12:0/2880751876 wait complete. 2024-05-17T22:43:36.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.465+0000 7f410b1d3b80 1 -- start start 2024-05-17T22:43:36.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.466+0000 7f410b1d3b80 1 Processor -- start 2024-05-17T22:43:36.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.466+0000 7f410b1d3b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecb1e18a70 0x55ecb1fa01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:36.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.466+0000 7f410b1d3b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb1f971f0 0x55ecb1fa0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:36.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.467+0000 7f40fa594700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecb1e18a70 0x55ecb1fa01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:36.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.467+0000 7f40fa594700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecb1e18a70 0x55ecb1fa01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:60316/0 (socket says 172.21.15.12:60316) 2024-05-17T22:43:36.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.467+0000 7f40fa594700 1 -- 172.21.15.12:0/2351165128 learned_addr learned my addr 172.21.15.12:0/2351165128 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:43:36.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.467+0000 7f40f9d93700 1 --2- 172.21.15.12:0/2351165128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb1f971f0 0x55ecb1fa0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:36.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.467+0000 7f410b1d3b80 1 --2- 172.21.15.12:0/2351165128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ecb1f9cb20 0x55ecb1fa0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:36.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.467+0000 7f410b1d3b80 1 -- 172.21.15.12:0/2351165128 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ecb1dbc1d0 con 0x55ecb1f971f0 2024-05-17T22:43:36.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.467+0000 7f410b1d3b80 1 -- 172.21.15.12:0/2351165128 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ecb1e55f00 con 0x55ecb1e18a70 2024-05-17T22:43:36.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.468+0000 7f410b1d3b80 1 -- 172.21.15.12:0/2351165128 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ecb1fa3410 con 0x55ecb1f9cb20 2024-05-17T22:43:36.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.468+0000 7f410b1d3b80 10 client.0 _ll_register_callbacks cb 0x55ecb1e4ff90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:43:36.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.468+0000 7f40f9d93700 1 -- 172.21.15.12:0/2351165128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ecb1f9cb20 msgr2=0x55ecb1fa0c70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T22:43:36.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.468+0000 7f40f9d93700 1 --2- 172.21.15.12:0/2351165128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ecb1f9cb20 0x55ecb1fa0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:36.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.468+0000 7f40f9d93700 1 -- 172.21.15.12:0/2351165128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecb1e18a70 msgr2=0x55ecb1fa01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:43:36.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.468+0000 7f40f9d93700 1 --2- 172.21.15.12:0/2351165128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecb1e18a70 0x55ecb1fa01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:36.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.468+0000 7f40f9d93700 1 -- 172.21.15.12:0/2351165128 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecb1e05500 con 0x55ecb1f971f0 2024-05-17T22:43:36.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.469+0000 7f40f9d93700 1 --2- 172.21.15.12:0/2351165128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb1f971f0 0x55ecb1fa0730 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0a61f0 tx=0x7f40ec0a9700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dc9d4b344306b21 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:43:36.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.469+0000 7f40ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:43:36.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.469+0000 7f40ddffb700 1 -- 172.21.15.12:0/2351165128 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec0b1de0 con 0x55ecb1f971f0 2024-05-17T22:43:36.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.469+0000 7f410b1d3b80 1 -- 172.21.15.12:0/2351165128 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40f0022d40 con 0x55ecb1f971f0 2024-05-17T22:43:36.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.469+0000 7f40ddffb700 1 -- 172.21.15.12:0/2351165128 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f40ec0b1950 con 0x55ecb1f971f0 2024-05-17T22:43:36.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.470+0000 7f40ddffb700 1 -- 172.21.15.12:0/2351165128 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f40ec0b1b50 con 0x55ecb1f971f0 2024-05-17T22:43:36.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.470+0000 7f410b1d3b80 10 client.15732 fetch_fsmap learned FSMap version 33 2024-05-17T22:43:36.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.470+0000 7f410b1d3b80 1 -- 172.21.15.12:0/2351165128 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ecb1fade90 con 0x55ecb1f971f0 2024-05-17T22:43:36.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.470+0000 7f40ddffb700 1 -- 172.21.15.12:0/2351165128 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40f0022d40 con 0x55ecb1f971f0 2024-05-17T22:43:36.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.470+0000 7f40ddffb700 1 -- 172.21.15.12:0/2351165128 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f40ec0a7ce0 con 0x55ecb1f971f0 2024-05-17T22:43:36.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.471+0000 7f410b1d3b80 10 client.15732 fetch_fsmap finished waiting for FSMap version 33 2024-05-17T22:43:36.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.471+0000 7f410b1d3b80 10 client.15732 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:43:36.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.471+0000 7f410b1d3b80 20 client.15732 populate_metadata read hostname 'smithi012' 2024-05-17T22:43:36.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.471+0000 7f410b1d3b80 1 --2- 172.21.15.12:0/2351165128 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55ecb1fb04f0 0x55ecb1fb29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:36.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.471+0000 7f410b1d3b80 4 client.15732 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:43:36.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.471+0000 7f410b1d3b80 1 -- 172.21.15.12:0/2351165128 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ecb1ec7260 con 0x55ecb1fb04f0 2024-05-17T22:43:36.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.472+0000 7f40fa594700 1 --2- 172.21.15.12:0/2351165128 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55ecb1fb04f0 0x55ecb1fb29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:36.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.473+0000 7f40fa594700 1 --2- 172.21.15.12:0/2351165128 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55ecb1fb04f0 0x55ecb1fb29c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d6d507f71cae34 server_cookie=3af02699ed2cb645 in_seq=0 out_seq=0 2024-05-17T22:43:36.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.473+0000 7f40ddffb700 10 client.15732 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:43:36.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.474+0000 7f40ddffb700 1 -- 172.21.15.12:0/2351165128 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ecb1ec7260 con 0x55ecb1fb04f0 2024-05-17T22:43:36.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.474+0000 7f40ddffb700 10 client.15732 handle_command_reply: tid=0 2024-05-17T22:43:36.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.474+0000 7f410b1d3b80 1 client.15732 shutdown 2024-05-17T22:43:36.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.474+0000 7f410b1d3b80 1 -- 172.21.15.12:0/2351165128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb1f971f0 msgr2=0x55ecb1fa0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:36.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.474+0000 7f410b1d3b80 1 --2- 172.21.15.12:0/2351165128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb1f971f0 0x55ecb1fa0730 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0a61f0 tx=0x7f40ec0a9700 comp rx=0 tx=0).stop 2024-05-17T22:43:36.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.474+0000 7f410b1d3b80 1 -- 172.21.15.12:0/2351165128 shutdown_connections 2024-05-17T22:43:36.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.474+0000 7f410b1d3b80 1 --2- 172.21.15.12:0/2351165128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ecb1f9cb20 0x55ecb1fa0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:36.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.474+0000 7f410b1d3b80 1 --2- 172.21.15.12:0/2351165128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecb1e18a70 0x55ecb1fa01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:36.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.474+0000 7f410b1d3b80 1 --2- 172.21.15.12:0/2351165128 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55ecb1fb04f0 0x55ecb1fb29c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:36.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.474+0000 7f410b1d3b80 1 --2- 172.21.15.12:0/2351165128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb1f971f0 0x55ecb1fa0730 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:36.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.474+0000 7f410b1d3b80 1 -- 172.21.15.12:0/2351165128 >> 172.21.15.12:0/2351165128 conn(0x55ecb1e5cf80 msgr2=0x55ecb1f8bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:43:36.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.475+0000 7f40ddffb700 0 client.15732 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:43:36.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.475+0000 7f410b1d3b80 1 -- 172.21.15.12:0/2351165128 shutdown_connections 2024-05-17T22:43:36.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.475+0000 7f410b1d3b80 1 -- 172.21.15.12:0/2351165128 wait complete. 2024-05-17T22:43:36.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.475+0000 7f410b1d3b80 20 client.15732 trim_cache size 0 max 16384 2024-05-17T22:43:36.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.510+0000 7f410b1d3b80 1 Processor -- start 2024-05-17T22:43:36.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.511+0000 7f410b1d3b80 1 -- start start 2024-05-17T22:43:36.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.511+0000 7f410b1d3b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ecb1e18a70 0x55ecb1f96cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:36.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.511+0000 7f410b1d3b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb1f971f0 0x55ecb1f9a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:36.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.511+0000 7f410b1d3b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecb1f9cb20 0x55ecb1f8bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:36.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.511+0000 7f410b1d3b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f40f41b89f0 con 0x55ecb1f971f0 2024-05-17T22:43:36.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.511+0000 7f410b1d3b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f40f41b6310 con 0x55ecb1f9cb20 2024-05-17T22:43:36.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.511+0000 7f410b1d3b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f40f4118d50 con 0x55ecb1e18a70 2024-05-17T22:43:36.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.512+0000 7f40fad95700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecb1f9cb20 0x55ecb1f8bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:36.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.512+0000 7f40fa594700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ecb1e18a70 0x55ecb1f96cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:36.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.512+0000 7f40fad95700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecb1f9cb20 0x55ecb1f8bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:60320/0 (socket says 172.21.15.12:60320) 2024-05-17T22:43:36.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.512+0000 7f40fad95700 1 -- 172.21.15.12:0/3216269142 learned_addr learned my addr 172.21.15.12:0/3216269142 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:43:36.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.513+0000 7f40f9d93700 1 --2- 172.21.15.12:0/3216269142 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb1f971f0 0x55ecb1f9a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:36.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.513+0000 7f40fa594700 1 -- 172.21.15.12:0/3216269142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecb1f9cb20 msgr2=0x55ecb1f8bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:43:36.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.513+0000 7f40fa594700 1 --2- 172.21.15.12:0/3216269142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecb1f9cb20 0x55ecb1f8bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:36.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.513+0000 7f40fa594700 1 -- 172.21.15.12:0/3216269142 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb1f971f0 msgr2=0x55ecb1f9a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:43:36.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.513+0000 7f40fa594700 1 --2- 172.21.15.12:0/3216269142 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb1f971f0 0x55ecb1f9a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:36.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.513+0000 7f40fa594700 1 -- 172.21.15.12:0/3216269142 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecb1f9e740 con 0x55ecb1e18a70 2024-05-17T22:43:36.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.514+0000 7f40fa594700 1 --2- 172.21.15.12:0/3216269142 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ecb1e18a70 0x55ecb1f96cb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55ecb1fb30f0 tx=0x7f40e4026040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=620c478c202aff49 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:43:36.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.515+0000 7f40dffff700 1 -- 172.21.15.12:0/3216269142 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e400bdd0 con 0x55ecb1e18a70 2024-05-17T22:43:36.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.515+0000 7f40dffff700 1 -- 172.21.15.12:0/3216269142 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40e402f8e0 con 0x55ecb1e18a70 2024-05-17T22:43:36.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.515+0000 7f40dffff700 1 -- 172.21.15.12:0/3216269142 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e40236f0 con 0x55ecb1e18a70 2024-05-17T22:43:36.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.516+0000 7f410b1d3b80 1 -- 172.21.15.12:0/3216269142 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ecb1e18a70 msgr2=0x55ecb1f96cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:36.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.516+0000 7f410b1d3b80 1 --2- 172.21.15.12:0/3216269142 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ecb1e18a70 0x55ecb1f96cb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55ecb1fb30f0 tx=0x7f40e4026040 comp rx=0 tx=0).stop 2024-05-17T22:43:36.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.520+0000 7f410b1d3b80 1 -- 172.21.15.12:0/3216269142 shutdown_connections 2024-05-17T22:43:36.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.520+0000 7f410b1d3b80 1 --2- 172.21.15.12:0/3216269142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecb1f9cb20 0x55ecb1f8bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:36.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.520+0000 7f410b1d3b80 1 --2- 172.21.15.12:0/3216269142 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ecb1e18a70 0x55ecb1f96cb0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:36.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.520+0000 7f410b1d3b80 1 --2- 172.21.15.12:0/3216269142 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb1f971f0 0x55ecb1f9a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:36.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.520+0000 7f410b1d3b80 1 -- 172.21.15.12:0/3216269142 >> 172.21.15.12:0/3216269142 conn(0x55ecb1e5cf80 msgr2=0x55ecb1f9b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:43:36.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.523+0000 7f410b1d3b80 1 -- 172.21.15.12:0/3216269142 shutdown_connections 2024-05-17T22:43:36.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.525+0000 7f410b1d3b80 1 -- 172.21.15.12:0/3216269142 wait complete. 2024-05-17T22:43:36.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.526+0000 7f410b1d3b80 1 -- start start 2024-05-17T22:43:36.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.526+0000 7f410b1d3b80 1 Processor -- start 2024-05-17T22:43:36.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.530+0000 7f410b1d3b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecb1e18a70 0x55ecb1fb7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:36.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.530+0000 7f410b1d3b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb1f971f0 0x55ecb1fb8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:36.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.530+0000 7f410b1d3b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ecb1f9cb20 0x55ecb2019d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:36.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.530+0000 7f410b1d3b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f40f41193c0 con 0x55ecb1f971f0 2024-05-17T22:43:36.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.530+0000 7f410b1d3b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f40f411a830 con 0x55ecb1e18a70 2024-05-17T22:43:36.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.530+0000 7f410b1d3b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f40f4119a90 con 0x55ecb1f9cb20 2024-05-17T22:43:36.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.530+0000 7f410b1d3b80 10 client.0 _ll_register_callbacks cb 0x55ecb1e4ff90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:43:36.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.533+0000 7f40fad95700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ecb1f9cb20 0x55ecb2019d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:36.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.535+0000 7f40fa594700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecb1e18a70 0x55ecb1fb7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:36.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.535+0000 7f40fa594700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecb1e18a70 0x55ecb1fb7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:60336/0 (socket says 172.21.15.12:60336) 2024-05-17T22:43:36.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.535+0000 7f40fa594700 1 -- 172.21.15.12:0/610812613 learned_addr learned my addr 172.21.15.12:0/610812613 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:43:36.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.539+0000 7f40fa594700 1 -- 172.21.15.12:0/610812613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ecb1f9cb20 msgr2=0x55ecb2019d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:43:36.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.539+0000 7f40fa594700 1 --2- 172.21.15.12:0/610812613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ecb1f9cb20 0x55ecb2019d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:36.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.539+0000 7f40fa594700 1 -- 172.21.15.12:0/610812613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb1f971f0 msgr2=0x55ecb1fb8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:43:36.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.539+0000 7f40fa594700 1 --2- 172.21.15.12:0/610812613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb1f971f0 0x55ecb1fb8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:36.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.539+0000 7f40fa594700 1 -- 172.21.15.12:0/610812613 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecb1fb8af0 con 0x55ecb1e18a70 2024-05-17T22:43:36.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.540+0000 7f40fa594700 1 --2- 172.21.15.12:0/610812613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecb1e18a70 0x55ecb1fb7d20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f40e40059d0 tx=0x7f40e4009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67563021798cc06d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:43:36.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.541+0000 7f40ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:43:36.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.541+0000 7f40ddffb700 1 -- 172.21.15.12:0/610812613 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e403e020 con 0x55ecb1e18a70 2024-05-17T22:43:36.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.541+0000 7f410b1d3b80 1 -- 172.21.15.12:0/610812613 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40e402f8e0 con 0x55ecb1e18a70 2024-05-17T22:43:36.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.541+0000 7f40ddffb700 1 -- 172.21.15.12:0/610812613 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f40e400fdb0 con 0x55ecb1e18a70 2024-05-17T22:43:36.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.546+0000 7f40ddffb700 1 -- 172.21.15.12:0/610812613 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f40e4049050 con 0x55ecb1e18a70 2024-05-17T22:43:36.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.546+0000 7f410b1d3b80 10 client.25400 fetch_fsmap learned FSMap version 33 2024-05-17T22:43:36.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.546+0000 7f410b1d3b80 1 -- 172.21.15.12:0/610812613 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ecb201b2d0 con 0x55ecb1e18a70 2024-05-17T22:43:36.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.549+0000 7f40ddffb700 1 -- 172.21.15.12:0/610812613 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40e402f8e0 con 0x55ecb1e18a70 2024-05-17T22:43:36.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.549+0000 7f40ddffb700 1 -- 172.21.15.12:0/610812613 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f40e4038440 con 0x55ecb1e18a70 2024-05-17T22:43:36.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.549+0000 7f410b1d3b80 10 client.25400 fetch_fsmap finished waiting for FSMap version 33 2024-05-17T22:43:36.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.549+0000 7f410b1d3b80 10 client.25400 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:43:36.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.549+0000 7f410b1d3b80 20 client.25400 populate_metadata read hostname 'smithi012' 2024-05-17T22:43:36.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.549+0000 7f410b1d3b80 1 --2- 172.21.15.12:0/610812613 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55ecb201b980 0x55ecb201dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:36.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.549+0000 7f410b1d3b80 4 client.25400 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-17T22:43:36.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.549+0000 7f410b1d3b80 1 -- 172.21.15.12:0/610812613 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ecb1ec0a40 con 0x55ecb201b980 2024-05-17T22:43:36.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.551+0000 7f40f9d93700 1 --2- 172.21.15.12:0/610812613 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55ecb201b980 0x55ecb201dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:36.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.555+0000 7f40ddffb700 10 client.25400 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:43:36.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.555+0000 7f40f9d93700 1 --2- 172.21.15.12:0/610812613 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55ecb201b980 0x55ecb201dda0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b328e888b8d41791 server_cookie=b4d68d40104f5b2f in_seq=0 out_seq=0 2024-05-17T22:43:36.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.558+0000 7f40ddffb700 1 -- 172.21.15.12:0/610812613 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ecb1ec0a40 con 0x55ecb201b980 2024-05-17T22:43:36.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.558+0000 7f40ddffb700 10 client.25400 handle_command_reply: tid=0 2024-05-17T22:43:36.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.558+0000 7f410b1d3b80 1 client.25400 shutdown 2024-05-17T22:43:36.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.558+0000 7f410b1d3b80 1 -- 172.21.15.12:0/610812613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecb1e18a70 msgr2=0x55ecb1fb7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:36.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.558+0000 7f410b1d3b80 1 --2- 172.21.15.12:0/610812613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecb1e18a70 0x55ecb1fb7d20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f40e40059d0 tx=0x7f40e4009310 comp rx=0 tx=0).stop 2024-05-17T22:43:36.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.558+0000 7f410b1d3b80 1 -- 172.21.15.12:0/610812613 shutdown_connections 2024-05-17T22:43:36.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.558+0000 7f410b1d3b80 1 --2- 172.21.15.12:0/610812613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ecb1f9cb20 0x55ecb2019d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:36.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.558+0000 7f410b1d3b80 1 --2- 172.21.15.12:0/610812613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecb1e18a70 0x55ecb1fb7d20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:36.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.558+0000 7f410b1d3b80 1 --2- 172.21.15.12:0/610812613 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55ecb201b980 0x55ecb201dda0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:36.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.558+0000 7f410b1d3b80 1 --2- 172.21.15.12:0/610812613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb1f971f0 0x55ecb1fb8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:36.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.559+0000 7f410b1d3b80 1 -- 172.21.15.12:0/610812613 >> 172.21.15.12:0/610812613 conn(0x55ecb1e5cf80 msgr2=0x55ecb1f9d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:43:36.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.559+0000 7f40ddffb700 0 client.25400 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:43:36.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.562+0000 7f410b1d3b80 1 -- 172.21.15.12:0/610812613 shutdown_connections 2024-05-17T22:43:36.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.562+0000 7f410b1d3b80 1 -- 172.21.15.12:0/610812613 wait complete. 2024-05-17T22:43:36.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.562+0000 7f410b1d3b80 20 client.25400 trim_cache size 0 max 16384 2024-05-17T22:43:36.567 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:43:36.567 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-17T22:43:36.567 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "a1508799-50f7-4762-ba61-ba4579e29f3d", 2024-05-17T22:43:36.567 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-17T22:43:36.567 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:43:36.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.568+0000 7f40fc7f8700 1 -- 172.21.15.12:0/2415840104 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f40e00b2920 msgr2=0x7f40e00b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:36.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.568+0000 7f40fc7f8700 1 --2- 172.21.15.12:0/2415840104 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f40e00b2920 0x7f40e00b4e30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f40e4009750 tx=0x7f40e400b610 comp rx=0 tx=0).stop 2024-05-17T22:43:36.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.568+0000 7f40fc7f8700 1 -- 172.21.15.12:0/2415840104 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40f410b180 msgr2=0x7f40f41a9c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:36.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.568+0000 7f40fc7f8700 1 --2- 172.21.15.12:0/2415840104 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40f410b180 0x7f40f41a9c00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f40ec00ba50 tx=0x7f40ec00be10 comp rx=0 tx=0).stop 2024-05-17T22:43:36.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.571+0000 7f40fc7f8700 1 -- 172.21.15.12:0/2415840104 shutdown_connections 2024-05-17T22:43:36.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.571+0000 7f40fc7f8700 1 --2- 172.21.15.12:0/2415840104 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f40e00b2920 0x7f40e00b4e30 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:36.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.571+0000 7f40fc7f8700 1 --2- 172.21.15.12:0/2415840104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40f411d550 0x7f40f41ade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:36.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.571+0000 7f40fc7f8700 1 --2- 172.21.15.12:0/2415840104 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40f410a830 0x7f40f41a9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:36.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.571+0000 7f40fc7f8700 1 --2- 172.21.15.12:0/2415840104 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40f410b180 0x7f40f41a9c00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:36.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.571+0000 7f40fc7f8700 1 -- 172.21.15.12:0/2415840104 >> 172.21.15.12:0/2415840104 conn(0x7f40f406efc0 msgr2=0x7f40f411b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:43:36.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.572+0000 7f40fc7f8700 1 -- 172.21.15.12:0/2415840104 shutdown_connections 2024-05-17T22:43:36.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:36.573+0000 7f40fc7f8700 1 -- 172.21.15.12:0/2415840104 wait complete. 2024-05-17T22:43:37.387 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-17T22:43:37.809 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:43:38.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:37 smithi151 ceph-mon[36101]: scrub queued for path: / 2024-05-17T22:43:38.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:37 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:43:38.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:37 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:43:38.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:37 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:43:38.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:37 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:43:38.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:37 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:43:38.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:37 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:43:38.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:37 smithi151 ceph-mon[36101]: pgmap v570: 129 pgs: 129 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.0 MiB/s wr, 730 op/s 2024-05-17T22:43:38.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:37 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:43:38.256 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:37 smithi012 ceph-mon[31632]: scrub queued for path: / 2024-05-17T22:43:38.257 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:37 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:43:38.257 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:37 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:43:38.257 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:37 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:43:38.257 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:37 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:43:38.257 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:37 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:43:38.257 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:37 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:43:38.257 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:37 smithi012 ceph-mon[31632]: pgmap v570: 129 pgs: 129 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.0 MiB/s wr, 730 op/s 2024-05-17T22:43:38.257 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:37 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:43:38.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:37 smithi096 ceph-mon[36065]: scrub queued for path: / 2024-05-17T22:43:38.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:37 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:43:38.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:37 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:43:38.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:37 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:43:38.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:37 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:43:38.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:37 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:43:38.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:37 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:43:38.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:37 smithi096 ceph-mon[36065]: pgmap v570: 129 pgs: 129 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.0 MiB/s wr, 730 op/s 2024-05-17T22:43:38.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:37 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:43:39.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.864+0000 7f8f72156700 1 -- 172.21.15.12:0/35473721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f6c10e220 msgr2=0x7f8f6c110730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:39.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.864+0000 7f8f72156700 1 --2- 172.21.15.12:0/35473721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f6c10e220 0x7f8f6c110730 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8f64009990 tx=0x7f8f64009ca0 comp rx=0 tx=0).stop 2024-05-17T22:43:39.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.865+0000 7f8f72156700 1 -- 172.21.15.12:0/35473721 shutdown_connections 2024-05-17T22:43:39.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.865+0000 7f8f72156700 1 --2- 172.21.15.12:0/35473721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f6c10e220 0x7f8f6c110730 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:39.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.865+0000 7f8f72156700 1 --2- 172.21.15.12:0/35473721 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f6c104dc0 0x7f8f6c1051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:39.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.865+0000 7f8f72156700 1 --2- 172.21.15.12:0/35473721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f6c105710 0x7f8f6c10dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:39.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.865+0000 7f8f72156700 1 -- 172.21.15.12:0/35473721 >> 172.21.15.12:0/35473721 conn(0x7f8f6c079af0 msgr2=0x7f8f6c079f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:43:39.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.869+0000 7f8f72156700 1 -- 172.21.15.12:0/35473721 shutdown_connections 2024-05-17T22:43:39.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.869+0000 7f8f72156700 1 -- 172.21.15.12:0/35473721 wait complete. 2024-05-17T22:43:39.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.871+0000 7f8f72156700 1 Processor -- start 2024-05-17T22:43:39.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.875+0000 7f8f72156700 1 -- start start 2024-05-17T22:43:39.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.876+0000 7f8f72156700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f6c104dc0 0x7f8f6c10b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:39.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.876+0000 7f8f72156700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f6c105710 0x7f8f6c10b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:39.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.876+0000 7f8f72156700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f6c10e220 0x7f8f6c1acc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:39.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.876+0000 7f8f72156700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c104540 con 0x7f8f6c10e220 2024-05-17T22:43:39.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.876+0000 7f8f72156700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c1043c0 con 0x7f8f6c104dc0 2024-05-17T22:43:39.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.876+0000 7f8f72156700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c1046c0 con 0x7f8f6c105710 2024-05-17T22:43:39.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.877+0000 7f8f6b7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f6c104dc0 0x7f8f6c10b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:39.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.877+0000 7f8f6bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f6c10e220 0x7f8f6c1acc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:39.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.878+0000 7f8f6bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f6c10e220 0x7f8f6c1acc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36948/0 (socket says 172.21.15.12:36948) 2024-05-17T22:43:39.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.878+0000 7f8f6bfff700 1 -- 172.21.15.12:0/1208587751 learned_addr learned my addr 172.21.15.12:0/1208587751 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:43:39.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.878+0000 7f8f6bfff700 1 -- 172.21.15.12:0/1208587751 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f6c105710 msgr2=0x7f8f6c10b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:39.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.878+0000 7f8f6bfff700 1 --2- 172.21.15.12:0/1208587751 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f6c105710 0x7f8f6c10b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:39.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.878+0000 7f8f6bfff700 1 -- 172.21.15.12:0/1208587751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f6c104dc0 msgr2=0x7f8f6c10b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:39.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.878+0000 7f8f6bfff700 1 --2- 172.21.15.12:0/1208587751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f6c104dc0 0x7f8f6c10b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:39.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.878+0000 7f8f6bfff700 1 -- 172.21.15.12:0/1208587751 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f6c10bfc0 con 0x7f8f6c10e220 2024-05-17T22:43:39.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.878+0000 7f8f6bfff700 1 --2- 172.21.15.12:0/1208587751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f6c10e220 0x7f8f6c1acc20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8f640047b0 tx=0x7f8f64009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:43:39.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.881+0000 7f8f697fa700 1 -- 172.21.15.12:0/1208587751 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f640047e0 con 0x7f8f6c10e220 2024-05-17T22:43:39.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.881+0000 7f8f697fa700 1 -- 172.21.15.12:0/1208587751 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f64004e30 con 0x7f8f6c10e220 2024-05-17T22:43:39.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.882+0000 7f8f697fa700 1 -- 172.21.15.12:0/1208587751 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6400fab0 con 0x7f8f6c10e220 2024-05-17T22:43:39.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.883+0000 7f8f72156700 1 -- 172.21.15.12:0/1208587751 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f6c1ad160 con 0x7f8f6c10e220 2024-05-17T22:43:39.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.883+0000 7f8f72156700 1 -- 172.21.15.12:0/1208587751 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f6c1ad560 con 0x7f8f6c10e220 2024-05-17T22:43:39.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.888+0000 7f8f80b31b80 1 Processor -- start 2024-05-17T22:43:39.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.890+0000 7f8f80b31b80 1 -- start start 2024-05-17T22:43:39.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.891+0000 7f8f80b31b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e1d3749f0 0x562e1d4f85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:39.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.891+0000 7f8f697fa700 1 -- 172.21.15.12:0/1208587751 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8f64028020 con 0x7f8f6c10e220 2024-05-17T22:43:39.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.891+0000 7f8f80b31b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e1d4f8b10 0x562e1d4f2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:39.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.891+0000 7f8f80b31b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e1d4f3170 0x562e1d4f3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:39.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.891+0000 7f8f80b31b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x562e1d421290 con 0x562e1d3749f0 2024-05-17T22:43:39.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.891+0000 7f8f80b31b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562e1d410fd0 con 0x562e1d4f8b10 2024-05-17T22:43:39.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.891+0000 7f8f80b31b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562e1d414e70 con 0x562e1d4f3170 2024-05-17T22:43:39.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.892+0000 7f8f697fa700 1 --2- 172.21.15.12:0/1208587751 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8f580b28e0 0x7f8f580b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:39.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.892+0000 7f8f697fa700 1 -- 172.21.15.12:0/1208587751 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f8f640e2f80 con 0x7f8f6c10e220 2024-05-17T22:43:39.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.893+0000 7f8f6bfff700 1 --2- 172.21.15.12:0/1208587751 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8f580b28e0 0x7f8f580b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:39.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.894+0000 7f8f6bfff700 1 --2- 172.21.15.12:0/1208587751 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8f580b28e0 0x7f8f580b4df0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8f640a74b0 tx=0x7f8f6401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:43:39.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.894+0000 7f8f62ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e1d3749f0 0x562e1d4f85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:39.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.894+0000 7f8f62ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e1d3749f0 0x562e1d4f85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36960/0 (socket says 172.21.15.12:36960) 2024-05-17T22:43:39.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.894+0000 7f8f62ffd700 1 -- 172.21.15.12:0/4084312566 learned_addr learned my addr 172.21.15.12:0/4084312566 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:43:39.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.894+0000 7f8f62ffd700 1 --2- 172.21.15.12:0/4084312566 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e1d4f3170 0x562e1d4f3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:39.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.894+0000 7f8f6b7fe700 1 --2- 172.21.15.12:0/4084312566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e1d4f8b10 0x562e1d4f2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:39.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.898+0000 7f8f62ffd700 1 -- 172.21.15.12:0/4084312566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e1d4f8b10 msgr2=0x562e1d4f2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:43:39.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.898+0000 7f8f62ffd700 1 --2- 172.21.15.12:0/4084312566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e1d4f8b10 0x562e1d4f2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:39.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.898+0000 7f8f62ffd700 1 -- 172.21.15.12:0/4084312566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e1d3749f0 msgr2=0x562e1d4f85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:43:39.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.898+0000 7f8f62ffd700 1 --2- 172.21.15.12:0/4084312566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e1d3749f0 0x562e1d4f85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:39.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.898+0000 7f8f62ffd700 1 -- 172.21.15.12:0/4084312566 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e1d3ba590 con 0x562e1d4f3170 2024-05-17T22:43:39.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.898+0000 7f8f62ffd700 1 --2- 172.21.15.12:0/4084312566 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e1d4f3170 0x562e1d4f3580 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8f5400afd0 tx=0x7f8f54005970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee679f87f09a7bfe server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:43:39.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.899+0000 7f8f72156700 1 -- 172.21.15.12:0/4084312566 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f54004020 con 0x562e1d4f3170 2024-05-17T22:43:39.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.899+0000 7f8f72156700 1 -- 172.21.15.12:0/4084312566 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f54013e50 con 0x562e1d4f3170 2024-05-17T22:43:39.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.899+0000 7f8f72156700 1 -- 172.21.15.12:0/4084312566 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5400e460 con 0x562e1d4f3170 2024-05-17T22:43:39.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.900+0000 7f8f80b31b80 1 -- 172.21.15.12:0/4084312566 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e1d4f3170 msgr2=0x562e1d4f3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:39.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.900+0000 7f8f80b31b80 1 --2- 172.21.15.12:0/4084312566 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e1d4f3170 0x562e1d4f3580 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8f5400afd0 tx=0x7f8f54005970 comp rx=0 tx=0).stop 2024-05-17T22:43:39.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.902+0000 7f8f80b31b80 1 -- 172.21.15.12:0/4084312566 shutdown_connections 2024-05-17T22:43:39.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.902+0000 7f8f80b31b80 1 --2- 172.21.15.12:0/4084312566 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e1d4f3170 0x562e1d4f3580 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:39.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.902+0000 7f8f80b31b80 1 --2- 172.21.15.12:0/4084312566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e1d3749f0 0x562e1d4f85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:39.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.902+0000 7f8f80b31b80 1 --2- 172.21.15.12:0/4084312566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e1d4f8b10 0x562e1d4f2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:39.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.902+0000 7f8f80b31b80 1 -- 172.21.15.12:0/4084312566 >> 172.21.15.12:0/4084312566 conn(0x562e1d3b8f00 msgr2=0x562e1d4edf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:43:39.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.903+0000 7f8f80b31b80 1 -- 172.21.15.12:0/4084312566 shutdown_connections 2024-05-17T22:43:39.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.903+0000 7f8f80b31b80 1 -- 172.21.15.12:0/4084312566 wait complete. 2024-05-17T22:43:39.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.903+0000 7f8f80b31b80 1 -- start start 2024-05-17T22:43:39.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.904+0000 7f8f80b31b80 1 Processor -- start 2024-05-17T22:43:39.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.904+0000 7f8f80b31b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e1d3749f0 0x562e1d4fc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:39.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.904+0000 7f8f80b31b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e1d4f3170 0x562e1d4fc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:39.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.904+0000 7f8f80b31b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e1d4f8b10 0x562e1d519650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:39.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.905+0000 7f8f62ffd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e1d3749f0 0x562e1d4fc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:39.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.905+0000 7f8f62ffd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e1d3749f0 0x562e1d4fc2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:34374/0 (socket says 172.21.15.12:34374) 2024-05-17T22:43:39.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.905+0000 7f8f6b7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e1d4f3170 0x562e1d4fc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:39.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.905+0000 7f8f6b7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e1d4f3170 0x562e1d4fc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:47170/0 (socket says 172.21.15.12:47170) 2024-05-17T22:43:39.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.905+0000 7f8f62ffd700 1 -- 172.21.15.12:0/262897581 learned_addr learned my addr 172.21.15.12:0/262897581 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:43:39.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.905+0000 7f8f80b31b80 1 -- 172.21.15.12:0/262897581 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x562e1d318150 con 0x562e1d4f8b10 2024-05-17T22:43:39.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.905+0000 7f8f80b31b80 1 -- 172.21.15.12:0/262897581 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562e1d3b1e80 con 0x562e1d4f3170 2024-05-17T22:43:39.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.905+0000 7f8f80b31b80 1 -- 172.21.15.12:0/262897581 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562e1d510380 con 0x562e1d3749f0 2024-05-17T22:43:39.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.905+0000 7f8f62ffd700 1 --2- 172.21.15.12:0/262897581 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e1d4f8b10 0x562e1d519650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:39.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.905+0000 7f8f80b31b80 10 client.0 _ll_register_callbacks cb 0x562e1d3abf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:43:39.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.905+0000 7f8f62ffd700 1 -- 172.21.15.12:0/262897581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e1d4f3170 msgr2=0x562e1d4fc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:43:39.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.905+0000 7f8f62ffd700 1 --2- 172.21.15.12:0/262897581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e1d4f3170 0x562e1d4fc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:39.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.905+0000 7f8f62ffd700 1 -- 172.21.15.12:0/262897581 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e1d4f8b10 msgr2=0x562e1d519650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:43:39.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.905+0000 7f8f62ffd700 1 --2- 172.21.15.12:0/262897581 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e1d4f8b10 0x562e1d519650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:39.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.905+0000 7f8f62ffd700 1 -- 172.21.15.12:0/262897581 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e1d361480 con 0x562e1d3749f0 2024-05-17T22:43:39.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.911+0000 7f8f62ffd700 1 --2- 172.21.15.12:0/262897581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e1d3749f0 0x562e1d4fc2a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8f54005fd0 tx=0x7f8f54003b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a5794ca8e7550c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:43:39.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.912+0000 7f8f4affd700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:43:39.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.916+0000 7f8f4affd700 1 -- 172.21.15.12:0/262897581 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f54004290 con 0x562e1d3749f0 2024-05-17T22:43:39.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.916+0000 7f8f4affd700 1 -- 172.21.15.12:0/262897581 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f8f54004430 con 0x562e1d3749f0 2024-05-17T22:43:39.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.917+0000 7f8f80b31b80 1 -- 172.21.15.12:0/262897581 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f54013e50 con 0x562e1d3749f0 2024-05-17T22:43:39.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.919+0000 7f8f4affd700 1 -- 172.21.15.12:0/262897581 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8f54004a40 con 0x562e1d3749f0 2024-05-17T22:43:39.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.921+0000 7f8f80b31b80 10 client.25357 fetch_fsmap learned FSMap version 33 2024-05-17T22:43:39.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.921+0000 7f8f80b31b80 1 -- 172.21.15.12:0/262897581 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562e1d5004b0 con 0x562e1d3749f0 2024-05-17T22:43:39.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.922+0000 7f8f4affd700 1 -- 172.21.15.12:0/262897581 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f54013e50 con 0x562e1d3749f0 2024-05-17T22:43:39.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.922+0000 7f8f4affd700 1 -- 172.21.15.12:0/262897581 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f8f5402c070 con 0x562e1d3749f0 2024-05-17T22:43:39.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.923+0000 7f8f80b31b80 10 client.25357 fetch_fsmap finished waiting for FSMap version 33 2024-05-17T22:43:39.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.923+0000 7f8f80b31b80 10 client.25357 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:43:39.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.923+0000 7f8f80b31b80 20 client.25357 populate_metadata read hostname 'smithi012' 2024-05-17T22:43:39.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.923+0000 7f8f80b31b80 1 --2- 172.21.15.12:0/262897581 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x562e1d5029e0 0x562e1d504e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:39.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.923+0000 7f8f80b31b80 4 client.25357 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:43:39.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.923+0000 7f8f80b31b80 1 -- 172.21.15.12:0/262897581 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e1d4231e0 con 0x562e1d5029e0 2024-05-17T22:43:39.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.924+0000 7f8f6b7fe700 1 --2- 172.21.15.12:0/262897581 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x562e1d5029e0 0x562e1d504e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:39.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.928+0000 7f8f6b7fe700 1 --2- 172.21.15.12:0/262897581 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x562e1d5029e0 0x562e1d504e00 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fba5daa59cd7fa40 server_cookie=3507d787578f5d5d in_seq=0 out_seq=0 2024-05-17T22:43:39.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.928+0000 7f8f4affd700 10 client.25357 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:43:39.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.929+0000 7f8f4affd700 1 -- 172.21.15.12:0/262897581 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562e1d4231e0 con 0x562e1d5029e0 2024-05-17T22:43:39.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.929+0000 7f8f4affd700 10 client.25357 handle_command_reply: tid=0 2024-05-17T22:43:39.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.929+0000 7f8f80b31b80 1 client.25357 shutdown 2024-05-17T22:43:39.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.931+0000 7f8f80b31b80 1 -- 172.21.15.12:0/262897581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e1d3749f0 msgr2=0x562e1d4fc2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:39.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.931+0000 7f8f80b31b80 1 --2- 172.21.15.12:0/262897581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e1d3749f0 0x562e1d4fc2a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8f54005fd0 tx=0x7f8f54003b70 comp rx=0 tx=0).stop 2024-05-17T22:43:39.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.931+0000 7f8f80b31b80 1 -- 172.21.15.12:0/262897581 shutdown_connections 2024-05-17T22:43:39.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.931+0000 7f8f80b31b80 1 --2- 172.21.15.12:0/262897581 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x562e1d5029e0 0x562e1d504e00 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:39.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.931+0000 7f8f80b31b80 1 --2- 172.21.15.12:0/262897581 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e1d4f8b10 0x562e1d519650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:39.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.931+0000 7f8f80b31b80 1 --2- 172.21.15.12:0/262897581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e1d3749f0 0x562e1d4fc2a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:39.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.931+0000 7f8f80b31b80 1 --2- 172.21.15.12:0/262897581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e1d4f3170 0x562e1d4fc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:39.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.931+0000 7f8f80b31b80 1 -- 172.21.15.12:0/262897581 >> 172.21.15.12:0/262897581 conn(0x562e1d3b8f00 msgr2=0x562e1d4f1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:43:39.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.931+0000 7f8f4affd700 0 client.25357 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:43:39.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.932+0000 7f8f80b31b80 1 -- 172.21.15.12:0/262897581 shutdown_connections 2024-05-17T22:43:39.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.932+0000 7f8f80b31b80 1 -- 172.21.15.12:0/262897581 wait complete. 2024-05-17T22:43:39.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.932+0000 7f8f80b31b80 20 client.25357 trim_cache size 0 max 16384 2024-05-17T22:43:39.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.965+0000 7f8f80b31b80 1 Processor -- start 2024-05-17T22:43:39.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.965+0000 7f8f80b31b80 1 -- start start 2024-05-17T22:43:39.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.965+0000 7f8f80b31b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e1d3749f0 0x562e1d4f2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:39.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.965+0000 7f8f80b31b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e1d4f3170 0x562e1d50fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:39.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.966+0000 7f8f80b31b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e1d4f8b10 0x562e1d4f7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:39.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.966+0000 7f8f80b31b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c1aff60 con 0x562e1d4f8b10 2024-05-17T22:43:39.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.966+0000 7f8f80b31b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c1ad850 con 0x562e1d4f3170 2024-05-17T22:43:39.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.966+0000 7f8f80b31b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c0737f0 con 0x562e1d3749f0 2024-05-17T22:43:39.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.966+0000 7f8f6b7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e1d4f3170 0x562e1d50fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:39.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.966+0000 7f8f6b7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e1d4f3170 0x562e1d50fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:47174/0 (socket says 172.21.15.12:47174) 2024-05-17T22:43:39.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.966+0000 7f8f6b7fe700 1 -- 172.21.15.12:0/55271729 learned_addr learned my addr 172.21.15.12:0/55271729 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:43:39.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.966+0000 7f8f6b7fe700 1 -- 172.21.15.12:0/55271729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e1d3749f0 msgr2=0x562e1d4f2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:43:39.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.966+0000 7f8f6b7fe700 1 --2- 172.21.15.12:0/55271729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e1d3749f0 0x562e1d4f2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:39.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.966+0000 7f8f6b7fe700 1 -- 172.21.15.12:0/55271729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e1d4f8b10 msgr2=0x562e1d4f7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:43:39.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.966+0000 7f8f6b7fe700 1 --2- 172.21.15.12:0/55271729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e1d4f8b10 0x562e1d4f7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:39.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.966+0000 7f8f6b7fe700 1 -- 172.21.15.12:0/55271729 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e1d4fcb70 con 0x562e1d4f3170 2024-05-17T22:43:39.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.967+0000 7f8f6b7fe700 1 --2- 172.21.15.12:0/55271729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e1d4f3170 0x562e1d50fe60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c006130 tx=0x7f8f5c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ee7a2b697391faa server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:43:39.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.968+0000 7f8f617fa700 1 -- 172.21.15.12:0/55271729 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c023070 con 0x562e1d4f3170 2024-05-17T22:43:39.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.968+0000 7f8f617fa700 1 -- 172.21.15.12:0/55271729 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f5c01d300 con 0x562e1d4f3170 2024-05-17T22:43:39.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.968+0000 7f8f617fa700 1 -- 172.21.15.12:0/55271729 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c004270 con 0x562e1d4f3170 2024-05-17T22:43:39.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.969+0000 7f8f80b31b80 1 -- 172.21.15.12:0/55271729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e1d4f3170 msgr2=0x562e1d50fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:39.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.969+0000 7f8f80b31b80 1 --2- 172.21.15.12:0/55271729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e1d4f3170 0x562e1d50fe60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c006130 tx=0x7f8f5c004ad0 comp rx=0 tx=0).stop 2024-05-17T22:43:39.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.969+0000 7f8f80b31b80 1 -- 172.21.15.12:0/55271729 shutdown_connections 2024-05-17T22:43:39.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.969+0000 7f8f80b31b80 1 --2- 172.21.15.12:0/55271729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e1d4f8b10 0x562e1d4f7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:39.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.969+0000 7f8f80b31b80 1 --2- 172.21.15.12:0/55271729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e1d3749f0 0x562e1d4f2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:39.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.969+0000 7f8f80b31b80 1 --2- 172.21.15.12:0/55271729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e1d4f3170 0x562e1d50fe60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:39.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.969+0000 7f8f80b31b80 1 -- 172.21.15.12:0/55271729 >> 172.21.15.12:0/55271729 conn(0x562e1d3b8f00 msgr2=0x562e1d4fe500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:43:39.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.969+0000 7f8f80b31b80 1 -- 172.21.15.12:0/55271729 shutdown_connections 2024-05-17T22:43:39.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.969+0000 7f8f80b31b80 1 -- 172.21.15.12:0/55271729 wait complete. 2024-05-17T22:43:39.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.970+0000 7f8f80b31b80 1 -- start start 2024-05-17T22:43:39.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.970+0000 7f8f80b31b80 1 Processor -- start 2024-05-17T22:43:39.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.971+0000 7f8f80b31b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e1d3749f0 0x562e1d512420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:39.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.971+0000 7f8f80b31b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e1d4f3170 0x562e1d512960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:39.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.971+0000 7f8f62ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e1d3749f0 0x562e1d512420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:39.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.971+0000 7f8f62ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e1d3749f0 0x562e1d512420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36976/0 (socket says 172.21.15.12:36976) 2024-05-17T22:43:39.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.971+0000 7f8f62ffd700 1 -- 172.21.15.12:0/141067417 learned_addr learned my addr 172.21.15.12:0/141067417 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:43:39.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.971+0000 7f8f80b31b80 1 --2- 172.21.15.12:0/141067417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e1d4f8b10 0x562e1d512ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:39.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.971+0000 7f8f80b31b80 1 -- 172.21.15.12:0/141067417 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c073e60 con 0x562e1d3749f0 2024-05-17T22:43:39.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.971+0000 7f8f80b31b80 1 -- 172.21.15.12:0/141067417 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c0752d0 con 0x562e1d4f8b10 2024-05-17T22:43:39.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.971+0000 7f8f80b31b80 1 -- 172.21.15.12:0/141067417 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c074530 con 0x562e1d4f3170 2024-05-17T22:43:39.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.971+0000 7f8f6b7fe700 1 --2- 172.21.15.12:0/141067417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e1d4f3170 0x562e1d512960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:39.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.971+0000 7f8f80b31b80 10 client.0 _ll_register_callbacks cb 0x562e1d3abf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:43:39.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.972+0000 7f8f6b7fe700 1 -- 172.21.15.12:0/141067417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e1d4f8b10 msgr2=0x562e1d512ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:43:39.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.972+0000 7f8f6b7fe700 1 --2- 172.21.15.12:0/141067417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e1d4f8b10 0x562e1d512ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:39.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.972+0000 7f8f6b7fe700 1 -- 172.21.15.12:0/141067417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e1d3749f0 msgr2=0x562e1d512420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:43:39.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.972+0000 7f8f6b7fe700 1 --2- 172.21.15.12:0/141067417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e1d3749f0 0x562e1d512420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:39.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.972+0000 7f8f6b7fe700 1 -- 172.21.15.12:0/141067417 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e1d597af0 con 0x562e1d4f3170 2024-05-17T22:43:39.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.972+0000 7f8f62ffd700 1 --2- 172.21.15.12:0/141067417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e1d3749f0 0x562e1d512420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:43:39.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.972+0000 7f8f80b31b80 1 -- 172.21.15.12:0/141067417 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f5c01d300 con 0x562e1d4f3170 2024-05-17T22:43:39.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.972+0000 7f8f6b7fe700 1 --2- 172.21.15.12:0/141067417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e1d4f3170 0x562e1d512960 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x562e1d4f8f60 tx=0x7f8f5c007680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=921990f6d094adce server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:43:39.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.972+0000 7f8f4affd700 10 client.25360 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:43:39.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.972+0000 7f8f4affd700 1 -- 172.21.15.12:0/141067417 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c023040 con 0x562e1d4f3170 2024-05-17T22:43:39.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.972+0000 7f8f4affd700 1 -- 172.21.15.12:0/141067417 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f8f5c01d4b0 con 0x562e1d4f3170 2024-05-17T22:43:39.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.972+0000 7f8f4affd700 1 -- 172.21.15.12:0/141067417 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8f5c01d6f0 con 0x562e1d4f3170 2024-05-17T22:43:39.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.972+0000 7f8f80b31b80 10 client.25360 fetch_fsmap learned FSMap version 33 2024-05-17T22:43:39.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.972+0000 7f8f80b31b80 1 -- 172.21.15.12:0/141067417 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({fsmap=33}) -- 0x562e1d598b10 con 0x562e1d4f3170 2024-05-17T22:43:39.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.973+0000 7f8f4affd700 1 -- 172.21.15.12:0/141067417 <== mon.1 v2:172.21.15.151:3300/0 4 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f8f5c01d890 con 0x562e1d4f3170 2024-05-17T22:43:39.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.973+0000 7f8f80b31b80 10 client.25360 fetch_fsmap finished waiting for FSMap version 33 2024-05-17T22:43:39.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.973+0000 7f8f80b31b80 10 client.25360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:43:39.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.973+0000 7f8f80b31b80 20 client.25360 populate_metadata read hostname 'smithi012' 2024-05-17T22:43:39.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.973+0000 7f8f80b31b80 1 --2- 172.21.15.12:0/141067417 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x562e1d598fb0 0x562e1d59b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:43:39.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.973+0000 7f8f80b31b80 4 client.25360 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:43:39.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.973+0000 7f8f80b31b80 1 -- 172.21.15.12:0/141067417 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562e1d41c9c0 con 0x562e1d598fb0 2024-05-17T22:43:39.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.973+0000 7f8f62ffd700 1 --2- 172.21.15.12:0/141067417 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x562e1d598fb0 0x562e1d59b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:43:39.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.975+0000 7f8f62ffd700 1 --2- 172.21.15.12:0/141067417 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x562e1d598fb0 0x562e1d59b3f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad1c9d28aeb3e5f7 server_cookie=e31ae326e80617b7 in_seq=0 out_seq=0 2024-05-17T22:43:39.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.975+0000 7f8f4affd700 10 client.25360 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:43:39.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.976+0000 7f8f4affd700 1 -- 172.21.15.12:0/141067417 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x562e1d41c9c0 con 0x562e1d598fb0 2024-05-17T22:43:39.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.976+0000 7f8f4affd700 10 client.25360 handle_command_reply: tid=0 2024-05-17T22:43:39.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.976+0000 7f8f80b31b80 1 client.25360 shutdown 2024-05-17T22:43:39.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.976+0000 7f8f80b31b80 1 -- 172.21.15.12:0/141067417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e1d4f3170 msgr2=0x562e1d512960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:39.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.976+0000 7f8f80b31b80 1 --2- 172.21.15.12:0/141067417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e1d4f3170 0x562e1d512960 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x562e1d4f8f60 tx=0x7f8f5c007680 comp rx=0 tx=0).stop 2024-05-17T22:43:39.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.976+0000 7f8f80b31b80 1 -- 172.21.15.12:0/141067417 shutdown_connections 2024-05-17T22:43:39.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.976+0000 7f8f80b31b80 1 --2- 172.21.15.12:0/141067417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e1d4f8b10 0x562e1d512ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:39.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.976+0000 7f8f80b31b80 1 --2- 172.21.15.12:0/141067417 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x562e1d598fb0 0x562e1d59b3f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:39.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.976+0000 7f8f80b31b80 1 --2- 172.21.15.12:0/141067417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e1d3749f0 0x562e1d512420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:39.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.976+0000 7f8f80b31b80 1 --2- 172.21.15.12:0/141067417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e1d4f3170 0x562e1d512960 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:39.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.976+0000 7f8f4affd700 0 client.25360 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:43:39.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.976+0000 7f8f80b31b80 1 -- 172.21.15.12:0/141067417 >> 172.21.15.12:0/141067417 conn(0x562e1d3b8f00 msgr2=0x562e1d4f91b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:43:39.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.977+0000 7f8f80b31b80 1 -- 172.21.15.12:0/141067417 shutdown_connections 2024-05-17T22:43:39.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.977+0000 7f8f80b31b80 1 -- 172.21.15.12:0/141067417 wait complete. 2024-05-17T22:43:39.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.977+0000 7f8f80b31b80 20 client.25360 trim_cache size 0 max 16384 2024-05-17T22:43:39.981 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:43:39.981 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-17T22:43:39.981 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-17T22:43:39.981 INFO:teuthology.orchestra.run.smithi012.stdout: "a1508799-50f7-4762-ba61-ba4579e29f3d": { 2024-05-17T22:43:39.981 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-17T22:43:39.982 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "a1508799-50f7-4762-ba61-ba4579e29f3d", 2024-05-17T22:43:39.982 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-17T22:43:39.982 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:43:39.982 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:43:39.982 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:43:39.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.978+0000 7f8f627fc700 1 -- 172.21.15.12:0/1208587751 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8f580b28e0 msgr2=0x7f8f580b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:39.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.978+0000 7f8f627fc700 1 --2- 172.21.15.12:0/1208587751 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8f580b28e0 0x7f8f580b4df0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8f640a74b0 tx=0x7f8f6401a040 comp rx=0 tx=0).stop 2024-05-17T22:43:39.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.978+0000 7f8f627fc700 1 -- 172.21.15.12:0/1208587751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f6c10e220 msgr2=0x7f8f6c1acc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:43:39.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.978+0000 7f8f627fc700 1 --2- 172.21.15.12:0/1208587751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f6c10e220 0x7f8f6c1acc20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8f640047b0 tx=0x7f8f64009f90 comp rx=0 tx=0).stop 2024-05-17T22:43:39.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.978+0000 7f8f627fc700 1 -- 172.21.15.12:0/1208587751 shutdown_connections 2024-05-17T22:43:39.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.978+0000 7f8f627fc700 1 --2- 172.21.15.12:0/1208587751 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8f580b28e0 0x7f8f580b4df0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:39.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.978+0000 7f8f627fc700 1 --2- 172.21.15.12:0/1208587751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f6c10e220 0x7f8f6c1acc20 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:39.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.978+0000 7f8f627fc700 1 --2- 172.21.15.12:0/1208587751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f6c104dc0 0x7f8f6c10b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:39.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.978+0000 7f8f627fc700 1 --2- 172.21.15.12:0/1208587751 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f6c105710 0x7f8f6c10b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:43:39.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.978+0000 7f8f627fc700 1 -- 172.21.15.12:0/1208587751 >> 172.21.15.12:0/1208587751 conn(0x7f8f6c079af0 msgr2=0x7f8f6c100b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:43:39.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.978+0000 7f8f627fc700 1 -- 172.21.15.12:0/1208587751 shutdown_connections 2024-05-17T22:43:39.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:43:39.978+0000 7f8f627fc700 1 -- 172.21.15.12:0/1208587751 wait complete. 2024-05-17T22:43:40.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:39 smithi151 ceph-mon[36101]: pgmap v571: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.5 MiB/s wr, 785 op/s 2024-05-17T22:43:40.297 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:39 smithi012 ceph-mon[31632]: pgmap v571: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.5 MiB/s wr, 785 op/s 2024-05-17T22:43:40.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:39 smithi096 ceph-mon[36065]: pgmap v571: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.5 MiB/s wr, 785 op/s 2024-05-17T22:43:41.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 260.0 s, 870.4 tps, lat 114.773 ms stddev 126.408 2024-05-17T22:43:41.725 INFO:tasks.cephfs.filesystem:scrub status for tag:a1508799-50f7-4762-ba61-ba4579e29f3d - {'path': '/', 'tag': 'a1508799-50f7-4762-ba61-ba4579e29f3d', 'options': 'recursive,force'} 2024-05-17T22:43:42.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:41 smithi151 ceph-mon[36101]: pgmap v572: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.8 MiB/s wr, 573 op/s 2024-05-17T22:43:42.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:41 smithi096 ceph-mon[36065]: pgmap v572: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.8 MiB/s wr, 573 op/s 2024-05-17T22:43:42.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:41 smithi012 ceph-mon[31632]: pgmap v572: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.8 MiB/s wr, 573 op/s 2024-05-17T22:43:44.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:43 smithi151 ceph-mon[36101]: pgmap v573: 129 pgs: 129 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.1 MiB/s wr, 580 op/s 2024-05-17T22:43:44.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:43 smithi151 ceph-mon[36101]: scrub summary: idle 2024-05-17T22:43:44.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:43 smithi096 ceph-mon[36065]: pgmap v573: 129 pgs: 129 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.1 MiB/s wr, 580 op/s 2024-05-17T22:43:44.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:43 smithi096 ceph-mon[36065]: scrub summary: idle 2024-05-17T22:43:44.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:43 smithi012 ceph-mon[31632]: pgmap v573: 129 pgs: 129 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.1 MiB/s wr, 580 op/s 2024-05-17T22:43:44.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:43 smithi012 ceph-mon[31632]: scrub summary: idle 2024-05-17T22:43:45.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:44 smithi151 ceph-mon[36101]: scrub summary: idle 2024-05-17T22:43:45.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:44 smithi151 ceph-mon[36101]: scrub summary: idle 2024-05-17T22:43:45.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:44 smithi096 ceph-mon[36065]: scrub summary: idle 2024-05-17T22:43:45.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:44 smithi096 ceph-mon[36065]: scrub summary: idle 2024-05-17T22:43:45.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:44 smithi012 ceph-mon[31632]: scrub summary: idle 2024-05-17T22:43:45.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:44 smithi012 ceph-mon[31632]: scrub summary: idle 2024-05-17T22:43:46.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:45 smithi151 ceph-mon[36101]: pgmap v574: 129 pgs: 129 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.2 MiB/s wr, 561 op/s 2024-05-17T22:43:46.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:45 smithi096 ceph-mon[36065]: pgmap v574: 129 pgs: 129 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.2 MiB/s wr, 561 op/s 2024-05-17T22:43:46.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:45 smithi012 ceph-mon[31632]: pgmap v574: 129 pgs: 129 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.2 MiB/s wr, 561 op/s 2024-05-17T22:43:46.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 265.0 s, 1146.4 tps, lat 88.675 ms stddev 107.835 2024-05-17T22:43:48.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:47 smithi151 ceph-mon[36101]: pgmap v575: 129 pgs: 129 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.7 MiB/s wr, 528 op/s 2024-05-17T22:43:48.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:47 smithi096 ceph-mon[36065]: pgmap v575: 129 pgs: 129 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.7 MiB/s wr, 528 op/s 2024-05-17T22:43:48.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:47 smithi012 ceph-mon[31632]: pgmap v575: 129 pgs: 129 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.7 MiB/s wr, 528 op/s 2024-05-17T22:43:50.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:49 smithi096 ceph-mon[36065]: pgmap v576: 129 pgs: 129 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.6 MiB/s wr, 613 op/s 2024-05-17T22:43:50.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:49 smithi012 ceph-mon[31632]: pgmap v576: 129 pgs: 129 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.6 MiB/s wr, 613 op/s 2024-05-17T22:43:50.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:49 smithi151 ceph-mon[36101]: pgmap v576: 129 pgs: 129 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.6 MiB/s wr, 613 op/s 2024-05-17T22:43:51.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 270.0 s, 783.8 tps, lat 125.183 ms stddev 145.827 2024-05-17T22:43:52.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:51 smithi096 ceph-mon[36065]: pgmap v577: 129 pgs: 129 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.6 MiB/s wr, 506 op/s 2024-05-17T22:43:52.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:51 smithi012 ceph-mon[31632]: pgmap v577: 129 pgs: 129 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.6 MiB/s wr, 506 op/s 2024-05-17T22:43:52.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:51 smithi151 ceph-mon[36101]: pgmap v577: 129 pgs: 129 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.6 MiB/s wr, 506 op/s 2024-05-17T22:43:54.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:54 smithi096 ceph-mon[36065]: pgmap v578: 129 pgs: 129 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.9 MiB/s wr, 574 op/s 2024-05-17T22:43:54.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:54 smithi012 ceph-mon[31632]: pgmap v578: 129 pgs: 129 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.9 MiB/s wr, 574 op/s 2024-05-17T22:43:54.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:54 smithi151 ceph-mon[36101]: pgmap v578: 129 pgs: 129 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.9 MiB/s wr, 574 op/s 2024-05-17T22:43:56.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:56 smithi096 ceph-mon[36065]: pgmap v579: 129 pgs: 129 active+clean; 1.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.4 MiB/s wr, 578 op/s 2024-05-17T22:43:56.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:56 smithi012 ceph-mon[31632]: pgmap v579: 129 pgs: 129 active+clean; 1.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.4 MiB/s wr, 578 op/s 2024-05-17T22:43:56.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 275.0 s, 683.4 tps, lat 145.821 ms stddev 161.960 2024-05-17T22:43:56.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:56 smithi151 ceph-mon[36101]: pgmap v579: 129 pgs: 129 active+clean; 1.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.4 MiB/s wr, 578 op/s 2024-05-17T22:43:57.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:57 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:43:57.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:57 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:43:57.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:57 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:43:58.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:58 smithi096 ceph-mon[36065]: pgmap v580: 129 pgs: 129 active+clean; 1.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 4.6 MiB/s wr, 529 op/s 2024-05-17T22:43:58.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:58 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:43:58.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:58 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:43:58.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:58 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:43:58.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:58 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:43:58.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:58 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:43:58.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:58 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:43:58.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:58 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:43:58.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:58 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:43:58.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:58 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:43:58.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:58 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:43:58.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:58 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:43:58.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:58 smithi012 ceph-mon[31632]: pgmap v580: 129 pgs: 129 active+clean; 1.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 4.6 MiB/s wr, 529 op/s 2024-05-17T22:43:58.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:58 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:43:58.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:58 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:43:58.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:58 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:43:58.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:58 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:43:58.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:58 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:43:58.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:58 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:43:58.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:58 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:43:58.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:58 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:43:58.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:58 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:43:58.343 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:58 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:43:58.343 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:58 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:43:58.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:58 smithi151 ceph-mon[36101]: pgmap v580: 129 pgs: 129 active+clean; 1.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 4.6 MiB/s wr, 529 op/s 2024-05-17T22:43:58.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:58 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:43:58.492 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:58 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:43:58.492 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:58 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:43:58.492 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:58 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:43:58.492 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:58 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:43:58.492 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:58 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:43:58.492 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:58 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:43:58.492 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:58 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:43:58.492 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:58 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:43:58.492 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:58 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:43:58.493 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:58 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:43:59.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:59 smithi096 ceph-mon[36065]: Adjusting osd_memory_target on smithi096 to 176.2M 2024-05-17T22:43:59.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:59 smithi096 ceph-mon[36065]: Unable to set osd_memory_target on smithi096 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-05-17T22:43:59.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:59 smithi096 ceph-mon[36065]: Adjusting osd_memory_target on smithi151 to 176.2M 2024-05-17T22:43:59.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:43:59 smithi096 ceph-mon[36065]: Unable to set osd_memory_target on smithi151 to 184786739: error parsing value: Value '184786739' is below minimum 939524096 2024-05-17T22:43:59.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:59 smithi012 ceph-mon[31632]: Adjusting osd_memory_target on smithi096 to 176.2M 2024-05-17T22:43:59.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:59 smithi012 ceph-mon[31632]: Unable to set osd_memory_target on smithi096 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-05-17T22:43:59.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:59 smithi012 ceph-mon[31632]: Adjusting osd_memory_target on smithi151 to 176.2M 2024-05-17T22:43:59.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:43:59 smithi012 ceph-mon[31632]: Unable to set osd_memory_target on smithi151 to 184786739: error parsing value: Value '184786739' is below minimum 939524096 2024-05-17T22:43:59.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:59 smithi151 ceph-mon[36101]: Adjusting osd_memory_target on smithi096 to 176.2M 2024-05-17T22:43:59.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:59 smithi151 ceph-mon[36101]: Unable to set osd_memory_target on smithi096 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-05-17T22:43:59.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:59 smithi151 ceph-mon[36101]: Adjusting osd_memory_target on smithi151 to 176.2M 2024-05-17T22:43:59.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:43:59 smithi151 ceph-mon[36101]: Unable to set osd_memory_target on smithi151 to 184786739: error parsing value: Value '184786739' is below minimum 939524096 2024-05-17T22:44:00.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:00 smithi096 ceph-mon[36065]: pgmap v581: 129 pgs: 129 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 7.6 MiB/s wr, 752 op/s 2024-05-17T22:44:00.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:00 smithi012 ceph-mon[31632]: pgmap v581: 129 pgs: 129 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 7.6 MiB/s wr, 752 op/s 2024-05-17T22:44:00.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:00 smithi151 ceph-mon[36101]: pgmap v581: 129 pgs: 129 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 7.6 MiB/s wr, 752 op/s 2024-05-17T22:44:01.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 280.0 s, 650.2 tps, lat 158.193 ms stddev 176.421 2024-05-17T22:44:02.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:02 smithi012 ceph-mon[31632]: pgmap v582: 129 pgs: 129 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 7.6 MiB/s wr, 703 op/s 2024-05-17T22:44:02.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:02 smithi151 ceph-mon[36101]: pgmap v582: 129 pgs: 129 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 7.6 MiB/s wr, 703 op/s 2024-05-17T22:44:02.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:02 smithi096 ceph-mon[36065]: pgmap v582: 129 pgs: 129 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 7.6 MiB/s wr, 703 op/s 2024-05-17T22:44:04.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:04 smithi012 ceph-mon[31632]: pgmap v583: 129 pgs: 129 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8.0 MiB/s wr, 744 op/s 2024-05-17T22:44:04.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:04 smithi151 ceph-mon[36101]: pgmap v583: 129 pgs: 129 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8.0 MiB/s wr, 744 op/s 2024-05-17T22:44:04.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:04 smithi096 ceph-mon[36065]: pgmap v583: 129 pgs: 129 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8.0 MiB/s wr, 744 op/s 2024-05-17T22:44:06.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:06 smithi012 ceph-mon[31632]: pgmap v584: 129 pgs: 129 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 8.2 MiB/s wr, 774 op/s 2024-05-17T22:44:06.350 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:06 smithi096 ceph-mon[36065]: pgmap v584: 129 pgs: 129 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 8.2 MiB/s wr, 774 op/s 2024-05-17T22:44:06.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 285.0 s, 1435.6 tps, lat 68.357 ms stddev 88.923 2024-05-17T22:44:06.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:06 smithi151 ceph-mon[36101]: pgmap v584: 129 pgs: 129 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 8.2 MiB/s wr, 774 op/s 2024-05-17T22:44:08.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:08 smithi012 ceph-mon[31632]: pgmap v585: 129 pgs: 129 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 7.4 MiB/s wr, 700 op/s 2024-05-17T22:44:08.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:08 smithi151 ceph-mon[36101]: pgmap v585: 129 pgs: 129 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 7.4 MiB/s wr, 700 op/s 2024-05-17T22:44:08.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:08 smithi096 ceph-mon[36065]: pgmap v585: 129 pgs: 129 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 7.4 MiB/s wr, 700 op/s 2024-05-17T22:44:10.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:10 smithi012 ceph-mon[31632]: pgmap v586: 129 pgs: 129 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 8.9 MiB/s wr, 854 op/s 2024-05-17T22:44:10.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:10 smithi151 ceph-mon[36101]: pgmap v586: 129 pgs: 129 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 8.9 MiB/s wr, 854 op/s 2024-05-17T22:44:10.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:10 smithi096 ceph-mon[36065]: pgmap v586: 129 pgs: 129 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 8.9 MiB/s wr, 854 op/s 2024-05-17T22:44:11.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 290.0 s, 964.0 tps, lat 102.441 ms stddev 124.732 2024-05-17T22:44:11.726 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-17T22:44:12.240 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:44:12.373 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:12 smithi012 ceph-mon[31632]: pgmap v587: 129 pgs: 129 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.9 MiB/s wr, 612 op/s 2024-05-17T22:44:12.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:12 smithi151 ceph-mon[36101]: pgmap v587: 129 pgs: 129 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.9 MiB/s wr, 612 op/s 2024-05-17T22:44:12.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:12 smithi096 ceph-mon[36065]: pgmap v587: 129 pgs: 129 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.9 MiB/s wr, 612 op/s 2024-05-17T22:44:14.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:14 smithi151 ceph-mon[36101]: pgmap v588: 129 pgs: 129 active+clean; 1.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.9 MiB/s wr, 556 op/s 2024-05-17T22:44:14.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:14 smithi096 ceph-mon[36065]: pgmap v588: 129 pgs: 129 active+clean; 1.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.9 MiB/s wr, 556 op/s 2024-05-17T22:44:14.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:14 smithi012 ceph-mon[31632]: pgmap v588: 129 pgs: 129 active+clean; 1.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.9 MiB/s wr, 556 op/s 2024-05-17T22:44:16.359 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:16 smithi012 ceph-mon[31632]: pgmap v589: 129 pgs: 129 active+clean; 1.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.0 MiB/s wr, 595 op/s 2024-05-17T22:44:16.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 295.0 s, 625.6 tps, lat 153.438 ms stddev 189.580 2024-05-17T22:44:16.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:16 smithi151 ceph-mon[36101]: pgmap v589: 129 pgs: 129 active+clean; 1.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.0 MiB/s wr, 595 op/s 2024-05-17T22:44:16.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:16 smithi096 ceph-mon[36065]: pgmap v589: 129 pgs: 129 active+clean; 1.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.0 MiB/s wr, 595 op/s 2024-05-17T22:44:17.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.956+0000 7f2fbf1c2700 1 -- 172.21.15.12:0/22869599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fb8101ed0 msgr2=0x7f2fb81022e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:17.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.956+0000 7f2fbd1be700 1 -- 172.21.15.12:0/22869599 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb001cda0 con 0x7f2fb8101ed0 2024-05-17T22:44:17.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.956+0000 7f2fbf1c2700 1 --2- 172.21.15.12:0/22869599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fb8101ed0 0x7f2fb81022e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2fb001c0d0 tx=0x7f2fb001c3e0 comp rx=0 tx=0).stop 2024-05-17T22:44:17.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.957+0000 7f2fbf1c2700 1 -- 172.21.15.12:0/22869599 shutdown_connections 2024-05-17T22:44:17.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.957+0000 7f2fbf1c2700 1 --2- 172.21.15.12:0/22869599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fb8102820 0x7f2fb8107660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:17.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.957+0000 7f2fbf1c2700 1 --2- 172.21.15.12:0/22869599 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2fb8108990 0x7f2fb8108da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:17.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.957+0000 7f2fbf1c2700 1 --2- 172.21.15.12:0/22869599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fb8101ed0 0x7f2fb81022e0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:17.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.957+0000 7f2fbf1c2700 1 -- 172.21.15.12:0/22869599 >> 172.21.15.12:0/22869599 conn(0x7f2fb80fde20 msgr2=0x7f2fb8100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:44:17.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.960+0000 7f2fbf1c2700 1 -- 172.21.15.12:0/22869599 shutdown_connections 2024-05-17T22:44:17.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.960+0000 7f2fbf1c2700 1 -- 172.21.15.12:0/22869599 wait complete. 2024-05-17T22:44:17.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.961+0000 7f2fbf1c2700 1 Processor -- start 2024-05-17T22:44:17.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.961+0000 7f2fbf1c2700 1 -- start start 2024-05-17T22:44:17.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.961+0000 7f2fbf1c2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fb8101ed0 0x7f2fb81983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:17.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.961+0000 7f2fbf1c2700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2fb8102820 0x7f2fb8198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:17.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.961+0000 7f2fbf1c2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fb8108990 0x7f2fb819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:17.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.961+0000 7f2fbf1c2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb810ba60 con 0x7f2fb8108990 2024-05-17T22:44:17.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.961+0000 7f2fbf1c2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb810b8e0 con 0x7f2fb8101ed0 2024-05-17T22:44:17.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.961+0000 7f2fbf1c2700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb810bbe0 con 0x7f2fb8102820 2024-05-17T22:44:17.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.963+0000 7f2fbe9c1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fb8108990 0x7f2fb819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:17.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.963+0000 7f2fbe9c1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fb8108990 0x7f2fb819cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51290/0 (socket says 172.21.15.12:51290) 2024-05-17T22:44:17.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.963+0000 7f2fbe9c1700 1 -- 172.21.15.12:0/4220365629 learned_addr learned my addr 172.21.15.12:0/4220365629 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:44:17.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.963+0000 7f2fbd9bf700 1 --2- 172.21.15.12:0/4220365629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2fb8102820 0x7f2fb8198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:17.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.964+0000 7f2fbe9c1700 1 -- 172.21.15.12:0/4220365629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2fb8102820 msgr2=0x7f2fb8198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:17.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.964+0000 7f2fbe9c1700 1 --2- 172.21.15.12:0/4220365629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2fb8102820 0x7f2fb8198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:17.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.964+0000 7f2fbe9c1700 1 -- 172.21.15.12:0/4220365629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fb8101ed0 msgr2=0x7f2fb81983d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-17T22:44:17.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.964+0000 7f2fbe9c1700 1 --2- 172.21.15.12:0/4220365629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fb8101ed0 0x7f2fb81983d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:17.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.964+0000 7f2fbe9c1700 1 -- 172.21.15.12:0/4220365629 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb819d2a0 con 0x7f2fb8108990 2024-05-17T22:44:17.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.964+0000 7f2fbe9c1700 1 --2- 172.21.15.12:0/4220365629 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fb8108990 0x7f2fb819cc00 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2fb400c970 tx=0x7f2fb400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:44:17.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.966+0000 7f2faf7fe700 1 -- 172.21.15.12:0/4220365629 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb400ce60 con 0x7f2fb8108990 2024-05-17T22:44:17.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.966+0000 7f2faf7fe700 1 -- 172.21.15.12:0/4220365629 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fb4004830 con 0x7f2fb8108990 2024-05-17T22:44:17.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.967+0000 7f2faf7fe700 1 -- 172.21.15.12:0/4220365629 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4005550 con 0x7f2fb8108990 2024-05-17T22:44:17.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.969+0000 7f2fbf1c2700 1 -- 172.21.15.12:0/4220365629 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fb819d590 con 0x7f2fb8108990 2024-05-17T22:44:17.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.969+0000 7f2fbf1c2700 1 -- 172.21.15.12:0/4220365629 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fb81c6ec0 con 0x7f2fb8108990 2024-05-17T22:44:17.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.978+0000 7f2fcedffb80 1 Processor -- start 2024-05-17T22:44:17.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.979+0000 7f2faf7fe700 1 -- 172.21.15.12:0/4220365629 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2fb4007b00 con 0x7f2fb8108990 2024-05-17T22:44:17.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.979+0000 7f2fcedffb80 1 -- start start 2024-05-17T22:44:17.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.980+0000 7f2fcedffb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c68e3579f0 0x55c68e4db770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:17.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.980+0000 7f2fcedffb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c68e4dbcb0 0x55c68e4d5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:17.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.980+0000 7f2fcedffb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c68e4d6170 0x55c68e4d6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:17.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.980+0000 7f2fcedffb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c68e404290 con 0x55c68e4dbcb0 2024-05-17T22:44:17.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.980+0000 7f2fcedffb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c68e3f3fd0 con 0x55c68e4d6170 2024-05-17T22:44:17.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.980+0000 7f2fcedffb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c68e3f7e70 con 0x55c68e3579f0 2024-05-17T22:44:17.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.980+0000 7f2fbe1c0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c68e4dbcb0 0x55c68e4d5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:17.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.980+0000 7f2fbe1c0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c68e4dbcb0 0x55c68e4d5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51300/0 (socket says 172.21.15.12:51300) 2024-05-17T22:44:17.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.980+0000 7f2fbe1c0700 1 -- 172.21.15.12:0/2386718900 learned_addr learned my addr 172.21.15.12:0/2386718900 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:44:17.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.980+0000 7f2fbd9bf700 1 --2- 172.21.15.12:0/2386718900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c68e4d6170 0x55c68e4d6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:17.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.980+0000 7f2fbd9bf700 1 --2- 172.21.15.12:0/2386718900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c68e3579f0 0x55c68e4db770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:17.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.981+0000 7f2fbe1c0700 1 -- 172.21.15.12:0/2386718900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c68e3579f0 msgr2=0x55c68e4db770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:44:17.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.981+0000 7f2fbe1c0700 1 --2- 172.21.15.12:0/2386718900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c68e3579f0 0x55c68e4db770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:17.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.981+0000 7f2fbe1c0700 1 -- 172.21.15.12:0/2386718900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c68e4d6170 msgr2=0x55c68e4d6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:44:17.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.981+0000 7f2fbe1c0700 1 --2- 172.21.15.12:0/2386718900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c68e4d6170 0x55c68e4d6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:17.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.981+0000 7f2fbe1c0700 1 -- 172.21.15.12:0/2386718900 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c68e39d590 con 0x55c68e4dbcb0 2024-05-17T22:44:17.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.981+0000 7f2fbe1c0700 1 --2- 172.21.15.12:0/2386718900 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c68e4dbcb0 0x55c68e4d5c30 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2fa800baa0 tx=0x7f2fa800bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ccbac7c8c23f0ad server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:44:17.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.982+0000 7f2facd46700 1 -- 172.21.15.12:0/2386718900 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8016070 con 0x55c68e4dbcb0 2024-05-17T22:44:17.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.982+0000 7f2facd46700 1 -- 172.21.15.12:0/2386718900 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fa800d7d0 con 0x55c68e4dbcb0 2024-05-17T22:44:17.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.982+0000 7f2faf7fe700 1 --2- 172.21.15.12:0/4220365629 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f2fa40b2a80 0x7f2fa40b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:17.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.982+0000 7f2facd46700 1 -- 172.21.15.12:0/2386718900 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa80054c0 con 0x55c68e4dbcb0 2024-05-17T22:44:17.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.982+0000 7f2faf7fe700 1 -- 172.21.15.12:0/4220365629 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f2fb40e2310 con 0x7f2fb8108990 2024-05-17T22:44:17.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.982+0000 7f2fbd9bf700 1 --2- 172.21.15.12:0/4220365629 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f2fa40b2a80 0x7f2fa40b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:17.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.983+0000 7f2fcedffb80 1 -- 172.21.15.12:0/2386718900 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c68e4dbcb0 msgr2=0x55c68e4d5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:17.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.983+0000 7f2fcedffb80 1 --2- 172.21.15.12:0/2386718900 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c68e4dbcb0 0x55c68e4d5c30 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2fa800baa0 tx=0x7f2fa800bdb0 comp rx=0 tx=0).stop 2024-05-17T22:44:17.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.983+0000 7f2fcedffb80 1 -- 172.21.15.12:0/2386718900 shutdown_connections 2024-05-17T22:44:17.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.983+0000 7f2fcedffb80 1 --2- 172.21.15.12:0/2386718900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c68e4d6170 0x55c68e4d6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:17.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.984+0000 7f2fcedffb80 1 --2- 172.21.15.12:0/2386718900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c68e3579f0 0x55c68e4db770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:17.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.984+0000 7f2fcedffb80 1 --2- 172.21.15.12:0/2386718900 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c68e4dbcb0 0x55c68e4d5c30 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:17.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.984+0000 7f2fcedffb80 1 -- 172.21.15.12:0/2386718900 >> 172.21.15.12:0/2386718900 conn(0x55c68e39bf00 msgr2=0x55c68e4d0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:44:17.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.984+0000 7f2fbd9bf700 1 --2- 172.21.15.12:0/4220365629 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f2fa40b2a80 0x7f2fa40b4f90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8102660 tx=0x7f2fb001a9f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:44:17.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.984+0000 7f2fcedffb80 1 -- 172.21.15.12:0/2386718900 shutdown_connections 2024-05-17T22:44:17.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.984+0000 7f2fcedffb80 1 -- 172.21.15.12:0/2386718900 wait complete. 2024-05-17T22:44:17.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.985+0000 7f2fcedffb80 1 -- start start 2024-05-17T22:44:17.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.985+0000 7f2fcedffb80 1 Processor -- start 2024-05-17T22:44:17.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.985+0000 7f2fcedffb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c68e3579f0 0x55c68e4df2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:17.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.985+0000 7f2fcedffb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c68e4d6170 0x55c68e4df830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:17.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.985+0000 7f2fcedffb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c68e4dbcb0 0x55c68e4dfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:17.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.985+0000 7f2fcedffb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c68e2fb150 con 0x55c68e3579f0 2024-05-17T22:44:17.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.985+0000 7f2fcedffb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c68e394e80 con 0x55c68e4dbcb0 2024-05-17T22:44:17.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.985+0000 7f2fcedffb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c68e4e25a0 con 0x55c68e4d6170 2024-05-17T22:44:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.985+0000 7f2fcedffb80 10 client.0 _ll_register_callbacks cb 0x55c68e38ef10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:44:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.986+0000 7f2fbd9bf700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c68e4d6170 0x55c68e4df830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.986+0000 7f2fbd9bf700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c68e4d6170 0x55c68e4df830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:43740/0 (socket says 172.21.15.12:43740) 2024-05-17T22:44:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.986+0000 7f2fbd9bf700 1 -- 172.21.15.12:0/802450324 learned_addr learned my addr 172.21.15.12:0/802450324 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:44:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.986+0000 7f2fbe9c1700 1 --2- 172.21.15.12:0/802450324 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c68e4dbcb0 0x55c68e4dfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.986+0000 7f2fbe1c0700 1 --2- 172.21.15.12:0/802450324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c68e3579f0 0x55c68e4df2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.986+0000 7f2fbd9bf700 1 -- 172.21.15.12:0/802450324 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c68e4dbcb0 msgr2=0x55c68e4dfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:44:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.986+0000 7f2fbd9bf700 1 --2- 172.21.15.12:0/802450324 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c68e4dbcb0 0x55c68e4dfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.986+0000 7f2fbd9bf700 1 -- 172.21.15.12:0/802450324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c68e3579f0 msgr2=0x55c68e4df2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:44:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.986+0000 7f2fbd9bf700 1 --2- 172.21.15.12:0/802450324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c68e3579f0 0x55c68e4df2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.986+0000 7f2fbd9bf700 1 -- 172.21.15.12:0/802450324 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c68e344480 con 0x55c68e4d6170 2024-05-17T22:44:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.987+0000 7f2fbd9bf700 1 --2- 172.21.15.12:0/802450324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c68e4d6170 0x55c68e4df830 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0018af0 tx=0x7f2fb001c040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af1084848e98dc90 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:44:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.987+0000 7f2fa1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:44:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.987+0000 7f2fa1ffb700 1 -- 172.21.15.12:0/802450324 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb0030030 con 0x55c68e4d6170 2024-05-17T22:44:17.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.987+0000 7f2fa1ffb700 1 -- 172.21.15.12:0/802450324 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f2fb0018260 con 0x55c68e4d6170 2024-05-17T22:44:17.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.987+0000 7f2fcedffb80 1 -- 172.21.15.12:0/802450324 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fa800d7d0 con 0x55c68e4d6170 2024-05-17T22:44:17.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.988+0000 7f2fa1ffb700 1 -- 172.21.15.12:0/802450324 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2fb003d050 con 0x55c68e4d6170 2024-05-17T22:44:17.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.989+0000 7f2fcedffb80 10 client.25372 fetch_fsmap learned FSMap version 33 2024-05-17T22:44:17.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.989+0000 7f2fcedffb80 1 -- 172.21.15.12:0/802450324 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c68e4ecf00 con 0x55c68e4d6170 2024-05-17T22:44:17.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.989+0000 7f2fa1ffb700 1 -- 172.21.15.12:0/802450324 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fa800d7d0 con 0x55c68e4d6170 2024-05-17T22:44:17.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.989+0000 7f2fa1ffb700 1 -- 172.21.15.12:0/802450324 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f2fb0022070 con 0x55c68e4d6170 2024-05-17T22:44:17.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.989+0000 7f2fcedffb80 10 client.25372 fetch_fsmap finished waiting for FSMap version 33 2024-05-17T22:44:17.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.989+0000 7f2fcedffb80 10 client.25372 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:44:17.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.989+0000 7f2fcedffb80 20 client.25372 populate_metadata read hostname 'smithi012' 2024-05-17T22:44:17.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.989+0000 7f2fcedffb80 1 --2- 172.21.15.12:0/802450324 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55c68e4ef940 0x55c68e4f1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:17.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.989+0000 7f2fcedffb80 4 client.25372 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:44:17.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.989+0000 7f2fcedffb80 1 -- 172.21.15.12:0/802450324 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c68e4061e0 con 0x55c68e4ef940 2024-05-17T22:44:17.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.990+0000 7f2fbe1c0700 1 --2- 172.21.15.12:0/802450324 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55c68e4ef940 0x55c68e4f1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:17.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.990+0000 7f2fa1ffb700 10 client.25372 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:44:17.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.990+0000 7f2fbe1c0700 1 --2- 172.21.15.12:0/802450324 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55c68e4ef940 0x55c68e4f1e10 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a64565a393453c server_cookie=e27a2dc4fcc0c464 in_seq=0 out_seq=0 2024-05-17T22:44:17.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.991+0000 7f2fa1ffb700 1 -- 172.21.15.12:0/802450324 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c68e4061e0 con 0x55c68e4ef940 2024-05-17T22:44:17.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.991+0000 7f2fa1ffb700 10 client.25372 handle_command_reply: tid=0 2024-05-17T22:44:17.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.991+0000 7f2fcedffb80 1 client.25372 shutdown 2024-05-17T22:44:17.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.992+0000 7f2fcedffb80 1 -- 172.21.15.12:0/802450324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c68e4d6170 msgr2=0x55c68e4df830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:17.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.992+0000 7f2fcedffb80 1 --2- 172.21.15.12:0/802450324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c68e4d6170 0x55c68e4df830 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0018af0 tx=0x7f2fb001c040 comp rx=0 tx=0).stop 2024-05-17T22:44:17.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.992+0000 7f2fcedffb80 1 -- 172.21.15.12:0/802450324 shutdown_connections 2024-05-17T22:44:17.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.992+0000 7f2fcedffb80 1 --2- 172.21.15.12:0/802450324 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c68e4dbcb0 0x55c68e4dfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:17.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.992+0000 7f2fcedffb80 1 --2- 172.21.15.12:0/802450324 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55c68e4ef940 0x55c68e4f1e10 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:17.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.992+0000 7f2fcedffb80 1 --2- 172.21.15.12:0/802450324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c68e3579f0 0x55c68e4df2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:17.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.992+0000 7f2fcedffb80 1 --2- 172.21.15.12:0/802450324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c68e4d6170 0x55c68e4df830 secure :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0018af0 tx=0x7f2fb001c040 comp rx=0 tx=0).stop 2024-05-17T22:44:17.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.992+0000 7f2fcedffb80 1 -- 172.21.15.12:0/802450324 >> 172.21.15.12:0/802450324 conn(0x55c68e39bf00 msgr2=0x55c68e4d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:44:17.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.992+0000 7f2fa1ffb700 0 client.25372 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:44:17.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.992+0000 7f2fcedffb80 1 -- 172.21.15.12:0/802450324 shutdown_connections 2024-05-17T22:44:17.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.992+0000 7f2fcedffb80 1 -- 172.21.15.12:0/802450324 wait complete. 2024-05-17T22:44:17.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:16.993+0000 7f2fcedffb80 20 client.25372 trim_cache size 0 max 16384 2024-05-17T22:44:17.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.026+0000 7f2fcedffb80 1 Processor -- start 2024-05-17T22:44:17.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.026+0000 7f2fcedffb80 1 -- start start 2024-05-17T22:44:17.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.027+0000 7f2fcedffb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c68e3579f0 0x55c68e4e12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:17.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.027+0000 7f2fcedffb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c68e4dbcb0 0x55c68e4d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:17.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.027+0000 7f2fcedffb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c68e4e1830 0x55c68e4d0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:17.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.027+0000 7f2fbe1c0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c68e3579f0 0x55c68e4e12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:17.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.027+0000 7f2fcedffb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb81d22f0 con 0x55c68e3579f0 2024-05-17T22:44:17.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.027+0000 7f2fcedffb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb81cfb20 con 0x55c68e4dbcb0 2024-05-17T22:44:17.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.027+0000 7f2fcedffb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb81cd970 con 0x55c68e4e1830 2024-05-17T22:44:17.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.027+0000 7f2fbe1c0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c68e3579f0 0x55c68e4e12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51304/0 (socket says 172.21.15.12:51304) 2024-05-17T22:44:17.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.027+0000 7f2fbe1c0700 1 -- 172.21.15.12:0/1693633634 learned_addr learned my addr 172.21.15.12:0/1693633634 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:44:17.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.027+0000 7f2fbd9bf700 1 --2- 172.21.15.12:0/1693633634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c68e4dbcb0 0x55c68e4d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:17.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.027+0000 7f2fbe9c1700 1 --2- 172.21.15.12:0/1693633634 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c68e4e1830 0x55c68e4d0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:17.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.028+0000 7f2fbe1c0700 1 -- 172.21.15.12:0/1693633634 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c68e4e1830 msgr2=0x55c68e4d0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:44:17.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.028+0000 7f2fbe1c0700 1 --2- 172.21.15.12:0/1693633634 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c68e4e1830 0x55c68e4d0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:17.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.028+0000 7f2fbe1c0700 1 -- 172.21.15.12:0/1693633634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c68e4dbcb0 msgr2=0x55c68e4d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:44:17.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.028+0000 7f2fbe1c0700 1 --2- 172.21.15.12:0/1693633634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c68e4dbcb0 0x55c68e4d0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:17.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.028+0000 7f2fbe1c0700 1 -- 172.21.15.12:0/1693633634 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c68e4d1660 con 0x55c68e3579f0 2024-05-17T22:44:17.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.028+0000 7f2fbe1c0700 1 --2- 172.21.15.12:0/1693633634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c68e3579f0 0x55c68e4e12f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f2fa800dfd0 tx=0x7f2fa800da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5221af92fa965456 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:44:17.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.029+0000 7f2fa3fff700 1 -- 172.21.15.12:0/1693633634 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa802a070 con 0x55c68e3579f0 2024-05-17T22:44:17.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.029+0000 7f2fa3fff700 1 -- 172.21.15.12:0/1693633634 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fa800de10 con 0x55c68e3579f0 2024-05-17T22:44:17.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.029+0000 7f2fa3fff700 1 -- 172.21.15.12:0/1693633634 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8016040 con 0x55c68e3579f0 2024-05-17T22:44:17.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.030+0000 7f2fcedffb80 1 -- 172.21.15.12:0/1693633634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c68e3579f0 msgr2=0x55c68e4e12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:17.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.030+0000 7f2fcedffb80 1 --2- 172.21.15.12:0/1693633634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c68e3579f0 0x55c68e4e12f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f2fa800dfd0 tx=0x7f2fa800da80 comp rx=0 tx=0).stop 2024-05-17T22:44:17.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.030+0000 7f2fcedffb80 1 -- 172.21.15.12:0/1693633634 shutdown_connections 2024-05-17T22:44:17.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.030+0000 7f2fcedffb80 1 --2- 172.21.15.12:0/1693633634 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c68e4e1830 0x55c68e4d0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:17.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.030+0000 7f2fcedffb80 1 --2- 172.21.15.12:0/1693633634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c68e3579f0 0x55c68e4e12f0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:17.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.030+0000 7f2fcedffb80 1 --2- 172.21.15.12:0/1693633634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c68e4dbcb0 0x55c68e4d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:17.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.030+0000 7f2fcedffb80 1 -- 172.21.15.12:0/1693633634 >> 172.21.15.12:0/1693633634 conn(0x55c68e39bf00 msgr2=0x55c68e4e0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:44:17.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.031+0000 7f2fcedffb80 1 -- 172.21.15.12:0/1693633634 shutdown_connections 2024-05-17T22:44:17.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.031+0000 7f2fcedffb80 1 -- 172.21.15.12:0/1693633634 wait complete. 2024-05-17T22:44:17.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.031+0000 7f2fcedffb80 1 -- start start 2024-05-17T22:44:17.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.032+0000 7f2fcedffb80 1 Processor -- start 2024-05-17T22:44:17.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.032+0000 7f2fcedffb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c68e3579f0 0x55c68e4e2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:17.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.033+0000 7f2fcedffb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c68e4dbcb0 0x55c68e4e2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:17.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.033+0000 7f2fcedffb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c68e4e1830 0x55c68e5581b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:17.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.033+0000 7f2fbe1c0700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c68e3579f0 0x55c68e4e2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:17.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.033+0000 7f2fbe1c0700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c68e3579f0 0x55c68e4e2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:43766/0 (socket says 172.21.15.12:43766) 2024-05-17T22:44:17.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.033+0000 7f2fbe1c0700 1 -- 172.21.15.12:0/2502426300 learned_addr learned my addr 172.21.15.12:0/2502426300 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:44:17.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.033+0000 7f2fcedffb80 1 -- 172.21.15.12:0/2502426300 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb81cdfe0 con 0x55c68e4e1830 2024-05-17T22:44:17.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.033+0000 7f2fbd9bf700 1 --2- 172.21.15.12:0/2502426300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c68e4dbcb0 0x55c68e4e2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:17.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.033+0000 7f2fcedffb80 1 -- 172.21.15.12:0/2502426300 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb81cf450 con 0x55c68e4dbcb0 2024-05-17T22:44:17.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.033+0000 7f2fbe9c1700 1 --2- 172.21.15.12:0/2502426300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c68e4e1830 0x55c68e5581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:17.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.033+0000 7f2fcedffb80 1 -- 172.21.15.12:0/2502426300 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb81ce6b0 con 0x55c68e3579f0 2024-05-17T22:44:17.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.034+0000 7f2fbe1c0700 1 -- 172.21.15.12:0/2502426300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c68e4dbcb0 msgr2=0x55c68e4e2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:44:17.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.034+0000 7f2fbe1c0700 1 --2- 172.21.15.12:0/2502426300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c68e4dbcb0 0x55c68e4e2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:17.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.034+0000 7f2fbe1c0700 1 -- 172.21.15.12:0/2502426300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c68e4e1830 msgr2=0x55c68e5581b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:44:17.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.034+0000 7f2fbe1c0700 1 --2- 172.21.15.12:0/2502426300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c68e4e1830 0x55c68e5581b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:17.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.034+0000 7f2fbe1c0700 1 -- 172.21.15.12:0/2502426300 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c68e5586f0 con 0x55c68e3579f0 2024-05-17T22:44:17.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.034+0000 7f2fcedffb80 10 client.0 _ll_register_callbacks cb 0x55c68e38ef10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:44:17.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.034+0000 7f2fbe1c0700 1 --2- 172.21.15.12:0/2502426300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c68e3579f0 0x55c68e4e2640 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8024650 tx=0x7f2fa800dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=954fd2c201ee96c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:44:17.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.034+0000 7f2fa1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:44:17.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.034+0000 7f2fcedffb80 1 -- 172.21.15.12:0/2502426300 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c68e3ff9c0 con 0x55c68e3579f0 2024-05-17T22:44:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.034+0000 7f2fa1ffb700 1 -- 172.21.15.12:0/2502426300 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa802a040 con 0x55c68e3579f0 2024-05-17T22:44:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.034+0000 7f2fa1ffb700 1 -- 172.21.15.12:0/2502426300 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f2fa80246b0 con 0x55c68e3579f0 2024-05-17T22:44:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.034+0000 7f2fa1ffb700 1 -- 172.21.15.12:0/2502426300 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2fa8016040 con 0x55c68e3579f0 2024-05-17T22:44:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.035+0000 7f2fcedffb80 10 client.25378 fetch_fsmap learned FSMap version 33 2024-05-17T22:44:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.035+0000 7f2fcedffb80 1 -- 172.21.15.12:0/2502426300 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({fsmap=33}) -- 0x55c68e5596c0 con 0x55c68e3579f0 2024-05-17T22:44:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.036+0000 7f2fa1ffb700 1 -- 172.21.15.12:0/2502426300 <== mon.1 v2:172.21.15.151:3300/0 4 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f2fa8024930 con 0x55c68e3579f0 2024-05-17T22:44:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.036+0000 7f2fcedffb80 10 client.25378 fetch_fsmap finished waiting for FSMap version 33 2024-05-17T22:44:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.036+0000 7f2fcedffb80 10 client.25378 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:44:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.036+0000 7f2fcedffb80 20 client.25378 populate_metadata read hostname 'smithi012' 2024-05-17T22:44:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.036+0000 7f2fcedffb80 1 --2- 172.21.15.12:0/2502426300 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55c68e559ad0 0x55c68e55bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.036+0000 7f2fcedffb80 4 client.25378 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:44:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.036+0000 7f2fcedffb80 1 -- 172.21.15.12:0/2502426300 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c68e2d0360 con 0x55c68e559ad0 2024-05-17T22:44:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.037+0000 7f2fbd9bf700 1 --2- 172.21.15.12:0/2502426300 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55c68e559ad0 0x55c68e55bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.039+0000 7f2fbd9bf700 1 --2- 172.21.15.12:0/2502426300 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55c68e559ad0 0x55c68e55bef0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7ddf74cdbb8ff1d server_cookie=e736fd79f99e7292 in_seq=0 out_seq=0 2024-05-17T22:44:17.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.039+0000 7f2fa1ffb700 10 client.25378 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:44:17.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.039+0000 7f2fa1ffb700 1 -- 172.21.15.12:0/2502426300 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c68e2d0360 con 0x55c68e559ad0 2024-05-17T22:44:17.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.039+0000 7f2fa1ffb700 10 client.25378 handle_command_reply: tid=0 2024-05-17T22:44:17.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.040+0000 7f2fcedffb80 1 client.25378 shutdown 2024-05-17T22:44:17.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.040+0000 7f2fcedffb80 1 -- 172.21.15.12:0/2502426300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c68e3579f0 msgr2=0x55c68e4e2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:17.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.040+0000 7f2fcedffb80 1 --2- 172.21.15.12:0/2502426300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c68e3579f0 0x55c68e4e2640 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8024650 tx=0x7f2fa800dc10 comp rx=0 tx=0).stop 2024-05-17T22:44:17.714 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:44:17.714 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-17T22:44:17.714 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-17T22:44:17.715 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:44:17.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.040+0000 7f2fcedffb80 1 -- 172.21.15.12:0/2502426300 shutdown_connections 2024-05-17T22:44:17.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.040+0000 7f2fcedffb80 1 --2- 172.21.15.12:0/2502426300 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55c68e559ad0 0x55c68e55bef0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:17.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.040+0000 7f2fcedffb80 1 --2- 172.21.15.12:0/2502426300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c68e4e1830 0x55c68e5581b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:17.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.040+0000 7f2fcedffb80 1 --2- 172.21.15.12:0/2502426300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c68e3579f0 0x55c68e4e2640 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:17.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.040+0000 7f2fcedffb80 1 --2- 172.21.15.12:0/2502426300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c68e4dbcb0 0x55c68e4e2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:17.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.040+0000 7f2fcedffb80 1 -- 172.21.15.12:0/2502426300 >> 172.21.15.12:0/2502426300 conn(0x55c68e39bf00 msgr2=0x55c68e4dc240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:44:17.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.040+0000 7f2fa1ffb700 0 client.25378 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:44:17.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.040+0000 7f2fcedffb80 1 -- 172.21.15.12:0/2502426300 shutdown_connections 2024-05-17T22:44:17.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.040+0000 7f2fcedffb80 1 -- 172.21.15.12:0/2502426300 wait complete. 2024-05-17T22:44:17.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.041+0000 7f2fcedffb80 20 client.25378 trim_cache size 0 max 16384 2024-05-17T22:44:17.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.041+0000 7f2fbf1c2700 1 -- 172.21.15.12:0/4220365629 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f2fa40b2a80 msgr2=0x7f2fa40b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:17.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.041+0000 7f2fbf1c2700 1 --2- 172.21.15.12:0/4220365629 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f2fa40b2a80 0x7f2fa40b4f90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8102660 tx=0x7f2fb001a9f0 comp rx=0 tx=0).stop 2024-05-17T22:44:17.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.041+0000 7f2fbf1c2700 1 -- 172.21.15.12:0/4220365629 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fb8108990 msgr2=0x7f2fb819cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:17.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.041+0000 7f2fbf1c2700 1 --2- 172.21.15.12:0/4220365629 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fb8108990 0x7f2fb819cc00 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2fb400c970 tx=0x7f2fb400cc80 comp rx=0 tx=0).stop 2024-05-17T22:44:17.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.041+0000 7f2fbf1c2700 1 -- 172.21.15.12:0/4220365629 shutdown_connections 2024-05-17T22:44:17.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.041+0000 7f2fbf1c2700 1 --2- 172.21.15.12:0/4220365629 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f2fa40b2a80 0x7f2fa40b4f90 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:17.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.041+0000 7f2fbf1c2700 1 --2- 172.21.15.12:0/4220365629 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fb8108990 0x7f2fb819cc00 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:17.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.042+0000 7f2fbf1c2700 1 --2- 172.21.15.12:0/4220365629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fb8101ed0 0x7f2fb81983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:17.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.042+0000 7f2fbf1c2700 1 --2- 172.21.15.12:0/4220365629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2fb8102820 0x7f2fb8198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:17.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.042+0000 7f2fbf1c2700 1 -- 172.21.15.12:0/4220365629 >> 172.21.15.12:0/4220365629 conn(0x7f2fb80fde20 msgr2=0x7f2fb8105630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:44:17.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.042+0000 7f2fbf1c2700 1 -- 172.21.15.12:0/4220365629 shutdown_connections 2024-05-17T22:44:17.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:17.042+0000 7f2fbf1c2700 1 -- 172.21.15.12:0/4220365629 wait complete. 2024-05-17T22:44:18.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:18 smithi151 ceph-mon[36101]: pgmap v590: 129 pgs: 129 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.5 MiB/s wr, 518 op/s 2024-05-17T22:44:18.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:18 smithi096 ceph-mon[36065]: pgmap v590: 129 pgs: 129 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.5 MiB/s wr, 518 op/s 2024-05-17T22:44:18.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:18 smithi012 ceph-mon[31632]: pgmap v590: 129 pgs: 129 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.5 MiB/s wr, 518 op/s 2024-05-17T22:44:20.227 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-17T22:44:20.228 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T22:44:20.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:20 smithi151 ceph-mon[36101]: pgmap v591: 129 pgs: 129 active+clean; 1.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.3 MiB/s wr, 650 op/s 2024-05-17T22:44:20.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:20 smithi096 ceph-mon[36065]: pgmap v591: 129 pgs: 129 active+clean; 1.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.3 MiB/s wr, 650 op/s 2024-05-17T22:44:20.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:20 smithi012 ceph-mon[31632]: pgmap v591: 129 pgs: 129 active+clean; 1.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.3 MiB/s wr, 650 op/s 2024-05-17T22:44:20.721 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:44:21.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 300.0 s, 521.0 tps, lat 201.861 ms stddev 307.534 2024-05-17T22:44:22.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:22 smithi012 ceph-mon[31632]: pgmap v592: 129 pgs: 129 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.3 MiB/s wr, 503 op/s 2024-05-17T22:44:22.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:22 smithi151 ceph-mon[36101]: pgmap v592: 129 pgs: 129 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.3 MiB/s wr, 503 op/s 2024-05-17T22:44:22.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:22 smithi096 ceph-mon[36065]: pgmap v592: 129 pgs: 129 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.3 MiB/s wr, 503 op/s 2024-05-17T22:44:22.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:22.883+0000 7f32b0b25700 1 -- 172.21.15.12:0/697047327 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32ac10b210 msgr2=0x7f32ac118830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:22.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:22.883+0000 7f32b0b25700 1 --2- 172.21.15.12:0/697047327 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32ac10b210 0x7f32ac118830 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f329c0098f0 tx=0x7f329c009c00 comp rx=0 tx=0).stop 2024-05-17T22:44:22.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:22.883+0000 7f32b0b25700 1 -- 172.21.15.12:0/697047327 shutdown_connections 2024-05-17T22:44:22.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:22.883+0000 7f32b0b25700 1 --2- 172.21.15.12:0/697047327 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32ac118d70 0x7f32ac11b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:22.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:22.883+0000 7f32b0b25700 1 --2- 172.21.15.12:0/697047327 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32ac10a830 0x7f32ac10ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:22.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:22.883+0000 7f32b0b25700 1 --2- 172.21.15.12:0/697047327 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32ac10b210 0x7f32ac118830 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:22.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:22.883+0000 7f32b0b25700 1 -- 172.21.15.12:0/697047327 >> 172.21.15.12:0/697047327 conn(0x7f32ac06f420 msgr2=0x7f32ac06f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:44:22.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:22.883+0000 7f32b0b25700 1 -- 172.21.15.12:0/697047327 shutdown_connections 2024-05-17T22:44:22.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:22.883+0000 7f32b0b25700 1 -- 172.21.15.12:0/697047327 wait complete. 2024-05-17T22:44:22.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:22.884+0000 7f32b0b25700 1 Processor -- start 2024-05-17T22:44:22.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:22.884+0000 7f32b0b25700 1 -- start start 2024-05-17T22:44:22.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:22.884+0000 7f32b0b25700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32ac10a830 0x7f32ac10e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:22.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:22.884+0000 7f32b0b25700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32ac10b210 0x7f32ac10e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:22.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:22.884+0000 7f32b0b25700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32ac118d70 0x7f32ac10ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:22.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:22.884+0000 7f32b0b25700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac11e7c0 con 0x7f32ac10b210 2024-05-17T22:44:22.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:22.884+0000 7f32b0b25700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac11e640 con 0x7f32ac118d70 2024-05-17T22:44:22.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:22.884+0000 7f32b0b25700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac11e940 con 0x7f32ac10a830 2024-05-17T22:44:22.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:22.884+0000 7f32aad9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32ac118d70 0x7f32ac10ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:22.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:22.884+0000 7f32aad9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32ac118d70 0x7f32ac10ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:57494/0 (socket says 172.21.15.12:57494) 2024-05-17T22:44:22.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:22.884+0000 7f32aad9d700 1 -- 172.21.15.12:0/1222755180 learned_addr learned my addr 172.21.15.12:0/1222755180 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:44:22.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:22.884+0000 7f32aa59c700 1 --2- 172.21.15.12:0/1222755180 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32ac10a830 0x7f32ac10e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:22.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:22.885+0000 7f32aad9d700 1 -- 172.21.15.12:0/1222755180 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32ac10a830 msgr2=0x7f32ac10e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:22.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:22.885+0000 7f32aad9d700 1 --2- 172.21.15.12:0/1222755180 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32ac10a830 0x7f32ac10e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:22.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:22.885+0000 7f32aad9d700 1 -- 172.21.15.12:0/1222755180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32ac10b210 msgr2=0x7f32ac10e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:22.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:22.885+0000 7f32aad9d700 1 --2- 172.21.15.12:0/1222755180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32ac10b210 0x7f32ac10e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:22.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:22.885+0000 7f32aad9d700 1 -- 172.21.15.12:0/1222755180 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32ac1b5700 con 0x7f32ac118d70 2024-05-17T22:44:22.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:22.885+0000 7f32aad9d700 1 --2- 172.21.15.12:0/1222755180 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32ac118d70 0x7f32ac10ef30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f32a400e3c0 tx=0x7f32a400e6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:44:22.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:22.885+0000 7f32a37fe700 1 -- 172.21.15.12:0/1222755180 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a40092d0 con 0x7f32ac118d70 2024-05-17T22:44:22.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:22.885+0000 7f32a37fe700 1 -- 172.21.15.12:0/1222755180 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32a400ee70 con 0x7f32ac118d70 2024-05-17T22:44:22.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:22.885+0000 7f32b0b25700 1 -- 172.21.15.12:0/1222755180 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32ac1b5990 con 0x7f32ac118d70 2024-05-17T22:44:22.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:22.885+0000 7f32a37fe700 1 -- 172.21.15.12:0/1222755180 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a4004030 con 0x7f32ac118d70 2024-05-17T22:44:22.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:22.885+0000 7f32b0b25700 1 -- 172.21.15.12:0/1222755180 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32ac1b5e80 con 0x7f32ac118d70 2024-05-17T22:44:22.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:22.887+0000 7f32b0b25700 1 -- 172.21.15.12:0/1222755180 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32ac113f70 con 0x7f32ac118d70 2024-05-17T22:44:22.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:22.889+0000 7f32a37fe700 1 -- 172.21.15.12:0/1222755180 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f32a4012050 con 0x7f32ac118d70 2024-05-17T22:44:22.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:22.890+0000 7f32a37fe700 1 --2- 172.21.15.12:0/1222755180 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f32980b2880 0x7f32980b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:22.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:22.890+0000 7f32a37fe700 1 -- 172.21.15.12:0/1222755180 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f32a40e2280 con 0x7f32ac118d70 2024-05-17T22:44:22.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:22.890+0000 7f32aa59c700 1 --2- 172.21.15.12:0/1222755180 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f32980b2880 0x7f32980b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:22.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:22.890+0000 7f32aa59c700 1 --2- 172.21.15.12:0/1222755180 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f32980b2880 0x7f32980b4d90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3294009c60 tx=0x7f3294009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:44:22.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:22.895+0000 7f32a37fe700 1 -- 172.21.15.12:0/1222755180 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f32a4022020 con 0x7f32ac118d70 2024-05-17T22:44:23.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:23.345+0000 7f32b0b25700 1 -- 172.21.15.12:0/1222755180 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32ac02ea80 con 0x7f32ac118d70 2024-05-17T22:44:23.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:23.345+0000 7f32a37fe700 1 -- 172.21.15.12:0/1222755180 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11873 (secure 0 0 0) 0x7f32a4010040 con 0x7f32ac118d70 2024-05-17T22:44:23.348 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:44:23.348 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1407219319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1407219319},{"type":"v1","addr":"172.21.15.12:6835","nonce":1407219319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/630977886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":630977886},{"type":"v1","addr":"172.21.15.12:6837","nonce":630977886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3731914742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3731914742},{"type":"v1","addr":"172.21.15.12:6839","nonce":3731914742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/3642142501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3642142501},{"type":"v1","addr":"172.21.15.12:6841","nonce":3642142501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3554053059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3554053059},{"type":"v1","addr":"172.21.15.151:6833","nonce":3554053059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/4228325569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":4228325569},{"type":"v1","addr":"172.21.15.96:6833","nonce":4228325569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T22:34:42.642111+0000","modified":"2024-05-17T22:42:34.928791+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24461,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/542323754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":542323754},{"type":"v1","addr":"172.21.15.151:6835","nonce":542323754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1758442232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1758442232},{"type":"v1","addr":"172.21.15.96:6835","nonce":1758442232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3403324038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3403324038},{"type":"v1","addr":"172.21.15.151:6837","nonce":3403324038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.151:6839/2338353849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2338353849},{"type":"v1","addr":"172.21.15.151:6839","nonce":2338353849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.96:6837/4222574338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":4222574338},{"type":"v1","addr":"172.21.15.96:6837","nonce":4222574338}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/2986295519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2986295519},{"type":"v1","addr":"172.21.15.96:6839","nonce":2986295519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24457,24461,24482]},"id":1}]} 2024-05-17T22:44:23.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:23.348+0000 7f32b0b25700 1 -- 172.21.15.12:0/1222755180 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f32980b2880 msgr2=0x7f32980b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:23.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:23.348+0000 7f32b0b25700 1 --2- 172.21.15.12:0/1222755180 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f32980b2880 0x7f32980b4d90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3294009c60 tx=0x7f3294009260 comp rx=0 tx=0).stop 2024-05-17T22:44:23.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:23.348+0000 7f32b0b25700 1 -- 172.21.15.12:0/1222755180 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32ac118d70 msgr2=0x7f32ac10ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:23.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:23.348+0000 7f32b0b25700 1 --2- 172.21.15.12:0/1222755180 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32ac118d70 0x7f32ac10ef30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f32a400e3c0 tx=0x7f32a400e6d0 comp rx=0 tx=0).stop 2024-05-17T22:44:23.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:23.348+0000 7f32b0b25700 1 -- 172.21.15.12:0/1222755180 shutdown_connections 2024-05-17T22:44:23.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:23.348+0000 7f32b0b25700 1 --2- 172.21.15.12:0/1222755180 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f32980b2880 0x7f32980b4d90 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:23.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:23.348+0000 7f32b0b25700 1 --2- 172.21.15.12:0/1222755180 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32ac118d70 0x7f32ac10ef30 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:23.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:23.348+0000 7f32b0b25700 1 --2- 172.21.15.12:0/1222755180 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32ac10a830 0x7f32ac10e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:23.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:23.348+0000 7f32b0b25700 1 --2- 172.21.15.12:0/1222755180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32ac10b210 0x7f32ac10e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:23.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:23.348+0000 7f32b0b25700 1 -- 172.21.15.12:0/1222755180 >> 172.21.15.12:0/1222755180 conn(0x7f32ac06f420 msgr2=0x7f32ac119320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:44:23.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:23.348+0000 7f32b0b25700 1 -- 172.21.15.12:0/1222755180 shutdown_connections 2024-05-17T22:44:23.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:23.348+0000 7f32b0b25700 1 -- 172.21.15.12:0/1222755180 wait complete. 2024-05-17T22:44:23.352 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 33 2024-05-17T22:44:24.259 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-17T22:44:24.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:24 smithi151 ceph-mon[36101]: pgmap v593: 129 pgs: 129 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.3 MiB/s wr, 532 op/s 2024-05-17T22:44:24.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:24 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/1222755180' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:44:24.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:24 smithi096 ceph-mon[36065]: pgmap v593: 129 pgs: 129 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.3 MiB/s wr, 532 op/s 2024-05-17T22:44:24.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:24 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/1222755180' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:44:24.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:24 smithi012 ceph-mon[31632]: pgmap v593: 129 pgs: 129 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.3 MiB/s wr, 532 op/s 2024-05-17T22:44:24.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:24 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/1222755180' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:44:24.761 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:44:26.180 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:26 smithi012 ceph-mon[31632]: pgmap v594: 129 pgs: 129 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 10 MiB/s wr, 638 op/s 2024-05-17T22:44:26.374 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 305.0 s, 587.1 tps, lat 166.550 ms stddev 186.317 2024-05-17T22:44:26.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:26 smithi151 ceph-mon[36101]: pgmap v594: 129 pgs: 129 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 10 MiB/s wr, 638 op/s 2024-05-17T22:44:26.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:26 smithi096 ceph-mon[36065]: pgmap v594: 129 pgs: 129 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 10 MiB/s wr, 638 op/s 2024-05-17T22:44:27.478 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:44:27.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.669+0000 7f55ca219700 1 -- 172.21.15.12:0/2347221073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55c4102890 msgr2=0x7f55c410bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:27.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.669+0000 7f55ca219700 1 --2- 172.21.15.12:0/2347221073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55c4102890 0x7f55c410bcf0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009990 tx=0x7f55b4009ca0 comp rx=0 tx=0).stop 2024-05-17T22:44:27.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.669+0000 7f55ca219700 1 -- 172.21.15.12:0/2347221073 shutdown_connections 2024-05-17T22:44:27.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.669+0000 7f55ca219700 1 --2- 172.21.15.12:0/2347221073 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55c410c3b0 0x7f55c4112d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:27.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.669+0000 7f55ca219700 1 --2- 172.21.15.12:0/2347221073 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55c4101f40 0x7f55c4102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:27.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.669+0000 7f55ca219700 1 --2- 172.21.15.12:0/2347221073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55c4102890 0x7f55c410bcf0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:27.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.669+0000 7f55ca219700 1 -- 172.21.15.12:0/2347221073 >> 172.21.15.12:0/2347221073 conn(0x7f55c40fdd60 msgr2=0x7f55c4100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:44:27.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.669+0000 7f55ca219700 1 -- 172.21.15.12:0/2347221073 shutdown_connections 2024-05-17T22:44:27.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.669+0000 7f55ca219700 1 -- 172.21.15.12:0/2347221073 wait complete. 2024-05-17T22:44:27.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.670+0000 7f55ca219700 1 Processor -- start 2024-05-17T22:44:27.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.670+0000 7f55ca219700 1 -- start start 2024-05-17T22:44:27.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.670+0000 7f55ca219700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55c4101f40 0x7f55c41a25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:27.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.670+0000 7f55ca219700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55c4102890 0x7f55c41a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:27.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.670+0000 7f55ca219700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55c410c3b0 0x7f55c419c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:27.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.670+0000 7f55ca219700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f55c4115fc0 con 0x7f55c410c3b0 2024-05-17T22:44:27.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.670+0000 7f55ca219700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f55c4115e40 con 0x7f55c4102890 2024-05-17T22:44:27.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.670+0000 7f55ca219700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f55c4116140 con 0x7f55c4101f40 2024-05-17T22:44:27.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.670+0000 7f55c3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55c410c3b0 0x7f55c419c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:27.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.670+0000 7f55c3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55c410c3b0 0x7f55c419c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47836/0 (socket says 172.21.15.12:47836) 2024-05-17T22:44:27.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.670+0000 7f55c3fff700 1 -- 172.21.15.12:0/3995008165 learned_addr learned my addr 172.21.15.12:0/3995008165 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:44:27.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.670+0000 7f55c37fe700 1 --2- 172.21.15.12:0/3995008165 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55c4101f40 0x7f55c41a25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:27.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.670+0000 7f55bbfff700 1 --2- 172.21.15.12:0/3995008165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55c4102890 0x7f55c41a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:27.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.671+0000 7f55c3fff700 1 -- 172.21.15.12:0/3995008165 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55c4101f40 msgr2=0x7f55c41a25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:27.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.671+0000 7f55c3fff700 1 --2- 172.21.15.12:0/3995008165 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55c4101f40 0x7f55c41a25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:27.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.671+0000 7f55c3fff700 1 -- 172.21.15.12:0/3995008165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55c4102890 msgr2=0x7f55c41a2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:27.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.671+0000 7f55c3fff700 1 --2- 172.21.15.12:0/3995008165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55c4102890 0x7f55c41a2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:27.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.671+0000 7f55c3fff700 1 -- 172.21.15.12:0/3995008165 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55c419cf60 con 0x7f55c410c3b0 2024-05-17T22:44:27.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.671+0000 7f55bbfff700 1 --2- 172.21.15.12:0/3995008165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55c4102890 0x7f55c41a2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:44:27.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.671+0000 7f55c3fff700 1 --2- 172.21.15.12:0/3995008165 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55c410c3b0 0x7f55c419c830 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f55bc00ef30 tx=0x7f55bc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:44:27.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.671+0000 7f55c17fa700 1 -- 172.21.15.12:0/3995008165 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc0092e0 con 0x7f55c410c3b0 2024-05-17T22:44:27.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.671+0000 7f55ca219700 1 -- 172.21.15.12:0/3995008165 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55c419d1f0 con 0x7f55c410c3b0 2024-05-17T22:44:27.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.671+0000 7f55c17fa700 1 -- 172.21.15.12:0/3995008165 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55bc004bb0 con 0x7f55c410c3b0 2024-05-17T22:44:27.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.671+0000 7f55ca219700 1 -- 172.21.15.12:0/3995008165 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55c41a9450 con 0x7f55c410c3b0 2024-05-17T22:44:27.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.671+0000 7f55c17fa700 1 -- 172.21.15.12:0/3995008165 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc004020 con 0x7f55c410c3b0 2024-05-17T22:44:27.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.677+0000 7f55c17fa700 1 -- 172.21.15.12:0/3995008165 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f55bc01b030 con 0x7f55c410c3b0 2024-05-17T22:44:27.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.677+0000 7f55d8bf4b80 1 Processor -- start 2024-05-17T22:44:27.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.677+0000 7f55d8bf4b80 1 -- start start 2024-05-17T22:44:27.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.678+0000 7f55d8bf4b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5620fa7369f0 0x5620fa8c1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:27.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.678+0000 7f55d8bf4b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620fa8c21a0 0x5620fa8bd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:27.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.678+0000 7f55d8bf4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620fa8bd980 0x5620fa8bdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:27.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.678+0000 7f55d8bf4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5620fa7e3290 con 0x5620fa8bd980 2024-05-17T22:44:27.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.678+0000 7f55d8bf4b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5620fa7d2fd0 con 0x5620fa8c21a0 2024-05-17T22:44:27.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.678+0000 7f55d8bf4b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5620fa7d6e70 con 0x5620fa7369f0 2024-05-17T22:44:27.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.678+0000 7f55c37fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620fa8c21a0 0x5620fa8bd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:27.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.678+0000 7f55c37fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620fa8c21a0 0x5620fa8bd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:57524/0 (socket says 172.21.15.12:57524) 2024-05-17T22:44:27.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.678+0000 7f55c37fe700 1 -- 172.21.15.12:0/716772433 learned_addr learned my addr 172.21.15.12:0/716772433 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:44:27.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.678+0000 7f55c17fa700 1 --2- 172.21.15.12:0/3995008165 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f55b00b2900 0x7f55b00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:27.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.678+0000 7f55c3fff700 1 --2- 172.21.15.12:0/3995008165 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f55b00b2900 0x7f55b00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:27.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.678+0000 7f55bbfff700 1 --2- 172.21.15.12:0/716772433 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5620fa7369f0 0x5620fa8c1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:27.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.678+0000 7f55bbfff700 1 --2- 172.21.15.12:0/716772433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620fa8bd980 0x5620fa8bdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:27.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.679+0000 7f55c17fa700 1 -- 172.21.15.12:0/3995008165 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f55bc0e1e60 con 0x7f55c410c3b0 2024-05-17T22:44:27.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.679+0000 7f55c37fe700 1 -- 172.21.15.12:0/716772433 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5620fa7369f0 msgr2=0x5620fa8c1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:44:27.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.679+0000 7f55c37fe700 1 --2- 172.21.15.12:0/716772433 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5620fa7369f0 0x5620fa8c1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:27.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.679+0000 7f55c37fe700 1 -- 172.21.15.12:0/716772433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620fa8bd980 msgr2=0x5620fa8bdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:44:27.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.679+0000 7f55c37fe700 1 --2- 172.21.15.12:0/716772433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620fa8bd980 0x5620fa8bdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:27.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.679+0000 7f55c37fe700 1 -- 172.21.15.12:0/716772433 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620fa77c590 con 0x5620fa8c21a0 2024-05-17T22:44:27.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.679+0000 7f55bbfff700 1 --2- 172.21.15.12:0/716772433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620fa8bd980 0x5620fa8bdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:44:27.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.679+0000 7f55c37fe700 1 --2- 172.21.15.12:0/716772433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620fa8c21a0 0x5620fa8bd440 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f55ac009cf0 tx=0x7f55ac011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5262a7184c3b406 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:44:27.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.679+0000 7f55b9ffb700 1 -- 172.21.15.12:0/716772433 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac011e30 con 0x5620fa8c21a0 2024-05-17T22:44:27.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.679+0000 7f55b9ffb700 1 -- 172.21.15.12:0/716772433 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55ac0044d0 con 0x5620fa8c21a0 2024-05-17T22:44:27.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.679+0000 7f55b9ffb700 1 -- 172.21.15.12:0/716772433 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac005260 con 0x5620fa8c21a0 2024-05-17T22:44:27.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.680+0000 7f55d8bf4b80 1 -- 172.21.15.12:0/716772433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620fa8c21a0 msgr2=0x5620fa8bd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:27.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.680+0000 7f55d8bf4b80 1 --2- 172.21.15.12:0/716772433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620fa8c21a0 0x5620fa8bd440 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f55ac009cf0 tx=0x7f55ac011440 comp rx=0 tx=0).stop 2024-05-17T22:44:27.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.680+0000 7f55c3fff700 1 --2- 172.21.15.12:0/3995008165 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f55b00b2900 0x7f55b00b4e10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f55bc0a63f0 tx=0x7f55bc06e6c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:44:27.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.680+0000 7f55d8bf4b80 1 -- 172.21.15.12:0/716772433 shutdown_connections 2024-05-17T22:44:27.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.680+0000 7f55d8bf4b80 1 --2- 172.21.15.12:0/716772433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620fa8bd980 0x5620fa8bdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:27.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.680+0000 7f55d8bf4b80 1 --2- 172.21.15.12:0/716772433 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5620fa7369f0 0x5620fa8c1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:27.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.680+0000 7f55d8bf4b80 1 --2- 172.21.15.12:0/716772433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620fa8c21a0 0x5620fa8bd440 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:27.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.680+0000 7f55d8bf4b80 1 -- 172.21.15.12:0/716772433 >> 172.21.15.12:0/716772433 conn(0x5620fa77af00 msgr2=0x5620fa8afb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:44:27.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.680+0000 7f55d8bf4b80 1 -- 172.21.15.12:0/716772433 shutdown_connections 2024-05-17T22:44:27.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.680+0000 7f55d8bf4b80 1 -- 172.21.15.12:0/716772433 wait complete. 2024-05-17T22:44:27.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.681+0000 7f55d8bf4b80 1 -- start start 2024-05-17T22:44:27.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.681+0000 7f55d8bf4b80 1 Processor -- start 2024-05-17T22:44:27.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.682+0000 7f55d8bf4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620fa7369f0 0x5620fa8c6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:27.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.682+0000 7f55d8bf4b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620fa8bd980 0x5620fa8c6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:27.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.682+0000 7f55d8bf4b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5620fa8c21a0 0x5620fa8c73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:27.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.682+0000 7f55d8bf4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5620fa6da150 con 0x5620fa7369f0 2024-05-17T22:44:27.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.682+0000 7f55d8bf4b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5620fa773e80 con 0x5620fa8bd980 2024-05-17T22:44:27.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.682+0000 7f55d8bf4b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5620fa8c9c80 con 0x5620fa8c21a0 2024-05-17T22:44:27.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.682+0000 7f55d8bf4b80 10 client.0 _ll_register_callbacks cb 0x5620fa76df10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:44:27.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.683+0000 7f55c37fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620fa8bd980 0x5620fa8c6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:27.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.683+0000 7f55c37fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620fa8bd980 0x5620fa8c6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:57528/0 (socket says 172.21.15.12:57528) 2024-05-17T22:44:27.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.683+0000 7f55c37fe700 1 -- 172.21.15.12:0/2957160080 learned_addr learned my addr 172.21.15.12:0/2957160080 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:44:27.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.683+0000 7f55bbfff700 1 --2- 172.21.15.12:0/2957160080 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5620fa8c21a0 0x5620fa8c73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:27.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.684+0000 7f55bbfff700 1 --2- 172.21.15.12:0/2957160080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620fa7369f0 0x5620fa8c6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:27.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.684+0000 7f55c37fe700 1 -- 172.21.15.12:0/2957160080 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5620fa8c21a0 msgr2=0x5620fa8c73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:44:27.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.684+0000 7f55c37fe700 1 --2- 172.21.15.12:0/2957160080 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5620fa8c21a0 0x5620fa8c73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:27.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.684+0000 7f55c37fe700 1 -- 172.21.15.12:0/2957160080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620fa7369f0 msgr2=0x5620fa8c6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:44:27.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.684+0000 7f55c37fe700 1 --2- 172.21.15.12:0/2957160080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620fa7369f0 0x5620fa8c6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:27.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.684+0000 7f55c37fe700 1 -- 172.21.15.12:0/2957160080 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620fa723480 con 0x5620fa8bd980 2024-05-17T22:44:27.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.684+0000 7f55bbfff700 1 --2- 172.21.15.12:0/2957160080 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5620fa8c21a0 0x5620fa8c73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:44:27.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.684+0000 7f55c37fe700 1 --2- 172.21.15.12:0/2957160080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620fa8bd980 0x5620fa8c6eb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f55ac006010 tx=0x7f55ac004a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c43336416b944b87 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:44:27.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.684+0000 7f55a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:44:27.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.684+0000 7f55a2ffd700 1 -- 172.21.15.12:0/2957160080 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac028020 con 0x5620fa8bd980 2024-05-17T22:44:27.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.684+0000 7f55a2ffd700 1 -- 172.21.15.12:0/2957160080 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f55ac005210 con 0x5620fa8bd980 2024-05-17T22:44:27.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.684+0000 7f55d8bf4b80 1 -- 172.21.15.12:0/2957160080 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55ac0044d0 con 0x5620fa8bd980 2024-05-17T22:44:27.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.685+0000 7f55a2ffd700 1 -- 172.21.15.12:0/2957160080 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f55ac033050 con 0x5620fa8bd980 2024-05-17T22:44:27.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.685+0000 7f55d8bf4b80 10 client.25442 fetch_fsmap learned FSMap version 33 2024-05-17T22:44:27.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.685+0000 7f55d8bf4b80 1 -- 172.21.15.12:0/2957160080 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5620fa8d4610 con 0x5620fa8bd980 2024-05-17T22:44:27.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.685+0000 7f55a2ffd700 1 -- 172.21.15.12:0/2957160080 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55ac0044d0 con 0x5620fa8bd980 2024-05-17T22:44:27.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.685+0000 7f55a2ffd700 1 -- 172.21.15.12:0/2957160080 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f55ac023070 con 0x5620fa8bd980 2024-05-17T22:44:27.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.685+0000 7f55d8bf4b80 10 client.25442 fetch_fsmap finished waiting for FSMap version 33 2024-05-17T22:44:27.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.685+0000 7f55d8bf4b80 10 client.25442 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-17T22:44:27.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.685+0000 7f55d8bf4b80 20 client.25442 populate_metadata read hostname 'smithi012' 2024-05-17T22:44:27.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.685+0000 7f55d8bf4b80 1 --2- 172.21.15.12:0/2957160080 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x5620fa8d6bf0 0x5620fa8d9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:27.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.685+0000 7f55d8bf4b80 4 client.25442 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:44:27.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.685+0000 7f55d8bf4b80 1 -- 172.21.15.12:0/2957160080 --> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620fa7e51e0 con 0x5620fa8d6bf0 2024-05-17T22:44:27.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.686+0000 7f55bbfff700 1 --2- 172.21.15.12:0/2957160080 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x5620fa8d6bf0 0x5620fa8d9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:27.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.686+0000 7f55a2ffd700 10 client.25442 ms_handle_connect on v2:172.21.15.151:6838/2338353849 2024-05-17T22:44:27.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.686+0000 7f55bbfff700 1 --2- 172.21.15.12:0/2957160080 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x5620fa8d6bf0 0x5620fa8d9010 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=40e8da856b22829f server_cookie=1142c612a8a90c34 in_seq=0 out_seq=0 2024-05-17T22:44:27.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.687+0000 7f55a2ffd700 1 -- 172.21.15.12:0/2957160080 <== mds.2 v2:172.21.15.151:6838/2338353849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5620fa7e51e0 con 0x5620fa8d6bf0 2024-05-17T22:44:27.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.687+0000 7f55a2ffd700 10 client.25442 handle_command_reply: tid=0 2024-05-17T22:44:27.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.687+0000 7f55d8bf4b80 1 client.25442 shutdown 2024-05-17T22:44:27.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.687+0000 7f55d8bf4b80 1 -- 172.21.15.12:0/2957160080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620fa8bd980 msgr2=0x5620fa8c6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:27.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.687+0000 7f55d8bf4b80 1 --2- 172.21.15.12:0/2957160080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620fa8bd980 0x5620fa8c6eb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f55ac006010 tx=0x7f55ac004a30 comp rx=0 tx=0).stop 2024-05-17T22:44:27.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.687+0000 7f55d8bf4b80 1 -- 172.21.15.12:0/2957160080 shutdown_connections 2024-05-17T22:44:27.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.687+0000 7f55d8bf4b80 1 --2- 172.21.15.12:0/2957160080 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x5620fa8d6bf0 0x5620fa8d9010 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:27.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.687+0000 7f55d8bf4b80 1 --2- 172.21.15.12:0/2957160080 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5620fa8c21a0 0x5620fa8c73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:27.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.687+0000 7f55d8bf4b80 1 --2- 172.21.15.12:0/2957160080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620fa7369f0 0x5620fa8c6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:27.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.687+0000 7f55d8bf4b80 1 --2- 172.21.15.12:0/2957160080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620fa8bd980 0x5620fa8c6eb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:27.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.687+0000 7f55d8bf4b80 1 -- 172.21.15.12:0/2957160080 >> 172.21.15.12:0/2957160080 conn(0x5620fa77af00 msgr2=0x5620fa8b3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:44:27.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.687+0000 7f55a2ffd700 0 client.25442 ms_handle_reset on v2:172.21.15.151:6838/2338353849 2024-05-17T22:44:27.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.687+0000 7f55d8bf4b80 1 -- 172.21.15.12:0/2957160080 shutdown_connections 2024-05-17T22:44:27.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.687+0000 7f55d8bf4b80 1 -- 172.21.15.12:0/2957160080 wait complete. 2024-05-17T22:44:27.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.687+0000 7f55d8bf4b80 20 client.25442 trim_cache size 0 max 16384 2024-05-17T22:44:27.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.716+0000 7f55d8bf4b80 1 Processor -- start 2024-05-17T22:44:27.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.716+0000 7f55d8bf4b80 1 -- start start 2024-05-17T22:44:27.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.716+0000 7f55d8bf4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620fa7369f0 0x5620fa8d9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:27.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.717+0000 7f55d8bf4b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620fa8bd980 0x5620fa8d3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:27.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.717+0000 7f55d8bf4b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5620fa8c21a0 0x5620fa8a9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:27.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.717+0000 7f55d8bf4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f55a40113e0 con 0x5620fa7369f0 2024-05-17T22:44:27.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.717+0000 7f55d8bf4b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f55a400ec10 con 0x5620fa8bd980 2024-05-17T22:44:27.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.717+0000 7f55d8bf4b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f55a400ca60 con 0x5620fa8c21a0 2024-05-17T22:44:27.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.719+0000 7f55c37fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620fa8bd980 0x5620fa8d3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:27.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.720+0000 7f55bbfff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5620fa8c21a0 0x5620fa8a9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:27.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.720+0000 7f55bbfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620fa7369f0 0x5620fa8d9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:27.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.720+0000 7f55bbfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620fa7369f0 0x5620fa8d9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47868/0 (socket says 172.21.15.12:47868) 2024-05-17T22:44:27.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.720+0000 7f55bbfff700 1 -- 172.21.15.12:0/3739741801 learned_addr learned my addr 172.21.15.12:0/3739741801 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:44:27.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.722+0000 7f55c37fe700 1 -- 172.21.15.12:0/3739741801 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5620fa8c21a0 msgr2=0x5620fa8a9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:44:27.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.722+0000 7f55c37fe700 1 --2- 172.21.15.12:0/3739741801 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5620fa8c21a0 0x5620fa8a9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:27.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.722+0000 7f55c37fe700 1 -- 172.21.15.12:0/3739741801 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620fa7369f0 msgr2=0x5620fa8d9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:44:27.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.722+0000 7f55c37fe700 1 --2- 172.21.15.12:0/3739741801 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620fa7369f0 0x5620fa8d9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:27.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.722+0000 7f55c37fe700 1 -- 172.21.15.12:0/3739741801 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620fa8be100 con 0x5620fa8bd980 2024-05-17T22:44:27.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.725+0000 7f55c37fe700 1 --2- 172.21.15.12:0/3739741801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620fa8bd980 0x5620fa8d3050 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f55ac009c10 tx=0x7f55ac011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcd3b84a6c8433ea server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:44:27.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.729+0000 7f55b97fa700 1 -- 172.21.15.12:0/3739741801 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac028040 con 0x5620fa8bd980 2024-05-17T22:44:27.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.729+0000 7f55b97fa700 1 -- 172.21.15.12:0/3739741801 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55ac0056c0 con 0x5620fa8bd980 2024-05-17T22:44:27.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.729+0000 7f55b97fa700 1 -- 172.21.15.12:0/3739741801 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac01d030 con 0x5620fa8bd980 2024-05-17T22:44:27.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.731+0000 7f55d8bf4b80 1 -- 172.21.15.12:0/3739741801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620fa8bd980 msgr2=0x5620fa8d3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:27.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.731+0000 7f55d8bf4b80 1 --2- 172.21.15.12:0/3739741801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620fa8bd980 0x5620fa8d3050 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f55ac009c10 tx=0x7f55ac011440 comp rx=0 tx=0).stop 2024-05-17T22:44:27.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.731+0000 7f55d8bf4b80 1 -- 172.21.15.12:0/3739741801 shutdown_connections 2024-05-17T22:44:27.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.731+0000 7f55d8bf4b80 1 --2- 172.21.15.12:0/3739741801 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5620fa8c21a0 0x5620fa8a9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:27.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.731+0000 7f55d8bf4b80 1 --2- 172.21.15.12:0/3739741801 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620fa7369f0 0x5620fa8d9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:27.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.731+0000 7f55d8bf4b80 1 --2- 172.21.15.12:0/3739741801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620fa8bd980 0x5620fa8d3050 secure :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f55ac009c10 tx=0x7f55ac011440 comp rx=0 tx=0).stop 2024-05-17T22:44:27.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.731+0000 7f55d8bf4b80 1 -- 172.21.15.12:0/3739741801 >> 172.21.15.12:0/3739741801 conn(0x5620fa77af00 msgr2=0x5620fa8b3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:44:27.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.732+0000 7f55d8bf4b80 1 -- 172.21.15.12:0/3739741801 shutdown_connections 2024-05-17T22:44:27.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.732+0000 7f55d8bf4b80 1 -- 172.21.15.12:0/3739741801 wait complete. 2024-05-17T22:44:27.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.733+0000 7f55d8bf4b80 1 -- start start 2024-05-17T22:44:27.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.733+0000 7f55d8bf4b80 1 Processor -- start 2024-05-17T22:44:27.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.734+0000 7f55d8bf4b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620fa7369f0 0x5620fa8caac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:27.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.734+0000 7f55d8bf4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620fa8c21a0 0x5620fa8cb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:27.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.735+0000 7f55d8bf4b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5620fa8cb540 0x5620fa8bac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:27.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.735+0000 7f55bbfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620fa7369f0 0x5620fa8caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:27.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.735+0000 7f55bbfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620fa7369f0 0x5620fa8caac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:57542/0 (socket says 172.21.15.12:57542) 2024-05-17T22:44:27.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.735+0000 7f55bbfff700 1 -- 172.21.15.12:0/1484861260 learned_addr learned my addr 172.21.15.12:0/1484861260 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:44:27.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.735+0000 7f55d8bf4b80 1 -- 172.21.15.12:0/1484861260 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f55a400d0d0 con 0x5620fa8c21a0 2024-05-17T22:44:27.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.735+0000 7f55d8bf4b80 1 -- 172.21.15.12:0/1484861260 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f55a400e540 con 0x5620fa7369f0 2024-05-17T22:44:27.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.735+0000 7f55d8bf4b80 1 -- 172.21.15.12:0/1484861260 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f55a400d7a0 con 0x5620fa8cb540 2024-05-17T22:44:27.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.735+0000 7f55c37fe700 1 --2- 172.21.15.12:0/1484861260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620fa8c21a0 0x5620fa8cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:27.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.735+0000 7f55d8bf4b80 10 client.0 _ll_register_callbacks cb 0x5620fa76df10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:44:27.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.735+0000 7f55bbfff700 1 --2- 172.21.15.12:0/1484861260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5620fa8cb540 0x5620fa8bac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:27.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.735+0000 7f55c37fe700 1 -- 172.21.15.12:0/1484861260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5620fa8cb540 msgr2=0x5620fa8bac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:44:27.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.735+0000 7f55c37fe700 1 --2- 172.21.15.12:0/1484861260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5620fa8cb540 0x5620fa8bac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:27.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.735+0000 7f55c37fe700 1 -- 172.21.15.12:0/1484861260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620fa7369f0 msgr2=0x5620fa8caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:44:27.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.735+0000 7f55c37fe700 1 --2- 172.21.15.12:0/1484861260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620fa7369f0 0x5620fa8caac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:27.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.735+0000 7f55c37fe700 1 -- 172.21.15.12:0/1484861260 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620fa8bb250 con 0x5620fa8c21a0 2024-05-17T22:44:27.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.735+0000 7f55bbfff700 1 --2- 172.21.15.12:0/1484861260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5620fa8cb540 0x5620fa8bac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:44:27.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.735+0000 7f55c37fe700 1 --2- 172.21.15.12:0/1484861260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620fa8c21a0 0x5620fa8cb000 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x5620fa8c7be0 tx=0x7f55ac0098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218a4350f26a4c78 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:44:27.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.735+0000 7f55a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:44:27.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.735+0000 7f55a2ffd700 1 -- 172.21.15.12:0/1484861260 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac01d040 con 0x5620fa8c21a0 2024-05-17T22:44:27.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.736+0000 7f55d8bf4b80 1 -- 172.21.15.12:0/1484861260 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5620fa7de9c0 con 0x5620fa8c21a0 2024-05-17T22:44:27.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.736+0000 7f55a2ffd700 1 -- 172.21.15.12:0/1484861260 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f55ac005490 con 0x5620fa8c21a0 2024-05-17T22:44:27.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.736+0000 7f55a2ffd700 1 -- 172.21.15.12:0/1484861260 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f55ac028040 con 0x5620fa8c21a0 2024-05-17T22:44:27.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.736+0000 7f55d8bf4b80 10 client.15786 fetch_fsmap learned FSMap version 33 2024-05-17T22:44:27.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.736+0000 7f55d8bf4b80 1 -- 172.21.15.12:0/1484861260 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5620fa8bc490 con 0x5620fa8c21a0 2024-05-17T22:44:27.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.736+0000 7f55a2ffd700 1 -- 172.21.15.12:0/1484861260 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5620fa7de9c0 con 0x5620fa8c21a0 2024-05-17T22:44:27.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.737+0000 7f55a2ffd700 1 -- 172.21.15.12:0/1484861260 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f55ac023040 con 0x5620fa8c21a0 2024-05-17T22:44:27.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.737+0000 7f55d8bf4b80 10 client.15786 fetch_fsmap finished waiting for FSMap version 33 2024-05-17T22:44:27.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.737+0000 7f55d8bf4b80 10 client.15786 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-17T22:44:27.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.737+0000 7f55d8bf4b80 20 client.15786 populate_metadata read hostname 'smithi012' 2024-05-17T22:44:27.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.737+0000 7f55d8bf4b80 1 --2- 172.21.15.12:0/1484861260 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x5620fa8bcb30 0x5620fa94c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:27.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.737+0000 7f55d8bf4b80 4 client.15786 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:44:27.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.737+0000 7f55d8bf4b80 1 -- 172.21.15.12:0/1484861260 --> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5620fa6af360 con 0x5620fa8bcb30 2024-05-17T22:44:27.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.737+0000 7f55bbfff700 1 --2- 172.21.15.12:0/1484861260 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x5620fa8bcb30 0x5620fa94c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:27.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.738+0000 7f55a2ffd700 10 client.15786 ms_handle_connect on v2:172.21.15.151:6838/2338353849 2024-05-17T22:44:27.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.738+0000 7f55bbfff700 1 --2- 172.21.15.12:0/1484861260 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x5620fa8bcb30 0x5620fa94c1a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8fb4e2f71a705f59 server_cookie=c34e836921f4102d in_seq=0 out_seq=0 2024-05-17T22:44:27.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.738+0000 7f55a2ffd700 1 -- 172.21.15.12:0/1484861260 <== mds.2 v2:172.21.15.151:6838/2338353849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5620fa6af360 con 0x5620fa8bcb30 2024-05-17T22:44:27.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.738+0000 7f55a2ffd700 10 client.15786 handle_command_reply: tid=0 2024-05-17T22:44:27.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.738+0000 7f55d8bf4b80 1 client.15786 shutdown 2024-05-17T22:44:27.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.738+0000 7f55d8bf4b80 1 -- 172.21.15.12:0/1484861260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620fa8c21a0 msgr2=0x5620fa8cb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:27.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.738+0000 7f55d8bf4b80 1 --2- 172.21.15.12:0/1484861260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620fa8c21a0 0x5620fa8cb000 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x5620fa8c7be0 tx=0x7f55ac0098b0 comp rx=0 tx=0).stop 2024-05-17T22:44:27.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.738+0000 7f55d8bf4b80 1 -- 172.21.15.12:0/1484861260 shutdown_connections 2024-05-17T22:44:27.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.738+0000 7f55d8bf4b80 1 --2- 172.21.15.12:0/1484861260 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x5620fa8bcb30 0x5620fa94c1a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:27.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.738+0000 7f55d8bf4b80 1 --2- 172.21.15.12:0/1484861260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5620fa8cb540 0x5620fa8bac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:27.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.738+0000 7f55a2ffd700 0 client.15786 ms_handle_reset on v2:172.21.15.151:6838/2338353849 2024-05-17T22:44:27.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.738+0000 7f55d8bf4b80 1 --2- 172.21.15.12:0/1484861260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620fa7369f0 0x5620fa8caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:27.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.738+0000 7f55d8bf4b80 1 --2- 172.21.15.12:0/1484861260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5620fa8c21a0 0x5620fa8cb000 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:27.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.738+0000 7f55d8bf4b80 1 -- 172.21.15.12:0/1484861260 >> 172.21.15.12:0/1484861260 conn(0x5620fa77af00 msgr2=0x5620fa8c12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:44:27.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.738+0000 7f55d8bf4b80 1 -- 172.21.15.12:0/1484861260 shutdown_connections 2024-05-17T22:44:27.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.738+0000 7f55d8bf4b80 1 -- 172.21.15.12:0/1484861260 wait complete. 2024-05-17T22:44:27.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.739+0000 7f55d8bf4b80 20 client.15786 trim_cache size 0 max 16384 2024-05-17T22:44:27.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.743+0000 7f55ca219700 1 -- 172.21.15.12:0/3995008165 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f55b00b2900 msgr2=0x7f55b00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:27.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.743+0000 7f55ca219700 1 --2- 172.21.15.12:0/3995008165 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f55b00b2900 0x7f55b00b4e10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f55bc0a63f0 tx=0x7f55bc06e6c0 comp rx=0 tx=0).stop 2024-05-17T22:44:27.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.743+0000 7f55ca219700 1 -- 172.21.15.12:0/3995008165 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55c410c3b0 msgr2=0x7f55c419c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:27.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.743+0000 7f55ca219700 1 --2- 172.21.15.12:0/3995008165 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55c410c3b0 0x7f55c419c830 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f55bc00ef30 tx=0x7f55bc00c3a0 comp rx=0 tx=0).stop 2024-05-17T22:44:27.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.745+0000 7f55ca219700 1 -- 172.21.15.12:0/3995008165 shutdown_connections 2024-05-17T22:44:27.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.745+0000 7f55ca219700 1 --2- 172.21.15.12:0/3995008165 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f55b00b2900 0x7f55b00b4e10 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:27.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.745+0000 7f55ca219700 1 --2- 172.21.15.12:0/3995008165 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55c410c3b0 0x7f55c419c830 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:27.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.745+0000 7f55ca219700 1 --2- 172.21.15.12:0/3995008165 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55c4101f40 0x7f55c41a25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:27.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.745+0000 7f55ca219700 1 --2- 172.21.15.12:0/3995008165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55c4102890 0x7f55c41a2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:27.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.745+0000 7f55ca219700 1 -- 172.21.15.12:0/3995008165 >> 172.21.15.12:0/3995008165 conn(0x7f55c40fdd60 msgr2=0x7f55c4100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:44:27.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.745+0000 7f55ca219700 1 -- 172.21.15.12:0/3995008165 shutdown_connections 2024-05-17T22:44:27.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:26.745+0000 7f55ca219700 1 -- 172.21.15.12:0/3995008165 wait complete. 2024-05-17T22:44:28.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:28 smithi151 ceph-mon[36101]: pgmap v595: 129 pgs: 129 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 576 op/s 2024-05-17T22:44:28.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:28 smithi096 ceph-mon[36065]: pgmap v595: 129 pgs: 129 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 576 op/s 2024-05-17T22:44:28.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:28 smithi012 ceph-mon[31632]: pgmap v595: 129 pgs: 129 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 576 op/s 2024-05-17T22:44:29.183 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-17T22:44:29.724 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:44:30.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:30 smithi151 ceph-mon[36101]: pgmap v596: 129 pgs: 129 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 MiB/s wr, 715 op/s 2024-05-17T22:44:30.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:30 smithi096 ceph-mon[36065]: pgmap v596: 129 pgs: 129 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 MiB/s wr, 715 op/s 2024-05-17T22:44:30.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:30 smithi012 ceph-mon[31632]: pgmap v596: 129 pgs: 129 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 MiB/s wr, 715 op/s 2024-05-17T22:44:31.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 310.0 s, 715.6 tps, lat 143.986 ms stddev 171.130 2024-05-17T22:44:31.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:31 smithi151 ceph-mon[36101]: pgmap v597: 129 pgs: 129 active+clean; 1.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 597 op/s 2024-05-17T22:44:31.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:31 smithi096 ceph-mon[36065]: pgmap v597: 129 pgs: 129 active+clean; 1.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 597 op/s 2024-05-17T22:44:31.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:31 smithi012 ceph-mon[31632]: pgmap v597: 129 pgs: 129 active+clean; 1.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 597 op/s 2024-05-17T22:44:33.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.248+0000 7f3e1a342700 1 -- 172.21.15.12:0/1085264789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e14112230 msgr2=0x7f3e1410aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:33.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.248+0000 7f3e1a342700 1 --2- 172.21.15.12:0/1085264789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e14112230 0x7f3e1410aa80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3e100098f0 tx=0x7f3e10009c00 comp rx=0 tx=0).stop 2024-05-17T22:44:33.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.252+0000 7f3e1a342700 1 -- 172.21.15.12:0/1085264789 shutdown_connections 2024-05-17T22:44:33.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.252+0000 7f3e1a342700 1 --2- 172.21.15.12:0/1085264789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e1411d010 0x7f3e1411f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:33.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.252+0000 7f3e1a342700 1 --2- 172.21.15.12:0/1085264789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e14112230 0x7f3e1410aa80 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:33.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.252+0000 7f3e1a342700 1 --2- 172.21.15.12:0/1085264789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e1410afc0 0x7f3e1410b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:33.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.252+0000 7f3e1a342700 1 -- 172.21.15.12:0/1085264789 >> 172.21.15.12:0/1085264789 conn(0x7f3e1406eda0 msgr2=0x7f3e1406f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:44:33.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.257+0000 7f3e1a342700 1 -- 172.21.15.12:0/1085264789 shutdown_connections 2024-05-17T22:44:33.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.258+0000 7f3e1a342700 1 -- 172.21.15.12:0/1085264789 wait complete. 2024-05-17T22:44:33.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.263+0000 7f3e1a342700 1 Processor -- start 2024-05-17T22:44:33.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.265+0000 7f3e1a342700 1 -- start start 2024-05-17T22:44:33.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.265+0000 7f3e1a342700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e1410afc0 0x7f3e1411ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:33.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.265+0000 7f3e1a342700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e14112230 0x7f3e14114b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:33.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.265+0000 7f3e1a342700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e1411d010 0x7f3e14115220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:33.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.265+0000 7f3e1a342700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3e14122a20 con 0x7f3e14112230 2024-05-17T22:44:33.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.265+0000 7f3e1a342700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3e141228a0 con 0x7f3e1410afc0 2024-05-17T22:44:33.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.265+0000 7f3e1a342700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3e14122ba0 con 0x7f3e1411d010 2024-05-17T22:44:33.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.268+0000 7f3e19b41700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e1411d010 0x7f3e14115220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:33.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.268+0000 7f3e19340700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e1410afc0 0x7f3e1411ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:33.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.269+0000 7f3e19340700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e1410afc0 0x7f3e1411ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:60524/0 (socket says 172.21.15.12:60524) 2024-05-17T22:44:33.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.269+0000 7f3e19340700 1 -- 172.21.15.12:0/3873339883 learned_addr learned my addr 172.21.15.12:0/3873339883 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:44:33.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.271+0000 7f3e18b3f700 1 --2- 172.21.15.12:0/3873339883 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e14112230 0x7f3e14114b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:33.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.280+0000 7f3e19b41700 1 -- 172.21.15.12:0/3873339883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e1410afc0 msgr2=0x7f3e1411ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:33.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.280+0000 7f3e19b41700 1 --2- 172.21.15.12:0/3873339883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e1410afc0 0x7f3e1411ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:33.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.280+0000 7f3e19b41700 1 -- 172.21.15.12:0/3873339883 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e14112230 msgr2=0x7f3e14114b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:33.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.280+0000 7f3e19b41700 1 --2- 172.21.15.12:0/3873339883 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e14112230 0x7f3e14114b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:33.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.280+0000 7f3e19b41700 1 -- 172.21.15.12:0/3873339883 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e14119580 con 0x7f3e1411d010 2024-05-17T22:44:33.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.281+0000 7f3e18b3f700 1 --2- 172.21.15.12:0/3873339883 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e14112230 0x7f3e14114b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:44:33.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.286+0000 7f3e19b41700 1 --2- 172.21.15.12:0/3873339883 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e1411d010 0x7f3e14115220 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c00efc0 tx=0x7f3e0c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:44:33.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.289+0000 7f3e0a7fc700 1 -- 172.21.15.12:0/3873339883 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c0092e0 con 0x7f3e1411d010 2024-05-17T22:44:33.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.289+0000 7f3e0a7fc700 1 -- 172.21.15.12:0/3873339883 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e0c004bb0 con 0x7f3e1411d010 2024-05-17T22:44:33.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.290+0000 7f3e0a7fc700 1 -- 172.21.15.12:0/3873339883 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c004020 con 0x7f3e1411d010 2024-05-17T22:44:33.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.291+0000 7f3e1a342700 1 -- 172.21.15.12:0/3873339883 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e14119810 con 0x7f3e1411d010 2024-05-17T22:44:33.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.291+0000 7f3e1a342700 1 -- 172.21.15.12:0/3873339883 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e14119cd0 con 0x7f3e1411d010 2024-05-17T22:44:33.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.299+0000 7f3e29f7fb80 1 Processor -- start 2024-05-17T22:44:33.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.301+0000 7f3e29f7fb80 1 -- start start 2024-05-17T22:44:33.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.301+0000 7f3e29f7fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a94ab7d9f0 0x55a94ad01770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:33.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.301+0000 7f3e29f7fb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a94ad01cb0 0x55a94acfbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:33.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.301+0000 7f3e29f7fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a94acfc170 0x55a94acfc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:33.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.301+0000 7f3e29f7fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a94ac2a290 con 0x55a94ab7d9f0 2024-05-17T22:44:33.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.301+0000 7f3e29f7fb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a94ac19fd0 con 0x55a94ad01cb0 2024-05-17T22:44:33.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.301+0000 7f3e29f7fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a94ac1de70 con 0x55a94acfc170 2024-05-17T22:44:33.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.302+0000 7f3e19340700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a94ad01cb0 0x55a94acfbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:33.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.302+0000 7f3e19340700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a94ad01cb0 0x55a94acfbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:60532/0 (socket says 172.21.15.12:60532) 2024-05-17T22:44:33.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.302+0000 7f3e19340700 1 -- 172.21.15.12:0/3493403462 learned_addr learned my addr 172.21.15.12:0/3493403462 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:44:33.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.303+0000 7f3e0a7fc700 1 -- 172.21.15.12:0/3873339883 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3e0c012050 con 0x7f3e1411d010 2024-05-17T22:44:33.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.303+0000 7f3e19340700 1 -- 172.21.15.12:0/3493403462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a94acfc170 msgr2=0x55a94acfc580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T22:44:33.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.303+0000 7f3e19340700 1 --2- 172.21.15.12:0/3493403462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a94acfc170 0x55a94acfc580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:33.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.303+0000 7f3e19340700 1 -- 172.21.15.12:0/3493403462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a94ab7d9f0 msgr2=0x55a94ad01770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:44:33.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.303+0000 7f3e18b3f700 1 --2- 172.21.15.12:0/3493403462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a94ab7d9f0 0x55a94ad01770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:33.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.303+0000 7f3e19340700 1 --2- 172.21.15.12:0/3493403462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a94ab7d9f0 0x55a94ad01770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:33.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.303+0000 7f3e19340700 1 -- 172.21.15.12:0/3493403462 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a94abc3590 con 0x55a94ad01cb0 2024-05-17T22:44:33.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.303+0000 7f3e19340700 1 --2- 172.21.15.12:0/3493403462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a94ad01cb0 0x55a94acfbc30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3e10009230 tx=0x7f3e1000bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0bb18eab3d8ebf0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:44:33.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.304+0000 7f3e18b3f700 1 --2- 172.21.15.12:0/3493403462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a94ab7d9f0 0x55a94ad01770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:44:33.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.304+0000 7f3e0a7fc700 1 --2- 172.21.15.12:0/3873339883 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f3e000b28a0 0x7f3e000b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:33.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.305+0000 7f3e0a7fc700 1 -- 172.21.15.12:0/3873339883 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f3e0c0e1710 con 0x7f3e1411d010 2024-05-17T22:44:33.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.306+0000 7f3dff7fe700 1 -- 172.21.15.12:0/3493403462 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e10026070 con 0x55a94ad01cb0 2024-05-17T22:44:33.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.306+0000 7f3dff7fe700 1 -- 172.21.15.12:0/3493403462 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e1000f8a0 con 0x55a94ad01cb0 2024-05-17T22:44:33.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.306+0000 7f3dff7fe700 1 -- 172.21.15.12:0/3493403462 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e10003b90 con 0x55a94ad01cb0 2024-05-17T22:44:33.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.306+0000 7f3e18b3f700 1 --2- 172.21.15.12:0/3873339883 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f3e000b28a0 0x7f3e000b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:33.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.308+0000 7f3e29f7fb80 1 -- 172.21.15.12:0/3493403462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a94ad01cb0 msgr2=0x55a94acfbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:33.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.308+0000 7f3e29f7fb80 1 --2- 172.21.15.12:0/3493403462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a94ad01cb0 0x55a94acfbc30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3e10009230 tx=0x7f3e1000bf60 comp rx=0 tx=0).stop 2024-05-17T22:44:33.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.308+0000 7f3e29f7fb80 1 -- 172.21.15.12:0/3493403462 shutdown_connections 2024-05-17T22:44:33.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.308+0000 7f3e29f7fb80 1 --2- 172.21.15.12:0/3493403462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a94acfc170 0x55a94acfc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:33.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.308+0000 7f3e29f7fb80 1 --2- 172.21.15.12:0/3493403462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a94ab7d9f0 0x55a94ad01770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:33.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.308+0000 7f3e29f7fb80 1 --2- 172.21.15.12:0/3493403462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a94ad01cb0 0x55a94acfbc30 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:33.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.308+0000 7f3e29f7fb80 1 -- 172.21.15.12:0/3493403462 >> 172.21.15.12:0/3493403462 conn(0x55a94abc1f00 msgr2=0x55a94acf6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:44:33.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.309+0000 7f3e29f7fb80 1 -- 172.21.15.12:0/3493403462 shutdown_connections 2024-05-17T22:44:33.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.310+0000 7f3e29f7fb80 1 -- 172.21.15.12:0/3493403462 wait complete. 2024-05-17T22:44:33.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.310+0000 7f3e29f7fb80 1 -- start start 2024-05-17T22:44:33.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.310+0000 7f3e29f7fb80 1 Processor -- start 2024-05-17T22:44:33.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.311+0000 7f3e18b3f700 1 --2- 172.21.15.12:0/3873339883 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f3e000b28a0 0x7f3e000b4db0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55a94acf7ae0 tx=0x7f3e04009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:44:33.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.311+0000 7f3e29f7fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a94ab7d9f0 0x55a94ad052b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:33.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.311+0000 7f3e29f7fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a94acfc170 0x55a94ad057f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:33.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.311+0000 7f3e29f7fb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a94ad01cb0 0x55a94ad05d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:33.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.311+0000 7f3e29f7fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a94ab21150 con 0x55a94acfc170 2024-05-17T22:44:33.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.311+0000 7f3e29f7fb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a94abbae80 con 0x55a94ad01cb0 2024-05-17T22:44:33.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.311+0000 7f3e29f7fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a94ad085a0 con 0x55a94ab7d9f0 2024-05-17T22:44:33.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.311+0000 7f3e29f7fb80 10 client.0 _ll_register_callbacks cb 0x55a94abb4f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:44:33.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.312+0000 7f3e19340700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a94ab7d9f0 0x55a94ad052b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:33.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.312+0000 7f3e19b41700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a94ad01cb0 0x55a94ad05d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:33.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.312+0000 7f3e19340700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a94ab7d9f0 0x55a94ad052b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:54230/0 (socket says 172.21.15.12:54230) 2024-05-17T22:44:33.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.313+0000 7f3e19340700 1 -- 172.21.15.12:0/4140611726 learned_addr learned my addr 172.21.15.12:0/4140611726 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:44:33.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.314+0000 7f3e18b3f700 1 --2- 172.21.15.12:0/4140611726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a94acfc170 0x55a94ad057f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:33.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.316+0000 7f3e19b41700 1 -- 172.21.15.12:0/4140611726 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a94ab7d9f0 msgr2=0x55a94ad052b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:44:33.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.316+0000 7f3e19b41700 1 --2- 172.21.15.12:0/4140611726 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a94ab7d9f0 0x55a94ad052b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:33.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.316+0000 7f3e19b41700 1 -- 172.21.15.12:0/4140611726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a94acfc170 msgr2=0x55a94ad057f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:44:33.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.316+0000 7f3e19b41700 1 --2- 172.21.15.12:0/4140611726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a94acfc170 0x55a94ad057f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:33.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.316+0000 7f3e19b41700 1 -- 172.21.15.12:0/4140611726 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a94ab6a480 con 0x55a94ad01cb0 2024-05-17T22:44:33.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.317+0000 7f3e19b41700 1 --2- 172.21.15.12:0/4140611726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a94ad01cb0 0x55a94ad05d30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c00ef90 tx=0x7f3e0c0a4150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4336a28062f6d86 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:44:33.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.318+0000 7f3dfcff9700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:44:33.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.320+0000 7f3dfcff9700 1 -- 172.21.15.12:0/4140611726 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c0a8940 con 0x55a94ad01cb0 2024-05-17T22:44:33.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.320+0000 7f3e29f7fb80 1 -- 172.21.15.12:0/4140611726 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e1000f8a0 con 0x55a94ad01cb0 2024-05-17T22:44:33.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.320+0000 7f3dfcff9700 1 -- 172.21.15.12:0/4140611726 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f3e0c0cb070 con 0x55a94ad01cb0 2024-05-17T22:44:33.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.322+0000 7f3dfcff9700 1 -- 172.21.15.12:0/4140611726 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3e0c0cb270 con 0x55a94ad01cb0 2024-05-17T22:44:33.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.322+0000 7f3e29f7fb80 10 client.25460 fetch_fsmap learned FSMap version 33 2024-05-17T22:44:33.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.322+0000 7f3e29f7fb80 1 -- 172.21.15.12:0/4140611726 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a94ad12ee0 con 0x55a94ad01cb0 2024-05-17T22:44:33.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.328+0000 7f3dfcff9700 1 -- 172.21.15.12:0/4140611726 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e1000f8a0 con 0x55a94ad01cb0 2024-05-17T22:44:33.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.328+0000 7f3dfcff9700 1 -- 172.21.15.12:0/4140611726 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f3e0c0b1310 con 0x55a94ad01cb0 2024-05-17T22:44:33.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.328+0000 7f3e29f7fb80 10 client.25460 fetch_fsmap finished waiting for FSMap version 33 2024-05-17T22:44:33.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.328+0000 7f3e29f7fb80 10 client.25460 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-17T22:44:33.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.329+0000 7f3e29f7fb80 20 client.25460 populate_metadata read hostname 'smithi012' 2024-05-17T22:44:33.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.329+0000 7f3e29f7fb80 1 --2- 172.21.15.12:0/4140611726 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55a94ad15540 0x55a94ad179d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:33.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.329+0000 7f3e29f7fb80 4 client.25460 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:44:33.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.329+0000 7f3e29f7fb80 1 -- 172.21.15.12:0/4140611726 --> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a94ac2c1e0 con 0x55a94ad15540 2024-05-17T22:44:33.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.330+0000 7f3e19340700 1 --2- 172.21.15.12:0/4140611726 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55a94ad15540 0x55a94ad179d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:33.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.352+0000 7f3e19340700 1 --2- 172.21.15.12:0/4140611726 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55a94ad15540 0x55a94ad179d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ac3cd1d268f61c36 server_cookie=db7d4e9c9176405e in_seq=0 out_seq=0 2024-05-17T22:44:33.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.352+0000 7f3dfcff9700 10 client.25460 ms_handle_connect on v2:172.21.15.96:6836/4222574338 2024-05-17T22:44:33.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.354+0000 7f3dfcff9700 1 -- 172.21.15.12:0/4140611726 <== mds.1 v2:172.21.15.96:6836/4222574338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a94ac2c1e0 con 0x55a94ad15540 2024-05-17T22:44:33.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.354+0000 7f3dfcff9700 10 client.25460 handle_command_reply: tid=0 2024-05-17T22:44:33.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.357+0000 7f3e29f7fb80 1 client.25460 shutdown 2024-05-17T22:44:33.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.357+0000 7f3e29f7fb80 1 -- 172.21.15.12:0/4140611726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a94ad01cb0 msgr2=0x55a94ad05d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:33.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.357+0000 7f3e29f7fb80 1 --2- 172.21.15.12:0/4140611726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a94ad01cb0 0x55a94ad05d30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c00ef90 tx=0x7f3e0c0a4150 comp rx=0 tx=0).stop 2024-05-17T22:44:33.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.358+0000 7f3e29f7fb80 1 -- 172.21.15.12:0/4140611726 shutdown_connections 2024-05-17T22:44:33.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.358+0000 7f3e29f7fb80 1 --2- 172.21.15.12:0/4140611726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a94ad01cb0 0x55a94ad05d30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:33.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.358+0000 7f3e29f7fb80 1 --2- 172.21.15.12:0/4140611726 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a94ab7d9f0 0x55a94ad052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:33.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.358+0000 7f3e29f7fb80 1 --2- 172.21.15.12:0/4140611726 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55a94ad15540 0x55a94ad179d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:33.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.358+0000 7f3e29f7fb80 1 --2- 172.21.15.12:0/4140611726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a94acfc170 0x55a94ad057f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:33.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.358+0000 7f3e29f7fb80 1 -- 172.21.15.12:0/4140611726 >> 172.21.15.12:0/4140611726 conn(0x55a94abc1f00 msgr2=0x55a94acf6110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:44:33.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.358+0000 7f3dfcff9700 0 client.25460 ms_handle_reset on v2:172.21.15.96:6836/4222574338 2024-05-17T22:44:33.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.361+0000 7f3e29f7fb80 1 -- 172.21.15.12:0/4140611726 shutdown_connections 2024-05-17T22:44:33.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.361+0000 7f3e29f7fb80 1 -- 172.21.15.12:0/4140611726 wait complete. 2024-05-17T22:44:33.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.361+0000 7f3e29f7fb80 20 client.25460 trim_cache size 0 max 16384 2024-05-17T22:44:33.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.395+0000 7f3e29f7fb80 1 Processor -- start 2024-05-17T22:44:33.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.395+0000 7f3e29f7fb80 1 -- start start 2024-05-17T22:44:33.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.395+0000 7f3e29f7fb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a94ab7d9f0 0x55a94acfbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:33.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.395+0000 7f3e29f7fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a94acfc170 0x55a94ad00c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:33.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.395+0000 7f3e29f7fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a94ad01cb0 0x55a94ad11350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:33.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.395+0000 7f3e29f7fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3e141c99b0 con 0x55a94ad01cb0 2024-05-17T22:44:33.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.395+0000 7f3e29f7fb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3e141c71e0 con 0x55a94ab7d9f0 2024-05-17T22:44:33.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.395+0000 7f3e29f7fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3e141c5030 con 0x55a94acfc170 2024-05-17T22:44:33.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.395+0000 7f3e19b41700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a94ad01cb0 0x55a94ad11350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:33.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.395+0000 7f3e19b41700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a94ad01cb0 0x55a94ad11350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38130/0 (socket says 172.21.15.12:38130) 2024-05-17T22:44:33.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.395+0000 7f3e19b41700 1 -- 172.21.15.12:0/2666098321 learned_addr learned my addr 172.21.15.12:0/2666098321 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:44:33.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.396+0000 7f3e19b41700 1 -- 172.21.15.12:0/2666098321 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a94acfc170 msgr2=0x55a94ad00c50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T22:44:33.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.396+0000 7f3e19b41700 1 --2- 172.21.15.12:0/2666098321 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a94acfc170 0x55a94ad00c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:33.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.396+0000 7f3e19b41700 1 -- 172.21.15.12:0/2666098321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a94ab7d9f0 msgr2=0x55a94acfbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:44:33.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.396+0000 7f3e19b41700 1 --2- 172.21.15.12:0/2666098321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a94ab7d9f0 0x55a94acfbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:33.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.396+0000 7f3e19b41700 1 -- 172.21.15.12:0/2666098321 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a94ad08400 con 0x55a94ad01cb0 2024-05-17T22:44:33.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.396+0000 7f3e19b41700 1 --2- 172.21.15.12:0/2666098321 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a94ad01cb0 0x55a94ad11350 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c0a4240 tx=0x7f3e0c0a8160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd3504fa12f3be8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:44:33.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.400+0000 7f3dfeffd700 1 -- 172.21.15.12:0/2666098321 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c0b6080 con 0x55a94ad01cb0 2024-05-17T22:44:33.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.400+0000 7f3dfeffd700 1 -- 172.21.15.12:0/2666098321 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e0c0a8940 con 0x55a94ad01cb0 2024-05-17T22:44:33.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.400+0000 7f3dfeffd700 1 -- 172.21.15.12:0/2666098321 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c0a8ba0 con 0x55a94ad01cb0 2024-05-17T22:44:33.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.402+0000 7f3e29f7fb80 1 -- 172.21.15.12:0/2666098321 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a94ad01cb0 msgr2=0x55a94ad11350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:33.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.402+0000 7f3e29f7fb80 1 --2- 172.21.15.12:0/2666098321 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a94ad01cb0 0x55a94ad11350 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c0a4240 tx=0x7f3e0c0a8160 comp rx=0 tx=0).stop 2024-05-17T22:44:33.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.402+0000 7f3e29f7fb80 1 -- 172.21.15.12:0/2666098321 shutdown_connections 2024-05-17T22:44:33.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.402+0000 7f3e29f7fb80 1 --2- 172.21.15.12:0/2666098321 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a94ad01cb0 0x55a94ad11350 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:33.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.402+0000 7f3e29f7fb80 1 --2- 172.21.15.12:0/2666098321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a94ab7d9f0 0x55a94acfbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:33.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.402+0000 7f3e29f7fb80 1 --2- 172.21.15.12:0/2666098321 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a94acfc170 0x55a94ad00c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:33.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.403+0000 7f3e29f7fb80 1 -- 172.21.15.12:0/2666098321 >> 172.21.15.12:0/2666098321 conn(0x55a94abc1f00 msgr2=0x55a94acff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:44:33.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.405+0000 7f3e29f7fb80 1 -- 172.21.15.12:0/2666098321 shutdown_connections 2024-05-17T22:44:33.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.405+0000 7f3e29f7fb80 1 -- 172.21.15.12:0/2666098321 wait complete. 2024-05-17T22:44:33.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.406+0000 7f3e29f7fb80 1 -- start start 2024-05-17T22:44:33.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.406+0000 7f3e29f7fb80 1 Processor -- start 2024-05-17T22:44:33.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.408+0000 7f3e29f7fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a94ab7d9f0 0x55a94ad0dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:33.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.408+0000 7f3e29f7fb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a94acfc170 0x55a94ad1c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:33.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.408+0000 7f3e29f7fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a94ad01cb0 0x55a94ad1cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:33.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.408+0000 7f3e29f7fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3e141c56a0 con 0x55a94ab7d9f0 2024-05-17T22:44:33.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.408+0000 7f3e18b3f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a94acfc170 0x55a94ad1c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:33.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.408+0000 7f3e18b3f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a94acfc170 0x55a94ad1c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:60566/0 (socket says 172.21.15.12:60566) 2024-05-17T22:44:33.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.408+0000 7f3e18b3f700 1 -- 172.21.15.12:0/1903217797 learned_addr learned my addr 172.21.15.12:0/1903217797 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:44:33.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.409+0000 7f3e19b41700 1 --2- 172.21.15.12:0/1903217797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a94ad01cb0 0x55a94ad1cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:33.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.409+0000 7f3e29f7fb80 1 -- 172.21.15.12:0/1903217797 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3e141c6b10 con 0x55a94acfc170 2024-05-17T22:44:33.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.409+0000 7f3e29f7fb80 1 -- 172.21.15.12:0/1903217797 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3e141c5d70 con 0x55a94ad01cb0 2024-05-17T22:44:33.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.409+0000 7f3e29f7fb80 10 client.0 _ll_register_callbacks cb 0x55a94abb4f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:44:33.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.410+0000 7f3e19b41700 1 -- 172.21.15.12:0/1903217797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a94acfc170 msgr2=0x55a94ad1c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:44:33.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.410+0000 7f3e19b41700 1 --2- 172.21.15.12:0/1903217797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a94acfc170 0x55a94ad1c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:33.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.410+0000 7f3e19b41700 1 -- 172.21.15.12:0/1903217797 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a94ab7d9f0 msgr2=0x55a94ad0dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:44:33.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.410+0000 7f3e19b41700 1 --2- 172.21.15.12:0/1903217797 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a94ab7d9f0 0x55a94ad0dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:33.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.410+0000 7f3e19b41700 1 -- 172.21.15.12:0/1903217797 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a94ad1d3e0 con 0x55a94ad01cb0 2024-05-17T22:44:33.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.414+0000 7f3e19b41700 1 --2- 172.21.15.12:0/1903217797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a94ad01cb0 0x55a94ad1cc60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c0a4240 tx=0x7f3e0c0a8dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c6f8b2a4c75589b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:44:33.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.414+0000 7f3dfcff9700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:44:33.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.414+0000 7f3dfcff9700 1 -- 172.21.15.12:0/1903217797 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c0cb0b0 con 0x55a94ad01cb0 2024-05-17T22:44:33.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.414+0000 7f3dfcff9700 1 -- 172.21.15.12:0/1903217797 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f3e0c0a4f30 con 0x55a94ad01cb0 2024-05-17T22:44:33.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.414+0000 7f3e29f7fb80 1 -- 172.21.15.12:0/1903217797 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a94ac259c0 con 0x55a94ad01cb0 2024-05-17T22:44:33.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.415+0000 7f3dfcff9700 1 -- 172.21.15.12:0/1903217797 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3e0c09d050 con 0x55a94ad01cb0 2024-05-17T22:44:33.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.415+0000 7f3e29f7fb80 10 client.25411 fetch_fsmap learned FSMap version 33 2024-05-17T22:44:33.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.415+0000 7f3e29f7fb80 1 -- 172.21.15.12:0/1903217797 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a94ad7fc50 con 0x55a94ad01cb0 2024-05-17T22:44:33.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.418+0000 7f3dfcff9700 1 -- 172.21.15.12:0/1903217797 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a94ac259c0 con 0x55a94ad01cb0 2024-05-17T22:44:33.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.418+0000 7f3dfcff9700 1 -- 172.21.15.12:0/1903217797 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f3e0c0ae020 con 0x55a94ad01cb0 2024-05-17T22:44:33.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.418+0000 7f3e29f7fb80 10 client.25411 fetch_fsmap finished waiting for FSMap version 33 2024-05-17T22:44:33.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.418+0000 7f3e29f7fb80 10 client.25411 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-17T22:44:33.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.418+0000 7f3e29f7fb80 20 client.25411 populate_metadata read hostname 'smithi012' 2024-05-17T22:44:33.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.418+0000 7f3e29f7fb80 1 --2- 172.21.15.12:0/1903217797 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55a94ad80070 0x55a94ad82490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:33.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.418+0000 7f3e29f7fb80 4 client.25411 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:44:33.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.418+0000 7f3e29f7fb80 1 -- 172.21.15.12:0/1903217797 --> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a94aaf6360 con 0x55a94ad80070 2024-05-17T22:44:33.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.422+0000 7f3e19340700 1 --2- 172.21.15.12:0/1903217797 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55a94ad80070 0x55a94ad82490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:33.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.422+0000 7f3e19340700 1 --2- 172.21.15.12:0/1903217797 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55a94ad80070 0x55a94ad82490 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4f0441bb84df4a59 server_cookie=8fd2d4c575d4f0a9 in_seq=0 out_seq=0 2024-05-17T22:44:33.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.422+0000 7f3dfcff9700 10 client.25411 ms_handle_connect on v2:172.21.15.96:6836/4222574338 2024-05-17T22:44:33.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.422+0000 7f3dfcff9700 1 -- 172.21.15.12:0/1903217797 <== mds.1 v2:172.21.15.96:6836/4222574338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a94aaf6360 con 0x55a94ad80070 2024-05-17T22:44:33.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.423+0000 7f3dfcff9700 10 client.25411 handle_command_reply: tid=0 2024-05-17T22:44:33.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.426+0000 7f3e29f7fb80 1 client.25411 shutdown 2024-05-17T22:44:33.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.433+0000 7f3e29f7fb80 1 -- 172.21.15.12:0/1903217797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a94ad01cb0 msgr2=0x55a94ad1cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:33.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.433+0000 7f3e29f7fb80 1 --2- 172.21.15.12:0/1903217797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a94ad01cb0 0x55a94ad1cc60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c0a4240 tx=0x7f3e0c0a8dd0 comp rx=0 tx=0).stop 2024-05-17T22:44:33.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.434+0000 7f3e29f7fb80 1 -- 172.21.15.12:0/1903217797 shutdown_connections 2024-05-17T22:44:33.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.434+0000 7f3e29f7fb80 1 --2- 172.21.15.12:0/1903217797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a94ad01cb0 0x55a94ad1cc60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:33.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.434+0000 7f3e29f7fb80 1 --2- 172.21.15.12:0/1903217797 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55a94ad80070 0x55a94ad82490 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:33.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.434+0000 7f3e29f7fb80 1 --2- 172.21.15.12:0/1903217797 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a94ab7d9f0 0x55a94ad0dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:33.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.434+0000 7f3e29f7fb80 1 --2- 172.21.15.12:0/1903217797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a94acfc170 0x55a94ad1c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:33.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.434+0000 7f3e29f7fb80 1 -- 172.21.15.12:0/1903217797 >> 172.21.15.12:0/1903217797 conn(0x55a94abc1f00 msgr2=0x55a94acff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:44:33.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.434+0000 7f3dfcff9700 0 client.25411 ms_handle_reset on v2:172.21.15.96:6836/4222574338 2024-05-17T22:44:33.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.436+0000 7f3e29f7fb80 1 -- 172.21.15.12:0/1903217797 shutdown_connections 2024-05-17T22:44:33.441 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:44:33.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.438+0000 7f3e29f7fb80 1 -- 172.21.15.12:0/1903217797 wait complete. 2024-05-17T22:44:33.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.438+0000 7f3e29f7fb80 20 client.25411 trim_cache size 0 max 16384 2024-05-17T22:44:33.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.440+0000 7f3e1a342700 1 -- 172.21.15.12:0/3873339883 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f3e000b28a0 msgr2=0x7f3e000b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:33.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.440+0000 7f3e1a342700 1 --2- 172.21.15.12:0/3873339883 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f3e000b28a0 0x7f3e000b4db0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55a94acf7ae0 tx=0x7f3e04009260 comp rx=0 tx=0).stop 2024-05-17T22:44:33.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.440+0000 7f3e1a342700 1 -- 172.21.15.12:0/3873339883 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e1411d010 msgr2=0x7f3e14115220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:33.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.440+0000 7f3e1a342700 1 --2- 172.21.15.12:0/3873339883 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e1411d010 0x7f3e14115220 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c00efc0 tx=0x7f3e0c00c3a0 comp rx=0 tx=0).stop 2024-05-17T22:44:33.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.441+0000 7f3e1a342700 1 -- 172.21.15.12:0/3873339883 shutdown_connections 2024-05-17T22:44:33.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.441+0000 7f3e1a342700 1 --2- 172.21.15.12:0/3873339883 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f3e000b28a0 0x7f3e000b4db0 secure :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55a94acf7ae0 tx=0x7f3e04009260 comp rx=0 tx=0).stop 2024-05-17T22:44:33.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.441+0000 7f3e1a342700 1 --2- 172.21.15.12:0/3873339883 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e1411d010 0x7f3e14115220 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:33.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.441+0000 7f3e1a342700 1 --2- 172.21.15.12:0/3873339883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e1410afc0 0x7f3e1411ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:33.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.441+0000 7f3e1a342700 1 --2- 172.21.15.12:0/3873339883 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e14112230 0x7f3e14114b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:33.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.441+0000 7f3e1a342700 1 -- 172.21.15.12:0/3873339883 >> 172.21.15.12:0/3873339883 conn(0x7f3e1406eda0 msgr2=0x7f3e14072020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:44:33.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.442+0000 7f3e1a342700 1 -- 172.21.15.12:0/3873339883 shutdown_connections 2024-05-17T22:44:33.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:33.443+0000 7f3e1a342700 1 -- 172.21.15.12:0/3873339883 wait complete. 2024-05-17T22:44:34.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:33 smithi096 ceph-mon[36065]: pgmap v598: 129 pgs: 129 active+clean; 1.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 626 op/s 2024-05-17T22:44:34.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:33 smithi012 ceph-mon[31632]: pgmap v598: 129 pgs: 129 active+clean; 1.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 626 op/s 2024-05-17T22:44:34.181 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-17T22:44:34.192 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:33 smithi151 ceph-mon[36101]: pgmap v598: 129 pgs: 129 active+clean; 1.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 626 op/s 2024-05-17T22:44:34.707 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:44:36.030 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:35 smithi012 ceph-mon[31632]: pgmap v599: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 MiB/s wr, 689 op/s 2024-05-17T22:44:36.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:35 smithi096 ceph-mon[36065]: pgmap v599: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 MiB/s wr, 689 op/s 2024-05-17T22:44:36.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:35 smithi151 ceph-mon[36101]: pgmap v599: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 MiB/s wr, 689 op/s 2024-05-17T22:44:36.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 315.0 s, 654.8 tps, lat 148.732 ms stddev 180.733 2024-05-17T22:44:36.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.731+0000 7f136ebe9700 1 -- 172.21.15.12:0/3170452556 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1368101ed0 msgr2=0x7f13681022e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:36.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.731+0000 7f136ebe9700 1 --2- 172.21.15.12:0/3170452556 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1368101ed0 0x7f13681022e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f1358009990 tx=0x7f1358009ca0 comp rx=0 tx=0).stop 2024-05-17T22:44:36.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.731+0000 7f136ebe9700 1 -- 172.21.15.12:0/3170452556 shutdown_connections 2024-05-17T22:44:36.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.731+0000 7f136ebe9700 1 --2- 172.21.15.12:0/3170452556 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1368102820 0x7f1368107660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:36.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.731+0000 7f136ebe9700 1 --2- 172.21.15.12:0/3170452556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1368108990 0x7f1368108da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:36.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.731+0000 7f136ebe9700 1 --2- 172.21.15.12:0/3170452556 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1368101ed0 0x7f13681022e0 secure :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f1358009990 tx=0x7f1358009ca0 comp rx=0 tx=0).stop 2024-05-17T22:44:36.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.731+0000 7f136ebe9700 1 -- 172.21.15.12:0/3170452556 >> 172.21.15.12:0/3170452556 conn(0x7f13680fdf80 msgr2=0x7f13681003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:44:36.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.733+0000 7f136ebe9700 1 -- 172.21.15.12:0/3170452556 shutdown_connections 2024-05-17T22:44:36.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.733+0000 7f136ebe9700 1 -- 172.21.15.12:0/3170452556 wait complete. 2024-05-17T22:44:36.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.734+0000 7f136ebe9700 1 Processor -- start 2024-05-17T22:44:36.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.734+0000 7f136ebe9700 1 -- start start 2024-05-17T22:44:36.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.734+0000 7f136ebe9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1368102820 0x7f136806f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:36.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.734+0000 7f136ebe9700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1368108990 0x7f136806fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:36.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.734+0000 7f136ebe9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1368070070 0x7f13681b5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:36.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.734+0000 7f136ebe9700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f136806d9b0 con 0x7f1368070070 2024-05-17T22:44:36.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.734+0000 7f136ebe9700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f136806d830 con 0x7f1368102820 2024-05-17T22:44:36.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.734+0000 7f136ebe9700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f136806db30 con 0x7f1368108990 2024-05-17T22:44:36.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.734+0000 7f136dbe7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1368102820 0x7f136806f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:36.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.735+0000 7f136e3e8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1368070070 0x7f13681b5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:36.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.735+0000 7f136dbe7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1368102820 0x7f136806f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:60578/0 (socket says 172.21.15.12:60578) 2024-05-17T22:44:36.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.735+0000 7f136dbe7700 1 -- 172.21.15.12:0/3273609909 learned_addr learned my addr 172.21.15.12:0/3273609909 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:44:36.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.735+0000 7f136d3e6700 1 --2- 172.21.15.12:0/3273609909 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1368108990 0x7f136806fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:36.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.735+0000 7f136dbe7700 1 -- 172.21.15.12:0/3273609909 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1368108990 msgr2=0x7f136806fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:36.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.735+0000 7f136dbe7700 1 --2- 172.21.15.12:0/3273609909 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1368108990 0x7f136806fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:36.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.735+0000 7f136dbe7700 1 -- 172.21.15.12:0/3273609909 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1368070070 msgr2=0x7f13681b5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:36.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.735+0000 7f136dbe7700 1 --2- 172.21.15.12:0/3273609909 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1368070070 0x7f13681b5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:36.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.735+0000 7f136dbe7700 1 -- 172.21.15.12:0/3273609909 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13681b5db0 con 0x7f1368102820 2024-05-17T22:44:36.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.735+0000 7f136dbe7700 1 --2- 172.21.15.12:0/3273609909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1368102820 0x7f136806f5f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f136400c920 tx=0x7f136400cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:44:36.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.735+0000 7f135effd700 1 -- 172.21.15.12:0/3273609909 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1364007b00 con 0x7f1368102820 2024-05-17T22:44:36.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.735+0000 7f136ebe9700 1 -- 172.21.15.12:0/3273609909 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13681b5f80 con 0x7f1368102820 2024-05-17T22:44:36.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.736+0000 7f136ebe9700 1 -- 172.21.15.12:0/3273609909 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13681b63d0 con 0x7f1368102820 2024-05-17T22:44:36.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.736+0000 7f135effd700 1 -- 172.21.15.12:0/3273609909 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1364004ce0 con 0x7f1368102820 2024-05-17T22:44:36.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.736+0000 7f135effd700 1 -- 172.21.15.12:0/3273609909 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1364005500 con 0x7f1368102820 2024-05-17T22:44:36.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.738+0000 7f135effd700 1 -- 172.21.15.12:0/3273609909 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f136400f450 con 0x7f1368102820 2024-05-17T22:44:36.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.739+0000 7f135effd700 1 --2- 172.21.15.12:0/3273609909 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f13540b2900 0x7f13540b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:36.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.739+0000 7f135effd700 1 -- 172.21.15.12:0/3273609909 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f13640e2150 con 0x7f1368102820 2024-05-17T22:44:36.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.740+0000 7f136d3e6700 1 --2- 172.21.15.12:0/3273609909 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f13540b2900 0x7f13540b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:36.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.740+0000 7f137e826b80 1 Processor -- start 2024-05-17T22:44:36.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.740+0000 7f137e826b80 1 -- start start 2024-05-17T22:44:36.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.740+0000 7f136d3e6700 1 --2- 172.21.15.12:0/3273609909 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f13540b2900 0x7f13540b4e10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1358008000 tx=0x7f1358005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:44:36.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.741+0000 7f137e826b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e317899f0 0x555e3190cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:36.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.741+0000 7f137e826b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e3190d510 0x555e319075c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:36.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.741+0000 7f137e826b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e31907b00 0x555e31907f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:36.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.741+0000 7f137e826b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x555e318360d0 con 0x555e3190d510 2024-05-17T22:44:36.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.741+0000 7f137e826b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555e31825c70 con 0x555e317899f0 2024-05-17T22:44:36.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.741+0000 7f137e826b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555e31829cb0 con 0x555e31907b00 2024-05-17T22:44:36.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.741+0000 7f136d3e6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e3190d510 0x555e319075c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:36.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.741+0000 7f136dbe7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e317899f0 0x555e3190cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:36.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.741+0000 7f136dbe7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e317899f0 0x555e3190cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:60584/0 (socket says 172.21.15.12:60584) 2024-05-17T22:44:36.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.741+0000 7f136dbe7700 1 -- 172.21.15.12:0/979929066 learned_addr learned my addr 172.21.15.12:0/979929066 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:44:36.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.741+0000 7f136e3e8700 1 --2- 172.21.15.12:0/979929066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e31907b00 0x555e31907f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:36.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.741+0000 7f136dbe7700 1 -- 172.21.15.12:0/979929066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e31907b00 msgr2=0x555e31907f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:44:36.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.741+0000 7f136dbe7700 1 --2- 172.21.15.12:0/979929066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e31907b00 0x555e31907f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:36.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.741+0000 7f136dbe7700 1 -- 172.21.15.12:0/979929066 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e3190d510 msgr2=0x555e319075c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:44:36.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.741+0000 7f136dbe7700 1 --2- 172.21.15.12:0/979929066 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e3190d510 0x555e319075c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:36.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.741+0000 7f136dbe7700 1 -- 172.21.15.12:0/979929066 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e317bf5c0 con 0x555e317899f0 2024-05-17T22:44:36.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.741+0000 7f136d3e6700 1 --2- 172.21.15.12:0/979929066 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e3190d510 0x555e319075c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:44:36.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.741+0000 7f136dbe7700 1 --2- 172.21.15.12:0/979929066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e317899f0 0x555e3190cfd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f13640a9fa0 tx=0x7f1364007d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a676f94a7ff7e6fa server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:44:36.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.741+0000 7f1353fff700 1 -- 172.21.15.12:0/979929066 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136401d030 con 0x555e317899f0 2024-05-17T22:44:36.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.742+0000 7f1353fff700 1 -- 172.21.15.12:0/979929066 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13640b1070 con 0x555e317899f0 2024-05-17T22:44:36.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.742+0000 7f1353fff700 1 -- 172.21.15.12:0/979929066 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13640a9700 con 0x555e317899f0 2024-05-17T22:44:36.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.742+0000 7f137e826b80 1 -- 172.21.15.12:0/979929066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e317899f0 msgr2=0x555e3190cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:36.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.742+0000 7f137e826b80 1 --2- 172.21.15.12:0/979929066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e317899f0 0x555e3190cfd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f13640a9fa0 tx=0x7f1364007d20 comp rx=0 tx=0).stop 2024-05-17T22:44:36.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.743+0000 7f137e826b80 1 -- 172.21.15.12:0/979929066 shutdown_connections 2024-05-17T22:44:36.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.743+0000 7f137e826b80 1 --2- 172.21.15.12:0/979929066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e31907b00 0x555e31907f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:36.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.743+0000 7f137e826b80 1 --2- 172.21.15.12:0/979929066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e317899f0 0x555e3190cfd0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:36.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.743+0000 7f137e826b80 1 --2- 172.21.15.12:0/979929066 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e3190d510 0x555e319075c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:36.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.743+0000 7f137e826b80 1 -- 172.21.15.12:0/979929066 >> 172.21.15.12:0/979929066 conn(0x555e317e97d0 msgr2=0x555e319021a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:44:36.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.743+0000 7f137e826b80 1 -- 172.21.15.12:0/979929066 shutdown_connections 2024-05-17T22:44:36.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.743+0000 7f137e826b80 1 -- 172.21.15.12:0/979929066 wait complete. 2024-05-17T22:44:36.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.743+0000 7f137e826b80 1 -- start start 2024-05-17T22:44:36.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.744+0000 7f137e826b80 1 Processor -- start 2024-05-17T22:44:36.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.745+0000 7f137e826b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e317899f0 0x555e31910d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:36.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.745+0000 7f137e826b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e31907b00 0x555e31911250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:36.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.745+0000 7f137e826b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e3190d510 0x555e3191d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:36.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.745+0000 7f137e826b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x555e3172d150 con 0x555e31907b00 2024-05-17T22:44:36.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.745+0000 7f137e826b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555e317c7940 con 0x555e3190d510 2024-05-17T22:44:36.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.745+0000 7f137e826b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555e31913ec0 con 0x555e317899f0 2024-05-17T22:44:36.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.745+0000 7f136d3e6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e31907b00 0x555e31911250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:36.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.745+0000 7f136dbe7700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e317899f0 0x555e31910d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:36.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.745+0000 7f136e3e8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e3190d510 0x555e3191d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:36.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.745+0000 7f136dbe7700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e317899f0 0x555e31910d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:54308/0 (socket says 172.21.15.12:54308) 2024-05-17T22:44:36.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.745+0000 7f136d3e6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e31907b00 0x555e31911250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38164/0 (socket says 172.21.15.12:38164) 2024-05-17T22:44:36.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.745+0000 7f136dbe7700 1 -- 172.21.15.12:0/2430945773 learned_addr learned my addr 172.21.15.12:0/2430945773 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:44:36.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.745+0000 7f137e826b80 10 client.0 _ll_register_callbacks cb 0x555e317bcb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:44:36.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.745+0000 7f136e3e8700 1 -- 172.21.15.12:0/2430945773 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e317899f0 msgr2=0x555e31910d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:44:36.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.745+0000 7f136e3e8700 1 --2- 172.21.15.12:0/2430945773 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e317899f0 0x555e31910d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:36.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.745+0000 7f136e3e8700 1 -- 172.21.15.12:0/2430945773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e31907b00 msgr2=0x555e31911250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:44:36.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.745+0000 7f136e3e8700 1 --2- 172.21.15.12:0/2430945773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e31907b00 0x555e31911250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:36.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.745+0000 7f136e3e8700 1 -- 172.21.15.12:0/2430945773 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e31776480 con 0x555e3190d510 2024-05-17T22:44:36.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.746+0000 7f136dbe7700 1 --2- 172.21.15.12:0/2430945773 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e317899f0 0x555e31910d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:44:36.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.746+0000 7f136e3e8700 1 --2- 172.21.15.12:0/2430945773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e3190d510 0x555e3191d1b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f136000a900 tx=0x7f136000a930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff686822417b9998 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:44:36.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.746+0000 7f13517fa700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:44:36.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.746+0000 7f13517fa700 1 -- 172.21.15.12:0/2430945773 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1360015780 con 0x555e3190d510 2024-05-17T22:44:36.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.746+0000 7f13517fa700 1 -- 172.21.15.12:0/2430945773 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f1360015920 con 0x555e3190d510 2024-05-17T22:44:36.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.746+0000 7f137e826b80 1 -- 172.21.15.12:0/2430945773 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13640b1070 con 0x555e3190d510 2024-05-17T22:44:36.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.746+0000 7f13517fa700 1 -- 172.21.15.12:0/2430945773 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1360015b20 con 0x555e3190d510 2024-05-17T22:44:36.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.746+0000 7f137e826b80 10 client.25472 fetch_fsmap learned FSMap version 34 2024-05-17T22:44:36.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.746+0000 7f137e826b80 1 -- 172.21.15.12:0/2430945773 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555e3191ea30 con 0x555e3190d510 2024-05-17T22:44:36.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.746+0000 7f13517fa700 1 -- 172.21.15.12:0/2430945773 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13640b1070 con 0x555e3190d510 2024-05-17T22:44:36.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.747+0000 7f13517fa700 1 -- 172.21.15.12:0/2430945773 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f136001f3b0 con 0x555e3190d510 2024-05-17T22:44:36.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.747+0000 7f137e826b80 10 client.25472 fetch_fsmap finished waiting for FSMap version 34 2024-05-17T22:44:36.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.747+0000 7f137e826b80 10 client.25472 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:44:36.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.747+0000 7f137e826b80 20 client.25472 populate_metadata read hostname 'smithi012' 2024-05-17T22:44:36.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.747+0000 7f137e826b80 1 --2- 172.21.15.12:0/2430945773 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555e31921470 0x555e31923900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:36.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.747+0000 7f137e826b80 4 client.25472 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:44:36.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.747+0000 7f137e826b80 1 -- 172.21.15.12:0/2430945773 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e31838020 con 0x555e31921470 2024-05-17T22:44:36.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.747+0000 7f136dbe7700 1 --2- 172.21.15.12:0/2430945773 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555e31921470 0x555e31923900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:36.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.747+0000 7f13517fa700 10 client.25472 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:44:36.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.748+0000 7f136dbe7700 1 --2- 172.21.15.12:0/2430945773 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555e31921470 0x555e31923900 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec096bb60cdb08dc server_cookie=a4c812078311c2e2 in_seq=0 out_seq=0 2024-05-17T22:44:36.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.748+0000 7f13517fa700 1 -- 172.21.15.12:0/2430945773 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555e31838020 con 0x555e31921470 2024-05-17T22:44:36.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.748+0000 7f13517fa700 10 client.25472 handle_command_reply: tid=0 2024-05-17T22:44:36.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.748+0000 7f137e826b80 1 client.25472 shutdown 2024-05-17T22:44:36.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.749+0000 7f137e826b80 1 -- 172.21.15.12:0/2430945773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e3190d510 msgr2=0x555e3191d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:36.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.749+0000 7f137e826b80 1 --2- 172.21.15.12:0/2430945773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e3190d510 0x555e3191d1b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f136000a900 tx=0x7f136000a930 comp rx=0 tx=0).stop 2024-05-17T22:44:36.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.749+0000 7f137e826b80 1 -- 172.21.15.12:0/2430945773 shutdown_connections 2024-05-17T22:44:36.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.749+0000 7f137e826b80 1 --2- 172.21.15.12:0/2430945773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e3190d510 0x555e3191d1b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:36.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.749+0000 7f137e826b80 1 --2- 172.21.15.12:0/2430945773 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e317899f0 0x555e31910d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:36.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.749+0000 7f137e826b80 1 --2- 172.21.15.12:0/2430945773 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555e31921470 0x555e31923900 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:36.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.749+0000 7f137e826b80 1 --2- 172.21.15.12:0/2430945773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e31907b00 0x555e31911250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:36.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.749+0000 7f137e826b80 1 -- 172.21.15.12:0/2430945773 >> 172.21.15.12:0/2430945773 conn(0x555e317e97d0 msgr2=0x555e31902140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:44:36.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.749+0000 7f13517fa700 0 client.25472 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:44:36.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.750+0000 7f137e826b80 1 -- 172.21.15.12:0/2430945773 shutdown_connections 2024-05-17T22:44:36.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.750+0000 7f137e826b80 1 -- 172.21.15.12:0/2430945773 wait complete. 2024-05-17T22:44:36.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.750+0000 7f137e826b80 20 client.25472 trim_cache size 0 max 16384 2024-05-17T22:44:36.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.774+0000 7f137e826b80 1 Processor -- start 2024-05-17T22:44:36.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.774+0000 7f137e826b80 1 -- start start 2024-05-17T22:44:36.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.774+0000 7f137e826b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e317e97d0 0x555e3190c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:36.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.774+0000 7f137e826b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e317899f0 0x555e319115e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:36.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.774+0000 7f136dbe7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e317e97d0 0x555e3190c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:36.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.774+0000 7f136dbe7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e317e97d0 0x555e3190c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38176/0 (socket says 172.21.15.12:38176) 2024-05-17T22:44:36.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.775+0000 7f136dbe7700 1 -- 172.21.15.12:0/1100987320 learned_addr learned my addr 172.21.15.12:0/1100987320 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:44:36.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.775+0000 7f137e826b80 1 --2- 172.21.15.12:0/1100987320 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e31907b00 0x555e31929100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:36.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.775+0000 7f137e826b80 1 -- 172.21.15.12:0/1100987320 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f13681c1aa0 con 0x555e317e97d0 2024-05-17T22:44:36.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.775+0000 7f137e826b80 1 -- 172.21.15.12:0/1100987320 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f13681bf300 con 0x555e317899f0 2024-05-17T22:44:36.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.775+0000 7f137e826b80 1 -- 172.21.15.12:0/1100987320 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f13681bd150 con 0x555e31907b00 2024-05-17T22:44:36.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.775+0000 7f136d3e6700 1 --2- 172.21.15.12:0/1100987320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e317899f0 0x555e319115e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:36.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.775+0000 7f136e3e8700 1 --2- 172.21.15.12:0/1100987320 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e31907b00 0x555e31929100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:36.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.775+0000 7f136d3e6700 1 -- 172.21.15.12:0/1100987320 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e31907b00 msgr2=0x555e31929100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:44:36.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.775+0000 7f136d3e6700 1 --2- 172.21.15.12:0/1100987320 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e31907b00 0x555e31929100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:36.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.775+0000 7f136d3e6700 1 -- 172.21.15.12:0/1100987320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e317e97d0 msgr2=0x555e3190c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:44:36.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.775+0000 7f136d3e6700 1 --2- 172.21.15.12:0/1100987320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e317e97d0 0x555e3190c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:36.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.775+0000 7f136d3e6700 1 -- 172.21.15.12:0/1100987320 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e3190f280 con 0x555e317899f0 2024-05-17T22:44:36.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.775+0000 7f136e3e8700 1 --2- 172.21.15.12:0/1100987320 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e31907b00 0x555e31929100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:44:36.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.776+0000 7f136d3e6700 1 --2- 172.21.15.12:0/1100987320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e317899f0 0x555e319115e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f136806bd20 tx=0x7f1358020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cfc67b86a0aba66 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:44:36.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.776+0000 7f13537fe700 1 -- 172.21.15.12:0/1100987320 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135800f8c0 con 0x555e317899f0 2024-05-17T22:44:36.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.776+0000 7f13537fe700 1 -- 172.21.15.12:0/1100987320 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1358005110 con 0x555e317899f0 2024-05-17T22:44:36.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.776+0000 7f13537fe700 1 -- 172.21.15.12:0/1100987320 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1358003ae0 con 0x555e317899f0 2024-05-17T22:44:36.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.777+0000 7f137e826b80 1 -- 172.21.15.12:0/1100987320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e317899f0 msgr2=0x555e319115e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:36.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.777+0000 7f137e826b80 1 --2- 172.21.15.12:0/1100987320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e317899f0 0x555e319115e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f136806bd20 tx=0x7f1358020000 comp rx=0 tx=0).stop 2024-05-17T22:44:36.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.777+0000 7f137e826b80 1 -- 172.21.15.12:0/1100987320 shutdown_connections 2024-05-17T22:44:36.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.777+0000 7f137e826b80 1 --2- 172.21.15.12:0/1100987320 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e31907b00 0x555e31929100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:36.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.777+0000 7f137e826b80 1 --2- 172.21.15.12:0/1100987320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e317e97d0 0x555e3190c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:36.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.778+0000 7f137e826b80 1 --2- 172.21.15.12:0/1100987320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e317899f0 0x555e319115e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:36.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.778+0000 7f137e826b80 1 -- 172.21.15.12:0/1100987320 >> 172.21.15.12:0/1100987320 conn(0x555e319024a0 msgr2=0x555e3190be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:44:36.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.778+0000 7f137e826b80 1 -- 172.21.15.12:0/1100987320 shutdown_connections 2024-05-17T22:44:36.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.778+0000 7f137e826b80 1 -- 172.21.15.12:0/1100987320 wait complete. 2024-05-17T22:44:36.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.778+0000 7f137e826b80 1 -- start start 2024-05-17T22:44:36.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.779+0000 7f137e826b80 1 Processor -- start 2024-05-17T22:44:36.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.779+0000 7f137e826b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e317899f0 0x555e31919110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:36.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.780+0000 7f137e826b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e317e97d0 0x555e31919650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:36.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.780+0000 7f136dbe7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e317899f0 0x555e31919110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:36.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.780+0000 7f137e826b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e31907b00 0x555e3191bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:36.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.780+0000 7f136dbe7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e317899f0 0x555e31919110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38186/0 (socket says 172.21.15.12:38186) 2024-05-17T22:44:36.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.780+0000 7f137e826b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f13681bd7c0 con 0x555e317899f0 2024-05-17T22:44:36.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.780+0000 7f136dbe7700 1 -- 172.21.15.12:0/3944269154 learned_addr learned my addr 172.21.15.12:0/3944269154 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:44:36.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.780+0000 7f136d3e6700 1 --2- 172.21.15.12:0/3944269154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e317e97d0 0x555e31919650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:36.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.780+0000 7f137e826b80 1 -- 172.21.15.12:0/3944269154 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f13681bec30 con 0x555e317e97d0 2024-05-17T22:44:36.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.780+0000 7f137e826b80 1 -- 172.21.15.12:0/3944269154 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f13681bde90 con 0x555e31907b00 2024-05-17T22:44:36.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.780+0000 7f136e3e8700 1 --2- 172.21.15.12:0/3944269154 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e31907b00 0x555e3191bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:36.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.781+0000 7f136d3e6700 1 -- 172.21.15.12:0/3944269154 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e31907b00 msgr2=0x555e3191bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:44:36.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.781+0000 7f136d3e6700 1 --2- 172.21.15.12:0/3944269154 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e31907b00 0x555e3191bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:36.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.781+0000 7f136d3e6700 1 -- 172.21.15.12:0/3944269154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e317899f0 msgr2=0x555e31919110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:44:36.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.781+0000 7f136d3e6700 1 --2- 172.21.15.12:0/3944269154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e317899f0 0x555e31919110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:36.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.781+0000 7f136d3e6700 1 -- 172.21.15.12:0/3944269154 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e3191c2c0 con 0x555e317e97d0 2024-05-17T22:44:36.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.781+0000 7f137e826b80 10 client.0 _ll_register_callbacks cb 0x555e317bcb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:44:36.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.781+0000 7f137e826b80 1 -- 172.21.15.12:0/3944269154 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1358005110 con 0x555e317e97d0 2024-05-17T22:44:36.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.781+0000 7f13517fa700 10 client.25478 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:44:36.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.781+0000 7f136d3e6700 1 --2- 172.21.15.12:0/3944269154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e317e97d0 0x555e31919650 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x555e3191ea00 tx=0x7f135800bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ac030cc548886c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:44:36.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.781+0000 7f13517fa700 1 -- 172.21.15.12:0/3944269154 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135800fdc0 con 0x555e317e97d0 2024-05-17T22:44:36.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.782+0000 7f13517fa700 1 -- 172.21.15.12:0/3944269154 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f1358003880 con 0x555e317e97d0 2024-05-17T22:44:36.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.782+0000 7f13517fa700 1 -- 172.21.15.12:0/3944269154 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1358003aa0 con 0x555e317e97d0 2024-05-17T22:44:36.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.782+0000 7f137e826b80 10 client.25478 fetch_fsmap learned FSMap version 34 2024-05-17T22:44:36.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.782+0000 7f137e826b80 1 -- 172.21.15.12:0/3944269154 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({fsmap=34}) -- 0x555e3198b050 con 0x555e317e97d0 2024-05-17T22:44:36.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.783+0000 7f13517fa700 1 -- 172.21.15.12:0/3944269154 <== mon.2 v2:172.21.15.96:3300/0 4 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f135802f020 con 0x555e317e97d0 2024-05-17T22:44:36.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.783+0000 7f137e826b80 10 client.25478 fetch_fsmap finished waiting for FSMap version 34 2024-05-17T22:44:36.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.783+0000 7f137e826b80 10 client.25478 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:44:36.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.783+0000 7f137e826b80 20 client.25478 populate_metadata read hostname 'smithi012' 2024-05-17T22:44:36.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.783+0000 7f137e826b80 1 --2- 172.21.15.12:0/3944269154 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555e3190d510 0x555e3198d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:36.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.783+0000 7f137e826b80 4 client.25478 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:44:36.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.783+0000 7f137e826b80 1 -- 172.21.15.12:0/3944269154 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555e31831800 con 0x555e3190d510 2024-05-17T22:44:36.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.783+0000 7f136dbe7700 1 --2- 172.21.15.12:0/3944269154 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555e3190d510 0x555e3198d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:36.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.784+0000 7f136dbe7700 1 --2- 172.21.15.12:0/3944269154 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555e3190d510 0x555e3198d630 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adc18b8ac45cf4f4 server_cookie=4c7207d472f955f0 in_seq=0 out_seq=0 2024-05-17T22:44:36.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.784+0000 7f13517fa700 10 client.25478 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:44:36.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.784+0000 7f13517fa700 1 -- 172.21.15.12:0/3944269154 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555e31831800 con 0x555e3190d510 2024-05-17T22:44:36.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.784+0000 7f13517fa700 10 client.25478 handle_command_reply: tid=0 2024-05-17T22:44:36.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.784+0000 7f137e826b80 1 client.25478 shutdown 2024-05-17T22:44:36.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.785+0000 7f137e826b80 1 -- 172.21.15.12:0/3944269154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e317e97d0 msgr2=0x555e31919650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:36.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.785+0000 7f137e826b80 1 --2- 172.21.15.12:0/3944269154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e317e97d0 0x555e31919650 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x555e3191ea00 tx=0x7f135800bf90 comp rx=0 tx=0).stop 2024-05-17T22:44:36.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.785+0000 7f137e826b80 1 -- 172.21.15.12:0/3944269154 shutdown_connections 2024-05-17T22:44:36.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.785+0000 7f137e826b80 1 --2- 172.21.15.12:0/3944269154 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e31907b00 0x555e3191bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:36.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.785+0000 7f137e826b80 1 --2- 172.21.15.12:0/3944269154 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x555e3190d510 0x555e3198d630 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:36.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.785+0000 7f13517fa700 0 client.25478 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:44:36.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.785+0000 7f137e826b80 1 --2- 172.21.15.12:0/3944269154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e317899f0 0x555e31919110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:36.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.785+0000 7f137e826b80 1 --2- 172.21.15.12:0/3944269154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e317e97d0 0x555e31919650 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:36.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.785+0000 7f137e826b80 1 -- 172.21.15.12:0/3944269154 >> 172.21.15.12:0/3944269154 conn(0x555e319024a0 msgr2=0x555e318fc220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:44:36.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.786+0000 7f137e826b80 1 -- 172.21.15.12:0/3944269154 shutdown_connections 2024-05-17T22:44:36.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.786+0000 7f137e826b80 1 -- 172.21.15.12:0/3944269154 wait complete. 2024-05-17T22:44:36.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.786+0000 7f137e826b80 20 client.25478 trim_cache size 0 max 16384 2024-05-17T22:44:36.790 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:44:36.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.787+0000 7f136ebe9700 1 -- 172.21.15.12:0/3273609909 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f13540b2900 msgr2=0x7f13540b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:36.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.787+0000 7f136ebe9700 1 --2- 172.21.15.12:0/3273609909 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f13540b2900 0x7f13540b4e10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1358008000 tx=0x7f1358005fb0 comp rx=0 tx=0).stop 2024-05-17T22:44:36.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.787+0000 7f136ebe9700 1 -- 172.21.15.12:0/3273609909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1368102820 msgr2=0x7f136806f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:36.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.787+0000 7f136ebe9700 1 --2- 172.21.15.12:0/3273609909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1368102820 0x7f136806f5f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f136400c920 tx=0x7f136400cc30 comp rx=0 tx=0).stop 2024-05-17T22:44:36.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.787+0000 7f136ebe9700 1 -- 172.21.15.12:0/3273609909 shutdown_connections 2024-05-17T22:44:36.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.787+0000 7f136ebe9700 1 --2- 172.21.15.12:0/3273609909 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f13540b2900 0x7f13540b4e10 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:36.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.787+0000 7f136ebe9700 1 --2- 172.21.15.12:0/3273609909 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1368070070 0x7f13681b5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:36.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.787+0000 7f136ebe9700 1 --2- 172.21.15.12:0/3273609909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1368102820 0x7f136806f5f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:36.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.787+0000 7f136ebe9700 1 --2- 172.21.15.12:0/3273609909 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1368108990 0x7f136806fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:36.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.787+0000 7f136ebe9700 1 -- 172.21.15.12:0/3273609909 >> 172.21.15.12:0/3273609909 conn(0x7f13680fdf80 msgr2=0x7f1368105630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:44:36.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.788+0000 7f136ebe9700 1 -- 172.21.15.12:0/3273609909 shutdown_connections 2024-05-17T22:44:36.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:36.788+0000 7f136ebe9700 1 -- 172.21.15.12:0/3273609909 wait complete. 2024-05-17T22:44:37.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:36 smithi096 ceph-mon[36065]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:44:37.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:36 smithi012 ceph-mon[31632]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:44:37.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:36 smithi151 ceph-mon[36101]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:44:38.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:37 smithi012 ceph-mon[31632]: pgmap v600: 129 pgs: 129 active+clean; 1.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 568 op/s 2024-05-17T22:44:38.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:37 smithi151 ceph-mon[36101]: pgmap v600: 129 pgs: 129 active+clean; 1.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 568 op/s 2024-05-17T22:44:38.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:37 smithi096 ceph-mon[36065]: pgmap v600: 129 pgs: 129 active+clean; 1.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 568 op/s 2024-05-17T22:44:38.557 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-17T22:44:38.558 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-17T22:44:38.994 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:44:40.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:39 smithi012 ceph-mon[31632]: pgmap v601: 129 pgs: 129 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 665 op/s 2024-05-17T22:44:40.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:39 smithi151 ceph-mon[36101]: pgmap v601: 129 pgs: 129 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 665 op/s 2024-05-17T22:44:40.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:39 smithi096 ceph-mon[36065]: pgmap v601: 129 pgs: 129 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 665 op/s 2024-05-17T22:44:41.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.330+0000 7fdf67ef9700 1 -- 172.21.15.12:0/2253984524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf60109190 msgr2=0x7fdf60110af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:41.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.330+0000 7fdf67ef9700 1 --2- 172.21.15.12:0/2253984524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf60109190 0x7fdf60110af0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdf58009f90 tx=0x7fdf5800b330 comp rx=0 tx=0).stop 2024-05-17T22:44:41.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.331+0000 7fdf67ef9700 1 -- 172.21.15.12:0/2253984524 shutdown_connections 2024-05-17T22:44:41.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.331+0000 7fdf67ef9700 1 --2- 172.21.15.12:0/2253984524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf60109190 0x7fdf60110af0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:41.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.331+0000 7fdf67ef9700 1 --2- 172.21.15.12:0/2253984524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf60103f10 0x7fdf60104320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:41.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.331+0000 7fdf67ef9700 1 --2- 172.21.15.12:0/2253984524 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf60108840 0x7fdf60108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:41.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.331+0000 7fdf67ef9700 1 -- 172.21.15.12:0/2253984524 >> 172.21.15.12:0/2253984524 conn(0x7fdf600ff9b0 msgr2=0x7fdf60101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:44:41.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.332+0000 7fdf67ef9700 1 -- 172.21.15.12:0/2253984524 shutdown_connections 2024-05-17T22:44:41.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.332+0000 7fdf67ef9700 1 -- 172.21.15.12:0/2253984524 wait complete. 2024-05-17T22:44:41.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.333+0000 7fdf67ef9700 1 Processor -- start 2024-05-17T22:44:41.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.333+0000 7fdf67ef9700 1 -- start start 2024-05-17T22:44:41.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.334+0000 7fdf67ef9700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf60103f10 0x7fdf6019a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:41.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.334+0000 7fdf67ef9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf60108840 0x7fdf6019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:41.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.334+0000 7fdf67ef9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf60109190 0x7fdf6019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:41.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.334+0000 7fdf67ef9700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdf60113e20 con 0x7fdf60109190 2024-05-17T22:44:41.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.334+0000 7fdf67ef9700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdf60113ca0 con 0x7fdf60108840 2024-05-17T22:44:41.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.334+0000 7fdf67ef9700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdf60113fa0 con 0x7fdf60103f10 2024-05-17T22:44:41.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.334+0000 7fdf65c95700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf60103f10 0x7fdf6019a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:41.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.334+0000 7fdf65c95700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf60103f10 0x7fdf6019a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:50102/0 (socket says 172.21.15.12:50102) 2024-05-17T22:44:41.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.334+0000 7fdf65c95700 1 -- 172.21.15.12:0/2508218086 learned_addr learned my addr 172.21.15.12:0/2508218086 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:44:41.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.335+0000 7fdf65494700 1 --2- 172.21.15.12:0/2508218086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf60108840 0x7fdf6019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:41.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.335+0000 7fdf66496700 1 --2- 172.21.15.12:0/2508218086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf60109190 0x7fdf6019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:41.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.335+0000 7fdf65c95700 1 -- 172.21.15.12:0/2508218086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf60108840 msgr2=0x7fdf6019aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:41.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.335+0000 7fdf65c95700 1 --2- 172.21.15.12:0/2508218086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf60108840 0x7fdf6019aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:41.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.335+0000 7fdf65c95700 1 -- 172.21.15.12:0/2508218086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf60109190 msgr2=0x7fdf6019ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:41.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.335+0000 7fdf65c95700 1 --2- 172.21.15.12:0/2508218086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf60109190 0x7fdf6019ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:41.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.335+0000 7fdf65c95700 1 -- 172.21.15.12:0/2508218086 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf6019f400 con 0x7fdf60103f10 2024-05-17T22:44:41.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.335+0000 7fdf66496700 1 --2- 172.21.15.12:0/2508218086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf60109190 0x7fdf6019ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:44:41.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.335+0000 7fdf65c95700 1 --2- 172.21.15.12:0/2508218086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf60103f10 0x7fdf6019a530 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00e9a0 tx=0x7fdf5c00ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:44:41.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.335+0000 7fdf56ffd700 1 -- 172.21.15.12:0/2508218086 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c00cbd0 con 0x7fdf60103f10 2024-05-17T22:44:41.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.335+0000 7fdf56ffd700 1 -- 172.21.15.12:0/2508218086 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf5c00cd70 con 0x7fdf60103f10 2024-05-17T22:44:41.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.335+0000 7fdf56ffd700 1 -- 172.21.15.12:0/2508218086 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c018680 con 0x7fdf60103f10 2024-05-17T22:44:41.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.336+0000 7fdf67ef9700 1 -- 172.21.15.12:0/2508218086 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf6019f6f0 con 0x7fdf60103f10 2024-05-17T22:44:41.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.336+0000 7fdf67ef9700 1 -- 172.21.15.12:0/2508218086 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf601a7010 con 0x7fdf60103f10 2024-05-17T22:44:41.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.340+0000 7fdf56ffd700 1 -- 172.21.15.12:0/2508218086 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fdf5c010970 con 0x7fdf60103f10 2024-05-17T22:44:41.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.342+0000 7fdf56ffd700 1 --2- 172.21.15.12:0/2508218086 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fdf4c0b2900 0x7fdf4c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:41.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.343+0000 7fdf768d4b80 1 Processor -- start 2024-05-17T22:44:41.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.343+0000 7fdf56ffd700 1 -- 172.21.15.12:0/2508218086 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fdf5c0e28b0 con 0x7fdf60103f10 2024-05-17T22:44:41.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.343+0000 7fdf768d4b80 1 -- start start 2024-05-17T22:44:41.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.344+0000 7fdf65494700 1 --2- 172.21.15.12:0/2508218086 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fdf4c0b2900 0x7fdf4c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:41.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.344+0000 7fdf768d4b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55611562aa70 0x5561157ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:41.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.344+0000 7fdf768d4b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561157aeb20 0x5561157a8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:41.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.344+0000 7fdf768d4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561157a91f0 0x5561157a9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:41.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.344+0000 7fdf768d4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5561156d7310 con 0x5561157a91f0 2024-05-17T22:44:41.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.344+0000 7fdf66496700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561157a91f0 0x5561157a9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:41.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.344+0000 7fdf66496700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561157a91f0 0x5561157a9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41882/0 (socket says 172.21.15.12:41882) 2024-05-17T22:44:41.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.344+0000 7fdf66496700 1 -- 172.21.15.12:0/1218213746 learned_addr learned my addr 172.21.15.12:0/1218213746 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:44:41.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.344+0000 7fdf65494700 1 --2- 172.21.15.12:0/1218213746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561157aeb20 0x5561157a8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:41.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.344+0000 7fdf768d4b80 1 -- 172.21.15.12:0/1218213746 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5561156c7050 con 0x55611562aa70 2024-05-17T22:44:41.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.344+0000 7fdf768d4b80 1 -- 172.21.15.12:0/1218213746 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5561156caef0 con 0x5561157aeb20 2024-05-17T22:44:41.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.345+0000 7fdf65c95700 1 --2- 172.21.15.12:0/1218213746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55611562aa70 0x5561157ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:41.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.345+0000 7fdf65494700 1 --2- 172.21.15.12:0/2508218086 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fdf4c0b2900 0x7fdf4c0b4e10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fdf601072c0 tx=0x7fdf50009f80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:44:41.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.345+0000 7fdf66496700 1 -- 172.21.15.12:0/1218213746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561157aeb20 msgr2=0x5561157a8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:44:41.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.345+0000 7fdf66496700 1 --2- 172.21.15.12:0/1218213746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561157aeb20 0x5561157a8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:41.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.345+0000 7fdf66496700 1 -- 172.21.15.12:0/1218213746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55611562aa70 msgr2=0x5561157ae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:44:41.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.345+0000 7fdf66496700 1 --2- 172.21.15.12:0/1218213746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55611562aa70 0x5561157ae5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:41.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.345+0000 7fdf66496700 1 -- 172.21.15.12:0/1218213746 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556115670610 con 0x5561157a91f0 2024-05-17T22:44:41.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.345+0000 7fdf65494700 1 --2- 172.21.15.12:0/1218213746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561157aeb20 0x5561157a8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:44:41.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.346+0000 7fdf66496700 1 --2- 172.21.15.12:0/1218213746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561157a91f0 0x5561157a9600 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fdf58009470 tx=0x7fdf580094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a360cba7c23449b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:44:41.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.346+0000 7fdf4bfff700 1 -- 172.21.15.12:0/1218213746 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf58021070 con 0x5561157a91f0 2024-05-17T22:44:41.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.346+0000 7fdf4bfff700 1 -- 172.21.15.12:0/1218213746 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf58004c50 con 0x5561157a91f0 2024-05-17T22:44:41.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.346+0000 7fdf4bfff700 1 -- 172.21.15.12:0/1218213746 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf58007c10 con 0x5561157a91f0 2024-05-17T22:44:41.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.347+0000 7fdf768d4b80 1 -- 172.21.15.12:0/1218213746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561157a91f0 msgr2=0x5561157a9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:41.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.347+0000 7fdf768d4b80 1 --2- 172.21.15.12:0/1218213746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561157a91f0 0x5561157a9600 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fdf58009470 tx=0x7fdf580094a0 comp rx=0 tx=0).stop 2024-05-17T22:44:41.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.347+0000 7fdf768d4b80 1 -- 172.21.15.12:0/1218213746 shutdown_connections 2024-05-17T22:44:41.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.347+0000 7fdf768d4b80 1 --2- 172.21.15.12:0/1218213746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561157a91f0 0x5561157a9600 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:41.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.347+0000 7fdf768d4b80 1 --2- 172.21.15.12:0/1218213746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55611562aa70 0x5561157ae5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:41.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.347+0000 7fdf768d4b80 1 --2- 172.21.15.12:0/1218213746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561157aeb20 0x5561157a8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:41.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.347+0000 7fdf768d4b80 1 -- 172.21.15.12:0/1218213746 >> 172.21.15.12:0/1218213746 conn(0x55611566ef80 msgr2=0x5561157a3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:44:41.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.347+0000 7fdf768d4b80 1 -- 172.21.15.12:0/1218213746 shutdown_connections 2024-05-17T22:44:41.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.347+0000 7fdf768d4b80 1 -- 172.21.15.12:0/1218213746 wait complete. 2024-05-17T22:44:41.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.348+0000 7fdf768d4b80 1 -- start start 2024-05-17T22:44:41.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.348+0000 7fdf768d4b80 1 Processor -- start 2024-05-17T22:44:41.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.349+0000 7fdf768d4b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55611562aa70 0x5561157b2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:41.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.349+0000 7fdf768d4b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561157a91f0 0x5561157b2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:41.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.349+0000 7fdf768d4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561157aeb20 0x5561157b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:41.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.349+0000 7fdf768d4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5561155ce1d0 con 0x5561157aeb20 2024-05-17T22:44:41.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.349+0000 7fdf768d4b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556115667f00 con 0x55611562aa70 2024-05-17T22:44:41.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.349+0000 7fdf768d4b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5561157b5410 con 0x5561157a91f0 2024-05-17T22:44:41.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.349+0000 7fdf768d4b80 10 client.0 _ll_register_callbacks cb 0x556115661f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:44:41.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.349+0000 7fdf65494700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561157a91f0 0x5561157b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:41.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.349+0000 7fdf65494700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561157a91f0 0x5561157b2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:50130/0 (socket says 172.21.15.12:50130) 2024-05-17T22:44:41.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.349+0000 7fdf65494700 1 -- 172.21.15.12:0/2632555510 learned_addr learned my addr 172.21.15.12:0/2632555510 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:44:41.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.349+0000 7fdf65c95700 1 --2- 172.21.15.12:0/2632555510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55611562aa70 0x5561157b2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:41.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.349+0000 7fdf66496700 1 --2- 172.21.15.12:0/2632555510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561157aeb20 0x5561157b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:41.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.350+0000 7fdf66496700 1 -- 172.21.15.12:0/2632555510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561157a91f0 msgr2=0x5561157b2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:44:41.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.350+0000 7fdf66496700 1 --2- 172.21.15.12:0/2632555510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561157a91f0 0x5561157b2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:41.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.350+0000 7fdf66496700 1 -- 172.21.15.12:0/2632555510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55611562aa70 msgr2=0x5561157b2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:44:41.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.350+0000 7fdf66496700 1 --2- 172.21.15.12:0/2632555510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55611562aa70 0x5561157b2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:41.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.350+0000 7fdf66496700 1 -- 172.21.15.12:0/2632555510 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556115617500 con 0x5561157aeb20 2024-05-17T22:44:41.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.350+0000 7fdf66496700 1 --2- 172.21.15.12:0/2632555510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561157aeb20 0x5561157b2cd0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fdf5800d010 tx=0x7fdf5800be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cda1551aff3e16c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:44:41.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.350+0000 7fdf497fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:44:41.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.350+0000 7fdf497fa700 1 -- 172.21.15.12:0/2632555510 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5802b070 con 0x5561157aeb20 2024-05-17T22:44:41.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.350+0000 7fdf497fa700 1 -- 172.21.15.12:0/2632555510 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fdf58004e00 con 0x5561157aeb20 2024-05-17T22:44:41.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.350+0000 7fdf768d4b80 1 -- 172.21.15.12:0/2632555510 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf58004c50 con 0x5561157aeb20 2024-05-17T22:44:41.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.351+0000 7fdf497fa700 1 -- 172.21.15.12:0/2632555510 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fdf58037050 con 0x5561157aeb20 2024-05-17T22:44:41.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.351+0000 7fdf768d4b80 10 client.15825 fetch_fsmap learned FSMap version 34 2024-05-17T22:44:41.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.351+0000 7fdf768d4b80 1 -- 172.21.15.12:0/2632555510 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5561157bfe30 con 0x5561157aeb20 2024-05-17T22:44:41.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.351+0000 7fdf497fa700 1 -- 172.21.15.12:0/2632555510 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf58004c50 con 0x5561157aeb20 2024-05-17T22:44:41.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.351+0000 7fdf497fa700 1 -- 172.21.15.12:0/2632555510 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7fdf58011070 con 0x5561157aeb20 2024-05-17T22:44:41.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.351+0000 7fdf768d4b80 10 client.15825 fetch_fsmap finished waiting for FSMap version 34 2024-05-17T22:44:41.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.351+0000 7fdf768d4b80 10 client.15825 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:44:41.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.351+0000 7fdf768d4b80 20 client.15825 populate_metadata read hostname 'smithi012' 2024-05-17T22:44:41.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.351+0000 7fdf768d4b80 1 --2- 172.21.15.12:0/2632555510 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5561157c2300 0x5561157c4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:41.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.351+0000 7fdf768d4b80 4 client.15825 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:44:41.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.351+0000 7fdf768d4b80 1 -- 172.21.15.12:0/2632555510 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561156d9260 con 0x5561157c2300 2024-05-17T22:44:41.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.354+0000 7fdf65c95700 1 --2- 172.21.15.12:0/2632555510 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5561157c2300 0x5561157c4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:41.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.355+0000 7fdf65c95700 1 --2- 172.21.15.12:0/2632555510 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5561157c2300 0x5561157c4720 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19def8a313afa44e server_cookie=2e057af49b815aae in_seq=0 out_seq=0 2024-05-17T22:44:41.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.355+0000 7fdf497fa700 10 client.15825 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:44:41.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.356+0000 7fdf497fa700 1 -- 172.21.15.12:0/2632555510 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5561156d9260 con 0x5561157c2300 2024-05-17T22:44:41.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.356+0000 7fdf497fa700 10 client.15825 handle_command_reply: tid=0 2024-05-17T22:44:41.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.356+0000 7fdf768d4b80 1 client.15825 shutdown 2024-05-17T22:44:41.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.356+0000 7fdf768d4b80 1 -- 172.21.15.12:0/2632555510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561157aeb20 msgr2=0x5561157b2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:41.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.356+0000 7fdf768d4b80 1 --2- 172.21.15.12:0/2632555510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561157aeb20 0x5561157b2cd0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fdf5800d010 tx=0x7fdf5800be40 comp rx=0 tx=0).stop 2024-05-17T22:44:41.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.356+0000 7fdf768d4b80 1 -- 172.21.15.12:0/2632555510 shutdown_connections 2024-05-17T22:44:41.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.356+0000 7fdf768d4b80 1 --2- 172.21.15.12:0/2632555510 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5561157c2300 0x5561157c4720 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:41.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.357+0000 7fdf768d4b80 1 --2- 172.21.15.12:0/2632555510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561157aeb20 0x5561157b2cd0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:41.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.357+0000 7fdf768d4b80 1 --2- 172.21.15.12:0/2632555510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55611562aa70 0x5561157b2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:41.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.357+0000 7fdf768d4b80 1 --2- 172.21.15.12:0/2632555510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561157a91f0 0x5561157b2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:41.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.357+0000 7fdf768d4b80 1 -- 172.21.15.12:0/2632555510 >> 172.21.15.12:0/2632555510 conn(0x55611566ef80 msgr2=0x55611579df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:44:41.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.357+0000 7fdf497fa700 0 client.15825 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:44:41.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.357+0000 7fdf768d4b80 1 -- 172.21.15.12:0/2632555510 shutdown_connections 2024-05-17T22:44:41.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.357+0000 7fdf768d4b80 1 -- 172.21.15.12:0/2632555510 wait complete. 2024-05-17T22:44:41.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.357+0000 7fdf768d4b80 20 client.15825 trim_cache size 0 max 16384 2024-05-17T22:44:41.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 320.0 s, 801.8 tps, lat 126.556 ms stddev 167.804 2024-05-17T22:44:41.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.398+0000 7fdf768d4b80 1 Processor -- start 2024-05-17T22:44:41.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.398+0000 7fdf768d4b80 1 -- start start 2024-05-17T22:44:41.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.398+0000 7fdf768d4b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55611562aa70 0x5561157a8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:41.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.399+0000 7fdf768d4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561157a91f0 0x55611579df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:41.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.399+0000 7fdf65c95700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55611562aa70 0x5561157a8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:41.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.399+0000 7fdf65c95700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55611562aa70 0x5561157a8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:59732/0 (socket says 172.21.15.12:59732) 2024-05-17T22:44:41.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.399+0000 7fdf65c95700 1 -- 172.21.15.12:0/1613588094 learned_addr learned my addr 172.21.15.12:0/1613588094 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:44:41.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.399+0000 7fdf768d4b80 1 --2- 172.21.15.12:0/1613588094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561157aeb20 0x5561157b2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:41.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.399+0000 7fdf768d4b80 1 -- 172.21.15.12:0/1613588094 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdf601b2580 con 0x5561157a91f0 2024-05-17T22:44:41.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.399+0000 7fdf768d4b80 1 -- 172.21.15.12:0/1613588094 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdf601afde0 con 0x55611562aa70 2024-05-17T22:44:41.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.399+0000 7fdf66496700 1 --2- 172.21.15.12:0/1613588094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561157aeb20 0x5561157b2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:41.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.399+0000 7fdf768d4b80 1 -- 172.21.15.12:0/1613588094 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdf601adc30 con 0x5561157aeb20 2024-05-17T22:44:41.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.401+0000 7fdf65494700 1 --2- 172.21.15.12:0/1613588094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561157a91f0 0x55611579df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:41.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.406+0000 7fdf65494700 1 -- 172.21.15.12:0/1613588094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561157aeb20 msgr2=0x5561157b2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:44:41.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.406+0000 7fdf65494700 1 --2- 172.21.15.12:0/1613588094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561157aeb20 0x5561157b2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:41.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.406+0000 7fdf65494700 1 -- 172.21.15.12:0/1613588094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55611562aa70 msgr2=0x5561157a8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:44:41.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.406+0000 7fdf65494700 1 --2- 172.21.15.12:0/1613588094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55611562aa70 0x5561157a8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:41.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.406+0000 7fdf65494700 1 -- 172.21.15.12:0/1613588094 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561157b07a0 con 0x5561157a91f0 2024-05-17T22:44:41.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.406+0000 7fdf65c95700 1 --2- 172.21.15.12:0/1613588094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55611562aa70 0x5561157a8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:44:41.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.406+0000 7fdf66496700 1 --2- 172.21.15.12:0/1613588094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561157aeb20 0x5561157b2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:44:41.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.411+0000 7fdf65494700 1 --2- 172.21.15.12:0/1613588094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561157a91f0 0x55611579df30 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x5561157b3a10 tx=0x7fdf50016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcaa4584eace9a48 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:44:41.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.416+0000 7fdf4b7fe700 1 -- 172.21.15.12:0/1613588094 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf50003a10 con 0x5561157a91f0 2024-05-17T22:44:41.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.416+0000 7fdf4b7fe700 1 -- 172.21.15.12:0/1613588094 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf50012450 con 0x5561157a91f0 2024-05-17T22:44:41.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.416+0000 7fdf4b7fe700 1 -- 172.21.15.12:0/1613588094 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5001f680 con 0x5561157a91f0 2024-05-17T22:44:41.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.417+0000 7fdf768d4b80 1 -- 172.21.15.12:0/1613588094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561157a91f0 msgr2=0x55611579df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:41.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.417+0000 7fdf768d4b80 1 --2- 172.21.15.12:0/1613588094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561157a91f0 0x55611579df30 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x5561157b3a10 tx=0x7fdf50016000 comp rx=0 tx=0).stop 2024-05-17T22:44:41.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.417+0000 7fdf768d4b80 1 -- 172.21.15.12:0/1613588094 shutdown_connections 2024-05-17T22:44:41.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.417+0000 7fdf768d4b80 1 --2- 172.21.15.12:0/1613588094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561157aeb20 0x5561157b2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:41.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.417+0000 7fdf768d4b80 1 --2- 172.21.15.12:0/1613588094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55611562aa70 0x5561157a8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:41.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.417+0000 7fdf768d4b80 1 --2- 172.21.15.12:0/1613588094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561157a91f0 0x55611579df30 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:41.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.417+0000 7fdf768d4b80 1 -- 172.21.15.12:0/1613588094 >> 172.21.15.12:0/1613588094 conn(0x55611566ef80 msgr2=0x5561157b4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:44:41.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.422+0000 7fdf768d4b80 1 -- 172.21.15.12:0/1613588094 shutdown_connections 2024-05-17T22:44:41.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.422+0000 7fdf768d4b80 1 -- 172.21.15.12:0/1613588094 wait complete. 2024-05-17T22:44:41.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.422+0000 7fdf768d4b80 1 -- start start 2024-05-17T22:44:41.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.423+0000 7fdf768d4b80 1 Processor -- start 2024-05-17T22:44:41.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.424+0000 7fdf768d4b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55611562aa70 0x5561157c52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:41.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.424+0000 7fdf768d4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561157a91f0 0x5561157c5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:41.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.424+0000 7fdf768d4b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561157aeb20 0x55611582b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:41.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.424+0000 7fdf768d4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdf601ae2a0 con 0x5561157a91f0 2024-05-17T22:44:41.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.424+0000 7fdf768d4b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdf601af710 con 0x55611562aa70 2024-05-17T22:44:41.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.424+0000 7fdf768d4b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdf601ae970 con 0x5561157aeb20 2024-05-17T22:44:41.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.424+0000 7fdf66496700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561157aeb20 0x55611582b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:41.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.424+0000 7fdf65c95700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55611562aa70 0x5561157c52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:41.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.425+0000 7fdf66496700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561157aeb20 0x55611582b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:50152/0 (socket says 172.21.15.12:50152) 2024-05-17T22:44:41.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.425+0000 7fdf66496700 1 -- 172.21.15.12:0/2751317085 learned_addr learned my addr 172.21.15.12:0/2751317085 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:44:41.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.425+0000 7fdf768d4b80 10 client.0 _ll_register_callbacks cb 0x556115661f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:44:41.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.425+0000 7fdf66496700 1 -- 172.21.15.12:0/2751317085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55611562aa70 msgr2=0x5561157c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:44:41.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.425+0000 7fdf66496700 1 --2- 172.21.15.12:0/2751317085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55611562aa70 0x5561157c52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:41.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.425+0000 7fdf66496700 1 -- 172.21.15.12:0/2751317085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561157a91f0 msgr2=0x5561157c5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:44:41.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.425+0000 7fdf66496700 1 --2- 172.21.15.12:0/2751317085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561157a91f0 0x5561157c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:41.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.425+0000 7fdf66496700 1 -- 172.21.15.12:0/2751317085 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561157c60f0 con 0x5561157aeb20 2024-05-17T22:44:41.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.426+0000 7fdf66496700 1 --2- 172.21.15.12:0/2751317085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561157aeb20 0x55611582b850 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdf5800d010 tx=0x7fdf5802f5e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed9e2b719ff7692e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:44:41.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.426+0000 7fdf768d4b80 1 -- 172.21.15.12:0/2751317085 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf50012450 con 0x5561157aeb20 2024-05-17T22:44:41.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.426+0000 7fdf497fa700 10 client.25438 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:44:41.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.426+0000 7fdf497fa700 1 -- 172.21.15.12:0/2751317085 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5802b070 con 0x5561157aeb20 2024-05-17T22:44:41.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.426+0000 7fdf497fa700 1 -- 172.21.15.12:0/2751317085 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fdf58007aa0 con 0x5561157aeb20 2024-05-17T22:44:41.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.428+0000 7fdf497fa700 1 -- 172.21.15.12:0/2751317085 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fdf58021040 con 0x5561157aeb20 2024-05-17T22:44:41.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.428+0000 7fdf768d4b80 10 client.25438 fetch_fsmap learned FSMap version 34 2024-05-17T22:44:41.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.428+0000 7fdf768d4b80 1 -- 172.21.15.12:0/2751317085 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({fsmap=34}) -- 0x55611582cc90 con 0x5561157aeb20 2024-05-17T22:44:41.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.430+0000 7fdf497fa700 1 -- 172.21.15.12:0/2751317085 <== mon.1 v2:172.21.15.151:3300/0 4 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7fdf58011040 con 0x5561157aeb20 2024-05-17T22:44:41.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.431+0000 7fdf768d4b80 10 client.25438 fetch_fsmap finished waiting for FSMap version 34 2024-05-17T22:44:41.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.431+0000 7fdf768d4b80 10 client.25438 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:44:41.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.431+0000 7fdf768d4b80 20 client.25438 populate_metadata read hostname 'smithi012' 2024-05-17T22:44:41.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.431+0000 7fdf768d4b80 1 --2- 172.21.15.12:0/2751317085 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55611582d000 0x55611582f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:41.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.431+0000 7fdf768d4b80 4 client.25438 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-17T22:44:41.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.431+0000 7fdf768d4b80 1 -- 172.21.15.12:0/2751317085 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fdf50012450 con 0x55611582d000 2024-05-17T22:44:41.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.435+0000 7fdf65c95700 1 --2- 172.21.15.12:0/2751317085 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55611582d000 0x55611582f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:41.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.441+0000 7fdf65c95700 1 --2- 172.21.15.12:0/2751317085 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55611582d000 0x55611582f440 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=276dbef88e730913 server_cookie=e130388a44e2af03 in_seq=0 out_seq=0 2024-05-17T22:44:41.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.441+0000 7fdf497fa700 10 client.25438 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:44:41.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.442+0000 7fdf497fa700 1 -- 172.21.15.12:0/2751317085 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fdf50012450 con 0x55611582d000 2024-05-17T22:44:41.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.442+0000 7fdf497fa700 10 client.25438 handle_command_reply: tid=0 2024-05-17T22:44:41.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.444+0000 7fdf768d4b80 1 client.25438 shutdown 2024-05-17T22:44:41.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.449+0000 7fdf768d4b80 1 -- 172.21.15.12:0/2751317085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561157aeb20 msgr2=0x55611582b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:41.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.449+0000 7fdf768d4b80 1 --2- 172.21.15.12:0/2751317085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561157aeb20 0x55611582b850 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdf5800d010 tx=0x7fdf5802f5e0 comp rx=0 tx=0).stop 2024-05-17T22:44:41.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.449+0000 7fdf768d4b80 1 -- 172.21.15.12:0/2751317085 shutdown_connections 2024-05-17T22:44:41.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.449+0000 7fdf768d4b80 1 --2- 172.21.15.12:0/2751317085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561157aeb20 0x55611582b850 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:41.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.449+0000 7fdf768d4b80 1 --2- 172.21.15.12:0/2751317085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55611562aa70 0x5561157c52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:41.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.449+0000 7fdf768d4b80 1 --2- 172.21.15.12:0/2751317085 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55611582d000 0x55611582f440 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:41.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.449+0000 7fdf768d4b80 1 --2- 172.21.15.12:0/2751317085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5561157a91f0 0x5561157c5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:41.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.449+0000 7fdf768d4b80 1 -- 172.21.15.12:0/2751317085 >> 172.21.15.12:0/2751317085 conn(0x55611566ef80 msgr2=0x5561157bde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:44:41.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.450+0000 7fdf497fa700 0 client.25438 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:44:41.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.452+0000 7fdf768d4b80 1 -- 172.21.15.12:0/2751317085 shutdown_connections 2024-05-17T22:44:41.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.454+0000 7fdf768d4b80 1 -- 172.21.15.12:0/2751317085 wait complete. 2024-05-17T22:44:41.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.454+0000 7fdf768d4b80 20 client.25438 trim_cache size 0 max 16384 2024-05-17T22:44:41.459 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:44:41.460 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-17T22:44:41.460 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "ed106853-1c61-4fce-b527-6c2f90a3daf1", 2024-05-17T22:44:41.460 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-17T22:44:41.460 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:44:41.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.456+0000 7fdf67ef9700 1 -- 172.21.15.12:0/2508218086 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fdf4c0b2900 msgr2=0x7fdf4c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:41.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.456+0000 7fdf67ef9700 1 --2- 172.21.15.12:0/2508218086 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fdf4c0b2900 0x7fdf4c0b4e10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fdf601072c0 tx=0x7fdf50009f80 comp rx=0 tx=0).stop 2024-05-17T22:44:41.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.456+0000 7fdf67ef9700 1 -- 172.21.15.12:0/2508218086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf60103f10 msgr2=0x7fdf6019a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:41.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.456+0000 7fdf67ef9700 1 --2- 172.21.15.12:0/2508218086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf60103f10 0x7fdf6019a530 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00e9a0 tx=0x7fdf5c00ed60 comp rx=0 tx=0).stop 2024-05-17T22:44:41.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.457+0000 7fdf67ef9700 1 -- 172.21.15.12:0/2508218086 shutdown_connections 2024-05-17T22:44:41.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.457+0000 7fdf67ef9700 1 --2- 172.21.15.12:0/2508218086 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fdf4c0b2900 0x7fdf4c0b4e10 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:41.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.457+0000 7fdf67ef9700 1 --2- 172.21.15.12:0/2508218086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf60109190 0x7fdf6019ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:41.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.457+0000 7fdf67ef9700 1 --2- 172.21.15.12:0/2508218086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf60103f10 0x7fdf6019a530 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:41.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.457+0000 7fdf67ef9700 1 --2- 172.21.15.12:0/2508218086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf60108840 0x7fdf6019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:41.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.457+0000 7fdf67ef9700 1 -- 172.21.15.12:0/2508218086 >> 172.21.15.12:0/2508218086 conn(0x7fdf600ff9b0 msgr2=0x7fdf6010a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:44:41.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.457+0000 7fdf67ef9700 1 -- 172.21.15.12:0/2508218086 shutdown_connections 2024-05-17T22:44:41.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:41.457+0000 7fdf67ef9700 1 -- 172.21.15.12:0/2508218086 wait complete. 2024-05-17T22:44:42.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:41 smithi012 ceph-mon[31632]: pgmap v602: 129 pgs: 129 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 512 op/s 2024-05-17T22:44:42.194 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-17T22:44:42.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:41 smithi151 ceph-mon[36101]: pgmap v602: 129 pgs: 129 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 512 op/s 2024-05-17T22:44:42.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:41 smithi096 ceph-mon[36065]: pgmap v602: 129 pgs: 129 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 512 op/s 2024-05-17T22:44:42.734 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:44:43.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:42 smithi012 ceph-mon[31632]: scrub queued for path: / 2024-05-17T22:44:43.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:42 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:44:43.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:42 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:44:43.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:42 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:44:43.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:42 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:44:43.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:42 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:44:43.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:42 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:44:43.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:42 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:44:43.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:42 smithi151 ceph-mon[36101]: scrub queued for path: / 2024-05-17T22:44:43.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:42 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:44:43.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:42 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:44:43.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:42 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:44:43.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:42 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:44:43.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:42 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:44:43.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:42 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:44:43.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:42 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:44:43.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:42 smithi096 ceph-mon[36065]: scrub queued for path: / 2024-05-17T22:44:43.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:42 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:44:43.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:42 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:44:43.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:42 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:44:43.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:42 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:44:43.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:42 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:44:43.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:42 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:44:43.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:42 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:44:44.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:43 smithi012 ceph-mon[31632]: pgmap v603: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 556 op/s 2024-05-17T22:44:44.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:43 smithi151 ceph-mon[36101]: pgmap v603: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 556 op/s 2024-05-17T22:44:44.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:43 smithi096 ceph-mon[36065]: pgmap v603: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 556 op/s 2024-05-17T22:44:46.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:45 smithi012 ceph-mon[31632]: pgmap v604: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 597 op/s 2024-05-17T22:44:46.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:45 smithi151 ceph-mon[36101]: pgmap v604: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 597 op/s 2024-05-17T22:44:46.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:45 smithi096 ceph-mon[36065]: pgmap v604: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 597 op/s 2024-05-17T22:44:46.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.101+0000 7fab275ca700 1 -- 172.21.15.12:0/2944351993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab2011d550 msgr2=0x7fab2011f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:46.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.101+0000 7fab275ca700 1 --2- 172.21.15.12:0/2944351993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab2011d550 0x7fab2011f970 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fab1800b0a0 tx=0x7fab1800b3b0 comp rx=0 tx=0).stop 2024-05-17T22:44:46.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.105+0000 7fab275ca700 1 -- 172.21.15.12:0/2944351993 shutdown_connections 2024-05-17T22:44:46.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.105+0000 7fab275ca700 1 --2- 172.21.15.12:0/2944351993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab2011d550 0x7fab2011f970 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:46.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.105+0000 7fab275ca700 1 --2- 172.21.15.12:0/2944351993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab2010a830 0x7fab2010ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:46.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.105+0000 7fab275ca700 1 --2- 172.21.15.12:0/2944351993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab2010b180 0x7fab2011d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:46.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.105+0000 7fab275ca700 1 -- 172.21.15.12:0/2944351993 >> 172.21.15.12:0/2944351993 conn(0x7fab2006efc0 msgr2=0x7fab2006f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:44:46.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.106+0000 7fab275ca700 1 -- 172.21.15.12:0/2944351993 shutdown_connections 2024-05-17T22:44:46.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.106+0000 7fab275ca700 1 -- 172.21.15.12:0/2944351993 wait complete. 2024-05-17T22:44:46.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.107+0000 7fab275ca700 1 Processor -- start 2024-05-17T22:44:46.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.107+0000 7fab275ca700 1 -- start start 2024-05-17T22:44:46.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.108+0000 7fab275ca700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab2010a830 0x7fab201a9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:46.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.108+0000 7fab275ca700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab2010b180 0x7fab201a9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:46.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.108+0000 7fab275ca700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab2011d550 0x7fab201adf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:46.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.108+0000 7fab275ca700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fab20122cc0 con 0x7fab2010a830 2024-05-17T22:44:46.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.108+0000 7fab275ca700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fab20122b40 con 0x7fab2010b180 2024-05-17T22:44:46.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.108+0000 7fab275ca700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fab20122e40 con 0x7fab2011d550 2024-05-17T22:44:46.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.108+0000 7fab25366700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab2010a830 0x7fab201a9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:46.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.108+0000 7fab25366700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab2010a830 0x7fab201a9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41920/0 (socket says 172.21.15.12:41920) 2024-05-17T22:44:46.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.108+0000 7fab25366700 1 -- 172.21.15.12:0/2179040224 learned_addr learned my addr 172.21.15.12:0/2179040224 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:44:46.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.110+0000 7fab24b65700 1 --2- 172.21.15.12:0/2179040224 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab2010b180 0x7fab201a9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:46.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.110+0000 7fab25b67700 1 --2- 172.21.15.12:0/2179040224 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab2011d550 0x7fab201adf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:46.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.110+0000 7fab25b67700 1 -- 172.21.15.12:0/2179040224 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab2010b180 msgr2=0x7fab201a9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:46.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.110+0000 7fab25b67700 1 --2- 172.21.15.12:0/2179040224 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab2010b180 0x7fab201a9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:46.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.110+0000 7fab25b67700 1 -- 172.21.15.12:0/2179040224 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab2010a830 msgr2=0x7fab201a9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:46.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.110+0000 7fab25b67700 1 --2- 172.21.15.12:0/2179040224 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab2010a830 0x7fab201a9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:46.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.110+0000 7fab25b67700 1 -- 172.21.15.12:0/2179040224 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab201ae630 con 0x7fab2011d550 2024-05-17T22:44:46.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.111+0000 7fab25b67700 1 --2- 172.21.15.12:0/2179040224 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab2011d550 0x7fab201adf90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fab18009fd0 tx=0x7fab18004cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:44:46.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.114+0000 7fab167fc700 1 -- 172.21.15.12:0/2179040224 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab18021070 con 0x7fab2011d550 2024-05-17T22:44:46.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.114+0000 7fab275ca700 1 -- 172.21.15.12:0/2179040224 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab201ae8c0 con 0x7fab2011d550 2024-05-17T22:44:46.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.114+0000 7fab275ca700 1 -- 172.21.15.12:0/2179040224 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab201b6240 con 0x7fab2011d550 2024-05-17T22:44:46.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.119+0000 7fab35fa5b80 1 Processor -- start 2024-05-17T22:44:46.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.120+0000 7fab35fa5b80 1 -- start start 2024-05-17T22:44:46.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.120+0000 7fab35fa5b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570e85b89f0 0x5570e873c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:46.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.120+0000 7fab35fa5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570e873caa0 0x5570e8736c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:46.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.121+0000 7fab35fa5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e8737170 0x5570e8737580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:46.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.121+0000 7fab35fa5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5570e8665290 con 0x5570e873caa0 2024-05-17T22:44:46.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.121+0000 7fab35fa5b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5570e8654fd0 con 0x5570e85b89f0 2024-05-17T22:44:46.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.121+0000 7fab35fa5b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5570e8658e70 con 0x5570e8737170 2024-05-17T22:44:46.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.121+0000 7fab167fc700 1 -- 172.21.15.12:0/2179040224 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab18007b80 con 0x7fab2011d550 2024-05-17T22:44:46.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.121+0000 7fab167fc700 1 -- 172.21.15.12:0/2179040224 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab180052a0 con 0x7fab2011d550 2024-05-17T22:44:46.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.122+0000 7fab25366700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570e873caa0 0x5570e8736c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:46.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.122+0000 7fab25366700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570e873caa0 0x5570e8736c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41936/0 (socket says 172.21.15.12:41936) 2024-05-17T22:44:46.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.122+0000 7fab25366700 1 -- 172.21.15.12:0/265882856 learned_addr learned my addr 172.21.15.12:0/265882856 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:44:46.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.125+0000 7fab25366700 1 -- 172.21.15.12:0/265882856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e8737170 msgr2=0x5570e8737580 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T22:44:46.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.125+0000 7fab25366700 1 --2- 172.21.15.12:0/265882856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e8737170 0x5570e8737580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:46.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.125+0000 7fab25366700 1 -- 172.21.15.12:0/265882856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570e85b89f0 msgr2=0x5570e873c560 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T22:44:46.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.125+0000 7fab25366700 1 --2- 172.21.15.12:0/265882856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570e85b89f0 0x5570e873c560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:46.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.125+0000 7fab25366700 1 -- 172.21.15.12:0/265882856 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570e85fe590 con 0x5570e873caa0 2024-05-17T22:44:46.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.131+0000 7fab167fc700 1 -- 172.21.15.12:0/2179040224 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fab18024070 con 0x7fab2011d550 2024-05-17T22:44:46.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.132+0000 7fab167fc700 1 --2- 172.21.15.12:0/2179040224 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fab0c0b2a90 0x7fab0c0b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:46.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.132+0000 7fab25366700 1 --2- 172.21.15.12:0/265882856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570e873caa0 0x5570e8736c30 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fab1c009bb0 tx=0x7fab1c009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cab189e370e4a6aa server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:44:46.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.132+0000 7fab167fc700 1 -- 172.21.15.12:0/2179040224 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fab180e38e0 con 0x7fab2011d550 2024-05-17T22:44:46.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.133+0000 7fab0b7fe700 1 -- 172.21.15.12:0/265882856 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1c017070 con 0x5570e873caa0 2024-05-17T22:44:46.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.133+0000 7fab0b7fe700 1 -- 172.21.15.12:0/265882856 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab1c013470 con 0x5570e873caa0 2024-05-17T22:44:46.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.135+0000 7fab35fa5b80 1 -- 172.21.15.12:0/265882856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570e873caa0 msgr2=0x5570e8736c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:46.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.135+0000 7fab35fa5b80 1 --2- 172.21.15.12:0/265882856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570e873caa0 0x5570e8736c30 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fab1c009bb0 tx=0x7fab1c009ec0 comp rx=0 tx=0).stop 2024-05-17T22:44:46.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.135+0000 7fab35fa5b80 1 -- 172.21.15.12:0/265882856 shutdown_connections 2024-05-17T22:44:46.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.135+0000 7fab35fa5b80 1 --2- 172.21.15.12:0/265882856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e8737170 0x5570e8737580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:46.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.135+0000 7fab35fa5b80 1 --2- 172.21.15.12:0/265882856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570e85b89f0 0x5570e873c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:46.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.135+0000 7fab35fa5b80 1 --2- 172.21.15.12:0/265882856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570e873caa0 0x5570e8736c30 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:46.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.135+0000 7fab35fa5b80 1 -- 172.21.15.12:0/265882856 >> 172.21.15.12:0/265882856 conn(0x5570e85fcf00 msgr2=0x5570e8731b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:44:46.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.140+0000 7fab35fa5b80 1 -- 172.21.15.12:0/265882856 shutdown_connections 2024-05-17T22:44:46.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.142+0000 7fab24b65700 1 --2- 172.21.15.12:0/2179040224 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fab0c0b2a90 0x7fab0c0b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:46.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.144+0000 7fab35fa5b80 1 -- 172.21.15.12:0/265882856 wait complete. 2024-05-17T22:44:46.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.144+0000 7fab35fa5b80 1 -- start start 2024-05-17T22:44:46.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.144+0000 7fab35fa5b80 1 Processor -- start 2024-05-17T22:44:46.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.145+0000 7fab35fa5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570e85b89f0 0x5570e87401d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:46.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.145+0000 7fab35fa5b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570e8737170 0x5570e8740710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:46.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.145+0000 7fab35fa5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e873caa0 0x5570e8740c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:46.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.145+0000 7fab35fa5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5570e855c150 con 0x5570e85b89f0 2024-05-17T22:44:46.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.145+0000 7fab35fa5b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5570e85f5e80 con 0x5570e8737170 2024-05-17T22:44:46.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.145+0000 7fab35fa5b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5570e87433e0 con 0x5570e873caa0 2024-05-17T22:44:46.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.145+0000 7fab35fa5b80 10 client.0 _ll_register_callbacks cb 0x5570e85eff10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:44:46.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.146+0000 7fab25366700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570e85b89f0 0x5570e87401d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:46.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.146+0000 7fab25366700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570e85b89f0 0x5570e87401d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41942/0 (socket says 172.21.15.12:41942) 2024-05-17T22:44:46.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.146+0000 7fab25366700 1 -- 172.21.15.12:0/2734965871 learned_addr learned my addr 172.21.15.12:0/2734965871 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:44:46.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.146+0000 7fab25b67700 1 --2- 172.21.15.12:0/2734965871 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e873caa0 0x5570e8740c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:46.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.150+0000 7fab25b67700 1 -- 172.21.15.12:0/2734965871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570e8737170 msgr2=0x5570e8740710 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T22:44:46.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.150+0000 7fab25b67700 1 --2- 172.21.15.12:0/2734965871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570e8737170 0x5570e8740710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:46.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.150+0000 7fab25b67700 1 -- 172.21.15.12:0/2734965871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570e85b89f0 msgr2=0x5570e87401d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:44:46.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.150+0000 7fab25b67700 1 --2- 172.21.15.12:0/2734965871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570e85b89f0 0x5570e87401d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:46.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.150+0000 7fab25b67700 1 -- 172.21.15.12:0/2734965871 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570e85a5480 con 0x5570e873caa0 2024-05-17T22:44:46.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.152+0000 7fab25b67700 1 --2- 172.21.15.12:0/2734965871 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e873caa0 0x5570e8740c50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fab180e0ff0 tx=0x7fab180aa310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4177a6ae26e2820c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:44:46.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.153+0000 7fab08ff9700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:44:46.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.153+0000 7fab24b65700 1 --2- 172.21.15.12:0/2179040224 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fab0c0b2a90 0x7fab0c0b4fa0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x5570e8732ae0 tx=0x7fab10009490 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:44:46.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.154+0000 7fab08ff9700 1 -- 172.21.15.12:0/2734965871 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1802b070 con 0x5570e873caa0 2024-05-17T22:44:46.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.154+0000 7fab08ff9700 1 -- 172.21.15.12:0/2734965871 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fab180a7190 con 0x5570e873caa0 2024-05-17T22:44:46.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.155+0000 7fab35fa5b80 1 -- 172.21.15.12:0/2734965871 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5570e86671e0 con 0x5570e873caa0 2024-05-17T22:44:46.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.159+0000 7fab08ff9700 1 -- 172.21.15.12:0/2734965871 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fab180aa6b0 con 0x5570e873caa0 2024-05-17T22:44:46.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.159+0000 7fab35fa5b80 10 client.25447 fetch_fsmap learned FSMap version 34 2024-05-17T22:44:46.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.159+0000 7fab35fa5b80 1 -- 172.21.15.12:0/2734965871 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5570e875ef30 con 0x5570e873caa0 2024-05-17T22:44:46.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.160+0000 7fab08ff9700 1 -- 172.21.15.12:0/2734965871 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5570e86671e0 con 0x5570e873caa0 2024-05-17T22:44:46.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.160+0000 7fab08ff9700 1 -- 172.21.15.12:0/2734965871 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7fab18011070 con 0x5570e873caa0 2024-05-17T22:44:46.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.160+0000 7fab35fa5b80 10 client.25447 fetch_fsmap finished waiting for FSMap version 34 2024-05-17T22:44:46.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.160+0000 7fab35fa5b80 10 client.25447 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:44:46.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.160+0000 7fab35fa5b80 20 client.25447 populate_metadata read hostname 'smithi012' 2024-05-17T22:44:46.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.160+0000 7fab35fa5b80 1 --2- 172.21.15.12:0/2734965871 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5570e8761570 0x5570e8763a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:46.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.160+0000 7fab35fa5b80 4 client.25447 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:44:46.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.160+0000 7fab35fa5b80 1 -- 172.21.15.12:0/2734965871 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570e86609c0 con 0x5570e8761570 2024-05-17T22:44:46.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.161+0000 7fab25366700 1 --2- 172.21.15.12:0/2734965871 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5570e8761570 0x5570e8763a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:46.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.161+0000 7fab08ff9700 10 client.25447 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:44:46.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.162+0000 7fab25366700 1 --2- 172.21.15.12:0/2734965871 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5570e8761570 0x5570e8763a00 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db947b44f40362ab server_cookie=309c2916762b192a in_seq=0 out_seq=0 2024-05-17T22:44:46.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.166+0000 7fab08ff9700 1 -- 172.21.15.12:0/2734965871 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5570e86609c0 con 0x5570e8761570 2024-05-17T22:44:46.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.167+0000 7fab08ff9700 10 client.25447 handle_command_reply: tid=0 2024-05-17T22:44:46.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.167+0000 7fab35fa5b80 1 client.25447 shutdown 2024-05-17T22:44:46.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.171+0000 7fab35fa5b80 1 -- 172.21.15.12:0/2734965871 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e873caa0 msgr2=0x5570e8740c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:46.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.171+0000 7fab35fa5b80 1 --2- 172.21.15.12:0/2734965871 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e873caa0 0x5570e8740c50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fab180e0ff0 tx=0x7fab180aa310 comp rx=0 tx=0).stop 2024-05-17T22:44:46.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.172+0000 7fab35fa5b80 1 -- 172.21.15.12:0/2734965871 shutdown_connections 2024-05-17T22:44:46.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.172+0000 7fab35fa5b80 1 --2- 172.21.15.12:0/2734965871 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e873caa0 0x5570e8740c50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:46.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.172+0000 7fab35fa5b80 1 --2- 172.21.15.12:0/2734965871 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5570e8761570 0x5570e8763a00 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:46.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.172+0000 7fab35fa5b80 1 --2- 172.21.15.12:0/2734965871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570e85b89f0 0x5570e87401d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:46.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.172+0000 7fab35fa5b80 1 --2- 172.21.15.12:0/2734965871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570e8737170 0x5570e8740710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:46.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.172+0000 7fab35fa5b80 1 -- 172.21.15.12:0/2734965871 >> 172.21.15.12:0/2734965871 conn(0x5570e85fcf00 msgr2=0x5570e872bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:44:46.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.172+0000 7fab08ff9700 0 client.25447 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:44:46.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.172+0000 7fab35fa5b80 1 -- 172.21.15.12:0/2734965871 shutdown_connections 2024-05-17T22:44:46.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.172+0000 7fab35fa5b80 1 -- 172.21.15.12:0/2734965871 wait complete. 2024-05-17T22:44:46.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.172+0000 7fab35fa5b80 20 client.25447 trim_cache size 0 max 16384 2024-05-17T22:44:46.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.197+0000 7fab35fa5b80 1 Processor -- start 2024-05-17T22:44:46.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.198+0000 7fab35fa5b80 1 -- start start 2024-05-17T22:44:46.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.198+0000 7fab35fa5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570e85b89f0 0x5570e8736c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:46.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.198+0000 7fab35fa5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e8737170 0x5570e8742980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:46.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.198+0000 7fab35fa5b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570e873caa0 0x5570e872beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:46.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.198+0000 7fab35fa5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fab04011560 con 0x5570e85b89f0 2024-05-17T22:44:46.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.198+0000 7fab25366700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570e85b89f0 0x5570e8736c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:46.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.198+0000 7fab25366700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570e85b89f0 0x5570e8736c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41948/0 (socket says 172.21.15.12:41948) 2024-05-17T22:44:46.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.198+0000 7fab25366700 1 -- 172.21.15.12:0/1798981709 learned_addr learned my addr 172.21.15.12:0/1798981709 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:44:46.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.198+0000 7fab35fa5b80 1 -- 172.21.15.12:0/1798981709 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fab0400edc0 con 0x5570e873caa0 2024-05-17T22:44:46.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.198+0000 7fab35fa5b80 1 -- 172.21.15.12:0/1798981709 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fab0400cc10 con 0x5570e8737170 2024-05-17T22:44:46.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.198+0000 7fab24b65700 1 --2- 172.21.15.12:0/1798981709 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e8737170 0x5570e8742980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:46.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.198+0000 7fab25b67700 1 --2- 172.21.15.12:0/1798981709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570e873caa0 0x5570e872beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:46.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.199+0000 7fab25366700 1 -- 172.21.15.12:0/1798981709 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e8737170 msgr2=0x5570e8742980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:44:46.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.199+0000 7fab25366700 1 --2- 172.21.15.12:0/1798981709 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e8737170 0x5570e8742980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:46.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.199+0000 7fab25366700 1 -- 172.21.15.12:0/1798981709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570e873caa0 msgr2=0x5570e872beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:44:46.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.199+0000 7fab25366700 1 --2- 172.21.15.12:0/1798981709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570e873caa0 0x5570e872beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:46.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.199+0000 7fab25366700 1 -- 172.21.15.12:0/1798981709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570e875e290 con 0x5570e85b89f0 2024-05-17T22:44:46.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.199+0000 7fab25366700 1 --2- 172.21.15.12:0/1798981709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570e85b89f0 0x5570e8736c30 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fab1c009b00 tx=0x7fab1c009400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8cd9308fdacd5f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:44:46.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.199+0000 7fab0affd700 1 -- 172.21.15.12:0/1798981709 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1c02d070 con 0x5570e85b89f0 2024-05-17T22:44:46.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.199+0000 7fab0affd700 1 -- 172.21.15.12:0/1798981709 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab1c009660 con 0x5570e85b89f0 2024-05-17T22:44:46.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.199+0000 7fab0affd700 1 -- 172.21.15.12:0/1798981709 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1c017040 con 0x5570e85b89f0 2024-05-17T22:44:46.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.200+0000 7fab35fa5b80 1 -- 172.21.15.12:0/1798981709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570e85b89f0 msgr2=0x5570e8736c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:46.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.200+0000 7fab35fa5b80 1 --2- 172.21.15.12:0/1798981709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570e85b89f0 0x5570e8736c30 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fab1c009b00 tx=0x7fab1c009400 comp rx=0 tx=0).stop 2024-05-17T22:44:46.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.200+0000 7fab35fa5b80 1 -- 172.21.15.12:0/1798981709 shutdown_connections 2024-05-17T22:44:46.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.200+0000 7fab35fa5b80 1 --2- 172.21.15.12:0/1798981709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570e873caa0 0x5570e872beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:46.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.200+0000 7fab35fa5b80 1 --2- 172.21.15.12:0/1798981709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570e85b89f0 0x5570e8736c30 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:46.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.200+0000 7fab35fa5b80 1 --2- 172.21.15.12:0/1798981709 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e8737170 0x5570e8742980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:46.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.200+0000 7fab35fa5b80 1 -- 172.21.15.12:0/1798981709 >> 172.21.15.12:0/1798981709 conn(0x5570e85fcf00 msgr2=0x5570e873a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:44:46.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.200+0000 7fab35fa5b80 1 -- 172.21.15.12:0/1798981709 shutdown_connections 2024-05-17T22:44:46.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.200+0000 7fab35fa5b80 1 -- 172.21.15.12:0/1798981709 wait complete. 2024-05-17T22:44:46.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.200+0000 7fab35fa5b80 1 -- start start 2024-05-17T22:44:46.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.200+0000 7fab35fa5b80 1 Processor -- start 2024-05-17T22:44:46.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.201+0000 7fab35fa5b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570e85b89f0 0x5570e8748860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:46.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.201+0000 7fab35fa5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e8737170 0x5570e8748da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:46.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.201+0000 7fab35fa5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570e873caa0 0x5570e8768740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:46.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.201+0000 7fab35fa5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fab0400d280 con 0x5570e873caa0 2024-05-17T22:44:46.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.201+0000 7fab35fa5b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fab0400e6f0 con 0x5570e85b89f0 2024-05-17T22:44:46.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.201+0000 7fab35fa5b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fab0400d950 con 0x5570e8737170 2024-05-17T22:44:46.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.201+0000 7fab35fa5b80 10 client.0 _ll_register_callbacks cb 0x5570e85eff10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:44:46.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.201+0000 7fab25b67700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570e873caa0 0x5570e8768740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:46.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.201+0000 7fab25b67700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570e873caa0 0x5570e8768740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41960/0 (socket says 172.21.15.12:41960) 2024-05-17T22:44:46.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.201+0000 7fab25b67700 1 -- 172.21.15.12:0/731660280 learned_addr learned my addr 172.21.15.12:0/731660280 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:44:46.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.201+0000 7fab24b65700 1 --2- 172.21.15.12:0/731660280 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e8737170 0x5570e8748da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:46.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.201+0000 7fab25366700 1 --2- 172.21.15.12:0/731660280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570e85b89f0 0x5570e8748860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:46.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.201+0000 7fab25b67700 1 -- 172.21.15.12:0/731660280 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e8737170 msgr2=0x5570e8748da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:44:46.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.201+0000 7fab25b67700 1 --2- 172.21.15.12:0/731660280 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e8737170 0x5570e8748da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:46.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.201+0000 7fab25b67700 1 -- 172.21.15.12:0/731660280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570e85b89f0 msgr2=0x5570e8748860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:44:46.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.201+0000 7fab25b67700 1 --2- 172.21.15.12:0/731660280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570e85b89f0 0x5570e8748860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:46.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.201+0000 7fab25b67700 1 -- 172.21.15.12:0/731660280 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570e8768eb0 con 0x5570e873caa0 2024-05-17T22:44:46.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.201+0000 7fab25366700 1 --2- 172.21.15.12:0/731660280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570e85b89f0 0x5570e8748860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:44:46.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.201+0000 7fab24b65700 1 --2- 172.21.15.12:0/731660280 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e8737170 0x5570e8748da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:44:46.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.202+0000 7fab25b67700 1 --2- 172.21.15.12:0/731660280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570e873caa0 0x5570e8768740 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fab180a6c20 tx=0x7fab1801dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f73b55e4d433153 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:44:46.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.202+0000 7fab08ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:44:46.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.202+0000 7fab08ff9700 1 -- 172.21.15.12:0/731660280 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1802b070 con 0x5570e873caa0 2024-05-17T22:44:46.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.202+0000 7fab08ff9700 1 -- 172.21.15.12:0/731660280 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fab180a7190 con 0x5570e873caa0 2024-05-17T22:44:46.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.202+0000 7fab35fa5b80 1 -- 172.21.15.12:0/731660280 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab1c009660 con 0x5570e873caa0 2024-05-17T22:44:46.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.202+0000 7fab08ff9700 1 -- 172.21.15.12:0/731660280 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fab180b3970 con 0x5570e873caa0 2024-05-17T22:44:46.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.202+0000 7fab35fa5b80 10 client.15843 fetch_fsmap learned FSMap version 34 2024-05-17T22:44:46.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.202+0000 7fab35fa5b80 1 -- 172.21.15.12:0/731660280 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5570e87cb940 con 0x5570e873caa0 2024-05-17T22:44:46.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.202+0000 7fab08ff9700 1 -- 172.21.15.12:0/731660280 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab1c009660 con 0x5570e873caa0 2024-05-17T22:44:46.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.202+0000 7fab08ff9700 1 -- 172.21.15.12:0/731660280 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7fab18011070 con 0x5570e873caa0 2024-05-17T22:44:46.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.202+0000 7fab35fa5b80 10 client.15843 fetch_fsmap finished waiting for FSMap version 34 2024-05-17T22:44:46.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.202+0000 7fab35fa5b80 10 client.15843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:44:46.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.202+0000 7fab35fa5b80 20 client.15843 populate_metadata read hostname 'smithi012' 2024-05-17T22:44:46.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.202+0000 7fab35fa5b80 1 --2- 172.21.15.12:0/731660280 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5570e87cbe60 0x5570e87ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:44:46.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.202+0000 7fab35fa5b80 4 client.15843 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:44:46.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.202+0000 7fab35fa5b80 1 -- 172.21.15.12:0/731660280 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5570e8531360 con 0x5570e87cbe60 2024-05-17T22:44:46.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.203+0000 7fab25366700 1 --2- 172.21.15.12:0/731660280 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5570e87cbe60 0x5570e87ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:44:46.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.203+0000 7fab25366700 1 --2- 172.21.15.12:0/731660280 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5570e87cbe60 0x5570e87ce280 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b317f676880ea6a6 server_cookie=61201bb7817ba94 in_seq=0 out_seq=0 2024-05-17T22:44:46.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.203+0000 7fab08ff9700 10 client.15843 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:44:46.340 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:44:46.340 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-17T22:44:46.341 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-17T22:44:46.341 INFO:teuthology.orchestra.run.smithi012.stdout: "ed106853-1c61-4fce-b527-6c2f90a3daf1": { 2024-05-17T22:44:46.341 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-17T22:44:46.341 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "ed106853-1c61-4fce-b527-6c2f90a3daf1", 2024-05-17T22:44:46.341 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-17T22:44:46.341 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:44:46.341 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:44:46.341 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:44:46.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.203+0000 7fab08ff9700 1 -- 172.21.15.12:0/731660280 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5570e8531360 con 0x5570e87cbe60 2024-05-17T22:44:46.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.203+0000 7fab08ff9700 10 client.15843 handle_command_reply: tid=0 2024-05-17T22:44:46.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.203+0000 7fab35fa5b80 1 client.15843 shutdown 2024-05-17T22:44:46.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.203+0000 7fab35fa5b80 1 -- 172.21.15.12:0/731660280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570e873caa0 msgr2=0x5570e8768740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:46.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.203+0000 7fab35fa5b80 1 --2- 172.21.15.12:0/731660280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570e873caa0 0x5570e8768740 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fab180a6c20 tx=0x7fab1801dc40 comp rx=0 tx=0).stop 2024-05-17T22:44:46.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.203+0000 7fab35fa5b80 1 -- 172.21.15.12:0/731660280 shutdown_connections 2024-05-17T22:44:46.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.203+0000 7fab35fa5b80 1 --2- 172.21.15.12:0/731660280 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5570e87cbe60 0x5570e87ce280 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:46.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.203+0000 7fab35fa5b80 1 --2- 172.21.15.12:0/731660280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570e873caa0 0x5570e8768740 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:46.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.203+0000 7fab35fa5b80 1 --2- 172.21.15.12:0/731660280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570e85b89f0 0x5570e8748860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:46.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.203+0000 7fab35fa5b80 1 --2- 172.21.15.12:0/731660280 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e8737170 0x5570e8748da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:46.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.204+0000 7fab08ff9700 0 client.15843 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:44:46.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.203+0000 7fab35fa5b80 1 -- 172.21.15.12:0/731660280 >> 172.21.15.12:0/731660280 conn(0x5570e85fcf00 msgr2=0x5570e873a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:44:46.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.204+0000 7fab35fa5b80 1 -- 172.21.15.12:0/731660280 shutdown_connections 2024-05-17T22:44:46.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.204+0000 7fab35fa5b80 1 -- 172.21.15.12:0/731660280 wait complete. 2024-05-17T22:44:46.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.204+0000 7fab35fa5b80 20 client.15843 trim_cache size 0 max 16384 2024-05-17T22:44:46.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.204+0000 7fab275ca700 1 -- 172.21.15.12:0/2179040224 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fab0c0b2a90 msgr2=0x7fab0c0b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:46.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.204+0000 7fab275ca700 1 --2- 172.21.15.12:0/2179040224 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fab0c0b2a90 0x7fab0c0b4fa0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x5570e8732ae0 tx=0x7fab10009490 comp rx=0 tx=0).stop 2024-05-17T22:44:46.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.204+0000 7fab275ca700 1 -- 172.21.15.12:0/2179040224 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab2011d550 msgr2=0x7fab201adf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:44:46.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.204+0000 7fab275ca700 1 --2- 172.21.15.12:0/2179040224 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab2011d550 0x7fab201adf90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fab18009fd0 tx=0x7fab18004cd0 comp rx=0 tx=0).stop 2024-05-17T22:44:46.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.204+0000 7fab275ca700 1 -- 172.21.15.12:0/2179040224 shutdown_connections 2024-05-17T22:44:46.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.204+0000 7fab275ca700 1 --2- 172.21.15.12:0/2179040224 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fab0c0b2a90 0x7fab0c0b4fa0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:46.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.204+0000 7fab275ca700 1 --2- 172.21.15.12:0/2179040224 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab2011d550 0x7fab201adf90 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:46.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.204+0000 7fab275ca700 1 --2- 172.21.15.12:0/2179040224 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab2010a830 0x7fab201a9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:46.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.204+0000 7fab275ca700 1 --2- 172.21.15.12:0/2179040224 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab2010b180 0x7fab201a9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:44:46.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.204+0000 7fab275ca700 1 -- 172.21.15.12:0/2179040224 >> 172.21.15.12:0/2179040224 conn(0x7fab2006efc0 msgr2=0x7fab2011b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:44:46.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.204+0000 7fab275ca700 1 -- 172.21.15.12:0/2179040224 shutdown_connections 2024-05-17T22:44:46.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:44:46.204+0000 7fab275ca700 1 -- 172.21.15.12:0/2179040224 wait complete. 2024-05-17T22:44:46.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 325.0 s, 587.6 tps, lat 169.636 ms stddev 198.729 2024-05-17T22:44:47.531 INFO:tasks.cephfs.filesystem:scrub status for tag:ed106853-1c61-4fce-b527-6c2f90a3daf1 - {'path': '/', 'tag': 'ed106853-1c61-4fce-b527-6c2f90a3daf1', 'options': 'recursive,force'} 2024-05-17T22:44:48.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:47 smithi151 ceph-mon[36101]: pgmap v605: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 491 op/s 2024-05-17T22:44:48.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:47 smithi096 ceph-mon[36065]: pgmap v605: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 491 op/s 2024-05-17T22:44:48.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:47 smithi012 ceph-mon[31632]: pgmap v605: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 491 op/s 2024-05-17T22:44:49.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:48 smithi151 ceph-mon[36101]: scrub summary: idle 2024-05-17T22:44:49.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:48 smithi096 ceph-mon[36065]: scrub summary: idle 2024-05-17T22:44:49.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:48 smithi012 ceph-mon[31632]: scrub summary: idle 2024-05-17T22:44:50.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:49 smithi151 ceph-mon[36101]: scrub summary: idle 2024-05-17T22:44:50.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:49 smithi151 ceph-mon[36101]: scrub summary: idle 2024-05-17T22:44:50.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:49 smithi151 ceph-mon[36101]: pgmap v606: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 594 op/s 2024-05-17T22:44:50.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:49 smithi096 ceph-mon[36065]: scrub summary: idle 2024-05-17T22:44:50.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:49 smithi096 ceph-mon[36065]: scrub summary: idle 2024-05-17T22:44:50.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:49 smithi096 ceph-mon[36065]: pgmap v606: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 594 op/s 2024-05-17T22:44:50.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:49 smithi012 ceph-mon[31632]: scrub summary: idle 2024-05-17T22:44:50.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:49 smithi012 ceph-mon[31632]: scrub summary: idle 2024-05-17T22:44:50.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:49 smithi012 ceph-mon[31632]: pgmap v606: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 594 op/s 2024-05-17T22:44:51.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 330.0 s, 615.0 tps, lat 162.554 ms stddev 181.730 2024-05-17T22:44:52.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:51 smithi151 ceph-mon[36101]: pgmap v607: 129 pgs: 129 active+clean; 1.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 11 MiB/s wr, 498 op/s 2024-05-17T22:44:52.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:51 smithi096 ceph-mon[36065]: pgmap v607: 129 pgs: 129 active+clean; 1.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 11 MiB/s wr, 498 op/s 2024-05-17T22:44:52.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:51 smithi012 ceph-mon[31632]: pgmap v607: 129 pgs: 129 active+clean; 1.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 11 MiB/s wr, 498 op/s 2024-05-17T22:44:54.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:53 smithi151 ceph-mon[36101]: pgmap v608: 129 pgs: 129 active+clean; 1.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 13 MiB/s wr, 557 op/s 2024-05-17T22:44:54.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:53 smithi096 ceph-mon[36065]: pgmap v608: 129 pgs: 129 active+clean; 1.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 13 MiB/s wr, 557 op/s 2024-05-17T22:44:54.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:53 smithi012 ceph-mon[31632]: pgmap v608: 129 pgs: 129 active+clean; 1.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 13 MiB/s wr, 557 op/s 2024-05-17T22:44:55.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:54 smithi151 ceph-mon[36101]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:44:55.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:54 smithi096 ceph-mon[36065]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:44:55.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:54 smithi012 ceph-mon[31632]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:44:56.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:55 smithi151 ceph-mon[36101]: pgmap v609: 129 pgs: 129 active+clean; 1.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 599 op/s 2024-05-17T22:44:56.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:55 smithi096 ceph-mon[36065]: pgmap v609: 129 pgs: 129 active+clean; 1.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 599 op/s 2024-05-17T22:44:56.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:55 smithi012 ceph-mon[31632]: pgmap v609: 129 pgs: 129 active+clean; 1.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 599 op/s 2024-05-17T22:44:56.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 335.0 s, 791.2 tps, lat 123.571 ms stddev 156.476 2024-05-17T22:44:58.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:57 smithi151 ceph-mon[36101]: pgmap v610: 129 pgs: 129 active+clean; 1.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 521 op/s 2024-05-17T22:44:58.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:57 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:44:58.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:57 smithi096 ceph-mon[36065]: pgmap v610: 129 pgs: 129 active+clean; 1.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 521 op/s 2024-05-17T22:44:58.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:57 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:44:58.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:57 smithi012 ceph-mon[31632]: pgmap v610: 129 pgs: 129 active+clean; 1.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 521 op/s 2024-05-17T22:44:58.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:57 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:45:00.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:59 smithi151 ceph-mon[36101]: pgmap v611: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 14 MiB/s wr, 670 op/s 2024-05-17T22:45:00.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:59 smithi151 ceph-mon[36101]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:45:00.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:59 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:45:00.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:59 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:45:00.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:44:59 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:45:00.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:59 smithi096 ceph-mon[36065]: pgmap v611: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 14 MiB/s wr, 670 op/s 2024-05-17T22:45:00.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:59 smithi096 ceph-mon[36065]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:45:00.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:59 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:45:00.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:59 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:45:00.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:44:59 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:45:00.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:59 smithi012 ceph-mon[31632]: pgmap v611: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 14 MiB/s wr, 670 op/s 2024-05-17T22:45:00.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:59 smithi012 ceph-mon[31632]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:45:00.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:59 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:45:00.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:59 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:45:00.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:44:59 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:45:01.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 340.0 s, 617.0 tps, lat 163.508 ms stddev 198.120 2024-05-17T22:45:02.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:01 smithi151 ceph-mon[36101]: pgmap v612: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 12 MiB/s wr, 554 op/s 2024-05-17T22:45:02.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:01 smithi151 ceph-mon[36101]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:45:02.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:01 smithi096 ceph-mon[36065]: pgmap v612: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 12 MiB/s wr, 554 op/s 2024-05-17T22:45:02.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:01 smithi096 ceph-mon[36065]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:45:02.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:01 smithi012 ceph-mon[31632]: pgmap v612: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 12 MiB/s wr, 554 op/s 2024-05-17T22:45:02.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:01 smithi012 ceph-mon[31632]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:45:04.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:03 smithi151 ceph-mon[36101]: pgmap v613: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 13 MiB/s wr, 623 op/s 2024-05-17T22:45:04.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:03 smithi096 ceph-mon[36065]: pgmap v613: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 13 MiB/s wr, 623 op/s 2024-05-17T22:45:04.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:03 smithi012 ceph-mon[31632]: pgmap v613: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 13 MiB/s wr, 623 op/s 2024-05-17T22:45:06.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:05 smithi151 ceph-mon[36101]: pgmap v614: 129 pgs: 129 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 13 MiB/s wr, 619 op/s 2024-05-17T22:45:06.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:05 smithi096 ceph-mon[36065]: pgmap v614: 129 pgs: 129 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 13 MiB/s wr, 619 op/s 2024-05-17T22:45:06.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:05 smithi012 ceph-mon[31632]: pgmap v614: 129 pgs: 129 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 13 MiB/s wr, 619 op/s 2024-05-17T22:45:06.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 345.0 s, 665.2 tps, lat 155.426 ms stddev 191.302 2024-05-17T22:45:08.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:07 smithi151 ceph-mon[36101]: pgmap v615: 129 pgs: 129 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 10 MiB/s wr, 520 op/s 2024-05-17T22:45:08.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:07 smithi096 ceph-mon[36065]: pgmap v615: 129 pgs: 129 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 10 MiB/s wr, 520 op/s 2024-05-17T22:45:08.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:07 smithi012 ceph-mon[31632]: pgmap v615: 129 pgs: 129 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 10 MiB/s wr, 520 op/s 2024-05-17T22:45:10.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:09 smithi151 ceph-mon[36101]: pgmap v616: 129 pgs: 129 active+clean; 1.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 14 MiB/s wr, 653 op/s 2024-05-17T22:45:10.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:09 smithi096 ceph-mon[36065]: pgmap v616: 129 pgs: 129 active+clean; 1.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 14 MiB/s wr, 653 op/s 2024-05-17T22:45:10.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:09 smithi012 ceph-mon[31632]: pgmap v616: 129 pgs: 129 active+clean; 1.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 14 MiB/s wr, 653 op/s 2024-05-17T22:45:11.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 350.0 s, 574.0 tps, lat 172.076 ms stddev 180.321 2024-05-17T22:45:12.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:11 smithi151 ceph-mon[36101]: pgmap v617: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.1 MiB/s wr, 478 op/s 2024-05-17T22:45:12.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:11 smithi096 ceph-mon[36065]: pgmap v617: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.1 MiB/s wr, 478 op/s 2024-05-17T22:45:12.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:11 smithi012 ceph-mon[31632]: pgmap v617: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.1 MiB/s wr, 478 op/s 2024-05-17T22:45:14.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:13 smithi151 ceph-mon[36101]: pgmap v618: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 MiB/s wr, 538 op/s 2024-05-17T22:45:14.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:13 smithi096 ceph-mon[36065]: pgmap v618: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 MiB/s wr, 538 op/s 2024-05-17T22:45:14.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:13 smithi012 ceph-mon[31632]: pgmap v618: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 MiB/s wr, 538 op/s 2024-05-17T22:45:16.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:15 smithi151 ceph-mon[36101]: pgmap v619: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 546 op/s 2024-05-17T22:45:16.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:15 smithi096 ceph-mon[36065]: pgmap v619: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 546 op/s 2024-05-17T22:45:16.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:15 smithi012 ceph-mon[31632]: pgmap v619: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 546 op/s 2024-05-17T22:45:16.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 355.0 s, 772.6 tps, lat 131.085 ms stddev 165.661 2024-05-17T22:45:17.532 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-17T22:45:18.041 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:45:18.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:17 smithi151 ceph-mon[36101]: pgmap v620: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.8 MiB/s wr, 485 op/s 2024-05-17T22:45:18.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:17 smithi096 ceph-mon[36065]: pgmap v620: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.8 MiB/s wr, 485 op/s 2024-05-17T22:45:18.328 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:17 smithi012 ceph-mon[31632]: pgmap v620: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.8 MiB/s wr, 485 op/s 2024-05-17T22:45:20.238 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:19 smithi012 ceph-mon[31632]: pgmap v621: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 12 MiB/s wr, 620 op/s 2024-05-17T22:45:20.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:19 smithi096 ceph-mon[36065]: pgmap v621: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 12 MiB/s wr, 620 op/s 2024-05-17T22:45:20.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:19 smithi151 ceph-mon[36101]: pgmap v621: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 12 MiB/s wr, 620 op/s 2024-05-17T22:45:21.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.366+0000 7f09c9f6a700 1 -- 172.21.15.12:0/2525476039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09c410b180 msgr2=0x7f09c411d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:21.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.366+0000 7f09c9f6a700 1 --2- 172.21.15.12:0/2525476039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09c410b180 0x7f09c411d010 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f09b8009990 tx=0x7f09b8009ca0 comp rx=0 tx=0).stop 2024-05-17T22:45:21.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.366+0000 7f09c9f6a700 1 -- 172.21.15.12:0/2525476039 shutdown_connections 2024-05-17T22:45:21.370 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 360.0 s, 923.6 tps, lat 104.547 ms stddev 121.334 2024-05-17T22:45:21.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.366+0000 7f09c9f6a700 1 --2- 172.21.15.12:0/2525476039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09c411d550 0x7f09c411f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:21.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.366+0000 7f09c9f6a700 1 --2- 172.21.15.12:0/2525476039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09c410a830 0x7f09c410ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:21.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.366+0000 7f09c9f6a700 1 --2- 172.21.15.12:0/2525476039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09c410b180 0x7f09c411d010 secure :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f09b8009990 tx=0x7f09b8009ca0 comp rx=0 tx=0).stop 2024-05-17T22:45:21.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.366+0000 7f09c9f6a700 1 -- 172.21.15.12:0/2525476039 >> 172.21.15.12:0/2525476039 conn(0x7f09c406efc0 msgr2=0x7f09c406f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:45:21.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.367+0000 7f09c9f6a700 1 -- 172.21.15.12:0/2525476039 shutdown_connections 2024-05-17T22:45:21.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.369+0000 7f09c9f6a700 1 -- 172.21.15.12:0/2525476039 wait complete. 2024-05-17T22:45:21.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.371+0000 7f09c9f6a700 1 Processor -- start 2024-05-17T22:45:21.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.371+0000 7f09c9f6a700 1 -- start start 2024-05-17T22:45:21.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.372+0000 7f09c9f6a700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09c410a830 0x7f09c41a9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:21.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.372+0000 7f09c9f6a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09c411d550 0x7f09c41a9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:21.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.372+0000 7f09c9f6a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09c41adfc0 0x7f09c41ae3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:21.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.372+0000 7f09c9f6a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f09c4122bf0 con 0x7f09c411d550 2024-05-17T22:45:21.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.372+0000 7f09c9f6a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f09c4122a70 con 0x7f09c41adfc0 2024-05-17T22:45:21.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.372+0000 7f09c9f6a700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f09c4122d70 con 0x7f09c410a830 2024-05-17T22:45:21.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.372+0000 7f09c2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09c411d550 0x7f09c41a9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:21.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.372+0000 7f09c2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09c411d550 0x7f09c41a9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46950/0 (socket says 172.21.15.12:46950) 2024-05-17T22:45:21.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.372+0000 7f09c2ffd700 1 -- 172.21.15.12:0/3212946537 learned_addr learned my addr 172.21.15.12:0/3212946537 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:45:21.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.373+0000 7f09c2ffd700 1 -- 172.21.15.12:0/3212946537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09c410a830 msgr2=0x7f09c41a9850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-17T22:45:21.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.373+0000 7f09c2ffd700 1 --2- 172.21.15.12:0/3212946537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09c410a830 0x7f09c41a9850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:21.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.373+0000 7f09c2ffd700 1 -- 172.21.15.12:0/3212946537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09c41adfc0 msgr2=0x7f09c41ae3d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-17T22:45:21.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.373+0000 7f09c2ffd700 1 --2- 172.21.15.12:0/3212946537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09c41adfc0 0x7f09c41ae3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:21.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.373+0000 7f09c2ffd700 1 -- 172.21.15.12:0/3212946537 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c41aeb10 con 0x7f09c411d550 2024-05-17T22:45:21.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.373+0000 7f09c2ffd700 1 --2- 172.21.15.12:0/3212946537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09c411d550 0x7f09c41a9df0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f09b8015000 tx=0x7f09b8005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:45:21.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.375+0000 7f09c0ff9700 1 -- 172.21.15.12:0/3212946537 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b8026070 con 0x7f09c411d550 2024-05-17T22:45:21.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.375+0000 7f09c0ff9700 1 -- 172.21.15.12:0/3212946537 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09b8021410 con 0x7f09c411d550 2024-05-17T22:45:21.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.375+0000 7f09c0ff9700 1 -- 172.21.15.12:0/3212946537 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b8017650 con 0x7f09c411d550 2024-05-17T22:45:21.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.378+0000 7f09c9f6a700 1 -- 172.21.15.12:0/3212946537 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09c41aeda0 con 0x7f09c411d550 2024-05-17T22:45:21.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.378+0000 7f09c9f6a700 1 -- 172.21.15.12:0/3212946537 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09c4112a20 con 0x7f09c411d550 2024-05-17T22:45:21.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.383+0000 7f09d8945b80 1 Processor -- start 2024-05-17T22:45:21.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.384+0000 7f09d8945b80 1 -- start start 2024-05-17T22:45:21.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.384+0000 7f09d8945b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560fb37909f0 0x560fb3914560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:21.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.384+0000 7f09d8945b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fb3914aa0 0x560fb390ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:21.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.384+0000 7f09d8945b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fb390f170 0x560fb390f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:21.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.384+0000 7f09d8945b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x560fb383d290 con 0x560fb37909f0 2024-05-17T22:45:21.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.384+0000 7f09d8945b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560fb382cfd0 con 0x560fb390f170 2024-05-17T22:45:21.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.384+0000 7f09c3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560fb37909f0 0x560fb3914560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:21.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.384+0000 7f09c3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560fb37909f0 0x560fb3914560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46952/0 (socket says 172.21.15.12:46952) 2024-05-17T22:45:21.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.384+0000 7f09c3fff700 1 -- 172.21.15.12:0/515766296 learned_addr learned my addr 172.21.15.12:0/515766296 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:45:21.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.384+0000 7f09d8945b80 1 -- 172.21.15.12:0/515766296 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560fb3830e70 con 0x560fb3914aa0 2024-05-17T22:45:21.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.385+0000 7f09c37fe700 1 --2- 172.21.15.12:0/515766296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fb3914aa0 0x560fb390ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:21.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.385+0000 7f09c3fff700 1 -- 172.21.15.12:0/515766296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fb3914aa0 msgr2=0x560fb390ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:21.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.386+0000 7f09c3fff700 1 --2- 172.21.15.12:0/515766296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fb3914aa0 0x560fb390ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:21.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.386+0000 7f09c3fff700 1 -- 172.21.15.12:0/515766296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fb390f170 msgr2=0x560fb390f580 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T22:45:21.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.386+0000 7f09c3fff700 1 --2- 172.21.15.12:0/515766296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fb390f170 0x560fb390f580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:21.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.386+0000 7f09c3fff700 1 -- 172.21.15.12:0/515766296 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fb37d6590 con 0x560fb37909f0 2024-05-17T22:45:21.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.386+0000 7f09c3fff700 1 --2- 172.21.15.12:0/515766296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560fb37909f0 0x560fb3914560 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f09b402c8e0 tx=0x7f09b402cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca3e45f4541ada server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:45:21.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.390+0000 7f09c0ff9700 1 -- 172.21.15.12:0/3212946537 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f09b8009d10 con 0x7f09c411d550 2024-05-17T22:45:21.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.391+0000 7f09a9ffb700 1 -- 172.21.15.12:0/515766296 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b4004220 con 0x560fb37909f0 2024-05-17T22:45:21.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.391+0000 7f09a9ffb700 1 -- 172.21.15.12:0/515766296 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09b402ed10 con 0x560fb37909f0 2024-05-17T22:45:21.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.391+0000 7f09a9ffb700 1 -- 172.21.15.12:0/515766296 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b4005020 con 0x560fb37909f0 2024-05-17T22:45:21.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.392+0000 7f09c0ff9700 1 --2- 172.21.15.12:0/3212946537 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f09ac0b2a50 0x7f09ac0b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:21.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.392+0000 7f09c0ff9700 1 -- 172.21.15.12:0/3212946537 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f09b80e2d00 con 0x7f09c411d550 2024-05-17T22:45:21.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.392+0000 7f09d8945b80 1 -- 172.21.15.12:0/515766296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560fb37909f0 msgr2=0x560fb3914560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:21.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.392+0000 7f09d8945b80 1 --2- 172.21.15.12:0/515766296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560fb37909f0 0x560fb3914560 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f09b402c8e0 tx=0x7f09b402cbf0 comp rx=0 tx=0).stop 2024-05-17T22:45:21.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.396+0000 7f09c37fe700 1 --2- 172.21.15.12:0/3212946537 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f09ac0b2a50 0x7f09ac0b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:21.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.396+0000 7f09d8945b80 1 -- 172.21.15.12:0/515766296 shutdown_connections 2024-05-17T22:45:21.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.396+0000 7f09d8945b80 1 --2- 172.21.15.12:0/515766296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fb390f170 0x560fb390f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:21.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.396+0000 7f09d8945b80 1 --2- 172.21.15.12:0/515766296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560fb37909f0 0x560fb3914560 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:21.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.396+0000 7f09d8945b80 1 --2- 172.21.15.12:0/515766296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fb3914aa0 0x560fb390ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:21.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.396+0000 7f09d8945b80 1 -- 172.21.15.12:0/515766296 >> 172.21.15.12:0/515766296 conn(0x560fb37d4f00 msgr2=0x560fb3909b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:45:21.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.397+0000 7f09c37fe700 1 --2- 172.21.15.12:0/3212946537 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f09ac0b2a50 0x7f09ac0b4f60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x560fb390b320 tx=0x7f09bc00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:45:21.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.400+0000 7f09d8945b80 1 -- 172.21.15.12:0/515766296 shutdown_connections 2024-05-17T22:45:21.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.406+0000 7f09d8945b80 1 -- 172.21.15.12:0/515766296 wait complete. 2024-05-17T22:45:21.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.406+0000 7f09d8945b80 1 -- start start 2024-05-17T22:45:21.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.406+0000 7f09d8945b80 1 Processor -- start 2024-05-17T22:45:21.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.407+0000 7f09d8945b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fb37909f0 0x560fb3918270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:21.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.407+0000 7f09d8945b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560fb390f170 0x560fb39187b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:21.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.407+0000 7f09d8945b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fb3914aa0 0x560fb3924690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:21.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.407+0000 7f09d8945b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x560fb3734150 con 0x560fb390f170 2024-05-17T22:45:21.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.407+0000 7f09d8945b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560fb37cde80 con 0x560fb37909f0 2024-05-17T22:45:21.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.407+0000 7f09d8945b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560fb391b390 con 0x560fb3914aa0 2024-05-17T22:45:21.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.407+0000 7f09d8945b80 10 client.0 _ll_register_callbacks cb 0x560fb37c7f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:45:21.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.408+0000 7f09c3fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fb3914aa0 0x560fb3924690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:21.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.408+0000 7f09c3fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fb3914aa0 0x560fb3924690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:52962/0 (socket says 172.21.15.12:52962) 2024-05-17T22:45:21.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.408+0000 7f09c3fff700 1 -- 172.21.15.12:0/1096963761 learned_addr learned my addr 172.21.15.12:0/1096963761 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:45:21.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.408+0000 7f09c37fe700 1 --2- 172.21.15.12:0/1096963761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fb37909f0 0x560fb3918270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:21.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.412+0000 7f09c37fe700 1 -- 172.21.15.12:0/1096963761 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fb3914aa0 msgr2=0x560fb3924690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:21.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.412+0000 7f09c37fe700 1 --2- 172.21.15.12:0/1096963761 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fb3914aa0 0x560fb3924690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:21.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.412+0000 7f09c37fe700 1 -- 172.21.15.12:0/1096963761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560fb390f170 msgr2=0x560fb39187b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:21.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.412+0000 7f09c37fe700 1 --2- 172.21.15.12:0/1096963761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560fb390f170 0x560fb39187b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:21.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.412+0000 7f09c37fe700 1 -- 172.21.15.12:0/1096963761 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fb377d480 con 0x560fb37909f0 2024-05-17T22:45:21.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.413+0000 7f09c37fe700 1 --2- 172.21.15.12:0/1096963761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fb37909f0 0x560fb3918270 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x560fb3914030 tx=0x7f09bc004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1002928171b6f002 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:45:21.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.415+0000 7f09a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:45:21.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.415+0000 7f09a2ffd700 1 -- 172.21.15.12:0/1096963761 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09bc026060 con 0x560fb37909f0 2024-05-17T22:45:21.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.415+0000 7f09d8945b80 1 -- 172.21.15.12:0/1096963761 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09b402ed10 con 0x560fb37909f0 2024-05-17T22:45:21.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.416+0000 7f09a2ffd700 1 -- 172.21.15.12:0/1096963761 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f09bc015670 con 0x560fb37909f0 2024-05-17T22:45:21.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.416+0000 7f09a2ffd700 1 -- 172.21.15.12:0/1096963761 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f09bc02d050 con 0x560fb37909f0 2024-05-17T22:45:21.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.416+0000 7f09d8945b80 10 client.25511 fetch_fsmap learned FSMap version 37 2024-05-17T22:45:21.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.416+0000 7f09d8945b80 1 -- 172.21.15.12:0/1096963761 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x560fb3925f50 con 0x560fb37909f0 2024-05-17T22:45:21.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.417+0000 7f09a2ffd700 1 -- 172.21.15.12:0/1096963761 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09b402ed10 con 0x560fb37909f0 2024-05-17T22:45:21.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.417+0000 7f09a2ffd700 1 -- 172.21.15.12:0/1096963761 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f09bc0219f0 con 0x560fb37909f0 2024-05-17T22:45:21.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.417+0000 7f09d8945b80 10 client.25511 fetch_fsmap finished waiting for FSMap version 37 2024-05-17T22:45:21.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.417+0000 7f09d8945b80 10 client.25511 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:45:21.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.417+0000 7f09d8945b80 20 client.25511 populate_metadata read hostname 'smithi012' 2024-05-17T22:45:21.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.417+0000 7f09d8945b80 1 --2- 172.21.15.12:0/1096963761 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x560fb39289c0 0x560fb392ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:21.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.417+0000 7f09d8945b80 4 client.25511 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:45:21.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.417+0000 7f09d8945b80 1 -- 172.21.15.12:0/1096963761 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fb383f1e0 con 0x560fb39289c0 2024-05-17T22:45:21.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.422+0000 7f09c2ffd700 1 --2- 172.21.15.12:0/1096963761 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x560fb39289c0 0x560fb392ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:21.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.428+0000 7f09a2ffd700 10 client.25511 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:45:21.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.428+0000 7f09c2ffd700 1 --2- 172.21.15.12:0/1096963761 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x560fb39289c0 0x560fb392ae70 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=830289402afc0d5 server_cookie=fc9a31f4ec26a510 in_seq=0 out_seq=0 2024-05-17T22:45:21.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.431+0000 7f09a2ffd700 1 -- 172.21.15.12:0/1096963761 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560fb383f1e0 con 0x560fb39289c0 2024-05-17T22:45:21.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.431+0000 7f09a2ffd700 10 client.25511 handle_command_reply: tid=0 2024-05-17T22:45:21.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.431+0000 7f09d8945b80 1 client.25511 shutdown 2024-05-17T22:45:21.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.435+0000 7f09d8945b80 1 -- 172.21.15.12:0/1096963761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fb37909f0 msgr2=0x560fb3918270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:21.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.435+0000 7f09d8945b80 1 --2- 172.21.15.12:0/1096963761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fb37909f0 0x560fb3918270 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x560fb3914030 tx=0x7f09bc004ad0 comp rx=0 tx=0).stop 2024-05-17T22:45:21.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.435+0000 7f09d8945b80 1 -- 172.21.15.12:0/1096963761 shutdown_connections 2024-05-17T22:45:21.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.435+0000 7f09d8945b80 1 --2- 172.21.15.12:0/1096963761 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fb3914aa0 0x560fb3924690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:21.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.435+0000 7f09d8945b80 1 --2- 172.21.15.12:0/1096963761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fb37909f0 0x560fb3918270 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:21.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.435+0000 7f09d8945b80 1 --2- 172.21.15.12:0/1096963761 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x560fb39289c0 0x560fb392ae70 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:21.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.435+0000 7f09d8945b80 1 --2- 172.21.15.12:0/1096963761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560fb390f170 0x560fb39187b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:21.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.435+0000 7f09a2ffd700 0 client.25511 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:45:21.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.435+0000 7f09d8945b80 1 -- 172.21.15.12:0/1096963761 >> 172.21.15.12:0/1096963761 conn(0x560fb37d4f00 msgr2=0x560fb3903ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:45:21.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.436+0000 7f09d8945b80 1 -- 172.21.15.12:0/1096963761 shutdown_connections 2024-05-17T22:45:21.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.437+0000 7f09d8945b80 1 -- 172.21.15.12:0/1096963761 wait complete. 2024-05-17T22:45:21.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.438+0000 7f09d8945b80 20 client.25511 trim_cache size 0 max 16384 2024-05-17T22:45:21.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.481+0000 7f09d8945b80 1 Processor -- start 2024-05-17T22:45:21.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.481+0000 7f09d8945b80 1 -- start start 2024-05-17T22:45:21.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.481+0000 7f09d8945b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fb37909f0 0x560fb3918750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:21.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.481+0000 7f09d8945b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560fb390f170 0x560fb3903eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:21.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.482+0000 7f09d8945b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fb3914aa0 0x560fb391aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:21.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.482+0000 7f09c37fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fb37909f0 0x560fb3918750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:21.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.482+0000 7f09c37fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fb37909f0 0x560fb3918750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:52978/0 (socket says 172.21.15.12:52978) 2024-05-17T22:45:21.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.482+0000 7f09c37fe700 1 -- 172.21.15.12:0/2767363006 learned_addr learned my addr 172.21.15.12:0/2767363006 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:45:21.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.482+0000 7f09c3fff700 1 --2- 172.21.15.12:0/2767363006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fb3914aa0 0x560fb391aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:21.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.482+0000 7f09d8945b80 1 -- 172.21.15.12:0/2767363006 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f09c41b9270 con 0x560fb390f170 2024-05-17T22:45:21.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.482+0000 7f09d8945b80 1 -- 172.21.15.12:0/2767363006 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f09c41b6b00 con 0x560fb3914aa0 2024-05-17T22:45:21.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.482+0000 7f09d8945b80 1 -- 172.21.15.12:0/2767363006 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f09c41196e0 con 0x560fb37909f0 2024-05-17T22:45:21.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.483+0000 7f09c2ffd700 1 --2- 172.21.15.12:0/2767363006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560fb390f170 0x560fb3903eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:21.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.483+0000 7f09c37fe700 1 -- 172.21.15.12:0/2767363006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fb3914aa0 msgr2=0x560fb391aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:21.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.483+0000 7f09c37fe700 1 --2- 172.21.15.12:0/2767363006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fb3914aa0 0x560fb391aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:21.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.483+0000 7f09c37fe700 1 -- 172.21.15.12:0/2767363006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560fb390f170 msgr2=0x560fb3903eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:21.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.483+0000 7f09c37fe700 1 --2- 172.21.15.12:0/2767363006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560fb390f170 0x560fb3903eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:21.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.483+0000 7f09c37fe700 1 -- 172.21.15.12:0/2767363006 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fb39140d0 con 0x560fb37909f0 2024-05-17T22:45:21.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.483+0000 7f09c37fe700 1 --2- 172.21.15.12:0/2767363006 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fb37909f0 0x560fb3918750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f09bc004020 tx=0x7f09bc014560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d529515bc2ac0af server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:45:21.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.484+0000 7f09a97fa700 1 -- 172.21.15.12:0/2767363006 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09bc005e60 con 0x560fb37909f0 2024-05-17T22:45:21.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.484+0000 7f09a97fa700 1 -- 172.21.15.12:0/2767363006 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09bc0145d0 con 0x560fb37909f0 2024-05-17T22:45:21.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.484+0000 7f09a97fa700 1 -- 172.21.15.12:0/2767363006 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09bc021ae0 con 0x560fb37909f0 2024-05-17T22:45:21.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.485+0000 7f09d8945b80 1 -- 172.21.15.12:0/2767363006 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fb37909f0 msgr2=0x560fb3918750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:21.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.485+0000 7f09d8945b80 1 --2- 172.21.15.12:0/2767363006 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fb37909f0 0x560fb3918750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f09bc004020 tx=0x7f09bc014560 comp rx=0 tx=0).stop 2024-05-17T22:45:21.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.485+0000 7f09d8945b80 1 -- 172.21.15.12:0/2767363006 shutdown_connections 2024-05-17T22:45:21.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.485+0000 7f09d8945b80 1 --2- 172.21.15.12:0/2767363006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fb3914aa0 0x560fb391aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:21.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.485+0000 7f09d8945b80 1 --2- 172.21.15.12:0/2767363006 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fb37909f0 0x560fb3918750 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:21.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.485+0000 7f09d8945b80 1 --2- 172.21.15.12:0/2767363006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560fb390f170 0x560fb3903eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:21.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.485+0000 7f09d8945b80 1 -- 172.21.15.12:0/2767363006 >> 172.21.15.12:0/2767363006 conn(0x560fb37d4f00 msgr2=0x560fb3915200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:45:21.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.485+0000 7f09d8945b80 1 -- 172.21.15.12:0/2767363006 shutdown_connections 2024-05-17T22:45:21.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.485+0000 7f09d8945b80 1 -- 172.21.15.12:0/2767363006 wait complete. 2024-05-17T22:45:21.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.486+0000 7f09d8945b80 1 -- start start 2024-05-17T22:45:21.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.486+0000 7f09d8945b80 1 Processor -- start 2024-05-17T22:45:21.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.487+0000 7f09d8945b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560fb37909f0 0x560fb39203a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:21.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.487+0000 7f09d8945b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fb390f170 0x560fb39208e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:21.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.487+0000 7f09c37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560fb37909f0 0x560fb39203a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:21.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.487+0000 7f09c37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560fb37909f0 0x560fb39203a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46972/0 (socket says 172.21.15.12:46972) 2024-05-17T22:45:21.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.487+0000 7f09c37fe700 1 -- 172.21.15.12:0/955986361 learned_addr learned my addr 172.21.15.12:0/955986361 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:45:21.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.488+0000 7f09d8945b80 1 --2- 172.21.15.12:0/955986361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fb3914aa0 0x560fb391b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:21.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.488+0000 7f09c2ffd700 1 --2- 172.21.15.12:0/955986361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fb390f170 0x560fb39208e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:21.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.488+0000 7f09d8945b80 1 -- 172.21.15.12:0/955986361 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f09c4119d50 con 0x560fb37909f0 2024-05-17T22:45:21.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.488+0000 7f09d8945b80 1 -- 172.21.15.12:0/955986361 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f09c41b6430 con 0x560fb3914aa0 2024-05-17T22:45:21.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.488+0000 7f09d8945b80 1 -- 172.21.15.12:0/955986361 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f09c411a420 con 0x560fb390f170 2024-05-17T22:45:21.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.488+0000 7f09c3fff700 1 --2- 172.21.15.12:0/955986361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fb3914aa0 0x560fb391b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:21.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.488+0000 7f09d8945b80 10 client.0 _ll_register_callbacks cb 0x560fb37c7f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:45:21.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.488+0000 7f09c2ffd700 1 -- 172.21.15.12:0/955986361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fb3914aa0 msgr2=0x560fb391b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:21.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.488+0000 7f09c2ffd700 1 --2- 172.21.15.12:0/955986361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fb3914aa0 0x560fb391b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:21.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.488+0000 7f09c2ffd700 1 -- 172.21.15.12:0/955986361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560fb37909f0 msgr2=0x560fb39203a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:21.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.488+0000 7f09c2ffd700 1 --2- 172.21.15.12:0/955986361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560fb37909f0 0x560fb39203a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:21.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.488+0000 7f09c2ffd700 1 -- 172.21.15.12:0/955986361 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fb391bf20 con 0x560fb390f170 2024-05-17T22:45:21.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.488+0000 7f09c37fe700 1 --2- 172.21.15.12:0/955986361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560fb37909f0 0x560fb39203a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:45:21.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.489+0000 7f09c2ffd700 1 --2- 172.21.15.12:0/955986361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fb390f170 0x560fb39208e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x560fb3919a30 tx=0x7f09b8009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29934c12e4826470 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:45:21.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.489+0000 7f09a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:45:21.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.489+0000 7f09a2ffd700 1 -- 172.21.15.12:0/955986361 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b80aa750 con 0x560fb390f170 2024-05-17T22:45:21.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.489+0000 7f09a2ffd700 1 -- 172.21.15.12:0/955986361 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f09b80218c0 con 0x560fb390f170 2024-05-17T22:45:21.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.489+0000 7f09d8945b80 1 -- 172.21.15.12:0/955986361 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09bc0145d0 con 0x560fb390f170 2024-05-17T22:45:21.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.489+0000 7f09a2ffd700 1 -- 172.21.15.12:0/955986361 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f09b80aa050 con 0x560fb390f170 2024-05-17T22:45:21.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.490+0000 7f09d8945b80 10 client.25462 fetch_fsmap learned FSMap version 37 2024-05-17T22:45:21.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.490+0000 7f09d8945b80 1 -- 172.21.15.12:0/955986361 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x560fb3992d10 con 0x560fb390f170 2024-05-17T22:45:21.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.490+0000 7f09a2ffd700 1 -- 172.21.15.12:0/955986361 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09b80ae400 con 0x560fb390f170 2024-05-17T22:45:21.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.490+0000 7f09a2ffd700 1 -- 172.21.15.12:0/955986361 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f09b80b2990 con 0x560fb390f170 2024-05-17T22:45:21.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.490+0000 7f09d8945b80 10 client.25462 fetch_fsmap finished waiting for FSMap version 37 2024-05-17T22:45:21.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.490+0000 7f09d8945b80 10 client.25462 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:45:21.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.490+0000 7f09d8945b80 20 client.25462 populate_metadata read hostname 'smithi012' 2024-05-17T22:45:21.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.490+0000 7f09d8945b80 1 --2- 172.21.15.12:0/955986361 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x560fb3993450 0x560fb3995870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:21.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.490+0000 7f09d8945b80 4 client.25462 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:45:21.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.490+0000 7f09d8945b80 1 -- 172.21.15.12:0/955986361 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f09bc0145d0 con 0x560fb3993450 2024-05-17T22:45:21.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.490+0000 7f09c37fe700 1 --2- 172.21.15.12:0/955986361 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x560fb3993450 0x560fb3995870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:21.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.491+0000 7f09a2ffd700 10 client.25462 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:45:21.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.491+0000 7f09c37fe700 1 --2- 172.21.15.12:0/955986361 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x560fb3993450 0x560fb3995870 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dec593bfffbb6a7f server_cookie=61384a5fe8560a12 in_seq=0 out_seq=0 2024-05-17T22:45:21.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.491+0000 7f09a2ffd700 1 -- 172.21.15.12:0/955986361 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f09bc0145d0 con 0x560fb3993450 2024-05-17T22:45:21.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.491+0000 7f09a2ffd700 10 client.25462 handle_command_reply: tid=0 2024-05-17T22:45:21.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.491+0000 7f09d8945b80 1 client.25462 shutdown 2024-05-17T22:45:21.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.492+0000 7f09d8945b80 1 -- 172.21.15.12:0/955986361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fb390f170 msgr2=0x560fb39208e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:21.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.492+0000 7f09d8945b80 1 --2- 172.21.15.12:0/955986361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fb390f170 0x560fb39208e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x560fb3919a30 tx=0x7f09b8009ca0 comp rx=0 tx=0).stop 2024-05-17T22:45:21.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.492+0000 7f09d8945b80 1 -- 172.21.15.12:0/955986361 shutdown_connections 2024-05-17T22:45:21.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.492+0000 7f09d8945b80 1 --2- 172.21.15.12:0/955986361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fb3914aa0 0x560fb391b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:21.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.492+0000 7f09d8945b80 1 --2- 172.21.15.12:0/955986361 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x560fb3993450 0x560fb3995870 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:21.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.492+0000 7f09a2ffd700 0 client.25462 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:45:21.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.492+0000 7f09d8945b80 1 --2- 172.21.15.12:0/955986361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560fb37909f0 0x560fb39203a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:21.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.492+0000 7f09d8945b80 1 --2- 172.21.15.12:0/955986361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fb390f170 0x560fb39208e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:21.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.492+0000 7f09d8945b80 1 -- 172.21.15.12:0/955986361 >> 172.21.15.12:0/955986361 conn(0x560fb37d4f00 msgr2=0x560fb39125a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:45:21.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.492+0000 7f09d8945b80 1 -- 172.21.15.12:0/955986361 shutdown_connections 2024-05-17T22:45:21.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.492+0000 7f09d8945b80 1 -- 172.21.15.12:0/955986361 wait complete. 2024-05-17T22:45:21.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.493+0000 7f09d8945b80 20 client.25462 trim_cache size 0 max 16384 2024-05-17T22:45:21.498 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:45:21.498 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-17T22:45:21.498 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-17T22:45:21.498 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:45:21.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.493+0000 7f09c9f6a700 1 -- 172.21.15.12:0/3212946537 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f09ac0b2a50 msgr2=0x7f09ac0b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:21.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.493+0000 7f09c9f6a700 1 --2- 172.21.15.12:0/3212946537 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f09ac0b2a50 0x7f09ac0b4f60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x560fb390b320 tx=0x7f09bc00b040 comp rx=0 tx=0).stop 2024-05-17T22:45:21.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.493+0000 7f09c9f6a700 1 -- 172.21.15.12:0/3212946537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09c411d550 msgr2=0x7f09c41a9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:21.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.493+0000 7f09c9f6a700 1 --2- 172.21.15.12:0/3212946537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09c411d550 0x7f09c41a9df0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f09b8015000 tx=0x7f09b8005e80 comp rx=0 tx=0).stop 2024-05-17T22:45:21.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.493+0000 7f09c9f6a700 1 -- 172.21.15.12:0/3212946537 shutdown_connections 2024-05-17T22:45:21.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.493+0000 7f09c9f6a700 1 --2- 172.21.15.12:0/3212946537 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f09ac0b2a50 0x7f09ac0b4f60 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:21.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.493+0000 7f09c9f6a700 1 --2- 172.21.15.12:0/3212946537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09c41adfc0 0x7f09c41ae3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:21.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.493+0000 7f09c9f6a700 1 --2- 172.21.15.12:0/3212946537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09c410a830 0x7f09c41a9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:21.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.493+0000 7f09c9f6a700 1 --2- 172.21.15.12:0/3212946537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09c411d550 0x7f09c41a9df0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:21.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.493+0000 7f09c9f6a700 1 -- 172.21.15.12:0/3212946537 >> 172.21.15.12:0/3212946537 conn(0x7f09c406efc0 msgr2=0x7f09c411b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:45:21.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.494+0000 7f09c9f6a700 1 -- 172.21.15.12:0/3212946537 shutdown_connections 2024-05-17T22:45:21.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:21.494+0000 7f09c9f6a700 1 -- 172.21.15.12:0/3212946537 wait complete. 2024-05-17T22:45:22.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:22 smithi096 ceph-mon[36065]: pgmap v622: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.7 MiB/s wr, 476 op/s 2024-05-17T22:45:22.318 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:22 smithi012 ceph-mon[31632]: pgmap v622: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.7 MiB/s wr, 476 op/s 2024-05-17T22:45:22.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:22 smithi151 ceph-mon[36101]: pgmap v622: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.7 MiB/s wr, 476 op/s 2024-05-17T22:45:22.663 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-17T22:45:22.664 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T22:45:23.080 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:45:24.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:24 smithi096 ceph-mon[36065]: pgmap v623: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.8 MiB/s wr, 550 op/s 2024-05-17T22:45:24.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:24 smithi012 ceph-mon[31632]: pgmap v623: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.8 MiB/s wr, 550 op/s 2024-05-17T22:45:24.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:24 smithi151 ceph-mon[36101]: pgmap v623: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.8 MiB/s wr, 550 op/s 2024-05-17T22:45:25.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:25.996+0000 7fb817cf9700 1 -- 172.21.15.12:0/2849797863 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb81010b180 msgr2=0x7fb81011d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:25.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:25.996+0000 7fb817cf9700 1 --2- 172.21.15.12:0/2849797863 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb81010b180 0x7fb81011d010 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb800009990 tx=0x7fb800009ca0 comp rx=0 tx=0).stop 2024-05-17T22:45:26.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:25.999+0000 7fb817cf9700 1 -- 172.21.15.12:0/2849797863 shutdown_connections 2024-05-17T22:45:26.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:25.999+0000 7fb817cf9700 1 --2- 172.21.15.12:0/2849797863 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb81011d550 0x7fb81011f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:26.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:25.999+0000 7fb817cf9700 1 --2- 172.21.15.12:0/2849797863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb81010a830 0x7fb81010ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:26.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:25.999+0000 7fb817cf9700 1 --2- 172.21.15.12:0/2849797863 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb81010b180 0x7fb81011d010 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:26.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:25.999+0000 7fb817cf9700 1 -- 172.21.15.12:0/2849797863 >> 172.21.15.12:0/2849797863 conn(0x7fb81006efc0 msgr2=0x7fb81006f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:45:26.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:25.999+0000 7fb817cf9700 1 -- 172.21.15.12:0/2849797863 shutdown_connections 2024-05-17T22:45:26.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:25.999+0000 7fb817cf9700 1 -- 172.21.15.12:0/2849797863 wait complete. 2024-05-17T22:45:26.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.000+0000 7fb817cf9700 1 Processor -- start 2024-05-17T22:45:26.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.000+0000 7fb817cf9700 1 -- start start 2024-05-17T22:45:26.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.000+0000 7fb817cf9700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb81010a830 0x7fb81011b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:26.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.000+0000 7fb817cf9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb81010b180 0x7fb810114b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:26.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.000+0000 7fb817cf9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb81011d550 0x7fb810115150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:26.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.000+0000 7fb817cf9700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb810122d40 con 0x7fb81011d550 2024-05-17T22:45:26.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.000+0000 7fb817cf9700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb810122bc0 con 0x7fb81010b180 2024-05-17T22:45:26.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.000+0000 7fb817cf9700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb810122ec0 con 0x7fb81010a830 2024-05-17T22:45:26.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.001+0000 7fb815a95700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb81010a830 0x7fb81011b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:26.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.001+0000 7fb815a95700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb81010a830 0x7fb81011b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:53014/0 (socket says 172.21.15.12:53014) 2024-05-17T22:45:26.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.001+0000 7fb815a95700 1 -- 172.21.15.12:0/2568425231 learned_addr learned my addr 172.21.15.12:0/2568425231 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:45:26.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.001+0000 7fb815294700 1 --2- 172.21.15.12:0/2568425231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb81010b180 0x7fb810114b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:26.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.001+0000 7fb815294700 1 -- 172.21.15.12:0/2568425231 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb81010a830 msgr2=0x7fb81011b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:26.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.001+0000 7fb815294700 1 --2- 172.21.15.12:0/2568425231 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb81010a830 0x7fb81011b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:26.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.001+0000 7fb815294700 1 -- 172.21.15.12:0/2568425231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb81011d550 msgr2=0x7fb810115150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:26.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.001+0000 7fb815294700 1 --2- 172.21.15.12:0/2568425231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb81011d550 0x7fb810115150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:26.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.001+0000 7fb815294700 1 -- 172.21.15.12:0/2568425231 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb810119500 con 0x7fb81010b180 2024-05-17T22:45:26.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.002+0000 7fb815294700 1 --2- 172.21.15.12:0/2568425231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb81010b180 0x7fb810114b00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb800000c00 tx=0x7fb8000057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:45:26.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.003+0000 7fb806ffd700 1 -- 172.21.15.12:0/2568425231 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb800026070 con 0x7fb81010b180 2024-05-17T22:45:26.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.003+0000 7fb806ffd700 1 -- 172.21.15.12:0/2568425231 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb800021410 con 0x7fb81010b180 2024-05-17T22:45:26.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.003+0000 7fb806ffd700 1 -- 172.21.15.12:0/2568425231 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb800017650 con 0x7fb81010b180 2024-05-17T22:45:26.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.003+0000 7fb817cf9700 1 -- 172.21.15.12:0/2568425231 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb810119790 con 0x7fb81010b180 2024-05-17T22:45:26.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.003+0000 7fb817cf9700 1 -- 172.21.15.12:0/2568425231 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb810119c20 con 0x7fb81010b180 2024-05-17T22:45:26.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.010+0000 7fb806ffd700 1 -- 172.21.15.12:0/2568425231 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb800009d10 con 0x7fb81010b180 2024-05-17T22:45:26.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.010+0000 7fb817cf9700 1 -- 172.21.15.12:0/2568425231 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7f4005360 con 0x7fb81010b180 2024-05-17T22:45:26.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.012+0000 7fb806ffd700 1 --2- 172.21.15.12:0/2568425231 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb7fc0b2a90 0x7fb7fc0b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:26.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.012+0000 7fb806ffd700 1 -- 172.21.15.12:0/2568425231 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fb8000e3ef0 con 0x7fb81010b180 2024-05-17T22:45:26.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.014+0000 7fb815a95700 1 --2- 172.21.15.12:0/2568425231 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb7fc0b2a90 0x7fb7fc0b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:26.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.014+0000 7fb815a95700 1 --2- 172.21.15.12:0/2568425231 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb7fc0b2a90 0x7fb7fc0b4fa0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb810116350 tx=0x7fb80c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:45:26.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.019+0000 7fb806ffd700 1 -- 172.21.15.12:0/2568425231 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb800013070 con 0x7fb81010b180 2024-05-17T22:45:26.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:26 smithi096 ceph-mon[36065]: pgmap v624: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 595 op/s 2024-05-17T22:45:26.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:26 smithi012 ceph-mon[31632]: pgmap v624: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 595 op/s 2024-05-17T22:45:26.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 365.0 s, 511.2 tps, lat 196.816 ms stddev 232.144 2024-05-17T22:45:26.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:26 smithi151 ceph-mon[36101]: pgmap v624: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 595 op/s 2024-05-17T22:45:26.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.529+0000 7fb817cf9700 1 -- 172.21.15.12:0/2568425231 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7f4005a70 con 0x7fb81010b180 2024-05-17T22:45:26.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.534+0000 7fb806ffd700 1 -- 172.21.15.12:0/2568425231 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11873 (secure 0 0 0) 0x7fb8000095a0 con 0x7fb81010b180 2024-05-17T22:45:26.539 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:45:26.539 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1407219319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1407219319},{"type":"v1","addr":"172.21.15.12:6835","nonce":1407219319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/630977886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":630977886},{"type":"v1","addr":"172.21.15.12:6837","nonce":630977886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3731914742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3731914742},{"type":"v1","addr":"172.21.15.12:6839","nonce":3731914742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/3642142501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3642142501},{"type":"v1","addr":"172.21.15.12:6841","nonce":3642142501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3554053059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3554053059},{"type":"v1","addr":"172.21.15.151:6833","nonce":3554053059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/4228325569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":4228325569},{"type":"v1","addr":"172.21.15.96:6833","nonce":4228325569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T22:34:42.642111+0000","modified":"2024-05-17T22:44:59.930707+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24461,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/542323754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":542323754},{"type":"v1","addr":"172.21.15.151:6835","nonce":542323754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1758442232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1758442232},{"type":"v1","addr":"172.21.15.96:6835","nonce":1758442232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3403324038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3403324038},{"type":"v1","addr":"172.21.15.151:6837","nonce":3403324038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.151:6839/2338353849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2338353849},{"type":"v1","addr":"172.21.15.151:6839","nonce":2338353849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.96:6837/4222574338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":4222574338},{"type":"v1","addr":"172.21.15.96:6837","nonce":4222574338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/2986295519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2986295519},{"type":"v1","addr":"172.21.15.96:6839","nonce":2986295519}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24457,24461,24482]},"id":1}]} 2024-05-17T22:45:26.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.542+0000 7fb804ff9700 1 -- 172.21.15.12:0/2568425231 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb7fc0b2a90 msgr2=0x7fb7fc0b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:26.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.542+0000 7fb804ff9700 1 --2- 172.21.15.12:0/2568425231 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb7fc0b2a90 0x7fb7fc0b4fa0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb810116350 tx=0x7fb80c009260 comp rx=0 tx=0).stop 2024-05-17T22:45:26.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.542+0000 7fb804ff9700 1 -- 172.21.15.12:0/2568425231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb81010b180 msgr2=0x7fb810114b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:26.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.542+0000 7fb804ff9700 1 --2- 172.21.15.12:0/2568425231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb81010b180 0x7fb810114b00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb800000c00 tx=0x7fb8000057c0 comp rx=0 tx=0).stop 2024-05-17T22:45:26.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.543+0000 7fb804ff9700 1 -- 172.21.15.12:0/2568425231 shutdown_connections 2024-05-17T22:45:26.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.543+0000 7fb804ff9700 1 --2- 172.21.15.12:0/2568425231 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb7fc0b2a90 0x7fb7fc0b4fa0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:26.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.543+0000 7fb804ff9700 1 --2- 172.21.15.12:0/2568425231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb81011d550 0x7fb810115150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:26.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.543+0000 7fb804ff9700 1 --2- 172.21.15.12:0/2568425231 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb81010a830 0x7fb81011b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:26.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.543+0000 7fb804ff9700 1 --2- 172.21.15.12:0/2568425231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb81010b180 0x7fb810114b00 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:26.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.543+0000 7fb804ff9700 1 -- 172.21.15.12:0/2568425231 >> 172.21.15.12:0/2568425231 conn(0x7fb81006efc0 msgr2=0x7fb810072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:45:26.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.543+0000 7fb804ff9700 1 -- 172.21.15.12:0/2568425231 shutdown_connections 2024-05-17T22:45:26.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:26.544+0000 7fb804ff9700 1 -- 172.21.15.12:0/2568425231 wait complete. 2024-05-17T22:45:26.550 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 37 2024-05-17T22:45:27.238 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:27 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/2568425231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:45:27.279 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-17T22:45:27.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:27 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/2568425231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:45:27.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:27 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/2568425231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:45:27.710 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:45:28.289 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:28 smithi012 ceph-mon[31632]: pgmap v625: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 522 op/s 2024-05-17T22:45:28.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:28 smithi151 ceph-mon[36101]: pgmap v625: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 522 op/s 2024-05-17T22:45:28.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:28 smithi096 ceph-mon[36065]: pgmap v625: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 522 op/s 2024-05-17T22:45:29.415 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:29 smithi012 ceph-mon[31632]: pgmap v626: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 670 op/s 2024-05-17T22:45:29.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:29 smithi096 ceph-mon[36065]: pgmap v626: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 670 op/s 2024-05-17T22:45:29.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:29 smithi151 ceph-mon[36101]: pgmap v626: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 670 op/s 2024-05-17T22:45:30.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.143+0000 7f454a77e700 1 -- 172.21.15.12:0/982550633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4544102b90 msgr2=0x7f4544102fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:30.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.143+0000 7f454a77e700 1 --2- 172.21.15.12:0/982550633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4544102b90 0x7f4544102fa0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f453c01c0d0 tx=0x7f453c01c3e0 comp rx=0 tx=0).stop 2024-05-17T22:45:30.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.149+0000 7f454a77e700 1 -- 172.21.15.12:0/982550633 shutdown_connections 2024-05-17T22:45:30.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.149+0000 7f454a77e700 1 --2- 172.21.15.12:0/982550633 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4544107fd0 0x7f45441084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:30.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.149+0000 7f454a77e700 1 --2- 172.21.15.12:0/982550633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4544102b90 0x7f4544102fa0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:30.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.149+0000 7f454a77e700 1 --2- 172.21.15.12:0/982550633 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45441034e0 0x7f4544107a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:30.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.149+0000 7f454a77e700 1 -- 172.21.15.12:0/982550633 >> 172.21.15.12:0/982550633 conn(0x7f4544079bd0 msgr2=0x7f4544079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:45:30.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.153+0000 7f454a77e700 1 -- 172.21.15.12:0/982550633 shutdown_connections 2024-05-17T22:45:30.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.155+0000 7f454a77e700 1 -- 172.21.15.12:0/982550633 wait complete. 2024-05-17T22:45:30.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.156+0000 7f454a77e700 1 Processor -- start 2024-05-17T22:45:30.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.156+0000 7f454a77e700 1 -- start start 2024-05-17T22:45:30.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.157+0000 7f454a77e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4544102b90 0x7f454410f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:30.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.157+0000 7f454a77e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45441034e0 0x7f454410feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:30.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.158+0000 7f454a77e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4544107fd0 0x7f45440745b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:30.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.158+0000 7f454a77e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f454410b800 con 0x7f45441034e0 2024-05-17T22:45:30.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.158+0000 7f454a77e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f454410b680 con 0x7f4544102b90 2024-05-17T22:45:30.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.158+0000 7f454a77e700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f454410b980 con 0x7f4544107fd0 2024-05-17T22:45:30.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.158+0000 7f45437fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45441034e0 0x7f454410feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:30.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.159+0000 7f45437fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45441034e0 0x7f454410feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37428/0 (socket says 172.21.15.12:37428) 2024-05-17T22:45:30.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.159+0000 7f4548d1b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4544107fd0 0x7f45440745b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:30.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.159+0000 7f4548d1b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4544107fd0 0x7f45440745b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:54996/0 (socket says 172.21.15.12:54996) 2024-05-17T22:45:30.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.159+0000 7f45437fe700 1 -- 172.21.15.12:0/664088503 learned_addr learned my addr 172.21.15.12:0/664088503 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:45:30.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.159+0000 7f4548d1b700 1 -- 172.21.15.12:0/664088503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4544102b90 msgr2=0x7f454410f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:30.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.160+0000 7f4548d1b700 1 --2- 172.21.15.12:0/664088503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4544102b90 0x7f454410f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:30.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.160+0000 7f4548d1b700 1 -- 172.21.15.12:0/664088503 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45441034e0 msgr2=0x7f454410feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:30.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.160+0000 7f4548d1b700 1 --2- 172.21.15.12:0/664088503 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45441034e0 0x7f454410feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:30.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.160+0000 7f4548d1b700 1 -- 172.21.15.12:0/664088503 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4544074cb0 con 0x7f4544107fd0 2024-05-17T22:45:30.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.160+0000 7f4548d1b700 1 --2- 172.21.15.12:0/664088503 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4544107fd0 0x7f45440745b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f453800b3f0 tx=0x7f453800b700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:45:30.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.162+0000 7f45417fa700 1 -- 172.21.15.12:0/664088503 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4538015020 con 0x7f4544107fd0 2024-05-17T22:45:30.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.162+0000 7f45417fa700 1 -- 172.21.15.12:0/664088503 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4538011ce0 con 0x7f4544107fd0 2024-05-17T22:45:30.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.162+0000 7f45417fa700 1 -- 172.21.15.12:0/664088503 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4538010c40 con 0x7f4544107fd0 2024-05-17T22:45:30.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.162+0000 7f454a77e700 1 -- 172.21.15.12:0/664088503 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4544074fa0 con 0x7f4544107fd0 2024-05-17T22:45:30.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.162+0000 7f454a77e700 1 -- 172.21.15.12:0/664088503 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45441ad9a0 con 0x7f4544107fd0 2024-05-17T22:45:30.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.167+0000 7f4559159b80 1 Processor -- start 2024-05-17T22:45:30.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.170+0000 7f45417fa700 1 -- 172.21.15.12:0/664088503 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4538018020 con 0x7f4544107fd0 2024-05-17T22:45:30.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.170+0000 7f4559159b80 1 -- start start 2024-05-17T22:45:30.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.171+0000 7f4559159b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559069cf59f0 0x559069e78ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:30.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.171+0000 7f4559159b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559069e79400 0x559069e735c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:30.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.171+0000 7f4559159b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559069e73b00 0x559069e73f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:30.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.171+0000 7f4559159b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x559069da20d0 con 0x559069e79400 2024-05-17T22:45:30.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.171+0000 7f4559159b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559069d91c70 con 0x559069cf59f0 2024-05-17T22:45:30.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.171+0000 7f45437fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559069cf59f0 0x559069e78ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:30.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.171+0000 7f4543fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559069e79400 0x559069e735c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:30.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.171+0000 7f45437fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559069cf59f0 0x559069e78ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:55020/0 (socket says 172.21.15.12:55020) 2024-05-17T22:45:30.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.171+0000 7f45437fe700 1 -- 172.21.15.12:0/1431622712 learned_addr learned my addr 172.21.15.12:0/1431622712 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:45:30.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.171+0000 7f45417fa700 1 --2- 172.21.15.12:0/664088503 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f452c0b2a10 0x7f452c0b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:30.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.171+0000 7f4559159b80 1 -- 172.21.15.12:0/1431622712 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559069d95cb0 con 0x559069e73b00 2024-05-17T22:45:30.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.172+0000 7f45417fa700 1 -- 172.21.15.12:0/664088503 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f45380e1bb0 con 0x7f4544107fd0 2024-05-17T22:45:30.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.172+0000 7f45437fe700 1 --2- 172.21.15.12:0/1431622712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559069e73b00 0x559069e73f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:30.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.172+0000 7f45437fe700 1 -- 172.21.15.12:0/1431622712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559069e73b00 msgr2=0x559069e73f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:30.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.172+0000 7f4548d1b700 1 --2- 172.21.15.12:0/664088503 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f452c0b2a10 0x7f452c0b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:30.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.172+0000 7f45437fe700 1 --2- 172.21.15.12:0/1431622712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559069e73b00 0x559069e73f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:30.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.172+0000 7f45437fe700 1 -- 172.21.15.12:0/1431622712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559069e79400 msgr2=0x559069e735c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:30.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.172+0000 7f45437fe700 1 --2- 172.21.15.12:0/1431622712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559069e79400 0x559069e735c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:30.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.172+0000 7f45437fe700 1 -- 172.21.15.12:0/1431622712 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559069d2b5c0 con 0x559069cf59f0 2024-05-17T22:45:30.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.173+0000 7f4543fff700 1 --2- 172.21.15.12:0/1431622712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559069e79400 0x559069e735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:45:30.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.173+0000 7f4548d1b700 1 --2- 172.21.15.12:0/664088503 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f452c0b2a10 0x7f452c0b4f20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f45380e1e50 tx=0x7f45380106e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:45:30.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.173+0000 7f45437fe700 1 --2- 172.21.15.12:0/1431622712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559069cf59f0 0x559069e78ec0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4534013670 tx=0x7f453400a8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4d25e48cb1110b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:45:30.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.173+0000 7f452a7fc700 1 -- 172.21.15.12:0/1431622712 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4534019030 con 0x559069cf59f0 2024-05-17T22:45:30.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.173+0000 7f452a7fc700 1 -- 172.21.15.12:0/1431622712 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4534004ce0 con 0x559069cf59f0 2024-05-17T22:45:30.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.173+0000 7f452a7fc700 1 -- 172.21.15.12:0/1431622712 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453400f630 con 0x559069cf59f0 2024-05-17T22:45:30.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.174+0000 7f4559159b80 1 -- 172.21.15.12:0/1431622712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559069cf59f0 msgr2=0x559069e78ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:30.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.174+0000 7f4559159b80 1 --2- 172.21.15.12:0/1431622712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559069cf59f0 0x559069e78ec0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4534013670 tx=0x7f453400a8a0 comp rx=0 tx=0).stop 2024-05-17T22:45:30.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.174+0000 7f4559159b80 1 -- 172.21.15.12:0/1431622712 shutdown_connections 2024-05-17T22:45:30.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.174+0000 7f4559159b80 1 --2- 172.21.15.12:0/1431622712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559069e73b00 0x559069e73f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:30.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.174+0000 7f4559159b80 1 --2- 172.21.15.12:0/1431622712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559069cf59f0 0x559069e78ec0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:30.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.174+0000 7f4559159b80 1 --2- 172.21.15.12:0/1431622712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559069e79400 0x559069e735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:30.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.174+0000 7f4559159b80 1 -- 172.21.15.12:0/1431622712 >> 172.21.15.12:0/1431622712 conn(0x559069d557d0 msgr2=0x559069e6e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:45:30.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.174+0000 7f4559159b80 1 -- 172.21.15.12:0/1431622712 shutdown_connections 2024-05-17T22:45:30.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.174+0000 7f4559159b80 1 -- 172.21.15.12:0/1431622712 wait complete. 2024-05-17T22:45:30.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.175+0000 7f4559159b80 1 -- start start 2024-05-17T22:45:30.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.175+0000 7f4559159b80 1 Processor -- start 2024-05-17T22:45:30.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.176+0000 7f4559159b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559069cf59f0 0x559069e7ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:30.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.176+0000 7f4559159b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559069e73b00 0x559069e7cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:30.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.176+0000 7f4559159b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559069e79400 0x559069e7d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:30.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.176+0000 7f4559159b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x559069c99150 con 0x559069cf59f0 2024-05-17T22:45:30.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.176+0000 7f4559159b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559069d33940 con 0x559069e73b00 2024-05-17T22:45:30.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.176+0000 7f4559159b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559069e7fdd0 con 0x559069e79400 2024-05-17T22:45:30.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.176+0000 7f4559159b80 10 client.0 _ll_register_callbacks cb 0x559069d28b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:45:30.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.176+0000 7f45437fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559069e79400 0x559069e7d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:30.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.176+0000 7f45437fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559069e79400 0x559069e7d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:55018/0 (socket says 172.21.15.12:55018) 2024-05-17T22:45:30.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.176+0000 7f4543fff700 1 --2- 172.21.15.12:0/2257599355 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559069e73b00 0x559069e7cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:30.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.176+0000 7f45437fe700 1 -- 172.21.15.12:0/2257599355 learned_addr learned my addr 172.21.15.12:0/2257599355 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:45:30.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.177+0000 7f45437fe700 1 --2- 172.21.15.12:0/2257599355 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559069cf59f0 0x559069e7ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:30.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.177+0000 7f4543fff700 1 -- 172.21.15.12:0/2257599355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559069e79400 msgr2=0x559069e7d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:30.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.177+0000 7f4543fff700 1 --2- 172.21.15.12:0/2257599355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559069e79400 0x559069e7d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:30.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.177+0000 7f4543fff700 1 -- 172.21.15.12:0/2257599355 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559069cf59f0 msgr2=0x559069e7ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:30.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.177+0000 7f4543fff700 1 --2- 172.21.15.12:0/2257599355 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559069cf59f0 0x559069e7ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:30.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.177+0000 7f4543fff700 1 -- 172.21.15.12:0/2257599355 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559069ce2480 con 0x559069e73b00 2024-05-17T22:45:30.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.178+0000 7f45437fe700 1 --2- 172.21.15.12:0/2257599355 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559069cf59f0 0x559069e7ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:45:30.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.178+0000 7f4543fff700 1 --2- 172.21.15.12:0/2257599355 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559069e73b00 0x559069e7cfc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f453c01acb0 tx=0x7f453c003d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69d9be270540a118 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:45:30.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.178+0000 7f45237fe700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:45:30.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.178+0000 7f45237fe700 1 -- 172.21.15.12:0/2257599355 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453c031070 con 0x559069e73b00 2024-05-17T22:45:30.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.178+0000 7f45237fe700 1 -- 172.21.15.12:0/2257599355 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f453c018aa0 con 0x559069e73b00 2024-05-17T22:45:30.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.178+0000 7f4559159b80 1 -- 172.21.15.12:0/2257599355 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4534004ce0 con 0x559069e73b00 2024-05-17T22:45:30.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.179+0000 7f45237fe700 1 -- 172.21.15.12:0/2257599355 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f453c03f050 con 0x559069e73b00 2024-05-17T22:45:30.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.179+0000 7f4559159b80 10 client.25532 fetch_fsmap learned FSMap version 37 2024-05-17T22:45:30.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.179+0000 7f4559159b80 1 -- 172.21.15.12:0/2257599355 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559069e8a8e0 con 0x559069e73b00 2024-05-17T22:45:30.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.179+0000 7f45237fe700 1 -- 172.21.15.12:0/2257599355 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4534004ce0 con 0x559069e73b00 2024-05-17T22:45:30.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.179+0000 7f45237fe700 1 -- 172.21.15.12:0/2257599355 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f453c022070 con 0x559069e73b00 2024-05-17T22:45:30.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.179+0000 7f4559159b80 10 client.25532 fetch_fsmap finished waiting for FSMap version 37 2024-05-17T22:45:30.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.179+0000 7f4559159b80 10 client.25532 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-17T22:45:30.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.179+0000 7f4559159b80 20 client.25532 populate_metadata read hostname 'smithi012' 2024-05-17T22:45:30.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.179+0000 7f4559159b80 1 --2- 172.21.15.12:0/2257599355 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x559069e8cf20 0x559069e8f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:30.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.179+0000 7f4559159b80 4 client.25532 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:45:30.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.179+0000 7f4559159b80 1 -- 172.21.15.12:0/2257599355 --> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559069da4020 con 0x559069e8cf20 2024-05-17T22:45:30.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.180+0000 7f45437fe700 1 --2- 172.21.15.12:0/2257599355 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x559069e8cf20 0x559069e8f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:30.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.181+0000 7f45237fe700 10 client.25532 ms_handle_connect on v2:172.21.15.151:6838/2338353849 2024-05-17T22:45:30.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.181+0000 7f45437fe700 1 --2- 172.21.15.12:0/2257599355 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x559069e8cf20 0x559069e8f390 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=22e71ee6db9f893a server_cookie=d5edf1b7997c3178 in_seq=0 out_seq=0 2024-05-17T22:45:30.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.182+0000 7f45237fe700 1 -- 172.21.15.12:0/2257599355 <== mds.2 v2:172.21.15.151:6838/2338353849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559069da4020 con 0x559069e8cf20 2024-05-17T22:45:30.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.182+0000 7f45237fe700 10 client.25532 handle_command_reply: tid=0 2024-05-17T22:45:30.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.182+0000 7f4559159b80 1 client.25532 shutdown 2024-05-17T22:45:30.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.182+0000 7f4559159b80 1 -- 172.21.15.12:0/2257599355 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559069e73b00 msgr2=0x559069e7cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:30.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.182+0000 7f4559159b80 1 --2- 172.21.15.12:0/2257599355 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559069e73b00 0x559069e7cfc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f453c01acb0 tx=0x7f453c003d70 comp rx=0 tx=0).stop 2024-05-17T22:45:30.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.185+0000 7f4559159b80 1 -- 172.21.15.12:0/2257599355 shutdown_connections 2024-05-17T22:45:30.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.185+0000 7f4559159b80 1 --2- 172.21.15.12:0/2257599355 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x559069e8cf20 0x559069e8f390 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:30.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.185+0000 7f4559159b80 1 --2- 172.21.15.12:0/2257599355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559069e79400 0x559069e7d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:30.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.185+0000 7f4559159b80 1 --2- 172.21.15.12:0/2257599355 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559069cf59f0 0x559069e7ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:30.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.185+0000 7f4559159b80 1 --2- 172.21.15.12:0/2257599355 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559069e73b00 0x559069e7cfc0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:30.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.185+0000 7f4559159b80 1 -- 172.21.15.12:0/2257599355 >> 172.21.15.12:0/2257599355 conn(0x559069d557d0 msgr2=0x559069e6e160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:45:30.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.185+0000 7f45237fe700 0 client.25532 ms_handle_reset on v2:172.21.15.151:6838/2338353849 2024-05-17T22:45:30.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.185+0000 7f4559159b80 1 -- 172.21.15.12:0/2257599355 shutdown_connections 2024-05-17T22:45:30.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.186+0000 7f4559159b80 1 -- 172.21.15.12:0/2257599355 wait complete. 2024-05-17T22:45:30.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.186+0000 7f4559159b80 20 client.25532 trim_cache size 0 max 16384 2024-05-17T22:45:30.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.211+0000 7f4559159b80 1 Processor -- start 2024-05-17T22:45:30.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.211+0000 7f4559159b80 1 -- start start 2024-05-17T22:45:30.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.212+0000 7f4559159b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559069cf59f0 0x559069e94ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:30.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.212+0000 7f4559159b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559069e73b00 0x559069e950e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:30.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.212+0000 7f4559159b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559069e79400 0x559069e77e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:30.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.212+0000 7f4559159b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f45441b8bf0 con 0x559069e79400 2024-05-17T22:45:30.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.212+0000 7f4559159b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f45441b6480 con 0x559069cf59f0 2024-05-17T22:45:30.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.212+0000 7f4559159b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f45441b42d0 con 0x559069e73b00 2024-05-17T22:45:30.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.212+0000 7f4543fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559069e73b00 0x559069e950e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:30.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.212+0000 7f4543fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559069e73b00 0x559069e950e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:55030/0 (socket says 172.21.15.12:55030) 2024-05-17T22:45:30.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.212+0000 7f4543fff700 1 -- 172.21.15.12:0/1525606014 learned_addr learned my addr 172.21.15.12:0/1525606014 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:45:30.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.212+0000 7f45437fe700 1 --2- 172.21.15.12:0/1525606014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559069e79400 0x559069e77e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:30.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.212+0000 7f45437fe700 1 --2- 172.21.15.12:0/1525606014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559069cf59f0 0x559069e94ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:30.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.217+0000 7f4543fff700 1 -- 172.21.15.12:0/1525606014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559069cf59f0 msgr2=0x559069e94ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:30.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.217+0000 7f4543fff700 1 --2- 172.21.15.12:0/1525606014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559069cf59f0 0x559069e94ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:30.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.217+0000 7f4543fff700 1 -- 172.21.15.12:0/1525606014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559069e79400 msgr2=0x559069e77e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:30.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.217+0000 7f4543fff700 1 --2- 172.21.15.12:0/1525606014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559069e79400 0x559069e77e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:30.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.217+0000 7f4543fff700 1 -- 172.21.15.12:0/1525606014 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559069e7f280 con 0x559069e73b00 2024-05-17T22:45:30.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.217+0000 7f4543fff700 1 --2- 172.21.15.12:0/1525606014 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559069e73b00 0x559069e950e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f453c000f80 tx=0x7f453c01a9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34323424c732ba89 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:45:30.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.225+0000 7f4529ffb700 1 -- 172.21.15.12:0/1525606014 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453c031040 con 0x559069e73b00 2024-05-17T22:45:30.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.225+0000 7f4529ffb700 1 -- 172.21.15.12:0/1525606014 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f453c02d5e0 con 0x559069e73b00 2024-05-17T22:45:30.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.225+0000 7f4529ffb700 1 -- 172.21.15.12:0/1525606014 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453c018ca0 con 0x559069e73b00 2024-05-17T22:45:30.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.227+0000 7f4559159b80 1 -- 172.21.15.12:0/1525606014 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559069e73b00 msgr2=0x559069e950e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:30.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.227+0000 7f4559159b80 1 --2- 172.21.15.12:0/1525606014 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559069e73b00 0x559069e950e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f453c000f80 tx=0x7f453c01a9f0 comp rx=0 tx=0).stop 2024-05-17T22:45:30.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.232+0000 7f4559159b80 1 -- 172.21.15.12:0/1525606014 shutdown_connections 2024-05-17T22:45:30.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.232+0000 7f4559159b80 1 --2- 172.21.15.12:0/1525606014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559069e79400 0x559069e77e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:30.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.232+0000 7f4559159b80 1 --2- 172.21.15.12:0/1525606014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559069cf59f0 0x559069e94ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:30.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.232+0000 7f4559159b80 1 --2- 172.21.15.12:0/1525606014 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559069e73b00 0x559069e950e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:30.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.232+0000 7f4559159b80 1 -- 172.21.15.12:0/1525606014 >> 172.21.15.12:0/1525606014 conn(0x559069d557d0 msgr2=0x559069e8ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:45:30.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.236+0000 7f4559159b80 1 -- 172.21.15.12:0/1525606014 shutdown_connections 2024-05-17T22:45:30.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.236+0000 7f4559159b80 1 -- 172.21.15.12:0/1525606014 wait complete. 2024-05-17T22:45:30.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.236+0000 7f4559159b80 1 -- start start 2024-05-17T22:45:30.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.237+0000 7f4559159b80 1 Processor -- start 2024-05-17T22:45:30.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.238+0000 7f4559159b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559069cf59f0 0x559069e85040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:30.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.238+0000 7f4559159b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559069e73b00 0x559069e85580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:30.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.238+0000 7f4559159b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559069e79400 0x559069e82450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:30.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.238+0000 7f4559159b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f45441b4940 con 0x559069e79400 2024-05-17T22:45:30.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.238+0000 7f4559159b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f45441b5db0 con 0x559069cf59f0 2024-05-17T22:45:30.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.238+0000 7f45437fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559069cf59f0 0x559069e85040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:30.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.238+0000 7f4543fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559069e73b00 0x559069e85580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:30.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.238+0000 7f4543fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559069e73b00 0x559069e85580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:55040/0 (socket says 172.21.15.12:55040) 2024-05-17T22:45:30.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.238+0000 7f4543fff700 1 -- 172.21.15.12:0/3043135826 learned_addr learned my addr 172.21.15.12:0/3043135826 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:45:30.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.239+0000 7f4559159b80 1 -- 172.21.15.12:0/3043135826 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f45441b5010 con 0x559069e73b00 2024-05-17T22:45:30.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.239+0000 7f4559159b80 10 client.0 _ll_register_callbacks cb 0x559069d28b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:45:30.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.240+0000 7f4543fff700 1 -- 172.21.15.12:0/3043135826 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559069cf59f0 msgr2=0x559069e85040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:30.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.240+0000 7f4543fff700 1 --2- 172.21.15.12:0/3043135826 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559069cf59f0 0x559069e85040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:30.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.240+0000 7f4543fff700 1 -- 172.21.15.12:0/3043135826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559069e79400 msgr2=0x559069e82450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:30.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.240+0000 7f4543fff700 1 --2- 172.21.15.12:0/3043135826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559069e79400 0x559069e82450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:30.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.240+0000 7f4543fff700 1 -- 172.21.15.12:0/3043135826 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559069e82ba0 con 0x559069e73b00 2024-05-17T22:45:30.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.240+0000 7f4543fff700 1 --2- 172.21.15.12:0/3043135826 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559069e73b00 0x559069e85580 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f453c01c0d0 tx=0x7f453c018e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73f2e0a64d2be2cf server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:45:30.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.241+0000 7f45237fe700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:45:30.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.246+0000 7f45237fe700 1 -- 172.21.15.12:0/3043135826 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453c02daf0 con 0x559069e73b00 2024-05-17T22:45:30.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.246+0000 7f45237fe700 1 -- 172.21.15.12:0/3043135826 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f453c02dc90 con 0x559069e73b00 2024-05-17T22:45:30.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.247+0000 7f4559159b80 1 -- 172.21.15.12:0/3043135826 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f453c02d5e0 con 0x559069e73b00 2024-05-17T22:45:30.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.258+0000 7f45237fe700 1 -- 172.21.15.12:0/3043135826 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f453c055050 con 0x559069e73b00 2024-05-17T22:45:30.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.259+0000 7f4559159b80 10 client.25480 fetch_fsmap learned FSMap version 37 2024-05-17T22:45:30.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.259+0000 7f4559159b80 1 -- 172.21.15.12:0/3043135826 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559069ef7100 con 0x559069e73b00 2024-05-17T22:45:30.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.262+0000 7f45237fe700 1 -- 172.21.15.12:0/3043135826 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f453c02d5e0 con 0x559069e73b00 2024-05-17T22:45:30.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.262+0000 7f45237fe700 1 -- 172.21.15.12:0/3043135826 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f453c042070 con 0x559069e73b00 2024-05-17T22:45:30.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.262+0000 7f4559159b80 10 client.25480 fetch_fsmap finished waiting for FSMap version 37 2024-05-17T22:45:30.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.262+0000 7f4559159b80 10 client.25480 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-17T22:45:30.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.262+0000 7f4559159b80 20 client.25480 populate_metadata read hostname 'smithi012' 2024-05-17T22:45:30.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.262+0000 7f4559159b80 1 --2- 172.21.15.12:0/3043135826 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x559069ef7720 0x559069ef9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:30.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.262+0000 7f4559159b80 4 client.25480 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:45:30.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.262+0000 7f4559159b80 1 -- 172.21.15.12:0/3043135826 --> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559069d9d800 con 0x559069ef7720 2024-05-17T22:45:30.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.263+0000 7f45437fe700 1 --2- 172.21.15.12:0/3043135826 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x559069ef7720 0x559069ef9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:30.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.269+0000 7f45437fe700 1 --2- 172.21.15.12:0/3043135826 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x559069ef7720 0x559069ef9b40 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=39daf81bc64454b server_cookie=14817f283180d1e5 in_seq=0 out_seq=0 2024-05-17T22:45:30.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.271+0000 7f45237fe700 10 client.25480 ms_handle_connect on v2:172.21.15.151:6838/2338353849 2024-05-17T22:45:30.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.274+0000 7f45237fe700 1 -- 172.21.15.12:0/3043135826 <== mds.2 v2:172.21.15.151:6838/2338353849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559069d9d800 con 0x559069ef7720 2024-05-17T22:45:30.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.274+0000 7f45237fe700 10 client.25480 handle_command_reply: tid=0 2024-05-17T22:45:30.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.274+0000 7f4559159b80 1 client.25480 shutdown 2024-05-17T22:45:30.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.282+0000 7f4559159b80 1 -- 172.21.15.12:0/3043135826 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559069e73b00 msgr2=0x559069e85580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:30.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.282+0000 7f4559159b80 1 --2- 172.21.15.12:0/3043135826 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559069e73b00 0x559069e85580 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f453c01c0d0 tx=0x7f453c018e10 comp rx=0 tx=0).stop 2024-05-17T22:45:30.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.286+0000 7f4559159b80 1 -- 172.21.15.12:0/3043135826 shutdown_connections 2024-05-17T22:45:30.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.286+0000 7f4559159b80 1 --2- 172.21.15.12:0/3043135826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559069e79400 0x559069e82450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:30.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.286+0000 7f4559159b80 1 --2- 172.21.15.12:0/3043135826 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559069cf59f0 0x559069e85040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:30.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.286+0000 7f4559159b80 1 --2- 172.21.15.12:0/3043135826 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x559069ef7720 0x559069ef9b40 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:30.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.286+0000 7f4559159b80 1 --2- 172.21.15.12:0/3043135826 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559069e73b00 0x559069e85580 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:30.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.286+0000 7f4559159b80 1 -- 172.21.15.12:0/3043135826 >> 172.21.15.12:0/3043135826 conn(0x559069d557d0 msgr2=0x559069e77e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:45:30.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.286+0000 7f45237fe700 0 client.25480 ms_handle_reset on v2:172.21.15.151:6838/2338353849 2024-05-17T22:45:30.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.289+0000 7f4559159b80 1 -- 172.21.15.12:0/3043135826 shutdown_connections 2024-05-17T22:45:30.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.289+0000 7f4559159b80 1 -- 172.21.15.12:0/3043135826 wait complete. 2024-05-17T22:45:30.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.289+0000 7f4559159b80 20 client.25480 trim_cache size 0 max 16384 2024-05-17T22:45:30.292 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:45:30.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.292+0000 7f454a77e700 1 -- 172.21.15.12:0/664088503 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f452c0b2a10 msgr2=0x7f452c0b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:30.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.292+0000 7f454a77e700 1 --2- 172.21.15.12:0/664088503 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f452c0b2a10 0x7f452c0b4f20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f45380e1e50 tx=0x7f45380106e0 comp rx=0 tx=0).stop 2024-05-17T22:45:30.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.292+0000 7f454a77e700 1 -- 172.21.15.12:0/664088503 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4544107fd0 msgr2=0x7f45440745b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:30.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.292+0000 7f454a77e700 1 --2- 172.21.15.12:0/664088503 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4544107fd0 0x7f45440745b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f453800b3f0 tx=0x7f453800b700 comp rx=0 tx=0).stop 2024-05-17T22:45:30.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.293+0000 7f454a77e700 1 -- 172.21.15.12:0/664088503 shutdown_connections 2024-05-17T22:45:30.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.293+0000 7f454a77e700 1 --2- 172.21.15.12:0/664088503 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f452c0b2a10 0x7f452c0b4f20 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:30.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.293+0000 7f454a77e700 1 --2- 172.21.15.12:0/664088503 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4544107fd0 0x7f45440745b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:30.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.293+0000 7f454a77e700 1 --2- 172.21.15.12:0/664088503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4544102b90 0x7f454410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:30.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.293+0000 7f454a77e700 1 --2- 172.21.15.12:0/664088503 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45441034e0 0x7f454410feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:30.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.293+0000 7f454a77e700 1 -- 172.21.15.12:0/664088503 >> 172.21.15.12:0/664088503 conn(0x7f4544079bd0 msgr2=0x7f45441059e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:45:30.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.295+0000 7f454a77e700 1 -- 172.21.15.12:0/664088503 shutdown_connections 2024-05-17T22:45:30.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:30.296+0000 7f454a77e700 1 -- 172.21.15.12:0/664088503 wait complete. 2024-05-17T22:45:31.188 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-17T22:45:31.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 370.0 s, 621.2 tps, lat 154.249 ms stddev 207.720 2024-05-17T22:45:31.635 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:45:32.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:31 smithi096 ceph-mon[36065]: pgmap v627: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.2 MiB/s wr, 526 op/s 2024-05-17T22:45:32.079 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:31 smithi012 ceph-mon[31632]: pgmap v627: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.2 MiB/s wr, 526 op/s 2024-05-17T22:45:32.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:31 smithi151 ceph-mon[36101]: pgmap v627: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.2 MiB/s wr, 526 op/s 2024-05-17T22:45:33.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.857+0000 7f13980cf700 1 -- 172.21.15.12:0/136252672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1390108080 msgr2=0x7f139010f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:33.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.857+0000 7f13980cf700 1 --2- 172.21.15.12:0/136252672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1390108080 0x7f139010f9e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1388009f90 tx=0x7f138800b330 comp rx=0 tx=0).stop 2024-05-17T22:45:33.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.857+0000 7f13980cf700 1 -- 172.21.15.12:0/136252672 shutdown_connections 2024-05-17T22:45:33.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.857+0000 7f13980cf700 1 --2- 172.21.15.12:0/136252672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1390108080 0x7f139010f9e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:33.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.857+0000 7f13980cf700 1 --2- 172.21.15.12:0/136252672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1390102e00 0x7f1390103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:33.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.857+0000 7f13980cf700 1 --2- 172.21.15.12:0/136252672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1390107730 0x7f1390107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:33.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.857+0000 7f13980cf700 1 -- 172.21.15.12:0/136252672 >> 172.21.15.12:0/136252672 conn(0x7f1390079bd0 msgr2=0x7f1390079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:45:33.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.858+0000 7f13980cf700 1 -- 172.21.15.12:0/136252672 shutdown_connections 2024-05-17T22:45:33.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.858+0000 7f13980cf700 1 -- 172.21.15.12:0/136252672 wait complete. 2024-05-17T22:45:33.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.859+0000 7f13980cf700 1 Processor -- start 2024-05-17T22:45:33.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.859+0000 7f13980cf700 1 -- start start 2024-05-17T22:45:33.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.860+0000 7f13980cf700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1390102e00 0x7f1390198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:33.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.860+0000 7f13980cf700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1390107730 0x7f1390198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:33.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.860+0000 7f13980cf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1390108080 0x7f139019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:33.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.860+0000 7f13980cf700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1390111c40 con 0x7f1390108080 2024-05-17T22:45:33.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.860+0000 7f13980cf700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1390111ac0 con 0x7f1390102e00 2024-05-17T22:45:33.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.860+0000 7f13980cf700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1390111dc0 con 0x7f1390107730 2024-05-17T22:45:33.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.860+0000 7f1395e6b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1390102e00 0x7f1390198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:33.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.860+0000 7f1395e6b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1390102e00 0x7f1390198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:55066/0 (socket says 172.21.15.12:55066) 2024-05-17T22:45:33.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.860+0000 7f1395e6b700 1 -- 172.21.15.12:0/1258279275 learned_addr learned my addr 172.21.15.12:0/1258279275 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:45:33.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.860+0000 7f139566a700 1 --2- 172.21.15.12:0/1258279275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1390107730 0x7f1390198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:33.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.860+0000 7f139666c700 1 --2- 172.21.15.12:0/1258279275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1390108080 0x7f139019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:33.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.861+0000 7f1395e6b700 1 -- 172.21.15.12:0/1258279275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1390107730 msgr2=0x7f1390198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:33.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.861+0000 7f1395e6b700 1 --2- 172.21.15.12:0/1258279275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1390107730 0x7f1390198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:33.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.861+0000 7f1395e6b700 1 -- 172.21.15.12:0/1258279275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1390108080 msgr2=0x7f139019cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:33.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.861+0000 7f1395e6b700 1 --2- 172.21.15.12:0/1258279275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1390108080 0x7f139019cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:33.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.861+0000 7f1395e6b700 1 -- 172.21.15.12:0/1258279275 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f139019d3f0 con 0x7f1390102e00 2024-05-17T22:45:33.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.863+0000 7f1395e6b700 1 --2- 172.21.15.12:0/1258279275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1390102e00 0x7f1390198520 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f138c00e9a0 tx=0x7f138c00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:45:33.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.863+0000 7f1386ffd700 1 -- 172.21.15.12:0/1258279275 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138c00cbd0 con 0x7f1390102e00 2024-05-17T22:45:33.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.864+0000 7f1386ffd700 1 -- 172.21.15.12:0/1258279275 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f138c00cd70 con 0x7f1390102e00 2024-05-17T22:45:33.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.865+0000 7f1386ffd700 1 -- 172.21.15.12:0/1258279275 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138c018680 con 0x7f1390102e00 2024-05-17T22:45:33.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.866+0000 7f13980cf700 1 -- 172.21.15.12:0/1258279275 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f139019d6e0 con 0x7f1390102e00 2024-05-17T22:45:33.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.866+0000 7f13980cf700 1 -- 172.21.15.12:0/1258279275 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f139019dae0 con 0x7f1390102e00 2024-05-17T22:45:33.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.869+0000 7f1386ffd700 1 -- 172.21.15.12:0/1258279275 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f138c018820 con 0x7f1390102e00 2024-05-17T22:45:33.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.870+0000 7f13a6aaab80 1 Processor -- start 2024-05-17T22:45:33.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.871+0000 7f13a6aaab80 1 -- start start 2024-05-17T22:45:33.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.871+0000 7f1386ffd700 1 --2- 172.21.15.12:0/1258279275 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f137c0b2920 0x7f137c0b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:33.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.871+0000 7f139666c700 1 --2- 172.21.15.12:0/1258279275 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f137c0b2920 0x7f137c0b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:33.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.871+0000 7f1386ffd700 1 -- 172.21.15.12:0/1258279275 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f138c0e2850 con 0x7f1390102e00 2024-05-17T22:45:33.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.871+0000 7f13a6aaab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55998e3739f0 0x55998e4f6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:33.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.871+0000 7f13a6aaab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55998e4f7400 0x55998e4f15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:33.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.871+0000 7f13a6aaab80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55998e4f1b00 0x55998e4f1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:33.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.872+0000 7f13a6aaab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55998e4200d0 con 0x55998e4f7400 2024-05-17T22:45:33.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.872+0000 7f13a6aaab80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55998e40fc70 con 0x55998e4f1b00 2024-05-17T22:45:33.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.872+0000 7f13a6aaab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55998e413cb0 con 0x55998e3739f0 2024-05-17T22:45:33.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.872+0000 7f139666c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55998e4f1b00 0x55998e4f1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:33.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.872+0000 7f139666c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55998e4f1b00 0x55998e4f1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:55072/0 (socket says 172.21.15.12:55072) 2024-05-17T22:45:33.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.872+0000 7f139666c700 1 -- 172.21.15.12:0/1980582091 learned_addr learned my addr 172.21.15.12:0/1980582091 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:45:33.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.872+0000 7f139566a700 1 --2- 172.21.15.12:0/1980582091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55998e4f7400 0x55998e4f15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:33.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.872+0000 7f1395e6b700 1 --2- 172.21.15.12:0/1980582091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55998e3739f0 0x55998e4f6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:33.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.872+0000 7f139666c700 1 -- 172.21.15.12:0/1980582091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55998e3739f0 msgr2=0x55998e4f6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:33.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.872+0000 7f139666c700 1 --2- 172.21.15.12:0/1980582091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55998e3739f0 0x55998e4f6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:33.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.872+0000 7f139666c700 1 -- 172.21.15.12:0/1980582091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55998e4f7400 msgr2=0x55998e4f15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:33.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.872+0000 7f139666c700 1 --2- 172.21.15.12:0/1980582091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55998e4f7400 0x55998e4f15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:33.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.872+0000 7f139666c700 1 -- 172.21.15.12:0/1980582091 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55998e3a95c0 con 0x55998e4f1b00 2024-05-17T22:45:33.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.872+0000 7f139666c700 1 --2- 172.21.15.12:0/1980582091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55998e4f1b00 0x55998e4f1f10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f138800ba20 tx=0x7f138800ba50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d3547b8551bdd19 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:45:33.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.873+0000 7f137bfff700 1 -- 172.21.15.12:0/1980582091 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1388031070 con 0x55998e4f1b00 2024-05-17T22:45:33.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.873+0000 7f137bfff700 1 -- 172.21.15.12:0/1980582091 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1388021070 con 0x55998e4f1b00 2024-05-17T22:45:33.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.873+0000 7f137bfff700 1 -- 172.21.15.12:0/1980582091 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1388026050 con 0x55998e4f1b00 2024-05-17T22:45:33.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.874+0000 7f139666c700 1 --2- 172.21.15.12:0/1258279275 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f137c0b2920 0x7f137c0b4e30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f138802e000 tx=0x7f1388021000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:45:33.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.874+0000 7f13a6aaab80 1 -- 172.21.15.12:0/1980582091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55998e4f1b00 msgr2=0x55998e4f1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:33.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.874+0000 7f13a6aaab80 1 --2- 172.21.15.12:0/1980582091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55998e4f1b00 0x55998e4f1f10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f138800ba20 tx=0x7f138800ba50 comp rx=0 tx=0).stop 2024-05-17T22:45:33.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.874+0000 7f13a6aaab80 1 -- 172.21.15.12:0/1980582091 shutdown_connections 2024-05-17T22:45:33.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.874+0000 7f13a6aaab80 1 --2- 172.21.15.12:0/1980582091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55998e4f1b00 0x55998e4f1f10 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:33.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.874+0000 7f13a6aaab80 1 --2- 172.21.15.12:0/1980582091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55998e3739f0 0x55998e4f6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:33.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.874+0000 7f13a6aaab80 1 --2- 172.21.15.12:0/1980582091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55998e4f7400 0x55998e4f15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:33.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.874+0000 7f13a6aaab80 1 -- 172.21.15.12:0/1980582091 >> 172.21.15.12:0/1980582091 conn(0x55998e3d37d0 msgr2=0x55998e4ec1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:45:33.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.874+0000 7f13a6aaab80 1 -- 172.21.15.12:0/1980582091 shutdown_connections 2024-05-17T22:45:33.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.874+0000 7f13a6aaab80 1 -- 172.21.15.12:0/1980582091 wait complete. 2024-05-17T22:45:33.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.875+0000 7f13a6aaab80 1 -- start start 2024-05-17T22:45:33.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.875+0000 7f13a6aaab80 1 Processor -- start 2024-05-17T22:45:33.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.876+0000 7f13a6aaab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55998e3739f0 0x55998e4faaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:33.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.876+0000 7f13a6aaab80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55998e4f1b00 0x55998e4fafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:33.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.876+0000 7f13a6aaab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55998e4f7400 0x55998e4fb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:33.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.876+0000 7f13a6aaab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55998e317150 con 0x55998e4f7400 2024-05-17T22:45:33.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.876+0000 7f13a6aaab80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55998e3b1940 con 0x55998e4f1b00 2024-05-17T22:45:33.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.876+0000 7f139566a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55998e4f1b00 0x55998e4fafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:33.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.877+0000 7f139566a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55998e4f1b00 0x55998e4fafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:55086/0 (socket says 172.21.15.12:55086) 2024-05-17T22:45:33.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.877+0000 7f139566a700 1 -- 172.21.15.12:0/2640648834 learned_addr learned my addr 172.21.15.12:0/2640648834 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:45:33.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.877+0000 7f13a6aaab80 1 -- 172.21.15.12:0/2640648834 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55998e4fddd0 con 0x55998e3739f0 2024-05-17T22:45:33.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.877+0000 7f13a6aaab80 10 client.0 _ll_register_callbacks cb 0x55998e3a6b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:45:33.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.877+0000 7f1395e6b700 1 --2- 172.21.15.12:0/2640648834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55998e3739f0 0x55998e4faaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:33.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.877+0000 7f139666c700 1 --2- 172.21.15.12:0/2640648834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55998e4f7400 0x55998e4fb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:33.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.877+0000 7f139566a700 1 -- 172.21.15.12:0/2640648834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55998e3739f0 msgr2=0x55998e4faaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:33.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.877+0000 7f139566a700 1 --2- 172.21.15.12:0/2640648834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55998e3739f0 0x55998e4faaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:33.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.877+0000 7f139566a700 1 -- 172.21.15.12:0/2640648834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55998e4f7400 msgr2=0x55998e4fb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:33.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.877+0000 7f139566a700 1 --2- 172.21.15.12:0/2640648834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55998e4f7400 0x55998e4fb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:33.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.877+0000 7f139566a700 1 -- 172.21.15.12:0/2640648834 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55998e360480 con 0x55998e4f1b00 2024-05-17T22:45:33.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.877+0000 7f139666c700 1 --2- 172.21.15.12:0/2640648834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55998e4f7400 0x55998e4fb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:45:33.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.877+0000 7f139566a700 1 --2- 172.21.15.12:0/2640648834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55998e4f1b00 0x55998e4fafe0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f138000c380 tx=0x7f138000c690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65460c131e6ee69e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:45:33.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.877+0000 7f13797fa700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:45:33.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.878+0000 7f13797fa700 1 -- 172.21.15.12:0/2640648834 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1380004440 con 0x55998e4f1b00 2024-05-17T22:45:33.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.878+0000 7f13797fa700 1 -- 172.21.15.12:0/2640648834 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f1380003680 con 0x55998e4f1b00 2024-05-17T22:45:33.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.878+0000 7f13a6aaab80 1 -- 172.21.15.12:0/2640648834 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1388021070 con 0x55998e4f1b00 2024-05-17T22:45:33.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.878+0000 7f13797fa700 1 -- 172.21.15.12:0/2640648834 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1380026050 con 0x55998e4f1b00 2024-05-17T22:45:33.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.878+0000 7f13a6aaab80 10 client.25550 fetch_fsmap learned FSMap version 37 2024-05-17T22:45:33.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.878+0000 7f13a6aaab80 1 -- 172.21.15.12:0/2640648834 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55998e5086f0 con 0x55998e4f1b00 2024-05-17T22:45:33.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.878+0000 7f13797fa700 1 -- 172.21.15.12:0/2640648834 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1380003d30 con 0x55998e4f1b00 2024-05-17T22:45:33.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.878+0000 7f13797fa700 1 -- 172.21.15.12:0/2640648834 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f1380021030 con 0x55998e4f1b00 2024-05-17T22:45:33.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.879+0000 7f13a6aaab80 10 client.25550 fetch_fsmap finished waiting for FSMap version 37 2024-05-17T22:45:33.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.879+0000 7f13a6aaab80 10 client.25550 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-17T22:45:33.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.879+0000 7f13a6aaab80 20 client.25550 populate_metadata read hostname 'smithi012' 2024-05-17T22:45:33.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.879+0000 7f13a6aaab80 1 --2- 172.21.15.12:0/2640648834 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55998e50ad80 0x55998e50d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:33.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.879+0000 7f13a6aaab80 4 client.25550 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:45:33.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.879+0000 7f13a6aaab80 1 -- 172.21.15.12:0/2640648834 --> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1388021070 con 0x55998e50ad80 2024-05-17T22:45:33.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.879+0000 7f1395e6b700 1 --2- 172.21.15.12:0/2640648834 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55998e50ad80 0x55998e50d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:33.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.880+0000 7f1395e6b700 1 --2- 172.21.15.12:0/2640648834 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55998e50ad80 0x55998e50d1e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aa08fb59bf3a2ff9 server_cookie=f908eb948d24e802 in_seq=0 out_seq=0 2024-05-17T22:45:33.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.880+0000 7f13797fa700 10 client.25550 ms_handle_connect on v2:172.21.15.96:6836/4222574338 2024-05-17T22:45:33.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.880+0000 7f13797fa700 1 -- 172.21.15.12:0/2640648834 <== mds.1 v2:172.21.15.96:6836/4222574338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f1388021070 con 0x55998e50ad80 2024-05-17T22:45:33.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.880+0000 7f13797fa700 10 client.25550 handle_command_reply: tid=0 2024-05-17T22:45:33.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.880+0000 7f13a6aaab80 1 client.25550 shutdown 2024-05-17T22:45:33.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.880+0000 7f13a6aaab80 1 -- 172.21.15.12:0/2640648834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55998e4f1b00 msgr2=0x55998e4fafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:33.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.880+0000 7f13a6aaab80 1 --2- 172.21.15.12:0/2640648834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55998e4f1b00 0x55998e4fafe0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f138000c380 tx=0x7f138000c690 comp rx=0 tx=0).stop 2024-05-17T22:45:33.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.881+0000 7f13a6aaab80 1 -- 172.21.15.12:0/2640648834 shutdown_connections 2024-05-17T22:45:33.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.881+0000 7f13a6aaab80 1 --2- 172.21.15.12:0/2640648834 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55998e50ad80 0x55998e50d1e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:33.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.881+0000 7f13a6aaab80 1 --2- 172.21.15.12:0/2640648834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55998e4f7400 0x55998e4fb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:33.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.881+0000 7f13a6aaab80 1 --2- 172.21.15.12:0/2640648834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55998e3739f0 0x55998e4faaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:33.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.881+0000 7f13a6aaab80 1 --2- 172.21.15.12:0/2640648834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55998e4f1b00 0x55998e4fafe0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:33.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.881+0000 7f13a6aaab80 1 -- 172.21.15.12:0/2640648834 >> 172.21.15.12:0/2640648834 conn(0x55998e3d37d0 msgr2=0x55998e4ec160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:45:33.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.881+0000 7f13797fa700 0 client.25550 ms_handle_reset on v2:172.21.15.96:6836/4222574338 2024-05-17T22:45:33.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.881+0000 7f13a6aaab80 1 -- 172.21.15.12:0/2640648834 shutdown_connections 2024-05-17T22:45:33.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.881+0000 7f13a6aaab80 1 -- 172.21.15.12:0/2640648834 wait complete. 2024-05-17T22:45:33.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.881+0000 7f13a6aaab80 20 client.25550 trim_cache size 0 max 16384 2024-05-17T22:45:33.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.907+0000 7f13a6aaab80 1 Processor -- start 2024-05-17T22:45:33.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.907+0000 7f13a6aaab80 1 -- start start 2024-05-17T22:45:33.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.907+0000 7f13a6aaab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55998e3739f0 0x55998e4fb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:33.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.907+0000 7f13a6aaab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55998e4f1b00 0x55998e506ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:33.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.907+0000 7f13a6aaab80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55998e4f7400 0x55998e507220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:33.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.907+0000 7f13a6aaab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f13901aff10 con 0x55998e3739f0 2024-05-17T22:45:33.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.907+0000 7f13a6aaab80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f13901ad7a0 con 0x55998e4f7400 2024-05-17T22:45:33.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.907+0000 7f13a6aaab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f13901ab5f0 con 0x55998e4f1b00 2024-05-17T22:45:33.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.907+0000 7f1395e6b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55998e3739f0 0x55998e4fb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:33.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.907+0000 7f1395e6b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55998e3739f0 0x55998e4fb370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37516/0 (socket says 172.21.15.12:37516) 2024-05-17T22:45:33.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.907+0000 7f1395e6b700 1 -- 172.21.15.12:0/4238415978 learned_addr learned my addr 172.21.15.12:0/4238415978 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:45:33.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.908+0000 7f139566a700 1 --2- 172.21.15.12:0/4238415978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55998e4f1b00 0x55998e506ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:33.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.908+0000 7f139666c700 1 --2- 172.21.15.12:0/4238415978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55998e4f7400 0x55998e507220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:33.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.908+0000 7f1395e6b700 1 -- 172.21.15.12:0/4238415978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55998e4f1b00 msgr2=0x55998e506ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:33.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.908+0000 7f1395e6b700 1 --2- 172.21.15.12:0/4238415978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55998e4f1b00 0x55998e506ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:33.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.908+0000 7f1395e6b700 1 -- 172.21.15.12:0/4238415978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55998e4f7400 msgr2=0x55998e507220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:33.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.908+0000 7f1395e6b700 1 --2- 172.21.15.12:0/4238415978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55998e4f7400 0x55998e507220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:33.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.908+0000 7f1395e6b700 1 -- 172.21.15.12:0/4238415978 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55998e4fd550 con 0x55998e3739f0 2024-05-17T22:45:33.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.908+0000 7f139666c700 1 --2- 172.21.15.12:0/4238415978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55998e4f7400 0x55998e507220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:45:33.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.908+0000 7f1395e6b700 1 --2- 172.21.15.12:0/4238415978 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55998e3739f0 0x55998e4fb370 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f138c0a6cf0 tx=0x7f138c0ac040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=407e121327b23b3e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:45:33.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.908+0000 7f137b7fe700 1 -- 172.21.15.12:0/4238415978 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138c0aa180 con 0x55998e3739f0 2024-05-17T22:45:33.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.909+0000 7f137b7fe700 1 -- 172.21.15.12:0/4238415978 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f138c0a9a80 con 0x55998e3739f0 2024-05-17T22:45:33.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.909+0000 7f137b7fe700 1 -- 172.21.15.12:0/4238415978 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138c0a9ce0 con 0x55998e3739f0 2024-05-17T22:45:33.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.910+0000 7f13a6aaab80 1 -- 172.21.15.12:0/4238415978 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55998e3739f0 msgr2=0x55998e4fb370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:33.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.910+0000 7f13a6aaab80 1 --2- 172.21.15.12:0/4238415978 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55998e3739f0 0x55998e4fb370 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f138c0a6cf0 tx=0x7f138c0ac040 comp rx=0 tx=0).stop 2024-05-17T22:45:33.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.910+0000 7f13a6aaab80 1 -- 172.21.15.12:0/4238415978 shutdown_connections 2024-05-17T22:45:33.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.910+0000 7f13a6aaab80 1 --2- 172.21.15.12:0/4238415978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55998e4f7400 0x55998e507220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:33.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.910+0000 7f13a6aaab80 1 --2- 172.21.15.12:0/4238415978 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55998e3739f0 0x55998e4fb370 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:33.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.910+0000 7f13a6aaab80 1 --2- 172.21.15.12:0/4238415978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55998e4f1b00 0x55998e506ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:33.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.910+0000 7f13a6aaab80 1 -- 172.21.15.12:0/4238415978 >> 172.21.15.12:0/4238415978 conn(0x55998e3d37d0 msgr2=0x55998e4f9390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:45:33.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.910+0000 7f13a6aaab80 1 -- 172.21.15.12:0/4238415978 shutdown_connections 2024-05-17T22:45:33.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.910+0000 7f13a6aaab80 1 -- 172.21.15.12:0/4238415978 wait complete. 2024-05-17T22:45:33.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.910+0000 7f13a6aaab80 1 -- start start 2024-05-17T22:45:33.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.910+0000 7f13a6aaab80 1 Processor -- start 2024-05-17T22:45:33.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.911+0000 7f13a6aaab80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55998e3739f0 0x55998e503330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:33.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.911+0000 7f13a6aaab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55998e4f1b00 0x55998e511d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:33.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.911+0000 7f13a6aaab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55998e4f7400 0x55998e5122d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:33.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.911+0000 7f13a6aaab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f13901abc60 con 0x55998e4f1b00 2024-05-17T22:45:33.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.911+0000 7f13a6aaab80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f13901ad0d0 con 0x55998e3739f0 2024-05-17T22:45:33.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.911+0000 7f13a6aaab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f13901ac330 con 0x55998e4f7400 2024-05-17T22:45:33.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.911+0000 7f13a6aaab80 10 client.0 _ll_register_callbacks cb 0x55998e3a6b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:45:33.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.912+0000 7f1395e6b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55998e3739f0 0x55998e503330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:33.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.913+0000 7f1395e6b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55998e3739f0 0x55998e503330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:55104/0 (socket says 172.21.15.12:55104) 2024-05-17T22:45:33.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.913+0000 7f1395e6b700 1 -- 172.21.15.12:0/685448932 learned_addr learned my addr 172.21.15.12:0/685448932 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:45:33.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.913+0000 7f1395e6b700 1 -- 172.21.15.12:0/685448932 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55998e4f7400 msgr2=0x55998e5122d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T22:45:33.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.913+0000 7f1395e6b700 1 --2- 172.21.15.12:0/685448932 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55998e4f7400 0x55998e5122d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:33.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.913+0000 7f139566a700 1 --2- 172.21.15.12:0/685448932 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55998e4f1b00 0x55998e511d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:33.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.913+0000 7f1395e6b700 1 -- 172.21.15.12:0/685448932 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55998e4f1b00 msgr2=0x55998e511d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:33.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.913+0000 7f1395e6b700 1 --2- 172.21.15.12:0/685448932 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55998e4f1b00 0x55998e511d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:33.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.914+0000 7f1395e6b700 1 -- 172.21.15.12:0/685448932 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55998e512840 con 0x55998e3739f0 2024-05-17T22:45:33.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.914+0000 7f139566a700 1 --2- 172.21.15.12:0/685448932 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55998e4f1b00 0x55998e511d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:45:33.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.914+0000 7f1395e6b700 1 --2- 172.21.15.12:0/685448932 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55998e3739f0 0x55998e503330 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f138c010fd0 tx=0x7f138c0c4280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2ef9ccb568340a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:45:33.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.914+0000 7f13797fa700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:45:33.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.915+0000 7f13797fa700 1 -- 172.21.15.12:0/685448932 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138c0b6070 con 0x55998e3739f0 2024-05-17T22:45:33.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.915+0000 7f13797fa700 1 -- 172.21.15.12:0/685448932 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f138c0a9ef0 con 0x55998e3739f0 2024-05-17T22:45:33.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.915+0000 7f13a6aaab80 1 -- 172.21.15.12:0/685448932 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f138c0a9a80 con 0x55998e3739f0 2024-05-17T22:45:33.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.915+0000 7f13797fa700 1 -- 172.21.15.12:0/685448932 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f138c0be050 con 0x55998e3739f0 2024-05-17T22:45:33.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.915+0000 7f13a6aaab80 10 client.25553 fetch_fsmap learned FSMap version 37 2024-05-17T22:45:33.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.915+0000 7f13a6aaab80 1 -- 172.21.15.12:0/685448932 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55998e57da20 con 0x55998e3739f0 2024-05-17T22:45:33.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.915+0000 7f13797fa700 1 -- 172.21.15.12:0/685448932 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f138c0aa180 con 0x55998e3739f0 2024-05-17T22:45:33.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.915+0000 7f13797fa700 1 -- 172.21.15.12:0/685448932 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f138c0b9030 con 0x55998e3739f0 2024-05-17T22:45:33.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.916+0000 7f13a6aaab80 10 client.25553 fetch_fsmap finished waiting for FSMap version 37 2024-05-17T22:45:33.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.916+0000 7f13a6aaab80 10 client.25553 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-17T22:45:33.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.916+0000 7f13a6aaab80 20 client.25553 populate_metadata read hostname 'smithi012' 2024-05-17T22:45:33.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.916+0000 7f13a6aaab80 1 --2- 172.21.15.12:0/685448932 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55998e57dfc0 0x55998e5803e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:33.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.916+0000 7f13a6aaab80 4 client.25553 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:45:33.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.916+0000 7f13a6aaab80 1 -- 172.21.15.12:0/685448932 --> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f138c0a9a80 con 0x55998e57dfc0 2024-05-17T22:45:33.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.916+0000 7f139566a700 1 --2- 172.21.15.12:0/685448932 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55998e57dfc0 0x55998e5803e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:33.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.916+0000 7f13797fa700 10 client.25553 ms_handle_connect on v2:172.21.15.96:6836/4222574338 2024-05-17T22:45:33.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.916+0000 7f139566a700 1 --2- 172.21.15.12:0/685448932 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55998e57dfc0 0x55998e5803e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c6f3a223ae686553 server_cookie=fe1d394268c36b84 in_seq=0 out_seq=0 2024-05-17T22:45:33.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.917+0000 7f13797fa700 1 -- 172.21.15.12:0/685448932 <== mds.1 v2:172.21.15.96:6836/4222574338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f138c0a9a80 con 0x55998e57dfc0 2024-05-17T22:45:33.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.917+0000 7f13797fa700 10 client.25553 handle_command_reply: tid=0 2024-05-17T22:45:33.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.917+0000 7f13a6aaab80 1 client.25553 shutdown 2024-05-17T22:45:33.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.917+0000 7f13a6aaab80 1 -- 172.21.15.12:0/685448932 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55998e3739f0 msgr2=0x55998e503330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:33.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.917+0000 7f13a6aaab80 1 --2- 172.21.15.12:0/685448932 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55998e3739f0 0x55998e503330 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f138c010fd0 tx=0x7f138c0c4280 comp rx=0 tx=0).stop 2024-05-17T22:45:33.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.917+0000 7f13a6aaab80 1 -- 172.21.15.12:0/685448932 shutdown_connections 2024-05-17T22:45:33.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.917+0000 7f13a6aaab80 1 --2- 172.21.15.12:0/685448932 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55998e4f7400 0x55998e5122d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:33.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.917+0000 7f13a6aaab80 1 --2- 172.21.15.12:0/685448932 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55998e3739f0 0x55998e503330 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:33.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.917+0000 7f13a6aaab80 1 --2- 172.21.15.12:0/685448932 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55998e57dfc0 0x55998e5803e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:33.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.917+0000 7f13797fa700 0 client.25553 ms_handle_reset on v2:172.21.15.96:6836/4222574338 2024-05-17T22:45:33.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.917+0000 7f13a6aaab80 1 --2- 172.21.15.12:0/685448932 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55998e4f1b00 0x55998e511d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:33.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.917+0000 7f13a6aaab80 1 -- 172.21.15.12:0/685448932 >> 172.21.15.12:0/685448932 conn(0x55998e3d37d0 msgr2=0x55998e4f5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:45:33.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.917+0000 7f13a6aaab80 1 -- 172.21.15.12:0/685448932 shutdown_connections 2024-05-17T22:45:33.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.917+0000 7f13a6aaab80 1 -- 172.21.15.12:0/685448932 wait complete. 2024-05-17T22:45:33.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.917+0000 7f13a6aaab80 20 client.25553 trim_cache size 0 max 16384 2024-05-17T22:45:33.923 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:45:33.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.918+0000 7f13980cf700 1 -- 172.21.15.12:0/1258279275 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f137c0b2920 msgr2=0x7f137c0b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:33.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.918+0000 7f13980cf700 1 --2- 172.21.15.12:0/1258279275 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f137c0b2920 0x7f137c0b4e30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f138802e000 tx=0x7f1388021000 comp rx=0 tx=0).stop 2024-05-17T22:45:33.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.918+0000 7f13980cf700 1 -- 172.21.15.12:0/1258279275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1390102e00 msgr2=0x7f1390198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:33.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.919+0000 7f13980cf700 1 --2- 172.21.15.12:0/1258279275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1390102e00 0x7f1390198520 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f138c00e9a0 tx=0x7f138c00ed60 comp rx=0 tx=0).stop 2024-05-17T22:45:33.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.919+0000 7f13980cf700 1 -- 172.21.15.12:0/1258279275 shutdown_connections 2024-05-17T22:45:33.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.919+0000 7f13980cf700 1 --2- 172.21.15.12:0/1258279275 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f137c0b2920 0x7f137c0b4e30 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:33.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.919+0000 7f13980cf700 1 --2- 172.21.15.12:0/1258279275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1390108080 0x7f139019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:33.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.919+0000 7f13980cf700 1 --2- 172.21.15.12:0/1258279275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1390102e00 0x7f1390198520 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:33.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.919+0000 7f13980cf700 1 --2- 172.21.15.12:0/1258279275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1390107730 0x7f1390198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:33.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.919+0000 7f13980cf700 1 -- 172.21.15.12:0/1258279275 >> 172.21.15.12:0/1258279275 conn(0x7f1390079bd0 msgr2=0x7f1390105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:45:33.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.919+0000 7f13980cf700 1 -- 172.21.15.12:0/1258279275 shutdown_connections 2024-05-17T22:45:33.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:33.919+0000 7f13980cf700 1 -- 172.21.15.12:0/1258279275 wait complete. 2024-05-17T22:45:34.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:33 smithi096 ceph-mon[36065]: pgmap v628: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 629 op/s 2024-05-17T22:45:34.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:33 smithi012 ceph-mon[31632]: pgmap v628: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 629 op/s 2024-05-17T22:45:34.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:33 smithi151 ceph-mon[36101]: pgmap v628: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 629 op/s 2024-05-17T22:45:34.858 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-17T22:45:35.358 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:45:36.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:35 smithi012 ceph-mon[31632]: pgmap v629: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 623 op/s 2024-05-17T22:45:36.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:35 smithi151 ceph-mon[36101]: pgmap v629: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 623 op/s 2024-05-17T22:45:36.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:35 smithi096 ceph-mon[36065]: pgmap v629: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 623 op/s 2024-05-17T22:45:36.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 375.0 s, 638.2 tps, lat 165.109 ms stddev 233.377 2024-05-17T22:45:37.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.236+0000 7f555740d700 1 -- 172.21.15.12:0/360974827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5548095030 msgr2=0x7f5548095440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:37.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.236+0000 7f555740d700 1 --2- 172.21.15.12:0/360974827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5548095030 0x7f5548095440 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f554c009990 tx=0x7f554c009ca0 comp rx=0 tx=0).stop 2024-05-17T22:45:37.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.237+0000 7f555740d700 1 -- 172.21.15.12:0/360974827 shutdown_connections 2024-05-17T22:45:37.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.237+0000 7f555740d700 1 --2- 172.21.15.12:0/360974827 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f554809a2b0 0x7f55480a1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:37.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.237+0000 7f555740d700 1 --2- 172.21.15.12:0/360974827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5548095030 0x7f5548095440 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:37.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.237+0000 7f555740d700 1 --2- 172.21.15.12:0/360974827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5548099960 0x7f5548099d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:37.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.237+0000 7f555740d700 1 -- 172.21.15.12:0/360974827 >> 172.21.15.12:0/360974827 conn(0x7f554800ab00 msgr2=0x7f554800af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:45:37.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.238+0000 7f555740d700 1 -- 172.21.15.12:0/360974827 shutdown_connections 2024-05-17T22:45:37.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.238+0000 7f555740d700 1 -- 172.21.15.12:0/360974827 wait complete. 2024-05-17T22:45:37.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.239+0000 7f555740d700 1 Processor -- start 2024-05-17T22:45:37.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.240+0000 7f555740d700 1 -- start start 2024-05-17T22:45:37.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.240+0000 7f555740d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5548095030 0x7f554812a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:37.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.240+0000 7f555740d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5548099960 0x7f554812a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:37.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.240+0000 7f555740d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f554809a2b0 0x7f554812af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:37.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.240+0000 7f555740d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f55480a3ec0 con 0x7f5548099960 2024-05-17T22:45:37.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.240+0000 7f555740d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f55480a3d40 con 0x7f554809a2b0 2024-05-17T22:45:37.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.240+0000 7f555740d700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f55480a4040 con 0x7f5548095030 2024-05-17T22:45:37.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.244+0000 7f55559aa700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f554809a2b0 0x7f554812af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:37.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.244+0000 7f55559aa700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f554809a2b0 0x7f554812af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:55132/0 (socket says 172.21.15.12:55132) 2024-05-17T22:45:37.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.244+0000 7f55559aa700 1 -- 172.21.15.12:0/3352695670 learned_addr learned my addr 172.21.15.12:0/3352695670 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:45:37.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.244+0000 7f55551a9700 1 --2- 172.21.15.12:0/3352695670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5548095030 0x7f554812a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:37.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.245+0000 7f55549a8700 1 --2- 172.21.15.12:0/3352695670 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5548099960 0x7f554812a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:37.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.246+0000 7f55551a9700 1 -- 172.21.15.12:0/3352695670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f554809a2b0 msgr2=0x7f554812af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:37.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.246+0000 7f55551a9700 1 --2- 172.21.15.12:0/3352695670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f554809a2b0 0x7f554812af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:37.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.246+0000 7f55551a9700 1 -- 172.21.15.12:0/3352695670 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5548099960 msgr2=0x7f554812a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:37.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.246+0000 7f55551a9700 1 --2- 172.21.15.12:0/3352695670 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5548099960 0x7f554812a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:37.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.246+0000 7f55551a9700 1 -- 172.21.15.12:0/3352695670 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f554812e760 con 0x7f5548095030 2024-05-17T22:45:37.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.253+0000 7f55551a9700 1 --2- 172.21.15.12:0/3352695670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5548095030 0x7f554812a4a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f554c0098e0 tx=0x7f554c0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:45:37.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.256+0000 7f55467fc700 1 -- 172.21.15.12:0/3352695670 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554c00fbd0 con 0x7f5548095030 2024-05-17T22:45:37.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.256+0000 7f55467fc700 1 -- 172.21.15.12:0/3352695670 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f554c00fd70 con 0x7f5548095030 2024-05-17T22:45:37.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.256+0000 7f55467fc700 1 -- 172.21.15.12:0/3352695670 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554c022470 con 0x7f5548095030 2024-05-17T22:45:37.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.260+0000 7f555740d700 1 -- 172.21.15.12:0/3352695670 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f554812e9f0 con 0x7f5548095030 2024-05-17T22:45:37.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.260+0000 7f555740d700 1 -- 172.21.15.12:0/3352695670 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f554812edf0 con 0x7f5548095030 2024-05-17T22:45:37.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.277+0000 7f5565de8b80 1 Processor -- start 2024-05-17T22:45:37.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.277+0000 7f5565de8b80 1 -- start start 2024-05-17T22:45:37.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.277+0000 7f5565de8b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28656e9f0 0x55b2866f2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:37.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.278+0000 7f5565de8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2866f2cb0 0x55b2866ecc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:37.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.278+0000 7f55559aa700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28656e9f0 0x55b2866f2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:37.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.278+0000 7f55559aa700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28656e9f0 0x55b2866f2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:55094/0 (socket says 172.21.15.12:55094) 2024-05-17T22:45:37.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.278+0000 7f5565de8b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2866ed170 0x55b2866ed580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:37.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.278+0000 7f55559aa700 1 -- 172.21.15.12:0/798380643 learned_addr learned my addr 172.21.15.12:0/798380643 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:45:37.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.278+0000 7f5565de8b80 1 -- 172.21.15.12:0/798380643 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b28661b290 con 0x55b2866f2cb0 2024-05-17T22:45:37.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.278+0000 7f5565de8b80 1 -- 172.21.15.12:0/798380643 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b28660afd0 con 0x55b2866ed170 2024-05-17T22:45:37.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.278+0000 7f5565de8b80 1 -- 172.21.15.12:0/798380643 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b28660ee70 con 0x55b28656e9f0 2024-05-17T22:45:37.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.282+0000 7f55559aa700 1 -- 172.21.15.12:0/798380643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2866ed170 msgr2=0x55b2866ed580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T22:45:37.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.282+0000 7f55559aa700 1 --2- 172.21.15.12:0/798380643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2866ed170 0x55b2866ed580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:37.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.282+0000 7f55559aa700 1 -- 172.21.15.12:0/798380643 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2866f2cb0 msgr2=0x55b2866ecc30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T22:45:37.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.282+0000 7f55559aa700 1 --2- 172.21.15.12:0/798380643 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2866f2cb0 0x55b2866ecc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:37.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.282+0000 7f55559aa700 1 -- 172.21.15.12:0/798380643 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2865b4590 con 0x55b28656e9f0 2024-05-17T22:45:37.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.282+0000 7f55467fc700 1 -- 172.21.15.12:0/3352695670 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f554c009d10 con 0x7f5548095030 2024-05-17T22:45:37.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.282+0000 7f55559aa700 1 --2- 172.21.15.12:0/798380643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28656e9f0 0x55b2866f2770 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5550051e50 tx=0x7f555006d1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81b984f5eeac6012 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:45:37.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.283+0000 7f55467fc700 1 --2- 172.21.15.12:0/3352695670 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f553c0b2a50 0x7f553c0b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:37.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.284+0000 7f55551a9700 1 --2- 172.21.15.12:0/3352695670 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f553c0b2a50 0x7f553c0b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:37.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.284+0000 7f55467fc700 1 -- 172.21.15.12:0/3352695670 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f554c0e3350 con 0x7f5548095030 2024-05-17T22:45:37.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.284+0000 7f55551a9700 1 --2- 172.21.15.12:0/3352695670 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f553c0b2a50 0x7f553c0b4f60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f554c0a7880 tx=0x7f554c06d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:45:37.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.284+0000 7f553b7fe700 1 -- 172.21.15.12:0/798380643 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5550075da0 con 0x55b28656e9f0 2024-05-17T22:45:37.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.284+0000 7f553b7fe700 1 -- 172.21.15.12:0/798380643 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55500818f0 con 0x55b28656e9f0 2024-05-17T22:45:37.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.286+0000 7f5565de8b80 1 -- 172.21.15.12:0/798380643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28656e9f0 msgr2=0x55b2866f2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:37.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.286+0000 7f5565de8b80 1 --2- 172.21.15.12:0/798380643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28656e9f0 0x55b2866f2770 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5550051e50 tx=0x7f555006d1e0 comp rx=0 tx=0).stop 2024-05-17T22:45:37.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.291+0000 7f5565de8b80 1 -- 172.21.15.12:0/798380643 shutdown_connections 2024-05-17T22:45:37.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.291+0000 7f5565de8b80 1 --2- 172.21.15.12:0/798380643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2866ed170 0x55b2866ed580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:37.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.291+0000 7f5565de8b80 1 --2- 172.21.15.12:0/798380643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28656e9f0 0x55b2866f2770 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:37.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.291+0000 7f5565de8b80 1 --2- 172.21.15.12:0/798380643 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2866f2cb0 0x55b2866ecc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:37.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.291+0000 7f5565de8b80 1 -- 172.21.15.12:0/798380643 >> 172.21.15.12:0/798380643 conn(0x55b2865b2f00 msgr2=0x55b2866e7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:45:37.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.291+0000 7f5565de8b80 1 -- 172.21.15.12:0/798380643 shutdown_connections 2024-05-17T22:45:37.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.292+0000 7f5565de8b80 1 -- 172.21.15.12:0/798380643 wait complete. 2024-05-17T22:45:37.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.292+0000 7f5565de8b80 1 -- start start 2024-05-17T22:45:37.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.292+0000 7f5565de8b80 1 Processor -- start 2024-05-17T22:45:37.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.294+0000 7f5565de8b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28656e9f0 0x55b2866f61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:37.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.294+0000 7f5565de8b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2866ed170 0x55b2866f6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:37.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.294+0000 7f5565de8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2866f2cb0 0x55b2866f6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:37.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.294+0000 7f5565de8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b286512150 con 0x55b2866f2cb0 2024-05-17T22:45:37.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.294+0000 7f5565de8b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b2865abe80 con 0x55b2866ed170 2024-05-17T22:45:37.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.294+0000 7f5565de8b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b2866f95a0 con 0x55b28656e9f0 2024-05-17T22:45:37.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.294+0000 7f5565de8b80 10 client.0 _ll_register_callbacks cb 0x55b2865a5f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:45:37.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.294+0000 7f55549a8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2866ed170 0x55b2866f6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:37.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.294+0000 7f55549a8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2866ed170 0x55b2866f6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:55146/0 (socket says 172.21.15.12:55146) 2024-05-17T22:45:37.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.294+0000 7f55549a8700 1 -- 172.21.15.12:0/403393855 learned_addr learned my addr 172.21.15.12:0/403393855 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:45:37.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.298+0000 7f55559aa700 1 --2- 172.21.15.12:0/403393855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2866f2cb0 0x55b2866f6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:37.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.298+0000 7f55559aa700 1 --2- 172.21.15.12:0/403393855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28656e9f0 0x55b2866f61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:37.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.299+0000 7f55549a8700 1 -- 172.21.15.12:0/403393855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28656e9f0 msgr2=0x55b2866f61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:37.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.299+0000 7f55549a8700 1 --2- 172.21.15.12:0/403393855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28656e9f0 0x55b2866f61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:37.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.299+0000 7f55549a8700 1 -- 172.21.15.12:0/403393855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2866f2cb0 msgr2=0x55b2866f6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:37.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.299+0000 7f55549a8700 1 --2- 172.21.15.12:0/403393855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2866f2cb0 0x55b2866f6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:37.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.299+0000 7f55549a8700 1 -- 172.21.15.12:0/403393855 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b28655b480 con 0x55b2866ed170 2024-05-17T22:45:37.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.300+0000 7f55549a8700 1 --2- 172.21.15.12:0/403393855 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2866ed170 0x55b2866f6710 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f554000a4d0 tx=0x7f554000a7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb75a30e42750fb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:45:37.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.302+0000 7f5538ff9700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:45:37.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.302+0000 7f5538ff9700 1 -- 172.21.15.12:0/403393855 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5540014770 con 0x55b2866ed170 2024-05-17T22:45:37.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.302+0000 7f5538ff9700 1 -- 172.21.15.12:0/403393855 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f5540014910 con 0x55b2866ed170 2024-05-17T22:45:37.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.302+0000 7f5565de8b80 1 -- 172.21.15.12:0/403393855 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55500818f0 con 0x55b2866ed170 2024-05-17T22:45:37.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.305+0000 7f5538ff9700 1 -- 172.21.15.12:0/403393855 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5540023050 con 0x55b2866ed170 2024-05-17T22:45:37.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.307+0000 7f5565de8b80 10 client.25562 fetch_fsmap learned FSMap version 37 2024-05-17T22:45:37.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.307+0000 7f5565de8b80 1 -- 172.21.15.12:0/403393855 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b286714ef0 con 0x55b2866ed170 2024-05-17T22:45:37.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.309+0000 7f5538ff9700 1 -- 172.21.15.12:0/403393855 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55500818f0 con 0x55b2866ed170 2024-05-17T22:45:37.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.309+0000 7f5538ff9700 1 -- 172.21.15.12:0/403393855 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f5540015b00 con 0x55b2866ed170 2024-05-17T22:45:37.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.309+0000 7f5565de8b80 10 client.25562 fetch_fsmap finished waiting for FSMap version 37 2024-05-17T22:45:37.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.309+0000 7f5565de8b80 10 client.25562 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:45:37.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.309+0000 7f5565de8b80 20 client.25562 populate_metadata read hostname 'smithi012' 2024-05-17T22:45:37.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.309+0000 7f5565de8b80 1 --2- 172.21.15.12:0/403393855 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55b286717580 0x55b2867199a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:37.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.309+0000 7f5565de8b80 4 client.25562 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:45:37.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.309+0000 7f5565de8b80 1 -- 172.21.15.12:0/403393855 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b28661d1e0 con 0x55b286717580 2024-05-17T22:45:37.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.311+0000 7f55559aa700 1 --2- 172.21.15.12:0/403393855 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55b286717580 0x55b2867199a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:37.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.312+0000 7f55559aa700 1 --2- 172.21.15.12:0/403393855 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55b286717580 0x55b2867199a0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb9d4d2e99fadec2 server_cookie=b0349e995c6126ce in_seq=0 out_seq=0 2024-05-17T22:45:37.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.312+0000 7f5538ff9700 10 client.25562 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:45:37.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.313+0000 7f5538ff9700 1 -- 172.21.15.12:0/403393855 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b28661d1e0 con 0x55b286717580 2024-05-17T22:45:37.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.313+0000 7f5538ff9700 10 client.25562 handle_command_reply: tid=0 2024-05-17T22:45:37.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.313+0000 7f5565de8b80 1 client.25562 shutdown 2024-05-17T22:45:37.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.317+0000 7f5565de8b80 1 -- 172.21.15.12:0/403393855 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2866ed170 msgr2=0x55b2866f6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:37.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.317+0000 7f5565de8b80 1 --2- 172.21.15.12:0/403393855 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2866ed170 0x55b2866f6710 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f554000a4d0 tx=0x7f554000a7e0 comp rx=0 tx=0).stop 2024-05-17T22:45:37.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.317+0000 7f5565de8b80 1 -- 172.21.15.12:0/403393855 shutdown_connections 2024-05-17T22:45:37.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.317+0000 7f5565de8b80 1 --2- 172.21.15.12:0/403393855 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55b286717580 0x55b2867199a0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:37.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.317+0000 7f5565de8b80 1 --2- 172.21.15.12:0/403393855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2866f2cb0 0x55b2866f6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:37.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.317+0000 7f5565de8b80 1 --2- 172.21.15.12:0/403393855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28656e9f0 0x55b2866f61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:37.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.317+0000 7f5565de8b80 1 --2- 172.21.15.12:0/403393855 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2866ed170 0x55b2866f6710 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:37.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.317+0000 7f5565de8b80 1 -- 172.21.15.12:0/403393855 >> 172.21.15.12:0/403393855 conn(0x55b2865b2f00 msgr2=0x55b2866e7110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:45:37.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.321+0000 7f5538ff9700 0 client.25562 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:45:37.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.321+0000 7f5565de8b80 1 -- 172.21.15.12:0/403393855 shutdown_connections 2024-05-17T22:45:37.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.321+0000 7f5565de8b80 1 -- 172.21.15.12:0/403393855 wait complete. 2024-05-17T22:45:37.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.321+0000 7f5565de8b80 20 client.25562 trim_cache size 0 max 16384 2024-05-17T22:45:37.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.353+0000 7f5565de8b80 1 Processor -- start 2024-05-17T22:45:37.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.353+0000 7f5565de8b80 1 -- start start 2024-05-17T22:45:37.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.353+0000 7f5565de8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b28656e9f0 0x55b2866ecc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:37.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.353+0000 7f5565de8b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2866ed170 0x55b2866f5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:37.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.353+0000 7f5565de8b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2866f2cb0 0x55b2866ed580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:37.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.353+0000 7f5565de8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5548153680 con 0x55b28656e9f0 2024-05-17T22:45:37.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.353+0000 7f5565de8b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5548150f10 con 0x55b2866ed170 2024-05-17T22:45:37.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.353+0000 7f5565de8b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5548135cf0 con 0x55b2866f2cb0 2024-05-17T22:45:37.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.353+0000 7f55549a8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2866ed170 0x55b2866f5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:37.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.353+0000 7f55559aa700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2866f2cb0 0x55b2866ed580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:37.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.353+0000 7f55559aa700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2866f2cb0 0x55b2866ed580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:55112/0 (socket says 172.21.15.12:55112) 2024-05-17T22:45:37.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.353+0000 7f55549a8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2866ed170 0x55b2866f5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:55156/0 (socket says 172.21.15.12:55156) 2024-05-17T22:45:37.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.353+0000 7f55559aa700 1 -- 172.21.15.12:0/1599675 learned_addr learned my addr 172.21.15.12:0/1599675 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:45:37.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.354+0000 7f55559aa700 1 --2- 172.21.15.12:0/1599675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b28656e9f0 0x55b2866ecc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:37.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.354+0000 7f55549a8700 1 -- 172.21.15.12:0/1599675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2866f2cb0 msgr2=0x55b2866ed580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:37.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.354+0000 7f55549a8700 1 --2- 172.21.15.12:0/1599675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2866f2cb0 0x55b2866ed580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:37.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.354+0000 7f55549a8700 1 -- 172.21.15.12:0/1599675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b28656e9f0 msgr2=0x55b2866ecc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:37.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.354+0000 7f55549a8700 1 --2- 172.21.15.12:0/1599675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b28656e9f0 0x55b2866ecc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:37.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.354+0000 7f55549a8700 1 -- 172.21.15.12:0/1599675 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2866e3af0 con 0x55b2866ed170 2024-05-17T22:45:37.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.354+0000 7f55549a8700 1 --2- 172.21.15.12:0/1599675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2866ed170 0x55b2866f5110 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f554000d010 tx=0x7f5540005900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ebd593e2768e3e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:45:37.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.355+0000 7f553affd700 1 -- 172.21.15.12:0/1599675 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554001e6f0 con 0x55b2866ed170 2024-05-17T22:45:37.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.355+0000 7f553affd700 1 -- 172.21.15.12:0/1599675 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f554002b070 con 0x55b2866ed170 2024-05-17T22:45:37.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.355+0000 7f553affd700 1 -- 172.21.15.12:0/1599675 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5540003900 con 0x55b2866ed170 2024-05-17T22:45:37.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.357+0000 7f5565de8b80 1 -- 172.21.15.12:0/1599675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2866ed170 msgr2=0x55b2866f5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:37.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.357+0000 7f5565de8b80 1 --2- 172.21.15.12:0/1599675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2866ed170 0x55b2866f5110 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f554000d010 tx=0x7f5540005900 comp rx=0 tx=0).stop 2024-05-17T22:45:37.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.357+0000 7f5565de8b80 1 -- 172.21.15.12:0/1599675 shutdown_connections 2024-05-17T22:45:37.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.357+0000 7f5565de8b80 1 --2- 172.21.15.12:0/1599675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2866f2cb0 0x55b2866ed580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:37.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.358+0000 7f5565de8b80 1 --2- 172.21.15.12:0/1599675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b28656e9f0 0x55b2866ecc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:37.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.358+0000 7f5565de8b80 1 --2- 172.21.15.12:0/1599675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2866ed170 0x55b2866f5110 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:37.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.358+0000 7f5565de8b80 1 -- 172.21.15.12:0/1599675 >> 172.21.15.12:0/1599675 conn(0x55b2865b2f00 msgr2=0x55b2866f1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:45:37.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.358+0000 7f5565de8b80 1 -- 172.21.15.12:0/1599675 shutdown_connections 2024-05-17T22:45:37.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.358+0000 7f5565de8b80 1 -- 172.21.15.12:0/1599675 wait complete. 2024-05-17T22:45:37.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.359+0000 7f5565de8b80 1 -- start start 2024-05-17T22:45:37.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.360+0000 7f5565de8b80 1 Processor -- start 2024-05-17T22:45:37.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.360+0000 7f5565de8b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28656e9f0 0x55b28671f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:37.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.361+0000 7f5565de8b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2866ed170 0x55b2866fe720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:37.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.361+0000 7f55559aa700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28656e9f0 0x55b28671f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:37.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.361+0000 7f55559aa700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28656e9f0 0x55b28671f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:55114/0 (socket says 172.21.15.12:55114) 2024-05-17T22:45:37.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.361+0000 7f55559aa700 1 -- 172.21.15.12:0/2503441378 learned_addr learned my addr 172.21.15.12:0/2503441378 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:45:37.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.361+0000 7f55549a8700 1 --2- 172.21.15.12:0/2503441378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2866ed170 0x55b2866fe720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:37.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.361+0000 7f5565de8b80 1 --2- 172.21.15.12:0/2503441378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2866f2cb0 0x55b2866feca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:37.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.361+0000 7f5565de8b80 1 -- 172.21.15.12:0/2503441378 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5548136360 con 0x55b2866f2cb0 2024-05-17T22:45:37.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.361+0000 7f5565de8b80 1 -- 172.21.15.12:0/2503441378 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5548150840 con 0x55b2866ed170 2024-05-17T22:45:37.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.361+0000 7f5565de8b80 1 -- 172.21.15.12:0/2503441378 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5548136a30 con 0x55b28656e9f0 2024-05-17T22:45:37.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.362+0000 7f5565de8b80 10 client.0 _ll_register_callbacks cb 0x55b2865a5f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:45:37.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.362+0000 7f55549a8700 1 -- 172.21.15.12:0/2503441378 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28656e9f0 msgr2=0x55b28671f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:37.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.362+0000 7f55549a8700 1 --2- 172.21.15.12:0/2503441378 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28656e9f0 0x55b28671f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:37.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.362+0000 7f55549a8700 1 -- 172.21.15.12:0/2503441378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2866f2cb0 msgr2=0x55b2866feca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:37.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.362+0000 7f55549a8700 1 --2- 172.21.15.12:0/2503441378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2866f2cb0 0x55b2866feca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:37.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.362+0000 7f55549a8700 1 -- 172.21.15.12:0/2503441378 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b28671e220 con 0x55b2866ed170 2024-05-17T22:45:37.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.362+0000 7f5565de8b80 1 -- 172.21.15.12:0/2503441378 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f554002b070 con 0x55b2866ed170 2024-05-17T22:45:37.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.362+0000 7f55549a8700 1 --2- 172.21.15.12:0/2503441378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2866ed170 0x55b2866fe720 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x55b2866f8b10 tx=0x7f554001e6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2b99286ffa22421 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:45:37.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.362+0000 7f5538ff9700 10 client.25568 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:45:37.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.363+0000 7f5538ff9700 1 -- 172.21.15.12:0/2503441378 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5540005270 con 0x55b2866ed170 2024-05-17T22:45:37.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.363+0000 7f5538ff9700 1 -- 172.21.15.12:0/2503441378 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f5540005410 con 0x55b2866ed170 2024-05-17T22:45:37.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.363+0000 7f55559aa700 1 --2- 172.21.15.12:0/2503441378 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28656e9f0 0x55b28671f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:45:37.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.363+0000 7f5538ff9700 1 -- 172.21.15.12:0/2503441378 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f554001eb00 con 0x55b2866ed170 2024-05-17T22:45:37.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.363+0000 7f5565de8b80 10 client.25568 fetch_fsmap learned FSMap version 37 2024-05-17T22:45:37.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.363+0000 7f5565de8b80 1 -- 172.21.15.12:0/2503441378 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({fsmap=37}) -- 0x55b286780f30 con 0x55b2866ed170 2024-05-17T22:45:37.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.364+0000 7f5538ff9700 1 -- 172.21.15.12:0/2503441378 <== mon.2 v2:172.21.15.96:3300/0 4 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f5540036020 con 0x55b2866ed170 2024-05-17T22:45:37.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.364+0000 7f5565de8b80 10 client.25568 fetch_fsmap finished waiting for FSMap version 37 2024-05-17T22:45:37.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.364+0000 7f5565de8b80 10 client.25568 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:45:37.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.364+0000 7f5565de8b80 20 client.25568 populate_metadata read hostname 'smithi012' 2024-05-17T22:45:37.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.364+0000 7f5565de8b80 1 --2- 172.21.15.12:0/2503441378 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55b286781380 0x55b2867837a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:37.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.364+0000 7f5565de8b80 4 client.25568 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:45:37.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.364+0000 7f5565de8b80 1 -- 172.21.15.12:0/2503441378 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b2866169c0 con 0x55b286781380 2024-05-17T22:45:37.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.364+0000 7f55559aa700 1 --2- 172.21.15.12:0/2503441378 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55b286781380 0x55b2867837a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:37.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.365+0000 7f55559aa700 1 --2- 172.21.15.12:0/2503441378 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55b286781380 0x55b2867837a0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c108656d18721a7 server_cookie=6ee715a14edf9f30 in_seq=0 out_seq=0 2024-05-17T22:45:37.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.365+0000 7f5538ff9700 10 client.25568 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:45:37.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.366+0000 7f5538ff9700 1 -- 172.21.15.12:0/2503441378 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b2866169c0 con 0x55b286781380 2024-05-17T22:45:37.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.366+0000 7f5538ff9700 10 client.25568 handle_command_reply: tid=0 2024-05-17T22:45:37.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.366+0000 7f5565de8b80 1 client.25568 shutdown 2024-05-17T22:45:37.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.366+0000 7f5565de8b80 1 -- 172.21.15.12:0/2503441378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2866ed170 msgr2=0x55b2866fe720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:37.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.366+0000 7f5565de8b80 1 --2- 172.21.15.12:0/2503441378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2866ed170 0x55b2866fe720 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x55b2866f8b10 tx=0x7f554001e6d0 comp rx=0 tx=0).stop 2024-05-17T22:45:37.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.366+0000 7f5565de8b80 1 -- 172.21.15.12:0/2503441378 shutdown_connections 2024-05-17T22:45:37.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.366+0000 7f5565de8b80 1 --2- 172.21.15.12:0/2503441378 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55b286781380 0x55b2867837a0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:37.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.366+0000 7f5565de8b80 1 --2- 172.21.15.12:0/2503441378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2866f2cb0 0x55b2866feca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:37.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.366+0000 7f5538ff9700 0 client.25568 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:45:37.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.367+0000 7f5565de8b80 1 --2- 172.21.15.12:0/2503441378 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28656e9f0 0x55b28671f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:37.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.367+0000 7f5565de8b80 1 --2- 172.21.15.12:0/2503441378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2866ed170 0x55b2866fe720 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:37.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.367+0000 7f5565de8b80 1 -- 172.21.15.12:0/2503441378 >> 172.21.15.12:0/2503441378 conn(0x55b2865b2f00 msgr2=0x55b2866f79c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:45:37.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.367+0000 7f5565de8b80 1 -- 172.21.15.12:0/2503441378 shutdown_connections 2024-05-17T22:45:37.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.368+0000 7f5565de8b80 1 -- 172.21.15.12:0/2503441378 wait complete. 2024-05-17T22:45:37.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.368+0000 7f5565de8b80 20 client.25568 trim_cache size 0 max 16384 2024-05-17T22:45:37.373 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:45:37.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.369+0000 7f555740d700 1 -- 172.21.15.12:0/3352695670 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f553c0b2a50 msgr2=0x7f553c0b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:37.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.369+0000 7f555740d700 1 --2- 172.21.15.12:0/3352695670 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f553c0b2a50 0x7f553c0b4f60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f554c0a7880 tx=0x7f554c06d040 comp rx=0 tx=0).stop 2024-05-17T22:45:37.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.369+0000 7f555740d700 1 -- 172.21.15.12:0/3352695670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5548095030 msgr2=0x7f554812a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:37.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.369+0000 7f555740d700 1 --2- 172.21.15.12:0/3352695670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5548095030 0x7f554812a4a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f554c0098e0 tx=0x7f554c0047b0 comp rx=0 tx=0).stop 2024-05-17T22:45:37.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.369+0000 7f555740d700 1 -- 172.21.15.12:0/3352695670 shutdown_connections 2024-05-17T22:45:37.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.369+0000 7f555740d700 1 --2- 172.21.15.12:0/3352695670 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f553c0b2a50 0x7f553c0b4f60 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:37.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.369+0000 7f555740d700 1 --2- 172.21.15.12:0/3352695670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f554809a2b0 0x7f554812af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:37.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.369+0000 7f555740d700 1 --2- 172.21.15.12:0/3352695670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5548095030 0x7f554812a4a0 secure :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f554c0098e0 tx=0x7f554c0047b0 comp rx=0 tx=0).stop 2024-05-17T22:45:37.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.369+0000 7f555740d700 1 --2- 172.21.15.12:0/3352695670 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5548099960 0x7f554812a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:37.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.369+0000 7f555740d700 1 -- 172.21.15.12:0/3352695670 >> 172.21.15.12:0/3352695670 conn(0x7f554800ab00 msgr2=0x7f554809bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:45:37.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.369+0000 7f555740d700 1 -- 172.21.15.12:0/3352695670 shutdown_connections 2024-05-17T22:45:37.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:37.369+0000 7f555740d700 1 -- 172.21.15.12:0/3352695670 wait complete. 2024-05-17T22:45:38.010 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:37 smithi012 ceph-mon[31632]: pgmap v630: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.3 MiB/s wr, 527 op/s 2024-05-17T22:45:38.100 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:37 smithi096 ceph-mon[36065]: pgmap v630: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.3 MiB/s wr, 527 op/s 2024-05-17T22:45:38.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:37 smithi151 ceph-mon[36101]: pgmap v630: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.3 MiB/s wr, 527 op/s 2024-05-17T22:45:39.055 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-17T22:45:39.056 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-17T22:45:39.573 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:45:40.008 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:39 smithi012 ceph-mon[31632]: pgmap v631: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.5 MiB/s wr, 628 op/s 2024-05-17T22:45:40.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:39 smithi151 ceph-mon[36101]: pgmap v631: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.5 MiB/s wr, 628 op/s 2024-05-17T22:45:40.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:39 smithi096 ceph-mon[36065]: pgmap v631: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.5 MiB/s wr, 628 op/s 2024-05-17T22:45:41.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 380.0 s, 615.2 tps, lat 163.371 ms stddev 208.845 2024-05-17T22:45:42.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:41 smithi012 ceph-mon[31632]: pgmap v632: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 MiB/s wr, 458 op/s 2024-05-17T22:45:42.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:41 smithi151 ceph-mon[36101]: pgmap v632: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 MiB/s wr, 458 op/s 2024-05-17T22:45:42.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:41 smithi096 ceph-mon[36065]: pgmap v632: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 MiB/s wr, 458 op/s 2024-05-17T22:45:43.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.069+0000 7f73b10f2700 1 -- 172.21.15.12:0/3043524392 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73ac11d550 msgr2=0x7f73ac11f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:43.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.069+0000 7f73b10f2700 1 --2- 172.21.15.12:0/3043524392 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73ac11d550 0x7f73ac11f970 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f739c01c0d0 tx=0x7f739c01c3e0 comp rx=0 tx=0).stop 2024-05-17T22:45:43.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.070+0000 7f73b10f2700 1 -- 172.21.15.12:0/3043524392 shutdown_connections 2024-05-17T22:45:43.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.070+0000 7f73b10f2700 1 --2- 172.21.15.12:0/3043524392 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73ac11d550 0x7f73ac11f970 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:43.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.070+0000 7f73b10f2700 1 --2- 172.21.15.12:0/3043524392 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73ac10a830 0x7f73ac10ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:43.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.070+0000 7f73b10f2700 1 --2- 172.21.15.12:0/3043524392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73ac10b180 0x7f73ac11d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:43.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.070+0000 7f73b10f2700 1 -- 172.21.15.12:0/3043524392 >> 172.21.15.12:0/3043524392 conn(0x7f73ac06efc0 msgr2=0x7f73ac06f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:45:43.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.072+0000 7f73b10f2700 1 -- 172.21.15.12:0/3043524392 shutdown_connections 2024-05-17T22:45:43.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.075+0000 7f73b10f2700 1 -- 172.21.15.12:0/3043524392 wait complete. 2024-05-17T22:45:43.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.076+0000 7f73b10f2700 1 Processor -- start 2024-05-17T22:45:43.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.080+0000 7f73b10f2700 1 -- start start 2024-05-17T22:45:43.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.080+0000 7f73b10f2700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73ac10a830 0x7f73ac11ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:43.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.080+0000 7f73b10f2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73ac10b180 0x7f73ac118f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:43.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.080+0000 7f73b10f2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73ac11d550 0x7f73ac1194a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:43.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.080+0000 7f73b10f2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac114fc0 con 0x7f73ac11d550 2024-05-17T22:45:43.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.080+0000 7f73b10f2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac114e40 con 0x7f73ac10b180 2024-05-17T22:45:43.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.080+0000 7f73b10f2700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac115140 con 0x7f73ac10a830 2024-05-17T22:45:43.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.084+0000 7f73aad9d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73ac10a830 0x7f73ac11ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:43.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.084+0000 7f73aa59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73ac10b180 0x7f73ac118f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:43.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.084+0000 7f73aa59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73ac10b180 0x7f73ac118f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:35364/0 (socket says 172.21.15.12:35364) 2024-05-17T22:45:43.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.084+0000 7f73aa59c700 1 -- 172.21.15.12:0/1151835835 learned_addr learned my addr 172.21.15.12:0/1151835835 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:45:43.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.085+0000 7f73ab59e700 1 --2- 172.21.15.12:0/1151835835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73ac11d550 0x7f73ac1194a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:43.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.086+0000 7f73aad9d700 1 -- 172.21.15.12:0/1151835835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73ac10b180 msgr2=0x7f73ac118f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:43.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.086+0000 7f73aad9d700 1 --2- 172.21.15.12:0/1151835835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73ac10b180 0x7f73ac118f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:43.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.086+0000 7f73aad9d700 1 -- 172.21.15.12:0/1151835835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73ac11d550 msgr2=0x7f73ac1194a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:43.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.086+0000 7f73aad9d700 1 --2- 172.21.15.12:0/1151835835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73ac11d550 0x7f73ac1194a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:43.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.086+0000 7f73aad9d700 1 -- 172.21.15.12:0/1151835835 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73ac119c40 con 0x7f73ac10a830 2024-05-17T22:45:43.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.086+0000 7f73aad9d700 1 --2- 172.21.15.12:0/1151835835 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73ac10a830 0x7f73ac11ce20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f73a400e9f0 tx=0x7f73a400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:45:43.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.087+0000 7f73a3fff700 1 -- 172.21.15.12:0/1151835835 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a400c9c0 con 0x7f73ac10a830 2024-05-17T22:45:43.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.087+0000 7f73b10f2700 1 -- 172.21.15.12:0/1151835835 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73ac1ad250 con 0x7f73ac10a830 2024-05-17T22:45:43.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.087+0000 7f73b10f2700 1 -- 172.21.15.12:0/1151835835 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73ac1ad760 con 0x7f73ac10a830 2024-05-17T22:45:43.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.087+0000 7f73a3fff700 1 -- 172.21.15.12:0/1151835835 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73a400cb60 con 0x7f73ac10a830 2024-05-17T22:45:43.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.091+0000 7f73a3fff700 1 -- 172.21.15.12:0/1151835835 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a4018680 con 0x7f73ac10a830 2024-05-17T22:45:43.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.108+0000 7f73bfacdb80 1 Processor -- start 2024-05-17T22:45:43.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.109+0000 7f73a3fff700 1 -- 172.21.15.12:0/1151835835 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f73a4018820 con 0x7f73ac10a830 2024-05-17T22:45:43.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.109+0000 7f73bfacdb80 1 -- start start 2024-05-17T22:45:43.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.110+0000 7f73bfacdb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643e564ca70 0x5643e57cff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:43.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.110+0000 7f73bfacdb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643e57d0480 0x5643e57ca640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:43.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.110+0000 7f73bfacdb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643e57cab80 0x5643e57caf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:43.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.110+0000 7f73bfacdb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5643e56f9150 con 0x5643e57d0480 2024-05-17T22:45:43.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.110+0000 7f73bfacdb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5643e56e8cf0 con 0x5643e564ca70 2024-05-17T22:45:43.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.110+0000 7f73bfacdb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5643e56ecd30 con 0x5643e57cab80 2024-05-17T22:45:43.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.110+0000 7f73ab59e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643e564ca70 0x5643e57cff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:43.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.110+0000 7f73ab59e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643e564ca70 0x5643e57cff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:35376/0 (socket says 172.21.15.12:35376) 2024-05-17T22:45:43.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.110+0000 7f73ab59e700 1 -- 172.21.15.12:0/2742724361 learned_addr learned my addr 172.21.15.12:0/2742724361 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:45:43.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.110+0000 7f73aad9d700 1 --2- 172.21.15.12:0/2742724361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643e57d0480 0x5643e57ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:43.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.117+0000 7f73aa59c700 1 --2- 172.21.15.12:0/2742724361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643e57cab80 0x5643e57caf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:43.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.117+0000 7f73ab59e700 1 -- 172.21.15.12:0/2742724361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643e57cab80 msgr2=0x5643e57caf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:43.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.117+0000 7f73ab59e700 1 --2- 172.21.15.12:0/2742724361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643e57cab80 0x5643e57caf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:43.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.117+0000 7f73ab59e700 1 -- 172.21.15.12:0/2742724361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643e57d0480 msgr2=0x5643e57ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:43.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.117+0000 7f73ab59e700 1 --2- 172.21.15.12:0/2742724361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643e57d0480 0x5643e57ca640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:43.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.117+0000 7f73ab59e700 1 -- 172.21.15.12:0/2742724361 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643e5682640 con 0x5643e564ca70 2024-05-17T22:45:43.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.117+0000 7f73ab59e700 1 --2- 172.21.15.12:0/2742724361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643e564ca70 0x5643e57cff40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f739c01a310 tx=0x7f739c01a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29b23f13c3cde434 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:45:43.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.118+0000 7f73a17fa700 1 -- 172.21.15.12:0/2742724361 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739c053070 con 0x5643e564ca70 2024-05-17T22:45:43.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.118+0000 7f73a17fa700 1 -- 172.21.15.12:0/2742724361 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f739c01cc80 con 0x5643e564ca70 2024-05-17T22:45:43.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.118+0000 7f73a17fa700 1 -- 172.21.15.12:0/2742724361 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739c018e00 con 0x5643e564ca70 2024-05-17T22:45:43.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.118+0000 7f73a3fff700 1 --2- 172.21.15.12:0/1151835835 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f73980b2a90 0x7f73980b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:43.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.119+0000 7f73aad9d700 1 --2- 172.21.15.12:0/1151835835 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f73980b2a90 0x7f73980b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:43.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.119+0000 7f73a3fff700 1 -- 172.21.15.12:0/1151835835 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f73a40e2570 con 0x7f73ac10a830 2024-05-17T22:45:43.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.120+0000 7f73bfacdb80 1 -- 172.21.15.12:0/2742724361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643e564ca70 msgr2=0x5643e57cff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:43.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.120+0000 7f73bfacdb80 1 --2- 172.21.15.12:0/2742724361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643e564ca70 0x5643e57cff40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f739c01a310 tx=0x7f739c01a340 comp rx=0 tx=0).stop 2024-05-17T22:45:43.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.120+0000 7f73aad9d700 1 --2- 172.21.15.12:0/1151835835 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f73980b2a90 0x7f73980b4fa0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f73a40a42c0 tx=0x7f73a40e7040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:45:43.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.121+0000 7f73bfacdb80 1 -- 172.21.15.12:0/2742724361 shutdown_connections 2024-05-17T22:45:43.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.121+0000 7f73bfacdb80 1 --2- 172.21.15.12:0/2742724361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643e57cab80 0x5643e57caf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:43.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.121+0000 7f73bfacdb80 1 --2- 172.21.15.12:0/2742724361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643e564ca70 0x5643e57cff40 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:43.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.121+0000 7f73bfacdb80 1 --2- 172.21.15.12:0/2742724361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643e57d0480 0x5643e57ca640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:43.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.121+0000 7f73bfacdb80 1 -- 172.21.15.12:0/2742724361 >> 172.21.15.12:0/2742724361 conn(0x5643e56ac850 msgr2=0x5643e57c5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:45:43.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.124+0000 7f73bfacdb80 1 -- 172.21.15.12:0/2742724361 shutdown_connections 2024-05-17T22:45:43.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.124+0000 7f73bfacdb80 1 -- 172.21.15.12:0/2742724361 wait complete. 2024-05-17T22:45:43.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.125+0000 7f73bfacdb80 1 -- start start 2024-05-17T22:45:43.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.125+0000 7f73bfacdb80 1 Processor -- start 2024-05-17T22:45:43.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.125+0000 7f73bfacdb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643e564ca70 0x5643e57d3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:43.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.125+0000 7f73bfacdb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643e57cab80 0x5643e57d4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:43.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.125+0000 7f73bfacdb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643e57d0480 0x5643e57d45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:43.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.125+0000 7f73bfacdb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5643e55f01d0 con 0x5643e57d0480 2024-05-17T22:45:43.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.125+0000 7f73bfacdb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5643e568a9c0 con 0x5643e57cab80 2024-05-17T22:45:43.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.125+0000 7f73bfacdb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5643e57d6df0 con 0x5643e564ca70 2024-05-17T22:45:43.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.125+0000 7f73bfacdb80 10 client.0 _ll_register_callbacks cb 0x5643e567fbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:45:43.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.126+0000 7f73ab59e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643e57d0480 0x5643e57d45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:43.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.126+0000 7f73ab59e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643e57d0480 0x5643e57d45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37972/0 (socket says 172.21.15.12:37972) 2024-05-17T22:45:43.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.126+0000 7f73ab59e700 1 -- 172.21.15.12:0/2617257850 learned_addr learned my addr 172.21.15.12:0/2617257850 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:45:43.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.126+0000 7f73ab59e700 1 --2- 172.21.15.12:0/2617257850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643e564ca70 0x5643e57d3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:43.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.131+0000 7f73ab59e700 1 -- 172.21.15.12:0/2617257850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643e57cab80 msgr2=0x5643e57d4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:43.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.131+0000 7f73ab59e700 1 --2- 172.21.15.12:0/2617257850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643e57cab80 0x5643e57d4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:43.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.131+0000 7f73ab59e700 1 -- 172.21.15.12:0/2617257850 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643e57d0480 msgr2=0x5643e57d45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:43.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.131+0000 7f73ab59e700 1 --2- 172.21.15.12:0/2617257850 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643e57d0480 0x5643e57d45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:43.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.131+0000 7f73ab59e700 1 -- 172.21.15.12:0/2617257850 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643e5639500 con 0x5643e564ca70 2024-05-17T22:45:43.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.132+0000 7f73ab59e700 1 --2- 172.21.15.12:0/2617257850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643e564ca70 0x5643e57d3b20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f739c01a8d0 tx=0x7f739c01c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4303c0064a59df12 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:45:43.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.132+0000 7f73927fc700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:45:43.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.132+0000 7f73927fc700 1 -- 172.21.15.12:0/2617257850 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739c053070 con 0x5643e564ca70 2024-05-17T22:45:43.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.132+0000 7f73927fc700 1 -- 172.21.15.12:0/2617257850 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f739c003c10 con 0x5643e564ca70 2024-05-17T22:45:43.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.135+0000 7f73bfacdb80 1 -- 172.21.15.12:0/2617257850 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f739c01cc80 con 0x5643e564ca70 2024-05-17T22:45:43.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.137+0000 7f73927fc700 1 -- 172.21.15.12:0/2617257850 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f739c003e10 con 0x5643e564ca70 2024-05-17T22:45:43.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.138+0000 7f73bfacdb80 10 client.25522 fetch_fsmap learned FSMap version 37 2024-05-17T22:45:43.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.138+0000 7f73bfacdb80 1 -- 172.21.15.12:0/2617257850 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5643e57e1950 con 0x5643e564ca70 2024-05-17T22:45:43.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.140+0000 7f73927fc700 1 -- 172.21.15.12:0/2617257850 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f739c01cc80 con 0x5643e564ca70 2024-05-17T22:45:43.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.140+0000 7f73927fc700 1 -- 172.21.15.12:0/2617257850 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f739c022070 con 0x5643e564ca70 2024-05-17T22:45:43.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.142+0000 7f73bfacdb80 10 client.25522 fetch_fsmap finished waiting for FSMap version 37 2024-05-17T22:45:43.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.142+0000 7f73bfacdb80 10 client.25522 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:45:43.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.142+0000 7f73bfacdb80 20 client.25522 populate_metadata read hostname 'smithi012' 2024-05-17T22:45:43.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.142+0000 7f73bfacdb80 1 --2- 172.21.15.12:0/2617257850 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5643e57e3e20 0x5643e57e6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:43.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.142+0000 7f73bfacdb80 4 client.25522 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:45:43.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.142+0000 7f73bfacdb80 1 -- 172.21.15.12:0/2617257850 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643e56fb0a0 con 0x5643e57e3e20 2024-05-17T22:45:43.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.145+0000 7f73aa59c700 1 --2- 172.21.15.12:0/2617257850 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5643e57e3e20 0x5643e57e6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:43.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.147+0000 7f73aa59c700 1 --2- 172.21.15.12:0/2617257850 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5643e57e3e20 0x5643e57e6240 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1405a4b0ad233de server_cookie=d073fc5c9badae8c in_seq=0 out_seq=0 2024-05-17T22:45:43.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.147+0000 7f73927fc700 10 client.25522 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:45:43.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.147+0000 7f73927fc700 1 -- 172.21.15.12:0/2617257850 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5643e56fb0a0 con 0x5643e57e3e20 2024-05-17T22:45:43.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.148+0000 7f73927fc700 10 client.25522 handle_command_reply: tid=0 2024-05-17T22:45:43.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.148+0000 7f73bfacdb80 1 client.25522 shutdown 2024-05-17T22:45:43.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.150+0000 7f73bfacdb80 1 -- 172.21.15.12:0/2617257850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643e564ca70 msgr2=0x5643e57d3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:43.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.150+0000 7f73bfacdb80 1 --2- 172.21.15.12:0/2617257850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643e564ca70 0x5643e57d3b20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f739c01a8d0 tx=0x7f739c01c3e0 comp rx=0 tx=0).stop 2024-05-17T22:45:43.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.150+0000 7f73bfacdb80 1 -- 172.21.15.12:0/2617257850 shutdown_connections 2024-05-17T22:45:43.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.150+0000 7f73bfacdb80 1 --2- 172.21.15.12:0/2617257850 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5643e57e3e20 0x5643e57e6240 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:43.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.150+0000 7f73bfacdb80 1 --2- 172.21.15.12:0/2617257850 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643e57d0480 0x5643e57d45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:43.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.150+0000 7f73bfacdb80 1 --2- 172.21.15.12:0/2617257850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643e564ca70 0x5643e57d3b20 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:43.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.150+0000 7f73bfacdb80 1 --2- 172.21.15.12:0/2617257850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643e57cab80 0x5643e57d4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:43.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.150+0000 7f73bfacdb80 1 -- 172.21.15.12:0/2617257850 >> 172.21.15.12:0/2617257850 conn(0x5643e56ac850 msgr2=0x5643e57c51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:45:43.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.151+0000 7f73927fc700 0 client.25522 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:45:43.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.151+0000 7f73bfacdb80 1 -- 172.21.15.12:0/2617257850 shutdown_connections 2024-05-17T22:45:43.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.151+0000 7f73bfacdb80 1 -- 172.21.15.12:0/2617257850 wait complete. 2024-05-17T22:45:43.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.154+0000 7f73bfacdb80 20 client.25522 trim_cache size 0 max 16384 2024-05-17T22:45:43.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.197+0000 7f73bfacdb80 1 Processor -- start 2024-05-17T22:45:43.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.197+0000 7f73bfacdb80 1 -- start start 2024-05-17T22:45:43.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.197+0000 7f73bfacdb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643e564ca70 0x5643e57d2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:43.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.197+0000 7f73bfacdb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643e57cab80 0x5643e57d43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:43.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.198+0000 7f73bfacdb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643e57d0480 0x5643e57e0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:43.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.198+0000 7f73bfacdb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f738c011460 con 0x5643e57cab80 2024-05-17T22:45:43.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.198+0000 7f73bfacdb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f738c00ecf0 con 0x5643e57d0480 2024-05-17T22:45:43.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.198+0000 7f73bfacdb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f738c00cb40 con 0x5643e564ca70 2024-05-17T22:45:43.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.198+0000 7f73ab59e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643e564ca70 0x5643e57d2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:43.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.198+0000 7f73ab59e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643e564ca70 0x5643e57d2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:37244/0 (socket says 172.21.15.12:37244) 2024-05-17T22:45:43.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.198+0000 7f73ab59e700 1 -- 172.21.15.12:0/3757911655 learned_addr learned my addr 172.21.15.12:0/3757911655 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:45:43.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.198+0000 7f73aa59c700 1 --2- 172.21.15.12:0/3757911655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643e57cab80 0x5643e57d43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:43.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.198+0000 7f73ab59e700 1 --2- 172.21.15.12:0/3757911655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643e57d0480 0x5643e57e0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:43.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.198+0000 7f73ab59e700 1 -- 172.21.15.12:0/3757911655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643e57d0480 msgr2=0x5643e57e0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:43.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.198+0000 7f73ab59e700 1 --2- 172.21.15.12:0/3757911655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643e57d0480 0x5643e57e0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:43.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.198+0000 7f73ab59e700 1 -- 172.21.15.12:0/3757911655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643e57cab80 msgr2=0x5643e57d43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:43.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.198+0000 7f73ab59e700 1 --2- 172.21.15.12:0/3757911655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643e57cab80 0x5643e57d43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:43.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.198+0000 7f73ab59e700 1 -- 172.21.15.12:0/3757911655 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643e57c9bf0 con 0x5643e564ca70 2024-05-17T22:45:43.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.198+0000 7f73ab59e700 1 --2- 172.21.15.12:0/3757911655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643e564ca70 0x5643e57d2a60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f739c01a380 tx=0x7f739c018680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6a208c5155716a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:45:43.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.199+0000 7f73a0ff9700 1 -- 172.21.15.12:0/3757911655 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739c053040 con 0x5643e564ca70 2024-05-17T22:45:43.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.199+0000 7f73a0ff9700 1 -- 172.21.15.12:0/3757911655 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f739c067930 con 0x5643e564ca70 2024-05-17T22:45:43.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.199+0000 7f73a0ff9700 1 -- 172.21.15.12:0/3757911655 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739c005ea0 con 0x5643e564ca70 2024-05-17T22:45:43.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.200+0000 7f73bfacdb80 1 -- 172.21.15.12:0/3757911655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643e564ca70 msgr2=0x5643e57d2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:43.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.200+0000 7f73bfacdb80 1 --2- 172.21.15.12:0/3757911655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643e564ca70 0x5643e57d2a60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f739c01a380 tx=0x7f739c018680 comp rx=0 tx=0).stop 2024-05-17T22:45:43.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.200+0000 7f73bfacdb80 1 -- 172.21.15.12:0/3757911655 shutdown_connections 2024-05-17T22:45:43.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.200+0000 7f73bfacdb80 1 --2- 172.21.15.12:0/3757911655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643e57d0480 0x5643e57e0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:43.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.200+0000 7f73bfacdb80 1 --2- 172.21.15.12:0/3757911655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643e564ca70 0x5643e57d2a60 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:43.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.200+0000 7f73bfacdb80 1 --2- 172.21.15.12:0/3757911655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643e57cab80 0x5643e57d43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:43.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.200+0000 7f73bfacdb80 1 -- 172.21.15.12:0/3757911655 >> 172.21.15.12:0/3757911655 conn(0x5643e56ac850 msgr2=0x5643e57bf190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:45:43.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.200+0000 7f73bfacdb80 1 -- 172.21.15.12:0/3757911655 shutdown_connections 2024-05-17T22:45:43.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.200+0000 7f73bfacdb80 1 -- 172.21.15.12:0/3757911655 wait complete. 2024-05-17T22:45:43.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.201+0000 7f73bfacdb80 1 -- start start 2024-05-17T22:45:43.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.201+0000 7f73bfacdb80 1 Processor -- start 2024-05-17T22:45:43.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.202+0000 7f73bfacdb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643e564ca70 0x5643e57e6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:43.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.202+0000 7f73bfacdb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643e57cab80 0x5643e57e7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:43.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.202+0000 7f73bfacdb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643e57d0480 0x5643e57e76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:43.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.202+0000 7f73ab59e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643e564ca70 0x5643e57e6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:43.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.202+0000 7f73bfacdb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f738c00d1b0 con 0x5643e57cab80 2024-05-17T22:45:43.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.202+0000 7f73bfacdb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f738c00e620 con 0x5643e57d0480 2024-05-17T22:45:43.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.202+0000 7f73bfacdb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f738c00d880 con 0x5643e564ca70 2024-05-17T22:45:43.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.202+0000 7f73ab59e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643e564ca70 0x5643e57e6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:37258/0 (socket says 172.21.15.12:37258) 2024-05-17T22:45:43.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.202+0000 7f73ab59e700 1 -- 172.21.15.12:0/1266864133 learned_addr learned my addr 172.21.15.12:0/1266864133 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:45:43.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.202+0000 7f73bfacdb80 10 client.0 _ll_register_callbacks cb 0x5643e567fbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:45:43.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.203+0000 7f73ab59e700 1 --2- 172.21.15.12:0/1266864133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643e57d0480 0x5643e57e76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:43.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.203+0000 7f73ab59e700 1 -- 172.21.15.12:0/1266864133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643e57d0480 msgr2=0x5643e57e76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:43.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.203+0000 7f73ab59e700 1 --2- 172.21.15.12:0/1266864133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643e57d0480 0x5643e57e76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:43.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.203+0000 7f73ab59e700 1 -- 172.21.15.12:0/1266864133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643e57cab80 msgr2=0x5643e57e7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:43.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.203+0000 7f73ab59e700 1 --2- 172.21.15.12:0/1266864133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643e57cab80 0x5643e57e7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:43.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.203+0000 7f73ab59e700 1 -- 172.21.15.12:0/1266864133 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643e584d560 con 0x5643e564ca70 2024-05-17T22:45:43.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.203+0000 7f73ab59e700 1 --2- 172.21.15.12:0/1266864133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643e564ca70 0x5643e57e6c20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f739c01c0d0 tx=0x7f739c067500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7c9e53290a87585 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:45:43.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.204+0000 7f73927fc700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:45:43.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.204+0000 7f73927fc700 1 -- 172.21.15.12:0/1266864133 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739c053070 con 0x5643e564ca70 2024-05-17T22:45:43.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.204+0000 7f73927fc700 1 -- 172.21.15.12:0/1266864133 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f739c061040 con 0x5643e564ca70 2024-05-17T22:45:43.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.204+0000 7f73bfacdb80 1 -- 172.21.15.12:0/1266864133 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f739c067930 con 0x5643e564ca70 2024-05-17T22:45:43.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.205+0000 7f73927fc700 1 -- 172.21.15.12:0/1266864133 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f739c07d050 con 0x5643e564ca70 2024-05-17T22:45:43.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.205+0000 7f73bfacdb80 10 client.25528 fetch_fsmap learned FSMap version 37 2024-05-17T22:45:43.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.205+0000 7f73bfacdb80 1 -- 172.21.15.12:0/1266864133 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5643e584e7e0 con 0x5643e564ca70 2024-05-17T22:45:43.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.205+0000 7f73927fc700 1 -- 172.21.15.12:0/1266864133 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f739c005fa0 con 0x5643e564ca70 2024-05-17T22:45:43.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.205+0000 7f73927fc700 1 -- 172.21.15.12:0/1266864133 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f739c022070 con 0x5643e564ca70 2024-05-17T22:45:43.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.205+0000 7f73bfacdb80 10 client.25528 fetch_fsmap finished waiting for FSMap version 37 2024-05-17T22:45:43.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.205+0000 7f73bfacdb80 10 client.25528 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:45:43.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.205+0000 7f73bfacdb80 20 client.25528 populate_metadata read hostname 'smithi012' 2024-05-17T22:45:43.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.205+0000 7f73bfacdb80 1 --2- 172.21.15.12:0/1266864133 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5643e584edc0 0x5643e58511e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:43.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.206+0000 7f73bfacdb80 4 client.25528 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-17T22:45:43.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.206+0000 7f73bfacdb80 1 -- 172.21.15.12:0/1266864133 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f739c067930 con 0x5643e584edc0 2024-05-17T22:45:43.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.206+0000 7f73aa59c700 1 --2- 172.21.15.12:0/1266864133 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5643e584edc0 0x5643e58511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:43.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.207+0000 7f73927fc700 10 client.25528 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:45:43.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.207+0000 7f73aa59c700 1 --2- 172.21.15.12:0/1266864133 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5643e584edc0 0x5643e58511e0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24643faad18f4f69 server_cookie=f7170d3838718b6c in_seq=0 out_seq=0 2024-05-17T22:45:43.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.208+0000 7f73927fc700 1 -- 172.21.15.12:0/1266864133 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f739c067930 con 0x5643e584edc0 2024-05-17T22:45:43.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.208+0000 7f73927fc700 10 client.25528 handle_command_reply: tid=0 2024-05-17T22:45:43.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.208+0000 7f73bfacdb80 1 client.25528 shutdown 2024-05-17T22:45:43.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.208+0000 7f73bfacdb80 1 -- 172.21.15.12:0/1266864133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643e564ca70 msgr2=0x5643e57e6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:43.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.208+0000 7f73bfacdb80 1 --2- 172.21.15.12:0/1266864133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643e564ca70 0x5643e57e6c20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f739c01c0d0 tx=0x7f739c067500 comp rx=0 tx=0).stop 2024-05-17T22:45:43.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.208+0000 7f73bfacdb80 1 -- 172.21.15.12:0/1266864133 shutdown_connections 2024-05-17T22:45:43.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.208+0000 7f73bfacdb80 1 --2- 172.21.15.12:0/1266864133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643e57d0480 0x5643e57e76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:43.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.208+0000 7f73bfacdb80 1 --2- 172.21.15.12:0/1266864133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643e564ca70 0x5643e57e6c20 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:43.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.208+0000 7f73bfacdb80 1 --2- 172.21.15.12:0/1266864133 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5643e584edc0 0x5643e58511e0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:43.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.208+0000 7f73bfacdb80 1 --2- 172.21.15.12:0/1266864133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643e57cab80 0x5643e57e7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:43.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.208+0000 7f73bfacdb80 1 -- 172.21.15.12:0/1266864133 >> 172.21.15.12:0/1266864133 conn(0x5643e56ac850 msgr2=0x5643e57cd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:45:43.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.208+0000 7f73927fc700 0 client.25528 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:45:43.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.208+0000 7f73bfacdb80 1 -- 172.21.15.12:0/1266864133 shutdown_connections 2024-05-17T22:45:43.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.208+0000 7f73bfacdb80 1 -- 172.21.15.12:0/1266864133 wait complete. 2024-05-17T22:45:43.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.209+0000 7f73bfacdb80 20 client.25528 trim_cache size 0 max 16384 2024-05-17T22:45:43.228 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:45:43.228 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-17T22:45:43.228 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "a05fdcd6-f2db-4e73-ba06-bc91cbe08e1c", 2024-05-17T22:45:43.229 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-17T22:45:43.229 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:45:43.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.209+0000 7f73b10f2700 1 -- 172.21.15.12:0/1151835835 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f73980b2a90 msgr2=0x7f73980b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:43.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.209+0000 7f73b10f2700 1 --2- 172.21.15.12:0/1151835835 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f73980b2a90 0x7f73980b4fa0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f73a40a42c0 tx=0x7f73a40e7040 comp rx=0 tx=0).stop 2024-05-17T22:45:43.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.209+0000 7f73b10f2700 1 -- 172.21.15.12:0/1151835835 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73ac10a830 msgr2=0x7f73ac11ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:43.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.209+0000 7f73b10f2700 1 --2- 172.21.15.12:0/1151835835 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73ac10a830 0x7f73ac11ce20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f73a400e9f0 tx=0x7f73a400ed00 comp rx=0 tx=0).stop 2024-05-17T22:45:43.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.209+0000 7f73b10f2700 1 -- 172.21.15.12:0/1151835835 shutdown_connections 2024-05-17T22:45:43.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.209+0000 7f73b10f2700 1 --2- 172.21.15.12:0/1151835835 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f73980b2a90 0x7f73980b4fa0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:43.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.209+0000 7f73b10f2700 1 --2- 172.21.15.12:0/1151835835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73ac11d550 0x7f73ac1194a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:43.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.209+0000 7f73b10f2700 1 --2- 172.21.15.12:0/1151835835 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73ac10a830 0x7f73ac11ce20 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:43.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.209+0000 7f73b10f2700 1 --2- 172.21.15.12:0/1151835835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73ac10b180 0x7f73ac118f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:43.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.209+0000 7f73b10f2700 1 -- 172.21.15.12:0/1151835835 >> 172.21.15.12:0/1151835835 conn(0x7f73ac06efc0 msgr2=0x7f73ac11db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:45:43.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.210+0000 7f73b10f2700 1 -- 172.21.15.12:0/1151835835 shutdown_connections 2024-05-17T22:45:43.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:43.210+0000 7f73b10f2700 1 -- 172.21.15.12:0/1151835835 wait complete. 2024-05-17T22:45:44.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:43 smithi012 ceph-mon[31632]: pgmap v633: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 MiB/s wr, 575 op/s 2024-05-17T22:45:44.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:43 smithi012 ceph-mon[31632]: scrub queued for path: / 2024-05-17T22:45:44.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:43 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:45:44.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:43 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:45:44.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:43 smithi151 ceph-mon[36101]: pgmap v633: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 MiB/s wr, 575 op/s 2024-05-17T22:45:44.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:43 smithi151 ceph-mon[36101]: scrub queued for path: / 2024-05-17T22:45:44.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:43 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:45:44.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:43 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:45:44.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:43 smithi096 ceph-mon[36065]: pgmap v633: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 MiB/s wr, 575 op/s 2024-05-17T22:45:44.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:43 smithi096 ceph-mon[36065]: scrub queued for path: / 2024-05-17T22:45:44.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:43 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:45:44.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:43 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:45:44.327 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-17T22:45:44.787 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:45:45.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:44 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:45:45.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:44 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:45:45.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:44 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:45:45.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:44 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:45:45.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:44 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:45:45.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:44 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:45:45.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:44 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:45:45.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:44 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:45:45.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:44 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:45:45.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:44 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:45:45.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:44 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:45:45.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:44 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:45:45.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:44 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:45:45.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:44 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:45:45.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:44 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:45:46.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:45 smithi012 ceph-mon[31632]: pgmap v634: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.0 MiB/s wr, 537 op/s 2024-05-17T22:45:46.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:45 smithi151 ceph-mon[36101]: pgmap v634: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.0 MiB/s wr, 537 op/s 2024-05-17T22:45:46.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:45 smithi096 ceph-mon[36065]: pgmap v634: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.0 MiB/s wr, 537 op/s 2024-05-17T22:45:46.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 385.0 s, 549.8 tps, lat 176.899 ms stddev 208.018 2024-05-17T22:45:47.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.852+0000 7f797bbd5700 1 -- 172.21.15.12:0/2769121457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f797410b350 msgr2=0x7f7974116440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:47.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.853+0000 7f797bbd5700 1 --2- 172.21.15.12:0/2769121457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f797410b350 0x7f7974116440 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f7964009990 tx=0x7f7964009ca0 comp rx=0 tx=0).stop 2024-05-17T22:45:47.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.853+0000 7f797bbd5700 1 -- 172.21.15.12:0/2769121457 shutdown_connections 2024-05-17T22:45:47.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.853+0000 7f797bbd5700 1 --2- 172.21.15.12:0/2769121457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7974116a60 0x7f7974118f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:47.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.853+0000 7f797bbd5700 1 --2- 172.21.15.12:0/2769121457 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f797410aa00 0x7f797410ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:47.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.853+0000 7f797bbd5700 1 --2- 172.21.15.12:0/2769121457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f797410b350 0x7f7974116440 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:47.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.853+0000 7f797bbd5700 1 -- 172.21.15.12:0/2769121457 >> 172.21.15.12:0/2769121457 conn(0x7f797406efc0 msgr2=0x7f797406f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:45:47.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.853+0000 7f797bbd5700 1 -- 172.21.15.12:0/2769121457 shutdown_connections 2024-05-17T22:45:47.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.853+0000 7f797bbd5700 1 -- 172.21.15.12:0/2769121457 wait complete. 2024-05-17T22:45:47.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.854+0000 7f797bbd5700 1 Processor -- start 2024-05-17T22:45:47.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.854+0000 7f797bbd5700 1 -- start start 2024-05-17T22:45:47.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.854+0000 7f797bbd5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f797410aa00 0x7f797410c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:47.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.854+0000 7f797bbd5700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f797410b350 0x7f797410c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:47.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.854+0000 7f797bbd5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7974116a60 0x7f797410cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:47.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.854+0000 7f797bbd5700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f797411c640 con 0x7f7974116a60 2024-05-17T22:45:47.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.854+0000 7f797bbd5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f797411c4c0 con 0x7f797410aa00 2024-05-17T22:45:47.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.854+0000 7f797bbd5700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f797411c7c0 con 0x7f797410b350 2024-05-17T22:45:47.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.854+0000 7f7979971700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f797410aa00 0x7f797410c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:47.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.854+0000 7f7979170700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f797410b350 0x7f797410c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:47.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.854+0000 7f7979971700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f797410aa00 0x7f797410c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:35414/0 (socket says 172.21.15.12:35414) 2024-05-17T22:45:47.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.854+0000 7f7979971700 1 -- 172.21.15.12:0/4241882103 learned_addr learned my addr 172.21.15.12:0/4241882103 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:45:47.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.854+0000 7f7979971700 1 -- 172.21.15.12:0/4241882103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f797410b350 msgr2=0x7f797410c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:47.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.854+0000 7f7979971700 1 --2- 172.21.15.12:0/4241882103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f797410b350 0x7f797410c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:47.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.854+0000 7f7979971700 1 -- 172.21.15.12:0/4241882103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7974116a60 msgr2=0x7f797410cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:47.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.854+0000 7f7979971700 1 --2- 172.21.15.12:0/4241882103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7974116a60 0x7f797410cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:47.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.854+0000 7f7979971700 1 -- 172.21.15.12:0/4241882103 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79741b1440 con 0x7f797410aa00 2024-05-17T22:45:47.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.855+0000 7f7979971700 1 --2- 172.21.15.12:0/4241882103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f797410aa00 0x7f797410c2a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f797000d950 tx=0x7f797000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:45:47.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.855+0000 7f796affd700 1 -- 172.21.15.12:0/4241882103 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7970016070 con 0x7f797410aa00 2024-05-17T22:45:47.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.855+0000 7f796affd700 1 -- 172.21.15.12:0/4241882103 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7970009d70 con 0x7f797410aa00 2024-05-17T22:45:47.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.855+0000 7f796affd700 1 -- 172.21.15.12:0/4241882103 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797000f3c0 con 0x7f797410aa00 2024-05-17T22:45:47.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.855+0000 7f797bbd5700 1 -- 172.21.15.12:0/4241882103 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79741b1730 con 0x7f797410aa00 2024-05-17T22:45:47.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.855+0000 7f797bbd5700 1 -- 172.21.15.12:0/4241882103 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79741b1c20 con 0x7f797410aa00 2024-05-17T22:45:47.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.861+0000 7f796affd700 1 -- 172.21.15.12:0/4241882103 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7970019070 con 0x7f797410aa00 2024-05-17T22:45:47.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.862+0000 7f796affd700 1 --2- 172.21.15.12:0/4241882103 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f79600b2900 0x7f79600b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:47.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.862+0000 7f796affd700 1 -- 172.21.15.12:0/4241882103 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f79700e2350 con 0x7f797410aa00 2024-05-17T22:45:47.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.862+0000 7f7979170700 1 --2- 172.21.15.12:0/4241882103 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f79600b2900 0x7f79600b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:47.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.863+0000 7f798a5b0b80 1 Processor -- start 2024-05-17T22:45:47.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.863+0000 7f798a5b0b80 1 -- start start 2024-05-17T22:45:47.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.863+0000 7f7979170700 1 --2- 172.21.15.12:0/4241882103 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f79600b2900 0x7f79600b4e10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7974119310 tx=0x7f796400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:45:47.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.864+0000 7f798a5b0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622badd79f0 0x5622baf5b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:47.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.864+0000 7f798a5b0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622baf5baa0 0x5622baf55c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:47.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.864+0000 7f798a5b0b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622baf56170 0x5622baf56580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:47.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.864+0000 7f798a5b0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5622bae84290 con 0x5622badd79f0 2024-05-17T22:45:47.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.864+0000 7f798a5b0b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5622bae73fd0 con 0x5622baf56170 2024-05-17T22:45:47.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.864+0000 7f798a5b0b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5622bae77e70 con 0x5622baf5baa0 2024-05-17T22:45:47.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.864+0000 7f7979971700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622badd79f0 0x5622baf5b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:47.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.864+0000 7f797a172700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622baf56170 0x5622baf56580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:47.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.864+0000 7f7979971700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622badd79f0 0x5622baf5b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38020/0 (socket says 172.21.15.12:38020) 2024-05-17T22:45:47.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.864+0000 7f7979971700 1 -- 172.21.15.12:0/2290952208 learned_addr learned my addr 172.21.15.12:0/2290952208 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:45:47.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.864+0000 7f7979170700 1 --2- 172.21.15.12:0/2290952208 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622baf5baa0 0x5622baf55c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:47.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.864+0000 7f7979971700 1 -- 172.21.15.12:0/2290952208 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622baf5baa0 msgr2=0x5622baf55c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:47.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.864+0000 7f7979971700 1 --2- 172.21.15.12:0/2290952208 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622baf5baa0 0x5622baf55c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:47.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.864+0000 7f7979971700 1 -- 172.21.15.12:0/2290952208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622baf56170 msgr2=0x5622baf56580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:47.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.864+0000 7f7979971700 1 --2- 172.21.15.12:0/2290952208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622baf56170 0x5622baf56580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:47.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.864+0000 7f7979971700 1 -- 172.21.15.12:0/2290952208 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622bae1d590 con 0x5622badd79f0 2024-05-17T22:45:47.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.865+0000 7f7979971700 1 --2- 172.21.15.12:0/2290952208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622badd79f0 0x5622baf5b560 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f79700dfb00 tx=0x7f79700a8ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9dd24e503d138bf server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:45:47.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.865+0000 7f795ffff700 1 -- 172.21.15.12:0/2290952208 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7970020070 con 0x5622badd79f0 2024-05-17T22:45:47.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.865+0000 7f795ffff700 1 -- 172.21.15.12:0/2290952208 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79700e2170 con 0x5622badd79f0 2024-05-17T22:45:47.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.865+0000 7f795ffff700 1 -- 172.21.15.12:0/2290952208 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79700a98e0 con 0x5622badd79f0 2024-05-17T22:45:47.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.866+0000 7f798a5b0b80 1 -- 172.21.15.12:0/2290952208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622badd79f0 msgr2=0x5622baf5b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:47.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.866+0000 7f798a5b0b80 1 --2- 172.21.15.12:0/2290952208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622badd79f0 0x5622baf5b560 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f79700dfb00 tx=0x7f79700a8ec0 comp rx=0 tx=0).stop 2024-05-17T22:45:47.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.866+0000 7f798a5b0b80 1 -- 172.21.15.12:0/2290952208 shutdown_connections 2024-05-17T22:45:47.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.866+0000 7f798a5b0b80 1 --2- 172.21.15.12:0/2290952208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622baf56170 0x5622baf56580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:47.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.866+0000 7f798a5b0b80 1 --2- 172.21.15.12:0/2290952208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622badd79f0 0x5622baf5b560 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:47.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.866+0000 7f798a5b0b80 1 --2- 172.21.15.12:0/2290952208 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622baf5baa0 0x5622baf55c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:47.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.866+0000 7f798a5b0b80 1 -- 172.21.15.12:0/2290952208 >> 172.21.15.12:0/2290952208 conn(0x5622bae1bf00 msgr2=0x5622baf50b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:45:47.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.866+0000 7f798a5b0b80 1 -- 172.21.15.12:0/2290952208 shutdown_connections 2024-05-17T22:45:47.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.866+0000 7f798a5b0b80 1 -- 172.21.15.12:0/2290952208 wait complete. 2024-05-17T22:45:47.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.867+0000 7f798a5b0b80 1 -- start start 2024-05-17T22:45:47.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.867+0000 7f798a5b0b80 1 Processor -- start 2024-05-17T22:45:47.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.868+0000 7f798a5b0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622badd79f0 0x5622baf5f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:47.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.868+0000 7f798a5b0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622baf56170 0x5622baf5f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:47.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.868+0000 7f798a5b0b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622baf5baa0 0x5622baf5fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:47.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.868+0000 7f798a5b0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5622bad7b150 con 0x5622badd79f0 2024-05-17T22:45:47.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.868+0000 7f798a5b0b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5622bae14e80 con 0x5622baf5baa0 2024-05-17T22:45:47.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.868+0000 7f798a5b0b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5622baf62390 con 0x5622baf56170 2024-05-17T22:45:47.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.868+0000 7f797a172700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622baf5baa0 0x5622baf5fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:47.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.868+0000 7f798a5b0b80 10 client.0 _ll_register_callbacks cb 0x5622bae0ef10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:45:47.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.868+0000 7f797a172700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622baf5baa0 0x5622baf5fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:35424/0 (socket says 172.21.15.12:35424) 2024-05-17T22:45:47.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.868+0000 7f797a172700 1 -- 172.21.15.12:0/2034964600 learned_addr learned my addr 172.21.15.12:0/2034964600 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:45:47.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.869+0000 7f7979971700 1 --2- 172.21.15.12:0/2034964600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622badd79f0 0x5622baf5f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:47.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.869+0000 7f7979170700 1 --2- 172.21.15.12:0/2034964600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622baf56170 0x5622baf5f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:47.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.869+0000 7f797a172700 1 -- 172.21.15.12:0/2034964600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622baf56170 msgr2=0x5622baf5f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:47.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.869+0000 7f797a172700 1 --2- 172.21.15.12:0/2034964600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622baf56170 0x5622baf5f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:47.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.869+0000 7f797a172700 1 -- 172.21.15.12:0/2034964600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622badd79f0 msgr2=0x5622baf5f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:47.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.869+0000 7f797a172700 1 --2- 172.21.15.12:0/2034964600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622badd79f0 0x5622baf5f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:47.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.869+0000 7f797a172700 1 -- 172.21.15.12:0/2034964600 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622badc4480 con 0x5622baf5baa0 2024-05-17T22:45:47.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.869+0000 7f7979971700 1 --2- 172.21.15.12:0/2034964600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622badd79f0 0x5622baf5f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:45:47.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.869+0000 7f797a172700 1 --2- 172.21.15.12:0/2034964600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622baf5baa0 0x5622baf5fd10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f796c007af0 tx=0x7f796c007eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4344453d6158251 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:45:47.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.869+0000 7f795d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:45:47.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.869+0000 7f795d7fa700 1 -- 172.21.15.12:0/2034964600 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796c010ab0 con 0x5622baf5baa0 2024-05-17T22:45:47.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.869+0000 7f795d7fa700 1 -- 172.21.15.12:0/2034964600 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f796c010c50 con 0x5622baf5baa0 2024-05-17T22:45:47.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.869+0000 7f798a5b0b80 1 -- 172.21.15.12:0/2034964600 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79700e2170 con 0x5622baf5baa0 2024-05-17T22:45:47.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.870+0000 7f795d7fa700 1 -- 172.21.15.12:0/2034964600 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f796c023050 con 0x5622baf5baa0 2024-05-17T22:45:47.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.870+0000 7f798a5b0b80 10 client.25595 fetch_fsmap learned FSMap version 37 2024-05-17T22:45:47.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.870+0000 7f798a5b0b80 1 -- 172.21.15.12:0/2034964600 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5622baf6ccb0 con 0x5622baf5baa0 2024-05-17T22:45:47.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.870+0000 7f795d7fa700 1 -- 172.21.15.12:0/2034964600 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79700e2170 con 0x5622baf5baa0 2024-05-17T22:45:47.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.870+0000 7f795d7fa700 1 -- 172.21.15.12:0/2034964600 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f796c01e420 con 0x5622baf5baa0 2024-05-17T22:45:47.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.870+0000 7f798a5b0b80 10 client.25595 fetch_fsmap finished waiting for FSMap version 37 2024-05-17T22:45:47.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.870+0000 7f798a5b0b80 10 client.25595 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:45:47.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.870+0000 7f798a5b0b80 20 client.25595 populate_metadata read hostname 'smithi012' 2024-05-17T22:45:47.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.870+0000 7f798a5b0b80 1 --2- 172.21.15.12:0/2034964600 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5622baf6f390 0x5622baf717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:47.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.870+0000 7f798a5b0b80 4 client.25595 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:45:47.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.870+0000 7f798a5b0b80 1 -- 172.21.15.12:0/2034964600 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622bae861e0 con 0x5622baf6f390 2024-05-17T22:45:47.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.871+0000 7f7979971700 1 --2- 172.21.15.12:0/2034964600 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5622baf6f390 0x5622baf717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:47.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.871+0000 7f795d7fa700 10 client.25595 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:45:47.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.871+0000 7f7979971700 1 --2- 172.21.15.12:0/2034964600 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5622baf6f390 0x5622baf717f0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb22130d079cfc7e server_cookie=e175bc15eb4b181d in_seq=0 out_seq=0 2024-05-17T22:45:47.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.872+0000 7f795d7fa700 1 -- 172.21.15.12:0/2034964600 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5622bae861e0 con 0x5622baf6f390 2024-05-17T22:45:47.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.872+0000 7f795d7fa700 10 client.25595 handle_command_reply: tid=0 2024-05-17T22:45:47.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.872+0000 7f798a5b0b80 1 client.25595 shutdown 2024-05-17T22:45:47.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.873+0000 7f798a5b0b80 1 -- 172.21.15.12:0/2034964600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622baf5baa0 msgr2=0x5622baf5fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:47.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.873+0000 7f798a5b0b80 1 --2- 172.21.15.12:0/2034964600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622baf5baa0 0x5622baf5fd10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f796c007af0 tx=0x7f796c007eb0 comp rx=0 tx=0).stop 2024-05-17T22:45:47.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.873+0000 7f798a5b0b80 1 -- 172.21.15.12:0/2034964600 shutdown_connections 2024-05-17T22:45:47.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.873+0000 7f798a5b0b80 1 --2- 172.21.15.12:0/2034964600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622baf5baa0 0x5622baf5fd10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:47.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.873+0000 7f798a5b0b80 1 --2- 172.21.15.12:0/2034964600 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5622baf6f390 0x5622baf717f0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:47.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.873+0000 7f795d7fa700 0 client.25595 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:45:47.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.873+0000 7f798a5b0b80 1 --2- 172.21.15.12:0/2034964600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622badd79f0 0x5622baf5f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:47.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.873+0000 7f798a5b0b80 1 --2- 172.21.15.12:0/2034964600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622baf56170 0x5622baf5f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:47.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.873+0000 7f798a5b0b80 1 -- 172.21.15.12:0/2034964600 >> 172.21.15.12:0/2034964600 conn(0x5622bae1bf00 msgr2=0x5622baf4aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:45:47.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.873+0000 7f798a5b0b80 1 -- 172.21.15.12:0/2034964600 shutdown_connections 2024-05-17T22:45:47.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.873+0000 7f798a5b0b80 1 -- 172.21.15.12:0/2034964600 wait complete. 2024-05-17T22:45:47.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.873+0000 7f798a5b0b80 20 client.25595 trim_cache size 0 max 16384 2024-05-17T22:45:47.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.900+0000 7f798a5b0b80 1 Processor -- start 2024-05-17T22:45:47.903 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:47 smithi012 ceph-mon[31632]: pgmap v635: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.8 MiB/s wr, 492 op/s 2024-05-17T22:45:47.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.900+0000 7f798a5b0b80 1 -- start start 2024-05-17T22:45:47.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.901+0000 7f798a5b0b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622badd79f0 0x5622baf55c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:47.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.901+0000 7f798a5b0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622baf56170 0x5622baf61880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:47.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.901+0000 7f798a5b0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622baf5baa0 0x5622baf4aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:47.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.901+0000 7f798a5b0b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f79741bd190 con 0x5622baf56170 2024-05-17T22:45:47.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.901+0000 7f7979971700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622badd79f0 0x5622baf55c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:47.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.901+0000 7f7979971700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622badd79f0 0x5622baf55c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:35430/0 (socket says 172.21.15.12:35430) 2024-05-17T22:45:47.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.901+0000 7f7979971700 1 -- 172.21.15.12:0/2086647475 learned_addr learned my addr 172.21.15.12:0/2086647475 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:45:47.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.901+0000 7f7979170700 1 --2- 172.21.15.12:0/2086647475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622baf56170 0x5622baf61880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:47.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.901+0000 7f798a5b0b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f79741baa20 con 0x5622badd79f0 2024-05-17T22:45:47.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.902+0000 7f798a5b0b80 1 -- 172.21.15.12:0/2086647475 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f79741b8870 con 0x5622baf5baa0 2024-05-17T22:45:47.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.902+0000 7f797a172700 1 --2- 172.21.15.12:0/2086647475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622baf5baa0 0x5622baf4aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:47.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.902+0000 7f7979971700 1 -- 172.21.15.12:0/2086647475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622baf5baa0 msgr2=0x5622baf4aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:47.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.902+0000 7f7979971700 1 --2- 172.21.15.12:0/2086647475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622baf5baa0 0x5622baf4aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:47.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.902+0000 7f7979971700 1 -- 172.21.15.12:0/2086647475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622baf56170 msgr2=0x5622baf61880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:47.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.902+0000 7f7979971700 1 --2- 172.21.15.12:0/2086647475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622baf56170 0x5622baf61880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:47.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.902+0000 7f7979971700 1 -- 172.21.15.12:0/2086647475 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622baf4b3f0 con 0x5622badd79f0 2024-05-17T22:45:47.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.902+0000 7f797a172700 1 --2- 172.21.15.12:0/2086647475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622baf5baa0 0x5622baf4aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:45:47.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.902+0000 7f7979170700 1 --2- 172.21.15.12:0/2086647475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622baf56170 0x5622baf61880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:45:47.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.902+0000 7f7979971700 1 --2- 172.21.15.12:0/2086647475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622badd79f0 0x5622baf55c30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7970000c00 tx=0x7f7970005910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f8356393118763a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:45:47.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.903+0000 7f795f7fe700 1 -- 172.21.15.12:0/2086647475 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7970020040 con 0x5622badd79f0 2024-05-17T22:45:47.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.903+0000 7f795f7fe700 1 -- 172.21.15.12:0/2086647475 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79700b24e0 con 0x5622badd79f0 2024-05-17T22:45:47.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.903+0000 7f795f7fe700 1 -- 172.21.15.12:0/2086647475 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79700a98e0 con 0x5622badd79f0 2024-05-17T22:45:47.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.904+0000 7f798a5b0b80 1 -- 172.21.15.12:0/2086647475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622badd79f0 msgr2=0x5622baf55c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:47.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.904+0000 7f798a5b0b80 1 --2- 172.21.15.12:0/2086647475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622badd79f0 0x5622baf55c30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7970000c00 tx=0x7f7970005910 comp rx=0 tx=0).stop 2024-05-17T22:45:47.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.904+0000 7f798a5b0b80 1 -- 172.21.15.12:0/2086647475 shutdown_connections 2024-05-17T22:45:47.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.904+0000 7f798a5b0b80 1 --2- 172.21.15.12:0/2086647475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622baf5baa0 0x5622baf4aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:47.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.904+0000 7f798a5b0b80 1 --2- 172.21.15.12:0/2086647475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622badd79f0 0x5622baf55c30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:47.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.904+0000 7f798a5b0b80 1 --2- 172.21.15.12:0/2086647475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622baf56170 0x5622baf61880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:47.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.904+0000 7f798a5b0b80 1 -- 172.21.15.12:0/2086647475 >> 172.21.15.12:0/2086647475 conn(0x5622bae1bf00 msgr2=0x5622baf59610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:45:47.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.904+0000 7f798a5b0b80 1 -- 172.21.15.12:0/2086647475 shutdown_connections 2024-05-17T22:45:47.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.904+0000 7f798a5b0b80 1 -- 172.21.15.12:0/2086647475 wait complete. 2024-05-17T22:45:47.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.905+0000 7f798a5b0b80 1 -- start start 2024-05-17T22:45:47.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.905+0000 7f798a5b0b80 1 Processor -- start 2024-05-17T22:45:47.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.906+0000 7f798a5b0b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622badd79f0 0x5622baf76b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:47.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.906+0000 7f798a5b0b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622baf56170 0x5622baf770d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:47.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.906+0000 7f7979971700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622badd79f0 0x5622baf76b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:47.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.906+0000 7f7979971700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622badd79f0 0x5622baf76b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38046/0 (socket says 172.21.15.12:38046) 2024-05-17T22:45:47.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.906+0000 7f7979971700 1 -- 172.21.15.12:0/659254339 learned_addr learned my addr 172.21.15.12:0/659254339 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:45:47.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.906+0000 7f798a5b0b80 1 --2- 172.21.15.12:0/659254339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622baf5baa0 0x5622bafd8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:47.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.906+0000 7f7979170700 1 --2- 172.21.15.12:0/659254339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622baf56170 0x5622baf770d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:47.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.906+0000 7f798a5b0b80 1 -- 172.21.15.12:0/659254339 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f79741b8ee0 con 0x5622badd79f0 2024-05-17T22:45:47.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.906+0000 7f798a5b0b80 1 -- 172.21.15.12:0/659254339 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f79741ba350 con 0x5622baf56170 2024-05-17T22:45:47.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.906+0000 7f798a5b0b80 1 -- 172.21.15.12:0/659254339 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f79741b95b0 con 0x5622baf5baa0 2024-05-17T22:45:47.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.906+0000 7f798a5b0b80 10 client.0 _ll_register_callbacks cb 0x5622bae0ef10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:45:47.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.906+0000 7f797a172700 1 --2- 172.21.15.12:0/659254339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622baf5baa0 0x5622bafd8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:47.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.906+0000 7f7979170700 1 -- 172.21.15.12:0/659254339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622baf5baa0 msgr2=0x5622bafd8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:47.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.906+0000 7f7979170700 1 --2- 172.21.15.12:0/659254339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622baf5baa0 0x5622bafd8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:47.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.906+0000 7f7979170700 1 -- 172.21.15.12:0/659254339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622badd79f0 msgr2=0x5622baf76b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:45:47.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.906+0000 7f7979170700 1 --2- 172.21.15.12:0/659254339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622badd79f0 0x5622baf76b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:47.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.906+0000 7f7979170700 1 -- 172.21.15.12:0/659254339 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622baf77930 con 0x5622baf56170 2024-05-17T22:45:47.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.907+0000 7f797a172700 1 --2- 172.21.15.12:0/659254339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622baf5baa0 0x5622bafd8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:45:47.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.907+0000 7f7979170700 1 --2- 172.21.15.12:0/659254339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622baf56170 0x5622baf770d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f79640098e0 tx=0x7f7964009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc8f0e8135c9448a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:45:47.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.907+0000 7f795d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:45:47.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.907+0000 7f795d7fa700 1 -- 172.21.15.12:0/659254339 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7964037060 con 0x5622baf56170 2024-05-17T22:45:47.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.907+0000 7f795d7fa700 1 -- 172.21.15.12:0/659254339 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f7964032660 con 0x5622baf56170 2024-05-17T22:45:47.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.907+0000 7f798a5b0b80 1 -- 172.21.15.12:0/659254339 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79700b24e0 con 0x5622baf56170 2024-05-17T22:45:47.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.907+0000 7f795d7fa700 1 -- 172.21.15.12:0/659254339 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f796403e050 con 0x5622baf56170 2024-05-17T22:45:47.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.908+0000 7f798a5b0b80 10 client.25601 fetch_fsmap learned FSMap version 37 2024-05-17T22:45:47.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.908+0000 7f798a5b0b80 1 -- 172.21.15.12:0/659254339 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5622bafda0a0 con 0x5622baf56170 2024-05-17T22:45:47.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.908+0000 7f795d7fa700 1 -- 172.21.15.12:0/659254339 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f796400be50 con 0x5622baf56170 2024-05-17T22:45:47.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.908+0000 7f795d7fa700 1 -- 172.21.15.12:0/659254339 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f7964022070 con 0x5622baf56170 2024-05-17T22:45:47.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.908+0000 7f798a5b0b80 10 client.25601 fetch_fsmap finished waiting for FSMap version 37 2024-05-17T22:45:47.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.908+0000 7f798a5b0b80 10 client.25601 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:45:47.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.908+0000 7f798a5b0b80 20 client.25601 populate_metadata read hostname 'smithi012' 2024-05-17T22:45:47.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.908+0000 7f798a5b0b80 1 --2- 172.21.15.12:0/659254339 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5622baf6f390 0x5622bafdc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:45:47.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.908+0000 7f798a5b0b80 4 client.25601 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:45:47.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.908+0000 7f798a5b0b80 1 -- 172.21.15.12:0/659254339 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f79700b24e0 con 0x5622baf6f390 2024-05-17T22:45:47.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.908+0000 7f7979971700 1 --2- 172.21.15.12:0/659254339 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5622baf6f390 0x5622bafdc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:45:47.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.909+0000 7f795d7fa700 10 client.25601 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:45:47.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.909+0000 7f7979971700 1 --2- 172.21.15.12:0/659254339 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5622baf6f390 0x5622bafdc850 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8e24e8fbfa13076 server_cookie=c89f0f2362bca77 in_seq=0 out_seq=0 2024-05-17T22:45:47.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.909+0000 7f795d7fa700 1 -- 172.21.15.12:0/659254339 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f79700b24e0 con 0x5622baf6f390 2024-05-17T22:45:47.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.909+0000 7f795d7fa700 10 client.25601 handle_command_reply: tid=0 2024-05-17T22:45:47.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.909+0000 7f798a5b0b80 1 client.25601 shutdown 2024-05-17T22:45:47.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.910+0000 7f798a5b0b80 1 -- 172.21.15.12:0/659254339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622baf56170 msgr2=0x5622baf770d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:47.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.910+0000 7f798a5b0b80 1 --2- 172.21.15.12:0/659254339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622baf56170 0x5622baf770d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f79640098e0 tx=0x7f7964009ca0 comp rx=0 tx=0).stop 2024-05-17T22:45:47.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.911+0000 7f798a5b0b80 1 -- 172.21.15.12:0/659254339 shutdown_connections 2024-05-17T22:45:47.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.911+0000 7f798a5b0b80 1 --2- 172.21.15.12:0/659254339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622baf5baa0 0x5622bafd8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:47.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.911+0000 7f798a5b0b80 1 --2- 172.21.15.12:0/659254339 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5622baf6f390 0x5622bafdc850 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:47.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.911+0000 7f798a5b0b80 1 --2- 172.21.15.12:0/659254339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622badd79f0 0x5622baf76b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:47.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.911+0000 7f798a5b0b80 1 --2- 172.21.15.12:0/659254339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622baf56170 0x5622baf770d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:47.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.911+0000 7f798a5b0b80 1 -- 172.21.15.12:0/659254339 >> 172.21.15.12:0/659254339 conn(0x5622bae1bf00 msgr2=0x5622baf5cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:45:47.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.911+0000 7f795d7fa700 0 client.25601 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:45:47.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.911+0000 7f798a5b0b80 1 -- 172.21.15.12:0/659254339 shutdown_connections 2024-05-17T22:45:47.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.911+0000 7f798a5b0b80 1 -- 172.21.15.12:0/659254339 wait complete. 2024-05-17T22:45:47.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.911+0000 7f798a5b0b80 20 client.25601 trim_cache size 0 max 16384 2024-05-17T22:45:47.916 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:45:47.916 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-17T22:45:47.916 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-17T22:45:47.916 INFO:teuthology.orchestra.run.smithi012.stdout: "a05fdcd6-f2db-4e73-ba06-bc91cbe08e1c": { 2024-05-17T22:45:47.916 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-17T22:45:47.916 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "a05fdcd6-f2db-4e73-ba06-bc91cbe08e1c", 2024-05-17T22:45:47.916 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-17T22:45:47.916 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:45:47.916 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:45:47.917 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:45:47.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.912+0000 7f797bbd5700 1 -- 172.21.15.12:0/4241882103 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f79600b2900 msgr2=0x7f79600b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:47.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.912+0000 7f797bbd5700 1 --2- 172.21.15.12:0/4241882103 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f79600b2900 0x7f79600b4e10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7974119310 tx=0x7f796400b260 comp rx=0 tx=0).stop 2024-05-17T22:45:47.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.912+0000 7f797bbd5700 1 -- 172.21.15.12:0/4241882103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f797410aa00 msgr2=0x7f797410c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:45:47.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.912+0000 7f797bbd5700 1 --2- 172.21.15.12:0/4241882103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f797410aa00 0x7f797410c2a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f797000d950 tx=0x7f797000dc60 comp rx=0 tx=0).stop 2024-05-17T22:45:47.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.912+0000 7f797bbd5700 1 -- 172.21.15.12:0/4241882103 shutdown_connections 2024-05-17T22:45:47.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.912+0000 7f797bbd5700 1 --2- 172.21.15.12:0/4241882103 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f79600b2900 0x7f79600b4e10 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:47.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.912+0000 7f797bbd5700 1 --2- 172.21.15.12:0/4241882103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7974116a60 0x7f797410cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:47.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.912+0000 7f797bbd5700 1 --2- 172.21.15.12:0/4241882103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f797410aa00 0x7f797410c2a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:47.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.912+0000 7f797bbd5700 1 --2- 172.21.15.12:0/4241882103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f797410b350 0x7f797410c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:45:47.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.912+0000 7f797bbd5700 1 -- 172.21.15.12:0/4241882103 >> 172.21.15.12:0/4241882103 conn(0x7f797406efc0 msgr2=0x7f79740724c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:45:47.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.912+0000 7f797bbd5700 1 -- 172.21.15.12:0/4241882103 shutdown_connections 2024-05-17T22:45:47.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:45:47.912+0000 7f797bbd5700 1 -- 172.21.15.12:0/4241882103 wait complete. 2024-05-17T22:45:48.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:47 smithi151 ceph-mon[36101]: pgmap v635: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.8 MiB/s wr, 492 op/s 2024-05-17T22:45:48.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:47 smithi096 ceph-mon[36065]: pgmap v635: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.8 MiB/s wr, 492 op/s 2024-05-17T22:45:48.644 INFO:tasks.cephfs.filesystem:scrub status for tag:a05fdcd6-f2db-4e73-ba06-bc91cbe08e1c - {'path': '/', 'tag': 'a05fdcd6-f2db-4e73-ba06-bc91cbe08e1c', 'options': 'recursive,force'} 2024-05-17T22:45:50.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:49 smithi012 ceph-mon[31632]: pgmap v636: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 MiB/s wr, 609 op/s 2024-05-17T22:45:50.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:49 smithi151 ceph-mon[36101]: pgmap v636: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 MiB/s wr, 609 op/s 2024-05-17T22:45:50.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:49 smithi096 ceph-mon[36065]: pgmap v636: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 MiB/s wr, 609 op/s 2024-05-17T22:45:51.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 390.0 s, 573.4 tps, lat 174.148 ms stddev 213.500 2024-05-17T22:45:52.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:51 smithi151 ceph-mon[36101]: pgmap v637: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.9 MiB/s wr, 465 op/s 2024-05-17T22:45:52.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:51 smithi096 ceph-mon[36065]: pgmap v637: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.9 MiB/s wr, 465 op/s 2024-05-17T22:45:52.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:51 smithi012 ceph-mon[31632]: pgmap v637: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.9 MiB/s wr, 465 op/s 2024-05-17T22:45:54.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:53 smithi151 ceph-mon[36101]: pgmap v638: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.6 MiB/s wr, 536 op/s 2024-05-17T22:45:54.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:53 smithi151 ceph-mon[36101]: scrub summary: idle 2024-05-17T22:45:54.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:53 smithi096 ceph-mon[36065]: pgmap v638: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.6 MiB/s wr, 536 op/s 2024-05-17T22:45:54.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:53 smithi096 ceph-mon[36065]: scrub summary: idle 2024-05-17T22:45:54.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:53 smithi012 ceph-mon[31632]: pgmap v638: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.6 MiB/s wr, 536 op/s 2024-05-17T22:45:54.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:53 smithi012 ceph-mon[31632]: scrub summary: idle 2024-05-17T22:45:55.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:54 smithi151 ceph-mon[36101]: scrub summary: idle 2024-05-17T22:45:55.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:54 smithi151 ceph-mon[36101]: scrub summary: idle 2024-05-17T22:45:55.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:54 smithi096 ceph-mon[36065]: scrub summary: idle 2024-05-17T22:45:55.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:54 smithi096 ceph-mon[36065]: scrub summary: idle 2024-05-17T22:45:55.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:54 smithi012 ceph-mon[31632]: scrub summary: idle 2024-05-17T22:45:55.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:54 smithi012 ceph-mon[31632]: scrub summary: idle 2024-05-17T22:45:56.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:55 smithi151 ceph-mon[36101]: pgmap v639: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 MiB/s wr, 532 op/s 2024-05-17T22:45:56.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:55 smithi096 ceph-mon[36065]: pgmap v639: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 MiB/s wr, 532 op/s 2024-05-17T22:45:56.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:55 smithi012 ceph-mon[31632]: pgmap v639: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 MiB/s wr, 532 op/s 2024-05-17T22:45:56.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 395.0 s, 544.4 tps, lat 188.666 ms stddev 222.509 2024-05-17T22:45:58.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:57 smithi151 ceph-mon[36101]: pgmap v640: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.6 MiB/s wr, 479 op/s 2024-05-17T22:45:58.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:57 smithi096 ceph-mon[36065]: pgmap v640: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.6 MiB/s wr, 479 op/s 2024-05-17T22:45:58.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:57 smithi012 ceph-mon[31632]: pgmap v640: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.6 MiB/s wr, 479 op/s 2024-05-17T22:46:00.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:59 smithi151 ceph-mon[36101]: pgmap v641: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.3 MiB/s wr, 634 op/s 2024-05-17T22:46:00.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:45:59 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:46:00.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:59 smithi096 ceph-mon[36065]: pgmap v641: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.3 MiB/s wr, 634 op/s 2024-05-17T22:46:00.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:45:59 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:46:00.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:59 smithi012 ceph-mon[31632]: pgmap v641: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.3 MiB/s wr, 634 op/s 2024-05-17T22:46:00.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:45:59 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:46:01.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:46:01.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:46:01.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:46:01.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:00 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:46:01.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:00 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:46:01.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:00 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:46:01.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:00 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:46:01.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:00 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:46:01.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:00 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:46:01.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 400.0 s, 570.2 tps, lat 173.720 ms stddev 190.151 2024-05-17T22:46:02.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:01 smithi151 ceph-mon[36101]: pgmap v642: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.8 MiB/s wr, 500 op/s 2024-05-17T22:46:02.244 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:01 smithi012 ceph-mon[31632]: pgmap v642: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.8 MiB/s wr, 500 op/s 2024-05-17T22:46:02.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:01 smithi096 ceph-mon[36065]: pgmap v642: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.8 MiB/s wr, 500 op/s 2024-05-17T22:46:04.034 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:03 smithi012 ceph-mon[31632]: pgmap v643: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.7 MiB/s wr, 573 op/s 2024-05-17T22:46:04.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:03 smithi151 ceph-mon[36101]: pgmap v643: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.7 MiB/s wr, 573 op/s 2024-05-17T22:46:04.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:03 smithi096 ceph-mon[36065]: pgmap v643: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.7 MiB/s wr, 573 op/s 2024-05-17T22:46:06.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:05 smithi151 ceph-mon[36101]: pgmap v644: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 MiB/s wr, 780 op/s 2024-05-17T22:46:06.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:05 smithi096 ceph-mon[36065]: pgmap v644: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 MiB/s wr, 780 op/s 2024-05-17T22:46:06.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:05 smithi012 ceph-mon[31632]: pgmap v644: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 MiB/s wr, 780 op/s 2024-05-17T22:46:06.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 405.0 s, 664.2 tps, lat 150.057 ms stddev 188.990 2024-05-17T22:46:08.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:07 smithi151 ceph-mon[36101]: pgmap v645: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 844 op/s 2024-05-17T22:46:08.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:07 smithi096 ceph-mon[36065]: pgmap v645: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 844 op/s 2024-05-17T22:46:08.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:07 smithi012 ceph-mon[31632]: pgmap v645: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 844 op/s 2024-05-17T22:46:10.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:09 smithi151 ceph-mon[36101]: pgmap v646: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 14 MiB/s wr, 1.15k op/s 2024-05-17T22:46:10.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:09 smithi096 ceph-mon[36065]: pgmap v646: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 14 MiB/s wr, 1.15k op/s 2024-05-17T22:46:10.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:09 smithi012 ceph-mon[31632]: pgmap v646: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 14 MiB/s wr, 1.15k op/s 2024-05-17T22:46:11.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 410.0 s, 740.8 tps, lat 136.339 ms stddev 172.112 2024-05-17T22:46:12.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:11 smithi151 ceph-mon[36101]: pgmap v647: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 11 MiB/s wr, 1.01k op/s 2024-05-17T22:46:12.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:11 smithi096 ceph-mon[36065]: pgmap v647: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 11 MiB/s wr, 1.01k op/s 2024-05-17T22:46:12.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:11 smithi012 ceph-mon[31632]: pgmap v647: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 11 MiB/s wr, 1.01k op/s 2024-05-17T22:46:14.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:13 smithi151 ceph-mon[36101]: pgmap v648: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 12 MiB/s wr, 1.09k op/s 2024-05-17T22:46:14.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:13 smithi096 ceph-mon[36065]: pgmap v648: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 12 MiB/s wr, 1.09k op/s 2024-05-17T22:46:14.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:13 smithi012 ceph-mon[31632]: pgmap v648: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 12 MiB/s wr, 1.09k op/s 2024-05-17T22:46:16.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:15 smithi151 ceph-mon[36101]: pgmap v649: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 13 MiB/s wr, 1.12k op/s 2024-05-17T22:46:16.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:15 smithi096 ceph-mon[36065]: pgmap v649: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 13 MiB/s wr, 1.12k op/s 2024-05-17T22:46:16.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:15 smithi012 ceph-mon[31632]: pgmap v649: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 13 MiB/s wr, 1.12k op/s 2024-05-17T22:46:16.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 415.0 s, 769.2 tps, lat 129.270 ms stddev 156.987 2024-05-17T22:46:18.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:17 smithi151 ceph-mon[36101]: pgmap v650: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 10 MiB/s wr, 881 op/s 2024-05-17T22:46:18.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:17 smithi096 ceph-mon[36065]: pgmap v650: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 10 MiB/s wr, 881 op/s 2024-05-17T22:46:18.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:17 smithi012 ceph-mon[31632]: pgmap v650: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 10 MiB/s wr, 881 op/s 2024-05-17T22:46:18.645 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-17T22:46:19.174 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:46:19.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:19 smithi096 ceph-mon[36065]: pgmap v651: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 9.9 MiB/s wr, 816 op/s 2024-05-17T22:46:19.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:19 smithi012 ceph-mon[31632]: pgmap v651: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 9.9 MiB/s wr, 816 op/s 2024-05-17T22:46:19.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:19 smithi151 ceph-mon[36101]: pgmap v651: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 9.9 MiB/s wr, 816 op/s 2024-05-17T22:46:21.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 420.0 s, 693.8 tps, lat 141.008 ms stddev 182.239 2024-05-17T22:46:22.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:21 smithi012 ceph-mon[31632]: pgmap v652: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 6.1 MiB/s wr, 526 op/s 2024-05-17T22:46:22.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:21 smithi151 ceph-mon[36101]: pgmap v652: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 6.1 MiB/s wr, 526 op/s 2024-05-17T22:46:22.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:21 smithi096 ceph-mon[36065]: pgmap v652: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 6.1 MiB/s wr, 526 op/s 2024-05-17T22:46:23.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.281+0000 7fee910c4700 1 -- 172.21.15.12:0/1381715802 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee8c11d550 msgr2=0x7fee8c11f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:23.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.281+0000 7fee910c4700 1 --2- 172.21.15.12:0/1381715802 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee8c11d550 0x7fee8c11f970 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fee8400b0a0 tx=0x7fee8400b3b0 comp rx=0 tx=0).stop 2024-05-17T22:46:23.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.282+0000 7fee910c4700 1 -- 172.21.15.12:0/1381715802 shutdown_connections 2024-05-17T22:46:23.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.282+0000 7fee910c4700 1 --2- 172.21.15.12:0/1381715802 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee8c11d550 0x7fee8c11f970 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:23.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.282+0000 7fee910c4700 1 --2- 172.21.15.12:0/1381715802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee8c10a830 0x7fee8c10ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:23.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.282+0000 7fee910c4700 1 --2- 172.21.15.12:0/1381715802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee8c10b180 0x7fee8c11d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:23.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.282+0000 7fee910c4700 1 -- 172.21.15.12:0/1381715802 >> 172.21.15.12:0/1381715802 conn(0x7fee8c06efc0 msgr2=0x7fee8c06f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:46:23.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.285+0000 7fee910c4700 1 -- 172.21.15.12:0/1381715802 shutdown_connections 2024-05-17T22:46:23.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.285+0000 7fee910c4700 1 -- 172.21.15.12:0/1381715802 wait complete. 2024-05-17T22:46:23.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.286+0000 7fee910c4700 1 Processor -- start 2024-05-17T22:46:23.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.287+0000 7fee910c4700 1 -- start start 2024-05-17T22:46:23.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.287+0000 7fee910c4700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee8c10a830 0x7fee8c11ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:23.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.287+0000 7fee910c4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee8c10b180 0x7fee8c114c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:23.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.287+0000 7fee910c4700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee8c11d550 0x7fee8c115320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:23.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.287+0000 7fee910c4700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c122ce0 con 0x7fee8c10b180 2024-05-17T22:46:23.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.287+0000 7fee910c4700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c122b60 con 0x7fee8c10a830 2024-05-17T22:46:23.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.287+0000 7fee910c4700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c122e60 con 0x7fee8c11d550 2024-05-17T22:46:23.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.287+0000 7fee8b59e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee8c11d550 0x7fee8c115320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:23.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.287+0000 7fee8a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee8c10b180 0x7fee8c114c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:23.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.288+0000 7fee8b59e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee8c11d550 0x7fee8c115320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:52886/0 (socket says 172.21.15.12:52886) 2024-05-17T22:46:23.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.288+0000 7fee8b59e700 1 -- 172.21.15.12:0/2664958698 learned_addr learned my addr 172.21.15.12:0/2664958698 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:46:23.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.288+0000 7fee8b59e700 1 -- 172.21.15.12:0/2664958698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee8c10a830 msgr2=0x7fee8c11ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:23.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.288+0000 7fee8b59e700 1 --2- 172.21.15.12:0/2664958698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee8c10a830 0x7fee8c11ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:23.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.288+0000 7fee8b59e700 1 -- 172.21.15.12:0/2664958698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee8c10b180 msgr2=0x7fee8c114c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:23.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.288+0000 7fee8b59e700 1 --2- 172.21.15.12:0/2664958698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee8c10b180 0x7fee8c114c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:23.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.288+0000 7fee8b59e700 1 -- 172.21.15.12:0/2664958698 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee8c119600 con 0x7fee8c11d550 2024-05-17T22:46:23.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.288+0000 7fee8b59e700 1 --2- 172.21.15.12:0/2664958698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee8c11d550 0x7fee8c115320 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fee84009c40 tx=0x7fee8400cdb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:46:23.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.289+0000 7fee73fff700 1 -- 172.21.15.12:0/2664958698 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee84021070 con 0x7fee8c11d550 2024-05-17T22:46:23.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.289+0000 7fee73fff700 1 -- 172.21.15.12:0/2664958698 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee84003c00 con 0x7fee8c11d550 2024-05-17T22:46:23.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.289+0000 7fee73fff700 1 -- 172.21.15.12:0/2664958698 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee840047f0 con 0x7fee8c11d550 2024-05-17T22:46:23.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.289+0000 7fee910c4700 1 -- 172.21.15.12:0/2664958698 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee8c119890 con 0x7fee8c11d550 2024-05-17T22:46:23.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.289+0000 7fee910c4700 1 -- 172.21.15.12:0/2664958698 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee8c119d80 con 0x7fee8c11d550 2024-05-17T22:46:23.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.295+0000 7fee9fa9fb80 1 Processor -- start 2024-05-17T22:46:23.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.297+0000 7fee73fff700 1 -- 172.21.15.12:0/2664958698 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fee84007a00 con 0x7fee8c11d550 2024-05-17T22:46:23.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.298+0000 7fee9fa9fb80 1 -- start start 2024-05-17T22:46:23.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.299+0000 7fee73fff700 1 --2- 172.21.15.12:0/2664958698 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fee740b29d0 0x7fee740b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:23.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.299+0000 7fee73fff700 1 -- 172.21.15.12:0/2664958698 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fee8400f020 con 0x7fee8c11d550 2024-05-17T22:46:23.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.299+0000 7fee9fa9fb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea59be9f0 0x55dea5b42560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:23.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.299+0000 7fee9fa9fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dea5b42aa0 0x55dea5b3cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:23.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.300+0000 7fee9fa9fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dea5b3d170 0x55dea5b3d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:23.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.300+0000 7fee9fa9fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55dea5a6b290 con 0x55dea5b3d170 2024-05-17T22:46:23.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.300+0000 7fee9fa9fb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55dea5a5afd0 con 0x55dea59be9f0 2024-05-17T22:46:23.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.300+0000 7fee8b59e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dea5b3d170 0x55dea5b3d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:23.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.300+0000 7fee8b59e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dea5b3d170 0x55dea5b3d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48604/0 (socket says 172.21.15.12:48604) 2024-05-17T22:46:23.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.300+0000 7fee8b59e700 1 -- 172.21.15.12:0/4015270667 learned_addr learned my addr 172.21.15.12:0/4015270667 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:46:23.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.300+0000 7fee9fa9fb80 1 -- 172.21.15.12:0/4015270667 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55dea5a5ee70 con 0x55dea5b42aa0 2024-05-17T22:46:23.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.300+0000 7fee8a59c700 1 --2- 172.21.15.12:0/2664958698 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fee740b29d0 0x7fee740b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:23.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.301+0000 7fee8a59c700 1 --2- 172.21.15.12:0/4015270667 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dea5b42aa0 0x55dea5b3cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:23.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.301+0000 7fee8b59e700 1 -- 172.21.15.12:0/4015270667 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dea5b42aa0 msgr2=0x55dea5b3cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:46:23.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.301+0000 7fee8ad9d700 1 --2- 172.21.15.12:0/4015270667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea59be9f0 0x55dea5b42560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:23.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.301+0000 7fee8b59e700 1 --2- 172.21.15.12:0/4015270667 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dea5b42aa0 0x55dea5b3cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:23.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.301+0000 7fee8b59e700 1 -- 172.21.15.12:0/4015270667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea59be9f0 msgr2=0x55dea5b42560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:46:23.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.301+0000 7fee8b59e700 1 --2- 172.21.15.12:0/4015270667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea59be9f0 0x55dea5b42560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:23.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.301+0000 7fee8b59e700 1 -- 172.21.15.12:0/4015270667 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dea5a04590 con 0x55dea5b3d170 2024-05-17T22:46:23.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.301+0000 7fee8a59c700 1 --2- 172.21.15.12:0/4015270667 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dea5b42aa0 0x55dea5b3cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:46:23.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.301+0000 7fee8b59e700 1 --2- 172.21.15.12:0/4015270667 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dea5b3d170 0x55dea5b3d580 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fee840aa980 tx=0x7fee8400cfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=913f4b6bdfe671c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:46:23.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.302+0000 7fee8a59c700 1 --2- 172.21.15.12:0/2664958698 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fee740b29d0 0x7fee740b4ee0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55dea5b39320 tx=0x7fee8000a110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:46:23.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.305+0000 7fee717fa700 1 -- 172.21.15.12:0/4015270667 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee840af030 con 0x55dea5b3d170 2024-05-17T22:46:23.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.305+0000 7fee717fa700 1 -- 172.21.15.12:0/4015270667 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee840b3a50 con 0x55dea5b3d170 2024-05-17T22:46:23.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.305+0000 7fee717fa700 1 -- 172.21.15.12:0/4015270667 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8402d350 con 0x55dea5b3d170 2024-05-17T22:46:23.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.307+0000 7fee9fa9fb80 1 -- 172.21.15.12:0/4015270667 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dea5b3d170 msgr2=0x55dea5b3d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:23.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.307+0000 7fee9fa9fb80 1 --2- 172.21.15.12:0/4015270667 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dea5b3d170 0x55dea5b3d580 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fee840aa980 tx=0x7fee8400cfd0 comp rx=0 tx=0).stop 2024-05-17T22:46:23.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.307+0000 7fee9fa9fb80 1 -- 172.21.15.12:0/4015270667 shutdown_connections 2024-05-17T22:46:23.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.307+0000 7fee9fa9fb80 1 --2- 172.21.15.12:0/4015270667 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dea5b3d170 0x55dea5b3d580 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:23.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.307+0000 7fee9fa9fb80 1 --2- 172.21.15.12:0/4015270667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea59be9f0 0x55dea5b42560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:23.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.308+0000 7fee9fa9fb80 1 --2- 172.21.15.12:0/4015270667 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dea5b42aa0 0x55dea5b3cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:23.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.308+0000 7fee9fa9fb80 1 -- 172.21.15.12:0/4015270667 >> 172.21.15.12:0/4015270667 conn(0x55dea5a02f00 msgr2=0x55dea5b37b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:46:23.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.311+0000 7fee9fa9fb80 1 -- 172.21.15.12:0/4015270667 shutdown_connections 2024-05-17T22:46:23.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.311+0000 7fee9fa9fb80 1 -- 172.21.15.12:0/4015270667 wait complete. 2024-05-17T22:46:23.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.312+0000 7fee9fa9fb80 1 -- start start 2024-05-17T22:46:23.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.312+0000 7fee9fa9fb80 1 Processor -- start 2024-05-17T22:46:23.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.312+0000 7fee9fa9fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dea59be9f0 0x55dea5b46240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:23.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.312+0000 7fee9fa9fb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea5b3d170 0x55dea5b46780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:23.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.312+0000 7fee9fa9fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dea5b42aa0 0x55dea5b46d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:23.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.313+0000 7fee9fa9fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55dea5962150 con 0x55dea5b42aa0 2024-05-17T22:46:23.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.313+0000 7fee9fa9fb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55dea59fbe80 con 0x55dea5b3d170 2024-05-17T22:46:23.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.313+0000 7fee9fa9fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55dea5a6b290 con 0x55dea59be9f0 2024-05-17T22:46:23.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.313+0000 7fee9fa9fb80 10 client.0 _ll_register_callbacks cb 0x55dea59f5f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:46:23.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.313+0000 7fee8ad9d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dea59be9f0 0x55dea5b46240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:23.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.313+0000 7fee8ad9d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dea59be9f0 0x55dea5b46240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:52910/0 (socket says 172.21.15.12:52910) 2024-05-17T22:46:23.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.313+0000 7fee8a59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea5b3d170 0x55dea5b46780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:23.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.313+0000 7fee8ad9d700 1 -- 172.21.15.12:0/1513377133 learned_addr learned my addr 172.21.15.12:0/1513377133 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:46:23.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.313+0000 7fee8b59e700 1 --2- 172.21.15.12:0/1513377133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dea5b42aa0 0x55dea5b46d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:23.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.314+0000 7fee8ad9d700 1 -- 172.21.15.12:0/1513377133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea5b3d170 msgr2=0x55dea5b46780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:46:23.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.314+0000 7fee8ad9d700 1 --2- 172.21.15.12:0/1513377133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea5b3d170 0x55dea5b46780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:23.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.314+0000 7fee8ad9d700 1 -- 172.21.15.12:0/1513377133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dea5b42aa0 msgr2=0x55dea5b46d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:46:23.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.314+0000 7fee8ad9d700 1 --2- 172.21.15.12:0/1513377133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dea5b42aa0 0x55dea5b46d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:23.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.314+0000 7fee8ad9d700 1 -- 172.21.15.12:0/1513377133 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dea59ab480 con 0x55dea59be9f0 2024-05-17T22:46:23.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.314+0000 7fee6eff5700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:46:23.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.314+0000 7fee8ad9d700 1 --2- 172.21.15.12:0/1513377133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dea59be9f0 0x55dea5b46240 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fee7c010620 tx=0x7fee7c010930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af9a2c23a729ae13 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:46:23.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.315+0000 7fee6eff5700 1 -- 172.21.15.12:0/1513377133 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7c004a30 con 0x55dea59be9f0 2024-05-17T22:46:23.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.315+0000 7fee6eff5700 1 -- 172.21.15.12:0/1513377133 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fee7c004830 con 0x55dea59be9f0 2024-05-17T22:46:23.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.315+0000 7fee9fa9fb80 1 -- 172.21.15.12:0/1513377133 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee840b3a50 con 0x55dea59be9f0 2024-05-17T22:46:23.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.315+0000 7fee6eff5700 1 -- 172.21.15.12:0/1513377133 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fee7c004a30 con 0x55dea59be9f0 2024-05-17T22:46:23.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.316+0000 7fee9fa9fb80 10 client.25549 fetch_fsmap learned FSMap version 37 2024-05-17T22:46:23.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.316+0000 7fee9fa9fb80 1 -- 172.21.15.12:0/1513377133 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55dea5b53b60 con 0x55dea59be9f0 2024-05-17T22:46:23.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.316+0000 7fee6eff5700 1 -- 172.21.15.12:0/1513377133 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee840b3a50 con 0x55dea59be9f0 2024-05-17T22:46:23.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.316+0000 7fee6eff5700 1 -- 172.21.15.12:0/1513377133 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fee7c00f040 con 0x55dea59be9f0 2024-05-17T22:46:23.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.317+0000 7fee9fa9fb80 10 client.25549 fetch_fsmap finished waiting for FSMap version 37 2024-05-17T22:46:23.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.317+0000 7fee9fa9fb80 10 client.25549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:46:23.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.317+0000 7fee9fa9fb80 20 client.25549 populate_metadata read hostname 'smithi012' 2024-05-17T22:46:23.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.317+0000 7fee9fa9fb80 1 --2- 172.21.15.12:0/1513377133 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55dea5b561a0 0x55dea5b585e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:23.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.317+0000 7fee9fa9fb80 4 client.25549 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:46:23.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.317+0000 7fee9fa9fb80 1 -- 172.21.15.12:0/1513377133 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dea5a6d1e0 con 0x55dea5b561a0 2024-05-17T22:46:23.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.317+0000 7fee8a59c700 1 --2- 172.21.15.12:0/1513377133 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55dea5b561a0 0x55dea5b585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:23.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.317+0000 7fee6eff5700 10 client.25549 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:46:23.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.318+0000 7fee8a59c700 1 --2- 172.21.15.12:0/1513377133 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55dea5b561a0 0x55dea5b585e0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=963b581b4b3ff018 server_cookie=85a31e6519b6ccd1 in_seq=0 out_seq=0 2024-05-17T22:46:23.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.318+0000 7fee6eff5700 1 -- 172.21.15.12:0/1513377133 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dea5a6d1e0 con 0x55dea5b561a0 2024-05-17T22:46:23.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.318+0000 7fee6eff5700 10 client.25549 handle_command_reply: tid=0 2024-05-17T22:46:23.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.319+0000 7fee9fa9fb80 1 client.25549 shutdown 2024-05-17T22:46:23.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.319+0000 7fee9fa9fb80 1 -- 172.21.15.12:0/1513377133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dea59be9f0 msgr2=0x55dea5b46240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:23.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.319+0000 7fee9fa9fb80 1 --2- 172.21.15.12:0/1513377133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dea59be9f0 0x55dea5b46240 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fee7c010620 tx=0x7fee7c010930 comp rx=0 tx=0).stop 2024-05-17T22:46:23.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.320+0000 7fee9fa9fb80 1 -- 172.21.15.12:0/1513377133 shutdown_connections 2024-05-17T22:46:23.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.320+0000 7fee9fa9fb80 1 --2- 172.21.15.12:0/1513377133 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55dea5b561a0 0x55dea5b585e0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:23.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.320+0000 7fee9fa9fb80 1 --2- 172.21.15.12:0/1513377133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dea5b42aa0 0x55dea5b46d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:23.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.320+0000 7fee9fa9fb80 1 --2- 172.21.15.12:0/1513377133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dea59be9f0 0x55dea5b46240 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:23.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.321+0000 7fee9fa9fb80 1 --2- 172.21.15.12:0/1513377133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea5b3d170 0x55dea5b46780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:23.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.321+0000 7fee9fa9fb80 1 -- 172.21.15.12:0/1513377133 >> 172.21.15.12:0/1513377133 conn(0x55dea5a02f00 msgr2=0x55dea5b31ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:46:23.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.321+0000 7fee6eff5700 0 client.25549 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:46:23.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.321+0000 7fee9fa9fb80 1 -- 172.21.15.12:0/1513377133 shutdown_connections 2024-05-17T22:46:23.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.321+0000 7fee9fa9fb80 1 -- 172.21.15.12:0/1513377133 wait complete. 2024-05-17T22:46:23.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.321+0000 7fee9fa9fb80 20 client.25549 trim_cache size 0 max 16384 2024-05-17T22:46:23.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.353+0000 7fee9fa9fb80 1 Processor -- start 2024-05-17T22:46:23.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.353+0000 7fee9fa9fb80 1 -- start start 2024-05-17T22:46:23.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.354+0000 7fee9fa9fb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea59be9f0 0x55dea5b3cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:23.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.354+0000 7fee9fa9fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dea5b3d170 0x55dea5b407a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:23.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.354+0000 7fee9fa9fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dea5b42aa0 0x55dea5b31eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:23.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.354+0000 7fee9fa9fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c1c9e00 con 0x55dea5b42aa0 2024-05-17T22:46:23.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.354+0000 7fee9fa9fb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c1c7690 con 0x55dea59be9f0 2024-05-17T22:46:23.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.354+0000 7fee9fa9fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c1c5480 con 0x55dea5b3d170 2024-05-17T22:46:23.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.357+0000 7fee8a59c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dea5b3d170 0x55dea5b407a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:23.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.357+0000 7fee8a59c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dea5b3d170 0x55dea5b407a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:52912/0 (socket says 172.21.15.12:52912) 2024-05-17T22:46:23.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.357+0000 7fee8a59c700 1 -- 172.21.15.12:0/2833927729 learned_addr learned my addr 172.21.15.12:0/2833927729 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:46:23.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.357+0000 7fee8ad9d700 1 --2- 172.21.15.12:0/2833927729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea59be9f0 0x55dea5b3cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:23.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.359+0000 7fee8b59e700 1 --2- 172.21.15.12:0/2833927729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dea5b42aa0 0x55dea5b31eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:23.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.362+0000 7fee8a59c700 1 -- 172.21.15.12:0/2833927729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea59be9f0 msgr2=0x55dea5b3cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:46:23.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.362+0000 7fee8a59c700 1 --2- 172.21.15.12:0/2833927729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea59be9f0 0x55dea5b3cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:23.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.362+0000 7fee8a59c700 1 -- 172.21.15.12:0/2833927729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dea5b42aa0 msgr2=0x55dea5b31eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:46:23.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.362+0000 7fee8a59c700 1 --2- 172.21.15.12:0/2833927729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dea5b42aa0 0x55dea5b31eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:23.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.362+0000 7fee8a59c700 1 -- 172.21.15.12:0/2833927729 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dea5b47330 con 0x55dea5b3d170 2024-05-17T22:46:23.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.364+0000 7fee8a59c700 1 --2- 172.21.15.12:0/2833927729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dea5b3d170 0x55dea5b407a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fee8c11bfe0 tx=0x7fee80012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd0645095926a054 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:46:23.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.365+0000 7fee70ff9700 1 -- 172.21.15.12:0/2833927729 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8002a070 con 0x55dea5b3d170 2024-05-17T22:46:23.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.366+0000 7fee70ff9700 1 -- 172.21.15.12:0/2833927729 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee800238d0 con 0x55dea5b3d170 2024-05-17T22:46:23.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.366+0000 7fee70ff9700 1 -- 172.21.15.12:0/2833927729 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee800137a0 con 0x55dea5b3d170 2024-05-17T22:46:23.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.368+0000 7fee9fa9fb80 1 -- 172.21.15.12:0/2833927729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dea5b3d170 msgr2=0x55dea5b407a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:23.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.368+0000 7fee9fa9fb80 1 --2- 172.21.15.12:0/2833927729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dea5b3d170 0x55dea5b407a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fee8c11bfe0 tx=0x7fee80012000 comp rx=0 tx=0).stop 2024-05-17T22:46:23.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.373+0000 7fee9fa9fb80 1 -- 172.21.15.12:0/2833927729 shutdown_connections 2024-05-17T22:46:23.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.373+0000 7fee9fa9fb80 1 --2- 172.21.15.12:0/2833927729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dea5b42aa0 0x55dea5b31eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:23.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.373+0000 7fee9fa9fb80 1 --2- 172.21.15.12:0/2833927729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea59be9f0 0x55dea5b3cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:23.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.373+0000 7fee9fa9fb80 1 --2- 172.21.15.12:0/2833927729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dea5b3d170 0x55dea5b407a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:23.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.373+0000 7fee9fa9fb80 1 -- 172.21.15.12:0/2833927729 >> 172.21.15.12:0/2833927729 conn(0x55dea5a02f00 msgr2=0x55dea5b43190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:46:23.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.377+0000 7fee9fa9fb80 1 -- 172.21.15.12:0/2833927729 shutdown_connections 2024-05-17T22:46:23.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.378+0000 7fee9fa9fb80 1 -- 172.21.15.12:0/2833927729 wait complete. 2024-05-17T22:46:23.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.378+0000 7fee9fa9fb80 1 -- start start 2024-05-17T22:46:23.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.378+0000 7fee9fa9fb80 1 Processor -- start 2024-05-17T22:46:23.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.379+0000 7fee9fa9fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dea59be9f0 0x55dea5b5d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:23.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.379+0000 7fee9fa9fb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea5b3d170 0x55dea5b5dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:23.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.379+0000 7fee9fa9fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dea5b42aa0 0x55dea5b5e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:23.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.379+0000 7fee9fa9fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c1c5b50 con 0x55dea5b42aa0 2024-05-17T22:46:23.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.379+0000 7fee9fa9fb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c1c6fc0 con 0x55dea5b3d170 2024-05-17T22:46:23.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.379+0000 7fee9fa9fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c1c6220 con 0x55dea59be9f0 2024-05-17T22:46:23.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.379+0000 7fee9fa9fb80 10 client.0 _ll_register_callbacks cb 0x55dea59f5f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:46:23.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.382+0000 7fee8a59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea5b3d170 0x55dea5b5dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:23.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.382+0000 7fee8ad9d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dea59be9f0 0x55dea5b5d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:23.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.382+0000 7fee8ad9d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dea59be9f0 0x55dea5b5d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:52918/0 (socket says 172.21.15.12:52918) 2024-05-17T22:46:23.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.382+0000 7fee8ad9d700 1 -- 172.21.15.12:0/2306405193 learned_addr learned my addr 172.21.15.12:0/2306405193 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:46:23.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.384+0000 7fee8b59e700 1 --2- 172.21.15.12:0/2306405193 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dea5b42aa0 0x55dea5b5e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:23.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.389+0000 7fee8a59c700 1 -- 172.21.15.12:0/2306405193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dea59be9f0 msgr2=0x55dea5b5d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:46:23.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.389+0000 7fee8a59c700 1 --2- 172.21.15.12:0/2306405193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dea59be9f0 0x55dea5b5d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:23.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.389+0000 7fee8a59c700 1 -- 172.21.15.12:0/2306405193 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dea5b42aa0 msgr2=0x55dea5b5e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:46:23.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.389+0000 7fee8a59c700 1 --2- 172.21.15.12:0/2306405193 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dea5b42aa0 0x55dea5b5e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:23.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.389+0000 7fee8a59c700 1 -- 172.21.15.12:0/2306405193 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dea5bbf950 con 0x55dea5b3d170 2024-05-17T22:46:23.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.389+0000 7fee8a59c700 1 --2- 172.21.15.12:0/2306405193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea5b3d170 0x55dea5b5dbb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fee8000a490 tx=0x7fee80023420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c567f0023fdfc18e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:46:23.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.389+0000 7fee6eff5700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:46:23.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.394+0000 7fee6eff5700 1 -- 172.21.15.12:0/2306405193 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8002a040 con 0x55dea5b3d170 2024-05-17T22:46:23.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.394+0000 7fee6eff5700 1 -- 172.21.15.12:0/2306405193 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fee80023690 con 0x55dea5b3d170 2024-05-17T22:46:23.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.395+0000 7fee9fa9fb80 1 -- 172.21.15.12:0/2306405193 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee800238d0 con 0x55dea5b3d170 2024-05-17T22:46:23.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.400+0000 7fee6eff5700 1 -- 172.21.15.12:0/2306405193 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fee80023a80 con 0x55dea5b3d170 2024-05-17T22:46:23.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.400+0000 7fee9fa9fb80 10 client.25610 fetch_fsmap learned FSMap version 37 2024-05-17T22:46:23.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.400+0000 7fee9fa9fb80 1 -- 172.21.15.12:0/2306405193 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55dea5bc0bc0 con 0x55dea5b3d170 2024-05-17T22:46:23.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.403+0000 7fee6eff5700 1 -- 172.21.15.12:0/2306405193 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee800238d0 con 0x55dea5b3d170 2024-05-17T22:46:23.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.403+0000 7fee6eff5700 1 -- 172.21.15.12:0/2306405193 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fee8002d030 con 0x55dea5b3d170 2024-05-17T22:46:23.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.403+0000 7fee9fa9fb80 10 client.25610 fetch_fsmap finished waiting for FSMap version 37 2024-05-17T22:46:23.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.403+0000 7fee9fa9fb80 10 client.25610 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:46:23.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.403+0000 7fee9fa9fb80 20 client.25610 populate_metadata read hostname 'smithi012' 2024-05-17T22:46:23.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.403+0000 7fee9fa9fb80 1 --2- 172.21.15.12:0/2306405193 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55dea5bc1250 0x55dea5bc3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:23.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.403+0000 7fee9fa9fb80 4 client.25610 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:46:23.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.403+0000 7fee9fa9fb80 1 -- 172.21.15.12:0/2306405193 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dea5a669c0 con 0x55dea5bc1250 2024-05-17T22:46:23.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.403+0000 7fee8ad9d700 1 --2- 172.21.15.12:0/2306405193 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55dea5bc1250 0x55dea5bc3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:23.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.404+0000 7fee6eff5700 10 client.25610 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:46:23.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.404+0000 7fee8ad9d700 1 --2- 172.21.15.12:0/2306405193 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55dea5bc1250 0x55dea5bc3670 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bfeb0318bc8f33c server_cookie=3160970db3384859 in_seq=0 out_seq=0 2024-05-17T22:46:23.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.408+0000 7fee6eff5700 1 -- 172.21.15.12:0/2306405193 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55dea5a669c0 con 0x55dea5bc1250 2024-05-17T22:46:23.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.408+0000 7fee6eff5700 10 client.25610 handle_command_reply: tid=0 2024-05-17T22:46:23.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.409+0000 7fee9fa9fb80 1 client.25610 shutdown 2024-05-17T22:46:23.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.411+0000 7fee9fa9fb80 1 -- 172.21.15.12:0/2306405193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea5b3d170 msgr2=0x55dea5b5dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:23.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.411+0000 7fee9fa9fb80 1 --2- 172.21.15.12:0/2306405193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea5b3d170 0x55dea5b5dbb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fee8000a490 tx=0x7fee80023420 comp rx=0 tx=0).stop 2024-05-17T22:46:23.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.411+0000 7fee9fa9fb80 1 -- 172.21.15.12:0/2306405193 shutdown_connections 2024-05-17T22:46:23.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.411+0000 7fee9fa9fb80 1 --2- 172.21.15.12:0/2306405193 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55dea5bc1250 0x55dea5bc3670 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:23.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.411+0000 7fee9fa9fb80 1 --2- 172.21.15.12:0/2306405193 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dea5b42aa0 0x55dea5b5e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:23.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.411+0000 7fee9fa9fb80 1 --2- 172.21.15.12:0/2306405193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dea59be9f0 0x55dea5b5d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:23.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.411+0000 7fee9fa9fb80 1 --2- 172.21.15.12:0/2306405193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea5b3d170 0x55dea5b5dbb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:23.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.411+0000 7fee9fa9fb80 1 -- 172.21.15.12:0/2306405193 >> 172.21.15.12:0/2306405193 conn(0x55dea5a02f00 msgr2=0x55dea5b40570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:46:23.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.411+0000 7fee6eff5700 0 client.25610 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:46:23.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.412+0000 7fee9fa9fb80 1 -- 172.21.15.12:0/2306405193 shutdown_connections 2024-05-17T22:46:23.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.412+0000 7fee9fa9fb80 1 -- 172.21.15.12:0/2306405193 wait complete. 2024-05-17T22:46:23.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.413+0000 7fee9fa9fb80 20 client.25610 trim_cache size 0 max 16384 2024-05-17T22:46:23.415 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:46:23.415 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-17T22:46:23.416 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-17T22:46:23.416 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:46:23.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.416+0000 7fee910c4700 1 -- 172.21.15.12:0/2664958698 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fee740b29d0 msgr2=0x7fee740b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:23.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.416+0000 7fee910c4700 1 --2- 172.21.15.12:0/2664958698 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fee740b29d0 0x7fee740b4ee0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55dea5b39320 tx=0x7fee8000a110 comp rx=0 tx=0).stop 2024-05-17T22:46:23.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.416+0000 7fee910c4700 1 -- 172.21.15.12:0/2664958698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee8c11d550 msgr2=0x7fee8c115320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:23.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.416+0000 7fee910c4700 1 --2- 172.21.15.12:0/2664958698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee8c11d550 0x7fee8c115320 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fee84009c40 tx=0x7fee8400cdb0 comp rx=0 tx=0).stop 2024-05-17T22:46:23.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.421+0000 7fee910c4700 1 -- 172.21.15.12:0/2664958698 shutdown_connections 2024-05-17T22:46:23.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.422+0000 7fee910c4700 1 --2- 172.21.15.12:0/2664958698 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fee740b29d0 0x7fee740b4ee0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:23.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.422+0000 7fee910c4700 1 --2- 172.21.15.12:0/2664958698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee8c11d550 0x7fee8c115320 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:23.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.422+0000 7fee910c4700 1 --2- 172.21.15.12:0/2664958698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee8c10a830 0x7fee8c11ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:23.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.422+0000 7fee910c4700 1 --2- 172.21.15.12:0/2664958698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee8c10b180 0x7fee8c114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:23.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.422+0000 7fee910c4700 1 -- 172.21.15.12:0/2664958698 >> 172.21.15.12:0/2664958698 conn(0x7fee8c06efc0 msgr2=0x7fee8c072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:46:23.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.422+0000 7fee910c4700 1 -- 172.21.15.12:0/2664958698 shutdown_connections 2024-05-17T22:46:23.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:23.422+0000 7fee910c4700 1 -- 172.21.15.12:0/2664958698 wait complete. 2024-05-17T22:46:24.083 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:23 smithi012 ceph-mon[31632]: pgmap v653: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 6.2 MiB/s wr, 553 op/s 2024-05-17T22:46:24.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:23 smithi151 ceph-mon[36101]: pgmap v653: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 6.2 MiB/s wr, 553 op/s 2024-05-17T22:46:24.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:23 smithi096 ceph-mon[36065]: pgmap v653: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 6.2 MiB/s wr, 553 op/s 2024-05-17T22:46:25.202 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-17T22:46:25.203 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T22:46:25.602 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:46:26.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:25 smithi012 ceph-mon[31632]: pgmap v654: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.6 MiB/s wr, 574 op/s 2024-05-17T22:46:26.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:25 smithi151 ceph-mon[36101]: pgmap v654: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.6 MiB/s wr, 574 op/s 2024-05-17T22:46:26.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:25 smithi096 ceph-mon[36065]: pgmap v654: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.6 MiB/s wr, 574 op/s 2024-05-17T22:46:26.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 425.0 s, 493.6 tps, lat 203.702 ms stddev 277.162 2024-05-17T22:46:27.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:27.776+0000 7f14b9359700 1 -- 172.21.15.12:0/433358962 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14b4110710 msgr2=0x7f14b4110b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:27.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:27.776+0000 7f14b9359700 1 --2- 172.21.15.12:0/433358962 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14b4110710 0x7f14b4110b20 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f14a400fb90 tx=0x7f14a400fea0 comp rx=0 tx=0).stop 2024-05-17T22:46:27.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:27.782+0000 7f14b9359700 1 -- 172.21.15.12:0/433358962 shutdown_connections 2024-05-17T22:46:27.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:27.782+0000 7f14b9359700 1 --2- 172.21.15.12:0/433358962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14b410b180 0x7f14b4117540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:27.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:27.782+0000 7f14b9359700 1 --2- 172.21.15.12:0/433358962 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14b4110710 0x7f14b4110b20 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:27.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:27.782+0000 7f14b9359700 1 --2- 172.21.15.12:0/433358962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14b410a830 0x7f14b410ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:27.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:27.782+0000 7f14b9359700 1 -- 172.21.15.12:0/433358962 >> 172.21.15.12:0/433358962 conn(0x7f14b406f420 msgr2=0x7f14b406f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:46:27.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:27.783+0000 7f14b9359700 1 -- 172.21.15.12:0/433358962 shutdown_connections 2024-05-17T22:46:27.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:27.784+0000 7f14b9359700 1 -- 172.21.15.12:0/433358962 wait complete. 2024-05-17T22:46:27.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:27.785+0000 7f14b9359700 1 Processor -- start 2024-05-17T22:46:27.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:27.786+0000 7f14b9359700 1 -- start start 2024-05-17T22:46:27.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:27.787+0000 7f14b9359700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14b410a830 0x7f14b41a95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:27.787+0000 7f14b9359700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14b410b180 0x7f14b41a9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:27.787+0000 7f14b9359700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14b4110710 0x7f14b41adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:27.787+0000 7f14b9359700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f14b411a4c0 con 0x7f14b410a830 2024-05-17T22:46:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:27.787+0000 7f14b9359700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f14b411a340 con 0x7f14b410b180 2024-05-17T22:46:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:27.787+0000 7f14b9359700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f14b411a640 con 0x7f14b4110710 2024-05-17T22:46:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:27.788+0000 7f14b37fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14b4110710 0x7f14b41adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:27.788+0000 7f14b37fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14b4110710 0x7f14b41adde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:52942/0 (socket says 172.21.15.12:52942) 2024-05-17T22:46:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:27.788+0000 7f14b37fe700 1 -- 172.21.15.12:0/3493948456 learned_addr learned my addr 172.21.15.12:0/3493948456 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:46:27.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:27.791+0000 7f14b2ffd700 1 --2- 172.21.15.12:0/3493948456 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14b410a830 0x7f14b41a95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:27.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:27.792+0000 7f14b27fc700 1 --2- 172.21.15.12:0/3493948456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14b410b180 0x7f14b41a9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:27.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:27.792+0000 7f14b2ffd700 1 -- 172.21.15.12:0/3493948456 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14b4110710 msgr2=0x7f14b41adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:27.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:27.792+0000 7f14b2ffd700 1 --2- 172.21.15.12:0/3493948456 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14b4110710 0x7f14b41adde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:27.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:27.792+0000 7f14b2ffd700 1 -- 172.21.15.12:0/3493948456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14b410b180 msgr2=0x7f14b41a9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:27.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:27.792+0000 7f14b2ffd700 1 --2- 172.21.15.12:0/3493948456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14b410b180 0x7f14b41a9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:27.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:27.792+0000 7f14b2ffd700 1 -- 172.21.15.12:0/3493948456 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b41ae480 con 0x7f14b410a830 2024-05-17T22:46:27.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:27.793+0000 7f14b2ffd700 1 --2- 172.21.15.12:0/3493948456 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14b410a830 0x7f14b41a95b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f14a4009720 tx=0x7f14a400e700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:46:27.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:27.794+0000 7f149bfff700 1 -- 172.21.15.12:0/3493948456 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a401c070 con 0x7f14b410a830 2024-05-17T22:46:27.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:27.794+0000 7f14b9359700 1 -- 172.21.15.12:0/3493948456 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14b41ae710 con 0x7f14b410a830 2024-05-17T22:46:27.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:27.794+0000 7f14b9359700 1 -- 172.21.15.12:0/3493948456 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14b41aeba0 con 0x7f14b410a830 2024-05-17T22:46:27.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:27.794+0000 7f149bfff700 1 -- 172.21.15.12:0/3493948456 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14a400edd0 con 0x7f14b410a830 2024-05-17T22:46:27.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:27.794+0000 7f149bfff700 1 -- 172.21.15.12:0/3493948456 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a40038e0 con 0x7f14b410a830 2024-05-17T22:46:27.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:27.800+0000 7f149bfff700 1 -- 172.21.15.12:0/3493948456 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f14a4003a80 con 0x7f14b410a830 2024-05-17T22:46:27.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:27.801+0000 7f14b9359700 1 -- 172.21.15.12:0/3493948456 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14a0005330 con 0x7f14b410a830 2024-05-17T22:46:27.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:27.802+0000 7f149bfff700 1 --2- 172.21.15.12:0/3493948456 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f149c0b2a90 0x7f149c0b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:27.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:27.802+0000 7f149bfff700 1 -- 172.21.15.12:0/3493948456 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f14a40e2d50 con 0x7f14b410a830 2024-05-17T22:46:27.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:27.809+0000 7f14b27fc700 1 --2- 172.21.15.12:0/3493948456 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f149c0b2a90 0x7f149c0b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:27.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:27.812+0000 7f149bfff700 1 -- 172.21.15.12:0/3493948456 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f14a40ab640 con 0x7f14b410a830 2024-05-17T22:46:27.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:27.815+0000 7f14b27fc700 1 --2- 172.21.15.12:0/3493948456 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f149c0b2a90 0x7f149c0b4fa0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f14b4073350 tx=0x7f14a800c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:46:28.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:27 smithi012 ceph-mon[31632]: pgmap v655: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 5.1 MiB/s wr, 498 op/s 2024-05-17T22:46:28.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:27 smithi151 ceph-mon[36101]: pgmap v655: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 5.1 MiB/s wr, 498 op/s 2024-05-17T22:46:28.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:27 smithi096 ceph-mon[36065]: pgmap v655: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 5.1 MiB/s wr, 498 op/s 2024-05-17T22:46:28.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:28.296+0000 7f14b9359700 1 -- 172.21.15.12:0/3493948456 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14a0005a40 con 0x7f14b410a830 2024-05-17T22:46:28.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:28.298+0000 7f149bfff700 1 -- 172.21.15.12:0/3493948456 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11873 (secure 0 0 0) 0x7f14a40aaf40 con 0x7f14b410a830 2024-05-17T22:46:28.301 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:46:28.301 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1407219319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1407219319},{"type":"v1","addr":"172.21.15.12:6835","nonce":1407219319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/630977886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":630977886},{"type":"v1","addr":"172.21.15.12:6837","nonce":630977886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3731914742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3731914742},{"type":"v1","addr":"172.21.15.12:6839","nonce":3731914742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/3642142501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3642142501},{"type":"v1","addr":"172.21.15.12:6841","nonce":3642142501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3554053059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3554053059},{"type":"v1","addr":"172.21.15.151:6833","nonce":3554053059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/4228325569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":4228325569},{"type":"v1","addr":"172.21.15.96:6833","nonce":4228325569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T22:34:42.642111+0000","modified":"2024-05-17T22:44:59.930707+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24461,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/542323754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":542323754},{"type":"v1","addr":"172.21.15.151:6835","nonce":542323754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1758442232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1758442232},{"type":"v1","addr":"172.21.15.96:6835","nonce":1758442232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3403324038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3403324038},{"type":"v1","addr":"172.21.15.151:6837","nonce":3403324038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.151:6839/2338353849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2338353849},{"type":"v1","addr":"172.21.15.151:6839","nonce":2338353849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.96:6837/4222574338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":4222574338},{"type":"v1","addr":"172.21.15.96:6837","nonce":4222574338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/2986295519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2986295519},{"type":"v1","addr":"172.21.15.96:6839","nonce":2986295519}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24457,24461,24482]},"id":1}]} 2024-05-17T22:46:28.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:28.302+0000 7f14b9359700 1 -- 172.21.15.12:0/3493948456 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f149c0b2a90 msgr2=0x7f149c0b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:28.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:28.302+0000 7f14b9359700 1 --2- 172.21.15.12:0/3493948456 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f149c0b2a90 0x7f149c0b4fa0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f14b4073350 tx=0x7f14a800c040 comp rx=0 tx=0).stop 2024-05-17T22:46:28.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:28.302+0000 7f14b9359700 1 -- 172.21.15.12:0/3493948456 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14b410a830 msgr2=0x7f14b41a95b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:28.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:28.302+0000 7f14b9359700 1 --2- 172.21.15.12:0/3493948456 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14b410a830 0x7f14b41a95b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f14a4009720 tx=0x7f14a400e700 comp rx=0 tx=0).stop 2024-05-17T22:46:28.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:28.305+0000 7f14b9359700 1 -- 172.21.15.12:0/3493948456 shutdown_connections 2024-05-17T22:46:28.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:28.305+0000 7f14b9359700 1 --2- 172.21.15.12:0/3493948456 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f149c0b2a90 0x7f149c0b4fa0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:28.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:28.305+0000 7f14b9359700 1 --2- 172.21.15.12:0/3493948456 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14b4110710 0x7f14b41adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:28.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:28.305+0000 7f14b9359700 1 --2- 172.21.15.12:0/3493948456 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14b410a830 0x7f14b41a95b0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:28.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:28.305+0000 7f14b9359700 1 --2- 172.21.15.12:0/3493948456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14b410b180 0x7f14b41a9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:28.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:28.305+0000 7f14b9359700 1 -- 172.21.15.12:0/3493948456 >> 172.21.15.12:0/3493948456 conn(0x7f14b406f420 msgr2=0x7f14b4073e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:46:28.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:28.309+0000 7f14b9359700 1 -- 172.21.15.12:0/3493948456 shutdown_connections 2024-05-17T22:46:28.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:28.311+0000 7f14b9359700 1 -- 172.21.15.12:0/3493948456 wait complete. 2024-05-17T22:46:28.323 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 37 2024-05-17T22:46:28.979 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:28 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/3493948456' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:46:29.033 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-17T22:46:29.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:28 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/3493948456' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:46:29.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:28 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/3493948456' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:46:29.558 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:46:30.022 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:29 smithi012 ceph-mon[31632]: pgmap v656: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.3 MiB/s wr, 592 op/s 2024-05-17T22:46:30.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:29 smithi151 ceph-mon[36101]: pgmap v656: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.3 MiB/s wr, 592 op/s 2024-05-17T22:46:30.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:29 smithi096 ceph-mon[36065]: pgmap v656: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.3 MiB/s wr, 592 op/s 2024-05-17T22:46:31.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 430.0 s, 702.2 tps, lat 145.810 ms stddev 195.001 2024-05-17T22:46:32.022 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:31 smithi012 ceph-mon[31632]: pgmap v657: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.1 MiB/s wr, 507 op/s 2024-05-17T22:46:32.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:31 smithi151 ceph-mon[36101]: pgmap v657: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.1 MiB/s wr, 507 op/s 2024-05-17T22:46:32.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:31 smithi096 ceph-mon[36065]: pgmap v657: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.1 MiB/s wr, 507 op/s 2024-05-17T22:46:32.502 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:46:32.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.605+0000 7f32f1ffb700 1 -- 172.21.15.12:0/1661393811 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ec012650 con 0x7f32f410b180 2024-05-17T22:46:32.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.606+0000 7f32f98ae700 1 -- 172.21.15.12:0/1661393811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32f410b180 msgr2=0x7f32f411d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:32.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.606+0000 7f32f98ae700 1 --2- 172.21.15.12:0/1661393811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32f410b180 0x7f32f411d010 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f32ec00b0a0 tx=0x7f32ec00b3b0 comp rx=0 tx=0).stop 2024-05-17T22:46:32.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.613+0000 7f32f98ae700 1 -- 172.21.15.12:0/1661393811 shutdown_connections 2024-05-17T22:46:32.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.613+0000 7f32f98ae700 1 --2- 172.21.15.12:0/1661393811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32f411d550 0x7f32f411f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:32.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.613+0000 7f32f98ae700 1 --2- 172.21.15.12:0/1661393811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32f410a830 0x7f32f410ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:32.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.613+0000 7f32f98ae700 1 --2- 172.21.15.12:0/1661393811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32f410b180 0x7f32f411d010 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:32.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.613+0000 7f32f98ae700 1 -- 172.21.15.12:0/1661393811 >> 172.21.15.12:0/1661393811 conn(0x7f32f406efc0 msgr2=0x7f32f406f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:46:32.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.614+0000 7f32f98ae700 1 -- 172.21.15.12:0/1661393811 shutdown_connections 2024-05-17T22:46:32.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.615+0000 7f32f98ae700 1 -- 172.21.15.12:0/1661393811 wait complete. 2024-05-17T22:46:32.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.616+0000 7f32f98ae700 1 Processor -- start 2024-05-17T22:46:32.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.618+0000 7f32f98ae700 1 -- start start 2024-05-17T22:46:32.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.619+0000 7f32f98ae700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32f410a830 0x7f32f411b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:32.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.619+0000 7f32f98ae700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32f410b180 0x7f32f4114a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:32.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.619+0000 7f32f98ae700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32f411d550 0x7f32f4114fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:32.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.619+0000 7f32f98ae700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f32f4122dd0 con 0x7f32f411d550 2024-05-17T22:46:32.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.619+0000 7f32f98ae700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f32f4122c50 con 0x7f32f410a830 2024-05-17T22:46:32.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.619+0000 7f32f98ae700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f32f4122f50 con 0x7f32f410b180 2024-05-17T22:46:32.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.620+0000 7f32f37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32f411d550 0x7f32f4114fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:32.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.620+0000 7f32f37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32f411d550 0x7f32f4114fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56804/0 (socket says 172.21.15.12:56804) 2024-05-17T22:46:32.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.620+0000 7f32f37fe700 1 -- 172.21.15.12:0/2786222418 learned_addr learned my addr 172.21.15.12:0/2786222418 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:46:32.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.620+0000 7f32f2ffd700 1 --2- 172.21.15.12:0/2786222418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32f410a830 0x7f32f411b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:32.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.620+0000 7f32f37fe700 1 -- 172.21.15.12:0/2786222418 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32f410b180 msgr2=0x7f32f4114a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:32.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.620+0000 7f32f37fe700 1 --2- 172.21.15.12:0/2786222418 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32f410b180 0x7f32f4114a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:32.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.620+0000 7f32f37fe700 1 -- 172.21.15.12:0/2786222418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32f410a830 msgr2=0x7f32f411b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:32.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.620+0000 7f32f37fe700 1 --2- 172.21.15.12:0/2786222418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32f410a830 0x7f32f411b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:32.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.620+0000 7f32f37fe700 1 -- 172.21.15.12:0/2786222418 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f41157f0 con 0x7f32f411d550 2024-05-17T22:46:32.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.620+0000 7f32f2ffd700 1 --2- 172.21.15.12:0/2786222418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32f410a830 0x7f32f411b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:46:32.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.621+0000 7f32f37fe700 1 --2- 172.21.15.12:0/2786222418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32f411d550 0x7f32f4114fb0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f32e400b4a0 tx=0x7f32e400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:46:32.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.621+0000 7f32f88ac700 1 -- 172.21.15.12:0/2786222418 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e4015020 con 0x7f32f411d550 2024-05-17T22:46:32.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.621+0000 7f32f98ae700 1 -- 172.21.15.12:0/2786222418 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32f41194d0 con 0x7f32f411d550 2024-05-17T22:46:32.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.621+0000 7f32f98ae700 1 -- 172.21.15.12:0/2786222418 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32f4119a60 con 0x7f32f411d550 2024-05-17T22:46:32.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.623+0000 7f32f88ac700 1 -- 172.21.15.12:0/2786222418 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32e4011ce0 con 0x7f32f411d550 2024-05-17T22:46:32.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.623+0000 7f32f88ac700 1 -- 172.21.15.12:0/2786222418 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e4015020 con 0x7f32f411d550 2024-05-17T22:46:32.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.635+0000 7f32f88ac700 1 -- 172.21.15.12:0/2786222418 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f32e4018020 con 0x7f32f411d550 2024-05-17T22:46:32.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.636+0000 7f32f88ac700 1 --2- 172.21.15.12:0/2786222418 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f32dc0b2d40 0x7f32dc0b5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:32.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.636+0000 7f32f88ac700 1 -- 172.21.15.12:0/2786222418 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f32e4011630 con 0x7f32f411d550 2024-05-17T22:46:32.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.638+0000 7f32f2ffd700 1 --2- 172.21.15.12:0/2786222418 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f32dc0b2d40 0x7f32dc0b5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:32.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.644+0000 7f32f2ffd700 1 --2- 172.21.15.12:0/2786222418 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f32dc0b2d40 0x7f32dc0b5250 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f32e8005fd0 tx=0x7f32e8005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:46:32.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.644+0000 7f3308289b80 1 Processor -- start 2024-05-17T22:46:32.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.644+0000 7f3308289b80 1 -- start start 2024-05-17T22:46:32.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.644+0000 7f3308289b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55767b7cd9f0 0x55767b951560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:32.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.644+0000 7f3308289b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55767b951aa0 0x55767b94bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:32.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.644+0000 7f3308289b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55767b94c170 0x55767b94c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:32.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.644+0000 7f3308289b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55767b87a290 con 0x55767b7cd9f0 2024-05-17T22:46:32.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.644+0000 7f3308289b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55767b869fd0 con 0x55767b94c170 2024-05-17T22:46:32.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.644+0000 7f3308289b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55767b86de70 con 0x55767b951aa0 2024-05-17T22:46:32.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.646+0000 7f32f27fc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55767b951aa0 0x55767b94bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:32.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.646+0000 7f32f27fc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55767b951aa0 0x55767b94bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:47100/0 (socket says 172.21.15.12:47100) 2024-05-17T22:46:32.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.646+0000 7f32f27fc700 1 -- 172.21.15.12:0/3674337964 learned_addr learned my addr 172.21.15.12:0/3674337964 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:46:32.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.647+0000 7f32f27fc700 1 -- 172.21.15.12:0/3674337964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55767b94c170 msgr2=0x55767b94c580 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T22:46:32.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.647+0000 7f32f27fc700 1 --2- 172.21.15.12:0/3674337964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55767b94c170 0x55767b94c580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:32.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.647+0000 7f32f27fc700 1 -- 172.21.15.12:0/3674337964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55767b7cd9f0 msgr2=0x55767b951560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:46:32.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.647+0000 7f32f27fc700 1 --2- 172.21.15.12:0/3674337964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55767b7cd9f0 0x55767b951560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:32.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.647+0000 7f32f27fc700 1 -- 172.21.15.12:0/3674337964 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55767b813590 con 0x55767b951aa0 2024-05-17T22:46:32.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.650+0000 7f32f27fc700 1 --2- 172.21.15.12:0/3674337964 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55767b951aa0 0x55767b94bc30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f32ec00b070 tx=0x7f32ec009880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9925575d60828229 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:46:32.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.653+0000 7f32d9ffb700 1 -- 172.21.15.12:0/3674337964 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ec0092d0 con 0x55767b951aa0 2024-05-17T22:46:32.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.653+0000 7f32d9ffb700 1 -- 172.21.15.12:0/3674337964 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32ec0075c0 con 0x55767b951aa0 2024-05-17T22:46:32.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.653+0000 7f32d9ffb700 1 -- 172.21.15.12:0/3674337964 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ec01d690 con 0x55767b951aa0 2024-05-17T22:46:32.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.656+0000 7f3308289b80 1 -- 172.21.15.12:0/3674337964 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55767b951aa0 msgr2=0x55767b94bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:32.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.656+0000 7f3308289b80 1 --2- 172.21.15.12:0/3674337964 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55767b951aa0 0x55767b94bc30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f32ec00b070 tx=0x7f32ec009880 comp rx=0 tx=0).stop 2024-05-17T22:46:32.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.662+0000 7f3308289b80 1 -- 172.21.15.12:0/3674337964 shutdown_connections 2024-05-17T22:46:32.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.662+0000 7f3308289b80 1 --2- 172.21.15.12:0/3674337964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55767b94c170 0x55767b94c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:32.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.662+0000 7f3308289b80 1 --2- 172.21.15.12:0/3674337964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55767b7cd9f0 0x55767b951560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:32.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.662+0000 7f3308289b80 1 --2- 172.21.15.12:0/3674337964 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55767b951aa0 0x55767b94bc30 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:32.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.662+0000 7f3308289b80 1 -- 172.21.15.12:0/3674337964 >> 172.21.15.12:0/3674337964 conn(0x55767b811f00 msgr2=0x55767b946b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:46:32.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.666+0000 7f3308289b80 1 -- 172.21.15.12:0/3674337964 shutdown_connections 2024-05-17T22:46:32.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.666+0000 7f3308289b80 1 -- 172.21.15.12:0/3674337964 wait complete. 2024-05-17T22:46:32.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.667+0000 7f3308289b80 1 -- start start 2024-05-17T22:46:32.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.667+0000 7f3308289b80 1 Processor -- start 2024-05-17T22:46:32.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.668+0000 7f3308289b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55767b7cd9f0 0x55767b955340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:32.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.668+0000 7f3308289b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55767b94c170 0x55767b955880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:32.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.668+0000 7f3308289b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55767b951aa0 0x55767b9616c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:32.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.668+0000 7f3308289b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55767b771150 con 0x55767b7cd9f0 2024-05-17T22:46:32.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.668+0000 7f3308289b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55767b80ae80 con 0x55767b94c170 2024-05-17T22:46:32.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.668+0000 7f3308289b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55767b958390 con 0x55767b951aa0 2024-05-17T22:46:32.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.668+0000 7f3308289b80 10 client.0 _ll_register_callbacks cb 0x55767b804f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:46:32.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.668+0000 7f32f2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55767b7cd9f0 0x55767b955340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:32.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.668+0000 7f32f27fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55767b94c170 0x55767b955880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:32.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.668+0000 7f32f2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55767b7cd9f0 0x55767b955340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56810/0 (socket says 172.21.15.12:56810) 2024-05-17T22:46:32.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.668+0000 7f32f2ffd700 1 -- 172.21.15.12:0/2355402636 learned_addr learned my addr 172.21.15.12:0/2355402636 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:46:32.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.668+0000 7f32f37fe700 1 --2- 172.21.15.12:0/2355402636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55767b951aa0 0x55767b9616c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:32.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.669+0000 7f32f2ffd700 1 -- 172.21.15.12:0/2355402636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55767b951aa0 msgr2=0x55767b9616c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:46:32.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.669+0000 7f32f2ffd700 1 --2- 172.21.15.12:0/2355402636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55767b951aa0 0x55767b9616c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:32.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.669+0000 7f32f2ffd700 1 -- 172.21.15.12:0/2355402636 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55767b94c170 msgr2=0x55767b955880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:46:32.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.669+0000 7f32f2ffd700 1 --2- 172.21.15.12:0/2355402636 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55767b94c170 0x55767b955880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:32.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.669+0000 7f32f2ffd700 1 -- 172.21.15.12:0/2355402636 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55767b7ba480 con 0x55767b7cd9f0 2024-05-17T22:46:32.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.669+0000 7f32f2ffd700 1 --2- 172.21.15.12:0/2355402636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55767b7cd9f0 0x55767b955340 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f32e80116f0 tx=0x7f32e8011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce14f8d44d7ddaf server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:46:32.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.669+0000 7f32d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:46:32.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.669+0000 7f32d2ffd700 1 -- 172.21.15.12:0/2355402636 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e8022030 con 0x55767b7cd9f0 2024-05-17T22:46:32.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.669+0000 7f32d2ffd700 1 -- 172.21.15.12:0/2355402636 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f32e8004020 con 0x55767b7cd9f0 2024-05-17T22:46:32.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.669+0000 7f3308289b80 1 -- 172.21.15.12:0/2355402636 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32ec0075c0 con 0x55767b7cd9f0 2024-05-17T22:46:32.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.670+0000 7f32d2ffd700 1 -- 172.21.15.12:0/2355402636 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f32e802c050 con 0x55767b7cd9f0 2024-05-17T22:46:32.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.670+0000 7f3308289b80 10 client.15957 fetch_fsmap learned FSMap version 37 2024-05-17T22:46:32.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.670+0000 7f3308289b80 1 -- 172.21.15.12:0/2355402636 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55767b963000 con 0x55767b7cd9f0 2024-05-17T22:46:32.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.670+0000 7f32d2ffd700 1 -- 172.21.15.12:0/2355402636 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32ec0075c0 con 0x55767b7cd9f0 2024-05-17T22:46:32.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.670+0000 7f32d2ffd700 1 -- 172.21.15.12:0/2355402636 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f32e800f040 con 0x55767b7cd9f0 2024-05-17T22:46:32.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.670+0000 7f3308289b80 10 client.15957 fetch_fsmap finished waiting for FSMap version 37 2024-05-17T22:46:32.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.670+0000 7f3308289b80 10 client.15957 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-17T22:46:32.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.670+0000 7f3308289b80 20 client.15957 populate_metadata read hostname 'smithi012' 2024-05-17T22:46:32.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.670+0000 7f3308289b80 1 --2- 172.21.15.12:0/2355402636 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55767b9658d0 0x55767b967d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:32.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.670+0000 7f3308289b80 4 client.15957 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:46:32.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.670+0000 7f3308289b80 1 -- 172.21.15.12:0/2355402636 --> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55767b87c1e0 con 0x55767b9658d0 2024-05-17T22:46:32.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.671+0000 7f32f27fc700 1 --2- 172.21.15.12:0/2355402636 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55767b9658d0 0x55767b967d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:32.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.675+0000 7f32f27fc700 1 --2- 172.21.15.12:0/2355402636 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55767b9658d0 0x55767b967d20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3b633c66f298eafd server_cookie=3901e1f2f12b6704 in_seq=0 out_seq=0 2024-05-17T22:46:32.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.675+0000 7f32d2ffd700 10 client.15957 ms_handle_connect on v2:172.21.15.151:6838/2338353849 2024-05-17T22:46:32.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.677+0000 7f32d2ffd700 1 -- 172.21.15.12:0/2355402636 <== mds.2 v2:172.21.15.151:6838/2338353849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55767b87c1e0 con 0x55767b9658d0 2024-05-17T22:46:32.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.677+0000 7f32d2ffd700 10 client.15957 handle_command_reply: tid=0 2024-05-17T22:46:32.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.677+0000 7f3308289b80 1 client.15957 shutdown 2024-05-17T22:46:32.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.677+0000 7f3308289b80 1 -- 172.21.15.12:0/2355402636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55767b7cd9f0 msgr2=0x55767b955340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:32.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.677+0000 7f3308289b80 1 --2- 172.21.15.12:0/2355402636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55767b7cd9f0 0x55767b955340 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f32e80116f0 tx=0x7f32e8011a00 comp rx=0 tx=0).stop 2024-05-17T22:46:32.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.677+0000 7f3308289b80 1 -- 172.21.15.12:0/2355402636 shutdown_connections 2024-05-17T22:46:32.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.677+0000 7f3308289b80 1 --2- 172.21.15.12:0/2355402636 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55767b9658d0 0x55767b967d20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:32.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.677+0000 7f3308289b80 1 --2- 172.21.15.12:0/2355402636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55767b951aa0 0x55767b9616c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:32.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.677+0000 7f3308289b80 1 --2- 172.21.15.12:0/2355402636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55767b7cd9f0 0x55767b955340 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:32.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.677+0000 7f3308289b80 1 --2- 172.21.15.12:0/2355402636 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55767b94c170 0x55767b955880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:32.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.677+0000 7f3308289b80 1 -- 172.21.15.12:0/2355402636 >> 172.21.15.12:0/2355402636 conn(0x55767b811f00 msgr2=0x55767b940ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:46:32.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.677+0000 7f32d2ffd700 0 client.15957 ms_handle_reset on v2:172.21.15.151:6838/2338353849 2024-05-17T22:46:32.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.677+0000 7f3308289b80 1 -- 172.21.15.12:0/2355402636 shutdown_connections 2024-05-17T22:46:32.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.677+0000 7f3308289b80 1 -- 172.21.15.12:0/2355402636 wait complete. 2024-05-17T22:46:32.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.677+0000 7f3308289b80 20 client.15957 trim_cache size 0 max 16384 2024-05-17T22:46:32.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.692+0000 7f3308289b80 1 Processor -- start 2024-05-17T22:46:32.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.692+0000 7f3308289b80 1 -- start start 2024-05-17T22:46:32.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.693+0000 7f3308289b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55767b7cd9f0 0x55767b94bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:32.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.693+0000 7f3308289b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55767b94c170 0x55767b940eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:32.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.693+0000 7f3308289b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55767b951aa0 0x55767b957a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:32.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.693+0000 7f3308289b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f32e00113b0 con 0x55767b951aa0 2024-05-17T22:46:32.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.693+0000 7f3308289b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f32e000ec40 con 0x55767b7cd9f0 2024-05-17T22:46:32.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.693+0000 7f3308289b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f32e000ca90 con 0x55767b94c170 2024-05-17T22:46:32.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.693+0000 7f32f37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55767b951aa0 0x55767b957a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:32.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.693+0000 7f32f37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55767b951aa0 0x55767b957a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56824/0 (socket says 172.21.15.12:56824) 2024-05-17T22:46:32.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.693+0000 7f32f37fe700 1 -- 172.21.15.12:0/2530718810 learned_addr learned my addr 172.21.15.12:0/2530718810 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:46:32.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.693+0000 7f32f27fc700 1 --2- 172.21.15.12:0/2530718810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55767b94c170 0x55767b940eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:32.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.693+0000 7f32f2ffd700 1 --2- 172.21.15.12:0/2530718810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55767b7cd9f0 0x55767b94bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:32.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.693+0000 7f32f37fe700 1 -- 172.21.15.12:0/2530718810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55767b94c170 msgr2=0x55767b940eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:46:32.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.693+0000 7f32f37fe700 1 --2- 172.21.15.12:0/2530718810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55767b94c170 0x55767b940eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:32.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.693+0000 7f32f37fe700 1 -- 172.21.15.12:0/2530718810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55767b7cd9f0 msgr2=0x55767b94bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:46:32.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.693+0000 7f32f37fe700 1 --2- 172.21.15.12:0/2530718810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55767b7cd9f0 0x55767b94bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:32.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.693+0000 7f32f37fe700 1 -- 172.21.15.12:0/2530718810 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55767b953890 con 0x55767b951aa0 2024-05-17T22:46:32.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.693+0000 7f32f2ffd700 1 --2- 172.21.15.12:0/2530718810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55767b7cd9f0 0x55767b94bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:46:32.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.693+0000 7f32f37fe700 1 --2- 172.21.15.12:0/2530718810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55767b951aa0 0x55767b957a90 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f32e40a5a60 tx=0x7f32e4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2198fd0fffb8fc63 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:46:32.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.693+0000 7f32d97fa700 1 -- 172.21.15.12:0/2530718810 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e40c3070 con 0x55767b951aa0 2024-05-17T22:46:32.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.693+0000 7f32d97fa700 1 -- 172.21.15.12:0/2530718810 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32e40aa0e0 con 0x55767b951aa0 2024-05-17T22:46:32.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.693+0000 7f32d97fa700 1 -- 172.21.15.12:0/2530718810 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e4015040 con 0x55767b951aa0 2024-05-17T22:46:32.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.694+0000 7f3308289b80 1 -- 172.21.15.12:0/2530718810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55767b951aa0 msgr2=0x55767b957a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:32.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.694+0000 7f3308289b80 1 --2- 172.21.15.12:0/2530718810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55767b951aa0 0x55767b957a90 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f32e40a5a60 tx=0x7f32e4000bc0 comp rx=0 tx=0).stop 2024-05-17T22:46:32.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.694+0000 7f3308289b80 1 -- 172.21.15.12:0/2530718810 shutdown_connections 2024-05-17T22:46:32.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.694+0000 7f3308289b80 1 --2- 172.21.15.12:0/2530718810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55767b951aa0 0x55767b957a90 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:32.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.694+0000 7f3308289b80 1 --2- 172.21.15.12:0/2530718810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55767b7cd9f0 0x55767b94bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:32.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.694+0000 7f3308289b80 1 --2- 172.21.15.12:0/2530718810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55767b94c170 0x55767b940eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:32.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.694+0000 7f3308289b80 1 -- 172.21.15.12:0/2530718810 >> 172.21.15.12:0/2530718810 conn(0x55767b811f00 msgr2=0x55767b94ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:46:32.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.694+0000 7f3308289b80 1 -- 172.21.15.12:0/2530718810 shutdown_connections 2024-05-17T22:46:32.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.694+0000 7f3308289b80 1 -- 172.21.15.12:0/2530718810 wait complete. 2024-05-17T22:46:32.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.695+0000 7f3308289b80 1 -- start start 2024-05-17T22:46:32.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.695+0000 7f3308289b80 1 Processor -- start 2024-05-17T22:46:32.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.695+0000 7f3308289b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55767b7cd9f0 0x55767b958250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:32.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.695+0000 7f3308289b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55767b94c170 0x55767b958790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:32.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.695+0000 7f3308289b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55767b951aa0 0x55767b958cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:32.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.695+0000 7f32f27fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55767b94c170 0x55767b958790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:32.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.695+0000 7f32f27fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55767b94c170 0x55767b958790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56832/0 (socket says 172.21.15.12:56832) 2024-05-17T22:46:32.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.695+0000 7f32f2ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55767b7cd9f0 0x55767b958250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:32.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.695+0000 7f32f27fc700 1 -- 172.21.15.12:0/3276724658 learned_addr learned my addr 172.21.15.12:0/3276724658 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:46:32.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.695+0000 7f3308289b80 1 -- 172.21.15.12:0/3276724658 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f32e000d100 con 0x55767b94c170 2024-05-17T22:46:32.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.695+0000 7f3308289b80 1 -- 172.21.15.12:0/3276724658 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f32e000e570 con 0x55767b7cd9f0 2024-05-17T22:46:32.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.695+0000 7f3308289b80 1 -- 172.21.15.12:0/3276724658 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f32e000d7d0 con 0x55767b951aa0 2024-05-17T22:46:32.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.696+0000 7f32f37fe700 1 --2- 172.21.15.12:0/3276724658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55767b951aa0 0x55767b958cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:32.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.696+0000 7f3308289b80 10 client.0 _ll_register_callbacks cb 0x55767b804f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:46:32.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.696+0000 7f32f27fc700 1 -- 172.21.15.12:0/3276724658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55767b951aa0 msgr2=0x55767b958cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:46:32.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.696+0000 7f32f27fc700 1 --2- 172.21.15.12:0/3276724658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55767b951aa0 0x55767b958cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:32.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.696+0000 7f32f27fc700 1 -- 172.21.15.12:0/3276724658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55767b7cd9f0 msgr2=0x55767b958250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:46:32.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.696+0000 7f32f27fc700 1 --2- 172.21.15.12:0/3276724658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55767b7cd9f0 0x55767b958250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:32.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.696+0000 7f32f27fc700 1 -- 172.21.15.12:0/3276724658 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55767b9ce1c0 con 0x55767b94c170 2024-05-17T22:46:32.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.696+0000 7f32f2ffd700 1 --2- 172.21.15.12:0/3276724658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55767b7cd9f0 0x55767b958250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:46:32.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.696+0000 7f32f27fc700 1 --2- 172.21.15.12:0/3276724658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55767b94c170 0x55767b958790 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f32ec00b0a0 tx=0x7f32ec009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321f681f0f5cacac server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:46:32.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.696+0000 7f32d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:46:32.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.696+0000 7f32d2ffd700 1 -- 172.21.15.12:0/3276724658 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ec032060 con 0x55767b94c170 2024-05-17T22:46:32.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.696+0000 7f32d2ffd700 1 -- 172.21.15.12:0/3276724658 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f32ec003a00 con 0x55767b94c170 2024-05-17T22:46:32.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.696+0000 7f3308289b80 1 -- 172.21.15.12:0/3276724658 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32e40aa0e0 con 0x55767b94c170 2024-05-17T22:46:32.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.696+0000 7f32d2ffd700 1 -- 172.21.15.12:0/3276724658 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f32ec039050 con 0x55767b94c170 2024-05-17T22:46:32.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.696+0000 7f3308289b80 10 client.15963 fetch_fsmap learned FSMap version 37 2024-05-17T22:46:32.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.696+0000 7f3308289b80 1 -- 172.21.15.12:0/3276724658 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55767b9cf390 con 0x55767b94c170 2024-05-17T22:46:32.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.696+0000 7f32d2ffd700 1 -- 172.21.15.12:0/3276724658 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32ec003c20 con 0x55767b94c170 2024-05-17T22:46:32.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.697+0000 7f32d2ffd700 1 -- 172.21.15.12:0/3276724658 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f32ec011070 con 0x55767b94c170 2024-05-17T22:46:32.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.697+0000 7f3308289b80 10 client.15963 fetch_fsmap finished waiting for FSMap version 37 2024-05-17T22:46:32.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.697+0000 7f3308289b80 10 client.15963 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-17T22:46:32.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.697+0000 7f3308289b80 20 client.15963 populate_metadata read hostname 'smithi012' 2024-05-17T22:46:32.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.697+0000 7f3308289b80 1 --2- 172.21.15.12:0/3276724658 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55767b9cf620 0x55767b9d1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:32.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.697+0000 7f3308289b80 4 client.15963 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:46:32.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.697+0000 7f3308289b80 1 -- 172.21.15.12:0/3276724658 --> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f32e40aa0e0 con 0x55767b9cf620 2024-05-17T22:46:32.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.697+0000 7f32f2ffd700 1 --2- 172.21.15.12:0/3276724658 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55767b9cf620 0x55767b9d1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:32.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.697+0000 7f32d2ffd700 10 client.15963 ms_handle_connect on v2:172.21.15.151:6838/2338353849 2024-05-17T22:46:32.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.697+0000 7f32f2ffd700 1 --2- 172.21.15.12:0/3276724658 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55767b9cf620 0x55767b9d1a40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d4fa79263c692395 server_cookie=6e7aef70d814122 in_seq=0 out_seq=0 2024-05-17T22:46:32.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.698+0000 7f32d2ffd700 1 -- 172.21.15.12:0/3276724658 <== mds.2 v2:172.21.15.151:6838/2338353849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f32e40aa0e0 con 0x55767b9cf620 2024-05-17T22:46:32.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.698+0000 7f32d2ffd700 10 client.15963 handle_command_reply: tid=0 2024-05-17T22:46:32.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.698+0000 7f3308289b80 1 client.15963 shutdown 2024-05-17T22:46:32.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.698+0000 7f3308289b80 1 -- 172.21.15.12:0/3276724658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55767b94c170 msgr2=0x55767b958790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:32.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.698+0000 7f3308289b80 1 --2- 172.21.15.12:0/3276724658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55767b94c170 0x55767b958790 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f32ec00b0a0 tx=0x7f32ec009290 comp rx=0 tx=0).stop 2024-05-17T22:46:32.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.698+0000 7f3308289b80 1 -- 172.21.15.12:0/3276724658 shutdown_connections 2024-05-17T22:46:32.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.698+0000 7f3308289b80 1 --2- 172.21.15.12:0/3276724658 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55767b9cf620 0x55767b9d1a40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:32.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.698+0000 7f3308289b80 1 --2- 172.21.15.12:0/3276724658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55767b951aa0 0x55767b958cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:32.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.698+0000 7f3308289b80 1 --2- 172.21.15.12:0/3276724658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55767b7cd9f0 0x55767b958250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:32.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.698+0000 7f3308289b80 1 --2- 172.21.15.12:0/3276724658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55767b94c170 0x55767b958790 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:32.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.698+0000 7f32d2ffd700 0 client.15963 ms_handle_reset on v2:172.21.15.151:6838/2338353849 2024-05-17T22:46:32.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.698+0000 7f3308289b80 1 -- 172.21.15.12:0/3276724658 >> 172.21.15.12:0/3276724658 conn(0x55767b811f00 msgr2=0x55767b94f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:46:32.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.698+0000 7f3308289b80 1 -- 172.21.15.12:0/3276724658 shutdown_connections 2024-05-17T22:46:32.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.698+0000 7f3308289b80 1 -- 172.21.15.12:0/3276724658 wait complete. 2024-05-17T22:46:32.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.698+0000 7f3308289b80 20 client.15963 trim_cache size 0 max 16384 2024-05-17T22:46:32.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.699+0000 7f32f98ae700 1 -- 172.21.15.12:0/2786222418 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f32dc0b2d40 msgr2=0x7f32dc0b5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:32.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.699+0000 7f32f98ae700 1 --2- 172.21.15.12:0/2786222418 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f32dc0b2d40 0x7f32dc0b5250 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f32e8005fd0 tx=0x7f32e8005f00 comp rx=0 tx=0).stop 2024-05-17T22:46:32.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.699+0000 7f32f98ae700 1 -- 172.21.15.12:0/2786222418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32f411d550 msgr2=0x7f32f4114fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:32.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.699+0000 7f32f98ae700 1 --2- 172.21.15.12:0/2786222418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32f411d550 0x7f32f4114fb0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f32e400b4a0 tx=0x7f32e400b860 comp rx=0 tx=0).stop 2024-05-17T22:46:32.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.699+0000 7f32f98ae700 1 -- 172.21.15.12:0/2786222418 shutdown_connections 2024-05-17T22:46:32.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.699+0000 7f32f98ae700 1 --2- 172.21.15.12:0/2786222418 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f32dc0b2d40 0x7f32dc0b5250 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:32.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.699+0000 7f32f98ae700 1 --2- 172.21.15.12:0/2786222418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32f411d550 0x7f32f4114fb0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:32.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.699+0000 7f32f98ae700 1 --2- 172.21.15.12:0/2786222418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32f410a830 0x7f32f411b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:32.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.699+0000 7f32f98ae700 1 --2- 172.21.15.12:0/2786222418 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32f410b180 0x7f32f4114a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:32.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.699+0000 7f32f98ae700 1 -- 172.21.15.12:0/2786222418 >> 172.21.15.12:0/2786222418 conn(0x7f32f406efc0 msgr2=0x7f32f4072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:46:32.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.699+0000 7f32f98ae700 1 -- 172.21.15.12:0/2786222418 shutdown_connections 2024-05-17T22:46:32.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:31.699+0000 7f32f98ae700 1 -- 172.21.15.12:0/2786222418 wait complete. 2024-05-17T22:46:33.381 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-17T22:46:33.892 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:46:34.211 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:33 smithi012 ceph-mon[31632]: pgmap v658: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 5.4 MiB/s wr, 552 op/s 2024-05-17T22:46:34.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:33 smithi151 ceph-mon[36101]: pgmap v658: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 5.4 MiB/s wr, 552 op/s 2024-05-17T22:46:34.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:33 smithi096 ceph-mon[36065]: pgmap v658: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 5.4 MiB/s wr, 552 op/s 2024-05-17T22:46:36.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:35 smithi012 ceph-mon[31632]: pgmap v659: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 MiB/s wr, 588 op/s 2024-05-17T22:46:36.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:35 smithi151 ceph-mon[36101]: pgmap v659: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 MiB/s wr, 588 op/s 2024-05-17T22:46:36.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.283+0000 7f299b59e700 1 -- 172.21.15.12:0/1446425368 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f299c1032c0 msgr2=0x7f299c1077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:36.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.283+0000 7f299b59e700 1 --2- 172.21.15.12:0/1446425368 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f299c1032c0 0x7f299c1077f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f299001a9a0 tx=0x7f299001acb0 comp rx=0 tx=0).stop 2024-05-17T22:46:36.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.284+0000 7f299b59e700 1 -- 172.21.15.12:0/1446425368 shutdown_connections 2024-05-17T22:46:36.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.284+0000 7f299b59e700 1 --2- 172.21.15.12:0/1446425368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f299c107db0 0x7f299c1082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:36.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.284+0000 7f299b59e700 1 --2- 172.21.15.12:0/1446425368 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f299c102970 0x7f299c102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:36.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.284+0000 7f299b59e700 1 --2- 172.21.15.12:0/1446425368 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f299c1032c0 0x7f299c1077f0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:36.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.284+0000 7f299b59e700 1 -- 172.21.15.12:0/1446425368 >> 172.21.15.12:0/1446425368 conn(0x7f299c0fe660 msgr2=0x7f299c100ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:46:36.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.284+0000 7f299b59e700 1 -- 172.21.15.12:0/1446425368 shutdown_connections 2024-05-17T22:46:36.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.284+0000 7f299b59e700 1 -- 172.21.15.12:0/1446425368 wait complete. 2024-05-17T22:46:36.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.285+0000 7f299b59e700 1 Processor -- start 2024-05-17T22:46:36.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.285+0000 7f299b59e700 1 -- start start 2024-05-17T22:46:36.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.286+0000 7f299b59e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f299c102970 0x7f299c10f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:36.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.286+0000 7f299b59e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f299c1032c0 0x7f299c10fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:36.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.286+0000 7f299b59e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f299c107db0 0x7f299c074680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:36.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.286+0000 7f299b59e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f299c10b5b0 con 0x7f299c102970 2024-05-17T22:46:36.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.286+0000 7f299b59e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f299c10b430 con 0x7f299c1032c0 2024-05-17T22:46:36.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.286+0000 7f299b59e700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f299c10b730 con 0x7f299c107db0 2024-05-17T22:46:36.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.287+0000 7f2999d9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f299c1032c0 0x7f299c10fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:36.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.287+0000 7f2999d9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f299c1032c0 0x7f299c10fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:47000/0 (socket says 172.21.15.12:47000) 2024-05-17T22:46:36.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.287+0000 7f2999d9b700 1 -- 172.21.15.12:0/850060604 learned_addr learned my addr 172.21.15.12:0/850060604 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:46:36.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.288+0000 7f299ad9d700 1 --2- 172.21.15.12:0/850060604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f299c107db0 0x7f299c074680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:36.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.288+0000 7f299a59c700 1 --2- 172.21.15.12:0/850060604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f299c102970 0x7f299c10f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:36.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.288+0000 7f2999d9b700 1 -- 172.21.15.12:0/850060604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f299c107db0 msgr2=0x7f299c074680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:36.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.288+0000 7f2999d9b700 1 --2- 172.21.15.12:0/850060604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f299c107db0 0x7f299c074680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:36.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.288+0000 7f2999d9b700 1 -- 172.21.15.12:0/850060604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f299c102970 msgr2=0x7f299c10f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:36.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.288+0000 7f2999d9b700 1 --2- 172.21.15.12:0/850060604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f299c102970 0x7f299c10f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:36.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.288+0000 7f2999d9b700 1 -- 172.21.15.12:0/850060604 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f299c074db0 con 0x7f299c1032c0 2024-05-17T22:46:36.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.288+0000 7f2999d9b700 1 --2- 172.21.15.12:0/850060604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f299c1032c0 0x7f299c10fc40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f299001afd0 tx=0x7f2990003bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:46:36.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.290+0000 7f298b7fe700 1 -- 172.21.15.12:0/850060604 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2990036070 con 0x7f299c1032c0 2024-05-17T22:46:36.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.290+0000 7f298b7fe700 1 -- 172.21.15.12:0/850060604 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2990031410 con 0x7f299c1032c0 2024-05-17T22:46:36.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.290+0000 7f298b7fe700 1 -- 172.21.15.12:0/850060604 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29900283f0 con 0x7f299c1032c0 2024-05-17T22:46:36.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.292+0000 7f299b59e700 1 -- 172.21.15.12:0/850060604 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f299c074fe0 con 0x7f299c1032c0 2024-05-17T22:46:36.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.292+0000 7f299b59e700 1 -- 172.21.15.12:0/850060604 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f299c075470 con 0x7f299c1032c0 2024-05-17T22:46:36.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:35 smithi096 ceph-mon[36065]: pgmap v659: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 MiB/s wr, 588 op/s 2024-05-17T22:46:36.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.305+0000 7f298b7fe700 1 -- 172.21.15.12:0/850060604 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f29900318c0 con 0x7f299c1032c0 2024-05-17T22:46:36.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.308+0000 7f29af6d6b80 1 Processor -- start 2024-05-17T22:46:36.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.313+0000 7f298b7fe700 1 --2- 172.21.15.12:0/850060604 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f29840b2a50 0x7f29840b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:36.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.313+0000 7f298b7fe700 1 -- 172.21.15.12:0/850060604 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f29900f3bd0 con 0x7f299c1032c0 2024-05-17T22:46:36.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.314+0000 7f29af6d6b80 1 -- start start 2024-05-17T22:46:36.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.314+0000 7f29af6d6b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3c7a069f0 0x55b3c7b89fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:36.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.314+0000 7f29af6d6b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3c7b8a510 0x55b3c7b845c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:36.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.314+0000 7f29af6d6b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3c7b84b00 0x55b3c7b84f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:36.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.314+0000 7f29af6d6b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b3c7ab30d0 con 0x55b3c7b8a510 2024-05-17T22:46:36.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.314+0000 7f29af6d6b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b3c7aa2c70 con 0x55b3c7b84b00 2024-05-17T22:46:36.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.314+0000 7f29af6d6b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b3c7aa6cb0 con 0x55b3c7a069f0 2024-05-17T22:46:36.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.314+0000 7f299ad9d700 1 --2- 172.21.15.12:0/850060604 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f29840b2a50 0x7f29840b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:36.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.315+0000 7f299a59c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3c7a069f0 0x55b3c7b89fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:36.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.315+0000 7f299ad9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3c7b84b00 0x55b3c7b84f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:36.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.315+0000 7f299ad9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3c7b84b00 0x55b3c7b84f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:47010/0 (socket says 172.21.15.12:47010) 2024-05-17T22:46:36.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.315+0000 7f299ad9d700 1 -- 172.21.15.12:0/3061958655 learned_addr learned my addr 172.21.15.12:0/3061958655 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:46:36.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.315+0000 7f299ad9d700 1 -- 172.21.15.12:0/3061958655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3c7a069f0 msgr2=0x55b3c7b89fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:46:36.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.315+0000 7f299ad9d700 1 --2- 172.21.15.12:0/3061958655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3c7a069f0 0x55b3c7b89fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:36.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.315+0000 7f299ad9d700 1 -- 172.21.15.12:0/3061958655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3c7b8a510 msgr2=0x55b3c7b845c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:46:36.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.315+0000 7f299ad9d700 1 --2- 172.21.15.12:0/3061958655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3c7b8a510 0x55b3c7b845c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:36.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.315+0000 7f299ad9d700 1 -- 172.21.15.12:0/3061958655 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3c7a3c5c0 con 0x55b3c7b84b00 2024-05-17T22:46:36.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.316+0000 7f299ad9d700 1 --2- 172.21.15.12:0/3061958655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3c7b84b00 0x55b3c7b84f10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f298c014790 tx=0x7f298c014aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=273cad8e645ac23d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:46:36.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.319+0000 7f2988ff9700 1 -- 172.21.15.12:0/3061958655 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c01c020 con 0x55b3c7b84b00 2024-05-17T22:46:36.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.319+0000 7f2988ff9700 1 -- 172.21.15.12:0/3061958655 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f298c010040 con 0x55b3c7b84b00 2024-05-17T22:46:36.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.319+0000 7f2988ff9700 1 -- 172.21.15.12:0/3061958655 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c018c30 con 0x55b3c7b84b00 2024-05-17T22:46:36.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.319+0000 7f299ad9d700 1 --2- 172.21.15.12:0/850060604 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f29840b2a50 0x7f29840b4f60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f298c01d000 tx=0x7f298c017660 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:46:36.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.323+0000 7f29af6d6b80 1 -- 172.21.15.12:0/3061958655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3c7b84b00 msgr2=0x55b3c7b84f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:36.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.323+0000 7f29af6d6b80 1 --2- 172.21.15.12:0/3061958655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3c7b84b00 0x55b3c7b84f10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f298c014790 tx=0x7f298c014aa0 comp rx=0 tx=0).stop 2024-05-17T22:46:36.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.328+0000 7f29af6d6b80 1 -- 172.21.15.12:0/3061958655 shutdown_connections 2024-05-17T22:46:36.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.328+0000 7f29af6d6b80 1 --2- 172.21.15.12:0/3061958655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3c7b84b00 0x55b3c7b84f10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:36.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.328+0000 7f29af6d6b80 1 --2- 172.21.15.12:0/3061958655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3c7a069f0 0x55b3c7b89fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:36.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.328+0000 7f29af6d6b80 1 --2- 172.21.15.12:0/3061958655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3c7b8a510 0x55b3c7b845c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:36.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.328+0000 7f29af6d6b80 1 -- 172.21.15.12:0/3061958655 >> 172.21.15.12:0/3061958655 conn(0x55b3c7a667d0 msgr2=0x55b3c7b7f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:46:36.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.330+0000 7f29af6d6b80 1 -- 172.21.15.12:0/3061958655 shutdown_connections 2024-05-17T22:46:36.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.330+0000 7f29af6d6b80 1 -- 172.21.15.12:0/3061958655 wait complete. 2024-05-17T22:46:36.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.330+0000 7f29af6d6b80 1 -- start start 2024-05-17T22:46:36.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.330+0000 7f29af6d6b80 1 Processor -- start 2024-05-17T22:46:36.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.331+0000 7f29af6d6b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3c7a069f0 0x55b3c7b8dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:36.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.331+0000 7f29af6d6b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3c7b84b00 0x55b3c7b8e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:36.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.331+0000 7f29af6d6b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3c7b8a510 0x55b3c7b8e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:36.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.331+0000 7f29af6d6b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b3c79aa150 con 0x55b3c7b84b00 2024-05-17T22:46:36.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.331+0000 7f29af6d6b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b3c7a44940 con 0x55b3c7b8a510 2024-05-17T22:46:36.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.331+0000 7f29af6d6b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b3c7b90e60 con 0x55b3c7a069f0 2024-05-17T22:46:36.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.331+0000 7f2999d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3c7b84b00 0x55b3c7b8e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:36.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.331+0000 7f2999d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3c7b84b00 0x55b3c7b8e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56874/0 (socket says 172.21.15.12:56874) 2024-05-17T22:46:36.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.331+0000 7f2999d9b700 1 -- 172.21.15.12:0/3800849209 learned_addr learned my addr 172.21.15.12:0/3800849209 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:46:36.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.331+0000 7f29af6d6b80 10 client.0 _ll_register_callbacks cb 0x55b3c7a39b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:46:36.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.331+0000 7f299a59c700 1 --2- 172.21.15.12:0/3800849209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3c7a069f0 0x55b3c7b8dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:36.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.332+0000 7f299ad9d700 1 --2- 172.21.15.12:0/3800849209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3c7b8a510 0x55b3c7b8e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:36.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.335+0000 7f299ad9d700 1 -- 172.21.15.12:0/3800849209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3c7a069f0 msgr2=0x55b3c7b8dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:46:36.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.335+0000 7f299ad9d700 1 --2- 172.21.15.12:0/3800849209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3c7a069f0 0x55b3c7b8dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:36.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.335+0000 7f299ad9d700 1 -- 172.21.15.12:0/3800849209 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3c7b84b00 msgr2=0x55b3c7b8e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:46:36.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.335+0000 7f299ad9d700 1 --2- 172.21.15.12:0/3800849209 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3c7b84b00 0x55b3c7b8e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:36.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.335+0000 7f299ad9d700 1 -- 172.21.15.12:0/3800849209 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3c79f3480 con 0x55b3c7b8a510 2024-05-17T22:46:36.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.335+0000 7f299ad9d700 1 --2- 172.21.15.12:0/3800849209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3c7b8a510 0x55b3c7b8e750 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f298c0092f0 tx=0x7f298c004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=214eba1b69fe4244 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:46:36.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.336+0000 7f2981ffb700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:46:36.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.336+0000 7f2981ffb700 1 -- 172.21.15.12:0/3800849209 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c01c040 con 0x55b3c7b8a510 2024-05-17T22:46:36.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.336+0000 7f2981ffb700 1 -- 172.21.15.12:0/3800849209 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f298c005c10 con 0x55b3c7b8a510 2024-05-17T22:46:36.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.336+0000 7f29af6d6b80 1 -- 172.21.15.12:0/3800849209 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f298c010040 con 0x55b3c7b8a510 2024-05-17T22:46:36.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.336+0000 7f2981ffb700 1 -- 172.21.15.12:0/3800849209 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f298c005e10 con 0x55b3c7b8a510 2024-05-17T22:46:36.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.336+0000 7f29af6d6b80 10 client.25631 fetch_fsmap learned FSMap version 37 2024-05-17T22:46:36.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.337+0000 7f29af6d6b80 1 -- 172.21.15.12:0/3800849209 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b3c7b9b770 con 0x55b3c7b8a510 2024-05-17T22:46:36.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.340+0000 7f2981ffb700 1 -- 172.21.15.12:0/3800849209 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f298c031070 con 0x55b3c7b8a510 2024-05-17T22:46:36.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.340+0000 7f2981ffb700 1 -- 172.21.15.12:0/3800849209 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f298c016950 con 0x55b3c7b8a510 2024-05-17T22:46:36.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.340+0000 7f29af6d6b80 10 client.25631 fetch_fsmap finished waiting for FSMap version 37 2024-05-17T22:46:36.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.340+0000 7f29af6d6b80 10 client.25631 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-17T22:46:36.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.340+0000 7f29af6d6b80 20 client.25631 populate_metadata read hostname 'smithi012' 2024-05-17T22:46:36.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.340+0000 7f29af6d6b80 1 --2- 172.21.15.12:0/3800849209 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55b3c7b9dcf0 0x55b3c7ba0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:36.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.340+0000 7f29af6d6b80 4 client.25631 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:46:36.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.340+0000 7f29af6d6b80 1 -- 172.21.15.12:0/3800849209 --> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f298c010040 con 0x55b3c7b9dcf0 2024-05-17T22:46:36.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.341+0000 7f299a59c700 1 --2- 172.21.15.12:0/3800849209 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55b3c7b9dcf0 0x55b3c7ba0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:36.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.342+0000 7f299a59c700 1 --2- 172.21.15.12:0/3800849209 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55b3c7b9dcf0 0x55b3c7ba0110 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c5e78e1832a8b66d server_cookie=bbbc7de7b8ecead3 in_seq=0 out_seq=0 2024-05-17T22:46:36.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.344+0000 7f2981ffb700 10 client.25631 ms_handle_connect on v2:172.21.15.96:6836/4222574338 2024-05-17T22:46:36.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.344+0000 7f2981ffb700 1 -- 172.21.15.12:0/3800849209 <== mds.1 v2:172.21.15.96:6836/4222574338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f298c010040 con 0x55b3c7b9dcf0 2024-05-17T22:46:36.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.344+0000 7f2981ffb700 10 client.25631 handle_command_reply: tid=0 2024-05-17T22:46:36.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.344+0000 7f29af6d6b80 1 client.25631 shutdown 2024-05-17T22:46:36.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.348+0000 7f29af6d6b80 1 -- 172.21.15.12:0/3800849209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3c7b8a510 msgr2=0x55b3c7b8e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:36.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.348+0000 7f29af6d6b80 1 --2- 172.21.15.12:0/3800849209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3c7b8a510 0x55b3c7b8e750 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f298c0092f0 tx=0x7f298c004770 comp rx=0 tx=0).stop 2024-05-17T22:46:36.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.348+0000 7f29af6d6b80 1 -- 172.21.15.12:0/3800849209 shutdown_connections 2024-05-17T22:46:36.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.348+0000 7f29af6d6b80 1 --2- 172.21.15.12:0/3800849209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3c7b8a510 0x55b3c7b8e750 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:36.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.348+0000 7f29af6d6b80 1 --2- 172.21.15.12:0/3800849209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3c7a069f0 0x55b3c7b8dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:36.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.348+0000 7f29af6d6b80 1 --2- 172.21.15.12:0/3800849209 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55b3c7b9dcf0 0x55b3c7ba0110 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:36.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.348+0000 7f29af6d6b80 1 --2- 172.21.15.12:0/3800849209 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3c7b84b00 0x55b3c7b8e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:36.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.349+0000 7f29af6d6b80 1 -- 172.21.15.12:0/3800849209 >> 172.21.15.12:0/3800849209 conn(0x55b3c7a667d0 msgr2=0x55b3c7b7f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:46:36.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.350+0000 7f2981ffb700 0 client.25631 ms_handle_reset on v2:172.21.15.96:6836/4222574338 2024-05-17T22:46:36.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.350+0000 7f29af6d6b80 1 -- 172.21.15.12:0/3800849209 shutdown_connections 2024-05-17T22:46:36.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.350+0000 7f29af6d6b80 1 -- 172.21.15.12:0/3800849209 wait complete. 2024-05-17T22:46:36.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.350+0000 7f29af6d6b80 20 client.25631 trim_cache size 0 max 16384 2024-05-17T22:46:36.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 435.0 s, 507.6 tps, lat 190.967 ms stddev 218.175 2024-05-17T22:46:36.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.380+0000 7f29af6d6b80 1 Processor -- start 2024-05-17T22:46:36.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.380+0000 7f29af6d6b80 1 -- start start 2024-05-17T22:46:36.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.381+0000 7f29af6d6b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3c7a069f0 0x55b3c7b845c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:36.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.381+0000 7f29af6d6b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3c7b84b00 0x55b3c7b8cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:36.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.381+0000 7f29af6d6b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3c7b8a510 0x55b3c7b8e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:36.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.381+0000 7f29af6d6b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f299c1b8a60 con 0x55b3c7b84b00 2024-05-17T22:46:36.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.381+0000 7f29af6d6b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f299c1b62f0 con 0x55b3c7b8a510 2024-05-17T22:46:36.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.381+0000 7f29af6d6b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f299c1b4160 con 0x55b3c7a069f0 2024-05-17T22:46:36.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.383+0000 7f2999d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3c7b84b00 0x55b3c7b8cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:36.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.383+0000 7f2999d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3c7b84b00 0x55b3c7b8cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56890/0 (socket says 172.21.15.12:56890) 2024-05-17T22:46:36.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.383+0000 7f2999d9b700 1 -- 172.21.15.12:0/2440270727 learned_addr learned my addr 172.21.15.12:0/2440270727 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:46:36.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.384+0000 7f299ad9d700 1 --2- 172.21.15.12:0/2440270727 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3c7b8a510 0x55b3c7b8e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:36.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.384+0000 7f299a59c700 1 --2- 172.21.15.12:0/2440270727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3c7a069f0 0x55b3c7b845c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:36.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.385+0000 7f2999d9b700 1 -- 172.21.15.12:0/2440270727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3c7a069f0 msgr2=0x55b3c7b845c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:46:36.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.385+0000 7f2999d9b700 1 --2- 172.21.15.12:0/2440270727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3c7a069f0 0x55b3c7b845c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:36.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.385+0000 7f2999d9b700 1 -- 172.21.15.12:0/2440270727 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3c7b8a510 msgr2=0x55b3c7b8e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:46:36.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.385+0000 7f2999d9b700 1 --2- 172.21.15.12:0/2440270727 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3c7b8a510 0x55b3c7b8e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:36.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.385+0000 7f2999d9b700 1 -- 172.21.15.12:0/2440270727 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3c7b8c220 con 0x55b3c7b84b00 2024-05-17T22:46:36.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.387+0000 7f299ad9d700 1 --2- 172.21.15.12:0/2440270727 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3c7b8a510 0x55b3c7b8e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:46:36.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.388+0000 7f2999d9b700 1 --2- 172.21.15.12:0/2440270727 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3c7b84b00 0x55b3c7b8cf60 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2990005b40 tx=0x7f29900ba960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0d92afbdfc87580 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:46:36.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.392+0000 7f2983fff700 1 -- 172.21.15.12:0/2440270727 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29900baf10 con 0x55b3c7b84b00 2024-05-17T22:46:36.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.392+0000 7f2983fff700 1 -- 172.21.15.12:0/2440270727 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29900bf2c0 con 0x55b3c7b84b00 2024-05-17T22:46:36.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.392+0000 7f2983fff700 1 -- 172.21.15.12:0/2440270727 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29900b6880 con 0x55b3c7b84b00 2024-05-17T22:46:36.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.394+0000 7f29af6d6b80 1 -- 172.21.15.12:0/2440270727 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3c7b84b00 msgr2=0x55b3c7b8cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:36.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.394+0000 7f29af6d6b80 1 --2- 172.21.15.12:0/2440270727 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3c7b84b00 0x55b3c7b8cf60 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2990005b40 tx=0x7f29900ba960 comp rx=0 tx=0).stop 2024-05-17T22:46:36.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.398+0000 7f29af6d6b80 1 -- 172.21.15.12:0/2440270727 shutdown_connections 2024-05-17T22:46:36.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.398+0000 7f29af6d6b80 1 --2- 172.21.15.12:0/2440270727 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3c7b8a510 0x55b3c7b8e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:36.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.398+0000 7f29af6d6b80 1 --2- 172.21.15.12:0/2440270727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3c7a069f0 0x55b3c7b845c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:36.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.398+0000 7f29af6d6b80 1 --2- 172.21.15.12:0/2440270727 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3c7b84b00 0x55b3c7b8cf60 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:36.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.398+0000 7f29af6d6b80 1 -- 172.21.15.12:0/2440270727 >> 172.21.15.12:0/2440270727 conn(0x55b3c7a667d0 msgr2=0x55b3c7b893f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:46:36.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.402+0000 7f29af6d6b80 1 -- 172.21.15.12:0/2440270727 shutdown_connections 2024-05-17T22:46:36.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.402+0000 7f29af6d6b80 1 -- 172.21.15.12:0/2440270727 wait complete. 2024-05-17T22:46:36.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.402+0000 7f29af6d6b80 1 -- start start 2024-05-17T22:46:36.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.403+0000 7f29af6d6b80 1 Processor -- start 2024-05-17T22:46:36.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.406+0000 7f29af6d6b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3c7a069f0 0x55b3c7b96500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:36.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.406+0000 7f29af6d6b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3c7b84b00 0x55b3c7ba4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:36.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.406+0000 7f29af6d6b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3c7b8a510 0x55b3c7ba51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:36.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.407+0000 7f29af6d6b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f299c1b47b0 con 0x55b3c7b8a510 2024-05-17T22:46:36.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.407+0000 7f29af6d6b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f299c1b5c20 con 0x55b3c7a069f0 2024-05-17T22:46:36.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.407+0000 7f29af6d6b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f299c1b4e80 con 0x55b3c7b84b00 2024-05-17T22:46:36.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.407+0000 7f29af6d6b80 10 client.0 _ll_register_callbacks cb 0x55b3c7a39b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:46:36.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.407+0000 7f2999d9b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3c7b84b00 0x55b3c7ba4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:36.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.407+0000 7f2999d9b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3c7b84b00 0x55b3c7ba4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:47198/0 (socket says 172.21.15.12:47198) 2024-05-17T22:46:36.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.407+0000 7f2999d9b700 1 -- 172.21.15.12:0/873451018 learned_addr learned my addr 172.21.15.12:0/873451018 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:46:36.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.408+0000 7f2999d9b700 1 -- 172.21.15.12:0/873451018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3c7a069f0 msgr2=0x55b3c7b96500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T22:46:36.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.408+0000 7f2999d9b700 1 --2- 172.21.15.12:0/873451018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3c7a069f0 0x55b3c7b96500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:36.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.408+0000 7f2999d9b700 1 -- 172.21.15.12:0/873451018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3c7b8a510 msgr2=0x55b3c7ba51d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T22:46:36.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.408+0000 7f2999d9b700 1 --2- 172.21.15.12:0/873451018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3c7b8a510 0x55b3c7ba51d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:36.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.408+0000 7f2999d9b700 1 -- 172.21.15.12:0/873451018 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3c7ba5810 con 0x55b3c7b84b00 2024-05-17T22:46:36.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.408+0000 7f2999d9b700 1 --2- 172.21.15.12:0/873451018 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3c7b84b00 0x55b3c7ba4c90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f29900f1330 tx=0x7f29900bafc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d284da92d643cfec server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:46:36.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.408+0000 7f2981ffb700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:46:36.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.408+0000 7f2981ffb700 1 -- 172.21.15.12:0/873451018 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29900c8060 con 0x55b3c7b84b00 2024-05-17T22:46:36.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.409+0000 7f29af6d6b80 1 -- 172.21.15.12:0/873451018 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29900bf2c0 con 0x55b3c7b84b00 2024-05-17T22:46:36.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.409+0000 7f2981ffb700 1 -- 172.21.15.12:0/873451018 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f29900bb370 con 0x55b3c7b84b00 2024-05-17T22:46:36.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.409+0000 7f2981ffb700 1 -- 172.21.15.12:0/873451018 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f29900ba170 con 0x55b3c7b84b00 2024-05-17T22:46:36.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.411+0000 7f29af6d6b80 10 client.25591 fetch_fsmap learned FSMap version 37 2024-05-17T22:46:36.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.411+0000 7f29af6d6b80 1 -- 172.21.15.12:0/873451018 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b3c7c080d0 con 0x55b3c7b84b00 2024-05-17T22:46:36.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.414+0000 7f2981ffb700 1 -- 172.21.15.12:0/873451018 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29900ba310 con 0x55b3c7b84b00 2024-05-17T22:46:36.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.414+0000 7f2981ffb700 1 -- 172.21.15.12:0/873451018 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f29900cb050 con 0x55b3c7b84b00 2024-05-17T22:46:36.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.418+0000 7f29af6d6b80 10 client.25591 fetch_fsmap finished waiting for FSMap version 37 2024-05-17T22:46:36.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.418+0000 7f29af6d6b80 10 client.25591 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-17T22:46:36.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.418+0000 7f29af6d6b80 20 client.25591 populate_metadata read hostname 'smithi012' 2024-05-17T22:46:36.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.418+0000 7f29af6d6b80 1 --2- 172.21.15.12:0/873451018 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55b3c7c086f0 0x55b3c7c0ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:36.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.418+0000 7f29af6d6b80 4 client.25591 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:46:36.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.418+0000 7f29af6d6b80 1 -- 172.21.15.12:0/873451018 --> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f29900bf2c0 con 0x55b3c7c086f0 2024-05-17T22:46:36.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.420+0000 7f299a59c700 1 --2- 172.21.15.12:0/873451018 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55b3c7c086f0 0x55b3c7c0ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:36.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.421+0000 7f299a59c700 1 --2- 172.21.15.12:0/873451018 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55b3c7c086f0 0x55b3c7c0ab10 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b8a816b86439c56f server_cookie=58f0107b0dd51661 in_seq=0 out_seq=0 2024-05-17T22:46:36.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.423+0000 7f2981ffb700 10 client.25591 ms_handle_connect on v2:172.21.15.96:6836/4222574338 2024-05-17T22:46:36.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.423+0000 7f2981ffb700 1 -- 172.21.15.12:0/873451018 <== mds.1 v2:172.21.15.96:6836/4222574338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f29900bf2c0 con 0x55b3c7c086f0 2024-05-17T22:46:36.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.423+0000 7f2981ffb700 10 client.25591 handle_command_reply: tid=0 2024-05-17T22:46:36.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.424+0000 7f29af6d6b80 1 client.25591 shutdown 2024-05-17T22:46:36.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.433+0000 7f29af6d6b80 1 -- 172.21.15.12:0/873451018 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3c7b84b00 msgr2=0x55b3c7ba4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:36.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.433+0000 7f29af6d6b80 1 --2- 172.21.15.12:0/873451018 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3c7b84b00 0x55b3c7ba4c90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f29900f1330 tx=0x7f29900bafc0 comp rx=0 tx=0).stop 2024-05-17T22:46:36.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.433+0000 7f29af6d6b80 1 -- 172.21.15.12:0/873451018 shutdown_connections 2024-05-17T22:46:36.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.433+0000 7f29af6d6b80 1 --2- 172.21.15.12:0/873451018 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55b3c7c086f0 0x55b3c7c0ab10 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:36.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.433+0000 7f29af6d6b80 1 --2- 172.21.15.12:0/873451018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3c7b8a510 0x55b3c7ba51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:36.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.433+0000 7f29af6d6b80 1 --2- 172.21.15.12:0/873451018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3c7a069f0 0x55b3c7b96500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:36.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.433+0000 7f29af6d6b80 1 --2- 172.21.15.12:0/873451018 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3c7b84b00 0x55b3c7ba4c90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:36.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.433+0000 7f29af6d6b80 1 -- 172.21.15.12:0/873451018 >> 172.21.15.12:0/873451018 conn(0x55b3c7a667d0 msgr2=0x55b3c7b87920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:46:36.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.434+0000 7f2981ffb700 0 client.25591 ms_handle_reset on v2:172.21.15.96:6836/4222574338 2024-05-17T22:46:36.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.435+0000 7f29af6d6b80 1 -- 172.21.15.12:0/873451018 shutdown_connections 2024-05-17T22:46:36.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.435+0000 7f29af6d6b80 1 -- 172.21.15.12:0/873451018 wait complete. 2024-05-17T22:46:36.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.435+0000 7f29af6d6b80 20 client.25591 trim_cache size 0 max 16384 2024-05-17T22:46:36.439 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:46:36.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.437+0000 7f299b59e700 1 -- 172.21.15.12:0/850060604 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f29840b2a50 msgr2=0x7f29840b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:36.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.437+0000 7f299b59e700 1 --2- 172.21.15.12:0/850060604 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f29840b2a50 0x7f29840b4f60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f298c01d000 tx=0x7f298c017660 comp rx=0 tx=0).stop 2024-05-17T22:46:36.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.437+0000 7f299b59e700 1 -- 172.21.15.12:0/850060604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f299c1032c0 msgr2=0x7f299c10fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:36.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.437+0000 7f299b59e700 1 --2- 172.21.15.12:0/850060604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f299c1032c0 0x7f299c10fc40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f299001afd0 tx=0x7f2990003bd0 comp rx=0 tx=0).stop 2024-05-17T22:46:36.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.438+0000 7f299b59e700 1 -- 172.21.15.12:0/850060604 shutdown_connections 2024-05-17T22:46:36.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.438+0000 7f299b59e700 1 --2- 172.21.15.12:0/850060604 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f29840b2a50 0x7f29840b4f60 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:36.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.438+0000 7f299b59e700 1 --2- 172.21.15.12:0/850060604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f299c107db0 0x7f299c074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:36.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.438+0000 7f299b59e700 1 --2- 172.21.15.12:0/850060604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f299c102970 0x7f299c10f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:36.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.438+0000 7f299b59e700 1 --2- 172.21.15.12:0/850060604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f299c1032c0 0x7f299c10fc40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:36.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.438+0000 7f299b59e700 1 -- 172.21.15.12:0/850060604 >> 172.21.15.12:0/850060604 conn(0x7f299c0fe660 msgr2=0x7f299c1057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:46:36.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.438+0000 7f299b59e700 1 -- 172.21.15.12:0/850060604 shutdown_connections 2024-05-17T22:46:36.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:36.438+0000 7f299b59e700 1 -- 172.21.15.12:0/850060604 wait complete. 2024-05-17T22:46:37.205 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-17T22:46:37.656 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:46:38.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:37 smithi012 ceph-mon[31632]: pgmap v660: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.3 MiB/s wr, 531 op/s 2024-05-17T22:46:38.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:37 smithi151 ceph-mon[36101]: pgmap v660: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.3 MiB/s wr, 531 op/s 2024-05-17T22:46:38.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:37 smithi096 ceph-mon[36065]: pgmap v660: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.3 MiB/s wr, 531 op/s 2024-05-17T22:46:39.920 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:39 smithi012 ceph-mon[31632]: pgmap v661: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.0 MiB/s wr, 598 op/s 2024-05-17T22:46:40.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.075+0000 7f9a20780700 1 -- 172.21.15.12:0/3519991169 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a18102e00 msgr2=0x7f9a18103210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:40.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.075+0000 7f9a20780700 1 --2- 172.21.15.12:0/3519991169 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a18102e00 0x7f9a18103210 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9a14009990 tx=0x7f9a14009ca0 comp rx=0 tx=0).stop 2024-05-17T22:46:40.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.075+0000 7f9a20780700 1 -- 172.21.15.12:0/3519991169 shutdown_connections 2024-05-17T22:46:40.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.075+0000 7f9a20780700 1 --2- 172.21.15.12:0/3519991169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a18108080 0x7f9a1810f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:40.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.075+0000 7f9a20780700 1 --2- 172.21.15.12:0/3519991169 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a18102e00 0x7f9a18103210 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:40.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.075+0000 7f9a20780700 1 --2- 172.21.15.12:0/3519991169 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a18107730 0x7f9a18107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:40.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.075+0000 7f9a20780700 1 -- 172.21.15.12:0/3519991169 >> 172.21.15.12:0/3519991169 conn(0x7f9a18079bd0 msgr2=0x7f9a18079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:46:40.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.075+0000 7f9a20780700 1 -- 172.21.15.12:0/3519991169 shutdown_connections 2024-05-17T22:46:40.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.075+0000 7f9a20780700 1 -- 172.21.15.12:0/3519991169 wait complete. 2024-05-17T22:46:40.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.076+0000 7f9a20780700 1 Processor -- start 2024-05-17T22:46:40.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.076+0000 7f9a20780700 1 -- start start 2024-05-17T22:46:40.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.077+0000 7f9a20780700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a18102e00 0x7f9a1810ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:40.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.077+0000 7f9a20780700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a18107730 0x7f9a1810f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:40.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.077+0000 7f9a20780700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a18108080 0x7f9a181a5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:40.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.077+0000 7f9a20780700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9a18111cc0 con 0x7f9a18107730 2024-05-17T22:46:40.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.077+0000 7f9a20780700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9a18111b40 con 0x7f9a18102e00 2024-05-17T22:46:40.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.077+0000 7f9a20780700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9a18111e40 con 0x7f9a18108080 2024-05-17T22:46:40.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.077+0000 7f9a1e51c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a18102e00 0x7f9a1810ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:40.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.077+0000 7f9a1dd1b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a18107730 0x7f9a1810f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:40.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.077+0000 7f9a1e51c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a18102e00 0x7f9a1810ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:37820/0 (socket says 172.21.15.12:37820) 2024-05-17T22:46:40.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.077+0000 7f9a1dd1b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a18107730 0x7f9a1810f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36806/0 (socket says 172.21.15.12:36806) 2024-05-17T22:46:40.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.077+0000 7f9a1e51c700 1 -- 172.21.15.12:0/1712493971 learned_addr learned my addr 172.21.15.12:0/1712493971 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:46:40.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.077+0000 7f9a1ed1d700 1 --2- 172.21.15.12:0/1712493971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a18108080 0x7f9a181a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:40.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.077+0000 7f9a1e51c700 1 -- 172.21.15.12:0/1712493971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a18108080 msgr2=0x7f9a181a5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:40.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.077+0000 7f9a1e51c700 1 --2- 172.21.15.12:0/1712493971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a18108080 0x7f9a181a5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:40.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.077+0000 7f9a1e51c700 1 -- 172.21.15.12:0/1712493971 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a18107730 msgr2=0x7f9a1810f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:40.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.077+0000 7f9a1e51c700 1 --2- 172.21.15.12:0/1712493971 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a18107730 0x7f9a1810f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:40.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.077+0000 7f9a1e51c700 1 -- 172.21.15.12:0/1712493971 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a181a5d60 con 0x7f9a18102e00 2024-05-17T22:46:40.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.078+0000 7f9a1e51c700 1 --2- 172.21.15.12:0/1712493971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a18102e00 0x7f9a1810ef30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9a14000c00 tx=0x7f9a140057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:46:40.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.078+0000 7f9a0f7fe700 1 -- 172.21.15.12:0/1712493971 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a14026070 con 0x7f9a18102e00 2024-05-17T22:46:40.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.078+0000 7f9a0f7fe700 1 -- 172.21.15.12:0/1712493971 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a14021410 con 0x7f9a18102e00 2024-05-17T22:46:40.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.078+0000 7f9a0f7fe700 1 -- 172.21.15.12:0/1712493971 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a14017650 con 0x7f9a18102e00 2024-05-17T22:46:40.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.078+0000 7f9a20780700 1 -- 172.21.15.12:0/1712493971 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a181a5ff0 con 0x7f9a18102e00 2024-05-17T22:46:40.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.078+0000 7f9a20780700 1 -- 172.21.15.12:0/1712493971 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a181a63c0 con 0x7f9a18102e00 2024-05-17T22:46:40.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.081+0000 7f9a0f7fe700 1 -- 172.21.15.12:0/1712493971 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9a14029070 con 0x7f9a18102e00 2024-05-17T22:46:40.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.083+0000 7f9a2f15bb80 1 Processor -- start 2024-05-17T22:46:40.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.083+0000 7f9a2f15bb80 1 -- start start 2024-05-17T22:46:40.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.084+0000 7f9a2f15bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fac13359f0 0x55fac14b9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:40.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.084+0000 7f9a2f15bb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fac14b9aa0 0x55fac14b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:40.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.084+0000 7f9a2f15bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fac14b4170 0x55fac14b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:40.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.084+0000 7f9a2f15bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55fac13e2290 con 0x55fac13359f0 2024-05-17T22:46:40.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.084+0000 7f9a0f7fe700 1 --2- 172.21.15.12:0/1712493971 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9a040b2930 0x7f9a040b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:40.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.084+0000 7f9a2f15bb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fac13d1fd0 con 0x55fac14b9aa0 2024-05-17T22:46:40.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.084+0000 7f9a2f15bb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55fac13d5e70 con 0x55fac14b4170 2024-05-17T22:46:40.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.084+0000 7f9a0f7fe700 1 -- 172.21.15.12:0/1712493971 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f9a140e3150 con 0x7f9a18102e00 2024-05-17T22:46:40.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.085+0000 7f9a1e51c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fac14b9aa0 0x55fac14b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:40.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.085+0000 7f9a1e51c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fac14b9aa0 0x55fac14b3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:37836/0 (socket says 172.21.15.12:37836) 2024-05-17T22:46:40.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.085+0000 7f9a1e51c700 1 -- 172.21.15.12:0/1939410665 learned_addr learned my addr 172.21.15.12:0/1939410665 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:46:40.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.086+0000 7f9a1e51c700 1 -- 172.21.15.12:0/1939410665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fac14b4170 msgr2=0x55fac14b4580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T22:46:40.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.086+0000 7f9a1e51c700 1 --2- 172.21.15.12:0/1939410665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fac14b4170 0x55fac14b4580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:40.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.086+0000 7f9a1e51c700 1 -- 172.21.15.12:0/1939410665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fac13359f0 msgr2=0x55fac14b9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:46:40.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.086+0000 7f9a1e51c700 1 --2- 172.21.15.12:0/1939410665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fac13359f0 0x55fac14b9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:40.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.086+0000 7f9a1e51c700 1 -- 172.21.15.12:0/1939410665 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fac137b590 con 0x55fac14b9aa0 2024-05-17T22:46:40.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.086+0000 7f9a1e51c700 1 --2- 172.21.15.12:0/1939410665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fac14b9aa0 0x55fac14b3c30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9a140aa400 tx=0x7f9a140aa050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fda3ca972207ec9a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:46:40.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.086+0000 7f9a0cff9700 1 -- 172.21.15.12:0/1939410665 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a140ae820 con 0x55fac14b9aa0 2024-05-17T22:46:40.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.087+0000 7f9a0cff9700 1 -- 172.21.15.12:0/1939410665 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a140b3240 con 0x55fac14b9aa0 2024-05-17T22:46:40.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.087+0000 7f9a0cff9700 1 -- 172.21.15.12:0/1939410665 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a140aa960 con 0x55fac14b9aa0 2024-05-17T22:46:40.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.092+0000 7f9a2f15bb80 1 -- 172.21.15.12:0/1939410665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fac14b9aa0 msgr2=0x55fac14b3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:40.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.092+0000 7f9a2f15bb80 1 --2- 172.21.15.12:0/1939410665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fac14b9aa0 0x55fac14b3c30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9a140aa400 tx=0x7f9a140aa050 comp rx=0 tx=0).stop 2024-05-17T22:46:40.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.110+0000 7f9a1ed1d700 1 --2- 172.21.15.12:0/1712493971 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9a040b2930 0x7f9a040b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:40.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.111+0000 7f9a2f15bb80 1 -- 172.21.15.12:0/1939410665 shutdown_connections 2024-05-17T22:46:40.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.111+0000 7f9a2f15bb80 1 --2- 172.21.15.12:0/1939410665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fac14b4170 0x55fac14b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:40.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.111+0000 7f9a2f15bb80 1 --2- 172.21.15.12:0/1939410665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fac13359f0 0x55fac14b9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:40.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.111+0000 7f9a2f15bb80 1 --2- 172.21.15.12:0/1939410665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fac14b9aa0 0x55fac14b3c30 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:40.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.111+0000 7f9a2f15bb80 1 -- 172.21.15.12:0/1939410665 >> 172.21.15.12:0/1939410665 conn(0x55fac1379f00 msgr2=0x55fac14aeb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:46:40.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.116+0000 7f9a2f15bb80 1 -- 172.21.15.12:0/1939410665 shutdown_connections 2024-05-17T22:46:40.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.117+0000 7f9a2f15bb80 1 -- 172.21.15.12:0/1939410665 wait complete. 2024-05-17T22:46:40.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.117+0000 7f9a2f15bb80 1 -- start start 2024-05-17T22:46:40.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.117+0000 7f9a2f15bb80 1 Processor -- start 2024-05-17T22:46:40.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.119+0000 7f9a1ed1d700 1 --2- 172.21.15.12:0/1712493971 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9a040b2930 0x7f9a040b4e40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9a10009900 tx=0x7f9a1000d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:46:40.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.121+0000 7f9a2f15bb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fac13359f0 0x55fac14bd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:40.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.121+0000 7f9a2f15bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fac14b4170 0x55fac14bd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:40.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.121+0000 7f9a2f15bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fac14b9aa0 0x55fac14bdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:40.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.121+0000 7f9a1e51c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fac13359f0 0x55fac14bd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:40.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.121+0000 7f9a1e51c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fac13359f0 0x55fac14bd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:37852/0 (socket says 172.21.15.12:37852) 2024-05-17T22:46:40.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.121+0000 7f9a1e51c700 1 -- 172.21.15.12:0/1872768132 learned_addr learned my addr 172.21.15.12:0/1872768132 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:46:40.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.122+0000 7f9a1dd1b700 1 --2- 172.21.15.12:0/1872768132 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fac14b4170 0x55fac14bd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:40.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.122+0000 7f9a2f15bb80 1 -- 172.21.15.12:0/1872768132 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55fac12d9150 con 0x55fac14b9aa0 2024-05-17T22:46:40.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.122+0000 7f9a2f15bb80 1 -- 172.21.15.12:0/1872768132 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fac1372e80 con 0x55fac13359f0 2024-05-17T22:46:40.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.122+0000 7f9a2f15bb80 1 -- 172.21.15.12:0/1872768132 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55fac13d1fd0 con 0x55fac14b4170 2024-05-17T22:46:40.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.122+0000 7f9a2f15bb80 10 client.0 _ll_register_callbacks cb 0x55fac136cf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:46:40.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.123+0000 7f9a1ed1d700 1 --2- 172.21.15.12:0/1872768132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fac14b9aa0 0x55fac14bdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:40.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.123+0000 7f9a1e51c700 1 -- 172.21.15.12:0/1872768132 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fac14b4170 msgr2=0x55fac14bd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:46:40.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.123+0000 7f9a1e51c700 1 --2- 172.21.15.12:0/1872768132 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fac14b4170 0x55fac14bd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:40.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.123+0000 7f9a1e51c700 1 -- 172.21.15.12:0/1872768132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fac14b9aa0 msgr2=0x55fac14bdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:46:40.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.123+0000 7f9a1e51c700 1 --2- 172.21.15.12:0/1872768132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fac14b9aa0 0x55fac14bdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:40.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.123+0000 7f9a1e51c700 1 -- 172.21.15.12:0/1872768132 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fac1322480 con 0x55fac13359f0 2024-05-17T22:46:40.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.129+0000 7f9a1e51c700 1 --2- 172.21.15.12:0/1872768132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fac13359f0 0x55fac14bd1a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9a140e0910 tx=0x7f9a140aaa10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=134373bd95b9c2b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:46:40.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.130+0000 7f9a01ffb700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:46:40.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.135+0000 7f9a01ffb700 1 -- 172.21.15.12:0/1872768132 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a140c4060 con 0x55fac13359f0 2024-05-17T22:46:40.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.135+0000 7f9a01ffb700 1 -- 172.21.15.12:0/1872768132 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f9a140094b0 con 0x55fac13359f0 2024-05-17T22:46:40.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.135+0000 7f9a2f15bb80 1 -- 172.21.15.12:0/1872768132 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a140b3240 con 0x55fac13359f0 2024-05-17T22:46:40.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.141+0000 7f9a01ffb700 1 -- 172.21.15.12:0/1872768132 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f9a140a4b90 con 0x55fac13359f0 2024-05-17T22:46:40.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.144+0000 7f9a2f15bb80 10 client.25643 fetch_fsmap learned FSMap version 37 2024-05-17T22:46:40.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.144+0000 7f9a2f15bb80 1 -- 172.21.15.12:0/1872768132 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55fac14dbaa0 con 0x55fac13359f0 2024-05-17T22:46:40.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.149+0000 7f9a01ffb700 1 -- 172.21.15.12:0/1872768132 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a140b3240 con 0x55fac13359f0 2024-05-17T22:46:40.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.149+0000 7f9a01ffb700 1 -- 172.21.15.12:0/1872768132 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f9a14005220 con 0x55fac13359f0 2024-05-17T22:46:40.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.150+0000 7f9a2f15bb80 10 client.25643 fetch_fsmap finished waiting for FSMap version 37 2024-05-17T22:46:40.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.150+0000 7f9a2f15bb80 10 client.25643 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:46:40.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.150+0000 7f9a2f15bb80 20 client.25643 populate_metadata read hostname 'smithi012' 2024-05-17T22:46:40.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.150+0000 7f9a2f15bb80 1 --2- 172.21.15.12:0/1872768132 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55fac14ddda0 0x55fac14e01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:40.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.150+0000 7f9a2f15bb80 4 client.25643 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:46:40.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.150+0000 7f9a2f15bb80 1 -- 172.21.15.12:0/1872768132 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fac13e41e0 con 0x55fac14ddda0 2024-05-17T22:46:40.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.152+0000 7f9a1dd1b700 1 --2- 172.21.15.12:0/1872768132 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55fac14ddda0 0x55fac14e01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:40.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.156+0000 7f9a1dd1b700 1 --2- 172.21.15.12:0/1872768132 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55fac14ddda0 0x55fac14e01c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e61bf17ad0ca0e2 server_cookie=510a9558577ccecf in_seq=0 out_seq=0 2024-05-17T22:46:40.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.156+0000 7f9a01ffb700 10 client.25643 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:46:40.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.158+0000 7f9a01ffb700 1 -- 172.21.15.12:0/1872768132 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fac13e41e0 con 0x55fac14ddda0 2024-05-17T22:46:40.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.158+0000 7f9a01ffb700 10 client.25643 handle_command_reply: tid=0 2024-05-17T22:46:40.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.158+0000 7f9a2f15bb80 1 client.25643 shutdown 2024-05-17T22:46:40.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.165+0000 7f9a2f15bb80 1 -- 172.21.15.12:0/1872768132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fac13359f0 msgr2=0x55fac14bd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:40.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.165+0000 7f9a2f15bb80 1 --2- 172.21.15.12:0/1872768132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fac13359f0 0x55fac14bd1a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9a140e0910 tx=0x7f9a140aaa10 comp rx=0 tx=0).stop 2024-05-17T22:46:40.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.169+0000 7f9a2f15bb80 1 -- 172.21.15.12:0/1872768132 shutdown_connections 2024-05-17T22:46:40.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.169+0000 7f9a2f15bb80 1 --2- 172.21.15.12:0/1872768132 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55fac14ddda0 0x55fac14e01c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:40.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.169+0000 7f9a2f15bb80 1 --2- 172.21.15.12:0/1872768132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fac14b9aa0 0x55fac14bdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:40.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.169+0000 7f9a2f15bb80 1 --2- 172.21.15.12:0/1872768132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fac13359f0 0x55fac14bd1a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:40.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.169+0000 7f9a2f15bb80 1 --2- 172.21.15.12:0/1872768132 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fac14b4170 0x55fac14bd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:40.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.169+0000 7f9a2f15bb80 1 -- 172.21.15.12:0/1872768132 >> 172.21.15.12:0/1872768132 conn(0x55fac1379f00 msgr2=0x55fac14a8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:46:40.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.169+0000 7f9a01ffb700 0 client.25643 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:46:40.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.169+0000 7f9a2f15bb80 1 -- 172.21.15.12:0/1872768132 shutdown_connections 2024-05-17T22:46:40.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.169+0000 7f9a2f15bb80 1 -- 172.21.15.12:0/1872768132 wait complete. 2024-05-17T22:46:40.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.170+0000 7f9a2f15bb80 20 client.25643 trim_cache size 0 max 16384 2024-05-17T22:46:40.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.201+0000 7f9a2f15bb80 1 Processor -- start 2024-05-17T22:46:40.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.201+0000 7f9a2f15bb80 1 -- start start 2024-05-17T22:46:40.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.202+0000 7f9a2f15bb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fac13359f0 0x55fac14b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:40.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.202+0000 7f9a2f15bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fac14b4170 0x55fac14a8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:40.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.202+0000 7f9a2f15bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fac14b9aa0 0x55fac14b88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:40.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.202+0000 7f9a1dd1b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fac14b4170 0x55fac14a8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:40.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.202+0000 7f9a1dd1b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fac14b4170 0x55fac14a8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36824/0 (socket says 172.21.15.12:36824) 2024-05-17T22:46:40.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.202+0000 7f9a1e51c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fac13359f0 0x55fac14b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:40.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.202+0000 7f9a1e51c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fac13359f0 0x55fac14b4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:37866/0 (socket says 172.21.15.12:37866) 2024-05-17T22:46:40.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.203+0000 7f9a1ed1d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fac14b9aa0 0x55fac14b88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:40.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.203+0000 7f9a1ed1d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fac14b9aa0 0x55fac14b88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:59512/0 (socket says 172.21.15.12:59512) 2024-05-17T22:46:40.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.203+0000 7f9a1dd1b700 1 -- 172.21.15.12:0/1160601830 learned_addr learned my addr 172.21.15.12:0/1160601830 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:46:40.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.203+0000 7f9a2f15bb80 1 -- 172.21.15.12:0/1160601830 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9a181b0460 con 0x55fac14b4170 2024-05-17T22:46:40.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.203+0000 7f9a2f15bb80 1 -- 172.21.15.12:0/1160601830 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9a181adcf0 con 0x55fac13359f0 2024-05-17T22:46:40.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.203+0000 7f9a2f15bb80 1 -- 172.21.15.12:0/1160601830 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9a18073bc0 con 0x55fac14b9aa0 2024-05-17T22:46:40.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.206+0000 7f9a1ed1d700 1 -- 172.21.15.12:0/1160601830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fac13359f0 msgr2=0x55fac14b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:46:40.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.206+0000 7f9a1ed1d700 1 --2- 172.21.15.12:0/1160601830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fac13359f0 0x55fac14b4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:40.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.206+0000 7f9a1ed1d700 1 -- 172.21.15.12:0/1160601830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fac14b4170 msgr2=0x55fac14a8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:46:40.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.206+0000 7f9a1ed1d700 1 --2- 172.21.15.12:0/1160601830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fac14b4170 0x55fac14a8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:40.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.206+0000 7f9a1ed1d700 1 -- 172.21.15.12:0/1160601830 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fac14bb750 con 0x55fac14b9aa0 2024-05-17T22:46:40.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.209+0000 7f9a1ed1d700 1 --2- 172.21.15.12:0/1160601830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fac14b9aa0 0x55fac14b88f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x55fac14ba070 tx=0x7f9a10015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=443c568819b9e7e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:46:40.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.213+0000 7f9a03fff700 1 -- 172.21.15.12:0/1160601830 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a10022470 con 0x55fac14b9aa0 2024-05-17T22:46:40.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.214+0000 7f9a03fff700 1 -- 172.21.15.12:0/1160601830 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a10027070 con 0x55fac14b9aa0 2024-05-17T22:46:40.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.214+0000 7f9a03fff700 1 -- 172.21.15.12:0/1160601830 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a100158a0 con 0x55fac14b9aa0 2024-05-17T22:46:40.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.216+0000 7f9a2f15bb80 1 -- 172.21.15.12:0/1160601830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fac14b9aa0 msgr2=0x55fac14b88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:40.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.216+0000 7f9a2f15bb80 1 --2- 172.21.15.12:0/1160601830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fac14b9aa0 0x55fac14b88f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x55fac14ba070 tx=0x7f9a10015000 comp rx=0 tx=0).stop 2024-05-17T22:46:40.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.218+0000 7f9a2f15bb80 1 -- 172.21.15.12:0/1160601830 shutdown_connections 2024-05-17T22:46:40.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.218+0000 7f9a2f15bb80 1 --2- 172.21.15.12:0/1160601830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fac14b9aa0 0x55fac14b88f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:40.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.218+0000 7f9a2f15bb80 1 --2- 172.21.15.12:0/1160601830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fac13359f0 0x55fac14b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:40.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.218+0000 7f9a2f15bb80 1 --2- 172.21.15.12:0/1160601830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fac14b4170 0x55fac14a8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:40.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.218+0000 7f9a2f15bb80 1 -- 172.21.15.12:0/1160601830 >> 172.21.15.12:0/1160601830 conn(0x55fac1379f00 msgr2=0x55fac14b81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:46:40.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.218+0000 7f9a2f15bb80 1 -- 172.21.15.12:0/1160601830 shutdown_connections 2024-05-17T22:46:40.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.218+0000 7f9a2f15bb80 1 -- 172.21.15.12:0/1160601830 wait complete. 2024-05-17T22:46:40.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.219+0000 7f9a2f15bb80 1 -- start start 2024-05-17T22:46:40.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.219+0000 7f9a2f15bb80 1 Processor -- start 2024-05-17T22:46:40.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.220+0000 7f9a2f15bb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fac13359f0 0x55fac14c53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:40.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.220+0000 7f9a2f15bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fac14b4170 0x55fac14c5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:40.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.221+0000 7f9a1e51c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fac13359f0 0x55fac14c53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:40.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.221+0000 7f9a1e51c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fac13359f0 0x55fac14c53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:37878/0 (socket says 172.21.15.12:37878) 2024-05-17T22:46:40.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.221+0000 7f9a1e51c700 1 -- 172.21.15.12:0/2547097152 learned_addr learned my addr 172.21.15.12:0/2547097152 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:46:40.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.221+0000 7f9a1dd1b700 1 --2- 172.21.15.12:0/2547097152 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fac14b4170 0x55fac14c5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:40.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.221+0000 7f9a2f15bb80 1 --2- 172.21.15.12:0/2547097152 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fac14b9aa0 0x55fac14e4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:40.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.221+0000 7f9a2f15bb80 1 -- 172.21.15.12:0/2547097152 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9a18074230 con 0x55fac14b4170 2024-05-17T22:46:40.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.222+0000 7f9a1ed1d700 1 --2- 172.21.15.12:0/2547097152 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fac14b9aa0 0x55fac14e4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:40.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.222+0000 7f9a2f15bb80 1 -- 172.21.15.12:0/2547097152 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9a180756a0 con 0x55fac13359f0 2024-05-17T22:46:40.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.222+0000 7f9a2f15bb80 1 -- 172.21.15.12:0/2547097152 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9a18074900 con 0x55fac14b9aa0 2024-05-17T22:46:40.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.226+0000 7f9a1dd1b700 1 -- 172.21.15.12:0/2547097152 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fac14b9aa0 msgr2=0x55fac14e4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:46:40.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.226+0000 7f9a1dd1b700 1 --2- 172.21.15.12:0/2547097152 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fac14b9aa0 0x55fac14e4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:40.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.226+0000 7f9a1dd1b700 1 -- 172.21.15.12:0/2547097152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fac13359f0 msgr2=0x55fac14c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:46:40.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.226+0000 7f9a1dd1b700 1 --2- 172.21.15.12:0/2547097152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fac13359f0 0x55fac14c53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:40.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.226+0000 7f9a1dd1b700 1 -- 172.21.15.12:0/2547097152 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fac14e5280 con 0x55fac14b4170 2024-05-17T22:46:40.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.226+0000 7f9a1e51c700 1 --2- 172.21.15.12:0/2547097152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fac13359f0 0x55fac14c53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:46:40.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.226+0000 7f9a2f15bb80 10 client.0 _ll_register_callbacks cb 0x55fac136cf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:46:40.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.226+0000 7f9a2f15bb80 1 -- 172.21.15.12:0/2547097152 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a10027070 con 0x55fac14b4170 2024-05-17T22:46:40.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.227+0000 7f9a1dd1b700 1 --2- 172.21.15.12:0/2547097152 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fac14b4170 0x55fac14c5970 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9a1807a3d0 tx=0x7f9a08004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59c438cddbcb3839 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:46:40.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.227+0000 7f9a01ffb700 10 client.15990 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:46:40.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.227+0000 7f9a01ffb700 1 -- 172.21.15.12:0/2547097152 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a08003e50 con 0x55fac14b4170 2024-05-17T22:46:40.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.227+0000 7f9a01ffb700 1 -- 172.21.15.12:0/2547097152 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f9a08005050 con 0x55fac14b4170 2024-05-17T22:46:40.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.228+0000 7f9a01ffb700 1 -- 172.21.15.12:0/2547097152 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f9a08005270 con 0x55fac14b4170 2024-05-17T22:46:40.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.228+0000 7f9a2f15bb80 10 client.15990 fetch_fsmap learned FSMap version 37 2024-05-17T22:46:40.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.228+0000 7f9a2f15bb80 1 -- 172.21.15.12:0/2547097152 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({fsmap=37}) -- 0x55fac15474f0 con 0x55fac14b4170 2024-05-17T22:46:40.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.229+0000 7f9a01ffb700 1 -- 172.21.15.12:0/2547097152 <== mon.0 v2:172.21.15.12:3300/0 4 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f9a08005410 con 0x55fac14b4170 2024-05-17T22:46:40.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.229+0000 7f9a2f15bb80 10 client.15990 fetch_fsmap finished waiting for FSMap version 37 2024-05-17T22:46:40.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.229+0000 7f9a2f15bb80 10 client.15990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:46:40.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.229+0000 7f9a2f15bb80 20 client.15990 populate_metadata read hostname 'smithi012' 2024-05-17T22:46:40.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.229+0000 7f9a2f15bb80 1 --2- 172.21.15.12:0/2547097152 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55fac1547930 0x55fac1549dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:40.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.229+0000 7f9a2f15bb80 4 client.15990 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:46:40.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.229+0000 7f9a2f15bb80 1 -- 172.21.15.12:0/2547097152 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fac13dd9c0 con 0x55fac1547930 2024-05-17T22:46:40.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.229+0000 7f9a1e51c700 1 --2- 172.21.15.12:0/2547097152 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55fac1547930 0x55fac1549dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:40.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.231+0000 7f9a1e51c700 1 --2- 172.21.15.12:0/2547097152 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55fac1547930 0x55fac1549dd0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffee01d045be106d server_cookie=3a32309881642a1 in_seq=0 out_seq=0 2024-05-17T22:46:40.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.231+0000 7f9a01ffb700 10 client.15990 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:46:40.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.231+0000 7f9a01ffb700 1 -- 172.21.15.12:0/2547097152 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fac13dd9c0 con 0x55fac1547930 2024-05-17T22:46:40.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.231+0000 7f9a01ffb700 10 client.15990 handle_command_reply: tid=0 2024-05-17T22:46:40.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.231+0000 7f9a2f15bb80 1 client.15990 shutdown 2024-05-17T22:46:40.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.234+0000 7f9a2f15bb80 1 -- 172.21.15.12:0/2547097152 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fac14b4170 msgr2=0x55fac14c5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:40.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.234+0000 7f9a2f15bb80 1 --2- 172.21.15.12:0/2547097152 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fac14b4170 0x55fac14c5970 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9a1807a3d0 tx=0x7f9a08004320 comp rx=0 tx=0).stop 2024-05-17T22:46:40.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.234+0000 7f9a2f15bb80 1 -- 172.21.15.12:0/2547097152 shutdown_connections 2024-05-17T22:46:40.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.234+0000 7f9a2f15bb80 1 --2- 172.21.15.12:0/2547097152 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fac14b9aa0 0x55fac14e4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:40.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.234+0000 7f9a2f15bb80 1 --2- 172.21.15.12:0/2547097152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fac13359f0 0x55fac14c53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:40.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.234+0000 7f9a2f15bb80 1 --2- 172.21.15.12:0/2547097152 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55fac1547930 0x55fac1549dd0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:40.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.234+0000 7f9a2f15bb80 1 --2- 172.21.15.12:0/2547097152 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fac14b4170 0x55fac14c5970 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:40.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.234+0000 7f9a2f15bb80 1 -- 172.21.15.12:0/2547097152 >> 172.21.15.12:0/2547097152 conn(0x55fac1379f00 msgr2=0x55fac14b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:46:40.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.234+0000 7f9a01ffb700 0 client.15990 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:46:40.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.236+0000 7f9a2f15bb80 1 -- 172.21.15.12:0/2547097152 shutdown_connections 2024-05-17T22:46:40.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.236+0000 7f9a2f15bb80 1 -- 172.21.15.12:0/2547097152 wait complete. 2024-05-17T22:46:40.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.236+0000 7f9a2f15bb80 20 client.15990 trim_cache size 0 max 16384 2024-05-17T22:46:40.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:39 smithi151 ceph-mon[36101]: pgmap v661: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.0 MiB/s wr, 598 op/s 2024-05-17T22:46:40.241 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:46:40.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.239+0000 7f9a20780700 1 -- 172.21.15.12:0/1712493971 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9a040b2930 msgr2=0x7f9a040b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:40.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.239+0000 7f9a20780700 1 --2- 172.21.15.12:0/1712493971 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9a040b2930 0x7f9a040b4e40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9a10009900 tx=0x7f9a1000d040 comp rx=0 tx=0).stop 2024-05-17T22:46:40.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.239+0000 7f9a20780700 1 -- 172.21.15.12:0/1712493971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a18102e00 msgr2=0x7f9a1810ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:40.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.239+0000 7f9a20780700 1 --2- 172.21.15.12:0/1712493971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a18102e00 0x7f9a1810ef30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9a14000c00 tx=0x7f9a140057c0 comp rx=0 tx=0).stop 2024-05-17T22:46:40.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.239+0000 7f9a20780700 1 -- 172.21.15.12:0/1712493971 shutdown_connections 2024-05-17T22:46:40.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.240+0000 7f9a20780700 1 --2- 172.21.15.12:0/1712493971 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9a040b2930 0x7f9a040b4e40 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:40.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.240+0000 7f9a20780700 1 --2- 172.21.15.12:0/1712493971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a18108080 0x7f9a181a5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:40.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.240+0000 7f9a20780700 1 --2- 172.21.15.12:0/1712493971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a18102e00 0x7f9a1810ef30 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:40.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.240+0000 7f9a20780700 1 --2- 172.21.15.12:0/1712493971 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a18107730 0x7f9a1810f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:40.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.240+0000 7f9a20780700 1 -- 172.21.15.12:0/1712493971 >> 172.21.15.12:0/1712493971 conn(0x7f9a18079bd0 msgr2=0x7f9a18106710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:46:40.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.240+0000 7f9a20780700 1 -- 172.21.15.12:0/1712493971 shutdown_connections 2024-05-17T22:46:40.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:40.240+0000 7f9a20780700 1 -- 172.21.15.12:0/1712493971 wait complete. 2024-05-17T22:46:40.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:39 smithi096 ceph-mon[36065]: pgmap v661: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.0 MiB/s wr, 598 op/s 2024-05-17T22:46:41.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 440.0 s, 487.4 tps, lat 199.545 ms stddev 270.207 2024-05-17T22:46:42.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:41 smithi012 ceph-mon[31632]: pgmap v662: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.9 MiB/s wr, 499 op/s 2024-05-17T22:46:42.242 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:41 smithi151 ceph-mon[36101]: pgmap v662: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.9 MiB/s wr, 499 op/s 2024-05-17T22:46:42.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:41 smithi096 ceph-mon[36065]: pgmap v662: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.9 MiB/s wr, 499 op/s 2024-05-17T22:46:43.114 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-17T22:46:43.114 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-17T22:46:43.667 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:46:44.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:43 smithi151 ceph-mon[36101]: pgmap v663: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.1 MiB/s wr, 526 op/s 2024-05-17T22:46:44.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:43 smithi096 ceph-mon[36065]: pgmap v663: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.1 MiB/s wr, 526 op/s 2024-05-17T22:46:44.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:43 smithi012 ceph-mon[31632]: pgmap v663: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.1 MiB/s wr, 526 op/s 2024-05-17T22:46:46.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:45 smithi151 ceph-mon[36101]: pgmap v664: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.2 MiB/s wr, 530 op/s 2024-05-17T22:46:46.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:45 smithi096 ceph-mon[36065]: pgmap v664: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.2 MiB/s wr, 530 op/s 2024-05-17T22:46:46.300 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:45 smithi012 ceph-mon[31632]: pgmap v664: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.2 MiB/s wr, 530 op/s 2024-05-17T22:46:46.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 445.0 s, 548.6 tps, lat 186.827 ms stddev 217.465 2024-05-17T22:46:48.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.000+0000 7f1e7eaa7700 1 -- 172.21.15.12:0/3319132593 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e7807c6d0 msgr2=0x7f1e7807cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:48.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.000+0000 7f1e7eaa7700 1 --2- 172.21.15.12:0/3319132593 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e7807c6d0 0x7f1e7807cae0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c0098f0 tx=0x7f1e6c009c00 comp rx=0 tx=0).stop 2024-05-17T22:46:48.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.000+0000 7f1e7eaa7700 1 -- 172.21.15.12:0/3319132593 shutdown_connections 2024-05-17T22:46:48.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.000+0000 7f1e7eaa7700 1 --2- 172.21.15.12:0/3319132593 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e7811d1b0 0x7f1e7811f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:48.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.000+0000 7f1e7eaa7700 1 --2- 172.21.15.12:0/3319132593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e78084330 0x7f1e7807c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:48.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.000+0000 7f1e7eaa7700 1 --2- 172.21.15.12:0/3319132593 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e7807c6d0 0x7f1e7807cae0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:48.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.000+0000 7f1e7eaa7700 1 -- 172.21.15.12:0/3319132593 >> 172.21.15.12:0/3319132593 conn(0x7f1e78078570 msgr2=0x7f1e78108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:46:48.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.001+0000 7f1e7eaa7700 1 -- 172.21.15.12:0/3319132593 shutdown_connections 2024-05-17T22:46:48.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.001+0000 7f1e7eaa7700 1 -- 172.21.15.12:0/3319132593 wait complete. 2024-05-17T22:46:48.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.002+0000 7f1e7eaa7700 1 Processor -- start 2024-05-17T22:46:48.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.002+0000 7f1e7eaa7700 1 -- start start 2024-05-17T22:46:48.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.002+0000 7f1e7eaa7700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e7807c6d0 0x7f1e781a96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:48.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.002+0000 7f1e7eaa7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e78084330 0x7f1e781a9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:48.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.002+0000 7f1e7eaa7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e7811d1b0 0x7f1e781adf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:48.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.002+0000 7f1e7eaa7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1e78122cf0 con 0x7f1e78084330 2024-05-17T22:46:48.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.002+0000 7f1e7eaa7700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1e78122b70 con 0x7f1e7811d1b0 2024-05-17T22:46:48.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.002+0000 7f1e7eaa7700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1e78122e70 con 0x7f1e7807c6d0 2024-05-17T22:46:48.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.003+0000 7f1e7d044700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e7811d1b0 0x7f1e781adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:48.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.003+0000 7f1e7c843700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e7807c6d0 0x7f1e781a96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:48.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.003+0000 7f1e7d044700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e7811d1b0 0x7f1e781adf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:37896/0 (socket says 172.21.15.12:37896) 2024-05-17T22:46:48.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.003+0000 7f1e7c843700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e7807c6d0 0x7f1e781a96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:59534/0 (socket says 172.21.15.12:59534) 2024-05-17T22:46:48.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.003+0000 7f1e7d044700 1 -- 172.21.15.12:0/2546467383 learned_addr learned my addr 172.21.15.12:0/2546467383 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:46:48.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.003+0000 7f1e7c843700 1 -- 172.21.15.12:0/2546467383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e7811d1b0 msgr2=0x7f1e781adf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:48.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.003+0000 7f1e7c843700 1 --2- 172.21.15.12:0/2546467383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e7811d1b0 0x7f1e781adf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:48.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.003+0000 7f1e7c843700 1 -- 172.21.15.12:0/2546467383 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e78084330 msgr2=0x7f1e781a9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:48.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.003+0000 7f1e7c843700 1 --2- 172.21.15.12:0/2546467383 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e78084330 0x7f1e781a9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:48.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.003+0000 7f1e7c843700 1 -- 172.21.15.12:0/2546467383 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e781ae5a0 con 0x7f1e7807c6d0 2024-05-17T22:46:48.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.003+0000 7f1e7d044700 1 --2- 172.21.15.12:0/2546467383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e7811d1b0 0x7f1e781adf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:46:48.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.004+0000 7f1e7c843700 1 --2- 172.21.15.12:0/2546467383 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e7807c6d0 0x7f1e781a96d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1e6800e9c0 tx=0x7f1e6800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:46:48.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.005+0000 7f1e75ffb700 1 -- 172.21.15.12:0/2546467383 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6800c9c0 con 0x7f1e7807c6d0 2024-05-17T22:46:48.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.005+0000 7f1e7eaa7700 1 -- 172.21.15.12:0/2546467383 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e781ae890 con 0x7f1e7807c6d0 2024-05-17T22:46:48.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.006+0000 7f1e7eaa7700 1 -- 172.21.15.12:0/2546467383 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e781b61b0 con 0x7f1e7807c6d0 2024-05-17T22:46:48.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.006+0000 7f1e75ffb700 1 -- 172.21.15.12:0/2546467383 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e6800cb60 con 0x7f1e7807c6d0 2024-05-17T22:46:48.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.006+0000 7f1e75ffb700 1 -- 172.21.15.12:0/2546467383 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e68018680 con 0x7f1e7807c6d0 2024-05-17T22:46:48.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.009+0000 7f1e75ffb700 1 -- 172.21.15.12:0/2546467383 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1e68018820 con 0x7f1e7807c6d0 2024-05-17T22:46:48.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.011+0000 7f1e75ffb700 1 --2- 172.21.15.12:0/2546467383 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f1e600b2a90 0x7f1e600b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:48.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.011+0000 7f1e77fff700 1 --2- 172.21.15.12:0/2546467383 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f1e600b2a90 0x7f1e600b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:48.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.011+0000 7f1e75ffb700 1 -- 172.21.15.12:0/2546467383 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f1e680e2540 con 0x7f1e7807c6d0 2024-05-17T22:46:48.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.012+0000 7f1e8d482b80 1 Processor -- start 2024-05-17T22:46:48.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.012+0000 7f1e8d482b80 1 -- start start 2024-05-17T22:46:48.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.012+0000 7f1e77fff700 1 --2- 172.21.15.12:0/2546467383 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f1e600b2a90 0x7f1e600b4fa0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c009680 tx=0x7f1e6c009f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:46:48.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.012+0000 7f1e8d482b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9105b2a70 0x55f910735f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:48.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.012+0000 7f1e8d482b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f910736480 0x55f910730640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:48.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.012+0000 7f1e8d482b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f910730b80 0x55f910730f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:48.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.013+0000 7f1e8d482b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f91065f150 con 0x55f9105b2a70 2024-05-17T22:46:48.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.013+0000 7f1e8d482b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f91064ecf0 con 0x55f910730b80 2024-05-17T22:46:48.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.013+0000 7f1e8d482b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f910652d30 con 0x55f910736480 2024-05-17T22:46:48.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.013+0000 7f1e7d044700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f910730b80 0x55f910730f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:48.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.013+0000 7f1e7d044700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f910730b80 0x55f910730f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:37902/0 (socket says 172.21.15.12:37902) 2024-05-17T22:46:48.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.013+0000 7f1e7d044700 1 -- 172.21.15.12:0/3085038860 learned_addr learned my addr 172.21.15.12:0/3085038860 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:46:48.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.013+0000 7f1e77fff700 1 --2- 172.21.15.12:0/3085038860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f910736480 0x55f910730640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:48.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.013+0000 7f1e7c843700 1 --2- 172.21.15.12:0/3085038860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9105b2a70 0x55f910735f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:48.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.013+0000 7f1e7d044700 1 -- 172.21.15.12:0/3085038860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f910736480 msgr2=0x55f910730640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:46:48.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.013+0000 7f1e7d044700 1 --2- 172.21.15.12:0/3085038860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f910736480 0x55f910730640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:48.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.013+0000 7f1e7d044700 1 -- 172.21.15.12:0/3085038860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9105b2a70 msgr2=0x55f910735f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:46:48.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.013+0000 7f1e7d044700 1 --2- 172.21.15.12:0/3085038860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9105b2a70 0x55f910735f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:48.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.013+0000 7f1e7d044700 1 -- 172.21.15.12:0/3085038860 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9105e8640 con 0x55f910730b80 2024-05-17T22:46:48.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.013+0000 7f1e7c843700 1 --2- 172.21.15.12:0/3085038860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9105b2a70 0x55f910735f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:46:48.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.014+0000 7f1e7d044700 1 --2- 172.21.15.12:0/3085038860 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f910730b80 0x55f910730f90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000e5e0 tx=0x7f1e70009710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a096b0a807f7ba7a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:46:48.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.014+0000 7f1e5effd700 1 -- 172.21.15.12:0/3085038860 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70017020 con 0x55f910730b80 2024-05-17T22:46:48.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.014+0000 7f1e5effd700 1 -- 172.21.15.12:0/3085038860 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e70013d80 con 0x55f910730b80 2024-05-17T22:46:48.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.014+0000 7f1e5effd700 1 -- 172.21.15.12:0/3085038860 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70007d90 con 0x55f910730b80 2024-05-17T22:46:48.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.015+0000 7f1e8d482b80 1 -- 172.21.15.12:0/3085038860 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f910730b80 msgr2=0x55f910730f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:48.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.015+0000 7f1e8d482b80 1 --2- 172.21.15.12:0/3085038860 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f910730b80 0x55f910730f90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000e5e0 tx=0x7f1e70009710 comp rx=0 tx=0).stop 2024-05-17T22:46:48.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.015+0000 7f1e8d482b80 1 -- 172.21.15.12:0/3085038860 shutdown_connections 2024-05-17T22:46:48.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.015+0000 7f1e8d482b80 1 --2- 172.21.15.12:0/3085038860 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f910730b80 0x55f910730f90 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:48.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.015+0000 7f1e8d482b80 1 --2- 172.21.15.12:0/3085038860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9105b2a70 0x55f910735f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:48.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.015+0000 7f1e8d482b80 1 --2- 172.21.15.12:0/3085038860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f910736480 0x55f910730640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:48.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.015+0000 7f1e8d482b80 1 -- 172.21.15.12:0/3085038860 >> 172.21.15.12:0/3085038860 conn(0x55f910612850 msgr2=0x55f91072b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:46:48.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.015+0000 7f1e8d482b80 1 -- 172.21.15.12:0/3085038860 shutdown_connections 2024-05-17T22:46:48.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.015+0000 7f1e8d482b80 1 -- 172.21.15.12:0/3085038860 wait complete. 2024-05-17T22:46:48.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.016+0000 7f1e8d482b80 1 -- start start 2024-05-17T22:46:48.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.016+0000 7f1e8d482b80 1 Processor -- start 2024-05-17T22:46:48.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.017+0000 7f1e8d482b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f9105b2a70 0x55f910739ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:48.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.017+0000 7f1e8d482b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f910730b80 0x55f910739ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:48.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.017+0000 7f1e7c843700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f9105b2a70 0x55f910739ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:48.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.017+0000 7f1e7c843700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f9105b2a70 0x55f910739ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:59552/0 (socket says 172.21.15.12:59552) 2024-05-17T22:46:48.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.017+0000 7f1e7c843700 1 -- 172.21.15.12:0/3757356421 learned_addr learned my addr 172.21.15.12:0/3757356421 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:46:48.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.017+0000 7f1e8d482b80 1 --2- 172.21.15.12:0/3757356421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f910736480 0x55f91073a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:48.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.017+0000 7f1e77fff700 1 --2- 172.21.15.12:0/3757356421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f910730b80 0x55f910739ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:48.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.017+0000 7f1e8d482b80 1 -- 172.21.15.12:0/3757356421 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f9105561d0 con 0x55f910736480 2024-05-17T22:46:48.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.018+0000 7f1e7d044700 1 --2- 172.21.15.12:0/3757356421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f910736480 0x55f91073a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:48.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.018+0000 7f1e8d482b80 1 -- 172.21.15.12:0/3757356421 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f9105f09c0 con 0x55f910730b80 2024-05-17T22:46:48.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.018+0000 7f1e8d482b80 1 -- 172.21.15.12:0/3757356421 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f91073ce50 con 0x55f9105b2a70 2024-05-17T22:46:48.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.018+0000 7f1e8d482b80 10 client.0 _ll_register_callbacks cb 0x55f9105e5bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:46:48.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.018+0000 7f1e7d044700 1 -- 172.21.15.12:0/3757356421 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f9105b2a70 msgr2=0x55f910739ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:46:48.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.018+0000 7f1e7d044700 1 --2- 172.21.15.12:0/3757356421 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f9105b2a70 0x55f910739ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:48.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.018+0000 7f1e7d044700 1 -- 172.21.15.12:0/3757356421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f910730b80 msgr2=0x55f910739ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:46:48.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.018+0000 7f1e7d044700 1 --2- 172.21.15.12:0/3757356421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f910730b80 0x55f910739ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:48.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.018+0000 7f1e7d044700 1 -- 172.21.15.12:0/3757356421 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f91059f500 con 0x55f910736480 2024-05-17T22:46:48.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.018+0000 7f1e77fff700 1 --2- 172.21.15.12:0/3757356421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f910730b80 0x55f910739ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:46:48.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.019+0000 7f1e7d044700 1 --2- 172.21.15.12:0/3757356421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f910736480 0x55f91073a530 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000b010 tx=0x7f1e70007c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f5e0e8cb2a4cac6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:46:48.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.019+0000 7f1e57fff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:46:48.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.019+0000 7f1e57fff700 1 -- 172.21.15.12:0/3757356421 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70017040 con 0x55f910736480 2024-05-17T22:46:48.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.019+0000 7f1e8d482b80 1 -- 172.21.15.12:0/3757356421 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e70013d80 con 0x55f910736480 2024-05-17T22:46:48.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.019+0000 7f1e57fff700 1 -- 172.21.15.12:0/3757356421 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f1e700138e0 con 0x55f910736480 2024-05-17T22:46:48.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.019+0000 7f1e57fff700 1 -- 172.21.15.12:0/3757356421 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1e70031050 con 0x55f910736480 2024-05-17T22:46:48.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.019+0000 7f1e8d482b80 10 client.15993 fetch_fsmap learned FSMap version 37 2024-05-17T22:46:48.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.019+0000 7f1e8d482b80 1 -- 172.21.15.12:0/3757356421 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f910747870 con 0x55f910736480 2024-05-17T22:46:48.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.019+0000 7f1e57fff700 1 -- 172.21.15.12:0/3757356421 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e70013d80 con 0x55f910736480 2024-05-17T22:46:48.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.020+0000 7f1e57fff700 1 -- 172.21.15.12:0/3757356421 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f1e7001c430 con 0x55f910736480 2024-05-17T22:46:48.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.020+0000 7f1e8d482b80 10 client.15993 fetch_fsmap finished waiting for FSMap version 37 2024-05-17T22:46:48.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.020+0000 7f1e8d482b80 10 client.15993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:46:48.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.020+0000 7f1e8d482b80 20 client.15993 populate_metadata read hostname 'smithi012' 2024-05-17T22:46:48.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.020+0000 7f1e8d482b80 1 --2- 172.21.15.12:0/3757356421 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55f910749da0 0x55f91074c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:48.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.020+0000 7f1e8d482b80 4 client.15993 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:46:48.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.020+0000 7f1e8d482b80 1 -- 172.21.15.12:0/3757356421 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9106610a0 con 0x55f910749da0 2024-05-17T22:46:48.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.020+0000 7f1e7c843700 1 --2- 172.21.15.12:0/3757356421 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55f910749da0 0x55f91074c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:48.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.020+0000 7f1e57fff700 10 client.15993 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:46:48.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.020+0000 7f1e7c843700 1 --2- 172.21.15.12:0/3757356421 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55f910749da0 0x55f91074c1c0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fb2812361afd71c server_cookie=7fb8fde315669985 in_seq=0 out_seq=0 2024-05-17T22:46:48.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.021+0000 7f1e57fff700 1 -- 172.21.15.12:0/3757356421 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f9106610a0 con 0x55f910749da0 2024-05-17T22:46:48.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.021+0000 7f1e57fff700 10 client.15993 handle_command_reply: tid=0 2024-05-17T22:46:48.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.021+0000 7f1e8d482b80 1 client.15993 shutdown 2024-05-17T22:46:48.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.021+0000 7f1e8d482b80 1 -- 172.21.15.12:0/3757356421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f910736480 msgr2=0x55f91073a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:48.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.021+0000 7f1e8d482b80 1 --2- 172.21.15.12:0/3757356421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f910736480 0x55f91073a530 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000b010 tx=0x7f1e70007c40 comp rx=0 tx=0).stop 2024-05-17T22:46:48.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.021+0000 7f1e8d482b80 1 -- 172.21.15.12:0/3757356421 shutdown_connections 2024-05-17T22:46:48.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.021+0000 7f1e8d482b80 1 --2- 172.21.15.12:0/3757356421 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55f910749da0 0x55f91074c1c0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:48.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.022+0000 7f1e8d482b80 1 --2- 172.21.15.12:0/3757356421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f910736480 0x55f91073a530 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:48.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.022+0000 7f1e57fff700 0 client.15993 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:46:48.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.022+0000 7f1e8d482b80 1 --2- 172.21.15.12:0/3757356421 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f9105b2a70 0x55f910739ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:48.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.022+0000 7f1e8d482b80 1 --2- 172.21.15.12:0/3757356421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f910730b80 0x55f910739ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:48.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.022+0000 7f1e8d482b80 1 -- 172.21.15.12:0/3757356421 >> 172.21.15.12:0/3757356421 conn(0x55f910612850 msgr2=0x55f91072b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:46:48.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.022+0000 7f1e8d482b80 1 -- 172.21.15.12:0/3757356421 shutdown_connections 2024-05-17T22:46:48.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.022+0000 7f1e8d482b80 1 -- 172.21.15.12:0/3757356421 wait complete. 2024-05-17T22:46:48.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.022+0000 7f1e8d482b80 20 client.15993 trim_cache size 0 max 16384 2024-05-17T22:46:48.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.047+0000 7f1e8d482b80 1 Processor -- start 2024-05-17T22:46:48.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.047+0000 7f1e8d482b80 1 -- start start 2024-05-17T22:46:48.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.047+0000 7f1e8d482b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f910612850 0x55f910730640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:48.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.048+0000 7f1e8d482b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f9105b2a70 0x55f910725b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:48.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.048+0000 7f1e8d482b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f910730b80 0x55f910751760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:48.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.048+0000 7f1e8d482b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1e64011380 con 0x55f910612850 2024-05-17T22:46:48.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.048+0000 7f1e8d482b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6400ec10 con 0x55f910730b80 2024-05-17T22:46:48.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.048+0000 7f1e8d482b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6400ca60 con 0x55f9105b2a70 2024-05-17T22:46:48.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.048+0000 7f1e77fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f9105b2a70 0x55f910725b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:48.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.048+0000 7f1e7c843700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f910612850 0x55f910730640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:48.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.048+0000 7f1e7c843700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f910612850 0x55f910730640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36892/0 (socket says 172.21.15.12:36892) 2024-05-17T22:46:48.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.048+0000 7f1e7c843700 1 -- 172.21.15.12:0/1770527147 learned_addr learned my addr 172.21.15.12:0/1770527147 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:46:48.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.048+0000 7f1e7d044700 1 --2- 172.21.15.12:0/1770527147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f910730b80 0x55f910751760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:48.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.048+0000 7f1e77fff700 1 -- 172.21.15.12:0/1770527147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f910730b80 msgr2=0x55f910751760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:46:48.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.048+0000 7f1e77fff700 1 --2- 172.21.15.12:0/1770527147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f910730b80 0x55f910751760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:48.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.048+0000 7f1e77fff700 1 -- 172.21.15.12:0/1770527147 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f910612850 msgr2=0x55f910730640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:46:48.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.048+0000 7f1e77fff700 1 --2- 172.21.15.12:0/1770527147 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f910612850 0x55f910730640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:48.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.048+0000 7f1e77fff700 1 -- 172.21.15.12:0/1770527147 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f910723600 con 0x55f9105b2a70 2024-05-17T22:46:48.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.049+0000 7f1e77fff700 1 --2- 172.21.15.12:0/1770527147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f9105b2a70 0x55f910725b50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c009230 tx=0x7f1e6c01c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c1f58768b569107 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:46:48.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.049+0000 7f1e5e7fc700 1 -- 172.21.15.12:0/1770527147 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c00f9b0 con 0x55f9105b2a70 2024-05-17T22:46:48.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.049+0000 7f1e5e7fc700 1 -- 172.21.15.12:0/1770527147 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e6c033070 con 0x55f9105b2a70 2024-05-17T22:46:48.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.049+0000 7f1e5e7fc700 1 -- 172.21.15.12:0/1770527147 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c00b7c0 con 0x55f9105b2a70 2024-05-17T22:46:48.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.050+0000 7f1e8d482b80 1 -- 172.21.15.12:0/1770527147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f9105b2a70 msgr2=0x55f910725b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:48.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.050+0000 7f1e8d482b80 1 --2- 172.21.15.12:0/1770527147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f9105b2a70 0x55f910725b50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c009230 tx=0x7f1e6c01c000 comp rx=0 tx=0).stop 2024-05-17T22:46:48.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.050+0000 7f1e8d482b80 1 -- 172.21.15.12:0/1770527147 shutdown_connections 2024-05-17T22:46:48.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.050+0000 7f1e8d482b80 1 --2- 172.21.15.12:0/1770527147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f910730b80 0x55f910751760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:48.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.050+0000 7f1e8d482b80 1 --2- 172.21.15.12:0/1770527147 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f910612850 0x55f910730640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:48.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.050+0000 7f1e8d482b80 1 --2- 172.21.15.12:0/1770527147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f9105b2a70 0x55f910725b50 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:48.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.050+0000 7f1e8d482b80 1 -- 172.21.15.12:0/1770527147 >> 172.21.15.12:0/1770527147 conn(0x55f91072b540 msgr2=0x55f910735160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:46:48.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.051+0000 7f1e8d482b80 1 -- 172.21.15.12:0/1770527147 shutdown_connections 2024-05-17T22:46:48.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.051+0000 7f1e8d482b80 1 -- 172.21.15.12:0/1770527147 wait complete. 2024-05-17T22:46:48.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.051+0000 7f1e8d482b80 1 -- start start 2024-05-17T22:46:48.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.051+0000 7f1e8d482b80 1 Processor -- start 2024-05-17T22:46:48.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.051+0000 7f1e8d482b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9105b2a70 0x55f910740600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:48.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.052+0000 7f1e8d482b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f910612850 0x55f910740b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:48.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.052+0000 7f1e8d482b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f910730b80 0x55f9107ad360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:48.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.052+0000 7f1e8d482b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6400d0d0 con 0x55f910612850 2024-05-17T22:46:48.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.052+0000 7f1e8d482b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6400e540 con 0x55f9105b2a70 2024-05-17T22:46:48.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.052+0000 7f1e8d482b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6400d7a0 con 0x55f910730b80 2024-05-17T22:46:48.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.052+0000 7f1e8d482b80 10 client.0 _ll_register_callbacks cb 0x55f9105e5bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:46:48.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.052+0000 7f1e7c843700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9105b2a70 0x55f910740600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:48.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.052+0000 7f1e7c843700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9105b2a70 0x55f910740600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:37922/0 (socket says 172.21.15.12:37922) 2024-05-17T22:46:48.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.052+0000 7f1e7c843700 1 -- 172.21.15.12:0/64227213 learned_addr learned my addr 172.21.15.12:0/64227213 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:46:48.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.052+0000 7f1e77fff700 1 --2- 172.21.15.12:0/64227213 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f910612850 0x55f910740b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:48.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.052+0000 7f1e7d044700 1 --2- 172.21.15.12:0/64227213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f910730b80 0x55f9107ad360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:48.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.052+0000 7f1e7c843700 1 -- 172.21.15.12:0/64227213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f910730b80 msgr2=0x55f9107ad360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:46:48.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.052+0000 7f1e7c843700 1 --2- 172.21.15.12:0/64227213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f910730b80 0x55f9107ad360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:48.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.052+0000 7f1e7c843700 1 -- 172.21.15.12:0/64227213 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f910612850 msgr2=0x55f910740b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:46:48.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.052+0000 7f1e7c843700 1 --2- 172.21.15.12:0/64227213 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f910612850 0x55f910740b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:48.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.052+0000 7f1e7c843700 1 -- 172.21.15.12:0/64227213 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9107ad920 con 0x55f9105b2a70 2024-05-17T22:46:48.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.052+0000 7f1e77fff700 1 --2- 172.21.15.12:0/64227213 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f910612850 0x55f910740b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:46:48.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.052+0000 7f1e7d044700 1 --2- 172.21.15.12:0/64227213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f910730b80 0x55f9107ad360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:46:48.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.052+0000 7f1e7c843700 1 --2- 172.21.15.12:0/64227213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9105b2a70 0x55f910740600 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1e68010c50 tx=0x7f1e680b3040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c729885e52cce1f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:46:48.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.052+0000 7f1e57fff700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:46:48.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.053+0000 7f1e57fff700 1 -- 172.21.15.12:0/64227213 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6808a070 con 0x55f9105b2a70 2024-05-17T22:46:48.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.053+0000 7f1e8d482b80 1 -- 172.21.15.12:0/64227213 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e6c033070 con 0x55f9105b2a70 2024-05-17T22:46:48.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.053+0000 7f1e57fff700 1 -- 172.21.15.12:0/64227213 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f1e680aaa10 con 0x55f9105b2a70 2024-05-17T22:46:48.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.053+0000 7f1e57fff700 1 -- 172.21.15.12:0/64227213 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1e68097050 con 0x55f9105b2a70 2024-05-17T22:46:48.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.053+0000 7f1e8d482b80 10 client.25667 fetch_fsmap learned FSMap version 37 2024-05-17T22:46:48.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.053+0000 7f1e8d482b80 1 -- 172.21.15.12:0/64227213 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f9107b4d00 con 0x55f9105b2a70 2024-05-17T22:46:48.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.053+0000 7f1e57fff700 1 -- 172.21.15.12:0/64227213 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e6c033070 con 0x55f9105b2a70 2024-05-17T22:46:48.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.053+0000 7f1e57fff700 1 -- 172.21.15.12:0/64227213 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f1e680921b0 con 0x55f9105b2a70 2024-05-17T22:46:48.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.053+0000 7f1e8d482b80 10 client.25667 fetch_fsmap finished waiting for FSMap version 37 2024-05-17T22:46:48.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.053+0000 7f1e8d482b80 10 client.25667 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:46:48.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.053+0000 7f1e8d482b80 20 client.25667 populate_metadata read hostname 'smithi012' 2024-05-17T22:46:48.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.053+0000 7f1e8d482b80 1 --2- 172.21.15.12:0/64227213 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55f910736480 0x55f9107b7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:48.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.053+0000 7f1e8d482b80 4 client.25667 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-17T22:46:48.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.053+0000 7f1e8d482b80 1 -- 172.21.15.12:0/64227213 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f91065a880 con 0x55f910736480 2024-05-17T22:46:48.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.053+0000 7f1e77fff700 1 --2- 172.21.15.12:0/64227213 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55f910736480 0x55f9107b7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:48.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.054+0000 7f1e57fff700 10 client.25667 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:46:48.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.054+0000 7f1e77fff700 1 --2- 172.21.15.12:0/64227213 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55f910736480 0x55f9107b7460 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f35f5f9c278242de server_cookie=6e51c5543b9ebfaf in_seq=0 out_seq=0 2024-05-17T22:46:48.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.055+0000 7f1e57fff700 1 -- 172.21.15.12:0/64227213 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f91065a880 con 0x55f910736480 2024-05-17T22:46:48.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.055+0000 7f1e57fff700 10 client.25667 handle_command_reply: tid=0 2024-05-17T22:46:48.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.055+0000 7f1e8d482b80 1 client.25667 shutdown 2024-05-17T22:46:48.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.055+0000 7f1e8d482b80 1 -- 172.21.15.12:0/64227213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9105b2a70 msgr2=0x55f910740600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:48.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.055+0000 7f1e8d482b80 1 --2- 172.21.15.12:0/64227213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9105b2a70 0x55f910740600 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1e68010c50 tx=0x7f1e680b3040 comp rx=0 tx=0).stop 2024-05-17T22:46:48.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.055+0000 7f1e8d482b80 1 -- 172.21.15.12:0/64227213 shutdown_connections 2024-05-17T22:46:48.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.055+0000 7f1e8d482b80 1 --2- 172.21.15.12:0/64227213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f910730b80 0x55f9107ad360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:48.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.055+0000 7f1e8d482b80 1 --2- 172.21.15.12:0/64227213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9105b2a70 0x55f910740600 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:48.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.055+0000 7f1e8d482b80 1 --2- 172.21.15.12:0/64227213 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55f910736480 0x55f9107b7460 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:48.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.055+0000 7f1e8d482b80 1 --2- 172.21.15.12:0/64227213 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f910612850 0x55f910740b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:48.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.055+0000 7f1e8d482b80 1 -- 172.21.15.12:0/64227213 >> 172.21.15.12:0/64227213 conn(0x55f91072b540 msgr2=0x55f91073b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:46:48.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.055+0000 7f1e57fff700 0 client.25667 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:46:48.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.056+0000 7f1e8d482b80 1 -- 172.21.15.12:0/64227213 shutdown_connections 2024-05-17T22:46:48.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.056+0000 7f1e8d482b80 1 -- 172.21.15.12:0/64227213 wait complete. 2024-05-17T22:46:48.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.056+0000 7f1e8d482b80 20 client.25667 trim_cache size 0 max 16384 2024-05-17T22:46:48.064 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:46:48.064 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-17T22:46:48.065 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "d547c817-b870-4c6a-9e34-d3378d9e03e6", 2024-05-17T22:46:48.065 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-17T22:46:48.065 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:46:48.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.057+0000 7f1e7eaa7700 1 -- 172.21.15.12:0/2546467383 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f1e600b2a90 msgr2=0x7f1e600b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:48.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.057+0000 7f1e7eaa7700 1 --2- 172.21.15.12:0/2546467383 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f1e600b2a90 0x7f1e600b4fa0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c009680 tx=0x7f1e6c009f50 comp rx=0 tx=0).stop 2024-05-17T22:46:48.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.057+0000 7f1e7eaa7700 1 -- 172.21.15.12:0/2546467383 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e7807c6d0 msgr2=0x7f1e781a96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:48.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.057+0000 7f1e7eaa7700 1 --2- 172.21.15.12:0/2546467383 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e7807c6d0 0x7f1e781a96d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1e6800e9c0 tx=0x7f1e6800ecd0 comp rx=0 tx=0).stop 2024-05-17T22:46:48.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.057+0000 7f1e7eaa7700 1 -- 172.21.15.12:0/2546467383 shutdown_connections 2024-05-17T22:46:48.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.057+0000 7f1e7eaa7700 1 --2- 172.21.15.12:0/2546467383 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f1e600b2a90 0x7f1e600b4fa0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:48.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.057+0000 7f1e7eaa7700 1 --2- 172.21.15.12:0/2546467383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e7811d1b0 0x7f1e781adf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:48.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.057+0000 7f1e7eaa7700 1 --2- 172.21.15.12:0/2546467383 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e7807c6d0 0x7f1e781a96d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:48.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.057+0000 7f1e7eaa7700 1 --2- 172.21.15.12:0/2546467383 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e78084330 0x7f1e781a9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:48.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.057+0000 7f1e7eaa7700 1 -- 172.21.15.12:0/2546467383 >> 172.21.15.12:0/2546467383 conn(0x7f1e78078570 msgr2=0x7f1e7811b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:46:48.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.057+0000 7f1e7eaa7700 1 -- 172.21.15.12:0/2546467383 shutdown_connections 2024-05-17T22:46:48.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:48.057+0000 7f1e7eaa7700 1 -- 172.21.15.12:0/2546467383 wait complete. 2024-05-17T22:46:48.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:47 smithi012 ceph-mon[31632]: pgmap v665: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.7 MiB/s wr, 471 op/s 2024-05-17T22:46:48.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:47 smithi151 ceph-mon[36101]: pgmap v665: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.7 MiB/s wr, 471 op/s 2024-05-17T22:46:48.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:47 smithi096 ceph-mon[36065]: pgmap v665: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.7 MiB/s wr, 471 op/s 2024-05-17T22:46:49.176 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:48 smithi012 ceph-mon[31632]: scrub queued for path: / 2024-05-17T22:46:49.176 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:48 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:46:49.176 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:48 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:46:49.178 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-17T22:46:49.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:48 smithi151 ceph-mon[36101]: scrub queued for path: / 2024-05-17T22:46:49.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:48 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:46:49.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:48 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:46:49.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:48 smithi096 ceph-mon[36065]: scrub queued for path: / 2024-05-17T22:46:49.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:48 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:46:49.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:48 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:46:49.684 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:46:50.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:49 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:46:50.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:49 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:46:50.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:49 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:46:50.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:49 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:46:50.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:49 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:46:50.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:49 smithi012 ceph-mon[31632]: pgmap v666: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 5.1 MiB/s wr, 533 op/s 2024-05-17T22:46:50.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:49 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:46:50.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:49 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:46:50.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:49 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:46:50.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:49 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:46:50.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:49 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:46:50.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:49 smithi151 ceph-mon[36101]: pgmap v666: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 5.1 MiB/s wr, 533 op/s 2024-05-17T22:46:50.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:49 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:46:50.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:49 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:46:50.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:49 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:46:50.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:49 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:46:50.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:49 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:46:50.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:49 smithi096 ceph-mon[36065]: pgmap v666: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 5.1 MiB/s wr, 533 op/s 2024-05-17T22:46:51.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 450.0 s, 511.2 tps, lat 200.570 ms stddev 223.319 2024-05-17T22:46:51.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.723+0000 7fcedf59e700 1 -- 172.21.15.12:0/3846029788 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcee0116980 msgr2=0x7fcee0118da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:51.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.723+0000 7fcedf59e700 1 --2- 172.21.15.12:0/3846029788 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcee0116980 0x7fcee0118da0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fced8009990 tx=0x7fced8009ca0 comp rx=0 tx=0).stop 2024-05-17T22:46:51.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.723+0000 7fcedf59e700 1 -- 172.21.15.12:0/3846029788 shutdown_connections 2024-05-17T22:46:51.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.723+0000 7fcedf59e700 1 --2- 172.21.15.12:0/3846029788 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcee0116980 0x7fcee0118da0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:51.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.723+0000 7fcedf59e700 1 --2- 172.21.15.12:0/3846029788 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcee010aa60 0x7fcee010ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:51.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.723+0000 7fcedf59e700 1 --2- 172.21.15.12:0/3846029788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcee010b3b0 0x7fcee0116440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:51.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.723+0000 7fcedf59e700 1 -- 172.21.15.12:0/3846029788 >> 172.21.15.12:0/3846029788 conn(0x7fcee006eda0 msgr2=0x7fcee006f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:46:51.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.724+0000 7fcedf59e700 1 -- 172.21.15.12:0/3846029788 shutdown_connections 2024-05-17T22:46:51.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.724+0000 7fcedf59e700 1 -- 172.21.15.12:0/3846029788 wait complete. 2024-05-17T22:46:51.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.725+0000 7fcedf59e700 1 Processor -- start 2024-05-17T22:46:51.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.725+0000 7fcedf59e700 1 -- start start 2024-05-17T22:46:51.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.725+0000 7fcedf59e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcee010aa60 0x7fcee01138c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:51.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.725+0000 7fcedf59e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcee010b3b0 0x7fcee0113e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:51.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.725+0000 7fcedf59e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcee0116980 0x7fcee0114340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:51.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.725+0000 7fcedf59e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcee010dcb0 con 0x7fcee0116980 2024-05-17T22:46:51.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.725+0000 7fcedf59e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcee010db30 con 0x7fcee010b3b0 2024-05-17T22:46:51.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.725+0000 7fcedf59e700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcee010de30 con 0x7fcee010aa60 2024-05-17T22:46:51.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.726+0000 7fceddd9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcee010b3b0 0x7fcee0113e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:51.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.726+0000 7fceddd9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcee010b3b0 0x7fcee0113e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:44872/0 (socket says 172.21.15.12:44872) 2024-05-17T22:46:51.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.726+0000 7fceddd9b700 1 -- 172.21.15.12:0/1764752226 learned_addr learned my addr 172.21.15.12:0/1764752226 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:46:51.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.726+0000 7fceddd9b700 1 -- 172.21.15.12:0/1764752226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcee010aa60 msgr2=0x7fcee01138c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-17T22:46:51.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.726+0000 7fceddd9b700 1 --2- 172.21.15.12:0/1764752226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcee010aa60 0x7fcee01138c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:51.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.726+0000 7fceddd9b700 1 -- 172.21.15.12:0/1764752226 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcee0116980 msgr2=0x7fcee0114340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:51.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.726+0000 7fceddd9b700 1 --2- 172.21.15.12:0/1764752226 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcee0116980 0x7fcee0114340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:51.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.726+0000 7fceddd9b700 1 -- 172.21.15.12:0/1764752226 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee01b5450 con 0x7fcee010b3b0 2024-05-17T22:46:51.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.726+0000 7fceddd9b700 1 --2- 172.21.15.12:0/1764752226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcee010b3b0 0x7fcee0113e00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fced0007ee0 tx=0x7fced000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:46:51.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.727+0000 7fcecf7fe700 1 -- 172.21.15.12:0/1764752226 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced0003a00 con 0x7fcee010b3b0 2024-05-17T22:46:51.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.727+0000 7fcedf59e700 1 -- 172.21.15.12:0/1764752226 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcee01b5740 con 0x7fcee010b3b0 2024-05-17T22:46:51.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.727+0000 7fcedf59e700 1 -- 172.21.15.12:0/1764752226 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcee01b5cd0 con 0x7fcee010b3b0 2024-05-17T22:46:51.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.729+0000 7fcecf7fe700 1 -- 172.21.15.12:0/1764752226 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fced0003ba0 con 0x7fcee010b3b0 2024-05-17T22:46:51.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.729+0000 7fcecf7fe700 1 -- 172.21.15.12:0/1764752226 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced0015400 con 0x7fcee010b3b0 2024-05-17T22:46:51.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.732+0000 7fcef38d5b80 1 Processor -- start 2024-05-17T22:46:51.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.732+0000 7fcecf7fe700 1 -- 172.21.15.12:0/1764752226 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fced0015640 con 0x7fcee010b3b0 2024-05-17T22:46:51.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.733+0000 7fcef38d5b80 1 -- start start 2024-05-17T22:46:51.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.733+0000 7fcef38d5b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e9e5c9f0 0x55c8e9fdffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:51.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.733+0000 7fcef38d5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8e9fe0510 0x55c8e9fda5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:51.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.733+0000 7fcef38d5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8e9fdab00 0x55c8e9fdaf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:51.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.733+0000 7fcef38d5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c8e9f090d0 con 0x55c8e9fdab00 2024-05-17T22:46:51.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.733+0000 7fcef38d5b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c8e9ef8c70 con 0x55c8e9e5c9f0 2024-05-17T22:46:51.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.733+0000 7fcef38d5b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c8e9efccb0 con 0x55c8e9fe0510 2024-05-17T22:46:51.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.733+0000 7fceded9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e9e5c9f0 0x55c8e9fdffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:51.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.733+0000 7fceded9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e9e5c9f0 0x55c8e9fdffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:44882/0 (socket says 172.21.15.12:44882) 2024-05-17T22:46:51.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.733+0000 7fceded9d700 1 -- 172.21.15.12:0/26973632 learned_addr learned my addr 172.21.15.12:0/26973632 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:46:51.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.733+0000 7fcede59c700 1 --2- 172.21.15.12:0/26973632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8e9fe0510 0x55c8e9fda5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:51.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.733+0000 7fceddd9b700 1 --2- 172.21.15.12:0/26973632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8e9fdab00 0x55c8e9fdaf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:51.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.734+0000 7fceded9d700 1 -- 172.21.15.12:0/26973632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8e9fe0510 msgr2=0x55c8e9fda5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:46:51.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.734+0000 7fceded9d700 1 --2- 172.21.15.12:0/26973632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8e9fe0510 0x55c8e9fda5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:51.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.734+0000 7fceded9d700 1 -- 172.21.15.12:0/26973632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8e9fdab00 msgr2=0x55c8e9fdaf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:46:51.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.734+0000 7fceded9d700 1 --2- 172.21.15.12:0/26973632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8e9fdab00 0x55c8e9fdaf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:51.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.734+0000 7fceded9d700 1 -- 172.21.15.12:0/26973632 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8e9e925c0 con 0x55c8e9e5c9f0 2024-05-17T22:46:51.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.734+0000 7fcede59c700 1 --2- 172.21.15.12:0/26973632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8e9fe0510 0x55c8e9fda5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:46:51.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.734+0000 7fceded9d700 1 --2- 172.21.15.12:0/26973632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e9e5c9f0 0x55c8e9fdffd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fced8005b40 tx=0x7fced8009ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=865aaf013dc66ea5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:46:51.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.734+0000 7fceccff9700 1 -- 172.21.15.12:0/26973632 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced8004230 con 0x55c8e9e5c9f0 2024-05-17T22:46:51.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.734+0000 7fceccff9700 1 -- 172.21.15.12:0/26973632 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fced8003e30 con 0x55c8e9e5c9f0 2024-05-17T22:46:51.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.734+0000 7fcecf7fe700 1 --2- 172.21.15.12:0/1764752226 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fcec80b2900 0x7fcec80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:51.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.735+0000 7fcede59c700 1 --2- 172.21.15.12:0/1764752226 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fcec80b2900 0x7fcec80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:51.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.735+0000 7fceccff9700 1 -- 172.21.15.12:0/26973632 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced800b8c0 con 0x55c8e9e5c9f0 2024-05-17T22:46:51.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.735+0000 7fcecf7fe700 1 -- 172.21.15.12:0/1764752226 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fced00e20b0 con 0x7fcee010b3b0 2024-05-17T22:46:51.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.735+0000 7fcef38d5b80 1 -- 172.21.15.12:0/26973632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e9e5c9f0 msgr2=0x55c8e9fdffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:51.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.735+0000 7fcef38d5b80 1 --2- 172.21.15.12:0/26973632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e9e5c9f0 0x55c8e9fdffd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fced8005b40 tx=0x7fced8009ee0 comp rx=0 tx=0).stop 2024-05-17T22:46:51.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.735+0000 7fcede59c700 1 --2- 172.21.15.12:0/1764752226 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fcec80b2900 0x7fcec80b4e10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55c8e9fd6ba0 tx=0x7fced4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:46:51.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.735+0000 7fcef38d5b80 1 -- 172.21.15.12:0/26973632 shutdown_connections 2024-05-17T22:46:51.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.735+0000 7fcef38d5b80 1 --2- 172.21.15.12:0/26973632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8e9fdab00 0x55c8e9fdaf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:51.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.735+0000 7fcef38d5b80 1 --2- 172.21.15.12:0/26973632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e9e5c9f0 0x55c8e9fdffd0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:51.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.735+0000 7fcef38d5b80 1 --2- 172.21.15.12:0/26973632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8e9fe0510 0x55c8e9fda5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:51.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.735+0000 7fcef38d5b80 1 -- 172.21.15.12:0/26973632 >> 172.21.15.12:0/26973632 conn(0x55c8e9ebc7d0 msgr2=0x55c8e9fd51a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:46:51.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.736+0000 7fcef38d5b80 1 -- 172.21.15.12:0/26973632 shutdown_connections 2024-05-17T22:46:51.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.736+0000 7fcef38d5b80 1 -- 172.21.15.12:0/26973632 wait complete. 2024-05-17T22:46:51.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.737+0000 7fcef38d5b80 1 -- start start 2024-05-17T22:46:51.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.737+0000 7fcef38d5b80 1 Processor -- start 2024-05-17T22:46:51.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.737+0000 7fcef38d5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8e9e5c9f0 0x55c8e9fe3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:51.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.737+0000 7fcef38d5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8e9fdab00 0x55c8e9fe4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:51.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.737+0000 7fcef38d5b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e9fe0510 0x55c8e9fe45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:51.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.737+0000 7fcef38d5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c8e9e00150 con 0x55c8e9e5c9f0 2024-05-17T22:46:51.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.737+0000 7fcef38d5b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c8e9e9a940 con 0x55c8e9fe0510 2024-05-17T22:46:51.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.737+0000 7fcef38d5b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c8e9fe6e50 con 0x55c8e9fdab00 2024-05-17T22:46:51.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.737+0000 7fcef38d5b80 10 client.0 _ll_register_callbacks cb 0x55c8e9e8fb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:46:51.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.737+0000 7fcede59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8e9e5c9f0 0x55c8e9fe3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:51.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.737+0000 7fceddd9b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8e9fdab00 0x55c8e9fe4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:51.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.737+0000 7fceded9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e9fe0510 0x55c8e9fe45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:51.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.737+0000 7fceddd9b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8e9fdab00 0x55c8e9fe4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:52380/0 (socket says 172.21.15.12:52380) 2024-05-17T22:46:51.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.737+0000 7fceded9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e9fe0510 0x55c8e9fe45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:44896/0 (socket says 172.21.15.12:44896) 2024-05-17T22:46:51.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.738+0000 7fceddd9b700 1 -- 172.21.15.12:0/686455500 learned_addr learned my addr 172.21.15.12:0/686455500 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:46:51.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.738+0000 7fceded9d700 1 -- 172.21.15.12:0/686455500 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8e9fdab00 msgr2=0x55c8e9fe4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:46:51.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.738+0000 7fceded9d700 1 --2- 172.21.15.12:0/686455500 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8e9fdab00 0x55c8e9fe4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:51.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.738+0000 7fceded9d700 1 -- 172.21.15.12:0/686455500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8e9e5c9f0 msgr2=0x55c8e9fe3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:46:51.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.738+0000 7fceded9d700 1 --2- 172.21.15.12:0/686455500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8e9e5c9f0 0x55c8e9fe3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:51.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.738+0000 7fceded9d700 1 -- 172.21.15.12:0/686455500 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8e9e49480 con 0x55c8e9fe0510 2024-05-17T22:46:51.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.739+0000 7fceded9d700 1 --2- 172.21.15.12:0/686455500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e9fe0510 0x55c8e9fe45d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fced8009990 tx=0x7fced80056d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=256898b0d65f1e30 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:46:51.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.739+0000 7fcec5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:46:51.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.739+0000 7fcec5ffb700 1 -- 172.21.15.12:0/686455500 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced800b9e0 con 0x55c8e9fe0510 2024-05-17T22:46:51.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.739+0000 7fcec5ffb700 1 -- 172.21.15.12:0/686455500 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fced800bb80 con 0x55c8e9fe0510 2024-05-17T22:46:51.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.739+0000 7fcef38d5b80 1 -- 172.21.15.12:0/686455500 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fced8003e30 con 0x55c8e9fe0510 2024-05-17T22:46:51.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.739+0000 7fcec5ffb700 1 -- 172.21.15.12:0/686455500 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fced800bd80 con 0x55c8e9fe0510 2024-05-17T22:46:51.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.739+0000 7fcef38d5b80 10 client.25679 fetch_fsmap learned FSMap version 37 2024-05-17T22:46:51.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.739+0000 7fcef38d5b80 1 -- 172.21.15.12:0/686455500 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c8e9ff1880 con 0x55c8e9fe0510 2024-05-17T22:46:51.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.740+0000 7fcec5ffb700 1 -- 172.21.15.12:0/686455500 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fced8030070 con 0x55c8e9fe0510 2024-05-17T22:46:51.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.740+0000 7fcec5ffb700 1 -- 172.21.15.12:0/686455500 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fced802b540 con 0x55c8e9fe0510 2024-05-17T22:46:51.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.740+0000 7fcef38d5b80 10 client.25679 fetch_fsmap finished waiting for FSMap version 37 2024-05-17T22:46:51.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.740+0000 7fcef38d5b80 10 client.25679 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:46:51.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.740+0000 7fcef38d5b80 20 client.25679 populate_metadata read hostname 'smithi012' 2024-05-17T22:46:51.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.740+0000 7fcef38d5b80 1 --2- 172.21.15.12:0/686455500 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55c8e9ff3d50 0x55c8e9ff6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:51.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.740+0000 7fcef38d5b80 4 client.25679 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:46:51.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.740+0000 7fcef38d5b80 1 -- 172.21.15.12:0/686455500 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fced8003e30 con 0x55c8e9ff3d50 2024-05-17T22:46:51.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.740+0000 7fcede59c700 1 --2- 172.21.15.12:0/686455500 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55c8e9ff3d50 0x55c8e9ff6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:51.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.741+0000 7fcec5ffb700 10 client.25679 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:46:51.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.741+0000 7fcede59c700 1 --2- 172.21.15.12:0/686455500 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55c8e9ff3d50 0x55c8e9ff6170 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c01f380c82b7eeac server_cookie=481786326079f2a5 in_seq=0 out_seq=0 2024-05-17T22:46:51.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.742+0000 7fcec5ffb700 1 -- 172.21.15.12:0/686455500 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fced8003e30 con 0x55c8e9ff3d50 2024-05-17T22:46:51.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.742+0000 7fcec5ffb700 10 client.25679 handle_command_reply: tid=0 2024-05-17T22:46:51.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.742+0000 7fcef38d5b80 1 client.25679 shutdown 2024-05-17T22:46:51.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.742+0000 7fcef38d5b80 1 -- 172.21.15.12:0/686455500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e9fe0510 msgr2=0x55c8e9fe45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:51.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.742+0000 7fcef38d5b80 1 --2- 172.21.15.12:0/686455500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e9fe0510 0x55c8e9fe45d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fced8009990 tx=0x7fced80056d0 comp rx=0 tx=0).stop 2024-05-17T22:46:51.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.742+0000 7fcef38d5b80 1 -- 172.21.15.12:0/686455500 shutdown_connections 2024-05-17T22:46:51.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.742+0000 7fcef38d5b80 1 --2- 172.21.15.12:0/686455500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e9fe0510 0x55c8e9fe45d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:51.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.742+0000 7fcef38d5b80 1 --2- 172.21.15.12:0/686455500 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55c8e9ff3d50 0x55c8e9ff6170 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:51.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.742+0000 7fcef38d5b80 1 --2- 172.21.15.12:0/686455500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8e9e5c9f0 0x55c8e9fe3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:51.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.742+0000 7fcef38d5b80 1 --2- 172.21.15.12:0/686455500 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8e9fdab00 0x55c8e9fe4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:51.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.742+0000 7fcef38d5b80 1 -- 172.21.15.12:0/686455500 >> 172.21.15.12:0/686455500 conn(0x55c8e9ebc7d0 msgr2=0x55c8e9fd5140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:46:51.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.742+0000 7fcec5ffb700 0 client.25679 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:46:51.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.742+0000 7fcef38d5b80 1 -- 172.21.15.12:0/686455500 shutdown_connections 2024-05-17T22:46:51.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.742+0000 7fcef38d5b80 1 -- 172.21.15.12:0/686455500 wait complete. 2024-05-17T22:46:51.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.743+0000 7fcef38d5b80 20 client.25679 trim_cache size 0 max 16384 2024-05-17T22:46:51.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.777+0000 7fcef38d5b80 1 Processor -- start 2024-05-17T22:46:51.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.777+0000 7fcef38d5b80 1 -- start start 2024-05-17T22:46:51.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.777+0000 7fcef38d5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8e9e5c9f0 0x55c8e9fda5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:51.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.777+0000 7fcef38d5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8e9fdab00 0x55c8e9fcfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:51.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.777+0000 7fcef38d5b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e9fe0510 0x55c8e9fe40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:51.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.777+0000 7fcef38d5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcec0011470 con 0x55c8e9fdab00 2024-05-17T22:46:51.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.777+0000 7fcef38d5b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcec000ed00 con 0x55c8e9fe0510 2024-05-17T22:46:51.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.777+0000 7fcef38d5b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcec000cb50 con 0x55c8e9e5c9f0 2024-05-17T22:46:51.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.779+0000 7fceded9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e9fe0510 0x55c8e9fe40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:51.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.779+0000 7fceded9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e9fe0510 0x55c8e9fe40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:44912/0 (socket says 172.21.15.12:44912) 2024-05-17T22:46:51.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.779+0000 7fceded9d700 1 -- 172.21.15.12:0/3113002458 learned_addr learned my addr 172.21.15.12:0/3113002458 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:46:51.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.779+0000 7fcede59c700 1 --2- 172.21.15.12:0/3113002458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8e9e5c9f0 0x55c8e9fda5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:51.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.780+0000 7fcede59c700 1 -- 172.21.15.12:0/3113002458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e9fe0510 msgr2=0x55c8e9fe40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:46:51.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.780+0000 7fcede59c700 1 --2- 172.21.15.12:0/3113002458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e9fe0510 0x55c8e9fe40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:51.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.780+0000 7fcede59c700 1 -- 172.21.15.12:0/3113002458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8e9fdab00 msgr2=0x55c8e9fcfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:46:51.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.780+0000 7fcede59c700 1 --2- 172.21.15.12:0/3113002458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8e9fdab00 0x55c8e9fcfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:51.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.780+0000 7fcede59c700 1 -- 172.21.15.12:0/3113002458 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8e9fe6ee0 con 0x55c8e9e5c9f0 2024-05-17T22:46:51.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.780+0000 7fcede59c700 1 --2- 172.21.15.12:0/3113002458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8e9e5c9f0 0x55c8e9fda5c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x55c8e9fe6e80 tx=0x7fced4013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=878d57d502cfb928 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:46:51.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.783+0000 7fcec7fff700 1 -- 172.21.15.12:0/3113002458 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced4004020 con 0x55c8e9e5c9f0 2024-05-17T22:46:51.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.783+0000 7fcec7fff700 1 -- 172.21.15.12:0/3113002458 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fced40118e0 con 0x55c8e9e5c9f0 2024-05-17T22:46:51.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.783+0000 7fcec7fff700 1 -- 172.21.15.12:0/3113002458 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced4026610 con 0x55c8e9e5c9f0 2024-05-17T22:46:51.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.784+0000 7fcef38d5b80 1 -- 172.21.15.12:0/3113002458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8e9e5c9f0 msgr2=0x55c8e9fda5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:51.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.784+0000 7fcef38d5b80 1 --2- 172.21.15.12:0/3113002458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8e9e5c9f0 0x55c8e9fda5c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x55c8e9fe6e80 tx=0x7fced4013000 comp rx=0 tx=0).stop 2024-05-17T22:46:51.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.788+0000 7fcef38d5b80 1 -- 172.21.15.12:0/3113002458 shutdown_connections 2024-05-17T22:46:51.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.788+0000 7fcef38d5b80 1 --2- 172.21.15.12:0/3113002458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e9fe0510 0x55c8e9fe40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:51.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.788+0000 7fcef38d5b80 1 --2- 172.21.15.12:0/3113002458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8e9e5c9f0 0x55c8e9fda5c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:51.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.788+0000 7fcef38d5b80 1 --2- 172.21.15.12:0/3113002458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8e9fdab00 0x55c8e9fcfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:51.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.788+0000 7fcef38d5b80 1 -- 172.21.15.12:0/3113002458 >> 172.21.15.12:0/3113002458 conn(0x55c8e9ebc7d0 msgr2=0x55c8e9fe2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:46:51.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.793+0000 7fcef38d5b80 1 -- 172.21.15.12:0/3113002458 shutdown_connections 2024-05-17T22:46:51.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.797+0000 7fcef38d5b80 1 -- 172.21.15.12:0/3113002458 wait complete. 2024-05-17T22:46:51.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.797+0000 7fcef38d5b80 1 -- start start 2024-05-17T22:46:51.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.797+0000 7fcef38d5b80 1 Processor -- start 2024-05-17T22:46:51.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.798+0000 7fcef38d5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8e9e5c9f0 0x55c8e9ff6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:51.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.798+0000 7fcef38d5b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e9fdab00 0x55c8e9ff6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:51.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.798+0000 7fcef38d5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8e9fe0510 0x55c8e9ff74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:51.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.798+0000 7fceddd9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e9fdab00 0x55c8e9ff6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:51.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.798+0000 7fceddd9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e9fdab00 0x55c8e9ff6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:44914/0 (socket says 172.21.15.12:44914) 2024-05-17T22:46:51.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.798+0000 7fceddd9b700 1 -- 172.21.15.12:0/1631679017 learned_addr learned my addr 172.21.15.12:0/1631679017 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:46:51.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.798+0000 7fceded9d700 1 --2- 172.21.15.12:0/1631679017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8e9fe0510 0x55c8e9ff74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:51.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.798+0000 7fcef38d5b80 1 -- 172.21.15.12:0/1631679017 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcec000d1c0 con 0x55c8e9fe0510 2024-05-17T22:46:51.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.798+0000 7fcef38d5b80 1 -- 172.21.15.12:0/1631679017 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcec000e630 con 0x55c8e9fdab00 2024-05-17T22:46:51.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.798+0000 7fcef38d5b80 1 -- 172.21.15.12:0/1631679017 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcec000d890 con 0x55c8e9e5c9f0 2024-05-17T22:46:51.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.798+0000 7fcef38d5b80 10 client.0 _ll_register_callbacks cb 0x55c8e9e8fb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:46:51.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.799+0000 7fceddd9b700 1 -- 172.21.15.12:0/1631679017 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8e9e5c9f0 msgr2=0x55c8e9ff6a70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T22:46:51.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.799+0000 7fceddd9b700 1 --2- 172.21.15.12:0/1631679017 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8e9e5c9f0 0x55c8e9ff6a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:51.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.799+0000 7fceddd9b700 1 -- 172.21.15.12:0/1631679017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8e9fe0510 msgr2=0x55c8e9ff74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:46:51.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.799+0000 7fceddd9b700 1 --2- 172.21.15.12:0/1631679017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8e9fe0510 0x55c8e9ff74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:51.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.799+0000 7fceddd9b700 1 -- 172.21.15.12:0/1631679017 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8ea06e3b0 con 0x55c8e9fdab00 2024-05-17T22:46:51.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.799+0000 7fceddd9b700 1 --2- 172.21.15.12:0/1631679017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e9fdab00 0x55c8e9ff6fb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fced000d7b0 tx=0x7fced00a91b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e5ad80382ccfec5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:46:51.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.799+0000 7fcec5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:46:51.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.800+0000 7fcec5ffb700 1 -- 172.21.15.12:0/1631679017 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced00a92e0 con 0x55c8e9fdab00 2024-05-17T22:46:51.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.800+0000 7fcef38d5b80 1 -- 172.21.15.12:0/1631679017 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fced40118e0 con 0x55c8e9fdab00 2024-05-17T22:46:51.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.800+0000 7fcec5ffb700 1 -- 172.21.15.12:0/1631679017 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fced00e40d0 con 0x55c8e9fdab00 2024-05-17T22:46:51.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.800+0000 7fcec5ffb700 1 -- 172.21.15.12:0/1631679017 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fced00ad690 con 0x55c8e9fdab00 2024-05-17T22:46:51.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.806+0000 7fcef38d5b80 10 client.25685 fetch_fsmap learned FSMap version 37 2024-05-17T22:46:51.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.806+0000 7fcef38d5b80 1 -- 172.21.15.12:0/1631679017 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c8ea0552a0 con 0x55c8e9fdab00 2024-05-17T22:46:51.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.807+0000 7fcec5ffb700 1 -- 172.21.15.12:0/1631679017 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fced40118e0 con 0x55c8e9fdab00 2024-05-17T22:46:51.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.807+0000 7fcec5ffb700 1 -- 172.21.15.12:0/1631679017 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fced00b1cb0 con 0x55c8e9fdab00 2024-05-17T22:46:51.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.807+0000 7fcef38d5b80 10 client.25685 fetch_fsmap finished waiting for FSMap version 37 2024-05-17T22:46:51.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.807+0000 7fcef38d5b80 10 client.25685 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:46:51.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.807+0000 7fcef38d5b80 20 client.25685 populate_metadata read hostname 'smithi012' 2024-05-17T22:46:51.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.807+0000 7fcef38d5b80 1 --2- 172.21.15.12:0/1631679017 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55c8ea055b10 0x55c8ea057f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:46:51.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.807+0000 7fcef38d5b80 4 client.25685 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:46:51.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.807+0000 7fcef38d5b80 1 -- 172.21.15.12:0/1631679017 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c8e9f0b020 con 0x55c8ea055b10 2024-05-17T22:46:51.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.812+0000 7fcede59c700 1 --2- 172.21.15.12:0/1631679017 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55c8ea055b10 0x55c8ea057f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:46:51.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.819+0000 7fcede59c700 1 --2- 172.21.15.12:0/1631679017 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55c8ea055b10 0x55c8ea057f30 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c004f1808905e619 server_cookie=7556adcdc6006827 in_seq=0 out_seq=0 2024-05-17T22:46:51.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.820+0000 7fcec5ffb700 10 client.25685 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:46:51.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.824+0000 7fcec5ffb700 1 -- 172.21.15.12:0/1631679017 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c8e9f0b020 con 0x55c8ea055b10 2024-05-17T22:46:51.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.824+0000 7fcec5ffb700 10 client.25685 handle_command_reply: tid=0 2024-05-17T22:46:51.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.824+0000 7fcef38d5b80 1 client.25685 shutdown 2024-05-17T22:46:51.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.830+0000 7fcef38d5b80 1 -- 172.21.15.12:0/1631679017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e9fdab00 msgr2=0x55c8e9ff6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:51.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.830+0000 7fcef38d5b80 1 --2- 172.21.15.12:0/1631679017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e9fdab00 0x55c8e9ff6fb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fced000d7b0 tx=0x7fced00a91b0 comp rx=0 tx=0).stop 2024-05-17T22:46:51.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.833+0000 7fcef38d5b80 1 -- 172.21.15.12:0/1631679017 shutdown_connections 2024-05-17T22:46:51.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.833+0000 7fcef38d5b80 1 --2- 172.21.15.12:0/1631679017 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55c8ea055b10 0x55c8ea057f30 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:51.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.833+0000 7fcec5ffb700 0 client.25685 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:46:51.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.833+0000 7fcef38d5b80 1 --2- 172.21.15.12:0/1631679017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8e9fe0510 0x55c8e9ff74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:51.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.834+0000 7fcef38d5b80 1 --2- 172.21.15.12:0/1631679017 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8e9e5c9f0 0x55c8e9ff6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:51.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.834+0000 7fcef38d5b80 1 --2- 172.21.15.12:0/1631679017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e9fdab00 0x55c8e9ff6fb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:51.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.834+0000 7fcef38d5b80 1 -- 172.21.15.12:0/1631679017 >> 172.21.15.12:0/1631679017 conn(0x55c8e9ebc7d0 msgr2=0x55c8e9fdd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:46:51.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.834+0000 7fcef38d5b80 1 -- 172.21.15.12:0/1631679017 shutdown_connections 2024-05-17T22:46:51.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.834+0000 7fcef38d5b80 1 -- 172.21.15.12:0/1631679017 wait complete. 2024-05-17T22:46:51.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.834+0000 7fcef38d5b80 20 client.25685 trim_cache size 0 max 16384 2024-05-17T22:46:51.838 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:46:51.839 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-17T22:46:51.839 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-17T22:46:51.839 INFO:teuthology.orchestra.run.smithi012.stdout: "d547c817-b870-4c6a-9e34-d3378d9e03e6": { 2024-05-17T22:46:51.839 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-17T22:46:51.839 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "d547c817-b870-4c6a-9e34-d3378d9e03e6", 2024-05-17T22:46:51.839 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-17T22:46:51.839 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:46:51.839 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:46:51.839 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:46:51.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.835+0000 7fcedf59e700 1 -- 172.21.15.12:0/1764752226 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fcec80b2900 msgr2=0x7fcec80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:51.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.835+0000 7fcedf59e700 1 --2- 172.21.15.12:0/1764752226 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fcec80b2900 0x7fcec80b4e10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55c8e9fd6ba0 tx=0x7fced4005f90 comp rx=0 tx=0).stop 2024-05-17T22:46:51.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.835+0000 7fcedf59e700 1 -- 172.21.15.12:0/1764752226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcee010b3b0 msgr2=0x7fcee0113e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:46:51.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.835+0000 7fcedf59e700 1 --2- 172.21.15.12:0/1764752226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcee010b3b0 0x7fcee0113e00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fced0007ee0 tx=0x7fced000d3b0 comp rx=0 tx=0).stop 2024-05-17T22:46:51.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.835+0000 7fcedf59e700 1 -- 172.21.15.12:0/1764752226 shutdown_connections 2024-05-17T22:46:51.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.835+0000 7fcedf59e700 1 --2- 172.21.15.12:0/1764752226 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fcec80b2900 0x7fcec80b4e10 secure :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55c8e9fd6ba0 tx=0x7fced4005f90 comp rx=0 tx=0).stop 2024-05-17T22:46:51.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.835+0000 7fcedf59e700 1 --2- 172.21.15.12:0/1764752226 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcee0116980 0x7fcee0114340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:51.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.835+0000 7fcedf59e700 1 --2- 172.21.15.12:0/1764752226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcee010aa60 0x7fcee01138c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:51.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.835+0000 7fcedf59e700 1 --2- 172.21.15.12:0/1764752226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcee010b3b0 0x7fcee0113e00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:46:51.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.836+0000 7fcedf59e700 1 -- 172.21.15.12:0/1764752226 >> 172.21.15.12:0/1764752226 conn(0x7fcee006eda0 msgr2=0x7fcee0072020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:46:51.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.836+0000 7fcedf59e700 1 -- 172.21.15.12:0/1764752226 shutdown_connections 2024-05-17T22:46:51.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:46:51.836+0000 7fcedf59e700 1 -- 172.21.15.12:0/1764752226 wait complete. 2024-05-17T22:46:52.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:51 smithi012 ceph-mon[31632]: pgmap v667: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.4 MiB/s wr, 460 op/s 2024-05-17T22:46:52.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:51 smithi151 ceph-mon[36101]: pgmap v667: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.4 MiB/s wr, 460 op/s 2024-05-17T22:46:52.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:51 smithi096 ceph-mon[36065]: pgmap v667: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.4 MiB/s wr, 460 op/s 2024-05-17T22:46:52.875 INFO:tasks.cephfs.filesystem:scrub status for tag:d547c817-b870-4c6a-9e34-d3378d9e03e6 - {'path': '/', 'tag': 'd547c817-b870-4c6a-9e34-d3378d9e03e6', 'options': 'recursive,force'} 2024-05-17T22:46:54.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:53 smithi151 ceph-mon[36101]: pgmap v668: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.0 MiB/s wr, 505 op/s 2024-05-17T22:46:54.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:53 smithi096 ceph-mon[36065]: pgmap v668: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.0 MiB/s wr, 505 op/s 2024-05-17T22:46:54.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:53 smithi012 ceph-mon[31632]: pgmap v668: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.0 MiB/s wr, 505 op/s 2024-05-17T22:46:56.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:55 smithi151 ceph-mon[36101]: pgmap v669: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 5.4 MiB/s wr, 516 op/s 2024-05-17T22:46:56.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:55 smithi096 ceph-mon[36065]: pgmap v669: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 5.4 MiB/s wr, 516 op/s 2024-05-17T22:46:56.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:55 smithi012 ceph-mon[31632]: pgmap v669: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 5.4 MiB/s wr, 516 op/s 2024-05-17T22:46:56.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 455.0 s, 500.6 tps, lat 200.537 ms stddev 253.487 2024-05-17T22:46:58.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:57 smithi151 ceph-mon[36101]: pgmap v670: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.3 MiB/s wr, 485 op/s 2024-05-17T22:46:58.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:57 smithi096 ceph-mon[36065]: pgmap v670: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.3 MiB/s wr, 485 op/s 2024-05-17T22:46:58.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:57 smithi012 ceph-mon[31632]: pgmap v670: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.3 MiB/s wr, 485 op/s 2024-05-17T22:46:59.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:58 smithi151 ceph-mon[36101]: scrub summary: idle 2024-05-17T22:46:59.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:58 smithi096 ceph-mon[36065]: scrub summary: idle 2024-05-17T22:46:59.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:58 smithi012 ceph-mon[31632]: scrub summary: idle 2024-05-17T22:47:00.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:59 smithi151 ceph-mon[36101]: scrub summary: idle 2024-05-17T22:47:00.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:59 smithi151 ceph-mon[36101]: scrub summary: idle 2024-05-17T22:47:00.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:46:59 smithi151 ceph-mon[36101]: pgmap v671: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.6 MiB/s wr, 567 op/s 2024-05-17T22:47:00.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:59 smithi096 ceph-mon[36065]: scrub summary: idle 2024-05-17T22:47:00.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:59 smithi096 ceph-mon[36065]: scrub summary: idle 2024-05-17T22:47:00.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:46:59 smithi096 ceph-mon[36065]: pgmap v671: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.6 MiB/s wr, 567 op/s 2024-05-17T22:47:00.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:59 smithi012 ceph-mon[31632]: scrub summary: idle 2024-05-17T22:47:00.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:59 smithi012 ceph-mon[31632]: scrub summary: idle 2024-05-17T22:47:00.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:46:59 smithi012 ceph-mon[31632]: pgmap v671: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.6 MiB/s wr, 567 op/s 2024-05-17T22:47:01.242 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:00 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:47:01.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:00 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:47:01.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:00 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:47:01.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 460.0 s, 527.2 tps, lat 184.015 ms stddev 197.966 2024-05-17T22:47:02.243 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:01 smithi012 ceph-mon[31632]: pgmap v672: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.9 MiB/s wr, 495 op/s 2024-05-17T22:47:02.243 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:01 smithi151 ceph-mon[36101]: pgmap v672: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.9 MiB/s wr, 495 op/s 2024-05-17T22:47:02.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:01 smithi096 ceph-mon[36065]: pgmap v672: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.9 MiB/s wr, 495 op/s 2024-05-17T22:47:04.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:03 smithi012 ceph-mon[31632]: pgmap v673: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.6 MiB/s wr, 547 op/s 2024-05-17T22:47:04.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:03 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:47:04.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:03 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:47:04.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:03 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:47:04.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:03 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:47:04.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:03 smithi151 ceph-mon[36101]: pgmap v673: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.6 MiB/s wr, 547 op/s 2024-05-17T22:47:04.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:03 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:47:04.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:03 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:47:04.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:03 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:47:04.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:03 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:47:04.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:03 smithi096 ceph-mon[36065]: pgmap v673: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.6 MiB/s wr, 547 op/s 2024-05-17T22:47:04.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:03 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:47:04.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:03 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:47:04.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:03 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:47:04.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:03 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:47:06.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 465.0 s, 484.6 tps, lat 206.249 ms stddev 239.517 2024-05-17T22:47:06.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:06 smithi151 ceph-mon[36101]: pgmap v674: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.5 MiB/s wr, 553 op/s 2024-05-17T22:47:06.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:06 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:47:06.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:06 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:47:06.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:06 smithi096 ceph-mon[36065]: pgmap v674: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.5 MiB/s wr, 553 op/s 2024-05-17T22:47:06.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:06 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:47:06.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:06 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:47:06.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:06 smithi012 ceph-mon[31632]: pgmap v674: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.5 MiB/s wr, 553 op/s 2024-05-17T22:47:06.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:06 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:47:06.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:06 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:47:07.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:07 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:47:07.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:07 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:47:07.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:07 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:47:07.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:07 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:47:07.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:07 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:47:07.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:07 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:47:07.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:07 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:47:07.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:07 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:47:07.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:07 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:47:08.449 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:08 smithi096 ceph-mon[36065]: pgmap v675: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.8 MiB/s wr, 504 op/s 2024-05-17T22:47:08.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:08 smithi151 ceph-mon[36101]: pgmap v675: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.8 MiB/s wr, 504 op/s 2024-05-17T22:47:08.594 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:08 smithi012 ceph-mon[31632]: pgmap v675: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.8 MiB/s wr, 504 op/s 2024-05-17T22:47:09.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:09 smithi151 ceph-mon[36101]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:47:09.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:09 smithi151 ceph-mon[36101]: pgmap v676: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.1 MiB/s wr, 556 op/s 2024-05-17T22:47:09.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:09 smithi096 ceph-mon[36065]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:47:09.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:09 smithi096 ceph-mon[36065]: pgmap v676: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.1 MiB/s wr, 556 op/s 2024-05-17T22:47:09.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:09 smithi012 ceph-mon[31632]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:47:09.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:09 smithi012 ceph-mon[31632]: pgmap v676: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.1 MiB/s wr, 556 op/s 2024-05-17T22:47:11.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 470.0 s, 523.8 tps, lat 198.007 ms stddev 249.157 2024-05-17T22:47:12.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:11 smithi151 ceph-mon[36101]: pgmap v677: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.6 MiB/s wr, 469 op/s 2024-05-17T22:47:12.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:11 smithi096 ceph-mon[36065]: pgmap v677: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.6 MiB/s wr, 469 op/s 2024-05-17T22:47:12.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:11 smithi012 ceph-mon[31632]: pgmap v677: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.6 MiB/s wr, 469 op/s 2024-05-17T22:47:14.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:13 smithi012 ceph-mon[31632]: pgmap v678: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 4.8 MiB/s wr, 496 op/s 2024-05-17T22:47:14.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:13 smithi151 ceph-mon[36101]: pgmap v678: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 4.8 MiB/s wr, 496 op/s 2024-05-17T22:47:14.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:13 smithi096 ceph-mon[36065]: pgmap v678: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 4.8 MiB/s wr, 496 op/s 2024-05-17T22:47:16.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:15 smithi151 ceph-mon[36101]: pgmap v679: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.3 MiB/s wr, 471 op/s 2024-05-17T22:47:16.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:15 smithi096 ceph-mon[36065]: pgmap v679: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.3 MiB/s wr, 471 op/s 2024-05-17T22:47:16.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:15 smithi012 ceph-mon[31632]: pgmap v679: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.3 MiB/s wr, 471 op/s 2024-05-17T22:47:16.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 475.0 s, 521.4 tps, lat 184.681 ms stddev 214.857 2024-05-17T22:47:18.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:17 smithi151 ceph-mon[36101]: pgmap v680: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.2 MiB/s wr, 456 op/s 2024-05-17T22:47:18.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:17 smithi096 ceph-mon[36065]: pgmap v680: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.2 MiB/s wr, 456 op/s 2024-05-17T22:47:18.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:17 smithi012 ceph-mon[31632]: pgmap v680: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.2 MiB/s wr, 456 op/s 2024-05-17T22:47:20.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:19 smithi151 ceph-mon[36101]: pgmap v681: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.0 MiB/s wr, 527 op/s 2024-05-17T22:47:20.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:19 smithi096 ceph-mon[36065]: pgmap v681: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.0 MiB/s wr, 527 op/s 2024-05-17T22:47:20.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:19 smithi012 ceph-mon[31632]: pgmap v681: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.0 MiB/s wr, 527 op/s 2024-05-17T22:47:21.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 480.0 s, 555.4 tps, lat 185.821 ms stddev 244.513 2024-05-17T22:47:22.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:21 smithi151 ceph-mon[36101]: pgmap v682: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.4 MiB/s wr, 473 op/s 2024-05-17T22:47:22.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:21 smithi096 ceph-mon[36065]: pgmap v682: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.4 MiB/s wr, 473 op/s 2024-05-17T22:47:22.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:21 smithi012 ceph-mon[31632]: pgmap v682: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.4 MiB/s wr, 473 op/s 2024-05-17T22:47:22.876 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-17T22:47:23.356 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:47:24.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:23 smithi012 ceph-mon[31632]: pgmap v683: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.8 MiB/s wr, 519 op/s 2024-05-17T22:47:24.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:23 smithi151 ceph-mon[36101]: pgmap v683: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.8 MiB/s wr, 519 op/s 2024-05-17T22:47:24.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:23 smithi096 ceph-mon[36065]: pgmap v683: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.8 MiB/s wr, 519 op/s 2024-05-17T22:47:26.219 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:25 smithi012 ceph-mon[31632]: pgmap v684: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.8 MiB/s wr, 512 op/s 2024-05-17T22:47:26.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:25 smithi151 ceph-mon[36101]: pgmap v684: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.8 MiB/s wr, 512 op/s 2024-05-17T22:47:26.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:25 smithi096 ceph-mon[36065]: pgmap v684: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.8 MiB/s wr, 512 op/s 2024-05-17T22:47:26.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 485.0 s, 425.4 tps, lat 221.146 ms stddev 250.126 2024-05-17T22:47:26.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.617+0000 7fda7ffff700 1 -- 172.21.15.12:0/628260909 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda8011d010 msgr2=0x7fda8011f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:26.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.617+0000 7fda7ffff700 1 --2- 172.21.15.12:0/628260909 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda8011d010 0x7fda8011f430 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fda78009f90 tx=0x7fda7800b330 comp rx=0 tx=0).stop 2024-05-17T22:47:26.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.620+0000 7fda7ffff700 1 -- 172.21.15.12:0/628260909 shutdown_connections 2024-05-17T22:47:26.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.620+0000 7fda7ffff700 1 --2- 172.21.15.12:0/628260909 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda8011d010 0x7fda8011f430 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:26.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.620+0000 7fda7ffff700 1 --2- 172.21.15.12:0/628260909 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda80112230 0x7fda8010aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:26.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.620+0000 7fda7ffff700 1 --2- 172.21.15.12:0/628260909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda8010afc0 0x7fda8010b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:26.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.620+0000 7fda7ffff700 1 -- 172.21.15.12:0/628260909 >> 172.21.15.12:0/628260909 conn(0x7fda8006eda0 msgr2=0x7fda8006f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:47:26.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.625+0000 7fda7ffff700 1 -- 172.21.15.12:0/628260909 shutdown_connections 2024-05-17T22:47:26.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.630+0000 7fda7ffff700 1 -- 172.21.15.12:0/628260909 wait complete. 2024-05-17T22:47:26.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.631+0000 7fda7ffff700 1 Processor -- start 2024-05-17T22:47:26.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.632+0000 7fda7ffff700 1 -- start start 2024-05-17T22:47:26.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.632+0000 7fda7ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda8010afc0 0x7fda801a9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:26.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.632+0000 7fda7ffff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda80112230 0x7fda801a9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:26.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.632+0000 7fda7ffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda8011d010 0x7fda801adb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:26.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.633+0000 7fda7ffff700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fda80122a60 con 0x7fda8010afc0 2024-05-17T22:47:26.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.633+0000 7fda7ffff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fda801228e0 con 0x7fda8011d010 2024-05-17T22:47:26.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.633+0000 7fda7ffff700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fda80122be0 con 0x7fda80112230 2024-05-17T22:47:26.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.633+0000 7fda7effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda8010afc0 0x7fda801a9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:26.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.633+0000 7fda7effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda8010afc0 0x7fda801a9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46638/0 (socket says 172.21.15.12:46638) 2024-05-17T22:47:26.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.633+0000 7fda7effd700 1 -- 172.21.15.12:0/2745025689 learned_addr learned my addr 172.21.15.12:0/2745025689 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:47:26.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.634+0000 7fda7e7fc700 1 --2- 172.21.15.12:0/2745025689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda80112230 0x7fda801a9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:26.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.634+0000 7fda7effd700 1 -- 172.21.15.12:0/2745025689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda80112230 msgr2=0x7fda801a9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:26.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.634+0000 7fda7effd700 1 --2- 172.21.15.12:0/2745025689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda80112230 0x7fda801a9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:26.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.634+0000 7fda7effd700 1 -- 172.21.15.12:0/2745025689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda8011d010 msgr2=0x7fda801adb90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-17T22:47:26.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.634+0000 7fda7effd700 1 --2- 172.21.15.12:0/2745025689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda8011d010 0x7fda801adb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:26.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.634+0000 7fda7effd700 1 -- 172.21.15.12:0/2745025689 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda801ae230 con 0x7fda8010afc0 2024-05-17T22:47:26.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.635+0000 7fda7effd700 1 --2- 172.21.15.12:0/2745025689 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda8010afc0 0x7fda801a9360 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fda7000b500 tx=0x7fda7000b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:47:26.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.637+0000 7fda67fff700 1 -- 172.21.15.12:0/2745025689 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda70015020 con 0x7fda8010afc0 2024-05-17T22:47:26.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.637+0000 7fda67fff700 1 -- 172.21.15.12:0/2745025689 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda70011ce0 con 0x7fda8010afc0 2024-05-17T22:47:26.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.637+0000 7fda67fff700 1 -- 172.21.15.12:0/2745025689 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda70010de0 con 0x7fda8010afc0 2024-05-17T22:47:26.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.639+0000 7fda7ffff700 1 -- 172.21.15.12:0/2745025689 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda801ae520 con 0x7fda8010afc0 2024-05-17T22:47:26.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.639+0000 7fda7ffff700 1 -- 172.21.15.12:0/2745025689 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda801127f0 con 0x7fda8010afc0 2024-05-17T22:47:26.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.653+0000 7fda93ff5b80 1 Processor -- start 2024-05-17T22:47:26.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.655+0000 7fda67fff700 1 -- 172.21.15.12:0/2745025689 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fda70018020 con 0x7fda8010afc0 2024-05-17T22:47:26.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.655+0000 7fda93ff5b80 1 -- start start 2024-05-17T22:47:26.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.655+0000 7fda93ff5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f7da379f0 0x561f7dbbafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:26.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.655+0000 7fda93ff5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561f7dbbb510 0x561f7dbb55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:26.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.655+0000 7fda7f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f7da379f0 0x561f7dbbafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:26.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.655+0000 7fda7f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f7da379f0 0x561f7dbbafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46646/0 (socket says 172.21.15.12:46646) 2024-05-17T22:47:26.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.656+0000 7fda7f7fe700 1 -- 172.21.15.12:0/907242680 learned_addr learned my addr 172.21.15.12:0/907242680 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:47:26.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.656+0000 7fda93ff5b80 1 --2- 172.21.15.12:0/907242680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f7dbb5b00 0x561f7dbb5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:26.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.656+0000 7fda7effd700 1 --2- 172.21.15.12:0/907242680 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561f7dbbb510 0x561f7dbb55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:26.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.656+0000 7fda93ff5b80 1 -- 172.21.15.12:0/907242680 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x561f7dae40d0 con 0x561f7da379f0 2024-05-17T22:47:26.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.656+0000 7fda93ff5b80 1 -- 172.21.15.12:0/907242680 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561f7dad3c70 con 0x561f7dbb5b00 2024-05-17T22:47:26.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.656+0000 7fda93ff5b80 1 -- 172.21.15.12:0/907242680 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561f7dad7cb0 con 0x561f7dbbb510 2024-05-17T22:47:26.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.656+0000 7fda7e7fc700 1 --2- 172.21.15.12:0/907242680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f7dbb5b00 0x561f7dbb5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:26.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.656+0000 7fda7e7fc700 1 -- 172.21.15.12:0/907242680 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561f7dbbb510 msgr2=0x561f7dbb55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:47:26.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.656+0000 7fda7e7fc700 1 --2- 172.21.15.12:0/907242680 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561f7dbbb510 0x561f7dbb55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:26.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.656+0000 7fda7e7fc700 1 -- 172.21.15.12:0/907242680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f7da379f0 msgr2=0x561f7dbbafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:47:26.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.656+0000 7fda7e7fc700 1 --2- 172.21.15.12:0/907242680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f7da379f0 0x561f7dbbafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:26.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.656+0000 7fda7e7fc700 1 -- 172.21.15.12:0/907242680 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f7da6d5c0 con 0x561f7dbb5b00 2024-05-17T22:47:26.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.657+0000 7fda7e7fc700 1 --2- 172.21.15.12:0/907242680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f7dbb5b00 0x561f7dbb5f10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fda74005fd0 tx=0x7fda74009ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cd8928c299aae8b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:47:26.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.657+0000 7fda67fff700 1 --2- 172.21.15.12:0/2745025689 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fda680c3aa0 0x7fda680c5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:26.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.657+0000 7fda657fa700 1 -- 172.21.15.12:0/907242680 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda74011b90 con 0x561f7dbb5b00 2024-05-17T22:47:26.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.657+0000 7fda7f7fe700 1 --2- 172.21.15.12:0/2745025689 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fda680c3aa0 0x7fda680c5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:26.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.657+0000 7fda67fff700 1 -- 172.21.15.12:0/2745025689 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fda700e1b30 con 0x7fda8010afc0 2024-05-17T22:47:26.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.657+0000 7fda657fa700 1 -- 172.21.15.12:0/907242680 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda74011d30 con 0x561f7dbb5b00 2024-05-17T22:47:26.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.657+0000 7fda657fa700 1 -- 172.21.15.12:0/907242680 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7400f6c0 con 0x561f7dbb5b00 2024-05-17T22:47:26.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.658+0000 7fda93ff5b80 1 -- 172.21.15.12:0/907242680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f7dbb5b00 msgr2=0x561f7dbb5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:26.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.658+0000 7fda7f7fe700 1 --2- 172.21.15.12:0/2745025689 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fda680c3aa0 0x7fda680c5fb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fda7800b6c0 tx=0x7fda780097e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:47:26.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.658+0000 7fda93ff5b80 1 --2- 172.21.15.12:0/907242680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f7dbb5b00 0x561f7dbb5f10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fda74005fd0 tx=0x7fda74009ee0 comp rx=0 tx=0).stop 2024-05-17T22:47:26.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.658+0000 7fda93ff5b80 1 -- 172.21.15.12:0/907242680 shutdown_connections 2024-05-17T22:47:26.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.659+0000 7fda93ff5b80 1 --2- 172.21.15.12:0/907242680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f7dbb5b00 0x561f7dbb5f10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:26.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.659+0000 7fda93ff5b80 1 --2- 172.21.15.12:0/907242680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f7da379f0 0x561f7dbbafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:26.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.659+0000 7fda93ff5b80 1 --2- 172.21.15.12:0/907242680 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561f7dbbb510 0x561f7dbb55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:26.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.659+0000 7fda93ff5b80 1 -- 172.21.15.12:0/907242680 >> 172.21.15.12:0/907242680 conn(0x561f7da977d0 msgr2=0x561f7dbb01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:47:26.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.659+0000 7fda93ff5b80 1 -- 172.21.15.12:0/907242680 shutdown_connections 2024-05-17T22:47:26.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.659+0000 7fda93ff5b80 1 -- 172.21.15.12:0/907242680 wait complete. 2024-05-17T22:47:26.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.659+0000 7fda93ff5b80 1 -- start start 2024-05-17T22:47:26.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.660+0000 7fda93ff5b80 1 Processor -- start 2024-05-17T22:47:26.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.660+0000 7fda93ff5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561f7da379f0 0x561f7dbbec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:26.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.660+0000 7fda93ff5b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f7dbb5b00 0x561f7dbbf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:26.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.660+0000 7fda93ff5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f7dbbb510 0x561f7dbbf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:26.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.660+0000 7fda93ff5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x561f7d9db150 con 0x561f7dbbb510 2024-05-17T22:47:26.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.660+0000 7fda93ff5b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561f7da75940 con 0x561f7dbb5b00 2024-05-17T22:47:26.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.660+0000 7fda93ff5b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561f7dbc1ec0 con 0x561f7da379f0 2024-05-17T22:47:26.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.660+0000 7fda93ff5b80 10 client.0 _ll_register_callbacks cb 0x561f7da6ab50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:47:26.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.662+0000 7fda7effd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561f7da379f0 0x561f7dbbec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:26.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.662+0000 7fda7effd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561f7da379f0 0x561f7dbbec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:44362/0 (socket says 172.21.15.12:44362) 2024-05-17T22:47:26.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.662+0000 7fda7effd700 1 -- 172.21.15.12:0/2188875939 learned_addr learned my addr 172.21.15.12:0/2188875939 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:47:26.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.663+0000 7fda7e7fc700 1 --2- 172.21.15.12:0/2188875939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f7dbb5b00 0x561f7dbbf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:26.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.663+0000 7fda7f7fe700 1 --2- 172.21.15.12:0/2188875939 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f7dbbb510 0x561f7dbbf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:26.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.664+0000 7fda7e7fc700 1 -- 172.21.15.12:0/2188875939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561f7da379f0 msgr2=0x561f7dbbec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:47:26.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.664+0000 7fda7e7fc700 1 --2- 172.21.15.12:0/2188875939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561f7da379f0 0x561f7dbbec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:26.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.664+0000 7fda7e7fc700 1 -- 172.21.15.12:0/2188875939 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f7dbbb510 msgr2=0x561f7dbbf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:47:26.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.664+0000 7fda7e7fc700 1 --2- 172.21.15.12:0/2188875939 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f7dbbb510 0x561f7dbbf6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:26.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.664+0000 7fda7e7fc700 1 -- 172.21.15.12:0/2188875939 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f7da24480 con 0x561f7dbb5b00 2024-05-17T22:47:26.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.664+0000 7fda7f7fe700 1 --2- 172.21.15.12:0/2188875939 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f7dbbb510 0x561f7dbbf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:47:26.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.664+0000 7fda62ff5700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:47:26.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.664+0000 7fda7e7fc700 1 --2- 172.21.15.12:0/2188875939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f7dbb5b00 0x561f7dbbf1b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fda7401c040 tx=0x7fda74012660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47b8e6ca598d648e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:47:26.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.665+0000 7fda62ff5700 1 -- 172.21.15.12:0/2188875939 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda74028030 con 0x561f7dbb5b00 2024-05-17T22:47:26.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.665+0000 7fda62ff5700 1 -- 172.21.15.12:0/2188875939 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fda74004020 con 0x561f7dbb5b00 2024-05-17T22:47:26.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.665+0000 7fda93ff5b80 1 -- 172.21.15.12:0/2188875939 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda74011d30 con 0x561f7dbb5b00 2024-05-17T22:47:26.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.665+0000 7fda62ff5700 1 -- 172.21.15.12:0/2188875939 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fda74032050 con 0x561f7dbb5b00 2024-05-17T22:47:26.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.665+0000 7fda93ff5b80 10 client.25694 fetch_fsmap learned FSMap version 38 2024-05-17T22:47:26.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.665+0000 7fda93ff5b80 1 -- 172.21.15.12:0/2188875939 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x561f7dbcc9c0 con 0x561f7dbb5b00 2024-05-17T22:47:26.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.665+0000 7fda62ff5700 1 -- 172.21.15.12:0/2188875939 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda74011d30 con 0x561f7dbb5b00 2024-05-17T22:47:26.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.666+0000 7fda62ff5700 1 -- 172.21.15.12:0/2188875939 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 38) ==== 8350+0+0 (secure 0 0 0) 0x7fda74004750 con 0x561f7dbb5b00 2024-05-17T22:47:26.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.666+0000 7fda93ff5b80 10 client.25694 fetch_fsmap finished waiting for FSMap version 38 2024-05-17T22:47:26.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.666+0000 7fda93ff5b80 10 client.25694 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:47:26.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.666+0000 7fda93ff5b80 20 client.25694 populate_metadata read hostname 'smithi012' 2024-05-17T22:47:26.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.666+0000 7fda93ff5b80 1 --2- 172.21.15.12:0/2188875939 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x561f7dbceff0 0x561f7dbd1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:26.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.666+0000 7fda93ff5b80 4 client.25694 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:47:26.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.666+0000 7fda93ff5b80 1 -- 172.21.15.12:0/2188875939 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f7dae6020 con 0x561f7dbceff0 2024-05-17T22:47:26.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.666+0000 7fda7effd700 1 --2- 172.21.15.12:0/2188875939 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x561f7dbceff0 0x561f7dbd1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:26.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.666+0000 7fda62ff5700 10 client.25694 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:47:26.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.667+0000 7fda7effd700 1 --2- 172.21.15.12:0/2188875939 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x561f7dbceff0 0x561f7dbd1410 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a33a9a632bf3fe3d server_cookie=f1b06f18a5dd70a1 in_seq=0 out_seq=0 2024-05-17T22:47:26.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.668+0000 7fda62ff5700 1 -- 172.21.15.12:0/2188875939 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561f7dae6020 con 0x561f7dbceff0 2024-05-17T22:47:26.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.668+0000 7fda62ff5700 10 client.25694 handle_command_reply: tid=0 2024-05-17T22:47:26.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.668+0000 7fda93ff5b80 1 client.25694 shutdown 2024-05-17T22:47:26.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.668+0000 7fda93ff5b80 1 -- 172.21.15.12:0/2188875939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f7dbb5b00 msgr2=0x561f7dbbf1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:26.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.668+0000 7fda93ff5b80 1 --2- 172.21.15.12:0/2188875939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f7dbb5b00 0x561f7dbbf1b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fda7401c040 tx=0x7fda74012660 comp rx=0 tx=0).stop 2024-05-17T22:47:26.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.668+0000 7fda93ff5b80 1 -- 172.21.15.12:0/2188875939 shutdown_connections 2024-05-17T22:47:26.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.668+0000 7fda93ff5b80 1 --2- 172.21.15.12:0/2188875939 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x561f7dbceff0 0x561f7dbd1410 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:26.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.668+0000 7fda62ff5700 0 client.25694 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:47:26.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.669+0000 7fda93ff5b80 1 --2- 172.21.15.12:0/2188875939 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f7dbbb510 0x561f7dbbf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:26.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.669+0000 7fda93ff5b80 1 --2- 172.21.15.12:0/2188875939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561f7da379f0 0x561f7dbbec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:26.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.669+0000 7fda93ff5b80 1 --2- 172.21.15.12:0/2188875939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f7dbb5b00 0x561f7dbbf1b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:26.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.669+0000 7fda93ff5b80 1 -- 172.21.15.12:0/2188875939 >> 172.21.15.12:0/2188875939 conn(0x561f7da977d0 msgr2=0x561f7dbb0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:47:26.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.669+0000 7fda93ff5b80 1 -- 172.21.15.12:0/2188875939 shutdown_connections 2024-05-17T22:47:26.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.669+0000 7fda93ff5b80 1 -- 172.21.15.12:0/2188875939 wait complete. 2024-05-17T22:47:26.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.669+0000 7fda93ff5b80 20 client.25694 trim_cache size 0 max 16384 2024-05-17T22:47:26.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.697+0000 7fda93ff5b80 1 Processor -- start 2024-05-17T22:47:26.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.697+0000 7fda93ff5b80 1 -- start start 2024-05-17T22:47:26.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.698+0000 7fda93ff5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f7da977d0 0x561f7dbb55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:26.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.698+0000 7fda93ff5b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f7da379f0 0x561f7dbaabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:26.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.698+0000 7fda93ff5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561f7dbb5b00 0x561f7dbbf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:26.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.698+0000 7fda93ff5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fda801b89e0 con 0x561f7da977d0 2024-05-17T22:47:26.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.698+0000 7fda93ff5b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fda801b6270 con 0x561f7da379f0 2024-05-17T22:47:26.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.698+0000 7fda93ff5b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fda80119280 con 0x561f7dbb5b00 2024-05-17T22:47:26.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.698+0000 7fda7effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f7da977d0 0x561f7dbb55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:26.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.698+0000 7fda7effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f7da977d0 0x561f7dbb55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46660/0 (socket says 172.21.15.12:46660) 2024-05-17T22:47:26.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.698+0000 7fda7effd700 1 -- 172.21.15.12:0/3386576720 learned_addr learned my addr 172.21.15.12:0/3386576720 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:47:26.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.698+0000 7fda7e7fc700 1 --2- 172.21.15.12:0/3386576720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f7da379f0 0x561f7dbaabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:26.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.698+0000 7fda7f7fe700 1 --2- 172.21.15.12:0/3386576720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561f7dbb5b00 0x561f7dbbf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:26.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.698+0000 7fda7e7fc700 1 -- 172.21.15.12:0/3386576720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561f7dbb5b00 msgr2=0x561f7dbbf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:47:26.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.698+0000 7fda7e7fc700 1 --2- 172.21.15.12:0/3386576720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561f7dbb5b00 0x561f7dbbf540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:26.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.698+0000 7fda7e7fc700 1 -- 172.21.15.12:0/3386576720 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f7da977d0 msgr2=0x561f7dbb55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:47:26.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.698+0000 7fda7e7fc700 1 --2- 172.21.15.12:0/3386576720 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f7da977d0 0x561f7dbb55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:26.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.698+0000 7fda7e7fc700 1 -- 172.21.15.12:0/3386576720 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f7dbb8ff0 con 0x561f7da379f0 2024-05-17T22:47:26.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.699+0000 7fda7effd700 1 --2- 172.21.15.12:0/3386576720 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f7da977d0 0x561f7dbb55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:47:26.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.699+0000 7fda7e7fc700 1 --2- 172.21.15.12:0/3386576720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f7da379f0 0x561f7dbaabe0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fda74004ba0 tx=0x7fda740116e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=857a511a2cde13f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:47:26.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.699+0000 7fda64ff9700 1 -- 172.21.15.12:0/3386576720 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda74028040 con 0x561f7da379f0 2024-05-17T22:47:26.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.699+0000 7fda64ff9700 1 -- 172.21.15.12:0/3386576720 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda74011d30 con 0x561f7da379f0 2024-05-17T22:47:26.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.699+0000 7fda64ff9700 1 -- 172.21.15.12:0/3386576720 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda74003e60 con 0x561f7da379f0 2024-05-17T22:47:26.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.700+0000 7fda93ff5b80 1 -- 172.21.15.12:0/3386576720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f7da379f0 msgr2=0x561f7dbaabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:26.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.700+0000 7fda93ff5b80 1 --2- 172.21.15.12:0/3386576720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f7da379f0 0x561f7dbaabe0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fda74004ba0 tx=0x7fda740116e0 comp rx=0 tx=0).stop 2024-05-17T22:47:26.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.700+0000 7fda93ff5b80 1 -- 172.21.15.12:0/3386576720 shutdown_connections 2024-05-17T22:47:26.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.700+0000 7fda93ff5b80 1 --2- 172.21.15.12:0/3386576720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561f7dbb5b00 0x561f7dbbf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:26.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.700+0000 7fda93ff5b80 1 --2- 172.21.15.12:0/3386576720 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f7da977d0 0x561f7dbb55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:26.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.700+0000 7fda93ff5b80 1 --2- 172.21.15.12:0/3386576720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f7da379f0 0x561f7dbaabe0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:26.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.700+0000 7fda93ff5b80 1 -- 172.21.15.12:0/3386576720 >> 172.21.15.12:0/3386576720 conn(0x561f7dbb04a0 msgr2=0x561f7dbc0560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:47:26.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.700+0000 7fda93ff5b80 1 -- 172.21.15.12:0/3386576720 shutdown_connections 2024-05-17T22:47:26.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.701+0000 7fda93ff5b80 1 -- 172.21.15.12:0/3386576720 wait complete. 2024-05-17T22:47:26.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.701+0000 7fda93ff5b80 1 -- start start 2024-05-17T22:47:26.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.701+0000 7fda93ff5b80 1 Processor -- start 2024-05-17T22:47:26.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.701+0000 7fda93ff5b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f7da379f0 0x561f7dbd23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:26.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.702+0000 7fda93ff5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f7da977d0 0x561f7dbd2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:26.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.702+0000 7fda93ff5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561f7dbb5b00 0x561f7dc38540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:26.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.702+0000 7fda93ff5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fda801198f0 con 0x561f7da977d0 2024-05-17T22:47:26.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.702+0000 7fda7effd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f7da379f0 0x561f7dbd23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:26.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.702+0000 7fda7effd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f7da379f0 0x561f7dbd23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:39848/0 (socket says 172.21.15.12:39848) 2024-05-17T22:47:26.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.702+0000 7fda7effd700 1 -- 172.21.15.12:0/831039374 learned_addr learned my addr 172.21.15.12:0/831039374 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:47:26.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.702+0000 7fda93ff5b80 1 -- 172.21.15.12:0/831039374 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fda8011ad60 con 0x561f7da379f0 2024-05-17T22:47:26.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.702+0000 7fda7e7fc700 1 --2- 172.21.15.12:0/831039374 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f7da977d0 0x561f7dbd2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:26.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.702+0000 7fda93ff5b80 1 -- 172.21.15.12:0/831039374 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fda80119fc0 con 0x561f7dbb5b00 2024-05-17T22:47:26.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.702+0000 7fda7f7fe700 1 --2- 172.21.15.12:0/831039374 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561f7dbb5b00 0x561f7dc38540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:26.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.702+0000 7fda93ff5b80 10 client.0 _ll_register_callbacks cb 0x561f7da6ab50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:47:26.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.703+0000 7fda7effd700 1 -- 172.21.15.12:0/831039374 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561f7dbb5b00 msgr2=0x561f7dc38540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:47:26.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.703+0000 7fda7effd700 1 --2- 172.21.15.12:0/831039374 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561f7dbb5b00 0x561f7dc38540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:26.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.703+0000 7fda7effd700 1 -- 172.21.15.12:0/831039374 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f7da977d0 msgr2=0x561f7dbd2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:47:26.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.703+0000 7fda7effd700 1 --2- 172.21.15.12:0/831039374 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f7da977d0 0x561f7dbd2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:26.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.703+0000 7fda7effd700 1 -- 172.21.15.12:0/831039374 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f7dbd3190 con 0x561f7da379f0 2024-05-17T22:47:26.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.703+0000 7fda7effd700 1 --2- 172.21.15.12:0/831039374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f7da379f0 0x561f7dbd23c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fda700a4de0 tx=0x7fda700e54f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d355cf327fd8255d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:47:26.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.703+0000 7fda62ff5700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:47:26.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.703+0000 7fda62ff5700 1 -- 172.21.15.12:0/831039374 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda700a84d0 con 0x561f7da379f0 2024-05-17T22:47:26.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.703+0000 7fda62ff5700 1 -- 172.21.15.12:0/831039374 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fda700c33e0 con 0x561f7da379f0 2024-05-17T22:47:26.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.703+0000 7fda93ff5b80 1 -- 172.21.15.12:0/831039374 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda74011d30 con 0x561f7da379f0 2024-05-17T22:47:26.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.703+0000 7fda62ff5700 1 -- 172.21.15.12:0/831039374 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fda700a8870 con 0x561f7da379f0 2024-05-17T22:47:26.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.703+0000 7fda93ff5b80 10 client.25700 fetch_fsmap learned FSMap version 38 2024-05-17T22:47:26.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.704+0000 7fda93ff5b80 1 -- 172.21.15.12:0/831039374 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x561f7dc39ae0 con 0x561f7da379f0 2024-05-17T22:47:26.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.704+0000 7fda62ff5700 1 -- 172.21.15.12:0/831039374 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda700a9460 con 0x561f7da379f0 2024-05-17T22:47:26.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.704+0000 7fda62ff5700 1 -- 172.21.15.12:0/831039374 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 38) ==== 8350+0+0 (secure 0 0 0) 0x7fda70022060 con 0x561f7da379f0 2024-05-17T22:47:26.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.704+0000 7fda93ff5b80 10 client.25700 fetch_fsmap finished waiting for FSMap version 38 2024-05-17T22:47:26.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.704+0000 7fda93ff5b80 10 client.25700 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:47:26.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.704+0000 7fda93ff5b80 20 client.25700 populate_metadata read hostname 'smithi012' 2024-05-17T22:47:26.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.704+0000 7fda93ff5b80 1 --2- 172.21.15.12:0/831039374 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x561f7dbbb510 0x561f7dc3c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:26.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.704+0000 7fda93ff5b80 4 client.25700 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:47:26.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.704+0000 7fda93ff5b80 1 -- 172.21.15.12:0/831039374 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fda74011d30 con 0x561f7dbbb510 2024-05-17T22:47:26.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.705+0000 7fda7e7fc700 1 --2- 172.21.15.12:0/831039374 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x561f7dbbb510 0x561f7dc3c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:26.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.705+0000 7fda7e7fc700 1 --2- 172.21.15.12:0/831039374 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x561f7dbbb510 0x561f7dc3c1e0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=442059f4bf636aa7 server_cookie=8e477c5f3550edf2 in_seq=0 out_seq=0 2024-05-17T22:47:26.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.705+0000 7fda62ff5700 10 client.25700 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:47:26.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.705+0000 7fda62ff5700 1 -- 172.21.15.12:0/831039374 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fda74011d30 con 0x561f7dbbb510 2024-05-17T22:47:26.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.705+0000 7fda62ff5700 10 client.25700 handle_command_reply: tid=0 2024-05-17T22:47:26.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.705+0000 7fda93ff5b80 1 client.25700 shutdown 2024-05-17T22:47:26.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.706+0000 7fda93ff5b80 1 -- 172.21.15.12:0/831039374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f7da379f0 msgr2=0x561f7dbd23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:26.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.706+0000 7fda93ff5b80 1 --2- 172.21.15.12:0/831039374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f7da379f0 0x561f7dbd23c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fda700a4de0 tx=0x7fda700e54f0 comp rx=0 tx=0).stop 2024-05-17T22:47:26.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.706+0000 7fda93ff5b80 1 -- 172.21.15.12:0/831039374 shutdown_connections 2024-05-17T22:47:26.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.706+0000 7fda93ff5b80 1 --2- 172.21.15.12:0/831039374 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561f7dbb5b00 0x561f7dc38540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:26.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.706+0000 7fda93ff5b80 1 --2- 172.21.15.12:0/831039374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f7da379f0 0x561f7dbd23c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:26.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.706+0000 7fda93ff5b80 1 --2- 172.21.15.12:0/831039374 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x561f7dbbb510 0x561f7dc3c1e0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:26.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.706+0000 7fda93ff5b80 1 --2- 172.21.15.12:0/831039374 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f7da977d0 0x561f7dbd2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:26.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.706+0000 7fda93ff5b80 1 -- 172.21.15.12:0/831039374 >> 172.21.15.12:0/831039374 conn(0x561f7dbb04a0 msgr2=0x561f7dbba5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:47:26.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.706+0000 7fda62ff5700 0 client.25700 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:47:26.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.706+0000 7fda93ff5b80 1 -- 172.21.15.12:0/831039374 shutdown_connections 2024-05-17T22:47:26.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.706+0000 7fda93ff5b80 1 -- 172.21.15.12:0/831039374 wait complete. 2024-05-17T22:47:26.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.706+0000 7fda93ff5b80 20 client.25700 trim_cache size 0 max 16384 2024-05-17T22:47:26.713 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:47:26.713 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-17T22:47:26.713 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-17T22:47:26.714 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:47:26.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.707+0000 7fda7ffff700 1 -- 172.21.15.12:0/2745025689 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fda680c3aa0 msgr2=0x7fda680c5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:26.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.707+0000 7fda7ffff700 1 --2- 172.21.15.12:0/2745025689 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fda680c3aa0 0x7fda680c5fb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fda7800b6c0 tx=0x7fda780097e0 comp rx=0 tx=0).stop 2024-05-17T22:47:26.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.707+0000 7fda7ffff700 1 -- 172.21.15.12:0/2745025689 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda8010afc0 msgr2=0x7fda801a9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:26.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.707+0000 7fda7ffff700 1 --2- 172.21.15.12:0/2745025689 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda8010afc0 0x7fda801a9360 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fda7000b500 tx=0x7fda7000b8c0 comp rx=0 tx=0).stop 2024-05-17T22:47:26.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.707+0000 7fda7ffff700 1 -- 172.21.15.12:0/2745025689 shutdown_connections 2024-05-17T22:47:26.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.707+0000 7fda7ffff700 1 --2- 172.21.15.12:0/2745025689 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fda680c3aa0 0x7fda680c5fb0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:26.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.707+0000 7fda7ffff700 1 --2- 172.21.15.12:0/2745025689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda8011d010 0x7fda801adb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:26.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.707+0000 7fda7ffff700 1 --2- 172.21.15.12:0/2745025689 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda8010afc0 0x7fda801a9360 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:26.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.707+0000 7fda7ffff700 1 --2- 172.21.15.12:0/2745025689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda80112230 0x7fda801a9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:26.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.707+0000 7fda7ffff700 1 -- 172.21.15.12:0/2745025689 >> 172.21.15.12:0/2745025689 conn(0x7fda8006eda0 msgr2=0x7fda8011b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:47:26.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.707+0000 7fda7ffff700 1 -- 172.21.15.12:0/2745025689 shutdown_connections 2024-05-17T22:47:26.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:26.707+0000 7fda7ffff700 1 -- 172.21.15.12:0/2745025689 wait complete. 2024-05-17T22:47:27.954 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-17T22:47:27.955 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T22:47:28.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:27 smithi151 ceph-mon[36101]: pgmap v685: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.0 MiB/s wr, 529 op/s 2024-05-17T22:47:28.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:27 smithi096 ceph-mon[36065]: pgmap v685: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.0 MiB/s wr, 529 op/s 2024-05-17T22:47:28.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:27 smithi012 ceph-mon[31632]: pgmap v685: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.0 MiB/s wr, 529 op/s 2024-05-17T22:47:28.409 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:47:30.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:29 smithi151 ceph-mon[36101]: pgmap v686: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.0 MiB/s wr, 549 op/s 2024-05-17T22:47:30.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:29 smithi096 ceph-mon[36065]: pgmap v686: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.0 MiB/s wr, 549 op/s 2024-05-17T22:47:30.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:29 smithi012 ceph-mon[31632]: pgmap v686: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.0 MiB/s wr, 549 op/s 2024-05-17T22:47:31.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.319+0000 7f1d1689e700 1 -- 172.21.15.12:0/3539409473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d1010b180 msgr2=0x7f1d10117540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:31.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.319+0000 7f1d1689e700 1 --2- 172.21.15.12:0/3539409473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d1010b180 0x7f1d10117540 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1d0800c340 tx=0x7f1d0800c650 comp rx=0 tx=0).stop 2024-05-17T22:47:31.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.320+0000 7f1d1689e700 1 -- 172.21.15.12:0/3539409473 shutdown_connections 2024-05-17T22:47:31.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.320+0000 7f1d1689e700 1 --2- 172.21.15.12:0/3539409473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d1010b180 0x7f1d10117540 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:31.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.320+0000 7f1d1689e700 1 --2- 172.21.15.12:0/3539409473 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d10110710 0x7f1d10110b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:31.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.320+0000 7f1d1689e700 1 --2- 172.21.15.12:0/3539409473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1d1010a830 0x7f1d1010ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:31.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.320+0000 7f1d1689e700 1 -- 172.21.15.12:0/3539409473 >> 172.21.15.12:0/3539409473 conn(0x7f1d1006f420 msgr2=0x7f1d1006f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:47:31.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.328+0000 7f1d1689e700 1 -- 172.21.15.12:0/3539409473 shutdown_connections 2024-05-17T22:47:31.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.328+0000 7f1d1689e700 1 -- 172.21.15.12:0/3539409473 wait complete. 2024-05-17T22:47:31.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.330+0000 7f1d1689e700 1 Processor -- start 2024-05-17T22:47:31.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.330+0000 7f1d1689e700 1 -- start start 2024-05-17T22:47:31.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.331+0000 7f1d1689e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1d1010a830 0x7f1d101a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:31.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.331+0000 7f1d1689e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d1010b180 0x7f1d101a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:31.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.331+0000 7f1d1689e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d10110710 0x7f1d101a7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:31.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.331+0000 7f1d1689e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1011a470 con 0x7f1d1010a830 2024-05-17T22:47:31.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.331+0000 7f1d0ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1d1010a830 0x7f1d101a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:31.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.331+0000 7f1d0f7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d1010b180 0x7f1d101a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:31.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.331+0000 7f1d0ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1d1010a830 0x7f1d101a2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36810/0 (socket says 172.21.15.12:36810) 2024-05-17T22:47:31.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.331+0000 7f1d0ffff700 1 -- 172.21.15.12:0/2448629871 learned_addr learned my addr 172.21.15.12:0/2448629871 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:47:31.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.331+0000 7f1d14e3b700 1 --2- 172.21.15.12:0/2448629871 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d10110710 0x7f1d101a7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:31.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.331+0000 7f1d1689e700 1 -- 172.21.15.12:0/2448629871 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1011a2f0 con 0x7f1d1010b180 2024-05-17T22:47:31.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.332+0000 7f1d1689e700 1 -- 172.21.15.12:0/2448629871 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1011a5f0 con 0x7f1d10110710 2024-05-17T22:47:31.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.335+0000 7f1d0ffff700 1 -- 172.21.15.12:0/2448629871 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d10110710 msgr2=0x7f1d101a7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:31.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.335+0000 7f1d0ffff700 1 --2- 172.21.15.12:0/2448629871 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d10110710 0x7f1d101a7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:31.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.335+0000 7f1d0ffff700 1 -- 172.21.15.12:0/2448629871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d1010b180 msgr2=0x7f1d101a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:31.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.335+0000 7f1d0ffff700 1 --2- 172.21.15.12:0/2448629871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d1010b180 0x7f1d101a3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:31.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.335+0000 7f1d0ffff700 1 -- 172.21.15.12:0/2448629871 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d101a7ea0 con 0x7f1d1010a830 2024-05-17T22:47:31.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.335+0000 7f1d0ffff700 1 --2- 172.21.15.12:0/2448629871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1d1010a830 0x7f1d101a2fd0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f1d0000c920 tx=0x7f1d0000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:47:31.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.341+0000 7f1d0d7fa700 1 -- 172.21.15.12:0/2448629871 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d00004a30 con 0x7f1d1010a830 2024-05-17T22:47:31.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.341+0000 7f1d0d7fa700 1 -- 172.21.15.12:0/2448629871 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d000054d0 con 0x7f1d1010a830 2024-05-17T22:47:31.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.341+0000 7f1d0d7fa700 1 -- 172.21.15.12:0/2448629871 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d00007c70 con 0x7f1d1010a830 2024-05-17T22:47:31.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.341+0000 7f1d1689e700 1 -- 172.21.15.12:0/2448629871 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d101a8190 con 0x7f1d1010a830 2024-05-17T22:47:31.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.341+0000 7f1d1689e700 1 -- 172.21.15.12:0/2448629871 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d10111510 con 0x7f1d1010a830 2024-05-17T22:47:31.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.344+0000 7f1d1689e700 1 -- 172.21.15.12:0/2448629871 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d1010be10 con 0x7f1d1010a830 2024-05-17T22:47:31.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.349+0000 7f1d0d7fa700 1 -- 172.21.15.12:0/2448629871 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1d00004c60 con 0x7f1d1010a830 2024-05-17T22:47:31.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.350+0000 7f1d0d7fa700 1 --2- 172.21.15.12:0/2448629871 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f1cf80b6ea0 0x7f1cf80b93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:31.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.350+0000 7f1d0d7fa700 1 -- 172.21.15.12:0/2448629871 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f1d000e23d0 con 0x7f1d1010a830 2024-05-17T22:47:31.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.357+0000 7f1d0f7fe700 1 --2- 172.21.15.12:0/2448629871 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f1cf80b6ea0 0x7f1cf80b93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:31.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.358+0000 7f1d0d7fa700 1 -- 172.21.15.12:0/2448629871 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1d00014030 con 0x7f1d1010a830 2024-05-17T22:47:31.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 490.0 s, 460.2 tps, lat 218.424 ms stddev 268.108 2024-05-17T22:47:31.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.373+0000 7f1d0f7fe700 1 --2- 172.21.15.12:0/2448629871 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f1cf80b6ea0 0x7f1cf80b93b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1d1006b2c0 tx=0x7f1d04009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:47:31.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.789+0000 7f1d1689e700 1 -- 172.21.15.12:0/2448629871 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d1002ea80 con 0x7f1d1010a830 2024-05-17T22:47:31.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.792+0000 7f1d0d7fa700 1 -- 172.21.15.12:0/2448629871 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11871 (secure 0 0 0) 0x7f1d000a9d00 con 0x7f1d1010a830 2024-05-17T22:47:31.795 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:47:31.795 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1407219319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1407219319},{"type":"v1","addr":"172.21.15.12:6835","nonce":1407219319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/630977886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":630977886},{"type":"v1","addr":"172.21.15.12:6837","nonce":630977886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3731914742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3731914742},{"type":"v1","addr":"172.21.15.12:6839","nonce":3731914742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/3642142501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3642142501},{"type":"v1","addr":"172.21.15.12:6841","nonce":3642142501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3554053059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3554053059},{"type":"v1","addr":"172.21.15.151:6833","nonce":3554053059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/4228325569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":4228325569},{"type":"v1","addr":"172.21.15.96:6833","nonce":4228325569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T22:34:42.642111+0000","modified":"2024-05-17T22:47:08.154490+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24461,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/542323754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":542323754},{"type":"v1","addr":"172.21.15.151:6835","nonce":542323754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1758442232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1758442232},{"type":"v1","addr":"172.21.15.96:6835","nonce":1758442232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3403324038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3403324038},{"type":"v1","addr":"172.21.15.151:6837","nonce":3403324038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.151:6839/2338353849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2338353849},{"type":"v1","addr":"172.21.15.151:6839","nonce":2338353849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.96:6837/4222574338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":4222574338},{"type":"v1","addr":"172.21.15.96:6837","nonce":4222574338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/2986295519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2986295519},{"type":"v1","addr":"172.21.15.96:6839","nonce":2986295519}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24457,24482,24461]},"id":1}]} 2024-05-17T22:47:31.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.796+0000 7f1d1689e700 1 -- 172.21.15.12:0/2448629871 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f1cf80b6ea0 msgr2=0x7f1cf80b93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:31.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.796+0000 7f1d1689e700 1 --2- 172.21.15.12:0/2448629871 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f1cf80b6ea0 0x7f1cf80b93b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1d1006b2c0 tx=0x7f1d04009190 comp rx=0 tx=0).stop 2024-05-17T22:47:31.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.796+0000 7f1d1689e700 1 -- 172.21.15.12:0/2448629871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1d1010a830 msgr2=0x7f1d101a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:31.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.796+0000 7f1d1689e700 1 --2- 172.21.15.12:0/2448629871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1d1010a830 0x7f1d101a2fd0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f1d0000c920 tx=0x7f1d0000cc30 comp rx=0 tx=0).stop 2024-05-17T22:47:31.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.796+0000 7f1d1689e700 1 -- 172.21.15.12:0/2448629871 shutdown_connections 2024-05-17T22:47:31.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.796+0000 7f1d1689e700 1 --2- 172.21.15.12:0/2448629871 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f1cf80b6ea0 0x7f1cf80b93b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:31.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.796+0000 7f1d1689e700 1 --2- 172.21.15.12:0/2448629871 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d10110710 0x7f1d101a7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:31.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.796+0000 7f1d1689e700 1 --2- 172.21.15.12:0/2448629871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1d1010a830 0x7f1d101a2fd0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:31.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.796+0000 7f1d1689e700 1 --2- 172.21.15.12:0/2448629871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d1010b180 0x7f1d101a3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:31.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.796+0000 7f1d1689e700 1 -- 172.21.15.12:0/2448629871 >> 172.21.15.12:0/2448629871 conn(0x7f1d1006f420 msgr2=0x7f1d10073be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:47:31.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.796+0000 7f1d1689e700 1 -- 172.21.15.12:0/2448629871 shutdown_connections 2024-05-17T22:47:31.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:31.796+0000 7f1d1689e700 1 -- 172.21.15.12:0/2448629871 wait complete. 2024-05-17T22:47:31.800 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 38 2024-05-17T22:47:32.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:31 smithi012 ceph-mon[31632]: pgmap v687: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.6 MiB/s wr, 511 op/s 2024-05-17T22:47:32.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:31 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/2448629871' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:47:32.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:31 smithi151 ceph-mon[36101]: pgmap v687: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.6 MiB/s wr, 511 op/s 2024-05-17T22:47:32.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:31 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/2448629871' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:47:32.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:31 smithi096 ceph-mon[36065]: pgmap v687: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.6 MiB/s wr, 511 op/s 2024-05-17T22:47:32.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:31 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/2448629871' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:47:33.618 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-17T22:47:34.023 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:47:34.150 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:33 smithi012 ceph-mon[31632]: pgmap v688: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.0 MiB/s wr, 528 op/s 2024-05-17T22:47:34.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:33 smithi151 ceph-mon[36101]: pgmap v688: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.0 MiB/s wr, 528 op/s 2024-05-17T22:47:34.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:33 smithi096 ceph-mon[36065]: pgmap v688: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.0 MiB/s wr, 528 op/s 2024-05-17T22:47:36.195 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:35 smithi012 ceph-mon[31632]: pgmap v689: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.5 MiB/s wr, 498 op/s 2024-05-17T22:47:36.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:35 smithi151 ceph-mon[36101]: pgmap v689: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.5 MiB/s wr, 498 op/s 2024-05-17T22:47:36.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:35 smithi096 ceph-mon[36065]: pgmap v689: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.5 MiB/s wr, 498 op/s 2024-05-17T22:47:36.370 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 495.0 s, 507.3 tps, lat 201.657 ms stddev 227.984 2024-05-17T22:47:36.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.382+0000 7f8bbd1dc700 1 -- 172.21.15.12:0/711782498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8bb8112200 msgr2=0x7f8bb810aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:36.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.382+0000 7f8bbd1dc700 1 --2- 172.21.15.12:0/711782498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8bb8112200 0x7f8bb810aa60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8009990 tx=0x7f8ba8009ca0 comp rx=0 tx=0).stop 2024-05-17T22:47:36.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.389+0000 7f8bbd1dc700 1 -- 172.21.15.12:0/711782498 shutdown_connections 2024-05-17T22:47:36.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.389+0000 7f8bbd1dc700 1 --2- 172.21.15.12:0/711782498 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bb811d010 0x7f8bb811f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:36.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.389+0000 7f8bbd1dc700 1 --2- 172.21.15.12:0/711782498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8bb8112200 0x7f8bb810aa60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:36.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.389+0000 7f8bbd1dc700 1 --2- 172.21.15.12:0/711782498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8bb810afa0 0x7f8bb810b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:36.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.389+0000 7f8bbd1dc700 1 -- 172.21.15.12:0/711782498 >> 172.21.15.12:0/711782498 conn(0x7f8bb806edf0 msgr2=0x7f8bb806f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:47:36.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.392+0000 7f8bbd1dc700 1 -- 172.21.15.12:0/711782498 shutdown_connections 2024-05-17T22:47:36.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.393+0000 7f8bbd1dc700 1 -- 172.21.15.12:0/711782498 wait complete. 2024-05-17T22:47:36.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.394+0000 7f8bbd1dc700 1 Processor -- start 2024-05-17T22:47:36.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.398+0000 7f8bbd1dc700 1 -- start start 2024-05-17T22:47:36.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.398+0000 7f8bbd1dc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8bb810afa0 0x7f8bb81a95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:36.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.398+0000 7f8bbd1dc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bb8112200 0x7f8bb81a9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:36.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.398+0000 7f8bbd1dc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8bb811d010 0x7f8bb81addf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:36.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.398+0000 7f8bbd1dc700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb8122be0 con 0x7f8bb8112200 2024-05-17T22:47:36.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.398+0000 7f8bbd1dc700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb8122a60 con 0x7f8bb810afa0 2024-05-17T22:47:36.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.398+0000 7f8bbd1dc700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb8122d60 con 0x7f8bb811d010 2024-05-17T22:47:36.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.399+0000 7f8bbc9db700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8bb811d010 0x7f8bb81addf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:36.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.399+0000 7f8bbc9db700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8bb811d010 0x7f8bb81addf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:54066/0 (socket says 172.21.15.12:54066) 2024-05-17T22:47:36.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.399+0000 7f8bbc9db700 1 -- 172.21.15.12:0/858984357 learned_addr learned my addr 172.21.15.12:0/858984357 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:47:36.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.399+0000 7f8bb7fff700 1 --2- 172.21.15.12:0/858984357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8bb810afa0 0x7f8bb81a95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:36.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.399+0000 7f8bbc9db700 1 -- 172.21.15.12:0/858984357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8bb810afa0 msgr2=0x7f8bb81a95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:36.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.399+0000 7f8bbc9db700 1 --2- 172.21.15.12:0/858984357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8bb810afa0 0x7f8bb81a95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:36.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.399+0000 7f8bbc9db700 1 -- 172.21.15.12:0/858984357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bb8112200 msgr2=0x7f8bb81a9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:36.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.399+0000 7f8bbc9db700 1 --2- 172.21.15.12:0/858984357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bb8112200 0x7f8bb81a9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:36.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.399+0000 7f8bbc9db700 1 -- 172.21.15.12:0/858984357 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bb81ae490 con 0x7f8bb811d010 2024-05-17T22:47:36.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.400+0000 7f8bbc9db700 1 --2- 172.21.15.12:0/858984357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8bb811d010 0x7f8bb81addf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8bb000b970 tx=0x7f8bb000bd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:47:36.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.400+0000 7f8bb57fa700 1 -- 172.21.15.12:0/858984357 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb0019070 con 0x7f8bb811d010 2024-05-17T22:47:36.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.400+0000 7f8bb57fa700 1 -- 172.21.15.12:0/858984357 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bb0014440 con 0x7f8bb811d010 2024-05-17T22:47:36.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.400+0000 7f8bb57fa700 1 -- 172.21.15.12:0/858984357 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb000ce60 con 0x7f8bb811d010 2024-05-17T22:47:36.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.400+0000 7f8bbd1dc700 1 -- 172.21.15.12:0/858984357 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bb81ae780 con 0x7f8bb811d010 2024-05-17T22:47:36.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.401+0000 7f8bbd1dc700 1 -- 172.21.15.12:0/858984357 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bb81127f0 con 0x7f8bb811d010 2024-05-17T22:47:36.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.408+0000 7f8bb57fa700 1 -- 172.21.15.12:0/858984357 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8bb001c070 con 0x7f8bb811d010 2024-05-17T22:47:36.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.409+0000 7f8bcce19b80 1 Processor -- start 2024-05-17T22:47:36.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.409+0000 7f8bcce19b80 1 -- start start 2024-05-17T22:47:36.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.409+0000 7f8bcce19b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a8dc379f0 0x558a8ddbafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:36.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.409+0000 7f8bcce19b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a8ddbb510 0x558a8ddb55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:36.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.409+0000 7f8bcce19b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a8ddb5b00 0x558a8ddb5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:36.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.409+0000 7f8bcce19b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x558a8dce40d0 con 0x558a8ddbb510 2024-05-17T22:47:36.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.409+0000 7f8bcce19b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558a8dcd3c70 con 0x558a8ddb5b00 2024-05-17T22:47:36.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.409+0000 7f8bcce19b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558a8dcd7cb0 con 0x558a8dc379f0 2024-05-17T22:47:36.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.410+0000 7f8bb77fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a8dc379f0 0x558a8ddbafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:36.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.410+0000 7f8bb77fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a8ddb5b00 0x558a8ddb5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:36.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.410+0000 7f8bb77fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a8ddb5b00 0x558a8ddb5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:47750/0 (socket says 172.21.15.12:47750) 2024-05-17T22:47:36.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.410+0000 7f8bb77fe700 1 -- 172.21.15.12:0/3680495382 learned_addr learned my addr 172.21.15.12:0/3680495382 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:47:36.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.410+0000 7f8bb77fe700 1 -- 172.21.15.12:0/3680495382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a8dc379f0 msgr2=0x558a8ddbafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:47:36.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.410+0000 7f8bb77fe700 1 --2- 172.21.15.12:0/3680495382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a8dc379f0 0x558a8ddbafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:36.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.410+0000 7f8bb77fe700 1 -- 172.21.15.12:0/3680495382 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a8ddbb510 msgr2=0x558a8ddb55c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T22:47:36.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.410+0000 7f8bb77fe700 1 --2- 172.21.15.12:0/3680495382 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a8ddbb510 0x558a8ddb55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:36.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.410+0000 7f8bb77fe700 1 -- 172.21.15.12:0/3680495382 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a8dc6d5c0 con 0x558a8ddb5b00 2024-05-17T22:47:36.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.410+0000 7f8bb57fa700 1 --2- 172.21.15.12:0/858984357 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8ba00b2a00 0x7f8ba00b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:36.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.410+0000 7f8bb77fe700 1 --2- 172.21.15.12:0/3680495382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a8ddb5b00 0x558a8ddb5f10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8bac015430 tx=0x7f8bac001140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccd8cc2a2727e4c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:47:36.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.410+0000 7f8bb57fa700 1 -- 172.21.15.12:0/858984357 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f8bb00e1960 con 0x7f8bb811d010 2024-05-17T22:47:36.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.411+0000 7f8bb7fff700 1 --2- 172.21.15.12:0/858984357 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8ba00b2a00 0x7f8ba00b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:36.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.412+0000 7f8ba67fc700 1 -- 172.21.15.12:0/3680495382 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bac012040 con 0x558a8ddb5b00 2024-05-17T22:47:36.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.412+0000 7f8bb7fff700 1 --2- 172.21.15.12:0/858984357 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8ba00b2a00 0x7f8ba00b4f10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x558a8ddb1930 tx=0x7f8ba801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:47:36.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.412+0000 7f8ba67fc700 1 -- 172.21.15.12:0/3680495382 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bac015d10 con 0x558a8ddb5b00 2024-05-17T22:47:36.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.413+0000 7f8ba67fc700 1 -- 172.21.15.12:0/3680495382 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bac004800 con 0x558a8ddb5b00 2024-05-17T22:47:36.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.413+0000 7f8bcce19b80 1 -- 172.21.15.12:0/3680495382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a8ddb5b00 msgr2=0x558a8ddb5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:36.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.413+0000 7f8bcce19b80 1 --2- 172.21.15.12:0/3680495382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a8ddb5b00 0x558a8ddb5f10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8bac015430 tx=0x7f8bac001140 comp rx=0 tx=0).stop 2024-05-17T22:47:36.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.413+0000 7f8bcce19b80 1 -- 172.21.15.12:0/3680495382 shutdown_connections 2024-05-17T22:47:36.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.413+0000 7f8bcce19b80 1 --2- 172.21.15.12:0/3680495382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a8ddb5b00 0x558a8ddb5f10 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:36.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.413+0000 7f8bcce19b80 1 --2- 172.21.15.12:0/3680495382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a8dc379f0 0x558a8ddbafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:36.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.413+0000 7f8bcce19b80 1 --2- 172.21.15.12:0/3680495382 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a8ddbb510 0x558a8ddb55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:36.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.413+0000 7f8bcce19b80 1 -- 172.21.15.12:0/3680495382 >> 172.21.15.12:0/3680495382 conn(0x558a8dc977d0 msgr2=0x558a8ddb01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:47:36.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.413+0000 7f8bcce19b80 1 -- 172.21.15.12:0/3680495382 shutdown_connections 2024-05-17T22:47:36.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.413+0000 7f8bcce19b80 1 -- 172.21.15.12:0/3680495382 wait complete. 2024-05-17T22:47:36.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.414+0000 7f8bcce19b80 1 -- start start 2024-05-17T22:47:36.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.414+0000 7f8bcce19b80 1 Processor -- start 2024-05-17T22:47:36.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.415+0000 7f8bcce19b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a8dc379f0 0x558a8ddbece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:36.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.415+0000 7f8bcce19b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a8ddb5b00 0x558a8ddbf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:36.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.415+0000 7f8bcce19b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a8ddbb510 0x558a8ddcb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:36.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.415+0000 7f8bcce19b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x558a8dbdb150 con 0x558a8ddbb510 2024-05-17T22:47:36.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.415+0000 7f8bcce19b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558a8dc75940 con 0x558a8dc379f0 2024-05-17T22:47:36.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.415+0000 7f8bcce19b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558a8ddc1f00 con 0x558a8ddb5b00 2024-05-17T22:47:36.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.415+0000 7f8bcce19b80 10 client.0 _ll_register_callbacks cb 0x558a8dc6ab50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:47:36.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.415+0000 7f8bbc9db700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a8ddbb510 0x558a8ddcb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:36.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.415+0000 7f8bbc9db700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a8ddbb510 0x558a8ddcb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36842/0 (socket says 172.21.15.12:36842) 2024-05-17T22:47:36.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.415+0000 7f8bbc9db700 1 -- 172.21.15.12:0/2677946016 learned_addr learned my addr 172.21.15.12:0/2677946016 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:47:36.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.416+0000 7f8bbc9db700 1 -- 172.21.15.12:0/2677946016 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a8ddb5b00 msgr2=0x558a8ddbf220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:47:36.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.416+0000 7f8bbc9db700 1 --2- 172.21.15.12:0/2677946016 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a8ddb5b00 0x558a8ddbf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:36.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.416+0000 7f8bbc9db700 1 -- 172.21.15.12:0/2677946016 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a8dc379f0 msgr2=0x558a8ddbece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:47:36.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.416+0000 7f8bbc9db700 1 --2- 172.21.15.12:0/2677946016 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a8dc379f0 0x558a8ddbece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:36.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.416+0000 7f8bbc9db700 1 -- 172.21.15.12:0/2677946016 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a8dc24480 con 0x558a8ddbb510 2024-05-17T22:47:36.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.416+0000 7f8bbc9db700 1 --2- 172.21.15.12:0/2677946016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a8ddbb510 0x558a8ddcb280 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8bb00a4c10 tx=0x7f8bb0014b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab1a15a1ce9ea1e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:47:36.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.418+0000 7f8b9b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:47:36.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.418+0000 7f8b9b7fe700 1 -- 172.21.15.12:0/2677946016 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb00a8b90 con 0x558a8ddbb510 2024-05-17T22:47:36.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.418+0000 7f8bcce19b80 1 -- 172.21.15.12:0/2677946016 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bac015d10 con 0x558a8ddbb510 2024-05-17T22:47:36.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.418+0000 7f8b9b7fe700 1 -- 172.21.15.12:0/2677946016 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f8bb00b1770 con 0x558a8ddbb510 2024-05-17T22:47:36.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.419+0000 7f8b9b7fe700 1 -- 172.21.15.12:0/2677946016 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f8bb00acf40 con 0x558a8ddbb510 2024-05-17T22:47:36.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.419+0000 7f8bcce19b80 10 client.16032 fetch_fsmap learned FSMap version 38 2024-05-17T22:47:36.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.419+0000 7f8bcce19b80 1 -- 172.21.15.12:0/2677946016 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x558a8ddccb80 con 0x558a8ddbb510 2024-05-17T22:47:36.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.420+0000 7f8b9b7fe700 1 -- 172.21.15.12:0/2677946016 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bac015d10 con 0x558a8ddbb510 2024-05-17T22:47:36.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.420+0000 7f8b9b7fe700 1 -- 172.21.15.12:0/2677946016 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 38) ==== 8350+0+0 (secure 0 0 0) 0x7f8bb00a4500 con 0x558a8ddbb510 2024-05-17T22:47:36.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.421+0000 7f8bcce19b80 10 client.16032 fetch_fsmap finished waiting for FSMap version 38 2024-05-17T22:47:36.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.421+0000 7f8bcce19b80 10 client.16032 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-17T22:47:36.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.421+0000 7f8bcce19b80 20 client.16032 populate_metadata read hostname 'smithi012' 2024-05-17T22:47:36.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.421+0000 7f8bcce19b80 1 --2- 172.21.15.12:0/2677946016 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x558a8ddcf660 0x558a8ddd1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:36.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.421+0000 7f8bcce19b80 4 client.16032 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:47:36.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.421+0000 7f8bcce19b80 1 -- 172.21.15.12:0/2677946016 --> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a8dce6020 con 0x558a8ddcf660 2024-05-17T22:47:36.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.422+0000 7f8bb7fff700 1 --2- 172.21.15.12:0/2677946016 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x558a8ddcf660 0x558a8ddd1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:36.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.426+0000 7f8bb7fff700 1 --2- 172.21.15.12:0/2677946016 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x558a8ddcf660 0x558a8ddd1b70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a8f66de7f17c3791 server_cookie=1ec709b8bf82dbb3 in_seq=0 out_seq=0 2024-05-17T22:47:36.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.426+0000 7f8b9b7fe700 10 client.16032 ms_handle_connect on v2:172.21.15.151:6838/2338353849 2024-05-17T22:47:36.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.427+0000 7f8b9b7fe700 1 -- 172.21.15.12:0/2677946016 <== mds.2 v2:172.21.15.151:6838/2338353849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558a8dce6020 con 0x558a8ddcf660 2024-05-17T22:47:36.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.427+0000 7f8b9b7fe700 10 client.16032 handle_command_reply: tid=0 2024-05-17T22:47:36.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.427+0000 7f8bcce19b80 1 client.16032 shutdown 2024-05-17T22:47:36.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.429+0000 7f8bcce19b80 1 -- 172.21.15.12:0/2677946016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a8ddbb510 msgr2=0x558a8ddcb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:36.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.429+0000 7f8bcce19b80 1 --2- 172.21.15.12:0/2677946016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a8ddbb510 0x558a8ddcb280 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8bb00a4c10 tx=0x7f8bb0014b40 comp rx=0 tx=0).stop 2024-05-17T22:47:36.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.429+0000 7f8bcce19b80 1 -- 172.21.15.12:0/2677946016 shutdown_connections 2024-05-17T22:47:36.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.429+0000 7f8bcce19b80 1 --2- 172.21.15.12:0/2677946016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a8ddbb510 0x558a8ddcb280 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:36.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.429+0000 7f8bcce19b80 1 --2- 172.21.15.12:0/2677946016 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a8dc379f0 0x558a8ddbece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:36.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.429+0000 7f8bcce19b80 1 --2- 172.21.15.12:0/2677946016 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x558a8ddcf660 0x558a8ddd1b70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:36.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.429+0000 7f8bcce19b80 1 --2- 172.21.15.12:0/2677946016 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a8ddb5b00 0x558a8ddbf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:36.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.429+0000 7f8bcce19b80 1 -- 172.21.15.12:0/2677946016 >> 172.21.15.12:0/2677946016 conn(0x558a8dc977d0 msgr2=0x558a8ddb0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:47:36.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.429+0000 7f8b9b7fe700 0 client.16032 ms_handle_reset on v2:172.21.15.151:6838/2338353849 2024-05-17T22:47:36.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.429+0000 7f8bcce19b80 1 -- 172.21.15.12:0/2677946016 shutdown_connections 2024-05-17T22:47:36.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.430+0000 7f8bcce19b80 1 -- 172.21.15.12:0/2677946016 wait complete. 2024-05-17T22:47:36.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.430+0000 7f8bcce19b80 20 client.16032 trim_cache size 0 max 16384 2024-05-17T22:47:36.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.453+0000 7f8bcce19b80 1 Processor -- start 2024-05-17T22:47:36.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.453+0000 7f8bcce19b80 1 -- start start 2024-05-17T22:47:36.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.454+0000 7f8bcce19b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a8dc379f0 0x558a8ddb55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:36.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.454+0000 7f8bcce19b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a8ddb5b00 0x558a8ddbf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:36.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.454+0000 7f8bcce19b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a8ddbb510 0x558a8ddd1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:36.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.454+0000 7f8bcce19b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb81c9af0 con 0x558a8ddbb510 2024-05-17T22:47:36.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.454+0000 7f8bcce19b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb81c7380 con 0x558a8ddb5b00 2024-05-17T22:47:36.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.454+0000 7f8bcce19b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb8119110 con 0x558a8dc379f0 2024-05-17T22:47:36.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.454+0000 7f8bb77fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a8ddb5b00 0x558a8ddbf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:36.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.455+0000 7f8bb77fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a8ddb5b00 0x558a8ddbf250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:47770/0 (socket says 172.21.15.12:47770) 2024-05-17T22:47:36.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.455+0000 7f8bb77fe700 1 -- 172.21.15.12:0/3617505758 learned_addr learned my addr 172.21.15.12:0/3617505758 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:47:36.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.455+0000 7f8bb7fff700 1 --2- 172.21.15.12:0/3617505758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a8dc379f0 0x558a8ddb55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:36.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.455+0000 7f8bbc9db700 1 --2- 172.21.15.12:0/3617505758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a8ddbb510 0x558a8ddd1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:36.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.456+0000 7f8bb77fe700 1 -- 172.21.15.12:0/3617505758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a8dc379f0 msgr2=0x558a8ddb55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:47:36.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.456+0000 7f8bb77fe700 1 --2- 172.21.15.12:0/3617505758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a8dc379f0 0x558a8ddb55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:36.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.456+0000 7f8bb77fe700 1 -- 172.21.15.12:0/3617505758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a8ddbb510 msgr2=0x558a8ddd1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:47:36.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.456+0000 7f8bb77fe700 1 --2- 172.21.15.12:0/3617505758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a8ddbb510 0x558a8ddd1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:36.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.456+0000 7f8bb77fe700 1 -- 172.21.15.12:0/3617505758 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a8ddacc20 con 0x558a8ddb5b00 2024-05-17T22:47:36.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.458+0000 7f8bb77fe700 1 --2- 172.21.15.12:0/3617505758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a8ddb5b00 0x558a8ddbf250 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8bac001da0 tx=0x7f8bac000ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66fa309ccb33d49c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:47:36.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.460+0000 7f8ba5ffb700 1 -- 172.21.15.12:0/3617505758 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bac015d10 con 0x558a8ddb5b00 2024-05-17T22:47:36.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.460+0000 7f8ba5ffb700 1 -- 172.21.15.12:0/3617505758 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bac007120 con 0x558a8ddb5b00 2024-05-17T22:47:36.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.462+0000 7f8bcce19b80 1 -- 172.21.15.12:0/3617505758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a8ddb5b00 msgr2=0x558a8ddbf250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:36.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.462+0000 7f8bcce19b80 1 --2- 172.21.15.12:0/3617505758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a8ddb5b00 0x558a8ddbf250 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8bac001da0 tx=0x7f8bac000ee0 comp rx=0 tx=0).stop 2024-05-17T22:47:36.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.462+0000 7f8bcce19b80 1 -- 172.21.15.12:0/3617505758 shutdown_connections 2024-05-17T22:47:36.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.462+0000 7f8bcce19b80 1 --2- 172.21.15.12:0/3617505758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a8ddbb510 0x558a8ddd1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:36.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.462+0000 7f8bcce19b80 1 --2- 172.21.15.12:0/3617505758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a8dc379f0 0x558a8ddb55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:36.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.462+0000 7f8bcce19b80 1 --2- 172.21.15.12:0/3617505758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a8ddb5b00 0x558a8ddbf250 secure :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8bac001da0 tx=0x7f8bac000ee0 comp rx=0 tx=0).stop 2024-05-17T22:47:36.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.462+0000 7f8bcce19b80 1 -- 172.21.15.12:0/3617505758 >> 172.21.15.12:0/3617505758 conn(0x558a8dc977d0 msgr2=0x558a8ddb8930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:47:36.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.469+0000 7f8bcce19b80 1 -- 172.21.15.12:0/3617505758 shutdown_connections 2024-05-17T22:47:36.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.470+0000 7f8bcce19b80 1 -- 172.21.15.12:0/3617505758 wait complete. 2024-05-17T22:47:36.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.471+0000 7f8bcce19b80 1 -- start start 2024-05-17T22:47:36.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.471+0000 7f8bcce19b80 1 Processor -- start 2024-05-17T22:47:36.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.472+0000 7f8bcce19b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a8dc379f0 0x558a8ddc76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:36.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.472+0000 7f8bcce19b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a8ddbb510 0x558a8ddc9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:36.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.472+0000 7f8bcce19b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a8ddca390 0x558a8ddca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:36.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.472+0000 7f8bcce19b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb8119780 con 0x558a8ddca390 2024-05-17T22:47:36.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.472+0000 7f8bcce19b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb811abf0 con 0x558a8ddbb510 2024-05-17T22:47:36.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.472+0000 7f8bcce19b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb8119e50 con 0x558a8dc379f0 2024-05-17T22:47:36.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.472+0000 7f8bcce19b80 10 client.0 _ll_register_callbacks cb 0x558a8dc6ab50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:47:36.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.473+0000 7f8bb77fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a8ddbb510 0x558a8ddc9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:36.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.473+0000 7f8bbc9db700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a8ddca390 0x558a8ddca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:36.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.473+0000 7f8bbc9db700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a8ddca390 0x558a8ddca7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36854/0 (socket says 172.21.15.12:36854) 2024-05-17T22:47:36.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.473+0000 7f8bbc9db700 1 -- 172.21.15.12:0/1083017576 learned_addr learned my addr 172.21.15.12:0/1083017576 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:47:36.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.473+0000 7f8bb7fff700 1 --2- 172.21.15.12:0/1083017576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a8dc379f0 0x558a8ddc76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:36.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.475+0000 7f8bbc9db700 1 -- 172.21.15.12:0/1083017576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a8dc379f0 msgr2=0x558a8ddc76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:47:36.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.475+0000 7f8bbc9db700 1 --2- 172.21.15.12:0/1083017576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a8dc379f0 0x558a8ddc76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:36.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.475+0000 7f8bbc9db700 1 -- 172.21.15.12:0/1083017576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a8ddbb510 msgr2=0x558a8ddc9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:47:36.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.475+0000 7f8bbc9db700 1 --2- 172.21.15.12:0/1083017576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a8ddbb510 0x558a8ddc9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:36.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.475+0000 7f8bbc9db700 1 -- 172.21.15.12:0/1083017576 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a8ddcaea0 con 0x558a8ddca390 2024-05-17T22:47:36.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.481+0000 7f8bbc9db700 1 --2- 172.21.15.12:0/1083017576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a8ddca390 0x558a8ddca7a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f8bb00a5e00 tx=0x7f8bb000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fae4f02dad1c204 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:47:36.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.481+0000 7f8b9b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:47:36.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.484+0000 7f8b9b7fe700 1 -- 172.21.15.12:0/1083017576 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb00a8b90 con 0x558a8ddca390 2024-05-17T22:47:36.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.484+0000 7f8bcce19b80 1 -- 172.21.15.12:0/1083017576 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bac007120 con 0x558a8ddca390 2024-05-17T22:47:36.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.484+0000 7f8b9b7fe700 1 -- 172.21.15.12:0/1083017576 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f8bb000cbc0 con 0x558a8ddca390 2024-05-17T22:47:36.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.489+0000 7f8b9b7fe700 1 -- 172.21.15.12:0/1083017576 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f8bb00b19f0 con 0x558a8ddca390 2024-05-17T22:47:36.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.489+0000 7f8bcce19b80 10 client.16038 fetch_fsmap learned FSMap version 38 2024-05-17T22:47:36.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.489+0000 7f8bcce19b80 1 -- 172.21.15.12:0/1083017576 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x558a8de39310 con 0x558a8ddca390 2024-05-17T22:47:36.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.491+0000 7f8b9b7fe700 1 -- 172.21.15.12:0/1083017576 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bac007120 con 0x558a8ddca390 2024-05-17T22:47:36.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.492+0000 7f8b9b7fe700 1 -- 172.21.15.12:0/1083017576 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 38) ==== 8350+0+0 (secure 0 0 0) 0x7f8bb00cb0d0 con 0x558a8ddca390 2024-05-17T22:47:36.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.492+0000 7f8bcce19b80 10 client.16038 fetch_fsmap finished waiting for FSMap version 38 2024-05-17T22:47:36.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.492+0000 7f8bcce19b80 10 client.16038 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-17T22:47:36.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.492+0000 7f8bcce19b80 20 client.16038 populate_metadata read hostname 'smithi012' 2024-05-17T22:47:36.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.492+0000 7f8bcce19b80 1 --2- 172.21.15.12:0/1083017576 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x558a8de39830 0x558a8de3bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:36.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.492+0000 7f8bcce19b80 4 client.16038 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:47:36.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.492+0000 7f8bcce19b80 1 -- 172.21.15.12:0/1083017576 --> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558a8dcdf800 con 0x558a8de39830 2024-05-17T22:47:36.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.493+0000 7f8bb7fff700 1 --2- 172.21.15.12:0/1083017576 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x558a8de39830 0x558a8de3bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:36.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.501+0000 7f8bb7fff700 1 --2- 172.21.15.12:0/1083017576 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x558a8de39830 0x558a8de3bc50 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=22a7ccc3e249e1db server_cookie=6c1bd1a4fc53bfd5 in_seq=0 out_seq=0 2024-05-17T22:47:36.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.501+0000 7f8b9b7fe700 10 client.16038 ms_handle_connect on v2:172.21.15.151:6838/2338353849 2024-05-17T22:47:36.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.505+0000 7f8b9b7fe700 1 -- 172.21.15.12:0/1083017576 <== mds.2 v2:172.21.15.151:6838/2338353849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558a8dcdf800 con 0x558a8de39830 2024-05-17T22:47:36.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.505+0000 7f8b9b7fe700 10 client.16038 handle_command_reply: tid=0 2024-05-17T22:47:36.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.506+0000 7f8bcce19b80 1 client.16038 shutdown 2024-05-17T22:47:36.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.509+0000 7f8bcce19b80 1 -- 172.21.15.12:0/1083017576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a8ddca390 msgr2=0x558a8ddca7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:36.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.509+0000 7f8bcce19b80 1 --2- 172.21.15.12:0/1083017576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a8ddca390 0x558a8ddca7a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f8bb00a5e00 tx=0x7f8bb000c3a0 comp rx=0 tx=0).stop 2024-05-17T22:47:36.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.510+0000 7f8bcce19b80 1 -- 172.21.15.12:0/1083017576 shutdown_connections 2024-05-17T22:47:36.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.510+0000 7f8bcce19b80 1 --2- 172.21.15.12:0/1083017576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a8ddca390 0x558a8ddca7a0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:36.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.510+0000 7f8bcce19b80 1 --2- 172.21.15.12:0/1083017576 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x558a8de39830 0x558a8de3bc50 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:36.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.510+0000 7f8bcce19b80 1 --2- 172.21.15.12:0/1083017576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a8dc379f0 0x558a8ddc76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:36.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.510+0000 7f8bcce19b80 1 --2- 172.21.15.12:0/1083017576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a8ddbb510 0x558a8ddc9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:36.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.510+0000 7f8bcce19b80 1 -- 172.21.15.12:0/1083017576 >> 172.21.15.12:0/1083017576 conn(0x558a8dc977d0 msgr2=0x558a8ddaa110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:47:36.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.510+0000 7f8b9b7fe700 0 client.16038 ms_handle_reset on v2:172.21.15.151:6838/2338353849 2024-05-17T22:47:36.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.513+0000 7f8bcce19b80 1 -- 172.21.15.12:0/1083017576 shutdown_connections 2024-05-17T22:47:36.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.515+0000 7f8bcce19b80 1 -- 172.21.15.12:0/1083017576 wait complete. 2024-05-17T22:47:36.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.515+0000 7f8bcce19b80 20 client.16038 trim_cache size 0 max 16384 2024-05-17T22:47:36.518 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:47:36.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.517+0000 7f8bbd1dc700 1 -- 172.21.15.12:0/858984357 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8ba00b2a00 msgr2=0x7f8ba00b4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:36.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.517+0000 7f8bbd1dc700 1 --2- 172.21.15.12:0/858984357 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8ba00b2a00 0x7f8ba00b4f10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x558a8ddb1930 tx=0x7f8ba801a040 comp rx=0 tx=0).stop 2024-05-17T22:47:36.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.517+0000 7f8bbd1dc700 1 -- 172.21.15.12:0/858984357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8bb811d010 msgr2=0x7f8bb81addf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:36.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.517+0000 7f8bbd1dc700 1 --2- 172.21.15.12:0/858984357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8bb811d010 0x7f8bb81addf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8bb000b970 tx=0x7f8bb000bd30 comp rx=0 tx=0).stop 2024-05-17T22:47:36.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.519+0000 7f8bbd1dc700 1 -- 172.21.15.12:0/858984357 shutdown_connections 2024-05-17T22:47:36.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.519+0000 7f8bbd1dc700 1 --2- 172.21.15.12:0/858984357 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8ba00b2a00 0x7f8ba00b4f10 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:36.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.519+0000 7f8bbd1dc700 1 --2- 172.21.15.12:0/858984357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8bb811d010 0x7f8bb81addf0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:36.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.519+0000 7f8bbd1dc700 1 --2- 172.21.15.12:0/858984357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8bb810afa0 0x7f8bb81a95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:36.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.520+0000 7f8bbd1dc700 1 --2- 172.21.15.12:0/858984357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bb8112200 0x7f8bb81a9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:36.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.520+0000 7f8bbd1dc700 1 -- 172.21.15.12:0/858984357 >> 172.21.15.12:0/858984357 conn(0x7f8bb806edf0 msgr2=0x7f8bb811b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:47:36.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.522+0000 7f8bbd1dc700 1 -- 172.21.15.12:0/858984357 shutdown_connections 2024-05-17T22:47:36.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:36.522+0000 7f8bbd1dc700 1 -- 172.21.15.12:0/858984357 wait complete. 2024-05-17T22:47:37.358 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-17T22:47:37.853 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:47:38.176 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:37 smithi012 ceph-mon[31632]: pgmap v690: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.5 MiB/s wr, 503 op/s 2024-05-17T22:47:38.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:37 smithi151 ceph-mon[36101]: pgmap v690: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.5 MiB/s wr, 503 op/s 2024-05-17T22:47:38.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:37 smithi096 ceph-mon[36065]: pgmap v690: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.5 MiB/s wr, 503 op/s 2024-05-17T22:47:40.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.007+0000 7f5d7722b700 1 -- 172.21.15.12:0/848385275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d70104e70 msgr2=0x7f5d70105280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:40.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.007+0000 7f5d7722b700 1 --2- 172.21.15.12:0/848385275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d70104e70 0x7f5d70105280 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f5d64009990 tx=0x7f5d64009ca0 comp rx=0 tx=0).stop 2024-05-17T22:47:40.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.008+0000 7f5d7722b700 1 -- 172.21.15.12:0/848385275 shutdown_connections 2024-05-17T22:47:40.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.008+0000 7f5d7722b700 1 --2- 172.21.15.12:0/848385275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d7010e2a0 0x7f5d70110740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:40.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.008+0000 7f5d7722b700 1 --2- 172.21.15.12:0/848385275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d70104e70 0x7f5d70105280 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:40.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.008+0000 7f5d7722b700 1 --2- 172.21.15.12:0/848385275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d701057c0 0x7f5d7010dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:40.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.008+0000 7f5d7722b700 1 -- 172.21.15.12:0/848385275 >> 172.21.15.12:0/848385275 conn(0x7f5d70079ae0 msgr2=0x7f5d70079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:47:40.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.008+0000 7f5d7722b700 1 -- 172.21.15.12:0/848385275 shutdown_connections 2024-05-17T22:47:40.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.008+0000 7f5d7722b700 1 -- 172.21.15.12:0/848385275 wait complete. 2024-05-17T22:47:40.012 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:39 smithi012 ceph-mon[31632]: pgmap v691: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.4 MiB/s wr, 500 op/s 2024-05-17T22:47:40.013 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:39 smithi012 ceph-mon[31632]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:47:40.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.009+0000 7f5d7722b700 1 Processor -- start 2024-05-17T22:47:40.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.009+0000 7f5d7722b700 1 -- start start 2024-05-17T22:47:40.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.010+0000 7f5d7722b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d70104e70 0x7f5d70104700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:40.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.010+0000 7f5d7722b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d701057c0 0x7f5d70102b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:40.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.010+0000 7f5d7722b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d7010e2a0 0x7f5d70103140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:40.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.010+0000 7f5d7722b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5d70113db0 con 0x7f5d701057c0 2024-05-17T22:47:40.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.010+0000 7f5d7722b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5d70113c30 con 0x7f5d7010e2a0 2024-05-17T22:47:40.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.010+0000 7f5d7722b700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5d70113f30 con 0x7f5d70104e70 2024-05-17T22:47:40.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.010+0000 7f5d74fc7700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d70104e70 0x7f5d70104700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:40.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.010+0000 7f5d74fc7700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d70104e70 0x7f5d70104700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:41084/0 (socket says 172.21.15.12:41084) 2024-05-17T22:47:40.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.010+0000 7f5d74fc7700 1 -- 172.21.15.12:0/3233774620 learned_addr learned my addr 172.21.15.12:0/3233774620 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:47:40.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.010+0000 7f5d757c8700 1 --2- 172.21.15.12:0/3233774620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d7010e2a0 0x7f5d70103140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:40.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.011+0000 7f5d6ffff700 1 --2- 172.21.15.12:0/3233774620 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d701057c0 0x7f5d70102b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:40.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.011+0000 7f5d74fc7700 1 -- 172.21.15.12:0/3233774620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d7010e2a0 msgr2=0x7f5d70103140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:40.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.011+0000 7f5d74fc7700 1 --2- 172.21.15.12:0/3233774620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d7010e2a0 0x7f5d70103140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:40.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.011+0000 7f5d74fc7700 1 -- 172.21.15.12:0/3233774620 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d701057c0 msgr2=0x7f5d70102b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:40.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.011+0000 7f5d74fc7700 1 --2- 172.21.15.12:0/3233774620 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d701057c0 0x7f5d70102b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:40.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.011+0000 7f5d74fc7700 1 -- 172.21.15.12:0/3233774620 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d70103970 con 0x7f5d70104e70 2024-05-17T22:47:40.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.011+0000 7f5d74fc7700 1 --2- 172.21.15.12:0/3233774620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d70104e70 0x7f5d70104700 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5d640047b0 tx=0x7f5d64005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:47:40.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.012+0000 7f5d6dffb700 1 -- 172.21.15.12:0/3233774620 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d64026070 con 0x7f5d70104e70 2024-05-17T22:47:40.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.012+0000 7f5d6dffb700 1 -- 172.21.15.12:0/3233774620 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d64021410 con 0x7f5d70104e70 2024-05-17T22:47:40.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.012+0000 7f5d6dffb700 1 -- 172.21.15.12:0/3233774620 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d64017650 con 0x7f5d70104e70 2024-05-17T22:47:40.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.012+0000 7f5d7722b700 1 -- 172.21.15.12:0/3233774620 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d701ad1c0 con 0x7f5d70104e70 2024-05-17T22:47:40.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.012+0000 7f5d7722b700 1 -- 172.21.15.12:0/3233774620 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d701ad600 con 0x7f5d70104e70 2024-05-17T22:47:40.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.015+0000 7f5d6dffb700 1 -- 172.21.15.12:0/3233774620 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5d64029070 con 0x7f5d70104e70 2024-05-17T22:47:40.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.016+0000 7f5d85c06b80 1 Processor -- start 2024-05-17T22:47:40.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.016+0000 7f5d6dffb700 1 --2- 172.21.15.12:0/3233774620 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5d600b2980 0x7f5d600b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:40.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.016+0000 7f5d85c06b80 1 -- start start 2024-05-17T22:47:40.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.016+0000 7f5d6dffb700 1 -- 172.21.15.12:0/3233774620 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f5d640e33e0 con 0x7f5d70104e70 2024-05-17T22:47:40.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.017+0000 7f5d6ffff700 1 --2- 172.21.15.12:0/3233774620 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5d600b2980 0x7f5d600b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:40.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.017+0000 7f5d85c06b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586fe01a9f0 0x5586fe19e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:40.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.017+0000 7f5d85c06b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586fe19eaa0 0x5586fe198c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:40.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.017+0000 7f5d85c06b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5586fe199170 0x5586fe199580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:40.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.017+0000 7f5d85c06b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5586fe0c7290 con 0x5586fe19eaa0 2024-05-17T22:47:40.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.017+0000 7f5d85c06b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5586fe0b6fd0 con 0x5586fe01a9f0 2024-05-17T22:47:40.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.017+0000 7f5d85c06b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5586fe0bae70 con 0x5586fe199170 2024-05-17T22:47:40.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.017+0000 7f5d757c8700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5586fe199170 0x5586fe199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:40.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.017+0000 7f5d757c8700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5586fe199170 0x5586fe199580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:41098/0 (socket says 172.21.15.12:41098) 2024-05-17T22:47:40.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.017+0000 7f5d757c8700 1 -- 172.21.15.12:0/152312342 learned_addr learned my addr 172.21.15.12:0/152312342 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:47:40.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.017+0000 7f5d74fc7700 1 --2- 172.21.15.12:0/152312342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586fe01a9f0 0x5586fe19e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:40.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.017+0000 7f5d6ffff700 1 --2- 172.21.15.12:0/152312342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586fe19eaa0 0x5586fe198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:40.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.017+0000 7f5d757c8700 1 -- 172.21.15.12:0/152312342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586fe01a9f0 msgr2=0x5586fe19e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:47:40.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.017+0000 7f5d757c8700 1 --2- 172.21.15.12:0/152312342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586fe01a9f0 0x5586fe19e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:40.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.017+0000 7f5d757c8700 1 -- 172.21.15.12:0/152312342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586fe19eaa0 msgr2=0x5586fe198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:47:40.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.017+0000 7f5d757c8700 1 --2- 172.21.15.12:0/152312342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586fe19eaa0 0x5586fe198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:40.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.017+0000 7f5d757c8700 1 -- 172.21.15.12:0/152312342 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586fe060590 con 0x5586fe199170 2024-05-17T22:47:40.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.017+0000 7f5d6ffff700 1 --2- 172.21.15.12:0/152312342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586fe19eaa0 0x5586fe198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:47:40.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.018+0000 7f5d6ffff700 1 --2- 172.21.15.12:0/3233774620 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5d600b2980 0x7f5d600b4e90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x5586fe195320 tx=0x7f5d5c007760 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:47:40.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.018+0000 7f5d757c8700 1 --2- 172.21.15.12:0/152312342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5586fe199170 0x5586fe199580 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5d68011680 tx=0x7f5d68009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=412230c7d2d97993 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:47:40.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.018+0000 7f5d5affd700 1 -- 172.21.15.12:0/152312342 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d68019030 con 0x5586fe199170 2024-05-17T22:47:40.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.018+0000 7f5d5affd700 1 -- 172.21.15.12:0/152312342 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d680093a0 con 0x5586fe199170 2024-05-17T22:47:40.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.018+0000 7f5d5affd700 1 -- 172.21.15.12:0/152312342 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d68007500 con 0x5586fe199170 2024-05-17T22:47:40.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.019+0000 7f5d85c06b80 1 -- 172.21.15.12:0/152312342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5586fe199170 msgr2=0x5586fe199580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:40.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.019+0000 7f5d85c06b80 1 --2- 172.21.15.12:0/152312342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5586fe199170 0x5586fe199580 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5d68011680 tx=0x7f5d68009920 comp rx=0 tx=0).stop 2024-05-17T22:47:40.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.019+0000 7f5d85c06b80 1 -- 172.21.15.12:0/152312342 shutdown_connections 2024-05-17T22:47:40.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.019+0000 7f5d85c06b80 1 --2- 172.21.15.12:0/152312342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5586fe199170 0x5586fe199580 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:40.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.019+0000 7f5d85c06b80 1 --2- 172.21.15.12:0/152312342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586fe01a9f0 0x5586fe19e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:40.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.019+0000 7f5d85c06b80 1 --2- 172.21.15.12:0/152312342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586fe19eaa0 0x5586fe198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:40.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.019+0000 7f5d85c06b80 1 -- 172.21.15.12:0/152312342 >> 172.21.15.12:0/152312342 conn(0x5586fe05ef00 msgr2=0x5586fe193b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:47:40.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.019+0000 7f5d85c06b80 1 -- 172.21.15.12:0/152312342 shutdown_connections 2024-05-17T22:47:40.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.019+0000 7f5d85c06b80 1 -- 172.21.15.12:0/152312342 wait complete. 2024-05-17T22:47:40.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.020+0000 7f5d85c06b80 1 -- start start 2024-05-17T22:47:40.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.020+0000 7f5d85c06b80 1 Processor -- start 2024-05-17T22:47:40.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.020+0000 7f5d85c06b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5586fe01a9f0 0x5586fe1a2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:40.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.020+0000 7f5d85c06b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586fe199170 0x5586fe1a2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:40.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.020+0000 7f5d85c06b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586fe19eaa0 0x5586fe1a2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:40.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.021+0000 7f5d74fc7700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5586fe01a9f0 0x5586fe1a2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:40.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.021+0000 7f5d85c06b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5586fdfbe150 con 0x5586fe19eaa0 2024-05-17T22:47:40.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.021+0000 7f5d85c06b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5586fe057e80 con 0x5586fe199170 2024-05-17T22:47:40.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.021+0000 7f5d85c06b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5586fe1a5390 con 0x5586fe01a9f0 2024-05-17T22:47:40.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.021+0000 7f5d74fc7700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5586fe01a9f0 0x5586fe1a2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:41114/0 (socket says 172.21.15.12:41114) 2024-05-17T22:47:40.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.021+0000 7f5d74fc7700 1 -- 172.21.15.12:0/1492795070 learned_addr learned my addr 172.21.15.12:0/1492795070 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:47:40.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.021+0000 7f5d6ffff700 1 --2- 172.21.15.12:0/1492795070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586fe199170 0x5586fe1a2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:40.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.021+0000 7f5d85c06b80 10 client.0 _ll_register_callbacks cb 0x5586fe051f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:47:40.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.021+0000 7f5d757c8700 1 --2- 172.21.15.12:0/1492795070 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586fe19eaa0 0x5586fe1a2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:40.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.021+0000 7f5d6ffff700 1 -- 172.21.15.12:0/1492795070 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5586fe01a9f0 msgr2=0x5586fe1a2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:47:40.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.021+0000 7f5d6ffff700 1 --2- 172.21.15.12:0/1492795070 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5586fe01a9f0 0x5586fe1a2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:40.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.021+0000 7f5d6ffff700 1 -- 172.21.15.12:0/1492795070 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586fe19eaa0 msgr2=0x5586fe1a2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:47:40.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.021+0000 7f5d6ffff700 1 --2- 172.21.15.12:0/1492795070 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586fe19eaa0 0x5586fe1a2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:40.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.021+0000 7f5d6ffff700 1 -- 172.21.15.12:0/1492795070 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586fe007480 con 0x5586fe199170 2024-05-17T22:47:40.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.021+0000 7f5d74fc7700 1 --2- 172.21.15.12:0/1492795070 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5586fe01a9f0 0x5586fe1a2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:47:40.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.022+0000 7f5d6ffff700 1 --2- 172.21.15.12:0/1492795070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586fe199170 0x5586fe1a2740 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c007fd0 tx=0x7f5d5c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b42cf701177aa95 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:47:40.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.022+0000 7f5d4ffff700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:47:40.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.022+0000 7f5d4ffff700 1 -- 172.21.15.12:0/1492795070 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c023070 con 0x5586fe199170 2024-05-17T22:47:40.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.022+0000 7f5d4ffff700 1 -- 172.21.15.12:0/1492795070 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f5d5c004050 con 0x5586fe199170 2024-05-17T22:47:40.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.022+0000 7f5d85c06b80 1 -- 172.21.15.12:0/1492795070 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5586fe0c91e0 con 0x5586fe199170 2024-05-17T22:47:40.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.022+0000 7f5d4ffff700 1 -- 172.21.15.12:0/1492795070 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5d5c02d050 con 0x5586fe199170 2024-05-17T22:47:40.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.022+0000 7f5d85c06b80 10 client.25733 fetch_fsmap learned FSMap version 39 2024-05-17T22:47:40.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.022+0000 7f5d85c06b80 1 -- 172.21.15.12:0/1492795070 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5586fe1afb00 con 0x5586fe199170 2024-05-17T22:47:40.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.023+0000 7f5d4ffff700 1 -- 172.21.15.12:0/1492795070 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5586fe0c91e0 con 0x5586fe199170 2024-05-17T22:47:40.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.023+0000 7f5d4ffff700 1 -- 172.21.15.12:0/1492795070 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f5d5c01c040 con 0x5586fe199170 2024-05-17T22:47:40.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.023+0000 7f5d85c06b80 10 client.25733 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:47:40.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.023+0000 7f5d85c06b80 10 client.25733 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-17T22:47:40.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.023+0000 7f5d85c06b80 20 client.25733 populate_metadata read hostname 'smithi012' 2024-05-17T22:47:40.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.023+0000 7f5d85c06b80 1 --2- 172.21.15.12:0/1492795070 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5586fe1b21e0 0x5586fe1b4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:40.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.023+0000 7f5d85c06b80 4 client.25733 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:47:40.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.023+0000 7f5d85c06b80 1 -- 172.21.15.12:0/1492795070 --> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586fe0c29c0 con 0x5586fe1b21e0 2024-05-17T22:47:40.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.023+0000 7f5d74fc7700 1 --2- 172.21.15.12:0/1492795070 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5586fe1b21e0 0x5586fe1b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:40.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.024+0000 7f5d4ffff700 10 client.25733 ms_handle_connect on v2:172.21.15.96:6836/4222574338 2024-05-17T22:47:40.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.024+0000 7f5d74fc7700 1 --2- 172.21.15.12:0/1492795070 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5586fe1b21e0 0x5586fe1b4600 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=419d192bdb4bd2f9 server_cookie=1b20a1a38a25c13d in_seq=0 out_seq=0 2024-05-17T22:47:40.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.025+0000 7f5d4ffff700 1 -- 172.21.15.12:0/1492795070 <== mds.1 v2:172.21.15.96:6836/4222574338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5586fe0c29c0 con 0x5586fe1b21e0 2024-05-17T22:47:40.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.025+0000 7f5d4ffff700 10 client.25733 handle_command_reply: tid=0 2024-05-17T22:47:40.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.025+0000 7f5d85c06b80 1 client.25733 shutdown 2024-05-17T22:47:40.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.025+0000 7f5d85c06b80 1 -- 172.21.15.12:0/1492795070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586fe199170 msgr2=0x5586fe1a2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:40.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.025+0000 7f5d85c06b80 1 --2- 172.21.15.12:0/1492795070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586fe199170 0x5586fe1a2740 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c007fd0 tx=0x7f5d5c004750 comp rx=0 tx=0).stop 2024-05-17T22:47:40.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.025+0000 7f5d85c06b80 1 -- 172.21.15.12:0/1492795070 shutdown_connections 2024-05-17T22:47:40.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.025+0000 7f5d85c06b80 1 --2- 172.21.15.12:0/1492795070 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5586fe1b21e0 0x5586fe1b4600 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:40.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.025+0000 7f5d4ffff700 0 client.25733 ms_handle_reset on v2:172.21.15.96:6836/4222574338 2024-05-17T22:47:40.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.025+0000 7f5d85c06b80 1 --2- 172.21.15.12:0/1492795070 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586fe19eaa0 0x5586fe1a2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:40.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.026+0000 7f5d85c06b80 1 --2- 172.21.15.12:0/1492795070 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5586fe01a9f0 0x5586fe1a2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:40.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.026+0000 7f5d85c06b80 1 --2- 172.21.15.12:0/1492795070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586fe199170 0x5586fe1a2740 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:40.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.026+0000 7f5d85c06b80 1 -- 172.21.15.12:0/1492795070 >> 172.21.15.12:0/1492795070 conn(0x5586fe05ef00 msgr2=0x5586fe18ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:47:40.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.027+0000 7f5d85c06b80 1 -- 172.21.15.12:0/1492795070 shutdown_connections 2024-05-17T22:47:40.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.027+0000 7f5d85c06b80 1 -- 172.21.15.12:0/1492795070 wait complete. 2024-05-17T22:47:40.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.027+0000 7f5d85c06b80 20 client.25733 trim_cache size 0 max 16384 2024-05-17T22:47:40.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.050+0000 7f5d85c06b80 1 Processor -- start 2024-05-17T22:47:40.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.050+0000 7f5d85c06b80 1 -- start start 2024-05-17T22:47:40.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.050+0000 7f5d85c06b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5586fe01a9f0 0x5586fe198c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:40.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.050+0000 7f5d85c06b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586fe199170 0x5586fe19c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:40.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.051+0000 7f5d85c06b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586fe19eaa0 0x5586fe18deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:40.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.051+0000 7f5d85c06b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5d701b8b50 con 0x5586fe19eaa0 2024-05-17T22:47:40.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.051+0000 7f5d85c06b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5d701b63e0 con 0x5586fe199170 2024-05-17T22:47:40.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.051+0000 7f5d85c06b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5d701b4230 con 0x5586fe01a9f0 2024-05-17T22:47:40.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.051+0000 7f5d74fc7700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5586fe01a9f0 0x5586fe198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:40.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.051+0000 7f5d757c8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586fe19eaa0 0x5586fe18deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:40.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.051+0000 7f5d74fc7700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5586fe01a9f0 0x5586fe198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:41116/0 (socket says 172.21.15.12:41116) 2024-05-17T22:47:40.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.051+0000 7f5d757c8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586fe19eaa0 0x5586fe18deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60192/0 (socket says 172.21.15.12:60192) 2024-05-17T22:47:40.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.051+0000 7f5d74fc7700 1 -- 172.21.15.12:0/2754048918 learned_addr learned my addr 172.21.15.12:0/2754048918 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:47:40.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.051+0000 7f5d6ffff700 1 --2- 172.21.15.12:0/2754048918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586fe199170 0x5586fe19c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:40.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.051+0000 7f5d6ffff700 1 -- 172.21.15.12:0/2754048918 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5586fe01a9f0 msgr2=0x5586fe198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:47:40.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.051+0000 7f5d6ffff700 1 --2- 172.21.15.12:0/2754048918 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5586fe01a9f0 0x5586fe198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:40.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.051+0000 7f5d6ffff700 1 -- 172.21.15.12:0/2754048918 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586fe19eaa0 msgr2=0x5586fe18deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:47:40.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.051+0000 7f5d6ffff700 1 --2- 172.21.15.12:0/2754048918 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586fe19eaa0 0x5586fe18deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:40.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.051+0000 7f5d6ffff700 1 -- 172.21.15.12:0/2754048918 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586fe1a0750 con 0x5586fe199170 2024-05-17T22:47:40.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.052+0000 7f5d6ffff700 1 --2- 172.21.15.12:0/2754048918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586fe199170 0x5586fe19c820 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c01f590 tx=0x7f5d5c00fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1980a58e33e82281 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:47:40.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.052+0000 7f5d5a7fc700 1 -- 172.21.15.12:0/2754048918 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c023040 con 0x5586fe199170 2024-05-17T22:47:40.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.052+0000 7f5d5a7fc700 1 -- 172.21.15.12:0/2754048918 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d5c003680 con 0x5586fe199170 2024-05-17T22:47:40.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.052+0000 7f5d5a7fc700 1 -- 172.21.15.12:0/2754048918 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c01f870 con 0x5586fe199170 2024-05-17T22:47:40.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.053+0000 7f5d85c06b80 1 -- 172.21.15.12:0/2754048918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586fe199170 msgr2=0x5586fe19c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:40.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.053+0000 7f5d85c06b80 1 --2- 172.21.15.12:0/2754048918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586fe199170 0x5586fe19c820 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c01f590 tx=0x7f5d5c00fa00 comp rx=0 tx=0).stop 2024-05-17T22:47:40.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.053+0000 7f5d85c06b80 1 -- 172.21.15.12:0/2754048918 shutdown_connections 2024-05-17T22:47:40.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.053+0000 7f5d85c06b80 1 --2- 172.21.15.12:0/2754048918 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586fe19eaa0 0x5586fe18deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:40.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.053+0000 7f5d85c06b80 1 --2- 172.21.15.12:0/2754048918 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5586fe01a9f0 0x5586fe198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:40.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.053+0000 7f5d85c06b80 1 --2- 172.21.15.12:0/2754048918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586fe199170 0x5586fe19c820 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:40.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.053+0000 7f5d85c06b80 1 -- 172.21.15.12:0/2754048918 >> 172.21.15.12:0/2754048918 conn(0x5586fe05ef00 msgr2=0x5586fe19d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:47:40.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.053+0000 7f5d85c06b80 1 -- 172.21.15.12:0/2754048918 shutdown_connections 2024-05-17T22:47:40.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.053+0000 7f5d85c06b80 1 -- 172.21.15.12:0/2754048918 wait complete. 2024-05-17T22:47:40.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.054+0000 7f5d85c06b80 1 -- start start 2024-05-17T22:47:40.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.054+0000 7f5d85c06b80 1 Processor -- start 2024-05-17T22:47:40.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.055+0000 7f5d85c06b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5586fe01a9f0 0x5586fe1aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:40.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.055+0000 7f5d85c06b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586fe199170 0x5586fe1aa9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:40.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.055+0000 7f5d85c06b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586fe19eaa0 0x5586fe1b8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:40.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.055+0000 7f5d85c06b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5d701b48a0 con 0x5586fe19eaa0 2024-05-17T22:47:40.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.055+0000 7f5d85c06b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5d701b5d10 con 0x5586fe199170 2024-05-17T22:47:40.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.055+0000 7f5d85c06b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5d701b4f70 con 0x5586fe01a9f0 2024-05-17T22:47:40.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.055+0000 7f5d757c8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586fe19eaa0 0x5586fe1b8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:40.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.055+0000 7f5d74fc7700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5586fe01a9f0 0x5586fe1aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:40.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.055+0000 7f5d757c8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586fe19eaa0 0x5586fe1b8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60194/0 (socket says 172.21.15.12:60194) 2024-05-17T22:47:40.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.055+0000 7f5d757c8700 1 -- 172.21.15.12:0/1609163629 learned_addr learned my addr 172.21.15.12:0/1609163629 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:47:40.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.055+0000 7f5d85c06b80 10 client.0 _ll_register_callbacks cb 0x5586fe051f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:47:40.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.055+0000 7f5d6ffff700 1 --2- 172.21.15.12:0/1609163629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586fe199170 0x5586fe1aa9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:40.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.055+0000 7f5d74fc7700 1 -- 172.21.15.12:0/1609163629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586fe199170 msgr2=0x5586fe1aa9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:47:40.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.055+0000 7f5d74fc7700 1 --2- 172.21.15.12:0/1609163629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586fe199170 0x5586fe1aa9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:40.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.055+0000 7f5d74fc7700 1 -- 172.21.15.12:0/1609163629 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586fe19eaa0 msgr2=0x5586fe1b8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:47:40.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.055+0000 7f5d74fc7700 1 --2- 172.21.15.12:0/1609163629 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586fe19eaa0 0x5586fe1b8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:40.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.055+0000 7f5d74fc7700 1 -- 172.21.15.12:0/1609163629 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586fe1b93f0 con 0x5586fe01a9f0 2024-05-17T22:47:40.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.056+0000 7f5d74fc7700 1 --2- 172.21.15.12:0/1609163629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5586fe01a9f0 0x5586fe1aa470 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5d640a66d0 tx=0x7f5d640e09c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f7fa6c66d403807 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:47:40.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.056+0000 7f5d4ffff700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:47:40.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.056+0000 7f5d4ffff700 1 -- 172.21.15.12:0/1609163629 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d64021880 con 0x5586fe01a9f0 2024-05-17T22:47:40.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.056+0000 7f5d4ffff700 1 -- 172.21.15.12:0/1609163629 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f5d640215c0 con 0x5586fe01a9f0 2024-05-17T22:47:40.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.056+0000 7f5d85c06b80 1 -- 172.21.15.12:0/1609163629 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d5c003680 con 0x5586fe01a9f0 2024-05-17T22:47:40.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.056+0000 7f5d4ffff700 1 -- 172.21.15.12:0/1609163629 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5d640aada0 con 0x5586fe01a9f0 2024-05-17T22:47:40.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.056+0000 7f5d85c06b80 10 client.25687 fetch_fsmap learned FSMap version 39 2024-05-17T22:47:40.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.056+0000 7f5d85c06b80 1 -- 172.21.15.12:0/1609163629 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5586fe21c250 con 0x5586fe01a9f0 2024-05-17T22:47:40.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.056+0000 7f5d4ffff700 1 -- 172.21.15.12:0/1609163629 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d640aa6a0 con 0x5586fe01a9f0 2024-05-17T22:47:40.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.056+0000 7f5d4ffff700 1 -- 172.21.15.12:0/1609163629 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f5d640b2fe0 con 0x5586fe01a9f0 2024-05-17T22:47:40.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.056+0000 7f5d85c06b80 10 client.25687 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:47:40.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.056+0000 7f5d85c06b80 10 client.25687 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-17T22:47:40.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.056+0000 7f5d85c06b80 20 client.25687 populate_metadata read hostname 'smithi012' 2024-05-17T22:47:40.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.056+0000 7f5d85c06b80 1 --2- 172.21.15.12:0/1609163629 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5586fe1b21e0 0x5586fe21e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:40.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.057+0000 7f5d85c06b80 4 client.25687 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:47:40.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.057+0000 7f5d85c06b80 1 -- 172.21.15.12:0/1609163629 --> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f5d5c003680 con 0x5586fe1b21e0 2024-05-17T22:47:40.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.057+0000 7f5d6ffff700 1 --2- 172.21.15.12:0/1609163629 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5586fe1b21e0 0x5586fe21e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:40.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.057+0000 7f5d6ffff700 1 --2- 172.21.15.12:0/1609163629 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5586fe1b21e0 0x5586fe21e970 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=19c67ebe863c7aa8 server_cookie=4bfe9a85c0ac4fa3 in_seq=0 out_seq=0 2024-05-17T22:47:40.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.059+0000 7f5d4ffff700 10 client.25687 ms_handle_connect on v2:172.21.15.96:6836/4222574338 2024-05-17T22:47:40.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.059+0000 7f5d4ffff700 1 -- 172.21.15.12:0/1609163629 <== mds.1 v2:172.21.15.96:6836/4222574338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f5d5c003680 con 0x5586fe1b21e0 2024-05-17T22:47:40.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.059+0000 7f5d4ffff700 10 client.25687 handle_command_reply: tid=0 2024-05-17T22:47:40.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.059+0000 7f5d85c06b80 1 client.25687 shutdown 2024-05-17T22:47:40.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.060+0000 7f5d85c06b80 1 -- 172.21.15.12:0/1609163629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5586fe01a9f0 msgr2=0x5586fe1aa470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:40.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.060+0000 7f5d85c06b80 1 --2- 172.21.15.12:0/1609163629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5586fe01a9f0 0x5586fe1aa470 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5d640a66d0 tx=0x7f5d640e09c0 comp rx=0 tx=0).stop 2024-05-17T22:47:40.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.060+0000 7f5d85c06b80 1 -- 172.21.15.12:0/1609163629 shutdown_connections 2024-05-17T22:47:40.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.060+0000 7f5d85c06b80 1 --2- 172.21.15.12:0/1609163629 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5586fe1b21e0 0x5586fe21e970 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:40.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.060+0000 7f5d4ffff700 0 client.25687 ms_handle_reset on v2:172.21.15.96:6836/4222574338 2024-05-17T22:47:40.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.060+0000 7f5d85c06b80 1 --2- 172.21.15.12:0/1609163629 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586fe19eaa0 0x5586fe1b8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:40.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.060+0000 7f5d85c06b80 1 --2- 172.21.15.12:0/1609163629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5586fe01a9f0 0x5586fe1aa470 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:40.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.060+0000 7f5d85c06b80 1 --2- 172.21.15.12:0/1609163629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586fe199170 0x5586fe1aa9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:40.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.060+0000 7f5d85c06b80 1 -- 172.21.15.12:0/1609163629 >> 172.21.15.12:0/1609163629 conn(0x5586fe05ef00 msgr2=0x5586fe19d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:47:40.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.061+0000 7f5d85c06b80 1 -- 172.21.15.12:0/1609163629 shutdown_connections 2024-05-17T22:47:40.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.061+0000 7f5d85c06b80 1 -- 172.21.15.12:0/1609163629 wait complete. 2024-05-17T22:47:40.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.061+0000 7f5d85c06b80 20 client.25687 trim_cache size 0 max 16384 2024-05-17T22:47:40.065 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:47:40.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.062+0000 7f5d7722b700 1 -- 172.21.15.12:0/3233774620 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5d600b2980 msgr2=0x7f5d600b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:40.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.062+0000 7f5d7722b700 1 --2- 172.21.15.12:0/3233774620 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5d600b2980 0x7f5d600b4e90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x5586fe195320 tx=0x7f5d5c007760 comp rx=0 tx=0).stop 2024-05-17T22:47:40.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.062+0000 7f5d7722b700 1 -- 172.21.15.12:0/3233774620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d70104e70 msgr2=0x7f5d70104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:40.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.062+0000 7f5d7722b700 1 --2- 172.21.15.12:0/3233774620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d70104e70 0x7f5d70104700 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5d640047b0 tx=0x7f5d64005e80 comp rx=0 tx=0).stop 2024-05-17T22:47:40.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.062+0000 7f5d7722b700 1 -- 172.21.15.12:0/3233774620 shutdown_connections 2024-05-17T22:47:40.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.062+0000 7f5d7722b700 1 --2- 172.21.15.12:0/3233774620 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5d600b2980 0x7f5d600b4e90 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:40.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.062+0000 7f5d7722b700 1 --2- 172.21.15.12:0/3233774620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d7010e2a0 0x7f5d70103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:40.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.062+0000 7f5d7722b700 1 --2- 172.21.15.12:0/3233774620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d70104e70 0x7f5d70104700 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:40.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.062+0000 7f5d7722b700 1 --2- 172.21.15.12:0/3233774620 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d701057c0 0x7f5d70102b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:40.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.062+0000 7f5d7722b700 1 -- 172.21.15.12:0/3233774620 >> 172.21.15.12:0/3233774620 conn(0x7f5d70079ae0 msgr2=0x7f5d700ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:47:40.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.062+0000 7f5d7722b700 1 -- 172.21.15.12:0/3233774620 shutdown_connections 2024-05-17T22:47:40.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:40.063+0000 7f5d7722b700 1 -- 172.21.15.12:0/3233774620 wait complete. 2024-05-17T22:47:40.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:39 smithi151 ceph-mon[36101]: pgmap v691: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.4 MiB/s wr, 500 op/s 2024-05-17T22:47:40.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:39 smithi151 ceph-mon[36101]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:47:40.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:39 smithi096 ceph-mon[36065]: pgmap v691: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.4 MiB/s wr, 500 op/s 2024-05-17T22:47:40.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:39 smithi096 ceph-mon[36065]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:47:41.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 500.0 s, 487.1 tps, lat 212.263 ms stddev 238.034 2024-05-17T22:47:42.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:41 smithi151 ceph-mon[36101]: pgmap v692: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.5 MiB/s wr, 474 op/s 2024-05-17T22:47:42.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:41 smithi096 ceph-mon[36065]: pgmap v692: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.5 MiB/s wr, 474 op/s 2024-05-17T22:47:42.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:41 smithi012 ceph-mon[31632]: pgmap v692: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.5 MiB/s wr, 474 op/s 2024-05-17T22:47:43.093 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-17T22:47:43.669 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:47:44.093 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:43 smithi012 ceph-mon[31632]: pgmap v693: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.5 MiB/s wr, 479 op/s 2024-05-17T22:47:44.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:43 smithi151 ceph-mon[36101]: pgmap v693: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.5 MiB/s wr, 479 op/s 2024-05-17T22:47:44.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:43 smithi096 ceph-mon[36065]: pgmap v693: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.5 MiB/s wr, 479 op/s 2024-05-17T22:47:45.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.743+0000 7f5a3a21b700 1 -- 172.21.15.12:0/375336450 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a34107730 msgr2=0x7f5a34107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:45.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.743+0000 7f5a3a21b700 1 --2- 172.21.15.12:0/375336450 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a34107730 0x7f5a34107b40 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c009990 tx=0x7f5a1c009ca0 comp rx=0 tx=0).stop 2024-05-17T22:47:45.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.743+0000 7f5a3a21b700 1 -- 172.21.15.12:0/375336450 shutdown_connections 2024-05-17T22:47:45.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.743+0000 7f5a3a21b700 1 --2- 172.21.15.12:0/375336450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a34108080 0x7f5a3410f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:45.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.743+0000 7f5a3a21b700 1 --2- 172.21.15.12:0/375336450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a34102e00 0x7f5a34103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:45.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.743+0000 7f5a3a21b700 1 --2- 172.21.15.12:0/375336450 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a34107730 0x7f5a34107b40 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:45.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.743+0000 7f5a3a21b700 1 -- 172.21.15.12:0/375336450 >> 172.21.15.12:0/375336450 conn(0x7f5a34079bd0 msgr2=0x7f5a34079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:47:45.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.743+0000 7f5a3a21b700 1 -- 172.21.15.12:0/375336450 shutdown_connections 2024-05-17T22:47:45.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.743+0000 7f5a3a21b700 1 -- 172.21.15.12:0/375336450 wait complete. 2024-05-17T22:47:45.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.744+0000 7f5a3a21b700 1 Processor -- start 2024-05-17T22:47:45.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.744+0000 7f5a3a21b700 1 -- start start 2024-05-17T22:47:45.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.745+0000 7f5a3a21b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a34102e00 0x7f5a3419a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:45.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.745+0000 7f5a3a21b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a34107730 0x7f5a3419ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:45.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.745+0000 7f5a3a21b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a34108080 0x7f5a3419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:45.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.745+0000 7f5a3a21b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5a34112c90 con 0x7f5a34108080 2024-05-17T22:47:45.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.745+0000 7f5a3a21b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5a34112b10 con 0x7f5a34102e00 2024-05-17T22:47:45.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.746+0000 7f5a3a21b700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5a34112e10 con 0x7f5a34107730 2024-05-17T22:47:45.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.749+0000 7f5a32ffd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a34107730 0x7f5a3419ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:45.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.749+0000 7f5a32ffd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a34107730 0x7f5a3419ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:41140/0 (socket says 172.21.15.12:41140) 2024-05-17T22:47:45.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.749+0000 7f5a32ffd700 1 -- 172.21.15.12:0/2028411543 learned_addr learned my addr 172.21.15.12:0/2028411543 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:47:45.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.749+0000 7f5a32ffd700 1 -- 172.21.15.12:0/2028411543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a34102e00 msgr2=0x7f5a3419a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:45.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.749+0000 7f5a32ffd700 1 --2- 172.21.15.12:0/2028411543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a34102e00 0x7f5a3419a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:45.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.749+0000 7f5a32ffd700 1 -- 172.21.15.12:0/2028411543 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a34108080 msgr2=0x7f5a3419efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:45.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.749+0000 7f5a32ffd700 1 --2- 172.21.15.12:0/2028411543 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a34108080 0x7f5a3419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:45.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.749+0000 7f5a32ffd700 1 -- 172.21.15.12:0/2028411543 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a3419f650 con 0x7f5a34107730 2024-05-17T22:47:45.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.750+0000 7f5a32ffd700 1 --2- 172.21.15.12:0/2028411543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a34107730 0x7f5a3419ad20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c000c00 tx=0x7f5a1c00bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:47:45.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.751+0000 7f5a30ff9700 1 -- 172.21.15.12:0/2028411543 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c00fa10 con 0x7f5a34107730 2024-05-17T22:47:45.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.751+0000 7f5a30ff9700 1 -- 172.21.15.12:0/2028411543 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a1c004ce0 con 0x7f5a34107730 2024-05-17T22:47:45.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.751+0000 7f5a3a21b700 1 -- 172.21.15.12:0/2028411543 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a3419f8e0 con 0x7f5a34107730 2024-05-17T22:47:45.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.751+0000 7f5a3a21b700 1 -- 172.21.15.12:0/2028411543 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a3419fd00 con 0x7f5a34107730 2024-05-17T22:47:45.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.752+0000 7f5a30ff9700 1 -- 172.21.15.12:0/2028411543 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c005620 con 0x7f5a34107730 2024-05-17T22:47:45.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.756+0000 7f5a30ff9700 1 -- 172.21.15.12:0/2028411543 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5a1c009d10 con 0x7f5a34107730 2024-05-17T22:47:45.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.772+0000 7f5a30ff9700 1 --2- 172.21.15.12:0/2028411543 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5a200b28a0 0x7f5a200b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:45.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.773+0000 7f5a30ff9700 1 -- 172.21.15.12:0/2028411543 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f5a1c0e30e0 con 0x7f5a34107730 2024-05-17T22:47:45.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.777+0000 7f5a48bf6b80 1 Processor -- start 2024-05-17T22:47:45.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.779+0000 7f5a337fe700 1 --2- 172.21.15.12:0/2028411543 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5a200b28a0 0x7f5a200b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:45.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.781+0000 7f5a48bf6b80 1 -- start start 2024-05-17T22:47:45.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.781+0000 7f5a48bf6b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3e8ee09f0 0x55f3e9064560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:45.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.781+0000 7f5a48bf6b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3e9064aa0 0x55f3e905ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:45.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.782+0000 7f5a48bf6b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3e905f170 0x55f3e905f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:45.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.782+0000 7f5a32ffd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3e9064aa0 0x55f3e905ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:45.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.782+0000 7f5a32ffd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3e9064aa0 0x55f3e905ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:41146/0 (socket says 172.21.15.12:41146) 2024-05-17T22:47:45.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.782+0000 7f5a32ffd700 1 -- 172.21.15.12:0/2178119797 learned_addr learned my addr 172.21.15.12:0/2178119797 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:47:45.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.782+0000 7f5a48bf6b80 1 -- 172.21.15.12:0/2178119797 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f3e8f8d290 con 0x55f3e8ee09f0 2024-05-17T22:47:45.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.782+0000 7f5a48bf6b80 1 -- 172.21.15.12:0/2178119797 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f3e8f7cfd0 con 0x55f3e905f170 2024-05-17T22:47:45.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.782+0000 7f5a48bf6b80 1 -- 172.21.15.12:0/2178119797 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f3e8f80e70 con 0x55f3e9064aa0 2024-05-17T22:47:45.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.786+0000 7f5a33fff700 1 --2- 172.21.15.12:0/2178119797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3e905f170 0x55f3e905f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:45.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.786+0000 7f5a32ffd700 1 -- 172.21.15.12:0/2178119797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3e905f170 msgr2=0x55f3e905f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:47:45.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.786+0000 7f5a32ffd700 1 --2- 172.21.15.12:0/2178119797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3e905f170 0x55f3e905f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:45.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.786+0000 7f5a32ffd700 1 -- 172.21.15.12:0/2178119797 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3e8ee09f0 msgr2=0x55f3e9064560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:47:45.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.786+0000 7f5a32ffd700 1 --2- 172.21.15.12:0/2178119797 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3e8ee09f0 0x55f3e9064560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:45.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.786+0000 7f5a32ffd700 1 -- 172.21.15.12:0/2178119797 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3e8f26590 con 0x55f3e9064aa0 2024-05-17T22:47:45.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.790+0000 7f5a32ffd700 1 --2- 172.21.15.12:0/2178119797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3e9064aa0 0x55f3e905ec30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c0aa100 tx=0x7f5a1c005020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=935788b9e2471672 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:47:45.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.794+0000 7f5a337fe700 1 --2- 172.21.15.12:0/2028411543 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5a200b28a0 0x7f5a200b4db0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5a24005950 tx=0x7f5a24009f70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:47:45.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.796+0000 7f5a29ffb700 1 -- 172.21.15.12:0/2178119797 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c0aab00 con 0x55f3e9064aa0 2024-05-17T22:47:45.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.796+0000 7f5a29ffb700 1 -- 172.21.15.12:0/2178119797 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a1c0aa400 con 0x55f3e9064aa0 2024-05-17T22:47:45.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.796+0000 7f5a29ffb700 1 -- 172.21.15.12:0/2178119797 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c00fe10 con 0x55f3e9064aa0 2024-05-17T22:47:45.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.800+0000 7f5a48bf6b80 1 -- 172.21.15.12:0/2178119797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3e9064aa0 msgr2=0x55f3e905ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:45.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.800+0000 7f5a48bf6b80 1 --2- 172.21.15.12:0/2178119797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3e9064aa0 0x55f3e905ec30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c0aa100 tx=0x7f5a1c005020 comp rx=0 tx=0).stop 2024-05-17T22:47:45.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.803+0000 7f5a48bf6b80 1 -- 172.21.15.12:0/2178119797 shutdown_connections 2024-05-17T22:47:45.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.803+0000 7f5a48bf6b80 1 --2- 172.21.15.12:0/2178119797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3e905f170 0x55f3e905f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:45.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.803+0000 7f5a48bf6b80 1 --2- 172.21.15.12:0/2178119797 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3e8ee09f0 0x55f3e9064560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:45.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.803+0000 7f5a48bf6b80 1 --2- 172.21.15.12:0/2178119797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3e9064aa0 0x55f3e905ec30 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:45.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.803+0000 7f5a48bf6b80 1 -- 172.21.15.12:0/2178119797 >> 172.21.15.12:0/2178119797 conn(0x55f3e8f24f00 msgr2=0x55f3e9059b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:47:45.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.808+0000 7f5a48bf6b80 1 -- 172.21.15.12:0/2178119797 shutdown_connections 2024-05-17T22:47:45.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.809+0000 7f5a48bf6b80 1 -- 172.21.15.12:0/2178119797 wait complete. 2024-05-17T22:47:45.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.809+0000 7f5a48bf6b80 1 -- start start 2024-05-17T22:47:45.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.810+0000 7f5a48bf6b80 1 Processor -- start 2024-05-17T22:47:45.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.810+0000 7f5a48bf6b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3e8ee09f0 0x55f3e9068340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:45.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.810+0000 7f5a48bf6b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3e905f170 0x55f3e9068880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:45.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.810+0000 7f5a48bf6b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3e9064aa0 0x55f3e9068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:45.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.811+0000 7f5a48bf6b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f3e8e84150 con 0x55f3e8ee09f0 2024-05-17T22:47:45.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.811+0000 7f5a48bf6b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f3e8f1de80 con 0x55f3e9064aa0 2024-05-17T22:47:45.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.811+0000 7f5a48bf6b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f3e8f80e70 con 0x55f3e905f170 2024-05-17T22:47:45.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.811+0000 7f5a32ffd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3e905f170 0x55f3e9068880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:45.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.811+0000 7f5a32ffd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3e905f170 0x55f3e9068880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:41152/0 (socket says 172.21.15.12:41152) 2024-05-17T22:47:45.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.811+0000 7f5a32ffd700 1 -- 172.21.15.12:0/3179037562 learned_addr learned my addr 172.21.15.12:0/3179037562 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:47:45.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.811+0000 7f5a33fff700 1 --2- 172.21.15.12:0/3179037562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3e9064aa0 0x55f3e9068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:45.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.811+0000 7f5a48bf6b80 10 client.0 _ll_register_callbacks cb 0x55f3e8f17f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:47:45.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.812+0000 7f5a32ffd700 1 -- 172.21.15.12:0/3179037562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3e9064aa0 msgr2=0x55f3e9068dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:47:45.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.812+0000 7f5a32ffd700 1 --2- 172.21.15.12:0/3179037562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3e9064aa0 0x55f3e9068dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:45.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.812+0000 7f5a32ffd700 1 -- 172.21.15.12:0/3179037562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3e8ee09f0 msgr2=0x55f3e9068340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:47:45.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.812+0000 7f5a32ffd700 1 --2- 172.21.15.12:0/3179037562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3e8ee09f0 0x55f3e9068340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:45.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.812+0000 7f5a32ffd700 1 -- 172.21.15.12:0/3179037562 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3e8ecd480 con 0x55f3e905f170 2024-05-17T22:47:45.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.816+0000 7f5a32ffd700 1 --2- 172.21.15.12:0/3179037562 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3e905f170 0x55f3e9068880 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c004ff0 tx=0x7f5a1c004a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23324b7f692c771c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:47:45.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.817+0000 7f5a12ffd700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:47:45.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.817+0000 7f5a12ffd700 1 -- 172.21.15.12:0/3179037562 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c0b31d0 con 0x55f3e905f170 2024-05-17T22:47:45.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.818+0000 7f5a12ffd700 1 -- 172.21.15.12:0/3179037562 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f5a1c0a9870 con 0x55f3e905f170 2024-05-17T22:47:45.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.818+0000 7f5a48bf6b80 1 -- 172.21.15.12:0/3179037562 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a1c0aa400 con 0x55f3e905f170 2024-05-17T22:47:45.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.818+0000 7f5a12ffd700 1 -- 172.21.15.12:0/3179037562 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5a1c0aab00 con 0x55f3e905f170 2024-05-17T22:47:45.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.818+0000 7f5a48bf6b80 10 client.25699 fetch_fsmap learned FSMap version 39 2024-05-17T22:47:45.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.818+0000 7f5a48bf6b80 1 -- 172.21.15.12:0/3179037562 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f3e9086b20 con 0x55f3e905f170 2024-05-17T22:47:45.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.818+0000 7f5a12ffd700 1 -- 172.21.15.12:0/3179037562 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a1c0cb340 con 0x55f3e905f170 2024-05-17T22:47:45.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.819+0000 7f5a12ffd700 1 -- 172.21.15.12:0/3179037562 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f5a1c0a90d0 con 0x55f3e905f170 2024-05-17T22:47:45.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.819+0000 7f5a48bf6b80 10 client.25699 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:47:45.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.819+0000 7f5a48bf6b80 10 client.25699 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:47:45.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.819+0000 7f5a48bf6b80 20 client.25699 populate_metadata read hostname 'smithi012' 2024-05-17T22:47:45.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.819+0000 7f5a48bf6b80 1 --2- 172.21.15.12:0/3179037562 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55f3e9088e20 0x55f3e908b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:45.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.819+0000 7f5a48bf6b80 4 client.25699 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:47:45.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.819+0000 7f5a48bf6b80 1 -- 172.21.15.12:0/3179037562 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5a1c0aa400 con 0x55f3e9088e20 2024-05-17T22:47:45.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.822+0000 7f5a337fe700 1 --2- 172.21.15.12:0/3179037562 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55f3e9088e20 0x55f3e908b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:45.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.836+0000 7f5a12ffd700 10 client.25699 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:47:45.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.836+0000 7f5a337fe700 1 --2- 172.21.15.12:0/3179037562 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55f3e9088e20 0x55f3e908b240 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=930945da3f6f6307 server_cookie=6815e51b4a92bdd0 in_seq=0 out_seq=0 2024-05-17T22:47:45.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.838+0000 7f5a12ffd700 1 -- 172.21.15.12:0/3179037562 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f5a1c0aa400 con 0x55f3e9088e20 2024-05-17T22:47:45.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.838+0000 7f5a12ffd700 10 client.25699 handle_command_reply: tid=0 2024-05-17T22:47:45.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.838+0000 7f5a48bf6b80 1 client.25699 shutdown 2024-05-17T22:47:45.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.839+0000 7f5a48bf6b80 1 -- 172.21.15.12:0/3179037562 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3e905f170 msgr2=0x55f3e9068880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:45.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.839+0000 7f5a48bf6b80 1 --2- 172.21.15.12:0/3179037562 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3e905f170 0x55f3e9068880 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c004ff0 tx=0x7f5a1c004a60 comp rx=0 tx=0).stop 2024-05-17T22:47:45.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.839+0000 7f5a48bf6b80 1 -- 172.21.15.12:0/3179037562 shutdown_connections 2024-05-17T22:47:45.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.839+0000 7f5a48bf6b80 1 --2- 172.21.15.12:0/3179037562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3e9064aa0 0x55f3e9068dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:45.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.839+0000 7f5a48bf6b80 1 --2- 172.21.15.12:0/3179037562 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55f3e9088e20 0x55f3e908b240 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:45.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.839+0000 7f5a48bf6b80 1 --2- 172.21.15.12:0/3179037562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3e8ee09f0 0x55f3e9068340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:45.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.839+0000 7f5a48bf6b80 1 --2- 172.21.15.12:0/3179037562 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3e905f170 0x55f3e9068880 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:45.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.839+0000 7f5a12ffd700 0 client.25699 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:47:45.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.839+0000 7f5a48bf6b80 1 -- 172.21.15.12:0/3179037562 >> 172.21.15.12:0/3179037562 conn(0x55f3e8f24f00 msgr2=0x55f3e9053ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:47:45.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.840+0000 7f5a48bf6b80 1 -- 172.21.15.12:0/3179037562 shutdown_connections 2024-05-17T22:47:45.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.840+0000 7f5a48bf6b80 1 -- 172.21.15.12:0/3179037562 wait complete. 2024-05-17T22:47:45.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.840+0000 7f5a48bf6b80 20 client.25699 trim_cache size 0 max 16384 2024-05-17T22:47:45.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.873+0000 7f5a48bf6b80 1 Processor -- start 2024-05-17T22:47:45.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.876+0000 7f5a48bf6b80 1 -- start start 2024-05-17T22:47:45.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.876+0000 7f5a48bf6b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3e8ee09f0 0x55f3e906a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:45.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.876+0000 7f5a48bf6b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3e905f170 0x55f3e9053eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:45.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.876+0000 7f5a48bf6b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3e9064aa0 0x55f3e9068c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:45.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.876+0000 7f5a48bf6b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5a341b00d0 con 0x55f3e8ee09f0 2024-05-17T22:47:45.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.876+0000 7f5a48bf6b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5a341ad960 con 0x55f3e905f170 2024-05-17T22:47:45.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.876+0000 7f5a48bf6b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5a341ab7d0 con 0x55f3e9064aa0 2024-05-17T22:47:45.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.878+0000 7f5a337fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3e8ee09f0 0x55f3e906a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:45.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.878+0000 7f5a337fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3e8ee09f0 0x55f3e906a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60246/0 (socket says 172.21.15.12:60246) 2024-05-17T22:47:45.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.878+0000 7f5a337fe700 1 -- 172.21.15.12:0/2229747203 learned_addr learned my addr 172.21.15.12:0/2229747203 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:47:45.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.882+0000 7f5a337fe700 1 -- 172.21.15.12:0/2229747203 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3e9064aa0 msgr2=0x55f3e9068c50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T22:47:45.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.882+0000 7f5a337fe700 1 --2- 172.21.15.12:0/2229747203 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3e9064aa0 0x55f3e9068c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:45.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.882+0000 7f5a337fe700 1 -- 172.21.15.12:0/2229747203 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3e905f170 msgr2=0x55f3e9053eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T22:47:45.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.882+0000 7f5a337fe700 1 --2- 172.21.15.12:0/2229747203 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3e905f170 0x55f3e9053eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:45.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.882+0000 7f5a337fe700 1 -- 172.21.15.12:0/2229747203 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3e90648a0 con 0x55f3e8ee09f0 2024-05-17T22:47:45.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.885+0000 7f5a337fe700 1 --2- 172.21.15.12:0/2229747203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3e8ee09f0 0x55f3e906a930 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x55f3e905aae0 tx=0x7f5a24009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0238872d91bb03b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:47:45.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.892+0000 7f5a297fa700 1 -- 172.21.15.12:0/2229747203 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a24012920 con 0x55f3e8ee09f0 2024-05-17T22:47:45.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.892+0000 7f5a297fa700 1 -- 172.21.15.12:0/2229747203 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a2402a070 con 0x55f3e8ee09f0 2024-05-17T22:47:45.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.892+0000 7f5a297fa700 1 -- 172.21.15.12:0/2229747203 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a24024a30 con 0x55f3e8ee09f0 2024-05-17T22:47:45.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.893+0000 7f5a48bf6b80 1 -- 172.21.15.12:0/2229747203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3e8ee09f0 msgr2=0x55f3e906a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:45.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.893+0000 7f5a48bf6b80 1 --2- 172.21.15.12:0/2229747203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3e8ee09f0 0x55f3e906a930 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x55f3e905aae0 tx=0x7f5a24009260 comp rx=0 tx=0).stop 2024-05-17T22:47:45.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.893+0000 7f5a48bf6b80 1 -- 172.21.15.12:0/2229747203 shutdown_connections 2024-05-17T22:47:45.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.893+0000 7f5a48bf6b80 1 --2- 172.21.15.12:0/2229747203 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3e9064aa0 0x55f3e9068c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:45.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.893+0000 7f5a48bf6b80 1 --2- 172.21.15.12:0/2229747203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3e8ee09f0 0x55f3e906a930 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:45.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.893+0000 7f5a48bf6b80 1 --2- 172.21.15.12:0/2229747203 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3e905f170 0x55f3e9053eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:45.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.893+0000 7f5a48bf6b80 1 -- 172.21.15.12:0/2229747203 >> 172.21.15.12:0/2229747203 conn(0x55f3e8f24f00 msgr2=0x55f3e90625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:47:45.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.894+0000 7f5a48bf6b80 1 -- 172.21.15.12:0/2229747203 shutdown_connections 2024-05-17T22:47:45.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.894+0000 7f5a48bf6b80 1 -- 172.21.15.12:0/2229747203 wait complete. 2024-05-17T22:47:45.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.894+0000 7f5a48bf6b80 1 -- start start 2024-05-17T22:47:45.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.895+0000 7f5a48bf6b80 1 Processor -- start 2024-05-17T22:47:45.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.895+0000 7f5a48bf6b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3e8ee09f0 0x55f3e9071710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:45.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.895+0000 7f5a48bf6b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3e905f170 0x55f3e9071c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:45.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.895+0000 7f5a48bf6b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3e9064aa0 0x55f3e906cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:45.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.895+0000 7f5a48bf6b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5a341abe20 con 0x55f3e8ee09f0 2024-05-17T22:47:45.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.895+0000 7f5a48bf6b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5a341ad290 con 0x55f3e905f170 2024-05-17T22:47:45.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.895+0000 7f5a48bf6b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5a341ac4f0 con 0x55f3e9064aa0 2024-05-17T22:47:45.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.895+0000 7f5a48bf6b80 10 client.0 _ll_register_callbacks cb 0x55f3e8f17f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:47:45.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.896+0000 7f5a33fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3e9064aa0 0x55f3e906cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:45.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.896+0000 7f5a32ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3e905f170 0x55f3e9071c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:45.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.896+0000 7f5a33fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3e9064aa0 0x55f3e906cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:41166/0 (socket says 172.21.15.12:41166) 2024-05-17T22:47:45.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.896+0000 7f5a33fff700 1 -- 172.21.15.12:0/2630836091 learned_addr learned my addr 172.21.15.12:0/2630836091 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:47:45.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.898+0000 7f5a337fe700 1 --2- 172.21.15.12:0/2630836091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3e8ee09f0 0x55f3e9071710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:45.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.898+0000 7f5a32ffd700 1 -- 172.21.15.12:0/2630836091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3e9064aa0 msgr2=0x55f3e906cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:47:45.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.898+0000 7f5a32ffd700 1 --2- 172.21.15.12:0/2630836091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3e9064aa0 0x55f3e906cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:45.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.898+0000 7f5a32ffd700 1 -- 172.21.15.12:0/2630836091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3e8ee09f0 msgr2=0x55f3e9071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:47:45.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.898+0000 7f5a32ffd700 1 --2- 172.21.15.12:0/2630836091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3e8ee09f0 0x55f3e9071710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:45.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.898+0000 7f5a32ffd700 1 -- 172.21.15.12:0/2630836091 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3e906d330 con 0x55f3e905f170 2024-05-17T22:47:45.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.902+0000 7f5a32ffd700 1 --2- 172.21.15.12:0/2630836091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3e905f170 0x55f3e9071c50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c0aa0b0 tx=0x7f5a1c005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a04adf0325c5b17a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:47:45.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.902+0000 7f5a12ffd700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:47:45.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.902+0000 7f5a12ffd700 1 -- 172.21.15.12:0/2630836091 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c0b31d0 con 0x55f3e905f170 2024-05-17T22:47:45.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.902+0000 7f5a12ffd700 1 -- 172.21.15.12:0/2630836091 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f5a1c0a4b20 con 0x55f3e905f170 2024-05-17T22:47:45.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.902+0000 7f5a48bf6b80 1 -- 172.21.15.12:0/2630836091 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a2402a070 con 0x55f3e905f170 2024-05-17T22:47:45.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.902+0000 7f5a12ffd700 1 -- 172.21.15.12:0/2630836091 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5a1c0aa400 con 0x55f3e905f170 2024-05-17T22:47:45.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.903+0000 7f5a48bf6b80 10 client.25748 fetch_fsmap learned FSMap version 39 2024-05-17T22:47:45.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.903+0000 7f5a48bf6b80 1 -- 172.21.15.12:0/2630836091 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f3e906d330 con 0x55f3e905f170 2024-05-17T22:47:45.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.908+0000 7f5a12ffd700 1 -- 172.21.15.12:0/2630836091 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a2402a070 con 0x55f3e905f170 2024-05-17T22:47:45.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.908+0000 7f5a12ffd700 1 -- 172.21.15.12:0/2630836091 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f5a1c0af940 con 0x55f3e905f170 2024-05-17T22:47:45.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.909+0000 7f5a48bf6b80 10 client.25748 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:47:45.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.909+0000 7f5a48bf6b80 10 client.25748 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:47:45.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.909+0000 7f5a48bf6b80 20 client.25748 populate_metadata read hostname 'smithi012' 2024-05-17T22:47:45.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.909+0000 7f5a48bf6b80 1 --2- 172.21.15.12:0/2630836091 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55f3e91041b0 0x55f3e91065d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:45.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.909+0000 7f5a48bf6b80 4 client.25748 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:47:45.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.909+0000 7f5a48bf6b80 1 -- 172.21.15.12:0/2630836091 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f3e8f8f1e0 con 0x55f3e91041b0 2024-05-17T22:47:45.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.912+0000 7f5a337fe700 1 --2- 172.21.15.12:0/2630836091 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55f3e91041b0 0x55f3e91065d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:45.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.914+0000 7f5a337fe700 1 --2- 172.21.15.12:0/2630836091 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55f3e91041b0 0x55f3e91065d0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8dc9fe9559e8767 server_cookie=973b4faf10edfc41 in_seq=0 out_seq=0 2024-05-17T22:47:45.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.915+0000 7f5a12ffd700 10 client.25748 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:47:45.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.915+0000 7f5a12ffd700 1 -- 172.21.15.12:0/2630836091 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f3e8f8f1e0 con 0x55f3e91041b0 2024-05-17T22:47:45.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.915+0000 7f5a12ffd700 10 client.25748 handle_command_reply: tid=0 2024-05-17T22:47:45.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.916+0000 7f5a48bf6b80 1 client.25748 shutdown 2024-05-17T22:47:45.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.917+0000 7f5a48bf6b80 1 -- 172.21.15.12:0/2630836091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3e905f170 msgr2=0x55f3e9071c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:45.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.917+0000 7f5a48bf6b80 1 --2- 172.21.15.12:0/2630836091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3e905f170 0x55f3e9071c50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c0aa0b0 tx=0x7f5a1c005fb0 comp rx=0 tx=0).stop 2024-05-17T22:47:45.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.917+0000 7f5a48bf6b80 1 -- 172.21.15.12:0/2630836091 shutdown_connections 2024-05-17T22:47:45.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.917+0000 7f5a48bf6b80 1 --2- 172.21.15.12:0/2630836091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3e9064aa0 0x55f3e906cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:45.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.917+0000 7f5a48bf6b80 1 --2- 172.21.15.12:0/2630836091 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55f3e91041b0 0x55f3e91065d0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:45.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.917+0000 7f5a48bf6b80 1 --2- 172.21.15.12:0/2630836091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3e8ee09f0 0x55f3e9071710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:45.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.917+0000 7f5a48bf6b80 1 --2- 172.21.15.12:0/2630836091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3e905f170 0x55f3e9071c50 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:45.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.917+0000 7f5a48bf6b80 1 -- 172.21.15.12:0/2630836091 >> 172.21.15.12:0/2630836091 conn(0x55f3e8f24f00 msgr2=0x55f3e9062370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:47:45.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.918+0000 7f5a12ffd700 0 client.25748 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:47:45.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.918+0000 7f5a48bf6b80 1 -- 172.21.15.12:0/2630836091 shutdown_connections 2024-05-17T22:47:45.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.919+0000 7f5a48bf6b80 1 -- 172.21.15.12:0/2630836091 wait complete. 2024-05-17T22:47:45.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.920+0000 7f5a48bf6b80 20 client.25748 trim_cache size 0 max 16384 2024-05-17T22:47:45.922 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:47:45.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.922+0000 7f5a3a21b700 1 -- 172.21.15.12:0/2028411543 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5a200b28a0 msgr2=0x7f5a200b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:45.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.922+0000 7f5a3a21b700 1 --2- 172.21.15.12:0/2028411543 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5a200b28a0 0x7f5a200b4db0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5a24005950 tx=0x7f5a24009f70 comp rx=0 tx=0).stop 2024-05-17T22:47:45.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.922+0000 7f5a3a21b700 1 -- 172.21.15.12:0/2028411543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a34107730 msgr2=0x7f5a3419ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:45.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.922+0000 7f5a3a21b700 1 --2- 172.21.15.12:0/2028411543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a34107730 0x7f5a3419ad20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c000c00 tx=0x7f5a1c00bfd0 comp rx=0 tx=0).stop 2024-05-17T22:47:45.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.923+0000 7f5a3a21b700 1 -- 172.21.15.12:0/2028411543 shutdown_connections 2024-05-17T22:47:45.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.923+0000 7f5a3a21b700 1 --2- 172.21.15.12:0/2028411543 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5a200b28a0 0x7f5a200b4db0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:45.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.923+0000 7f5a3a21b700 1 --2- 172.21.15.12:0/2028411543 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a34108080 0x7f5a3419efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:45.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.923+0000 7f5a3a21b700 1 --2- 172.21.15.12:0/2028411543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a34102e00 0x7f5a3419a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:45.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.923+0000 7f5a3a21b700 1 --2- 172.21.15.12:0/2028411543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a34107730 0x7f5a3419ad20 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:45.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.923+0000 7f5a3a21b700 1 -- 172.21.15.12:0/2028411543 >> 172.21.15.12:0/2028411543 conn(0x7f5a34079bd0 msgr2=0x7f5a3406b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:47:45.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.923+0000 7f5a3a21b700 1 -- 172.21.15.12:0/2028411543 shutdown_connections 2024-05-17T22:47:45.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:45.923+0000 7f5a3a21b700 1 -- 172.21.15.12:0/2028411543 wait complete. 2024-05-17T22:47:46.197 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:45 smithi012 ceph-mon[31632]: pgmap v694: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.3 MiB/s wr, 486 op/s 2024-05-17T22:47:46.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:45 smithi151 ceph-mon[36101]: pgmap v694: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.3 MiB/s wr, 486 op/s 2024-05-17T22:47:46.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:45 smithi096 ceph-mon[36065]: pgmap v694: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.3 MiB/s wr, 486 op/s 2024-05-17T22:47:46.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 505.0 s, 762.6 tps, lat 131.875 ms stddev 158.679 2024-05-17T22:47:47.914 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-17T22:47:47.915 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-17T22:47:48.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:47 smithi151 ceph-mon[36101]: pgmap v695: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.1 MiB/s wr, 462 op/s 2024-05-17T22:47:48.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:47 smithi096 ceph-mon[36065]: pgmap v695: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.1 MiB/s wr, 462 op/s 2024-05-17T22:47:48.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:47 smithi012 ceph-mon[31632]: pgmap v695: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.1 MiB/s wr, 462 op/s 2024-05-17T22:47:48.434 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:47:50.226 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:49 smithi012 ceph-mon[31632]: pgmap v696: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.8 MiB/s wr, 520 op/s 2024-05-17T22:47:50.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:49 smithi151 ceph-mon[36101]: pgmap v696: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.8 MiB/s wr, 520 op/s 2024-05-17T22:47:50.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:49 smithi096 ceph-mon[36065]: pgmap v696: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.8 MiB/s wr, 520 op/s 2024-05-17T22:47:50.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.674+0000 7f1e856ac700 1 -- 172.21.15.12:0/16368861 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e8010a830 msgr2=0x7f1e8010ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:50.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.674+0000 7f1e856ac700 1 --2- 172.21.15.12:0/16368861 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e8010a830 0x7f1e8010ac40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1e68009990 tx=0x7f1e68009ca0 comp rx=0 tx=0).stop 2024-05-17T22:47:50.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.681+0000 7f1e856ac700 1 -- 172.21.15.12:0/16368861 shutdown_connections 2024-05-17T22:47:50.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.681+0000 7f1e856ac700 1 --2- 172.21.15.12:0/16368861 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e8011d550 0x7f1e8011f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:50.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.681+0000 7f1e856ac700 1 --2- 172.21.15.12:0/16368861 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e8010a830 0x7f1e8010ac40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:50.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.681+0000 7f1e856ac700 1 --2- 172.21.15.12:0/16368861 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e8010b180 0x7f1e8011d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:50.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.681+0000 7f1e856ac700 1 -- 172.21.15.12:0/16368861 >> 172.21.15.12:0/16368861 conn(0x7f1e8006efc0 msgr2=0x7f1e8006f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:47:50.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.686+0000 7f1e856ac700 1 -- 172.21.15.12:0/16368861 shutdown_connections 2024-05-17T22:47:50.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.686+0000 7f1e856ac700 1 -- 172.21.15.12:0/16368861 wait complete. 2024-05-17T22:47:50.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.687+0000 7f1e856ac700 1 Processor -- start 2024-05-17T22:47:50.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.689+0000 7f1e856ac700 1 -- start start 2024-05-17T22:47:50.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.689+0000 7f1e856ac700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e8010a830 0x7f1e801a95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:50.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.689+0000 7f1e856ac700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e8010b180 0x7f1e801a9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:50.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.689+0000 7f1e856ac700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e8011d550 0x7f1e801ade10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:50.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.689+0000 7f1e856ac700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1e80122c40 con 0x7f1e8011d550 2024-05-17T22:47:50.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.689+0000 7f1e856ac700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1e80122ac0 con 0x7f1e8010b180 2024-05-17T22:47:50.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.689+0000 7f1e856ac700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1e80122dc0 con 0x7f1e8010a830 2024-05-17T22:47:50.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.690+0000 7f1e7effd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e8010a830 0x7f1e801a95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:50.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.690+0000 7f1e7effd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e8010a830 0x7f1e801a95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:49360/0 (socket says 172.21.15.12:49360) 2024-05-17T22:47:50.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.690+0000 7f1e7effd700 1 -- 172.21.15.12:0/2101656506 learned_addr learned my addr 172.21.15.12:0/2101656506 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:47:50.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.691+0000 7f1e7f7fe700 1 --2- 172.21.15.12:0/2101656506 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e8011d550 0x7f1e801ade10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:50.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.691+0000 7f1e7effd700 1 -- 172.21.15.12:0/2101656506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e8010b180 msgr2=0x7f1e801a9b80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-17T22:47:50.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.691+0000 7f1e7effd700 1 --2- 172.21.15.12:0/2101656506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e8010b180 0x7f1e801a9b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:50.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.691+0000 7f1e7effd700 1 -- 172.21.15.12:0/2101656506 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e8011d550 msgr2=0x7f1e801ade10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:50.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.691+0000 7f1e7effd700 1 --2- 172.21.15.12:0/2101656506 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e8011d550 0x7f1e801ade10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:50.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.691+0000 7f1e7effd700 1 -- 172.21.15.12:0/2101656506 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e801ae4b0 con 0x7f1e8010a830 2024-05-17T22:47:50.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.695+0000 7f1e7effd700 1 --2- 172.21.15.12:0/2101656506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e8010a830 0x7f1e801a95e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1e68006010 tx=0x7f1e6800b650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:47:50.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.697+0000 7f1e7cff9700 1 -- 172.21.15.12:0/2101656506 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e68027070 con 0x7f1e8010a830 2024-05-17T22:47:50.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.697+0000 7f1e856ac700 1 -- 172.21.15.12:0/2101656506 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e801ae740 con 0x7f1e8010a830 2024-05-17T22:47:50.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.698+0000 7f1e856ac700 1 -- 172.21.15.12:0/2101656506 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e801aec20 con 0x7f1e8010a830 2024-05-17T22:47:50.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.698+0000 7f1e7cff9700 1 -- 172.21.15.12:0/2101656506 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e68022470 con 0x7f1e8010a830 2024-05-17T22:47:50.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.698+0000 7f1e7cff9700 1 -- 172.21.15.12:0/2101656506 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e68017650 con 0x7f1e8010a830 2024-05-17T22:47:50.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.710+0000 7f1e94087b80 1 Processor -- start 2024-05-17T22:47:50.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.710+0000 7f1e94087b80 1 -- start start 2024-05-17T22:47:50.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.717+0000 7f1e94087b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f285bcba70 0x55f285d4ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:50.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.717+0000 7f1e94087b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f285d4f480 0x55f285d49640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:50.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.717+0000 7f1e7f7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f285bcba70 0x55f285d4ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:50.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.717+0000 7f1e7f7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f285bcba70 0x55f285d4ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:46548/0 (socket says 172.21.15.12:46548) 2024-05-17T22:47:50.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.717+0000 7f1e7f7fe700 1 -- 172.21.15.12:0/460973957 learned_addr learned my addr 172.21.15.12:0/460973957 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:47:50.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.717+0000 7f1e7effd700 1 --2- 172.21.15.12:0/460973957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f285d4f480 0x55f285d49640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:50.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.718+0000 7f1e94087b80 1 --2- 172.21.15.12:0/460973957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f285d49b80 0x55f285d49f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:50.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.718+0000 7f1e94087b80 1 -- 172.21.15.12:0/460973957 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f285c78150 con 0x55f285d49b80 2024-05-17T22:47:50.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.718+0000 7f1e94087b80 1 -- 172.21.15.12:0/460973957 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f285c67cf0 con 0x55f285bcba70 2024-05-17T22:47:50.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.718+0000 7f1e94087b80 1 -- 172.21.15.12:0/460973957 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f285c6bd30 con 0x55f285d4f480 2024-05-17T22:47:50.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.718+0000 7f1e7cff9700 1 -- 172.21.15.12:0/2101656506 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1e68009d10 con 0x7f1e8010a830 2024-05-17T22:47:50.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.719+0000 7f1e7f7fe700 1 -- 172.21.15.12:0/460973957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f285d4f480 msgr2=0x55f285d49640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:47:50.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.719+0000 7f1e7f7fe700 1 --2- 172.21.15.12:0/460973957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f285d4f480 0x55f285d49640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:50.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.719+0000 7f1e7f7fe700 1 -- 172.21.15.12:0/460973957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f285d49b80 msgr2=0x55f285d49f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T22:47:50.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.719+0000 7f1e7f7fe700 1 --2- 172.21.15.12:0/460973957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f285d49b80 0x55f285d49f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:50.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.719+0000 7f1e7f7fe700 1 -- 172.21.15.12:0/460973957 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f285c01640 con 0x55f285bcba70 2024-05-17T22:47:50.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.720+0000 7f1e7cff9700 1 --2- 172.21.15.12:0/2101656506 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f1e6c0b2a50 0x7f1e6c0b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:50.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.720+0000 7f1e7f7fe700 1 --2- 172.21.15.12:0/460973957 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f285bcba70 0x55f285d4ef40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1e78009cb0 tx=0x7f1e780103d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cadd126dc461b67 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:47:50.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.720+0000 7f1e7effd700 1 --2- 172.21.15.12:0/2101656506 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f1e6c0b2a50 0x7f1e6c0b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:50.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.720+0000 7f1e7cff9700 1 -- 172.21.15.12:0/2101656506 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f1e680e3080 con 0x7f1e8010a830 2024-05-17T22:47:50.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.721+0000 7f1e757fa700 1 -- 172.21.15.12:0/460973957 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e78010c50 con 0x55f285bcba70 2024-05-17T22:47:50.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.721+0000 7f1e757fa700 1 -- 172.21.15.12:0/460973957 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e78010df0 con 0x55f285bcba70 2024-05-17T22:47:50.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.721+0000 7f1e757fa700 1 -- 172.21.15.12:0/460973957 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e78019720 con 0x55f285bcba70 2024-05-17T22:47:50.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.726+0000 7f1e94087b80 1 -- 172.21.15.12:0/460973957 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f285bcba70 msgr2=0x55f285d4ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:50.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.726+0000 7f1e94087b80 1 --2- 172.21.15.12:0/460973957 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f285bcba70 0x55f285d4ef40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1e78009cb0 tx=0x7f1e780103d0 comp rx=0 tx=0).stop 2024-05-17T22:47:50.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.729+0000 7f1e7effd700 1 --2- 172.21.15.12:0/2101656506 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f1e6c0b2a50 0x7f1e6c0b4f60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1e680aa2d0 tx=0x7f1e680e5040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:47:50.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.730+0000 7f1e94087b80 1 -- 172.21.15.12:0/460973957 shutdown_connections 2024-05-17T22:47:50.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.730+0000 7f1e94087b80 1 --2- 172.21.15.12:0/460973957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f285d49b80 0x55f285d49f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:50.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.730+0000 7f1e94087b80 1 --2- 172.21.15.12:0/460973957 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f285bcba70 0x55f285d4ef40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:50.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.730+0000 7f1e94087b80 1 --2- 172.21.15.12:0/460973957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f285d4f480 0x55f285d49640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:50.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.730+0000 7f1e94087b80 1 -- 172.21.15.12:0/460973957 >> 172.21.15.12:0/460973957 conn(0x55f285c2b850 msgr2=0x55f285d44240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:47:50.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.734+0000 7f1e94087b80 1 -- 172.21.15.12:0/460973957 shutdown_connections 2024-05-17T22:47:50.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.735+0000 7f1e94087b80 1 -- 172.21.15.12:0/460973957 wait complete. 2024-05-17T22:47:50.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.735+0000 7f1e94087b80 1 -- start start 2024-05-17T22:47:50.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.736+0000 7f1e94087b80 1 Processor -- start 2024-05-17T22:47:50.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.739+0000 7f1e94087b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f285bcba70 0x55f285d52ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:50.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.739+0000 7f1e94087b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f285d49b80 0x55f285d53010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:50.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.739+0000 7f1e94087b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f285d4f480 0x55f285d53550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:50.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.739+0000 7f1e94087b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f285b6f1d0 con 0x55f285d4f480 2024-05-17T22:47:50.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.739+0000 7f1e94087b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f285c099c0 con 0x55f285d49b80 2024-05-17T22:47:50.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.739+0000 7f1e94087b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f285d55e50 con 0x55f285bcba70 2024-05-17T22:47:50.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.739+0000 7f1e94087b80 10 client.0 _ll_register_callbacks cb 0x55f285bfebd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:47:50.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.740+0000 7f1e767fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f285d49b80 0x55f285d53010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:50.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.740+0000 7f1e767fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f285d49b80 0x55f285d53010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:46562/0 (socket says 172.21.15.12:46562) 2024-05-17T22:47:50.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.740+0000 7f1e767fc700 1 -- 172.21.15.12:0/3225641758 learned_addr learned my addr 172.21.15.12:0/3225641758 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:47:50.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.741+0000 7f1e767fc700 1 -- 172.21.15.12:0/3225641758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f285bcba70 msgr2=0x55f285d52ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:47:50.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.741+0000 7f1e767fc700 1 --2- 172.21.15.12:0/3225641758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f285bcba70 0x55f285d52ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:50.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.741+0000 7f1e767fc700 1 -- 172.21.15.12:0/3225641758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f285d4f480 msgr2=0x55f285d53550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:47:50.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.741+0000 7f1e767fc700 1 --2- 172.21.15.12:0/3225641758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f285d4f480 0x55f285d53550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:50.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.741+0000 7f1e767fc700 1 -- 172.21.15.12:0/3225641758 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f285bb8500 con 0x55f285d49b80 2024-05-17T22:47:50.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.742+0000 7f1e767fc700 1 --2- 172.21.15.12:0/3225641758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f285d49b80 0x55f285d53010 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1e700077b0 tx=0x7f1e70007b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39713412885f4311 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:47:50.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.743+0000 7f1e667fc700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:47:50.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.743+0000 7f1e667fc700 1 -- 172.21.15.12:0/3225641758 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70019020 con 0x55f285d49b80 2024-05-17T22:47:50.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.743+0000 7f1e94087b80 1 -- 172.21.15.12:0/3225641758 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e78010df0 con 0x55f285d49b80 2024-05-17T22:47:50.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.743+0000 7f1e667fc700 1 -- 172.21.15.12:0/3225641758 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f1e7000d780 con 0x55f285d49b80 2024-05-17T22:47:50.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.743+0000 7f1e667fc700 1 -- 172.21.15.12:0/3225641758 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1e70021050 con 0x55f285d49b80 2024-05-17T22:47:50.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.743+0000 7f1e94087b80 10 client.25757 fetch_fsmap learned FSMap version 39 2024-05-17T22:47:50.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.743+0000 7f1e94087b80 1 -- 172.21.15.12:0/3225641758 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f285d71a00 con 0x55f285d49b80 2024-05-17T22:47:50.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.748+0000 7f1e667fc700 1 -- 172.21.15.12:0/3225641758 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e78010df0 con 0x55f285d49b80 2024-05-17T22:47:50.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.748+0000 7f1e667fc700 1 -- 172.21.15.12:0/3225641758 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f1e70015c00 con 0x55f285d49b80 2024-05-17T22:47:50.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.748+0000 7f1e94087b80 10 client.25757 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:47:50.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.748+0000 7f1e94087b80 10 client.25757 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:47:50.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.748+0000 7f1e94087b80 20 client.25757 populate_metadata read hostname 'smithi012' 2024-05-17T22:47:50.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.748+0000 7f1e94087b80 1 --2- 172.21.15.12:0/3225641758 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55f285d74400 0x55f285d76910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:50.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.748+0000 7f1e94087b80 4 client.25757 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:47:50.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.748+0000 7f1e94087b80 1 -- 172.21.15.12:0/3225641758 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f285c7a0a0 con 0x55f285d74400 2024-05-17T22:47:50.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.755+0000 7f1e7f7fe700 1 --2- 172.21.15.12:0/3225641758 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55f285d74400 0x55f285d76910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:50.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.767+0000 7f1e667fc700 10 client.25757 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:47:50.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.767+0000 7f1e7f7fe700 1 --2- 172.21.15.12:0/3225641758 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55f285d74400 0x55f285d76910 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d37a499cb51b852 server_cookie=e5b17b12374b86f2 in_seq=0 out_seq=0 2024-05-17T22:47:50.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.769+0000 7f1e667fc700 1 -- 172.21.15.12:0/3225641758 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f285c7a0a0 con 0x55f285d74400 2024-05-17T22:47:50.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.769+0000 7f1e667fc700 10 client.25757 handle_command_reply: tid=0 2024-05-17T22:47:50.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.769+0000 7f1e94087b80 1 client.25757 shutdown 2024-05-17T22:47:50.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.771+0000 7f1e94087b80 1 -- 172.21.15.12:0/3225641758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f285d49b80 msgr2=0x55f285d53010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:50.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.771+0000 7f1e94087b80 1 --2- 172.21.15.12:0/3225641758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f285d49b80 0x55f285d53010 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1e700077b0 tx=0x7f1e70007b70 comp rx=0 tx=0).stop 2024-05-17T22:47:50.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.771+0000 7f1e94087b80 1 -- 172.21.15.12:0/3225641758 shutdown_connections 2024-05-17T22:47:50.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.771+0000 7f1e94087b80 1 --2- 172.21.15.12:0/3225641758 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55f285d74400 0x55f285d76910 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:50.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.771+0000 7f1e94087b80 1 --2- 172.21.15.12:0/3225641758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f285d4f480 0x55f285d53550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:50.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.771+0000 7f1e94087b80 1 --2- 172.21.15.12:0/3225641758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f285bcba70 0x55f285d52ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:50.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.771+0000 7f1e94087b80 1 --2- 172.21.15.12:0/3225641758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f285d49b80 0x55f285d53010 secure :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1e700077b0 tx=0x7f1e70007b70 comp rx=0 tx=0).stop 2024-05-17T22:47:50.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.771+0000 7f1e94087b80 1 -- 172.21.15.12:0/3225641758 >> 172.21.15.12:0/3225641758 conn(0x55f285c2b850 msgr2=0x55f285d441e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:47:50.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.771+0000 7f1e667fc700 0 client.25757 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:47:50.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.771+0000 7f1e94087b80 1 -- 172.21.15.12:0/3225641758 shutdown_connections 2024-05-17T22:47:50.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.772+0000 7f1e94087b80 1 -- 172.21.15.12:0/3225641758 wait complete. 2024-05-17T22:47:50.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.772+0000 7f1e94087b80 20 client.25757 trim_cache size 0 max 16384 2024-05-17T22:47:50.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.817+0000 7f1e94087b80 1 Processor -- start 2024-05-17T22:47:50.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.817+0000 7f1e94087b80 1 -- start start 2024-05-17T22:47:50.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.818+0000 7f1e94087b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f285bcba70 0x55f285d55360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:50.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.818+0000 7f1e94087b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f285d4f480 0x55f285d3eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:50.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.818+0000 7f1e7f7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f285bcba70 0x55f285d55360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:50.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.818+0000 7f1e7f7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f285bcba70 0x55f285d55360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:46576/0 (socket says 172.21.15.12:46576) 2024-05-17T22:47:50.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.818+0000 7f1e7f7fe700 1 -- 172.21.15.12:0/1229493307 learned_addr learned my addr 172.21.15.12:0/1229493307 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:47:50.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.818+0000 7f1e94087b80 1 --2- 172.21.15.12:0/1229493307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f285d5edf0 0x55f285d533a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:50.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.818+0000 7f1e94087b80 1 -- 172.21.15.12:0/1229493307 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1e600114d0 con 0x55f285d4f480 2024-05-17T22:47:50.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.818+0000 7f1e94087b80 1 -- 172.21.15.12:0/1229493307 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6000ed60 con 0x55f285bcba70 2024-05-17T22:47:50.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.818+0000 7f1e94087b80 1 -- 172.21.15.12:0/1229493307 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6000cbb0 con 0x55f285d5edf0 2024-05-17T22:47:50.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.820+0000 7f1e767fc700 1 --2- 172.21.15.12:0/1229493307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f285d4f480 0x55f285d3eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:50.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.822+0000 7f1e7f7fe700 1 --2- 172.21.15.12:0/1229493307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f285d5edf0 0x55f285d533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:50.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.823+0000 7f1e7f7fe700 1 -- 172.21.15.12:0/1229493307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f285d5edf0 msgr2=0x55f285d533a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:47:50.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.823+0000 7f1e7f7fe700 1 --2- 172.21.15.12:0/1229493307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f285d5edf0 0x55f285d533a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:50.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.823+0000 7f1e7f7fe700 1 -- 172.21.15.12:0/1229493307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f285d4f480 msgr2=0x55f285d3eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:47:50.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.823+0000 7f1e7f7fe700 1 --2- 172.21.15.12:0/1229493307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f285d4f480 0x55f285d3eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:50.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.823+0000 7f1e7f7fe700 1 -- 172.21.15.12:0/1229493307 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f285d4e9b0 con 0x55f285bcba70 2024-05-17T22:47:50.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.831+0000 7f1e7f7fe700 1 --2- 172.21.15.12:0/1229493307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f285bcba70 0x55f285d55360 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1e78011ec0 tx=0x7f1e78011e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7235c0760507c181 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:47:50.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.837+0000 7f1e74ff9700 1 -- 172.21.15.12:0/1229493307 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7802c070 con 0x55f285bcba70 2024-05-17T22:47:50.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.837+0000 7f1e74ff9700 1 -- 172.21.15.12:0/1229493307 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e78010d70 con 0x55f285bcba70 2024-05-17T22:47:50.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.837+0000 7f1e74ff9700 1 -- 172.21.15.12:0/1229493307 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7801f040 con 0x55f285bcba70 2024-05-17T22:47:50.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.840+0000 7f1e94087b80 1 -- 172.21.15.12:0/1229493307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f285bcba70 msgr2=0x55f285d55360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:50.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.840+0000 7f1e94087b80 1 --2- 172.21.15.12:0/1229493307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f285bcba70 0x55f285d55360 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1e78011ec0 tx=0x7f1e78011e50 comp rx=0 tx=0).stop 2024-05-17T22:47:50.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.840+0000 7f1e94087b80 1 -- 172.21.15.12:0/1229493307 shutdown_connections 2024-05-17T22:47:50.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.840+0000 7f1e94087b80 1 --2- 172.21.15.12:0/1229493307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f285d5edf0 0x55f285d533a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:50.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.840+0000 7f1e94087b80 1 --2- 172.21.15.12:0/1229493307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f285bcba70 0x55f285d55360 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:50.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.840+0000 7f1e94087b80 1 --2- 172.21.15.12:0/1229493307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f285d4f480 0x55f285d3eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:50.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.840+0000 7f1e94087b80 1 -- 172.21.15.12:0/1229493307 >> 172.21.15.12:0/1229493307 conn(0x55f285c2b850 msgr2=0x55f285d51a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:47:50.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.841+0000 7f1e94087b80 1 -- 172.21.15.12:0/1229493307 shutdown_connections 2024-05-17T22:47:50.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.841+0000 7f1e94087b80 1 -- 172.21.15.12:0/1229493307 wait complete. 2024-05-17T22:47:50.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.841+0000 7f1e94087b80 1 -- start start 2024-05-17T22:47:50.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.842+0000 7f1e94087b80 1 Processor -- start 2024-05-17T22:47:50.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.842+0000 7f1e94087b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f285bcba70 0x55f285d5b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:50.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.842+0000 7f1e94087b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f285d4f480 0x55f285d57b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:50.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.842+0000 7f1e94087b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f285d5edf0 0x55f285d581c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:50.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.842+0000 7f1e94087b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6000d220 con 0x55f285d4f480 2024-05-17T22:47:50.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.842+0000 7f1e94087b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6000e690 con 0x55f285bcba70 2024-05-17T22:47:50.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.842+0000 7f1e94087b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6000d8f0 con 0x55f285d5edf0 2024-05-17T22:47:50.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.842+0000 7f1e94087b80 10 client.0 _ll_register_callbacks cb 0x55f285bfebd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:47:50.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.843+0000 7f1e7f7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f285bcba70 0x55f285d5b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:50.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.843+0000 7f1e7f7fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f285d5edf0 0x55f285d581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:50.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.843+0000 7f1e7f7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f285bcba70 0x55f285d5b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:46588/0 (socket says 172.21.15.12:46588) 2024-05-17T22:47:50.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.843+0000 7f1e7f7fe700 1 -- 172.21.15.12:0/3708065273 learned_addr learned my addr 172.21.15.12:0/3708065273 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:47:50.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.845+0000 7f1e767fc700 1 --2- 172.21.15.12:0/3708065273 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f285d4f480 0x55f285d57b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:50.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.846+0000 7f1e767fc700 1 -- 172.21.15.12:0/3708065273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f285d5edf0 msgr2=0x55f285d581c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:47:50.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.846+0000 7f1e767fc700 1 --2- 172.21.15.12:0/3708065273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f285d5edf0 0x55f285d581c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:50.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.846+0000 7f1e767fc700 1 -- 172.21.15.12:0/3708065273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f285bcba70 msgr2=0x55f285d5b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:47:50.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.846+0000 7f1e767fc700 1 --2- 172.21.15.12:0/3708065273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f285bcba70 0x55f285d5b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:50.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.846+0000 7f1e767fc700 1 -- 172.21.15.12:0/3708065273 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f285d58900 con 0x55f285d4f480 2024-05-17T22:47:50.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.846+0000 7f1e767fc700 1 --2- 172.21.15.12:0/3708065273 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f285d4f480 0x55f285d57b20 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x55f285d53bc0 tx=0x7f1e70006070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=add2211845af986c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:47:50.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.847+0000 7f1e667fc700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:47:50.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.849+0000 7f1e667fc700 1 -- 172.21.15.12:0/3708065273 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70019070 con 0x55f285d4f480 2024-05-17T22:47:50.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.849+0000 7f1e667fc700 1 -- 172.21.15.12:0/3708065273 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f1e700147e0 con 0x55f285d4f480 2024-05-17T22:47:50.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.850+0000 7f1e94087b80 1 -- 172.21.15.12:0/3708065273 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e78010d70 con 0x55f285d4f480 2024-05-17T22:47:50.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.853+0000 7f1e667fc700 1 -- 172.21.15.12:0/3708065273 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1e70014a00 con 0x55f285d4f480 2024-05-17T22:47:50.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.853+0000 7f1e94087b80 10 client.16074 fetch_fsmap learned FSMap version 39 2024-05-17T22:47:50.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.853+0000 7f1e94087b80 1 -- 172.21.15.12:0/3708065273 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f285ddfd50 con 0x55f285d4f480 2024-05-17T22:47:50.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.857+0000 7f1e667fc700 1 -- 172.21.15.12:0/3708065273 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e78010d70 con 0x55f285d4f480 2024-05-17T22:47:50.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.857+0000 7f1e667fc700 1 -- 172.21.15.12:0/3708065273 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f1e700050b0 con 0x55f285d4f480 2024-05-17T22:47:50.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.857+0000 7f1e94087b80 10 client.16074 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:47:50.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.857+0000 7f1e94087b80 10 client.16074 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:47:50.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.857+0000 7f1e94087b80 20 client.16074 populate_metadata read hostname 'smithi012' 2024-05-17T22:47:50.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.857+0000 7f1e94087b80 1 --2- 172.21.15.12:0/3708065273 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55f285de03b0 0x55f285de27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:50.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.857+0000 7f1e94087b80 4 client.16074 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-17T22:47:50.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.857+0000 7f1e94087b80 1 -- 172.21.15.12:0/3708065273 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f285c73880 con 0x55f285de03b0 2024-05-17T22:47:50.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.861+0000 7f1e7f7fe700 1 --2- 172.21.15.12:0/3708065273 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55f285de03b0 0x55f285de27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:50.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.863+0000 7f1e7f7fe700 1 --2- 172.21.15.12:0/3708065273 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55f285de03b0 0x55f285de27d0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd5e188a63c5cd90 server_cookie=ed805720d3b2d84b in_seq=0 out_seq=0 2024-05-17T22:47:50.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.863+0000 7f1e667fc700 10 client.16074 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:47:50.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.864+0000 7f1e667fc700 1 -- 172.21.15.12:0/3708065273 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f285c73880 con 0x55f285de03b0 2024-05-17T22:47:50.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.864+0000 7f1e667fc700 10 client.16074 handle_command_reply: tid=0 2024-05-17T22:47:50.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.864+0000 7f1e94087b80 1 client.16074 shutdown 2024-05-17T22:47:50.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.866+0000 7f1e94087b80 1 -- 172.21.15.12:0/3708065273 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f285d4f480 msgr2=0x55f285d57b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:50.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.866+0000 7f1e94087b80 1 --2- 172.21.15.12:0/3708065273 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f285d4f480 0x55f285d57b20 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x55f285d53bc0 tx=0x7f1e70006070 comp rx=0 tx=0).stop 2024-05-17T22:47:50.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.866+0000 7f1e94087b80 1 -- 172.21.15.12:0/3708065273 shutdown_connections 2024-05-17T22:47:50.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.866+0000 7f1e94087b80 1 --2- 172.21.15.12:0/3708065273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f285d5edf0 0x55f285d581c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:50.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.866+0000 7f1e94087b80 1 --2- 172.21.15.12:0/3708065273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f285bcba70 0x55f285d5b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:50.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.866+0000 7f1e94087b80 1 --2- 172.21.15.12:0/3708065273 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55f285de03b0 0x55f285de27d0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:50.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.866+0000 7f1e94087b80 1 --2- 172.21.15.12:0/3708065273 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f285d4f480 0x55f285d57b20 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:50.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.866+0000 7f1e94087b80 1 -- 172.21.15.12:0/3708065273 >> 172.21.15.12:0/3708065273 conn(0x55f285c2b850 msgr2=0x55f285d3e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:47:50.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.866+0000 7f1e667fc700 0 client.16074 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:47:50.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.867+0000 7f1e94087b80 1 -- 172.21.15.12:0/3708065273 shutdown_connections 2024-05-17T22:47:50.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.867+0000 7f1e94087b80 1 -- 172.21.15.12:0/3708065273 wait complete. 2024-05-17T22:47:50.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.867+0000 7f1e94087b80 20 client.16074 trim_cache size 0 max 16384 2024-05-17T22:47:50.871 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:47:50.871 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-17T22:47:50.871 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "715b0151-ef4c-4701-ba99-d9650a623cbe", 2024-05-17T22:47:50.871 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-17T22:47:50.871 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:47:50.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.868+0000 7f1e856ac700 1 -- 172.21.15.12:0/2101656506 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f1e6c0b2a50 msgr2=0x7f1e6c0b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:50.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.868+0000 7f1e856ac700 1 --2- 172.21.15.12:0/2101656506 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f1e6c0b2a50 0x7f1e6c0b4f60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1e680aa2d0 tx=0x7f1e680e5040 comp rx=0 tx=0).stop 2024-05-17T22:47:50.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.868+0000 7f1e856ac700 1 -- 172.21.15.12:0/2101656506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e8010a830 msgr2=0x7f1e801a95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:50.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.868+0000 7f1e856ac700 1 --2- 172.21.15.12:0/2101656506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e8010a830 0x7f1e801a95e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1e68006010 tx=0x7f1e6800b650 comp rx=0 tx=0).stop 2024-05-17T22:47:50.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.868+0000 7f1e856ac700 1 -- 172.21.15.12:0/2101656506 shutdown_connections 2024-05-17T22:47:50.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.868+0000 7f1e856ac700 1 --2- 172.21.15.12:0/2101656506 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f1e6c0b2a50 0x7f1e6c0b4f60 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:50.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.868+0000 7f1e856ac700 1 --2- 172.21.15.12:0/2101656506 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e8011d550 0x7f1e801ade10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:50.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.868+0000 7f1e856ac700 1 --2- 172.21.15.12:0/2101656506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e8010a830 0x7f1e801a95e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:50.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.868+0000 7f1e856ac700 1 --2- 172.21.15.12:0/2101656506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e8010b180 0x7f1e801a9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:50.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.868+0000 7f1e856ac700 1 -- 172.21.15.12:0/2101656506 >> 172.21.15.12:0/2101656506 conn(0x7f1e8006efc0 msgr2=0x7f1e8011b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:47:50.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.868+0000 7f1e856ac700 1 -- 172.21.15.12:0/2101656506 shutdown_connections 2024-05-17T22:47:50.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:50.868+0000 7f1e856ac700 1 -- 172.21.15.12:0/2101656506 wait complete. 2024-05-17T22:47:51.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 510.0 s, 836.6 tps, lat 117.590 ms stddev 124.714 2024-05-17T22:47:51.702 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-17T22:47:51.974 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:51 smithi012 ceph-mon[31632]: pgmap v697: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.9 MiB/s wr, 503 op/s 2024-05-17T22:47:51.974 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:51 smithi012 ceph-mon[31632]: scrub queued for path: / 2024-05-17T22:47:51.974 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:51 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:47:51.974 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:51 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:47:51.974 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:51 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:47:51.974 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:51 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:47:51.975 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:51 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:47:51.975 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:51 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:47:52.228 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:47:52.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:51 smithi151 ceph-mon[36101]: pgmap v697: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.9 MiB/s wr, 503 op/s 2024-05-17T22:47:52.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:51 smithi151 ceph-mon[36101]: scrub queued for path: / 2024-05-17T22:47:52.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:51 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:47:52.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:51 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:47:52.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:51 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:47:52.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:51 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:47:52.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:51 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:47:52.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:51 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:47:52.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:51 smithi096 ceph-mon[36065]: pgmap v697: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.9 MiB/s wr, 503 op/s 2024-05-17T22:47:52.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:51 smithi096 ceph-mon[36065]: scrub queued for path: / 2024-05-17T22:47:52.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:51 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:47:52.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:51 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:47:52.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:51 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:47:52.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:51 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:47:52.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:51 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:47:52.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:51 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:47:53.030 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:52 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:47:53.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:52 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:47:53.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:52 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:47:54.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:53 smithi151 ceph-mon[36101]: pgmap v698: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 5.1 MiB/s wr, 531 op/s 2024-05-17T22:47:54.259 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:53 smithi012 ceph-mon[31632]: pgmap v698: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 5.1 MiB/s wr, 531 op/s 2024-05-17T22:47:54.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:53 smithi096 ceph-mon[36065]: pgmap v698: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 5.1 MiB/s wr, 531 op/s 2024-05-17T22:47:54.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.437+0000 7f85b6b12700 1 -- 172.21.15.12:0/1076069989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85b010b180 msgr2=0x7f85b011d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:54.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.437+0000 7f85b6b12700 1 --2- 172.21.15.12:0/1076069989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85b010b180 0x7f85b011d010 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f85a40098f0 tx=0x7f85a4009c00 comp rx=0 tx=0).stop 2024-05-17T22:47:54.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.437+0000 7f85b6b12700 1 -- 172.21.15.12:0/1076069989 shutdown_connections 2024-05-17T22:47:54.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.437+0000 7f85b6b12700 1 --2- 172.21.15.12:0/1076069989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85b011d550 0x7f85b011f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:54.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.437+0000 7f85b6b12700 1 --2- 172.21.15.12:0/1076069989 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85b010a830 0x7f85b010ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:54.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.437+0000 7f85b6b12700 1 --2- 172.21.15.12:0/1076069989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85b010b180 0x7f85b011d010 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:54.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.437+0000 7f85b6b12700 1 -- 172.21.15.12:0/1076069989 >> 172.21.15.12:0/1076069989 conn(0x7f85b006efc0 msgr2=0x7f85b006f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:47:54.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.437+0000 7f85b6b12700 1 -- 172.21.15.12:0/1076069989 shutdown_connections 2024-05-17T22:47:54.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.437+0000 7f85b6b12700 1 -- 172.21.15.12:0/1076069989 wait complete. 2024-05-17T22:47:54.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.439+0000 7f85b6b12700 1 Processor -- start 2024-05-17T22:47:54.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.442+0000 7f85b6b12700 1 -- start start 2024-05-17T22:47:54.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.443+0000 7f85b6b12700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85b010a830 0x7f85b011ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:54.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.443+0000 7f85b6b12700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85b010b180 0x7f85b0118f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:54.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.443+0000 7f85b6b12700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85b011d550 0x7f85b0119510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:54.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.443+0000 7f85b6b12700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f85b0114f40 con 0x7f85b011d550 2024-05-17T22:47:54.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.443+0000 7f85b6b12700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f85b0114dc0 con 0x7f85b010a830 2024-05-17T22:47:54.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.443+0000 7f85b6b12700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f85b01150c0 con 0x7f85b010b180 2024-05-17T22:47:54.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.443+0000 7f85b48ae700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85b010a830 0x7f85b011ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:54.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.443+0000 7f85b50af700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85b011d550 0x7f85b0119510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:54.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.443+0000 7f85b50af700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85b011d550 0x7f85b0119510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44080/0 (socket says 172.21.15.12:44080) 2024-05-17T22:47:54.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.443+0000 7f85b50af700 1 -- 172.21.15.12:0/400930132 learned_addr learned my addr 172.21.15.12:0/400930132 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:47:54.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.444+0000 7f85b50af700 1 -- 172.21.15.12:0/400930132 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85b010b180 msgr2=0x7f85b0118f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:54.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.444+0000 7f85b50af700 1 --2- 172.21.15.12:0/400930132 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85b010b180 0x7f85b0118f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:54.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.444+0000 7f85b50af700 1 -- 172.21.15.12:0/400930132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85b010a830 msgr2=0x7f85b011ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:54.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.444+0000 7f85b50af700 1 --2- 172.21.15.12:0/400930132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85b010a830 0x7f85b011ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:54.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.445+0000 7f85b50af700 1 -- 172.21.15.12:0/400930132 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b0119d40 con 0x7f85b011d550 2024-05-17T22:47:54.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.446+0000 7f85b50af700 1 --2- 172.21.15.12:0/400930132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85b011d550 0x7f85b0119510 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f85a800ada0 tx=0x7f85a800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:47:54.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.446+0000 7f85adffb700 1 -- 172.21.15.12:0/400930132 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a8017020 con 0x7f85b011d550 2024-05-17T22:47:54.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.447+0000 7f85b6b12700 1 -- 172.21.15.12:0/400930132 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85b01cf320 con 0x7f85b011d550 2024-05-17T22:47:54.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.447+0000 7f85b6b12700 1 -- 172.21.15.12:0/400930132 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85b01cf830 con 0x7f85b011d550 2024-05-17T22:47:54.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.447+0000 7f85adffb700 1 -- 172.21.15.12:0/400930132 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85a800ce30 con 0x7f85b011d550 2024-05-17T22:47:54.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.447+0000 7f85adffb700 1 -- 172.21.15.12:0/400930132 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a80054a0 con 0x7f85b011d550 2024-05-17T22:47:54.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.457+0000 7f85c54edb80 1 Processor -- start 2024-05-17T22:47:54.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.457+0000 7f85c54edb80 1 -- start start 2024-05-17T22:47:54.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.458+0000 7f85c54edb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56485f5c59f0 0x56485f748ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:54.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.458+0000 7f85c54edb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56485f749400 0x56485f7435c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:54.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.458+0000 7f85c54edb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485f743b00 0x56485f743f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:54.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.458+0000 7f85c54edb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56485f6720d0 con 0x56485f749400 2024-05-17T22:47:54.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.458+0000 7f85c54edb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56485f661c70 con 0x56485f743b00 2024-05-17T22:47:54.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.458+0000 7f85c54edb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56485f665cb0 con 0x56485f5c59f0 2024-05-17T22:47:54.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.458+0000 7f85adffb700 1 -- 172.21.15.12:0/400930132 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f85a8004ad0 con 0x7f85b011d550 2024-05-17T22:47:54.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.459+0000 7f85adffb700 1 --2- 172.21.15.12:0/400930132 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f85980b2a80 0x7f85980b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:54.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.459+0000 7f85b50af700 1 --2- 172.21.15.12:0/400930132 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f85980b2a80 0x7f85980b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:54.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.460+0000 7f85adffb700 1 -- 172.21.15.12:0/400930132 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f85a80e1b90 con 0x7f85b011d550 2024-05-17T22:47:54.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.460+0000 7f85b48ae700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56485f749400 0x56485f7435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:54.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.461+0000 7f85b48ae700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56485f749400 0x56485f7435c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44082/0 (socket says 172.21.15.12:44082) 2024-05-17T22:47:54.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.461+0000 7f85b48ae700 1 -- 172.21.15.12:0/507395110 learned_addr learned my addr 172.21.15.12:0/507395110 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:47:54.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.461+0000 7f85b48ae700 1 -- 172.21.15.12:0/507395110 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56485f5c59f0 msgr2=0x56485f748ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T22:47:54.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.461+0000 7f85b48ae700 1 --2- 172.21.15.12:0/507395110 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56485f5c59f0 0x56485f748ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:54.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.461+0000 7f85b48ae700 1 -- 172.21.15.12:0/507395110 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485f743b00 msgr2=0x56485f743f10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T22:47:54.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.461+0000 7f85b48ae700 1 --2- 172.21.15.12:0/507395110 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485f743b00 0x56485f743f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:54.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.461+0000 7f85b48ae700 1 -- 172.21.15.12:0/507395110 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56485f5fb5c0 con 0x56485f749400 2024-05-17T22:47:54.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.462+0000 7f85b48ae700 1 --2- 172.21.15.12:0/507395110 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56485f749400 0x56485f7435c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f85a001ad20 tx=0x7f85a000d5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b5b00bf59b189c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:47:54.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.464+0000 7f85b50af700 1 --2- 172.21.15.12:0/400930132 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f85980b2a80 0x7f85980b4f90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f85a80a60c0 tx=0x7f85a80a5510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:47:54.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.466+0000 7f8596ffd700 1 -- 172.21.15.12:0/507395110 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a0014030 con 0x56485f749400 2024-05-17T22:47:54.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.467+0000 7f8596ffd700 1 -- 172.21.15.12:0/507395110 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85a000dbd0 con 0x56485f749400 2024-05-17T22:47:54.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.468+0000 7f85c54edb80 1 -- 172.21.15.12:0/507395110 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56485f749400 msgr2=0x56485f7435c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:54.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.468+0000 7f85c54edb80 1 --2- 172.21.15.12:0/507395110 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56485f749400 0x56485f7435c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f85a001ad20 tx=0x7f85a000d5c0 comp rx=0 tx=0).stop 2024-05-17T22:47:54.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.474+0000 7f85c54edb80 1 -- 172.21.15.12:0/507395110 shutdown_connections 2024-05-17T22:47:54.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.474+0000 7f85c54edb80 1 --2- 172.21.15.12:0/507395110 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485f743b00 0x56485f743f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:54.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.474+0000 7f85c54edb80 1 --2- 172.21.15.12:0/507395110 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56485f5c59f0 0x56485f748ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:54.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.474+0000 7f85c54edb80 1 --2- 172.21.15.12:0/507395110 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56485f749400 0x56485f7435c0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:54.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.474+0000 7f85c54edb80 1 -- 172.21.15.12:0/507395110 >> 172.21.15.12:0/507395110 conn(0x56485f6257d0 msgr2=0x56485f73e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:47:54.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.476+0000 7f85c54edb80 1 -- 172.21.15.12:0/507395110 shutdown_connections 2024-05-17T22:47:54.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.478+0000 7f85c54edb80 1 -- 172.21.15.12:0/507395110 wait complete. 2024-05-17T22:47:54.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.479+0000 7f85c54edb80 1 -- start start 2024-05-17T22:47:54.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.479+0000 7f85c54edb80 1 Processor -- start 2024-05-17T22:47:54.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.479+0000 7f85c54edb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56485f5c59f0 0x56485f74c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:54.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.479+0000 7f85c54edb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485f743b00 0x56485f74cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:54.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.480+0000 7f85c54edb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56485f749400 0x56485f74d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:54.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.480+0000 7f85c54edb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56485f569150 con 0x56485f5c59f0 2024-05-17T22:47:54.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.480+0000 7f85c54edb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56485f603940 con 0x56485f743b00 2024-05-17T22:47:54.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.480+0000 7f85c54edb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56485f74fe20 con 0x56485f749400 2024-05-17T22:47:54.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.480+0000 7f85c54edb80 10 client.0 _ll_register_callbacks cb 0x56485f5f8b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:47:54.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.482+0000 7f85b48ae700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485f743b00 0x56485f74cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:54.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.483+0000 7f85b48ae700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485f743b00 0x56485f74cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:46618/0 (socket says 172.21.15.12:46618) 2024-05-17T22:47:54.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.483+0000 7f85b48ae700 1 -- 172.21.15.12:0/3217739002 learned_addr learned my addr 172.21.15.12:0/3217739002 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:47:54.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.483+0000 7f85affff700 1 --2- 172.21.15.12:0/3217739002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56485f749400 0x56485f74d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:54.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.485+0000 7f85b48ae700 1 -- 172.21.15.12:0/3217739002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56485f749400 msgr2=0x56485f74d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:47:54.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.485+0000 7f85b48ae700 1 --2- 172.21.15.12:0/3217739002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56485f749400 0x56485f74d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:54.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.485+0000 7f85b48ae700 1 -- 172.21.15.12:0/3217739002 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56485f5c59f0 msgr2=0x56485f74c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:47:54.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.485+0000 7f85b48ae700 1 --2- 172.21.15.12:0/3217739002 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56485f5c59f0 0x56485f74c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:54.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.485+0000 7f85b48ae700 1 -- 172.21.15.12:0/3217739002 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56485f5b2480 con 0x56485f743b00 2024-05-17T22:47:54.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.505+0000 7f85b48ae700 1 --2- 172.21.15.12:0/3217739002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485f743b00 0x56485f74cf00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f85a00058b0 tx=0x7f85a000dfb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cc64f664bf4c64c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:47:54.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.507+0000 7f858ffff700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:47:54.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.507+0000 7f858ffff700 1 -- 172.21.15.12:0/3217739002 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a0014030 con 0x56485f743b00 2024-05-17T22:47:54.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.507+0000 7f858ffff700 1 -- 172.21.15.12:0/3217739002 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f85a00052c0 con 0x56485f743b00 2024-05-17T22:47:54.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.507+0000 7f85c54edb80 1 -- 172.21.15.12:0/3217739002 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85a000dbd0 con 0x56485f743b00 2024-05-17T22:47:54.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.511+0000 7f858ffff700 1 -- 172.21.15.12:0/3217739002 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f85a0032050 con 0x56485f743b00 2024-05-17T22:47:54.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.512+0000 7f85c54edb80 10 client.25772 fetch_fsmap learned FSMap version 39 2024-05-17T22:47:54.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.512+0000 7f85c54edb80 1 -- 172.21.15.12:0/3217739002 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56485f75a8f0 con 0x56485f743b00 2024-05-17T22:47:54.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.515+0000 7f858ffff700 1 -- 172.21.15.12:0/3217739002 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85a0024070 con 0x56485f743b00 2024-05-17T22:47:54.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.515+0000 7f858ffff700 1 -- 172.21.15.12:0/3217739002 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f85a00185e0 con 0x56485f743b00 2024-05-17T22:47:54.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.518+0000 7f85c54edb80 10 client.25772 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:47:54.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.518+0000 7f85c54edb80 10 client.25772 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:47:54.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.518+0000 7f85c54edb80 20 client.25772 populate_metadata read hostname 'smithi012' 2024-05-17T22:47:54.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.518+0000 7f85c54edb80 1 --2- 172.21.15.12:0/3217739002 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x56485f75ce20 0x56485f75f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:54.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.518+0000 7f85c54edb80 4 client.25772 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:47:54.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.518+0000 7f85c54edb80 1 -- 172.21.15.12:0/3217739002 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f85a000dbd0 con 0x56485f75ce20 2024-05-17T22:47:54.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.523+0000 7f85affff700 1 --2- 172.21.15.12:0/3217739002 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x56485f75ce20 0x56485f75f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:54.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.539+0000 7f85affff700 1 --2- 172.21.15.12:0/3217739002 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x56485f75ce20 0x56485f75f240 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc8abd4e1eff956 server_cookie=3c524d330d4c2aea in_seq=0 out_seq=0 2024-05-17T22:47:54.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.539+0000 7f858ffff700 10 client.25772 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:47:54.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.540+0000 7f858ffff700 1 -- 172.21.15.12:0/3217739002 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f85a000dbd0 con 0x56485f75ce20 2024-05-17T22:47:54.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.540+0000 7f858ffff700 10 client.25772 handle_command_reply: tid=0 2024-05-17T22:47:54.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.540+0000 7f85c54edb80 1 client.25772 shutdown 2024-05-17T22:47:54.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.540+0000 7f85c54edb80 1 -- 172.21.15.12:0/3217739002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485f743b00 msgr2=0x56485f74cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:54.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.540+0000 7f85c54edb80 1 --2- 172.21.15.12:0/3217739002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485f743b00 0x56485f74cf00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f85a00058b0 tx=0x7f85a000dfb0 comp rx=0 tx=0).stop 2024-05-17T22:47:54.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.541+0000 7f85c54edb80 1 -- 172.21.15.12:0/3217739002 shutdown_connections 2024-05-17T22:47:54.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.541+0000 7f85c54edb80 1 --2- 172.21.15.12:0/3217739002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56485f749400 0x56485f74d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:54.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.541+0000 7f85c54edb80 1 --2- 172.21.15.12:0/3217739002 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x56485f75ce20 0x56485f75f240 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:54.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.541+0000 7f85c54edb80 1 --2- 172.21.15.12:0/3217739002 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56485f5c59f0 0x56485f74c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:54.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.541+0000 7f85c54edb80 1 --2- 172.21.15.12:0/3217739002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485f743b00 0x56485f74cf00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:54.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.541+0000 7f858ffff700 0 client.25772 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:47:54.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.541+0000 7f85c54edb80 1 -- 172.21.15.12:0/3217739002 >> 172.21.15.12:0/3217739002 conn(0x56485f6257d0 msgr2=0x56485f73e160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:47:54.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.541+0000 7f85c54edb80 1 -- 172.21.15.12:0/3217739002 shutdown_connections 2024-05-17T22:47:54.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.541+0000 7f85c54edb80 1 -- 172.21.15.12:0/3217739002 wait complete. 2024-05-17T22:47:54.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.542+0000 7f85c54edb80 20 client.25772 trim_cache size 0 max 16384 2024-05-17T22:47:54.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.582+0000 7f85c54edb80 1 Processor -- start 2024-05-17T22:47:54.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.586+0000 7f85c54edb80 1 -- start start 2024-05-17T22:47:54.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.586+0000 7f85c54edb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485f5c59f0 0x56485f7435c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:54.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.586+0000 7f85c54edb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56485f743b00 0x56485f748690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:54.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.587+0000 7f85c54edb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56485f749400 0x56485f7471b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:54.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.587+0000 7f85c54edb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f859c011500 con 0x56485f743b00 2024-05-17T22:47:54.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.587+0000 7f85c54edb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f859c00ed90 con 0x56485f5c59f0 2024-05-17T22:47:54.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.587+0000 7f85c54edb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f859c00cbe0 con 0x56485f749400 2024-05-17T22:47:54.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.591+0000 7f85b48ae700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56485f743b00 0x56485f748690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:54.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.591+0000 7f85b48ae700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56485f743b00 0x56485f748690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44106/0 (socket says 172.21.15.12:44106) 2024-05-17T22:47:54.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.591+0000 7f85b48ae700 1 -- 172.21.15.12:0/25028569 learned_addr learned my addr 172.21.15.12:0/25028569 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:47:54.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.591+0000 7f85affff700 1 --2- 172.21.15.12:0/25028569 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56485f749400 0x56485f7471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:54.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.591+0000 7f85affff700 1 --2- 172.21.15.12:0/25028569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485f5c59f0 0x56485f7435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:54.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.595+0000 7f85b48ae700 1 -- 172.21.15.12:0/25028569 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56485f749400 msgr2=0x56485f7471b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:47:54.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.595+0000 7f85b48ae700 1 --2- 172.21.15.12:0/25028569 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56485f749400 0x56485f7471b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:54.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.595+0000 7f85b48ae700 1 -- 172.21.15.12:0/25028569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485f5c59f0 msgr2=0x56485f7435c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:47:54.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.595+0000 7f85b48ae700 1 --2- 172.21.15.12:0/25028569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485f5c59f0 0x56485f7435c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:54.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.595+0000 7f85b48ae700 1 -- 172.21.15.12:0/25028569 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56485f73a6d0 con 0x56485f743b00 2024-05-17T22:47:54.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.599+0000 7f85b48ae700 1 --2- 172.21.15.12:0/25028569 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56485f743b00 0x56485f748690 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f85a0018b50 tx=0x7f85a001a320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db4aafca1b8ee177 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:47:54.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.603+0000 7f85967fc700 1 -- 172.21.15.12:0/25028569 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a0014040 con 0x56485f743b00 2024-05-17T22:47:54.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.603+0000 7f85967fc700 1 -- 172.21.15.12:0/25028569 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85a0024030 con 0x56485f743b00 2024-05-17T22:47:54.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.603+0000 7f85967fc700 1 -- 172.21.15.12:0/25028569 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a0018760 con 0x56485f743b00 2024-05-17T22:47:54.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.605+0000 7f85c54edb80 1 -- 172.21.15.12:0/25028569 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56485f743b00 msgr2=0x56485f748690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:54.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.605+0000 7f85c54edb80 1 --2- 172.21.15.12:0/25028569 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56485f743b00 0x56485f748690 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f85a0018b50 tx=0x7f85a001a320 comp rx=0 tx=0).stop 2024-05-17T22:47:54.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.606+0000 7f85c54edb80 1 -- 172.21.15.12:0/25028569 shutdown_connections 2024-05-17T22:47:54.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.606+0000 7f85c54edb80 1 --2- 172.21.15.12:0/25028569 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56485f749400 0x56485f7471b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:54.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.606+0000 7f85c54edb80 1 --2- 172.21.15.12:0/25028569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485f5c59f0 0x56485f7435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:54.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.606+0000 7f85c54edb80 1 --2- 172.21.15.12:0/25028569 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56485f743b00 0x56485f748690 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:54.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.606+0000 7f85c54edb80 1 -- 172.21.15.12:0/25028569 >> 172.21.15.12:0/25028569 conn(0x56485f6257d0 msgr2=0x56485f738110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:47:54.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.613+0000 7f85c54edb80 1 -- 172.21.15.12:0/25028569 shutdown_connections 2024-05-17T22:47:54.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.614+0000 7f85c54edb80 1 -- 172.21.15.12:0/25028569 wait complete. 2024-05-17T22:47:54.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.615+0000 7f85c54edb80 1 -- start start 2024-05-17T22:47:54.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.615+0000 7f85c54edb80 1 Processor -- start 2024-05-17T22:47:54.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.616+0000 7f85c54edb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485f5c59f0 0x56485f7621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:54.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.616+0000 7f85c54edb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56485f743b00 0x56485f7626f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:54.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.616+0000 7f85c54edb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56485f749400 0x56485f7605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:54.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.616+0000 7f85c54edb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f859c00d250 con 0x56485f749400 2024-05-17T22:47:54.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.616+0000 7f85c54edb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f859c00e6c0 con 0x56485f5c59f0 2024-05-17T22:47:54.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.617+0000 7f85c54edb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f859c00d920 con 0x56485f743b00 2024-05-17T22:47:54.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.617+0000 7f85c54edb80 10 client.0 _ll_register_callbacks cb 0x56485f5f8b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:47:54.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.618+0000 7f85affff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56485f749400 0x56485f7605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:54.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.618+0000 7f85affff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56485f749400 0x56485f7605f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44116/0 (socket says 172.21.15.12:44116) 2024-05-17T22:47:54.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.618+0000 7f85affff700 1 -- 172.21.15.12:0/3027294145 learned_addr learned my addr 172.21.15.12:0/3027294145 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:47:54.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.618+0000 7f85affff700 1 --2- 172.21.15.12:0/3027294145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485f5c59f0 0x56485f7621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:54.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.623+0000 7f85affff700 1 -- 172.21.15.12:0/3027294145 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56485f743b00 msgr2=0x56485f7626f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T22:47:54.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.623+0000 7f85affff700 1 --2- 172.21.15.12:0/3027294145 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56485f743b00 0x56485f7626f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:54.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.623+0000 7f85affff700 1 -- 172.21.15.12:0/3027294145 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56485f749400 msgr2=0x56485f7605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:47:54.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.623+0000 7f85affff700 1 --2- 172.21.15.12:0/3027294145 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56485f749400 0x56485f7605f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:54.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.623+0000 7f85affff700 1 -- 172.21.15.12:0/3027294145 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56485f760cb0 con 0x56485f5c59f0 2024-05-17T22:47:54.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.627+0000 7f85affff700 1 --2- 172.21.15.12:0/3027294145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485f5c59f0 0x56485f7621b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f85a4009810 tx=0x7f85a4017400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7801f5b2314d2071 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:47:54.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.627+0000 7f858ffff700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:47:54.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.629+0000 7f858ffff700 1 -- 172.21.15.12:0/3027294145 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a4028070 con 0x56485f5c59f0 2024-05-17T22:47:54.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.629+0000 7f85c54edb80 1 -- 172.21.15.12:0/3027294145 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85a0024030 con 0x56485f5c59f0 2024-05-17T22:47:54.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.629+0000 7f858ffff700 1 -- 172.21.15.12:0/3027294145 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f85a4004020 con 0x56485f5c59f0 2024-05-17T22:47:54.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.634+0000 7f858ffff700 1 -- 172.21.15.12:0/3027294145 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f85a4004220 con 0x56485f5c59f0 2024-05-17T22:47:54.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.635+0000 7f85c54edb80 10 client.25778 fetch_fsmap learned FSMap version 39 2024-05-17T22:47:54.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.635+0000 7f85c54edb80 1 -- 172.21.15.12:0/3027294145 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56485f7c7e20 con 0x56485f5c59f0 2024-05-17T22:47:54.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.638+0000 7f858ffff700 1 -- 172.21.15.12:0/3027294145 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85a0024030 con 0x56485f5c59f0 2024-05-17T22:47:54.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.638+0000 7f858ffff700 1 -- 172.21.15.12:0/3027294145 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f85a4004880 con 0x56485f5c59f0 2024-05-17T22:47:54.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.638+0000 7f85c54edb80 10 client.25778 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:47:54.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.638+0000 7f85c54edb80 10 client.25778 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:47:54.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.638+0000 7f85c54edb80 20 client.25778 populate_metadata read hostname 'smithi012' 2024-05-17T22:47:54.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.638+0000 7f85c54edb80 1 --2- 172.21.15.12:0/3027294145 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x56485f7c8510 0x56485f7ca930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:47:54.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.638+0000 7f85c54edb80 4 client.25778 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:47:54.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.638+0000 7f85c54edb80 1 -- 172.21.15.12:0/3027294145 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56485f674020 con 0x56485f7c8510 2024-05-17T22:47:54.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.640+0000 7f85b48ae700 1 --2- 172.21.15.12:0/3027294145 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x56485f7c8510 0x56485f7ca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:47:54.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.643+0000 7f858ffff700 10 client.25778 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:47:54.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.643+0000 7f85b48ae700 1 --2- 172.21.15.12:0/3027294145 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x56485f7c8510 0x56485f7ca930 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f375a1a091b9a5c2 server_cookie=b62aa1a4790c6d8 in_seq=0 out_seq=0 2024-05-17T22:47:54.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.643+0000 7f858ffff700 1 -- 172.21.15.12:0/3027294145 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56485f674020 con 0x56485f7c8510 2024-05-17T22:47:54.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.643+0000 7f858ffff700 10 client.25778 handle_command_reply: tid=0 2024-05-17T22:47:54.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.643+0000 7f85c54edb80 1 client.25778 shutdown 2024-05-17T22:47:54.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.643+0000 7f85c54edb80 1 -- 172.21.15.12:0/3027294145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485f5c59f0 msgr2=0x56485f7621b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:54.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.643+0000 7f85c54edb80 1 --2- 172.21.15.12:0/3027294145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485f5c59f0 0x56485f7621b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f85a4009810 tx=0x7f85a4017400 comp rx=0 tx=0).stop 2024-05-17T22:47:54.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.644+0000 7f85c54edb80 1 -- 172.21.15.12:0/3027294145 shutdown_connections 2024-05-17T22:47:54.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.644+0000 7f85c54edb80 1 --2- 172.21.15.12:0/3027294145 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x56485f7c8510 0x56485f7ca930 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:54.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.644+0000 7f85c54edb80 1 --2- 172.21.15.12:0/3027294145 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56485f749400 0x56485f7605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:54.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.644+0000 7f85c54edb80 1 --2- 172.21.15.12:0/3027294145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485f5c59f0 0x56485f7621b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:54.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.644+0000 7f85c54edb80 1 --2- 172.21.15.12:0/3027294145 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56485f743b00 0x56485f7626f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:54.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.644+0000 7f85c54edb80 1 -- 172.21.15.12:0/3027294145 >> 172.21.15.12:0/3027294145 conn(0x56485f6257d0 msgr2=0x56485f746f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:47:54.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.644+0000 7f858ffff700 0 client.25778 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:47:54.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.644+0000 7f85c54edb80 1 -- 172.21.15.12:0/3027294145 shutdown_connections 2024-05-17T22:47:54.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.644+0000 7f85c54edb80 1 -- 172.21.15.12:0/3027294145 wait complete. 2024-05-17T22:47:54.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.644+0000 7f85c54edb80 20 client.25778 trim_cache size 0 max 16384 2024-05-17T22:47:54.648 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:47:54.648 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-17T22:47:54.649 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-17T22:47:54.649 INFO:teuthology.orchestra.run.smithi012.stdout: "715b0151-ef4c-4701-ba99-d9650a623cbe": { 2024-05-17T22:47:54.649 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-17T22:47:54.649 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "715b0151-ef4c-4701-ba99-d9650a623cbe", 2024-05-17T22:47:54.649 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-17T22:47:54.649 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:47:54.649 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:47:54.649 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:47:54.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.646+0000 7f85b6b12700 1 -- 172.21.15.12:0/400930132 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f85980b2a80 msgr2=0x7f85980b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:54.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.646+0000 7f85b6b12700 1 --2- 172.21.15.12:0/400930132 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f85980b2a80 0x7f85980b4f90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f85a80a60c0 tx=0x7f85a80a5510 comp rx=0 tx=0).stop 2024-05-17T22:47:54.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.646+0000 7f85b6b12700 1 -- 172.21.15.12:0/400930132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85b011d550 msgr2=0x7f85b0119510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:47:54.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.646+0000 7f85b6b12700 1 --2- 172.21.15.12:0/400930132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85b011d550 0x7f85b0119510 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f85a800ada0 tx=0x7f85a800c330 comp rx=0 tx=0).stop 2024-05-17T22:47:54.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.646+0000 7f85b6b12700 1 -- 172.21.15.12:0/400930132 shutdown_connections 2024-05-17T22:47:54.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.646+0000 7f85b6b12700 1 --2- 172.21.15.12:0/400930132 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f85980b2a80 0x7f85980b4f90 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:54.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.646+0000 7f85b6b12700 1 --2- 172.21.15.12:0/400930132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85b011d550 0x7f85b0119510 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:54.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.646+0000 7f85b6b12700 1 --2- 172.21.15.12:0/400930132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85b010a830 0x7f85b011ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:54.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.646+0000 7f85b6b12700 1 --2- 172.21.15.12:0/400930132 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85b010b180 0x7f85b0118f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:47:54.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.646+0000 7f85b6b12700 1 -- 172.21.15.12:0/400930132 >> 172.21.15.12:0/400930132 conn(0x7f85b006efc0 msgr2=0x7f85b011dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:47:54.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.649+0000 7f85b6b12700 1 -- 172.21.15.12:0/400930132 shutdown_connections 2024-05-17T22:47:54.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:47:54.649+0000 7f85b6b12700 1 -- 172.21.15.12:0/400930132 wait complete. 2024-05-17T22:47:56.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:55 smithi012 ceph-mon[31632]: pgmap v699: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 5.1 MiB/s wr, 546 op/s 2024-05-17T22:47:56.237 INFO:tasks.cephfs.filesystem:scrub status for tag:715b0151-ef4c-4701-ba99-d9650a623cbe - {'path': '/', 'tag': '715b0151-ef4c-4701-ba99-d9650a623cbe', 'options': 'recursive,force'} 2024-05-17T22:47:56.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:55 smithi151 ceph-mon[36101]: pgmap v699: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 5.1 MiB/s wr, 546 op/s 2024-05-17T22:47:56.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:55 smithi096 ceph-mon[36065]: pgmap v699: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 5.1 MiB/s wr, 546 op/s 2024-05-17T22:47:56.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 515.0 s, 592.2 tps, lat 167.852 ms stddev 185.949 2024-05-17T22:47:58.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:57 smithi151 ceph-mon[36101]: pgmap v700: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.7 MiB/s wr, 507 op/s 2024-05-17T22:47:58.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:57 smithi096 ceph-mon[36065]: pgmap v700: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.7 MiB/s wr, 507 op/s 2024-05-17T22:47:58.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:57 smithi012 ceph-mon[31632]: pgmap v700: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.7 MiB/s wr, 507 op/s 2024-05-17T22:47:59.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:47:58 smithi096 ceph-mon[36065]: scrub summary: idle 2024-05-17T22:47:59.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:47:58 smithi012 ceph-mon[31632]: scrub summary: idle 2024-05-17T22:47:59.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:47:58 smithi151 ceph-mon[36101]: scrub summary: idle 2024-05-17T22:48:00.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:00 smithi096 ceph-mon[36065]: scrub summary: idle 2024-05-17T22:48:00.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:00 smithi096 ceph-mon[36065]: scrub summary: idle 2024-05-17T22:48:00.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:00 smithi096 ceph-mon[36065]: pgmap v701: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.7 MiB/s wr, 600 op/s 2024-05-17T22:48:00.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:00 smithi012 ceph-mon[31632]: scrub summary: idle 2024-05-17T22:48:00.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:00 smithi012 ceph-mon[31632]: scrub summary: idle 2024-05-17T22:48:00.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:00 smithi012 ceph-mon[31632]: pgmap v701: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.7 MiB/s wr, 600 op/s 2024-05-17T22:48:00.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:00 smithi151 ceph-mon[36101]: scrub summary: idle 2024-05-17T22:48:00.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:00 smithi151 ceph-mon[36101]: scrub summary: idle 2024-05-17T22:48:00.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:00 smithi151 ceph-mon[36101]: pgmap v701: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.7 MiB/s wr, 600 op/s 2024-05-17T22:48:01.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 520.0 s, 537.4 tps, lat 188.763 ms stddev 224.346 2024-05-17T22:48:02.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:02 smithi096 ceph-mon[36065]: pgmap v702: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.2 MiB/s wr, 536 op/s 2024-05-17T22:48:02.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:02 smithi012 ceph-mon[31632]: pgmap v702: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.2 MiB/s wr, 536 op/s 2024-05-17T22:48:02.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:02 smithi151 ceph-mon[36101]: pgmap v702: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.2 MiB/s wr, 536 op/s 2024-05-17T22:48:04.292 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:04 smithi151 ceph-mon[36101]: pgmap v703: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.3 MiB/s wr, 575 op/s 2024-05-17T22:48:04.294 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:04 smithi012 ceph-mon[31632]: pgmap v703: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.3 MiB/s wr, 575 op/s 2024-05-17T22:48:04.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:04 smithi096 ceph-mon[36065]: pgmap v703: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.3 MiB/s wr, 575 op/s 2024-05-17T22:48:06.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:06 smithi096 ceph-mon[36065]: pgmap v704: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.1 MiB/s wr, 579 op/s 2024-05-17T22:48:06.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:06 smithi012 ceph-mon[31632]: pgmap v704: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.1 MiB/s wr, 579 op/s 2024-05-17T22:48:06.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 525.0 s, 531.4 tps, lat 187.443 ms stddev 225.283 2024-05-17T22:48:06.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:06 smithi151 ceph-mon[36101]: pgmap v704: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.1 MiB/s wr, 579 op/s 2024-05-17T22:48:07.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:07 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:48:07.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:07 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:48:07.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:07 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:48:08.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:08 smithi096 ceph-mon[36065]: pgmap v705: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.7 MiB/s wr, 534 op/s 2024-05-17T22:48:08.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:08 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:48:08.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:08 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:48:08.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:08 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:48:08.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:08 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:48:08.343 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:08 smithi012 ceph-mon[31632]: pgmap v705: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.7 MiB/s wr, 534 op/s 2024-05-17T22:48:08.343 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:08 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:48:08.344 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:08 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:48:08.344 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:08 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:48:08.344 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:08 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:48:08.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:08 smithi151 ceph-mon[36101]: pgmap v705: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.7 MiB/s wr, 534 op/s 2024-05-17T22:48:08.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:08 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:48:08.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:08 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:48:08.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:08 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:48:08.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:08 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:48:09.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:48:09.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:09 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:48:09.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:09 smithi096 ceph-mon[36065]: pgmap v706: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.6 MiB/s wr, 613 op/s 2024-05-17T22:48:09.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:09 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:48:09.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:09 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:48:09.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:09 smithi012 ceph-mon[31632]: pgmap v706: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.6 MiB/s wr, 613 op/s 2024-05-17T22:48:09.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:09 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:48:09.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:09 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:48:09.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:09 smithi151 ceph-mon[36101]: pgmap v706: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.6 MiB/s wr, 613 op/s 2024-05-17T22:48:10.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:10 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:48:10.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:10 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:48:10.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:10 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:48:10.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:10 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:48:10.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:10 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:48:10.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:10 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:48:10.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:10 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:48:10.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:10 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:48:10.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:10 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:48:11.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 530.0 s, 533.8 tps, lat 188.926 ms stddev 209.182 2024-05-17T22:48:11.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:11 smithi096 ceph-mon[36065]: pgmap v707: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.7 MiB/s wr, 520 op/s 2024-05-17T22:48:11.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:11 smithi012 ceph-mon[31632]: pgmap v707: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.7 MiB/s wr, 520 op/s 2024-05-17T22:48:11.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:11 smithi151 ceph-mon[36101]: pgmap v707: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.7 MiB/s wr, 520 op/s 2024-05-17T22:48:14.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:13 smithi151 ceph-mon[36101]: pgmap v708: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.2 MiB/s wr, 485 op/s 2024-05-17T22:48:14.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:13 smithi096 ceph-mon[36065]: pgmap v708: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.2 MiB/s wr, 485 op/s 2024-05-17T22:48:14.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:13 smithi012 ceph-mon[31632]: pgmap v708: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.2 MiB/s wr, 485 op/s 2024-05-17T22:48:16.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:15 smithi012 ceph-mon[31632]: pgmap v709: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.8 MiB/s wr, 544 op/s 2024-05-17T22:48:16.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:15 smithi151 ceph-mon[36101]: pgmap v709: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.8 MiB/s wr, 544 op/s 2024-05-17T22:48:16.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:15 smithi096 ceph-mon[36065]: pgmap v709: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.8 MiB/s wr, 544 op/s 2024-05-17T22:48:16.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 535.0 s, 514.0 tps, lat 190.501 ms stddev 306.221 2024-05-17T22:48:18.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:17 smithi151 ceph-mon[36101]: pgmap v710: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 MiB/s wr, 483 op/s 2024-05-17T22:48:18.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:17 smithi096 ceph-mon[36065]: pgmap v710: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 MiB/s wr, 483 op/s 2024-05-17T22:48:18.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:17 smithi012 ceph-mon[31632]: pgmap v710: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 MiB/s wr, 483 op/s 2024-05-17T22:48:19.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:19 smithi096 ceph-mon[36065]: pgmap v711: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 4.8 MiB/s wr, 531 op/s 2024-05-17T22:48:19.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:19 smithi012 ceph-mon[31632]: pgmap v711: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 4.8 MiB/s wr, 531 op/s 2024-05-17T22:48:19.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:19 smithi151 ceph-mon[36101]: pgmap v711: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 4.8 MiB/s wr, 531 op/s 2024-05-17T22:48:21.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 540.0 s, 529.4 tps, lat 190.669 ms stddev 224.436 2024-05-17T22:48:22.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:21 smithi151 ceph-mon[36101]: pgmap v712: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.2 MiB/s wr, 478 op/s 2024-05-17T22:48:22.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:21 smithi096 ceph-mon[36065]: pgmap v712: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.2 MiB/s wr, 478 op/s 2024-05-17T22:48:22.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:21 smithi012 ceph-mon[31632]: pgmap v712: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.2 MiB/s wr, 478 op/s 2024-05-17T22:48:24.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:23 smithi151 ceph-mon[36101]: pgmap v713: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.0 MiB/s wr, 464 op/s 2024-05-17T22:48:24.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:23 smithi096 ceph-mon[36065]: pgmap v713: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.0 MiB/s wr, 464 op/s 2024-05-17T22:48:24.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:23 smithi012 ceph-mon[31632]: pgmap v713: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.0 MiB/s wr, 464 op/s 2024-05-17T22:48:26.238 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-17T22:48:26.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:25 smithi151 ceph-mon[36101]: pgmap v714: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.4 MiB/s wr, 605 op/s 2024-05-17T22:48:26.295 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:25 smithi012 ceph-mon[31632]: pgmap v714: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.4 MiB/s wr, 605 op/s 2024-05-17T22:48:26.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:25 smithi096 ceph-mon[36065]: pgmap v714: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.4 MiB/s wr, 605 op/s 2024-05-17T22:48:26.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 545.0 s, 575.8 tps, lat 171.581 ms stddev 214.884 2024-05-17T22:48:26.748 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:48:28.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:27 smithi151 ceph-mon[36101]: pgmap v715: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.3 MiB/s wr, 488 op/s 2024-05-17T22:48:28.281 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:27 smithi012 ceph-mon[31632]: pgmap v715: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.3 MiB/s wr, 488 op/s 2024-05-17T22:48:28.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:27 smithi096 ceph-mon[36065]: pgmap v715: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.3 MiB/s wr, 488 op/s 2024-05-17T22:48:30.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:29 smithi151 ceph-mon[36101]: pgmap v716: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.8 MiB/s wr, 553 op/s 2024-05-17T22:48:30.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:29 smithi096 ceph-mon[36065]: pgmap v716: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.8 MiB/s wr, 553 op/s 2024-05-17T22:48:30.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:29 smithi012 ceph-mon[31632]: pgmap v716: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.8 MiB/s wr, 553 op/s 2024-05-17T22:48:31.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 550.0 s, 444.8 tps, lat 226.713 ms stddev 266.042 2024-05-17T22:48:32.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:31 smithi151 ceph-mon[36101]: pgmap v717: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.8 MiB/s wr, 551 op/s 2024-05-17T22:48:32.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:31 smithi096 ceph-mon[36065]: pgmap v717: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.8 MiB/s wr, 551 op/s 2024-05-17T22:48:32.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:31 smithi012 ceph-mon[31632]: pgmap v717: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.8 MiB/s wr, 551 op/s 2024-05-17T22:48:34.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:33 smithi151 ceph-mon[36101]: pgmap v718: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.6 MiB/s wr, 516 op/s 2024-05-17T22:48:34.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:33 smithi096 ceph-mon[36065]: pgmap v718: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.6 MiB/s wr, 516 op/s 2024-05-17T22:48:34.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.294+0000 7fbbc8be1700 1 -- 172.21.15.12:0/3411884420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbc4107750 msgr2=0x7fbbc4107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:34.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.294+0000 7fbbc8be1700 1 --2- 172.21.15.12:0/3411884420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbc4107750 0x7fbbc4107b60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbbb80098f0 tx=0x7fbbb8009c00 comp rx=0 tx=0).stop 2024-05-17T22:48:34.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.295+0000 7fbbc8be1700 1 -- 172.21.15.12:0/3411884420 shutdown_connections 2024-05-17T22:48:34.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.295+0000 7fbbc8be1700 1 --2- 172.21.15.12:0/3411884420 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbbc41080a0 0x7fbbc410fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:34.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.295+0000 7fbbc8be1700 1 --2- 172.21.15.12:0/3411884420 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbc4102e20 0x7fbbc4103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:34.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.295+0000 7fbbc8be1700 1 --2- 172.21.15.12:0/3411884420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbc4107750 0x7fbbc4107b60 secure :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbbb80098f0 tx=0x7fbbb8009c00 comp rx=0 tx=0).stop 2024-05-17T22:48:34.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.295+0000 7fbbc8be1700 1 -- 172.21.15.12:0/3411884420 >> 172.21.15.12:0/3411884420 conn(0x7fbbc4079bd0 msgr2=0x7fbbc4079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:48:34.299 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:33 smithi012 ceph-mon[31632]: pgmap v718: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.6 MiB/s wr, 516 op/s 2024-05-17T22:48:34.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.300+0000 7fbbc8be1700 1 -- 172.21.15.12:0/3411884420 shutdown_connections 2024-05-17T22:48:34.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.302+0000 7fbbc8be1700 1 -- 172.21.15.12:0/3411884420 wait complete. 2024-05-17T22:48:34.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.303+0000 7fbbc8be1700 1 Processor -- start 2024-05-17T22:48:34.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.305+0000 7fbbc8be1700 1 -- start start 2024-05-17T22:48:34.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.306+0000 7fbbc8be1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbc4102e20 0x7fbbc4198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:34.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.306+0000 7fbbc8be1700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbbc41080a0 0x7fbbc4198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:34.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.306+0000 7fbbc8be1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbc419cfd0 0x7fbbc419d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:34.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.306+0000 7fbbc8be1700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc4111ce0 con 0x7fbbc419cfd0 2024-05-17T22:48:34.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.306+0000 7fbbc2d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbc419cfd0 0x7fbbc419d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:34.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.306+0000 7fbbc2d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbc419cfd0 0x7fbbc419d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59570/0 (socket says 172.21.15.12:59570) 2024-05-17T22:48:34.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.306+0000 7fbbc2d9d700 1 -- 172.21.15.12:0/901544640 learned_addr learned my addr 172.21.15.12:0/901544640 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:48:34.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.306+0000 7fbbc8be1700 1 -- 172.21.15.12:0/901544640 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc4111b60 con 0x7fbbc4102e20 2024-05-17T22:48:34.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.306+0000 7fbbc8be1700 1 -- 172.21.15.12:0/901544640 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc4111e60 con 0x7fbbc41080a0 2024-05-17T22:48:34.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.307+0000 7fbbc259c700 1 --2- 172.21.15.12:0/901544640 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbc4102e20 0x7fbbc4198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:34.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.307+0000 7fbbc2d9d700 1 -- 172.21.15.12:0/901544640 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbbc41080a0 msgr2=0x7fbbc4198e00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-17T22:48:34.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.307+0000 7fbbc2d9d700 1 --2- 172.21.15.12:0/901544640 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbbc41080a0 0x7fbbc4198e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:34.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.307+0000 7fbbc2d9d700 1 -- 172.21.15.12:0/901544640 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbc4102e20 msgr2=0x7fbbc4198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:34.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.307+0000 7fbbc2d9d700 1 --2- 172.21.15.12:0/901544640 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbc4102e20 0x7fbbc4198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:34.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.307+0000 7fbbc2d9d700 1 -- 172.21.15.12:0/901544640 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbc419db20 con 0x7fbbc419cfd0 2024-05-17T22:48:34.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.308+0000 7fbbc2d9d700 1 --2- 172.21.15.12:0/901544640 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbc419cfd0 0x7fbbc419d3e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc00efc0 tx=0x7fbbbc00c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:48:34.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.312+0000 7fbbb37fe700 1 -- 172.21.15.12:0/901544640 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc004bb0 con 0x7fbbc419cfd0 2024-05-17T22:48:34.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.312+0000 7fbbc8be1700 1 -- 172.21.15.12:0/901544640 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbc419de10 con 0x7fbbc419cfd0 2024-05-17T22:48:34.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.312+0000 7fbbc8be1700 1 -- 172.21.15.12:0/901544640 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbc41a55c0 con 0x7fbbc419cfd0 2024-05-17T22:48:34.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.312+0000 7fbbb37fe700 1 -- 172.21.15.12:0/901544640 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbbc017070 con 0x7fbbc419cfd0 2024-05-17T22:48:34.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.312+0000 7fbbb37fe700 1 -- 172.21.15.12:0/901544640 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc007d70 con 0x7fbbc419cfd0 2024-05-17T22:48:34.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.322+0000 7fbbb37fe700 1 -- 172.21.15.12:0/901544640 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fbbbc005000 con 0x7fbbc419cfd0 2024-05-17T22:48:34.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.325+0000 7fbbb37fe700 1 --2- 172.21.15.12:0/901544640 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fbbac0bb220 0x7fbbac0bd730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:34.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.326+0000 7fbbb37fe700 1 -- 172.21.15.12:0/901544640 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fbbbc0e2d00 con 0x7fbbc419cfd0 2024-05-17T22:48:34.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.326+0000 7fbbc259c700 1 --2- 172.21.15.12:0/901544640 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fbbac0bb220 0x7fbbac0bd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:34.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.326+0000 7fbbd75bcb80 1 Processor -- start 2024-05-17T22:48:34.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.327+0000 7fbbc259c700 1 --2- 172.21.15.12:0/901544640 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fbbac0bb220 0x7fbbac0bd730 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4005950 tx=0x7fbbb400b580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:48:34.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.327+0000 7fbbd75bcb80 1 -- start start 2024-05-17T22:48:34.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.328+0000 7fbbd75bcb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56070d1319f0 0x56070d2c0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:34.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.328+0000 7fbbd75bcb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56070d2c0db0 0x56070d2c4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:34.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.328+0000 7fbbd75bcb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d2c4eb0 0x56070d2c52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:34.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.328+0000 7fbbd75bcb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56070d1de0d0 con 0x56070d2c0db0 2024-05-17T22:48:34.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.328+0000 7fbbd75bcb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56070d1cdc70 con 0x56070d1319f0 2024-05-17T22:48:34.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.328+0000 7fbbd75bcb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56070d1d1cb0 con 0x56070d2c4eb0 2024-05-17T22:48:34.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.328+0000 7fbbc259c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56070d1319f0 0x56070d2c0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:34.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.328+0000 7fbbc259c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56070d1319f0 0x56070d2c0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:33462/0 (socket says 172.21.15.12:33462) 2024-05-17T22:48:34.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.328+0000 7fbbc259c700 1 -- 172.21.15.12:0/3778599689 learned_addr learned my addr 172.21.15.12:0/3778599689 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:48:34.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.328+0000 7fbbc2d9d700 1 --2- 172.21.15.12:0/3778599689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d2c4eb0 0x56070d2c52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:34.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.329+0000 7fbbc1d9b700 1 --2- 172.21.15.12:0/3778599689 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56070d2c0db0 0x56070d2c4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:34.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.329+0000 7fbbc259c700 1 -- 172.21.15.12:0/3778599689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d2c4eb0 msgr2=0x56070d2c52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:48:34.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.329+0000 7fbbc259c700 1 --2- 172.21.15.12:0/3778599689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d2c4eb0 0x56070d2c52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:34.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.329+0000 7fbbc259c700 1 -- 172.21.15.12:0/3778599689 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56070d2c0db0 msgr2=0x56070d2c4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:48:34.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.329+0000 7fbbc259c700 1 --2- 172.21.15.12:0/3778599689 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56070d2c0db0 0x56070d2c4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:34.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.329+0000 7fbbc259c700 1 -- 172.21.15.12:0/3778599689 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56070d1675c0 con 0x56070d1319f0 2024-05-17T22:48:34.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.329+0000 7fbbc259c700 1 --2- 172.21.15.12:0/3778599689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56070d1319f0 0x56070d2c0870 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4009e20 tx=0x7fbbb4013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4736b88196769c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:48:34.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.330+0000 7fbbb0ff9700 1 -- 172.21.15.12:0/3778599689 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb4038070 con 0x56070d1319f0 2024-05-17T22:48:34.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.330+0000 7fbbb0ff9700 1 -- 172.21.15.12:0/3778599689 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbb4017e40 con 0x56070d1319f0 2024-05-17T22:48:34.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.330+0000 7fbbb0ff9700 1 -- 172.21.15.12:0/3778599689 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb40156b0 con 0x56070d1319f0 2024-05-17T22:48:34.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.331+0000 7fbbd75bcb80 1 -- 172.21.15.12:0/3778599689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56070d1319f0 msgr2=0x56070d2c0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:34.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.331+0000 7fbbd75bcb80 1 --2- 172.21.15.12:0/3778599689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56070d1319f0 0x56070d2c0870 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4009e20 tx=0x7fbbb4013000 comp rx=0 tx=0).stop 2024-05-17T22:48:34.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.333+0000 7fbbd75bcb80 1 -- 172.21.15.12:0/3778599689 shutdown_connections 2024-05-17T22:48:34.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.333+0000 7fbbd75bcb80 1 --2- 172.21.15.12:0/3778599689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d2c4eb0 0x56070d2c52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:34.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.333+0000 7fbbd75bcb80 1 --2- 172.21.15.12:0/3778599689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56070d1319f0 0x56070d2c0870 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:34.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.333+0000 7fbbd75bcb80 1 --2- 172.21.15.12:0/3778599689 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56070d2c0db0 0x56070d2c4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:34.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.333+0000 7fbbd75bcb80 1 -- 172.21.15.12:0/3778599689 >> 172.21.15.12:0/3778599689 conn(0x56070d1917d0 msgr2=0x56070d2aa1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:48:34.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.335+0000 7fbbd75bcb80 1 -- 172.21.15.12:0/3778599689 shutdown_connections 2024-05-17T22:48:34.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.335+0000 7fbbd75bcb80 1 -- 172.21.15.12:0/3778599689 wait complete. 2024-05-17T22:48:34.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.336+0000 7fbbd75bcb80 1 -- start start 2024-05-17T22:48:34.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.336+0000 7fbbd75bcb80 1 Processor -- start 2024-05-17T22:48:34.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.337+0000 7fbbd75bcb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56070d1319f0 0x56070d2c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:34.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.337+0000 7fbbd75bcb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56070d2c0db0 0x56070d2ca040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:34.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.337+0000 7fbbd75bcb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d2c4eb0 0x56070d2ca580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:34.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.337+0000 7fbbd75bcb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56070d0d5150 con 0x56070d2c0db0 2024-05-17T22:48:34.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.337+0000 7fbbd75bcb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56070d16f940 con 0x56070d1319f0 2024-05-17T22:48:34.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.337+0000 7fbbd75bcb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56070d2ccd20 con 0x56070d2c4eb0 2024-05-17T22:48:34.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.338+0000 7fbbd75bcb80 10 client.0 _ll_register_callbacks cb 0x56070d164b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:48:34.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.338+0000 7fbbc259c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56070d1319f0 0x56070d2c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:34.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.338+0000 7fbbc259c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56070d1319f0 0x56070d2c9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:33472/0 (socket says 172.21.15.12:33472) 2024-05-17T22:48:34.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.338+0000 7fbbc259c700 1 -- 172.21.15.12:0/2975625250 learned_addr learned my addr 172.21.15.12:0/2975625250 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:48:34.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.338+0000 7fbbc2d9d700 1 --2- 172.21.15.12:0/2975625250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d2c4eb0 0x56070d2ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:34.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.339+0000 7fbbc1d9b700 1 --2- 172.21.15.12:0/2975625250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56070d2c0db0 0x56070d2ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:34.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.339+0000 7fbbc2d9d700 1 -- 172.21.15.12:0/2975625250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56070d1319f0 msgr2=0x56070d2c9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:48:34.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.339+0000 7fbbc2d9d700 1 --2- 172.21.15.12:0/2975625250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56070d1319f0 0x56070d2c9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:34.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.339+0000 7fbbc2d9d700 1 -- 172.21.15.12:0/2975625250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56070d2c0db0 msgr2=0x56070d2ca040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:48:34.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.339+0000 7fbbc2d9d700 1 --2- 172.21.15.12:0/2975625250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56070d2c0db0 0x56070d2ca040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:34.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.340+0000 7fbbc2d9d700 1 -- 172.21.15.12:0/2975625250 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56070d11e480 con 0x56070d2c4eb0 2024-05-17T22:48:34.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.340+0000 7fbbc1d9b700 1 --2- 172.21.15.12:0/2975625250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56070d2c0db0 0x56070d2ca040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:48:34.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.341+0000 7fbbc2d9d700 1 --2- 172.21.15.12:0/2975625250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d2c4eb0 0x56070d2ca580 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc0e04c0 tx=0x7fbbbc0a9780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d643f9e4b89e5b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:48:34.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.341+0000 7fbba9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:48:34.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.342+0000 7fbba9ffb700 1 -- 172.21.15.12:0/2975625250 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc0ae340 con 0x56070d2c4eb0 2024-05-17T22:48:34.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.342+0000 7fbba9ffb700 1 -- 172.21.15.12:0/2975625250 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fbbbc0aa400 con 0x56070d2c4eb0 2024-05-17T22:48:34.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.342+0000 7fbbd75bcb80 1 -- 172.21.15.12:0/2975625250 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56070d1e0020 con 0x56070d2c4eb0 2024-05-17T22:48:34.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.342+0000 7fbba9ffb700 1 -- 172.21.15.12:0/2975625250 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fbbbc0b2df0 con 0x56070d2c4eb0 2024-05-17T22:48:34.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.344+0000 7fbbd75bcb80 10 client.25729 fetch_fsmap learned FSMap version 39 2024-05-17T22:48:34.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.344+0000 7fbbd75bcb80 1 -- 172.21.15.12:0/2975625250 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56070d2d7880 con 0x56070d2c4eb0 2024-05-17T22:48:34.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.347+0000 7fbba9ffb700 1 -- 172.21.15.12:0/2975625250 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56070d1e0020 con 0x56070d2c4eb0 2024-05-17T22:48:34.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.347+0000 7fbba9ffb700 1 -- 172.21.15.12:0/2975625250 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fbbbc0a5900 con 0x56070d2c4eb0 2024-05-17T22:48:34.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.347+0000 7fbbd75bcb80 10 client.25729 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:48:34.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.347+0000 7fbbd75bcb80 10 client.25729 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:48:34.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.347+0000 7fbbd75bcb80 20 client.25729 populate_metadata read hostname 'smithi012' 2024-05-17T22:48:34.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.347+0000 7fbbd75bcb80 1 --2- 172.21.15.12:0/2975625250 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x56070d2da1c0 0x56070d2dc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:34.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.347+0000 7fbbd75bcb80 4 client.25729 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:48:34.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.347+0000 7fbbd75bcb80 1 -- 172.21.15.12:0/2975625250 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56070d1d9800 con 0x56070d2da1c0 2024-05-17T22:48:34.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.348+0000 7fbbc259c700 1 --2- 172.21.15.12:0/2975625250 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x56070d2da1c0 0x56070d2dc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:34.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.349+0000 7fbba9ffb700 10 client.25729 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:48:34.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.350+0000 7fbbc259c700 1 --2- 172.21.15.12:0/2975625250 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x56070d2da1c0 0x56070d2dc650 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f51900eea4b81dfc server_cookie=c277d473085d9a35 in_seq=0 out_seq=0 2024-05-17T22:48:34.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.351+0000 7fbba9ffb700 1 -- 172.21.15.12:0/2975625250 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56070d1d9800 con 0x56070d2da1c0 2024-05-17T22:48:34.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.351+0000 7fbba9ffb700 10 client.25729 handle_command_reply: tid=0 2024-05-17T22:48:34.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.351+0000 7fbbd75bcb80 1 client.25729 shutdown 2024-05-17T22:48:34.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.351+0000 7fbbd75bcb80 1 -- 172.21.15.12:0/2975625250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d2c4eb0 msgr2=0x56070d2ca580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:34.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.351+0000 7fbbd75bcb80 1 --2- 172.21.15.12:0/2975625250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d2c4eb0 0x56070d2ca580 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc0e04c0 tx=0x7fbbbc0a9780 comp rx=0 tx=0).stop 2024-05-17T22:48:34.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.351+0000 7fbbd75bcb80 1 -- 172.21.15.12:0/2975625250 shutdown_connections 2024-05-17T22:48:34.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.351+0000 7fbbd75bcb80 1 --2- 172.21.15.12:0/2975625250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d2c4eb0 0x56070d2ca580 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:34.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.351+0000 7fbbd75bcb80 1 --2- 172.21.15.12:0/2975625250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56070d1319f0 0x56070d2c9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:34.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.351+0000 7fbbd75bcb80 1 --2- 172.21.15.12:0/2975625250 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x56070d2da1c0 0x56070d2dc650 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:34.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.351+0000 7fbbd75bcb80 1 --2- 172.21.15.12:0/2975625250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56070d2c0db0 0x56070d2ca040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:34.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.351+0000 7fbbd75bcb80 1 -- 172.21.15.12:0/2975625250 >> 172.21.15.12:0/2975625250 conn(0x56070d1917d0 msgr2=0x56070d2aa160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:48:34.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.351+0000 7fbba9ffb700 0 client.25729 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:48:34.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.352+0000 7fbbd75bcb80 1 -- 172.21.15.12:0/2975625250 shutdown_connections 2024-05-17T22:48:34.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.352+0000 7fbbd75bcb80 1 -- 172.21.15.12:0/2975625250 wait complete. 2024-05-17T22:48:34.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.352+0000 7fbbd75bcb80 20 client.25729 trim_cache size 0 max 16384 2024-05-17T22:48:34.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.388+0000 7fbbd75bcb80 1 Processor -- start 2024-05-17T22:48:34.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.388+0000 7fbbd75bcb80 1 -- start start 2024-05-17T22:48:34.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.388+0000 7fbbd75bcb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d1917d0 0x56070d2c8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:34.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.388+0000 7fbbd75bcb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56070d1319f0 0x56070d2a4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:34.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.388+0000 7fbbd75bcb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56070d2c0db0 0x56070d2ca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:34.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.388+0000 7fbbd75bcb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbba4011420 con 0x56070d1319f0 2024-05-17T22:48:34.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.388+0000 7fbbd75bcb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbba400ecb0 con 0x56070d2c0db0 2024-05-17T22:48:34.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.388+0000 7fbbd75bcb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbba400cb00 con 0x56070d1917d0 2024-05-17T22:48:34.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.388+0000 7fbbc2d9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56070d2c0db0 0x56070d2ca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:34.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.388+0000 7fbbc2d9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56070d2c0db0 0x56070d2ca3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:33482/0 (socket says 172.21.15.12:33482) 2024-05-17T22:48:34.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.388+0000 7fbbc2d9d700 1 -- 172.21.15.12:0/3642574376 learned_addr learned my addr 172.21.15.12:0/3642574376 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:48:34.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.388+0000 7fbbc259c700 1 --2- 172.21.15.12:0/3642574376 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d1917d0 0x56070d2c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:34.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.388+0000 7fbbc2d9d700 1 -- 172.21.15.12:0/3642574376 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d1917d0 msgr2=0x56070d2c8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:48:34.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.388+0000 7fbbc2d9d700 1 --2- 172.21.15.12:0/3642574376 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d1917d0 0x56070d2c8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:34.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.388+0000 7fbbc2d9d700 1 -- 172.21.15.12:0/3642574376 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56070d1319f0 msgr2=0x56070d2a4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:48:34.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.388+0000 7fbbc2d9d700 1 --2- 172.21.15.12:0/3642574376 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56070d1319f0 0x56070d2a4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:34.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.389+0000 7fbbc2d9d700 1 -- 172.21.15.12:0/3642574376 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56070d2a6c20 con 0x56070d2c0db0 2024-05-17T22:48:34.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.389+0000 7fbbc259c700 1 --2- 172.21.15.12:0/3642574376 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d1917d0 0x56070d2c8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:48:34.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.389+0000 7fbbc2d9d700 1 --2- 172.21.15.12:0/3642574376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56070d2c0db0 0x56070d2ca3d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc0a6010 tx=0x7fbbbc0a5db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74b6509f87f6ac91 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:48:34.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.389+0000 7fbbabfff700 1 -- 172.21.15.12:0/3642574376 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc0a8c60 con 0x56070d2c0db0 2024-05-17T22:48:34.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.389+0000 7fbbabfff700 1 -- 172.21.15.12:0/3642574376 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbbc0aa400 con 0x56070d2c0db0 2024-05-17T22:48:34.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.389+0000 7fbbabfff700 1 -- 172.21.15.12:0/3642574376 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc00caa0 con 0x56070d2c0db0 2024-05-17T22:48:34.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.390+0000 7fbbd75bcb80 1 -- 172.21.15.12:0/3642574376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56070d2c0db0 msgr2=0x56070d2ca3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:34.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.390+0000 7fbbd75bcb80 1 --2- 172.21.15.12:0/3642574376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56070d2c0db0 0x56070d2ca3d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc0a6010 tx=0x7fbbbc0a5db0 comp rx=0 tx=0).stop 2024-05-17T22:48:34.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.390+0000 7fbbd75bcb80 1 -- 172.21.15.12:0/3642574376 shutdown_connections 2024-05-17T22:48:34.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.390+0000 7fbbd75bcb80 1 --2- 172.21.15.12:0/3642574376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56070d2c0db0 0x56070d2ca3d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:34.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.390+0000 7fbbd75bcb80 1 --2- 172.21.15.12:0/3642574376 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d1917d0 0x56070d2c8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:34.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.390+0000 7fbbd75bcb80 1 --2- 172.21.15.12:0/3642574376 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56070d1319f0 0x56070d2a4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:34.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.390+0000 7fbbd75bcb80 1 -- 172.21.15.12:0/3642574376 >> 172.21.15.12:0/3642574376 conn(0x56070d2aa4c0 msgr2=0x56070d2cb3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:48:34.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.390+0000 7fbbd75bcb80 1 -- 172.21.15.12:0/3642574376 shutdown_connections 2024-05-17T22:48:34.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.390+0000 7fbbd75bcb80 1 -- 172.21.15.12:0/3642574376 wait complete. 2024-05-17T22:48:34.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.391+0000 7fbbd75bcb80 1 -- start start 2024-05-17T22:48:34.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.391+0000 7fbbd75bcb80 1 Processor -- start 2024-05-17T22:48:34.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.391+0000 7fbbd75bcb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56070d1319f0 0x56070d2c6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:34.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.391+0000 7fbbd75bcb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56070d1917d0 0x56070d2c7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:34.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.391+0000 7fbbd75bcb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d2c0db0 0x56070d343a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:34.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.391+0000 7fbbd75bcb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbba400d170 con 0x56070d1319f0 2024-05-17T22:48:34.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.391+0000 7fbbd75bcb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbba400e5e0 con 0x56070d1917d0 2024-05-17T22:48:34.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.391+0000 7fbbd75bcb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbba400d840 con 0x56070d2c0db0 2024-05-17T22:48:34.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.391+0000 7fbbd75bcb80 10 client.0 _ll_register_callbacks cb 0x56070d164b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:48:34.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.391+0000 7fbbc1d9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56070d1917d0 0x56070d2c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:34.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.391+0000 7fbbc1d9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56070d1917d0 0x56070d2c7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:33492/0 (socket says 172.21.15.12:33492) 2024-05-17T22:48:34.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.391+0000 7fbbc2d9d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d2c0db0 0x56070d343a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:34.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.391+0000 7fbbc1d9b700 1 -- 172.21.15.12:0/1956647150 learned_addr learned my addr 172.21.15.12:0/1956647150 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:48:34.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.391+0000 7fbbc259c700 1 --2- 172.21.15.12:0/1956647150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56070d1319f0 0x56070d2c6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:34.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.392+0000 7fbbc1d9b700 1 -- 172.21.15.12:0/1956647150 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d2c0db0 msgr2=0x56070d343a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:48:34.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.392+0000 7fbbc1d9b700 1 --2- 172.21.15.12:0/1956647150 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d2c0db0 0x56070d343a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:34.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.392+0000 7fbbc1d9b700 1 -- 172.21.15.12:0/1956647150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56070d1319f0 msgr2=0x56070d2c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:48:34.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.392+0000 7fbbc1d9b700 1 --2- 172.21.15.12:0/1956647150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56070d1319f0 0x56070d2c6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:34.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.392+0000 7fbbc1d9b700 1 -- 172.21.15.12:0/1956647150 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56070d2c79e0 con 0x56070d1917d0 2024-05-17T22:48:34.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.392+0000 7fbbc2d9d700 1 --2- 172.21.15.12:0/1956647150 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d2c0db0 0x56070d343a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:48:34.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.392+0000 7fbbc1d9b700 1 --2- 172.21.15.12:0/1956647150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56070d1917d0 0x56070d2c7180 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8000c00 tx=0x7fbbb80094e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f22984b53ef5bf9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:48:34.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.392+0000 7fbba9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:48:34.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.392+0000 7fbba9ffb700 1 -- 172.21.15.12:0/1956647150 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8005250 con 0x56070d1917d0 2024-05-17T22:48:34.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.392+0000 7fbbd75bcb80 1 -- 172.21.15.12:0/1956647150 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbbc0aa400 con 0x56070d1917d0 2024-05-17T22:48:34.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.392+0000 7fbba9ffb700 1 -- 172.21.15.12:0/1956647150 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fbbb80053f0 con 0x56070d1917d0 2024-05-17T22:48:34.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.392+0000 7fbba9ffb700 1 -- 172.21.15.12:0/1956647150 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fbbb80055f0 con 0x56070d1917d0 2024-05-17T22:48:34.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.392+0000 7fbbd75bcb80 10 client.25796 fetch_fsmap learned FSMap version 39 2024-05-17T22:48:34.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.392+0000 7fbbd75bcb80 1 -- 172.21.15.12:0/1956647150 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56070d344ee0 con 0x56070d1917d0 2024-05-17T22:48:34.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.393+0000 7fbba9ffb700 1 -- 172.21.15.12:0/1956647150 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbbc0aa400 con 0x56070d1917d0 2024-05-17T22:48:34.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.393+0000 7fbba9ffb700 1 -- 172.21.15.12:0/1956647150 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fbbb800b6f0 con 0x56070d1917d0 2024-05-17T22:48:34.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.393+0000 7fbbd75bcb80 10 client.25796 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:48:34.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.393+0000 7fbbd75bcb80 10 client.25796 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:48:34.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.393+0000 7fbbd75bcb80 20 client.25796 populate_metadata read hostname 'smithi012' 2024-05-17T22:48:34.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.393+0000 7fbbd75bcb80 1 --2- 172.21.15.12:0/1956647150 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x56070d2c4eb0 0x56070d347630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:34.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.393+0000 7fbbd75bcb80 4 client.25796 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:48:34.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.393+0000 7fbbd75bcb80 1 -- 172.21.15.12:0/1956647150 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56070d0aa360 con 0x56070d2c4eb0 2024-05-17T22:48:34.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.394+0000 7fbbc259c700 1 --2- 172.21.15.12:0/1956647150 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x56070d2c4eb0 0x56070d347630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:34.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.394+0000 7fbba9ffb700 10 client.25796 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:48:34.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.394+0000 7fbbc259c700 1 --2- 172.21.15.12:0/1956647150 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x56070d2c4eb0 0x56070d347630 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d280780896cf3c server_cookie=cf0ca6f8d31d13d9 in_seq=0 out_seq=0 2024-05-17T22:48:34.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.394+0000 7fbba9ffb700 1 -- 172.21.15.12:0/1956647150 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56070d0aa360 con 0x56070d2c4eb0 2024-05-17T22:48:34.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.394+0000 7fbba9ffb700 10 client.25796 handle_command_reply: tid=0 2024-05-17T22:48:34.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.394+0000 7fbbd75bcb80 1 client.25796 shutdown 2024-05-17T22:48:34.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.394+0000 7fbbd75bcb80 1 -- 172.21.15.12:0/1956647150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56070d1917d0 msgr2=0x56070d2c7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:34.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.394+0000 7fbbd75bcb80 1 --2- 172.21.15.12:0/1956647150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56070d1917d0 0x56070d2c7180 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8000c00 tx=0x7fbbb80094e0 comp rx=0 tx=0).stop 2024-05-17T22:48:34.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.395+0000 7fbbd75bcb80 1 -- 172.21.15.12:0/1956647150 shutdown_connections 2024-05-17T22:48:34.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.395+0000 7fbbd75bcb80 1 --2- 172.21.15.12:0/1956647150 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d2c0db0 0x56070d343a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:34.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.395+0000 7fbbd75bcb80 1 --2- 172.21.15.12:0/1956647150 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x56070d2c4eb0 0x56070d347630 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:34.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.395+0000 7fbbd75bcb80 1 --2- 172.21.15.12:0/1956647150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56070d1319f0 0x56070d2c6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:34.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.395+0000 7fbbd75bcb80 1 --2- 172.21.15.12:0/1956647150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56070d1917d0 0x56070d2c7180 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:34.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.395+0000 7fbbd75bcb80 1 -- 172.21.15.12:0/1956647150 >> 172.21.15.12:0/1956647150 conn(0x56070d2aa4c0 msgr2=0x56070d2cbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:48:34.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.395+0000 7fbba9ffb700 0 client.25796 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:48:34.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.395+0000 7fbbd75bcb80 1 -- 172.21.15.12:0/1956647150 shutdown_connections 2024-05-17T22:48:34.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.395+0000 7fbbd75bcb80 1 -- 172.21.15.12:0/1956647150 wait complete. 2024-05-17T22:48:34.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.395+0000 7fbbd75bcb80 20 client.25796 trim_cache size 0 max 16384 2024-05-17T22:48:34.404 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:48:34.404 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-17T22:48:34.404 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-17T22:48:34.404 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:48:34.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.395+0000 7fbbc8be1700 1 -- 172.21.15.12:0/901544640 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fbbac0bb220 msgr2=0x7fbbac0bd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:34.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.395+0000 7fbbc8be1700 1 --2- 172.21.15.12:0/901544640 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fbbac0bb220 0x7fbbac0bd730 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4005950 tx=0x7fbbb400b580 comp rx=0 tx=0).stop 2024-05-17T22:48:34.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.395+0000 7fbbc8be1700 1 -- 172.21.15.12:0/901544640 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbc419cfd0 msgr2=0x7fbbc419d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:34.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.395+0000 7fbbc8be1700 1 --2- 172.21.15.12:0/901544640 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbc419cfd0 0x7fbbc419d3e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc00efc0 tx=0x7fbbbc00c3f0 comp rx=0 tx=0).stop 2024-05-17T22:48:34.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.396+0000 7fbbc8be1700 1 -- 172.21.15.12:0/901544640 shutdown_connections 2024-05-17T22:48:34.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.396+0000 7fbbc8be1700 1 --2- 172.21.15.12:0/901544640 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fbbac0bb220 0x7fbbac0bd730 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:34.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.396+0000 7fbbc8be1700 1 --2- 172.21.15.12:0/901544640 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbc419cfd0 0x7fbbc419d3e0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:34.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.396+0000 7fbbc8be1700 1 --2- 172.21.15.12:0/901544640 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbc4102e20 0x7fbbc4198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:34.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.396+0000 7fbbc8be1700 1 --2- 172.21.15.12:0/901544640 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbbc41080a0 0x7fbbc4198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:34.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.396+0000 7fbbc8be1700 1 -- 172.21.15.12:0/901544640 >> 172.21.15.12:0/901544640 conn(0x7fbbc4079bd0 msgr2=0x7fbbc4105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:48:34.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.396+0000 7fbbc8be1700 1 -- 172.21.15.12:0/901544640 shutdown_connections 2024-05-17T22:48:34.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:34.396+0000 7fbbc8be1700 1 -- 172.21.15.12:0/901544640 wait complete. 2024-05-17T22:48:35.260 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-17T22:48:35.260 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T22:48:35.710 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:48:36.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:35 smithi012 ceph-mon[31632]: pgmap v719: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 MiB/s wr, 608 op/s 2024-05-17T22:48:36.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:35 smithi151 ceph-mon[36101]: pgmap v719: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 MiB/s wr, 608 op/s 2024-05-17T22:48:36.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:35 smithi096 ceph-mon[36065]: pgmap v719: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 MiB/s wr, 608 op/s 2024-05-17T22:48:36.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 555.0 s, 445.2 tps, lat 219.277 ms stddev 248.412 2024-05-17T22:48:38.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:37 smithi012 ceph-mon[31632]: pgmap v720: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.9 MiB/s wr, 487 op/s 2024-05-17T22:48:38.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:37 smithi151 ceph-mon[36101]: pgmap v720: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.9 MiB/s wr, 487 op/s 2024-05-17T22:48:38.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:37 smithi096 ceph-mon[36065]: pgmap v720: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.9 MiB/s wr, 487 op/s 2024-05-17T22:48:39.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.131+0000 7fb7e7ff5700 1 -- 172.21.15.12:0/2869960217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7e0110330 msgr2=0x7fb7e0110740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:39.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.131+0000 7fb7e7ff5700 1 --2- 172.21.15.12:0/2869960217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7e0110330 0x7fb7e0110740 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0008580 tx=0x7fb7d0008890 comp rx=0 tx=0).stop 2024-05-17T22:48:39.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.131+0000 7fb7e7ff5700 1 -- 172.21.15.12:0/2869960217 shutdown_connections 2024-05-17T22:48:39.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.131+0000 7fb7e7ff5700 1 --2- 172.21.15.12:0/2869960217 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7e010b180 0x7fb7e007ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:39.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.131+0000 7fb7e7ff5700 1 --2- 172.21.15.12:0/2869960217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7e0110330 0x7fb7e0110740 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:39.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.131+0000 7fb7e7ff5700 1 --2- 172.21.15.12:0/2869960217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7e010a830 0x7fb7e010ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:39.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.131+0000 7fb7e7ff5700 1 -- 172.21.15.12:0/2869960217 >> 172.21.15.12:0/2869960217 conn(0x7fb7e006efc0 msgr2=0x7fb7e006f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:48:39.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.135+0000 7fb7e7ff5700 1 -- 172.21.15.12:0/2869960217 shutdown_connections 2024-05-17T22:48:39.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.139+0000 7fb7e7ff5700 1 -- 172.21.15.12:0/2869960217 wait complete. 2024-05-17T22:48:39.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.140+0000 7fb7e7ff5700 1 Processor -- start 2024-05-17T22:48:39.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.142+0000 7fb7e7ff5700 1 -- start start 2024-05-17T22:48:39.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.143+0000 7fb7e7ff5700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7e010b180 0x7fb7e0080b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:39.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.143+0000 7fb7e7ff5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7e0110330 0x7fb7e0081070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:39.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.143+0000 7fb7e7ff5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7e0081650 0x7fb7e01cedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:39.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.143+0000 7fb7e7ff5700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e007ef10 con 0x7fb7e0081650 2024-05-17T22:48:39.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.143+0000 7fb7e7ff5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e007ed90 con 0x7fb7e0110330 2024-05-17T22:48:39.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.143+0000 7fb7e7ff5700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e007f090 con 0x7fb7e010b180 2024-05-17T22:48:39.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.143+0000 7fb7e5590700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7e0110330 0x7fb7e0081070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:39.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.143+0000 7fb7e5590700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7e0110330 0x7fb7e0081070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:33498/0 (socket says 172.21.15.12:33498) 2024-05-17T22:48:39.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.143+0000 7fb7e5590700 1 -- 172.21.15.12:0/2451468409 learned_addr learned my addr 172.21.15.12:0/2451468409 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:48:39.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.148+0000 7fb7e5d91700 1 --2- 172.21.15.12:0/2451468409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7e010b180 0x7fb7e0080b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:39.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.148+0000 7fb7e5590700 1 -- 172.21.15.12:0/2451468409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7e010b180 msgr2=0x7fb7e0080b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:39.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.148+0000 7fb7e5590700 1 --2- 172.21.15.12:0/2451468409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7e010b180 0x7fb7e0080b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:39.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.148+0000 7fb7e5590700 1 -- 172.21.15.12:0/2451468409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7e0081650 msgr2=0x7fb7e01cedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:39.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.148+0000 7fb7e6592700 1 --2- 172.21.15.12:0/2451468409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7e0081650 0x7fb7e01cedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:39.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.148+0000 7fb7e5590700 1 --2- 172.21.15.12:0/2451468409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7e0081650 0x7fb7e01cedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:39.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.148+0000 7fb7e5590700 1 -- 172.21.15.12:0/2451468409 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7e01cf5d0 con 0x7fb7e0110330 2024-05-17T22:48:39.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.149+0000 7fb7e5590700 1 --2- 172.21.15.12:0/2451468409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7e0110330 0x7fb7e0081070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb7d800b410 tx=0x7fb7d800b7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:48:39.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.150+0000 7fb7d6ffd700 1 -- 172.21.15.12:0/2451468409 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d8015030 con 0x7fb7e0110330 2024-05-17T22:48:39.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.150+0000 7fb7d6ffd700 1 -- 172.21.15.12:0/2451468409 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7d8011ae0 con 0x7fb7e0110330 2024-05-17T22:48:39.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.150+0000 7fb7d6ffd700 1 -- 172.21.15.12:0/2451468409 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d8010b70 con 0x7fb7e0110330 2024-05-17T22:48:39.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.151+0000 7fb7e7ff5700 1 -- 172.21.15.12:0/2451468409 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7e01cf8c0 con 0x7fb7e0110330 2024-05-17T22:48:39.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.151+0000 7fb7e7ff5700 1 -- 172.21.15.12:0/2451468409 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7e01cfdb0 con 0x7fb7e0110330 2024-05-17T22:48:39.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.152+0000 7fb7e5d91700 1 --2- 172.21.15.12:0/2451468409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7e010b180 0x7fb7e0080b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:48:39.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.152+0000 7fb7e6592700 1 --2- 172.21.15.12:0/2451468409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7e0081650 0x7fb7e01cedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:48:39.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.153+0000 7fb7e7ff5700 1 -- 172.21.15.12:0/2451468409 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7e010ba50 con 0x7fb7e0110330 2024-05-17T22:48:39.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.157+0000 7fb7d6ffd700 1 -- 172.21.15.12:0/2451468409 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb7d8018070 con 0x7fb7e0110330 2024-05-17T22:48:39.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.158+0000 7fb7d6ffd700 1 --2- 172.21.15.12:0/2451468409 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb7cc0b2960 0x7fb7cc0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:39.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.159+0000 7fb7d6ffd700 1 -- 172.21.15.12:0/2451468409 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fb7d80e2060 con 0x7fb7e0110330 2024-05-17T22:48:39.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.160+0000 7fb7e5d91700 1 --2- 172.21.15.12:0/2451468409 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb7cc0b2960 0x7fb7cc0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:39.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.162+0000 7fb7d6ffd700 1 -- 172.21.15.12:0/2451468409 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb7d80a9990 con 0x7fb7e0110330 2024-05-17T22:48:39.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.162+0000 7fb7e5d91700 1 --2- 172.21.15.12:0/2451468409 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb7cc0b2960 0x7fb7cc0b4e70 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0000c00 tx=0x7fb7d0005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:48:39.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.634+0000 7fb7e7ff5700 1 -- 172.21.15.12:0/2451468409 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7e0082060 con 0x7fb7e0110330 2024-05-17T22:48:39.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.638+0000 7fb7d6ffd700 1 -- 172.21.15.12:0/2451468409 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11870 (secure 0 0 0) 0x7fb7d80a9290 con 0x7fb7e0110330 2024-05-17T22:48:39.641 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:48:39.642 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1407219319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1407219319},{"type":"v1","addr":"172.21.15.12:6835","nonce":1407219319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/630977886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":630977886},{"type":"v1","addr":"172.21.15.12:6837","nonce":630977886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3731914742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3731914742},{"type":"v1","addr":"172.21.15.12:6839","nonce":3731914742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/3642142501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3642142501},{"type":"v1","addr":"172.21.15.12:6841","nonce":3642142501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3554053059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3554053059},{"type":"v1","addr":"172.21.15.151:6833","nonce":3554053059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/4228325569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":4228325569},{"type":"v1","addr":"172.21.15.96:6833","nonce":4228325569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T22:34:42.642111+0000","modified":"2024-05-17T22:47:38.155618+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24461,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/542323754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":542323754},{"type":"v1","addr":"172.21.15.151:6835","nonce":542323754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1758442232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1758442232},{"type":"v1","addr":"172.21.15.96:6835","nonce":1758442232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3403324038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3403324038},{"type":"v1","addr":"172.21.15.151:6837","nonce":3403324038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.151:6839/2338353849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2338353849},{"type":"v1","addr":"172.21.15.151:6839","nonce":2338353849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.96:6837/4222574338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":4222574338},{"type":"v1","addr":"172.21.15.96:6837","nonce":4222574338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/2986295519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2986295519},{"type":"v1","addr":"172.21.15.96:6839","nonce":2986295519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24457,24461,24482]},"id":1}]} 2024-05-17T22:48:39.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.644+0000 7fb7e7ff5700 1 -- 172.21.15.12:0/2451468409 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb7cc0b2960 msgr2=0x7fb7cc0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:39.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.644+0000 7fb7e7ff5700 1 --2- 172.21.15.12:0/2451468409 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb7cc0b2960 0x7fb7cc0b4e70 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0000c00 tx=0x7fb7d0005d40 comp rx=0 tx=0).stop 2024-05-17T22:48:39.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.644+0000 7fb7e7ff5700 1 -- 172.21.15.12:0/2451468409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7e0110330 msgr2=0x7fb7e0081070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:39.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.644+0000 7fb7e7ff5700 1 --2- 172.21.15.12:0/2451468409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7e0110330 0x7fb7e0081070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb7d800b410 tx=0x7fb7d800b7d0 comp rx=0 tx=0).stop 2024-05-17T22:48:39.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.644+0000 7fb7e7ff5700 1 -- 172.21.15.12:0/2451468409 shutdown_connections 2024-05-17T22:48:39.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.644+0000 7fb7e7ff5700 1 --2- 172.21.15.12:0/2451468409 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb7cc0b2960 0x7fb7cc0b4e70 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:39.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.644+0000 7fb7e7ff5700 1 --2- 172.21.15.12:0/2451468409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7e0081650 0x7fb7e01cedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:39.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.644+0000 7fb7e7ff5700 1 --2- 172.21.15.12:0/2451468409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7e010b180 0x7fb7e0080b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:39.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.644+0000 7fb7e7ff5700 1 --2- 172.21.15.12:0/2451468409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7e0110330 0x7fb7e0081070 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:39.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.644+0000 7fb7e7ff5700 1 -- 172.21.15.12:0/2451468409 >> 172.21.15.12:0/2451468409 conn(0x7fb7e006efc0 msgr2=0x7fb7e0074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:48:39.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.645+0000 7fb7e7ff5700 1 -- 172.21.15.12:0/2451468409 shutdown_connections 2024-05-17T22:48:39.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:39.645+0000 7fb7e7ff5700 1 -- 172.21.15.12:0/2451468409 wait complete. 2024-05-17T22:48:39.649 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 39 2024-05-17T22:48:40.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:39 smithi151 ceph-mon[36101]: pgmap v721: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.1 MiB/s wr, 516 op/s 2024-05-17T22:48:40.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:39 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/2451468409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:48:40.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:39 smithi096 ceph-mon[36065]: pgmap v721: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.1 MiB/s wr, 516 op/s 2024-05-17T22:48:40.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:39 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/2451468409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:48:40.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:39 smithi012 ceph-mon[31632]: pgmap v721: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.1 MiB/s wr, 516 op/s 2024-05-17T22:48:40.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:39 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/2451468409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:48:40.468 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-17T22:48:40.930 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:48:41.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 560.0 s, 497.6 tps, lat 205.965 ms stddev 221.435 2024-05-17T22:48:42.176 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:41 smithi012 ceph-mon[31632]: pgmap v722: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.8 MiB/s wr, 469 op/s 2024-05-17T22:48:42.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:41 smithi151 ceph-mon[36101]: pgmap v722: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.8 MiB/s wr, 469 op/s 2024-05-17T22:48:42.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:41 smithi096 ceph-mon[36065]: pgmap v722: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.8 MiB/s wr, 469 op/s 2024-05-17T22:48:42.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.928+0000 7f709b15f700 1 -- 172.21.15.12:0/2573669227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7094108840 msgr2=0x7f7094108c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:42.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.928+0000 7f709b15f700 1 --2- 172.21.15.12:0/2573669227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7094108840 0x7f7094108c50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f7080009990 tx=0x7f7080009ca0 comp rx=0 tx=0).stop 2024-05-17T22:48:42.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.928+0000 7f709b15f700 1 -- 172.21.15.12:0/2573669227 shutdown_connections 2024-05-17T22:48:42.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.928+0000 7f709b15f700 1 --2- 172.21.15.12:0/2573669227 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7094109190 0x7f7094110af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:42.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.928+0000 7f709b15f700 1 --2- 172.21.15.12:0/2573669227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7094103f10 0x7f7094104320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:42.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.928+0000 7f709b15f700 1 --2- 172.21.15.12:0/2573669227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7094108840 0x7f7094108c50 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:42.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.928+0000 7f709b15f700 1 -- 172.21.15.12:0/2573669227 >> 172.21.15.12:0/2573669227 conn(0x7f70940ff990 msgr2=0x7f7094101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:48:42.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.928+0000 7f709b15f700 1 -- 172.21.15.12:0/2573669227 shutdown_connections 2024-05-17T22:48:42.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.928+0000 7f709b15f700 1 -- 172.21.15.12:0/2573669227 wait complete. 2024-05-17T22:48:42.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.929+0000 7f709b15f700 1 Processor -- start 2024-05-17T22:48:42.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.929+0000 7f709b15f700 1 -- start start 2024-05-17T22:48:42.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.930+0000 7f709b15f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7094103f10 0x7f7094075b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:42.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.930+0000 7f7098efb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7094103f10 0x7f7094075b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:42.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.930+0000 7f7098efb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7094103f10 0x7f7094075b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:36046/0 (socket says 172.21.15.12:36046) 2024-05-17T22:48:42.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.930+0000 7f709b15f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7094108840 0x7f7094076070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:42.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.930+0000 7f709b15f700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7094109190 0x7f70940765b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:42.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.930+0000 7f709b15f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7094113da0 con 0x7f7094108840 2024-05-17T22:48:42.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.930+0000 7f709b15f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7094113c20 con 0x7f7094103f10 2024-05-17T22:48:42.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.930+0000 7f709b15f700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7094113f20 con 0x7f7094109190 2024-05-17T22:48:42.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.931+0000 7f70996fc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7094109190 0x7f70940765b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:42.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.931+0000 7f70996fc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7094109190 0x7f70940765b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:54056/0 (socket says 172.21.15.12:54056) 2024-05-17T22:48:42.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.931+0000 7f70996fc700 1 -- 172.21.15.12:0/3185150596 learned_addr learned my addr 172.21.15.12:0/3185150596 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:48:42.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.932+0000 7f7093fff700 1 --2- 172.21.15.12:0/3185150596 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7094108840 0x7f7094076070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:42.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.932+0000 7f70996fc700 1 -- 172.21.15.12:0/3185150596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7094103f10 msgr2=0x7f7094075b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:42.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.932+0000 7f70996fc700 1 --2- 172.21.15.12:0/3185150596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7094103f10 0x7f7094075b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:42.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.932+0000 7f70996fc700 1 -- 172.21.15.12:0/3185150596 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7094108840 msgr2=0x7f7094076070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:42.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.932+0000 7f70996fc700 1 --2- 172.21.15.12:0/3185150596 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7094108840 0x7f7094076070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:42.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.932+0000 7f70996fc700 1 -- 172.21.15.12:0/3185150596 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f709419a6a0 con 0x7f7094109190 2024-05-17T22:48:42.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.932+0000 7f7098efb700 1 --2- 172.21.15.12:0/3185150596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7094103f10 0x7f7094075b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:48:42.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.932+0000 7f7093fff700 1 --2- 172.21.15.12:0/3185150596 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7094108840 0x7f7094076070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:48:42.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.933+0000 7f70996fc700 1 --2- 172.21.15.12:0/3185150596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7094109190 0x7f70940765b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f708c00c370 tx=0x7f708c00c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:48:42.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.933+0000 7f7091ffb700 1 -- 172.21.15.12:0/3185150596 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708c00ce20 con 0x7f7094109190 2024-05-17T22:48:42.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.933+0000 7f709b15f700 1 -- 172.21.15.12:0/3185150596 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f709419a930 con 0x7f7094109190 2024-05-17T22:48:42.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.933+0000 7f709b15f700 1 -- 172.21.15.12:0/3185150596 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f709419ae70 con 0x7f7094109190 2024-05-17T22:48:42.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.933+0000 7f7091ffb700 1 -- 172.21.15.12:0/3185150596 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f708c0090d0 con 0x7f7094109190 2024-05-17T22:48:42.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.933+0000 7f7091ffb700 1 -- 172.21.15.12:0/3185150596 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708c0047c0 con 0x7f7094109190 2024-05-17T22:48:42.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.937+0000 7f7091ffb700 1 -- 172.21.15.12:0/3185150596 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f708c01a020 con 0x7f7094109190 2024-05-17T22:48:42.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.938+0000 7f7091ffb700 1 --2- 172.21.15.12:0/3185150596 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f70840b2880 0x7f70840b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:42.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.938+0000 7f7098efb700 1 --2- 172.21.15.12:0/3185150596 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f70840b2880 0x7f70840b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:42.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.938+0000 7f7091ffb700 1 -- 172.21.15.12:0/3185150596 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f708c0e1ef0 con 0x7f7094109190 2024-05-17T22:48:42.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.938+0000 7f70a9b3ab80 1 Processor -- start 2024-05-17T22:48:42.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.939+0000 7f70a9b3ab80 1 -- start start 2024-05-17T22:48:42.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.939+0000 7f7098efb700 1 --2- 172.21.15.12:0/3185150596 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f70840b2880 0x7f70840b4d90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7088011490 tx=0x7f7088009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:48:42.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.939+0000 7f70a9b3ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6122fb9f0 0x55c61247f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:42.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.939+0000 7f70a9b3ab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c61247faa0 0x55c612479c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:42.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.939+0000 7f70a9b3ab80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c61247a170 0x55c61247a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:42.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.939+0000 7f70a9b3ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c6123a8290 con 0x55c6122fb9f0 2024-05-17T22:48:42.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.939+0000 7f70a9b3ab80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c612397fd0 con 0x55c61247a170 2024-05-17T22:48:42.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.939+0000 7f70a9b3ab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c61239be70 con 0x55c61247faa0 2024-05-17T22:48:42.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.940+0000 7f70996fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c61247a170 0x55c61247a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:42.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.940+0000 7f70996fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c61247a170 0x55c61247a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:36050/0 (socket says 172.21.15.12:36050) 2024-05-17T22:48:42.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.940+0000 7f70996fc700 1 -- 172.21.15.12:0/2331806417 learned_addr learned my addr 172.21.15.12:0/2331806417 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:48:42.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.940+0000 7f7093fff700 1 --2- 172.21.15.12:0/2331806417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c61247faa0 0x55c612479c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:42.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.940+0000 7f70996fc700 1 -- 172.21.15.12:0/2331806417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c61247faa0 msgr2=0x55c612479c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:48:42.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.940+0000 7f70996fc700 1 --2- 172.21.15.12:0/2331806417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c61247faa0 0x55c612479c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:42.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.940+0000 7f70996fc700 1 -- 172.21.15.12:0/2331806417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6122fb9f0 msgr2=0x55c61247f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:48:42.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.940+0000 7f70996fc700 1 --2- 172.21.15.12:0/2331806417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6122fb9f0 0x55c61247f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:42.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.940+0000 7f70996fc700 1 -- 172.21.15.12:0/2331806417 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c612341590 con 0x55c61247a170 2024-05-17T22:48:42.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.940+0000 7f7093fff700 1 --2- 172.21.15.12:0/2331806417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c61247faa0 0x55c612479c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:48:42.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.941+0000 7f70996fc700 1 --2- 172.21.15.12:0/2331806417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c61247a170 0x55c61247a580 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f708c0a63f0 tx=0x7f708c0a88f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52fe2088081c5b0a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:48:42.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.941+0000 7f707effd700 1 -- 172.21.15.12:0/2331806417 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708c0a8c90 con 0x55c61247a170 2024-05-17T22:48:42.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.941+0000 7f707effd700 1 -- 172.21.15.12:0/2331806417 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f708c0a9880 con 0x55c61247a170 2024-05-17T22:48:42.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.941+0000 7f707effd700 1 -- 172.21.15.12:0/2331806417 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708c0b1b50 con 0x55c61247a170 2024-05-17T22:48:42.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.942+0000 7f70a9b3ab80 1 -- 172.21.15.12:0/2331806417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c61247a170 msgr2=0x55c61247a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:42.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.942+0000 7f70a9b3ab80 1 --2- 172.21.15.12:0/2331806417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c61247a170 0x55c61247a580 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f708c0a63f0 tx=0x7f708c0a88f0 comp rx=0 tx=0).stop 2024-05-17T22:48:42.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.942+0000 7f70a9b3ab80 1 -- 172.21.15.12:0/2331806417 shutdown_connections 2024-05-17T22:48:42.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.942+0000 7f70a9b3ab80 1 --2- 172.21.15.12:0/2331806417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c61247a170 0x55c61247a580 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:42.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.942+0000 7f70a9b3ab80 1 --2- 172.21.15.12:0/2331806417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6122fb9f0 0x55c61247f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:42.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.942+0000 7f70a9b3ab80 1 --2- 172.21.15.12:0/2331806417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c61247faa0 0x55c612479c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:42.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.942+0000 7f70a9b3ab80 1 -- 172.21.15.12:0/2331806417 >> 172.21.15.12:0/2331806417 conn(0x55c61233ff00 msgr2=0x55c612474b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:48:42.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.942+0000 7f70a9b3ab80 1 -- 172.21.15.12:0/2331806417 shutdown_connections 2024-05-17T22:48:42.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.943+0000 7f70a9b3ab80 1 -- 172.21.15.12:0/2331806417 wait complete. 2024-05-17T22:48:42.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.943+0000 7f70a9b3ab80 1 -- start start 2024-05-17T22:48:42.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.943+0000 7f70a9b3ab80 1 Processor -- start 2024-05-17T22:48:42.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.944+0000 7f70a9b3ab80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6122fb9f0 0x55c6124832c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:42.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.944+0000 7f70a9b3ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c61247a170 0x55c612483800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:42.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.944+0000 7f70a9b3ab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c61247faa0 0x55c61248f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:42.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.944+0000 7f70a9b3ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c61229f150 con 0x55c61247a170 2024-05-17T22:48:42.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.944+0000 7f70a9b3ab80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c612338e80 con 0x55c6122fb9f0 2024-05-17T22:48:42.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.944+0000 7f70a9b3ab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c612486390 con 0x55c61247faa0 2024-05-17T22:48:42.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.944+0000 7f7098efb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6122fb9f0 0x55c6124832c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:42.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.945+0000 7f70a9b3ab80 10 client.0 _ll_register_callbacks cb 0x55c612332f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:48:42.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.945+0000 7f7098efb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6122fb9f0 0x55c6124832c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:36062/0 (socket says 172.21.15.12:36062) 2024-05-17T22:48:42.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.945+0000 7f7098efb700 1 -- 172.21.15.12:0/1087308207 learned_addr learned my addr 172.21.15.12:0/1087308207 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:48:42.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.945+0000 7f70996fc700 1 --2- 172.21.15.12:0/1087308207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c61247faa0 0x55c61248f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:42.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.945+0000 7f7098efb700 1 -- 172.21.15.12:0/1087308207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c61247faa0 msgr2=0x55c61248f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:48:42.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.945+0000 7f7093fff700 1 --2- 172.21.15.12:0/1087308207 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c61247a170 0x55c612483800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:42.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.945+0000 7f7098efb700 1 --2- 172.21.15.12:0/1087308207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c61247faa0 0x55c61248f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:42.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.945+0000 7f7098efb700 1 -- 172.21.15.12:0/1087308207 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c61247a170 msgr2=0x55c612483800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:48:42.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.945+0000 7f7098efb700 1 --2- 172.21.15.12:0/1087308207 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c61247a170 0x55c612483800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:42.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.945+0000 7f7098efb700 1 -- 172.21.15.12:0/1087308207 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6122e8480 con 0x55c6122fb9f0 2024-05-17T22:48:42.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.945+0000 7f7093fff700 1 --2- 172.21.15.12:0/1087308207 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c61247a170 0x55c612483800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:48:42.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.945+0000 7f7098efb700 1 --2- 172.21.15.12:0/1087308207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6122fb9f0 0x55c6124832c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7088003a00 tx=0x7f7088004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1034629f7d9c72e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:48:42.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.945+0000 7f7073fff700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:48:42.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.946+0000 7f7073fff700 1 -- 172.21.15.12:0/1087308207 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708800ebd0 con 0x55c6122fb9f0 2024-05-17T22:48:42.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.946+0000 7f70a9b3ab80 1 -- 172.21.15.12:0/1087308207 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f708c0a9880 con 0x55c6122fb9f0 2024-05-17T22:48:42.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.946+0000 7f7073fff700 1 -- 172.21.15.12:0/1087308207 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f708800ed70 con 0x55c6122fb9f0 2024-05-17T22:48:42.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.946+0000 7f7073fff700 1 -- 172.21.15.12:0/1087308207 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7088026400 con 0x55c6122fb9f0 2024-05-17T22:48:42.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.946+0000 7f70a9b3ab80 10 client.25811 fetch_fsmap learned FSMap version 39 2024-05-17T22:48:42.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.946+0000 7f70a9b3ab80 1 -- 172.21.15.12:0/1087308207 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c612490f50 con 0x55c6122fb9f0 2024-05-17T22:48:42.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.946+0000 7f7073fff700 1 -- 172.21.15.12:0/1087308207 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f708c0a9880 con 0x55c6122fb9f0 2024-05-17T22:48:42.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.947+0000 7f7073fff700 1 -- 172.21.15.12:0/1087308207 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f708801e4c0 con 0x55c6122fb9f0 2024-05-17T22:48:42.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.947+0000 7f70a9b3ab80 10 client.25811 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:48:42.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.947+0000 7f70a9b3ab80 10 client.25811 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-17T22:48:42.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.947+0000 7f70a9b3ab80 20 client.25811 populate_metadata read hostname 'smithi012' 2024-05-17T22:48:42.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.947+0000 7f70a9b3ab80 1 --2- 172.21.15.12:0/1087308207 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55c6124934e0 0x55c612495900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:42.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.947+0000 7f70a9b3ab80 4 client.25811 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:48:42.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.947+0000 7f70a9b3ab80 1 -- 172.21.15.12:0/1087308207 --> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6123aa1e0 con 0x55c6124934e0 2024-05-17T22:48:42.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.947+0000 7f7093fff700 1 --2- 172.21.15.12:0/1087308207 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55c6124934e0 0x55c612495900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:42.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.948+0000 7f7093fff700 1 --2- 172.21.15.12:0/1087308207 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55c6124934e0 0x55c612495900 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d2a4724d8a0a1b75 server_cookie=94ddbff781578849 in_seq=0 out_seq=0 2024-05-17T22:48:42.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.948+0000 7f7073fff700 10 client.25811 ms_handle_connect on v2:172.21.15.151:6838/2338353849 2024-05-17T22:48:42.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.949+0000 7f7073fff700 1 -- 172.21.15.12:0/1087308207 <== mds.2 v2:172.21.15.151:6838/2338353849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c6123aa1e0 con 0x55c6124934e0 2024-05-17T22:48:42.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.949+0000 7f7073fff700 10 client.25811 handle_command_reply: tid=0 2024-05-17T22:48:42.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.949+0000 7f70a9b3ab80 1 client.25811 shutdown 2024-05-17T22:48:42.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.949+0000 7f70a9b3ab80 1 -- 172.21.15.12:0/1087308207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6122fb9f0 msgr2=0x55c6124832c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:42.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.949+0000 7f70a9b3ab80 1 --2- 172.21.15.12:0/1087308207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6122fb9f0 0x55c6124832c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7088003a00 tx=0x7f7088004750 comp rx=0 tx=0).stop 2024-05-17T22:48:42.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.949+0000 7f70a9b3ab80 1 -- 172.21.15.12:0/1087308207 shutdown_connections 2024-05-17T22:48:42.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.949+0000 7f70a9b3ab80 1 --2- 172.21.15.12:0/1087308207 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55c6124934e0 0x55c612495900 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:42.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.949+0000 7f70a9b3ab80 1 --2- 172.21.15.12:0/1087308207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c61247faa0 0x55c61248f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:42.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.949+0000 7f70a9b3ab80 1 --2- 172.21.15.12:0/1087308207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6122fb9f0 0x55c6124832c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:42.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.949+0000 7f70a9b3ab80 1 --2- 172.21.15.12:0/1087308207 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c61247a170 0x55c612483800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:42.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.949+0000 7f70a9b3ab80 1 -- 172.21.15.12:0/1087308207 >> 172.21.15.12:0/1087308207 conn(0x55c61233ff00 msgr2=0x55c61246eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:48:42.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.949+0000 7f7073fff700 0 client.25811 ms_handle_reset on v2:172.21.15.151:6838/2338353849 2024-05-17T22:48:42.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.949+0000 7f70a9b3ab80 1 -- 172.21.15.12:0/1087308207 shutdown_connections 2024-05-17T22:48:42.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.949+0000 7f70a9b3ab80 1 -- 172.21.15.12:0/1087308207 wait complete. 2024-05-17T22:48:42.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.949+0000 7f70a9b3ab80 20 client.25811 trim_cache size 0 max 16384 2024-05-17T22:48:42.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.974+0000 7f70a9b3ab80 1 Processor -- start 2024-05-17T22:48:42.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.974+0000 7f70a9b3ab80 1 -- start start 2024-05-17T22:48:42.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.974+0000 7f70a9b3ab80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6122fb9f0 0x55c612479c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:42.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.974+0000 7f70a9b3ab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c61247a170 0x55c612476050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:42.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.974+0000 7f70a9b3ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c61247faa0 0x55c61246eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:42.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.974+0000 7f70a9b3ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f70941b8c40 con 0x55c61247faa0 2024-05-17T22:48:42.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.974+0000 7f70a9b3ab80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f70941b64d0 con 0x55c6122fb9f0 2024-05-17T22:48:42.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.974+0000 7f70a9b3ab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f70941b4320 con 0x55c61247a170 2024-05-17T22:48:42.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.974+0000 7f7098efb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6122fb9f0 0x55c612479c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:42.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.974+0000 7f7098efb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6122fb9f0 0x55c612479c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:36076/0 (socket says 172.21.15.12:36076) 2024-05-17T22:48:42.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.974+0000 7f7098efb700 1 -- 172.21.15.12:0/3359369313 learned_addr learned my addr 172.21.15.12:0/3359369313 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:48:42.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.974+0000 7f70996fc700 1 --2- 172.21.15.12:0/3359369313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c61247faa0 0x55c61246eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:42.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.974+0000 7f7093fff700 1 --2- 172.21.15.12:0/3359369313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c61247a170 0x55c612476050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:42.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.975+0000 7f7098efb700 1 -- 172.21.15.12:0/3359369313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c61247a170 msgr2=0x55c612476050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:48:42.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.975+0000 7f7098efb700 1 --2- 172.21.15.12:0/3359369313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c61247a170 0x55c612476050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:42.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.975+0000 7f7098efb700 1 -- 172.21.15.12:0/3359369313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c61247faa0 msgr2=0x55c61246eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:48:42.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.975+0000 7f7098efb700 1 --2- 172.21.15.12:0/3359369313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c61247faa0 0x55c61246eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:42.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.975+0000 7f7098efb700 1 -- 172.21.15.12:0/3359369313 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c612481810 con 0x55c6122fb9f0 2024-05-17T22:48:42.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.975+0000 7f7093fff700 1 --2- 172.21.15.12:0/3359369313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c61247a170 0x55c612476050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:48:42.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.975+0000 7f7098efb700 1 --2- 172.21.15.12:0/3359369313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6122fb9f0 0x55c612479c30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7088004210 tx=0x7f7088003e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5835a0b770bfbc9f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:48:42.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.975+0000 7f707e7fc700 1 -- 172.21.15.12:0/3359369313 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7088038070 con 0x55c6122fb9f0 2024-05-17T22:48:42.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.975+0000 7f707e7fc700 1 -- 172.21.15.12:0/3359369313 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f708802b070 con 0x55c6122fb9f0 2024-05-17T22:48:42.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.975+0000 7f707e7fc700 1 -- 172.21.15.12:0/3359369313 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708802f030 con 0x55c6122fb9f0 2024-05-17T22:48:42.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.976+0000 7f70a9b3ab80 1 -- 172.21.15.12:0/3359369313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6122fb9f0 msgr2=0x55c612479c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:42.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.976+0000 7f70a9b3ab80 1 --2- 172.21.15.12:0/3359369313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6122fb9f0 0x55c612479c30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7088004210 tx=0x7f7088003e70 comp rx=0 tx=0).stop 2024-05-17T22:48:42.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.977+0000 7f70a9b3ab80 1 -- 172.21.15.12:0/3359369313 shutdown_connections 2024-05-17T22:48:42.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.977+0000 7f70a9b3ab80 1 --2- 172.21.15.12:0/3359369313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c61247faa0 0x55c61246eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:42.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.977+0000 7f70a9b3ab80 1 --2- 172.21.15.12:0/3359369313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6122fb9f0 0x55c612479c30 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:42.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.977+0000 7f70a9b3ab80 1 --2- 172.21.15.12:0/3359369313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c61247a170 0x55c612476050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:42.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.977+0000 7f70a9b3ab80 1 -- 172.21.15.12:0/3359369313 >> 172.21.15.12:0/3359369313 conn(0x55c61233ff00 msgr2=0x55c61247e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:48:42.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.977+0000 7f70a9b3ab80 1 -- 172.21.15.12:0/3359369313 shutdown_connections 2024-05-17T22:48:42.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.977+0000 7f70a9b3ab80 1 -- 172.21.15.12:0/3359369313 wait complete. 2024-05-17T22:48:42.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.978+0000 7f70a9b3ab80 1 -- start start 2024-05-17T22:48:42.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.978+0000 7f70a9b3ab80 1 Processor -- start 2024-05-17T22:48:42.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.978+0000 7f70a9b3ab80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6122fb9f0 0x55c61248b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:42.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.979+0000 7f70a9b3ab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c61247a170 0x55c61248b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:42.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.979+0000 7f70a9b3ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c61247faa0 0x55c61248e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:42.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.979+0000 7f7098efb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6122fb9f0 0x55c61248b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:42.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.979+0000 7f7098efb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6122fb9f0 0x55c61248b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:36080/0 (socket says 172.21.15.12:36080) 2024-05-17T22:48:42.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.979+0000 7f7098efb700 1 -- 172.21.15.12:0/3982844421 learned_addr learned my addr 172.21.15.12:0/3982844421 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:48:42.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.979+0000 7f70a9b3ab80 1 -- 172.21.15.12:0/3982844421 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f70941b4990 con 0x55c61247faa0 2024-05-17T22:48:42.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.979+0000 7f70a9b3ab80 1 -- 172.21.15.12:0/3982844421 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f70941b5e00 con 0x55c6122fb9f0 2024-05-17T22:48:42.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.979+0000 7f7093fff700 1 --2- 172.21.15.12:0/3982844421 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c61247a170 0x55c61248b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:42.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.979+0000 7f70a9b3ab80 1 -- 172.21.15.12:0/3982844421 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f70941b5060 con 0x55c61247a170 2024-05-17T22:48:42.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.979+0000 7f70996fc700 1 --2- 172.21.15.12:0/3982844421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c61247faa0 0x55c61248e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:42.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.979+0000 7f70a9b3ab80 10 client.0 _ll_register_callbacks cb 0x55c612332f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:48:42.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.979+0000 7f7098efb700 1 -- 172.21.15.12:0/3982844421 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c61247a170 msgr2=0x55c61248b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:48:42.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.979+0000 7f7098efb700 1 --2- 172.21.15.12:0/3982844421 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c61247a170 0x55c61248b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:42.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.979+0000 7f7098efb700 1 -- 172.21.15.12:0/3982844421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c61247faa0 msgr2=0x55c61248e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:48:42.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.979+0000 7f7098efb700 1 --2- 172.21.15.12:0/3982844421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c61247faa0 0x55c61248e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:42.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.980+0000 7f7098efb700 1 -- 172.21.15.12:0/3982844421 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c61248e6c0 con 0x55c6122fb9f0 2024-05-17T22:48:42.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.980+0000 7f70a9b3ab80 1 -- 172.21.15.12:0/3982844421 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f708802b070 con 0x55c6122fb9f0 2024-05-17T22:48:42.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.980+0000 7f7098efb700 1 --2- 172.21.15.12:0/3982844421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6122fb9f0 0x55c61248b470 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f708800ec80 tx=0x7f708800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b66cfc5800a4c1c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:48:42.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.980+0000 7f7073fff700 10 client.25817 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:48:42.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.980+0000 7f7073fff700 1 -- 172.21.15.12:0/3982844421 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7088040030 con 0x55c6122fb9f0 2024-05-17T22:48:42.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.980+0000 7f7073fff700 1 -- 172.21.15.12:0/3982844421 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f7088005250 con 0x55c6122fb9f0 2024-05-17T22:48:42.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.980+0000 7f7073fff700 1 -- 172.21.15.12:0/3982844421 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7088038070 con 0x55c6122fb9f0 2024-05-17T22:48:42.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.980+0000 7f70a9b3ab80 10 client.25817 fetch_fsmap learned FSMap version 39 2024-05-17T22:48:42.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.980+0000 7f70a9b3ab80 1 -- 172.21.15.12:0/3982844421 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({fsmap=39}) -- 0x55c6124fd180 con 0x55c6122fb9f0 2024-05-17T22:48:42.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.981+0000 7f7073fff700 1 -- 172.21.15.12:0/3982844421 <== mon.2 v2:172.21.15.96:3300/0 4 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f7088005490 con 0x55c6122fb9f0 2024-05-17T22:48:42.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.981+0000 7f70a9b3ab80 10 client.25817 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:48:42.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.981+0000 7f70a9b3ab80 10 client.25817 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-17T22:48:42.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.981+0000 7f70a9b3ab80 20 client.25817 populate_metadata read hostname 'smithi012' 2024-05-17T22:48:42.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.981+0000 7f70a9b3ab80 1 --2- 172.21.15.12:0/3982844421 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55c6124fd770 0x55c6124ffb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:42.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.981+0000 7f70a9b3ab80 4 client.25817 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:48:42.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.981+0000 7f70a9b3ab80 1 -- 172.21.15.12:0/3982844421 --> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c6123a39c0 con 0x55c6124fd770 2024-05-17T22:48:42.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.981+0000 7f7093fff700 1 --2- 172.21.15.12:0/3982844421 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55c6124fd770 0x55c6124ffb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:42.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.982+0000 7f7073fff700 10 client.25817 ms_handle_connect on v2:172.21.15.151:6838/2338353849 2024-05-17T22:48:42.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.982+0000 7f7093fff700 1 --2- 172.21.15.12:0/3982844421 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55c6124fd770 0x55c6124ffb90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5c6d3fdc9f43f24a server_cookie=b9dd4dc978c96496 in_seq=0 out_seq=0 2024-05-17T22:48:42.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.982+0000 7f7073fff700 1 -- 172.21.15.12:0/3982844421 <== mds.2 v2:172.21.15.151:6838/2338353849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c6123a39c0 con 0x55c6124fd770 2024-05-17T22:48:42.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.982+0000 7f7073fff700 10 client.25817 handle_command_reply: tid=0 2024-05-17T22:48:42.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.982+0000 7f70a9b3ab80 1 client.25817 shutdown 2024-05-17T22:48:42.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.982+0000 7f70a9b3ab80 1 -- 172.21.15.12:0/3982844421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6122fb9f0 msgr2=0x55c61248b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:42.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.982+0000 7f70a9b3ab80 1 --2- 172.21.15.12:0/3982844421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6122fb9f0 0x55c61248b470 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f708800ec80 tx=0x7f708800ecb0 comp rx=0 tx=0).stop 2024-05-17T22:48:42.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.982+0000 7f70a9b3ab80 1 -- 172.21.15.12:0/3982844421 shutdown_connections 2024-05-17T22:48:42.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.983+0000 7f70a9b3ab80 1 --2- 172.21.15.12:0/3982844421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c61247faa0 0x55c61248e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:42.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.983+0000 7f70a9b3ab80 1 --2- 172.21.15.12:0/3982844421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6122fb9f0 0x55c61248b470 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:42.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.983+0000 7f70a9b3ab80 1 --2- 172.21.15.12:0/3982844421 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55c6124fd770 0x55c6124ffb90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:42.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.983+0000 7f7073fff700 0 client.25817 ms_handle_reset on v2:172.21.15.151:6838/2338353849 2024-05-17T22:48:42.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.983+0000 7f70a9b3ab80 1 --2- 172.21.15.12:0/3982844421 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c61247a170 0x55c61248b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:42.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.983+0000 7f70a9b3ab80 1 -- 172.21.15.12:0/3982844421 >> 172.21.15.12:0/3982844421 conn(0x55c61233ff00 msgr2=0x55c61247d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:48:42.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.983+0000 7f70a9b3ab80 1 -- 172.21.15.12:0/3982844421 shutdown_connections 2024-05-17T22:48:42.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.983+0000 7f70a9b3ab80 1 -- 172.21.15.12:0/3982844421 wait complete. 2024-05-17T22:48:42.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.983+0000 7f70a9b3ab80 20 client.25817 trim_cache size 0 max 16384 2024-05-17T22:48:42.991 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:48:42.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.984+0000 7f709b15f700 1 -- 172.21.15.12:0/3185150596 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f70840b2880 msgr2=0x7f70840b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:42.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.984+0000 7f709b15f700 1 --2- 172.21.15.12:0/3185150596 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f70840b2880 0x7f70840b4d90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7088011490 tx=0x7f7088009260 comp rx=0 tx=0).stop 2024-05-17T22:48:42.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.984+0000 7f709b15f700 1 -- 172.21.15.12:0/3185150596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7094109190 msgr2=0x7f70940765b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:42.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.984+0000 7f709b15f700 1 --2- 172.21.15.12:0/3185150596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7094109190 0x7f70940765b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f708c00c370 tx=0x7f708c00c680 comp rx=0 tx=0).stop 2024-05-17T22:48:42.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.984+0000 7f709b15f700 1 -- 172.21.15.12:0/3185150596 shutdown_connections 2024-05-17T22:48:42.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.984+0000 7f709b15f700 1 --2- 172.21.15.12:0/3185150596 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f70840b2880 0x7f70840b4d90 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:42.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.984+0000 7f709b15f700 1 --2- 172.21.15.12:0/3185150596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7094109190 0x7f70940765b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:42.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.984+0000 7f709b15f700 1 --2- 172.21.15.12:0/3185150596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7094103f10 0x7f7094075b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:42.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.984+0000 7f709b15f700 1 --2- 172.21.15.12:0/3185150596 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7094108840 0x7f7094076070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:42.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.984+0000 7f709b15f700 1 -- 172.21.15.12:0/3185150596 >> 172.21.15.12:0/3185150596 conn(0x7f70940ff990 msgr2=0x7f7094101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:48:42.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.986+0000 7f709b15f700 1 -- 172.21.15.12:0/3185150596 shutdown_connections 2024-05-17T22:48:42.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:42.986+0000 7f709b15f700 1 -- 172.21.15.12:0/3185150596 wait complete. 2024-05-17T22:48:44.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:43 smithi151 ceph-mon[36101]: pgmap v723: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.2 MiB/s wr, 411 op/s 2024-05-17T22:48:44.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:43 smithi096 ceph-mon[36065]: pgmap v723: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.2 MiB/s wr, 411 op/s 2024-05-17T22:48:44.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:43 smithi012 ceph-mon[31632]: pgmap v723: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.2 MiB/s wr, 411 op/s 2024-05-17T22:48:45.095 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-17T22:48:45.599 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:48:45.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:45 smithi012 ceph-mon[31632]: pgmap v724: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.9 MiB/s wr, 508 op/s 2024-05-17T22:48:46.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:45 smithi151 ceph-mon[36101]: pgmap v724: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.9 MiB/s wr, 508 op/s 2024-05-17T22:48:46.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:45 smithi096 ceph-mon[36065]: pgmap v724: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.9 MiB/s wr, 508 op/s 2024-05-17T22:48:46.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 565.0 s, 528.8 tps, lat 183.938 ms stddev 218.458 2024-05-17T22:48:47.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.539+0000 7efd099b5700 1 -- 172.21.15.12:0/1081247558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcfc095820 msgr2=0x7efcfc095c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:47.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.539+0000 7efd099b5700 1 --2- 172.21.15.12:0/1081247558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcfc095820 0x7efcfc095c30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7efcf400ba80 tx=0x7efcf400bd90 comp rx=0 tx=0).stop 2024-05-17T22:48:47.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.539+0000 7efd099b5700 1 -- 172.21.15.12:0/1081247558 shutdown_connections 2024-05-17T22:48:47.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.539+0000 7efd099b5700 1 --2- 172.21.15.12:0/1081247558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efcfc09a2e0 0x7efcfc0a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:47.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.539+0000 7efd099b5700 1 --2- 172.21.15.12:0/1081247558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcfc095820 0x7efcfc095c30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:47.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.539+0000 7efd099b5700 1 --2- 172.21.15.12:0/1081247558 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efcfc099990 0x7efcfc099da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:47.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.539+0000 7efd099b5700 1 -- 172.21.15.12:0/1081247558 >> 172.21.15.12:0/1081247558 conn(0x7efcfc00a2a0 msgr2=0x7efcfc00a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:48:47.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.540+0000 7efd099b5700 1 -- 172.21.15.12:0/1081247558 shutdown_connections 2024-05-17T22:48:47.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.540+0000 7efd099b5700 1 -- 172.21.15.12:0/1081247558 wait complete. 2024-05-17T22:48:47.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.540+0000 7efd099b5700 1 Processor -- start 2024-05-17T22:48:47.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.543+0000 7efd099b5700 1 -- start start 2024-05-17T22:48:47.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.543+0000 7efd099b5700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efcfc095820 0x7efcfc09bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:47.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.543+0000 7efd099b5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcfc099990 0x7efcfc09c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:47.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.544+0000 7efd099b5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efcfc09a2e0 0x7efcfc09c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:47.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.544+0000 7efd027fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcfc099990 0x7efcfc09c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:47.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.544+0000 7efd027fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcfc099990 0x7efcfc09c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:36088/0 (socket says 172.21.15.12:36088) 2024-05-17T22:48:47.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.544+0000 7efd027fc700 1 -- 172.21.15.12:0/1513854746 learned_addr learned my addr 172.21.15.12:0/1513854746 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:48:47.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.544+0000 7efd037fe700 1 --2- 172.21.15.12:0/1513854746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efcfc09a2e0 0x7efcfc09c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:47.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.544+0000 7efd099b5700 1 -- 172.21.15.12:0/1513854746 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc0a3e90 con 0x7efcfc09a2e0 2024-05-17T22:48:47.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.544+0000 7efd099b5700 1 -- 172.21.15.12:0/1513854746 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc0a3d10 con 0x7efcfc099990 2024-05-17T22:48:47.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.544+0000 7efd099b5700 1 -- 172.21.15.12:0/1513854746 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc0a4010 con 0x7efcfc095820 2024-05-17T22:48:47.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.544+0000 7efd02ffd700 1 --2- 172.21.15.12:0/1513854746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efcfc095820 0x7efcfc09bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:47.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.545+0000 7efd027fc700 1 -- 172.21.15.12:0/1513854746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efcfc095820 msgr2=0x7efcfc09bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:47.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.545+0000 7efd027fc700 1 --2- 172.21.15.12:0/1513854746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efcfc095820 0x7efcfc09bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:47.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.545+0000 7efd027fc700 1 -- 172.21.15.12:0/1513854746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efcfc09a2e0 msgr2=0x7efcfc09c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:47.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.545+0000 7efd027fc700 1 --2- 172.21.15.12:0/1513854746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efcfc09a2e0 0x7efcfc09c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:47.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.545+0000 7efd027fc700 1 -- 172.21.15.12:0/1513854746 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcfc13f1d0 con 0x7efcfc099990 2024-05-17T22:48:47.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.545+0000 7efd027fc700 1 --2- 172.21.15.12:0/1513854746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcfc099990 0x7efcfc09c1c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efcf800e9f0 tx=0x7efcf800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:48:47.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.545+0000 7efd089b3700 1 -- 172.21.15.12:0/1513854746 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf800c9c0 con 0x7efcfc099990 2024-05-17T22:48:47.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.545+0000 7efd089b3700 1 -- 172.21.15.12:0/1513854746 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcf800cb60 con 0x7efcfc099990 2024-05-17T22:48:47.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.545+0000 7efd099b5700 1 -- 172.21.15.12:0/1513854746 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcfc13f460 con 0x7efcfc099990 2024-05-17T22:48:47.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.545+0000 7efd089b3700 1 -- 172.21.15.12:0/1513854746 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf8018420 con 0x7efcfc099990 2024-05-17T22:48:47.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.546+0000 7efd099b5700 1 -- 172.21.15.12:0/1513854746 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcfc13f9f0 con 0x7efcfc099990 2024-05-17T22:48:47.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.550+0000 7efd089b3700 1 -- 172.21.15.12:0/1513854746 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7efcf80185c0 con 0x7efcfc099990 2024-05-17T22:48:47.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.551+0000 7efd18390b80 1 Processor -- start 2024-05-17T22:48:47.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.551+0000 7efd18390b80 1 -- start start 2024-05-17T22:48:47.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.551+0000 7efd18390b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b137b319f0 0x55b137cb5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:47.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.551+0000 7efd18390b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b137cb5cb0 0x55b137cafc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:47.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.551+0000 7efd18390b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b137cb0170 0x55b137cb0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:47.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.552+0000 7efd18390b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b137bde290 con 0x55b137b319f0 2024-05-17T22:48:47.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.552+0000 7efd18390b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b137bcdfd0 con 0x55b137cb5cb0 2024-05-17T22:48:47.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.552+0000 7efd18390b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b137bd1e70 con 0x55b137cb0170 2024-05-17T22:48:47.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.552+0000 7efd037fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b137b319f0 0x55b137cb5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:47.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.552+0000 7efd037fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b137b319f0 0x55b137cb5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60654/0 (socket says 172.21.15.12:60654) 2024-05-17T22:48:47.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.552+0000 7efd037fe700 1 -- 172.21.15.12:0/794275848 learned_addr learned my addr 172.21.15.12:0/794275848 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:48:47.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.552+0000 7efd02ffd700 1 --2- 172.21.15.12:0/794275848 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b137cb5cb0 0x55b137cafc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:47.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.552+0000 7efd027fc700 1 --2- 172.21.15.12:0/794275848 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b137cb0170 0x55b137cb0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:47.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.552+0000 7efd089b3700 1 --2- 172.21.15.12:0/1513854746 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7efcec0b2900 0x7efcec0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:47.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.552+0000 7efd037fe700 1 -- 172.21.15.12:0/794275848 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b137cb0170 msgr2=0x55b137cb0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:48:47.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.552+0000 7efd037fe700 1 --2- 172.21.15.12:0/794275848 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b137cb0170 0x55b137cb0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:47.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.552+0000 7efd037fe700 1 -- 172.21.15.12:0/794275848 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b137cb5cb0 msgr2=0x55b137cafc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:48:47.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.552+0000 7efd037fe700 1 --2- 172.21.15.12:0/794275848 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b137cb5cb0 0x55b137cafc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:47.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.552+0000 7efd037fe700 1 -- 172.21.15.12:0/794275848 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b137b77590 con 0x55b137b319f0 2024-05-17T22:48:47.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.553+0000 7efd089b3700 1 -- 172.21.15.12:0/1513854746 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7efcf80e2710 con 0x7efcfc099990 2024-05-17T22:48:47.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.553+0000 7efd037fe700 1 --2- 172.21.15.12:0/1513854746 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7efcec0b2900 0x7efcec0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:47.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.553+0000 7efd037fe700 1 --2- 172.21.15.12:0/794275848 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b137b319f0 0x55b137cb5770 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7efd04051a70 tx=0x7efd0406a7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd58eb30bc11e52c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:48:47.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.553+0000 7efce9ffb700 1 -- 172.21.15.12:0/794275848 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0406bbb0 con 0x55b137b319f0 2024-05-17T22:48:47.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.553+0000 7efce9ffb700 1 -- 172.21.15.12:0/794275848 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd04085040 con 0x55b137b319f0 2024-05-17T22:48:47.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.553+0000 7efce9ffb700 1 -- 172.21.15.12:0/794275848 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd04080cd0 con 0x55b137b319f0 2024-05-17T22:48:47.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.554+0000 7efd037fe700 1 --2- 172.21.15.12:0/1513854746 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7efcec0b2900 0x7efcec0b4e10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x55b137cb6320 tx=0x7efd0408f000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:48:47.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.554+0000 7efd18390b80 1 -- 172.21.15.12:0/794275848 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b137b319f0 msgr2=0x55b137cb5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:47.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.554+0000 7efd18390b80 1 --2- 172.21.15.12:0/794275848 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b137b319f0 0x55b137cb5770 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7efd04051a70 tx=0x7efd0406a7e0 comp rx=0 tx=0).stop 2024-05-17T22:48:47.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.554+0000 7efd18390b80 1 -- 172.21.15.12:0/794275848 shutdown_connections 2024-05-17T22:48:47.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.554+0000 7efd18390b80 1 --2- 172.21.15.12:0/794275848 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b137cb0170 0x55b137cb0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:47.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.554+0000 7efd18390b80 1 --2- 172.21.15.12:0/794275848 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b137b319f0 0x55b137cb5770 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:47.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.554+0000 7efd18390b80 1 --2- 172.21.15.12:0/794275848 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b137cb5cb0 0x55b137cafc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:47.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.554+0000 7efd18390b80 1 -- 172.21.15.12:0/794275848 >> 172.21.15.12:0/794275848 conn(0x55b137b75f00 msgr2=0x55b137caab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:48:47.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.555+0000 7efd18390b80 1 -- 172.21.15.12:0/794275848 shutdown_connections 2024-05-17T22:48:47.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.555+0000 7efd18390b80 1 -- 172.21.15.12:0/794275848 wait complete. 2024-05-17T22:48:47.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.555+0000 7efd18390b80 1 -- start start 2024-05-17T22:48:47.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.555+0000 7efd18390b80 1 Processor -- start 2024-05-17T22:48:47.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.556+0000 7efd18390b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b137b319f0 0x55b137cb9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:47.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.556+0000 7efd18390b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b137cb0170 0x55b137cb97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:47.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.556+0000 7efd18390b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b137cb5cb0 0x55b137cb9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:47.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.556+0000 7efd18390b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b137ad5150 con 0x55b137cb5cb0 2024-05-17T22:48:47.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.556+0000 7efd18390b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b137b6ee80 con 0x55b137cb0170 2024-05-17T22:48:47.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.556+0000 7efd18390b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b137cbc5a0 con 0x55b137b319f0 2024-05-17T22:48:47.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.556+0000 7efd027fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b137cb0170 0x55b137cb97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:47.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.556+0000 7efd027fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b137cb0170 0x55b137cb97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:36102/0 (socket says 172.21.15.12:36102) 2024-05-17T22:48:47.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.556+0000 7efd027fc700 1 -- 172.21.15.12:0/562096606 learned_addr learned my addr 172.21.15.12:0/562096606 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:48:47.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.556+0000 7efd037fe700 1 --2- 172.21.15.12:0/562096606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b137cb5cb0 0x55b137cb9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:47.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.556+0000 7efd18390b80 10 client.0 _ll_register_callbacks cb 0x55b137b68f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:48:47.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.556+0000 7efd02ffd700 1 --2- 172.21.15.12:0/562096606 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b137b319f0 0x55b137cb9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:47.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.556+0000 7efd027fc700 1 -- 172.21.15.12:0/562096606 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b137b319f0 msgr2=0x55b137cb9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:48:47.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.556+0000 7efd027fc700 1 --2- 172.21.15.12:0/562096606 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b137b319f0 0x55b137cb9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:47.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.556+0000 7efd027fc700 1 -- 172.21.15.12:0/562096606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b137cb5cb0 msgr2=0x55b137cb9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:48:47.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.556+0000 7efd027fc700 1 --2- 172.21.15.12:0/562096606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b137cb5cb0 0x55b137cb9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:47.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.557+0000 7efd027fc700 1 -- 172.21.15.12:0/562096606 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b137b1e480 con 0x55b137cb0170 2024-05-17T22:48:47.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.557+0000 7efd027fc700 1 --2- 172.21.15.12:0/562096606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b137cb0170 0x55b137cb97d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7efcf80a5a20 tx=0x7efcf80a9ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d079d4b51b1381a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:48:47.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.557+0000 7efce2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:48:47.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.557+0000 7efce2ffd700 1 -- 172.21.15.12:0/562096606 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf80a9110 con 0x55b137cb0170 2024-05-17T22:48:47.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.557+0000 7efce2ffd700 1 -- 172.21.15.12:0/562096606 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7efcf80e2440 con 0x55b137cb0170 2024-05-17T22:48:47.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.557+0000 7efd18390b80 1 -- 172.21.15.12:0/562096606 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd04085040 con 0x55b137cb0170 2024-05-17T22:48:47.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.557+0000 7efce2ffd700 1 -- 172.21.15.12:0/562096606 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7efcf80a5130 con 0x55b137cb0170 2024-05-17T22:48:47.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.557+0000 7efd18390b80 10 client.25829 fetch_fsmap learned FSMap version 39 2024-05-17T22:48:47.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.557+0000 7efd18390b80 1 -- 172.21.15.12:0/562096606 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b137cc6f20 con 0x55b137cb0170 2024-05-17T22:48:47.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.557+0000 7efce2ffd700 1 -- 172.21.15.12:0/562096606 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd04085040 con 0x55b137cb0170 2024-05-17T22:48:47.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.558+0000 7efce2ffd700 1 -- 172.21.15.12:0/562096606 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7efcf80cc0b0 con 0x55b137cb0170 2024-05-17T22:48:47.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.558+0000 7efd18390b80 10 client.25829 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:48:47.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.558+0000 7efd18390b80 10 client.25829 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-17T22:48:47.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.558+0000 7efd18390b80 20 client.25829 populate_metadata read hostname 'smithi012' 2024-05-17T22:48:47.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.558+0000 7efd18390b80 1 --2- 172.21.15.12:0/562096606 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55b137cc94b0 0x55b137ccb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:47.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.558+0000 7efd18390b80 4 client.25829 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:48:47.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.558+0000 7efd18390b80 1 -- 172.21.15.12:0/562096606 --> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b137be01e0 con 0x55b137cc94b0 2024-05-17T22:48:47.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.559+0000 7efd02ffd700 1 --2- 172.21.15.12:0/562096606 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55b137cc94b0 0x55b137ccb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:47.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.560+0000 7efd02ffd700 1 --2- 172.21.15.12:0/562096606 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55b137cc94b0 0x55b137ccb8d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=512636c783154601 server_cookie=faec875a11a60189 in_seq=0 out_seq=0 2024-05-17T22:48:47.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.560+0000 7efce2ffd700 10 client.25829 ms_handle_connect on v2:172.21.15.96:6836/4222574338 2024-05-17T22:48:47.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.561+0000 7efce2ffd700 1 -- 172.21.15.12:0/562096606 <== mds.1 v2:172.21.15.96:6836/4222574338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b137be01e0 con 0x55b137cc94b0 2024-05-17T22:48:47.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.561+0000 7efce2ffd700 10 client.25829 handle_command_reply: tid=0 2024-05-17T22:48:47.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.561+0000 7efd18390b80 1 client.25829 shutdown 2024-05-17T22:48:47.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.561+0000 7efd18390b80 1 -- 172.21.15.12:0/562096606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b137cb0170 msgr2=0x55b137cb97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:47.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.561+0000 7efd18390b80 1 --2- 172.21.15.12:0/562096606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b137cb0170 0x55b137cb97d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7efcf80a5a20 tx=0x7efcf80a9ee0 comp rx=0 tx=0).stop 2024-05-17T22:48:47.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.561+0000 7efd18390b80 1 -- 172.21.15.12:0/562096606 shutdown_connections 2024-05-17T22:48:47.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.561+0000 7efd18390b80 1 --2- 172.21.15.12:0/562096606 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55b137cc94b0 0x55b137ccb8d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:47.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.562+0000 7efd18390b80 1 --2- 172.21.15.12:0/562096606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b137cb5cb0 0x55b137cb9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:47.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.562+0000 7efd18390b80 1 --2- 172.21.15.12:0/562096606 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b137b319f0 0x55b137cb9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:47.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.562+0000 7efce2ffd700 0 client.25829 ms_handle_reset on v2:172.21.15.96:6836/4222574338 2024-05-17T22:48:47.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.562+0000 7efd18390b80 1 --2- 172.21.15.12:0/562096606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b137cb0170 0x55b137cb97d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:47.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.562+0000 7efd18390b80 1 -- 172.21.15.12:0/562096606 >> 172.21.15.12:0/562096606 conn(0x55b137b75f00 msgr2=0x55b137caa110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:48:47.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.566+0000 7efd18390b80 1 -- 172.21.15.12:0/562096606 shutdown_connections 2024-05-17T22:48:47.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.566+0000 7efd18390b80 1 -- 172.21.15.12:0/562096606 wait complete. 2024-05-17T22:48:47.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.566+0000 7efd18390b80 20 client.25829 trim_cache size 0 max 16384 2024-05-17T22:48:47.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.599+0000 7efd18390b80 1 Processor -- start 2024-05-17T22:48:47.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.604+0000 7efd18390b80 1 -- start start 2024-05-17T22:48:47.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.604+0000 7efd18390b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b137b319f0 0x55b137cafc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:47.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.605+0000 7efd18390b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b137cb5cb0 0x55b137cc5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:47.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.605+0000 7efd02ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b137b319f0 0x55b137cafc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:47.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.605+0000 7efd02ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b137b319f0 0x55b137cafc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:36106/0 (socket says 172.21.15.12:36106) 2024-05-17T22:48:47.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.605+0000 7efd02ffd700 1 -- 172.21.15.12:0/2798897133 learned_addr learned my addr 172.21.15.12:0/2798897133 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:48:47.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.605+0000 7efd18390b80 1 --2- 172.21.15.12:0/2798897133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b137cc94b0 0x55b137cb9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:47.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.605+0000 7efd18390b80 1 -- 172.21.15.12:0/2798897133 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc14afa0 con 0x55b137cc94b0 2024-05-17T22:48:47.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.605+0000 7efd18390b80 1 -- 172.21.15.12:0/2798897133 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc148830 con 0x55b137b319f0 2024-05-17T22:48:47.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.605+0000 7efd18390b80 1 -- 172.21.15.12:0/2798897133 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc146680 con 0x55b137cb5cb0 2024-05-17T22:48:47.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.609+0000 7efd02ffd700 1 -- 172.21.15.12:0/2798897133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b137cb5cb0 msgr2=0x55b137cc5350 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T22:48:47.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.609+0000 7efd02ffd700 1 --2- 172.21.15.12:0/2798897133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b137cb5cb0 0x55b137cc5350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:47.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.609+0000 7efd02ffd700 1 -- 172.21.15.12:0/2798897133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b137cc94b0 msgr2=0x55b137cb9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:48:47.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.609+0000 7efd02ffd700 1 --2- 172.21.15.12:0/2798897133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b137cc94b0 0x55b137cb9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:47.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.609+0000 7efd02ffd700 1 -- 172.21.15.12:0/2798897133 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b137cab860 con 0x55b137b319f0 2024-05-17T22:48:47.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.609+0000 7efd02ffd700 1 --2- 172.21.15.12:0/2798897133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b137b319f0 0x55b137cafc30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efcf400bfd0 tx=0x7efcf4014830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0c92334c72bb2f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:48:47.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.611+0000 7efce97fa700 1 -- 172.21.15.12:0/2798897133 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf402b020 con 0x55b137b319f0 2024-05-17T22:48:47.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.611+0000 7efce97fa700 1 -- 172.21.15.12:0/2798897133 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcf4019040 con 0x55b137b319f0 2024-05-17T22:48:47.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.611+0000 7efce97fa700 1 -- 172.21.15.12:0/2798897133 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf40053a0 con 0x55b137b319f0 2024-05-17T22:48:47.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.612+0000 7efd18390b80 1 -- 172.21.15.12:0/2798897133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b137b319f0 msgr2=0x55b137cafc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:47.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.612+0000 7efd18390b80 1 --2- 172.21.15.12:0/2798897133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b137b319f0 0x55b137cafc30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efcf400bfd0 tx=0x7efcf4014830 comp rx=0 tx=0).stop 2024-05-17T22:48:47.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.617+0000 7efd18390b80 1 -- 172.21.15.12:0/2798897133 shutdown_connections 2024-05-17T22:48:47.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.617+0000 7efd18390b80 1 --2- 172.21.15.12:0/2798897133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b137cc94b0 0x55b137cb9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:47.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.617+0000 7efd18390b80 1 --2- 172.21.15.12:0/2798897133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b137b319f0 0x55b137cafc30 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:47.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.617+0000 7efd18390b80 1 --2- 172.21.15.12:0/2798897133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b137cb5cb0 0x55b137cc5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:47.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.617+0000 7efd18390b80 1 -- 172.21.15.12:0/2798897133 >> 172.21.15.12:0/2798897133 conn(0x55b137b75f00 msgr2=0x55b137caaad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:48:47.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.626+0000 7efd18390b80 1 -- 172.21.15.12:0/2798897133 shutdown_connections 2024-05-17T22:48:47.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.627+0000 7efd18390b80 1 -- 172.21.15.12:0/2798897133 wait complete. 2024-05-17T22:48:47.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.627+0000 7efd18390b80 1 -- start start 2024-05-17T22:48:47.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.628+0000 7efd18390b80 1 Processor -- start 2024-05-17T22:48:47.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.629+0000 7efd18390b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b137b319f0 0x55b137cc1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:47.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.629+0000 7efd18390b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b137cb5cb0 0x55b137cd0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:47.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.629+0000 7efd02ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b137b319f0 0x55b137cc1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:47.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.629+0000 7efd18390b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b137cc94b0 0x55b137cd0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:47.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.629+0000 7efd02ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b137b319f0 0x55b137cc1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60676/0 (socket says 172.21.15.12:60676) 2024-05-17T22:48:47.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.630+0000 7efd18390b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc146cf0 con 0x55b137b319f0 2024-05-17T22:48:47.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.630+0000 7efd02ffd700 1 -- 172.21.15.12:0/843414021 learned_addr learned my addr 172.21.15.12:0/843414021 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:48:47.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.630+0000 7efd18390b80 1 -- 172.21.15.12:0/843414021 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc148160 con 0x55b137cb5cb0 2024-05-17T22:48:47.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.630+0000 7efd18390b80 1 -- 172.21.15.12:0/843414021 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc1473c0 con 0x55b137cc94b0 2024-05-17T22:48:47.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.630+0000 7efd037fe700 1 --2- 172.21.15.12:0/843414021 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b137cc94b0 0x55b137cd0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:47.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.630+0000 7efd18390b80 10 client.0 _ll_register_callbacks cb 0x55b137b68f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:48:47.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.631+0000 7efd02ffd700 1 -- 172.21.15.12:0/843414021 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b137cc94b0 msgr2=0x55b137cd0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:48:47.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.631+0000 7efd02ffd700 1 --2- 172.21.15.12:0/843414021 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b137cc94b0 0x55b137cd0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:47.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.631+0000 7efd02ffd700 1 -- 172.21.15.12:0/843414021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b137cb5cb0 msgr2=0x55b137cd0430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T22:48:47.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.631+0000 7efd02ffd700 1 --2- 172.21.15.12:0/843414021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b137cb5cb0 0x55b137cd0430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:47.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.632+0000 7efd02ffd700 1 -- 172.21.15.12:0/843414021 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b137cd0eb0 con 0x55b137b319f0 2024-05-17T22:48:47.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.632+0000 7efd02ffd700 1 --2- 172.21.15.12:0/843414021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b137b319f0 0x55b137cc1900 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7efcf400f010 tx=0x7efcf40148a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be361749a99069b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:48:47.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.633+0000 7efce2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:48:47.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.633+0000 7efce2ffd700 1 -- 172.21.15.12:0/843414021 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf40052f0 con 0x55b137b319f0 2024-05-17T22:48:47.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.634+0000 7efce2ffd700 1 -- 172.21.15.12:0/843414021 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7efcf4005490 con 0x55b137b319f0 2024-05-17T22:48:47.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.634+0000 7efd18390b80 1 -- 172.21.15.12:0/843414021 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcf4019040 con 0x55b137b319f0 2024-05-17T22:48:47.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.634+0000 7efce2ffd700 1 -- 172.21.15.12:0/843414021 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7efcf4005690 con 0x55b137b319f0 2024-05-17T22:48:47.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.637+0000 7efd18390b80 10 client.16122 fetch_fsmap learned FSMap version 39 2024-05-17T22:48:47.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.637+0000 7efd18390b80 1 -- 172.21.15.12:0/843414021 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b137d33810 con 0x55b137b319f0 2024-05-17T22:48:47.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.642+0000 7efce2ffd700 1 -- 172.21.15.12:0/843414021 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcf400dc80 con 0x55b137b319f0 2024-05-17T22:48:47.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.642+0000 7efce2ffd700 1 -- 172.21.15.12:0/843414021 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7efcf4013070 con 0x55b137b319f0 2024-05-17T22:48:47.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.642+0000 7efd18390b80 10 client.16122 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:48:47.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.642+0000 7efd18390b80 10 client.16122 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-17T22:48:47.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.642+0000 7efd18390b80 20 client.16122 populate_metadata read hostname 'smithi012' 2024-05-17T22:48:47.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.642+0000 7efd18390b80 1 --2- 172.21.15.12:0/843414021 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55b137d33d40 0x55b137d36160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:47.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.642+0000 7efd18390b80 4 client.16122 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:48:47.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.642+0000 7efd18390b80 1 -- 172.21.15.12:0/843414021 --> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7efcf4019040 con 0x55b137d33d40 2024-05-17T22:48:47.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.648+0000 7efd027fc700 1 --2- 172.21.15.12:0/843414021 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55b137d33d40 0x55b137d36160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:47.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.662+0000 7efd027fc700 1 --2- 172.21.15.12:0/843414021 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55b137d33d40 0x55b137d36160 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c0aed0e3a7702265 server_cookie=7050e079da410cb8 in_seq=0 out_seq=0 2024-05-17T22:48:47.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.662+0000 7efce2ffd700 10 client.16122 ms_handle_connect on v2:172.21.15.96:6836/4222574338 2024-05-17T22:48:47.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.666+0000 7efce2ffd700 1 -- 172.21.15.12:0/843414021 <== mds.1 v2:172.21.15.96:6836/4222574338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7efcf4019040 con 0x55b137d33d40 2024-05-17T22:48:47.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.666+0000 7efce2ffd700 10 client.16122 handle_command_reply: tid=0 2024-05-17T22:48:47.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.666+0000 7efd18390b80 1 client.16122 shutdown 2024-05-17T22:48:47.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.672+0000 7efd18390b80 1 -- 172.21.15.12:0/843414021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b137b319f0 msgr2=0x55b137cc1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:47.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.672+0000 7efd18390b80 1 --2- 172.21.15.12:0/843414021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b137b319f0 0x55b137cc1900 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7efcf400f010 tx=0x7efcf40148a0 comp rx=0 tx=0).stop 2024-05-17T22:48:47.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.672+0000 7efd18390b80 1 -- 172.21.15.12:0/843414021 shutdown_connections 2024-05-17T22:48:47.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.672+0000 7efd18390b80 1 --2- 172.21.15.12:0/843414021 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b137cc94b0 0x55b137cd0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:47.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.672+0000 7efd18390b80 1 --2- 172.21.15.12:0/843414021 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55b137d33d40 0x55b137d36160 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:47.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.672+0000 7efd18390b80 1 --2- 172.21.15.12:0/843414021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b137b319f0 0x55b137cc1900 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:47.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.672+0000 7efd18390b80 1 --2- 172.21.15.12:0/843414021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b137cb5cb0 0x55b137cd0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:47.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.672+0000 7efd18390b80 1 -- 172.21.15.12:0/843414021 >> 172.21.15.12:0/843414021 conn(0x55b137b75f00 msgr2=0x55b137caaad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:48:47.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.673+0000 7efce2ffd700 0 client.16122 ms_handle_reset on v2:172.21.15.96:6836/4222574338 2024-05-17T22:48:47.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.673+0000 7efd18390b80 1 -- 172.21.15.12:0/843414021 shutdown_connections 2024-05-17T22:48:47.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.673+0000 7efd18390b80 1 -- 172.21.15.12:0/843414021 wait complete. 2024-05-17T22:48:47.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.675+0000 7efd18390b80 20 client.16122 trim_cache size 0 max 16384 2024-05-17T22:48:47.680 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:48:47.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.682+0000 7efd099b5700 1 -- 172.21.15.12:0/1513854746 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7efcec0b2900 msgr2=0x7efcec0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:47.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.682+0000 7efd099b5700 1 --2- 172.21.15.12:0/1513854746 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7efcec0b2900 0x7efcec0b4e10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x55b137cb6320 tx=0x7efd0408f000 comp rx=0 tx=0).stop 2024-05-17T22:48:47.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.682+0000 7efd099b5700 1 -- 172.21.15.12:0/1513854746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcfc099990 msgr2=0x7efcfc09c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:47.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.682+0000 7efd099b5700 1 --2- 172.21.15.12:0/1513854746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcfc099990 0x7efcfc09c1c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efcf800e9f0 tx=0x7efcf800ed00 comp rx=0 tx=0).stop 2024-05-17T22:48:47.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.682+0000 7efd099b5700 1 -- 172.21.15.12:0/1513854746 shutdown_connections 2024-05-17T22:48:47.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.683+0000 7efd099b5700 1 --2- 172.21.15.12:0/1513854746 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7efcec0b2900 0x7efcec0b4e10 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:47.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.683+0000 7efd099b5700 1 --2- 172.21.15.12:0/1513854746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efcfc09a2e0 0x7efcfc09c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:47.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.683+0000 7efd099b5700 1 --2- 172.21.15.12:0/1513854746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efcfc095820 0x7efcfc09bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:47.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.683+0000 7efd099b5700 1 --2- 172.21.15.12:0/1513854746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcfc099990 0x7efcfc09c1c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:47.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.683+0000 7efd099b5700 1 -- 172.21.15.12:0/1513854746 >> 172.21.15.12:0/1513854746 conn(0x7efcfc00a2a0 msgr2=0x7efcfc0a18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:48:47.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.684+0000 7efd099b5700 1 -- 172.21.15.12:0/1513854746 shutdown_connections 2024-05-17T22:48:47.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:47.686+0000 7efd099b5700 1 -- 172.21.15.12:0/1513854746 wait complete. 2024-05-17T22:48:48.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:47 smithi151 ceph-mon[36101]: pgmap v725: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 3.8 MiB/s wr, 423 op/s 2024-05-17T22:48:48.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:47 smithi096 ceph-mon[36065]: pgmap v725: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 3.8 MiB/s wr, 423 op/s 2024-05-17T22:48:48.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:47 smithi012 ceph-mon[31632]: pgmap v725: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 3.8 MiB/s wr, 423 op/s 2024-05-17T22:48:48.400 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-17T22:48:48.943 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:48:50.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:49 smithi151 ceph-mon[36101]: pgmap v726: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.1 MiB/s wr, 480 op/s 2024-05-17T22:48:50.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:49 smithi096 ceph-mon[36065]: pgmap v726: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.1 MiB/s wr, 480 op/s 2024-05-17T22:48:50.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:49 smithi012 ceph-mon[31632]: pgmap v726: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.1 MiB/s wr, 480 op/s 2024-05-17T22:48:51.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.054+0000 7f4641097700 1 -- 172.21.15.12:0/2575353208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f463c102e70 msgr2=0x7f463c103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:51.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.054+0000 7f4641097700 1 --2- 172.21.15.12:0/2575353208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f463c102e70 0x7f463c103280 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f462c009990 tx=0x7f462c009ca0 comp rx=0 tx=0).stop 2024-05-17T22:48:51.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.054+0000 7f4641097700 1 -- 172.21.15.12:0/2575353208 shutdown_connections 2024-05-17T22:48:51.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.054+0000 7f4641097700 1 --2- 172.21.15.12:0/2575353208 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f463c1080a0 0x7f463c10fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:51.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.054+0000 7f4641097700 1 --2- 172.21.15.12:0/2575353208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f463c102e70 0x7f463c103280 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:51.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.054+0000 7f4641097700 1 --2- 172.21.15.12:0/2575353208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f463c107750 0x7f463c107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:51.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.054+0000 7f4641097700 1 -- 172.21.15.12:0/2575353208 >> 172.21.15.12:0/2575353208 conn(0x7f463c079ae0 msgr2=0x7f463c079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:48:51.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.054+0000 7f4641097700 1 -- 172.21.15.12:0/2575353208 shutdown_connections 2024-05-17T22:48:51.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.054+0000 7f4641097700 1 -- 172.21.15.12:0/2575353208 wait complete. 2024-05-17T22:48:51.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.055+0000 7f4641097700 1 Processor -- start 2024-05-17T22:48:51.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.055+0000 7f4641097700 1 -- start start 2024-05-17T22:48:51.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.056+0000 7f4641097700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f463c102e70 0x7f463c19b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:51.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.056+0000 7f4641097700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f463c107750 0x7f463c19bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:51.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.056+0000 7f4641097700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f463c1080a0 0x7f463c1a66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:51.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.056+0000 7f463ad9d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f463c102e70 0x7f463c19b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:51.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.056+0000 7f463ad9d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f463c102e70 0x7f463c19b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:35274/0 (socket says 172.21.15.12:35274) 2024-05-17T22:48:51.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.056+0000 7f463ad9d700 1 -- 172.21.15.12:0/2545184476 learned_addr learned my addr 172.21.15.12:0/2545184476 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:48:51.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.056+0000 7f4641097700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f463c112d20 con 0x7f463c107750 2024-05-17T22:48:51.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.056+0000 7f4641097700 1 -- 172.21.15.12:0/2545184476 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f463c112ba0 con 0x7f463c1080a0 2024-05-17T22:48:51.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.056+0000 7f4641097700 1 -- 172.21.15.12:0/2545184476 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f463c112ea0 con 0x7f463c102e70 2024-05-17T22:48:51.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.056+0000 7f463b59e700 1 --2- 172.21.15.12:0/2545184476 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f463c1080a0 0x7f463c1a66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:51.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.056+0000 7f463ad9d700 1 -- 172.21.15.12:0/2545184476 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f463c1080a0 msgr2=0x7f463c1a66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:51.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.056+0000 7f463ad9d700 1 --2- 172.21.15.12:0/2545184476 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f463c1080a0 0x7f463c1a66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:51.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.056+0000 7f463ad9d700 1 -- 172.21.15.12:0/2545184476 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f463c107750 msgr2=0x7f463c19bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:51.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.056+0000 7f463ad9d700 1 --2- 172.21.15.12:0/2545184476 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f463c107750 0x7f463c19bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:51.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.056+0000 7f463ad9d700 1 -- 172.21.15.12:0/2545184476 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f463c19c440 con 0x7f463c102e70 2024-05-17T22:48:51.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.057+0000 7f463ad9d700 1 --2- 172.21.15.12:0/2545184476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f463c102e70 0x7f463c19b670 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f462c000c00 tx=0x7f462c009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:48:51.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.057+0000 7f4623fff700 1 -- 172.21.15.12:0/2545184476 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462c027070 con 0x7f463c102e70 2024-05-17T22:48:51.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.057+0000 7f4623fff700 1 -- 172.21.15.12:0/2545184476 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f462c022470 con 0x7f463c102e70 2024-05-17T22:48:51.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.057+0000 7f4623fff700 1 -- 172.21.15.12:0/2545184476 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462c017650 con 0x7f463c102e70 2024-05-17T22:48:51.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.057+0000 7f4641097700 1 -- 172.21.15.12:0/2545184476 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f463c1a6c90 con 0x7f463c102e70 2024-05-17T22:48:51.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.057+0000 7f4641097700 1 -- 172.21.15.12:0/2545184476 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f463c1a7030 con 0x7f463c102e70 2024-05-17T22:48:51.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.065+0000 7f4623fff700 1 -- 172.21.15.12:0/2545184476 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f462c02a070 con 0x7f463c102e70 2024-05-17T22:48:51.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.066+0000 7f464fa72b80 1 Processor -- start 2024-05-17T22:48:51.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.074+0000 7f4623fff700 1 --2- 172.21.15.12:0/2545184476 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f46240b2980 0x7f46240b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:51.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.075+0000 7f4623fff700 1 -- 172.21.15.12:0/2545184476 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f462c0e33f0 con 0x7f463c102e70 2024-05-17T22:48:51.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.075+0000 7f463b59e700 1 --2- 172.21.15.12:0/2545184476 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f46240b2980 0x7f46240b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:51.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.076+0000 7f464fa72b80 1 -- start start 2024-05-17T22:48:51.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.076+0000 7f463b59e700 1 --2- 172.21.15.12:0/2545184476 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f46240b2980 0x7f46240b4e90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f463c06d120 tx=0x7f463400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:48:51.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.076+0000 7f464fa72b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558feef039f0 0x558fef086ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:51.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.077+0000 7f464fa72b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558fef087400 0x558fef0815c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:51.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.077+0000 7f464fa72b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fef081b00 0x558fef081f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:51.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.077+0000 7f464fa72b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x558feefb00d0 con 0x558fef087400 2024-05-17T22:48:51.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.077+0000 7f464fa72b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558feef9fc70 con 0x558fef081b00 2024-05-17T22:48:51.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.077+0000 7f464fa72b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558feefa3cb0 con 0x558feef039f0 2024-05-17T22:48:51.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.080+0000 7f463b59e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fef081b00 0x558fef081f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:51.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.080+0000 7f463b59e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fef081b00 0x558fef081f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:56838/0 (socket says 172.21.15.12:56838) 2024-05-17T22:48:51.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.080+0000 7f463b59e700 1 -- 172.21.15.12:0/4227378466 learned_addr learned my addr 172.21.15.12:0/4227378466 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:48:51.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.080+0000 7f463b59e700 1 -- 172.21.15.12:0/4227378466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558feef039f0 msgr2=0x558fef086ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T22:48:51.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.080+0000 7f463b59e700 1 --2- 172.21.15.12:0/4227378466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558feef039f0 0x558fef086ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:51.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.080+0000 7f463b59e700 1 -- 172.21.15.12:0/4227378466 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558fef087400 msgr2=0x558fef0815c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:48:51.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.080+0000 7f463b59e700 1 --2- 172.21.15.12:0/4227378466 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558fef087400 0x558fef0815c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:51.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.080+0000 7f463b59e700 1 -- 172.21.15.12:0/4227378466 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558feef395c0 con 0x558fef081b00 2024-05-17T22:48:51.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.080+0000 7f463b59e700 1 --2- 172.21.15.12:0/4227378466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fef081b00 0x558fef081f10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4634003d50 tx=0x7f4634014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fac13b68dd45bab3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:48:51.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.081+0000 7f46217fa700 1 -- 172.21.15.12:0/4227378466 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463401b070 con 0x558fef081b00 2024-05-17T22:48:51.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.081+0000 7f46217fa700 1 -- 172.21.15.12:0/4227378466 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4634003d80 con 0x558fef081b00 2024-05-17T22:48:51.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.081+0000 7f46217fa700 1 -- 172.21.15.12:0/4227378466 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4634022470 con 0x558fef081b00 2024-05-17T22:48:51.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.091+0000 7f464fa72b80 1 -- 172.21.15.12:0/4227378466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fef081b00 msgr2=0x558fef081f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:51.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.091+0000 7f464fa72b80 1 --2- 172.21.15.12:0/4227378466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fef081b00 0x558fef081f10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4634003d50 tx=0x7f4634014000 comp rx=0 tx=0).stop 2024-05-17T22:48:51.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.095+0000 7f464fa72b80 1 -- 172.21.15.12:0/4227378466 shutdown_connections 2024-05-17T22:48:51.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.095+0000 7f464fa72b80 1 --2- 172.21.15.12:0/4227378466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fef081b00 0x558fef081f10 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:51.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.095+0000 7f464fa72b80 1 --2- 172.21.15.12:0/4227378466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558feef039f0 0x558fef086ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:51.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.095+0000 7f464fa72b80 1 --2- 172.21.15.12:0/4227378466 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558fef087400 0x558fef0815c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:51.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.095+0000 7f464fa72b80 1 -- 172.21.15.12:0/4227378466 >> 172.21.15.12:0/4227378466 conn(0x558feef637d0 msgr2=0x558fef07c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:48:51.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.103+0000 7f464fa72b80 1 -- 172.21.15.12:0/4227378466 shutdown_connections 2024-05-17T22:48:51.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.107+0000 7f464fa72b80 1 -- 172.21.15.12:0/4227378466 wait complete. 2024-05-17T22:48:51.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.107+0000 7f464fa72b80 1 -- start start 2024-05-17T22:48:51.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.107+0000 7f464fa72b80 1 Processor -- start 2024-05-17T22:48:51.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.110+0000 7f464fa72b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558feef039f0 0x558fef08aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:51.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.110+0000 7f464fa72b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558fef081b00 0x558fef08b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:51.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.110+0000 7f464fa72b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fef087400 0x558fef08b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:51.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.110+0000 7f464fa72b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x558feeea7150 con 0x558fef081b00 2024-05-17T22:48:51.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.110+0000 7f464fa72b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558feef41940 con 0x558fef087400 2024-05-17T22:48:51.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.110+0000 7f464fa72b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558fef08ddd0 con 0x558feef039f0 2024-05-17T22:48:51.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.110+0000 7f464fa72b80 10 client.0 _ll_register_callbacks cb 0x558feef36b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:48:51.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.110+0000 7f463ad9d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558feef039f0 0x558fef08aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:51.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.110+0000 7f463ad9d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558feef039f0 0x558fef08aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:35300/0 (socket says 172.21.15.12:35300) 2024-05-17T22:48:51.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.110+0000 7f463ad9d700 1 -- 172.21.15.12:0/3908106769 learned_addr learned my addr 172.21.15.12:0/3908106769 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:48:51.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.110+0000 7f463b59e700 1 --2- 172.21.15.12:0/3908106769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fef087400 0x558fef08b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:51.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.111+0000 7f463ad9d700 1 -- 172.21.15.12:0/3908106769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fef087400 msgr2=0x558fef08b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:48:51.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.111+0000 7f463ad9d700 1 --2- 172.21.15.12:0/3908106769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fef087400 0x558fef08b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:51.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.111+0000 7f463ad9d700 1 -- 172.21.15.12:0/3908106769 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558fef081b00 msgr2=0x558fef08b020 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T22:48:51.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.111+0000 7f463ad9d700 1 --2- 172.21.15.12:0/3908106769 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558fef081b00 0x558fef08b020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:51.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.111+0000 7f463ad9d700 1 -- 172.21.15.12:0/3908106769 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558feeef0480 con 0x558feef039f0 2024-05-17T22:48:51.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.114+0000 7f463ad9d700 1 --2- 172.21.15.12:0/3908106769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558feef039f0 0x558fef08aae0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f462c0a6730 tx=0x7f462c0aa130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c8a99462c80c767 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:48:51.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.114+0000 7f461eff5700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:48:51.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.116+0000 7f461eff5700 1 -- 172.21.15.12:0/3908106769 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462c0aa1c0 con 0x558feef039f0 2024-05-17T22:48:51.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.116+0000 7f461eff5700 1 -- 172.21.15.12:0/3908106769 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f462c009580 con 0x558feef039f0 2024-05-17T22:48:51.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.121+0000 7f464fa72b80 1 -- 172.21.15.12:0/3908106769 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4634003d80 con 0x558feef039f0 2024-05-17T22:48:51.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.125+0000 7f461eff5700 1 -- 172.21.15.12:0/3908106769 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f462c0aadb0 con 0x558feef039f0 2024-05-17T22:48:51.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.125+0000 7f464fa72b80 10 client.25774 fetch_fsmap learned FSMap version 39 2024-05-17T22:48:51.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.125+0000 7f464fa72b80 1 -- 172.21.15.12:0/3908106769 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558fef08f760 con 0x558feef039f0 2024-05-17T22:48:51.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.131+0000 7f461eff5700 1 -- 172.21.15.12:0/3908106769 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4634003d80 con 0x558feef039f0 2024-05-17T22:48:51.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.131+0000 7f461eff5700 1 -- 172.21.15.12:0/3908106769 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f462c0b2ff0 con 0x558feef039f0 2024-05-17T22:48:51.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.132+0000 7f464fa72b80 10 client.25774 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:48:51.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.132+0000 7f464fa72b80 10 client.25774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:48:51.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.132+0000 7f464fa72b80 20 client.25774 populate_metadata read hostname 'smithi012' 2024-05-17T22:48:51.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.132+0000 7f464fa72b80 1 --2- 172.21.15.12:0/3908106769 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x558fef091cf0 0x558fef094110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:51.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.132+0000 7f464fa72b80 4 client.25774 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:48:51.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.132+0000 7f464fa72b80 1 -- 172.21.15.12:0/3908106769 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558feefb2020 con 0x558fef091cf0 2024-05-17T22:48:51.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.138+0000 7f463a59c700 1 --2- 172.21.15.12:0/3908106769 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x558fef091cf0 0x558fef094110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:51.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.147+0000 7f463a59c700 1 --2- 172.21.15.12:0/3908106769 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x558fef091cf0 0x558fef094110 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ea85135de38797c server_cookie=bdf0ce36a12490ec in_seq=0 out_seq=0 2024-05-17T22:48:51.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.148+0000 7f461eff5700 10 client.25774 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:48:51.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.150+0000 7f461eff5700 1 -- 172.21.15.12:0/3908106769 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558feefb2020 con 0x558fef091cf0 2024-05-17T22:48:51.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.151+0000 7f461eff5700 10 client.25774 handle_command_reply: tid=0 2024-05-17T22:48:51.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.151+0000 7f464fa72b80 1 client.25774 shutdown 2024-05-17T22:48:51.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.157+0000 7f464fa72b80 1 -- 172.21.15.12:0/3908106769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558feef039f0 msgr2=0x558fef08aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:51.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.157+0000 7f464fa72b80 1 --2- 172.21.15.12:0/3908106769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558feef039f0 0x558fef08aae0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f462c0a6730 tx=0x7f462c0aa130 comp rx=0 tx=0).stop 2024-05-17T22:48:51.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.158+0000 7f464fa72b80 1 -- 172.21.15.12:0/3908106769 shutdown_connections 2024-05-17T22:48:51.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.158+0000 7f464fa72b80 1 --2- 172.21.15.12:0/3908106769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fef087400 0x558fef08b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:51.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.158+0000 7f464fa72b80 1 --2- 172.21.15.12:0/3908106769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558feef039f0 0x558fef08aae0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:51.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.158+0000 7f464fa72b80 1 --2- 172.21.15.12:0/3908106769 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x558fef091cf0 0x558fef094110 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:51.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.158+0000 7f464fa72b80 1 --2- 172.21.15.12:0/3908106769 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558fef081b00 0x558fef08b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:51.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.158+0000 7f464fa72b80 1 -- 172.21.15.12:0/3908106769 >> 172.21.15.12:0/3908106769 conn(0x558feef637d0 msgr2=0x558fef07c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:48:51.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.159+0000 7f461eff5700 0 client.25774 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:48:51.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.163+0000 7f464fa72b80 1 -- 172.21.15.12:0/3908106769 shutdown_connections 2024-05-17T22:48:51.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.165+0000 7f464fa72b80 1 -- 172.21.15.12:0/3908106769 wait complete. 2024-05-17T22:48:51.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.165+0000 7f464fa72b80 20 client.25774 trim_cache size 0 max 16384 2024-05-17T22:48:51.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.201+0000 7f464fa72b80 1 Processor -- start 2024-05-17T22:48:51.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.201+0000 7f464fa72b80 1 -- start start 2024-05-17T22:48:51.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.202+0000 7f464fa72b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558feef039f0 0x558fef0815c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:51.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.202+0000 7f464fa72b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558fef081b00 0x558fef08d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:51.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.202+0000 7f464fa72b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fef087400 0x558fef0847f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:51.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.202+0000 7f464fa72b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f463c1b0480 con 0x558feef039f0 2024-05-17T22:48:51.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.202+0000 7f464fa72b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f463c1add10 con 0x558fef087400 2024-05-17T22:48:51.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.202+0000 7f463a59c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558fef081b00 0x558fef08d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:51.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.202+0000 7f463ad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558feef039f0 0x558fef0815c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:51.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.202+0000 7f463a59c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558fef081b00 0x558fef08d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:35314/0 (socket says 172.21.15.12:35314) 2024-05-17T22:48:51.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.202+0000 7f463ad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558feef039f0 0x558fef0815c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35872/0 (socket says 172.21.15.12:35872) 2024-05-17T22:48:51.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.202+0000 7f463a59c700 1 -- 172.21.15.12:0/731033523 learned_addr learned my addr 172.21.15.12:0/731033523 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:48:51.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.202+0000 7f463b59e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fef087400 0x558fef0847f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:51.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.203+0000 7f464fa72b80 1 -- 172.21.15.12:0/731033523 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f463c1abb60 con 0x558fef081b00 2024-05-17T22:48:51.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.205+0000 7f463ad9d700 1 -- 172.21.15.12:0/731033523 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558fef081b00 msgr2=0x558fef08d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:48:51.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.205+0000 7f463ad9d700 1 --2- 172.21.15.12:0/731033523 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558fef081b00 0x558fef08d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:51.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.205+0000 7f463ad9d700 1 -- 172.21.15.12:0/731033523 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fef087400 msgr2=0x558fef0847f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:48:51.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.205+0000 7f463ad9d700 1 --2- 172.21.15.12:0/731033523 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fef087400 0x558fef0847f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:51.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.205+0000 7f463ad9d700 1 -- 172.21.15.12:0/731033523 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fef08b3b0 con 0x558feef039f0 2024-05-17T22:48:51.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.209+0000 7f463ad9d700 1 --2- 172.21.15.12:0/731033523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558feef039f0 0x558fef0815c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f462c0e3210 tx=0x7f462c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e894d89465de5922 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:48:51.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.214+0000 7f4620ff9700 1 -- 172.21.15.12:0/731033523 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462c0aa6b0 con 0x558feef039f0 2024-05-17T22:48:51.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.214+0000 7f4620ff9700 1 -- 172.21.15.12:0/731033523 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f462c0b3480 con 0x558feef039f0 2024-05-17T22:48:51.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.214+0000 7f4620ff9700 1 -- 172.21.15.12:0/731033523 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462c0a9910 con 0x558feef039f0 2024-05-17T22:48:51.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.215+0000 7f464fa72b80 1 -- 172.21.15.12:0/731033523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558feef039f0 msgr2=0x558fef0815c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:51.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.215+0000 7f464fa72b80 1 --2- 172.21.15.12:0/731033523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558feef039f0 0x558fef0815c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f462c0e3210 tx=0x7f462c009ca0 comp rx=0 tx=0).stop 2024-05-17T22:48:51.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.222+0000 7f464fa72b80 1 -- 172.21.15.12:0/731033523 shutdown_connections 2024-05-17T22:48:51.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.222+0000 7f464fa72b80 1 --2- 172.21.15.12:0/731033523 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fef087400 0x558fef0847f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:51.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.222+0000 7f464fa72b80 1 --2- 172.21.15.12:0/731033523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558feef039f0 0x558fef0815c0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:51.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.222+0000 7f464fa72b80 1 --2- 172.21.15.12:0/731033523 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558fef081b00 0x558fef08d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:51.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.222+0000 7f464fa72b80 1 -- 172.21.15.12:0/731033523 >> 172.21.15.12:0/731033523 conn(0x558feef637d0 msgr2=0x558fef0893b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:48:51.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.230+0000 7f464fa72b80 1 -- 172.21.15.12:0/731033523 shutdown_connections 2024-05-17T22:48:51.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.232+0000 7f464fa72b80 1 -- 172.21.15.12:0/731033523 wait complete. 2024-05-17T22:48:51.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.233+0000 7f464fa72b80 1 -- start start 2024-05-17T22:48:51.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.233+0000 7f464fa72b80 1 Processor -- start 2024-05-17T22:48:51.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.233+0000 7f464fa72b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558feef039f0 0x558fef0a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:51.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.233+0000 7f464fa72b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fef081b00 0x558fef0a4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:51.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.233+0000 7f464fa72b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558fef087400 0x558fef0a2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:51.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.233+0000 7f464fa72b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f463c1ac1d0 con 0x558fef087400 2024-05-17T22:48:51.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.233+0000 7f464fa72b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f463c1ad640 con 0x558fef081b00 2024-05-17T22:48:51.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.233+0000 7f464fa72b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f463c1ac8a0 con 0x558feef039f0 2024-05-17T22:48:51.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.233+0000 7f464fa72b80 10 client.0 _ll_register_callbacks cb 0x558feef36b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:48:51.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.234+0000 7f463ad9d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558feef039f0 0x558fef0a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:51.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.234+0000 7f463b59e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558fef087400 0x558fef0a2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:51.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.234+0000 7f463b59e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558fef087400 0x558fef0a2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35880/0 (socket says 172.21.15.12:35880) 2024-05-17T22:48:51.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.234+0000 7f463b59e700 1 -- 172.21.15.12:0/2894175322 learned_addr learned my addr 172.21.15.12:0/2894175322 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:48:51.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.235+0000 7f463b59e700 1 -- 172.21.15.12:0/2894175322 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558feef039f0 msgr2=0x558fef0a48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:48:51.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.235+0000 7f463b59e700 1 --2- 172.21.15.12:0/2894175322 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558feef039f0 0x558fef0a48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:51.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.235+0000 7f463b59e700 1 -- 172.21.15.12:0/2894175322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fef081b00 msgr2=0x558fef0a4df0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T22:48:51.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.235+0000 7f463b59e700 1 --2- 172.21.15.12:0/2894175322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fef081b00 0x558fef0a4df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:51.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.235+0000 7f463b59e700 1 -- 172.21.15.12:0/2894175322 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fef0a2680 con 0x558fef087400 2024-05-17T22:48:51.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.236+0000 7f463b59e700 1 --2- 172.21.15.12:0/2894175322 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558fef087400 0x558fef0a2040 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4634009f80 tx=0x7f46340085c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49e2df5773dfa82e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:48:51.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.236+0000 7f461eff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:48:51.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.236+0000 7f461eff5700 1 -- 172.21.15.12:0/2894175322 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463401b040 con 0x558fef087400 2024-05-17T22:48:51.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.237+0000 7f464fa72b80 1 -- 172.21.15.12:0/2894175322 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f462c0b3480 con 0x558fef087400 2024-05-17T22:48:51.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.237+0000 7f461eff5700 1 -- 172.21.15.12:0/2894175322 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f4634004c80 con 0x558fef087400 2024-05-17T22:48:51.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.245+0000 7f461eff5700 1 -- 172.21.15.12:0/2894175322 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f463403d050 con 0x558fef087400 2024-05-17T22:48:51.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.247+0000 7f464fa72b80 10 client.16131 fetch_fsmap learned FSMap version 39 2024-05-17T22:48:51.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.247+0000 7f464fa72b80 1 -- 172.21.15.12:0/2894175322 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558fef105080 con 0x558fef087400 2024-05-17T22:48:51.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.248+0000 7f461eff5700 1 -- 172.21.15.12:0/2894175322 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f462c0b3480 con 0x558fef087400 2024-05-17T22:48:51.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.248+0000 7f461eff5700 1 -- 172.21.15.12:0/2894175322 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f46340054b0 con 0x558fef087400 2024-05-17T22:48:51.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.250+0000 7f464fa72b80 10 client.16131 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:48:51.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.250+0000 7f464fa72b80 10 client.16131 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:48:51.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.250+0000 7f464fa72b80 20 client.16131 populate_metadata read hostname 'smithi012' 2024-05-17T22:48:51.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.250+0000 7f464fa72b80 1 --2- 172.21.15.12:0/2894175322 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x558fef105400 0x558fef107820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:51.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.250+0000 7f464fa72b80 4 client.16131 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:48:51.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.250+0000 7f464fa72b80 1 -- 172.21.15.12:0/2894175322 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558feefab800 con 0x558fef105400 2024-05-17T22:48:51.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.251+0000 7f463ad9d700 1 --2- 172.21.15.12:0/2894175322 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x558fef105400 0x558fef107820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:51.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.254+0000 7f463ad9d700 1 --2- 172.21.15.12:0/2894175322 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x558fef105400 0x558fef107820 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bcf7dafb74cc9a6 server_cookie=27609d73d2804993 in_seq=0 out_seq=0 2024-05-17T22:48:51.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.254+0000 7f461eff5700 10 client.16131 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:48:51.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.254+0000 7f461eff5700 1 -- 172.21.15.12:0/2894175322 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558feefab800 con 0x558fef105400 2024-05-17T22:48:51.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.254+0000 7f461eff5700 10 client.16131 handle_command_reply: tid=0 2024-05-17T22:48:51.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.254+0000 7f464fa72b80 1 client.16131 shutdown 2024-05-17T22:48:51.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.255+0000 7f464fa72b80 1 -- 172.21.15.12:0/2894175322 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558fef087400 msgr2=0x558fef0a2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:51.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.255+0000 7f464fa72b80 1 --2- 172.21.15.12:0/2894175322 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558fef087400 0x558fef0a2040 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4634009f80 tx=0x7f46340085c0 comp rx=0 tx=0).stop 2024-05-17T22:48:51.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.256+0000 7f464fa72b80 1 -- 172.21.15.12:0/2894175322 shutdown_connections 2024-05-17T22:48:51.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.256+0000 7f464fa72b80 1 --2- 172.21.15.12:0/2894175322 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x558fef105400 0x558fef107820 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:51.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.256+0000 7f464fa72b80 1 --2- 172.21.15.12:0/2894175322 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558fef087400 0x558fef0a2040 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:51.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.256+0000 7f464fa72b80 1 --2- 172.21.15.12:0/2894175322 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558feef039f0 0x558fef0a48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:51.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.256+0000 7f464fa72b80 1 --2- 172.21.15.12:0/2894175322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fef081b00 0x558fef0a4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:51.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.256+0000 7f464fa72b80 1 -- 172.21.15.12:0/2894175322 >> 172.21.15.12:0/2894175322 conn(0x558feef637d0 msgr2=0x558fef0847f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:48:51.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.256+0000 7f461eff5700 0 client.16131 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:48:51.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.257+0000 7f464fa72b80 1 -- 172.21.15.12:0/2894175322 shutdown_connections 2024-05-17T22:48:51.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.257+0000 7f464fa72b80 1 -- 172.21.15.12:0/2894175322 wait complete. 2024-05-17T22:48:51.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.258+0000 7f464fa72b80 20 client.16131 trim_cache size 0 max 16384 2024-05-17T22:48:51.260 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:48:51.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.261+0000 7f4641097700 1 -- 172.21.15.12:0/2545184476 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f46240b2980 msgr2=0x7f46240b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:51.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.261+0000 7f4641097700 1 --2- 172.21.15.12:0/2545184476 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f46240b2980 0x7f46240b4e90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f463c06d120 tx=0x7f463400b040 comp rx=0 tx=0).stop 2024-05-17T22:48:51.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.261+0000 7f4641097700 1 -- 172.21.15.12:0/2545184476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f463c102e70 msgr2=0x7f463c19b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:51.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.261+0000 7f4641097700 1 --2- 172.21.15.12:0/2545184476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f463c102e70 0x7f463c19b670 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f462c000c00 tx=0x7f462c009510 comp rx=0 tx=0).stop 2024-05-17T22:48:51.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.261+0000 7f4641097700 1 -- 172.21.15.12:0/2545184476 shutdown_connections 2024-05-17T22:48:51.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.261+0000 7f4641097700 1 --2- 172.21.15.12:0/2545184476 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f46240b2980 0x7f46240b4e90 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:51.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.261+0000 7f4641097700 1 --2- 172.21.15.12:0/2545184476 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f463c1080a0 0x7f463c1a66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:51.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.261+0000 7f4641097700 1 --2- 172.21.15.12:0/2545184476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f463c102e70 0x7f463c19b670 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:51.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.261+0000 7f4641097700 1 --2- 172.21.15.12:0/2545184476 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f463c107750 0x7f463c19bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:51.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.261+0000 7f4641097700 1 -- 172.21.15.12:0/2545184476 >> 172.21.15.12:0/2545184476 conn(0x7f463c079ae0 msgr2=0x7f463c1065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:48:51.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.262+0000 7f4641097700 1 -- 172.21.15.12:0/2545184476 shutdown_connections 2024-05-17T22:48:51.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:51.262+0000 7f4641097700 1 -- 172.21.15.12:0/2545184476 wait complete. 2024-05-17T22:48:51.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 570.0 s, 482.8 tps, lat 204.310 ms stddev 260.937 2024-05-17T22:48:52.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:51 smithi151 ceph-mon[36101]: pgmap v727: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 4.2 MiB/s wr, 478 op/s 2024-05-17T22:48:52.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:51 smithi096 ceph-mon[36065]: pgmap v727: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 4.2 MiB/s wr, 478 op/s 2024-05-17T22:48:52.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:51 smithi012 ceph-mon[31632]: pgmap v727: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 4.2 MiB/s wr, 478 op/s 2024-05-17T22:48:52.980 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-17T22:48:52.980 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-17T22:48:53.451 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:48:54.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:53 smithi151 ceph-mon[36101]: pgmap v728: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.8 MiB/s wr, 443 op/s 2024-05-17T22:48:54.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:53 smithi096 ceph-mon[36065]: pgmap v728: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.8 MiB/s wr, 443 op/s 2024-05-17T22:48:54.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:53 smithi012 ceph-mon[31632]: pgmap v728: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.8 MiB/s wr, 443 op/s 2024-05-17T22:48:55.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.487+0000 7f4ebb3be700 1 -- 172.21.15.12:0/3395825468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4eb410b180 msgr2=0x7f4eb411d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:55.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.487+0000 7f4ebb3be700 1 --2- 172.21.15.12:0/3395825468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4eb410b180 0x7f4eb411d010 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4ea401a9a0 tx=0x7f4ea401acb0 comp rx=0 tx=0).stop 2024-05-17T22:48:55.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.490+0000 7f4ebb3be700 1 -- 172.21.15.12:0/3395825468 shutdown_connections 2024-05-17T22:48:55.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.490+0000 7f4ebb3be700 1 --2- 172.21.15.12:0/3395825468 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4eb411d550 0x7f4eb411f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:55.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.490+0000 7f4ebb3be700 1 --2- 172.21.15.12:0/3395825468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4eb410a830 0x7f4eb410ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:55.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.490+0000 7f4ebb3be700 1 --2- 172.21.15.12:0/3395825468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4eb410b180 0x7f4eb411d010 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:55.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.490+0000 7f4ebb3be700 1 -- 172.21.15.12:0/3395825468 >> 172.21.15.12:0/3395825468 conn(0x7f4eb406efc0 msgr2=0x7f4eb406f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:48:55.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.490+0000 7f4ebb3be700 1 -- 172.21.15.12:0/3395825468 shutdown_connections 2024-05-17T22:48:55.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.490+0000 7f4ebb3be700 1 -- 172.21.15.12:0/3395825468 wait complete. 2024-05-17T22:48:55.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.491+0000 7f4ebb3be700 1 Processor -- start 2024-05-17T22:48:55.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.491+0000 7f4ebb3be700 1 -- start start 2024-05-17T22:48:55.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.492+0000 7f4ebb3be700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4eb410a830 0x7f4eb411cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:55.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.492+0000 7f4ebb3be700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4eb410b180 0x7f4eb4119090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:55.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.492+0000 7f4ebb3be700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4eb411d550 0x7f4eb4119770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:55.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.492+0000 7f4ebb3be700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb41150d0 con 0x7f4eb410b180 2024-05-17T22:48:55.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.492+0000 7f4ebb3be700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb4114f50 con 0x7f4eb411d550 2024-05-17T22:48:55.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.492+0000 7f4ebb3be700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb4115250 con 0x7f4eb410a830 2024-05-17T22:48:55.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.494+0000 7f4eb8959700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4eb410b180 0x7f4eb4119090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:55.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.494+0000 7f4eb8959700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4eb410b180 0x7f4eb4119090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35910/0 (socket says 172.21.15.12:35910) 2024-05-17T22:48:55.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.494+0000 7f4eb8959700 1 -- 172.21.15.12:0/1190953540 learned_addr learned my addr 172.21.15.12:0/1190953540 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:48:55.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.496+0000 7f4eb995b700 1 --2- 172.21.15.12:0/1190953540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4eb411d550 0x7f4eb4119770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:55.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.496+0000 7f4eb915a700 1 --2- 172.21.15.12:0/1190953540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4eb410a830 0x7f4eb411cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:55.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.497+0000 7f4eb995b700 1 -- 172.21.15.12:0/1190953540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4eb410a830 msgr2=0x7f4eb411cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:55.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.497+0000 7f4eb995b700 1 --2- 172.21.15.12:0/1190953540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4eb410a830 0x7f4eb411cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:55.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.497+0000 7f4eb995b700 1 -- 172.21.15.12:0/1190953540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4eb410b180 msgr2=0x7f4eb4119090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:55.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.497+0000 7f4eb995b700 1 --2- 172.21.15.12:0/1190953540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4eb410b180 0x7f4eb4119090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:55.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.497+0000 7f4eb995b700 1 -- 172.21.15.12:0/1190953540 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb407ba90 con 0x7f4eb411d550 2024-05-17T22:48:55.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.497+0000 7f4eb995b700 1 --2- 172.21.15.12:0/1190953540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4eb411d550 0x7f4eb4119770 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4eb000b3f0 tx=0x7f4eb000b7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:48:55.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.497+0000 7f4eaa7fc700 1 -- 172.21.15.12:0/1190953540 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0015020 con 0x7f4eb411d550 2024-05-17T22:48:55.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.497+0000 7f4eaa7fc700 1 -- 172.21.15.12:0/1190953540 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4eb0011ce0 con 0x7f4eb411d550 2024-05-17T22:48:55.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.497+0000 7f4eaa7fc700 1 -- 172.21.15.12:0/1190953540 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0010d40 con 0x7f4eb411d550 2024-05-17T22:48:55.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.499+0000 7f4ebb3be700 1 -- 172.21.15.12:0/1190953540 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eb407bd80 con 0x7f4eb411d550 2024-05-17T22:48:55.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.499+0000 7f4ebb3be700 1 -- 172.21.15.12:0/1190953540 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eb407c270 con 0x7f4eb411d550 2024-05-17T22:48:55.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.510+0000 7f4eaa7fc700 1 -- 172.21.15.12:0/1190953540 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4eb0018020 con 0x7f4eb411d550 2024-05-17T22:48:55.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.516+0000 7f4ec9d99b80 1 Processor -- start 2024-05-17T22:48:55.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.516+0000 7f4ec9d99b80 1 -- start start 2024-05-17T22:48:55.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.516+0000 7f4ec9d99b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bee36e0a70 0x55bee38645e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:55.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.516+0000 7f4ec9d99b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bee3864b20 0x55bee385ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:55.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.516+0000 7f4ec9d99b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bee385f1f0 0x55bee385f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:55.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.516+0000 7f4ec9d99b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55bee378d310 con 0x55bee36e0a70 2024-05-17T22:48:55.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.516+0000 7f4ec9d99b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55bee377d050 con 0x55bee385f1f0 2024-05-17T22:48:55.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.516+0000 7f4ec9d99b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55bee3780ef0 con 0x55bee3864b20 2024-05-17T22:48:55.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.517+0000 7f4eaa7fc700 1 --2- 172.21.15.12:0/1190953540 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f4ea00b29b0 0x7f4ea00b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:55.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.517+0000 7f4eaa7fc700 1 -- 172.21.15.12:0/1190953540 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f4eb00e1b60 con 0x7f4eb411d550 2024-05-17T22:48:55.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.518+0000 7f4eb915a700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bee3864b20 0x55bee385ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:55.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.518+0000 7f4eb915a700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bee3864b20 0x55bee385ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:35354/0 (socket says 172.21.15.12:35354) 2024-05-17T22:48:55.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.518+0000 7f4eb915a700 1 -- 172.21.15.12:0/4188073089 learned_addr learned my addr 172.21.15.12:0/4188073089 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:48:55.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.519+0000 7f4eb8959700 1 --2- 172.21.15.12:0/4188073089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bee385f1f0 0x55bee385f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:55.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.520+0000 7f4eb995b700 1 --2- 172.21.15.12:0/1190953540 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f4ea00b29b0 0x7f4ea00b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:55.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.520+0000 7f4eb915a700 1 -- 172.21.15.12:0/4188073089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bee385f1f0 msgr2=0x55bee385f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:48:55.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.520+0000 7f4eb915a700 1 --2- 172.21.15.12:0/4188073089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bee385f1f0 0x55bee385f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:55.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.520+0000 7f4eb915a700 1 -- 172.21.15.12:0/4188073089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bee36e0a70 msgr2=0x55bee38645e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:48:55.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.520+0000 7f4eb915a700 1 --2- 172.21.15.12:0/4188073089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bee36e0a70 0x55bee38645e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:55.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.520+0000 7f4eb915a700 1 -- 172.21.15.12:0/4188073089 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bee3726610 con 0x55bee3864b20 2024-05-17T22:48:55.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.521+0000 7f4eb915a700 1 --2- 172.21.15.12:0/4188073089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bee3864b20 0x55bee385ecb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4eac009fd0 tx=0x7f4eac00c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac6c1f18ab2d44a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:48:55.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.521+0000 7f4e9f7fe700 1 -- 172.21.15.12:0/4188073089 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eac01a070 con 0x55bee3864b20 2024-05-17T22:48:55.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.521+0000 7f4e9f7fe700 1 -- 172.21.15.12:0/4188073089 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4eac00ce10 con 0x55bee3864b20 2024-05-17T22:48:55.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.521+0000 7f4e9f7fe700 1 -- 172.21.15.12:0/4188073089 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eac015a40 con 0x55bee3864b20 2024-05-17T22:48:55.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.522+0000 7f4ec9d99b80 1 -- 172.21.15.12:0/4188073089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bee3864b20 msgr2=0x55bee385ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:55.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.522+0000 7f4ec9d99b80 1 --2- 172.21.15.12:0/4188073089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bee3864b20 0x55bee385ecb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4eac009fd0 tx=0x7f4eac00c6e0 comp rx=0 tx=0).stop 2024-05-17T22:48:55.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.523+0000 7f4eb995b700 1 --2- 172.21.15.12:0/1190953540 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f4ea00b29b0 0x7f4ea00b4ec0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4eb00a4e10 tx=0x7f4eb0004860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:48:55.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.524+0000 7f4ec9d99b80 1 -- 172.21.15.12:0/4188073089 shutdown_connections 2024-05-17T22:48:55.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.524+0000 7f4ec9d99b80 1 --2- 172.21.15.12:0/4188073089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bee385f1f0 0x55bee385f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:55.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.524+0000 7f4ec9d99b80 1 --2- 172.21.15.12:0/4188073089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bee36e0a70 0x55bee38645e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:55.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.524+0000 7f4ec9d99b80 1 --2- 172.21.15.12:0/4188073089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bee3864b20 0x55bee385ecb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:55.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.525+0000 7f4ec9d99b80 1 -- 172.21.15.12:0/4188073089 >> 172.21.15.12:0/4188073089 conn(0x55bee3724f80 msgr2=0x55bee3859c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:48:55.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.528+0000 7f4ec9d99b80 1 -- 172.21.15.12:0/4188073089 shutdown_connections 2024-05-17T22:48:55.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.529+0000 7f4ec9d99b80 1 -- 172.21.15.12:0/4188073089 wait complete. 2024-05-17T22:48:55.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.529+0000 7f4ec9d99b80 1 -- start start 2024-05-17T22:48:55.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.529+0000 7f4ec9d99b80 1 Processor -- start 2024-05-17T22:48:55.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.530+0000 7f4ec9d99b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bee36e0a70 0x55bee38681f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:55.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.530+0000 7f4ec9d99b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bee385f1f0 0x55bee3868730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:55.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.530+0000 7f4ec9d99b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bee3864b20 0x55bee3868c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:55.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.530+0000 7f4ec9d99b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55bee36841d0 con 0x55bee3864b20 2024-05-17T22:48:55.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.530+0000 7f4ec9d99b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55bee371df00 con 0x55bee36e0a70 2024-05-17T22:48:55.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.530+0000 7f4ec9d99b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55bee386b410 con 0x55bee385f1f0 2024-05-17T22:48:55.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.530+0000 7f4ec9d99b80 10 client.0 _ll_register_callbacks cb 0x55bee3717f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:48:55.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.536+0000 7f4eb915a700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bee385f1f0 0x55bee3868730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:55.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.537+0000 7f4eb8959700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bee36e0a70 0x55bee38681f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:55.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.537+0000 7f4eb8959700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bee36e0a70 0x55bee38681f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:56890/0 (socket says 172.21.15.12:56890) 2024-05-17T22:48:55.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.537+0000 7f4eb8959700 1 -- 172.21.15.12:0/3699962301 learned_addr learned my addr 172.21.15.12:0/3699962301 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:48:55.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.537+0000 7f4eb8959700 1 --2- 172.21.15.12:0/3699962301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bee3864b20 0x55bee3868c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:55.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.537+0000 7f4eb8959700 1 -- 172.21.15.12:0/3699962301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bee385f1f0 msgr2=0x55bee3868730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:48:55.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.537+0000 7f4eb8959700 1 --2- 172.21.15.12:0/3699962301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bee385f1f0 0x55bee3868730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:55.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.537+0000 7f4eb8959700 1 -- 172.21.15.12:0/3699962301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bee3864b20 msgr2=0x55bee3868c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:48:55.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.537+0000 7f4eb8959700 1 --2- 172.21.15.12:0/3699962301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bee3864b20 0x55bee3868c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:55.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.537+0000 7f4eb8959700 1 -- 172.21.15.12:0/3699962301 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bee36cd500 con 0x55bee36e0a70 2024-05-17T22:48:55.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.542+0000 7f4eb8959700 1 --2- 172.21.15.12:0/3699962301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bee36e0a70 0x55bee38681f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4ea401a490 tx=0x7f4ea401afd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bddec583e191053 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:48:55.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.542+0000 7f4e9cff9700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:48:55.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.548+0000 7f4e9cff9700 1 -- 172.21.15.12:0/3699962301 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea401cc30 con 0x55bee36e0a70 2024-05-17T22:48:55.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.548+0000 7f4ec9d99b80 1 -- 172.21.15.12:0/3699962301 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4eac00ce10 con 0x55bee36e0a70 2024-05-17T22:48:55.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.548+0000 7f4e9cff9700 1 -- 172.21.15.12:0/3699962301 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f4ea401cdd0 con 0x55bee36e0a70 2024-05-17T22:48:55.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.555+0000 7f4e9cff9700 1 -- 172.21.15.12:0/3699962301 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4ea402e050 con 0x55bee36e0a70 2024-05-17T22:48:55.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.555+0000 7f4ec9d99b80 10 client.25853 fetch_fsmap learned FSMap version 39 2024-05-17T22:48:55.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.555+0000 7f4ec9d99b80 1 -- 172.21.15.12:0/3699962301 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55bee3875e90 con 0x55bee36e0a70 2024-05-17T22:48:55.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.558+0000 7f4e9cff9700 1 -- 172.21.15.12:0/3699962301 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4eac00ce10 con 0x55bee36e0a70 2024-05-17T22:48:55.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.559+0000 7f4e9cff9700 1 -- 172.21.15.12:0/3699962301 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f4ea4011930 con 0x55bee36e0a70 2024-05-17T22:48:55.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.559+0000 7f4ec9d99b80 10 client.25853 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:48:55.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.559+0000 7f4ec9d99b80 10 client.25853 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:48:55.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.559+0000 7f4ec9d99b80 20 client.25853 populate_metadata read hostname 'smithi012' 2024-05-17T22:48:55.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.559+0000 7f4ec9d99b80 1 --2- 172.21.15.12:0/3699962301 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55bee3878850 0x55bee387ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:55.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.559+0000 7f4ec9d99b80 4 client.25853 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:48:55.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.559+0000 7f4ec9d99b80 1 -- 172.21.15.12:0/3699962301 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bee378f260 con 0x55bee3878850 2024-05-17T22:48:55.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.561+0000 7f4eb915a700 1 --2- 172.21.15.12:0/3699962301 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55bee3878850 0x55bee387ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:55.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.564+0000 7f4e9cff9700 10 client.25853 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:48:55.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.564+0000 7f4eb915a700 1 --2- 172.21.15.12:0/3699962301 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55bee3878850 0x55bee387ace0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cea8a92a0b60947 server_cookie=a6b835207471b519 in_seq=0 out_seq=0 2024-05-17T22:48:55.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.565+0000 7f4e9cff9700 1 -- 172.21.15.12:0/3699962301 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bee378f260 con 0x55bee3878850 2024-05-17T22:48:55.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.565+0000 7f4e9cff9700 10 client.25853 handle_command_reply: tid=0 2024-05-17T22:48:55.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.565+0000 7f4ec9d99b80 1 client.25853 shutdown 2024-05-17T22:48:55.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.567+0000 7f4ec9d99b80 1 -- 172.21.15.12:0/3699962301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bee36e0a70 msgr2=0x55bee38681f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:55.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.567+0000 7f4ec9d99b80 1 --2- 172.21.15.12:0/3699962301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bee36e0a70 0x55bee38681f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4ea401a490 tx=0x7f4ea401afd0 comp rx=0 tx=0).stop 2024-05-17T22:48:55.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.569+0000 7f4ec9d99b80 1 -- 172.21.15.12:0/3699962301 shutdown_connections 2024-05-17T22:48:55.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.569+0000 7f4ec9d99b80 1 --2- 172.21.15.12:0/3699962301 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55bee3878850 0x55bee387ace0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:55.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.569+0000 7f4ec9d99b80 1 --2- 172.21.15.12:0/3699962301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bee3864b20 0x55bee3868c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:55.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.569+0000 7f4ec9d99b80 1 --2- 172.21.15.12:0/3699962301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bee36e0a70 0x55bee38681f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:55.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.569+0000 7f4ec9d99b80 1 --2- 172.21.15.12:0/3699962301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bee385f1f0 0x55bee3868730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:55.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.569+0000 7f4ec9d99b80 1 -- 172.21.15.12:0/3699962301 >> 172.21.15.12:0/3699962301 conn(0x55bee3724f80 msgr2=0x55bee3853f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:48:55.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.569+0000 7f4e9cff9700 0 client.25853 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:48:55.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.573+0000 7f4ec9d99b80 1 -- 172.21.15.12:0/3699962301 shutdown_connections 2024-05-17T22:48:55.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.573+0000 7f4ec9d99b80 1 -- 172.21.15.12:0/3699962301 wait complete. 2024-05-17T22:48:55.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.574+0000 7f4ec9d99b80 20 client.25853 trim_cache size 0 max 16384 2024-05-17T22:48:55.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.618+0000 7f4ec9d99b80 1 Processor -- start 2024-05-17T22:48:55.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.620+0000 7f4ec9d99b80 1 -- start start 2024-05-17T22:48:55.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.620+0000 7f4ec9d99b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bee3724f80 0x55bee385ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:55.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.620+0000 7f4ec9d99b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bee36e0a70 0x55bee3853f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:55.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.620+0000 7f4ec9d99b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bee385f1f0 0x55bee3868ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:55.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.620+0000 7f4ec9d99b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb41ebda0 con 0x55bee3724f80 2024-05-17T22:48:55.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.620+0000 7f4ec9d99b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb4084e60 con 0x55bee36e0a70 2024-05-17T22:48:55.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.620+0000 7f4ec9d99b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb4082cb0 con 0x55bee385f1f0 2024-05-17T22:48:55.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.625+0000 7f4eb8959700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bee3724f80 0x55bee385ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:55.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.625+0000 7f4eb915a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bee36e0a70 0x55bee3853f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:55.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.625+0000 7f4eb8959700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bee3724f80 0x55bee385ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35938/0 (socket says 172.21.15.12:35938) 2024-05-17T22:48:55.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.625+0000 7f4eb8959700 1 -- 172.21.15.12:0/477818720 learned_addr learned my addr 172.21.15.12:0/477818720 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:48:55.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.625+0000 7f4eb8959700 1 --2- 172.21.15.12:0/477818720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bee385f1f0 0x55bee3868ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:55.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.628+0000 7f4eb915a700 1 -- 172.21.15.12:0/477818720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bee385f1f0 msgr2=0x55bee3868ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:48:55.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.628+0000 7f4eb915a700 1 --2- 172.21.15.12:0/477818720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bee385f1f0 0x55bee3868ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:55.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.628+0000 7f4eb915a700 1 -- 172.21.15.12:0/477818720 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bee3724f80 msgr2=0x55bee385ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:48:55.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.628+0000 7f4eb915a700 1 --2- 172.21.15.12:0/477818720 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bee3724f80 0x55bee385ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:55.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.628+0000 7f4eb915a700 1 -- 172.21.15.12:0/477818720 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bee3866740 con 0x55bee36e0a70 2024-05-17T22:48:55.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.650+0000 7f4eb915a700 1 --2- 172.21.15.12:0/477818720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bee36e0a70 0x55bee3853f30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4eac018040 tx=0x7f4eac015710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e0cdd0b24d5dcd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:48:55.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.651+0000 7f4e9effd700 1 -- 172.21.15.12:0/477818720 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eac028070 con 0x55bee36e0a70 2024-05-17T22:48:55.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.651+0000 7f4e9effd700 1 -- 172.21.15.12:0/477818720 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4eac01b040 con 0x55bee36e0a70 2024-05-17T22:48:55.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.651+0000 7f4e9effd700 1 -- 172.21.15.12:0/477818720 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eac01a040 con 0x55bee36e0a70 2024-05-17T22:48:55.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.652+0000 7f4ec9d99b80 1 -- 172.21.15.12:0/477818720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bee36e0a70 msgr2=0x55bee3853f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:55.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.652+0000 7f4ec9d99b80 1 --2- 172.21.15.12:0/477818720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bee36e0a70 0x55bee3853f30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4eac018040 tx=0x7f4eac015710 comp rx=0 tx=0).stop 2024-05-17T22:48:55.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.652+0000 7f4ec9d99b80 1 -- 172.21.15.12:0/477818720 shutdown_connections 2024-05-17T22:48:55.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.652+0000 7f4ec9d99b80 1 --2- 172.21.15.12:0/477818720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bee385f1f0 0x55bee3868ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:55.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.652+0000 7f4ec9d99b80 1 --2- 172.21.15.12:0/477818720 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bee3724f80 0x55bee385ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:55.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.652+0000 7f4ec9d99b80 1 --2- 172.21.15.12:0/477818720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bee36e0a70 0x55bee3853f30 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:55.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.652+0000 7f4ec9d99b80 1 -- 172.21.15.12:0/477818720 >> 172.21.15.12:0/477818720 conn(0x55bee385ac90 msgr2=0x55bee3862690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:48:55.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.653+0000 7f4ec9d99b80 1 -- 172.21.15.12:0/477818720 shutdown_connections 2024-05-17T22:48:55.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.660+0000 7f4ec9d99b80 1 -- 172.21.15.12:0/477818720 wait complete. 2024-05-17T22:48:55.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.660+0000 7f4ec9d99b80 1 -- start start 2024-05-17T22:48:55.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.660+0000 7f4ec9d99b80 1 Processor -- start 2024-05-17T22:48:55.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.661+0000 7f4ec9d99b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bee36e0a70 0x55bee38702a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:55.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.661+0000 7f4ec9d99b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bee3724f80 0x55bee38707e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:55.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.661+0000 7f4ec9d99b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bee385f1f0 0x55bee386d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:55.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.661+0000 7f4ec9d99b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb4083320 con 0x55bee3724f80 2024-05-17T22:48:55.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.661+0000 7f4ec9d99b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb4084790 con 0x55bee385f1f0 2024-05-17T22:48:55.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.661+0000 7f4ec9d99b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb40839f0 con 0x55bee36e0a70 2024-05-17T22:48:55.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.661+0000 7f4ec9d99b80 10 client.0 _ll_register_callbacks cb 0x55bee3717f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:48:55.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.665+0000 7f4eb8959700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bee36e0a70 0x55bee38702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:55.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.665+0000 7f4eb8959700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bee385f1f0 0x55bee386d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:55.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.665+0000 7f4eb8959700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bee36e0a70 0x55bee38702a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:35374/0 (socket says 172.21.15.12:35374) 2024-05-17T22:48:55.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.665+0000 7f4eb8959700 1 -- 172.21.15.12:0/1522239811 learned_addr learned my addr 172.21.15.12:0/1522239811 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:48:55.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.665+0000 7f4eb915a700 1 --2- 172.21.15.12:0/1522239811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bee3724f80 0x55bee38707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:55.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.671+0000 7f4eb915a700 1 -- 172.21.15.12:0/1522239811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bee36e0a70 msgr2=0x55bee38702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:48:55.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.671+0000 7f4eb915a700 1 --2- 172.21.15.12:0/1522239811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bee36e0a70 0x55bee38702a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:55.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.671+0000 7f4eb915a700 1 -- 172.21.15.12:0/1522239811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bee385f1f0 msgr2=0x55bee386d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:48:55.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.671+0000 7f4eb915a700 1 --2- 172.21.15.12:0/1522239811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bee385f1f0 0x55bee386d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:55.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.671+0000 7f4eb915a700 1 -- 172.21.15.12:0/1522239811 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bee386d830 con 0x55bee3724f80 2024-05-17T22:48:55.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.676+0000 7f4eb915a700 1 --2- 172.21.15.12:0/1522239811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bee3724f80 0x55bee38707e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4eac015510 tx=0x7f4eac0098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19aec2845a0e63cc server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:48:55.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.676+0000 7f4e9cff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:48:55.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.677+0000 7f4e9cff9700 1 -- 172.21.15.12:0/1522239811 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eac031020 con 0x55bee3724f80 2024-05-17T22:48:55.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.677+0000 7f4e9cff9700 1 -- 172.21.15.12:0/1522239811 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f4eac015990 con 0x55bee3724f80 2024-05-17T22:48:55.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.677+0000 7f4ec9d99b80 1 -- 172.21.15.12:0/1522239811 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4eac01b040 con 0x55bee3724f80 2024-05-17T22:48:55.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.685+0000 7f4e9cff9700 1 -- 172.21.15.12:0/1522239811 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4eac01a070 con 0x55bee3724f80 2024-05-17T22:48:55.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.686+0000 7f4ec9d99b80 10 client.16143 fetch_fsmap learned FSMap version 39 2024-05-17T22:48:55.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.686+0000 7f4ec9d99b80 1 -- 172.21.15.12:0/1522239811 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55bee38e38c0 con 0x55bee3724f80 2024-05-17T22:48:55.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.689+0000 7f4e9cff9700 1 -- 172.21.15.12:0/1522239811 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4eac01b040 con 0x55bee3724f80 2024-05-17T22:48:55.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.689+0000 7f4e9cff9700 1 -- 172.21.15.12:0/1522239811 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f4eac0240f0 con 0x55bee3724f80 2024-05-17T22:48:55.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.689+0000 7f4ec9d99b80 10 client.16143 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:48:55.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.689+0000 7f4ec9d99b80 10 client.16143 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:48:55.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.689+0000 7f4ec9d99b80 20 client.16143 populate_metadata read hostname 'smithi012' 2024-05-17T22:48:55.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.689+0000 7f4ec9d99b80 1 --2- 172.21.15.12:0/1522239811 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55bee3864b20 0x55bee38e5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:48:55.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.689+0000 7f4ec9d99b80 4 client.16143 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-17T22:48:55.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.689+0000 7f4ec9d99b80 1 -- 172.21.15.12:0/1522239811 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bee3788a40 con 0x55bee3864b20 2024-05-17T22:48:55.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.689+0000 7f4eb8959700 1 --2- 172.21.15.12:0/1522239811 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55bee3864b20 0x55bee38e5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:48:55.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.693+0000 7f4e9cff9700 10 client.16143 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:48:55.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.693+0000 7f4eb8959700 1 --2- 172.21.15.12:0/1522239811 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55bee3864b20 0x55bee38e5cc0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=217973edd36eb0c9 server_cookie=8e5731c9c3b0de4b in_seq=0 out_seq=0 2024-05-17T22:48:55.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.695+0000 7f4e9cff9700 1 -- 172.21.15.12:0/1522239811 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bee3788a40 con 0x55bee3864b20 2024-05-17T22:48:55.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.695+0000 7f4e9cff9700 10 client.16143 handle_command_reply: tid=0 2024-05-17T22:48:55.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.695+0000 7f4ec9d99b80 1 client.16143 shutdown 2024-05-17T22:48:55.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.696+0000 7f4ec9d99b80 1 -- 172.21.15.12:0/1522239811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bee3724f80 msgr2=0x55bee38707e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:55.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.696+0000 7f4ec9d99b80 1 --2- 172.21.15.12:0/1522239811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bee3724f80 0x55bee38707e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4eac015510 tx=0x7f4eac0098a0 comp rx=0 tx=0).stop 2024-05-17T22:48:55.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.696+0000 7f4ec9d99b80 1 -- 172.21.15.12:0/1522239811 shutdown_connections 2024-05-17T22:48:55.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.696+0000 7f4ec9d99b80 1 --2- 172.21.15.12:0/1522239811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bee385f1f0 0x55bee386d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:55.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.696+0000 7f4ec9d99b80 1 --2- 172.21.15.12:0/1522239811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bee36e0a70 0x55bee38702a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:55.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.696+0000 7f4ec9d99b80 1 --2- 172.21.15.12:0/1522239811 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55bee3864b20 0x55bee38e5cc0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:55.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.696+0000 7f4ec9d99b80 1 --2- 172.21.15.12:0/1522239811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bee3724f80 0x55bee38707e0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:55.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.696+0000 7f4ec9d99b80 1 -- 172.21.15.12:0/1522239811 >> 172.21.15.12:0/1522239811 conn(0x55bee385ac90 msgr2=0x55bee3862690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:48:55.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.696+0000 7f4e9cff9700 0 client.16143 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:48:55.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.696+0000 7f4ec9d99b80 1 -- 172.21.15.12:0/1522239811 shutdown_connections 2024-05-17T22:48:55.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.696+0000 7f4ec9d99b80 1 -- 172.21.15.12:0/1522239811 wait complete. 2024-05-17T22:48:55.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.696+0000 7f4ec9d99b80 20 client.16143 trim_cache size 0 max 16384 2024-05-17T22:48:55.700 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:48:55.700 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-17T22:48:55.700 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "4fdbc97d-8f04-4808-b4c5-78678aca675a", 2024-05-17T22:48:55.700 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-17T22:48:55.700 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:48:55.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.699+0000 7f4ebb3be700 1 -- 172.21.15.12:0/1190953540 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f4ea00b29b0 msgr2=0x7f4ea00b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:55.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.699+0000 7f4ebb3be700 1 --2- 172.21.15.12:0/1190953540 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f4ea00b29b0 0x7f4ea00b4ec0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4eb00a4e10 tx=0x7f4eb0004860 comp rx=0 tx=0).stop 2024-05-17T22:48:55.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.699+0000 7f4ebb3be700 1 -- 172.21.15.12:0/1190953540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4eb411d550 msgr2=0x7f4eb4119770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:48:55.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.699+0000 7f4ebb3be700 1 --2- 172.21.15.12:0/1190953540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4eb411d550 0x7f4eb4119770 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4eb000b3f0 tx=0x7f4eb000b7b0 comp rx=0 tx=0).stop 2024-05-17T22:48:55.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.699+0000 7f4ebb3be700 1 -- 172.21.15.12:0/1190953540 shutdown_connections 2024-05-17T22:48:55.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.699+0000 7f4ebb3be700 1 --2- 172.21.15.12:0/1190953540 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f4ea00b29b0 0x7f4ea00b4ec0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:55.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.699+0000 7f4ebb3be700 1 --2- 172.21.15.12:0/1190953540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4eb411d550 0x7f4eb4119770 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:55.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.699+0000 7f4ebb3be700 1 --2- 172.21.15.12:0/1190953540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4eb410a830 0x7f4eb411cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:55.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.699+0000 7f4ebb3be700 1 --2- 172.21.15.12:0/1190953540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4eb410b180 0x7f4eb4119090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:48:55.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.699+0000 7f4ebb3be700 1 -- 172.21.15.12:0/1190953540 >> 172.21.15.12:0/1190953540 conn(0x7f4eb406efc0 msgr2=0x7f4eb411daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:48:55.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.700+0000 7f4ebb3be700 1 -- 172.21.15.12:0/1190953540 shutdown_connections 2024-05-17T22:48:55.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:48:55.700+0000 7f4ebb3be700 1 -- 172.21.15.12:0/1190953540 wait complete. 2024-05-17T22:48:56.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:55 smithi012 ceph-mon[31632]: pgmap v729: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.8 MiB/s wr, 561 op/s 2024-05-17T22:48:56.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:55 smithi151 ceph-mon[36101]: pgmap v729: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.8 MiB/s wr, 561 op/s 2024-05-17T22:48:56.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:55 smithi096 ceph-mon[36065]: pgmap v729: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.8 MiB/s wr, 561 op/s 2024-05-17T22:48:56.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 575.0 s, 500.0 tps, lat 205.980 ms stddev 266.751 2024-05-17T22:48:56.791 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-17T22:48:57.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:56 smithi012 ceph-mon[31632]: scrub queued for path: / 2024-05-17T22:48:57.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:56 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:48:57.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:56 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:48:57.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:56 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:48:57.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:56 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:48:57.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:56 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:48:57.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:56 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:48:57.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:56 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:48:57.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:56 smithi151 ceph-mon[36101]: scrub queued for path: / 2024-05-17T22:48:57.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:56 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:48:57.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:56 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:48:57.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:56 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:48:57.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:56 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:48:57.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:56 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:48:57.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:56 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:48:57.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:56 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:48:57.296 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:48:57.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:56 smithi096 ceph-mon[36065]: scrub queued for path: / 2024-05-17T22:48:57.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:56 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:48:57.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:56 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:48:57.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:56 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:48:57.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:56 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:48:57.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:56 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:48:57.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:56 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:48:57.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:56 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:48:58.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:48:57 smithi012 ceph-mon[31632]: pgmap v730: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.0 MiB/s wr, 450 op/s 2024-05-17T22:48:58.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:48:57 smithi151 ceph-mon[36101]: pgmap v730: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.0 MiB/s wr, 450 op/s 2024-05-17T22:48:58.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:48:57 smithi096 ceph-mon[36065]: pgmap v730: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.0 MiB/s wr, 450 op/s 2024-05-17T22:49:00.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:49:00 smithi096 ceph-mon[36065]: pgmap v731: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.4 MiB/s wr, 517 op/s 2024-05-17T22:49:00.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:49:00 smithi012 ceph-mon[31632]: pgmap v731: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.4 MiB/s wr, 517 op/s 2024-05-17T22:49:00.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:49:00 smithi151 ceph-mon[36101]: pgmap v731: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.4 MiB/s wr, 517 op/s 2024-05-17T22:49:01.120 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:49:01.121 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-17T22:49:01.121 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-17T22:49:01.121 INFO:teuthology.orchestra.run.smithi012.stdout: "4fdbc97d-8f04-4808-b4c5-78678aca675a": { 2024-05-17T22:49:01.121 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-17T22:49:01.121 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "4fdbc97d-8f04-4808-b4c5-78678aca675a", 2024-05-17T22:49:01.121 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-17T22:49:01.121 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:49:01.121 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:49:01.121 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:49:01.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.128+0000 7f0317662700 1 -- 172.21.15.12:0/3654726421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03101025b0 msgr2=0x7f03101029c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:01.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.128+0000 7f0317662700 1 --2- 172.21.15.12:0/3654726421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03101025b0 0x7f03101029c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f030c009990 tx=0x7f030c009ca0 comp rx=0 tx=0).stop 2024-05-17T22:49:01.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.129+0000 7f0317662700 1 -- 172.21.15.12:0/3654726421 shutdown_connections 2024-05-17T22:49:01.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.129+0000 7f0317662700 1 --2- 172.21.15.12:0/3654726421 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0310107830 0x7f031010f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:01.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.129+0000 7f0317662700 1 --2- 172.21.15.12:0/3654726421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03101025b0 0x7f03101029c0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:01.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.129+0000 7f0317662700 1 --2- 172.21.15.12:0/3654726421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0310106ee0 0x7f03101072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:01.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.129+0000 7f0317662700 1 -- 172.21.15.12:0/3654726421 >> 172.21.15.12:0/3654726421 conn(0x7f03100fe1a0 msgr2=0x7f03101005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:49:01.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.129+0000 7f0317662700 1 -- 172.21.15.12:0/3654726421 shutdown_connections 2024-05-17T22:49:01.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.129+0000 7f0317662700 1 -- 172.21.15.12:0/3654726421 wait complete. 2024-05-17T22:49:01.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.129+0000 7f0317662700 1 Processor -- start 2024-05-17T22:49:01.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.129+0000 7f0317662700 1 -- start start 2024-05-17T22:49:01.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.130+0000 7f0317662700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03101025b0 0x7f031019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:01.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.130+0000 7f0317662700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0310106ee0 0x7f031019ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:01.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.130+0000 7f0317662700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0310107830 0x7f031019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:01.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.130+0000 7f0317662700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0310111bd0 con 0x7f03101025b0 2024-05-17T22:49:01.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.130+0000 7f0317662700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0310111a50 con 0x7f0310106ee0 2024-05-17T22:49:01.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.130+0000 7f0317662700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0310111d50 con 0x7f0310107830 2024-05-17T22:49:01.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.130+0000 7f03153fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03101025b0 0x7f031019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:01.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.130+0000 7f03153fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03101025b0 0x7f031019a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47742/0 (socket says 172.21.15.12:47742) 2024-05-17T22:49:01.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.130+0000 7f03153fe700 1 -- 172.21.15.12:0/3832658089 learned_addr learned my addr 172.21.15.12:0/3832658089 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:49:01.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.130+0000 7f0315bff700 1 --2- 172.21.15.12:0/3832658089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0310107830 0x7f031019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:01.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.130+0000 7f0314bfd700 1 --2- 172.21.15.12:0/3832658089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0310106ee0 0x7f031019ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:01.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.130+0000 7f03153fe700 1 -- 172.21.15.12:0/3832658089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0310107830 msgr2=0x7f031019efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:01.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.130+0000 7f03153fe700 1 --2- 172.21.15.12:0/3832658089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0310107830 0x7f031019efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:01.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.130+0000 7f03153fe700 1 -- 172.21.15.12:0/3832658089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0310106ee0 msgr2=0x7f031019ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:01.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.130+0000 7f03153fe700 1 --2- 172.21.15.12:0/3832658089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0310106ee0 0x7f031019ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:01.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.130+0000 7f03153fe700 1 -- 172.21.15.12:0/3832658089 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f031019f650 con 0x7f03101025b0 2024-05-17T22:49:01.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.130+0000 7f03153fe700 1 --2- 172.21.15.12:0/3832658089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03101025b0 0x7f031019a780 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f030c000c00 tx=0x7f030c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:49:01.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.130+0000 7f03067fc700 1 -- 172.21.15.12:0/3832658089 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030c027070 con 0x7f03101025b0 2024-05-17T22:49:01.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.131+0000 7f03067fc700 1 -- 172.21.15.12:0/3832658089 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f030c022470 con 0x7f03101025b0 2024-05-17T22:49:01.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.131+0000 7f0317662700 1 -- 172.21.15.12:0/3832658089 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f031019f8e0 con 0x7f03101025b0 2024-05-17T22:49:01.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.131+0000 7f03067fc700 1 -- 172.21.15.12:0/3832658089 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030c017650 con 0x7f03101025b0 2024-05-17T22:49:01.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.131+0000 7f0317662700 1 -- 172.21.15.12:0/3832658089 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f031019fdc0 con 0x7f03101025b0 2024-05-17T22:49:01.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.133+0000 7f03067fc700 1 -- 172.21.15.12:0/3832658089 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f030c02a070 con 0x7f03101025b0 2024-05-17T22:49:01.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.134+0000 7f03067fc700 1 --2- 172.21.15.12:0/3832658089 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f02fc0b2880 0x7f02fc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:01.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.134+0000 7f032603db80 1 Processor -- start 2024-05-17T22:49:01.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.134+0000 7f0314bfd700 1 --2- 172.21.15.12:0/3832658089 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f02fc0b2880 0x7f02fc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:01.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.134+0000 7f032603db80 1 -- start start 2024-05-17T22:49:01.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.134+0000 7f03067fc700 1 -- 172.21.15.12:0/3832658089 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f030c0e3470 con 0x7f03101025b0 2024-05-17T22:49:01.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.135+0000 7f032603db80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d69f49f0 0x5574d6b77300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:01.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.135+0000 7f032603db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574d6b77840 0x5574d6b7b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:01.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.135+0000 7f032603db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574d6b7b9f0 0x5574d6b7be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:01.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.135+0000 7f032603db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5574d6aa1290 con 0x5574d6b7b9f0 2024-05-17T22:49:01.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.135+0000 7f032603db80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5574d6a90fd0 con 0x5574d69f49f0 2024-05-17T22:49:01.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.135+0000 7f032603db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5574d6a94e70 con 0x5574d6b77840 2024-05-17T22:49:01.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.135+0000 7f0314bfd700 1 --2- 172.21.15.12:0/3832658089 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f02fc0b2880 0x7f02fc0b4d90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0300005950 tx=0x7f03000058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:49:01.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.135+0000 7f0315bff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574d6b7b9f0 0x5574d6b7be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:01.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.135+0000 7f0314bfd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574d6b77840 0x5574d6b7b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:01.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.135+0000 7f0315bff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574d6b7b9f0 0x5574d6b7be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47750/0 (socket says 172.21.15.12:47750) 2024-05-17T22:49:01.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.135+0000 7f0315bff700 1 -- 172.21.15.12:0/2742733817 learned_addr learned my addr 172.21.15.12:0/2742733817 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:49:01.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.135+0000 7f03153fe700 1 --2- 172.21.15.12:0/2742733817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d69f49f0 0x5574d6b77300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:01.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.135+0000 7f0315bff700 1 -- 172.21.15.12:0/2742733817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574d6b77840 msgr2=0x5574d6b7b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:49:01.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.135+0000 7f0315bff700 1 --2- 172.21.15.12:0/2742733817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574d6b77840 0x5574d6b7b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:01.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.135+0000 7f0315bff700 1 -- 172.21.15.12:0/2742733817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d69f49f0 msgr2=0x5574d6b77300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:49:01.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.135+0000 7f0315bff700 1 --2- 172.21.15.12:0/2742733817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d69f49f0 0x5574d6b77300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:01.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.135+0000 7f0315bff700 1 -- 172.21.15.12:0/2742733817 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574d6a3a590 con 0x5574d6b7b9f0 2024-05-17T22:49:01.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.135+0000 7f0314bfd700 1 --2- 172.21.15.12:0/2742733817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574d6b77840 0x5574d6b7b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:49:01.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.135+0000 7f03153fe700 1 --2- 172.21.15.12:0/2742733817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d69f49f0 0x5574d6b77300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:49:01.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.135+0000 7f0315bff700 1 --2- 172.21.15.12:0/2742733817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574d6b7b9f0 0x5574d6b7be00 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f030800a610 tx=0x7f03080088f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b4e5d575e97ade server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:49:01.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.135+0000 7f02fb7fe700 1 -- 172.21.15.12:0/2742733817 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030801b070 con 0x5574d6b7b9f0 2024-05-17T22:49:01.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.135+0000 7f02fb7fe700 1 -- 172.21.15.12:0/2742733817 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0308007ca0 con 0x5574d6b7b9f0 2024-05-17T22:49:01.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.135+0000 7f02fb7fe700 1 -- 172.21.15.12:0/2742733817 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0308016e10 con 0x5574d6b7b9f0 2024-05-17T22:49:01.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.136+0000 7f032603db80 1 -- 172.21.15.12:0/2742733817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574d6b7b9f0 msgr2=0x5574d6b7be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:01.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.136+0000 7f032603db80 1 --2- 172.21.15.12:0/2742733817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574d6b7b9f0 0x5574d6b7be00 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f030800a610 tx=0x7f03080088f0 comp rx=0 tx=0).stop 2024-05-17T22:49:01.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.136+0000 7f032603db80 1 -- 172.21.15.12:0/2742733817 shutdown_connections 2024-05-17T22:49:01.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.136+0000 7f032603db80 1 --2- 172.21.15.12:0/2742733817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574d6b7b9f0 0x5574d6b7be00 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:01.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.136+0000 7f032603db80 1 --2- 172.21.15.12:0/2742733817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d69f49f0 0x5574d6b77300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:01.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.136+0000 7f032603db80 1 --2- 172.21.15.12:0/2742733817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574d6b77840 0x5574d6b7b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:01.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.136+0000 7f032603db80 1 -- 172.21.15.12:0/2742733817 >> 172.21.15.12:0/2742733817 conn(0x5574d6a38f00 msgr2=0x5574d6b6db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:49:01.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.136+0000 7f032603db80 1 -- 172.21.15.12:0/2742733817 shutdown_connections 2024-05-17T22:49:01.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.136+0000 7f032603db80 1 -- 172.21.15.12:0/2742733817 wait complete. 2024-05-17T22:49:01.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.137+0000 7f032603db80 1 -- start start 2024-05-17T22:49:01.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.137+0000 7f032603db80 1 Processor -- start 2024-05-17T22:49:01.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.137+0000 7f032603db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574d69f49f0 0x5574d6b80750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:01.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.137+0000 7f032603db80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d6b77840 0x5574d6b80c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:01.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.137+0000 7f032603db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574d6b7b9f0 0x5574d6b8ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:01.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.137+0000 7f032603db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5574d6998150 con 0x5574d69f49f0 2024-05-17T22:49:01.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.137+0000 7f032603db80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5574d6a31e80 con 0x5574d6b77840 2024-05-17T22:49:01.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.137+0000 7f032603db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5574d6b837c0 con 0x5574d6b7b9f0 2024-05-17T22:49:01.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.137+0000 7f032603db80 10 client.0 _ll_register_callbacks cb 0x5574d6a2bf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:49:01.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.137+0000 7f03153fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574d69f49f0 0x5574d6b80750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:01.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.137+0000 7f03153fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574d69f49f0 0x5574d6b80750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47766/0 (socket says 172.21.15.12:47766) 2024-05-17T22:49:01.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.137+0000 7f03153fe700 1 -- 172.21.15.12:0/4200191050 learned_addr learned my addr 172.21.15.12:0/4200191050 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:49:01.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.137+0000 7f0315bff700 1 --2- 172.21.15.12:0/4200191050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574d6b7b9f0 0x5574d6b8ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:01.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.137+0000 7f0314bfd700 1 --2- 172.21.15.12:0/4200191050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d6b77840 0x5574d6b80c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:01.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.138+0000 7f03153fe700 1 -- 172.21.15.12:0/4200191050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574d6b7b9f0 msgr2=0x5574d6b8ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:49:01.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.138+0000 7f03153fe700 1 --2- 172.21.15.12:0/4200191050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574d6b7b9f0 0x5574d6b8ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:01.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.138+0000 7f03153fe700 1 -- 172.21.15.12:0/4200191050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d6b77840 msgr2=0x5574d6b80c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:49:01.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.138+0000 7f03153fe700 1 --2- 172.21.15.12:0/4200191050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d6b77840 0x5574d6b80c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:01.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.138+0000 7f03153fe700 1 -- 172.21.15.12:0/4200191050 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574d69e1480 con 0x5574d69f49f0 2024-05-17T22:49:01.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.138+0000 7f03153fe700 1 --2- 172.21.15.12:0/4200191050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574d69f49f0 0x5574d6b80750 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f030c0e0b20 tx=0x7f030c0aac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=531c5af108bcf883 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:49:01.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.138+0000 7f02f8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:49:01.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.138+0000 7f02f8ff9700 1 -- 172.21.15.12:0/4200191050 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030c022940 con 0x5574d69f49f0 2024-05-17T22:49:01.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.138+0000 7f02f8ff9700 1 -- 172.21.15.12:0/4200191050 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f030c0e30c0 con 0x5574d69f49f0 2024-05-17T22:49:01.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.138+0000 7f032603db80 1 -- 172.21.15.12:0/4200191050 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0308007ca0 con 0x5574d69f49f0 2024-05-17T22:49:01.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.138+0000 7f02f8ff9700 1 -- 172.21.15.12:0/4200191050 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f030c0aa240 con 0x5574d69f49f0 2024-05-17T22:49:01.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.138+0000 7f032603db80 10 client.16155 fetch_fsmap learned FSMap version 39 2024-05-17T22:49:01.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.138+0000 7f032603db80 1 -- 172.21.15.12:0/4200191050 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5574d6b8e1c0 con 0x5574d69f49f0 2024-05-17T22:49:01.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.138+0000 7f02f8ff9700 1 -- 172.21.15.12:0/4200191050 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0308007ca0 con 0x5574d69f49f0 2024-05-17T22:49:01.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.138+0000 7f02f8ff9700 1 -- 172.21.15.12:0/4200191050 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f030c0a60a0 con 0x5574d69f49f0 2024-05-17T22:49:01.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.139+0000 7f032603db80 10 client.16155 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:49:01.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.139+0000 7f032603db80 10 client.16155 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:49:01.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.139+0000 7f032603db80 20 client.16155 populate_metadata read hostname 'smithi012' 2024-05-17T22:49:01.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.139+0000 7f032603db80 1 --2- 172.21.15.12:0/4200191050 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5574d6b90900 0x5574d6b92d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:01.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.139+0000 7f032603db80 4 client.16155 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:49:01.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.139+0000 7f032603db80 1 -- 172.21.15.12:0/4200191050 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574d6aa31e0 con 0x5574d6b90900 2024-05-17T22:49:01.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.139+0000 7f0314bfd700 1 --2- 172.21.15.12:0/4200191050 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5574d6b90900 0x5574d6b92d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:01.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.139+0000 7f02f8ff9700 10 client.16155 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:49:01.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.139+0000 7f0314bfd700 1 --2- 172.21.15.12:0/4200191050 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5574d6b90900 0x5574d6b92d90 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1292560f405723af server_cookie=227798fe29dcda13 in_seq=0 out_seq=0 2024-05-17T22:49:01.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.140+0000 7f02f8ff9700 1 -- 172.21.15.12:0/4200191050 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5574d6aa31e0 con 0x5574d6b90900 2024-05-17T22:49:01.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.140+0000 7f02f8ff9700 10 client.16155 handle_command_reply: tid=0 2024-05-17T22:49:01.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.140+0000 7f032603db80 1 client.16155 shutdown 2024-05-17T22:49:01.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.140+0000 7f032603db80 1 -- 172.21.15.12:0/4200191050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574d69f49f0 msgr2=0x5574d6b80750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:01.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.140+0000 7f032603db80 1 --2- 172.21.15.12:0/4200191050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574d69f49f0 0x5574d6b80750 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f030c0e0b20 tx=0x7f030c0aac70 comp rx=0 tx=0).stop 2024-05-17T22:49:01.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.140+0000 7f032603db80 1 -- 172.21.15.12:0/4200191050 shutdown_connections 2024-05-17T22:49:01.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.140+0000 7f032603db80 1 --2- 172.21.15.12:0/4200191050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574d6b7b9f0 0x5574d6b8ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:01.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.140+0000 7f032603db80 1 --2- 172.21.15.12:0/4200191050 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5574d6b90900 0x5574d6b92d90 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:01.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.140+0000 7f032603db80 1 --2- 172.21.15.12:0/4200191050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574d69f49f0 0x5574d6b80750 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:01.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.140+0000 7f032603db80 1 --2- 172.21.15.12:0/4200191050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d6b77840 0x5574d6b80c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:01.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.140+0000 7f02f8ff9700 0 client.16155 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:49:01.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.140+0000 7f032603db80 1 -- 172.21.15.12:0/4200191050 >> 172.21.15.12:0/4200191050 conn(0x5574d6a38f00 msgr2=0x5574d6b67ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:49:01.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.140+0000 7f032603db80 1 -- 172.21.15.12:0/4200191050 shutdown_connections 2024-05-17T22:49:01.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.140+0000 7f032603db80 1 -- 172.21.15.12:0/4200191050 wait complete. 2024-05-17T22:49:01.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.140+0000 7f032603db80 20 client.16155 trim_cache size 0 max 16384 2024-05-17T22:49:01.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.161+0000 7f032603db80 1 Processor -- start 2024-05-17T22:49:01.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.161+0000 7f032603db80 1 -- start start 2024-05-17T22:49:01.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.161+0000 7f032603db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574d69f49f0 0x5574d6b725e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:01.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.161+0000 7f032603db80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d6b77840 0x5574d6b67eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:01.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.161+0000 7f032603db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574d6b7b9f0 0x5574d6b7aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:01.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.161+0000 7f032603db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f03101b02f0 con 0x5574d69f49f0 2024-05-17T22:49:01.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.161+0000 7f032603db80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f03101adb80 con 0x5574d6b77840 2024-05-17T22:49:01.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.161+0000 7f032603db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f03101ab9d0 con 0x5574d6b7b9f0 2024-05-17T22:49:01.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.161+0000 7f03153fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574d69f49f0 0x5574d6b725e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:01.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.161+0000 7f03153fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574d69f49f0 0x5574d6b725e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47782/0 (socket says 172.21.15.12:47782) 2024-05-17T22:49:01.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.161+0000 7f03153fe700 1 -- 172.21.15.12:0/2254171544 learned_addr learned my addr 172.21.15.12:0/2254171544 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:49:01.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.161+0000 7f0314bfd700 1 --2- 172.21.15.12:0/2254171544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d6b77840 0x5574d6b67eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:01.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.161+0000 7f0315bff700 1 --2- 172.21.15.12:0/2254171544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574d6b7b9f0 0x5574d6b7aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:01.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.161+0000 7f03153fe700 1 -- 172.21.15.12:0/2254171544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574d6b7b9f0 msgr2=0x5574d6b7aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:49:01.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.161+0000 7f03153fe700 1 --2- 172.21.15.12:0/2254171544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574d6b7b9f0 0x5574d6b7aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:01.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.161+0000 7f03153fe700 1 -- 172.21.15.12:0/2254171544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d6b77840 msgr2=0x5574d6b67eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:49:01.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.161+0000 7f03153fe700 1 --2- 172.21.15.12:0/2254171544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d6b77840 0x5574d6b67eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:01.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.161+0000 7f03153fe700 1 -- 172.21.15.12:0/2254171544 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574d6b6f0c0 con 0x5574d69f49f0 2024-05-17T22:49:01.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.161+0000 7f0315bff700 1 --2- 172.21.15.12:0/2254171544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574d6b7b9f0 0x5574d6b7aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:49:01.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.162+0000 7f03153fe700 1 --2- 172.21.15.12:0/2254171544 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574d69f49f0 0x5574d6b725e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f030c005f50 tx=0x7f030c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b7ea31ac01461d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:49:01.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.162+0000 7f02faffd700 1 -- 172.21.15.12:0/2254171544 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030c022940 con 0x5574d69f49f0 2024-05-17T22:49:01.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.162+0000 7f02faffd700 1 -- 172.21.15.12:0/2254171544 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f030c0a4e50 con 0x5574d69f49f0 2024-05-17T22:49:01.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.162+0000 7f02faffd700 1 -- 172.21.15.12:0/2254171544 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030c0e30c0 con 0x5574d69f49f0 2024-05-17T22:49:01.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.162+0000 7f032603db80 1 -- 172.21.15.12:0/2254171544 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574d69f49f0 msgr2=0x5574d6b725e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:01.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.162+0000 7f032603db80 1 --2- 172.21.15.12:0/2254171544 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574d69f49f0 0x5574d6b725e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f030c005f50 tx=0x7f030c009310 comp rx=0 tx=0).stop 2024-05-17T22:49:01.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.162+0000 7f032603db80 1 -- 172.21.15.12:0/2254171544 shutdown_connections 2024-05-17T22:49:01.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.162+0000 7f032603db80 1 --2- 172.21.15.12:0/2254171544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574d6b7b9f0 0x5574d6b7aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:01.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.162+0000 7f032603db80 1 --2- 172.21.15.12:0/2254171544 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574d69f49f0 0x5574d6b725e0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:01.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.162+0000 7f032603db80 1 --2- 172.21.15.12:0/2254171544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d6b77840 0x5574d6b67eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:01.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.162+0000 7f032603db80 1 -- 172.21.15.12:0/2254171544 >> 172.21.15.12:0/2254171544 conn(0x5574d6a38f00 msgr2=0x5574d6b7a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:49:01.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.163+0000 7f032603db80 1 -- 172.21.15.12:0/2254171544 shutdown_connections 2024-05-17T22:49:01.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.163+0000 7f032603db80 1 -- 172.21.15.12:0/2254171544 wait complete. 2024-05-17T22:49:01.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.163+0000 7f032603db80 1 -- start start 2024-05-17T22:49:01.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.163+0000 7f032603db80 1 Processor -- start 2024-05-17T22:49:01.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.163+0000 7f032603db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574d69f49f0 0x5574d6b846d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:01.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.163+0000 7f032603db80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d6b77840 0x5574d6b84c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:01.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.163+0000 7f032603db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574d6b7b9f0 0x5574d6b85150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:01.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.163+0000 7f032603db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f03101ac040 con 0x5574d69f49f0 2024-05-17T22:49:01.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.163+0000 7f032603db80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f03101ad4b0 con 0x5574d6b77840 2024-05-17T22:49:01.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.163+0000 7f032603db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f03101ac710 con 0x5574d6b7b9f0 2024-05-17T22:49:01.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.163+0000 7f032603db80 10 client.0 _ll_register_callbacks cb 0x5574d6a2bf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:49:01.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.163+0000 7f03153fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574d69f49f0 0x5574d6b846d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:01.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.163+0000 7f03153fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574d69f49f0 0x5574d6b846d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47792/0 (socket says 172.21.15.12:47792) 2024-05-17T22:49:01.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.163+0000 7f03153fe700 1 -- 172.21.15.12:0/3264560745 learned_addr learned my addr 172.21.15.12:0/3264560745 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:49:01.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.163+0000 7f0314bfd700 1 --2- 172.21.15.12:0/3264560745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d6b77840 0x5574d6b84c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:01.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.164+0000 7f0315bff700 1 --2- 172.21.15.12:0/3264560745 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574d6b7b9f0 0x5574d6b85150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:01.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.164+0000 7f03153fe700 1 -- 172.21.15.12:0/3264560745 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574d6b7b9f0 msgr2=0x5574d6b85150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:49:01.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.164+0000 7f03153fe700 1 --2- 172.21.15.12:0/3264560745 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574d6b7b9f0 0x5574d6b85150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:01.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.164+0000 7f03153fe700 1 -- 172.21.15.12:0/3264560745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d6b77840 msgr2=0x5574d6b84c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:49:01.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.164+0000 7f03153fe700 1 --2- 172.21.15.12:0/3264560745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d6b77840 0x5574d6b84c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:01.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.164+0000 7f03153fe700 1 -- 172.21.15.12:0/3264560745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574d6bfa2a0 con 0x5574d69f49f0 2024-05-17T22:49:01.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.164+0000 7f0314bfd700 1 --2- 172.21.15.12:0/3264560745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d6b77840 0x5574d6b84c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:49:01.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.164+0000 7f03153fe700 1 --2- 172.21.15.12:0/3264560745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574d69f49f0 0x5574d6b846d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f030c0b3470 tx=0x7f030c0a6550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c9a70344c11b8db server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:49:01.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.164+0000 7f02f8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:49:01.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.164+0000 7f02f8ff9700 1 -- 172.21.15.12:0/3264560745 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030c0b3500 con 0x5574d69f49f0 2024-05-17T22:49:01.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.164+0000 7f02f8ff9700 1 -- 172.21.15.12:0/3264560745 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f030c0a9ba0 con 0x5574d69f49f0 2024-05-17T22:49:01.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.164+0000 7f032603db80 1 -- 172.21.15.12:0/3264560745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f030c0a4e50 con 0x5574d69f49f0 2024-05-17T22:49:01.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.164+0000 7f02f8ff9700 1 -- 172.21.15.12:0/3264560745 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f030c0aeae0 con 0x5574d69f49f0 2024-05-17T22:49:01.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.164+0000 7f032603db80 10 client.16161 fetch_fsmap learned FSMap version 39 2024-05-17T22:49:01.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.164+0000 7f032603db80 1 -- 172.21.15.12:0/3264560745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5574d6bfa2a0 con 0x5574d69f49f0 2024-05-17T22:49:01.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.164+0000 7f02f8ff9700 1 -- 172.21.15.12:0/3264560745 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f030c0a4e50 con 0x5574d69f49f0 2024-05-17T22:49:01.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.165+0000 7f02f8ff9700 1 -- 172.21.15.12:0/3264560745 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f030c0aba60 con 0x5574d69f49f0 2024-05-17T22:49:01.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.165+0000 7f032603db80 10 client.16161 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:49:01.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.165+0000 7f032603db80 10 client.16161 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:49:01.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.165+0000 7f032603db80 20 client.16161 populate_metadata read hostname 'smithi012' 2024-05-17T22:49:01.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.165+0000 7f032603db80 1 --2- 172.21.15.12:0/3264560745 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5574d6bfb780 0x5574d6bfdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:01.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.165+0000 7f032603db80 4 client.16161 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:49:01.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.165+0000 7f032603db80 1 -- 172.21.15.12:0/3264560745 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5574d6a9c9c0 con 0x5574d6bfb780 2024-05-17T22:49:01.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.165+0000 7f0314bfd700 1 --2- 172.21.15.12:0/3264560745 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5574d6bfb780 0x5574d6bfdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:01.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.165+0000 7f02f8ff9700 10 client.16161 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:49:01.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.165+0000 7f0314bfd700 1 --2- 172.21.15.12:0/3264560745 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5574d6bfb780 0x5574d6bfdba0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=267be5bb86600579 server_cookie=aa802c68dfc94d6f in_seq=0 out_seq=0 2024-05-17T22:49:01.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.166+0000 7f02f8ff9700 1 -- 172.21.15.12:0/3264560745 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5574d6a9c9c0 con 0x5574d6bfb780 2024-05-17T22:49:01.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.166+0000 7f02f8ff9700 10 client.16161 handle_command_reply: tid=0 2024-05-17T22:49:01.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.166+0000 7f032603db80 1 client.16161 shutdown 2024-05-17T22:49:01.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.166+0000 7f032603db80 1 -- 172.21.15.12:0/3264560745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574d69f49f0 msgr2=0x5574d6b846d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:01.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.166+0000 7f032603db80 1 --2- 172.21.15.12:0/3264560745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574d69f49f0 0x5574d6b846d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f030c0b3470 tx=0x7f030c0a6550 comp rx=0 tx=0).stop 2024-05-17T22:49:01.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.166+0000 7f032603db80 1 -- 172.21.15.12:0/3264560745 shutdown_connections 2024-05-17T22:49:01.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.166+0000 7f032603db80 1 --2- 172.21.15.12:0/3264560745 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574d6b7b9f0 0x5574d6b85150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:01.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.166+0000 7f032603db80 1 --2- 172.21.15.12:0/3264560745 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5574d6bfb780 0x5574d6bfdba0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:01.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.166+0000 7f032603db80 1 --2- 172.21.15.12:0/3264560745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574d69f49f0 0x5574d6b846d0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:01.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.166+0000 7f032603db80 1 --2- 172.21.15.12:0/3264560745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d6b77840 0x5574d6b84c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:01.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.166+0000 7f02f8ff9700 0 client.16161 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:49:01.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.166+0000 7f032603db80 1 -- 172.21.15.12:0/3264560745 >> 172.21.15.12:0/3264560745 conn(0x5574d6a38f00 msgr2=0x5574d6b723b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:49:01.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.166+0000 7f032603db80 1 -- 172.21.15.12:0/3264560745 shutdown_connections 2024-05-17T22:49:01.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.166+0000 7f032603db80 1 -- 172.21.15.12:0/3264560745 wait complete. 2024-05-17T22:49:01.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.166+0000 7f032603db80 20 client.16161 trim_cache size 0 max 16384 2024-05-17T22:49:01.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.166+0000 7f0317662700 1 -- 172.21.15.12:0/3832658089 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f02fc0b2880 msgr2=0x7f02fc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:01.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.166+0000 7f0317662700 1 --2- 172.21.15.12:0/3832658089 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f02fc0b2880 0x7f02fc0b4d90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0300005950 tx=0x7f03000058e0 comp rx=0 tx=0).stop 2024-05-17T22:49:01.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.166+0000 7f0317662700 1 -- 172.21.15.12:0/3832658089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03101025b0 msgr2=0x7f031019a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:01.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.166+0000 7f0317662700 1 --2- 172.21.15.12:0/3832658089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03101025b0 0x7f031019a780 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f030c000c00 tx=0x7f030c009580 comp rx=0 tx=0).stop 2024-05-17T22:49:01.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.166+0000 7f0317662700 1 -- 172.21.15.12:0/3832658089 shutdown_connections 2024-05-17T22:49:01.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.166+0000 7f0317662700 1 --2- 172.21.15.12:0/3832658089 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f02fc0b2880 0x7f02fc0b4d90 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:01.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.166+0000 7f0317662700 1 --2- 172.21.15.12:0/3832658089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0310107830 0x7f031019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:01.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.166+0000 7f0317662700 1 --2- 172.21.15.12:0/3832658089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03101025b0 0x7f031019a780 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:01.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.166+0000 7f0317662700 1 --2- 172.21.15.12:0/3832658089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0310106ee0 0x7f031019ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:01.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.167+0000 7f0317662700 1 -- 172.21.15.12:0/3832658089 >> 172.21.15.12:0/3832658089 conn(0x7f03100fe1a0 msgr2=0x7f0310106710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:49:01.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.167+0000 7f0317662700 1 -- 172.21.15.12:0/3832658089 shutdown_connections 2024-05-17T22:49:01.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:00.167+0000 7f0317662700 1 -- 172.21.15.12:0/3832658089 wait complete. 2024-05-17T22:49:01.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 580.0 s, 501.0 tps, lat 201.216 ms stddev 236.815 2024-05-17T22:49:02.071 INFO:tasks.cephfs.filesystem:scrub status for tag:4fdbc97d-8f04-4808-b4c5-78678aca675a - {'path': '/', 'tag': '4fdbc97d-8f04-4808-b4c5-78678aca675a', 'options': 'recursive,force'} 2024-05-17T22:49:02.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:49:02 smithi096 ceph-mon[36065]: pgmap v732: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.2 MiB/s wr, 478 op/s 2024-05-17T22:49:02.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:49:02 smithi012 ceph-mon[31632]: pgmap v732: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.2 MiB/s wr, 478 op/s 2024-05-17T22:49:02.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:49:02 smithi151 ceph-mon[36101]: pgmap v732: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.2 MiB/s wr, 478 op/s 2024-05-17T22:49:04.292 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:49:04 smithi151 ceph-mon[36101]: pgmap v733: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 3.9 MiB/s wr, 455 op/s 2024-05-17T22:49:04.292 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:49:04 smithi151 ceph-mon[36101]: scrub summary: idle 2024-05-17T22:49:04.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:49:04 smithi096 ceph-mon[36065]: pgmap v733: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 3.9 MiB/s wr, 455 op/s 2024-05-17T22:49:04.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:49:04 smithi096 ceph-mon[36065]: scrub summary: idle 2024-05-17T22:49:04.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:49:04 smithi012 ceph-mon[31632]: pgmap v733: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 3.9 MiB/s wr, 455 op/s 2024-05-17T22:49:04.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:49:04 smithi012 ceph-mon[31632]: scrub summary: idle 2024-05-17T22:49:05.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:49:05 smithi096 ceph-mon[36065]: scrub summary: idle 2024-05-17T22:49:05.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:49:05 smithi096 ceph-mon[36065]: scrub summary: idle 2024-05-17T22:49:05.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:49:05 smithi012 ceph-mon[31632]: scrub summary: idle 2024-05-17T22:49:05.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:49:05 smithi012 ceph-mon[31632]: scrub summary: idle 2024-05-17T22:49:05.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:49:05 smithi151 ceph-mon[36101]: scrub summary: idle 2024-05-17T22:49:05.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:49:05 smithi151 ceph-mon[36101]: scrub summary: idle 2024-05-17T22:49:06.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:49:06 smithi096 ceph-mon[36065]: pgmap v734: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.2 MiB/s wr, 599 op/s 2024-05-17T22:49:06.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:49:06 smithi012 ceph-mon[31632]: pgmap v734: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.2 MiB/s wr, 599 op/s 2024-05-17T22:49:06.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 585.0 s, 508.2 tps, lat 194.970 ms stddev 233.946 2024-05-17T22:49:06.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:49:06 smithi151 ceph-mon[36101]: pgmap v734: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.2 MiB/s wr, 599 op/s 2024-05-17T22:49:08.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:49:08 smithi096 ceph-mon[36065]: pgmap v735: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.1 MiB/s wr, 472 op/s 2024-05-17T22:49:08.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:49:08 smithi012 ceph-mon[31632]: pgmap v735: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.1 MiB/s wr, 472 op/s 2024-05-17T22:49:08.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:49:08 smithi151 ceph-mon[36101]: pgmap v735: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.1 MiB/s wr, 472 op/s 2024-05-17T22:49:10.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:49:10 smithi012 ceph-mon[31632]: pgmap v736: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.7 MiB/s wr, 553 op/s 2024-05-17T22:49:10.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:49:10 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:49:10.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:49:10 smithi151 ceph-mon[36101]: pgmap v736: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.7 MiB/s wr, 553 op/s 2024-05-17T22:49:10.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:49:10 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:49:10.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:49:10 smithi096 ceph-mon[36065]: pgmap v736: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.7 MiB/s wr, 553 op/s 2024-05-17T22:49:10.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:49:10 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:49:11.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 590.0 s, 524.6 tps, lat 187.477 ms stddev 204.911 2024-05-17T22:49:12.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:49:12 smithi012 ceph-mon[31632]: pgmap v737: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.5 MiB/s wr, 520 op/s 2024-05-17T22:49:12.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:49:12 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:49:12.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:49:12 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:49:12.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:49:12 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:49:12.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:49:12 smithi151 ceph-mon[36101]: pgmap v737: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.5 MiB/s wr, 520 op/s 2024-05-17T22:49:12.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:49:12 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:49:12.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:49:12 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:49:12.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:49:12 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:49:12.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:49:12 smithi096 ceph-mon[36065]: pgmap v737: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.5 MiB/s wr, 520 op/s 2024-05-17T22:49:12.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:49:12 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:49:12.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:49:12 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:49:12.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:49:12 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:49:14.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:49:14 smithi012 ceph-mon[31632]: pgmap v738: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.3 MiB/s wr, 498 op/s 2024-05-17T22:49:14.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:49:14 smithi151 ceph-mon[36101]: pgmap v738: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.3 MiB/s wr, 498 op/s 2024-05-17T22:49:14.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:49:14 smithi096 ceph-mon[36065]: pgmap v738: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.3 MiB/s wr, 498 op/s 2024-05-17T22:49:16.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:49:16 smithi012 ceph-mon[31632]: pgmap v739: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.4 MiB/s wr, 611 op/s 2024-05-17T22:49:16.368 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 595.0 s, 550.6 tps, lat 185.312 ms stddev 229.572 2024-05-17T22:49:16.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:49:16 smithi151 ceph-mon[36101]: pgmap v739: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.4 MiB/s wr, 611 op/s 2024-05-17T22:49:16.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:49:16 smithi096 ceph-mon[36065]: pgmap v739: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.4 MiB/s wr, 611 op/s 2024-05-17T22:49:18.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:49:18 smithi012 ceph-mon[31632]: pgmap v740: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.1 MiB/s wr, 462 op/s 2024-05-17T22:49:18.493 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:49:18 smithi151 ceph-mon[36101]: pgmap v740: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.1 MiB/s wr, 462 op/s 2024-05-17T22:49:18.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:49:18 smithi096 ceph-mon[36065]: pgmap v740: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.1 MiB/s wr, 462 op/s 2024-05-17T22:49:20.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:49:20 smithi151 ceph-mon[36101]: pgmap v741: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.8 MiB/s wr, 547 op/s 2024-05-17T22:49:20.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:49:20 smithi096 ceph-mon[36065]: pgmap v741: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.8 MiB/s wr, 547 op/s 2024-05-17T22:49:20.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:49:20 smithi012 ceph-mon[31632]: pgmap v741: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.8 MiB/s wr, 547 op/s 2024-05-17T22:49:21.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 600.0 s, 452.8 tps, lat 217.779 ms stddev 262.668 2024-05-17T22:49:22.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:49:22 smithi151 ceph-mon[36101]: pgmap v742: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.5 MiB/s wr, 509 op/s 2024-05-17T22:49:22.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:49:22 smithi096 ceph-mon[36065]: pgmap v742: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.5 MiB/s wr, 509 op/s 2024-05-17T22:49:22.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:49:22 smithi012 ceph-mon[31632]: pgmap v742: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.5 MiB/s wr, 509 op/s 2024-05-17T22:49:24.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:49:24 smithi151 ceph-mon[36101]: pgmap v743: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.4 MiB/s wr, 488 op/s 2024-05-17T22:49:24.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:49:24 smithi096 ceph-mon[36065]: pgmap v743: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.4 MiB/s wr, 488 op/s 2024-05-17T22:49:24.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:49:24 smithi012 ceph-mon[31632]: pgmap v743: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.4 MiB/s wr, 488 op/s 2024-05-17T22:49:26.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 605.0 s, 487.2 tps, lat 204.602 ms stddev 236.760 2024-05-17T22:49:26.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:49:26 smithi151 ceph-mon[36101]: pgmap v744: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 MiB/s wr, 591 op/s 2024-05-17T22:49:26.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:49:26 smithi096 ceph-mon[36065]: pgmap v744: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 MiB/s wr, 591 op/s 2024-05-17T22:49:26.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:49:26 smithi012 ceph-mon[31632]: pgmap v744: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 MiB/s wr, 591 op/s 2024-05-17T22:49:28.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:49:28 smithi151 ceph-mon[36101]: pgmap v745: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.7 MiB/s wr, 461 op/s 2024-05-17T22:49:28.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:49:28 smithi096 ceph-mon[36065]: pgmap v745: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.7 MiB/s wr, 461 op/s 2024-05-17T22:49:28.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:49:28 smithi012 ceph-mon[31632]: pgmap v745: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.7 MiB/s wr, 461 op/s 2024-05-17T22:49:30.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:49:30 smithi151 ceph-mon[36101]: pgmap v746: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.5 MiB/s wr, 546 op/s 2024-05-17T22:49:30.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:49:30 smithi096 ceph-mon[36065]: pgmap v746: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.5 MiB/s wr, 546 op/s 2024-05-17T22:49:30.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:49:30 smithi012 ceph-mon[31632]: pgmap v746: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.5 MiB/s wr, 546 op/s 2024-05-17T22:49:31.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 610.0 s, 546.4 tps, lat 190.450 ms stddev 214.411 2024-05-17T22:49:32.073 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-17T22:49:32.395 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:49:32 smithi012 ceph-mon[31632]: pgmap v747: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.1 MiB/s wr, 506 op/s 2024-05-17T22:49:32.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:49:32 smithi151 ceph-mon[36101]: pgmap v747: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.1 MiB/s wr, 506 op/s 2024-05-17T22:49:32.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:49:32 smithi096 ceph-mon[36065]: pgmap v747: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.1 MiB/s wr, 506 op/s 2024-05-17T22:49:32.579 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:49:34.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:49:34 smithi012 ceph-mon[31632]: pgmap v748: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.5 MiB/s wr, 496 op/s 2024-05-17T22:49:34.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:49:34 smithi151 ceph-mon[36101]: pgmap v748: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.5 MiB/s wr, 496 op/s 2024-05-17T22:49:34.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:49:34 smithi096 ceph-mon[36065]: pgmap v748: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.5 MiB/s wr, 496 op/s 2024-05-17T22:49:36.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 615.0 s, 482.4 tps, lat 196.658 ms stddev 234.195 2024-05-17T22:49:36.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:49:36 smithi151 ceph-mon[36101]: pgmap v749: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.6 MiB/s wr, 581 op/s 2024-05-17T22:49:36.530 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:49:36 smithi012 ceph-mon[31632]: pgmap v749: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.6 MiB/s wr, 581 op/s 2024-05-17T22:49:36.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:49:36 smithi096 ceph-mon[36065]: pgmap v749: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.6 MiB/s wr, 581 op/s 2024-05-17T22:49:36.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.526+0000 7fba73d76700 1 -- 172.21.15.12:0/766976478 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba6c110330 msgr2=0x7fba6c110740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:36.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.526+0000 7fba73d76700 1 --2- 172.21.15.12:0/766976478 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba6c110330 0x7fba6c110740 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fba68008580 tx=0x7fba6800ac60 comp rx=0 tx=0).stop 2024-05-17T22:49:36.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.526+0000 7fba73d76700 1 -- 172.21.15.12:0/766976478 shutdown_connections 2024-05-17T22:49:36.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.526+0000 7fba73d76700 1 --2- 172.21.15.12:0/766976478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba6c10b180 0x7fba6c07ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:36.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.526+0000 7fba73d76700 1 --2- 172.21.15.12:0/766976478 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba6c110330 0x7fba6c110740 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:36.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.526+0000 7fba73d76700 1 --2- 172.21.15.12:0/766976478 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba6c10a830 0x7fba6c10ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:36.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.526+0000 7fba73d76700 1 -- 172.21.15.12:0/766976478 >> 172.21.15.12:0/766976478 conn(0x7fba6c06efc0 msgr2=0x7fba6c06f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:49:36.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.527+0000 7fba73d76700 1 -- 172.21.15.12:0/766976478 shutdown_connections 2024-05-17T22:49:36.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.527+0000 7fba73d76700 1 -- 172.21.15.12:0/766976478 wait complete. 2024-05-17T22:49:36.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.528+0000 7fba73d76700 1 Processor -- start 2024-05-17T22:49:36.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.532+0000 7fba73d76700 1 -- start start 2024-05-17T22:49:36.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.532+0000 7fba73d76700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba6c10a830 0x7fba6c080950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:36.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.532+0000 7fba73d76700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba6c10b180 0x7fba6c080e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:36.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.532+0000 7fba73d76700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba6c110330 0x7fba6c0813d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:36.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.532+0000 7fba73d76700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fba6c07eff0 con 0x7fba6c110330 2024-05-17T22:49:36.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.532+0000 7fba73d76700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fba6c07ee70 con 0x7fba6c10b180 2024-05-17T22:49:36.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.532+0000 7fba73d76700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fba6c07f170 con 0x7fba6c10a830 2024-05-17T22:49:36.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.533+0000 7fba72313700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba6c110330 0x7fba6c0813d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:36.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.533+0000 7fba72313700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba6c110330 0x7fba6c0813d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44674/0 (socket says 172.21.15.12:44674) 2024-05-17T22:49:36.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.533+0000 7fba72313700 1 -- 172.21.15.12:0/470618023 learned_addr learned my addr 172.21.15.12:0/470618023 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:49:36.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.535+0000 7fba72313700 1 -- 172.21.15.12:0/470618023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba6c10a830 msgr2=0x7fba6c080950 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-17T22:49:36.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.535+0000 7fba72313700 1 --2- 172.21.15.12:0/470618023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba6c10a830 0x7fba6c080950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:36.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.535+0000 7fba72313700 1 -- 172.21.15.12:0/470618023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba6c10b180 msgr2=0x7fba6c080e90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-17T22:49:36.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.535+0000 7fba72313700 1 --2- 172.21.15.12:0/470618023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba6c10b180 0x7fba6c080e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:36.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.535+0000 7fba72313700 1 -- 172.21.15.12:0/470618023 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba6c1b7eb0 con 0x7fba6c110330 2024-05-17T22:49:36.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.542+0000 7fba72313700 1 --2- 172.21.15.12:0/470618023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba6c110330 0x7fba6c0813d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fba5c009840 tx=0x7fba5c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:49:36.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.542+0000 7fba62ffd700 1 -- 172.21.15.12:0/470618023 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5c004830 con 0x7fba6c110330 2024-05-17T22:49:36.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.543+0000 7fba62ffd700 1 -- 172.21.15.12:0/470618023 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba5c005020 con 0x7fba6c110330 2024-05-17T22:49:36.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.543+0000 7fba62ffd700 1 -- 172.21.15.12:0/470618023 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5c013860 con 0x7fba6c110330 2024-05-17T22:49:36.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.544+0000 7fba73d76700 1 -- 172.21.15.12:0/470618023 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba6c1b81a0 con 0x7fba6c110330 2024-05-17T22:49:36.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.544+0000 7fba73d76700 1 -- 172.21.15.12:0/470618023 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba6c1b8690 con 0x7fba6c110330 2024-05-17T22:49:36.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.549+0000 7fba82751b80 1 Processor -- start 2024-05-17T22:49:36.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.552+0000 7fba82751b80 1 -- start start 2024-05-17T22:49:36.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.552+0000 7fba82751b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee16dd09f0 0x55ee16f54560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:36.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.553+0000 7fba62ffd700 1 -- 172.21.15.12:0/470618023 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fba5c01b020 con 0x7fba6c110330 2024-05-17T22:49:36.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.553+0000 7fba82751b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee16f54aa0 0x55ee16f4ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:36.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.553+0000 7fba82751b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee16f4f170 0x55ee16f4f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:36.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.553+0000 7fba82751b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ee16e7d290 con 0x55ee16dd09f0 2024-05-17T22:49:36.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.553+0000 7fba82751b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ee16e6cfd0 con 0x55ee16f54aa0 2024-05-17T22:49:36.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.553+0000 7fba82751b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ee16e70e70 con 0x55ee16f4f170 2024-05-17T22:49:36.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.556+0000 7fba62ffd700 1 --2- 172.21.15.12:0/470618023 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fba580b2a10 0x7fba580b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:36.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.556+0000 7fba62ffd700 1 -- 172.21.15.12:0/470618023 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fba5c0e31c0 con 0x7fba6c110330 2024-05-17T22:49:36.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.556+0000 7fba71311700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee16dd09f0 0x55ee16f54560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:36.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.556+0000 7fba71311700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee16dd09f0 0x55ee16f54560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44676/0 (socket says 172.21.15.12:44676) 2024-05-17T22:49:36.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.556+0000 7fba71311700 1 -- 172.21.15.12:0/3121101092 learned_addr learned my addr 172.21.15.12:0/3121101092 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:49:36.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.558+0000 7fba71311700 1 --2- 172.21.15.12:0/3121101092 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee16f4f170 0x55ee16f4f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:36.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.559+0000 7fba72313700 1 --2- 172.21.15.12:0/470618023 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fba580b2a10 0x7fba580b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:36.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.559+0000 7fba71311700 1 -- 172.21.15.12:0/3121101092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee16f54aa0 msgr2=0x55ee16f4ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:49:36.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.559+0000 7fba71311700 1 --2- 172.21.15.12:0/3121101092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee16f54aa0 0x55ee16f4ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:36.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.559+0000 7fba71311700 1 -- 172.21.15.12:0/3121101092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee16dd09f0 msgr2=0x55ee16f54560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:49:36.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.559+0000 7fba71311700 1 --2- 172.21.15.12:0/3121101092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee16dd09f0 0x55ee16f54560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:36.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.559+0000 7fba71311700 1 -- 172.21.15.12:0/3121101092 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee16e16590 con 0x55ee16f4f170 2024-05-17T22:49:36.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.560+0000 7fba71311700 1 --2- 172.21.15.12:0/3121101092 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee16f4f170 0x55ee16f4f580 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x55ee16f4aae0 tx=0x7fba64004be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4685e418e6a9045 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:49:36.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.560+0000 7fba57fff700 1 -- 172.21.15.12:0/3121101092 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba64012040 con 0x55ee16f4f170 2024-05-17T22:49:36.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.561+0000 7fba57fff700 1 -- 172.21.15.12:0/3121101092 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba6400d030 con 0x55ee16f4f170 2024-05-17T22:49:36.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.562+0000 7fba82751b80 1 -- 172.21.15.12:0/3121101092 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee16f4f170 msgr2=0x55ee16f4f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:36.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.562+0000 7fba82751b80 1 --2- 172.21.15.12:0/3121101092 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee16f4f170 0x55ee16f4f580 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x55ee16f4aae0 tx=0x7fba64004be0 comp rx=0 tx=0).stop 2024-05-17T22:49:36.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.562+0000 7fba72313700 1 --2- 172.21.15.12:0/470618023 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fba580b2a10 0x7fba580b4f20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fba5c004ff0 tx=0x7fba5c016040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:49:36.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.566+0000 7fba82751b80 1 -- 172.21.15.12:0/3121101092 shutdown_connections 2024-05-17T22:49:36.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.566+0000 7fba82751b80 1 --2- 172.21.15.12:0/3121101092 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee16f4f170 0x55ee16f4f580 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:36.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.566+0000 7fba82751b80 1 --2- 172.21.15.12:0/3121101092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee16dd09f0 0x55ee16f54560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:36.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.566+0000 7fba82751b80 1 --2- 172.21.15.12:0/3121101092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee16f54aa0 0x55ee16f4ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:36.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.566+0000 7fba82751b80 1 -- 172.21.15.12:0/3121101092 >> 172.21.15.12:0/3121101092 conn(0x55ee16e14f00 msgr2=0x55ee16f49b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:49:36.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.572+0000 7fba82751b80 1 -- 172.21.15.12:0/3121101092 shutdown_connections 2024-05-17T22:49:36.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.577+0000 7fba82751b80 1 -- 172.21.15.12:0/3121101092 wait complete. 2024-05-17T22:49:36.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.577+0000 7fba82751b80 1 -- start start 2024-05-17T22:49:36.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.577+0000 7fba82751b80 1 Processor -- start 2024-05-17T22:49:36.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.582+0000 7fba82751b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee16dd09f0 0x55ee16f582c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:36.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.582+0000 7fba82751b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee16f4f170 0x55ee16f58800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:36.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.582+0000 7fba82751b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee16f54aa0 0x55ee16f64700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:36.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.582+0000 7fba82751b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ee16d74150 con 0x55ee16f4f170 2024-05-17T22:49:36.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.582+0000 7fba82751b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ee16e0de80 con 0x55ee16dd09f0 2024-05-17T22:49:36.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.582+0000 7fba82751b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ee16f5b3e0 con 0x55ee16f54aa0 2024-05-17T22:49:36.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.582+0000 7fba82751b80 10 client.0 _ll_register_callbacks cb 0x55ee16e07f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:49:36.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.584+0000 7fba71b12700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee16f4f170 0x55ee16f58800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:36.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.584+0000 7fba71b12700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee16f4f170 0x55ee16f58800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44678/0 (socket says 172.21.15.12:44678) 2024-05-17T22:49:36.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.584+0000 7fba71b12700 1 -- 172.21.15.12:0/4154721126 learned_addr learned my addr 172.21.15.12:0/4154721126 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:49:36.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.585+0000 7fba71311700 1 --2- 172.21.15.12:0/4154721126 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee16f54aa0 0x55ee16f64700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:36.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.585+0000 7fba71311700 1 --2- 172.21.15.12:0/4154721126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee16dd09f0 0x55ee16f582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:36.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.585+0000 7fba71311700 1 -- 172.21.15.12:0/4154721126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee16dd09f0 msgr2=0x55ee16f582c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:49:36.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.585+0000 7fba71311700 1 --2- 172.21.15.12:0/4154721126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee16dd09f0 0x55ee16f582c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:36.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.585+0000 7fba71311700 1 -- 172.21.15.12:0/4154721126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee16f4f170 msgr2=0x55ee16f58800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:49:36.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.585+0000 7fba71311700 1 --2- 172.21.15.12:0/4154721126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee16f4f170 0x55ee16f58800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:36.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.585+0000 7fba71311700 1 -- 172.21.15.12:0/4154721126 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee16dbd480 con 0x55ee16f54aa0 2024-05-17T22:49:36.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.590+0000 7fba71311700 1 --2- 172.21.15.12:0/4154721126 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee16f54aa0 0x55ee16f64700 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55ee16f57b50 tx=0x7fba64004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e0b2d4e676aa4a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:49:36.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.591+0000 7fba557fa700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:49:36.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.591+0000 7fba557fa700 1 -- 172.21.15.12:0/4154721126 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba64007730 con 0x55ee16f54aa0 2024-05-17T22:49:36.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.591+0000 7fba557fa700 1 -- 172.21.15.12:0/4154721126 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fba64024040 con 0x55ee16f54aa0 2024-05-17T22:49:36.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.593+0000 7fba82751b80 1 -- 172.21.15.12:0/4154721126 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ee16e7f1e0 con 0x55ee16f54aa0 2024-05-17T22:49:36.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.595+0000 7fba557fa700 1 -- 172.21.15.12:0/4154721126 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fba640078d0 con 0x55ee16f54aa0 2024-05-17T22:49:36.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.595+0000 7fba82751b80 10 client.25822 fetch_fsmap learned FSMap version 39 2024-05-17T22:49:36.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.595+0000 7fba82751b80 1 -- 172.21.15.12:0/4154721126 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ee16f66000 con 0x55ee16f54aa0 2024-05-17T22:49:36.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.597+0000 7fba557fa700 1 -- 172.21.15.12:0/4154721126 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ee16e7f1e0 con 0x55ee16f54aa0 2024-05-17T22:49:36.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.597+0000 7fba557fa700 1 -- 172.21.15.12:0/4154721126 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fba64028990 con 0x55ee16f54aa0 2024-05-17T22:49:36.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.597+0000 7fba82751b80 10 client.25822 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:49:36.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.597+0000 7fba82751b80 10 client.25822 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:49:36.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.597+0000 7fba82751b80 20 client.25822 populate_metadata read hostname 'smithi012' 2024-05-17T22:49:36.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.597+0000 7fba82751b80 1 --2- 172.21.15.12:0/4154721126 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55ee16f68990 0x55ee16f6ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:36.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.597+0000 7fba82751b80 4 client.25822 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:49:36.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.597+0000 7fba82751b80 1 -- 172.21.15.12:0/4154721126 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee16e789c0 con 0x55ee16f68990 2024-05-17T22:49:36.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.598+0000 7fba71b12700 1 --2- 172.21.15.12:0/4154721126 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55ee16f68990 0x55ee16f6ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:36.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.608+0000 7fba71b12700 1 --2- 172.21.15.12:0/4154721126 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55ee16f68990 0x55ee16f6ade0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=322a92e0c9784ecc server_cookie=473eaf322bc68a21 in_seq=0 out_seq=0 2024-05-17T22:49:36.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.608+0000 7fba557fa700 10 client.25822 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:49:36.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.609+0000 7fba557fa700 1 -- 172.21.15.12:0/4154721126 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ee16e789c0 con 0x55ee16f68990 2024-05-17T22:49:36.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.609+0000 7fba557fa700 10 client.25822 handle_command_reply: tid=0 2024-05-17T22:49:36.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.609+0000 7fba82751b80 1 client.25822 shutdown 2024-05-17T22:49:36.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.609+0000 7fba82751b80 1 -- 172.21.15.12:0/4154721126 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee16f54aa0 msgr2=0x55ee16f64700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:36.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.609+0000 7fba82751b80 1 --2- 172.21.15.12:0/4154721126 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee16f54aa0 0x55ee16f64700 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55ee16f57b50 tx=0x7fba64004ad0 comp rx=0 tx=0).stop 2024-05-17T22:49:36.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.612+0000 7fba82751b80 1 -- 172.21.15.12:0/4154721126 shutdown_connections 2024-05-17T22:49:36.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.612+0000 7fba82751b80 1 --2- 172.21.15.12:0/4154721126 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee16f54aa0 0x55ee16f64700 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:36.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.612+0000 7fba82751b80 1 --2- 172.21.15.12:0/4154721126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee16dd09f0 0x55ee16f582c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:36.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.612+0000 7fba82751b80 1 --2- 172.21.15.12:0/4154721126 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55ee16f68990 0x55ee16f6ade0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:36.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.612+0000 7fba82751b80 1 --2- 172.21.15.12:0/4154721126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee16f4f170 0x55ee16f58800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:36.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.612+0000 7fba557fa700 0 client.25822 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:49:36.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.612+0000 7fba82751b80 1 -- 172.21.15.12:0/4154721126 >> 172.21.15.12:0/4154721126 conn(0x55ee16e14f00 msgr2=0x55ee16f43ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:49:36.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.612+0000 7fba82751b80 1 -- 172.21.15.12:0/4154721126 shutdown_connections 2024-05-17T22:49:36.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.612+0000 7fba82751b80 1 -- 172.21.15.12:0/4154721126 wait complete. 2024-05-17T22:49:36.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.613+0000 7fba82751b80 20 client.25822 trim_cache size 0 max 16384 2024-05-17T22:49:36.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.645+0000 7fba82751b80 1 Processor -- start 2024-05-17T22:49:36.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.645+0000 7fba82751b80 1 -- start start 2024-05-17T22:49:36.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.645+0000 7fba82751b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee16dd09f0 0x55ee16f5aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:36.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.645+0000 7fba82751b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee16f4f170 0x55ee16f43eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:36.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.645+0000 7fba82751b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee16f54aa0 0x55ee16f70570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:36.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.645+0000 7fba82751b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fba6c1dae10 con 0x55ee16dd09f0 2024-05-17T22:49:36.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.645+0000 7fba82751b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fba6c1d86a0 con 0x55ee16f54aa0 2024-05-17T22:49:36.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.645+0000 7fba82751b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fba6c1bf380 con 0x55ee16f4f170 2024-05-17T22:49:36.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.645+0000 7fba71311700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee16dd09f0 0x55ee16f5aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:36.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.645+0000 7fba71311700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee16dd09f0 0x55ee16f5aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44692/0 (socket says 172.21.15.12:44692) 2024-05-17T22:49:36.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.645+0000 7fba71311700 1 -- 172.21.15.12:0/2188972936 learned_addr learned my addr 172.21.15.12:0/2188972936 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:49:36.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.645+0000 7fba71b12700 1 --2- 172.21.15.12:0/2188972936 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee16f4f170 0x55ee16f43eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:36.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.645+0000 7fba71311700 1 --2- 172.21.15.12:0/2188972936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee16f54aa0 0x55ee16f70570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:36.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.646+0000 7fba71b12700 1 -- 172.21.15.12:0/2188972936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee16f54aa0 msgr2=0x55ee16f70570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:49:36.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.646+0000 7fba71b12700 1 --2- 172.21.15.12:0/2188972936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee16f54aa0 0x55ee16f70570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:36.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.646+0000 7fba71b12700 1 -- 172.21.15.12:0/2188972936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee16dd09f0 msgr2=0x55ee16f5aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:49:36.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.646+0000 7fba71b12700 1 --2- 172.21.15.12:0/2188972936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee16dd09f0 0x55ee16f5aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:36.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.646+0000 7fba71b12700 1 -- 172.21.15.12:0/2188972936 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee16f4b2e0 con 0x55ee16f4f170 2024-05-17T22:49:36.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.646+0000 7fba71311700 1 --2- 172.21.15.12:0/2188972936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee16dd09f0 0x55ee16f5aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:49:36.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.646+0000 7fba71b12700 1 --2- 172.21.15.12:0/2188972936 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee16f4f170 0x55ee16f43eb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fba68008390 tx=0x7fba68008540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e69f346821da1cc2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:49:36.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.646+0000 7fba577fe700 1 -- 172.21.15.12:0/2188972936 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba68013480 con 0x55ee16f4f170 2024-05-17T22:49:36.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.646+0000 7fba577fe700 1 -- 172.21.15.12:0/2188972936 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba68013e40 con 0x55ee16f4f170 2024-05-17T22:49:36.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.646+0000 7fba577fe700 1 -- 172.21.15.12:0/2188972936 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba68003680 con 0x55ee16f4f170 2024-05-17T22:49:36.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.647+0000 7fba82751b80 1 -- 172.21.15.12:0/2188972936 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee16f4f170 msgr2=0x55ee16f43eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:36.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.647+0000 7fba82751b80 1 --2- 172.21.15.12:0/2188972936 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee16f4f170 0x55ee16f43eb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fba68008390 tx=0x7fba68008540 comp rx=0 tx=0).stop 2024-05-17T22:49:36.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.647+0000 7fba82751b80 1 -- 172.21.15.12:0/2188972936 shutdown_connections 2024-05-17T22:49:36.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.647+0000 7fba82751b80 1 --2- 172.21.15.12:0/2188972936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee16f54aa0 0x55ee16f70570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:36.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.647+0000 7fba82751b80 1 --2- 172.21.15.12:0/2188972936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee16dd09f0 0x55ee16f5aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:36.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.647+0000 7fba82751b80 1 --2- 172.21.15.12:0/2188972936 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee16f4f170 0x55ee16f43eb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:36.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.647+0000 7fba82751b80 1 -- 172.21.15.12:0/2188972936 >> 172.21.15.12:0/2188972936 conn(0x55ee16e14f00 msgr2=0x55ee16f58540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:49:36.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.649+0000 7fba82751b80 1 -- 172.21.15.12:0/2188972936 shutdown_connections 2024-05-17T22:49:36.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.650+0000 7fba82751b80 1 -- 172.21.15.12:0/2188972936 wait complete. 2024-05-17T22:49:36.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.650+0000 7fba82751b80 1 -- start start 2024-05-17T22:49:36.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.651+0000 7fba82751b80 1 Processor -- start 2024-05-17T22:49:36.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.651+0000 7fba82751b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee16dd09f0 0x55ee16f5d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:36.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.651+0000 7fba82751b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee16f4f170 0x55ee16f5d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:36.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.651+0000 7fba82751b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee16f54aa0 0x55ee16f5dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:36.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.651+0000 7fba71311700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee16dd09f0 0x55ee16f5d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:36.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.651+0000 7fba71311700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee16dd09f0 0x55ee16f5d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44694/0 (socket says 172.21.15.12:44694) 2024-05-17T22:49:36.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.652+0000 7fba71b12700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee16f4f170 0x55ee16f5d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:36.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.652+0000 7fba71b12700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee16f4f170 0x55ee16f5d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:49164/0 (socket says 172.21.15.12:49164) 2024-05-17T22:49:36.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.652+0000 7fba71311700 1 -- 172.21.15.12:0/2476809811 learned_addr learned my addr 172.21.15.12:0/2476809811 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:49:36.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.652+0000 7fba82751b80 1 -- 172.21.15.12:0/2476809811 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fba6c1bf9f0 con 0x55ee16dd09f0 2024-05-17T22:49:36.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.652+0000 7fba82751b80 1 -- 172.21.15.12:0/2476809811 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fba6c1d7fd0 con 0x55ee16f4f170 2024-05-17T22:49:36.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.652+0000 7fba82751b80 1 -- 172.21.15.12:0/2476809811 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fba6c1c00c0 con 0x55ee16f54aa0 2024-05-17T22:49:36.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.653+0000 7fba82751b80 10 client.0 _ll_register_callbacks cb 0x55ee16e07f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:49:36.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.653+0000 7fba71311700 1 --2- 172.21.15.12:0/2476809811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee16f54aa0 0x55ee16f5dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:36.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.653+0000 7fba71b12700 1 -- 172.21.15.12:0/2476809811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee16f54aa0 msgr2=0x55ee16f5dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:49:36.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.653+0000 7fba71b12700 1 --2- 172.21.15.12:0/2476809811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee16f54aa0 0x55ee16f5dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:36.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.653+0000 7fba71b12700 1 -- 172.21.15.12:0/2476809811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee16dd09f0 msgr2=0x55ee16f5d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:49:36.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.653+0000 7fba71b12700 1 --2- 172.21.15.12:0/2476809811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee16dd09f0 0x55ee16f5d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:36.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.653+0000 7fba71b12700 1 -- 172.21.15.12:0/2476809811 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee16fd11a0 con 0x55ee16f4f170 2024-05-17T22:49:36.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.653+0000 7fba71311700 1 --2- 172.21.15.12:0/2476809811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee16dd09f0 0x55ee16f5d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:49:36.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.653+0000 7fba71b12700 1 --2- 172.21.15.12:0/2476809811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee16f4f170 0x55ee16f5d6f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fba68000c00 tx=0x7fba68005280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=365100332684e2c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:49:36.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.653+0000 7fba557fa700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:49:36.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.653+0000 7fba82751b80 1 -- 172.21.15.12:0/2476809811 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ee16d49360 con 0x55ee16f4f170 2024-05-17T22:49:36.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.653+0000 7fba557fa700 1 -- 172.21.15.12:0/2476809811 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba68013990 con 0x55ee16f4f170 2024-05-17T22:49:36.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.653+0000 7fba557fa700 1 -- 172.21.15.12:0/2476809811 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fba68013b30 con 0x55ee16f4f170 2024-05-17T22:49:36.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.654+0000 7fba557fa700 1 -- 172.21.15.12:0/2476809811 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fba68013480 con 0x55ee16f4f170 2024-05-17T22:49:36.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.654+0000 7fba82751b80 10 client.25889 fetch_fsmap learned FSMap version 39 2024-05-17T22:49:36.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.654+0000 7fba82751b80 1 -- 172.21.15.12:0/2476809811 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({fsmap=39}) -- 0x55ee16fd2300 con 0x55ee16f4f170 2024-05-17T22:49:36.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.654+0000 7fba557fa700 1 -- 172.21.15.12:0/2476809811 <== mon.2 v2:172.21.15.96:3300/0 4 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fba68013620 con 0x55ee16f4f170 2024-05-17T22:49:36.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.654+0000 7fba82751b80 10 client.25889 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:49:36.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.654+0000 7fba82751b80 10 client.25889 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:49:36.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.654+0000 7fba82751b80 20 client.25889 populate_metadata read hostname 'smithi012' 2024-05-17T22:49:36.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.654+0000 7fba82751b80 1 --2- 172.21.15.12:0/2476809811 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55ee16fd2840 0x55ee16fd4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:36.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.654+0000 7fba82751b80 4 client.25889 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:49:36.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.654+0000 7fba82751b80 1 -- 172.21.15.12:0/2476809811 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ee16e41180 con 0x55ee16fd2840 2024-05-17T22:49:36.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.654+0000 7fba71311700 1 --2- 172.21.15.12:0/2476809811 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55ee16fd2840 0x55ee16fd4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:36.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.655+0000 7fba71311700 1 --2- 172.21.15.12:0/2476809811 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55ee16fd2840 0x55ee16fd4c80 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48d963facd01673b server_cookie=35dab24d7a75ed71 in_seq=0 out_seq=0 2024-05-17T22:49:36.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.655+0000 7fba557fa700 10 client.25889 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:49:36.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.655+0000 7fba557fa700 1 -- 172.21.15.12:0/2476809811 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ee16e41180 con 0x55ee16fd2840 2024-05-17T22:49:36.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.655+0000 7fba557fa700 10 client.25889 handle_command_reply: tid=0 2024-05-17T22:49:36.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.655+0000 7fba82751b80 1 client.25889 shutdown 2024-05-17T22:49:36.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.655+0000 7fba82751b80 1 -- 172.21.15.12:0/2476809811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee16f4f170 msgr2=0x55ee16f5d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:36.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.655+0000 7fba82751b80 1 --2- 172.21.15.12:0/2476809811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee16f4f170 0x55ee16f5d6f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fba68000c00 tx=0x7fba68005280 comp rx=0 tx=0).stop 2024-05-17T22:49:36.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.655+0000 7fba82751b80 1 -- 172.21.15.12:0/2476809811 shutdown_connections 2024-05-17T22:49:36.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.655+0000 7fba82751b80 1 --2- 172.21.15.12:0/2476809811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee16f54aa0 0x55ee16f5dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:36.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.655+0000 7fba82751b80 1 --2- 172.21.15.12:0/2476809811 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55ee16fd2840 0x55ee16fd4c80 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:36.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.655+0000 7fba82751b80 1 --2- 172.21.15.12:0/2476809811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ee16dd09f0 0x55ee16f5d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:36.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.655+0000 7fba557fa700 0 client.25889 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:49:36.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.655+0000 7fba82751b80 1 --2- 172.21.15.12:0/2476809811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee16f4f170 0x55ee16f5d6f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:36.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.655+0000 7fba82751b80 1 -- 172.21.15.12:0/2476809811 >> 172.21.15.12:0/2476809811 conn(0x55ee16e14f00 msgr2=0x55ee16f4dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:49:36.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.656+0000 7fba82751b80 1 -- 172.21.15.12:0/2476809811 shutdown_connections 2024-05-17T22:49:36.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.656+0000 7fba82751b80 1 -- 172.21.15.12:0/2476809811 wait complete. 2024-05-17T22:49:36.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.656+0000 7fba82751b80 20 client.25889 trim_cache size 0 max 16384 2024-05-17T22:49:36.661 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:49:36.661 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-17T22:49:36.662 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-17T22:49:36.662 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:49:36.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.656+0000 7fba73d76700 1 -- 172.21.15.12:0/470618023 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fba580b2a10 msgr2=0x7fba580b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:36.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.656+0000 7fba73d76700 1 --2- 172.21.15.12:0/470618023 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fba580b2a10 0x7fba580b4f20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fba5c004ff0 tx=0x7fba5c016040 comp rx=0 tx=0).stop 2024-05-17T22:49:36.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.656+0000 7fba73d76700 1 -- 172.21.15.12:0/470618023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba6c110330 msgr2=0x7fba6c0813d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:36.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.656+0000 7fba73d76700 1 --2- 172.21.15.12:0/470618023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba6c110330 0x7fba6c0813d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fba5c009840 tx=0x7fba5c009c00 comp rx=0 tx=0).stop 2024-05-17T22:49:36.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.657+0000 7fba73d76700 1 -- 172.21.15.12:0/470618023 shutdown_connections 2024-05-17T22:49:36.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.657+0000 7fba73d76700 1 --2- 172.21.15.12:0/470618023 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fba580b2a10 0x7fba580b4f20 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:36.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.657+0000 7fba73d76700 1 --2- 172.21.15.12:0/470618023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba6c110330 0x7fba6c0813d0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:36.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.657+0000 7fba73d76700 1 --2- 172.21.15.12:0/470618023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba6c10a830 0x7fba6c080950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:36.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.657+0000 7fba73d76700 1 --2- 172.21.15.12:0/470618023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba6c10b180 0x7fba6c080e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:36.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.657+0000 7fba73d76700 1 -- 172.21.15.12:0/470618023 >> 172.21.15.12:0/470618023 conn(0x7fba6c06efc0 msgr2=0x7fba6c073c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:49:36.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.657+0000 7fba73d76700 1 -- 172.21.15.12:0/470618023 shutdown_connections 2024-05-17T22:49:36.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:36.657+0000 7fba73d76700 1 -- 172.21.15.12:0/470618023 wait complete. 2024-05-17T22:49:37.473 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-17T22:49:37.473 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T22:49:37.977 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:49:38.364 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:49:38 smithi012 ceph-mon[31632]: pgmap v750: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.6 MiB/s wr, 479 op/s 2024-05-17T22:49:38.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:49:38 smithi151 ceph-mon[36101]: pgmap v750: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.6 MiB/s wr, 479 op/s 2024-05-17T22:49:38.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:49:38 smithi096 ceph-mon[36065]: pgmap v750: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.6 MiB/s wr, 479 op/s 2024-05-17T22:49:40.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.104+0000 7f0a4b61d700 1 -- 172.21.15.12:0/707307304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a4410a540 msgr2=0x7f0a4410a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:40.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.104+0000 7f0a4b61d700 1 --2- 172.21.15.12:0/707307304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a4410a540 0x7f0a4410a950 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0a40009990 tx=0x7f0a40009ca0 comp rx=0 tx=0).stop 2024-05-17T22:49:40.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.104+0000 7f0a4b61d700 1 -- 172.21.15.12:0/707307304 shutdown_connections 2024-05-17T22:49:40.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.104+0000 7f0a4b61d700 1 --2- 172.21.15.12:0/707307304 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a4410f7c0 0x7f0a44117120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:40.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.104+0000 7f0a4b61d700 1 --2- 172.21.15.12:0/707307304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a4410a540 0x7f0a4410a950 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:40.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.104+0000 7f0a4b61d700 1 --2- 172.21.15.12:0/707307304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a4410ee70 0x7f0a4410f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:40.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.104+0000 7f0a4b61d700 1 -- 172.21.15.12:0/707307304 >> 172.21.15.12:0/707307304 conn(0x7f0a44079c00 msgr2=0x7f0a44108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:49:40.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.104+0000 7f0a4b61d700 1 -- 172.21.15.12:0/707307304 shutdown_connections 2024-05-17T22:49:40.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.104+0000 7f0a4b61d700 1 -- 172.21.15.12:0/707307304 wait complete. 2024-05-17T22:49:40.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.105+0000 7f0a4b61d700 1 Processor -- start 2024-05-17T22:49:40.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.105+0000 7f0a4b61d700 1 -- start start 2024-05-17T22:49:40.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.106+0000 7f0a4b61d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a4410a540 0x7f0a441a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:40.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.106+0000 7f0a4b61d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a4410ee70 0x7f0a441a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:40.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.106+0000 7f0a4b61d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a4410f7c0 0x7f0a441a5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:40.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.106+0000 7f0a4b61d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4411a470 con 0x7f0a4410f7c0 2024-05-17T22:49:40.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.106+0000 7f0a4b61d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4411a2f0 con 0x7f0a4410a540 2024-05-17T22:49:40.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.106+0000 7f0a4b61d700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4411a5f0 con 0x7f0a4410ee70 2024-05-17T22:49:40.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.106+0000 7f0a48bb8700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a4410ee70 0x7f0a441a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:40.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.106+0000 7f0a49bba700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a4410f7c0 0x7f0a441a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:40.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.106+0000 7f0a48bb8700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a4410ee70 0x7f0a441a13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:47502/0 (socket says 172.21.15.12:47502) 2024-05-17T22:49:40.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.106+0000 7f0a48bb8700 1 -- 172.21.15.12:0/135300807 learned_addr learned my addr 172.21.15.12:0/135300807 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:49:40.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.106+0000 7f0a493b9700 1 --2- 172.21.15.12:0/135300807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a4410a540 0x7f0a441a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:40.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.107+0000 7f0a48bb8700 1 -- 172.21.15.12:0/135300807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a4410a540 msgr2=0x7f0a441a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:40.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.107+0000 7f0a48bb8700 1 --2- 172.21.15.12:0/135300807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a4410a540 0x7f0a441a0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:40.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.107+0000 7f0a48bb8700 1 -- 172.21.15.12:0/135300807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a4410f7c0 msgr2=0x7f0a441a5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:40.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.107+0000 7f0a48bb8700 1 --2- 172.21.15.12:0/135300807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a4410f7c0 0x7f0a441a5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:40.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.107+0000 7f0a48bb8700 1 -- 172.21.15.12:0/135300807 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a441a5ce0 con 0x7f0a4410ee70 2024-05-17T22:49:40.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.107+0000 7f0a49bba700 1 --2- 172.21.15.12:0/135300807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a4410f7c0 0x7f0a441a5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:49:40.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.108+0000 7f0a48bb8700 1 --2- 172.21.15.12:0/135300807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a4410ee70 0x7f0a441a13b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0a3400b500 tx=0x7f0a3400b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:49:40.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.108+0000 7f0a3a7fc700 1 -- 172.21.15.12:0/135300807 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a34015020 con 0x7f0a4410ee70 2024-05-17T22:49:40.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.108+0000 7f0a3a7fc700 1 -- 172.21.15.12:0/135300807 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a34011ce0 con 0x7f0a4410ee70 2024-05-17T22:49:40.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.108+0000 7f0a4b61d700 1 -- 172.21.15.12:0/135300807 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a441a5fd0 con 0x7f0a4410ee70 2024-05-17T22:49:40.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.109+0000 7f0a4b61d700 1 -- 172.21.15.12:0/135300807 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a441ad8f0 con 0x7f0a4410ee70 2024-05-17T22:49:40.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.109+0000 7f0a3a7fc700 1 -- 172.21.15.12:0/135300807 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a34010d20 con 0x7f0a4410ee70 2024-05-17T22:49:40.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.112+0000 7f0a4b61d700 1 -- 172.21.15.12:0/135300807 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a28005330 con 0x7f0a4410ee70 2024-05-17T22:49:40.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.114+0000 7f0a3a7fc700 1 -- 172.21.15.12:0/135300807 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0a34018020 con 0x7f0a4410ee70 2024-05-17T22:49:40.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.115+0000 7f0a3a7fc700 1 --2- 172.21.15.12:0/135300807 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f0a300b2a50 0x7f0a300b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:40.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.115+0000 7f0a493b9700 1 --2- 172.21.15.12:0/135300807 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f0a300b2a50 0x7f0a300b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:40.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.115+0000 7f0a3a7fc700 1 -- 172.21.15.12:0/135300807 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f0a340e1b50 con 0x7f0a4410ee70 2024-05-17T22:49:40.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.116+0000 7f0a493b9700 1 --2- 172.21.15.12:0/135300807 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f0a300b2a50 0x7f0a300b4f60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0a40006010 tx=0x7f0a4000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:49:40.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.120+0000 7f0a3a7fc700 1 -- 172.21.15.12:0/135300807 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0a3401f020 con 0x7f0a4410ee70 2024-05-17T22:49:40.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:49:40 smithi012 ceph-mon[31632]: pgmap v751: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 7.9 MiB/s wr, 546 op/s 2024-05-17T22:49:40.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:49:40 smithi151 ceph-mon[36101]: pgmap v751: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 7.9 MiB/s wr, 546 op/s 2024-05-17T22:49:40.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:49:40 smithi096 ceph-mon[36065]: pgmap v751: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 7.9 MiB/s wr, 546 op/s 2024-05-17T22:49:40.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.604+0000 7f0a4b61d700 1 -- 172.21.15.12:0/135300807 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a28005a40 con 0x7f0a4410ee70 2024-05-17T22:49:40.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.609+0000 7f0a3a7fc700 1 -- 172.21.15.12:0/135300807 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11870 (secure 0 0 0) 0x7f0a340a9480 con 0x7f0a4410ee70 2024-05-17T22:49:40.612 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:49:40.613 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1407219319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1407219319},{"type":"v1","addr":"172.21.15.12:6835","nonce":1407219319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/630977886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":630977886},{"type":"v1","addr":"172.21.15.12:6837","nonce":630977886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3731914742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3731914742},{"type":"v1","addr":"172.21.15.12:6839","nonce":3731914742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/3642142501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3642142501},{"type":"v1","addr":"172.21.15.12:6841","nonce":3642142501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3554053059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3554053059},{"type":"v1","addr":"172.21.15.151:6833","nonce":3554053059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/4228325569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":4228325569},{"type":"v1","addr":"172.21.15.96:6833","nonce":4228325569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T22:34:42.642111+0000","modified":"2024-05-17T22:47:38.155618+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24461,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/542323754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":542323754},{"type":"v1","addr":"172.21.15.151:6835","nonce":542323754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1758442232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1758442232},{"type":"v1","addr":"172.21.15.96:6835","nonce":1758442232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3403324038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3403324038},{"type":"v1","addr":"172.21.15.151:6837","nonce":3403324038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.151:6839/2338353849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2338353849},{"type":"v1","addr":"172.21.15.151:6839","nonce":2338353849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.96:6837/4222574338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":4222574338},{"type":"v1","addr":"172.21.15.96:6837","nonce":4222574338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/2986295519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2986295519},{"type":"v1","addr":"172.21.15.96:6839","nonce":2986295519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24457,24461,24482]},"id":1}]} 2024-05-17T22:49:40.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.624+0000 7f0a4b61d700 1 -- 172.21.15.12:0/135300807 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f0a300b2a50 msgr2=0x7f0a300b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:40.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.624+0000 7f0a4b61d700 1 --2- 172.21.15.12:0/135300807 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f0a300b2a50 0x7f0a300b4f60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0a40006010 tx=0x7f0a4000b260 comp rx=0 tx=0).stop 2024-05-17T22:49:40.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.624+0000 7f0a4b61d700 1 -- 172.21.15.12:0/135300807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a4410ee70 msgr2=0x7f0a441a13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:40.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.624+0000 7f0a4b61d700 1 --2- 172.21.15.12:0/135300807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a4410ee70 0x7f0a441a13b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0a3400b500 tx=0x7f0a3400b810 comp rx=0 tx=0).stop 2024-05-17T22:49:40.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.633+0000 7f0a4b61d700 1 -- 172.21.15.12:0/135300807 shutdown_connections 2024-05-17T22:49:40.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.633+0000 7f0a4b61d700 1 --2- 172.21.15.12:0/135300807 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f0a300b2a50 0x7f0a300b4f60 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:40.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.633+0000 7f0a4b61d700 1 --2- 172.21.15.12:0/135300807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a4410f7c0 0x7f0a441a5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:40.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.633+0000 7f0a4b61d700 1 --2- 172.21.15.12:0/135300807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a4410a540 0x7f0a441a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:40.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.633+0000 7f0a4b61d700 1 --2- 172.21.15.12:0/135300807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a4410ee70 0x7f0a441a13b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:40.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.633+0000 7f0a4b61d700 1 -- 172.21.15.12:0/135300807 >> 172.21.15.12:0/135300807 conn(0x7f0a44079c00 msgr2=0x7f0a4407ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:49:40.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.642+0000 7f0a4b61d700 1 -- 172.21.15.12:0/135300807 shutdown_connections 2024-05-17T22:49:40.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:40.645+0000 7f0a4b61d700 1 -- 172.21.15.12:0/135300807 wait complete. 2024-05-17T22:49:40.655 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 39 2024-05-17T22:49:41.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:49:41 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/135300807' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:49:41.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 620.0 s, 525.4 tps, lat 196.334 ms stddev 235.004 2024-05-17T22:49:41.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:49:41 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/135300807' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:49:41.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:49:41 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/135300807' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:49:41.601 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-17T22:49:42.110 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:49:42.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:49:42 smithi012 ceph-mon[31632]: pgmap v752: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.7 MiB/s wr, 485 op/s 2024-05-17T22:49:42.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:49:42 smithi151 ceph-mon[36101]: pgmap v752: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.7 MiB/s wr, 485 op/s 2024-05-17T22:49:42.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:49:42 smithi096 ceph-mon[36065]: pgmap v752: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.7 MiB/s wr, 485 op/s 2024-05-17T22:49:44.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.192+0000 7fb1e2430700 1 -- 172.21.15.12:0/304343841 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1dc107730 msgr2=0x7fb1dc107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:44.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.192+0000 7fb1e2430700 1 --2- 172.21.15.12:0/304343841 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1dc107730 0x7fb1dc107b40 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc009990 tx=0x7fb1cc009ca0 comp rx=0 tx=0).stop 2024-05-17T22:49:44.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.197+0000 7fb1e2430700 1 -- 172.21.15.12:0/304343841 shutdown_connections 2024-05-17T22:49:44.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.197+0000 7fb1e2430700 1 --2- 172.21.15.12:0/304343841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1dc108080 0x7fb1dc10f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:44.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.197+0000 7fb1e2430700 1 --2- 172.21.15.12:0/304343841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1dc102e00 0x7fb1dc103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:44.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.197+0000 7fb1e2430700 1 --2- 172.21.15.12:0/304343841 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1dc107730 0x7fb1dc107b40 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:44.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.197+0000 7fb1e2430700 1 -- 172.21.15.12:0/304343841 >> 172.21.15.12:0/304343841 conn(0x7fb1dc079bd0 msgr2=0x7fb1dc079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:49:44.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.200+0000 7fb1e2430700 1 -- 172.21.15.12:0/304343841 shutdown_connections 2024-05-17T22:49:44.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.200+0000 7fb1e2430700 1 -- 172.21.15.12:0/304343841 wait complete. 2024-05-17T22:49:44.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.201+0000 7fb1e2430700 1 Processor -- start 2024-05-17T22:49:44.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.203+0000 7fb1e2430700 1 -- start start 2024-05-17T22:49:44.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.203+0000 7fb1e2430700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1dc102e00 0x7fb1dc10c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:44.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.203+0000 7fb1e2430700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1dc107730 0x7fb1dc10c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:44.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.203+0000 7fb1e2430700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1dc108080 0x7fb1dc1acac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:44.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.203+0000 7fb1e2430700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb1dc111c60 con 0x7fb1dc107730 2024-05-17T22:49:44.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.203+0000 7fb1e2430700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb1dc111ae0 con 0x7fb1dc102e00 2024-05-17T22:49:44.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.203+0000 7fb1e2430700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb1dc111de0 con 0x7fb1dc108080 2024-05-17T22:49:44.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.205+0000 7fb1dbfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1dc102e00 0x7fb1dc10c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:44.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.206+0000 7fb1db7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1dc107730 0x7fb1dc10c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:44.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.206+0000 7fb1db7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1dc107730 0x7fb1dc10c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55422/0 (socket says 172.21.15.12:55422) 2024-05-17T22:49:44.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.206+0000 7fb1db7fe700 1 -- 172.21.15.12:0/140243798 learned_addr learned my addr 172.21.15.12:0/140243798 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:49:44.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.207+0000 7fb1e09cd700 1 --2- 172.21.15.12:0/140243798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1dc108080 0x7fb1dc1acac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:44.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.207+0000 7fb1db7fe700 1 -- 172.21.15.12:0/140243798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1dc108080 msgr2=0x7fb1dc1acac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:44.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.207+0000 7fb1db7fe700 1 --2- 172.21.15.12:0/140243798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1dc108080 0x7fb1dc1acac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:44.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.207+0000 7fb1db7fe700 1 -- 172.21.15.12:0/140243798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1dc102e00 msgr2=0x7fb1dc10c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:44.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.207+0000 7fb1db7fe700 1 --2- 172.21.15.12:0/140243798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1dc102e00 0x7fb1dc10c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:44.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.207+0000 7fb1db7fe700 1 -- 172.21.15.12:0/140243798 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1dc1ad000 con 0x7fb1dc107730 2024-05-17T22:49:44.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.211+0000 7fb1db7fe700 1 --2- 172.21.15.12:0/140243798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1dc107730 0x7fb1dc10c7b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc0057c0 tx=0x7fb1cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:49:44.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.216+0000 7fb1d97fa700 1 -- 172.21.15.12:0/140243798 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc003a90 con 0x7fb1dc107730 2024-05-17T22:49:44.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.216+0000 7fb1d97fa700 1 -- 172.21.15.12:0/140243798 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1cc0040e0 con 0x7fb1dc107730 2024-05-17T22:49:44.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.216+0000 7fb1d97fa700 1 -- 172.21.15.12:0/140243798 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc022420 con 0x7fb1dc107730 2024-05-17T22:49:44.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.218+0000 7fb1e2430700 1 -- 172.21.15.12:0/140243798 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1dc1ad1d0 con 0x7fb1dc107730 2024-05-17T22:49:44.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.218+0000 7fb1e2430700 1 -- 172.21.15.12:0/140243798 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1dc1ad5d0 con 0x7fb1dc107730 2024-05-17T22:49:44.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.225+0000 7fb1f0e0bb80 1 Processor -- start 2024-05-17T22:49:44.227 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:49:44 smithi012 ceph-mon[31632]: pgmap v753: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.8 MiB/s wr, 459 op/s 2024-05-17T22:49:44.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.225+0000 7fb1f0e0bb80 1 -- start start 2024-05-17T22:49:44.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.226+0000 7fb1f0e0bb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55818ed789f0 0x55818eefbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:44.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.226+0000 7fb1f0e0bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55818eefc400 0x55818eef65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:44.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.226+0000 7fb1e09cd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55818ed789f0 0x55818eefbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:44.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.226+0000 7fb1e09cd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55818ed789f0 0x55818eefbec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:43462/0 (socket says 172.21.15.12:43462) 2024-05-17T22:49:44.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.226+0000 7fb1e09cd700 1 -- 172.21.15.12:0/1640199307 learned_addr learned my addr 172.21.15.12:0/1640199307 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:49:44.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.229+0000 7fb1f0e0bb80 1 --2- 172.21.15.12:0/1640199307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55818eef6b00 0x55818eef6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:44.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.229+0000 7fb1f0e0bb80 1 -- 172.21.15.12:0/1640199307 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55818ee250d0 con 0x55818eef6b00 2024-05-17T22:49:44.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.229+0000 7fb1f0e0bb80 1 -- 172.21.15.12:0/1640199307 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55818ee14c70 con 0x55818ed789f0 2024-05-17T22:49:44.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.229+0000 7fb1f0e0bb80 1 -- 172.21.15.12:0/1640199307 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55818ee18cb0 con 0x55818eefc400 2024-05-17T22:49:44.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.230+0000 7fb1db7fe700 1 --2- 172.21.15.12:0/1640199307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55818eef6b00 0x55818eef6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:44.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.231+0000 7fb1dbfff700 1 --2- 172.21.15.12:0/1640199307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55818eefc400 0x55818eef65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:44.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.244+0000 7fb1e09cd700 1 -- 172.21.15.12:0/1640199307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55818eefc400 msgr2=0x55818eef65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:49:44.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.244+0000 7fb1e09cd700 1 --2- 172.21.15.12:0/1640199307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55818eefc400 0x55818eef65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:44.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.244+0000 7fb1e09cd700 1 -- 172.21.15.12:0/1640199307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55818eef6b00 msgr2=0x55818eef6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:49:44.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.244+0000 7fb1e09cd700 1 --2- 172.21.15.12:0/1640199307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55818eef6b00 0x55818eef6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:44.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.244+0000 7fb1e09cd700 1 -- 172.21.15.12:0/1640199307 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55818edae5c0 con 0x55818ed789f0 2024-05-17T22:49:44.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.244+0000 7fb1d97fa700 1 -- 172.21.15.12:0/140243798 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb1cc003c30 con 0x7fb1dc107730 2024-05-17T22:49:44.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.244+0000 7fb1e09cd700 1 --2- 172.21.15.12:0/1640199307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55818ed789f0 0x55818eefbec0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb1c400ab90 tx=0x7fb1c400aea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0619bdf4efde3c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:49:44.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.244+0000 7fb1db7fe700 1 --2- 172.21.15.12:0/1640199307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55818eef6b00 0x55818eef6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:49:44.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.245+0000 7fb1d27fc700 1 -- 172.21.15.12:0/1640199307 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c400fd80 con 0x55818ed789f0 2024-05-17T22:49:44.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.245+0000 7fb1d27fc700 1 -- 172.21.15.12:0/1640199307 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1c40044d0 con 0x55818ed789f0 2024-05-17T22:49:44.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.245+0000 7fb1d27fc700 1 -- 172.21.15.12:0/1640199307 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c4005310 con 0x55818ed789f0 2024-05-17T22:49:44.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.245+0000 7fb1d97fa700 1 --2- 172.21.15.12:0/140243798 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb1c80b2a90 0x7fb1c80b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:44.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.246+0000 7fb1d97fa700 1 -- 172.21.15.12:0/140243798 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fb1cc0e9080 con 0x7fb1dc107730 2024-05-17T22:49:44.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.246+0000 7fb1f0e0bb80 1 -- 172.21.15.12:0/1640199307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55818ed789f0 msgr2=0x55818eefbec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:44.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.246+0000 7fb1f0e0bb80 1 --2- 172.21.15.12:0/1640199307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55818ed789f0 0x55818eefbec0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb1c400ab90 tx=0x7fb1c400aea0 comp rx=0 tx=0).stop 2024-05-17T22:49:44.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.247+0000 7fb1dbfff700 1 --2- 172.21.15.12:0/140243798 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb1c80b2a90 0x7fb1c80b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:44.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.249+0000 7fb1f0e0bb80 1 -- 172.21.15.12:0/1640199307 shutdown_connections 2024-05-17T22:49:44.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.249+0000 7fb1f0e0bb80 1 --2- 172.21.15.12:0/1640199307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55818eef6b00 0x55818eef6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:44.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.249+0000 7fb1f0e0bb80 1 --2- 172.21.15.12:0/1640199307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55818ed789f0 0x55818eefbec0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:44.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.249+0000 7fb1f0e0bb80 1 --2- 172.21.15.12:0/1640199307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55818eefc400 0x55818eef65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:44.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.249+0000 7fb1f0e0bb80 1 -- 172.21.15.12:0/1640199307 >> 172.21.15.12:0/1640199307 conn(0x55818edd87d0 msgr2=0x55818eef11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:49:44.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.254+0000 7fb1f0e0bb80 1 -- 172.21.15.12:0/1640199307 shutdown_connections 2024-05-17T22:49:44.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.255+0000 7fb1f0e0bb80 1 -- 172.21.15.12:0/1640199307 wait complete. 2024-05-17T22:49:44.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.256+0000 7fb1f0e0bb80 1 -- start start 2024-05-17T22:49:44.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.256+0000 7fb1f0e0bb80 1 Processor -- start 2024-05-17T22:49:44.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.257+0000 7fb1f0e0bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55818ed789f0 0x55818eeffa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:44.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.257+0000 7fb1f0e0bb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55818eef6b00 0x55818eefff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:44.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.257+0000 7fb1f0e0bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55818eefc400 0x55818ef00490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:44.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.257+0000 7fb1f0e0bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55818ed1c150 con 0x55818eefc400 2024-05-17T22:49:44.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.257+0000 7fb1f0e0bb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55818edb6940 con 0x55818eef6b00 2024-05-17T22:49:44.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.257+0000 7fb1f0e0bb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55818ef02dd0 con 0x55818ed789f0 2024-05-17T22:49:44.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.257+0000 7fb1e09cd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55818eefc400 0x55818ef00490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:44.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.257+0000 7fb1db7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55818eef6b00 0x55818eefff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:44.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.257+0000 7fb1db7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55818eef6b00 0x55818eefff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:43474/0 (socket says 172.21.15.12:43474) 2024-05-17T22:49:44.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.257+0000 7fb1db7fe700 1 -- 172.21.15.12:0/539728590 learned_addr learned my addr 172.21.15.12:0/539728590 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:49:44.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.258+0000 7fb1f0e0bb80 10 client.0 _ll_register_callbacks cb 0x55818edabb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:49:44.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.258+0000 7fb1db7fe700 1 -- 172.21.15.12:0/539728590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55818ed789f0 msgr2=0x55818eeffa10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T22:49:44.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.258+0000 7fb1db7fe700 1 --2- 172.21.15.12:0/539728590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55818ed789f0 0x55818eeffa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:44.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.258+0000 7fb1db7fe700 1 -- 172.21.15.12:0/539728590 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55818eefc400 msgr2=0x55818ef00490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:49:44.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.258+0000 7fb1db7fe700 1 --2- 172.21.15.12:0/539728590 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55818eefc400 0x55818ef00490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:44.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.258+0000 7fb1db7fe700 1 -- 172.21.15.12:0/539728590 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55818ed65480 con 0x55818eef6b00 2024-05-17T22:49:44.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.258+0000 7fb1db7fe700 1 --2- 172.21.15.12:0/539728590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55818eef6b00 0x55818eefff50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc011040 tx=0x7fb1cc0b8f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3adf7ddb620273a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:49:44.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.258+0000 7fb1f0e0bb80 1 -- 172.21.15.12:0/539728590 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1c40044d0 con 0x55818eef6b00 2024-05-17T22:49:44.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.265+0000 7fb1bb7fe700 10 client.25901 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:49:44.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.265+0000 7fb1dbfff700 1 --2- 172.21.15.12:0/140243798 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb1c80b2a90 0x7fb1c80b4fa0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x55818eef26b0 tx=0x7fb1d4009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:49:44.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.265+0000 7fb1bb7fe700 1 -- 172.21.15.12:0/539728590 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc0b0080 con 0x55818eef6b00 2024-05-17T22:49:44.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.265+0000 7fb1bb7fe700 1 -- 172.21.15.12:0/539728590 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fb1cc0ac8a0 con 0x55818eef6b00 2024-05-17T22:49:44.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.265+0000 7fb1bb7fe700 1 -- 172.21.15.12:0/539728590 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb1cc0afa20 con 0x55818eef6b00 2024-05-17T22:49:44.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.266+0000 7fb1f0e0bb80 10 client.25901 fetch_fsmap learned FSMap version 39 2024-05-17T22:49:44.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.266+0000 7fb1f0e0bb80 1 -- 172.21.15.12:0/539728590 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({fsmap=39}) -- 0x55818ef0d740 con 0x55818eef6b00 2024-05-17T22:49:44.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.268+0000 7fb1bb7fe700 1 -- 172.21.15.12:0/539728590 <== mon.2 v2:172.21.15.96:3300/0 4 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fb1cc02a020 con 0x55818eef6b00 2024-05-17T22:49:44.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.270+0000 7fb1f0e0bb80 10 client.25901 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:49:44.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.270+0000 7fb1f0e0bb80 10 client.25901 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-17T22:49:44.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.270+0000 7fb1f0e0bb80 20 client.25901 populate_metadata read hostname 'smithi012' 2024-05-17T22:49:44.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.270+0000 7fb1f0e0bb80 1 --2- 172.21.15.12:0/539728590 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55818ef0fa40 0x55818ef11ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:44.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.270+0000 7fb1f0e0bb80 4 client.25901 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:49:44.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.270+0000 7fb1f0e0bb80 1 -- 172.21.15.12:0/539728590 --> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb1c40044d0 con 0x55818ef0fa40 2024-05-17T22:49:44.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.278+0000 7fb1dbfff700 1 --2- 172.21.15.12:0/539728590 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55818ef0fa40 0x55818ef11ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:44.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.280+0000 7fb1dbfff700 1 --2- 172.21.15.12:0/539728590 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55818ef0fa40 0x55818ef11ee0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9fb7ff24d2c21f12 server_cookie=773434eaac2d4f73 in_seq=0 out_seq=0 2024-05-17T22:49:44.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.281+0000 7fb1bb7fe700 10 client.25901 ms_handle_connect on v2:172.21.15.151:6838/2338353849 2024-05-17T22:49:44.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.281+0000 7fb1bb7fe700 1 -- 172.21.15.12:0/539728590 <== mds.2 v2:172.21.15.151:6838/2338353849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fb1c40044d0 con 0x55818ef0fa40 2024-05-17T22:49:44.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.281+0000 7fb1bb7fe700 10 client.25901 handle_command_reply: tid=0 2024-05-17T22:49:44.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.282+0000 7fb1f0e0bb80 1 client.25901 shutdown 2024-05-17T22:49:44.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.289+0000 7fb1f0e0bb80 1 -- 172.21.15.12:0/539728590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55818eef6b00 msgr2=0x55818eefff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:44.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.289+0000 7fb1f0e0bb80 1 --2- 172.21.15.12:0/539728590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55818eef6b00 0x55818eefff50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc011040 tx=0x7fb1cc0b8f60 comp rx=0 tx=0).stop 2024-05-17T22:49:44.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.289+0000 7fb1f0e0bb80 1 -- 172.21.15.12:0/539728590 shutdown_connections 2024-05-17T22:49:44.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.289+0000 7fb1f0e0bb80 1 --2- 172.21.15.12:0/539728590 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55818eefc400 0x55818ef00490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:44.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.289+0000 7fb1f0e0bb80 1 --2- 172.21.15.12:0/539728590 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55818ef0fa40 0x55818ef11ee0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:44.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.289+0000 7fb1f0e0bb80 1 --2- 172.21.15.12:0/539728590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55818ed789f0 0x55818eeffa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:44.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.289+0000 7fb1f0e0bb80 1 --2- 172.21.15.12:0/539728590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55818eef6b00 0x55818eefff50 secure :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc011040 tx=0x7fb1cc0b8f60 comp rx=0 tx=0).stop 2024-05-17T22:49:44.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.289+0000 7fb1f0e0bb80 1 -- 172.21.15.12:0/539728590 >> 172.21.15.12:0/539728590 conn(0x55818edd87d0 msgr2=0x55818eef1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:49:44.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.289+0000 7fb1bb7fe700 0 client.25901 ms_handle_reset on v2:172.21.15.151:6838/2338353849 2024-05-17T22:49:44.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.292+0000 7fb1f0e0bb80 1 -- 172.21.15.12:0/539728590 shutdown_connections 2024-05-17T22:49:44.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.295+0000 7fb1f0e0bb80 1 -- 172.21.15.12:0/539728590 wait complete. 2024-05-17T22:49:44.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.295+0000 7fb1f0e0bb80 20 client.25901 trim_cache size 0 max 16384 2024-05-17T22:49:44.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.328+0000 7fb1f0e0bb80 1 Processor -- start 2024-05-17T22:49:44.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.331+0000 7fb1f0e0bb80 1 -- start start 2024-05-17T22:49:44.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.331+0000 7fb1f0e0bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55818ed789f0 0x55818eefff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:44.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.331+0000 7fb1f0e0bb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55818eefc400 0x55818ef004a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:44.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.331+0000 7fb1f0e0bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55818eefd820 0x55818eefad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:44.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.331+0000 7fb1f0e0bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb1bc011450 con 0x55818eefd820 2024-05-17T22:49:44.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.331+0000 7fb1f0e0bb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb1bc00ece0 con 0x55818eefc400 2024-05-17T22:49:44.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.331+0000 7fb1f0e0bb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb1bc00cb30 con 0x55818ed789f0 2024-05-17T22:49:44.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.335+0000 7fb1db7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55818eefc400 0x55818ef004a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:44.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.335+0000 7fb1e09cd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55818eefd820 0x55818eefad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:44.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.336+0000 7fb1e09cd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55818eefd820 0x55818eefad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55442/0 (socket says 172.21.15.12:55442) 2024-05-17T22:49:44.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.336+0000 7fb1e09cd700 1 -- 172.21.15.12:0/3460002164 learned_addr learned my addr 172.21.15.12:0/3460002164 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:49:44.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.337+0000 7fb1e09cd700 1 -- 172.21.15.12:0/3460002164 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55818ed789f0 msgr2=0x55818eefff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:49:44.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.337+0000 7fb1e09cd700 1 --2- 172.21.15.12:0/3460002164 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55818ed789f0 0x55818eefff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:44.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.337+0000 7fb1e09cd700 1 -- 172.21.15.12:0/3460002164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55818eefc400 msgr2=0x55818ef004a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:49:44.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.337+0000 7fb1e09cd700 1 --2- 172.21.15.12:0/3460002164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55818eefc400 0x55818ef004a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:44.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.337+0000 7fb1e09cd700 1 -- 172.21.15.12:0/3460002164 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55818eef2970 con 0x55818eefd820 2024-05-17T22:49:44.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.337+0000 7fb1e09cd700 1 --2- 172.21.15.12:0/3460002164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55818eefd820 0x55818eefad70 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb1c400ab90 tx=0x7fb1c400a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de0d4a13fcc31a0a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:49:44.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.341+0000 7fb1d1ffb700 1 -- 172.21.15.12:0/3460002164 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c4026030 con 0x55818eefd820 2024-05-17T22:49:44.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.341+0000 7fb1d1ffb700 1 -- 172.21.15.12:0/3460002164 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1c401d040 con 0x55818eefd820 2024-05-17T22:49:44.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.341+0000 7fb1d1ffb700 1 -- 172.21.15.12:0/3460002164 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c401c040 con 0x55818eefd820 2024-05-17T22:49:44.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.343+0000 7fb1f0e0bb80 1 -- 172.21.15.12:0/3460002164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55818eefd820 msgr2=0x55818eefad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:44.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.343+0000 7fb1f0e0bb80 1 --2- 172.21.15.12:0/3460002164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55818eefd820 0x55818eefad70 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb1c400ab90 tx=0x7fb1c400a5d0 comp rx=0 tx=0).stop 2024-05-17T22:49:44.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.345+0000 7fb1f0e0bb80 1 -- 172.21.15.12:0/3460002164 shutdown_connections 2024-05-17T22:49:44.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.345+0000 7fb1f0e0bb80 1 --2- 172.21.15.12:0/3460002164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55818eefd820 0x55818eefad70 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:44.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.345+0000 7fb1f0e0bb80 1 --2- 172.21.15.12:0/3460002164 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55818ed789f0 0x55818eefff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:44.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.345+0000 7fb1f0e0bb80 1 --2- 172.21.15.12:0/3460002164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55818eefc400 0x55818ef004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:44.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.345+0000 7fb1f0e0bb80 1 -- 172.21.15.12:0/3460002164 >> 172.21.15.12:0/3460002164 conn(0x55818edd87d0 msgr2=0x55818eefe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:49:44.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.350+0000 7fb1f0e0bb80 1 -- 172.21.15.12:0/3460002164 shutdown_connections 2024-05-17T22:49:44.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.351+0000 7fb1f0e0bb80 1 -- 172.21.15.12:0/3460002164 wait complete. 2024-05-17T22:49:44.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.352+0000 7fb1f0e0bb80 1 -- start start 2024-05-17T22:49:44.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.352+0000 7fb1f0e0bb80 1 Processor -- start 2024-05-17T22:49:44.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.352+0000 7fb1f0e0bb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55818ed789f0 0x55818ef080f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:44.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.352+0000 7fb1f0e0bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55818eefc400 0x55818ef08630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:44.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.353+0000 7fb1f0e0bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55818ef166d0 0x55818ef16ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:44.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.353+0000 7fb1f0e0bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb1bc00d1a0 con 0x55818ef166d0 2024-05-17T22:49:44.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.353+0000 7fb1f0e0bb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb1bc00e610 con 0x55818ed789f0 2024-05-17T22:49:44.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.353+0000 7fb1f0e0bb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb1bc00d870 con 0x55818eefc400 2024-05-17T22:49:44.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.353+0000 7fb1f0e0bb80 10 client.0 _ll_register_callbacks cb 0x55818edabb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:49:44.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.353+0000 7fb1e09cd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55818ef166d0 0x55818ef16ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:44.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.353+0000 7fb1e09cd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55818ef166d0 0x55818ef16ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55450/0 (socket says 172.21.15.12:55450) 2024-05-17T22:49:44.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.353+0000 7fb1e09cd700 1 -- 172.21.15.12:0/3415899942 learned_addr learned my addr 172.21.15.12:0/3415899942 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:49:44.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.353+0000 7fb1dbfff700 1 --2- 172.21.15.12:0/3415899942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55818ed789f0 0x55818ef080f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:44.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.354+0000 7fb1db7fe700 1 --2- 172.21.15.12:0/3415899942 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55818eefc400 0x55818ef08630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:44.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.354+0000 7fb1e09cd700 1 -- 172.21.15.12:0/3415899942 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55818eefc400 msgr2=0x55818ef08630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:49:44.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.354+0000 7fb1e09cd700 1 --2- 172.21.15.12:0/3415899942 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55818eefc400 0x55818ef08630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:44.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.354+0000 7fb1e09cd700 1 -- 172.21.15.12:0/3415899942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55818ed789f0 msgr2=0x55818ef080f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:49:44.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.354+0000 7fb1e09cd700 1 --2- 172.21.15.12:0/3415899942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55818ed789f0 0x55818ef080f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:44.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.354+0000 7fb1e09cd700 1 -- 172.21.15.12:0/3415899942 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55818ef17190 con 0x55818ef166d0 2024-05-17T22:49:44.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.364+0000 7fb1e09cd700 1 --2- 172.21.15.12:0/3415899942 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55818ef166d0 0x55818ef16ae0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb1c401a040 tx=0x7fb1c40049f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34aff9e547bf662f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:49:44.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.364+0000 7fb1bb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:49:44.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.369+0000 7fb1bb7fe700 1 -- 172.21.15.12:0/3415899942 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c401c040 con 0x55818ef166d0 2024-05-17T22:49:44.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.369+0000 7fb1f0e0bb80 1 -- 172.21.15.12:0/3415899942 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1c401d040 con 0x55818ef166d0 2024-05-17T22:49:44.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.369+0000 7fb1bb7fe700 1 -- 172.21.15.12:0/3415899942 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fb1c4018a00 con 0x55818ef166d0 2024-05-17T22:49:44.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.373+0000 7fb1bb7fe700 1 -- 172.21.15.12:0/3415899942 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb1c400fe60 con 0x55818ef166d0 2024-05-17T22:49:44.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.375+0000 7fb1f0e0bb80 10 client.16200 fetch_fsmap learned FSMap version 39 2024-05-17T22:49:44.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.375+0000 7fb1f0e0bb80 1 -- 172.21.15.12:0/3415899942 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55818ef8afd0 con 0x55818ef166d0 2024-05-17T22:49:44.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.377+0000 7fb1bb7fe700 1 -- 172.21.15.12:0/3415899942 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1c401d040 con 0x55818ef166d0 2024-05-17T22:49:44.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.377+0000 7fb1bb7fe700 1 -- 172.21.15.12:0/3415899942 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fb1c4010c90 con 0x55818ef166d0 2024-05-17T22:49:44.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.377+0000 7fb1f0e0bb80 10 client.16200 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:49:44.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.377+0000 7fb1f0e0bb80 10 client.16200 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-17T22:49:44.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.377+0000 7fb1f0e0bb80 20 client.16200 populate_metadata read hostname 'smithi012' 2024-05-17T22:49:44.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.377+0000 7fb1f0e0bb80 1 --2- 172.21.15.12:0/3415899942 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55818ef8b4a0 0x55818ef8d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:44.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.377+0000 7fb1f0e0bb80 4 client.16200 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:49:44.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.377+0000 7fb1f0e0bb80 1 -- 172.21.15.12:0/3415899942 --> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55818ee27020 con 0x55818ef8b4a0 2024-05-17T22:49:44.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.379+0000 7fb1dbfff700 1 --2- 172.21.15.12:0/3415899942 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55818ef8b4a0 0x55818ef8d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:44.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.383+0000 7fb1bb7fe700 10 client.16200 ms_handle_connect on v2:172.21.15.151:6838/2338353849 2024-05-17T22:49:44.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.383+0000 7fb1dbfff700 1 --2- 172.21.15.12:0/3415899942 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55818ef8b4a0 0x55818ef8d8c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d0adcf034330080e server_cookie=dea924996aa295c1 in_seq=0 out_seq=0 2024-05-17T22:49:44.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.383+0000 7fb1bb7fe700 1 -- 172.21.15.12:0/3415899942 <== mds.2 v2:172.21.15.151:6838/2338353849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55818ee27020 con 0x55818ef8b4a0 2024-05-17T22:49:44.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.383+0000 7fb1bb7fe700 10 client.16200 handle_command_reply: tid=0 2024-05-17T22:49:44.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.383+0000 7fb1f0e0bb80 1 client.16200 shutdown 2024-05-17T22:49:44.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.386+0000 7fb1f0e0bb80 1 -- 172.21.15.12:0/3415899942 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55818ef166d0 msgr2=0x55818ef16ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:44.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.386+0000 7fb1f0e0bb80 1 --2- 172.21.15.12:0/3415899942 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55818ef166d0 0x55818ef16ae0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb1c401a040 tx=0x7fb1c40049f0 comp rx=0 tx=0).stop 2024-05-17T22:49:44.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.387+0000 7fb1f0e0bb80 1 -- 172.21.15.12:0/3415899942 shutdown_connections 2024-05-17T22:49:44.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.387+0000 7fb1f0e0bb80 1 --2- 172.21.15.12:0/3415899942 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55818ef166d0 0x55818ef16ae0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:44.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.387+0000 7fb1f0e0bb80 1 --2- 172.21.15.12:0/3415899942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55818ed789f0 0x55818ef080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:44.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.387+0000 7fb1f0e0bb80 1 --2- 172.21.15.12:0/3415899942 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55818ef8b4a0 0x55818ef8d8c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:44.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.387+0000 7fb1f0e0bb80 1 --2- 172.21.15.12:0/3415899942 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55818eefc400 0x55818ef08630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:44.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.388+0000 7fb1f0e0bb80 1 -- 172.21.15.12:0/3415899942 >> 172.21.15.12:0/3415899942 conn(0x55818edd87d0 msgr2=0x55818eefad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:49:44.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.388+0000 7fb1bb7fe700 0 client.16200 ms_handle_reset on v2:172.21.15.151:6838/2338353849 2024-05-17T22:49:44.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.389+0000 7fb1f0e0bb80 1 -- 172.21.15.12:0/3415899942 shutdown_connections 2024-05-17T22:49:44.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.389+0000 7fb1f0e0bb80 1 -- 172.21.15.12:0/3415899942 wait complete. 2024-05-17T22:49:44.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.390+0000 7fb1f0e0bb80 20 client.16200 trim_cache size 0 max 16384 2024-05-17T22:49:44.392 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:49:44.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.395+0000 7fb1e2430700 1 -- 172.21.15.12:0/140243798 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb1c80b2a90 msgr2=0x7fb1c80b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:44.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.395+0000 7fb1e2430700 1 --2- 172.21.15.12:0/140243798 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb1c80b2a90 0x7fb1c80b4fa0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x55818eef26b0 tx=0x7fb1d4009040 comp rx=0 tx=0).stop 2024-05-17T22:49:44.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.395+0000 7fb1e2430700 1 -- 172.21.15.12:0/140243798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1dc107730 msgr2=0x7fb1dc10c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:44.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.395+0000 7fb1e2430700 1 --2- 172.21.15.12:0/140243798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1dc107730 0x7fb1dc10c7b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc0057c0 tx=0x7fb1cc0057f0 comp rx=0 tx=0).stop 2024-05-17T22:49:44.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.397+0000 7fb1e2430700 1 -- 172.21.15.12:0/140243798 shutdown_connections 2024-05-17T22:49:44.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.397+0000 7fb1e2430700 1 --2- 172.21.15.12:0/140243798 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb1c80b2a90 0x7fb1c80b4fa0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:44.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.397+0000 7fb1e2430700 1 --2- 172.21.15.12:0/140243798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1dc108080 0x7fb1dc1acac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:44.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.397+0000 7fb1e2430700 1 --2- 172.21.15.12:0/140243798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1dc102e00 0x7fb1dc10c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:44.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.397+0000 7fb1e2430700 1 --2- 172.21.15.12:0/140243798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1dc107730 0x7fb1dc10c7b0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:44.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.397+0000 7fb1e2430700 1 -- 172.21.15.12:0/140243798 >> 172.21.15.12:0/140243798 conn(0x7fb1dc079bd0 msgr2=0x7fb1dc105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:49:44.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.397+0000 7fb1e2430700 1 -- 172.21.15.12:0/140243798 shutdown_connections 2024-05-17T22:49:44.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:44.400+0000 7fb1e2430700 1 -- 172.21.15.12:0/140243798 wait complete. 2024-05-17T22:49:44.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:49:44 smithi151 ceph-mon[36101]: pgmap v753: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.8 MiB/s wr, 459 op/s 2024-05-17T22:49:44.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:49:44 smithi096 ceph-mon[36065]: pgmap v753: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.8 MiB/s wr, 459 op/s 2024-05-17T22:49:45.122 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-17T22:49:45.574 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:49:46.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 625.0 s, 478.0 tps, lat 210.500 ms stddev 256.587 2024-05-17T22:49:46.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:49:46 smithi151 ceph-mon[36101]: pgmap v754: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.1 MiB/s wr, 522 op/s 2024-05-17T22:49:46.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:49:46 smithi096 ceph-mon[36065]: pgmap v754: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.1 MiB/s wr, 522 op/s 2024-05-17T22:49:46.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:49:46 smithi012 ceph-mon[31632]: pgmap v754: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.1 MiB/s wr, 522 op/s 2024-05-17T22:49:48.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:49:48 smithi151 ceph-mon[36101]: pgmap v755: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.9 MiB/s wr, 436 op/s 2024-05-17T22:49:48.494 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:49:48 smithi012 ceph-mon[31632]: pgmap v755: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.9 MiB/s wr, 436 op/s 2024-05-17T22:49:48.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:49:48 smithi096 ceph-mon[36065]: pgmap v755: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.9 MiB/s wr, 436 op/s 2024-05-17T22:49:48.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.732+0000 7fe18dc0d700 1 -- 172.21.15.12:0/4094756744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe18810b180 msgr2=0x7fe18811d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:48.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.732+0000 7fe18dc0d700 1 --2- 172.21.15.12:0/4094756744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe18810b180 0x7fe18811d010 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe1700098f0 tx=0x7fe170009c00 comp rx=0 tx=0).stop 2024-05-17T22:49:48.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.734+0000 7fe18dc0d700 1 -- 172.21.15.12:0/4094756744 shutdown_connections 2024-05-17T22:49:48.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.734+0000 7fe18dc0d700 1 --2- 172.21.15.12:0/4094756744 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe18811d550 0x7fe18811f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:48.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.734+0000 7fe18dc0d700 1 --2- 172.21.15.12:0/4094756744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe18810a830 0x7fe18810ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:48.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.734+0000 7fe18dc0d700 1 --2- 172.21.15.12:0/4094756744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe18810b180 0x7fe18811d010 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:48.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.734+0000 7fe18dc0d700 1 -- 172.21.15.12:0/4094756744 >> 172.21.15.12:0/4094756744 conn(0x7fe18806efc0 msgr2=0x7fe18806f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:49:48.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.742+0000 7fe18dc0d700 1 -- 172.21.15.12:0/4094756744 shutdown_connections 2024-05-17T22:49:48.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.742+0000 7fe18dc0d700 1 -- 172.21.15.12:0/4094756744 wait complete. 2024-05-17T22:49:48.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.743+0000 7fe18dc0d700 1 Processor -- start 2024-05-17T22:49:48.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.743+0000 7fe18dc0d700 1 -- start start 2024-05-17T22:49:48.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.743+0000 7fe18dc0d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe18810a830 0x7fe18811a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:48.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.743+0000 7fe18dc0d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe18810b180 0x7fe1881128c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:48.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.743+0000 7fe18dc0d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe18811d550 0x7fe188112e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:48.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.743+0000 7fe18dc0d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe188122c30 con 0x7fe18810a830 2024-05-17T22:49:48.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.743+0000 7fe18dc0d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe188122ab0 con 0x7fe18810b180 2024-05-17T22:49:48.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.743+0000 7fe18dc0d700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe188122db0 con 0x7fe18811d550 2024-05-17T22:49:48.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.744+0000 7fe187fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe18811d550 0x7fe188112e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:48.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.744+0000 7fe187fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe18811d550 0x7fe188112e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:47556/0 (socket says 172.21.15.12:47556) 2024-05-17T22:49:48.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.744+0000 7fe187fff700 1 -- 172.21.15.12:0/3374057454 learned_addr learned my addr 172.21.15.12:0/3374057454 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:49:48.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.744+0000 7fe17effd700 1 --2- 172.21.15.12:0/3374057454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe18810b180 0x7fe1881128c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:48.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.744+0000 7fe1877fe700 1 --2- 172.21.15.12:0/3374057454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe18810a830 0x7fe18811a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:48.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.751+0000 7fe187fff700 1 -- 172.21.15.12:0/3374057454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe18810b180 msgr2=0x7fe1881128c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:48.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.751+0000 7fe187fff700 1 --2- 172.21.15.12:0/3374057454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe18810b180 0x7fe1881128c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:48.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.751+0000 7fe187fff700 1 -- 172.21.15.12:0/3374057454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe18810a830 msgr2=0x7fe18811a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:48.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.751+0000 7fe187fff700 1 --2- 172.21.15.12:0/3374057454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe18810a830 0x7fe18811a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:48.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.751+0000 7fe187fff700 1 -- 172.21.15.12:0/3374057454 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe188113500 con 0x7fe18811d550 2024-05-17T22:49:48.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.754+0000 7fe187fff700 1 --2- 172.21.15.12:0/3374057454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe18811d550 0x7fe188112e00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe180030950 tx=0x7fe180030c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:49:48.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.757+0000 7fe1857fa700 1 -- 172.21.15.12:0/3374057454 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe180030ea0 con 0x7fe18811d550 2024-05-17T22:49:48.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.758+0000 7fe1857fa700 1 -- 172.21.15.12:0/3374057454 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe180004ce0 con 0x7fe18811d550 2024-05-17T22:49:48.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.758+0000 7fe1857fa700 1 -- 172.21.15.12:0/3374057454 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe180005600 con 0x7fe18811d550 2024-05-17T22:49:48.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.758+0000 7fe18dc0d700 1 -- 172.21.15.12:0/3374057454 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1881a9370 con 0x7fe18811d550 2024-05-17T22:49:48.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.758+0000 7fe18dc0d700 1 -- 172.21.15.12:0/3374057454 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1881a9860 con 0x7fe18811d550 2024-05-17T22:49:48.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.770+0000 7fe19c5e8b80 1 Processor -- start 2024-05-17T22:49:48.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.771+0000 7fe19c5e8b80 1 -- start start 2024-05-17T22:49:48.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.772+0000 7fe19c5e8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611c35ba9f0 0x5611c373e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:48.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.772+0000 7fe19c5e8b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5611c373eaa0 0x5611c3738c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:48.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.772+0000 7fe19c5e8b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611c3739170 0x5611c3739580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:48.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.772+0000 7fe19c5e8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5611c3667290 con 0x5611c35ba9f0 2024-05-17T22:49:48.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.772+0000 7fe19c5e8b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5611c3656fd0 con 0x5611c3739170 2024-05-17T22:49:48.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.772+0000 7fe19c5e8b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5611c365ae70 con 0x5611c373eaa0 2024-05-17T22:49:48.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.773+0000 7fe1857fa700 1 -- 172.21.15.12:0/3374057454 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe180004750 con 0x7fe18811d550 2024-05-17T22:49:48.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.773+0000 7fe1877fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5611c373eaa0 0x5611c3738c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:48.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.773+0000 7fe1877fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5611c373eaa0 0x5611c3738c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:47572/0 (socket says 172.21.15.12:47572) 2024-05-17T22:49:48.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.773+0000 7fe17effd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611c3739170 0x5611c3739580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:48.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.773+0000 7fe1877fe700 1 -- 172.21.15.12:0/851197811 learned_addr learned my addr 172.21.15.12:0/851197811 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:49:48.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.774+0000 7fe1857fa700 1 --2- 172.21.15.12:0/3374057454 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe1740b2a90 0x7fe1740b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:48.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.774+0000 7fe1857fa700 1 -- 172.21.15.12:0/3374057454 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe180104700 con 0x7fe18811d550 2024-05-17T22:49:48.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.776+0000 7fe17effd700 1 -- 172.21.15.12:0/851197811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5611c373eaa0 msgr2=0x5611c3738c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:49:48.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.776+0000 7fe17effd700 1 --2- 172.21.15.12:0/851197811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5611c373eaa0 0x5611c3738c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:48.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.776+0000 7fe17effd700 1 -- 172.21.15.12:0/851197811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611c35ba9f0 msgr2=0x5611c373e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:49:48.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.776+0000 7fe17effd700 1 --2- 172.21.15.12:0/851197811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611c35ba9f0 0x5611c373e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:48.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.776+0000 7fe17effd700 1 -- 172.21.15.12:0/851197811 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611c3600590 con 0x5611c3739170 2024-05-17T22:49:48.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.776+0000 7fe17effd700 1 --2- 172.21.15.12:0/851197811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611c3739170 0x5611c3739580 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe1700098c0 tx=0x7fe170009530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d147f2064fac060 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:49:48.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.776+0000 7fe17dffb700 1 -- 172.21.15.12:0/851197811 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17000fbc0 con 0x5611c3739170 2024-05-17T22:49:48.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.776+0000 7fe187fff700 1 --2- 172.21.15.12:0/3374057454 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe1740b2a90 0x7fe1740b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:48.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.777+0000 7fe17dffb700 1 -- 172.21.15.12:0/851197811 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe17000f910 con 0x5611c3739170 2024-05-17T22:49:48.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.777+0000 7fe17dffb700 1 -- 172.21.15.12:0/851197811 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe170003680 con 0x5611c3739170 2024-05-17T22:49:48.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.778+0000 7fe19c5e8b80 1 -- 172.21.15.12:0/851197811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611c3739170 msgr2=0x5611c3739580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:48.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.778+0000 7fe19c5e8b80 1 --2- 172.21.15.12:0/851197811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611c3739170 0x5611c3739580 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe1700098c0 tx=0x7fe170009530 comp rx=0 tx=0).stop 2024-05-17T22:49:48.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.780+0000 7fe19c5e8b80 1 -- 172.21.15.12:0/851197811 shutdown_connections 2024-05-17T22:49:48.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.780+0000 7fe19c5e8b80 1 --2- 172.21.15.12:0/851197811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611c3739170 0x5611c3739580 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:48.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.780+0000 7fe19c5e8b80 1 --2- 172.21.15.12:0/851197811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611c35ba9f0 0x5611c373e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:48.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.780+0000 7fe19c5e8b80 1 --2- 172.21.15.12:0/851197811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5611c373eaa0 0x5611c3738c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:48.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.780+0000 7fe19c5e8b80 1 -- 172.21.15.12:0/851197811 >> 172.21.15.12:0/851197811 conn(0x5611c35fef00 msgr2=0x5611c3733b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:49:48.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.785+0000 7fe187fff700 1 --2- 172.21.15.12:0/3374057454 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe1740b2a90 0x7fe1740b4fa0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe180009fd0 tx=0x7fe180005020 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:49:48.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.790+0000 7fe19c5e8b80 1 -- 172.21.15.12:0/851197811 shutdown_connections 2024-05-17T22:49:48.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.790+0000 7fe19c5e8b80 1 -- 172.21.15.12:0/851197811 wait complete. 2024-05-17T22:49:48.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.791+0000 7fe19c5e8b80 1 -- start start 2024-05-17T22:49:48.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.791+0000 7fe19c5e8b80 1 Processor -- start 2024-05-17T22:49:48.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.799+0000 7fe19c5e8b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611c35ba9f0 0x5611c3742240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:48.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.800+0000 7fe19c5e8b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5611c3739170 0x5611c3742780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:48.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.800+0000 7fe19c5e8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611c373eaa0 0x5611c3742d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:48.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.800+0000 7fe19c5e8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5611c355e150 con 0x5611c373eaa0 2024-05-17T22:49:48.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.800+0000 7fe19c5e8b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5611c35f7e80 con 0x5611c35ba9f0 2024-05-17T22:49:48.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.800+0000 7fe19c5e8b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5611c3745390 con 0x5611c3739170 2024-05-17T22:49:48.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.801+0000 7fe19c5e8b80 10 client.0 _ll_register_callbacks cb 0x5611c35f1f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:49:48.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.802+0000 7fe17effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611c373eaa0 0x5611c3742d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:48.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.802+0000 7fe17effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611c373eaa0 0x5611c3742d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55484/0 (socket says 172.21.15.12:55484) 2024-05-17T22:49:48.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.802+0000 7fe17effd700 1 -- 172.21.15.12:0/3123161475 learned_addr learned my addr 172.21.15.12:0/3123161475 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:49:48.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.802+0000 7fe17effd700 1 --2- 172.21.15.12:0/3123161475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611c35ba9f0 0x5611c3742240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:48.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.804+0000 7fe17effd700 1 -- 172.21.15.12:0/3123161475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5611c3739170 msgr2=0x5611c3742780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T22:49:48.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.804+0000 7fe17effd700 1 --2- 172.21.15.12:0/3123161475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5611c3739170 0x5611c3742780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:48.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.804+0000 7fe17effd700 1 -- 172.21.15.12:0/3123161475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611c373eaa0 msgr2=0x5611c3742d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:49:48.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.804+0000 7fe17effd700 1 --2- 172.21.15.12:0/3123161475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611c373eaa0 0x5611c3742d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:48.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.804+0000 7fe17effd700 1 -- 172.21.15.12:0/3123161475 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611c35a7480 con 0x5611c35ba9f0 2024-05-17T22:49:48.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.804+0000 7fe17effd700 1 --2- 172.21.15.12:0/3123161475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611c35ba9f0 0x5611c3742240 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe1700098f0 tx=0x7fe170009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4969763f2fffbd1c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:49:48.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.804+0000 7fe166ffd700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:49:48.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.805+0000 7fe166ffd700 1 -- 172.21.15.12:0/3123161475 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17000fbc0 con 0x5611c35ba9f0 2024-05-17T22:49:48.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.805+0000 7fe166ffd700 1 -- 172.21.15.12:0/3123161475 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe17000fd60 con 0x5611c35ba9f0 2024-05-17T22:49:48.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.805+0000 7fe19c5e8b80 1 -- 172.21.15.12:0/3123161475 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe17000f910 con 0x5611c35ba9f0 2024-05-17T22:49:48.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.805+0000 7fe166ffd700 1 -- 172.21.15.12:0/3123161475 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe17003a050 con 0x5611c35ba9f0 2024-05-17T22:49:48.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.805+0000 7fe19c5e8b80 10 client.25919 fetch_fsmap learned FSMap version 39 2024-05-17T22:49:48.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.805+0000 7fe19c5e8b80 1 -- 172.21.15.12:0/3123161475 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5611c374fe00 con 0x5611c35ba9f0 2024-05-17T22:49:48.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.806+0000 7fe166ffd700 1 -- 172.21.15.12:0/3123161475 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe17000f910 con 0x5611c35ba9f0 2024-05-17T22:49:48.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.806+0000 7fe166ffd700 1 -- 172.21.15.12:0/3123161475 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fe17002c020 con 0x5611c35ba9f0 2024-05-17T22:49:48.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.806+0000 7fe19c5e8b80 10 client.25919 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:49:48.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.806+0000 7fe19c5e8b80 10 client.25919 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-17T22:49:48.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.806+0000 7fe19c5e8b80 20 client.25919 populate_metadata read hostname 'smithi012' 2024-05-17T22:49:48.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.806+0000 7fe19c5e8b80 1 --2- 172.21.15.12:0/3123161475 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5611c37522d0 0x5611c3754710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:48.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.806+0000 7fe19c5e8b80 4 client.25919 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:49:48.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.806+0000 7fe19c5e8b80 1 -- 172.21.15.12:0/3123161475 --> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611c36691e0 con 0x5611c37522d0 2024-05-17T22:49:48.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.810+0000 7fe1877fe700 1 --2- 172.21.15.12:0/3123161475 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5611c37522d0 0x5611c3754710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:48.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.813+0000 7fe1877fe700 1 --2- 172.21.15.12:0/3123161475 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5611c37522d0 0x5611c3754710 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a4c23db105ba1e5b server_cookie=493f43300bac4d9b in_seq=0 out_seq=0 2024-05-17T22:49:48.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.814+0000 7fe166ffd700 10 client.25919 ms_handle_connect on v2:172.21.15.96:6836/4222574338 2024-05-17T22:49:48.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.815+0000 7fe166ffd700 1 -- 172.21.15.12:0/3123161475 <== mds.1 v2:172.21.15.96:6836/4222574338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5611c36691e0 con 0x5611c37522d0 2024-05-17T22:49:48.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.815+0000 7fe166ffd700 10 client.25919 handle_command_reply: tid=0 2024-05-17T22:49:48.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.815+0000 7fe19c5e8b80 1 client.25919 shutdown 2024-05-17T22:49:48.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.816+0000 7fe19c5e8b80 1 -- 172.21.15.12:0/3123161475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611c35ba9f0 msgr2=0x5611c3742240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:48.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.816+0000 7fe19c5e8b80 1 --2- 172.21.15.12:0/3123161475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611c35ba9f0 0x5611c3742240 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe1700098f0 tx=0x7fe170009c00 comp rx=0 tx=0).stop 2024-05-17T22:49:48.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.816+0000 7fe19c5e8b80 1 -- 172.21.15.12:0/3123161475 shutdown_connections 2024-05-17T22:49:48.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.816+0000 7fe19c5e8b80 1 --2- 172.21.15.12:0/3123161475 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5611c37522d0 0x5611c3754710 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:48.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.816+0000 7fe19c5e8b80 1 --2- 172.21.15.12:0/3123161475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611c373eaa0 0x5611c3742d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:48.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.816+0000 7fe19c5e8b80 1 --2- 172.21.15.12:0/3123161475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611c35ba9f0 0x5611c3742240 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:48.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.816+0000 7fe19c5e8b80 1 --2- 172.21.15.12:0/3123161475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5611c3739170 0x5611c3742780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:48.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.816+0000 7fe19c5e8b80 1 -- 172.21.15.12:0/3123161475 >> 172.21.15.12:0/3123161475 conn(0x5611c35fef00 msgr2=0x5611c372ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:49:48.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.816+0000 7fe166ffd700 0 client.25919 ms_handle_reset on v2:172.21.15.96:6836/4222574338 2024-05-17T22:49:48.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.817+0000 7fe19c5e8b80 1 -- 172.21.15.12:0/3123161475 shutdown_connections 2024-05-17T22:49:48.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.818+0000 7fe19c5e8b80 1 -- 172.21.15.12:0/3123161475 wait complete. 2024-05-17T22:49:48.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.818+0000 7fe19c5e8b80 20 client.25919 trim_cache size 0 max 16384 2024-05-17T22:49:48.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.839+0000 7fe19c5e8b80 1 Processor -- start 2024-05-17T22:49:48.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.840+0000 7fe19c5e8b80 1 -- start start 2024-05-17T22:49:48.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.840+0000 7fe19c5e8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611c35fef00 0x5611c3738c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:48.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.840+0000 7fe19c5e8b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5611c35ba9f0 0x5611c3741220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:48.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.840+0000 7fe19c5e8b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611c3739170 0x5611c3741b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:48.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.840+0000 7fe19c5e8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe1881b4bf0 con 0x5611c35fef00 2024-05-17T22:49:48.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.840+0000 7fe19c5e8b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe1881b2480 con 0x5611c3739170 2024-05-17T22:49:48.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.840+0000 7fe19c5e8b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe1881b02d0 con 0x5611c35ba9f0 2024-05-17T22:49:48.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.840+0000 7fe17effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611c35fef00 0x5611c3738c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:48.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.840+0000 7fe17effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611c35fef00 0x5611c3738c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55490/0 (socket says 172.21.15.12:55490) 2024-05-17T22:49:48.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.840+0000 7fe17effd700 1 -- 172.21.15.12:0/2160402272 learned_addr learned my addr 172.21.15.12:0/2160402272 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:49:48.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.840+0000 7fe17effd700 1 --2- 172.21.15.12:0/2160402272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611c3739170 0x5611c3741b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:48.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.841+0000 7fe17effd700 1 -- 172.21.15.12:0/2160402272 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5611c35ba9f0 msgr2=0x5611c3741220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:49:48.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.841+0000 7fe17effd700 1 --2- 172.21.15.12:0/2160402272 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5611c35ba9f0 0x5611c3741220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:48.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.841+0000 7fe17effd700 1 -- 172.21.15.12:0/2160402272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611c3739170 msgr2=0x5611c3741b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:49:48.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.841+0000 7fe17effd700 1 --2- 172.21.15.12:0/2160402272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611c3739170 0x5611c3741b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:48.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.841+0000 7fe17effd700 1 -- 172.21.15.12:0/2160402272 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611c373e0d0 con 0x5611c35fef00 2024-05-17T22:49:48.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.841+0000 7fe17effd700 1 --2- 172.21.15.12:0/2160402272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611c35fef00 0x5611c3738c30 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe170005f40 tx=0x7fe170001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d3b30f77d57abb server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:49:48.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.841+0000 7fe17d7fa700 1 -- 172.21.15.12:0/2160402272 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17000fbc0 con 0x5611c35fef00 2024-05-17T22:49:48.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.841+0000 7fe17d7fa700 1 -- 172.21.15.12:0/2160402272 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe17000fd60 con 0x5611c35fef00 2024-05-17T22:49:48.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.841+0000 7fe17d7fa700 1 -- 172.21.15.12:0/2160402272 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe170003c90 con 0x5611c35fef00 2024-05-17T22:49:48.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.842+0000 7fe19c5e8b80 1 -- 172.21.15.12:0/2160402272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611c35fef00 msgr2=0x5611c3738c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:48.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.842+0000 7fe19c5e8b80 1 --2- 172.21.15.12:0/2160402272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611c35fef00 0x5611c3738c30 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe170005f40 tx=0x7fe170001e50 comp rx=0 tx=0).stop 2024-05-17T22:49:48.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.842+0000 7fe19c5e8b80 1 -- 172.21.15.12:0/2160402272 shutdown_connections 2024-05-17T22:49:48.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.842+0000 7fe19c5e8b80 1 --2- 172.21.15.12:0/2160402272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611c3739170 0x5611c3741b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:48.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.842+0000 7fe19c5e8b80 1 --2- 172.21.15.12:0/2160402272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611c35fef00 0x5611c3738c30 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:48.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.842+0000 7fe19c5e8b80 1 --2- 172.21.15.12:0/2160402272 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5611c35ba9f0 0x5611c3741220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:48.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.842+0000 7fe19c5e8b80 1 -- 172.21.15.12:0/2160402272 >> 172.21.15.12:0/2160402272 conn(0x5611c374ea30 msgr2=0x5611c3740b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:49:48.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.844+0000 7fe19c5e8b80 1 -- 172.21.15.12:0/2160402272 shutdown_connections 2024-05-17T22:49:48.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.844+0000 7fe19c5e8b80 1 -- 172.21.15.12:0/2160402272 wait complete. 2024-05-17T22:49:48.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.845+0000 7fe19c5e8b80 1 -- start start 2024-05-17T22:49:48.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.845+0000 7fe19c5e8b80 1 Processor -- start 2024-05-17T22:49:48.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.846+0000 7fe19c5e8b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611c35ba9f0 0x5611c37454b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:48.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.846+0000 7fe19c5e8b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611c35fef00 0x5611c37459f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:48.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.846+0000 7fe19c5e8b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5611c3739170 0x5611c3745f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:48.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.846+0000 7fe17effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611c35ba9f0 0x5611c37454b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:48.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.846+0000 7fe1877fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611c35fef00 0x5611c37459f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:48.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.846+0000 7fe17effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611c35ba9f0 0x5611c37454b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55494/0 (socket says 172.21.15.12:55494) 2024-05-17T22:49:48.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.847+0000 7fe1877fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611c35fef00 0x5611c37459f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:43526/0 (socket says 172.21.15.12:43526) 2024-05-17T22:49:48.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.847+0000 7fe19c5e8b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe1881b0940 con 0x5611c35ba9f0 2024-05-17T22:49:48.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.847+0000 7fe19c5e8b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe1881b1db0 con 0x5611c35fef00 2024-05-17T22:49:48.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.847+0000 7fe19c5e8b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe1881b1010 con 0x5611c3739170 2024-05-17T22:49:48.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.847+0000 7fe17effd700 1 -- 172.21.15.12:0/2052587133 learned_addr learned my addr 172.21.15.12:0/2052587133 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:49:48.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.847+0000 7fe19c5e8b80 10 client.0 _ll_register_callbacks cb 0x5611c35f1f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:49:48.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.847+0000 7fe17effd700 1 --2- 172.21.15.12:0/2052587133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5611c3739170 0x5611c3745f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:48.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.847+0000 7fe1877fe700 1 -- 172.21.15.12:0/2052587133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5611c3739170 msgr2=0x5611c3745f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:49:48.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.847+0000 7fe1877fe700 1 --2- 172.21.15.12:0/2052587133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5611c3739170 0x5611c3745f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:48.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.847+0000 7fe1877fe700 1 -- 172.21.15.12:0/2052587133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611c35ba9f0 msgr2=0x5611c37454b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:49:48.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.847+0000 7fe1877fe700 1 --2- 172.21.15.12:0/2052587133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611c35ba9f0 0x5611c37454b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:48.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.847+0000 7fe1877fe700 1 -- 172.21.15.12:0/2052587133 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611c37baa70 con 0x5611c35fef00 2024-05-17T22:49:48.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.848+0000 7fe1877fe700 1 --2- 172.21.15.12:0/2052587133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611c35fef00 0x5611c37459f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x5611c3734cf0 tx=0x7fe178007980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29da79eee58d3a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:49:48.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.848+0000 7fe166ffd700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:49:48.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.848+0000 7fe166ffd700 1 -- 172.21.15.12:0/2052587133 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe178022060 con 0x5611c35fef00 2024-05-17T22:49:48.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.848+0000 7fe166ffd700 1 -- 172.21.15.12:0/2052587133 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe17801c250 con 0x5611c35fef00 2024-05-17T22:49:48.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.848+0000 7fe19c5e8b80 1 -- 172.21.15.12:0/2052587133 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe17000fd60 con 0x5611c35fef00 2024-05-17T22:49:48.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.848+0000 7fe166ffd700 1 -- 172.21.15.12:0/2052587133 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe178029050 con 0x5611c35fef00 2024-05-17T22:49:48.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.848+0000 7fe19c5e8b80 10 client.25922 fetch_fsmap learned FSMap version 39 2024-05-17T22:49:48.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.848+0000 7fe19c5e8b80 1 -- 172.21.15.12:0/2052587133 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5611c37bbc10 con 0x5611c35fef00 2024-05-17T22:49:48.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.849+0000 7fe166ffd700 1 -- 172.21.15.12:0/2052587133 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe17000fd60 con 0x5611c35fef00 2024-05-17T22:49:48.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.849+0000 7fe166ffd700 1 -- 172.21.15.12:0/2052587133 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fe1780044d0 con 0x5611c35fef00 2024-05-17T22:49:48.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.849+0000 7fe19c5e8b80 10 client.25922 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:49:48.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.849+0000 7fe19c5e8b80 10 client.25922 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-17T22:49:48.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.849+0000 7fe19c5e8b80 20 client.25922 populate_metadata read hostname 'smithi012' 2024-05-17T22:49:48.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.849+0000 7fe19c5e8b80 1 --2- 172.21.15.12:0/2052587133 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5611c373eaa0 0x5611c37be310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:48.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.849+0000 7fe19c5e8b80 4 client.25922 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:49:48.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.849+0000 7fe19c5e8b80 1 -- 172.21.15.12:0/2052587133 --> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5611c36629c0 con 0x5611c373eaa0 2024-05-17T22:49:48.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.850+0000 7fe17effd700 1 --2- 172.21.15.12:0/2052587133 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5611c373eaa0 0x5611c37be310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:48.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.851+0000 7fe166ffd700 10 client.25922 ms_handle_connect on v2:172.21.15.96:6836/4222574338 2024-05-17T22:49:48.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.851+0000 7fe17effd700 1 --2- 172.21.15.12:0/2052587133 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5611c373eaa0 0x5611c37be310 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2097a9878797fc66 server_cookie=97dc1b1e8782d9a1 in_seq=0 out_seq=0 2024-05-17T22:49:48.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.851+0000 7fe166ffd700 1 -- 172.21.15.12:0/2052587133 <== mds.1 v2:172.21.15.96:6836/4222574338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5611c36629c0 con 0x5611c373eaa0 2024-05-17T22:49:48.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.851+0000 7fe166ffd700 10 client.25922 handle_command_reply: tid=0 2024-05-17T22:49:48.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.851+0000 7fe19c5e8b80 1 client.25922 shutdown 2024-05-17T22:49:48.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.852+0000 7fe19c5e8b80 1 -- 172.21.15.12:0/2052587133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611c35fef00 msgr2=0x5611c37459f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:48.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.852+0000 7fe19c5e8b80 1 --2- 172.21.15.12:0/2052587133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611c35fef00 0x5611c37459f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x5611c3734cf0 tx=0x7fe178007980 comp rx=0 tx=0).stop 2024-05-17T22:49:48.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.852+0000 7fe19c5e8b80 1 -- 172.21.15.12:0/2052587133 shutdown_connections 2024-05-17T22:49:48.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.852+0000 7fe19c5e8b80 1 --2- 172.21.15.12:0/2052587133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5611c3739170 0x5611c3745f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:48.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.852+0000 7fe19c5e8b80 1 --2- 172.21.15.12:0/2052587133 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5611c373eaa0 0x5611c37be310 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:48.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.852+0000 7fe19c5e8b80 1 --2- 172.21.15.12:0/2052587133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611c35ba9f0 0x5611c37454b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:48.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.852+0000 7fe19c5e8b80 1 --2- 172.21.15.12:0/2052587133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611c35fef00 0x5611c37459f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:48.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.852+0000 7fe19c5e8b80 1 -- 172.21.15.12:0/2052587133 >> 172.21.15.12:0/2052587133 conn(0x5611c374ea30 msgr2=0x5611c3743a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:49:48.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.852+0000 7fe166ffd700 0 client.25922 ms_handle_reset on v2:172.21.15.96:6836/4222574338 2024-05-17T22:49:48.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.852+0000 7fe19c5e8b80 1 -- 172.21.15.12:0/2052587133 shutdown_connections 2024-05-17T22:49:48.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.852+0000 7fe19c5e8b80 1 -- 172.21.15.12:0/2052587133 wait complete. 2024-05-17T22:49:48.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.852+0000 7fe19c5e8b80 20 client.25922 trim_cache size 0 max 16384 2024-05-17T22:49:48.857 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:49:48.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.853+0000 7fe18dc0d700 1 -- 172.21.15.12:0/3374057454 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe1740b2a90 msgr2=0x7fe1740b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:48.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.853+0000 7fe18dc0d700 1 --2- 172.21.15.12:0/3374057454 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe1740b2a90 0x7fe1740b4fa0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe180009fd0 tx=0x7fe180005020 comp rx=0 tx=0).stop 2024-05-17T22:49:48.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.853+0000 7fe18dc0d700 1 -- 172.21.15.12:0/3374057454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe18811d550 msgr2=0x7fe188112e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:48.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.853+0000 7fe18dc0d700 1 --2- 172.21.15.12:0/3374057454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe18811d550 0x7fe188112e00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe180030950 tx=0x7fe180030c60 comp rx=0 tx=0).stop 2024-05-17T22:49:48.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.853+0000 7fe18dc0d700 1 -- 172.21.15.12:0/3374057454 shutdown_connections 2024-05-17T22:49:48.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.853+0000 7fe18dc0d700 1 --2- 172.21.15.12:0/3374057454 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fe1740b2a90 0x7fe1740b4fa0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:48.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.853+0000 7fe18dc0d700 1 --2- 172.21.15.12:0/3374057454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe18811d550 0x7fe188112e00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:48.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.853+0000 7fe18dc0d700 1 --2- 172.21.15.12:0/3374057454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe18810a830 0x7fe18811a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:48.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.853+0000 7fe18dc0d700 1 --2- 172.21.15.12:0/3374057454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe18810b180 0x7fe1881128c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:48.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.853+0000 7fe18dc0d700 1 -- 172.21.15.12:0/3374057454 >> 172.21.15.12:0/3374057454 conn(0x7fe18806efc0 msgr2=0x7fe18811b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:49:48.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.854+0000 7fe18dc0d700 1 -- 172.21.15.12:0/3374057454 shutdown_connections 2024-05-17T22:49:48.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:48.854+0000 7fe18dc0d700 1 -- 172.21.15.12:0/3374057454 wait complete. 2024-05-17T22:49:49.587 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-17T22:49:50.037 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:49:50.431 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:49:50 smithi012 ceph-mon[31632]: pgmap v756: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 7.1 MiB/s wr, 484 op/s 2024-05-17T22:49:50.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:49:50 smithi151 ceph-mon[36101]: pgmap v756: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 7.1 MiB/s wr, 484 op/s 2024-05-17T22:49:50.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:49:50 smithi096 ceph-mon[36065]: pgmap v756: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 7.1 MiB/s wr, 484 op/s 2024-05-17T22:49:51.373 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 630.0 s, 518.5 tps, lat 190.607 ms stddev 241.087 2024-05-17T22:49:52.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:49:52 smithi151 ceph-mon[36101]: pgmap v757: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 6.7 MiB/s wr, 434 op/s 2024-05-17T22:49:52.525 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:49:52 smithi012 ceph-mon[31632]: pgmap v757: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 6.7 MiB/s wr, 434 op/s 2024-05-17T22:49:52.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:49:52 smithi096 ceph-mon[36065]: pgmap v757: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 6.7 MiB/s wr, 434 op/s 2024-05-17T22:49:52.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.754+0000 7fea69002700 1 -- 172.21.15.12:0/3903550231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea641032a0 msgr2=0x7fea6410f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:52.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.754+0000 7fea69002700 1 --2- 172.21.15.12:0/3903550231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea641032a0 0x7fea6410f820 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fea5c00b0a0 tx=0x7fea5c00b3b0 comp rx=0 tx=0).stop 2024-05-17T22:49:52.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.754+0000 7fea69002700 1 -- 172.21.15.12:0/3903550231 shutdown_connections 2024-05-17T22:49:52.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.754+0000 7fea69002700 1 --2- 172.21.15.12:0/3903550231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea641032a0 0x7fea6410f820 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:52.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.754+0000 7fea69002700 1 --2- 172.21.15.12:0/3903550231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea64108980 0x7fea64108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:52.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.754+0000 7fea69002700 1 --2- 172.21.15.12:0/3903550231 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea64102950 0x7fea64102d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:52.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.754+0000 7fea69002700 1 -- 172.21.15.12:0/3903550231 >> 172.21.15.12:0/3903550231 conn(0x7fea640fe680 msgr2=0x7fea64100ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:49:52.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.754+0000 7fea69002700 1 -- 172.21.15.12:0/3903550231 shutdown_connections 2024-05-17T22:49:52.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.754+0000 7fea69002700 1 -- 172.21.15.12:0/3903550231 wait complete. 2024-05-17T22:49:52.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.755+0000 7fea69002700 1 Processor -- start 2024-05-17T22:49:52.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.755+0000 7fea69002700 1 -- start start 2024-05-17T22:49:52.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.756+0000 7fea69002700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea641032a0 0x7fea640754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:52.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.756+0000 7fea69002700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea64108980 0x7fea6406c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:52.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.756+0000 7fea69002700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea6406c580 0x7fea6406ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:52.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.756+0000 7fea69002700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fea64112b70 con 0x7fea64108980 2024-05-17T22:49:52.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.756+0000 7fea69002700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fea641129f0 con 0x7fea6406c580 2024-05-17T22:49:52.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.756+0000 7fea69002700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fea64112cf0 con 0x7fea641032a0 2024-05-17T22:49:52.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.757+0000 7fea68801700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea6406c580 0x7fea6406ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:52.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.757+0000 7fea63fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea641032a0 0x7fea640754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:52.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.757+0000 7fea68801700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea6406c580 0x7fea6406ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:36866/0 (socket says 172.21.15.12:36866) 2024-05-17T22:49:52.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.757+0000 7fea68801700 1 -- 172.21.15.12:0/2798568904 learned_addr learned my addr 172.21.15.12:0/2798568904 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:49:52.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.757+0000 7fea637fe700 1 --2- 172.21.15.12:0/2798568904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea64108980 0x7fea6406c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:52.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.757+0000 7fea63fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea641032a0 0x7fea640754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:54162/0 (socket says 172.21.15.12:54162) 2024-05-17T22:49:52.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.758+0000 7fea68801700 1 -- 172.21.15.12:0/2798568904 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea641032a0 msgr2=0x7fea640754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:52.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.758+0000 7fea68801700 1 --2- 172.21.15.12:0/2798568904 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea641032a0 0x7fea640754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:52.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.758+0000 7fea68801700 1 -- 172.21.15.12:0/2798568904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea64108980 msgr2=0x7fea6406c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:52.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.758+0000 7fea68801700 1 --2- 172.21.15.12:0/2798568904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea64108980 0x7fea6406c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:52.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.758+0000 7fea68801700 1 -- 172.21.15.12:0/2798568904 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea641a9180 con 0x7fea6406c580 2024-05-17T22:49:52.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.760+0000 7fea68801700 1 --2- 172.21.15.12:0/2798568904 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea6406c580 0x7fea6406ca50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fea5c00d010 tx=0x7fea5c004da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:49:52.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.760+0000 7fea617fa700 1 -- 172.21.15.12:0/2798568904 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c021070 con 0x7fea6406c580 2024-05-17T22:49:52.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.760+0000 7fea69002700 1 -- 172.21.15.12:0/2798568904 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea641a9410 con 0x7fea6406c580 2024-05-17T22:49:52.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.760+0000 7fea69002700 1 -- 172.21.15.12:0/2798568904 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea641a9950 con 0x7fea6406c580 2024-05-17T22:49:52.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.760+0000 7fea617fa700 1 -- 172.21.15.12:0/2798568904 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea5c007cc0 con 0x7fea6406c580 2024-05-17T22:49:52.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.763+0000 7fea617fa700 1 -- 172.21.15.12:0/2798568904 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c0047f0 con 0x7fea6406c580 2024-05-17T22:49:52.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.766+0000 7fea617fa700 1 -- 172.21.15.12:0/2798568904 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fea5c01d450 con 0x7fea6406c580 2024-05-17T22:49:52.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.767+0000 7fea78c3fb80 1 Processor -- start 2024-05-17T22:49:52.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.768+0000 7fea617fa700 1 --2- 172.21.15.12:0/2798568904 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fea4c0b2a50 0x7fea4c0b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:52.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.768+0000 7fea617fa700 1 -- 172.21.15.12:0/2798568904 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fea5c0e3570 con 0x7fea6406c580 2024-05-17T22:49:52.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.770+0000 7fea78c3fb80 1 -- start start 2024-05-17T22:49:52.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.771+0000 7fea78c3fb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e88e1099f0 0x55e88e28d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:52.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.771+0000 7fea78c3fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e88e28dcb0 0x55e88e287c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:52.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.771+0000 7fea78c3fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e88e288170 0x55e88e288580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:52.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.771+0000 7fea78c3fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e88e1b6290 con 0x55e88e288170 2024-05-17T22:49:52.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.771+0000 7fea78c3fb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e88e1a5fd0 con 0x55e88e1099f0 2024-05-17T22:49:52.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.771+0000 7fea78c3fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e88e1a9e70 con 0x55e88e28dcb0 2024-05-17T22:49:52.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.771+0000 7fea68801700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e88e288170 0x55e88e288580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:52.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.771+0000 7fea68801700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e88e288170 0x55e88e288580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49510/0 (socket says 172.21.15.12:49510) 2024-05-17T22:49:52.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.771+0000 7fea68801700 1 -- 172.21.15.12:0/3408620228 learned_addr learned my addr 172.21.15.12:0/3408620228 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:49:52.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.772+0000 7fea68801700 1 -- 172.21.15.12:0/3408620228 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e88e28dcb0 msgr2=0x55e88e287c30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T22:49:52.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.772+0000 7fea68801700 1 --2- 172.21.15.12:0/3408620228 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e88e28dcb0 0x55e88e287c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:52.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.772+0000 7fea68801700 1 -- 172.21.15.12:0/3408620228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e88e1099f0 msgr2=0x55e88e28d770 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T22:49:52.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.772+0000 7fea68801700 1 --2- 172.21.15.12:0/3408620228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e88e1099f0 0x55e88e28d770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:52.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.772+0000 7fea68801700 1 -- 172.21.15.12:0/3408620228 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e88e14f590 con 0x55e88e288170 2024-05-17T22:49:52.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.772+0000 7fea68801700 1 --2- 172.21.15.12:0/3408620228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e88e288170 0x55e88e288580 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fea5c0e0c80 tx=0x7fea5c00b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc2cce6fd29ad12e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:49:52.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.773+0000 7fea637fe700 1 --2- 172.21.15.12:0/2798568904 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fea4c0b2a50 0x7fea4c0b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:52.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.778+0000 7fea637fe700 1 --2- 172.21.15.12:0/2798568904 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fea4c0b2a50 0x7fea4c0b4f60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x55e88e28d240 tx=0x7fea58008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:49:52.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.779+0000 7fea4a7fc700 1 -- 172.21.15.12:0/3408620228 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c02a070 con 0x55e88e288170 2024-05-17T22:49:52.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.779+0000 7fea4a7fc700 1 -- 172.21.15.12:0/3408620228 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea5c0ab3f0 con 0x55e88e288170 2024-05-17T22:49:52.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.779+0000 7fea4a7fc700 1 -- 172.21.15.12:0/3408620228 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c0e3240 con 0x55e88e288170 2024-05-17T22:49:52.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.780+0000 7fea78c3fb80 1 -- 172.21.15.12:0/3408620228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e88e288170 msgr2=0x55e88e288580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:52.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.780+0000 7fea78c3fb80 1 --2- 172.21.15.12:0/3408620228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e88e288170 0x55e88e288580 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fea5c0e0c80 tx=0x7fea5c00b3b0 comp rx=0 tx=0).stop 2024-05-17T22:49:52.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.781+0000 7fea78c3fb80 1 -- 172.21.15.12:0/3408620228 shutdown_connections 2024-05-17T22:49:52.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.781+0000 7fea78c3fb80 1 --2- 172.21.15.12:0/3408620228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e88e288170 0x55e88e288580 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:52.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.781+0000 7fea78c3fb80 1 --2- 172.21.15.12:0/3408620228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e88e1099f0 0x55e88e28d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:52.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.781+0000 7fea78c3fb80 1 --2- 172.21.15.12:0/3408620228 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e88e28dcb0 0x55e88e287c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:52.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.781+0000 7fea78c3fb80 1 -- 172.21.15.12:0/3408620228 >> 172.21.15.12:0/3408620228 conn(0x55e88e14df00 msgr2=0x55e88e282b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:49:52.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.782+0000 7fea78c3fb80 1 -- 172.21.15.12:0/3408620228 shutdown_connections 2024-05-17T22:49:52.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.782+0000 7fea78c3fb80 1 -- 172.21.15.12:0/3408620228 wait complete. 2024-05-17T22:49:52.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.782+0000 7fea78c3fb80 1 -- start start 2024-05-17T22:49:52.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.782+0000 7fea78c3fb80 1 Processor -- start 2024-05-17T22:49:52.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.783+0000 7fea78c3fb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e88e1099f0 0x55e88e291330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:52.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.783+0000 7fea78c3fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e88e288170 0x55e88e291870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:52.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.783+0000 7fea78c3fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e88e28dcb0 0x55e88e291db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:52.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.783+0000 7fea78c3fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e88e0ad150 con 0x55e88e28dcb0 2024-05-17T22:49:52.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.783+0000 7fea78c3fb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e88e146e80 con 0x55e88e1099f0 2024-05-17T22:49:52.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.783+0000 7fea78c3fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e88e1b6290 con 0x55e88e288170 2024-05-17T22:49:52.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.783+0000 7fea78c3fb80 10 client.0 _ll_register_callbacks cb 0x55e88e140f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:49:52.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.783+0000 7fea63fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e88e1099f0 0x55e88e291330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:52.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.783+0000 7fea63fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e88e1099f0 0x55e88e291330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:36890/0 (socket says 172.21.15.12:36890) 2024-05-17T22:49:52.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.783+0000 7fea63fff700 1 -- 172.21.15.12:0/3331141755 learned_addr learned my addr 172.21.15.12:0/3331141755 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:49:52.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.783+0000 7fea68801700 1 --2- 172.21.15.12:0/3331141755 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e88e28dcb0 0x55e88e291db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:52.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.784+0000 7fea637fe700 1 --2- 172.21.15.12:0/3331141755 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e88e288170 0x55e88e291870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:52.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.785+0000 7fea68801700 1 -- 172.21.15.12:0/3331141755 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e88e288170 msgr2=0x55e88e291870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:49:52.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.785+0000 7fea68801700 1 --2- 172.21.15.12:0/3331141755 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e88e288170 0x55e88e291870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:52.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.785+0000 7fea68801700 1 -- 172.21.15.12:0/3331141755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e88e1099f0 msgr2=0x55e88e291330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:49:52.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.785+0000 7fea68801700 1 --2- 172.21.15.12:0/3331141755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e88e1099f0 0x55e88e291330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:52.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.785+0000 7fea68801700 1 -- 172.21.15.12:0/3331141755 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e88e0f6480 con 0x55e88e28dcb0 2024-05-17T22:49:52.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.787+0000 7fea437fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:49:52.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.787+0000 7fea68801700 1 --2- 172.21.15.12:0/3331141755 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e88e28dcb0 0x55e88e291db0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fea5c0aaeb0 tx=0x7fea5c0aada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52c9a9139fd135a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:49:52.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.788+0000 7fea437fe700 1 -- 172.21.15.12:0/3331141755 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c0c5030 con 0x55e88e28dcb0 2024-05-17T22:49:52.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.788+0000 7fea78c3fb80 1 -- 172.21.15.12:0/3331141755 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea5c0ab3f0 con 0x55e88e28dcb0 2024-05-17T22:49:52.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.788+0000 7fea437fe700 1 -- 172.21.15.12:0/3331141755 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fea5c00f040 con 0x55e88e28dcb0 2024-05-17T22:49:52.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.788+0000 7fea437fe700 1 -- 172.21.15.12:0/3331141755 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fea5c0c5030 con 0x55e88e28dcb0 2024-05-17T22:49:52.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.789+0000 7fea78c3fb80 10 client.16227 fetch_fsmap learned FSMap version 39 2024-05-17T22:49:52.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.789+0000 7fea78c3fb80 1 -- 172.21.15.12:0/3331141755 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e88e29ec60 con 0x55e88e28dcb0 2024-05-17T22:49:52.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.789+0000 7fea437fe700 1 -- 172.21.15.12:0/3331141755 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea5c0ab3f0 con 0x55e88e28dcb0 2024-05-17T22:49:52.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.791+0000 7fea437fe700 1 -- 172.21.15.12:0/3331141755 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fea5c011070 con 0x55e88e28dcb0 2024-05-17T22:49:52.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.791+0000 7fea78c3fb80 10 client.16227 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:49:52.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.791+0000 7fea78c3fb80 10 client.16227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:49:52.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.791+0000 7fea78c3fb80 20 client.16227 populate_metadata read hostname 'smithi012' 2024-05-17T22:49:52.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.791+0000 7fea78c3fb80 1 --2- 172.21.15.12:0/3331141755 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55e88e2a1130 0x55e88e2a3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:52.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.791+0000 7fea78c3fb80 4 client.16227 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:49:52.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.791+0000 7fea78c3fb80 1 -- 172.21.15.12:0/3331141755 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e88e1b81e0 con 0x55e88e2a1130 2024-05-17T22:49:52.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.791+0000 7fea63fff700 1 --2- 172.21.15.12:0/3331141755 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55e88e2a1130 0x55e88e2a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:52.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.792+0000 7fea63fff700 1 --2- 172.21.15.12:0/3331141755 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55e88e2a1130 0x55e88e2a3550 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c96076ca92c627d server_cookie=63c430423b6b334 in_seq=0 out_seq=0 2024-05-17T22:49:52.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.794+0000 7fea437fe700 10 client.16227 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:49:52.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.794+0000 7fea437fe700 1 -- 172.21.15.12:0/3331141755 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e88e1b81e0 con 0x55e88e2a1130 2024-05-17T22:49:52.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.794+0000 7fea437fe700 10 client.16227 handle_command_reply: tid=0 2024-05-17T22:49:52.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.794+0000 7fea78c3fb80 1 client.16227 shutdown 2024-05-17T22:49:52.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.794+0000 7fea78c3fb80 1 -- 172.21.15.12:0/3331141755 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e88e28dcb0 msgr2=0x55e88e291db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:52.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.794+0000 7fea78c3fb80 1 --2- 172.21.15.12:0/3331141755 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e88e28dcb0 0x55e88e291db0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fea5c0aaeb0 tx=0x7fea5c0aada0 comp rx=0 tx=0).stop 2024-05-17T22:49:52.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.795+0000 7fea78c3fb80 1 -- 172.21.15.12:0/3331141755 shutdown_connections 2024-05-17T22:49:52.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.795+0000 7fea78c3fb80 1 --2- 172.21.15.12:0/3331141755 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55e88e2a1130 0x55e88e2a3550 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:52.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.795+0000 7fea78c3fb80 1 --2- 172.21.15.12:0/3331141755 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e88e28dcb0 0x55e88e291db0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:52.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.795+0000 7fea78c3fb80 1 --2- 172.21.15.12:0/3331141755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e88e1099f0 0x55e88e291330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:52.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.795+0000 7fea78c3fb80 1 --2- 172.21.15.12:0/3331141755 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e88e288170 0x55e88e291870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:52.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.795+0000 7fea437fe700 0 client.16227 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:49:52.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.795+0000 7fea78c3fb80 1 -- 172.21.15.12:0/3331141755 >> 172.21.15.12:0/3331141755 conn(0x55e88e14df00 msgr2=0x55e88e282110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:49:52.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.797+0000 7fea78c3fb80 1 -- 172.21.15.12:0/3331141755 shutdown_connections 2024-05-17T22:49:52.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.800+0000 7fea78c3fb80 1 -- 172.21.15.12:0/3331141755 wait complete. 2024-05-17T22:49:52.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.800+0000 7fea78c3fb80 20 client.16227 trim_cache size 0 max 16384 2024-05-17T22:49:52.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.822+0000 7fea78c3fb80 1 Processor -- start 2024-05-17T22:49:52.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.822+0000 7fea78c3fb80 1 -- start start 2024-05-17T22:49:52.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.823+0000 7fea78c3fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e88e1099f0 0x55e88e287c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:52.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.823+0000 7fea78c3fb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e88e288170 0x55e88e284300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:52.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.823+0000 7fea78c3fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e88e28dcb0 0x55e88e292c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:52.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.823+0000 7fea78c3fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fea500116e0 con 0x55e88e1099f0 2024-05-17T22:49:52.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.823+0000 7fea78c3fb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fea5000ef70 con 0x55e88e288170 2024-05-17T22:49:52.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.823+0000 7fea78c3fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fea5000cd60 con 0x55e88e28dcb0 2024-05-17T22:49:52.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.823+0000 7fea68801700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e88e28dcb0 0x55e88e292c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:52.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.823+0000 7fea63fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e88e1099f0 0x55e88e287c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:52.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.823+0000 7fea68801700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e88e28dcb0 0x55e88e292c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:54194/0 (socket says 172.21.15.12:54194) 2024-05-17T22:49:52.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.823+0000 7fea68801700 1 -- 172.21.15.12:0/1817776595 learned_addr learned my addr 172.21.15.12:0/1817776595 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:49:52.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.823+0000 7fea637fe700 1 --2- 172.21.15.12:0/1817776595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e88e288170 0x55e88e284300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:52.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.823+0000 7fea68801700 1 -- 172.21.15.12:0/1817776595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e88e288170 msgr2=0x55e88e284300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:49:52.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.823+0000 7fea68801700 1 --2- 172.21.15.12:0/1817776595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e88e288170 0x55e88e284300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:52.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.823+0000 7fea68801700 1 -- 172.21.15.12:0/1817776595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e88e1099f0 msgr2=0x55e88e287c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:49:52.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.823+0000 7fea68801700 1 --2- 172.21.15.12:0/1817776595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e88e1099f0 0x55e88e287c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:52.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.823+0000 7fea68801700 1 -- 172.21.15.12:0/1817776595 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e88e28f8a0 con 0x55e88e28dcb0 2024-05-17T22:49:52.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.823+0000 7fea63fff700 1 --2- 172.21.15.12:0/1817776595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e88e1099f0 0x55e88e287c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:49:52.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.824+0000 7fea68801700 1 --2- 172.21.15.12:0/1817776595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e88e28dcb0 0x55e88e292c20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fea5c00b780 tx=0x7fea5c00b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca29f3b6e7da77c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:49:52.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.824+0000 7fea49ffb700 1 -- 172.21.15.12:0/1817776595 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c0c5030 con 0x55e88e28dcb0 2024-05-17T22:49:52.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.824+0000 7fea49ffb700 1 -- 172.21.15.12:0/1817776595 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea5c0ab3f0 con 0x55e88e28dcb0 2024-05-17T22:49:52.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.824+0000 7fea49ffb700 1 -- 172.21.15.12:0/1817776595 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c02a070 con 0x55e88e28dcb0 2024-05-17T22:49:52.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.825+0000 7fea78c3fb80 1 -- 172.21.15.12:0/1817776595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e88e28dcb0 msgr2=0x55e88e292c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:52.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.825+0000 7fea78c3fb80 1 --2- 172.21.15.12:0/1817776595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e88e28dcb0 0x55e88e292c20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fea5c00b780 tx=0x7fea5c00b3b0 comp rx=0 tx=0).stop 2024-05-17T22:49:52.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.825+0000 7fea78c3fb80 1 -- 172.21.15.12:0/1817776595 shutdown_connections 2024-05-17T22:49:52.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.825+0000 7fea78c3fb80 1 --2- 172.21.15.12:0/1817776595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e88e28dcb0 0x55e88e292c20 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:52.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.825+0000 7fea78c3fb80 1 --2- 172.21.15.12:0/1817776595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e88e1099f0 0x55e88e287c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:52.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.825+0000 7fea78c3fb80 1 --2- 172.21.15.12:0/1817776595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e88e288170 0x55e88e284300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:52.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.825+0000 7fea78c3fb80 1 -- 172.21.15.12:0/1817776595 >> 172.21.15.12:0/1817776595 conn(0x55e88e14df00 msgr2=0x55e88e282ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:49:52.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.825+0000 7fea78c3fb80 1 -- 172.21.15.12:0/1817776595 shutdown_connections 2024-05-17T22:49:52.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.825+0000 7fea78c3fb80 1 -- 172.21.15.12:0/1817776595 wait complete. 2024-05-17T22:49:52.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.826+0000 7fea78c3fb80 1 -- start start 2024-05-17T22:49:52.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.826+0000 7fea78c3fb80 1 Processor -- start 2024-05-17T22:49:52.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.826+0000 7fea78c3fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e88e1099f0 0x55e88e2a8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:52.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.826+0000 7fea78c3fb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e88e288170 0x55e88e2994c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:52.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.826+0000 7fea78c3fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e88e28dcb0 0x55e88e299a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:52.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.826+0000 7fea78c3fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fea5000d430 con 0x55e88e28dcb0 2024-05-17T22:49:52.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.826+0000 7fea78c3fb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fea5000e8a0 con 0x55e88e288170 2024-05-17T22:49:52.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.826+0000 7fea78c3fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fea5000db00 con 0x55e88e1099f0 2024-05-17T22:49:52.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.826+0000 7fea78c3fb80 10 client.0 _ll_register_callbacks cb 0x55e88e140f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:49:52.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.826+0000 7fea63fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e88e1099f0 0x55e88e2a8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:52.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.827+0000 7fea63fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e88e1099f0 0x55e88e2a8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:54200/0 (socket says 172.21.15.12:54200) 2024-05-17T22:49:52.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.827+0000 7fea63fff700 1 -- 172.21.15.12:0/2598312400 learned_addr learned my addr 172.21.15.12:0/2598312400 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:49:52.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.827+0000 7fea68801700 1 --2- 172.21.15.12:0/2598312400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e88e28dcb0 0x55e88e299a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:52.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.827+0000 7fea637fe700 1 --2- 172.21.15.12:0/2598312400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e88e288170 0x55e88e2994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:52.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.827+0000 7fea63fff700 1 -- 172.21.15.12:0/2598312400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e88e288170 msgr2=0x55e88e2994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:49:52.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.827+0000 7fea63fff700 1 --2- 172.21.15.12:0/2598312400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e88e288170 0x55e88e2994c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:52.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.827+0000 7fea63fff700 1 -- 172.21.15.12:0/2598312400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e88e28dcb0 msgr2=0x55e88e299a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:49:52.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.827+0000 7fea63fff700 1 --2- 172.21.15.12:0/2598312400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e88e28dcb0 0x55e88e299a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:52.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.827+0000 7fea63fff700 1 -- 172.21.15.12:0/2598312400 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e88e2a7b40 con 0x55e88e1099f0 2024-05-17T22:49:52.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.827+0000 7fea63fff700 1 --2- 172.21.15.12:0/2598312400 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e88e1099f0 0x55e88e2a8f70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fea54004790 tx=0x7fea5401a6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19acebfc68bec87a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:49:52.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.827+0000 7fea437fe700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:49:52.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.828+0000 7fea437fe700 1 -- 172.21.15.12:0/2598312400 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea54012910 con 0x55e88e1099f0 2024-05-17T22:49:52.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.828+0000 7fea437fe700 1 -- 172.21.15.12:0/2598312400 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fea54012ab0 con 0x55e88e1099f0 2024-05-17T22:49:52.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.828+0000 7fea78c3fb80 1 -- 172.21.15.12:0/2598312400 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea5c0ab3f0 con 0x55e88e1099f0 2024-05-17T22:49:52.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.828+0000 7fea437fe700 1 -- 172.21.15.12:0/2598312400 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fea54012cb0 con 0x55e88e1099f0 2024-05-17T22:49:52.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.828+0000 7fea78c3fb80 10 client.25867 fetch_fsmap learned FSMap version 39 2024-05-17T22:49:52.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.828+0000 7fea78c3fb80 1 -- 172.21.15.12:0/2598312400 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e88e30a940 con 0x55e88e1099f0 2024-05-17T22:49:52.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.829+0000 7fea437fe700 1 -- 172.21.15.12:0/2598312400 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea5c0ab3f0 con 0x55e88e1099f0 2024-05-17T22:49:52.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.829+0000 7fea437fe700 1 -- 172.21.15.12:0/2598312400 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fea54021070 con 0x55e88e1099f0 2024-05-17T22:49:52.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.829+0000 7fea78c3fb80 10 client.25867 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:49:52.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.829+0000 7fea78c3fb80 10 client.25867 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:49:52.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.829+0000 7fea78c3fb80 20 client.25867 populate_metadata read hostname 'smithi012' 2024-05-17T22:49:52.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.829+0000 7fea78c3fb80 1 --2- 172.21.15.12:0/2598312400 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55e88e30b0a0 0x55e88e30d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:49:52.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.829+0000 7fea78c3fb80 4 client.25867 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:49:52.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.829+0000 7fea78c3fb80 1 -- 172.21.15.12:0/2598312400 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e88e1b19c0 con 0x55e88e30b0a0 2024-05-17T22:49:52.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.830+0000 7fea637fe700 1 --2- 172.21.15.12:0/2598312400 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55e88e30b0a0 0x55e88e30d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:49:52.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.830+0000 7fea437fe700 10 client.25867 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:49:52.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.830+0000 7fea637fe700 1 --2- 172.21.15.12:0/2598312400 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55e88e30b0a0 0x55e88e30d4c0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf1104b090551168 server_cookie=630979a3f9fdc33d in_seq=0 out_seq=0 2024-05-17T22:49:52.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.830+0000 7fea437fe700 1 -- 172.21.15.12:0/2598312400 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e88e1b19c0 con 0x55e88e30b0a0 2024-05-17T22:49:52.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.830+0000 7fea437fe700 10 client.25867 handle_command_reply: tid=0 2024-05-17T22:49:52.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.830+0000 7fea78c3fb80 1 client.25867 shutdown 2024-05-17T22:49:52.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.830+0000 7fea78c3fb80 1 -- 172.21.15.12:0/2598312400 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e88e1099f0 msgr2=0x55e88e2a8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:52.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.830+0000 7fea78c3fb80 1 --2- 172.21.15.12:0/2598312400 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e88e1099f0 0x55e88e2a8f70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fea54004790 tx=0x7fea5401a6d0 comp rx=0 tx=0).stop 2024-05-17T22:49:52.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.830+0000 7fea78c3fb80 1 -- 172.21.15.12:0/2598312400 shutdown_connections 2024-05-17T22:49:52.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.830+0000 7fea78c3fb80 1 --2- 172.21.15.12:0/2598312400 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55e88e30b0a0 0x55e88e30d4c0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:52.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.830+0000 7fea78c3fb80 1 --2- 172.21.15.12:0/2598312400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e88e28dcb0 0x55e88e299a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:52.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.830+0000 7fea78c3fb80 1 --2- 172.21.15.12:0/2598312400 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e88e1099f0 0x55e88e2a8f70 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:52.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.830+0000 7fea78c3fb80 1 --2- 172.21.15.12:0/2598312400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e88e288170 0x55e88e2994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:52.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.831+0000 7fea78c3fb80 1 -- 172.21.15.12:0/2598312400 >> 172.21.15.12:0/2598312400 conn(0x55e88e14df00 msgr2=0x55e88e28cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:49:52.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.831+0000 7fea437fe700 0 client.25867 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:49:52.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.831+0000 7fea78c3fb80 1 -- 172.21.15.12:0/2598312400 shutdown_connections 2024-05-17T22:49:52.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.831+0000 7fea78c3fb80 1 -- 172.21.15.12:0/2598312400 wait complete. 2024-05-17T22:49:52.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.831+0000 7fea78c3fb80 20 client.25867 trim_cache size 0 max 16384 2024-05-17T22:49:52.839 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:49:52.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.831+0000 7fea69002700 1 -- 172.21.15.12:0/2798568904 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fea4c0b2a50 msgr2=0x7fea4c0b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:52.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.831+0000 7fea69002700 1 --2- 172.21.15.12:0/2798568904 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fea4c0b2a50 0x7fea4c0b4f60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x55e88e28d240 tx=0x7fea58008040 comp rx=0 tx=0).stop 2024-05-17T22:49:52.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.831+0000 7fea69002700 1 -- 172.21.15.12:0/2798568904 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea6406c580 msgr2=0x7fea6406ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:49:52.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.831+0000 7fea69002700 1 --2- 172.21.15.12:0/2798568904 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea6406c580 0x7fea6406ca50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fea5c00d010 tx=0x7fea5c004da0 comp rx=0 tx=0).stop 2024-05-17T22:49:52.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.832+0000 7fea69002700 1 -- 172.21.15.12:0/2798568904 shutdown_connections 2024-05-17T22:49:52.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.832+0000 7fea69002700 1 --2- 172.21.15.12:0/2798568904 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fea4c0b2a50 0x7fea4c0b4f60 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:52.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.832+0000 7fea69002700 1 --2- 172.21.15.12:0/2798568904 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea6406c580 0x7fea6406ca50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:52.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.832+0000 7fea69002700 1 --2- 172.21.15.12:0/2798568904 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea641032a0 0x7fea640754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:52.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.832+0000 7fea69002700 1 --2- 172.21.15.12:0/2798568904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea64108980 0x7fea6406c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:49:52.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.832+0000 7fea69002700 1 -- 172.21.15.12:0/2798568904 >> 172.21.15.12:0/2798568904 conn(0x7fea640fe680 msgr2=0x7fea64100aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:49:52.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.832+0000 7fea69002700 1 -- 172.21.15.12:0/2798568904 shutdown_connections 2024-05-17T22:49:52.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:49:52.832+0000 7fea69002700 1 -- 172.21.15.12:0/2798568904 wait complete. 2024-05-17T22:49:54.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:49:54 smithi151 ceph-mon[36101]: pgmap v758: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.4 MiB/s wr, 430 op/s 2024-05-17T22:49:54.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:49:54 smithi096 ceph-mon[36065]: pgmap v758: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.4 MiB/s wr, 430 op/s 2024-05-17T22:49:54.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:49:54 smithi012 ceph-mon[31632]: pgmap v758: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.4 MiB/s wr, 430 op/s 2024-05-17T22:49:56.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 635.0 s, 389.7 tps, lat 245.150 ms stddev 289.897 2024-05-17T22:49:56.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:49:56 smithi151 ceph-mon[36101]: pgmap v759: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.3 MiB/s wr, 497 op/s 2024-05-17T22:49:56.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:49:56 smithi096 ceph-mon[36065]: pgmap v759: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.3 MiB/s wr, 497 op/s 2024-05-17T22:49:56.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:49:56 smithi012 ceph-mon[31632]: pgmap v759: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.3 MiB/s wr, 497 op/s 2024-05-17T22:49:57.372 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-17T22:49:57.372 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-17T22:49:57.899 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:49:58.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:49:58 smithi012 ceph-mon[31632]: pgmap v760: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.9 MiB/s wr, 422 op/s 2024-05-17T22:49:58.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:49:58 smithi151 ceph-mon[36101]: pgmap v760: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.9 MiB/s wr, 422 op/s 2024-05-17T22:49:58.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:49:58 smithi096 ceph-mon[36065]: pgmap v760: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.9 MiB/s wr, 422 op/s 2024-05-17T22:50:00.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:00 smithi151 ceph-mon[36101]: pgmap v761: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.7 MiB/s wr, 473 op/s 2024-05-17T22:50:00.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:00 smithi151 ceph-mon[36101]: overall HEALTH_OK 2024-05-17T22:50:00.505 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:00 smithi012 ceph-mon[31632]: pgmap v761: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.7 MiB/s wr, 473 op/s 2024-05-17T22:50:00.505 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:00 smithi012 ceph-mon[31632]: overall HEALTH_OK 2024-05-17T22:50:00.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:00 smithi096 ceph-mon[36065]: pgmap v761: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.7 MiB/s wr, 473 op/s 2024-05-17T22:50:00.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:00 smithi096 ceph-mon[36065]: overall HEALTH_OK 2024-05-17T22:50:00.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.743+0000 7f5bb859f700 1 -- 172.21.15.12:0/2331816958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bb010a830 msgr2=0x7f5bb010ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:00.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.743+0000 7f5bb859f700 1 --2- 172.21.15.12:0/2331816958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bb010a830 0x7f5bb010ac40 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5bac009990 tx=0x7f5bac009ca0 comp rx=0 tx=0).stop 2024-05-17T22:50:00.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.744+0000 7f5bb859f700 1 -- 172.21.15.12:0/2331816958 shutdown_connections 2024-05-17T22:50:00.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.744+0000 7f5bb859f700 1 --2- 172.21.15.12:0/2331816958 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bb011d550 0x7f5bb011f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:00.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.744+0000 7f5bb859f700 1 --2- 172.21.15.12:0/2331816958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bb010a830 0x7f5bb010ac40 secure :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5bac009990 tx=0x7f5bac009ca0 comp rx=0 tx=0).stop 2024-05-17T22:50:00.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.744+0000 7f5bb859f700 1 --2- 172.21.15.12:0/2331816958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bb010b180 0x7f5bb011d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:00.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.744+0000 7f5bb859f700 1 -- 172.21.15.12:0/2331816958 >> 172.21.15.12:0/2331816958 conn(0x7f5bb006efc0 msgr2=0x7f5bb006f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:50:00.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.744+0000 7f5bb859f700 1 -- 172.21.15.12:0/2331816958 shutdown_connections 2024-05-17T22:50:00.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.749+0000 7f5bb859f700 1 -- 172.21.15.12:0/2331816958 wait complete. 2024-05-17T22:50:00.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.750+0000 7f5bb859f700 1 Processor -- start 2024-05-17T22:50:00.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.750+0000 7f5bb859f700 1 -- start start 2024-05-17T22:50:00.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.750+0000 7f5bb859f700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bb011d550 0x7f5bb01af6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:00.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.750+0000 7f5bb859f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bb01afbe0 0x7f5bb01a9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:00.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.750+0000 7f5bb859f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bb01a9d60 0x7f5bb01aa170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:00.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.750+0000 7f5bb859f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb0122dd0 con 0x7f5bb01afbe0 2024-05-17T22:50:00.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.750+0000 7f5bb859f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb0122c50 con 0x7f5bb01a9d60 2024-05-17T22:50:00.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.750+0000 7f5bb859f700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb0122f50 con 0x7f5bb011d550 2024-05-17T22:50:00.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.752+0000 7f5bb6b3c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bb01a9d60 0x7f5bb01aa170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:00.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.752+0000 7f5bb633b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bb011d550 0x7f5bb01af6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:00.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.752+0000 7f5bb633b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bb011d550 0x7f5bb01af6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:45408/0 (socket says 172.21.15.12:45408) 2024-05-17T22:50:00.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.752+0000 7f5bb633b700 1 -- 172.21.15.12:0/1251158971 learned_addr learned my addr 172.21.15.12:0/1251158971 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:50:00.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.757+0000 7f5bb5b3a700 1 --2- 172.21.15.12:0/1251158971 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bb01afbe0 0x7f5bb01a9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:00.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.762+0000 7f5bb633b700 1 -- 172.21.15.12:0/1251158971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bb01a9d60 msgr2=0x7f5bb01aa170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:00.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.762+0000 7f5bb633b700 1 --2- 172.21.15.12:0/1251158971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bb01a9d60 0x7f5bb01aa170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:00.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.762+0000 7f5bb633b700 1 -- 172.21.15.12:0/1251158971 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bb01afbe0 msgr2=0x7f5bb01a9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:00.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.762+0000 7f5bb633b700 1 --2- 172.21.15.12:0/1251158971 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bb01afbe0 0x7f5bb01a9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:00.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.762+0000 7f5bb633b700 1 -- 172.21.15.12:0/1251158971 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb0112870 con 0x7f5bb011d550 2024-05-17T22:50:00.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.771+0000 7f5bb633b700 1 --2- 172.21.15.12:0/1251158971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bb011d550 0x7f5bb01af6a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5bac000c00 tx=0x7f5bac009f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:50:00.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.775+0000 7f5ba77fe700 1 -- 172.21.15.12:0/1251158971 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac026070 con 0x7f5bb011d550 2024-05-17T22:50:00.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.775+0000 7f5bb859f700 1 -- 172.21.15.12:0/1251158971 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bb0112b00 con 0x7f5bb011d550 2024-05-17T22:50:00.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.775+0000 7f5bb859f700 1 -- 172.21.15.12:0/1251158971 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bb0112f90 con 0x7f5bb011d550 2024-05-17T22:50:00.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.775+0000 7f5ba77fe700 1 -- 172.21.15.12:0/1251158971 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bac003f20 con 0x7f5bb011d550 2024-05-17T22:50:00.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.776+0000 7f5ba77fe700 1 -- 172.21.15.12:0/1251158971 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac00fb80 con 0x7f5bb011d550 2024-05-17T22:50:00.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.785+0000 7f5ba77fe700 1 -- 172.21.15.12:0/1251158971 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5bac029070 con 0x7f5bb011d550 2024-05-17T22:50:00.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.786+0000 7f5ba77fe700 1 --2- 172.21.15.12:0/1251158971 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5b9c0b29c0 0x7f5b9c0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:00.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.786+0000 7f5ba77fe700 1 -- 172.21.15.12:0/1251158971 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f5bac0e3ee0 con 0x7f5bb011d550 2024-05-17T22:50:00.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.791+0000 7f5bb5b3a700 1 --2- 172.21.15.12:0/1251158971 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5b9c0b29c0 0x7f5b9c0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:00.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.847+0000 7f5bb5b3a700 1 --2- 172.21.15.12:0/1251158971 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5b9c0b29c0 0x7f5b9c0b4ed0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5bb01117b0 tx=0x7f5ba0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:50:00.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.849+0000 7f5bc6f7ab80 1 Processor -- start 2024-05-17T22:50:00.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.850+0000 7f5bc6f7ab80 1 -- start start 2024-05-17T22:50:00.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.850+0000 7f5bc6f7ab80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587c2d15a70 0x5587c2e995e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:00.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.850+0000 7f5bc6f7ab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587c2e99b20 0x5587c2e93cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:00.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.850+0000 7f5bc6f7ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c2e941f0 0x5587c2e94600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:00.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.852+0000 7f5bb633b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587c2d15a70 0x5587c2e995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:00.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.852+0000 7f5bb633b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587c2d15a70 0x5587c2e995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:60570/0 (socket says 172.21.15.12:60570) 2024-05-17T22:50:00.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.852+0000 7f5bc6f7ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5587c2dc2310 con 0x5587c2e941f0 2024-05-17T22:50:00.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.852+0000 7f5bc6f7ab80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5587c2db2050 con 0x5587c2d15a70 2024-05-17T22:50:00.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.852+0000 7f5bc6f7ab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5587c2db5ef0 con 0x5587c2e99b20 2024-05-17T22:50:00.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.852+0000 7f5bb633b700 1 -- 172.21.15.12:0/882162536 learned_addr learned my addr 172.21.15.12:0/882162536 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:50:00.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.853+0000 7f5bb633b700 1 -- 172.21.15.12:0/882162536 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587c2e99b20 msgr2=0x5587c2e93cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T22:50:00.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.853+0000 7f5bb633b700 1 --2- 172.21.15.12:0/882162536 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587c2e99b20 0x5587c2e93cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:00.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.853+0000 7f5bb633b700 1 -- 172.21.15.12:0/882162536 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c2e941f0 msgr2=0x5587c2e94600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:00.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.853+0000 7f5bb633b700 1 --2- 172.21.15.12:0/882162536 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c2e941f0 0x5587c2e94600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:00.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.853+0000 7f5bb633b700 1 -- 172.21.15.12:0/882162536 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587c2d5b610 con 0x5587c2d15a70 2024-05-17T22:50:00.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.854+0000 7f5bb633b700 1 --2- 172.21.15.12:0/882162536 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587c2d15a70 0x5587c2e995e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5bac00b5c0 tx=0x7f5bac0aabe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc51f540e033f626 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:50:00.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.854+0000 7f5ba4ff9700 1 -- 172.21.15.12:0/882162536 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac0ab8a0 con 0x5587c2d15a70 2024-05-17T22:50:00.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.855+0000 7f5ba4ff9700 1 -- 172.21.15.12:0/882162536 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bac0ab1a0 con 0x5587c2d15a70 2024-05-17T22:50:00.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.855+0000 7f5ba4ff9700 1 -- 172.21.15.12:0/882162536 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac0053a0 con 0x5587c2d15a70 2024-05-17T22:50:00.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.856+0000 7f5bc6f7ab80 1 -- 172.21.15.12:0/882162536 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587c2d15a70 msgr2=0x5587c2e995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:00.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.856+0000 7f5bc6f7ab80 1 --2- 172.21.15.12:0/882162536 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587c2d15a70 0x5587c2e995e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5bac00b5c0 tx=0x7f5bac0aabe0 comp rx=0 tx=0).stop 2024-05-17T22:50:00.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.856+0000 7f5bc6f7ab80 1 -- 172.21.15.12:0/882162536 shutdown_connections 2024-05-17T22:50:00.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.856+0000 7f5bc6f7ab80 1 --2- 172.21.15.12:0/882162536 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c2e941f0 0x5587c2e94600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:00.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.856+0000 7f5bc6f7ab80 1 --2- 172.21.15.12:0/882162536 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587c2d15a70 0x5587c2e995e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:00.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.856+0000 7f5bc6f7ab80 1 --2- 172.21.15.12:0/882162536 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587c2e99b20 0x5587c2e93cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:00.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.856+0000 7f5bc6f7ab80 1 -- 172.21.15.12:0/882162536 >> 172.21.15.12:0/882162536 conn(0x5587c2d59f80 msgr2=0x5587c2e8ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:50:00.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.856+0000 7f5bc6f7ab80 1 -- 172.21.15.12:0/882162536 shutdown_connections 2024-05-17T22:50:00.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.856+0000 7f5bc6f7ab80 1 -- 172.21.15.12:0/882162536 wait complete. 2024-05-17T22:50:00.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.857+0000 7f5bc6f7ab80 1 -- start start 2024-05-17T22:50:00.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.857+0000 7f5bc6f7ab80 1 Processor -- start 2024-05-17T22:50:00.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.858+0000 7f5bc6f7ab80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587c2d15a70 0x5587c2e9d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:00.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.858+0000 7f5bc6f7ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c2e941f0 0x5587c2e9d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:00.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.858+0000 7f5bc6f7ab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587c2e99b20 0x5587c2e9ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:00.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.858+0000 7f5bc6f7ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5587c2cb91d0 con 0x5587c2e941f0 2024-05-17T22:50:00.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.858+0000 7f5bc6f7ab80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5587c2d52f00 con 0x5587c2d15a70 2024-05-17T22:50:00.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.858+0000 7f5bb633b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587c2d15a70 0x5587c2e9d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:00.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.858+0000 7f5bb5b3a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c2e941f0 0x5587c2e9d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:00.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.858+0000 7f5bb633b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587c2d15a70 0x5587c2e9d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:60586/0 (socket says 172.21.15.12:60586) 2024-05-17T22:50:00.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.858+0000 7f5bb633b700 1 -- 172.21.15.12:0/3670126158 learned_addr learned my addr 172.21.15.12:0/3670126158 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:50:00.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.858+0000 7f5bc6f7ab80 1 -- 172.21.15.12:0/3670126158 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5587c2db2050 con 0x5587c2e99b20 2024-05-17T22:50:00.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.858+0000 7f5bc6f7ab80 10 client.0 _ll_register_callbacks cb 0x5587c2d4cf90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:50:00.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.858+0000 7f5bb6b3c700 1 --2- 172.21.15.12:0/3670126158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587c2e99b20 0x5587c2e9ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:00.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.859+0000 7f5bb6b3c700 1 -- 172.21.15.12:0/3670126158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587c2d15a70 msgr2=0x5587c2e9d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:00.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.859+0000 7f5bb6b3c700 1 --2- 172.21.15.12:0/3670126158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587c2d15a70 0x5587c2e9d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:00.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.859+0000 7f5bb6b3c700 1 -- 172.21.15.12:0/3670126158 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c2e941f0 msgr2=0x5587c2e9d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:00.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.859+0000 7f5bb6b3c700 1 --2- 172.21.15.12:0/3670126158 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c2e941f0 0x5587c2e9d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:00.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.859+0000 7f5bb6b3c700 1 -- 172.21.15.12:0/3670126158 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587c2d02500 con 0x5587c2e99b20 2024-05-17T22:50:00.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.859+0000 7f5bb5b3a700 1 --2- 172.21.15.12:0/3670126158 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c2e941f0 0x5587c2e9d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:50:00.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.859+0000 7f5bb6b3c700 1 --2- 172.21.15.12:0/3670126158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587c2e99b20 0x5587c2e9ddd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5ba800c720 tx=0x7f5ba800c750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea38689011d3ac03 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:50:00.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.859+0000 7f5b99ffb700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:50:00.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.860+0000 7f5b99ffb700 1 -- 172.21.15.12:0/3670126158 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba801a060 con 0x5587c2e99b20 2024-05-17T22:50:00.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.860+0000 7f5b99ffb700 1 -- 172.21.15.12:0/3670126158 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f5ba8015410 con 0x5587c2e99b20 2024-05-17T22:50:00.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.860+0000 7f5bc6f7ab80 1 -- 172.21.15.12:0/3670126158 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bac0ab1a0 con 0x5587c2e99b20 2024-05-17T22:50:00.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.860+0000 7f5b99ffb700 1 -- 172.21.15.12:0/3670126158 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5ba8021050 con 0x5587c2e99b20 2024-05-17T22:50:00.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.860+0000 7f5bc6f7ab80 10 client.25873 fetch_fsmap learned FSMap version 39 2024-05-17T22:50:00.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.860+0000 7f5bc6f7ab80 1 -- 172.21.15.12:0/3670126158 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5587c2eaaad0 con 0x5587c2e99b20 2024-05-17T22:50:00.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.861+0000 7f5b99ffb700 1 -- 172.21.15.12:0/3670126158 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ba8015ac0 con 0x5587c2e99b20 2024-05-17T22:50:00.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.861+0000 7f5b99ffb700 1 -- 172.21.15.12:0/3670126158 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f5ba800a650 con 0x5587c2e99b20 2024-05-17T22:50:00.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.861+0000 7f5bc6f7ab80 10 client.25873 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:50:00.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.861+0000 7f5bc6f7ab80 10 client.25873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:50:00.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.861+0000 7f5bc6f7ab80 20 client.25873 populate_metadata read hostname 'smithi012' 2024-05-17T22:50:00.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.861+0000 7f5bc6f7ab80 1 --2- 172.21.15.12:0/3670126158 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5587c2ead5d0 0x5587c2eafa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:00.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.861+0000 7f5bc6f7ab80 4 client.25873 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:50:00.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.861+0000 7f5bc6f7ab80 1 -- 172.21.15.12:0/3670126158 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5bac0ab1a0 con 0x5587c2ead5d0 2024-05-17T22:50:00.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.861+0000 7f5bb633b700 1 --2- 172.21.15.12:0/3670126158 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5587c2ead5d0 0x5587c2eafa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:00.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.862+0000 7f5b99ffb700 10 client.25873 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:50:00.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.862+0000 7f5bb633b700 1 --2- 172.21.15.12:0/3670126158 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5587c2ead5d0 0x5587c2eafa20 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fbe634399d6ed0f server_cookie=19eb4a82e5273e0 in_seq=0 out_seq=0 2024-05-17T22:50:00.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.862+0000 7f5b99ffb700 1 -- 172.21.15.12:0/3670126158 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f5bac0ab1a0 con 0x5587c2ead5d0 2024-05-17T22:50:00.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.862+0000 7f5b99ffb700 10 client.25873 handle_command_reply: tid=0 2024-05-17T22:50:00.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.862+0000 7f5bc6f7ab80 1 client.25873 shutdown 2024-05-17T22:50:00.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.863+0000 7f5bc6f7ab80 1 -- 172.21.15.12:0/3670126158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587c2e99b20 msgr2=0x5587c2e9ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:00.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.863+0000 7f5bc6f7ab80 1 --2- 172.21.15.12:0/3670126158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587c2e99b20 0x5587c2e9ddd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5ba800c720 tx=0x7f5ba800c750 comp rx=0 tx=0).stop 2024-05-17T22:50:00.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.863+0000 7f5bc6f7ab80 1 -- 172.21.15.12:0/3670126158 shutdown_connections 2024-05-17T22:50:00.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.863+0000 7f5bc6f7ab80 1 --2- 172.21.15.12:0/3670126158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587c2e99b20 0x5587c2e9ddd0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:00.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.863+0000 7f5bc6f7ab80 1 --2- 172.21.15.12:0/3670126158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587c2d15a70 0x5587c2e9d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:00.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.863+0000 7f5bc6f7ab80 1 --2- 172.21.15.12:0/3670126158 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5587c2ead5d0 0x5587c2eafa20 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:00.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.863+0000 7f5bc6f7ab80 1 --2- 172.21.15.12:0/3670126158 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c2e941f0 0x5587c2e9d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:00.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.863+0000 7f5bc6f7ab80 1 -- 172.21.15.12:0/3670126158 >> 172.21.15.12:0/3670126158 conn(0x5587c2d59f80 msgr2=0x5587c2e88f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:50:00.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.863+0000 7f5b99ffb700 0 client.25873 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:50:00.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.863+0000 7f5bc6f7ab80 1 -- 172.21.15.12:0/3670126158 shutdown_connections 2024-05-17T22:50:00.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.863+0000 7f5bc6f7ab80 1 -- 172.21.15.12:0/3670126158 wait complete. 2024-05-17T22:50:00.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.863+0000 7f5bc6f7ab80 20 client.25873 trim_cache size 0 max 16384 2024-05-17T22:50:00.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.889+0000 7f5bc6f7ab80 1 Processor -- start 2024-05-17T22:50:00.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.890+0000 7f5bc6f7ab80 1 -- start start 2024-05-17T22:50:00.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.890+0000 7f5bc6f7ab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587c2d15a70 0x5587c2e93cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:00.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.890+0000 7f5bc6f7ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c2e941f0 0x5587c2e9dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:00.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.890+0000 7f5bc6f7ab80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587c2e99b20 0x5587c2e9a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:00.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.890+0000 7f5bc6f7ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5b94011380 con 0x5587c2e941f0 2024-05-17T22:50:00.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.890+0000 7f5bc6f7ab80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9400ec10 con 0x5587c2e99b20 2024-05-17T22:50:00.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.890+0000 7f5bc6f7ab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9400ca60 con 0x5587c2d15a70 2024-05-17T22:50:00.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.890+0000 7f5bb6b3c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587c2e99b20 0x5587c2e9a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:00.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.890+0000 7f5bb6b3c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587c2e99b20 0x5587c2e9a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:60588/0 (socket says 172.21.15.12:60588) 2024-05-17T22:50:00.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.890+0000 7f5bb6b3c700 1 -- 172.21.15.12:0/3818074736 learned_addr learned my addr 172.21.15.12:0/3818074736 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:50:00.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.890+0000 7f5bb633b700 1 --2- 172.21.15.12:0/3818074736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587c2d15a70 0x5587c2e93cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:00.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.891+0000 7f5bb5b3a700 1 --2- 172.21.15.12:0/3818074736 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c2e941f0 0x5587c2e9dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:00.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.891+0000 7f5bb633b700 1 -- 172.21.15.12:0/3818074736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587c2e99b20 msgr2=0x5587c2e9a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:00.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.891+0000 7f5bb633b700 1 --2- 172.21.15.12:0/3818074736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587c2e99b20 0x5587c2e9a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:00.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.891+0000 7f5bb633b700 1 -- 172.21.15.12:0/3818074736 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c2e941f0 msgr2=0x5587c2e9dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:00.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.891+0000 7f5bb633b700 1 --2- 172.21.15.12:0/3818074736 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c2e941f0 0x5587c2e9dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:00.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.891+0000 7f5bb633b700 1 -- 172.21.15.12:0/3818074736 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587c2e9e440 con 0x5587c2d15a70 2024-05-17T22:50:00.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.891+0000 7f5bb6b3c700 1 --2- 172.21.15.12:0/3818074736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587c2e99b20 0x5587c2e9a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:50:00.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.891+0000 7f5bb5b3a700 1 --2- 172.21.15.12:0/3818074736 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c2e941f0 0x5587c2e9dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:50:00.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.891+0000 7f5bb633b700 1 --2- 172.21.15.12:0/3818074736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587c2d15a70 0x5587c2e93cb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5bac001eb0 tx=0x7f5bac009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35efa688b97ef93a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:50:00.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.891+0000 7f5b9bfff700 1 -- 172.21.15.12:0/3818074736 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac0c4070 con 0x5587c2d15a70 2024-05-17T22:50:00.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.891+0000 7f5b9bfff700 1 -- 172.21.15.12:0/3818074736 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bac0ab8a0 con 0x5587c2d15a70 2024-05-17T22:50:00.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.891+0000 7f5b9bfff700 1 -- 172.21.15.12:0/3818074736 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac0b3d40 con 0x5587c2d15a70 2024-05-17T22:50:00.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.892+0000 7f5bc6f7ab80 1 -- 172.21.15.12:0/3818074736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587c2d15a70 msgr2=0x5587c2e93cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:00.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.892+0000 7f5bc6f7ab80 1 --2- 172.21.15.12:0/3818074736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587c2d15a70 0x5587c2e93cb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5bac001eb0 tx=0x7f5bac009ca0 comp rx=0 tx=0).stop 2024-05-17T22:50:00.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.892+0000 7f5bc6f7ab80 1 -- 172.21.15.12:0/3818074736 shutdown_connections 2024-05-17T22:50:00.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.892+0000 7f5bc6f7ab80 1 --2- 172.21.15.12:0/3818074736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587c2e99b20 0x5587c2e9a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:00.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.892+0000 7f5bc6f7ab80 1 --2- 172.21.15.12:0/3818074736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587c2d15a70 0x5587c2e93cb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:00.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.892+0000 7f5bc6f7ab80 1 --2- 172.21.15.12:0/3818074736 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c2e941f0 0x5587c2e9dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:00.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.892+0000 7f5bc6f7ab80 1 -- 172.21.15.12:0/3818074736 >> 172.21.15.12:0/3818074736 conn(0x5587c2d59f80 msgr2=0x5587c2e9a280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:50:00.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.893+0000 7f5bc6f7ab80 1 -- 172.21.15.12:0/3818074736 shutdown_connections 2024-05-17T22:50:00.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.893+0000 7f5bc6f7ab80 1 -- 172.21.15.12:0/3818074736 wait complete. 2024-05-17T22:50:00.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.893+0000 7f5bc6f7ab80 1 -- start start 2024-05-17T22:50:00.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.894+0000 7f5bc6f7ab80 1 Processor -- start 2024-05-17T22:50:00.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.894+0000 7f5bc6f7ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c2d15a70 0x5587c2ea2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:00.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.894+0000 7f5bc6f7ab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587c2e941f0 0x5587c2ea3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:00.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.894+0000 7f5bc6f7ab80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587c2e99b20 0x5587c2f17d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:00.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.894+0000 7f5bc6f7ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9400d0d0 con 0x5587c2d15a70 2024-05-17T22:50:00.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.894+0000 7f5bc6f7ab80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9400e540 con 0x5587c2e99b20 2024-05-17T22:50:00.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.894+0000 7f5bc6f7ab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9400d7a0 con 0x5587c2e941f0 2024-05-17T22:50:00.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.894+0000 7f5bc6f7ab80 10 client.0 _ll_register_callbacks cb 0x5587c2d4cf90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:50:00.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.895+0000 7f5bb6b3c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587c2e99b20 0x5587c2f17d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:00.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.895+0000 7f5bb6b3c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587c2e99b20 0x5587c2f17d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:60602/0 (socket says 172.21.15.12:60602) 2024-05-17T22:50:00.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.895+0000 7f5bb6b3c700 1 -- 172.21.15.12:0/2015865389 learned_addr learned my addr 172.21.15.12:0/2015865389 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:50:00.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.895+0000 7f5bb5b3a700 1 --2- 172.21.15.12:0/2015865389 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587c2e941f0 0x5587c2ea3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:00.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.895+0000 7f5bb6b3c700 1 -- 172.21.15.12:0/2015865389 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587c2e941f0 msgr2=0x5587c2ea3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:00.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.895+0000 7f5bb6b3c700 1 --2- 172.21.15.12:0/2015865389 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587c2e941f0 0x5587c2ea3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:00.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.895+0000 7f5bb6b3c700 1 -- 172.21.15.12:0/2015865389 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c2d15a70 msgr2=0x5587c2ea2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:00.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.895+0000 7f5bb6b3c700 1 --2- 172.21.15.12:0/2015865389 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c2d15a70 0x5587c2ea2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:00.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.895+0000 7f5bb6b3c700 1 -- 172.21.15.12:0/2015865389 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587c2f182d0 con 0x5587c2e99b20 2024-05-17T22:50:00.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.895+0000 7f5bb6b3c700 1 --2- 172.21.15.12:0/2015865389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587c2e99b20 0x5587c2f17d90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8018040 tx=0x7f5ba80099b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3c2cdb2c5952663 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:50:00.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.895+0000 7f5b99ffb700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:50:00.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.895+0000 7f5b99ffb700 1 -- 172.21.15.12:0/2015865389 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba8029070 con 0x5587c2e99b20 2024-05-17T22:50:00.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.895+0000 7f5b99ffb700 1 -- 172.21.15.12:0/2015865389 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f5ba8003aa0 con 0x5587c2e99b20 2024-05-17T22:50:00.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.895+0000 7f5bc6f7ab80 1 -- 172.21.15.12:0/2015865389 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bac0ab8a0 con 0x5587c2e99b20 2024-05-17T22:50:00.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.896+0000 7f5b99ffb700 1 -- 172.21.15.12:0/2015865389 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5ba8035050 con 0x5587c2e99b20 2024-05-17T22:50:00.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.896+0000 7f5bc6f7ab80 10 client.25952 fetch_fsmap learned FSMap version 39 2024-05-17T22:50:00.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.896+0000 7f5bc6f7ab80 1 -- 172.21.15.12:0/2015865389 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5587c2f19400 con 0x5587c2e99b20 2024-05-17T22:50:00.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.896+0000 7f5b99ffb700 1 -- 172.21.15.12:0/2015865389 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bac0ab8a0 con 0x5587c2e99b20 2024-05-17T22:50:00.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.896+0000 7f5b99ffb700 1 -- 172.21.15.12:0/2015865389 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f5ba8007a10 con 0x5587c2e99b20 2024-05-17T22:50:00.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.896+0000 7f5bc6f7ab80 10 client.25952 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:50:00.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.896+0000 7f5bc6f7ab80 10 client.25952 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:50:00.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.896+0000 7f5bc6f7ab80 20 client.25952 populate_metadata read hostname 'smithi012' 2024-05-17T22:50:00.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.896+0000 7f5bc6f7ab80 1 --2- 172.21.15.12:0/2015865389 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5587c2f19ab0 0x5587c2f1bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:00.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.896+0000 7f5bc6f7ab80 4 client.25952 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-17T22:50:00.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.897+0000 7f5bb633b700 1 --2- 172.21.15.12:0/2015865389 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5587c2f19ab0 0x5587c2f1bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:00.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.897+0000 7f5bc6f7ab80 1 -- 172.21.15.12:0/2015865389 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5587c2dc4260 con 0x5587c2f19ab0 2024-05-17T22:50:00.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.897+0000 7f5b99ffb700 10 client.25952 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:50:00.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.897+0000 7f5bb633b700 1 --2- 172.21.15.12:0/2015865389 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5587c2f19ab0 0x5587c2f1bed0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f17005501d1d42ea server_cookie=16953368606fae86 in_seq=0 out_seq=0 2024-05-17T22:50:00.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.898+0000 7f5b99ffb700 1 -- 172.21.15.12:0/2015865389 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5587c2dc4260 con 0x5587c2f19ab0 2024-05-17T22:50:00.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.898+0000 7f5b99ffb700 10 client.25952 handle_command_reply: tid=0 2024-05-17T22:50:00.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.898+0000 7f5bc6f7ab80 1 client.25952 shutdown 2024-05-17T22:50:00.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.898+0000 7f5bc6f7ab80 1 -- 172.21.15.12:0/2015865389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587c2e99b20 msgr2=0x5587c2f17d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:00.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.898+0000 7f5bc6f7ab80 1 --2- 172.21.15.12:0/2015865389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587c2e99b20 0x5587c2f17d90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8018040 tx=0x7f5ba80099b0 comp rx=0 tx=0).stop 2024-05-17T22:50:00.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.899+0000 7f5bc6f7ab80 1 -- 172.21.15.12:0/2015865389 shutdown_connections 2024-05-17T22:50:00.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.899+0000 7f5bc6f7ab80 1 --2- 172.21.15.12:0/2015865389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587c2e99b20 0x5587c2f17d90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:00.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.899+0000 7f5bc6f7ab80 1 --2- 172.21.15.12:0/2015865389 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5587c2f19ab0 0x5587c2f1bed0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:00.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.899+0000 7f5b99ffb700 0 client.25952 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:50:00.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.899+0000 7f5bc6f7ab80 1 --2- 172.21.15.12:0/2015865389 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587c2d15a70 0x5587c2ea2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:00.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.899+0000 7f5bc6f7ab80 1 --2- 172.21.15.12:0/2015865389 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587c2e941f0 0x5587c2ea3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:00.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.899+0000 7f5bc6f7ab80 1 -- 172.21.15.12:0/2015865389 >> 172.21.15.12:0/2015865389 conn(0x5587c2d59f80 msgr2=0x5587c2e9aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:50:00.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.899+0000 7f5bc6f7ab80 1 -- 172.21.15.12:0/2015865389 shutdown_connections 2024-05-17T22:50:00.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.899+0000 7f5bc6f7ab80 1 -- 172.21.15.12:0/2015865389 wait complete. 2024-05-17T22:50:00.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.899+0000 7f5bc6f7ab80 20 client.25952 trim_cache size 0 max 16384 2024-05-17T22:50:00.907 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:50:00.907 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-17T22:50:00.907 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "2e29f53e-3c74-4a52-850d-542951802ba4", 2024-05-17T22:50:00.907 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-17T22:50:00.907 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:50:00.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.900+0000 7f5bb859f700 1 -- 172.21.15.12:0/1251158971 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5b9c0b29c0 msgr2=0x7f5b9c0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:00.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.900+0000 7f5bb859f700 1 --2- 172.21.15.12:0/1251158971 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5b9c0b29c0 0x7f5b9c0b4ed0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5bb01117b0 tx=0x7f5ba0008040 comp rx=0 tx=0).stop 2024-05-17T22:50:00.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.900+0000 7f5bb859f700 1 -- 172.21.15.12:0/1251158971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bb011d550 msgr2=0x7f5bb01af6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:00.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.900+0000 7f5bb859f700 1 --2- 172.21.15.12:0/1251158971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bb011d550 0x7f5bb01af6a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5bac000c00 tx=0x7f5bac009f70 comp rx=0 tx=0).stop 2024-05-17T22:50:00.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.900+0000 7f5bb859f700 1 -- 172.21.15.12:0/1251158971 shutdown_connections 2024-05-17T22:50:00.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.900+0000 7f5bb859f700 1 --2- 172.21.15.12:0/1251158971 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5b9c0b29c0 0x7f5b9c0b4ed0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:00.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.900+0000 7f5bb859f700 1 --2- 172.21.15.12:0/1251158971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bb01a9d60 0x7f5bb01aa170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:00.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.900+0000 7f5bb859f700 1 --2- 172.21.15.12:0/1251158971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bb011d550 0x7f5bb01af6a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:00.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.900+0000 7f5bb859f700 1 --2- 172.21.15.12:0/1251158971 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bb01afbe0 0x7f5bb01a9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:00.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.900+0000 7f5bb859f700 1 -- 172.21.15.12:0/1251158971 >> 172.21.15.12:0/1251158971 conn(0x7f5bb006efc0 msgr2=0x7f5bb011b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:50:00.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.900+0000 7f5bb859f700 1 -- 172.21.15.12:0/1251158971 shutdown_connections 2024-05-17T22:50:00.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:00.900+0000 7f5bb859f700 1 -- 172.21.15.12:0/1251158971 wait complete. 2024-05-17T22:50:01.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 640.0 s, 528.4 tps, lat 201.207 ms stddev 296.374 2024-05-17T22:50:01.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:01 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:50:01.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:01 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:50:01.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:01 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:50:01.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:01 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:50:01.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:01 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:50:01.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:01 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:50:01.549 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:01 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:50:01.550 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:01 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:50:01.550 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:01 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:50:01.596 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-17T22:50:02.116 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:50:02.470 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:02 smithi012 ceph-mon[31632]: pgmap v762: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.4 MiB/s wr, 441 op/s 2024-05-17T22:50:02.471 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:02 smithi012 ceph-mon[31632]: scrub queued for path: / 2024-05-17T22:50:02.471 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:02 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:50:02.471 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:02 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:50:02.471 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:02 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:50:02.471 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:02 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:50:02.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:02 smithi151 ceph-mon[36101]: pgmap v762: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.4 MiB/s wr, 441 op/s 2024-05-17T22:50:02.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:02 smithi151 ceph-mon[36101]: scrub queued for path: / 2024-05-17T22:50:02.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:02 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:50:02.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:02 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:50:02.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:02 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:50:02.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:02 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:50:02.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:02 smithi096 ceph-mon[36065]: pgmap v762: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.4 MiB/s wr, 441 op/s 2024-05-17T22:50:02.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:02 smithi096 ceph-mon[36065]: scrub queued for path: / 2024-05-17T22:50:02.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:02 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:50:02.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:02 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:50:02.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:02 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:50:02.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:02 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:50:04.482 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:04 smithi012 ceph-mon[31632]: pgmap v763: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.8 MiB/s wr, 439 op/s 2024-05-17T22:50:04.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:04 smithi151 ceph-mon[36101]: pgmap v763: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.8 MiB/s wr, 439 op/s 2024-05-17T22:50:04.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:04 smithi096 ceph-mon[36065]: pgmap v763: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.8 MiB/s wr, 439 op/s 2024-05-17T22:50:04.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.594+0000 7f13c37fe700 1 -- 172.21.15.12:0/3476777343 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c4012650 con 0x7f13cc1080a0 2024-05-17T22:50:04.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.595+0000 7f13d2d2d700 1 -- 172.21.15.12:0/3476777343 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13cc1080a0 msgr2=0x7f13cc10fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:04.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.595+0000 7f13d2d2d700 1 --2- 172.21.15.12:0/3476777343 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13cc1080a0 0x7f13cc10fa00 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f13c4009f90 tx=0x7f13c400b330 comp rx=0 tx=0).stop 2024-05-17T22:50:04.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.596+0000 7f13d2d2d700 1 -- 172.21.15.12:0/3476777343 shutdown_connections 2024-05-17T22:50:04.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.596+0000 7f13d2d2d700 1 --2- 172.21.15.12:0/3476777343 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13cc1080a0 0x7f13cc10fa00 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:04.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.596+0000 7f13d2d2d700 1 --2- 172.21.15.12:0/3476777343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13cc102e70 0x7f13cc103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:04.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.596+0000 7f13d2d2d700 1 --2- 172.21.15.12:0/3476777343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13cc107750 0x7f13cc107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:04.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.596+0000 7f13d2d2d700 1 -- 172.21.15.12:0/3476777343 >> 172.21.15.12:0/3476777343 conn(0x7f13cc079ae0 msgr2=0x7f13cc079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:50:04.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.596+0000 7f13d2d2d700 1 -- 172.21.15.12:0/3476777343 shutdown_connections 2024-05-17T22:50:04.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.596+0000 7f13d2d2d700 1 -- 172.21.15.12:0/3476777343 wait complete. 2024-05-17T22:50:04.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.596+0000 7f13d2d2d700 1 Processor -- start 2024-05-17T22:50:04.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.597+0000 7f13d2d2d700 1 -- start start 2024-05-17T22:50:04.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.597+0000 7f13d2d2d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13cc102e70 0x7f13cc1a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:04.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.597+0000 7f13d2d2d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13cc107750 0x7f13cc1a2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:04.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.597+0000 7f13d2d2d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13cc1080a0 0x7f13cc19c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:04.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.597+0000 7f13d2d2d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc111c40 con 0x7f13cc1080a0 2024-05-17T22:50:04.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.597+0000 7f13d2d2d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc111ac0 con 0x7f13cc107750 2024-05-17T22:50:04.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.597+0000 7f13d2d2d700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc111dc0 con 0x7f13cc102e70 2024-05-17T22:50:04.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.597+0000 7f13d0ac9700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13cc102e70 0x7f13cc1a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:04.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.597+0000 7f13d0ac9700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13cc102e70 0x7f13cc1a2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:45450/0 (socket says 172.21.15.12:45450) 2024-05-17T22:50:04.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.597+0000 7f13d12ca700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13cc1080a0 0x7f13cc19c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:04.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.597+0000 7f13d0ac9700 1 -- 172.21.15.12:0/3034531779 learned_addr learned my addr 172.21.15.12:0/3034531779 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:50:04.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.597+0000 7f13c3fff700 1 --2- 172.21.15.12:0/3034531779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13cc107750 0x7f13cc1a2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:04.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.598+0000 7f13d0ac9700 1 -- 172.21.15.12:0/3034531779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13cc107750 msgr2=0x7f13cc1a2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:04.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.598+0000 7f13d0ac9700 1 --2- 172.21.15.12:0/3034531779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13cc107750 0x7f13cc1a2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:04.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.598+0000 7f13d0ac9700 1 -- 172.21.15.12:0/3034531779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13cc1080a0 msgr2=0x7f13cc19c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:04.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.598+0000 7f13d0ac9700 1 --2- 172.21.15.12:0/3034531779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13cc1080a0 0x7f13cc19c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:04.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.598+0000 7f13d0ac9700 1 -- 172.21.15.12:0/3034531779 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13cc19ced0 con 0x7f13cc102e70 2024-05-17T22:50:04.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.598+0000 7f13d0ac9700 1 --2- 172.21.15.12:0/3034531779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13cc102e70 0x7f13cc1a2560 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f13c800e9c0 tx=0x7f13c800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:50:04.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.598+0000 7f13c1ffb700 1 -- 172.21.15.12:0/3034531779 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c8004a30 con 0x7f13cc102e70 2024-05-17T22:50:04.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.598+0000 7f13c1ffb700 1 -- 172.21.15.12:0/3034531779 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13c800cc70 con 0x7f13cc102e70 2024-05-17T22:50:04.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.598+0000 7f13d2d2d700 1 -- 172.21.15.12:0/3034531779 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13cc19d1c0 con 0x7f13cc102e70 2024-05-17T22:50:04.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.598+0000 7f13d2d2d700 1 -- 172.21.15.12:0/3034531779 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13cc1a93c0 con 0x7f13cc102e70 2024-05-17T22:50:04.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.598+0000 7f13c1ffb700 1 -- 172.21.15.12:0/3034531779 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c80054c0 con 0x7f13cc102e70 2024-05-17T22:50:04.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.602+0000 7f13c1ffb700 1 -- 172.21.15.12:0/3034531779 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f13c8018430 con 0x7f13cc102e70 2024-05-17T22:50:04.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.603+0000 7f13c1ffb700 1 --2- 172.21.15.12:0/3034531779 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f13b40b2960 0x7f13b40b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:04.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.604+0000 7f13c1ffb700 1 -- 172.21.15.12:0/3034531779 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f13c80e3640 con 0x7f13cc102e70 2024-05-17T22:50:04.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.604+0000 7f13c3fff700 1 --2- 172.21.15.12:0/3034531779 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f13b40b2960 0x7f13b40b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:04.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.606+0000 7f13c3fff700 1 --2- 172.21.15.12:0/3034531779 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f13b40b2960 0x7f13b40b4e70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f13cc104760 tx=0x7f13bc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:50:04.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.608+0000 7f13e1708b80 1 Processor -- start 2024-05-17T22:50:04.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.608+0000 7f13e1708b80 1 -- start start 2024-05-17T22:50:04.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.608+0000 7f13e1708b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55929259a9f0 0x55929271e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:04.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.608+0000 7f13e1708b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55929271eaa0 0x559292718c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:04.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.608+0000 7f13e1708b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559292719170 0x559292719580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:04.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.608+0000 7f13e1708b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x559292647290 con 0x559292719170 2024-05-17T22:50:04.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.608+0000 7f13e1708b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559292636fd0 con 0x55929271eaa0 2024-05-17T22:50:04.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.608+0000 7f13e1708b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55929263ae70 con 0x55929259a9f0 2024-05-17T22:50:04.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.609+0000 7f13d12ca700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559292719170 0x559292719580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:04.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.609+0000 7f13d12ca700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559292719170 0x559292719580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51438/0 (socket says 172.21.15.12:51438) 2024-05-17T22:50:04.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.609+0000 7f13d12ca700 1 -- 172.21.15.12:0/3273060512 learned_addr learned my addr 172.21.15.12:0/3273060512 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:50:04.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.609+0000 7f13c3fff700 1 --2- 172.21.15.12:0/3273060512 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55929271eaa0 0x559292718c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:04.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.609+0000 7f13d0ac9700 1 --2- 172.21.15.12:0/3273060512 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55929259a9f0 0x55929271e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:04.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.609+0000 7f13d0ac9700 1 -- 172.21.15.12:0/3273060512 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55929271eaa0 msgr2=0x559292718c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:04.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.609+0000 7f13d0ac9700 1 --2- 172.21.15.12:0/3273060512 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55929271eaa0 0x559292718c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:04.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.609+0000 7f13d0ac9700 1 -- 172.21.15.12:0/3273060512 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559292719170 msgr2=0x559292719580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:04.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.609+0000 7f13d0ac9700 1 --2- 172.21.15.12:0/3273060512 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559292719170 0x559292719580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:04.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.609+0000 7f13d0ac9700 1 -- 172.21.15.12:0/3273060512 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592925e0590 con 0x55929259a9f0 2024-05-17T22:50:04.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.609+0000 7f13c3fff700 1 --2- 172.21.15.12:0/3273060512 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55929271eaa0 0x559292718c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:50:04.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.609+0000 7f13d0ac9700 1 --2- 172.21.15.12:0/3273060512 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55929259a9f0 0x55929271e560 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f13c80e0d50 tx=0x7f13c8004f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b78c87fd7838d13e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:50:04.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.609+0000 7f13b2ffd700 1 -- 172.21.15.12:0/3273060512 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c80aa380 con 0x55929259a9f0 2024-05-17T22:50:04.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.610+0000 7f13b2ffd700 1 -- 172.21.15.12:0/3273060512 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13c80aaf70 con 0x55929259a9f0 2024-05-17T22:50:04.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.610+0000 7f13b2ffd700 1 -- 172.21.15.12:0/3273060512 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c80a6b00 con 0x55929259a9f0 2024-05-17T22:50:04.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.611+0000 7f13e1708b80 1 -- 172.21.15.12:0/3273060512 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55929259a9f0 msgr2=0x55929271e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:04.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.611+0000 7f13e1708b80 1 --2- 172.21.15.12:0/3273060512 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55929259a9f0 0x55929271e560 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f13c80e0d50 tx=0x7f13c8004f70 comp rx=0 tx=0).stop 2024-05-17T22:50:04.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.611+0000 7f13e1708b80 1 -- 172.21.15.12:0/3273060512 shutdown_connections 2024-05-17T22:50:04.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.611+0000 7f13e1708b80 1 --2- 172.21.15.12:0/3273060512 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559292719170 0x559292719580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:04.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.611+0000 7f13e1708b80 1 --2- 172.21.15.12:0/3273060512 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55929259a9f0 0x55929271e560 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:04.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.611+0000 7f13e1708b80 1 --2- 172.21.15.12:0/3273060512 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55929271eaa0 0x559292718c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:04.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.611+0000 7f13e1708b80 1 -- 172.21.15.12:0/3273060512 >> 172.21.15.12:0/3273060512 conn(0x5592925def00 msgr2=0x559292713b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:50:04.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.611+0000 7f13e1708b80 1 -- 172.21.15.12:0/3273060512 shutdown_connections 2024-05-17T22:50:04.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.611+0000 7f13e1708b80 1 -- 172.21.15.12:0/3273060512 wait complete. 2024-05-17T22:50:04.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.611+0000 7f13e1708b80 1 -- start start 2024-05-17T22:50:04.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.611+0000 7f13e1708b80 1 Processor -- start 2024-05-17T22:50:04.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.612+0000 7f13e1708b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55929259a9f0 0x559292722260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:04.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.612+0000 7f13e1708b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559292719170 0x5592927227a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:04.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.612+0000 7f13e1708b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55929271eaa0 0x55929272e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:04.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.612+0000 7f13e1708b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55929253e150 con 0x559292719170 2024-05-17T22:50:04.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.612+0000 7f13e1708b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5592925d7e80 con 0x55929259a9f0 2024-05-17T22:50:04.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.612+0000 7f13e1708b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559292725390 con 0x55929271eaa0 2024-05-17T22:50:04.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.612+0000 7f13e1708b80 10 client.0 _ll_register_callbacks cb 0x5592925d1f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:50:04.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.612+0000 7f13d12ca700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55929271eaa0 0x55929272e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:04.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.612+0000 7f13c3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559292719170 0x5592927227a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:04.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.612+0000 7f13d12ca700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55929271eaa0 0x55929272e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:45470/0 (socket says 172.21.15.12:45470) 2024-05-17T22:50:04.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.612+0000 7f13d12ca700 1 -- 172.21.15.12:0/477578409 learned_addr learned my addr 172.21.15.12:0/477578409 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:50:04.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.612+0000 7f13d0ac9700 1 --2- 172.21.15.12:0/477578409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55929259a9f0 0x559292722260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:04.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.612+0000 7f13d0ac9700 1 -- 172.21.15.12:0/477578409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55929271eaa0 msgr2=0x55929272e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:04.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.612+0000 7f13d0ac9700 1 --2- 172.21.15.12:0/477578409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55929271eaa0 0x55929272e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:04.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.612+0000 7f13d0ac9700 1 -- 172.21.15.12:0/477578409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559292719170 msgr2=0x5592927227a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:04.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.612+0000 7f13d0ac9700 1 --2- 172.21.15.12:0/477578409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559292719170 0x5592927227a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:04.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.612+0000 7f13d0ac9700 1 -- 172.21.15.12:0/477578409 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559292587480 con 0x55929259a9f0 2024-05-17T22:50:04.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.613+0000 7f13d12ca700 1 --2- 172.21.15.12:0/477578409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55929271eaa0 0x55929272e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:50:04.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.613+0000 7f13d0ac9700 1 --2- 172.21.15.12:0/477578409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55929259a9f0 0x559292722260 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x559292719b40 tx=0x7f13c80057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35295a1d07119d36 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:50:04.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.613+0000 7f13abfff700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:50:04.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.613+0000 7f13abfff700 1 -- 172.21.15.12:0/477578409 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c80c3070 con 0x55929259a9f0 2024-05-17T22:50:04.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.613+0000 7f13abfff700 1 -- 172.21.15.12:0/477578409 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f13c80aaad0 con 0x55929259a9f0 2024-05-17T22:50:04.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.613+0000 7f13e1708b80 1 -- 172.21.15.12:0/477578409 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13c80aaf70 con 0x55929259a9f0 2024-05-17T22:50:04.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.613+0000 7f13abfff700 1 -- 172.21.15.12:0/477578409 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f13c80aa380 con 0x55929259a9f0 2024-05-17T22:50:04.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.613+0000 7f13e1708b80 10 client.25964 fetch_fsmap learned FSMap version 39 2024-05-17T22:50:04.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.613+0000 7f13e1708b80 1 -- 172.21.15.12:0/477578409 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55929272fe50 con 0x55929259a9f0 2024-05-17T22:50:04.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.614+0000 7f13abfff700 1 -- 172.21.15.12:0/477578409 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13c80aa870 con 0x55929259a9f0 2024-05-17T22:50:04.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.614+0000 7f13abfff700 1 -- 172.21.15.12:0/477578409 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f13c80c9020 con 0x55929259a9f0 2024-05-17T22:50:04.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.614+0000 7f13e1708b80 10 client.25964 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:50:04.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.614+0000 7f13e1708b80 10 client.25964 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:50:04.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.614+0000 7f13e1708b80 20 client.25964 populate_metadata read hostname 'smithi012' 2024-05-17T22:50:04.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.614+0000 7f13e1708b80 1 --2- 172.21.15.12:0/477578409 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x559292732320 0x559292734740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:04.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.614+0000 7f13e1708b80 4 client.25964 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:50:04.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.614+0000 7f13e1708b80 1 -- 172.21.15.12:0/477578409 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f13c80aaf70 con 0x559292732320 2024-05-17T22:50:04.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.614+0000 7f13c3fff700 1 --2- 172.21.15.12:0/477578409 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x559292732320 0x559292734740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:04.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.615+0000 7f13abfff700 10 client.25964 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:50:04.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.615+0000 7f13c3fff700 1 --2- 172.21.15.12:0/477578409 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x559292732320 0x559292734740 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f39c88d685491d86 server_cookie=94226e59706e15dd in_seq=0 out_seq=0 2024-05-17T22:50:04.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.616+0000 7f13abfff700 1 -- 172.21.15.12:0/477578409 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f13c80aaf70 con 0x559292732320 2024-05-17T22:50:04.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.616+0000 7f13abfff700 10 client.25964 handle_command_reply: tid=0 2024-05-17T22:50:04.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.616+0000 7f13e1708b80 1 client.25964 shutdown 2024-05-17T22:50:04.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.616+0000 7f13e1708b80 1 -- 172.21.15.12:0/477578409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55929259a9f0 msgr2=0x559292722260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:04.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.616+0000 7f13e1708b80 1 --2- 172.21.15.12:0/477578409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55929259a9f0 0x559292722260 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x559292719b40 tx=0x7f13c80057f0 comp rx=0 tx=0).stop 2024-05-17T22:50:04.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.616+0000 7f13e1708b80 1 -- 172.21.15.12:0/477578409 shutdown_connections 2024-05-17T22:50:04.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.616+0000 7f13e1708b80 1 --2- 172.21.15.12:0/477578409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55929271eaa0 0x55929272e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:04.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.616+0000 7f13e1708b80 1 --2- 172.21.15.12:0/477578409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55929259a9f0 0x559292722260 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:04.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.616+0000 7f13e1708b80 1 --2- 172.21.15.12:0/477578409 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x559292732320 0x559292734740 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:04.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.616+0000 7f13abfff700 0 client.25964 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:50:04.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.616+0000 7f13e1708b80 1 --2- 172.21.15.12:0/477578409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559292719170 0x5592927227a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:04.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.616+0000 7f13e1708b80 1 -- 172.21.15.12:0/477578409 >> 172.21.15.12:0/477578409 conn(0x5592925def00 msgr2=0x55929270ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:50:04.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.616+0000 7f13e1708b80 1 -- 172.21.15.12:0/477578409 shutdown_connections 2024-05-17T22:50:04.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.616+0000 7f13e1708b80 1 -- 172.21.15.12:0/477578409 wait complete. 2024-05-17T22:50:04.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.617+0000 7f13e1708b80 20 client.25964 trim_cache size 0 max 16384 2024-05-17T22:50:04.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.644+0000 7f13e1708b80 1 Processor -- start 2024-05-17T22:50:04.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.650+0000 7f13e1708b80 1 -- start start 2024-05-17T22:50:04.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.650+0000 7f13e1708b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55929259a9f0 0x559292718c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:04.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.650+0000 7f13e1708b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559292719170 0x5592927211a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:04.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.650+0000 7f13e1708b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55929271eaa0 0x55929271c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:04.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.650+0000 7f13e1708b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc1b4960 con 0x55929259a9f0 2024-05-17T22:50:04.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.650+0000 7f13e1708b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc1b21f0 con 0x55929271eaa0 2024-05-17T22:50:04.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.651+0000 7f13c3fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559292719170 0x5592927211a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:04.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.651+0000 7f13c3fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559292719170 0x5592927211a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:45472/0 (socket says 172.21.15.12:45472) 2024-05-17T22:50:04.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.651+0000 7f13c3fff700 1 -- 172.21.15.12:0/3036596921 learned_addr learned my addr 172.21.15.12:0/3036596921 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:50:04.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.651+0000 7f13d12ca700 1 --2- 172.21.15.12:0/3036596921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55929271eaa0 0x55929271c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:04.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.651+0000 7f13e1708b80 1 -- 172.21.15.12:0/3036596921 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc1b0040 con 0x559292719170 2024-05-17T22:50:04.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.654+0000 7f13d0ac9700 1 --2- 172.21.15.12:0/3036596921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55929259a9f0 0x559292718c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:04.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.657+0000 7f13c3fff700 1 -- 172.21.15.12:0/3036596921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55929271eaa0 msgr2=0x55929271c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:04.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.657+0000 7f13c3fff700 1 --2- 172.21.15.12:0/3036596921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55929271eaa0 0x55929271c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:04.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.657+0000 7f13c3fff700 1 -- 172.21.15.12:0/3036596921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55929259a9f0 msgr2=0x559292718c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:04.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.657+0000 7f13c3fff700 1 --2- 172.21.15.12:0/3036596921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55929259a9f0 0x559292718c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:04.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.657+0000 7f13c3fff700 1 -- 172.21.15.12:0/3036596921 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592927207b0 con 0x559292719170 2024-05-17T22:50:04.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.665+0000 7f13c3fff700 1 --2- 172.21.15.12:0/3036596921 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559292719170 0x5592927211a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f13bc013010 tx=0x7f13bc014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54262a135a91810d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:50:04.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.671+0000 7f13b27fc700 1 -- 172.21.15.12:0/3036596921 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13bc003ae0 con 0x559292719170 2024-05-17T22:50:04.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.671+0000 7f13b27fc700 1 -- 172.21.15.12:0/3036596921 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13bc02f070 con 0x559292719170 2024-05-17T22:50:04.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.671+0000 7f13b27fc700 1 -- 172.21.15.12:0/3036596921 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13bc01e710 con 0x559292719170 2024-05-17T22:50:04.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.672+0000 7f13e1708b80 1 -- 172.21.15.12:0/3036596921 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559292719170 msgr2=0x5592927211a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:04.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.672+0000 7f13e1708b80 1 --2- 172.21.15.12:0/3036596921 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559292719170 0x5592927211a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f13bc013010 tx=0x7f13bc014000 comp rx=0 tx=0).stop 2024-05-17T22:50:04.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.681+0000 7f13e1708b80 1 -- 172.21.15.12:0/3036596921 shutdown_connections 2024-05-17T22:50:04.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.681+0000 7f13e1708b80 1 --2- 172.21.15.12:0/3036596921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55929271eaa0 0x55929271c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:04.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.681+0000 7f13e1708b80 1 --2- 172.21.15.12:0/3036596921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55929259a9f0 0x559292718c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:04.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.681+0000 7f13e1708b80 1 --2- 172.21.15.12:0/3036596921 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559292719170 0x5592927211a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:04.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.681+0000 7f13e1708b80 1 -- 172.21.15.12:0/3036596921 >> 172.21.15.12:0/3036596921 conn(0x5592925def00 msgr2=0x559292721b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:50:04.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.690+0000 7f13e1708b80 1 -- 172.21.15.12:0/3036596921 shutdown_connections 2024-05-17T22:50:04.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.690+0000 7f13e1708b80 1 -- 172.21.15.12:0/3036596921 wait complete. 2024-05-17T22:50:04.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.691+0000 7f13e1708b80 1 -- start start 2024-05-17T22:50:04.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.691+0000 7f13e1708b80 1 Processor -- start 2024-05-17T22:50:04.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.692+0000 7f13e1708b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55929259a9f0 0x559292737d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:04.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.692+0000 7f13e1708b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559292719170 0x55929279d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:04.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.697+0000 7f13c3fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559292719170 0x55929279d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:04.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.697+0000 7f13d0ac9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55929259a9f0 0x559292737d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:04.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.697+0000 7f13d0ac9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55929259a9f0 0x559292737d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51474/0 (socket says 172.21.15.12:51474) 2024-05-17T22:50:04.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.697+0000 7f13d0ac9700 1 -- 172.21.15.12:0/719447670 learned_addr learned my addr 172.21.15.12:0/719447670 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:50:04.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.697+0000 7f13e1708b80 1 --2- 172.21.15.12:0/719447670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55929271eaa0 0x55929279de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:04.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.697+0000 7f13e1708b80 1 -- 172.21.15.12:0/719447670 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc1b06b0 con 0x55929259a9f0 2024-05-17T22:50:04.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.697+0000 7f13e1708b80 1 -- 172.21.15.12:0/719447670 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc1b1b20 con 0x559292719170 2024-05-17T22:50:04.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.698+0000 7f13e1708b80 1 -- 172.21.15.12:0/719447670 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc1b0d80 con 0x55929271eaa0 2024-05-17T22:50:04.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.698+0000 7f13e1708b80 10 client.0 _ll_register_callbacks cb 0x5592925d1f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:50:04.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.700+0000 7f13d12ca700 1 --2- 172.21.15.12:0/719447670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55929271eaa0 0x55929279de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:04.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.706+0000 7f13d0ac9700 1 -- 172.21.15.12:0/719447670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55929271eaa0 msgr2=0x55929279de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:04.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.706+0000 7f13d0ac9700 1 --2- 172.21.15.12:0/719447670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55929271eaa0 0x55929279de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:04.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.706+0000 7f13d0ac9700 1 -- 172.21.15.12:0/719447670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559292719170 msgr2=0x55929279d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:04.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.706+0000 7f13d0ac9700 1 --2- 172.21.15.12:0/719447670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559292719170 0x55929279d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:04.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.706+0000 7f13d0ac9700 1 -- 172.21.15.12:0/719447670 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55929279e5b0 con 0x55929259a9f0 2024-05-17T22:50:04.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.707+0000 7f13d0ac9700 1 --2- 172.21.15.12:0/719447670 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55929259a9f0 0x559292737d00 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f13c80b3330 tx=0x7f13c80b35e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f4e087f3aec506 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:50:04.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.707+0000 7f13abfff700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:50:04.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.708+0000 7f13abfff700 1 -- 172.21.15.12:0/719447670 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c80c3070 con 0x55929259a9f0 2024-05-17T22:50:04.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.708+0000 7f13abfff700 1 -- 172.21.15.12:0/719447670 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f13c80a9ad0 con 0x55929259a9f0 2024-05-17T22:50:04.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.708+0000 7f13e1708b80 1 -- 172.21.15.12:0/719447670 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13bc02f070 con 0x55929259a9f0 2024-05-17T22:50:04.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.713+0000 7f13abfff700 1 -- 172.21.15.12:0/719447670 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f13c8090050 con 0x55929259a9f0 2024-05-17T22:50:04.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.713+0000 7f13e1708b80 10 client.16260 fetch_fsmap learned FSMap version 39 2024-05-17T22:50:04.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.713+0000 7f13e1708b80 1 -- 172.21.15.12:0/719447670 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5592927b6080 con 0x55929259a9f0 2024-05-17T22:50:04.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.716+0000 7f13abfff700 1 -- 172.21.15.12:0/719447670 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13c8089060 con 0x55929259a9f0 2024-05-17T22:50:04.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.716+0000 7f13abfff700 1 -- 172.21.15.12:0/719447670 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f13c80c9070 con 0x55929259a9f0 2024-05-17T22:50:04.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.716+0000 7f13e1708b80 10 client.16260 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:50:04.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.716+0000 7f13e1708b80 10 client.16260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:50:04.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.716+0000 7f13e1708b80 20 client.16260 populate_metadata read hostname 'smithi012' 2024-05-17T22:50:04.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.716+0000 7f13e1708b80 1 --2- 172.21.15.12:0/719447670 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5592927b6450 0x5592927b8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:04.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.717+0000 7f13e1708b80 4 client.16260 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:50:04.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.717+0000 7f13e1708b80 1 -- 172.21.15.12:0/719447670 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f13bc02f070 con 0x5592927b6450 2024-05-17T22:50:04.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.718+0000 7f13c3fff700 1 --2- 172.21.15.12:0/719447670 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5592927b6450 0x5592927b8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:04.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.727+0000 7f13c3fff700 1 --2- 172.21.15.12:0/719447670 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5592927b6450 0x5592927b8870 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cb439f3a5c10c0a server_cookie=e7bf6cd7720b7ca5 in_seq=0 out_seq=0 2024-05-17T22:50:04.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.728+0000 7f13abfff700 10 client.16260 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:50:04.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.728+0000 7f13abfff700 1 -- 172.21.15.12:0/719447670 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f13bc02f070 con 0x5592927b6450 2024-05-17T22:50:04.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.728+0000 7f13abfff700 10 client.16260 handle_command_reply: tid=0 2024-05-17T22:50:04.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.728+0000 7f13e1708b80 1 client.16260 shutdown 2024-05-17T22:50:04.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.731+0000 7f13e1708b80 1 -- 172.21.15.12:0/719447670 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55929259a9f0 msgr2=0x559292737d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:04.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.731+0000 7f13e1708b80 1 --2- 172.21.15.12:0/719447670 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55929259a9f0 0x559292737d00 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f13c80b3330 tx=0x7f13c80b35e0 comp rx=0 tx=0).stop 2024-05-17T22:50:04.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.732+0000 7f13e1708b80 1 -- 172.21.15.12:0/719447670 shutdown_connections 2024-05-17T22:50:04.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.732+0000 7f13e1708b80 1 --2- 172.21.15.12:0/719447670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55929271eaa0 0x55929279de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:04.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.732+0000 7f13e1708b80 1 --2- 172.21.15.12:0/719447670 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5592927b6450 0x5592927b8870 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:04.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.732+0000 7f13e1708b80 1 --2- 172.21.15.12:0/719447670 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55929259a9f0 0x559292737d00 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:04.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.732+0000 7f13e1708b80 1 --2- 172.21.15.12:0/719447670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559292719170 0x55929279d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:04.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.732+0000 7f13e1708b80 1 -- 172.21.15.12:0/719447670 >> 172.21.15.12:0/719447670 conn(0x5592925def00 msgr2=0x55929271c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:50:04.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.732+0000 7f13abfff700 0 client.16260 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:50:04.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.732+0000 7f13e1708b80 1 -- 172.21.15.12:0/719447670 shutdown_connections 2024-05-17T22:50:04.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.733+0000 7f13e1708b80 1 -- 172.21.15.12:0/719447670 wait complete. 2024-05-17T22:50:04.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.734+0000 7f13e1708b80 20 client.16260 trim_cache size 0 max 16384 2024-05-17T22:50:04.736 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:50:04.737 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-17T22:50:04.737 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-17T22:50:04.737 INFO:teuthology.orchestra.run.smithi012.stdout: "2e29f53e-3c74-4a52-850d-542951802ba4": { 2024-05-17T22:50:04.737 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-17T22:50:04.737 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "2e29f53e-3c74-4a52-850d-542951802ba4", 2024-05-17T22:50:04.737 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-17T22:50:04.737 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:50:04.737 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:50:04.738 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:50:04.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.735+0000 7f13d2d2d700 1 -- 172.21.15.12:0/3034531779 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f13b40b2960 msgr2=0x7f13b40b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:04.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.735+0000 7f13d2d2d700 1 --2- 172.21.15.12:0/3034531779 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f13b40b2960 0x7f13b40b4e70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f13cc104760 tx=0x7f13bc009260 comp rx=0 tx=0).stop 2024-05-17T22:50:04.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.735+0000 7f13d2d2d700 1 -- 172.21.15.12:0/3034531779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13cc102e70 msgr2=0x7f13cc1a2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:04.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.735+0000 7f13d2d2d700 1 --2- 172.21.15.12:0/3034531779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13cc102e70 0x7f13cc1a2560 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f13c800e9c0 tx=0x7f13c800ecd0 comp rx=0 tx=0).stop 2024-05-17T22:50:04.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.737+0000 7f13d2d2d700 1 -- 172.21.15.12:0/3034531779 shutdown_connections 2024-05-17T22:50:04.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.737+0000 7f13d2d2d700 1 --2- 172.21.15.12:0/3034531779 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f13b40b2960 0x7f13b40b4e70 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:04.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.737+0000 7f13d2d2d700 1 --2- 172.21.15.12:0/3034531779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13cc1080a0 0x7f13cc19c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:04.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.737+0000 7f13d2d2d700 1 --2- 172.21.15.12:0/3034531779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13cc102e70 0x7f13cc1a2560 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:04.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.737+0000 7f13d2d2d700 1 --2- 172.21.15.12:0/3034531779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13cc107750 0x7f13cc1a2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:04.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.737+0000 7f13d2d2d700 1 -- 172.21.15.12:0/3034531779 >> 172.21.15.12:0/3034531779 conn(0x7f13cc079ae0 msgr2=0x7f13cc1052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:50:04.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.738+0000 7f13d2d2d700 1 -- 172.21.15.12:0/3034531779 shutdown_connections 2024-05-17T22:50:04.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:04.738+0000 7f13d2d2d700 1 -- 172.21.15.12:0/3034531779 wait complete. 2024-05-17T22:50:05.570 INFO:tasks.cephfs.filesystem:scrub status for tag:2e29f53e-3c74-4a52-850d-542951802ba4 - {'path': '/', 'tag': '2e29f53e-3c74-4a52-850d-542951802ba4', 'options': 'recursive,force'} 2024-05-17T22:50:06.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 645.0 s, 488.2 tps, lat 195.717 ms stddev 221.949 2024-05-17T22:50:06.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:06 smithi151 ceph-mon[36101]: pgmap v764: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 MiB/s wr, 510 op/s 2024-05-17T22:50:06.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:06 smithi096 ceph-mon[36065]: pgmap v764: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 MiB/s wr, 510 op/s 2024-05-17T22:50:06.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:06 smithi012 ceph-mon[31632]: pgmap v764: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 MiB/s wr, 510 op/s 2024-05-17T22:50:08.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:08 smithi151 ceph-mon[36101]: pgmap v765: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.4 MiB/s wr, 466 op/s 2024-05-17T22:50:08.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:08 smithi096 ceph-mon[36065]: pgmap v765: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.4 MiB/s wr, 466 op/s 2024-05-17T22:50:08.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:08 smithi012 ceph-mon[31632]: pgmap v765: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.4 MiB/s wr, 466 op/s 2024-05-17T22:50:09.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:09 smithi096 ceph-mon[36065]: scrub summary: idle 2024-05-17T22:50:09.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:09 smithi096 ceph-mon[36065]: scrub summary: idle 2024-05-17T22:50:09.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:09 smithi096 ceph-mon[36065]: scrub summary: idle 2024-05-17T22:50:09.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:09 smithi012 ceph-mon[31632]: scrub summary: idle 2024-05-17T22:50:09.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:09 smithi012 ceph-mon[31632]: scrub summary: idle 2024-05-17T22:50:09.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:09 smithi012 ceph-mon[31632]: scrub summary: idle 2024-05-17T22:50:09.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:09 smithi151 ceph-mon[36101]: scrub summary: idle 2024-05-17T22:50:09.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:09 smithi151 ceph-mon[36101]: scrub summary: idle 2024-05-17T22:50:09.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:09 smithi151 ceph-mon[36101]: scrub summary: idle 2024-05-17T22:50:10.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:10 smithi096 ceph-mon[36065]: pgmap v766: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.2 MiB/s wr, 485 op/s 2024-05-17T22:50:10.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:10 smithi012 ceph-mon[31632]: pgmap v766: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.2 MiB/s wr, 485 op/s 2024-05-17T22:50:10.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:10 smithi151 ceph-mon[36101]: pgmap v766: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.2 MiB/s wr, 485 op/s 2024-05-17T22:50:11.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 650.0 s, 526.2 tps, lat 194.821 ms stddev 239.015 2024-05-17T22:50:12.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:12 smithi096 ceph-mon[36065]: pgmap v767: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.5 MiB/s wr, 485 op/s 2024-05-17T22:50:12.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:12 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:50:12.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:12 smithi012 ceph-mon[31632]: pgmap v767: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.5 MiB/s wr, 485 op/s 2024-05-17T22:50:12.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:12 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:50:12.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:12 smithi151 ceph-mon[36101]: pgmap v767: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.5 MiB/s wr, 485 op/s 2024-05-17T22:50:12.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:12 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:50:14.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:14 smithi096 ceph-mon[36065]: pgmap v768: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.6 MiB/s wr, 478 op/s 2024-05-17T22:50:14.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:14 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:50:14.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:14 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:50:14.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:14 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:50:14.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:14 smithi012 ceph-mon[31632]: pgmap v768: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.6 MiB/s wr, 478 op/s 2024-05-17T22:50:14.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:14 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:50:14.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:14 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:50:14.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:14 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:50:14.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:14 smithi151 ceph-mon[36101]: pgmap v768: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.6 MiB/s wr, 478 op/s 2024-05-17T22:50:14.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:14 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:50:14.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:14 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:50:14.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:14 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:50:16.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 655.0 s, 433.0 tps, lat 221.211 ms stddev 250.279 2024-05-17T22:50:16.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:16 smithi096 ceph-mon[36065]: pgmap v769: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.2 MiB/s wr, 534 op/s 2024-05-17T22:50:16.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:16 smithi012 ceph-mon[31632]: pgmap v769: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.2 MiB/s wr, 534 op/s 2024-05-17T22:50:16.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:16 smithi151 ceph-mon[36101]: pgmap v769: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.2 MiB/s wr, 534 op/s 2024-05-17T22:50:18.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:18 smithi096 ceph-mon[36065]: pgmap v770: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.0 MiB/s wr, 475 op/s 2024-05-17T22:50:18.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:18 smithi012 ceph-mon[31632]: pgmap v770: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.0 MiB/s wr, 475 op/s 2024-05-17T22:50:18.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:18 smithi151 ceph-mon[36101]: pgmap v770: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.0 MiB/s wr, 475 op/s 2024-05-17T22:50:19.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:19 smithi012 ceph-mon[31632]: pgmap v771: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 6.2 MiB/s wr, 477 op/s 2024-05-17T22:50:19.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:19 smithi151 ceph-mon[36101]: pgmap v771: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 6.2 MiB/s wr, 477 op/s 2024-05-17T22:50:19.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:19 smithi096 ceph-mon[36065]: pgmap v771: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 6.2 MiB/s wr, 477 op/s 2024-05-17T22:50:21.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 660.0 s, 500.8 tps, lat 210.802 ms stddev 291.073 2024-05-17T22:50:22.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:21 smithi151 ceph-mon[36101]: pgmap v772: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.3 MiB/s wr, 476 op/s 2024-05-17T22:50:22.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:21 smithi096 ceph-mon[36065]: pgmap v772: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.3 MiB/s wr, 476 op/s 2024-05-17T22:50:22.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:21 smithi012 ceph-mon[31632]: pgmap v772: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.3 MiB/s wr, 476 op/s 2024-05-17T22:50:24.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:23 smithi151 ceph-mon[36101]: pgmap v773: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.7 MiB/s wr, 449 op/s 2024-05-17T22:50:24.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:23 smithi096 ceph-mon[36065]: pgmap v773: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.7 MiB/s wr, 449 op/s 2024-05-17T22:50:24.343 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:23 smithi012 ceph-mon[31632]: pgmap v773: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.7 MiB/s wr, 449 op/s 2024-05-17T22:50:26.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:25 smithi151 ceph-mon[36101]: pgmap v774: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 MiB/s wr, 510 op/s 2024-05-17T22:50:26.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:25 smithi096 ceph-mon[36065]: pgmap v774: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 MiB/s wr, 510 op/s 2024-05-17T22:50:26.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:25 smithi012 ceph-mon[31632]: pgmap v774: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 MiB/s wr, 510 op/s 2024-05-17T22:50:26.370 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 665.0 s, 474.2 tps, lat 211.675 ms stddev 235.071 2024-05-17T22:50:28.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:27 smithi151 ceph-mon[36101]: pgmap v775: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.1 MiB/s wr, 463 op/s 2024-05-17T22:50:28.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:27 smithi096 ceph-mon[36065]: pgmap v775: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.1 MiB/s wr, 463 op/s 2024-05-17T22:50:28.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:27 smithi012 ceph-mon[31632]: pgmap v775: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.1 MiB/s wr, 463 op/s 2024-05-17T22:50:30.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:29 smithi151 ceph-mon[36101]: pgmap v776: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 MiB/s wr, 483 op/s 2024-05-17T22:50:30.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:29 smithi096 ceph-mon[36065]: pgmap v776: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 MiB/s wr, 483 op/s 2024-05-17T22:50:30.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:29 smithi012 ceph-mon[31632]: pgmap v776: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 MiB/s wr, 483 op/s 2024-05-17T22:50:31.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 670.0 s, 482.6 tps, lat 201.013 ms stddev 231.362 2024-05-17T22:50:32.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:31 smithi151 ceph-mon[36101]: pgmap v777: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.2 MiB/s wr, 471 op/s 2024-05-17T22:50:32.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:31 smithi096 ceph-mon[36065]: pgmap v777: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.2 MiB/s wr, 471 op/s 2024-05-17T22:50:32.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:31 smithi012 ceph-mon[31632]: pgmap v777: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.2 MiB/s wr, 471 op/s 2024-05-17T22:50:34.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:33 smithi151 ceph-mon[36101]: pgmap v778: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.5 MiB/s wr, 482 op/s 2024-05-17T22:50:34.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:33 smithi096 ceph-mon[36065]: pgmap v778: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.5 MiB/s wr, 482 op/s 2024-05-17T22:50:34.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:33 smithi012 ceph-mon[31632]: pgmap v778: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.5 MiB/s wr, 482 op/s 2024-05-17T22:50:35.571 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-17T22:50:36.015 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:50:36.238 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:35 smithi012 ceph-mon[31632]: pgmap v779: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.3 MiB/s wr, 508 op/s 2024-05-17T22:50:36.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:35 smithi151 ceph-mon[36101]: pgmap v779: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.3 MiB/s wr, 508 op/s 2024-05-17T22:50:36.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:35 smithi096 ceph-mon[36065]: pgmap v779: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.3 MiB/s wr, 508 op/s 2024-05-17T22:50:36.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 675.0 s, 472.2 tps, lat 216.961 ms stddev 266.978 2024-05-17T22:50:38.085 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:37 smithi012 ceph-mon[31632]: pgmap v780: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.5 MiB/s wr, 454 op/s 2024-05-17T22:50:38.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.222+0000 7f451bbbf700 1 -- 172.21.15.12:0/698610871 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4514076520 msgr2=0x7f4514114750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:38.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.222+0000 7f451bbbf700 1 --2- 172.21.15.12:0/698610871 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4514076520 0x7f4514114750 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f451000b0a0 tx=0x7f451000b3b0 comp rx=0 tx=0).stop 2024-05-17T22:50:38.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.223+0000 7f451bbbf700 1 -- 172.21.15.12:0/698610871 shutdown_connections 2024-05-17T22:50:38.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.223+0000 7f451bbbf700 1 --2- 172.21.15.12:0/698610871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4514114c90 0x7f45141170b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:38.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.223+0000 7f451bbbf700 1 --2- 172.21.15.12:0/698610871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4514075bd0 0x7f4514075fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:38.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.223+0000 7f451bbbf700 1 --2- 172.21.15.12:0/698610871 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4514076520 0x7f4514114750 secure :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f451000b0a0 tx=0x7f451000b3b0 comp rx=0 tx=0).stop 2024-05-17T22:50:38.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.223+0000 7f451bbbf700 1 -- 172.21.15.12:0/698610871 >> 172.21.15.12:0/698610871 conn(0x7f45140ff990 msgr2=0x7f4514101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:50:38.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.225+0000 7f451bbbf700 1 -- 172.21.15.12:0/698610871 shutdown_connections 2024-05-17T22:50:38.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.227+0000 7f451bbbf700 1 -- 172.21.15.12:0/698610871 wait complete. 2024-05-17T22:50:38.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.228+0000 7f451bbbf700 1 Processor -- start 2024-05-17T22:50:38.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.231+0000 7f451bbbf700 1 -- start start 2024-05-17T22:50:38.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.232+0000 7f451bbbf700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4514075bd0 0x7f451410d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:38.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.232+0000 7f451bbbf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4514114c90 0x7f451410dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:38.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.232+0000 7f451bbbf700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f451410e2d0 0x7f4514112250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:38.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.232+0000 7f451bbbf700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f451411a4d0 con 0x7f4514114c90 2024-05-17T22:50:38.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.232+0000 7f451bbbf700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f451411a350 con 0x7f4514075bd0 2024-05-17T22:50:38.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.232+0000 7f451bbbf700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f451411a650 con 0x7f451410e2d0 2024-05-17T22:50:38.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.232+0000 7f451a15c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f451410e2d0 0x7f4514112250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:38.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.232+0000 7f451a15c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f451410e2d0 0x7f4514112250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:59384/0 (socket says 172.21.15.12:59384) 2024-05-17T22:50:38.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.232+0000 7f451a15c700 1 -- 172.21.15.12:0/4255605092 learned_addr learned my addr 172.21.15.12:0/4255605092 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:50:38.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.233+0000 7f451915a700 1 --2- 172.21.15.12:0/4255605092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4514114c90 0x7f451410dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:38.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.234+0000 7f451995b700 1 --2- 172.21.15.12:0/4255605092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4514075bd0 0x7f451410d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:38.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:37 smithi151 ceph-mon[36101]: pgmap v780: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.5 MiB/s wr, 454 op/s 2024-05-17T22:50:38.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.237+0000 7f451915a700 1 -- 172.21.15.12:0/4255605092 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f451410e2d0 msgr2=0x7f4514112250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:38.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.237+0000 7f451915a700 1 --2- 172.21.15.12:0/4255605092 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f451410e2d0 0x7f4514112250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:38.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.237+0000 7f451915a700 1 -- 172.21.15.12:0/4255605092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4514075bd0 msgr2=0x7f451410d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:38.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.237+0000 7f451915a700 1 --2- 172.21.15.12:0/4255605092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4514075bd0 0x7f451410d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:38.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.237+0000 7f451915a700 1 -- 172.21.15.12:0/4255605092 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4514112790 con 0x7f4514114c90 2024-05-17T22:50:38.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.239+0000 7f451915a700 1 --2- 172.21.15.12:0/4255605092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4514114c90 0x7f451410dd90 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4510008000 tx=0x7f451000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:50:38.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.242+0000 7f450affd700 1 -- 172.21.15.12:0/4255605092 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4510022030 con 0x7f4514114c90 2024-05-17T22:50:38.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.242+0000 7f451bbbf700 1 -- 172.21.15.12:0/4255605092 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4514112a20 con 0x7f4514114c90 2024-05-17T22:50:38.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.242+0000 7f451bbbf700 1 -- 172.21.15.12:0/4255605092 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4514112ee0 con 0x7f4514114c90 2024-05-17T22:50:38.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.242+0000 7f450affd700 1 -- 172.21.15.12:0/4255605092 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4510003a00 con 0x7f4514114c90 2024-05-17T22:50:38.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.242+0000 7f450affd700 1 -- 172.21.15.12:0/4255605092 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4510007e20 con 0x7f4514114c90 2024-05-17T22:50:38.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.251+0000 7f452a59ab80 1 Processor -- start 2024-05-17T22:50:38.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.253+0000 7f452a59ab80 1 -- start start 2024-05-17T22:50:38.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.253+0000 7f452a59ab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edbf19f0 0x5603edd75560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:38.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.253+0000 7f452a59ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603edd75aa0 0x5603edd6fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:38.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.253+0000 7f452a59ab80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603edd70170 0x5603edd70580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:38.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.253+0000 7f452a59ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5603edc9e290 con 0x5603edd75aa0 2024-05-17T22:50:38.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.253+0000 7f452a59ab80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5603edc8dfd0 con 0x5603edd70170 2024-05-17T22:50:38.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.253+0000 7f452a59ab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5603edc91e70 con 0x5603edbf19f0 2024-05-17T22:50:38.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.255+0000 7f451995b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603edd75aa0 0x5603edd6fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:38.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.255+0000 7f451995b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603edd75aa0 0x5603edd6fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42192/0 (socket says 172.21.15.12:42192) 2024-05-17T22:50:38.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.255+0000 7f451995b700 1 -- 172.21.15.12:0/486858277 learned_addr learned my addr 172.21.15.12:0/486858277 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:50:38.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.257+0000 7f450affd700 1 -- 172.21.15.12:0/4255605092 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4510025070 con 0x7f4514114c90 2024-05-17T22:50:38.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.257+0000 7f451915a700 1 --2- 172.21.15.12:0/486858277 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603edd70170 0x5603edd70580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:38.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.258+0000 7f450affd700 1 --2- 172.21.15.12:0/4255605092 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f45000b2a80 0x7f45000b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:38.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.258+0000 7f450affd700 1 -- 172.21.15.12:0/4255605092 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f45100e2e20 con 0x7f4514114c90 2024-05-17T22:50:38.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.259+0000 7f451a15c700 1 --2- 172.21.15.12:0/486858277 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edbf19f0 0x5603edd75560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:38.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.260+0000 7f451a15c700 1 --2- 172.21.15.12:0/4255605092 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f45000b2a80 0x7f45000b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:38.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.263+0000 7f451995b700 1 -- 172.21.15.12:0/486858277 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edbf19f0 msgr2=0x5603edd75560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:38.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.263+0000 7f451995b700 1 --2- 172.21.15.12:0/486858277 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edbf19f0 0x5603edd75560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:38.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.263+0000 7f451995b700 1 -- 172.21.15.12:0/486858277 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603edd70170 msgr2=0x5603edd70580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:38.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.263+0000 7f451995b700 1 --2- 172.21.15.12:0/486858277 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603edd70170 0x5603edd70580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:38.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.263+0000 7f451995b700 1 -- 172.21.15.12:0/486858277 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603edc37590 con 0x5603edd75aa0 2024-05-17T22:50:38.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.263+0000 7f451a15c700 1 --2- 172.21.15.12:0/486858277 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edbf19f0 0x5603edd75560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:50:38.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.264+0000 7f451a15c700 1 --2- 172.21.15.12:0/4255605092 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f45000b2a80 0x7f45000b4f90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x5603edd6bae0 tx=0x7f45040047c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:50:38.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.264+0000 7f451995b700 1 --2- 172.21.15.12:0/486858277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603edd75aa0 0x5603edd6fc30 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f450c00a980 tx=0x7f450c00ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad692e40f338c46 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:50:38.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.267+0000 7f44fffff700 1 -- 172.21.15.12:0/486858277 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c015070 con 0x5603edd75aa0 2024-05-17T22:50:38.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.267+0000 7f44fffff700 1 -- 172.21.15.12:0/486858277 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f450c00cd80 con 0x5603edd75aa0 2024-05-17T22:50:38.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.267+0000 7f44fffff700 1 -- 172.21.15.12:0/486858277 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c018360 con 0x5603edd75aa0 2024-05-17T22:50:38.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.273+0000 7f452a59ab80 1 -- 172.21.15.12:0/486858277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603edd75aa0 msgr2=0x5603edd6fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:38.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.273+0000 7f452a59ab80 1 --2- 172.21.15.12:0/486858277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603edd75aa0 0x5603edd6fc30 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f450c00a980 tx=0x7f450c00ac90 comp rx=0 tx=0).stop 2024-05-17T22:50:38.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.275+0000 7f452a59ab80 1 -- 172.21.15.12:0/486858277 shutdown_connections 2024-05-17T22:50:38.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.275+0000 7f452a59ab80 1 --2- 172.21.15.12:0/486858277 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603edd70170 0x5603edd70580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:38.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.275+0000 7f452a59ab80 1 --2- 172.21.15.12:0/486858277 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edbf19f0 0x5603edd75560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:38.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.275+0000 7f452a59ab80 1 --2- 172.21.15.12:0/486858277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603edd75aa0 0x5603edd6fc30 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:38.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.275+0000 7f452a59ab80 1 -- 172.21.15.12:0/486858277 >> 172.21.15.12:0/486858277 conn(0x5603edc35f00 msgr2=0x5603edd6ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:50:38.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.282+0000 7f452a59ab80 1 -- 172.21.15.12:0/486858277 shutdown_connections 2024-05-17T22:50:38.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.284+0000 7f452a59ab80 1 -- 172.21.15.12:0/486858277 wait complete. 2024-05-17T22:50:38.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.285+0000 7f452a59ab80 1 -- start start 2024-05-17T22:50:38.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.285+0000 7f452a59ab80 1 Processor -- start 2024-05-17T22:50:38.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.285+0000 7f452a59ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603edbf19f0 0x5603edd79260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:38.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.285+0000 7f452a59ab80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603edd70170 0x5603edd797a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:38.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.285+0000 7f452a59ab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edd75aa0 0x5603edd79d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:38.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.285+0000 7f452a59ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5603edb95150 con 0x5603edbf19f0 2024-05-17T22:50:38.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.285+0000 7f452a59ab80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5603edc2ee80 con 0x5603edd70170 2024-05-17T22:50:38.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.285+0000 7f452a59ab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5603edd7c390 con 0x5603edd75aa0 2024-05-17T22:50:38.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.285+0000 7f452a59ab80 10 client.0 _ll_register_callbacks cb 0x5603edc28f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:50:38.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.286+0000 7f451915a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603edd70170 0x5603edd797a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:38.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.286+0000 7f451a15c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edd75aa0 0x5603edd79d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:38.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.286+0000 7f451a15c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edd75aa0 0x5603edd79d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:59404/0 (socket says 172.21.15.12:59404) 2024-05-17T22:50:38.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.286+0000 7f451915a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603edd70170 0x5603edd797a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:59234/0 (socket says 172.21.15.12:59234) 2024-05-17T22:50:38.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.286+0000 7f451a15c700 1 -- 172.21.15.12:0/1074382665 learned_addr learned my addr 172.21.15.12:0/1074382665 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:50:38.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.286+0000 7f451995b700 1 --2- 172.21.15.12:0/1074382665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603edbf19f0 0x5603edd79260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:38.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.286+0000 7f451915a700 1 -- 172.21.15.12:0/1074382665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edd75aa0 msgr2=0x5603edd79d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:38.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.286+0000 7f451915a700 1 --2- 172.21.15.12:0/1074382665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edd75aa0 0x5603edd79d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:38.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.286+0000 7f451915a700 1 -- 172.21.15.12:0/1074382665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603edbf19f0 msgr2=0x5603edd79260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:38.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.286+0000 7f451915a700 1 --2- 172.21.15.12:0/1074382665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603edbf19f0 0x5603edd79260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:38.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.286+0000 7f451915a700 1 -- 172.21.15.12:0/1074382665 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603edbde480 con 0x5603edd70170 2024-05-17T22:50:38.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.286+0000 7f451995b700 1 --2- 172.21.15.12:0/1074382665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603edbf19f0 0x5603edd79260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:50:38.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.286+0000 7f451915a700 1 --2- 172.21.15.12:0/1074382665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603edd70170 0x5603edd797a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4510009fd0 tx=0x7f45100a9b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bda722b35914f3d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:50:38.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.286+0000 7f44fd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:50:38.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.288+0000 7f44fd7fa700 1 -- 172.21.15.12:0/1074382665 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45100aa0e0 con 0x5603edd70170 2024-05-17T22:50:38.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.288+0000 7f44fd7fa700 1 -- 172.21.15.12:0/1074382665 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f45100aa340 con 0x5603edd70170 2024-05-17T22:50:38.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.288+0000 7f452a59ab80 1 -- 172.21.15.12:0/1074382665 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f450c00cd80 con 0x5603edd70170 2024-05-17T22:50:38.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.291+0000 7f44fd7fa700 1 -- 172.21.15.12:0/1074382665 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f45100ae490 con 0x5603edd70170 2024-05-17T22:50:38.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:37 smithi096 ceph-mon[36065]: pgmap v780: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.5 MiB/s wr, 454 op/s 2024-05-17T22:50:38.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.295+0000 7f452a59ab80 10 client.25976 fetch_fsmap learned FSMap version 39 2024-05-17T22:50:38.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.295+0000 7f452a59ab80 1 -- 172.21.15.12:0/1074382665 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5603edd86e90 con 0x5603edd70170 2024-05-17T22:50:38.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.300+0000 7f44fd7fa700 1 -- 172.21.15.12:0/1074382665 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f450c00cd80 con 0x5603edd70170 2024-05-17T22:50:38.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.300+0000 7f44fd7fa700 1 -- 172.21.15.12:0/1074382665 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f45100e5030 con 0x5603edd70170 2024-05-17T22:50:38.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.304+0000 7f452a59ab80 10 client.25976 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:50:38.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.304+0000 7f452a59ab80 10 client.25976 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:50:38.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.304+0000 7f452a59ab80 20 client.25976 populate_metadata read hostname 'smithi012' 2024-05-17T22:50:38.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.304+0000 7f452a59ab80 1 --2- 172.21.15.12:0/1074382665 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5603edd89450 0x5603edd8b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:38.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.304+0000 7f452a59ab80 4 client.25976 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:50:38.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.304+0000 7f452a59ab80 1 -- 172.21.15.12:0/1074382665 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603edca01e0 con 0x5603edd89450 2024-05-17T22:50:38.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.307+0000 7f451995b700 1 --2- 172.21.15.12:0/1074382665 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5603edd89450 0x5603edd8b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:38.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.317+0000 7f451995b700 1 --2- 172.21.15.12:0/1074382665 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5603edd89450 0x5603edd8b900 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72c7b83761016486 server_cookie=cbcbf890c92b2ccd in_seq=0 out_seq=0 2024-05-17T22:50:38.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.318+0000 7f44fd7fa700 10 client.25976 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:50:38.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.324+0000 7f44fd7fa700 1 -- 172.21.15.12:0/1074382665 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5603edca01e0 con 0x5603edd89450 2024-05-17T22:50:38.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.324+0000 7f44fd7fa700 10 client.25976 handle_command_reply: tid=0 2024-05-17T22:50:38.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.326+0000 7f452a59ab80 1 client.25976 shutdown 2024-05-17T22:50:38.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.326+0000 7f452a59ab80 1 -- 172.21.15.12:0/1074382665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603edd70170 msgr2=0x5603edd797a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:38.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.326+0000 7f452a59ab80 1 --2- 172.21.15.12:0/1074382665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603edd70170 0x5603edd797a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4510009fd0 tx=0x7f45100a9b60 comp rx=0 tx=0).stop 2024-05-17T22:50:38.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.328+0000 7f452a59ab80 1 -- 172.21.15.12:0/1074382665 shutdown_connections 2024-05-17T22:50:38.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.328+0000 7f452a59ab80 1 --2- 172.21.15.12:0/1074382665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edd75aa0 0x5603edd79d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:38.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.328+0000 7f452a59ab80 1 --2- 172.21.15.12:0/1074382665 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5603edd89450 0x5603edd8b900 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:38.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.328+0000 7f452a59ab80 1 --2- 172.21.15.12:0/1074382665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603edbf19f0 0x5603edd79260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:38.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.328+0000 7f452a59ab80 1 --2- 172.21.15.12:0/1074382665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603edd70170 0x5603edd797a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:38.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.328+0000 7f44fd7fa700 0 client.25976 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:50:38.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.328+0000 7f452a59ab80 1 -- 172.21.15.12:0/1074382665 >> 172.21.15.12:0/1074382665 conn(0x5603edc35f00 msgr2=0x5603edd64ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:50:38.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.328+0000 7f452a59ab80 1 -- 172.21.15.12:0/1074382665 shutdown_connections 2024-05-17T22:50:38.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.328+0000 7f452a59ab80 1 -- 172.21.15.12:0/1074382665 wait complete. 2024-05-17T22:50:38.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.329+0000 7f452a59ab80 20 client.25976 trim_cache size 0 max 16384 2024-05-17T22:50:38.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.368+0000 7f452a59ab80 1 Processor -- start 2024-05-17T22:50:38.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.369+0000 7f452a59ab80 1 -- start start 2024-05-17T22:50:38.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.369+0000 7f452a59ab80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603edbf19f0 0x5603edd6fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:38.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.369+0000 7f452a59ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603edd70170 0x5603edd85140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:38.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.369+0000 7f452a59ab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edd75aa0 0x5603edd64eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:38.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.369+0000 7f451995b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603edbf19f0 0x5603edd6fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:38.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.369+0000 7f451995b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603edbf19f0 0x5603edd6fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:59242/0 (socket says 172.21.15.12:59242) 2024-05-17T22:50:38.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.370+0000 7f451995b700 1 -- 172.21.15.12:0/4230664849 learned_addr learned my addr 172.21.15.12:0/4230664849 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:50:38.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.370+0000 7f451a15c700 1 --2- 172.21.15.12:0/4230664849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edd75aa0 0x5603edd64eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:38.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.370+0000 7f451915a700 1 --2- 172.21.15.12:0/4230664849 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603edd70170 0x5603edd85140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:38.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.370+0000 7f452a59ab80 1 -- 172.21.15.12:0/4230664849 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f44f8011410 con 0x5603edd70170 2024-05-17T22:50:38.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.370+0000 7f452a59ab80 1 -- 172.21.15.12:0/4230664849 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f44f800eca0 con 0x5603edbf19f0 2024-05-17T22:50:38.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.370+0000 7f452a59ab80 1 -- 172.21.15.12:0/4230664849 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f44f800caf0 con 0x5603edd75aa0 2024-05-17T22:50:38.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.371+0000 7f451a15c700 1 -- 172.21.15.12:0/4230664849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603edbf19f0 msgr2=0x5603edd6fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:38.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.371+0000 7f451a15c700 1 --2- 172.21.15.12:0/4230664849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603edbf19f0 0x5603edd6fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:38.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.371+0000 7f451a15c700 1 -- 172.21.15.12:0/4230664849 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603edd70170 msgr2=0x5603edd85140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:38.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.371+0000 7f451a15c700 1 --2- 172.21.15.12:0/4230664849 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603edd70170 0x5603edd85140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:38.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.371+0000 7f451a15c700 1 -- 172.21.15.12:0/4230664849 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603edd85680 con 0x5603edd75aa0 2024-05-17T22:50:38.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.372+0000 7f451a15c700 1 --2- 172.21.15.12:0/4230664849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edd75aa0 0x5603edd64eb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x5603edd6b6a0 tx=0x7f450400a790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eebeb7029acaf540 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:50:38.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.376+0000 7f44ff7fe700 1 -- 172.21.15.12:0/4230664849 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450400f870 con 0x5603edd75aa0 2024-05-17T22:50:38.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.376+0000 7f44ff7fe700 1 -- 172.21.15.12:0/4230664849 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4504011440 con 0x5603edd75aa0 2024-05-17T22:50:38.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.376+0000 7f44ff7fe700 1 -- 172.21.15.12:0/4230664849 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4504020660 con 0x5603edd75aa0 2024-05-17T22:50:38.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.377+0000 7f452a59ab80 1 -- 172.21.15.12:0/4230664849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edd75aa0 msgr2=0x5603edd64eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:38.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.377+0000 7f452a59ab80 1 --2- 172.21.15.12:0/4230664849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edd75aa0 0x5603edd64eb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x5603edd6b6a0 tx=0x7f450400a790 comp rx=0 tx=0).stop 2024-05-17T22:50:38.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.378+0000 7f452a59ab80 1 -- 172.21.15.12:0/4230664849 shutdown_connections 2024-05-17T22:50:38.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.378+0000 7f452a59ab80 1 --2- 172.21.15.12:0/4230664849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edd75aa0 0x5603edd64eb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:38.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.378+0000 7f452a59ab80 1 --2- 172.21.15.12:0/4230664849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603edbf19f0 0x5603edd6fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:38.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.378+0000 7f452a59ab80 1 --2- 172.21.15.12:0/4230664849 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603edd70170 0x5603edd85140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:38.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.378+0000 7f452a59ab80 1 -- 172.21.15.12:0/4230664849 >> 172.21.15.12:0/4230664849 conn(0x5603edc35f00 msgr2=0x5603edd794e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:50:38.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.384+0000 7f452a59ab80 1 -- 172.21.15.12:0/4230664849 shutdown_connections 2024-05-17T22:50:38.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.384+0000 7f452a59ab80 1 -- 172.21.15.12:0/4230664849 wait complete. 2024-05-17T22:50:38.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.385+0000 7f452a59ab80 1 -- start start 2024-05-17T22:50:38.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.385+0000 7f452a59ab80 1 Processor -- start 2024-05-17T22:50:38.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.386+0000 7f452a59ab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edbf19f0 0x5603edd81b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:38.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.386+0000 7f452a59ab80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603edd70170 0x5603edd90770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:38.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.386+0000 7f452a59ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603edd75aa0 0x5603edd90dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:38.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.386+0000 7f452a59ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f44f800d160 con 0x5603edd75aa0 2024-05-17T22:50:38.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.386+0000 7f452a59ab80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f44f800e5d0 con 0x5603edd70170 2024-05-17T22:50:38.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.386+0000 7f452a59ab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f44f800d830 con 0x5603edbf19f0 2024-05-17T22:50:38.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.386+0000 7f452a59ab80 10 client.0 _ll_register_callbacks cb 0x5603edc28f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:50:38.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.387+0000 7f451915a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603edd70170 0x5603edd90770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:38.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.387+0000 7f451915a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603edd70170 0x5603edd90770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:59244/0 (socket says 172.21.15.12:59244) 2024-05-17T22:50:38.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.387+0000 7f451915a700 1 -- 172.21.15.12:0/2230396582 learned_addr learned my addr 172.21.15.12:0/2230396582 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:50:38.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.387+0000 7f451995b700 1 --2- 172.21.15.12:0/2230396582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edbf19f0 0x5603edd81b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:38.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.388+0000 7f451915a700 1 -- 172.21.15.12:0/2230396582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edbf19f0 msgr2=0x5603edd81b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:38.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.388+0000 7f451915a700 1 --2- 172.21.15.12:0/2230396582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edbf19f0 0x5603edd81b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:38.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.388+0000 7f451915a700 1 -- 172.21.15.12:0/2230396582 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603edd75aa0 msgr2=0x5603edd90dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:38.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.388+0000 7f451915a700 1 --2- 172.21.15.12:0/2230396582 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603edd75aa0 0x5603edd90dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:38.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.388+0000 7f451915a700 1 -- 172.21.15.12:0/2230396582 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603edd91540 con 0x5603edd70170 2024-05-17T22:50:38.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.390+0000 7f451915a700 1 --2- 172.21.15.12:0/2230396582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603edd70170 0x5603edd90770 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f45100aa020 tx=0x7f451000b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5465530c3ef283d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:50:38.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.390+0000 7f44fd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:50:38.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.390+0000 7f44fd7fa700 1 -- 172.21.15.12:0/2230396582 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45100aa0e0 con 0x5603edd70170 2024-05-17T22:50:38.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.390+0000 7f452a59ab80 1 -- 172.21.15.12:0/2230396582 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4504011440 con 0x5603edd70170 2024-05-17T22:50:38.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.390+0000 7f44fd7fa700 1 -- 172.21.15.12:0/2230396582 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f4510012d50 con 0x5603edd70170 2024-05-17T22:50:38.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.390+0000 7f44fd7fa700 1 -- 172.21.15.12:0/2230396582 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f45100b2a20 con 0x5603edd70170 2024-05-17T22:50:38.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.392+0000 7f452a59ab80 10 client.25982 fetch_fsmap learned FSMap version 39 2024-05-17T22:50:38.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.392+0000 7f452a59ab80 1 -- 172.21.15.12:0/2230396582 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5603ede04d10 con 0x5603edd70170 2024-05-17T22:50:38.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.392+0000 7f44fd7fa700 1 -- 172.21.15.12:0/2230396582 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4504011440 con 0x5603edd70170 2024-05-17T22:50:38.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.392+0000 7f44fd7fa700 1 -- 172.21.15.12:0/2230396582 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f45100e5060 con 0x5603edd70170 2024-05-17T22:50:38.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.393+0000 7f452a59ab80 10 client.25982 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:50:38.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.393+0000 7f452a59ab80 10 client.25982 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:50:38.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.393+0000 7f452a59ab80 20 client.25982 populate_metadata read hostname 'smithi012' 2024-05-17T22:50:38.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.393+0000 7f452a59ab80 1 --2- 172.21.15.12:0/2230396582 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5603ede05290 0x5603ede076b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:38.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.393+0000 7f452a59ab80 4 client.25982 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:50:38.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.393+0000 7f452a59ab80 1 -- 172.21.15.12:0/2230396582 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5603edc999c0 con 0x5603ede05290 2024-05-17T22:50:38.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.393+0000 7f451995b700 1 --2- 172.21.15.12:0/2230396582 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5603ede05290 0x5603ede076b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:38.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.393+0000 7f451995b700 1 --2- 172.21.15.12:0/2230396582 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5603ede05290 0x5603ede076b0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16f88a1ed286232d server_cookie=67c613f8db7ba205 in_seq=0 out_seq=0 2024-05-17T22:50:38.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.393+0000 7f44fd7fa700 10 client.25982 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:50:38.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.394+0000 7f44fd7fa700 1 -- 172.21.15.12:0/2230396582 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5603edc999c0 con 0x5603ede05290 2024-05-17T22:50:38.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.394+0000 7f44fd7fa700 10 client.25982 handle_command_reply: tid=0 2024-05-17T22:50:38.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.396+0000 7f452a59ab80 1 client.25982 shutdown 2024-05-17T22:50:38.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.407+0000 7f452a59ab80 1 -- 172.21.15.12:0/2230396582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603edd70170 msgr2=0x5603edd90770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:38.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.407+0000 7f452a59ab80 1 --2- 172.21.15.12:0/2230396582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603edd70170 0x5603edd90770 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f45100aa020 tx=0x7f451000b3b0 comp rx=0 tx=0).stop 2024-05-17T22:50:38.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.407+0000 7f452a59ab80 1 -- 172.21.15.12:0/2230396582 shutdown_connections 2024-05-17T22:50:38.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.407+0000 7f452a59ab80 1 --2- 172.21.15.12:0/2230396582 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5603ede05290 0x5603ede076b0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:38.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.407+0000 7f452a59ab80 1 --2- 172.21.15.12:0/2230396582 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603edd75aa0 0x5603edd90dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:38.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.407+0000 7f452a59ab80 1 --2- 172.21.15.12:0/2230396582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edbf19f0 0x5603edd81b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:38.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.407+0000 7f452a59ab80 1 --2- 172.21.15.12:0/2230396582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603edd70170 0x5603edd90770 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:38.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.407+0000 7f452a59ab80 1 -- 172.21.15.12:0/2230396582 >> 172.21.15.12:0/2230396582 conn(0x5603edc35f00 msgr2=0x5603edd78af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:50:38.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.407+0000 7f44fd7fa700 0 client.25982 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:50:38.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.407+0000 7f452a59ab80 1 -- 172.21.15.12:0/2230396582 shutdown_connections 2024-05-17T22:50:38.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.407+0000 7f452a59ab80 1 -- 172.21.15.12:0/2230396582 wait complete. 2024-05-17T22:50:38.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.407+0000 7f452a59ab80 20 client.25982 trim_cache size 0 max 16384 2024-05-17T22:50:38.411 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:50:38.411 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-17T22:50:38.411 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-17T22:50:38.411 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:50:38.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.410+0000 7f451bbbf700 1 -- 172.21.15.12:0/4255605092 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f45000b2a80 msgr2=0x7f45000b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:38.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.410+0000 7f451bbbf700 1 --2- 172.21.15.12:0/4255605092 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f45000b2a80 0x7f45000b4f90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x5603edd6bae0 tx=0x7f45040047c0 comp rx=0 tx=0).stop 2024-05-17T22:50:38.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.410+0000 7f451bbbf700 1 -- 172.21.15.12:0/4255605092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4514114c90 msgr2=0x7f451410dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:38.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.410+0000 7f451bbbf700 1 --2- 172.21.15.12:0/4255605092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4514114c90 0x7f451410dd90 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4510008000 tx=0x7f451000be50 comp rx=0 tx=0).stop 2024-05-17T22:50:38.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.414+0000 7f451bbbf700 1 -- 172.21.15.12:0/4255605092 shutdown_connections 2024-05-17T22:50:38.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.414+0000 7f451bbbf700 1 --2- 172.21.15.12:0/4255605092 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f45000b2a80 0x7f45000b4f90 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:38.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.414+0000 7f451bbbf700 1 --2- 172.21.15.12:0/4255605092 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f451410e2d0 0x7f4514112250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:38.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.414+0000 7f451bbbf700 1 --2- 172.21.15.12:0/4255605092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4514075bd0 0x7f451410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:38.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.414+0000 7f451bbbf700 1 --2- 172.21.15.12:0/4255605092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4514114c90 0x7f451410dd90 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:38.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.414+0000 7f451bbbf700 1 -- 172.21.15.12:0/4255605092 >> 172.21.15.12:0/4255605092 conn(0x7f45140ff990 msgr2=0x7f4514100250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:50:38.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.416+0000 7f451bbbf700 1 -- 172.21.15.12:0/4255605092 shutdown_connections 2024-05-17T22:50:38.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:38.416+0000 7f451bbbf700 1 -- 172.21.15.12:0/4255605092 wait complete. 2024-05-17T22:50:40.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:39 smithi096 ceph-mon[36065]: pgmap v781: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.5 MiB/s wr, 456 op/s 2024-05-17T22:50:40.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:39 smithi012 ceph-mon[31632]: pgmap v781: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.5 MiB/s wr, 456 op/s 2024-05-17T22:50:40.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:39 smithi151 ceph-mon[36101]: pgmap v781: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.5 MiB/s wr, 456 op/s 2024-05-17T22:50:40.795 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-17T22:50:40.795 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T22:50:41.365 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:50:41.371 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 680.0 s, 464.0 tps, lat 218.677 ms stddev 255.070 2024-05-17T22:50:42.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:41 smithi096 ceph-mon[36065]: pgmap v782: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.1 MiB/s wr, 441 op/s 2024-05-17T22:50:42.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:41 smithi012 ceph-mon[31632]: pgmap v782: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.1 MiB/s wr, 441 op/s 2024-05-17T22:50:42.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:41 smithi151 ceph-mon[36101]: pgmap v782: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.1 MiB/s wr, 441 op/s 2024-05-17T22:50:44.264 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:43 smithi012 ceph-mon[31632]: pgmap v783: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.0 MiB/s wr, 426 op/s 2024-05-17T22:50:44.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:43 smithi096 ceph-mon[36065]: pgmap v783: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.0 MiB/s wr, 426 op/s 2024-05-17T22:50:44.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:43 smithi151 ceph-mon[36101]: pgmap v783: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.0 MiB/s wr, 426 op/s 2024-05-17T22:50:44.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:44.662+0000 7f2791e87700 1 -- 172.21.15.12:0/2769869142 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f278c10a110 msgr2=0x7f278c10a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:44.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:44.662+0000 7f2791e87700 1 --2- 172.21.15.12:0/2769869142 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f278c10a110 0x7f278c10a520 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2780009990 tx=0x7f2780009ca0 comp rx=0 tx=0).stop 2024-05-17T22:50:44.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:44.662+0000 7f2791e87700 1 -- 172.21.15.12:0/2769869142 shutdown_connections 2024-05-17T22:50:44.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:44.662+0000 7f2791e87700 1 --2- 172.21.15.12:0/2769869142 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f278c10aa60 0x7f278c116ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:44.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:44.662+0000 7f2791e87700 1 --2- 172.21.15.12:0/2769869142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f278c1100e0 0x7f278c1104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:44.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:44.662+0000 7f2791e87700 1 --2- 172.21.15.12:0/2769869142 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f278c10a110 0x7f278c10a520 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:44.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:44.662+0000 7f2791e87700 1 -- 172.21.15.12:0/2769869142 >> 172.21.15.12:0/2769869142 conn(0x7f278c079b90 msgr2=0x7f278c108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:50:44.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:44.664+0000 7f2791e87700 1 -- 172.21.15.12:0/2769869142 shutdown_connections 2024-05-17T22:50:44.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:44.664+0000 7f2791e87700 1 -- 172.21.15.12:0/2769869142 wait complete. 2024-05-17T22:50:44.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:44.665+0000 7f2791e87700 1 Processor -- start 2024-05-17T22:50:44.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:44.665+0000 7f2791e87700 1 -- start start 2024-05-17T22:50:44.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:44.666+0000 7f2791e87700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f278c10a110 0x7f278c1a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:44.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:44.666+0000 7f2791e87700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f278c10aa60 0x7f278c1a1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:44.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:44.666+0000 7f2790e85700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f278c10a110 0x7f278c1a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:44.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:44.667+0000 7f2790e85700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f278c10a110 0x7f278c1a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58940/0 (socket says 172.21.15.12:58940) 2024-05-17T22:50:44.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:44.667+0000 7f2790e85700 1 -- 172.21.15.12:0/4081326464 learned_addr learned my addr 172.21.15.12:0/4081326464 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:50:44.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:44.667+0000 7f2791e87700 1 --2- 172.21.15.12:0/4081326464 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f278c1100e0 0x7f278c1a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:44.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:44.667+0000 7f2791e87700 1 -- 172.21.15.12:0/4081326464 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f278c11a390 con 0x7f278c10a110 2024-05-17T22:50:44.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:44.667+0000 7f278bfff700 1 --2- 172.21.15.12:0/4081326464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f278c10aa60 0x7f278c1a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:44.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:44.667+0000 7f2791686700 1 --2- 172.21.15.12:0/4081326464 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f278c1100e0 0x7f278c1a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:44.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:44.667+0000 7f2791e87700 1 -- 172.21.15.12:0/4081326464 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f278c11a210 con 0x7f278c1100e0 2024-05-17T22:50:44.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:44.667+0000 7f2791e87700 1 -- 172.21.15.12:0/4081326464 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f278c11a510 con 0x7f278c10aa60 2024-05-17T22:50:44.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:44.668+0000 7f2791686700 1 -- 172.21.15.12:0/4081326464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f278c10aa60 msgr2=0x7f278c1a1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:44.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:44.668+0000 7f2791686700 1 --2- 172.21.15.12:0/4081326464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f278c10aa60 0x7f278c1a1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:44.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:44.668+0000 7f2791686700 1 -- 172.21.15.12:0/4081326464 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f278c10a110 msgr2=0x7f278c1a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:44.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:44.668+0000 7f2791686700 1 --2- 172.21.15.12:0/4081326464 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f278c10a110 0x7f278c1a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:44.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:44.668+0000 7f2791686700 1 -- 172.21.15.12:0/4081326464 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f278c1a5c60 con 0x7f278c1100e0 2024-05-17T22:50:44.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:44.668+0000 7f2790e85700 1 --2- 172.21.15.12:0/4081326464 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f278c10a110 0x7f278c1a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:50:44.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:44.668+0000 7f2791686700 1 --2- 172.21.15.12:0/4081326464 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f278c1100e0 0x7f278c1a55c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f278400be20 tx=0x7f2784009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:50:44.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:44.668+0000 7f2789ffb700 1 -- 172.21.15.12:0/4081326464 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2784019070 con 0x7f278c1100e0 2024-05-17T22:50:44.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:44.669+0000 7f2789ffb700 1 -- 172.21.15.12:0/4081326464 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f278400cc10 con 0x7f278c1100e0 2024-05-17T22:50:44.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:44.669+0000 7f2789ffb700 1 -- 172.21.15.12:0/4081326464 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27840125a0 con 0x7f278c1100e0 2024-05-17T22:50:44.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:44.669+0000 7f2791e87700 1 -- 172.21.15.12:0/4081326464 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f278c1a5f50 con 0x7f278c1100e0 2024-05-17T22:50:44.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:44.669+0000 7f2791e87700 1 -- 172.21.15.12:0/4081326464 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f278c1ad870 con 0x7f278c1100e0 2024-05-17T22:50:44.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:44.672+0000 7f27777fe700 1 -- 172.21.15.12:0/4081326464 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f278c10a5a0 con 0x7f278c1100e0 2024-05-17T22:50:44.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:44.674+0000 7f2789ffb700 1 -- 172.21.15.12:0/4081326464 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f278401c070 con 0x7f278c1100e0 2024-05-17T22:50:44.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:44.675+0000 7f2789ffb700 1 --2- 172.21.15.12:0/4081326464 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f277c0b2940 0x7f277c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:44.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:44.676+0000 7f2789ffb700 1 -- 172.21.15.12:0/4081326464 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f27840e1c00 con 0x7f278c1100e0 2024-05-17T22:50:44.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:44.680+0000 7f2790e85700 1 --2- 172.21.15.12:0/4081326464 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f277c0b2940 0x7f277c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:44.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:44.684+0000 7f2790e85700 1 --2- 172.21.15.12:0/4081326464 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f277c0b2940 0x7f277c0b4e50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2778009a70 tx=0x7f2778008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:50:44.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:44.696+0000 7f2789ffb700 1 -- 172.21.15.12:0/4081326464 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f278400cdc0 con 0x7f278c1100e0 2024-05-17T22:50:45.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:45.195+0000 7f27777fe700 1 -- 172.21.15.12:0/4081326464 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f278c064860 con 0x7f278c1100e0 2024-05-17T22:50:45.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:45.197+0000 7f2789ffb700 1 -- 172.21.15.12:0/4081326464 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11870 (secure 0 0 0) 0x7f27840a9530 con 0x7f278c1100e0 2024-05-17T22:50:45.201 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:50:45.201 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1407219319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1407219319},{"type":"v1","addr":"172.21.15.12:6835","nonce":1407219319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/630977886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":630977886},{"type":"v1","addr":"172.21.15.12:6837","nonce":630977886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3731914742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3731914742},{"type":"v1","addr":"172.21.15.12:6839","nonce":3731914742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/3642142501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3642142501},{"type":"v1","addr":"172.21.15.12:6841","nonce":3642142501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3554053059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3554053059},{"type":"v1","addr":"172.21.15.151:6833","nonce":3554053059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/4228325569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":4228325569},{"type":"v1","addr":"172.21.15.96:6833","nonce":4228325569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T22:34:42.642111+0000","modified":"2024-05-17T22:47:38.155618+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24461,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/542323754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":542323754},{"type":"v1","addr":"172.21.15.151:6835","nonce":542323754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1758442232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1758442232},{"type":"v1","addr":"172.21.15.96:6835","nonce":1758442232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3403324038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3403324038},{"type":"v1","addr":"172.21.15.151:6837","nonce":3403324038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.151:6839/2338353849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2338353849},{"type":"v1","addr":"172.21.15.151:6839","nonce":2338353849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.96:6837/4222574338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":4222574338},{"type":"v1","addr":"172.21.15.96:6837","nonce":4222574338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/2986295519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2986295519},{"type":"v1","addr":"172.21.15.96:6839","nonce":2986295519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24457,24461,24482]},"id":1}]} 2024-05-17T22:50:45.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:45.205+0000 7f27777fe700 1 -- 172.21.15.12:0/4081326464 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f277c0b2940 msgr2=0x7f277c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:45.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:45.205+0000 7f27777fe700 1 --2- 172.21.15.12:0/4081326464 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f277c0b2940 0x7f277c0b4e50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2778009a70 tx=0x7f2778008040 comp rx=0 tx=0).stop 2024-05-17T22:50:45.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:45.205+0000 7f27777fe700 1 -- 172.21.15.12:0/4081326464 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f278c1100e0 msgr2=0x7f278c1a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:45.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:45.205+0000 7f27777fe700 1 --2- 172.21.15.12:0/4081326464 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f278c1100e0 0x7f278c1a55c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f278400be20 tx=0x7f2784009ed0 comp rx=0 tx=0).stop 2024-05-17T22:50:45.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:45.209+0000 7f27777fe700 1 -- 172.21.15.12:0/4081326464 shutdown_connections 2024-05-17T22:50:45.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:45.209+0000 7f27777fe700 1 --2- 172.21.15.12:0/4081326464 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f277c0b2940 0x7f277c0b4e50 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:45.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:45.209+0000 7f27777fe700 1 --2- 172.21.15.12:0/4081326464 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f278c1100e0 0x7f278c1a55c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:45.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:45.209+0000 7f27777fe700 1 --2- 172.21.15.12:0/4081326464 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f278c10a110 0x7f278c1a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:45.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:45.209+0000 7f27777fe700 1 --2- 172.21.15.12:0/4081326464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f278c10aa60 0x7f278c1a1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:45.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:45.209+0000 7f27777fe700 1 -- 172.21.15.12:0/4081326464 >> 172.21.15.12:0/4081326464 conn(0x7f278c079b90 msgr2=0x7f278c07ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:50:45.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:45.210+0000 7f27777fe700 1 -- 172.21.15.12:0/4081326464 shutdown_connections 2024-05-17T22:50:45.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:45.210+0000 7f27777fe700 1 -- 172.21.15.12:0/4081326464 wait complete. 2024-05-17T22:50:45.214 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 39 2024-05-17T22:50:46.086 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-17T22:50:46.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:46 smithi096 ceph-mon[36065]: pgmap v784: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.1 MiB/s wr, 482 op/s 2024-05-17T22:50:46.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:46 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/4081326464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:50:46.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:46 smithi012 ceph-mon[31632]: pgmap v784: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.1 MiB/s wr, 482 op/s 2024-05-17T22:50:46.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:46 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/4081326464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:50:46.371 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 685.0 s, 540.0 tps, lat 179.116 ms stddev 212.684 2024-05-17T22:50:46.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:46 smithi151 ceph-mon[36101]: pgmap v784: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.1 MiB/s wr, 482 op/s 2024-05-17T22:50:46.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:46 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/4081326464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:50:46.532 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:50:48.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:48 smithi096 ceph-mon[36065]: pgmap v785: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.6 MiB/s wr, 441 op/s 2024-05-17T22:50:48.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:48 smithi012 ceph-mon[31632]: pgmap v785: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.6 MiB/s wr, 441 op/s 2024-05-17T22:50:48.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:48 smithi151 ceph-mon[36101]: pgmap v785: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.6 MiB/s wr, 441 op/s 2024-05-17T22:50:50.288 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:50 smithi012 ceph-mon[31632]: pgmap v786: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.4 MiB/s wr, 445 op/s 2024-05-17T22:50:50.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:50 smithi096 ceph-mon[36065]: pgmap v786: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.4 MiB/s wr, 445 op/s 2024-05-17T22:50:50.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:50 smithi151 ceph-mon[36101]: pgmap v786: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.4 MiB/s wr, 445 op/s 2024-05-17T22:50:50.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.785+0000 7f461ba7e700 1 -- 172.21.15.12:0/3932347559 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4614108870 msgr2=0x7f4614108c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:50.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.785+0000 7f461ba7e700 1 --2- 172.21.15.12:0/3932347559 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4614108870 0x7f4614108c80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f46100098f0 tx=0x7f4610009c00 comp rx=0 tx=0).stop 2024-05-17T22:50:50.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.785+0000 7f461ba7e700 1 -- 172.21.15.12:0/3932347559 shutdown_connections 2024-05-17T22:50:50.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.785+0000 7f461ba7e700 1 --2- 172.21.15.12:0/3932347559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46141091c0 0x7f4614110b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:50.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.785+0000 7f461ba7e700 1 --2- 172.21.15.12:0/3932347559 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4614102450 0x7f4614102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:50.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.785+0000 7f461ba7e700 1 --2- 172.21.15.12:0/3932347559 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4614108870 0x7f4614108c80 secure :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f46100098f0 tx=0x7f4610009c00 comp rx=0 tx=0).stop 2024-05-17T22:50:50.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.785+0000 7f461ba7e700 1 -- 172.21.15.12:0/3932347559 >> 172.21.15.12:0/3932347559 conn(0x7f46140fe030 msgr2=0x7f4614100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:50:50.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.785+0000 7f461ba7e700 1 -- 172.21.15.12:0/3932347559 shutdown_connections 2024-05-17T22:50:50.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.785+0000 7f461ba7e700 1 -- 172.21.15.12:0/3932347559 wait complete. 2024-05-17T22:50:50.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.786+0000 7f461ba7e700 1 Processor -- start 2024-05-17T22:50:50.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.786+0000 7f461ba7e700 1 -- start start 2024-05-17T22:50:50.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.786+0000 7f461ba7e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4614102450 0x7f4614199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:50.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.786+0000 7f461ba7e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46141091c0 0x7f4614199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:50.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.786+0000 7f461ba7e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46141a0340 0x7f46141a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:50.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.786+0000 7f461ba7e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4614113e70 con 0x7f46141a0340 2024-05-17T22:50:50.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.786+0000 7f461ba7e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4614113cf0 con 0x7f46141091c0 2024-05-17T22:50:50.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.786+0000 7f461ba7e700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4614113ff0 con 0x7f4614102450 2024-05-17T22:50:50.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.787+0000 7f4619019700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46141091c0 0x7f4614199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:50.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.787+0000 7f4619019700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46141091c0 0x7f4614199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:52988/0 (socket says 172.21.15.12:52988) 2024-05-17T22:50:50.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.787+0000 7f4619019700 1 -- 172.21.15.12:0/2124696474 learned_addr learned my addr 172.21.15.12:0/2124696474 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:50:50.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.787+0000 7f461a01b700 1 --2- 172.21.15.12:0/2124696474 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46141a0340 0x7f46141a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:50.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.787+0000 7f4619019700 1 -- 172.21.15.12:0/2124696474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4614102450 msgr2=0x7f4614199570 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-17T22:50:50.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.787+0000 7f4619019700 1 --2- 172.21.15.12:0/2124696474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4614102450 0x7f4614199570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:50.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.787+0000 7f4619019700 1 -- 172.21.15.12:0/2124696474 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46141a0340 msgr2=0x7f46141a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:50.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.787+0000 7f4619019700 1 --2- 172.21.15.12:0/2124696474 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46141a0340 0x7f46141a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:50.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.787+0000 7f4619019700 1 -- 172.21.15.12:0/2124696474 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46141a0e50 con 0x7f46141091c0 2024-05-17T22:50:50.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.791+0000 7f4619019700 1 --2- 172.21.15.12:0/2124696474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46141091c0 0x7f4614199ab0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4610000c00 tx=0x7f461000fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:50:50.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.792+0000 7f460affd700 1 -- 172.21.15.12:0/2124696474 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46100051c0 con 0x7f46141091c0 2024-05-17T22:50:50.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.792+0000 7f460affd700 1 -- 172.21.15.12:0/2124696474 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f461000b560 con 0x7f46141091c0 2024-05-17T22:50:50.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.792+0000 7f460affd700 1 -- 172.21.15.12:0/2124696474 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46100039b0 con 0x7f46141091c0 2024-05-17T22:50:50.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.792+0000 7f461ba7e700 1 -- 172.21.15.12:0/2124696474 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46141a10e0 con 0x7f46141091c0 2024-05-17T22:50:50.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.792+0000 7f461ba7e700 1 -- 172.21.15.12:0/2124696474 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46141a1540 con 0x7f46141091c0 2024-05-17T22:50:50.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.796+0000 7f460affd700 1 -- 172.21.15.12:0/2124696474 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f461000b8c0 con 0x7f46141091c0 2024-05-17T22:50:50.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.797+0000 7f460affd700 1 --2- 172.21.15.12:0/2124696474 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f46000b2900 0x7f46000b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:50.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.797+0000 7f460affd700 1 -- 172.21.15.12:0/2124696474 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f46100e3530 con 0x7f46141091c0 2024-05-17T22:50:50.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.797+0000 7f461981a700 1 --2- 172.21.15.12:0/2124696474 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f46000b2900 0x7f46000b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:50.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.798+0000 7f461981a700 1 --2- 172.21.15.12:0/2124696474 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f46000b2900 0x7f46000b4e10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4604009af0 tx=0x7f4604009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:50:50.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.807+0000 7f462a459b80 1 Processor -- start 2024-05-17T22:50:50.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.808+0000 7f462a459b80 1 -- start start 2024-05-17T22:50:50.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.808+0000 7f462a459b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1a38e9f0 0x55dd1a511ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:50.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.808+0000 7f462a459b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd1a512400 0x55dd1a50c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:50.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.808+0000 7f462a459b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dd1a50cb00 0x55dd1a50cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:50.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.808+0000 7f462a459b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55dd1a43b0d0 con 0x55dd1a512400 2024-05-17T22:50:50.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.809+0000 7f4619019700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd1a512400 0x55dd1a50c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:50.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.809+0000 7f4619019700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd1a512400 0x55dd1a50c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49478/0 (socket says 172.21.15.12:49478) 2024-05-17T22:50:50.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.809+0000 7f4619019700 1 -- 172.21.15.12:0/1506315135 learned_addr learned my addr 172.21.15.12:0/1506315135 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:50:50.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.809+0000 7f462a459b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55dd1a42ac70 con 0x55dd1a38e9f0 2024-05-17T22:50:50.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.809+0000 7f461981a700 1 --2- 172.21.15.12:0/1506315135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1a38e9f0 0x55dd1a511ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:50.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.809+0000 7f462a459b80 1 -- 172.21.15.12:0/1506315135 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55dd1a42ecb0 con 0x55dd1a50cb00 2024-05-17T22:50:50.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.809+0000 7f461a01b700 1 --2- 172.21.15.12:0/1506315135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dd1a50cb00 0x55dd1a50cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:50.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.809+0000 7f4619019700 1 -- 172.21.15.12:0/1506315135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dd1a50cb00 msgr2=0x55dd1a50cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:50.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.809+0000 7f4619019700 1 --2- 172.21.15.12:0/1506315135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dd1a50cb00 0x55dd1a50cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:50.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.809+0000 7f4619019700 1 -- 172.21.15.12:0/1506315135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1a38e9f0 msgr2=0x55dd1a511ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:50.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.809+0000 7f4619019700 1 --2- 172.21.15.12:0/1506315135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1a38e9f0 0x55dd1a511ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:50.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.809+0000 7f4619019700 1 -- 172.21.15.12:0/1506315135 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd1a3c45c0 con 0x55dd1a512400 2024-05-17T22:50:50.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.809+0000 7f461a01b700 1 --2- 172.21.15.12:0/1506315135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dd1a50cb00 0x55dd1a50cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:50:50.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.809+0000 7f461981a700 1 --2- 172.21.15.12:0/1506315135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1a38e9f0 0x55dd1a511ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:50:50.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.810+0000 7f4619019700 1 --2- 172.21.15.12:0/1506315135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd1a512400 0x55dd1a50c5c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f46100a68d0 tx=0x7f4610009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b98e6109829419 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:50:50.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.810+0000 7f45fffff700 1 -- 172.21.15.12:0/1506315135 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46100aa360 con 0x55dd1a512400 2024-05-17T22:50:50.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.810+0000 7f45fffff700 1 -- 172.21.15.12:0/1506315135 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46100aaf50 con 0x55dd1a512400 2024-05-17T22:50:50.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.810+0000 7f45fffff700 1 -- 172.21.15.12:0/1506315135 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461002bed0 con 0x55dd1a512400 2024-05-17T22:50:50.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.812+0000 7f462a459b80 1 -- 172.21.15.12:0/1506315135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd1a512400 msgr2=0x55dd1a50c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:50.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.812+0000 7f462a459b80 1 --2- 172.21.15.12:0/1506315135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd1a512400 0x55dd1a50c5c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f46100a68d0 tx=0x7f4610009c00 comp rx=0 tx=0).stop 2024-05-17T22:50:50.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.812+0000 7f462a459b80 1 -- 172.21.15.12:0/1506315135 shutdown_connections 2024-05-17T22:50:50.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.812+0000 7f462a459b80 1 --2- 172.21.15.12:0/1506315135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dd1a50cb00 0x55dd1a50cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:50.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.812+0000 7f462a459b80 1 --2- 172.21.15.12:0/1506315135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1a38e9f0 0x55dd1a511ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:50.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.812+0000 7f462a459b80 1 --2- 172.21.15.12:0/1506315135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd1a512400 0x55dd1a50c5c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:50.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.812+0000 7f462a459b80 1 -- 172.21.15.12:0/1506315135 >> 172.21.15.12:0/1506315135 conn(0x55dd1a3ee7d0 msgr2=0x55dd1a5071c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:50:50.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.812+0000 7f462a459b80 1 -- 172.21.15.12:0/1506315135 shutdown_connections 2024-05-17T22:50:50.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.812+0000 7f462a459b80 1 -- 172.21.15.12:0/1506315135 wait complete. 2024-05-17T22:50:50.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.813+0000 7f462a459b80 1 -- start start 2024-05-17T22:50:50.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.813+0000 7f462a459b80 1 Processor -- start 2024-05-17T22:50:50.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.813+0000 7f462a459b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd1a38e9f0 0x55dd1a515ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:50.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.813+0000 7f462a459b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1a50cb00 0x55dd1a516020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:50.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.813+0000 7f462a459b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dd1a512400 0x55dd1a5165a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:50.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.813+0000 7f461981a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd1a38e9f0 0x55dd1a515ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:50.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.813+0000 7f462a459b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55dd1a332150 con 0x55dd1a38e9f0 2024-05-17T22:50:50.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.813+0000 7f461981a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd1a38e9f0 0x55dd1a515ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49490/0 (socket says 172.21.15.12:49490) 2024-05-17T22:50:50.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.814+0000 7f461981a700 1 -- 172.21.15.12:0/2476745835 learned_addr learned my addr 172.21.15.12:0/2476745835 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:50:50.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.813+0000 7f462a459b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55dd1a3cc940 con 0x55dd1a50cb00 2024-05-17T22:50:50.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.814+0000 7f462a459b80 1 -- 172.21.15.12:0/2476745835 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55dd1a43b0d0 con 0x55dd1a512400 2024-05-17T22:50:50.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.814+0000 7f4619019700 1 --2- 172.21.15.12:0/2476745835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1a50cb00 0x55dd1a516020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:50.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.814+0000 7f462a459b80 10 client.0 _ll_register_callbacks cb 0x55dd1a3c1b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:50:50.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.814+0000 7f461a01b700 1 --2- 172.21.15.12:0/2476745835 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dd1a512400 0x55dd1a5165a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:50.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.815+0000 7f461a01b700 1 -- 172.21.15.12:0/2476745835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1a50cb00 msgr2=0x55dd1a516020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:50.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.815+0000 7f461a01b700 1 --2- 172.21.15.12:0/2476745835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1a50cb00 0x55dd1a516020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:50.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.815+0000 7f461a01b700 1 -- 172.21.15.12:0/2476745835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd1a38e9f0 msgr2=0x55dd1a515ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:50.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.815+0000 7f461a01b700 1 --2- 172.21.15.12:0/2476745835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd1a38e9f0 0x55dd1a515ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:50.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.815+0000 7f461a01b700 1 -- 172.21.15.12:0/2476745835 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd1a37b480 con 0x55dd1a512400 2024-05-17T22:50:50.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.815+0000 7f4619019700 1 --2- 172.21.15.12:0/2476745835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1a50cb00 0x55dd1a516020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:50:50.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.815+0000 7f461a01b700 1 --2- 172.21.15.12:0/2476745835 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dd1a512400 0x55dd1a5165a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f460c009cd0 tx=0x7f460c007bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=945980c92ec8ba76 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:50:50.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.815+0000 7f45fd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:50:50.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.816+0000 7f45fd7fa700 1 -- 172.21.15.12:0/2476745835 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460c01a030 con 0x55dd1a512400 2024-05-17T22:50:50.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.816+0000 7f45fd7fa700 1 -- 172.21.15.12:0/2476745835 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f460c015720 con 0x55dd1a512400 2024-05-17T22:50:50.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.816+0000 7f462a459b80 1 -- 172.21.15.12:0/2476745835 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46100aaf50 con 0x55dd1a512400 2024-05-17T22:50:50.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.816+0000 7f45fd7fa700 1 -- 172.21.15.12:0/2476745835 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f460c027050 con 0x55dd1a512400 2024-05-17T22:50:50.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.816+0000 7f462a459b80 10 client.25930 fetch_fsmap learned FSMap version 39 2024-05-17T22:50:50.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.816+0000 7f462a459b80 1 -- 172.21.15.12:0/2476745835 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55dd1a523560 con 0x55dd1a512400 2024-05-17T22:50:50.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.817+0000 7f45fd7fa700 1 -- 172.21.15.12:0/2476745835 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46100aaf50 con 0x55dd1a512400 2024-05-17T22:50:50.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.817+0000 7f45fd7fa700 1 -- 172.21.15.12:0/2476745835 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f460c016b50 con 0x55dd1a512400 2024-05-17T22:50:50.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.817+0000 7f462a459b80 10 client.25930 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:50:50.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.817+0000 7f462a459b80 10 client.25930 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-17T22:50:50.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.817+0000 7f462a459b80 20 client.25930 populate_metadata read hostname 'smithi012' 2024-05-17T22:50:50.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.817+0000 7f462a459b80 1 --2- 172.21.15.12:0/2476745835 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55dd1a525ba0 0x55dd1a527fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:50.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.817+0000 7f462a459b80 4 client.25930 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:50:50.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.817+0000 7f462a459b80 1 -- 172.21.15.12:0/2476745835 --> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd1a43d020 con 0x55dd1a525ba0 2024-05-17T22:50:50.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.817+0000 7f461981a700 1 --2- 172.21.15.12:0/2476745835 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55dd1a525ba0 0x55dd1a527fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:50.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.818+0000 7f45fd7fa700 10 client.25930 ms_handle_connect on v2:172.21.15.151:6838/2338353849 2024-05-17T22:50:50.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.818+0000 7f461981a700 1 --2- 172.21.15.12:0/2476745835 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55dd1a525ba0 0x55dd1a527fc0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b4f394e6f39e0693 server_cookie=6786664eb33570a0 in_seq=0 out_seq=0 2024-05-17T22:50:50.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.819+0000 7f45fd7fa700 1 -- 172.21.15.12:0/2476745835 <== mds.2 v2:172.21.15.151:6838/2338353849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dd1a43d020 con 0x55dd1a525ba0 2024-05-17T22:50:50.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.819+0000 7f45fd7fa700 10 client.25930 handle_command_reply: tid=0 2024-05-17T22:50:50.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.819+0000 7f462a459b80 1 client.25930 shutdown 2024-05-17T22:50:50.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.819+0000 7f462a459b80 1 -- 172.21.15.12:0/2476745835 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dd1a512400 msgr2=0x55dd1a5165a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:50.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.819+0000 7f462a459b80 1 --2- 172.21.15.12:0/2476745835 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dd1a512400 0x55dd1a5165a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f460c009cd0 tx=0x7f460c007bb0 comp rx=0 tx=0).stop 2024-05-17T22:50:50.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.819+0000 7f462a459b80 1 -- 172.21.15.12:0/2476745835 shutdown_connections 2024-05-17T22:50:50.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.819+0000 7f462a459b80 1 --2- 172.21.15.12:0/2476745835 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55dd1a525ba0 0x55dd1a527fc0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:50.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.820+0000 7f462a459b80 1 --2- 172.21.15.12:0/2476745835 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dd1a512400 0x55dd1a5165a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:50.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.820+0000 7f462a459b80 1 --2- 172.21.15.12:0/2476745835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd1a38e9f0 0x55dd1a515ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:50.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.820+0000 7f462a459b80 1 --2- 172.21.15.12:0/2476745835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1a50cb00 0x55dd1a516020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:50.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.820+0000 7f45fd7fa700 0 client.25930 ms_handle_reset on v2:172.21.15.151:6838/2338353849 2024-05-17T22:50:50.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.820+0000 7f462a459b80 1 -- 172.21.15.12:0/2476745835 >> 172.21.15.12:0/2476745835 conn(0x55dd1a3ee7d0 msgr2=0x55dd1a507160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:50:50.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.820+0000 7f462a459b80 1 -- 172.21.15.12:0/2476745835 shutdown_connections 2024-05-17T22:50:50.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.820+0000 7f462a459b80 1 -- 172.21.15.12:0/2476745835 wait complete. 2024-05-17T22:50:50.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.820+0000 7f462a459b80 20 client.25930 trim_cache size 0 max 16384 2024-05-17T22:50:50.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.857+0000 7f462a459b80 1 Processor -- start 2024-05-17T22:50:50.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.862+0000 7f462a459b80 1 -- start start 2024-05-17T22:50:50.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.862+0000 7f462a459b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd1a38e9f0 0x55dd1a50c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:50.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.862+0000 7f462a459b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1a50cb00 0x55dd1a50cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:50.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.862+0000 7f462a459b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dd1a512400 0x55dd1a516030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:50.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.862+0000 7f462a459b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f46141ca230 con 0x55dd1a38e9f0 2024-05-17T22:50:50.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.862+0000 7f462a459b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f46141c7b30 con 0x55dd1a50cb00 2024-05-17T22:50:50.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.862+0000 7f462a459b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4614073840 con 0x55dd1a512400 2024-05-17T22:50:50.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.866+0000 7f461a01b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dd1a512400 0x55dd1a516030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:50.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.868+0000 7f461981a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd1a38e9f0 0x55dd1a50c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:50.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.868+0000 7f461981a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd1a38e9f0 0x55dd1a50c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49498/0 (socket says 172.21.15.12:49498) 2024-05-17T22:50:50.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.868+0000 7f461981a700 1 -- 172.21.15.12:0/202568661 learned_addr learned my addr 172.21.15.12:0/202568661 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:50:50.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.870+0000 7f4619019700 1 --2- 172.21.15.12:0/202568661 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1a50cb00 0x55dd1a50cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:50.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.871+0000 7f461981a700 1 -- 172.21.15.12:0/202568661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dd1a512400 msgr2=0x55dd1a516030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:50.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.871+0000 7f461981a700 1 --2- 172.21.15.12:0/202568661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dd1a512400 0x55dd1a516030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:50.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.871+0000 7f461981a700 1 -- 172.21.15.12:0/202568661 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1a50cb00 msgr2=0x55dd1a50cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:50.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.871+0000 7f461981a700 1 --2- 172.21.15.12:0/202568661 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1a50cb00 0x55dd1a50cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:50.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.871+0000 7f461981a700 1 -- 172.21.15.12:0/202568661 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd1a507ab0 con 0x55dd1a38e9f0 2024-05-17T22:50:50.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.880+0000 7f461981a700 1 --2- 172.21.15.12:0/202568661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd1a38e9f0 0x55dd1a50c5c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f460400ee90 tx=0x7f4604011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=643d6edd697ed1f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:50:50.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.882+0000 7f45ff7fe700 1 -- 172.21.15.12:0/202568661 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460401e0d0 con 0x55dd1a38e9f0 2024-05-17T22:50:50.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.882+0000 7f45ff7fe700 1 -- 172.21.15.12:0/202568661 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f460401e270 con 0x55dd1a38e9f0 2024-05-17T22:50:50.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.882+0000 7f45ff7fe700 1 -- 172.21.15.12:0/202568661 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4604005020 con 0x55dd1a38e9f0 2024-05-17T22:50:50.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.884+0000 7f462a459b80 1 -- 172.21.15.12:0/202568661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd1a38e9f0 msgr2=0x55dd1a50c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:50.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.884+0000 7f462a459b80 1 --2- 172.21.15.12:0/202568661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd1a38e9f0 0x55dd1a50c5c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f460400ee90 tx=0x7f4604011000 comp rx=0 tx=0).stop 2024-05-17T22:50:50.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.895+0000 7f462a459b80 1 -- 172.21.15.12:0/202568661 shutdown_connections 2024-05-17T22:50:50.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.895+0000 7f462a459b80 1 --2- 172.21.15.12:0/202568661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dd1a512400 0x55dd1a516030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:50.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.895+0000 7f462a459b80 1 --2- 172.21.15.12:0/202568661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd1a38e9f0 0x55dd1a50c5c0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:50.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.895+0000 7f462a459b80 1 --2- 172.21.15.12:0/202568661 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1a50cb00 0x55dd1a50cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:50.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.895+0000 7f462a459b80 1 -- 172.21.15.12:0/202568661 >> 172.21.15.12:0/202568661 conn(0x55dd1a3ee7d0 msgr2=0x55dd1a5143b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:50:50.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.900+0000 7f462a459b80 1 -- 172.21.15.12:0/202568661 shutdown_connections 2024-05-17T22:50:50.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.900+0000 7f462a459b80 1 -- 172.21.15.12:0/202568661 wait complete. 2024-05-17T22:50:50.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.900+0000 7f462a459b80 1 -- start start 2024-05-17T22:50:50.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.901+0000 7f462a459b80 1 Processor -- start 2024-05-17T22:50:50.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.901+0000 7f462a459b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1a38e9f0 0x55dd1a51e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:50.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.902+0000 7f462a459b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dd1a50cb00 0x55dd1a5160a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:50.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.902+0000 7f462a459b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd1a512400 0x55dd1a5165e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:50.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.902+0000 7f462a459b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4614073eb0 con 0x55dd1a512400 2024-05-17T22:50:50.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.902+0000 7f462a459b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4614075320 con 0x55dd1a38e9f0 2024-05-17T22:50:50.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.902+0000 7f462a459b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4614074580 con 0x55dd1a50cb00 2024-05-17T22:50:50.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.902+0000 7f462a459b80 10 client.0 _ll_register_callbacks cb 0x55dd1a3c1b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:50:50.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.903+0000 7f461a01b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd1a512400 0x55dd1a5165e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:50.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.903+0000 7f461a01b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd1a512400 0x55dd1a5165e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49514/0 (socket says 172.21.15.12:49514) 2024-05-17T22:50:50.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.903+0000 7f461a01b700 1 -- 172.21.15.12:0/1253045664 learned_addr learned my addr 172.21.15.12:0/1253045664 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:50:50.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.907+0000 7f461981a700 1 --2- 172.21.15.12:0/1253045664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1a38e9f0 0x55dd1a51e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:50.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.909+0000 7f461a01b700 1 -- 172.21.15.12:0/1253045664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dd1a50cb00 msgr2=0x55dd1a5160a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T22:50:50.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.909+0000 7f461a01b700 1 --2- 172.21.15.12:0/1253045664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dd1a50cb00 0x55dd1a5160a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:50.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.909+0000 7f461a01b700 1 -- 172.21.15.12:0/1253045664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1a38e9f0 msgr2=0x55dd1a51e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:50.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.909+0000 7f461a01b700 1 --2- 172.21.15.12:0/1253045664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1a38e9f0 0x55dd1a51e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:50.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.909+0000 7f461a01b700 1 -- 172.21.15.12:0/1253045664 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd1a515020 con 0x55dd1a512400 2024-05-17T22:50:50.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.910+0000 7f461a01b700 1 --2- 172.21.15.12:0/1253045664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd1a512400 0x55dd1a5165e0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f460c009cd0 tx=0x7f460c005d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8120a53f08015135 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:50:50.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.910+0000 7f45fd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:50:50.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.911+0000 7f45fd7fa700 1 -- 172.21.15.12:0/1253045664 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460c01a040 con 0x55dd1a512400 2024-05-17T22:50:50.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.911+0000 7f45fd7fa700 1 -- 172.21.15.12:0/1253045664 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f460c003a00 con 0x55dd1a512400 2024-05-17T22:50:50.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.913+0000 7f462a459b80 1 -- 172.21.15.12:0/1253045664 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f460401e270 con 0x55dd1a512400 2024-05-17T22:50:50.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.913+0000 7f45fd7fa700 1 -- 172.21.15.12:0/1253045664 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f460c003c00 con 0x55dd1a512400 2024-05-17T22:50:50.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.917+0000 7f462a459b80 10 client.16290 fetch_fsmap learned FSMap version 39 2024-05-17T22:50:50.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.917+0000 7f462a459b80 1 -- 172.21.15.12:0/1253045664 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55dd1a58f430 con 0x55dd1a512400 2024-05-17T22:50:50.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.918+0000 7f45fd7fa700 1 -- 172.21.15.12:0/1253045664 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f460401e270 con 0x55dd1a512400 2024-05-17T22:50:50.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.919+0000 7f45fd7fa700 1 -- 172.21.15.12:0/1253045664 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f460c015c90 con 0x55dd1a512400 2024-05-17T22:50:50.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.921+0000 7f462a459b80 10 client.16290 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:50:50.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.921+0000 7f462a459b80 10 client.16290 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-17T22:50:50.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.921+0000 7f462a459b80 20 client.16290 populate_metadata read hostname 'smithi012' 2024-05-17T22:50:50.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.921+0000 7f462a459b80 1 --2- 172.21.15.12:0/1253045664 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55dd1a58fb10 0x55dd1a591f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:50.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.921+0000 7f462a459b80 4 client.16290 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:50:50.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.921+0000 7f462a459b80 1 -- 172.21.15.12:0/1253045664 --> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dd1a436800 con 0x55dd1a58fb10 2024-05-17T22:50:50.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.925+0000 7f461981a700 1 --2- 172.21.15.12:0/1253045664 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55dd1a58fb10 0x55dd1a591f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:50.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.935+0000 7f45fd7fa700 10 client.16290 ms_handle_connect on v2:172.21.15.151:6838/2338353849 2024-05-17T22:50:50.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.935+0000 7f461981a700 1 --2- 172.21.15.12:0/1253045664 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55dd1a58fb10 0x55dd1a591f30 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1da3241f0ca4efa9 server_cookie=5fcb8155d403cdb8 in_seq=0 out_seq=0 2024-05-17T22:50:50.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.940+0000 7f45fd7fa700 1 -- 172.21.15.12:0/1253045664 <== mds.2 v2:172.21.15.151:6838/2338353849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dd1a436800 con 0x55dd1a58fb10 2024-05-17T22:50:50.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.940+0000 7f45fd7fa700 10 client.16290 handle_command_reply: tid=0 2024-05-17T22:50:50.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.941+0000 7f462a459b80 1 client.16290 shutdown 2024-05-17T22:50:50.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.946+0000 7f462a459b80 1 -- 172.21.15.12:0/1253045664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd1a512400 msgr2=0x55dd1a5165e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:50.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.946+0000 7f462a459b80 1 --2- 172.21.15.12:0/1253045664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd1a512400 0x55dd1a5165e0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f460c009cd0 tx=0x7f460c005d10 comp rx=0 tx=0).stop 2024-05-17T22:50:50.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.946+0000 7f462a459b80 1 -- 172.21.15.12:0/1253045664 shutdown_connections 2024-05-17T22:50:50.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.946+0000 7f462a459b80 1 --2- 172.21.15.12:0/1253045664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dd1a512400 0x55dd1a5165e0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:50.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.946+0000 7f462a459b80 1 --2- 172.21.15.12:0/1253045664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1a38e9f0 0x55dd1a51e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:50.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.946+0000 7f462a459b80 1 --2- 172.21.15.12:0/1253045664 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55dd1a58fb10 0x55dd1a591f30 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:50.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.946+0000 7f462a459b80 1 --2- 172.21.15.12:0/1253045664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dd1a50cb00 0x55dd1a5160a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:50.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.946+0000 7f45fd7fa700 0 client.16290 ms_handle_reset on v2:172.21.15.151:6838/2338353849 2024-05-17T22:50:50.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.946+0000 7f462a459b80 1 -- 172.21.15.12:0/1253045664 >> 172.21.15.12:0/1253045664 conn(0x55dd1a3ee7d0 msgr2=0x55dd1a50f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:50:50.950 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:50:50.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.946+0000 7f462a459b80 1 -- 172.21.15.12:0/1253045664 shutdown_connections 2024-05-17T22:50:50.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.946+0000 7f462a459b80 1 -- 172.21.15.12:0/1253045664 wait complete. 2024-05-17T22:50:50.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.948+0000 7f462a459b80 20 client.16290 trim_cache size 0 max 16384 2024-05-17T22:50:50.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.950+0000 7f461ba7e700 1 -- 172.21.15.12:0/2124696474 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f46000b2900 msgr2=0x7f46000b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:50.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.950+0000 7f461ba7e700 1 --2- 172.21.15.12:0/2124696474 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f46000b2900 0x7f46000b4e10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4604009af0 tx=0x7f4604009260 comp rx=0 tx=0).stop 2024-05-17T22:50:50.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.950+0000 7f461ba7e700 1 -- 172.21.15.12:0/2124696474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46141091c0 msgr2=0x7f4614199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:50.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.950+0000 7f461ba7e700 1 --2- 172.21.15.12:0/2124696474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46141091c0 0x7f4614199ab0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4610000c00 tx=0x7f461000fa00 comp rx=0 tx=0).stop 2024-05-17T22:50:50.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.955+0000 7f461ba7e700 1 -- 172.21.15.12:0/2124696474 shutdown_connections 2024-05-17T22:50:50.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.955+0000 7f461ba7e700 1 --2- 172.21.15.12:0/2124696474 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f46000b2900 0x7f46000b4e10 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:50.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.955+0000 7f461ba7e700 1 --2- 172.21.15.12:0/2124696474 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46141a0340 0x7f46141a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:50.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.955+0000 7f461ba7e700 1 --2- 172.21.15.12:0/2124696474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4614102450 0x7f4614199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:50.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.955+0000 7f461ba7e700 1 --2- 172.21.15.12:0/2124696474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46141091c0 0x7f4614199ab0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:50.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.955+0000 7f461ba7e700 1 -- 172.21.15.12:0/2124696474 >> 172.21.15.12:0/2124696474 conn(0x7f46140fe030 msgr2=0x7f4614104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:50:50.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.956+0000 7f461ba7e700 1 -- 172.21.15.12:0/2124696474 shutdown_connections 2024-05-17T22:50:50.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:50.956+0000 7f461ba7e700 1 -- 172.21.15.12:0/2124696474 wait complete. 2024-05-17T22:50:51.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 690.0 s, 412.8 tps, lat 237.646 ms stddev 273.787 2024-05-17T22:50:51.842 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-17T22:50:52.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:52 smithi012 ceph-mon[31632]: pgmap v787: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.4 MiB/s wr, 446 op/s 2024-05-17T22:50:52.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:52 smithi096 ceph-mon[36065]: pgmap v787: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.4 MiB/s wr, 446 op/s 2024-05-17T22:50:52.412 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:50:52.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:52 smithi151 ceph-mon[36101]: pgmap v787: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.4 MiB/s wr, 446 op/s 2024-05-17T22:50:54.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:54 smithi096 ceph-mon[36065]: pgmap v788: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.8 MiB/s wr, 426 op/s 2024-05-17T22:50:54.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:54 smithi012 ceph-mon[31632]: pgmap v788: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.8 MiB/s wr, 426 op/s 2024-05-17T22:50:54.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:54 smithi151 ceph-mon[36101]: pgmap v788: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.8 MiB/s wr, 426 op/s 2024-05-17T22:50:55.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.487+0000 7f3b06650700 1 -- 172.21.15.12:0/1832090719 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b00107730 msgr2=0x7f3b00107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:55.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.487+0000 7f3b06650700 1 --2- 172.21.15.12:0/1832090719 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b00107730 0x7f3b00107b40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3af4009990 tx=0x7f3af4009ca0 comp rx=0 tx=0).stop 2024-05-17T22:50:55.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.487+0000 7f3b06650700 1 -- 172.21.15.12:0/1832090719 shutdown_connections 2024-05-17T22:50:55.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.487+0000 7f3b06650700 1 --2- 172.21.15.12:0/1832090719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b00108080 0x7f3b0010f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:55.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.487+0000 7f3b06650700 1 --2- 172.21.15.12:0/1832090719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b00102e00 0x7f3b00103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:55.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.487+0000 7f3b06650700 1 --2- 172.21.15.12:0/1832090719 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b00107730 0x7f3b00107b40 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:55.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.487+0000 7f3b06650700 1 -- 172.21.15.12:0/1832090719 >> 172.21.15.12:0/1832090719 conn(0x7f3b00079bd0 msgr2=0x7f3b00079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:50:55.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.487+0000 7f3b06650700 1 -- 172.21.15.12:0/1832090719 shutdown_connections 2024-05-17T22:50:55.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.487+0000 7f3b06650700 1 -- 172.21.15.12:0/1832090719 wait complete. 2024-05-17T22:50:55.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.488+0000 7f3b06650700 1 Processor -- start 2024-05-17T22:50:55.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.488+0000 7f3b06650700 1 -- start start 2024-05-17T22:50:55.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.489+0000 7f3b06650700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b00102e00 0x7f3b001103f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:55.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.489+0000 7f3b06650700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b00107730 0x7f3b00110930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:55.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.489+0000 7f3b06650700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b00108080 0x7f3b00110e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:55.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.489+0000 7f3b06650700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3b00113e40 con 0x7f3b00107730 2024-05-17T22:50:55.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.489+0000 7f3b06650700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3b00113cc0 con 0x7f3b00108080 2024-05-17T22:50:55.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.489+0000 7f3b06650700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3b00113fc0 con 0x7f3b00102e00 2024-05-17T22:50:55.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.489+0000 7f3aff7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b00107730 0x7f3b00110930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:55.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.489+0000 7f3afffff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b00102e00 0x7f3b001103f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:55.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.489+0000 7f3b04bed700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b00108080 0x7f3b00110e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:55.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.489+0000 7f3b04bed700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b00108080 0x7f3b00110e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:53036/0 (socket says 172.21.15.12:53036) 2024-05-17T22:50:55.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.489+0000 7f3b04bed700 1 -- 172.21.15.12:0/1522279308 learned_addr learned my addr 172.21.15.12:0/1522279308 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:50:55.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.489+0000 7f3b04bed700 1 -- 172.21.15.12:0/1522279308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b00102e00 msgr2=0x7f3b001103f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:55.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.489+0000 7f3b04bed700 1 --2- 172.21.15.12:0/1522279308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b00102e00 0x7f3b001103f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:55.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.489+0000 7f3b04bed700 1 -- 172.21.15.12:0/1522279308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b00107730 msgr2=0x7f3b00110930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:55.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.489+0000 7f3b04bed700 1 --2- 172.21.15.12:0/1522279308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b00107730 0x7f3b00110930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:55.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.489+0000 7f3b04bed700 1 -- 172.21.15.12:0/1522279308 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b001a8b80 con 0x7f3b00108080 2024-05-17T22:50:55.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.490+0000 7f3b04bed700 1 --2- 172.21.15.12:0/1522279308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b00108080 0x7f3b00110e70 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3af800ef30 tx=0x7f3af800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:50:55.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.490+0000 7f3afd7fa700 1 -- 172.21.15.12:0/1522279308 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af80092e0 con 0x7f3b00108080 2024-05-17T22:50:55.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.490+0000 7f3afd7fa700 1 -- 172.21.15.12:0/1522279308 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3af8004bb0 con 0x7f3b00108080 2024-05-17T22:50:55.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.490+0000 7f3b06650700 1 -- 172.21.15.12:0/1522279308 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b001a8e10 con 0x7f3b00108080 2024-05-17T22:50:55.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.490+0000 7f3b06650700 1 -- 172.21.15.12:0/1522279308 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b001a9230 con 0x7f3b00108080 2024-05-17T22:50:55.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.490+0000 7f3afd7fa700 1 -- 172.21.15.12:0/1522279308 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af8004020 con 0x7f3b00108080 2024-05-17T22:50:55.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.494+0000 7f3afd7fa700 1 -- 172.21.15.12:0/1522279308 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3af801b030 con 0x7f3b00108080 2024-05-17T22:50:55.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.497+0000 7f3afd7fa700 1 --2- 172.21.15.12:0/1522279308 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f3ae80b2980 0x7f3ae80b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:55.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.497+0000 7f3b1502bb80 1 Processor -- start 2024-05-17T22:50:55.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.497+0000 7f3b1502bb80 1 -- start start 2024-05-17T22:50:55.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.497+0000 7f3afffff700 1 --2- 172.21.15.12:0/1522279308 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f3ae80b2980 0x7f3ae80b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:55.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.497+0000 7f3b1502bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558fa3f89f0 0x5558fa57bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:55.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.497+0000 7f3b1502bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558fa57c400 0x5558fa5765c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:55.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.497+0000 7f3b1502bb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558fa576b00 0x5558fa576f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:55.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.497+0000 7f3b1502bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5558fa4a50d0 con 0x5558fa3f89f0 2024-05-17T22:50:55.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.497+0000 7f3b1502bb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5558fa494c70 con 0x5558fa576b00 2024-05-17T22:50:55.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.497+0000 7f3b1502bb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5558fa498cb0 con 0x5558fa57c400 2024-05-17T22:50:55.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.498+0000 7f3aff7fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558fa57c400 0x5558fa5765c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:55.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.498+0000 7f3aff7fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558fa57c400 0x5558fa5765c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:33826/0 (socket says 172.21.15.12:33826) 2024-05-17T22:50:55.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.498+0000 7f3aff7fe700 1 -- 172.21.15.12:0/249263108 learned_addr learned my addr 172.21.15.12:0/249263108 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:50:55.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.498+0000 7f3b04bed700 1 --2- 172.21.15.12:0/249263108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558fa576b00 0x5558fa576f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:55.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.498+0000 7f3aff7fe700 1 -- 172.21.15.12:0/249263108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558fa576b00 msgr2=0x5558fa576f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:55.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.498+0000 7f3aff7fe700 1 --2- 172.21.15.12:0/249263108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558fa576b00 0x5558fa576f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:55.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.498+0000 7f3aff7fe700 1 -- 172.21.15.12:0/249263108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558fa3f89f0 msgr2=0x5558fa57bec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T22:50:55.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.498+0000 7f3aff7fe700 1 --2- 172.21.15.12:0/249263108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558fa3f89f0 0x5558fa57bec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:55.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.498+0000 7f3aff7fe700 1 -- 172.21.15.12:0/249263108 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558fa42e5c0 con 0x5558fa57c400 2024-05-17T22:50:55.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.498+0000 7f3b04bed700 1 --2- 172.21.15.12:0/249263108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558fa576b00 0x5558fa576f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:50:55.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.498+0000 7f3afd7fa700 1 -- 172.21.15.12:0/1522279308 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f3af80e1fe0 con 0x7f3b00108080 2024-05-17T22:50:55.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.498+0000 7f3aff7fe700 1 --2- 172.21.15.12:0/249263108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558fa57c400 0x5558fa5765c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3af4009990 tx=0x7f3af4005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4341f101af64ba85 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:50:55.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.498+0000 7f3ae67fc700 1 -- 172.21.15.12:0/249263108 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af4027070 con 0x5558fa57c400 2024-05-17T22:50:55.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.499+0000 7f3ae67fc700 1 -- 172.21.15.12:0/249263108 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3af400bbd0 con 0x5558fa57c400 2024-05-17T22:50:55.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.499+0000 7f3ae67fc700 1 -- 172.21.15.12:0/249263108 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af4003d80 con 0x5558fa57c400 2024-05-17T22:50:55.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.500+0000 7f3b1502bb80 1 -- 172.21.15.12:0/249263108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558fa57c400 msgr2=0x5558fa5765c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:55.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.500+0000 7f3b1502bb80 1 --2- 172.21.15.12:0/249263108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558fa57c400 0x5558fa5765c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3af4009990 tx=0x7f3af4005e80 comp rx=0 tx=0).stop 2024-05-17T22:50:55.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.500+0000 7f3afffff700 1 --2- 172.21.15.12:0/1522279308 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f3ae80b2980 0x7f3ae80b4e90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x5558fa5721d0 tx=0x7f3af000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:50:55.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.500+0000 7f3b1502bb80 1 -- 172.21.15.12:0/249263108 shutdown_connections 2024-05-17T22:50:55.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.500+0000 7f3b1502bb80 1 --2- 172.21.15.12:0/249263108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558fa576b00 0x5558fa576f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:55.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.500+0000 7f3b1502bb80 1 --2- 172.21.15.12:0/249263108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558fa3f89f0 0x5558fa57bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:55.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.500+0000 7f3b1502bb80 1 --2- 172.21.15.12:0/249263108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558fa57c400 0x5558fa5765c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:55.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.500+0000 7f3b1502bb80 1 -- 172.21.15.12:0/249263108 >> 172.21.15.12:0/249263108 conn(0x5558fa4587d0 msgr2=0x5558fa5711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:50:55.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.500+0000 7f3b1502bb80 1 -- 172.21.15.12:0/249263108 shutdown_connections 2024-05-17T22:50:55.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.500+0000 7f3b1502bb80 1 -- 172.21.15.12:0/249263108 wait complete. 2024-05-17T22:50:55.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.501+0000 7f3b1502bb80 1 -- start start 2024-05-17T22:50:55.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.501+0000 7f3b1502bb80 1 Processor -- start 2024-05-17T22:50:55.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.501+0000 7f3b1502bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558fa3f89f0 0x5558fa57fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:55.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.501+0000 7f3b1502bb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558fa576b00 0x5558fa580050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:55.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.502+0000 7f3b1502bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558fa57c400 0x5558fa58c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:55.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.502+0000 7f3b1502bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5558fa39c150 con 0x5558fa57c400 2024-05-17T22:50:55.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.502+0000 7f3afffff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558fa3f89f0 0x5558fa57fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:55.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.502+0000 7f3b1502bb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5558fa436940 con 0x5558fa576b00 2024-05-17T22:50:55.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.502+0000 7f3afffff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558fa3f89f0 0x5558fa57fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:33834/0 (socket says 172.21.15.12:33834) 2024-05-17T22:50:55.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.502+0000 7f3afffff700 1 -- 172.21.15.12:0/2665731053 learned_addr learned my addr 172.21.15.12:0/2665731053 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:50:55.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.502+0000 7f3b1502bb80 1 -- 172.21.15.12:0/2665731053 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5558fa582d70 con 0x5558fa3f89f0 2024-05-17T22:50:55.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.502+0000 7f3aff7fe700 1 --2- 172.21.15.12:0/2665731053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558fa576b00 0x5558fa580050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:55.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.502+0000 7f3b1502bb80 10 client.0 _ll_register_callbacks cb 0x5558fa42bb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:50:55.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.502+0000 7f3b04bed700 1 --2- 172.21.15.12:0/2665731053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558fa57c400 0x5558fa58c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:55.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.502+0000 7f3aff7fe700 1 -- 172.21.15.12:0/2665731053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558fa3f89f0 msgr2=0x5558fa57fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:55.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.502+0000 7f3aff7fe700 1 --2- 172.21.15.12:0/2665731053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558fa3f89f0 0x5558fa57fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:55.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.502+0000 7f3aff7fe700 1 -- 172.21.15.12:0/2665731053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558fa57c400 msgr2=0x5558fa58c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:55.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.502+0000 7f3aff7fe700 1 --2- 172.21.15.12:0/2665731053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558fa57c400 0x5558fa58c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:55.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.502+0000 7f3aff7fe700 1 -- 172.21.15.12:0/2665731053 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558fa3e5480 con 0x5558fa576b00 2024-05-17T22:50:55.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.503+0000 7f3b04bed700 1 --2- 172.21.15.12:0/2665731053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558fa57c400 0x5558fa58c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:50:55.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.503+0000 7f3aff7fe700 1 --2- 172.21.15.12:0/2665731053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558fa576b00 0x5558fa580050 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3af40098e0 tx=0x7f3af4005a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfb0681b2e46fc44 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:50:55.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.503+0000 7f3adf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:50:55.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.503+0000 7f3adf7fe700 1 -- 172.21.15.12:0/2665731053 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af4027040 con 0x5558fa576b00 2024-05-17T22:50:55.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.503+0000 7f3b1502bb80 1 -- 172.21.15.12:0/2665731053 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3af400bbd0 con 0x5558fa576b00 2024-05-17T22:50:55.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.503+0000 7f3adf7fe700 1 -- 172.21.15.12:0/2665731053 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f3af400b720 con 0x5558fa576b00 2024-05-17T22:50:55.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.504+0000 7f3adf7fe700 1 -- 172.21.15.12:0/2665731053 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3af400b920 con 0x5558fa576b00 2024-05-17T22:50:55.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.504+0000 7f3b1502bb80 10 client.26012 fetch_fsmap learned FSMap version 39 2024-05-17T22:50:55.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.504+0000 7f3b1502bb80 1 -- 172.21.15.12:0/2665731053 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5558fa58d890 con 0x5558fa576b00 2024-05-17T22:50:55.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.504+0000 7f3adf7fe700 1 -- 172.21.15.12:0/2665731053 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3af400bbd0 con 0x5558fa576b00 2024-05-17T22:50:55.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.504+0000 7f3adf7fe700 1 -- 172.21.15.12:0/2665731053 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f3af4017950 con 0x5558fa576b00 2024-05-17T22:50:55.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.504+0000 7f3b1502bb80 10 client.26012 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:50:55.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.504+0000 7f3b1502bb80 10 client.26012 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-17T22:50:55.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.504+0000 7f3b1502bb80 20 client.26012 populate_metadata read hostname 'smithi012' 2024-05-17T22:50:55.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.504+0000 7f3b1502bb80 1 --2- 172.21.15.12:0/2665731053 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5558fa58fd60 0x5558fa592180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:55.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.504+0000 7f3b1502bb80 4 client.26012 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:50:55.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.504+0000 7f3b1502bb80 1 -- 172.21.15.12:0/2665731053 --> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558fa4a7020 con 0x5558fa58fd60 2024-05-17T22:50:55.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.504+0000 7f3afffff700 1 --2- 172.21.15.12:0/2665731053 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5558fa58fd60 0x5558fa592180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:55.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.505+0000 7f3adf7fe700 10 client.26012 ms_handle_connect on v2:172.21.15.96:6836/4222574338 2024-05-17T22:50:55.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.505+0000 7f3afffff700 1 --2- 172.21.15.12:0/2665731053 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5558fa58fd60 0x5558fa592180 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7aafcdd16d257926 server_cookie=bf94ffa21fe4de2e in_seq=0 out_seq=0 2024-05-17T22:50:55.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.505+0000 7f3adf7fe700 1 -- 172.21.15.12:0/2665731053 <== mds.1 v2:172.21.15.96:6836/4222574338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5558fa4a7020 con 0x5558fa58fd60 2024-05-17T22:50:55.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.505+0000 7f3adf7fe700 10 client.26012 handle_command_reply: tid=0 2024-05-17T22:50:55.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.505+0000 7f3b1502bb80 1 client.26012 shutdown 2024-05-17T22:50:55.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.506+0000 7f3b1502bb80 1 -- 172.21.15.12:0/2665731053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558fa576b00 msgr2=0x5558fa580050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:55.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.506+0000 7f3b1502bb80 1 --2- 172.21.15.12:0/2665731053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558fa576b00 0x5558fa580050 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3af40098e0 tx=0x7f3af4005a00 comp rx=0 tx=0).stop 2024-05-17T22:50:55.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.506+0000 7f3b1502bb80 1 -- 172.21.15.12:0/2665731053 shutdown_connections 2024-05-17T22:50:55.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.506+0000 7f3b1502bb80 1 --2- 172.21.15.12:0/2665731053 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5558fa58fd60 0x5558fa592180 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:55.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.506+0000 7f3b1502bb80 1 --2- 172.21.15.12:0/2665731053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558fa57c400 0x5558fa58c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:55.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.506+0000 7f3b1502bb80 1 --2- 172.21.15.12:0/2665731053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558fa3f89f0 0x5558fa57fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:55.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.506+0000 7f3b1502bb80 1 --2- 172.21.15.12:0/2665731053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558fa576b00 0x5558fa580050 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:55.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.506+0000 7f3b1502bb80 1 -- 172.21.15.12:0/2665731053 >> 172.21.15.12:0/2665731053 conn(0x5558fa4587d0 msgr2=0x5558fa571160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:50:55.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.506+0000 7f3adf7fe700 0 client.26012 ms_handle_reset on v2:172.21.15.96:6836/4222574338 2024-05-17T22:50:55.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.506+0000 7f3b1502bb80 1 -- 172.21.15.12:0/2665731053 shutdown_connections 2024-05-17T22:50:55.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.506+0000 7f3b1502bb80 1 -- 172.21.15.12:0/2665731053 wait complete. 2024-05-17T22:50:55.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.506+0000 7f3b1502bb80 20 client.26012 trim_cache size 0 max 16384 2024-05-17T22:50:55.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.538+0000 7f3b1502bb80 1 Processor -- start 2024-05-17T22:50:55.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.538+0000 7f3b1502bb80 1 -- start start 2024-05-17T22:50:55.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.538+0000 7f3b1502bb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558fa3f89f0 0x5558fa5765c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:55.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.538+0000 7f3b1502bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558fa576b00 0x5558fa57a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:55.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.538+0000 7f3b1502bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558fa57c400 0x5558fa58c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:55.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.539+0000 7f3b1502bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3b001b4760 con 0x5558fa576b00 2024-05-17T22:50:55.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.539+0000 7f3b1502bb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3b001b1ff0 con 0x5558fa3f89f0 2024-05-17T22:50:55.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.539+0000 7f3b1502bb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3b001afe40 con 0x5558fa57c400 2024-05-17T22:50:55.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.539+0000 7f3afffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558fa3f89f0 0x5558fa5765c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:55.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.539+0000 7f3afffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558fa3f89f0 0x5558fa5765c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:53066/0 (socket says 172.21.15.12:53066) 2024-05-17T22:50:55.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.539+0000 7f3afffff700 1 -- 172.21.15.12:0/2501708739 learned_addr learned my addr 172.21.15.12:0/2501708739 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:50:55.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.541+0000 7f3b04bed700 1 --2- 172.21.15.12:0/2501708739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558fa57c400 0x5558fa58c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:55.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.542+0000 7f3afffff700 1 -- 172.21.15.12:0/2501708739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558fa57c400 msgr2=0x5558fa58c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:55.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.542+0000 7f3afffff700 1 --2- 172.21.15.12:0/2501708739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558fa57c400 0x5558fa58c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:55.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.542+0000 7f3afffff700 1 -- 172.21.15.12:0/2501708739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558fa576b00 msgr2=0x5558fa57a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:55.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.542+0000 7f3afffff700 1 --2- 172.21.15.12:0/2501708739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558fa576b00 0x5558fa57a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:55.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.542+0000 7f3afffff700 1 -- 172.21.15.12:0/2501708739 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558fa571ce0 con 0x5558fa3f89f0 2024-05-17T22:50:55.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.545+0000 7f3afffff700 1 --2- 172.21.15.12:0/2501708739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558fa3f89f0 0x5558fa5765c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3af0011540 tx=0x7f3af0013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=129852bc525eb4b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:50:55.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.547+0000 7f3ae5ffb700 1 -- 172.21.15.12:0/2501708739 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af002c070 con 0x5558fa3f89f0 2024-05-17T22:50:55.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.547+0000 7f3ae5ffb700 1 -- 172.21.15.12:0/2501708739 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3af00262d0 con 0x5558fa3f89f0 2024-05-17T22:50:55.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.548+0000 7f3b1502bb80 1 -- 172.21.15.12:0/2501708739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558fa3f89f0 msgr2=0x5558fa5765c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:55.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.548+0000 7f3b1502bb80 1 --2- 172.21.15.12:0/2501708739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558fa3f89f0 0x5558fa5765c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3af0011540 tx=0x7f3af0013000 comp rx=0 tx=0).stop 2024-05-17T22:50:55.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.549+0000 7f3b1502bb80 1 -- 172.21.15.12:0/2501708739 shutdown_connections 2024-05-17T22:50:55.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.549+0000 7f3b1502bb80 1 --2- 172.21.15.12:0/2501708739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558fa57c400 0x5558fa58c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:55.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.549+0000 7f3b1502bb80 1 --2- 172.21.15.12:0/2501708739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558fa3f89f0 0x5558fa5765c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:55.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.549+0000 7f3b1502bb80 1 --2- 172.21.15.12:0/2501708739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558fa576b00 0x5558fa57a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:55.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.549+0000 7f3b1502bb80 1 -- 172.21.15.12:0/2501708739 >> 172.21.15.12:0/2501708739 conn(0x5558fa4587d0 msgr2=0x5558fa57b230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:50:55.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.562+0000 7f3b1502bb80 1 -- 172.21.15.12:0/2501708739 shutdown_connections 2024-05-17T22:50:55.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.566+0000 7f3b1502bb80 1 -- 172.21.15.12:0/2501708739 wait complete. 2024-05-17T22:50:55.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.566+0000 7f3b1502bb80 1 -- start start 2024-05-17T22:50:55.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.566+0000 7f3b1502bb80 1 Processor -- start 2024-05-17T22:50:55.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.567+0000 7f3b1502bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558fa3f89f0 0x5558fa5950d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:55.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.567+0000 7f3b1502bb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558fa576b00 0x5558fa592450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:55.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.567+0000 7f3b1502bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558fa57c400 0x5558fa592990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:55.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.567+0000 7f3b1502bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3b001b04b0 con 0x5558fa57c400 2024-05-17T22:50:55.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.567+0000 7f3b1502bb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3b001b1920 con 0x5558fa576b00 2024-05-17T22:50:55.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.567+0000 7f3b1502bb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3b001b0b80 con 0x5558fa3f89f0 2024-05-17T22:50:55.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.567+0000 7f3b1502bb80 10 client.0 _ll_register_callbacks cb 0x5558fa42bb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:50:55.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.573+0000 7f3afffff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558fa3f89f0 0x5558fa5950d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:55.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.574+0000 7f3b04bed700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558fa57c400 0x5558fa592990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:55.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.574+0000 7f3b04bed700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558fa57c400 0x5558fa592990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49560/0 (socket says 172.21.15.12:49560) 2024-05-17T22:50:55.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.574+0000 7f3b04bed700 1 -- 172.21.15.12:0/984764586 learned_addr learned my addr 172.21.15.12:0/984764586 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:50:55.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.579+0000 7f3aff7fe700 1 --2- 172.21.15.12:0/984764586 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558fa576b00 0x5558fa592450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:55.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.583+0000 7f3afffff700 1 -- 172.21.15.12:0/984764586 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558fa576b00 msgr2=0x5558fa592450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:55.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.583+0000 7f3afffff700 1 --2- 172.21.15.12:0/984764586 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558fa576b00 0x5558fa592450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:55.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.583+0000 7f3afffff700 1 -- 172.21.15.12:0/984764586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558fa57c400 msgr2=0x5558fa592990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:55.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.583+0000 7f3afffff700 1 --2- 172.21.15.12:0/984764586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558fa57c400 0x5558fa592990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:55.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.583+0000 7f3afffff700 1 -- 172.21.15.12:0/984764586 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558fa592ed0 con 0x5558fa3f89f0 2024-05-17T22:50:55.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.584+0000 7f3afffff700 1 --2- 172.21.15.12:0/984764586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558fa3f89f0 0x5558fa5950d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3af0011850 tx=0x7f3af0026ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddf5e6a6fe1f9570 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:50:55.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.585+0000 7f3adf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:50:55.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.589+0000 7f3adf7fe700 1 -- 172.21.15.12:0/984764586 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af0035060 con 0x5558fa3f89f0 2024-05-17T22:50:55.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.589+0000 7f3b1502bb80 1 -- 172.21.15.12:0/984764586 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3af00262d0 con 0x5558fa3f89f0 2024-05-17T22:50:55.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.589+0000 7f3adf7fe700 1 -- 172.21.15.12:0/984764586 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f3af0026480 con 0x5558fa3f89f0 2024-05-17T22:50:55.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.594+0000 7f3adf7fe700 1 -- 172.21.15.12:0/984764586 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3af002c040 con 0x5558fa3f89f0 2024-05-17T22:50:55.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.594+0000 7f3b1502bb80 10 client.25951 fetch_fsmap learned FSMap version 39 2024-05-17T22:50:55.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.594+0000 7f3b1502bb80 1 -- 172.21.15.12:0/984764586 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5558fa5f7f70 con 0x5558fa3f89f0 2024-05-17T22:50:55.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.598+0000 7f3adf7fe700 1 -- 172.21.15.12:0/984764586 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3af00262d0 con 0x5558fa3f89f0 2024-05-17T22:50:55.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.599+0000 7f3adf7fe700 1 -- 172.21.15.12:0/984764586 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f3af002f030 con 0x5558fa3f89f0 2024-05-17T22:50:55.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.599+0000 7f3b1502bb80 10 client.25951 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:50:55.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.599+0000 7f3b1502bb80 10 client.25951 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-17T22:50:55.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.599+0000 7f3b1502bb80 20 client.25951 populate_metadata read hostname 'smithi012' 2024-05-17T22:50:55.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.599+0000 7f3b1502bb80 1 --2- 172.21.15.12:0/984764586 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5558fa5f8560 0x5558fa60da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:55.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.599+0000 7f3b1502bb80 4 client.25951 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:50:55.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.599+0000 7f3b1502bb80 1 -- 172.21.15.12:0/984764586 --> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5558fa4a0800 con 0x5558fa5f8560 2024-05-17T22:50:55.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.601+0000 7f3aff7fe700 1 --2- 172.21.15.12:0/984764586 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5558fa5f8560 0x5558fa60da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:55.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.603+0000 7f3aff7fe700 1 --2- 172.21.15.12:0/984764586 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5558fa5f8560 0x5558fa60da90 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=87320647adaebd63 server_cookie=41d8bb98f0a68db8 in_seq=0 out_seq=0 2024-05-17T22:50:55.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.603+0000 7f3adf7fe700 10 client.25951 ms_handle_connect on v2:172.21.15.96:6836/4222574338 2024-05-17T22:50:55.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.603+0000 7f3adf7fe700 1 -- 172.21.15.12:0/984764586 <== mds.1 v2:172.21.15.96:6836/4222574338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5558fa4a0800 con 0x5558fa5f8560 2024-05-17T22:50:55.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.603+0000 7f3adf7fe700 10 client.25951 handle_command_reply: tid=0 2024-05-17T22:50:55.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.603+0000 7f3b1502bb80 1 client.25951 shutdown 2024-05-17T22:50:55.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.603+0000 7f3b1502bb80 1 -- 172.21.15.12:0/984764586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558fa3f89f0 msgr2=0x5558fa5950d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:55.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.603+0000 7f3b1502bb80 1 --2- 172.21.15.12:0/984764586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558fa3f89f0 0x5558fa5950d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3af0011850 tx=0x7f3af0026ae0 comp rx=0 tx=0).stop 2024-05-17T22:50:55.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.605+0000 7f3b1502bb80 1 -- 172.21.15.12:0/984764586 shutdown_connections 2024-05-17T22:50:55.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.605+0000 7f3b1502bb80 1 --2- 172.21.15.12:0/984764586 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5558fa5f8560 0x5558fa60da90 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:55.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.605+0000 7f3b1502bb80 1 --2- 172.21.15.12:0/984764586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558fa57c400 0x5558fa592990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:55.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.605+0000 7f3b1502bb80 1 --2- 172.21.15.12:0/984764586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558fa3f89f0 0x5558fa5950d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:55.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.605+0000 7f3b1502bb80 1 --2- 172.21.15.12:0/984764586 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558fa576b00 0x5558fa592450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:55.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.605+0000 7f3b1502bb80 1 -- 172.21.15.12:0/984764586 >> 172.21.15.12:0/984764586 conn(0x5558fa4587d0 msgr2=0x5558fa57b230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:50:55.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.605+0000 7f3adf7fe700 0 client.25951 ms_handle_reset on v2:172.21.15.96:6836/4222574338 2024-05-17T22:50:55.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.608+0000 7f3b1502bb80 1 -- 172.21.15.12:0/984764586 shutdown_connections 2024-05-17T22:50:55.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.608+0000 7f3b1502bb80 1 -- 172.21.15.12:0/984764586 wait complete. 2024-05-17T22:50:55.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.610+0000 7f3b1502bb80 20 client.25951 trim_cache size 0 max 16384 2024-05-17T22:50:55.614 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:50:55.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.613+0000 7f3b06650700 1 -- 172.21.15.12:0/1522279308 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f3ae80b2980 msgr2=0x7f3ae80b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:55.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.613+0000 7f3b06650700 1 --2- 172.21.15.12:0/1522279308 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f3ae80b2980 0x7f3ae80b4e90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x5558fa5721d0 tx=0x7f3af000b410 comp rx=0 tx=0).stop 2024-05-17T22:50:55.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.613+0000 7f3b06650700 1 -- 172.21.15.12:0/1522279308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b00108080 msgr2=0x7f3b00110e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:55.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.613+0000 7f3b06650700 1 --2- 172.21.15.12:0/1522279308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b00108080 0x7f3b00110e70 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3af800ef30 tx=0x7f3af800c3a0 comp rx=0 tx=0).stop 2024-05-17T22:50:55.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.613+0000 7f3b06650700 1 -- 172.21.15.12:0/1522279308 shutdown_connections 2024-05-17T22:50:55.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.613+0000 7f3b06650700 1 --2- 172.21.15.12:0/1522279308 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f3ae80b2980 0x7f3ae80b4e90 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:55.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.613+0000 7f3b06650700 1 --2- 172.21.15.12:0/1522279308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b00108080 0x7f3b00110e70 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:55.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.613+0000 7f3b06650700 1 --2- 172.21.15.12:0/1522279308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b00102e00 0x7f3b001103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:55.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.613+0000 7f3b06650700 1 --2- 172.21.15.12:0/1522279308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b00107730 0x7f3b00110930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:55.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.613+0000 7f3b06650700 1 -- 172.21.15.12:0/1522279308 >> 172.21.15.12:0/1522279308 conn(0x7f3b00079bd0 msgr2=0x7f3b001057d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:50:55.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.618+0000 7f3b06650700 1 -- 172.21.15.12:0/1522279308 shutdown_connections 2024-05-17T22:50:55.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:55.618+0000 7f3b06650700 1 -- 172.21.15.12:0/1522279308 wait complete. 2024-05-17T22:50:56.288 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:56 smithi012 ceph-mon[31632]: pgmap v789: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.0 MiB/s wr, 510 op/s 2024-05-17T22:50:56.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:56 smithi096 ceph-mon[36065]: pgmap v789: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.0 MiB/s wr, 510 op/s 2024-05-17T22:50:56.327 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-17T22:50:56.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 695.0 s, 484.8 tps, lat 214.374 ms stddev 258.623 2024-05-17T22:50:56.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:56 smithi151 ceph-mon[36101]: pgmap v789: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.0 MiB/s wr, 510 op/s 2024-05-17T22:50:56.780 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:50:58.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:50:58 smithi096 ceph-mon[36065]: pgmap v790: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.8 MiB/s wr, 442 op/s 2024-05-17T22:50:58.299 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:50:58 smithi012 ceph-mon[31632]: pgmap v790: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.8 MiB/s wr, 442 op/s 2024-05-17T22:50:58.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:50:58 smithi151 ceph-mon[36101]: pgmap v790: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.8 MiB/s wr, 442 op/s 2024-05-17T22:50:58.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.870+0000 7f7cc748e700 1 -- 172.21.15.12:0/1417421726 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cc0102b90 msgr2=0x7f7cc0102fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:58.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.870+0000 7f7cc748e700 1 --2- 172.21.15.12:0/1417421726 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cc0102b90 0x7f7cc0102fa0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0009990 tx=0x7f7cb0009ca0 comp rx=0 tx=0).stop 2024-05-17T22:50:58.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.875+0000 7f7cc748e700 1 -- 172.21.15.12:0/1417421726 shutdown_connections 2024-05-17T22:50:58.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.875+0000 7f7cc748e700 1 --2- 172.21.15.12:0/1417421726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7cc0107fd0 0x7f7cc01084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:58.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.875+0000 7f7cc748e700 1 --2- 172.21.15.12:0/1417421726 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cc0102b90 0x7f7cc0102fa0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:58.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.875+0000 7f7cc748e700 1 --2- 172.21.15.12:0/1417421726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7cc01034e0 0x7f7cc0107a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:58.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.875+0000 7f7cc748e700 1 -- 172.21.15.12:0/1417421726 >> 172.21.15.12:0/1417421726 conn(0x7f7cc0079bd0 msgr2=0x7f7cc0079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:50:58.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.876+0000 7f7cc748e700 1 -- 172.21.15.12:0/1417421726 shutdown_connections 2024-05-17T22:50:58.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.878+0000 7f7cc748e700 1 -- 172.21.15.12:0/1417421726 wait complete. 2024-05-17T22:50:58.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.879+0000 7f7cc748e700 1 Processor -- start 2024-05-17T22:50:58.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.881+0000 7f7cc748e700 1 -- start start 2024-05-17T22:50:58.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.882+0000 7f7cc748e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cc0102b90 0x7f7cc00754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:58.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.882+0000 7f7cc748e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7cc01034e0 0x7f7cc006e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:58.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.882+0000 7f7cc748e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7cc0107fd0 0x7f7cc006eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:58.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.882+0000 7f7cc748e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc011a4a0 con 0x7f7cc01034e0 2024-05-17T22:50:58.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.883+0000 7f7cc748e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc011a320 con 0x7f7cc0107fd0 2024-05-17T22:50:58.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.883+0000 7f7cc748e700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc011a620 con 0x7f7cc0102b90 2024-05-17T22:50:58.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.883+0000 7f7cc4a29700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7cc01034e0 0x7f7cc006e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:58.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.883+0000 7f7cc4a29700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7cc01034e0 0x7f7cc006e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49580/0 (socket says 172.21.15.12:49580) 2024-05-17T22:50:58.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.883+0000 7f7cc4a29700 1 -- 172.21.15.12:0/964777361 learned_addr learned my addr 172.21.15.12:0/964777361 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:50:58.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.883+0000 7f7cc522a700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cc0102b90 0x7f7cc00754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:58.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.887+0000 7f7cc5a2b700 1 --2- 172.21.15.12:0/964777361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7cc0107fd0 0x7f7cc006eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:58.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.889+0000 7f7cc4a29700 1 -- 172.21.15.12:0/964777361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cc0102b90 msgr2=0x7f7cc00754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:58.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.889+0000 7f7cc4a29700 1 --2- 172.21.15.12:0/964777361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cc0102b90 0x7f7cc00754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:58.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.889+0000 7f7cc4a29700 1 -- 172.21.15.12:0/964777361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7cc0107fd0 msgr2=0x7f7cc006eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:58.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.889+0000 7f7cc4a29700 1 --2- 172.21.15.12:0/964777361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7cc0107fd0 0x7f7cc006eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:58.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.889+0000 7f7cc4a29700 1 -- 172.21.15.12:0/964777361 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cc0072fe0 con 0x7f7cc01034e0 2024-05-17T22:50:58.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.890+0000 7f7cc4a29700 1 --2- 172.21.15.12:0/964777361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7cc01034e0 0x7f7cc006e5e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc00b530 tx=0x7f7cbc00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:50:58.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.890+0000 7f7cb67fc700 1 -- 172.21.15.12:0/964777361 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc015020 con 0x7f7cc01034e0 2024-05-17T22:50:58.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.890+0000 7f7cc748e700 1 -- 172.21.15.12:0/964777361 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cc00732d0 con 0x7f7cc01034e0 2024-05-17T22:50:58.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.890+0000 7f7cc748e700 1 -- 172.21.15.12:0/964777361 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cc0073860 con 0x7f7cc01034e0 2024-05-17T22:50:58.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.890+0000 7f7cb67fc700 1 -- 172.21.15.12:0/964777361 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cbc011da0 con 0x7f7cc01034e0 2024-05-17T22:50:58.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.890+0000 7f7cb67fc700 1 -- 172.21.15.12:0/964777361 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc00f2e0 con 0x7f7cc01034e0 2024-05-17T22:50:58.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.891+0000 7f7cc5a2b700 1 --2- 172.21.15.12:0/964777361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7cc0107fd0 0x7f7cc006eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:50:58.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.898+0000 7f7cd5e69b80 1 Processor -- start 2024-05-17T22:50:58.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.898+0000 7f7cd5e69b80 1 -- start start 2024-05-17T22:50:58.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.901+0000 7f7cb67fc700 1 -- 172.21.15.12:0/964777361 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7cbc018020 con 0x7f7cc01034e0 2024-05-17T22:50:58.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.902+0000 7f7cd5e69b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d1f89a9f0 0x564d1fa1dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:58.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.902+0000 7f7cd5e69b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d1fa1e400 0x564d1fa185c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:58.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.902+0000 7f7cd5e69b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d1fa18b00 0x564d1fa18f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:58.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.902+0000 7f7cd5e69b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x564d1f9470d0 con 0x564d1f89a9f0 2024-05-17T22:50:58.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.902+0000 7f7cd5e69b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564d1f936c70 con 0x564d1fa18b00 2024-05-17T22:50:58.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.902+0000 7f7cd5e69b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564d1f93acb0 con 0x564d1fa1e400 2024-05-17T22:50:58.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.906+0000 7f7cc5a2b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d1f89a9f0 0x564d1fa1dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:58.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.906+0000 7f7cc5a2b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d1f89a9f0 0x564d1fa1dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49588/0 (socket says 172.21.15.12:49588) 2024-05-17T22:50:58.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.906+0000 7f7cc5a2b700 1 -- 172.21.15.12:0/362496314 learned_addr learned my addr 172.21.15.12:0/362496314 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:50:58.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.909+0000 7f7cc4a29700 1 --2- 172.21.15.12:0/362496314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d1fa18b00 0x564d1fa18f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:58.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.909+0000 7f7cc522a700 1 --2- 172.21.15.12:0/362496314 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d1fa1e400 0x564d1fa185c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:58.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.910+0000 7f7cc5a2b700 1 -- 172.21.15.12:0/362496314 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d1fa1e400 msgr2=0x564d1fa185c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:58.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.910+0000 7f7cc5a2b700 1 --2- 172.21.15.12:0/362496314 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d1fa1e400 0x564d1fa185c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:58.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.910+0000 7f7cc5a2b700 1 -- 172.21.15.12:0/362496314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d1fa18b00 msgr2=0x564d1fa18f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:58.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.910+0000 7f7cc5a2b700 1 --2- 172.21.15.12:0/362496314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d1fa18b00 0x564d1fa18f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:58.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.910+0000 7f7cc5a2b700 1 -- 172.21.15.12:0/362496314 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d1f8d05c0 con 0x564d1f89a9f0 2024-05-17T22:50:58.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.911+0000 7f7cc5a2b700 1 --2- 172.21.15.12:0/362496314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d1f89a9f0 0x564d1fa1dec0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8003ee0 tx=0x7f7cb8009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3ff6b4c7bc0938f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:50:58.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.912+0000 7f7cb67fc700 1 --2- 172.21.15.12:0/964777361 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f7cac0b29c0 0x7f7cac0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:58.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.912+0000 7f7cb67fc700 1 -- 172.21.15.12:0/964777361 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f7cbc0e22b0 con 0x7f7cc01034e0 2024-05-17T22:50:58.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.915+0000 7f7cc522a700 1 --2- 172.21.15.12:0/964777361 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f7cac0b29c0 0x7f7cac0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:58.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.915+0000 7f7cc522a700 1 --2- 172.21.15.12:0/964777361 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f7cac0b29c0 0x7f7cac0b4ed0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0006010 tx=0x7f7cb00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:50:58.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.917+0000 7f7cab7fe700 1 -- 172.21.15.12:0/362496314 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8019020 con 0x564d1f89a9f0 2024-05-17T22:50:58.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.917+0000 7f7cab7fe700 1 -- 172.21.15.12:0/362496314 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cb80092d0 con 0x564d1f89a9f0 2024-05-17T22:50:58.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.917+0000 7f7cab7fe700 1 -- 172.21.15.12:0/362496314 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb800e8b0 con 0x564d1f89a9f0 2024-05-17T22:50:58.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.918+0000 7f7cd5e69b80 1 -- 172.21.15.12:0/362496314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d1f89a9f0 msgr2=0x564d1fa1dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:58.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.918+0000 7f7cd5e69b80 1 --2- 172.21.15.12:0/362496314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d1f89a9f0 0x564d1fa1dec0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8003ee0 tx=0x7f7cb8009920 comp rx=0 tx=0).stop 2024-05-17T22:50:58.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.920+0000 7f7cd5e69b80 1 -- 172.21.15.12:0/362496314 shutdown_connections 2024-05-17T22:50:58.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.920+0000 7f7cd5e69b80 1 --2- 172.21.15.12:0/362496314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d1fa18b00 0x564d1fa18f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:58.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.920+0000 7f7cd5e69b80 1 --2- 172.21.15.12:0/362496314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d1f89a9f0 0x564d1fa1dec0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:58.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.920+0000 7f7cd5e69b80 1 --2- 172.21.15.12:0/362496314 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d1fa1e400 0x564d1fa185c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:58.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.920+0000 7f7cd5e69b80 1 -- 172.21.15.12:0/362496314 >> 172.21.15.12:0/362496314 conn(0x564d1f8fa7d0 msgr2=0x564d1fa131c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:50:58.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.922+0000 7f7cd5e69b80 1 -- 172.21.15.12:0/362496314 shutdown_connections 2024-05-17T22:50:58.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.924+0000 7f7cd5e69b80 1 -- 172.21.15.12:0/362496314 wait complete. 2024-05-17T22:50:58.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.925+0000 7f7cd5e69b80 1 -- start start 2024-05-17T22:50:58.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.925+0000 7f7cd5e69b80 1 Processor -- start 2024-05-17T22:50:58.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.926+0000 7f7cd5e69b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d1f89a9f0 0x564d1fa21a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:58.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.926+0000 7f7cd5e69b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d1fa18b00 0x564d1fa21f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:58.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.926+0000 7f7cd5e69b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d1fa1e400 0x564d1fa224d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:58.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.926+0000 7f7cd5e69b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x564d1f83e150 con 0x564d1f89a9f0 2024-05-17T22:50:58.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.926+0000 7f7cd5e69b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564d1f8d8940 con 0x564d1fa1e400 2024-05-17T22:50:58.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.926+0000 7f7cd5e69b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564d1fa24dd0 con 0x564d1fa18b00 2024-05-17T22:50:58.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.926+0000 7f7cd5e69b80 10 client.0 _ll_register_callbacks cb 0x564d1f8cdb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:50:58.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.928+0000 7f7cc522a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d1f89a9f0 0x564d1fa21a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:58.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.928+0000 7f7cc522a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d1f89a9f0 0x564d1fa21a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49598/0 (socket says 172.21.15.12:49598) 2024-05-17T22:50:58.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.928+0000 7f7cc522a700 1 -- 172.21.15.12:0/2912965636 learned_addr learned my addr 172.21.15.12:0/2912965636 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:50:58.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.928+0000 7f7cc4a29700 1 --2- 172.21.15.12:0/2912965636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d1fa18b00 0x564d1fa21f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:58.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.929+0000 7f7cc4a29700 1 -- 172.21.15.12:0/2912965636 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d1fa1e400 msgr2=0x564d1fa224d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T22:50:58.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.929+0000 7f7cc4a29700 1 --2- 172.21.15.12:0/2912965636 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d1fa1e400 0x564d1fa224d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:58.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.929+0000 7f7cc4a29700 1 -- 172.21.15.12:0/2912965636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d1f89a9f0 msgr2=0x564d1fa21a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:58.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.929+0000 7f7cc4a29700 1 --2- 172.21.15.12:0/2912965636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d1f89a9f0 0x564d1fa21a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:58.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.929+0000 7f7cc4a29700 1 -- 172.21.15.12:0/2912965636 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d1f887480 con 0x564d1fa18b00 2024-05-17T22:50:58.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.930+0000 7f7cc4a29700 1 --2- 172.21.15.12:0/2912965636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d1fa18b00 0x564d1fa21f90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc00b500 tx=0x7f7cbc0a9ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dac5d60e79c5330 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:50:58.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.933+0000 7f7ca8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:50:58.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.933+0000 7f7ca8ff9700 1 -- 172.21.15.12:0/2912965636 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc010ec0 con 0x564d1fa18b00 2024-05-17T22:50:58.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.934+0000 7f7cd5e69b80 1 -- 172.21.15.12:0/2912965636 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cb80092d0 con 0x564d1fa18b00 2024-05-17T22:50:58.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.934+0000 7f7ca8ff9700 1 -- 172.21.15.12:0/2912965636 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f7cbc0aa410 con 0x564d1fa18b00 2024-05-17T22:50:58.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.936+0000 7f7ca8ff9700 1 -- 172.21.15.12:0/2912965636 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7cbc0c3050 con 0x564d1fa18b00 2024-05-17T22:50:58.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.936+0000 7f7cd5e69b80 10 client.25963 fetch_fsmap learned FSMap version 39 2024-05-17T22:50:58.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.936+0000 7f7cd5e69b80 1 -- 172.21.15.12:0/2912965636 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564d1fa2f8b0 con 0x564d1fa18b00 2024-05-17T22:50:58.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.936+0000 7f7ca8ff9700 1 -- 172.21.15.12:0/2912965636 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cb80092d0 con 0x564d1fa18b00 2024-05-17T22:50:58.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.936+0000 7f7ca8ff9700 1 -- 172.21.15.12:0/2912965636 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f7cbc0b2970 con 0x564d1fa18b00 2024-05-17T22:50:58.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.936+0000 7f7cd5e69b80 10 client.25963 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:50:58.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.936+0000 7f7cd5e69b80 10 client.25963 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:50:58.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.937+0000 7f7cd5e69b80 20 client.25963 populate_metadata read hostname 'smithi012' 2024-05-17T22:50:58.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.937+0000 7f7cd5e69b80 1 --2- 172.21.15.12:0/2912965636 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x564d1fa31f10 0x564d1fa34390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:58.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.937+0000 7f7cd5e69b80 4 client.25963 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:50:58.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.937+0000 7f7cd5e69b80 1 -- 172.21.15.12:0/2912965636 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d1f949020 con 0x564d1fa31f10 2024-05-17T22:50:58.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.937+0000 7f7cc522a700 1 --2- 172.21.15.12:0/2912965636 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x564d1fa31f10 0x564d1fa34390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:58.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.938+0000 7f7cc522a700 1 --2- 172.21.15.12:0/2912965636 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x564d1fa31f10 0x564d1fa34390 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67e52076c7d9b891 server_cookie=32a0c126e54f819 in_seq=0 out_seq=0 2024-05-17T22:50:58.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.938+0000 7f7ca8ff9700 10 client.25963 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:50:58.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.938+0000 7f7ca8ff9700 1 -- 172.21.15.12:0/2912965636 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564d1f949020 con 0x564d1fa31f10 2024-05-17T22:50:58.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.938+0000 7f7ca8ff9700 10 client.25963 handle_command_reply: tid=0 2024-05-17T22:50:58.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.938+0000 7f7cd5e69b80 1 client.25963 shutdown 2024-05-17T22:50:58.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.939+0000 7f7cd5e69b80 1 -- 172.21.15.12:0/2912965636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d1fa18b00 msgr2=0x564d1fa21f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:58.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.939+0000 7f7cd5e69b80 1 --2- 172.21.15.12:0/2912965636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d1fa18b00 0x564d1fa21f90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc00b500 tx=0x7f7cbc0a9ea0 comp rx=0 tx=0).stop 2024-05-17T22:50:58.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.939+0000 7f7cd5e69b80 1 -- 172.21.15.12:0/2912965636 shutdown_connections 2024-05-17T22:50:58.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.939+0000 7f7cd5e69b80 1 --2- 172.21.15.12:0/2912965636 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d1fa1e400 0x564d1fa224d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:58.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.939+0000 7f7cd5e69b80 1 --2- 172.21.15.12:0/2912965636 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x564d1fa31f10 0x564d1fa34390 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:58.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.939+0000 7f7cd5e69b80 1 --2- 172.21.15.12:0/2912965636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d1f89a9f0 0x564d1fa21a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:58.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.939+0000 7f7cd5e69b80 1 --2- 172.21.15.12:0/2912965636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d1fa18b00 0x564d1fa21f90 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:58.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.939+0000 7f7cd5e69b80 1 -- 172.21.15.12:0/2912965636 >> 172.21.15.12:0/2912965636 conn(0x564d1f8fa7d0 msgr2=0x564d1fa13160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:50:58.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.939+0000 7f7ca8ff9700 0 client.25963 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:50:58.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.939+0000 7f7cd5e69b80 1 -- 172.21.15.12:0/2912965636 shutdown_connections 2024-05-17T22:50:58.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.939+0000 7f7cd5e69b80 1 -- 172.21.15.12:0/2912965636 wait complete. 2024-05-17T22:50:58.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.939+0000 7f7cd5e69b80 20 client.25963 trim_cache size 0 max 16384 2024-05-17T22:50:58.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.959+0000 7f7cd5e69b80 1 Processor -- start 2024-05-17T22:50:58.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.959+0000 7f7cd5e69b80 1 -- start start 2024-05-17T22:50:58.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.960+0000 7f7cd5e69b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d1f89a9f0 0x564d1fa1d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:58.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.960+0000 7f7cd5e69b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d1fa18b00 0x564d1fa22320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:58.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.960+0000 7f7cd5e69b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d1fa1e400 0x564d1fa20320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:58.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.960+0000 7f7cd5e69b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca4011460 con 0x564d1f89a9f0 2024-05-17T22:50:58.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.960+0000 7f7cd5e69b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca400ecf0 con 0x564d1fa1e400 2024-05-17T22:50:58.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.960+0000 7f7cd5e69b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca400cb40 con 0x564d1fa18b00 2024-05-17T22:50:58.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.960+0000 7f7cc522a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d1f89a9f0 0x564d1fa1d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:58.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.960+0000 7f7cc522a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d1f89a9f0 0x564d1fa1d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49606/0 (socket says 172.21.15.12:49606) 2024-05-17T22:50:58.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.960+0000 7f7cc522a700 1 -- 172.21.15.12:0/376015067 learned_addr learned my addr 172.21.15.12:0/376015067 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:50:58.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.960+0000 7f7cc4a29700 1 --2- 172.21.15.12:0/376015067 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d1fa18b00 0x564d1fa22320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:58.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.960+0000 7f7cc5a2b700 1 --2- 172.21.15.12:0/376015067 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d1fa1e400 0x564d1fa20320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:58.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.960+0000 7f7cc522a700 1 -- 172.21.15.12:0/376015067 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d1fa18b00 msgr2=0x564d1fa22320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:58.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.960+0000 7f7cc522a700 1 --2- 172.21.15.12:0/376015067 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d1fa18b00 0x564d1fa22320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:58.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.960+0000 7f7cc522a700 1 -- 172.21.15.12:0/376015067 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d1fa1e400 msgr2=0x564d1fa20320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:58.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.960+0000 7f7cc522a700 1 --2- 172.21.15.12:0/376015067 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d1fa1e400 0x564d1fa20320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:58.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.960+0000 7f7cc522a700 1 -- 172.21.15.12:0/376015067 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d1fa348e0 con 0x564d1f89a9f0 2024-05-17T22:50:58.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.960+0000 7f7cc4a29700 1 --2- 172.21.15.12:0/376015067 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d1fa18b00 0x564d1fa22320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:50:58.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.961+0000 7f7cc522a700 1 --2- 172.21.15.12:0/376015067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d1f89a9f0 0x564d1fa1d4b0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7cb00098e0 tx=0x7f7cb001a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ccda524ef2473 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:50:58.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.963+0000 7f7caaffd700 1 -- 172.21.15.12:0/376015067 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0037070 con 0x564d1f89a9f0 2024-05-17T22:50:58.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.963+0000 7f7caaffd700 1 -- 172.21.15.12:0/376015067 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cb0032860 con 0x564d1f89a9f0 2024-05-17T22:50:58.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.963+0000 7f7caaffd700 1 -- 172.21.15.12:0/376015067 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0003cf0 con 0x564d1f89a9f0 2024-05-17T22:50:58.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.965+0000 7f7cd5e69b80 1 -- 172.21.15.12:0/376015067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d1f89a9f0 msgr2=0x564d1fa1d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:58.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.965+0000 7f7cd5e69b80 1 --2- 172.21.15.12:0/376015067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d1f89a9f0 0x564d1fa1d4b0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7cb00098e0 tx=0x7f7cb001a050 comp rx=0 tx=0).stop 2024-05-17T22:50:58.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.965+0000 7f7cd5e69b80 1 -- 172.21.15.12:0/376015067 shutdown_connections 2024-05-17T22:50:58.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.965+0000 7f7cd5e69b80 1 --2- 172.21.15.12:0/376015067 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d1fa1e400 0x564d1fa20320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:58.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.965+0000 7f7cd5e69b80 1 --2- 172.21.15.12:0/376015067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d1f89a9f0 0x564d1fa1d4b0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:58.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.965+0000 7f7cd5e69b80 1 --2- 172.21.15.12:0/376015067 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d1fa18b00 0x564d1fa22320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:58.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.965+0000 7f7cd5e69b80 1 -- 172.21.15.12:0/376015067 >> 172.21.15.12:0/376015067 conn(0x564d1f8fa7d0 msgr2=0x564d1fa1cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:50:58.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.965+0000 7f7cd5e69b80 1 -- 172.21.15.12:0/376015067 shutdown_connections 2024-05-17T22:50:58.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.965+0000 7f7cd5e69b80 1 -- 172.21.15.12:0/376015067 wait complete. 2024-05-17T22:50:58.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.966+0000 7f7cd5e69b80 1 -- start start 2024-05-17T22:50:58.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.966+0000 7f7cd5e69b80 1 Processor -- start 2024-05-17T22:50:58.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.967+0000 7f7cd5e69b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d1f89a9f0 0x564d1fa25080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:58.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.967+0000 7f7cd5e69b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d1fa18b00 0x564d1fa255c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:58.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.967+0000 7f7cd5e69b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d1fa1e400 0x564d1fa9a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:58.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.967+0000 7f7cc522a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d1f89a9f0 0x564d1fa25080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:58.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.967+0000 7f7cc4a29700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d1fa18b00 0x564d1fa255c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:58.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.967+0000 7f7cc522a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d1f89a9f0 0x564d1fa25080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:53130/0 (socket says 172.21.15.12:53130) 2024-05-17T22:50:58.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.967+0000 7f7cc4a29700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d1fa18b00 0x564d1fa255c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49614/0 (socket says 172.21.15.12:49614) 2024-05-17T22:50:58.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.967+0000 7f7cc522a700 1 -- 172.21.15.12:0/1873980149 learned_addr learned my addr 172.21.15.12:0/1873980149 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:50:58.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.967+0000 7f7cd5e69b80 1 -- 172.21.15.12:0/1873980149 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca400d1b0 con 0x564d1fa18b00 2024-05-17T22:50:58.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.967+0000 7f7cd5e69b80 1 -- 172.21.15.12:0/1873980149 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca400e620 con 0x564d1f89a9f0 2024-05-17T22:50:58.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.967+0000 7f7cd5e69b80 1 -- 172.21.15.12:0/1873980149 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca400d880 con 0x564d1fa1e400 2024-05-17T22:50:58.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.967+0000 7f7cd5e69b80 10 client.0 _ll_register_callbacks cb 0x564d1f8cdb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:50:58.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.967+0000 7f7cc5a2b700 1 --2- 172.21.15.12:0/1873980149 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d1fa1e400 0x564d1fa9a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:58.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.967+0000 7f7cc522a700 1 -- 172.21.15.12:0/1873980149 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d1fa1e400 msgr2=0x564d1fa9a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:58.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.967+0000 7f7cc522a700 1 --2- 172.21.15.12:0/1873980149 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d1fa1e400 0x564d1fa9a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:58.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.967+0000 7f7cc522a700 1 -- 172.21.15.12:0/1873980149 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d1fa18b00 msgr2=0x564d1fa255c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:50:58.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.967+0000 7f7cc522a700 1 --2- 172.21.15.12:0/1873980149 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d1fa18b00 0x564d1fa255c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:58.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.967+0000 7f7cc522a700 1 -- 172.21.15.12:0/1873980149 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d1fa25e50 con 0x564d1f89a9f0 2024-05-17T22:50:58.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.968+0000 7f7cc5a2b700 1 --2- 172.21.15.12:0/1873980149 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d1fa1e400 0x564d1fa9a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:50:58.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.968+0000 7f7cc522a700 1 --2- 172.21.15.12:0/1873980149 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d1f89a9f0 0x564d1fa25080 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7cb000b2e0 tx=0x7f7cb000b770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d738161be98fc9a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:50:58.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.968+0000 7f7ca8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:50:58.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.968+0000 7f7ca8ff9700 1 -- 172.21.15.12:0/1873980149 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb003f020 con 0x564d1f89a9f0 2024-05-17T22:50:58.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.968+0000 7f7ca8ff9700 1 -- 172.21.15.12:0/1873980149 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f7cb000bd40 con 0x564d1f89a9f0 2024-05-17T22:50:58.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.968+0000 7f7cd5e69b80 1 -- 172.21.15.12:0/1873980149 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cb0032860 con 0x564d1f89a9f0 2024-05-17T22:50:58.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.968+0000 7f7ca8ff9700 1 -- 172.21.15.12:0/1873980149 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7cb0047050 con 0x564d1f89a9f0 2024-05-17T22:50:58.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.968+0000 7f7cd5e69b80 10 client.26033 fetch_fsmap learned FSMap version 39 2024-05-17T22:50:58.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.968+0000 7f7cd5e69b80 1 -- 172.21.15.12:0/1873980149 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564d1fa9bc80 con 0x564d1f89a9f0 2024-05-17T22:50:58.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.969+0000 7f7ca8ff9700 1 -- 172.21.15.12:0/1873980149 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cb0032860 con 0x564d1f89a9f0 2024-05-17T22:50:58.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.969+0000 7f7ca8ff9700 1 -- 172.21.15.12:0/1873980149 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f7cb000f4f0 con 0x564d1f89a9f0 2024-05-17T22:50:58.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.969+0000 7f7cd5e69b80 10 client.26033 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:50:58.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.969+0000 7f7cd5e69b80 10 client.26033 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:50:58.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.969+0000 7f7cd5e69b80 20 client.26033 populate_metadata read hostname 'smithi012' 2024-05-17T22:50:58.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.969+0000 7f7cd5e69b80 1 --2- 172.21.15.12:0/1873980149 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x564d1fa9c1c0 0x564d1fa9e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:50:58.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.969+0000 7f7cd5e69b80 4 client.26033 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:50:58.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.969+0000 7f7cd5e69b80 1 -- 172.21.15.12:0/1873980149 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564d1f942800 con 0x564d1fa9c1c0 2024-05-17T22:50:58.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.969+0000 7f7cc4a29700 1 --2- 172.21.15.12:0/1873980149 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x564d1fa9c1c0 0x564d1fa9e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:50:58.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.970+0000 7f7ca8ff9700 10 client.26033 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:50:58.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.970+0000 7f7cc4a29700 1 --2- 172.21.15.12:0/1873980149 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x564d1fa9c1c0 0x564d1fa9e5e0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e87a8f1b6aac310d server_cookie=25e3d1f6884cc0a in_seq=0 out_seq=0 2024-05-17T22:50:58.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.970+0000 7f7ca8ff9700 1 -- 172.21.15.12:0/1873980149 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564d1f942800 con 0x564d1fa9c1c0 2024-05-17T22:50:58.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.970+0000 7f7ca8ff9700 10 client.26033 handle_command_reply: tid=0 2024-05-17T22:50:58.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.970+0000 7f7cd5e69b80 1 client.26033 shutdown 2024-05-17T22:50:58.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.970+0000 7f7cd5e69b80 1 -- 172.21.15.12:0/1873980149 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d1f89a9f0 msgr2=0x564d1fa25080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:58.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.970+0000 7f7cd5e69b80 1 --2- 172.21.15.12:0/1873980149 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d1f89a9f0 0x564d1fa25080 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7cb000b2e0 tx=0x7f7cb000b770 comp rx=0 tx=0).stop 2024-05-17T22:50:58.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.970+0000 7f7cd5e69b80 1 -- 172.21.15.12:0/1873980149 shutdown_connections 2024-05-17T22:50:58.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.970+0000 7f7cd5e69b80 1 --2- 172.21.15.12:0/1873980149 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d1fa1e400 0x564d1fa9a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:58.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.970+0000 7f7cd5e69b80 1 --2- 172.21.15.12:0/1873980149 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d1f89a9f0 0x564d1fa25080 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:58.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.970+0000 7f7cd5e69b80 1 --2- 172.21.15.12:0/1873980149 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x564d1fa9c1c0 0x564d1fa9e5e0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:58.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.970+0000 7f7cd5e69b80 1 --2- 172.21.15.12:0/1873980149 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d1fa18b00 0x564d1fa255c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:58.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.970+0000 7f7cd5e69b80 1 -- 172.21.15.12:0/1873980149 >> 172.21.15.12:0/1873980149 conn(0x564d1f8fa7d0 msgr2=0x564d1fa22330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:50:58.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.970+0000 7f7ca8ff9700 0 client.26033 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:50:58.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.971+0000 7f7cd5e69b80 1 -- 172.21.15.12:0/1873980149 shutdown_connections 2024-05-17T22:50:58.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.971+0000 7f7cd5e69b80 1 -- 172.21.15.12:0/1873980149 wait complete. 2024-05-17T22:50:58.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.971+0000 7f7cd5e69b80 20 client.26033 trim_cache size 0 max 16384 2024-05-17T22:50:58.978 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:50:58.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.972+0000 7f7cc748e700 1 -- 172.21.15.12:0/964777361 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f7cac0b29c0 msgr2=0x7f7cac0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:58.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.972+0000 7f7cc748e700 1 --2- 172.21.15.12:0/964777361 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f7cac0b29c0 0x7f7cac0b4ed0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0006010 tx=0x7f7cb00058e0 comp rx=0 tx=0).stop 2024-05-17T22:50:58.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.972+0000 7f7cc748e700 1 -- 172.21.15.12:0/964777361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7cc01034e0 msgr2=0x7f7cc006e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:50:58.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.972+0000 7f7cc748e700 1 --2- 172.21.15.12:0/964777361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7cc01034e0 0x7f7cc006e5e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc00b530 tx=0x7f7cbc00b840 comp rx=0 tx=0).stop 2024-05-17T22:50:58.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.972+0000 7f7cc748e700 1 -- 172.21.15.12:0/964777361 shutdown_connections 2024-05-17T22:50:58.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.972+0000 7f7cc748e700 1 --2- 172.21.15.12:0/964777361 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f7cac0b29c0 0x7f7cac0b4ed0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:58.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.972+0000 7f7cc748e700 1 --2- 172.21.15.12:0/964777361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7cc0107fd0 0x7f7cc006eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:58.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.972+0000 7f7cc748e700 1 --2- 172.21.15.12:0/964777361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cc0102b90 0x7f7cc00754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:58.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.972+0000 7f7cc748e700 1 --2- 172.21.15.12:0/964777361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7cc01034e0 0x7f7cc006e5e0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:50:58.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.972+0000 7f7cc748e700 1 -- 172.21.15.12:0/964777361 >> 172.21.15.12:0/964777361 conn(0x7f7cc0079bd0 msgr2=0x7f7cc01059e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:50:58.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.973+0000 7f7cc748e700 1 -- 172.21.15.12:0/964777361 shutdown_connections 2024-05-17T22:50:58.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:50:58.973+0000 7f7cc748e700 1 -- 172.21.15.12:0/964777361 wait complete. 2024-05-17T22:51:00.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:00 smithi012 ceph-mon[31632]: pgmap v791: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.2 MiB/s wr, 449 op/s 2024-05-17T22:51:00.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:00 smithi151 ceph-mon[36101]: pgmap v791: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.2 MiB/s wr, 449 op/s 2024-05-17T22:51:00.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:00 smithi096 ceph-mon[36065]: pgmap v791: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.2 MiB/s wr, 449 op/s 2024-05-17T22:51:00.744 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-17T22:51:00.744 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-17T22:51:01.269 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:51:01.372 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 700.0 s, 471.8 tps, lat 203.803 ms stddev 225.541 2024-05-17T22:51:02.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:02 smithi012 ceph-mon[31632]: pgmap v792: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.4 MiB/s wr, 449 op/s 2024-05-17T22:51:02.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:02 smithi151 ceph-mon[36101]: pgmap v792: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.4 MiB/s wr, 449 op/s 2024-05-17T22:51:02.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:02 smithi096 ceph-mon[36065]: pgmap v792: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.4 MiB/s wr, 449 op/s 2024-05-17T22:51:03.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.771+0000 7f50a602b700 1 -- 172.21.15.12:0/659565584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50980abae0 msgr2=0x7f50980abef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:51:03.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.771+0000 7f50a602b700 1 --2- 172.21.15.12:0/659565584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50980abae0 0x7f50980abef0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f509c0098f0 tx=0x7f509c009c00 comp rx=0 tx=0).stop 2024-05-17T22:51:03.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.772+0000 7f50a602b700 1 -- 172.21.15.12:0/659565584 shutdown_connections 2024-05-17T22:51:03.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.772+0000 7f50a602b700 1 --2- 172.21.15.12:0/659565584 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50980ac430 0x7f50980b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:03.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.772+0000 7f50a602b700 1 --2- 172.21.15.12:0/659565584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f50980a4f20 0x7f50980a5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:03.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.772+0000 7f50a602b700 1 --2- 172.21.15.12:0/659565584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50980abae0 0x7f50980abef0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:03.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.772+0000 7f50a602b700 1 -- 172.21.15.12:0/659565584 >> 172.21.15.12:0/659565584 conn(0x7f509801a9f0 msgr2=0x7f509801ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:51:03.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.780+0000 7f50a602b700 1 -- 172.21.15.12:0/659565584 shutdown_connections 2024-05-17T22:51:03.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.781+0000 7f50a602b700 1 -- 172.21.15.12:0/659565584 wait complete. 2024-05-17T22:51:03.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.782+0000 7f50a602b700 1 Processor -- start 2024-05-17T22:51:03.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.782+0000 7f50a602b700 1 -- start start 2024-05-17T22:51:03.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.782+0000 7f50a602b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50980a4f20 0x7f50980bb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:03.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.782+0000 7f50a602b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50980abae0 0x7f50980bbc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:03.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.782+0000 7f50a602b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f50980ac430 0x7f50980b68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:03.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.782+0000 7f50a602b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f50980b4e40 con 0x7f50980ac430 2024-05-17T22:51:03.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.782+0000 7f50a602b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f50980b4cc0 con 0x7f50980abae0 2024-05-17T22:51:03.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.782+0000 7f50a602b700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f50980b4fc0 con 0x7f50980a4f20 2024-05-17T22:51:03.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.785+0000 7f50a5029700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50980a4f20 0x7f50980bb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:03.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.785+0000 7f50a5029700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50980a4f20 0x7f50980bb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:34656/0 (socket says 172.21.15.12:34656) 2024-05-17T22:51:03.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.785+0000 7f50a5029700 1 -- 172.21.15.12:0/2353351328 learned_addr learned my addr 172.21.15.12:0/2353351328 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:51:03.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.785+0000 7f50a582a700 1 --2- 172.21.15.12:0/2353351328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f50980ac430 0x7f50980b68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:03.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.786+0000 7f50a4828700 1 --2- 172.21.15.12:0/2353351328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50980abae0 0x7f50980bbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:03.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.789+0000 7f50a5029700 1 -- 172.21.15.12:0/2353351328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50980abae0 msgr2=0x7f50980bbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:51:03.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.789+0000 7f50a5029700 1 --2- 172.21.15.12:0/2353351328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50980abae0 0x7f50980bbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:03.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.789+0000 7f50a5029700 1 -- 172.21.15.12:0/2353351328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f50980ac430 msgr2=0x7f50980b68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:51:03.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.789+0000 7f50a5029700 1 --2- 172.21.15.12:0/2353351328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f50980ac430 0x7f50980b68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:03.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.789+0000 7f50a5029700 1 -- 172.21.15.12:0/2353351328 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50980b7110 con 0x7f50980a4f20 2024-05-17T22:51:03.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.791+0000 7f50a5029700 1 --2- 172.21.15.12:0/2353351328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50980a4f20 0x7f50980bb720 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f50a0052cd0 tx=0x7f50a0075630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:51:03.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.793+0000 7f50967fc700 1 -- 172.21.15.12:0/2353351328 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a0077430 con 0x7f50980a4f20 2024-05-17T22:51:03.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.793+0000 7f50967fc700 1 -- 172.21.15.12:0/2353351328 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50a0077a80 con 0x7f50980a4f20 2024-05-17T22:51:03.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.793+0000 7f50967fc700 1 -- 172.21.15.12:0/2353351328 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a007fa20 con 0x7f50980a4f20 2024-05-17T22:51:03.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.794+0000 7f50a602b700 1 -- 172.21.15.12:0/2353351328 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50980b7400 con 0x7f50980a4f20 2024-05-17T22:51:03.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.794+0000 7f50a602b700 1 -- 172.21.15.12:0/2353351328 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50980c3450 con 0x7f50980a4f20 2024-05-17T22:51:03.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.799+0000 7f50b5c68b80 1 Processor -- start 2024-05-17T22:51:03.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.804+0000 7f50b5c68b80 1 -- start start 2024-05-17T22:51:03.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.805+0000 7f50b5c68b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eceb14da70 0x55eceb2d17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:03.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.805+0000 7f50b5c68b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eceb2d1d30 0x55eceb2cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:03.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.805+0000 7f50b5c68b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eceb2cc1f0 0x55eceb2cc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:03.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.805+0000 7f50b5c68b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55eceb1fa310 con 0x55eceb14da70 2024-05-17T22:51:03.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.805+0000 7f50b5c68b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55eceb1ea050 con 0x55eceb2d1d30 2024-05-17T22:51:03.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.805+0000 7f50b5c68b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55eceb1edef0 con 0x55eceb2cc1f0 2024-05-17T22:51:03.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.807+0000 7f50a4828700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eceb2cc1f0 0x55eceb2cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:03.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.807+0000 7f50967fc700 1 -- 172.21.15.12:0/2353351328 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f50a0077c30 con 0x7f50980a4f20 2024-05-17T22:51:03.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.807+0000 7f50a582a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eceb14da70 0x55eceb2d17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:03.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.807+0000 7f50a582a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eceb14da70 0x55eceb2d17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58232/0 (socket says 172.21.15.12:58232) 2024-05-17T22:51:03.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.807+0000 7f50a582a700 1 -- 172.21.15.12:0/101270189 learned_addr learned my addr 172.21.15.12:0/101270189 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:51:03.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.808+0000 7f50a582a700 1 -- 172.21.15.12:0/101270189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eceb2cc1f0 msgr2=0x55eceb2cc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:51:03.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.808+0000 7f50a582a700 1 --2- 172.21.15.12:0/101270189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eceb2cc1f0 0x55eceb2cc600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:03.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.808+0000 7f50a582a700 1 -- 172.21.15.12:0/101270189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eceb2d1d30 msgr2=0x55eceb2cbcb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T22:51:03.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.808+0000 7f50a582a700 1 --2- 172.21.15.12:0/101270189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eceb2d1d30 0x55eceb2cbcb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:03.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.808+0000 7f50a582a700 1 -- 172.21.15.12:0/101270189 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eceb193610 con 0x55eceb14da70 2024-05-17T22:51:03.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.808+0000 7f50a582a700 1 --2- 172.21.15.12:0/101270189 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eceb14da70 0x55eceb2d17f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f5090009bb0 tx=0x7f5090009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=969352e2ea02849d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:51:03.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.808+0000 7f50967fc700 1 --2- 172.21.15.12:0/2353351328 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f508c0b2a80 0x7f508c0b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:03.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.809+0000 7f50967fc700 1 -- 172.21.15.12:0/2353351328 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f50a0147d40 con 0x7f50980a4f20 2024-05-17T22:51:03.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.809+0000 7f50a5029700 1 --2- 172.21.15.12:0/2353351328 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f508c0b2a80 0x7f508c0b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:03.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.813+0000 7f50a5029700 1 --2- 172.21.15.12:0/2353351328 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f508c0b2a80 0x7f508c0b4f90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f50a010ec70 tx=0x7f50a0069220 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:51:03.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.815+0000 7f508b7fe700 1 -- 172.21.15.12:0/101270189 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5090011bf0 con 0x55eceb14da70 2024-05-17T22:51:03.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.815+0000 7f508b7fe700 1 -- 172.21.15.12:0/101270189 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5090011d90 con 0x55eceb14da70 2024-05-17T22:51:03.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.815+0000 7f508b7fe700 1 -- 172.21.15.12:0/101270189 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509000f730 con 0x55eceb14da70 2024-05-17T22:51:03.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.819+0000 7f50b5c68b80 1 -- 172.21.15.12:0/101270189 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eceb14da70 msgr2=0x55eceb2d17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:51:03.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.819+0000 7f50b5c68b80 1 --2- 172.21.15.12:0/101270189 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eceb14da70 0x55eceb2d17f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f5090009bb0 tx=0x7f5090009ec0 comp rx=0 tx=0).stop 2024-05-17T22:51:03.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.819+0000 7f50b5c68b80 1 -- 172.21.15.12:0/101270189 shutdown_connections 2024-05-17T22:51:03.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.819+0000 7f50b5c68b80 1 --2- 172.21.15.12:0/101270189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eceb2cc1f0 0x55eceb2cc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:03.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.819+0000 7f50b5c68b80 1 --2- 172.21.15.12:0/101270189 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eceb14da70 0x55eceb2d17f0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:03.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.819+0000 7f50b5c68b80 1 --2- 172.21.15.12:0/101270189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eceb2d1d30 0x55eceb2cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:03.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.819+0000 7f50b5c68b80 1 -- 172.21.15.12:0/101270189 >> 172.21.15.12:0/101270189 conn(0x55eceb191f80 msgr2=0x55eceb2c6c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:51:03.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.824+0000 7f50b5c68b80 1 -- 172.21.15.12:0/101270189 shutdown_connections 2024-05-17T22:51:03.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.828+0000 7f50b5c68b80 1 -- 172.21.15.12:0/101270189 wait complete. 2024-05-17T22:51:03.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.828+0000 7f50b5c68b80 1 -- start start 2024-05-17T22:51:03.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.828+0000 7f50b5c68b80 1 Processor -- start 2024-05-17T22:51:03.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.837+0000 7f50b5c68b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eceb14da70 0x55eceb2d53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:03.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.837+0000 7f50b5c68b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eceb2cc1f0 0x55eceb2d5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:03.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.837+0000 7f50b5c68b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eceb2d1d30 0x55eceb2d5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:03.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.837+0000 7f50b5c68b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55eceb0f11d0 con 0x55eceb2cc1f0 2024-05-17T22:51:03.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.837+0000 7f50b5c68b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55eceb18af00 con 0x55eceb2d1d30 2024-05-17T22:51:03.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.837+0000 7f50b5c68b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55eceb2d8620 con 0x55eceb14da70 2024-05-17T22:51:03.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.837+0000 7f50b5c68b80 10 client.0 _ll_register_callbacks cb 0x55eceb184f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:51:03.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.840+0000 7f50a582a700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eceb14da70 0x55eceb2d53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:03.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.840+0000 7f50a582a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eceb2d1d30 0x55eceb2d5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:03.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.840+0000 7f50a582a700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eceb14da70 0x55eceb2d53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:34676/0 (socket says 172.21.15.12:34676) 2024-05-17T22:51:03.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.840+0000 7f50a582a700 1 -- 172.21.15.12:0/2169109451 learned_addr learned my addr 172.21.15.12:0/2169109451 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:51:03.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.845+0000 7f50a582a700 1 -- 172.21.15.12:0/2169109451 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eceb2d1d30 msgr2=0x55eceb2d5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:51:03.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.845+0000 7f50a582a700 1 --2- 172.21.15.12:0/2169109451 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eceb2d1d30 0x55eceb2d5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:03.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.845+0000 7f50a582a700 1 -- 172.21.15.12:0/2169109451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eceb2cc1f0 msgr2=0x55eceb2d5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:51:03.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.845+0000 7f50a582a700 1 --2- 172.21.15.12:0/2169109451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eceb2cc1f0 0x55eceb2d5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:03.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.845+0000 7f50a582a700 1 -- 172.21.15.12:0/2169109451 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eceb13a500 con 0x55eceb14da70 2024-05-17T22:51:03.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.848+0000 7f50a582a700 1 --2- 172.21.15.12:0/2169109451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eceb14da70 0x55eceb2d53d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x55eceb2c7800 tx=0x7f5090012ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7adb92cbdf98993c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:51:03.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.850+0000 7f5088ff9700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:51:03.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.852+0000 7f5088ff9700 1 -- 172.21.15.12:0/2169109451 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5090012710 con 0x55eceb14da70 2024-05-17T22:51:03.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.852+0000 7f5088ff9700 1 -- 172.21.15.12:0/2169109451 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f50900128b0 con 0x55eceb14da70 2024-05-17T22:51:03.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.855+0000 7f50b5c68b80 1 -- 172.21.15.12:0/2169109451 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5090011d90 con 0x55eceb14da70 2024-05-17T22:51:03.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.861+0000 7f5088ff9700 1 -- 172.21.15.12:0/2169109451 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5090012ab0 con 0x55eceb14da70 2024-05-17T22:51:03.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.863+0000 7f50b5c68b80 10 client.25975 fetch_fsmap learned FSMap version 39 2024-05-17T22:51:03.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.863+0000 7f50b5c68b80 1 -- 172.21.15.12:0/2169109451 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55eceb2e3040 con 0x55eceb14da70 2024-05-17T22:51:03.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.865+0000 7f5088ff9700 1 -- 172.21.15.12:0/2169109451 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5090011d90 con 0x55eceb14da70 2024-05-17T22:51:03.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.865+0000 7f5088ff9700 1 -- 172.21.15.12:0/2169109451 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f5090026040 con 0x55eceb14da70 2024-05-17T22:51:03.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.867+0000 7f50b5c68b80 10 client.25975 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:51:03.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.867+0000 7f50b5c68b80 10 client.25975 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:51:03.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.867+0000 7f50b5c68b80 20 client.25975 populate_metadata read hostname 'smithi012' 2024-05-17T22:51:03.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.867+0000 7f50b5c68b80 1 --2- 172.21.15.12:0/2169109451 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55eceb2e55d0 0x55eceb2e79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:03.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.867+0000 7f50b5c68b80 4 client.25975 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:51:03.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.867+0000 7f50b5c68b80 1 -- 172.21.15.12:0/2169109451 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eceb1fc260 con 0x55eceb2e55d0 2024-05-17T22:51:03.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.868+0000 7f50a4828700 1 --2- 172.21.15.12:0/2169109451 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55eceb2e55d0 0x55eceb2e79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:03.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.869+0000 7f50a4828700 1 --2- 172.21.15.12:0/2169109451 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55eceb2e55d0 0x55eceb2e79f0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2083c58cd42ea4c3 server_cookie=96418d747a57e6d8 in_seq=0 out_seq=0 2024-05-17T22:51:03.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.869+0000 7f5088ff9700 10 client.25975 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:51:03.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.872+0000 7f5088ff9700 1 -- 172.21.15.12:0/2169109451 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55eceb1fc260 con 0x55eceb2e55d0 2024-05-17T22:51:03.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.872+0000 7f5088ff9700 10 client.25975 handle_command_reply: tid=0 2024-05-17T22:51:03.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.872+0000 7f50b5c68b80 1 client.25975 shutdown 2024-05-17T22:51:03.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.874+0000 7f50b5c68b80 1 -- 172.21.15.12:0/2169109451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eceb14da70 msgr2=0x55eceb2d53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:51:03.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.874+0000 7f50b5c68b80 1 --2- 172.21.15.12:0/2169109451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eceb14da70 0x55eceb2d53d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x55eceb2c7800 tx=0x7f5090012ed0 comp rx=0 tx=0).stop 2024-05-17T22:51:03.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.875+0000 7f50b5c68b80 1 -- 172.21.15.12:0/2169109451 shutdown_connections 2024-05-17T22:51:03.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.875+0000 7f50b5c68b80 1 --2- 172.21.15.12:0/2169109451 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eceb2d1d30 0x55eceb2d5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:03.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.875+0000 7f50b5c68b80 1 --2- 172.21.15.12:0/2169109451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eceb14da70 0x55eceb2d53d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:03.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.875+0000 7f50b5c68b80 1 --2- 172.21.15.12:0/2169109451 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55eceb2e55d0 0x55eceb2e79f0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:03.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.875+0000 7f50b5c68b80 1 --2- 172.21.15.12:0/2169109451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eceb2cc1f0 0x55eceb2d5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:03.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.875+0000 7f50b5c68b80 1 -- 172.21.15.12:0/2169109451 >> 172.21.15.12:0/2169109451 conn(0x55eceb191f80 msgr2=0x55eceb2c6190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:51:03.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.875+0000 7f5088ff9700 0 client.25975 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:51:03.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.876+0000 7f50b5c68b80 1 -- 172.21.15.12:0/2169109451 shutdown_connections 2024-05-17T22:51:03.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.878+0000 7f50b5c68b80 1 -- 172.21.15.12:0/2169109451 wait complete. 2024-05-17T22:51:03.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.878+0000 7f50b5c68b80 20 client.25975 trim_cache size 0 max 16384 2024-05-17T22:51:03.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.912+0000 7f50b5c68b80 1 Processor -- start 2024-05-17T22:51:03.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.912+0000 7f50b5c68b80 1 -- start start 2024-05-17T22:51:03.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.913+0000 7f50b5c68b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eceb14da70 0x55eceb2cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:03.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.913+0000 7f50b5c68b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eceb2cc1f0 0x55eceb2d2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:03.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.913+0000 7f50b5c68b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eceb2d1d30 0x55eceb2d7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:03.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.913+0000 7f50a582a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eceb14da70 0x55eceb2cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:03.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.913+0000 7f50a582a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eceb14da70 0x55eceb2cbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58236/0 (socket says 172.21.15.12:58236) 2024-05-17T22:51:03.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.913+0000 7f50a582a700 1 -- 172.21.15.12:0/2336324771 learned_addr learned my addr 172.21.15.12:0/2336324771 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:51:03.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.913+0000 7f50a4828700 1 --2- 172.21.15.12:0/2336324771 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eceb2cc1f0 0x55eceb2d2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:03.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.913+0000 7f50b5c68b80 1 -- 172.21.15.12:0/2336324771 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f50980cebc0 con 0x55eceb14da70 2024-05-17T22:51:03.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.913+0000 7f50b5c68b80 1 -- 172.21.15.12:0/2336324771 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f50980cc450 con 0x55eceb2d1d30 2024-05-17T22:51:03.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.913+0000 7f50b5c68b80 1 -- 172.21.15.12:0/2336324771 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f50980ca2a0 con 0x55eceb2cc1f0 2024-05-17T22:51:03.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.914+0000 7f50a582a700 1 --2- 172.21.15.12:0/2336324771 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eceb2d1d30 0x55eceb2d7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:03.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.914+0000 7f50a582a700 1 -- 172.21.15.12:0/2336324771 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eceb2cc1f0 msgr2=0x55eceb2d2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:51:03.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.914+0000 7f50a582a700 1 --2- 172.21.15.12:0/2336324771 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eceb2cc1f0 0x55eceb2d2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:03.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.914+0000 7f50a582a700 1 -- 172.21.15.12:0/2336324771 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eceb2d1d30 msgr2=0x55eceb2d7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:51:03.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.914+0000 7f50a582a700 1 --2- 172.21.15.12:0/2336324771 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eceb2d1d30 0x55eceb2d7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:03.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.914+0000 7f50a582a700 1 -- 172.21.15.12:0/2336324771 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eceb2d3920 con 0x55eceb14da70 2024-05-17T22:51:03.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.914+0000 7f50a582a700 1 --2- 172.21.15.12:0/2336324771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eceb14da70 0x55eceb2cbcb0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f5090005850 tx=0x7f50900057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=584c26b77dd478b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:51:03.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.914+0000 7f508affd700 1 -- 172.21.15.12:0/2336324771 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5090011c30 con 0x55eceb14da70 2024-05-17T22:51:03.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.914+0000 7f508affd700 1 -- 172.21.15.12:0/2336324771 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5090012710 con 0x55eceb14da70 2024-05-17T22:51:03.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.914+0000 7f508affd700 1 -- 172.21.15.12:0/2336324771 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5090003d90 con 0x55eceb14da70 2024-05-17T22:51:03.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.915+0000 7f50b5c68b80 1 -- 172.21.15.12:0/2336324771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eceb14da70 msgr2=0x55eceb2cbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:51:03.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.915+0000 7f50b5c68b80 1 --2- 172.21.15.12:0/2336324771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eceb14da70 0x55eceb2cbcb0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f5090005850 tx=0x7f50900057e0 comp rx=0 tx=0).stop 2024-05-17T22:51:03.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.915+0000 7f50b5c68b80 1 -- 172.21.15.12:0/2336324771 shutdown_connections 2024-05-17T22:51:03.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.915+0000 7f50b5c68b80 1 --2- 172.21.15.12:0/2336324771 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eceb2d1d30 0x55eceb2d7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:03.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.915+0000 7f50b5c68b80 1 --2- 172.21.15.12:0/2336324771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eceb14da70 0x55eceb2cbcb0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:03.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.915+0000 7f50b5c68b80 1 --2- 172.21.15.12:0/2336324771 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eceb2cc1f0 0x55eceb2d2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:03.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.915+0000 7f50b5c68b80 1 -- 172.21.15.12:0/2336324771 >> 172.21.15.12:0/2336324771 conn(0x55eceb191f80 msgr2=0x55eceb2d4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:51:03.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.915+0000 7f50b5c68b80 1 -- 172.21.15.12:0/2336324771 shutdown_connections 2024-05-17T22:51:03.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.915+0000 7f50b5c68b80 1 -- 172.21.15.12:0/2336324771 wait complete. 2024-05-17T22:51:03.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.916+0000 7f50b5c68b80 1 -- start start 2024-05-17T22:51:03.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.916+0000 7f50b5c68b80 1 Processor -- start 2024-05-17T22:51:03.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.917+0000 7f50b5c68b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eceb14da70 0x55eceb2eab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:03.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.917+0000 7f50b5c68b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eceb2cc1f0 0x55eceb2e8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:03.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.917+0000 7f50b5c68b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eceb2d1d30 0x55eceb2e9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:03.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.917+0000 7f50b5c68b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f50980ca910 con 0x55eceb14da70 2024-05-17T22:51:03.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.917+0000 7f50b5c68b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f50980cbd80 con 0x55eceb2d1d30 2024-05-17T22:51:03.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.917+0000 7f50b5c68b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f50980cafe0 con 0x55eceb2cc1f0 2024-05-17T22:51:03.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.917+0000 7f50b5c68b80 10 client.0 _ll_register_callbacks cb 0x55eceb184f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:51:03.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.917+0000 7f50a4828700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eceb2cc1f0 0x55eceb2e8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:03.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.917+0000 7f50a4828700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eceb2cc1f0 0x55eceb2e8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:34690/0 (socket says 172.21.15.12:34690) 2024-05-17T22:51:03.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.917+0000 7f50a4828700 1 -- 172.21.15.12:0/2168126109 learned_addr learned my addr 172.21.15.12:0/2168126109 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:51:03.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.917+0000 7f50a582a700 1 --2- 172.21.15.12:0/2168126109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eceb14da70 0x55eceb2eab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:03.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.918+0000 7f50a4828700 1 -- 172.21.15.12:0/2168126109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eceb2d1d30 msgr2=0x55eceb2e9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:51:03.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.918+0000 7f50a4828700 1 --2- 172.21.15.12:0/2168126109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eceb2d1d30 0x55eceb2e9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:03.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.918+0000 7f50a4828700 1 -- 172.21.15.12:0/2168126109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eceb14da70 msgr2=0x55eceb2eab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:51:03.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.918+0000 7f50a4828700 1 --2- 172.21.15.12:0/2168126109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eceb14da70 0x55eceb2eab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:03.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.918+0000 7f50a4828700 1 -- 172.21.15.12:0/2168126109 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eceb2e9ba0 con 0x55eceb2cc1f0 2024-05-17T22:51:03.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.918+0000 7f50a582a700 1 --2- 172.21.15.12:0/2168126109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eceb14da70 0x55eceb2eab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:51:03.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.918+0000 7f50a4828700 1 --2- 172.21.15.12:0/2168126109 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eceb2cc1f0 0x55eceb2e8ee0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f509c0098f0 tx=0x7f509c005e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2604068f0cb6f93 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:51:03.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.918+0000 7f5088ff9700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:51:03.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.918+0000 7f5088ff9700 1 -- 172.21.15.12:0/2168126109 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509c02a030 con 0x55eceb2cc1f0 2024-05-17T22:51:03.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.918+0000 7f5088ff9700 1 -- 172.21.15.12:0/2168126109 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f509c00f860 con 0x55eceb2cc1f0 2024-05-17T22:51:03.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.918+0000 7f50b5c68b80 1 -- 172.21.15.12:0/2168126109 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5090012710 con 0x55eceb2cc1f0 2024-05-17T22:51:03.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.919+0000 7f5088ff9700 1 -- 172.21.15.12:0/2168126109 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f509c036050 con 0x55eceb2cc1f0 2024-05-17T22:51:03.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.919+0000 7f50b5c68b80 10 client.25981 fetch_fsmap learned FSMap version 39 2024-05-17T22:51:03.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.919+0000 7f50b5c68b80 1 -- 172.21.15.12:0/2168126109 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55eceb350a80 con 0x55eceb2cc1f0 2024-05-17T22:51:03.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.919+0000 7f5088ff9700 1 -- 172.21.15.12:0/2168126109 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f509c0044d0 con 0x55eceb2cc1f0 2024-05-17T22:51:03.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.919+0000 7f5088ff9700 1 -- 172.21.15.12:0/2168126109 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f509c005040 con 0x55eceb2cc1f0 2024-05-17T22:51:03.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.919+0000 7f50b5c68b80 10 client.25981 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:51:03.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.919+0000 7f50b5c68b80 10 client.25981 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:51:03.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.919+0000 7f50b5c68b80 20 client.25981 populate_metadata read hostname 'smithi012' 2024-05-17T22:51:03.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.919+0000 7f50b5c68b80 1 --2- 172.21.15.12:0/2168126109 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55eceb3511c0 0x55eceb353600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:03.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.919+0000 7f50b5c68b80 4 client.25981 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-17T22:51:03.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.919+0000 7f50b5c68b80 1 -- 172.21.15.12:0/2168126109 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f5090012710 con 0x55eceb3511c0 2024-05-17T22:51:03.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.920+0000 7f50a582a700 1 --2- 172.21.15.12:0/2168126109 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55eceb3511c0 0x55eceb353600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:03.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.920+0000 7f5088ff9700 10 client.25981 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:51:03.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.920+0000 7f50a582a700 1 --2- 172.21.15.12:0/2168126109 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55eceb3511c0 0x55eceb353600 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83e3c6602e5cca94 server_cookie=baee516c7c90dbcc in_seq=0 out_seq=0 2024-05-17T22:51:03.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.921+0000 7f5088ff9700 1 -- 172.21.15.12:0/2168126109 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f5090012710 con 0x55eceb3511c0 2024-05-17T22:51:03.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.921+0000 7f5088ff9700 10 client.25981 handle_command_reply: tid=0 2024-05-17T22:51:03.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.921+0000 7f50b5c68b80 1 client.25981 shutdown 2024-05-17T22:51:03.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.922+0000 7f50b5c68b80 1 -- 172.21.15.12:0/2168126109 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eceb2cc1f0 msgr2=0x55eceb2e8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:51:03.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.922+0000 7f50b5c68b80 1 --2- 172.21.15.12:0/2168126109 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eceb2cc1f0 0x55eceb2e8ee0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f509c0098f0 tx=0x7f509c005e70 comp rx=0 tx=0).stop 2024-05-17T22:51:03.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.922+0000 7f50b5c68b80 1 -- 172.21.15.12:0/2168126109 shutdown_connections 2024-05-17T22:51:03.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.922+0000 7f50b5c68b80 1 --2- 172.21.15.12:0/2168126109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eceb2d1d30 0x55eceb2e9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:03.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.922+0000 7f50b5c68b80 1 --2- 172.21.15.12:0/2168126109 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55eceb3511c0 0x55eceb353600 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:03.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.922+0000 7f5088ff9700 0 client.25981 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:51:03.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.922+0000 7f50b5c68b80 1 --2- 172.21.15.12:0/2168126109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eceb14da70 0x55eceb2eab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:03.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.922+0000 7f50b5c68b80 1 --2- 172.21.15.12:0/2168126109 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eceb2cc1f0 0x55eceb2e8ee0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:03.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.922+0000 7f50b5c68b80 1 -- 172.21.15.12:0/2168126109 >> 172.21.15.12:0/2168126109 conn(0x55eceb191f80 msgr2=0x55eceb2e0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:51:03.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.922+0000 7f50b5c68b80 1 -- 172.21.15.12:0/2168126109 shutdown_connections 2024-05-17T22:51:03.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.922+0000 7f50b5c68b80 1 -- 172.21.15.12:0/2168126109 wait complete. 2024-05-17T22:51:03.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.922+0000 7f50b5c68b80 20 client.25981 trim_cache size 0 max 16384 2024-05-17T22:51:03.928 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:51:03.928 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-17T22:51:03.928 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "11c72a3d-f39b-46bc-a156-10a95d956679", 2024-05-17T22:51:03.928 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-17T22:51:03.928 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:51:03.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.923+0000 7f50a602b700 1 -- 172.21.15.12:0/2353351328 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f508c0b2a80 msgr2=0x7f508c0b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:51:03.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.923+0000 7f50a602b700 1 --2- 172.21.15.12:0/2353351328 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f508c0b2a80 0x7f508c0b4f90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f50a010ec70 tx=0x7f50a0069220 comp rx=0 tx=0).stop 2024-05-17T22:51:03.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.923+0000 7f50a602b700 1 -- 172.21.15.12:0/2353351328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50980a4f20 msgr2=0x7f50980bb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:51:03.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.923+0000 7f50a602b700 1 --2- 172.21.15.12:0/2353351328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50980a4f20 0x7f50980bb720 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f50a0052cd0 tx=0x7f50a0075630 comp rx=0 tx=0).stop 2024-05-17T22:51:03.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.923+0000 7f50a602b700 1 -- 172.21.15.12:0/2353351328 shutdown_connections 2024-05-17T22:51:03.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.923+0000 7f50a602b700 1 --2- 172.21.15.12:0/2353351328 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f508c0b2a80 0x7f508c0b4f90 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:03.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.923+0000 7f50a602b700 1 --2- 172.21.15.12:0/2353351328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f50980ac430 0x7f50980b68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:03.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.923+0000 7f50a602b700 1 --2- 172.21.15.12:0/2353351328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50980a4f20 0x7f50980bb720 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:03.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.923+0000 7f50a602b700 1 --2- 172.21.15.12:0/2353351328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50980abae0 0x7f50980bbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:03.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.923+0000 7f50a602b700 1 -- 172.21.15.12:0/2353351328 >> 172.21.15.12:0/2353351328 conn(0x7f509801a9f0 msgr2=0x7f50980a96b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:51:03.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.923+0000 7f50a602b700 1 -- 172.21.15.12:0/2353351328 shutdown_connections 2024-05-17T22:51:03.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:03.923+0000 7f50a602b700 1 -- 172.21.15.12:0/2353351328 wait complete. 2024-05-17T22:51:04.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:04 smithi012 ceph-mon[31632]: pgmap v793: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 4.9 MiB/s wr, 435 op/s 2024-05-17T22:51:04.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:04 smithi151 ceph-mon[36101]: pgmap v793: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 4.9 MiB/s wr, 435 op/s 2024-05-17T22:51:04.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:04 smithi096 ceph-mon[36065]: pgmap v793: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 4.9 MiB/s wr, 435 op/s 2024-05-17T22:51:04.950 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-17T22:51:05.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:05 smithi012 ceph-mon[31632]: scrub queued for path: / 2024-05-17T22:51:05.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:05 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:51:05.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:05 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:51:05.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:05 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:51:05.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:05 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:51:05.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:05 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:51:05.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:05 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:51:05.475 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:51:05.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:05 smithi151 ceph-mon[36101]: scrub queued for path: / 2024-05-17T22:51:05.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:05 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:51:05.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:05 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:51:05.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:05 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:51:05.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:05 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:51:05.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:05 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:51:05.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:05 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:51:05.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:05 smithi096 ceph-mon[36065]: scrub queued for path: / 2024-05-17T22:51:05.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:05 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:51:05.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:05 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:51:05.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:05 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:51:05.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:05 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:51:05.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:05 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:51:05.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:05 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:51:06.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:06 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:51:06.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:06 smithi012 ceph-mon[31632]: pgmap v794: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.4 MiB/s wr, 542 op/s 2024-05-17T22:51:06.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 705.0 s, 549.5 tps, lat 192.229 ms stddev 266.334 2024-05-17T22:51:06.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:06 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:51:06.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:06 smithi151 ceph-mon[36101]: pgmap v794: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.4 MiB/s wr, 542 op/s 2024-05-17T22:51:06.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:06 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:51:06.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:06 smithi096 ceph-mon[36065]: pgmap v794: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.4 MiB/s wr, 542 op/s 2024-05-17T22:51:07.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.612+0000 7f90dc64d700 1 -- 172.21.15.12:0/3297070960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90d410a830 msgr2=0x7f90d410ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:51:07.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.612+0000 7f90dc64d700 1 --2- 172.21.15.12:0/3297070960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90d410a830 0x7f90d410ac40 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f90d00098f0 tx=0x7f90d0009c00 comp rx=0 tx=0).stop 2024-05-17T22:51:07.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.617+0000 7f90dc64d700 1 -- 172.21.15.12:0/3297070960 shutdown_connections 2024-05-17T22:51:07.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.617+0000 7f90dc64d700 1 --2- 172.21.15.12:0/3297070960 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90d411d550 0x7f90d411f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:07.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.617+0000 7f90dc64d700 1 --2- 172.21.15.12:0/3297070960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90d410a830 0x7f90d410ac40 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:07.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.617+0000 7f90dc64d700 1 --2- 172.21.15.12:0/3297070960 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90d410b180 0x7f90d411d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:07.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.617+0000 7f90dc64d700 1 -- 172.21.15.12:0/3297070960 >> 172.21.15.12:0/3297070960 conn(0x7f90d406efc0 msgr2=0x7f90d406f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:51:07.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.625+0000 7f90dc64d700 1 -- 172.21.15.12:0/3297070960 shutdown_connections 2024-05-17T22:51:07.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.627+0000 7f90dc64d700 1 -- 172.21.15.12:0/3297070960 wait complete. 2024-05-17T22:51:07.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.628+0000 7f90dc64d700 1 Processor -- start 2024-05-17T22:51:07.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.628+0000 7f90dc64d700 1 -- start start 2024-05-17T22:51:07.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.629+0000 7f90dc64d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90d410a830 0x7f90d411cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:07.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.629+0000 7f90dc64d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90d410b180 0x7f90d4117db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:07.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.629+0000 7f90dc64d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90d411d550 0x7f90d41182f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:07.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.629+0000 7f90dc64d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f90d4113de0 con 0x7f90d410b180 2024-05-17T22:51:07.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.629+0000 7f90dc64d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f90d4113c60 con 0x7f90d411d550 2024-05-17T22:51:07.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.629+0000 7f90dc64d700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f90d4113f60 con 0x7f90d410a830 2024-05-17T22:51:07.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.629+0000 7f90da3e9700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90d410a830 0x7f90d411cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:07.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.629+0000 7f90da3e9700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90d410a830 0x7f90d411cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:34722/0 (socket says 172.21.15.12:34722) 2024-05-17T22:51:07.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.629+0000 7f90da3e9700 1 -- 172.21.15.12:0/2661715998 learned_addr learned my addr 172.21.15.12:0/2661715998 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:51:07.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.630+0000 7f90dabea700 1 --2- 172.21.15.12:0/2661715998 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90d411d550 0x7f90d41182f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:07.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.630+0000 7f90da3e9700 1 -- 172.21.15.12:0/2661715998 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90d411d550 msgr2=0x7f90d41182f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:51:07.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.630+0000 7f90da3e9700 1 --2- 172.21.15.12:0/2661715998 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90d411d550 0x7f90d41182f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:07.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.630+0000 7f90da3e9700 1 -- 172.21.15.12:0/2661715998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90d410b180 msgr2=0x7f90d4117db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-17T22:51:07.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.630+0000 7f90da3e9700 1 --2- 172.21.15.12:0/2661715998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90d410b180 0x7f90d4117db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:07.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.630+0000 7f90da3e9700 1 -- 172.21.15.12:0/2661715998 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d4118930 con 0x7f90d410a830 2024-05-17T22:51:07.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.634+0000 7f90da3e9700 1 --2- 172.21.15.12:0/2661715998 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90d410a830 0x7f90d411cc80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f90d0009810 tx=0x7f90d00050f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:51:07.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.635+0000 7f90cb7fe700 1 -- 172.21.15.12:0/2661715998 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d000b500 con 0x7f90d410a830 2024-05-17T22:51:07.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.635+0000 7f90cb7fe700 1 -- 172.21.15.12:0/2661715998 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90d000bb50 con 0x7f90d410a830 2024-05-17T22:51:07.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.635+0000 7f90dc64d700 1 -- 172.21.15.12:0/2661715998 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90d4118b60 con 0x7f90d410a830 2024-05-17T22:51:07.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.635+0000 7f90dc64d700 1 -- 172.21.15.12:0/2661715998 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90d41219c0 con 0x7f90d410a830 2024-05-17T22:51:07.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.636+0000 7f90cb7fe700 1 -- 172.21.15.12:0/2661715998 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d0017660 con 0x7f90d410a830 2024-05-17T22:51:07.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.646+0000 7f90eb028b80 1 Processor -- start 2024-05-17T22:51:07.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.648+0000 7f90cb7fe700 1 -- 172.21.15.12:0/2661715998 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f90d000b6a0 con 0x7f90d410a830 2024-05-17T22:51:07.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.648+0000 7f90eb028b80 1 -- start start 2024-05-17T22:51:07.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.649+0000 7f90eb028b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af9dfdb9f0 0x55af9e15f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:07.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.649+0000 7f90eb028b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af9e15faa0 0x55af9e159c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:07.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.649+0000 7f90eb028b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af9e15a170 0x55af9e15a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:07.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.649+0000 7f90eb028b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55af9e088290 con 0x55af9dfdb9f0 2024-05-17T22:51:07.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.649+0000 7f90eb028b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55af9e077fd0 con 0x55af9e15faa0 2024-05-17T22:51:07.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.649+0000 7f90eb028b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55af9e07be70 con 0x55af9e15a170 2024-05-17T22:51:07.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.649+0000 7f90dabea700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af9dfdb9f0 0x55af9e15f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:07.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.650+0000 7f90da3e9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af9e15faa0 0x55af9e159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:07.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.650+0000 7f90da3e9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af9e15faa0 0x55af9e159c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:48032/0 (socket says 172.21.15.12:48032) 2024-05-17T22:51:07.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.650+0000 7f90da3e9700 1 -- 172.21.15.12:0/1495000288 learned_addr learned my addr 172.21.15.12:0/1495000288 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:51:07.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.650+0000 7f90d9be8700 1 --2- 172.21.15.12:0/1495000288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af9e15a170 0x55af9e15a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:07.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.650+0000 7f90da3e9700 1 -- 172.21.15.12:0/1495000288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af9e15a170 msgr2=0x55af9e15a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:51:07.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.650+0000 7f90da3e9700 1 --2- 172.21.15.12:0/1495000288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af9e15a170 0x55af9e15a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:07.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.650+0000 7f90da3e9700 1 -- 172.21.15.12:0/1495000288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af9dfdb9f0 msgr2=0x55af9e15f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:51:07.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.650+0000 7f90da3e9700 1 --2- 172.21.15.12:0/1495000288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af9dfdb9f0 0x55af9e15f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:07.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.650+0000 7f90da3e9700 1 -- 172.21.15.12:0/1495000288 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af9e021590 con 0x55af9e15faa0 2024-05-17T22:51:07.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.652+0000 7f90cb7fe700 1 --2- 172.21.15.12:0/2661715998 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f90c00d4aa0 0x7f90c00d6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:07.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.652+0000 7f90cb7fe700 1 -- 172.21.15.12:0/2661715998 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f90d00e2fa0 con 0x7f90d410a830 2024-05-17T22:51:07.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.655+0000 7f90da3e9700 1 --2- 172.21.15.12:0/1495000288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af9e15faa0 0x55af9e159c30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f90d00e6fc0 tx=0x7f90d00eb0e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3de8c0147f92c63a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:51:07.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.655+0000 7f90dabea700 1 --2- 172.21.15.12:0/2661715998 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f90c00d4aa0 0x7f90c00d6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:07.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.656+0000 7f90c8ff9700 1 -- 172.21.15.12:0/1495000288 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d0026070 con 0x55af9e15faa0 2024-05-17T22:51:07.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.656+0000 7f90c8ff9700 1 -- 172.21.15.12:0/1495000288 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90d000b990 con 0x55af9e15faa0 2024-05-17T22:51:07.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.657+0000 7f90eb028b80 1 -- 172.21.15.12:0/1495000288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af9e15faa0 msgr2=0x55af9e159c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:51:07.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.657+0000 7f90eb028b80 1 --2- 172.21.15.12:0/1495000288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af9e15faa0 0x55af9e159c30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f90d00e6fc0 tx=0x7f90d00eb0e0 comp rx=0 tx=0).stop 2024-05-17T22:51:07.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.660+0000 7f90dabea700 1 --2- 172.21.15.12:0/2661715998 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f90c00d4aa0 0x7f90c00d6fb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x55af9e155ae0 tx=0x7f90cc009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:51:07.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.660+0000 7f90eb028b80 1 -- 172.21.15.12:0/1495000288 shutdown_connections 2024-05-17T22:51:07.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.660+0000 7f90eb028b80 1 --2- 172.21.15.12:0/1495000288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af9e15a170 0x55af9e15a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:07.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.660+0000 7f90eb028b80 1 --2- 172.21.15.12:0/1495000288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af9dfdb9f0 0x55af9e15f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:07.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.660+0000 7f90eb028b80 1 --2- 172.21.15.12:0/1495000288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af9e15faa0 0x55af9e159c30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:07.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.660+0000 7f90eb028b80 1 -- 172.21.15.12:0/1495000288 >> 172.21.15.12:0/1495000288 conn(0x55af9e01ff00 msgr2=0x55af9e154b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:51:07.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.663+0000 7f90eb028b80 1 -- 172.21.15.12:0/1495000288 shutdown_connections 2024-05-17T22:51:07.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.664+0000 7f90eb028b80 1 -- 172.21.15.12:0/1495000288 wait complete. 2024-05-17T22:51:07.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.664+0000 7f90eb028b80 1 -- start start 2024-05-17T22:51:07.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.664+0000 7f90eb028b80 1 Processor -- start 2024-05-17T22:51:07.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.668+0000 7f90eb028b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af9dfdb9f0 0x55af9e1630a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:07.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.668+0000 7f90eb028b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af9e15a170 0x55af9e1635e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:07.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.668+0000 7f90eb028b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af9e15faa0 0x55af9e163b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:07.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.668+0000 7f90eb028b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55af9df7f150 con 0x55af9dfdb9f0 2024-05-17T22:51:07.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.668+0000 7f90eb028b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55af9e018e80 con 0x55af9e15a170 2024-05-17T22:51:07.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.668+0000 7f90eb028b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55af9e088290 con 0x55af9e15faa0 2024-05-17T22:51:07.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.668+0000 7f90eb028b80 10 client.0 _ll_register_callbacks cb 0x55af9e012f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:51:07.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.670+0000 7f90da3e9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af9dfdb9f0 0x55af9e1630a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:07.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.670+0000 7f90da3e9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af9dfdb9f0 0x55af9e1630a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58270/0 (socket says 172.21.15.12:58270) 2024-05-17T22:51:07.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.670+0000 7f90da3e9700 1 -- 172.21.15.12:0/3736308551 learned_addr learned my addr 172.21.15.12:0/3736308551 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:51:07.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.672+0000 7f90dabea700 1 --2- 172.21.15.12:0/3736308551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af9e15faa0 0x55af9e163b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:07.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.672+0000 7f90d9be8700 1 --2- 172.21.15.12:0/3736308551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af9e15a170 0x55af9e1635e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:07.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.675+0000 7f90da3e9700 1 -- 172.21.15.12:0/3736308551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af9e15faa0 msgr2=0x55af9e163b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:51:07.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.675+0000 7f90da3e9700 1 --2- 172.21.15.12:0/3736308551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af9e15faa0 0x55af9e163b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:07.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.675+0000 7f90da3e9700 1 -- 172.21.15.12:0/3736308551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af9e15a170 msgr2=0x55af9e1635e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:51:07.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.675+0000 7f90da3e9700 1 --2- 172.21.15.12:0/3736308551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af9e15a170 0x55af9e1635e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:07.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.675+0000 7f90da3e9700 1 -- 172.21.15.12:0/3736308551 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af9dfc8480 con 0x55af9dfdb9f0 2024-05-17T22:51:07.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.675+0000 7f90dabea700 1 --2- 172.21.15.12:0/3736308551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af9e15faa0 0x55af9e163b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:51:07.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.678+0000 7f90da3e9700 1 --2- 172.21.15.12:0/3736308551 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af9dfdb9f0 0x55af9e1630a0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f90d00a74d0 tx=0x7f90d00a6720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f1ff1c05817357 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:51:07.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.680+0000 7f90bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:51:07.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.680+0000 7f90bdffb700 1 -- 172.21.15.12:0/3736308551 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d0026050 con 0x55af9dfdb9f0 2024-05-17T22:51:07.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.680+0000 7f90eb028b80 1 -- 172.21.15.12:0/3736308551 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55af9e08a1e0 con 0x55af9dfdb9f0 2024-05-17T22:51:07.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.680+0000 7f90bdffb700 1 -- 172.21.15.12:0/3736308551 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f90d00aa6d0 con 0x55af9dfdb9f0 2024-05-17T22:51:07.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.682+0000 7f90bdffb700 1 -- 172.21.15.12:0/3736308551 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f90d00a99d0 con 0x55af9dfdb9f0 2024-05-17T22:51:07.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.682+0000 7f90eb028b80 10 client.16338 fetch_fsmap learned FSMap version 39 2024-05-17T22:51:07.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.682+0000 7f90eb028b80 1 -- 172.21.15.12:0/3736308551 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55af9e170ac0 con 0x55af9dfdb9f0 2024-05-17T22:51:07.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.683+0000 7f90bdffb700 1 -- 172.21.15.12:0/3736308551 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55af9e08a1e0 con 0x55af9dfdb9f0 2024-05-17T22:51:07.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.683+0000 7f90bdffb700 1 -- 172.21.15.12:0/3736308551 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f90d00b2ba0 con 0x55af9dfdb9f0 2024-05-17T22:51:07.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.683+0000 7f90eb028b80 10 client.16338 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:51:07.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.683+0000 7f90eb028b80 10 client.16338 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:51:07.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.683+0000 7f90eb028b80 20 client.16338 populate_metadata read hostname 'smithi012' 2024-05-17T22:51:07.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.683+0000 7f90eb028b80 1 --2- 172.21.15.12:0/3736308551 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55af9e173060 0x55af9e1754f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:07.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.683+0000 7f90eb028b80 4 client.16338 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:51:07.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.683+0000 7f90eb028b80 1 -- 172.21.15.12:0/3736308551 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af9e0839c0 con 0x55af9e173060 2024-05-17T22:51:07.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.683+0000 7f90d9be8700 1 --2- 172.21.15.12:0/3736308551 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55af9e173060 0x55af9e1754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:07.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.690+0000 7f90bdffb700 10 client.16338 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:51:07.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.690+0000 7f90d9be8700 1 --2- 172.21.15.12:0/3736308551 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55af9e173060 0x55af9e1754f0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27c36a66fdf022dc server_cookie=c76518915ecea9ee in_seq=0 out_seq=0 2024-05-17T22:51:07.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.694+0000 7f90bdffb700 1 -- 172.21.15.12:0/3736308551 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55af9e0839c0 con 0x55af9e173060 2024-05-17T22:51:07.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.694+0000 7f90bdffb700 10 client.16338 handle_command_reply: tid=0 2024-05-17T22:51:07.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.694+0000 7f90eb028b80 1 client.16338 shutdown 2024-05-17T22:51:07.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.697+0000 7f90eb028b80 1 -- 172.21.15.12:0/3736308551 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af9dfdb9f0 msgr2=0x55af9e1630a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:51:07.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.697+0000 7f90eb028b80 1 --2- 172.21.15.12:0/3736308551 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af9dfdb9f0 0x55af9e1630a0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f90d00a74d0 tx=0x7f90d00a6720 comp rx=0 tx=0).stop 2024-05-17T22:51:07.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.697+0000 7f90eb028b80 1 -- 172.21.15.12:0/3736308551 shutdown_connections 2024-05-17T22:51:07.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.697+0000 7f90eb028b80 1 --2- 172.21.15.12:0/3736308551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af9e15faa0 0x55af9e163b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:07.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.697+0000 7f90eb028b80 1 --2- 172.21.15.12:0/3736308551 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55af9e173060 0x55af9e1754f0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:07.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.697+0000 7f90eb028b80 1 --2- 172.21.15.12:0/3736308551 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af9dfdb9f0 0x55af9e1630a0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:07.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.697+0000 7f90eb028b80 1 --2- 172.21.15.12:0/3736308551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af9e15a170 0x55af9e1635e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:07.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.697+0000 7f90eb028b80 1 -- 172.21.15.12:0/3736308551 >> 172.21.15.12:0/3736308551 conn(0x55af9e01ff00 msgr2=0x55af9e14eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:51:07.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.697+0000 7f90bdffb700 0 client.16338 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:51:07.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.697+0000 7f90eb028b80 1 -- 172.21.15.12:0/3736308551 shutdown_connections 2024-05-17T22:51:07.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.700+0000 7f90eb028b80 1 -- 172.21.15.12:0/3736308551 wait complete. 2024-05-17T22:51:07.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.700+0000 7f90eb028b80 20 client.16338 trim_cache size 0 max 16384 2024-05-17T22:51:07.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.734+0000 7f90eb028b80 1 Processor -- start 2024-05-17T22:51:07.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.734+0000 7f90eb028b80 1 -- start start 2024-05-17T22:51:07.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.734+0000 7f90eb028b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af9dfdb9f0 0x55af9e159c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:07.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.734+0000 7f90eb028b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af9e15a170 0x55af9e162040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:07.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.734+0000 7f90eb028b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af9e15faa0 0x55af9e165850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:07.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.735+0000 7f90eb028b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f90b800ece0 con 0x55af9dfdb9f0 2024-05-17T22:51:07.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.735+0000 7f90eb028b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f90b800cb30 con 0x55af9e15a170 2024-05-17T22:51:07.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.735+0000 7f90eb028b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f90b800e610 con 0x55af9e15faa0 2024-05-17T22:51:07.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.735+0000 7f90d9be8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af9e15a170 0x55af9e162040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:07.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.735+0000 7f90da3e9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af9dfdb9f0 0x55af9e159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:07.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.735+0000 7f90dabea700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af9e15faa0 0x55af9e165850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:07.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.735+0000 7f90da3e9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af9dfdb9f0 0x55af9e159c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58274/0 (socket says 172.21.15.12:58274) 2024-05-17T22:51:07.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.735+0000 7f90da3e9700 1 -- 172.21.15.12:0/925885157 learned_addr learned my addr 172.21.15.12:0/925885157 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:51:07.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.735+0000 7f90dabea700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af9e15faa0 0x55af9e165850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:34758/0 (socket says 172.21.15.12:34758) 2024-05-17T22:51:07.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.735+0000 7f90da3e9700 1 -- 172.21.15.12:0/925885157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af9e15faa0 msgr2=0x55af9e165850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:51:07.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.735+0000 7f90da3e9700 1 --2- 172.21.15.12:0/925885157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af9e15faa0 0x55af9e165850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:07.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.735+0000 7f90da3e9700 1 -- 172.21.15.12:0/925885157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af9e15a170 msgr2=0x55af9e162040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:51:07.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.735+0000 7f90da3e9700 1 --2- 172.21.15.12:0/925885157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af9e15a170 0x55af9e162040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:07.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.735+0000 7f90da3e9700 1 -- 172.21.15.12:0/925885157 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af9e165d90 con 0x55af9dfdb9f0 2024-05-17T22:51:07.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.735+0000 7f90da3e9700 1 --2- 172.21.15.12:0/925885157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af9dfdb9f0 0x55af9e159c30 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f90d00a9f60 tx=0x7f90d00e6960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a501e4322a5442e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:51:07.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.736+0000 7f90bffff700 1 -- 172.21.15.12:0/925885157 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d00a9bb0 con 0x55af9dfdb9f0 2024-05-17T22:51:07.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.736+0000 7f90bffff700 1 -- 172.21.15.12:0/925885157 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90d00e6730 con 0x55af9dfdb9f0 2024-05-17T22:51:07.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.736+0000 7f90bffff700 1 -- 172.21.15.12:0/925885157 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d00ae230 con 0x55af9dfdb9f0 2024-05-17T22:51:07.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.737+0000 7f90eb028b80 1 -- 172.21.15.12:0/925885157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af9dfdb9f0 msgr2=0x55af9e159c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:51:07.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.737+0000 7f90eb028b80 1 --2- 172.21.15.12:0/925885157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af9dfdb9f0 0x55af9e159c30 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f90d00a9f60 tx=0x7f90d00e6960 comp rx=0 tx=0).stop 2024-05-17T22:51:07.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.737+0000 7f90eb028b80 1 -- 172.21.15.12:0/925885157 shutdown_connections 2024-05-17T22:51:07.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.737+0000 7f90eb028b80 1 --2- 172.21.15.12:0/925885157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af9e15faa0 0x55af9e165850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:07.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.737+0000 7f90eb028b80 1 --2- 172.21.15.12:0/925885157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af9dfdb9f0 0x55af9e159c30 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:07.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.737+0000 7f90eb028b80 1 --2- 172.21.15.12:0/925885157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af9e15a170 0x55af9e162040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:07.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.737+0000 7f90eb028b80 1 -- 172.21.15.12:0/925885157 >> 172.21.15.12:0/925885157 conn(0x55af9e01ff00 msgr2=0x55af9e160b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:51:07.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.737+0000 7f90eb028b80 1 -- 172.21.15.12:0/925885157 shutdown_connections 2024-05-17T22:51:07.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.737+0000 7f90eb028b80 1 -- 172.21.15.12:0/925885157 wait complete. 2024-05-17T22:51:07.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.738+0000 7f90eb028b80 1 -- start start 2024-05-17T22:51:07.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.738+0000 7f90eb028b80 1 Processor -- start 2024-05-17T22:51:07.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.738+0000 7f90eb028b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af9dfdb9f0 0x55af9e16b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:07.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.738+0000 7f90eb028b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af9e15a170 0x55af9e16b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:07.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.739+0000 7f90eb028b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af9e15faa0 0x55af9e16e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:07.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.739+0000 7f90d9be8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af9e15a170 0x55af9e16b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:07.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.739+0000 7f90d9be8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af9e15a170 0x55af9e16b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58290/0 (socket says 172.21.15.12:58290) 2024-05-17T22:51:07.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.739+0000 7f90da3e9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af9dfdb9f0 0x55af9e16b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:07.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.739+0000 7f90da3e9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af9dfdb9f0 0x55af9e16b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:48056/0 (socket says 172.21.15.12:48056) 2024-05-17T22:51:07.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.739+0000 7f90d9be8700 1 -- 172.21.15.12:0/3371841447 learned_addr learned my addr 172.21.15.12:0/3371841447 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:51:07.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.739+0000 7f90dabea700 1 --2- 172.21.15.12:0/3371841447 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af9e15faa0 0x55af9e16e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:07.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.739+0000 7f90eb028b80 1 -- 172.21.15.12:0/3371841447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f90b800d870 con 0x55af9e15a170 2024-05-17T22:51:07.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.739+0000 7f90eb028b80 1 -- 172.21.15.12:0/3371841447 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f90b800d1a0 con 0x55af9dfdb9f0 2024-05-17T22:51:07.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.739+0000 7f90eb028b80 1 -- 172.21.15.12:0/3371841447 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f90b800df40 con 0x55af9e15faa0 2024-05-17T22:51:07.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.740+0000 7f90eb028b80 10 client.0 _ll_register_callbacks cb 0x55af9e012f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:51:07.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.740+0000 7f90da3e9700 1 -- 172.21.15.12:0/3371841447 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af9e15faa0 msgr2=0x55af9e16e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:51:07.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.740+0000 7f90da3e9700 1 --2- 172.21.15.12:0/3371841447 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af9e15faa0 0x55af9e16e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:07.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.740+0000 7f90da3e9700 1 -- 172.21.15.12:0/3371841447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af9e15a170 msgr2=0x55af9e16b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:51:07.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.740+0000 7f90da3e9700 1 --2- 172.21.15.12:0/3371841447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af9e15a170 0x55af9e16b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:07.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.740+0000 7f90da3e9700 1 -- 172.21.15.12:0/3371841447 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af9e16e850 con 0x55af9dfdb9f0 2024-05-17T22:51:07.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.740+0000 7f90eb028b80 1 -- 172.21.15.12:0/3371841447 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90d00e6730 con 0x55af9dfdb9f0 2024-05-17T22:51:07.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.740+0000 7f90da3e9700 1 --2- 172.21.15.12:0/3371841447 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af9dfdb9f0 0x55af9e16b120 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f90d00098c0 tx=0x7f90d0005f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=138db2cf2976344 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:51:07.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.740+0000 7f90bdffb700 10 client.26060 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:51:07.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.740+0000 7f90d9be8700 1 --2- 172.21.15.12:0/3371841447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af9e15a170 0x55af9e16b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:51:07.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.740+0000 7f90bdffb700 1 -- 172.21.15.12:0/3371841447 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d0026030 con 0x55af9dfdb9f0 2024-05-17T22:51:07.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.740+0000 7f90bdffb700 1 -- 172.21.15.12:0/3371841447 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f90d00e0b70 con 0x55af9dfdb9f0 2024-05-17T22:51:07.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.740+0000 7f90bdffb700 1 -- 172.21.15.12:0/3371841447 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f90d00a9bb0 con 0x55af9dfdb9f0 2024-05-17T22:51:07.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.740+0000 7f90eb028b80 10 client.26060 fetch_fsmap learned FSMap version 39 2024-05-17T22:51:07.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.740+0000 7f90eb028b80 1 -- 172.21.15.12:0/3371841447 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({fsmap=39}) -- 0x55af9e16e850 con 0x55af9dfdb9f0 2024-05-17T22:51:07.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.741+0000 7f90bdffb700 1 -- 172.21.15.12:0/3371841447 <== mon.2 v2:172.21.15.96:3300/0 4 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f90d00e6030 con 0x55af9dfdb9f0 2024-05-17T22:51:07.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.741+0000 7f90eb028b80 10 client.26060 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:51:07.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.741+0000 7f90eb028b80 10 client.26060 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:51:07.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.741+0000 7f90eb028b80 20 client.26060 populate_metadata read hostname 'smithi012' 2024-05-17T22:51:07.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.741+0000 7f90eb028b80 1 --2- 172.21.15.12:0/3371841447 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55af9e1dd190 0x55af9e1df5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:07.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.741+0000 7f90eb028b80 4 client.26060 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:51:07.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.741+0000 7f90eb028b80 1 -- 172.21.15.12:0/3371841447 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55af9df54360 con 0x55af9e1dd190 2024-05-17T22:51:07.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.742+0000 7f90d9be8700 1 --2- 172.21.15.12:0/3371841447 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55af9e1dd190 0x55af9e1df5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:07.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.742+0000 7f90bdffb700 10 client.26060 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:51:07.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.742+0000 7f90d9be8700 1 --2- 172.21.15.12:0/3371841447 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55af9e1dd190 0x55af9e1df5b0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d52c3eb5c2e4adc3 server_cookie=9b1a915dee1389d0 in_seq=0 out_seq=0 2024-05-17T22:51:07.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.743+0000 7f90bdffb700 1 -- 172.21.15.12:0/3371841447 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55af9df54360 con 0x55af9e1dd190 2024-05-17T22:51:07.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.743+0000 7f90bdffb700 10 client.26060 handle_command_reply: tid=0 2024-05-17T22:51:07.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.743+0000 7f90eb028b80 1 client.26060 shutdown 2024-05-17T22:51:07.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.743+0000 7f90eb028b80 1 -- 172.21.15.12:0/3371841447 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af9dfdb9f0 msgr2=0x55af9e16b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:51:07.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.743+0000 7f90eb028b80 1 --2- 172.21.15.12:0/3371841447 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af9dfdb9f0 0x55af9e16b120 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f90d00098c0 tx=0x7f90d0005f00 comp rx=0 tx=0).stop 2024-05-17T22:51:07.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.743+0000 7f90eb028b80 1 -- 172.21.15.12:0/3371841447 shutdown_connections 2024-05-17T22:51:07.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.743+0000 7f90eb028b80 1 --2- 172.21.15.12:0/3371841447 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af9e15faa0 0x55af9e16e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:07.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.743+0000 7f90eb028b80 1 --2- 172.21.15.12:0/3371841447 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af9dfdb9f0 0x55af9e16b120 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:07.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.743+0000 7f90eb028b80 1 --2- 172.21.15.12:0/3371841447 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55af9e1dd190 0x55af9e1df5b0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:07.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.743+0000 7f90eb028b80 1 --2- 172.21.15.12:0/3371841447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af9e15a170 0x55af9e16b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:07.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.743+0000 7f90bdffb700 0 client.26060 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:51:07.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.743+0000 7f90eb028b80 1 -- 172.21.15.12:0/3371841447 >> 172.21.15.12:0/3371841447 conn(0x55af9e01ff00 msgr2=0x55af9e160b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:51:07.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.743+0000 7f90eb028b80 1 -- 172.21.15.12:0/3371841447 shutdown_connections 2024-05-17T22:51:07.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.743+0000 7f90eb028b80 1 -- 172.21.15.12:0/3371841447 wait complete. 2024-05-17T22:51:07.748 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:51:07.749 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-17T22:51:07.749 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-17T22:51:07.749 INFO:teuthology.orchestra.run.smithi012.stdout: "11c72a3d-f39b-46bc-a156-10a95d956679": { 2024-05-17T22:51:07.749 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-17T22:51:07.749 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "11c72a3d-f39b-46bc-a156-10a95d956679", 2024-05-17T22:51:07.749 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-17T22:51:07.749 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:51:07.749 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:51:07.749 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:51:07.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.744+0000 7f90eb028b80 20 client.26060 trim_cache size 0 max 16384 2024-05-17T22:51:07.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.744+0000 7f90dc64d700 1 -- 172.21.15.12:0/2661715998 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f90c00d4aa0 msgr2=0x7f90c00d6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:51:07.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.744+0000 7f90dc64d700 1 --2- 172.21.15.12:0/2661715998 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f90c00d4aa0 0x7f90c00d6fb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x55af9e155ae0 tx=0x7f90cc009040 comp rx=0 tx=0).stop 2024-05-17T22:51:07.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.744+0000 7f90dc64d700 1 -- 172.21.15.12:0/2661715998 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90d410a830 msgr2=0x7f90d411cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:51:07.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.744+0000 7f90dc64d700 1 --2- 172.21.15.12:0/2661715998 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90d410a830 0x7f90d411cc80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f90d0009810 tx=0x7f90d00050f0 comp rx=0 tx=0).stop 2024-05-17T22:51:07.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.744+0000 7f90dc64d700 1 -- 172.21.15.12:0/2661715998 shutdown_connections 2024-05-17T22:51:07.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.744+0000 7f90dc64d700 1 --2- 172.21.15.12:0/2661715998 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f90c00d4aa0 0x7f90c00d6fb0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:07.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.744+0000 7f90dc64d700 1 --2- 172.21.15.12:0/2661715998 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90d411d550 0x7f90d41182f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:07.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.744+0000 7f90dc64d700 1 --2- 172.21.15.12:0/2661715998 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90d410a830 0x7f90d411cc80 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:07.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.744+0000 7f90dc64d700 1 --2- 172.21.15.12:0/2661715998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90d410b180 0x7f90d4117db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:07.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.744+0000 7f90dc64d700 1 -- 172.21.15.12:0/2661715998 >> 172.21.15.12:0/2661715998 conn(0x7f90d406efc0 msgr2=0x7f90d411db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:51:07.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.745+0000 7f90dc64d700 1 -- 172.21.15.12:0/2661715998 shutdown_connections 2024-05-17T22:51:07.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:07.745+0000 7f90dc64d700 1 -- 172.21.15.12:0/2661715998 wait complete. 2024-05-17T22:51:08.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:08 smithi012 ceph-mon[31632]: pgmap v795: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.3 MiB/s wr, 449 op/s 2024-05-17T22:51:08.350 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:08 smithi096 ceph-mon[36065]: pgmap v795: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.3 MiB/s wr, 449 op/s 2024-05-17T22:51:08.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:08 smithi151 ceph-mon[36101]: pgmap v795: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.3 MiB/s wr, 449 op/s 2024-05-17T22:51:08.745 INFO:tasks.cephfs.filesystem:scrub status for tag:11c72a3d-f39b-46bc-a156-10a95d956679 - {'path': '/', 'tag': '11c72a3d-f39b-46bc-a156-10a95d956679', 'options': 'recursive,force'} 2024-05-17T22:51:10.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:10 smithi012 ceph-mon[31632]: pgmap v796: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 5.3 MiB/s wr, 478 op/s 2024-05-17T22:51:10.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:10 smithi151 ceph-mon[36101]: pgmap v796: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 5.3 MiB/s wr, 478 op/s 2024-05-17T22:51:10.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:10 smithi096 ceph-mon[36065]: pgmap v796: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 5.3 MiB/s wr, 478 op/s 2024-05-17T22:51:11.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 710.0 s, 502.2 tps, lat 192.722 ms stddev 213.886 2024-05-17T22:51:12.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:12 smithi151 ceph-mon[36101]: pgmap v797: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.4 MiB/s wr, 480 op/s 2024-05-17T22:51:12.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:12 smithi096 ceph-mon[36065]: pgmap v797: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.4 MiB/s wr, 480 op/s 2024-05-17T22:51:12.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:12 smithi012 ceph-mon[31632]: pgmap v797: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.4 MiB/s wr, 480 op/s 2024-05-17T22:51:14.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:14 smithi151 ceph-mon[36101]: pgmap v798: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.0 MiB/s wr, 470 op/s 2024-05-17T22:51:14.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:14 smithi151 ceph-mon[36101]: scrub summary: idle 2024-05-17T22:51:14.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:14 smithi151 ceph-mon[36101]: scrub summary: idle 2024-05-17T22:51:14.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:14 smithi151 ceph-mon[36101]: scrub summary: idle 2024-05-17T22:51:14.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:14 smithi096 ceph-mon[36065]: pgmap v798: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.0 MiB/s wr, 470 op/s 2024-05-17T22:51:14.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:14 smithi096 ceph-mon[36065]: scrub summary: idle 2024-05-17T22:51:14.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:14 smithi096 ceph-mon[36065]: scrub summary: idle 2024-05-17T22:51:14.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:14 smithi096 ceph-mon[36065]: scrub summary: idle 2024-05-17T22:51:14.593 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:14 smithi012 ceph-mon[31632]: pgmap v798: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.0 MiB/s wr, 470 op/s 2024-05-17T22:51:14.593 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:14 smithi012 ceph-mon[31632]: scrub summary: idle 2024-05-17T22:51:14.593 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:14 smithi012 ceph-mon[31632]: scrub summary: idle 2024-05-17T22:51:14.593 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:14 smithi012 ceph-mon[31632]: scrub summary: idle 2024-05-17T22:51:15.426 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:15 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:51:15.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:15 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:51:15.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:15 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:51:16.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 715.0 s, 471.4 tps, lat 210.199 ms stddev 233.705 2024-05-17T22:51:16.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:16 smithi151 ceph-mon[36101]: pgmap v799: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.4 MiB/s wr, 563 op/s 2024-05-17T22:51:16.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:16 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:51:16.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:16 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:51:16.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:16 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:51:16.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:16 smithi096 ceph-mon[36065]: pgmap v799: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.4 MiB/s wr, 563 op/s 2024-05-17T22:51:16.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:16 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:51:16.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:16 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:51:16.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:16 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:51:16.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:16 smithi012 ceph-mon[31632]: pgmap v799: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.4 MiB/s wr, 563 op/s 2024-05-17T22:51:16.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:16 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:51:16.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:16 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:51:16.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:16 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:51:18.495 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:18 smithi151 ceph-mon[36101]: pgmap v800: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.2 MiB/s wr, 456 op/s 2024-05-17T22:51:18.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:18 smithi096 ceph-mon[36065]: pgmap v800: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.2 MiB/s wr, 456 op/s 2024-05-17T22:51:18.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:18 smithi012 ceph-mon[31632]: pgmap v800: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.2 MiB/s wr, 456 op/s 2024-05-17T22:51:20.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:20 smithi151 ceph-mon[36101]: pgmap v801: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.4 MiB/s wr, 509 op/s 2024-05-17T22:51:20.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:20 smithi096 ceph-mon[36065]: pgmap v801: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.4 MiB/s wr, 509 op/s 2024-05-17T22:51:20.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:20 smithi012 ceph-mon[31632]: pgmap v801: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.4 MiB/s wr, 509 op/s 2024-05-17T22:51:21.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 720.0 s, 474.2 tps, lat 213.189 ms stddev 255.585 2024-05-17T22:51:22.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:22 smithi151 ceph-mon[36101]: pgmap v802: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 MiB/s wr, 486 op/s 2024-05-17T22:51:22.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:22 smithi096 ceph-mon[36065]: pgmap v802: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 MiB/s wr, 486 op/s 2024-05-17T22:51:22.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:22 smithi012 ceph-mon[31632]: pgmap v802: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 MiB/s wr, 486 op/s 2024-05-17T22:51:24.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:24 smithi151 ceph-mon[36101]: pgmap v803: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.0 MiB/s wr, 463 op/s 2024-05-17T22:51:24.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:24 smithi096 ceph-mon[36065]: pgmap v803: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.0 MiB/s wr, 463 op/s 2024-05-17T22:51:24.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:24 smithi012 ceph-mon[31632]: pgmap v803: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.0 MiB/s wr, 463 op/s 2024-05-17T22:51:26.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 725.0 s, 516.0 tps, lat 195.489 ms stddev 241.638 2024-05-17T22:51:26.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:26 smithi151 ceph-mon[36101]: pgmap v804: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 MiB/s wr, 552 op/s 2024-05-17T22:51:26.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:26 smithi096 ceph-mon[36065]: pgmap v804: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 MiB/s wr, 552 op/s 2024-05-17T22:51:26.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:26 smithi012 ceph-mon[31632]: pgmap v804: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 MiB/s wr, 552 op/s 2024-05-17T22:51:28.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:28 smithi151 ceph-mon[36101]: pgmap v805: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.8 MiB/s wr, 430 op/s 2024-05-17T22:51:28.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:28 smithi096 ceph-mon[36065]: pgmap v805: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.8 MiB/s wr, 430 op/s 2024-05-17T22:51:28.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:28 smithi012 ceph-mon[31632]: pgmap v805: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.8 MiB/s wr, 430 op/s 2024-05-17T22:51:30.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:30 smithi151 ceph-mon[36101]: pgmap v806: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.3 MiB/s wr, 498 op/s 2024-05-17T22:51:30.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:30 smithi096 ceph-mon[36065]: pgmap v806: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.3 MiB/s wr, 498 op/s 2024-05-17T22:51:30.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:30 smithi012 ceph-mon[31632]: pgmap v806: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.3 MiB/s wr, 498 op/s 2024-05-17T22:51:31.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 730.0 s, 485.2 tps, lat 204.940 ms stddev 246.309 2024-05-17T22:51:32.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:32 smithi151 ceph-mon[36101]: pgmap v807: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.5 MiB/s wr, 485 op/s 2024-05-17T22:51:32.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:32 smithi096 ceph-mon[36065]: pgmap v807: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.5 MiB/s wr, 485 op/s 2024-05-17T22:51:32.593 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:32 smithi012 ceph-mon[31632]: pgmap v807: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.5 MiB/s wr, 485 op/s 2024-05-17T22:51:34.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:34 smithi151 ceph-mon[36101]: pgmap v808: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.1 MiB/s wr, 472 op/s 2024-05-17T22:51:34.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:34 smithi096 ceph-mon[36065]: pgmap v808: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.1 MiB/s wr, 472 op/s 2024-05-17T22:51:34.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:34 smithi012 ceph-mon[31632]: pgmap v808: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.1 MiB/s wr, 472 op/s 2024-05-17T22:51:36.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 735.0 s, 402.0 tps, lat 221.995 ms stddev 266.091 2024-05-17T22:51:36.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:36 smithi151 ceph-mon[36101]: pgmap v809: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.1 MiB/s wr, 543 op/s 2024-05-17T22:51:36.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:36 smithi096 ceph-mon[36065]: pgmap v809: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.1 MiB/s wr, 543 op/s 2024-05-17T22:51:36.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:36 smithi012 ceph-mon[31632]: pgmap v809: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.1 MiB/s wr, 543 op/s 2024-05-17T22:51:38.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:38 smithi151 ceph-mon[36101]: pgmap v810: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.9 MiB/s wr, 526 op/s 2024-05-17T22:51:38.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:38 smithi096 ceph-mon[36065]: pgmap v810: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.9 MiB/s wr, 526 op/s 2024-05-17T22:51:38.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:38 smithi012 ceph-mon[31632]: pgmap v810: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.9 MiB/s wr, 526 op/s 2024-05-17T22:51:38.746 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-17T22:51:39.380 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:51:40.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:40 smithi151 ceph-mon[36101]: pgmap v811: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 1.12k op/s 2024-05-17T22:51:40.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:40 smithi096 ceph-mon[36065]: pgmap v811: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 1.12k op/s 2024-05-17T22:51:40.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:40 smithi012 ceph-mon[31632]: pgmap v811: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 1.12k op/s 2024-05-17T22:51:41.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 740.0 s, 239.2 tps, lat 450.981 ms stddev 919.396 2024-05-17T22:51:42.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:42 smithi151 ceph-mon[36101]: pgmap v812: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 19 MiB/s wr, 1.40k op/s 2024-05-17T22:51:42.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:42 smithi096 ceph-mon[36065]: pgmap v812: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 19 MiB/s wr, 1.40k op/s 2024-05-17T22:51:42.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:42 smithi012 ceph-mon[31632]: pgmap v812: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 19 MiB/s wr, 1.40k op/s 2024-05-17T22:51:44.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:44 smithi012 ceph-mon[31632]: pgmap v813: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 19 MiB/s wr, 1.38k op/s 2024-05-17T22:51:44.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:44 smithi151 ceph-mon[36101]: pgmap v813: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 19 MiB/s wr, 1.38k op/s 2024-05-17T22:51:44.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:44 smithi096 ceph-mon[36065]: pgmap v813: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 19 MiB/s wr, 1.38k op/s 2024-05-17T22:51:46.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:46 smithi012 ceph-mon[31632]: pgmap v814: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 21 MiB/s wr, 1.53k op/s 2024-05-17T22:51:46.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 745.0 s, 494.2 tps, lat 207.311 ms stddev 246.960 2024-05-17T22:51:46.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:46 smithi151 ceph-mon[36101]: pgmap v814: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 21 MiB/s wr, 1.53k op/s 2024-05-17T22:51:46.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:46 smithi096 ceph-mon[36065]: pgmap v814: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 21 MiB/s wr, 1.53k op/s 2024-05-17T22:51:47.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.350+0000 7fb8ed4a3700 1 -- 172.21.15.12:0/328396789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8e810b180 msgr2=0x7fb8e811d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:51:47.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.350+0000 7fb8ed4a3700 1 --2- 172.21.15.12:0/328396789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8e810b180 0x7fb8e811d010 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb8e001c0d0 tx=0x7fb8e001c3e0 comp rx=0 tx=0).stop 2024-05-17T22:51:47.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.354+0000 7fb8ed4a3700 1 -- 172.21.15.12:0/328396789 shutdown_connections 2024-05-17T22:51:47.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.354+0000 7fb8ed4a3700 1 --2- 172.21.15.12:0/328396789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8e811d550 0x7fb8e811f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:47.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.354+0000 7fb8ed4a3700 1 --2- 172.21.15.12:0/328396789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb8e810a830 0x7fb8e810ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:47.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.354+0000 7fb8ed4a3700 1 --2- 172.21.15.12:0/328396789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8e810b180 0x7fb8e811d010 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:47.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.354+0000 7fb8ed4a3700 1 -- 172.21.15.12:0/328396789 >> 172.21.15.12:0/328396789 conn(0x7fb8e806efc0 msgr2=0x7fb8e806f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:51:47.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.355+0000 7fb8ed4a3700 1 -- 172.21.15.12:0/328396789 shutdown_connections 2024-05-17T22:51:47.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.357+0000 7fb8ed4a3700 1 -- 172.21.15.12:0/328396789 wait complete. 2024-05-17T22:51:47.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.358+0000 7fb8ed4a3700 1 Processor -- start 2024-05-17T22:51:47.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.358+0000 7fb8ed4a3700 1 -- start start 2024-05-17T22:51:47.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.358+0000 7fb8ed4a3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8e810a830 0x7fb8e811ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:47.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.358+0000 7fb8ed4a3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8e810b180 0x7fb8e8118f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:47.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.358+0000 7fb8ed4a3700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb8e811d550 0x7fb8e81194d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:47.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.358+0000 7fb8ed4a3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e8114fc0 con 0x7fb8e810b180 2024-05-17T22:51:47.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.358+0000 7fb8ed4a3700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e8114e40 con 0x7fb8e810a830 2024-05-17T22:51:47.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.358+0000 7fb8ed4a3700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e8115140 con 0x7fb8e811d550 2024-05-17T22:51:47.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.361+0000 7fb8e67fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8e810b180 0x7fb8e8118f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:47.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.361+0000 7fb8e67fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8e810b180 0x7fb8e8118f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48770/0 (socket says 172.21.15.12:48770) 2024-05-17T22:51:47.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.361+0000 7fb8e67fc700 1 -- 172.21.15.12:0/3875250254 learned_addr learned my addr 172.21.15.12:0/3875250254 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:51:47.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.361+0000 7fb8e67fc700 1 -- 172.21.15.12:0/3875250254 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb8e811d550 msgr2=0x7fb8e81194d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-17T22:51:47.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.361+0000 7fb8e67fc700 1 --2- 172.21.15.12:0/3875250254 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb8e811d550 0x7fb8e81194d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:47.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.361+0000 7fb8e67fc700 1 -- 172.21.15.12:0/3875250254 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8e810a830 msgr2=0x7fb8e811ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:51:47.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.361+0000 7fb8e67fc700 1 --2- 172.21.15.12:0/3875250254 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8e810a830 0x7fb8e811ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:47.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.361+0000 7fb8e67fc700 1 -- 172.21.15.12:0/3875250254 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e8119cb0 con 0x7fb8e810b180 2024-05-17T22:51:47.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.361+0000 7fb8e67fc700 1 --2- 172.21.15.12:0/3875250254 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8e810b180 0x7fb8e8118f90 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0003c30 tx=0x7fb8e0003d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:51:47.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.363+0000 7fb8cffff700 1 -- 172.21.15.12:0/3875250254 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0018b90 con 0x7fb8e810b180 2024-05-17T22:51:47.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.363+0000 7fb8cffff700 1 -- 172.21.15.12:0/3875250254 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8e0018d30 con 0x7fb8e810b180 2024-05-17T22:51:47.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.363+0000 7fb8cffff700 1 -- 172.21.15.12:0/3875250254 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0007820 con 0x7fb8e810b180 2024-05-17T22:51:47.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.364+0000 7fb8ed4a3700 1 -- 172.21.15.12:0/3875250254 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8e81cf260 con 0x7fb8e810b180 2024-05-17T22:51:47.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.365+0000 7fb8ed4a3700 1 -- 172.21.15.12:0/3875250254 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8e81cf6f0 con 0x7fb8e810b180 2024-05-17T22:51:47.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.371+0000 7fb8cffff700 1 -- 172.21.15.12:0/3875250254 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb8e00056b0 con 0x7fb8e810b180 2024-05-17T22:51:47.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.372+0000 7fb8fbe7eb80 1 Processor -- start 2024-05-17T22:51:47.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.372+0000 7fb8fbe7eb80 1 -- start start 2024-05-17T22:51:47.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.373+0000 7fb8fbe7eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56191bd5d9f0 0x56191bee0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:47.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.373+0000 7fb8cffff700 1 --2- 172.21.15.12:0/3875250254 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb8d00b2a50 0x7fb8d00b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:47.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.373+0000 7fb8fbe7eb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191bee1400 0x56191bedb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:47.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.373+0000 7fb8fbe7eb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191bedbb00 0x56191bedbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:47.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.374+0000 7fb8cffff700 1 -- 172.21.15.12:0/3875250254 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fb8e0020070 con 0x7fb8e810b180 2024-05-17T22:51:47.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.374+0000 7fb8e67fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191bee1400 0x56191bedb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:47.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.374+0000 7fb8e67fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191bee1400 0x56191bedb5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:48338/0 (socket says 172.21.15.12:48338) 2024-05-17T22:51:47.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.374+0000 7fb8e67fc700 1 -- 172.21.15.12:0/119984555 learned_addr learned my addr 172.21.15.12:0/119984555 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:51:47.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.374+0000 7fb8fbe7eb80 1 -- 172.21.15.12:0/119984555 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56191be0a0d0 con 0x56191bd5d9f0 2024-05-17T22:51:47.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.374+0000 7fb8fbe7eb80 1 -- 172.21.15.12:0/119984555 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56191bdf9c70 con 0x56191bee1400 2024-05-17T22:51:47.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.374+0000 7fb8fbe7eb80 1 -- 172.21.15.12:0/119984555 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56191bdfdcb0 con 0x56191bedbb00 2024-05-17T22:51:47.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.374+0000 7fb8e77fe700 1 --2- 172.21.15.12:0/119984555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191bedbb00 0x56191bedbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:47.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.374+0000 7fb8e77fe700 1 --2- 172.21.15.12:0/119984555 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56191bd5d9f0 0x56191bee0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:47.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.374+0000 7fb8e77fe700 1 -- 172.21.15.12:0/119984555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191bee1400 msgr2=0x56191bedb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:51:47.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.374+0000 7fb8e77fe700 1 --2- 172.21.15.12:0/119984555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191bee1400 0x56191bedb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:47.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.374+0000 7fb8e77fe700 1 -- 172.21.15.12:0/119984555 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56191bd5d9f0 msgr2=0x56191bee0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:51:47.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.374+0000 7fb8e77fe700 1 --2- 172.21.15.12:0/119984555 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56191bd5d9f0 0x56191bee0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:47.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.374+0000 7fb8e77fe700 1 -- 172.21.15.12:0/119984555 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56191bd935c0 con 0x56191bedbb00 2024-05-17T22:51:47.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.375+0000 7fb8e77fe700 1 --2- 172.21.15.12:0/119984555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191bedbb00 0x56191bedbf10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8009ed0 tx=0x7fb8d80096a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a092282334f2ac5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:51:47.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.377+0000 7fb8e67fc700 1 --2- 172.21.15.12:0/119984555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191bee1400 0x56191bedb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:51:47.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.378+0000 7fb8cd7fa700 1 -- 172.21.15.12:0/119984555 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8012b90 con 0x56191bedbb00 2024-05-17T22:51:47.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.378+0000 7fb8cd7fa700 1 -- 172.21.15.12:0/119984555 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8d80044e0 con 0x56191bedbb00 2024-05-17T22:51:47.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.379+0000 7fb8fbe7eb80 1 -- 172.21.15.12:0/119984555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191bedbb00 msgr2=0x56191bedbf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:51:47.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.379+0000 7fb8fbe7eb80 1 --2- 172.21.15.12:0/119984555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191bedbb00 0x56191bedbf10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8009ed0 tx=0x7fb8d80096a0 comp rx=0 tx=0).stop 2024-05-17T22:51:47.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.380+0000 7fb8e6ffd700 1 --2- 172.21.15.12:0/3875250254 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb8d00b2a50 0x7fb8d00b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:47.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.383+0000 7fb8fbe7eb80 1 -- 172.21.15.12:0/119984555 shutdown_connections 2024-05-17T22:51:47.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.383+0000 7fb8fbe7eb80 1 --2- 172.21.15.12:0/119984555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191bedbb00 0x56191bedbf10 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:47.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.383+0000 7fb8fbe7eb80 1 --2- 172.21.15.12:0/119984555 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56191bd5d9f0 0x56191bee0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:47.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.383+0000 7fb8fbe7eb80 1 --2- 172.21.15.12:0/119984555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191bee1400 0x56191bedb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:47.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.383+0000 7fb8fbe7eb80 1 -- 172.21.15.12:0/119984555 >> 172.21.15.12:0/119984555 conn(0x56191bdbd7d0 msgr2=0x56191bed61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:51:47.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.384+0000 7fb8e6ffd700 1 --2- 172.21.15.12:0/3875250254 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb8d00b2a50 0x7fb8d00b4f60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc005950 tx=0x7fb8dc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:51:47.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.384+0000 7fb8fbe7eb80 1 -- 172.21.15.12:0/119984555 shutdown_connections 2024-05-17T22:51:47.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.387+0000 7fb8fbe7eb80 1 -- 172.21.15.12:0/119984555 wait complete. 2024-05-17T22:51:47.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.387+0000 7fb8fbe7eb80 1 -- start start 2024-05-17T22:51:47.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.388+0000 7fb8fbe7eb80 1 Processor -- start 2024-05-17T22:51:47.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.388+0000 7fb8fbe7eb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191bd5d9f0 0x56191bee4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:47.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.389+0000 7fb8fbe7eb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191bedbb00 0x56191bee4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:47.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.389+0000 7fb8e6ffd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191bd5d9f0 0x56191bee4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:47.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.389+0000 7fb8e6ffd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191bd5d9f0 0x56191bee4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:60240/0 (socket says 172.21.15.12:60240) 2024-05-17T22:51:47.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.389+0000 7fb8e67fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191bedbb00 0x56191bee4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:47.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.389+0000 7fb8e67fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191bedbb00 0x56191bee4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:48346/0 (socket says 172.21.15.12:48346) 2024-05-17T22:51:47.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.390+0000 7fb8e6ffd700 1 -- 172.21.15.12:0/3925583029 learned_addr learned my addr 172.21.15.12:0/3925583029 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:51:47.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.390+0000 7fb8fbe7eb80 1 --2- 172.21.15.12:0/3925583029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56191bee1400 0x56191bee5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:47.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.390+0000 7fb8fbe7eb80 1 -- 172.21.15.12:0/3925583029 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56191bd01150 con 0x56191bee1400 2024-05-17T22:51:47.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.390+0000 7fb8e77fe700 1 --2- 172.21.15.12:0/3925583029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56191bee1400 0x56191bee5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:47.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.390+0000 7fb8fbe7eb80 1 -- 172.21.15.12:0/3925583029 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56191bd9b940 con 0x56191bedbb00 2024-05-17T22:51:47.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.391+0000 7fb8fbe7eb80 1 -- 172.21.15.12:0/3925583029 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56191bee7e20 con 0x56191bd5d9f0 2024-05-17T22:51:47.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.391+0000 7fb8fbe7eb80 10 client.0 _ll_register_callbacks cb 0x56191bd90b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:51:47.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.392+0000 7fb8e67fc700 1 -- 172.21.15.12:0/3925583029 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191bd5d9f0 msgr2=0x56191bee4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:51:47.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.392+0000 7fb8e67fc700 1 --2- 172.21.15.12:0/3925583029 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191bd5d9f0 0x56191bee4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:47.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.392+0000 7fb8e67fc700 1 -- 172.21.15.12:0/3925583029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56191bee1400 msgr2=0x56191bee5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:51:47.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.392+0000 7fb8e67fc700 1 --2- 172.21.15.12:0/3925583029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56191bee1400 0x56191bee5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:47.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.392+0000 7fb8e67fc700 1 -- 172.21.15.12:0/3925583029 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56191bd4a480 con 0x56191bedbb00 2024-05-17T22:51:47.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.399+0000 7fb8e67fc700 1 --2- 172.21.15.12:0/3925583029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191bedbb00 0x56191bee4fe0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb8e000a040 tx=0x7fb8e001ac70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba96ac1709a0503b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:51:47.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.399+0000 7fb8caff5700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:51:47.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.402+0000 7fb8caff5700 1 -- 172.21.15.12:0/3925583029 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e00aa450 con 0x56191bedbb00 2024-05-17T22:51:47.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.402+0000 7fb8caff5700 1 -- 172.21.15.12:0/3925583029 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fb8e00e3160 con 0x56191bedbb00 2024-05-17T22:51:47.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.402+0000 7fb8fbe7eb80 1 -- 172.21.15.12:0/3925583029 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8d80044e0 con 0x56191bedbb00 2024-05-17T22:51:47.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.402+0000 7fb8caff5700 1 -- 172.21.15.12:0/3925583029 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb8e00a9df0 con 0x56191bedbb00 2024-05-17T22:51:47.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.402+0000 7fb8fbe7eb80 10 client.26066 fetch_fsmap learned FSMap version 39 2024-05-17T22:51:47.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.402+0000 7fb8fbe7eb80 1 -- 172.21.15.12:0/3925583029 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56191bef2840 con 0x56191bedbb00 2024-05-17T22:51:47.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.405+0000 7fb8caff5700 1 -- 172.21.15.12:0/3925583029 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8e00185d0 con 0x56191bedbb00 2024-05-17T22:51:47.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.405+0000 7fb8caff5700 1 -- 172.21.15.12:0/3925583029 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fb8e0018070 con 0x56191bedbb00 2024-05-17T22:51:47.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.405+0000 7fb8fbe7eb80 10 client.26066 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:51:47.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.405+0000 7fb8fbe7eb80 10 client.26066 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:51:47.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.405+0000 7fb8fbe7eb80 20 client.26066 populate_metadata read hostname 'smithi012' 2024-05-17T22:51:47.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.405+0000 7fb8fbe7eb80 1 --2- 172.21.15.12:0/3925583029 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x56191bef4ea0 0x56191bef7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:47.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.405+0000 7fb8fbe7eb80 4 client.26066 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:51:47.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.406+0000 7fb8fbe7eb80 1 -- 172.21.15.12:0/3925583029 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb8d80044e0 con 0x56191bef4ea0 2024-05-17T22:51:47.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.407+0000 7fb8e6ffd700 1 --2- 172.21.15.12:0/3925583029 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x56191bef4ea0 0x56191bef7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:47.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.409+0000 7fb8e6ffd700 1 --2- 172.21.15.12:0/3925583029 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x56191bef4ea0 0x56191bef7350 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b0dc4b6fda5315c server_cookie=4db661c5cac1eb8c in_seq=0 out_seq=0 2024-05-17T22:51:47.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.410+0000 7fb8caff5700 10 client.26066 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:51:47.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.410+0000 7fb8caff5700 1 -- 172.21.15.12:0/3925583029 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fb8d80044e0 con 0x56191bef4ea0 2024-05-17T22:51:47.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.410+0000 7fb8caff5700 10 client.26066 handle_command_reply: tid=0 2024-05-17T22:51:47.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.411+0000 7fb8fbe7eb80 1 client.26066 shutdown 2024-05-17T22:51:47.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.413+0000 7fb8fbe7eb80 1 -- 172.21.15.12:0/3925583029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191bedbb00 msgr2=0x56191bee4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:51:47.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.413+0000 7fb8fbe7eb80 1 --2- 172.21.15.12:0/3925583029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191bedbb00 0x56191bee4fe0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb8e000a040 tx=0x7fb8e001ac70 comp rx=0 tx=0).stop 2024-05-17T22:51:47.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.417+0000 7fb8fbe7eb80 1 -- 172.21.15.12:0/3925583029 shutdown_connections 2024-05-17T22:51:47.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.417+0000 7fb8fbe7eb80 1 --2- 172.21.15.12:0/3925583029 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x56191bef4ea0 0x56191bef7350 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:47.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.417+0000 7fb8fbe7eb80 1 --2- 172.21.15.12:0/3925583029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56191bee1400 0x56191bee5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:47.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.417+0000 7fb8caff5700 0 client.26066 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:51:47.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.417+0000 7fb8fbe7eb80 1 --2- 172.21.15.12:0/3925583029 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191bd5d9f0 0x56191bee4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:47.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.417+0000 7fb8fbe7eb80 1 --2- 172.21.15.12:0/3925583029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191bedbb00 0x56191bee4fe0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:47.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.417+0000 7fb8fbe7eb80 1 -- 172.21.15.12:0/3925583029 >> 172.21.15.12:0/3925583029 conn(0x56191bdbd7d0 msgr2=0x56191bed6160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:51:47.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.418+0000 7fb8fbe7eb80 1 -- 172.21.15.12:0/3925583029 shutdown_connections 2024-05-17T22:51:47.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.419+0000 7fb8fbe7eb80 1 -- 172.21.15.12:0/3925583029 wait complete. 2024-05-17T22:51:47.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.421+0000 7fb8fbe7eb80 20 client.26066 trim_cache size 0 max 16384 2024-05-17T22:51:47.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.459+0000 7fb8fbe7eb80 1 Processor -- start 2024-05-17T22:51:47.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.459+0000 7fb8fbe7eb80 1 -- start start 2024-05-17T22:51:47.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.459+0000 7fb8fbe7eb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191bd5d9f0 0x56191bee0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:47.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.459+0000 7fb8fbe7eb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191bedbb00 0x56191bee4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:47.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.459+0000 7fb8fbe7eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56191bee1400 0x56191bee5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:47.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.459+0000 7fb8fbe7eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e81dacf0 con 0x56191bee1400 2024-05-17T22:51:47.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.459+0000 7fb8fbe7eb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e81d8580 con 0x56191bd5d9f0 2024-05-17T22:51:47.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.459+0000 7fb8fbe7eb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e81d63d0 con 0x56191bedbb00 2024-05-17T22:51:47.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.460+0000 7fb8e77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56191bee1400 0x56191bee5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:47.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.460+0000 7fb8e6ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191bd5d9f0 0x56191bee0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:47.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.460+0000 7fb8e6ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191bd5d9f0 0x56191bee0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:48350/0 (socket says 172.21.15.12:48350) 2024-05-17T22:51:47.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.460+0000 7fb8e6ffd700 1 -- 172.21.15.12:0/2355402234 learned_addr learned my addr 172.21.15.12:0/2355402234 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:51:47.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.460+0000 7fb8e67fc700 1 --2- 172.21.15.12:0/2355402234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191bedbb00 0x56191bee4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:47.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.460+0000 7fb8e6ffd700 1 -- 172.21.15.12:0/2355402234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191bedbb00 msgr2=0x56191bee4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:51:47.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.460+0000 7fb8e6ffd700 1 --2- 172.21.15.12:0/2355402234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191bedbb00 0x56191bee4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:47.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.460+0000 7fb8e6ffd700 1 -- 172.21.15.12:0/2355402234 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56191bee1400 msgr2=0x56191bee5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:51:47.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.460+0000 7fb8e6ffd700 1 --2- 172.21.15.12:0/2355402234 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56191bee1400 0x56191bee5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:47.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.460+0000 7fb8e6ffd700 1 -- 172.21.15.12:0/2355402234 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56191bee3d30 con 0x56191bd5d9f0 2024-05-17T22:51:47.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.461+0000 7fb8e6ffd700 1 --2- 172.21.15.12:0/2355402234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191bd5d9f0 0x56191bee0510 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc00edd0 tx=0x7fb8dc015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6203b10138a65a12 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:51:47.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.461+0000 7fb8ccff9700 1 -- 172.21.15.12:0/2355402234 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc004a30 con 0x56191bd5d9f0 2024-05-17T22:51:47.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.461+0000 7fb8ccff9700 1 -- 172.21.15.12:0/2355402234 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8dc02a030 con 0x56191bd5d9f0 2024-05-17T22:51:47.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.461+0000 7fb8ccff9700 1 -- 172.21.15.12:0/2355402234 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc005280 con 0x56191bd5d9f0 2024-05-17T22:51:47.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.462+0000 7fb8fbe7eb80 1 -- 172.21.15.12:0/2355402234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191bd5d9f0 msgr2=0x56191bee0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:51:47.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.462+0000 7fb8fbe7eb80 1 --2- 172.21.15.12:0/2355402234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191bd5d9f0 0x56191bee0510 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc00edd0 tx=0x7fb8dc015000 comp rx=0 tx=0).stop 2024-05-17T22:51:47.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.462+0000 7fb8fbe7eb80 1 -- 172.21.15.12:0/2355402234 shutdown_connections 2024-05-17T22:51:47.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.462+0000 7fb8fbe7eb80 1 --2- 172.21.15.12:0/2355402234 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56191bee1400 0x56191bee5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:47.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.462+0000 7fb8fbe7eb80 1 --2- 172.21.15.12:0/2355402234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191bd5d9f0 0x56191bee0510 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:47.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.462+0000 7fb8fbe7eb80 1 --2- 172.21.15.12:0/2355402234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191bedbb00 0x56191bee4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:47.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.462+0000 7fb8fbe7eb80 1 -- 172.21.15.12:0/2355402234 >> 172.21.15.12:0/2355402234 conn(0x56191bdbd7d0 msgr2=0x56191bedfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:51:47.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.462+0000 7fb8fbe7eb80 1 -- 172.21.15.12:0/2355402234 shutdown_connections 2024-05-17T22:51:47.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.462+0000 7fb8fbe7eb80 1 -- 172.21.15.12:0/2355402234 wait complete. 2024-05-17T22:51:47.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.462+0000 7fb8fbe7eb80 1 -- start start 2024-05-17T22:51:47.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.463+0000 7fb8fbe7eb80 1 Processor -- start 2024-05-17T22:51:47.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.463+0000 7fb8fbe7eb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56191bd5d9f0 0x56191befc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:47.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.463+0000 7fb8fbe7eb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191bedbb00 0x56191befca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:47.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.463+0000 7fb8fbe7eb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191bee1400 0x56191befcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:47.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.463+0000 7fb8fbe7eb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e81d6a40 con 0x56191bd5d9f0 2024-05-17T22:51:47.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.463+0000 7fb8fbe7eb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e81d7eb0 con 0x56191bee1400 2024-05-17T22:51:47.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.463+0000 7fb8fbe7eb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e81d7110 con 0x56191bedbb00 2024-05-17T22:51:47.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.463+0000 7fb8e6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56191bd5d9f0 0x56191befc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:47.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.463+0000 7fb8fbe7eb80 10 client.0 _ll_register_callbacks cb 0x56191bd90b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:51:47.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.463+0000 7fb8e6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56191bd5d9f0 0x56191befc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48812/0 (socket says 172.21.15.12:48812) 2024-05-17T22:51:47.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.463+0000 7fb8e6ffd700 1 -- 172.21.15.12:0/1307414907 learned_addr learned my addr 172.21.15.12:0/1307414907 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:51:47.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.463+0000 7fb8e67fc700 1 --2- 172.21.15.12:0/1307414907 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191bedbb00 0x56191befca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:47.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.463+0000 7fb8e77fe700 1 --2- 172.21.15.12:0/1307414907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191bee1400 0x56191befcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:47.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.464+0000 7fb8e67fc700 1 -- 172.21.15.12:0/1307414907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191bee1400 msgr2=0x56191befcf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:51:47.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.464+0000 7fb8e67fc700 1 --2- 172.21.15.12:0/1307414907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191bee1400 0x56191befcf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:47.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.464+0000 7fb8e67fc700 1 -- 172.21.15.12:0/1307414907 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56191bd5d9f0 msgr2=0x56191befc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:51:47.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.464+0000 7fb8e67fc700 1 --2- 172.21.15.12:0/1307414907 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56191bd5d9f0 0x56191befc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:47.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.464+0000 7fb8e67fc700 1 -- 172.21.15.12:0/1307414907 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56191bf5e830 con 0x56191bedbb00 2024-05-17T22:51:47.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.464+0000 7fb8e77fe700 1 --2- 172.21.15.12:0/1307414907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191bee1400 0x56191befcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:51:47.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.465+0000 7fb8e67fc700 1 --2- 172.21.15.12:0/1307414907 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191bedbb00 0x56191befca00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb8e00a6700 tx=0x7fb8e0009fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f487620e765390a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:51:47.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.465+0000 7fb8caff5700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:51:47.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.465+0000 7fb8caff5700 1 -- 172.21.15.12:0/1307414907 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e00a9fd0 con 0x56191bedbb00 2024-05-17T22:51:47.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.465+0000 7fb8caff5700 1 -- 172.21.15.12:0/1307414907 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fb8e00a6c70 con 0x56191bedbb00 2024-05-17T22:51:47.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.465+0000 7fb8fbe7eb80 1 -- 172.21.15.12:0/1307414907 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8dc02a030 con 0x56191bedbb00 2024-05-17T22:51:47.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.465+0000 7fb8caff5700 1 -- 172.21.15.12:0/1307414907 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb8e00bb050 con 0x56191bedbb00 2024-05-17T22:51:47.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.465+0000 7fb8fbe7eb80 10 client.26011 fetch_fsmap learned FSMap version 39 2024-05-17T22:51:47.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.465+0000 7fb8fbe7eb80 1 -- 172.21.15.12:0/1307414907 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56191bf5faa0 con 0x56191bedbb00 2024-05-17T22:51:47.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.465+0000 7fb8caff5700 1 -- 172.21.15.12:0/1307414907 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8dc02a030 con 0x56191bedbb00 2024-05-17T22:51:47.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.466+0000 7fb8caff5700 1 -- 172.21.15.12:0/1307414907 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fb8e000e070 con 0x56191bedbb00 2024-05-17T22:51:47.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.466+0000 7fb8fbe7eb80 10 client.26011 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:51:47.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.466+0000 7fb8fbe7eb80 10 client.26011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:51:47.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.466+0000 7fb8fbe7eb80 20 client.26011 populate_metadata read hostname 'smithi012' 2024-05-17T22:51:47.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.466+0000 7fb8fbe7eb80 1 --2- 172.21.15.12:0/1307414907 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x56191bef4ea0 0x56191bf61fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:47.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.466+0000 7fb8fbe7eb80 4 client.26011 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:51:47.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.466+0000 7fb8fbe7eb80 1 -- 172.21.15.12:0/1307414907 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56191be0c020 con 0x56191bef4ea0 2024-05-17T22:51:47.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.466+0000 7fb8e6ffd700 1 --2- 172.21.15.12:0/1307414907 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x56191bef4ea0 0x56191bf61fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:47.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.466+0000 7fb8caff5700 10 client.26011 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:51:47.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.467+0000 7fb8e6ffd700 1 --2- 172.21.15.12:0/1307414907 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x56191bef4ea0 0x56191bf61fe0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d52a5421c74d22f9 server_cookie=9fb8f8e9d2db695 in_seq=0 out_seq=0 2024-05-17T22:51:47.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.467+0000 7fb8caff5700 1 -- 172.21.15.12:0/1307414907 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56191be0c020 con 0x56191bef4ea0 2024-05-17T22:51:47.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.467+0000 7fb8caff5700 10 client.26011 handle_command_reply: tid=0 2024-05-17T22:51:47.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.467+0000 7fb8fbe7eb80 1 client.26011 shutdown 2024-05-17T22:51:47.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.467+0000 7fb8fbe7eb80 1 -- 172.21.15.12:0/1307414907 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191bedbb00 msgr2=0x56191befca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:51:47.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.467+0000 7fb8fbe7eb80 1 --2- 172.21.15.12:0/1307414907 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191bedbb00 0x56191befca00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb8e00a6700 tx=0x7fb8e0009fd0 comp rx=0 tx=0).stop 2024-05-17T22:51:47.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.467+0000 7fb8fbe7eb80 1 -- 172.21.15.12:0/1307414907 shutdown_connections 2024-05-17T22:51:47.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.467+0000 7fb8fbe7eb80 1 --2- 172.21.15.12:0/1307414907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191bee1400 0x56191befcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:47.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.467+0000 7fb8fbe7eb80 1 --2- 172.21.15.12:0/1307414907 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x56191bef4ea0 0x56191bf61fe0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:47.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.467+0000 7fb8caff5700 0 client.26011 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:51:47.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.467+0000 7fb8fbe7eb80 1 --2- 172.21.15.12:0/1307414907 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56191bd5d9f0 0x56191befc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:47.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.467+0000 7fb8fbe7eb80 1 --2- 172.21.15.12:0/1307414907 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191bedbb00 0x56191befca00 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:47.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.467+0000 7fb8fbe7eb80 1 -- 172.21.15.12:0/1307414907 >> 172.21.15.12:0/1307414907 conn(0x56191bdbd7d0 msgr2=0x56191bee6290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:51:47.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.468+0000 7fb8fbe7eb80 1 -- 172.21.15.12:0/1307414907 shutdown_connections 2024-05-17T22:51:47.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.468+0000 7fb8fbe7eb80 1 -- 172.21.15.12:0/1307414907 wait complete. 2024-05-17T22:51:47.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.468+0000 7fb8fbe7eb80 20 client.26011 trim_cache size 0 max 16384 2024-05-17T22:51:47.474 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:51:47.475 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-17T22:51:47.475 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-17T22:51:47.475 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:51:47.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.473+0000 7fb8ed4a3700 1 -- 172.21.15.12:0/3875250254 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb8d00b2a50 msgr2=0x7fb8d00b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:51:47.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.473+0000 7fb8ed4a3700 1 --2- 172.21.15.12:0/3875250254 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb8d00b2a50 0x7fb8d00b4f60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc005950 tx=0x7fb8dc0058e0 comp rx=0 tx=0).stop 2024-05-17T22:51:47.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.473+0000 7fb8ed4a3700 1 -- 172.21.15.12:0/3875250254 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8e810b180 msgr2=0x7fb8e8118f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:51:47.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.473+0000 7fb8ed4a3700 1 --2- 172.21.15.12:0/3875250254 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8e810b180 0x7fb8e8118f90 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0003c30 tx=0x7fb8e0003d10 comp rx=0 tx=0).stop 2024-05-17T22:51:47.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.477+0000 7fb8ed4a3700 1 -- 172.21.15.12:0/3875250254 shutdown_connections 2024-05-17T22:51:47.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.477+0000 7fb8ed4a3700 1 --2- 172.21.15.12:0/3875250254 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb8d00b2a50 0x7fb8d00b4f60 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:47.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.477+0000 7fb8ed4a3700 1 --2- 172.21.15.12:0/3875250254 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb8e811d550 0x7fb8e81194d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:47.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.477+0000 7fb8ed4a3700 1 --2- 172.21.15.12:0/3875250254 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8e810a830 0x7fb8e811ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:47.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.477+0000 7fb8ed4a3700 1 --2- 172.21.15.12:0/3875250254 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8e810b180 0x7fb8e8118f90 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:47.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.477+0000 7fb8ed4a3700 1 -- 172.21.15.12:0/3875250254 >> 172.21.15.12:0/3875250254 conn(0x7fb8e806efc0 msgr2=0x7fb8e811db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:51:47.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.478+0000 7fb8ed4a3700 1 -- 172.21.15.12:0/3875250254 shutdown_connections 2024-05-17T22:51:47.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:47.478+0000 7fb8ed4a3700 1 -- 172.21.15.12:0/3875250254 wait complete. 2024-05-17T22:51:48.380 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-17T22:51:48.380 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T22:51:48.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:48 smithi151 ceph-mon[36101]: pgmap v815: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 20 MiB/s wr, 1.45k op/s 2024-05-17T22:51:48.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:48 smithi096 ceph-mon[36065]: pgmap v815: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 20 MiB/s wr, 1.45k op/s 2024-05-17T22:51:48.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:48 smithi012 ceph-mon[31632]: pgmap v815: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 20 MiB/s wr, 1.45k op/s 2024-05-17T22:51:48.851 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:51:50.481 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:50 smithi012 ceph-mon[31632]: pgmap v816: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 19 MiB/s wr, 1.42k op/s 2024-05-17T22:51:50.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:50 smithi151 ceph-mon[36101]: pgmap v816: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 19 MiB/s wr, 1.42k op/s 2024-05-17T22:51:50.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:50 smithi096 ceph-mon[36065]: pgmap v816: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 19 MiB/s wr, 1.42k op/s 2024-05-17T22:51:51.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 750.0 s, 426.6 tps, lat 236.105 ms stddev 309.394 2024-05-17T22:51:51.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:51.527+0000 7f0164e15700 1 -- 172.21.15.12:0/1959713533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0160102e00 msgr2=0x7f0160103210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:51:51.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:51.527+0000 7f0164e15700 1 --2- 172.21.15.12:0/1959713533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0160102e00 0x7f0160103210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0154009990 tx=0x7f0154009ca0 comp rx=0 tx=0).stop 2024-05-17T22:51:51.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:51.530+0000 7f0164e15700 1 -- 172.21.15.12:0/1959713533 shutdown_connections 2024-05-17T22:51:51.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:51.530+0000 7f0164e15700 1 --2- 172.21.15.12:0/1959713533 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0160108080 0x7f016010f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:51.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:51.530+0000 7f0164e15700 1 --2- 172.21.15.12:0/1959713533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0160102e00 0x7f0160103210 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:51.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:51.530+0000 7f0164e15700 1 --2- 172.21.15.12:0/1959713533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0160107730 0x7f0160107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:51.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:51.530+0000 7f0164e15700 1 -- 172.21.15.12:0/1959713533 >> 172.21.15.12:0/1959713533 conn(0x7f0160079bd0 msgr2=0x7f0160079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:51:51.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:51.534+0000 7f0164e15700 1 -- 172.21.15.12:0/1959713533 shutdown_connections 2024-05-17T22:51:51.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:51.535+0000 7f0164e15700 1 -- 172.21.15.12:0/1959713533 wait complete. 2024-05-17T22:51:51.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:51.536+0000 7f0164e15700 1 Processor -- start 2024-05-17T22:51:51.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:51.536+0000 7f0164e15700 1 -- start start 2024-05-17T22:51:51.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:51.536+0000 7f0164e15700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0160102e00 0x7f016010ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:51.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:51.536+0000 7f0164e15700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0160107730 0x7f016010d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:51.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:51.536+0000 7f0164e15700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0160108080 0x7f016010d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:51.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:51.536+0000 7f0164e15700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0160111d60 con 0x7f0160102e00 2024-05-17T22:51:51.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:51.536+0000 7f0164e15700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0160111be0 con 0x7f0160107730 2024-05-17T22:51:51.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:51.536+0000 7f0164e15700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0160111ee0 con 0x7f0160108080 2024-05-17T22:51:51.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:51.538+0000 7f015dd9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0160107730 0x7f016010d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:51.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:51.538+0000 7f015ed9d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0160108080 0x7f016010d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:51.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:51.538+0000 7f015ed9d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0160108080 0x7f016010d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:42442/0 (socket says 172.21.15.12:42442) 2024-05-17T22:51:51.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:51.538+0000 7f015ed9d700 1 -- 172.21.15.12:0/1649223526 learned_addr learned my addr 172.21.15.12:0/1649223526 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:51:51.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:51.538+0000 7f015ed9d700 1 -- 172.21.15.12:0/1649223526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0160107730 msgr2=0x7f016010d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:51:51.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:51.538+0000 7f015ed9d700 1 --2- 172.21.15.12:0/1649223526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0160107730 0x7f016010d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:51.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:51.538+0000 7f015ed9d700 1 -- 172.21.15.12:0/1649223526 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0160102e00 msgr2=0x7f016010ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:51:51.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:51.538+0000 7f015ed9d700 1 --2- 172.21.15.12:0/1649223526 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0160102e00 0x7f016010ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:51.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:51.538+0000 7f015ed9d700 1 -- 172.21.15.12:0/1649223526 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0160074850 con 0x7f0160108080 2024-05-17T22:51:51.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:51.539+0000 7f015ed9d700 1 --2- 172.21.15.12:0/1649223526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0160108080 0x7f016010d8b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f015000b4a0 tx=0x7f015000b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:51:51.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:51.539+0000 7f014f7fe700 1 -- 172.21.15.12:0/1649223526 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0150015020 con 0x7f0160108080 2024-05-17T22:51:51.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:51.539+0000 7f0164e15700 1 -- 172.21.15.12:0/1649223526 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0160074b40 con 0x7f0160108080 2024-05-17T22:51:51.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:51.539+0000 7f0164e15700 1 -- 172.21.15.12:0/1649223526 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01600750d0 con 0x7f0160108080 2024-05-17T22:51:51.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:51.539+0000 7f014f7fe700 1 -- 172.21.15.12:0/1649223526 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0150011ce0 con 0x7f0160108080 2024-05-17T22:51:51.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:51.539+0000 7f014f7fe700 1 -- 172.21.15.12:0/1649223526 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0150015020 con 0x7f0160108080 2024-05-17T22:51:51.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:51.543+0000 7f014f7fe700 1 -- 172.21.15.12:0/1649223526 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0150018020 con 0x7f0160108080 2024-05-17T22:51:51.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:51.544+0000 7f0164e15700 1 -- 172.21.15.12:0/1649223526 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0140005330 con 0x7f0160108080 2024-05-17T22:51:51.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:51.548+0000 7f014f7fe700 1 --2- 172.21.15.12:0/1649223526 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f01480b2cb0 0x7f01480b51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:51.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:51.548+0000 7f015e59c700 1 --2- 172.21.15.12:0/1649223526 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f01480b2cb0 0x7f01480b51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:51.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:51.549+0000 7f014f7fe700 1 -- 172.21.15.12:0/1649223526 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f0150011630 con 0x7f0160108080 2024-05-17T22:51:51.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:51.549+0000 7f015e59c700 1 --2- 172.21.15.12:0/1649223526 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f01480b2cb0 0x7f01480b51c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0154009750 tx=0x7f0154000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:51:51.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:51.557+0000 7f014f7fe700 1 -- 172.21.15.12:0/1649223526 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f015001f020 con 0x7f0160108080 2024-05-17T22:51:52.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:52.033+0000 7f0164e15700 1 -- 172.21.15.12:0/1649223526 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0140005a40 con 0x7f0160108080 2024-05-17T22:51:52.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:52.037+0000 7f014f7fe700 1 -- 172.21.15.12:0/1649223526 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11870 (secure 0 0 0) 0x7f01500aa090 con 0x7f0160108080 2024-05-17T22:51:52.040 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:51:52.041 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1407219319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1407219319},{"type":"v1","addr":"172.21.15.12:6835","nonce":1407219319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/630977886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":630977886},{"type":"v1","addr":"172.21.15.12:6837","nonce":630977886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3731914742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3731914742},{"type":"v1","addr":"172.21.15.12:6839","nonce":3731914742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/3642142501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3642142501},{"type":"v1","addr":"172.21.15.12:6841","nonce":3642142501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3554053059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3554053059},{"type":"v1","addr":"172.21.15.151:6833","nonce":3554053059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/4228325569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":4228325569},{"type":"v1","addr":"172.21.15.96:6833","nonce":4228325569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T22:34:42.642111+0000","modified":"2024-05-17T22:47:38.155618+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24461,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/542323754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":542323754},{"type":"v1","addr":"172.21.15.151:6835","nonce":542323754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1758442232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1758442232},{"type":"v1","addr":"172.21.15.96:6835","nonce":1758442232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3403324038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3403324038},{"type":"v1","addr":"172.21.15.151:6837","nonce":3403324038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.151:6839/2338353849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2338353849},{"type":"v1","addr":"172.21.15.151:6839","nonce":2338353849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.96:6837/4222574338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":4222574338},{"type":"v1","addr":"172.21.15.96:6837","nonce":4222574338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/2986295519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2986295519},{"type":"v1","addr":"172.21.15.96:6839","nonce":2986295519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24457,24461,24482]},"id":1}]} 2024-05-17T22:51:52.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:52.045+0000 7f0164e15700 1 -- 172.21.15.12:0/1649223526 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f01480b2cb0 msgr2=0x7f01480b51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:51:52.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:52.045+0000 7f0164e15700 1 --2- 172.21.15.12:0/1649223526 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f01480b2cb0 0x7f01480b51c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0154009750 tx=0x7f0154000bc0 comp rx=0 tx=0).stop 2024-05-17T22:51:52.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:52.045+0000 7f0164e15700 1 -- 172.21.15.12:0/1649223526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0160108080 msgr2=0x7f016010d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:51:52.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:52.046+0000 7f0164e15700 1 --2- 172.21.15.12:0/1649223526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0160108080 0x7f016010d8b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f015000b4a0 tx=0x7f015000b860 comp rx=0 tx=0).stop 2024-05-17T22:51:52.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:52.049+0000 7f0164e15700 1 -- 172.21.15.12:0/1649223526 shutdown_connections 2024-05-17T22:51:52.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:52.049+0000 7f0164e15700 1 --2- 172.21.15.12:0/1649223526 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f01480b2cb0 0x7f01480b51c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:52.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:52.049+0000 7f0164e15700 1 --2- 172.21.15.12:0/1649223526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0160108080 0x7f016010d8b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:52.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:52.049+0000 7f0164e15700 1 --2- 172.21.15.12:0/1649223526 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0160102e00 0x7f016010ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:52.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:52.049+0000 7f0164e15700 1 --2- 172.21.15.12:0/1649223526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0160107730 0x7f016010d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:52.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:52.049+0000 7f0164e15700 1 -- 172.21.15.12:0/1649223526 >> 172.21.15.12:0/1649223526 conn(0x7f0160079bd0 msgr2=0x7f0160105870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:51:52.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:52.049+0000 7f0164e15700 1 -- 172.21.15.12:0/1649223526 shutdown_connections 2024-05-17T22:51:52.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:52.049+0000 7f0164e15700 1 -- 172.21.15.12:0/1649223526 wait complete. 2024-05-17T22:51:52.058 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 39 2024-05-17T22:51:52.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:52 smithi012 ceph-mon[31632]: pgmap v817: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 11 MiB/s wr, 867 op/s 2024-05-17T22:51:52.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:52 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/1649223526' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:51:52.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:52 smithi151 ceph-mon[36101]: pgmap v817: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 11 MiB/s wr, 867 op/s 2024-05-17T22:51:52.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:52 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/1649223526' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:51:52.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:52 smithi096 ceph-mon[36065]: pgmap v817: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 11 MiB/s wr, 867 op/s 2024-05-17T22:51:52.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:52 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/1649223526' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:51:52.897 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-17T22:51:53.376 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:51:54.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:54 smithi096 ceph-mon[36065]: pgmap v818: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.3 MiB/s wr, 490 op/s 2024-05-17T22:51:54.585 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:54 smithi012 ceph-mon[31632]: pgmap v818: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.3 MiB/s wr, 490 op/s 2024-05-17T22:51:54.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:54 smithi151 ceph-mon[36101]: pgmap v818: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.3 MiB/s wr, 490 op/s 2024-05-17T22:51:56.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 755.0 s, 431.8 tps, lat 230.303 ms stddev 290.237 2024-05-17T22:51:56.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:56 smithi096 ceph-mon[36065]: pgmap v819: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 MiB/s wr, 563 op/s 2024-05-17T22:51:56.554 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:56 smithi012 ceph-mon[31632]: pgmap v819: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 MiB/s wr, 563 op/s 2024-05-17T22:51:56.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.652+0000 7f3f0134a700 1 -- 172.21.15.12:0/3500379693 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3efc104e00 msgr2=0x7f3efc105210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:51:56.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.652+0000 7f3f0134a700 1 --2- 172.21.15.12:0/3500379693 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3efc104e00 0x7f3efc105210 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3eec009990 tx=0x7f3eec009ca0 comp rx=0 tx=0).stop 2024-05-17T22:51:56.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.652+0000 7f3f0134a700 1 -- 172.21.15.12:0/3500379693 shutdown_connections 2024-05-17T22:51:56.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.652+0000 7f3f0134a700 1 --2- 172.21.15.12:0/3500379693 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3efc10e230 0x7f3efc110740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:56.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.652+0000 7f3f0134a700 1 --2- 172.21.15.12:0/3500379693 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3efc104e00 0x7f3efc105210 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:56.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.652+0000 7f3f0134a700 1 --2- 172.21.15.12:0/3500379693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3efc105750 0x7f3efc10dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:56.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.652+0000 7f3f0134a700 1 -- 172.21.15.12:0/3500379693 >> 172.21.15.12:0/3500379693 conn(0x7f3efc079bd0 msgr2=0x7f3efc079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:51:56.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.653+0000 7f3f0134a700 1 -- 172.21.15.12:0/3500379693 shutdown_connections 2024-05-17T22:51:56.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.653+0000 7f3f0134a700 1 -- 172.21.15.12:0/3500379693 wait complete. 2024-05-17T22:51:56.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.654+0000 7f3f0134a700 1 Processor -- start 2024-05-17T22:51:56.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.654+0000 7f3f0134a700 1 -- start start 2024-05-17T22:51:56.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.654+0000 7f3f0134a700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3efc104e00 0x7f3efc102af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:56.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.655+0000 7f3f0134a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3efc105750 0x7f3efc103030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:56.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.655+0000 7f3f0134a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3efc10e230 0x7f3efc103570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:56.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.655+0000 7f3f0134a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc113dd0 con 0x7f3efc105750 2024-05-17T22:51:56.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.655+0000 7f3f0134a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc113c50 con 0x7f3efc10e230 2024-05-17T22:51:56.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.655+0000 7f3f0134a700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc113f50 con 0x7f3efc104e00 2024-05-17T22:51:56.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.655+0000 7f3efa7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3efc105750 0x7f3efc103030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:56.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.655+0000 7f3efaffd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3efc104e00 0x7f3efc102af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:56.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.655+0000 7f3efa7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3efc105750 0x7f3efc103030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32930/0 (socket says 172.21.15.12:32930) 2024-05-17T22:51:56.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.655+0000 7f3efaffd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3efc104e00 0x7f3efc102af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:42464/0 (socket says 172.21.15.12:42464) 2024-05-17T22:51:56.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.655+0000 7f3efa7fc700 1 -- 172.21.15.12:0/1296094732 learned_addr learned my addr 172.21.15.12:0/1296094732 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:51:56.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.655+0000 7f3efb7fe700 1 --2- 172.21.15.12:0/1296094732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3efc10e230 0x7f3efc103570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:56.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.655+0000 7f3efaffd700 1 -- 172.21.15.12:0/1296094732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3efc10e230 msgr2=0x7f3efc103570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:51:56.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.655+0000 7f3efaffd700 1 --2- 172.21.15.12:0/1296094732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3efc10e230 0x7f3efc103570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:56.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.655+0000 7f3efaffd700 1 -- 172.21.15.12:0/1296094732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3efc105750 msgr2=0x7f3efc103030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:51:56.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.655+0000 7f3efaffd700 1 --2- 172.21.15.12:0/1296094732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3efc105750 0x7f3efc103030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:56.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.655+0000 7f3efaffd700 1 -- 172.21.15.12:0/1296094732 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3efc1a8b30 con 0x7f3efc104e00 2024-05-17T22:51:56.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.655+0000 7f3efa7fc700 1 --2- 172.21.15.12:0/1296094732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3efc105750 0x7f3efc103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:51:56.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.656+0000 7f3efaffd700 1 --2- 172.21.15.12:0/1296094732 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3efc104e00 0x7f3efc102af0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3eec0059d0 tx=0x7f3eec0038b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:51:56.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.656+0000 7f3ee3fff700 1 -- 172.21.15.12:0/1296094732 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eec027070 con 0x7f3efc104e00 2024-05-17T22:51:56.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.656+0000 7f3ee3fff700 1 -- 172.21.15.12:0/1296094732 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3eec00be60 con 0x7f3efc104e00 2024-05-17T22:51:56.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.656+0000 7f3f0134a700 1 -- 172.21.15.12:0/1296094732 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3efc1a8dc0 con 0x7f3efc104e00 2024-05-17T22:51:56.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.656+0000 7f3ee3fff700 1 -- 172.21.15.12:0/1296094732 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eec004230 con 0x7f3efc104e00 2024-05-17T22:51:56.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.656+0000 7f3f0134a700 1 -- 172.21.15.12:0/1296094732 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3efc1a9200 con 0x7f3efc104e00 2024-05-17T22:51:56.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.660+0000 7f3ee3fff700 1 -- 172.21.15.12:0/1296094732 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3eec02a070 con 0x7f3efc104e00 2024-05-17T22:51:56.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.660+0000 7f3f0fd25b80 1 Processor -- start 2024-05-17T22:51:56.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.660+0000 7f3f0fd25b80 1 -- start start 2024-05-17T22:51:56.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.660+0000 7f3f0fd25b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570c72c39f0 0x5570c7447560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:56.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.660+0000 7f3f0fd25b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570c7447aa0 0x5570c7441c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:56.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.660+0000 7f3f0fd25b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570c7442170 0x5570c7442580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:56.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.660+0000 7f3f0fd25b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5570c7370290 con 0x5570c7447aa0 2024-05-17T22:51:56.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.660+0000 7f3ee3fff700 1 --2- 172.21.15.12:0/1296094732 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f3ee40b2900 0x7f3ee40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:56.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.660+0000 7f3f0fd25b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5570c735ffd0 con 0x5570c72c39f0 2024-05-17T22:51:56.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.660+0000 7f3f0fd25b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5570c7363e70 con 0x5570c7442170 2024-05-17T22:51:56.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.661+0000 7f3efa7fc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570c7442170 0x5570c7442580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:56.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.661+0000 7f3efb7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570c72c39f0 0x5570c7447560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:56.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.661+0000 7f3efb7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570c72c39f0 0x5570c7447560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:48196/0 (socket says 172.21.15.12:48196) 2024-05-17T22:51:56.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.661+0000 7f3efb7fe700 1 -- 172.21.15.12:0/3454884110 learned_addr learned my addr 172.21.15.12:0/3454884110 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:51:56.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.661+0000 7f3efa7fc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570c7442170 0x5570c7442580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:42468/0 (socket says 172.21.15.12:42468) 2024-05-17T22:51:56.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.661+0000 7f3efb7fe700 1 --2- 172.21.15.12:0/1296094732 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f3ee40b2900 0x7f3ee40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:56.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.661+0000 7f3efb7fe700 1 -- 172.21.15.12:0/3454884110 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570c7442170 msgr2=0x5570c7442580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:51:56.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.661+0000 7f3efb7fe700 1 --2- 172.21.15.12:0/3454884110 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570c7442170 0x5570c7442580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:56.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.661+0000 7f3efb7fe700 1 -- 172.21.15.12:0/3454884110 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570c7447aa0 msgr2=0x5570c7441c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:51:56.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.661+0000 7f3efaffd700 1 --2- 172.21.15.12:0/3454884110 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570c7447aa0 0x5570c7441c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:56.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.661+0000 7f3efb7fe700 1 --2- 172.21.15.12:0/3454884110 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570c7447aa0 0x5570c7441c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:56.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.661+0000 7f3efb7fe700 1 -- 172.21.15.12:0/3454884110 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570c7309590 con 0x5570c72c39f0 2024-05-17T22:51:56.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.661+0000 7f3ee3fff700 1 -- 172.21.15.12:0/1296094732 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f3eec0e36b0 con 0x7f3efc104e00 2024-05-17T22:51:56.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.661+0000 7f3efb7fe700 1 --2- 172.21.15.12:0/3454884110 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570c72c39f0 0x5570c7447560 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4009b90 tx=0x7f3ef4009ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=885b75ffb7d431c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:51:56.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.662+0000 7f3ee17fa700 1 -- 172.21.15.12:0/3454884110 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4016d50 con 0x5570c72c39f0 2024-05-17T22:51:56.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.662+0000 7f3efb7fe700 1 --2- 172.21.15.12:0/1296094732 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f3ee40b2900 0x7f3ee40b4e10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4020000 tx=0x7f3ef401e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:51:56.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.663+0000 7f3ee17fa700 1 -- 172.21.15.12:0/3454884110 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ef4005030 con 0x5570c72c39f0 2024-05-17T22:51:56.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.663+0000 7f3ee17fa700 1 -- 172.21.15.12:0/3454884110 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef40193f0 con 0x5570c72c39f0 2024-05-17T22:51:56.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.664+0000 7f3f0fd25b80 1 -- 172.21.15.12:0/3454884110 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570c72c39f0 msgr2=0x5570c7447560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:51:56.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.664+0000 7f3f0fd25b80 1 --2- 172.21.15.12:0/3454884110 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570c72c39f0 0x5570c7447560 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4009b90 tx=0x7f3ef4009ea0 comp rx=0 tx=0).stop 2024-05-17T22:51:56.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.664+0000 7f3f0fd25b80 1 -- 172.21.15.12:0/3454884110 shutdown_connections 2024-05-17T22:51:56.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.664+0000 7f3f0fd25b80 1 --2- 172.21.15.12:0/3454884110 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570c7442170 0x5570c7442580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:56.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.664+0000 7f3f0fd25b80 1 --2- 172.21.15.12:0/3454884110 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570c72c39f0 0x5570c7447560 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:56.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.664+0000 7f3f0fd25b80 1 --2- 172.21.15.12:0/3454884110 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570c7447aa0 0x5570c7441c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:56.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.664+0000 7f3f0fd25b80 1 -- 172.21.15.12:0/3454884110 >> 172.21.15.12:0/3454884110 conn(0x5570c7307f00 msgr2=0x5570c743cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:51:56.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.664+0000 7f3f0fd25b80 1 -- 172.21.15.12:0/3454884110 shutdown_connections 2024-05-17T22:51:56.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.664+0000 7f3f0fd25b80 1 -- 172.21.15.12:0/3454884110 wait complete. 2024-05-17T22:51:56.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.665+0000 7f3f0fd25b80 1 -- start start 2024-05-17T22:51:56.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.665+0000 7f3f0fd25b80 1 Processor -- start 2024-05-17T22:51:56.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.665+0000 7f3f0fd25b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570c72c39f0 0x5570c744b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:56.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.665+0000 7f3f0fd25b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570c7442170 0x5570c744b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:56.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.665+0000 7f3f0fd25b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570c7447aa0 0x5570c744bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:56.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.665+0000 7f3f0fd25b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5570c7267150 con 0x5570c72c39f0 2024-05-17T22:51:56.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.665+0000 7f3f0fd25b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5570c7300e80 con 0x5570c7447aa0 2024-05-17T22:51:56.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.665+0000 7f3f0fd25b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5570c744e390 con 0x5570c7442170 2024-05-17T22:51:56.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.665+0000 7f3f0fd25b80 10 client.0 _ll_register_callbacks cb 0x5570c72faf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:51:56.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.665+0000 7f3efaffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570c72c39f0 0x5570c744b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:56.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.665+0000 7f3efa7fc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570c7442170 0x5570c744b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:56.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.665+0000 7f3efb7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570c7447aa0 0x5570c744bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:56.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.665+0000 7f3efaffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570c72c39f0 0x5570c744b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32946/0 (socket says 172.21.15.12:32946) 2024-05-17T22:51:56.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.665+0000 7f3efb7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570c7447aa0 0x5570c744bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:48202/0 (socket says 172.21.15.12:48202) 2024-05-17T22:51:56.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.665+0000 7f3efaffd700 1 -- 172.21.15.12:0/4280719744 learned_addr learned my addr 172.21.15.12:0/4280719744 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:51:56.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.666+0000 7f3efb7fe700 1 -- 172.21.15.12:0/4280719744 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570c7442170 msgr2=0x5570c744b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:51:56.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.666+0000 7f3efb7fe700 1 --2- 172.21.15.12:0/4280719744 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570c7442170 0x5570c744b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:56.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.666+0000 7f3efb7fe700 1 -- 172.21.15.12:0/4280719744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570c72c39f0 msgr2=0x5570c744b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:51:56.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.666+0000 7f3efb7fe700 1 --2- 172.21.15.12:0/4280719744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570c72c39f0 0x5570c744b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:56.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.666+0000 7f3efb7fe700 1 -- 172.21.15.12:0/4280719744 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570c72b0480 con 0x5570c7447aa0 2024-05-17T22:51:56.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.666+0000 7f3efa7fc700 1 --2- 172.21.15.12:0/4280719744 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570c7442170 0x5570c744b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:51:56.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.666+0000 7f3efb7fe700 1 --2- 172.21.15.12:0/4280719744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570c7447aa0 0x5570c744bc60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4009b60 tx=0x7f3ef4005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f125a38377915801 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:51:56.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.666+0000 7f3edeff5700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:51:56.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.666+0000 7f3edeff5700 1 -- 172.21.15.12:0/4280719744 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef402f020 con 0x5570c7447aa0 2024-05-17T22:51:56.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.666+0000 7f3edeff5700 1 -- 172.21.15.12:0/4280719744 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f3ef4016d50 con 0x5570c7447aa0 2024-05-17T22:51:56.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.666+0000 7f3f0fd25b80 1 -- 172.21.15.12:0/4280719744 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ef4005030 con 0x5570c7447aa0 2024-05-17T22:51:56.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.666+0000 7f3edeff5700 1 -- 172.21.15.12:0/4280719744 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3ef4004a90 con 0x5570c7447aa0 2024-05-17T22:51:56.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.666+0000 7f3f0fd25b80 10 client.26084 fetch_fsmap learned FSMap version 39 2024-05-17T22:51:56.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.666+0000 7f3f0fd25b80 1 -- 172.21.15.12:0/4280719744 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5570c7458ba0 con 0x5570c7447aa0 2024-05-17T22:51:56.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.667+0000 7f3edeff5700 1 -- 172.21.15.12:0/4280719744 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ef40051e0 con 0x5570c7447aa0 2024-05-17T22:51:56.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.667+0000 7f3edeff5700 1 -- 172.21.15.12:0/4280719744 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f3ef40196d0 con 0x5570c7447aa0 2024-05-17T22:51:56.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.667+0000 7f3f0fd25b80 10 client.26084 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:51:56.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.667+0000 7f3f0fd25b80 10 client.26084 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-17T22:51:56.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.667+0000 7f3f0fd25b80 20 client.26084 populate_metadata read hostname 'smithi012' 2024-05-17T22:51:56.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.667+0000 7f3f0fd25b80 1 --2- 172.21.15.12:0/4280719744 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x5570c745b070 0x5570c745d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:56.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.667+0000 7f3f0fd25b80 4 client.26084 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:51:56.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.667+0000 7f3f0fd25b80 1 -- 172.21.15.12:0/4280719744 --> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3ef4005030 con 0x5570c745b070 2024-05-17T22:51:56.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.667+0000 7f3efaffd700 1 --2- 172.21.15.12:0/4280719744 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x5570c745b070 0x5570c745d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:56.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.668+0000 7f3efaffd700 1 --2- 172.21.15.12:0/4280719744 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x5570c745b070 0x5570c745d490 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=24ab6080cbec4f8e server_cookie=73aee8a1719ba217 in_seq=0 out_seq=0 2024-05-17T22:51:56.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.668+0000 7f3edeff5700 10 client.26084 ms_handle_connect on v2:172.21.15.151:6838/2338353849 2024-05-17T22:51:56.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.668+0000 7f3edeff5700 1 -- 172.21.15.12:0/4280719744 <== mds.2 v2:172.21.15.151:6838/2338353849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f3ef4005030 con 0x5570c745b070 2024-05-17T22:51:56.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.668+0000 7f3edeff5700 10 client.26084 handle_command_reply: tid=0 2024-05-17T22:51:56.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.668+0000 7f3f0fd25b80 1 client.26084 shutdown 2024-05-17T22:51:56.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.669+0000 7f3f0fd25b80 1 -- 172.21.15.12:0/4280719744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570c7447aa0 msgr2=0x5570c744bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:51:56.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.669+0000 7f3f0fd25b80 1 --2- 172.21.15.12:0/4280719744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570c7447aa0 0x5570c744bc60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4009b60 tx=0x7f3ef4005b50 comp rx=0 tx=0).stop 2024-05-17T22:51:56.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.669+0000 7f3f0fd25b80 1 -- 172.21.15.12:0/4280719744 shutdown_connections 2024-05-17T22:51:56.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.669+0000 7f3f0fd25b80 1 --2- 172.21.15.12:0/4280719744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570c7447aa0 0x5570c744bc60 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:56.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.669+0000 7f3f0fd25b80 1 --2- 172.21.15.12:0/4280719744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570c72c39f0 0x5570c744b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:56.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.669+0000 7f3f0fd25b80 1 --2- 172.21.15.12:0/4280719744 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x5570c745b070 0x5570c745d490 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:56.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.669+0000 7f3f0fd25b80 1 --2- 172.21.15.12:0/4280719744 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570c7442170 0x5570c744b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:56.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.669+0000 7f3f0fd25b80 1 -- 172.21.15.12:0/4280719744 >> 172.21.15.12:0/4280719744 conn(0x5570c7307f00 msgr2=0x5570c7436ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:51:56.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.669+0000 7f3edeff5700 0 client.26084 ms_handle_reset on v2:172.21.15.151:6838/2338353849 2024-05-17T22:51:56.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.669+0000 7f3f0fd25b80 1 -- 172.21.15.12:0/4280719744 shutdown_connections 2024-05-17T22:51:56.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.669+0000 7f3f0fd25b80 1 -- 172.21.15.12:0/4280719744 wait complete. 2024-05-17T22:51:56.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.669+0000 7f3f0fd25b80 20 client.26084 trim_cache size 0 max 16384 2024-05-17T22:51:56.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.695+0000 7f3f0fd25b80 1 Processor -- start 2024-05-17T22:51:56.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.699+0000 7f3f0fd25b80 1 -- start start 2024-05-17T22:51:56.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.699+0000 7f3f0fd25b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570c72c39f0 0x5570c7441c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:56.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.699+0000 7f3f0fd25b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570c7442170 0x5570c7436eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:56.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.699+0000 7f3f0fd25b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570c7447aa0 0x5570c744bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:56.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.699+0000 7f3efaffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570c72c39f0 0x5570c7441c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:56.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.699+0000 7f3efaffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570c72c39f0 0x5570c7441c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:48204/0 (socket says 172.21.15.12:48204) 2024-05-17T22:51:56.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.699+0000 7f3efa7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570c7442170 0x5570c7436eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:56.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.699+0000 7f3efa7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570c7442170 0x5570c7436eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32948/0 (socket says 172.21.15.12:32948) 2024-05-17T22:51:56.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.699+0000 7f3efaffd700 1 -- 172.21.15.12:0/2319500553 learned_addr learned my addr 172.21.15.12:0/2319500553 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:51:56.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.699+0000 7f3f0fd25b80 1 -- 172.21.15.12:0/2319500553 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc1b4730 con 0x5570c7442170 2024-05-17T22:51:56.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.699+0000 7f3f0fd25b80 1 -- 172.21.15.12:0/2319500553 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc1b1fc0 con 0x5570c72c39f0 2024-05-17T22:51:56.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.699+0000 7f3f0fd25b80 1 -- 172.21.15.12:0/2319500553 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc1afe10 con 0x5570c7447aa0 2024-05-17T22:51:56.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.700+0000 7f3efb7fe700 1 --2- 172.21.15.12:0/2319500553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570c7447aa0 0x5570c744bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:56.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.704+0000 7f3efaffd700 1 -- 172.21.15.12:0/2319500553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570c7447aa0 msgr2=0x5570c744bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:51:56.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.704+0000 7f3efaffd700 1 --2- 172.21.15.12:0/2319500553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570c7447aa0 0x5570c744bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:56.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.704+0000 7f3efaffd700 1 -- 172.21.15.12:0/2319500553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570c7442170 msgr2=0x5570c7436eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:51:56.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.704+0000 7f3efaffd700 1 --2- 172.21.15.12:0/2319500553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570c7442170 0x5570c7436eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:56.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.704+0000 7f3efaffd700 1 -- 172.21.15.12:0/2319500553 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570c7449750 con 0x5570c72c39f0 2024-05-17T22:51:56.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.705+0000 7f3efaffd700 1 --2- 172.21.15.12:0/2319500553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570c72c39f0 0x5570c7441c30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00c010 tx=0x7f3eec003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b221ffbae3c44dd server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:51:56.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.706+0000 7f3efa7fc700 1 --2- 172.21.15.12:0/2319500553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570c7442170 0x5570c7436eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:51:56.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.710+0000 7f3ee0ff9700 1 -- 172.21.15.12:0/2319500553 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eec0a60d0 con 0x5570c72c39f0 2024-05-17T22:51:56.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.710+0000 7f3ee0ff9700 1 -- 172.21.15.12:0/2319500553 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3eec0b30e0 con 0x5570c72c39f0 2024-05-17T22:51:56.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.710+0000 7f3ee0ff9700 1 -- 172.21.15.12:0/2319500553 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eec0a6800 con 0x5570c72c39f0 2024-05-17T22:51:56.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.713+0000 7f3f0fd25b80 1 -- 172.21.15.12:0/2319500553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570c72c39f0 msgr2=0x5570c7441c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:51:56.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.713+0000 7f3f0fd25b80 1 --2- 172.21.15.12:0/2319500553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570c72c39f0 0x5570c7441c30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00c010 tx=0x7f3eec003f90 comp rx=0 tx=0).stop 2024-05-17T22:51:56.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.717+0000 7f3f0fd25b80 1 -- 172.21.15.12:0/2319500553 shutdown_connections 2024-05-17T22:51:56.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.717+0000 7f3f0fd25b80 1 --2- 172.21.15.12:0/2319500553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570c7447aa0 0x5570c744bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:56.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.717+0000 7f3f0fd25b80 1 --2- 172.21.15.12:0/2319500553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570c72c39f0 0x5570c7441c30 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:56.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.717+0000 7f3f0fd25b80 1 --2- 172.21.15.12:0/2319500553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570c7442170 0x5570c7436eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:56.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.717+0000 7f3f0fd25b80 1 -- 172.21.15.12:0/2319500553 >> 172.21.15.12:0/2319500553 conn(0x5570c7307f00 msgr2=0x5570c7448200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:51:56.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.721+0000 7f3f0fd25b80 1 -- 172.21.15.12:0/2319500553 shutdown_connections 2024-05-17T22:51:56.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.722+0000 7f3f0fd25b80 1 -- 172.21.15.12:0/2319500553 wait complete. 2024-05-17T22:51:56.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.722+0000 7f3f0fd25b80 1 -- start start 2024-05-17T22:51:56.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.723+0000 7f3f0fd25b80 1 Processor -- start 2024-05-17T22:51:56.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.723+0000 7f3f0fd25b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570c72c39f0 0x5570c744e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:56.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.723+0000 7f3f0fd25b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570c7442170 0x5570c744ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:56.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.723+0000 7f3f0fd25b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570c7447aa0 0x5570c74cbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:56.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.724+0000 7f3f0fd25b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc1b0480 con 0x5570c7447aa0 2024-05-17T22:51:56.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.724+0000 7f3f0fd25b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc1b18f0 con 0x5570c7442170 2024-05-17T22:51:56.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.724+0000 7f3f0fd25b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc1b0b50 con 0x5570c72c39f0 2024-05-17T22:51:56.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.724+0000 7f3f0fd25b80 10 client.0 _ll_register_callbacks cb 0x5570c72faf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:51:56.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.725+0000 7f3efa7fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570c7442170 0x5570c744ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:56.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.725+0000 7f3efaffd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570c72c39f0 0x5570c744e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:56.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.725+0000 7f3efa7fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570c7442170 0x5570c744ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:48218/0 (socket says 172.21.15.12:48218) 2024-05-17T22:51:56.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.725+0000 7f3efa7fc700 1 -- 172.21.15.12:0/3513884908 learned_addr learned my addr 172.21.15.12:0/3513884908 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:51:56.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.728+0000 7f3efb7fe700 1 --2- 172.21.15.12:0/3513884908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570c7447aa0 0x5570c74cbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:56.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.728+0000 7f3efa7fc700 1 -- 172.21.15.12:0/3513884908 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570c72c39f0 msgr2=0x5570c744e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:51:56.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.728+0000 7f3efa7fc700 1 --2- 172.21.15.12:0/3513884908 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570c72c39f0 0x5570c744e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:56.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.728+0000 7f3efa7fc700 1 -- 172.21.15.12:0/3513884908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570c7447aa0 msgr2=0x5570c74cbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:51:56.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.728+0000 7f3efa7fc700 1 --2- 172.21.15.12:0/3513884908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570c7447aa0 0x5570c74cbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:56.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.728+0000 7f3efa7fc700 1 -- 172.21.15.12:0/3513884908 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570c74cc410 con 0x5570c7442170 2024-05-17T22:51:56.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:56 smithi151 ceph-mon[36101]: pgmap v819: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 MiB/s wr, 563 op/s 2024-05-17T22:51:56.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.738+0000 7f3efa7fc700 1 --2- 172.21.15.12:0/3513884908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570c7442170 0x5570c744ebf0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000f780 tx=0x7f3ef000fa90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57699cb64fba1084 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:51:56.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.739+0000 7f3edeff5700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:51:56.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.740+0000 7f3edeff5700 1 -- 172.21.15.12:0/3513884908 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0021030 con 0x5570c7442170 2024-05-17T22:51:56.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.740+0000 7f3f0fd25b80 1 -- 172.21.15.12:0/3513884908 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5570c73721e0 con 0x5570c7442170 2024-05-17T22:51:56.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.740+0000 7f3edeff5700 1 -- 172.21.15.12:0/3513884908 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f3ef0004020 con 0x5570c7442170 2024-05-17T22:51:56.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.747+0000 7f3edeff5700 1 -- 172.21.15.12:0/3513884908 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3ef002b050 con 0x5570c7442170 2024-05-17T22:51:56.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.748+0000 7f3f0fd25b80 10 client.26090 fetch_fsmap learned FSMap version 39 2024-05-17T22:51:56.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.748+0000 7f3f0fd25b80 1 -- 172.21.15.12:0/3513884908 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5570c74cd5a0 con 0x5570c7442170 2024-05-17T22:51:56.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.752+0000 7f3edeff5700 1 -- 172.21.15.12:0/3513884908 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5570c73721e0 con 0x5570c7442170 2024-05-17T22:51:56.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.752+0000 7f3edeff5700 1 -- 172.21.15.12:0/3513884908 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f3ef000e4d0 con 0x5570c7442170 2024-05-17T22:51:56.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.752+0000 7f3f0fd25b80 10 client.26090 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T22:51:56.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.752+0000 7f3f0fd25b80 10 client.26090 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-17T22:51:56.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.752+0000 7f3f0fd25b80 20 client.26090 populate_metadata read hostname 'smithi012' 2024-05-17T22:51:56.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.752+0000 7f3f0fd25b80 1 --2- 172.21.15.12:0/3513884908 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x5570c74cdc50 0x5570c74d0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:51:56.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.752+0000 7f3f0fd25b80 4 client.26090 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:51:56.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.752+0000 7f3f0fd25b80 1 -- 172.21.15.12:0/3513884908 --> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5570c736b9c0 con 0x5570c74cdc50 2024-05-17T22:51:56.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.753+0000 7f3efaffd700 1 --2- 172.21.15.12:0/3513884908 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x5570c74cdc50 0x5570c74d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:51:56.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.754+0000 7f3efaffd700 1 --2- 172.21.15.12:0/3513884908 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x5570c74cdc50 0x5570c74d0070 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cc4b5a0a1cde86ef server_cookie=783691c3ac5f3236 in_seq=0 out_seq=0 2024-05-17T22:51:56.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.754+0000 7f3edeff5700 10 client.26090 ms_handle_connect on v2:172.21.15.151:6838/2338353849 2024-05-17T22:51:56.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.755+0000 7f3edeff5700 1 -- 172.21.15.12:0/3513884908 <== mds.2 v2:172.21.15.151:6838/2338353849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5570c736b9c0 con 0x5570c74cdc50 2024-05-17T22:51:56.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.755+0000 7f3edeff5700 10 client.26090 handle_command_reply: tid=0 2024-05-17T22:51:56.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.755+0000 7f3f0fd25b80 1 client.26090 shutdown 2024-05-17T22:51:56.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.760+0000 7f3f0fd25b80 1 -- 172.21.15.12:0/3513884908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570c7442170 msgr2=0x5570c744ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:51:56.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.760+0000 7f3f0fd25b80 1 --2- 172.21.15.12:0/3513884908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570c7442170 0x5570c744ebf0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000f780 tx=0x7f3ef000fa90 comp rx=0 tx=0).stop 2024-05-17T22:51:56.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.761+0000 7f3f0fd25b80 1 -- 172.21.15.12:0/3513884908 shutdown_connections 2024-05-17T22:51:56.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.761+0000 7f3f0fd25b80 1 --2- 172.21.15.12:0/3513884908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570c7447aa0 0x5570c74cbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:56.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.761+0000 7f3f0fd25b80 1 --2- 172.21.15.12:0/3513884908 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x5570c74cdc50 0x5570c74d0070 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:56.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.761+0000 7f3f0fd25b80 1 --2- 172.21.15.12:0/3513884908 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570c72c39f0 0x5570c744e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:56.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.761+0000 7f3f0fd25b80 1 --2- 172.21.15.12:0/3513884908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570c7442170 0x5570c744ebf0 secure :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000f780 tx=0x7f3ef000fa90 comp rx=0 tx=0).stop 2024-05-17T22:51:56.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.761+0000 7f3f0fd25b80 1 -- 172.21.15.12:0/3513884908 >> 172.21.15.12:0/3513884908 conn(0x5570c7307f00 msgr2=0x5570c74453e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:51:56.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.761+0000 7f3edeff5700 0 client.26090 ms_handle_reset on v2:172.21.15.151:6838/2338353849 2024-05-17T22:51:56.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.761+0000 7f3f0fd25b80 1 -- 172.21.15.12:0/3513884908 shutdown_connections 2024-05-17T22:51:56.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.762+0000 7f3f0fd25b80 1 -- 172.21.15.12:0/3513884908 wait complete. 2024-05-17T22:51:56.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.763+0000 7f3f0fd25b80 20 client.26090 trim_cache size 0 max 16384 2024-05-17T22:51:56.769 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:51:56.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.771+0000 7f3f0134a700 1 -- 172.21.15.12:0/1296094732 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f3ee40b2900 msgr2=0x7f3ee40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:51:56.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.771+0000 7f3f0134a700 1 --2- 172.21.15.12:0/1296094732 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f3ee40b2900 0x7f3ee40b4e10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4020000 tx=0x7f3ef401e000 comp rx=0 tx=0).stop 2024-05-17T22:51:56.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.771+0000 7f3f0134a700 1 -- 172.21.15.12:0/1296094732 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3efc104e00 msgr2=0x7f3efc102af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:51:56.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.771+0000 7f3f0134a700 1 --2- 172.21.15.12:0/1296094732 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3efc104e00 0x7f3efc102af0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3eec0059d0 tx=0x7f3eec0038b0 comp rx=0 tx=0).stop 2024-05-17T22:51:56.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.774+0000 7f3f0134a700 1 -- 172.21.15.12:0/1296094732 shutdown_connections 2024-05-17T22:51:56.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.774+0000 7f3f0134a700 1 --2- 172.21.15.12:0/1296094732 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f3ee40b2900 0x7f3ee40b4e10 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:56.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.774+0000 7f3f0134a700 1 --2- 172.21.15.12:0/1296094732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3efc10e230 0x7f3efc103570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:56.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.774+0000 7f3f0134a700 1 --2- 172.21.15.12:0/1296094732 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3efc104e00 0x7f3efc102af0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:56.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.774+0000 7f3f0134a700 1 --2- 172.21.15.12:0/1296094732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3efc105750 0x7f3efc103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:51:56.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.774+0000 7f3f0134a700 1 -- 172.21.15.12:0/1296094732 >> 172.21.15.12:0/1296094732 conn(0x7f3efc079bd0 msgr2=0x7f3efc0ffb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:51:56.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.780+0000 7f3f0134a700 1 -- 172.21.15.12:0/1296094732 shutdown_connections 2024-05-17T22:51:56.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:51:56.780+0000 7f3f0134a700 1 -- 172.21.15.12:0/1296094732 wait complete. 2024-05-17T22:51:57.508 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-17T22:51:58.013 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:51:58.463 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:58 smithi012 ceph-mon[31632]: pgmap v820: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.8 MiB/s wr, 387 op/s 2024-05-17T22:51:58.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:58 smithi096 ceph-mon[36065]: pgmap v820: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.8 MiB/s wr, 387 op/s 2024-05-17T22:51:58.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:58 smithi151 ceph-mon[36101]: pgmap v820: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.8 MiB/s wr, 387 op/s 2024-05-17T22:51:59.510 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:51:59 smithi012 ceph-mon[31632]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:51:59.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:51:59 smithi096 ceph-mon[36065]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:51:59.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:51:59 smithi151 ceph-mon[36101]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:52:00.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.307+0000 7f0fa92ca700 1 -- 172.21.15.12:0/4230919546 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fa410b180 msgr2=0x7f0fa411d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:00.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.307+0000 7f0fa92ca700 1 --2- 172.21.15.12:0/4230919546 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fa410b180 0x7f0fa411d010 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f0f94009990 tx=0x7f0f94009ca0 comp rx=0 tx=0).stop 2024-05-17T22:52:00.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.308+0000 7f0fa92ca700 1 -- 172.21.15.12:0/4230919546 shutdown_connections 2024-05-17T22:52:00.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.308+0000 7f0fa92ca700 1 --2- 172.21.15.12:0/4230919546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fa411d550 0x7f0fa411f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:00.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.308+0000 7f0fa92ca700 1 --2- 172.21.15.12:0/4230919546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0fa410a830 0x7f0fa410ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:00.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.308+0000 7f0fa92ca700 1 --2- 172.21.15.12:0/4230919546 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fa410b180 0x7f0fa411d010 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:00.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.308+0000 7f0fa92ca700 1 -- 172.21.15.12:0/4230919546 >> 172.21.15.12:0/4230919546 conn(0x7f0fa406efc0 msgr2=0x7f0fa406f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:52:00.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.309+0000 7f0fa92ca700 1 -- 172.21.15.12:0/4230919546 shutdown_connections 2024-05-17T22:52:00.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.309+0000 7f0fa92ca700 1 -- 172.21.15.12:0/4230919546 wait complete. 2024-05-17T22:52:00.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.310+0000 7f0fa92ca700 1 Processor -- start 2024-05-17T22:52:00.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.310+0000 7f0fa92ca700 1 -- start start 2024-05-17T22:52:00.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.311+0000 7f0fa92ca700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fa410a830 0x7f0fa411a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:00.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.311+0000 7f0fa92ca700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0fa410b180 0x7f0fa41128c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:00.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.311+0000 7f0fa92ca700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fa411d550 0x7f0fa4112e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:00.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.311+0000 7f0fa92ca700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa4122ca0 con 0x7f0fa411d550 2024-05-17T22:52:00.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.311+0000 7f0fa92ca700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa4122b20 con 0x7f0fa410a830 2024-05-17T22:52:00.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.311+0000 7f0fa92ca700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa4122e20 con 0x7f0fa410b180 2024-05-17T22:52:00.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.311+0000 7f0fa37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fa411d550 0x7f0fa4112e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:00.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.311+0000 7f0fa37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fa411d550 0x7f0fa4112e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53504/0 (socket says 172.21.15.12:53504) 2024-05-17T22:52:00.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.311+0000 7f0fa37fe700 1 -- 172.21.15.12:0/3108174321 learned_addr learned my addr 172.21.15.12:0/3108174321 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:52:00.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.311+0000 7f0fa2ffd700 1 --2- 172.21.15.12:0/3108174321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fa410a830 0x7f0fa411a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:00.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.312+0000 7f0fa2ffd700 1 -- 172.21.15.12:0/3108174321 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0fa410b180 msgr2=0x7f0fa41128c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-17T22:52:00.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.312+0000 7f0fa2ffd700 1 --2- 172.21.15.12:0/3108174321 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0fa410b180 0x7f0fa41128c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:00.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.312+0000 7f0fa2ffd700 1 -- 172.21.15.12:0/3108174321 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fa411d550 msgr2=0x7f0fa4112e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:00.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.312+0000 7f0fa2ffd700 1 --2- 172.21.15.12:0/3108174321 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fa411d550 0x7f0fa4112e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:00.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.312+0000 7f0fa2ffd700 1 -- 172.21.15.12:0/3108174321 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa4113500 con 0x7f0fa410a830 2024-05-17T22:52:00.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.312+0000 7f0fa2ffd700 1 --2- 172.21.15.12:0/3108174321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fa410a830 0x7f0fa411a850 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c007ec0 tx=0x7f0f9c00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:52:00.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.317+0000 7f0f8bfff700 1 -- 172.21.15.12:0/3108174321 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c00ddd0 con 0x7f0fa410a830 2024-05-17T22:52:00.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.317+0000 7f0f8bfff700 1 -- 172.21.15.12:0/3108174321 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f9c004bb0 con 0x7f0fa410a830 2024-05-17T22:52:00.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.317+0000 7f0f8bfff700 1 -- 172.21.15.12:0/3108174321 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c015400 con 0x7f0fa410a830 2024-05-17T22:52:00.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.317+0000 7f0fa92ca700 1 -- 172.21.15.12:0/3108174321 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fa41a93c0 con 0x7f0fa410a830 2024-05-17T22:52:00.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.317+0000 7f0fa92ca700 1 -- 172.21.15.12:0/3108174321 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fa41a98b0 con 0x7f0fa410a830 2024-05-17T22:52:00.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.323+0000 7f0fb7ca5b80 1 Processor -- start 2024-05-17T22:52:00.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.323+0000 7f0fb7ca5b80 1 -- start start 2024-05-17T22:52:00.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.324+0000 7f0fb7ca5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6bc979f0 0x561a6be1aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:00.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.324+0000 7f0fb7ca5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a6be1b400 0x561a6be155c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:00.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.324+0000 7f0fb7ca5b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a6be15b00 0x561a6be15f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:00.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.324+0000 7f0fb7ca5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x561a6bd440d0 con 0x561a6bc979f0 2024-05-17T22:52:00.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.324+0000 7f0fb7ca5b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561a6bd33c70 con 0x561a6be15b00 2024-05-17T22:52:00.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.324+0000 7f0fb7ca5b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561a6bd37cb0 con 0x561a6be1b400 2024-05-17T22:52:00.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.325+0000 7f0fa37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6bc979f0 0x561a6be1aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:00.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.325+0000 7f0fa37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6bc979f0 0x561a6be1aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53520/0 (socket says 172.21.15.12:53520) 2024-05-17T22:52:00.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.325+0000 7f0fa37fe700 1 -- 172.21.15.12:0/1344084125 learned_addr learned my addr 172.21.15.12:0/1344084125 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:52:00.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.326+0000 7f0fa37fe700 1 -- 172.21.15.12:0/1344084125 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a6be1b400 msgr2=0x561a6be155c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T22:52:00.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.326+0000 7f0fa37fe700 1 --2- 172.21.15.12:0/1344084125 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a6be1b400 0x561a6be155c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:00.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.326+0000 7f0fa37fe700 1 -- 172.21.15.12:0/1344084125 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a6be15b00 msgr2=0x561a6be15f10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T22:52:00.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.326+0000 7f0fa37fe700 1 --2- 172.21.15.12:0/1344084125 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a6be15b00 0x561a6be15f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:00.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.326+0000 7f0fa37fe700 1 -- 172.21.15.12:0/1344084125 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a6bccd5c0 con 0x561a6bc979f0 2024-05-17T22:52:00.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.327+0000 7f0fa37fe700 1 --2- 172.21.15.12:0/1344084125 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6bc979f0 0x561a6be1aec0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f0f9802dd90 tx=0x7f0f980326c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39bfefd5e93717c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:52:00.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.327+0000 7f0f8bfff700 1 -- 172.21.15.12:0/3108174321 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0f9c005020 con 0x7f0fa410a830 2024-05-17T22:52:00.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.328+0000 7f0f8bfff700 1 --2- 172.21.15.12:0/3108174321 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f0f8c0b2a90 0x7f0f8c0b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:00.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.329+0000 7f0f8bfff700 1 -- 172.21.15.12:0/3108174321 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f0f9c0e1d90 con 0x7f0fa410a830 2024-05-17T22:52:00.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.329+0000 7f0f897fa700 1 -- 172.21.15.12:0/1344084125 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f98004830 con 0x561a6bc979f0 2024-05-17T22:52:00.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.329+0000 7f0f897fa700 1 -- 172.21.15.12:0/1344084125 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f98005020 con 0x561a6bc979f0 2024-05-17T22:52:00.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.329+0000 7f0f897fa700 1 -- 172.21.15.12:0/1344084125 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9803c6c0 con 0x561a6bc979f0 2024-05-17T22:52:00.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.330+0000 7f0fb7ca5b80 1 -- 172.21.15.12:0/1344084125 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6bc979f0 msgr2=0x561a6be1aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:00.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.330+0000 7f0fb7ca5b80 1 --2- 172.21.15.12:0/1344084125 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6bc979f0 0x561a6be1aec0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f0f9802dd90 tx=0x7f0f980326c0 comp rx=0 tx=0).stop 2024-05-17T22:52:00.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.331+0000 7f0fb7ca5b80 1 -- 172.21.15.12:0/1344084125 shutdown_connections 2024-05-17T22:52:00.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.331+0000 7f0fb7ca5b80 1 --2- 172.21.15.12:0/1344084125 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a6be15b00 0x561a6be15f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:00.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.331+0000 7f0fb7ca5b80 1 --2- 172.21.15.12:0/1344084125 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6bc979f0 0x561a6be1aec0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:00.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.331+0000 7f0fb7ca5b80 1 --2- 172.21.15.12:0/1344084125 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a6be1b400 0x561a6be155c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:00.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.331+0000 7f0fb7ca5b80 1 -- 172.21.15.12:0/1344084125 >> 172.21.15.12:0/1344084125 conn(0x561a6bcf77d0 msgr2=0x561a6be101c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:52:00.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.331+0000 7f0fa2ffd700 1 --2- 172.21.15.12:0/3108174321 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f0f8c0b2a90 0x7f0f8c0b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:00.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.333+0000 7f0fb7ca5b80 1 -- 172.21.15.12:0/1344084125 shutdown_connections 2024-05-17T22:52:00.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.333+0000 7f0fa2ffd700 1 --2- 172.21.15.12:0/3108174321 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f0f8c0b2a90 0x7f0f8c0b4fa0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c0a8f50 tx=0x7f0f9c00df70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:52:00.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.337+0000 7f0fb7ca5b80 1 -- 172.21.15.12:0/1344084125 wait complete. 2024-05-17T22:52:00.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.337+0000 7f0fb7ca5b80 1 -- start start 2024-05-17T22:52:00.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.337+0000 7f0fb7ca5b80 1 Processor -- start 2024-05-17T22:52:00.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.338+0000 7f0fb7ca5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a6bc979f0 0x561a6be1ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:00.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.338+0000 7f0fb7ca5b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a6be15b00 0x561a6be1f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:00.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.338+0000 7f0fb7ca5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6be1b400 0x561a6be1f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:00.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.338+0000 7f0fb7ca5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x561a6bc3b150 con 0x561a6be1b400 2024-05-17T22:52:00.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.338+0000 7f0fb7ca5b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561a6bcd5940 con 0x561a6be15b00 2024-05-17T22:52:00.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.338+0000 7f0fb7ca5b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561a6be21dd0 con 0x561a6bc979f0 2024-05-17T22:52:00.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.338+0000 7f0fb7ca5b80 10 client.0 _ll_register_callbacks cb 0x561a6bccab50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:52:00.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.344+0000 7f0fa27fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a6be15b00 0x561a6be1f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:00.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.344+0000 7f0fa37fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a6bc979f0 0x561a6be1ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:00.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.345+0000 7f0fa37fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a6bc979f0 0x561a6be1ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:34324/0 (socket says 172.21.15.12:34324) 2024-05-17T22:52:00.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.345+0000 7f0fa37fe700 1 -- 172.21.15.12:0/1753728930 learned_addr learned my addr 172.21.15.12:0/1753728930 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:52:00.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.345+0000 7f0fa37fe700 1 --2- 172.21.15.12:0/1753728930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6be1b400 0x561a6be1f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:00.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.345+0000 7f0fa37fe700 1 -- 172.21.15.12:0/1753728930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a6be15b00 msgr2=0x561a6be1f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:00.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.345+0000 7f0fa37fe700 1 --2- 172.21.15.12:0/1753728930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a6be15b00 0x561a6be1f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:00.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.345+0000 7f0fa37fe700 1 -- 172.21.15.12:0/1753728930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6be1b400 msgr2=0x561a6be1f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:00.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.345+0000 7f0fa37fe700 1 --2- 172.21.15.12:0/1753728930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6be1b400 0x561a6be1f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:00.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.345+0000 7f0fa37fe700 1 -- 172.21.15.12:0/1753728930 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a6bc84480 con 0x561a6bc979f0 2024-05-17T22:52:00.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.347+0000 7f0fa37fe700 1 --2- 172.21.15.12:0/1753728930 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a6bc979f0 0x561a6be1ead0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0f9802dd60 tx=0x7f0f9802dee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d5494a4a2858190 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:52:00.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.347+0000 7f0f86ff5700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:52:00.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.347+0000 7f0f86ff5700 1 -- 172.21.15.12:0/1753728930 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f98045040 con 0x561a6bc979f0 2024-05-17T22:52:00.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.347+0000 7f0f86ff5700 1 -- 172.21.15.12:0/1753728930 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f0f9803ccd0 con 0x561a6bc979f0 2024-05-17T22:52:00.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.347+0000 7f0fb7ca5b80 1 -- 172.21.15.12:0/1753728930 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f98005020 con 0x561a6bc979f0 2024-05-17T22:52:00.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.349+0000 7f0f86ff5700 1 -- 172.21.15.12:0/1753728930 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f0f98054050 con 0x561a6bc979f0 2024-05-17T22:52:00.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.349+0000 7f0fb7ca5b80 10 client.26041 fetch_fsmap learned FSMap version 40 2024-05-17T22:52:00.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.349+0000 7f0fb7ca5b80 1 -- 172.21.15.12:0/1753728930 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561a6be2c980 con 0x561a6bc979f0 2024-05-17T22:52:00.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.352+0000 7f0f86ff5700 1 -- 172.21.15.12:0/1753728930 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f98005020 con 0x561a6bc979f0 2024-05-17T22:52:00.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.352+0000 7f0f86ff5700 1 -- 172.21.15.12:0/1753728930 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7f0f98030040 con 0x561a6bc979f0 2024-05-17T22:52:00.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.352+0000 7f0fb7ca5b80 10 client.26041 fetch_fsmap finished waiting for FSMap version 40 2024-05-17T22:52:00.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.352+0000 7f0fb7ca5b80 10 client.26041 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-17T22:52:00.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.352+0000 7f0fb7ca5b80 20 client.26041 populate_metadata read hostname 'smithi012' 2024-05-17T22:52:00.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.352+0000 7f0fb7ca5b80 1 --2- 172.21.15.12:0/1753728930 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x561a6be2f2c0 0x561a6be31750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:00.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.352+0000 7f0fb7ca5b80 4 client.26041 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:52:00.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.352+0000 7f0fb7ca5b80 1 -- 172.21.15.12:0/1753728930 --> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a6bd46020 con 0x561a6be2f2c0 2024-05-17T22:52:00.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.355+0000 7f0fa27fc700 1 --2- 172.21.15.12:0/1753728930 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x561a6be2f2c0 0x561a6be31750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:00.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.356+0000 7f0f86ff5700 10 client.26041 ms_handle_connect on v2:172.21.15.96:6836/4222574338 2024-05-17T22:52:00.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.356+0000 7f0fa27fc700 1 --2- 172.21.15.12:0/1753728930 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x561a6be2f2c0 0x561a6be31750 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=16a6da3d6de8e72a server_cookie=abf55d6cc3be4470 in_seq=0 out_seq=0 2024-05-17T22:52:00.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.357+0000 7f0f86ff5700 1 -- 172.21.15.12:0/1753728930 <== mds.1 v2:172.21.15.96:6836/4222574338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561a6bd46020 con 0x561a6be2f2c0 2024-05-17T22:52:00.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.357+0000 7f0f86ff5700 10 client.26041 handle_command_reply: tid=0 2024-05-17T22:52:00.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.357+0000 7f0fb7ca5b80 1 client.26041 shutdown 2024-05-17T22:52:00.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.358+0000 7f0fb7ca5b80 1 -- 172.21.15.12:0/1753728930 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a6bc979f0 msgr2=0x561a6be1ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:00.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.358+0000 7f0fb7ca5b80 1 --2- 172.21.15.12:0/1753728930 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a6bc979f0 0x561a6be1ead0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0f9802dd60 tx=0x7f0f9802dee0 comp rx=0 tx=0).stop 2024-05-17T22:52:00.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.358+0000 7f0fb7ca5b80 1 -- 172.21.15.12:0/1753728930 shutdown_connections 2024-05-17T22:52:00.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.358+0000 7f0fb7ca5b80 1 --2- 172.21.15.12:0/1753728930 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x561a6be2f2c0 0x561a6be31750 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:00.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.358+0000 7f0fb7ca5b80 1 --2- 172.21.15.12:0/1753728930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6be1b400 0x561a6be1f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:00.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.358+0000 7f0fb7ca5b80 1 --2- 172.21.15.12:0/1753728930 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a6bc979f0 0x561a6be1ead0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:00.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.358+0000 7f0fb7ca5b80 1 --2- 172.21.15.12:0/1753728930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a6be15b00 0x561a6be1f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:00.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.358+0000 7f0fb7ca5b80 1 -- 172.21.15.12:0/1753728930 >> 172.21.15.12:0/1753728930 conn(0x561a6bcf77d0 msgr2=0x561a6be10160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:52:00.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.358+0000 7f0f86ff5700 0 client.26041 ms_handle_reset on v2:172.21.15.96:6836/4222574338 2024-05-17T22:52:00.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.358+0000 7f0fb7ca5b80 1 -- 172.21.15.12:0/1753728930 shutdown_connections 2024-05-17T22:52:00.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.359+0000 7f0fb7ca5b80 1 -- 172.21.15.12:0/1753728930 wait complete. 2024-05-17T22:52:00.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.359+0000 7f0fb7ca5b80 20 client.26041 trim_cache size 0 max 16384 2024-05-17T22:52:00.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.378+0000 7f0fb7ca5b80 1 Processor -- start 2024-05-17T22:52:00.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.378+0000 7f0fb7ca5b80 1 -- start start 2024-05-17T22:52:00.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.379+0000 7f0fb7ca5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a6bc979f0 0x561a6be1d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:00.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.379+0000 7f0fb7ca5b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a6be15b00 0x561a6be1d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:00.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.379+0000 7f0fb7ca5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6be1b400 0x561a6be19e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:00.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.379+0000 7f0fb7ca5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0f900114b0 con 0x561a6be1b400 2024-05-17T22:52:00.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.379+0000 7f0fb7ca5b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9000ed40 con 0x561a6be15b00 2024-05-17T22:52:00.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.379+0000 7f0fb7ca5b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9000cb90 con 0x561a6bc979f0 2024-05-17T22:52:00.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.379+0000 7f0fa27fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a6be15b00 0x561a6be1d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:00.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.379+0000 7f0fa27fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a6be15b00 0x561a6be1d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:47826/0 (socket says 172.21.15.12:47826) 2024-05-17T22:52:00.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.379+0000 7f0fa27fc700 1 -- 172.21.15.12:0/1304806311 learned_addr learned my addr 172.21.15.12:0/1304806311 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:52:00.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.379+0000 7f0fa37fe700 1 --2- 172.21.15.12:0/1304806311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a6bc979f0 0x561a6be1d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:00.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.379+0000 7f0fa37fe700 1 --2- 172.21.15.12:0/1304806311 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6be1b400 0x561a6be19e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:00.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.380+0000 7f0fa27fc700 1 -- 172.21.15.12:0/1304806311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a6bc979f0 msgr2=0x561a6be1d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:00.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.380+0000 7f0fa27fc700 1 --2- 172.21.15.12:0/1304806311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a6bc979f0 0x561a6be1d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:00.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.380+0000 7f0fa27fc700 1 -- 172.21.15.12:0/1304806311 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6be1b400 msgr2=0x561a6be19e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:00.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.380+0000 7f0fa27fc700 1 --2- 172.21.15.12:0/1304806311 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6be1b400 0x561a6be19e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:00.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.380+0000 7f0fa27fc700 1 -- 172.21.15.12:0/1304806311 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a6be0cc20 con 0x561a6be15b00 2024-05-17T22:52:00.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.380+0000 7f0fa37fe700 1 --2- 172.21.15.12:0/1304806311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a6bc979f0 0x561a6be1d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:52:00.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.380+0000 7f0fa27fc700 1 --2- 172.21.15.12:0/1304806311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a6be15b00 0x561a6be1d900 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0fa411ca30 tx=0x7f0f940098c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b40f5a6767b5de4e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:52:00.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.380+0000 7f0f88ff9700 1 -- 172.21.15.12:0/1304806311 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f94026020 con 0x561a6be15b00 2024-05-17T22:52:00.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.380+0000 7f0f88ff9700 1 -- 172.21.15.12:0/1304806311 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f94022420 con 0x561a6be15b00 2024-05-17T22:52:00.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.380+0000 7f0f88ff9700 1 -- 172.21.15.12:0/1304806311 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f940039d0 con 0x561a6be15b00 2024-05-17T22:52:00.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.381+0000 7f0fb7ca5b80 1 -- 172.21.15.12:0/1304806311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a6be15b00 msgr2=0x561a6be1d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:00.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.381+0000 7f0fb7ca5b80 1 --2- 172.21.15.12:0/1304806311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a6be15b00 0x561a6be1d900 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0fa411ca30 tx=0x7f0f940098c0 comp rx=0 tx=0).stop 2024-05-17T22:52:00.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.381+0000 7f0fb7ca5b80 1 -- 172.21.15.12:0/1304806311 shutdown_connections 2024-05-17T22:52:00.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.381+0000 7f0fb7ca5b80 1 --2- 172.21.15.12:0/1304806311 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6be1b400 0x561a6be19e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:00.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.381+0000 7f0fb7ca5b80 1 --2- 172.21.15.12:0/1304806311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a6bc979f0 0x561a6be1d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:00.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.381+0000 7f0fb7ca5b80 1 --2- 172.21.15.12:0/1304806311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a6be15b00 0x561a6be1d900 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:00.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.381+0000 7f0fb7ca5b80 1 -- 172.21.15.12:0/1304806311 >> 172.21.15.12:0/1304806311 conn(0x561a6bcf77d0 msgr2=0x561a6be312d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:52:00.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.382+0000 7f0fb7ca5b80 1 -- 172.21.15.12:0/1304806311 shutdown_connections 2024-05-17T22:52:00.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.382+0000 7f0fb7ca5b80 1 -- 172.21.15.12:0/1304806311 wait complete. 2024-05-17T22:52:00.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.382+0000 7f0fb7ca5b80 1 -- start start 2024-05-17T22:52:00.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.382+0000 7f0fb7ca5b80 1 Processor -- start 2024-05-17T22:52:00.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.383+0000 7f0fb7ca5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a6bc979f0 0x561a6be270f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:00.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.383+0000 7f0fb7ca5b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6be15b00 0x561a6be27630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:00.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.383+0000 7f0fb7ca5b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a6be1b400 0x561a6be1b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:00.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.383+0000 7f0fb7ca5b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9000d200 con 0x561a6be15b00 2024-05-17T22:52:00.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.383+0000 7f0fb7ca5b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9000e670 con 0x561a6be1b400 2024-05-17T22:52:00.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.383+0000 7f0fa27fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6be15b00 0x561a6be27630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:00.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.383+0000 7f0fa27fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6be15b00 0x561a6be27630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53554/0 (socket says 172.21.15.12:53554) 2024-05-17T22:52:00.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.383+0000 7f0fa27fc700 1 -- 172.21.15.12:0/1055779835 learned_addr learned my addr 172.21.15.12:0/1055779835 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:52:00.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.383+0000 7f0fa37fe700 1 --2- 172.21.15.12:0/1055779835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a6be1b400 0x561a6be1b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:00.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.383+0000 7f0fb7ca5b80 1 -- 172.21.15.12:0/1055779835 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9000d8d0 con 0x561a6bc979f0 2024-05-17T22:52:00.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.383+0000 7f0fb7ca5b80 10 client.0 _ll_register_callbacks cb 0x561a6bccab50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:52:00.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.383+0000 7f0fa37fe700 1 --2- 172.21.15.12:0/1055779835 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a6bc979f0 0x561a6be270f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:00.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.384+0000 7f0fa27fc700 1 -- 172.21.15.12:0/1055779835 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a6bc979f0 msgr2=0x561a6be270f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:00.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.384+0000 7f0fa27fc700 1 --2- 172.21.15.12:0/1055779835 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a6bc979f0 0x561a6be270f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:00.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.384+0000 7f0fa27fc700 1 -- 172.21.15.12:0/1055779835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a6be1b400 msgr2=0x561a6be1b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:00.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.384+0000 7f0fa27fc700 1 --2- 172.21.15.12:0/1055779835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a6be1b400 0x561a6be1b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:00.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.384+0000 7f0fa27fc700 1 -- 172.21.15.12:0/1055779835 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a6be19800 con 0x561a6be15b00 2024-05-17T22:52:00.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.384+0000 7f0fa37fe700 1 --2- 172.21.15.12:0/1055779835 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a6bc979f0 0x561a6be270f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:52:00.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.384+0000 7f0fa27fc700 1 --2- 172.21.15.12:0/1055779835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6be15b00 0x561a6be27630 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0f94005b40 tx=0x7f0f940225d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f02f3ae99b2e03 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:52:00.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.384+0000 7f0f86ff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:52:00.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.384+0000 7f0f86ff5700 1 -- 172.21.15.12:0/1055779835 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f94026070 con 0x561a6be15b00 2024-05-17T22:52:00.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.384+0000 7f0fb7ca5b80 1 -- 172.21.15.12:0/1055779835 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f94022420 con 0x561a6be15b00 2024-05-17T22:52:00.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.384+0000 7f0f86ff5700 1 -- 172.21.15.12:0/1055779835 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f0f94005100 con 0x561a6be15b00 2024-05-17T22:52:00.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.385+0000 7f0f86ff5700 1 -- 172.21.15.12:0/1055779835 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f0f9403e050 con 0x561a6be15b00 2024-05-17T22:52:00.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.385+0000 7f0fb7ca5b80 10 client.16395 fetch_fsmap learned FSMap version 40 2024-05-17T22:52:00.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.385+0000 7f0fb7ca5b80 1 -- 172.21.15.12:0/1055779835 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561a6be98fe0 con 0x561a6be15b00 2024-05-17T22:52:00.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.386+0000 7f0f86ff5700 1 -- 172.21.15.12:0/1055779835 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f94022420 con 0x561a6be15b00 2024-05-17T22:52:00.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.386+0000 7f0f86ff5700 1 -- 172.21.15.12:0/1055779835 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7f0f9400b700 con 0x561a6be15b00 2024-05-17T22:52:00.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.386+0000 7f0fb7ca5b80 10 client.16395 fetch_fsmap finished waiting for FSMap version 40 2024-05-17T22:52:00.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.386+0000 7f0fb7ca5b80 10 client.16395 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-17T22:52:00.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.386+0000 7f0fb7ca5b80 20 client.16395 populate_metadata read hostname 'smithi012' 2024-05-17T22:52:00.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.386+0000 7f0fb7ca5b80 1 --2- 172.21.15.12:0/1055779835 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x561a6be99660 0x561a6be9ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:00.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.386+0000 7f0fb7ca5b80 4 client.16395 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:52:00.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.386+0000 7f0fb7ca5b80 1 -- 172.21.15.12:0/1055779835 --> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561a6bd3f800 con 0x561a6be99660 2024-05-17T22:52:00.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.386+0000 7f0fa37fe700 1 --2- 172.21.15.12:0/1055779835 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x561a6be99660 0x561a6be9ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:00.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.387+0000 7f0f86ff5700 10 client.16395 ms_handle_connect on v2:172.21.15.96:6836/4222574338 2024-05-17T22:52:00.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.387+0000 7f0fa37fe700 1 --2- 172.21.15.12:0/1055779835 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x561a6be99660 0x561a6be9ba80 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e6cd561fca70c27c server_cookie=588a82b7f8ef529b in_seq=0 out_seq=0 2024-05-17T22:52:00.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.387+0000 7f0f86ff5700 1 -- 172.21.15.12:0/1055779835 <== mds.1 v2:172.21.15.96:6836/4222574338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561a6bd3f800 con 0x561a6be99660 2024-05-17T22:52:00.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.387+0000 7f0f86ff5700 10 client.16395 handle_command_reply: tid=0 2024-05-17T22:52:00.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.387+0000 7f0fb7ca5b80 1 client.16395 shutdown 2024-05-17T22:52:00.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.387+0000 7f0fb7ca5b80 1 -- 172.21.15.12:0/1055779835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6be15b00 msgr2=0x561a6be27630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:00.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.387+0000 7f0fb7ca5b80 1 --2- 172.21.15.12:0/1055779835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6be15b00 0x561a6be27630 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0f94005b40 tx=0x7f0f940225d0 comp rx=0 tx=0).stop 2024-05-17T22:52:00.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.387+0000 7f0fb7ca5b80 1 -- 172.21.15.12:0/1055779835 shutdown_connections 2024-05-17T22:52:00.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.387+0000 7f0fb7ca5b80 1 --2- 172.21.15.12:0/1055779835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a6be1b400 0x561a6be1b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:00.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.387+0000 7f0fb7ca5b80 1 --2- 172.21.15.12:0/1055779835 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a6bc979f0 0x561a6be270f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:00.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.387+0000 7f0fb7ca5b80 1 --2- 172.21.15.12:0/1055779835 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x561a6be99660 0x561a6be9ba80 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:00.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.387+0000 7f0fb7ca5b80 1 --2- 172.21.15.12:0/1055779835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561a6be15b00 0x561a6be27630 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:00.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.387+0000 7f0fb7ca5b80 1 -- 172.21.15.12:0/1055779835 >> 172.21.15.12:0/1055779835 conn(0x561a6bcf77d0 msgr2=0x561a6be19e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:52:00.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.387+0000 7f0f86ff5700 0 client.16395 ms_handle_reset on v2:172.21.15.96:6836/4222574338 2024-05-17T22:52:00.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.387+0000 7f0fb7ca5b80 1 -- 172.21.15.12:0/1055779835 shutdown_connections 2024-05-17T22:52:00.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.387+0000 7f0fb7ca5b80 1 -- 172.21.15.12:0/1055779835 wait complete. 2024-05-17T22:52:00.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.388+0000 7f0fb7ca5b80 20 client.16395 trim_cache size 0 max 16384 2024-05-17T22:52:00.395 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:52:00.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.388+0000 7f0fa92ca700 1 -- 172.21.15.12:0/3108174321 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f0f8c0b2a90 msgr2=0x7f0f8c0b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:00.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.388+0000 7f0fa92ca700 1 --2- 172.21.15.12:0/3108174321 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f0f8c0b2a90 0x7f0f8c0b4fa0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c0a8f50 tx=0x7f0f9c00df70 comp rx=0 tx=0).stop 2024-05-17T22:52:00.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.388+0000 7f0fa92ca700 1 -- 172.21.15.12:0/3108174321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fa410a830 msgr2=0x7f0fa411a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:00.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.388+0000 7f0fa92ca700 1 --2- 172.21.15.12:0/3108174321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fa410a830 0x7f0fa411a850 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c007ec0 tx=0x7f0f9c00d3b0 comp rx=0 tx=0).stop 2024-05-17T22:52:00.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.388+0000 7f0fa92ca700 1 -- 172.21.15.12:0/3108174321 shutdown_connections 2024-05-17T22:52:00.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.388+0000 7f0fa92ca700 1 --2- 172.21.15.12:0/3108174321 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f0f8c0b2a90 0x7f0f8c0b4fa0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:00.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.388+0000 7f0fa92ca700 1 --2- 172.21.15.12:0/3108174321 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fa411d550 0x7f0fa4112e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:00.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.388+0000 7f0fa92ca700 1 --2- 172.21.15.12:0/3108174321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fa410a830 0x7f0fa411a850 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:00.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.388+0000 7f0fa92ca700 1 --2- 172.21.15.12:0/3108174321 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0fa410b180 0x7f0fa41128c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:00.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.388+0000 7f0fa92ca700 1 -- 172.21.15.12:0/3108174321 >> 172.21.15.12:0/3108174321 conn(0x7f0fa406efc0 msgr2=0x7f0fa411b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:52:00.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.388+0000 7f0fa92ca700 1 -- 172.21.15.12:0/3108174321 shutdown_connections 2024-05-17T22:52:00.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:00.388+0000 7f0fa92ca700 1 -- 172.21.15.12:0/3108174321 wait complete. 2024-05-17T22:52:00.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:00 smithi096 ceph-mon[36065]: pgmap v821: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.4 MiB/s wr, 431 op/s 2024-05-17T22:52:00.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:00 smithi012 ceph-mon[31632]: pgmap v821: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.4 MiB/s wr, 431 op/s 2024-05-17T22:52:00.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:00 smithi151 ceph-mon[36101]: pgmap v821: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.4 MiB/s wr, 431 op/s 2024-05-17T22:52:01.315 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-17T22:52:01.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 760.0 s, 458.6 tps, lat 213.845 ms stddev 247.482 2024-05-17T22:52:01.749 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:52:02.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:02 smithi096 ceph-mon[36065]: pgmap v822: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.3 MiB/s wr, 422 op/s 2024-05-17T22:52:02.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:02 smithi012 ceph-mon[31632]: pgmap v822: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.3 MiB/s wr, 422 op/s 2024-05-17T22:52:02.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:02 smithi151 ceph-mon[36101]: pgmap v822: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.3 MiB/s wr, 422 op/s 2024-05-17T22:52:03.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.867+0000 7f8228406700 1 -- 172.21.15.12:0/3086951746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8220102b00 msgr2=0x7f8220110aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:03.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.867+0000 7f8228406700 1 --2- 172.21.15.12:0/3086951746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8220102b00 0x7f8220110aa0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f821800b0a0 tx=0x7f821800b3b0 comp rx=0 tx=0).stop 2024-05-17T22:52:03.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.868+0000 7f8228406700 1 -- 172.21.15.12:0/3086951746 shutdown_connections 2024-05-17T22:52:03.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.868+0000 7f8228406700 1 --2- 172.21.15.12:0/3086951746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8220102b00 0x7f8220110aa0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:03.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.868+0000 7f8228406700 1 --2- 172.21.15.12:0/3086951746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8220108c10 0x7f8220109020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:03.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.868+0000 7f8228406700 1 --2- 172.21.15.12:0/3086951746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82201021b0 0x7f82201025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:03.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.868+0000 7f8228406700 1 -- 172.21.15.12:0/3086951746 >> 172.21.15.12:0/3086951746 conn(0x7f8220078040 msgr2=0x7f82200784a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:52:03.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.868+0000 7f8228406700 1 -- 172.21.15.12:0/3086951746 shutdown_connections 2024-05-17T22:52:03.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.868+0000 7f8228406700 1 -- 172.21.15.12:0/3086951746 wait complete. 2024-05-17T22:52:03.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.870+0000 7f8228406700 1 Processor -- start 2024-05-17T22:52:03.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.870+0000 7f8228406700 1 -- start start 2024-05-17T22:52:03.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.870+0000 7f8228406700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82201021b0 0x7f82201a04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:03.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.870+0000 7f8228406700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8220102b00 0x7f82201a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:03.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.870+0000 7f8228406700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8220108c10 0x7f822019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:03.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.870+0000 7f8228406700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8220113ee0 con 0x7f82201021b0 2024-05-17T22:52:03.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.870+0000 7f8228406700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8220113d60 con 0x7f8220108c10 2024-05-17T22:52:03.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.870+0000 7f8228406700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8220114060 con 0x7f8220102b00 2024-05-17T22:52:03.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.870+0000 7f82261a2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82201021b0 0x7f82201a04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:03.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.870+0000 7f82261a2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82201021b0 0x7f82201a04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53570/0 (socket says 172.21.15.12:53570) 2024-05-17T22:52:03.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.870+0000 7f82261a2700 1 -- 172.21.15.12:0/747078317 learned_addr learned my addr 172.21.15.12:0/747078317 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:52:03.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.870+0000 7f82269a3700 1 --2- 172.21.15.12:0/747078317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8220108c10 0x7f822019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:03.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.871+0000 7f82269a3700 1 -- 172.21.15.12:0/747078317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8220102b00 msgr2=0x7f82201a0a20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-17T22:52:03.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.871+0000 7f82269a3700 1 --2- 172.21.15.12:0/747078317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8220102b00 0x7f82201a0a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:03.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.871+0000 7f82269a3700 1 -- 172.21.15.12:0/747078317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82201021b0 msgr2=0x7f82201a04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:03.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.871+0000 7f82269a3700 1 --2- 172.21.15.12:0/747078317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82201021b0 0x7f82201a04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:03.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.871+0000 7f82269a3700 1 -- 172.21.15.12:0/747078317 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f822019ada0 con 0x7f8220108c10 2024-05-17T22:52:03.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.871+0000 7f82269a3700 1 --2- 172.21.15.12:0/747078317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8220108c10 0x7f822019a670 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f821800b070 tx=0x7f8218009bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:52:03.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.871+0000 7f82177fe700 1 -- 172.21.15.12:0/747078317 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8218022070 con 0x7f8220108c10 2024-05-17T22:52:03.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.871+0000 7f82177fe700 1 -- 172.21.15.12:0/747078317 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8218007e30 con 0x7f8220108c10 2024-05-17T22:52:03.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.871+0000 7f8228406700 1 -- 172.21.15.12:0/747078317 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f822019b030 con 0x7f8220108c10 2024-05-17T22:52:03.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.871+0000 7f82177fe700 1 -- 172.21.15.12:0/747078317 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821801cc70 con 0x7f8220108c10 2024-05-17T22:52:03.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.871+0000 7f8228406700 1 -- 172.21.15.12:0/747078317 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82201032c0 con 0x7f8220108c10 2024-05-17T22:52:03.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.874+0000 7f82177fe700 1 -- 172.21.15.12:0/747078317 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8218003c00 con 0x7f8220108c10 2024-05-17T22:52:03.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.875+0000 7f82177fe700 1 --2- 172.21.15.12:0/747078317 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f820c0b2880 0x7f820c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:03.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.875+0000 7f82177fe700 1 -- 172.21.15.12:0/747078317 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f821800f030 con 0x7f8220108c10 2024-05-17T22:52:03.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.875+0000 7f82261a2700 1 --2- 172.21.15.12:0/747078317 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f820c0b2880 0x7f820c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:03.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.876+0000 7f82261a2700 1 --2- 172.21.15.12:0/747078317 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f820c0b2880 0x7f820c0b4d90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f821c005950 tx=0x7f821c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:52:03.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.877+0000 7f8236de1b80 1 Processor -- start 2024-05-17T22:52:03.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.877+0000 7f8236de1b80 1 -- start start 2024-05-17T22:52:03.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.877+0000 7f8236de1b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616fcad39f0 0x5616fcc5f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:03.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.877+0000 7f8236de1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5616fcc5fc10 0x5616fcc59dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:03.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.877+0000 7f8236de1b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616fcc5a310 0x5616fcc5a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:03.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.877+0000 7f8236de1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5616fcb800d0 con 0x5616fcc5fc10 2024-05-17T22:52:03.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.877+0000 7f8236de1b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5616fcb6fc70 con 0x5616fcc5a310 2024-05-17T22:52:03.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.877+0000 7f8236de1b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5616fcb73cb0 con 0x5616fcad39f0 2024-05-17T22:52:03.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.877+0000 7f82259a1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5616fcc5fc10 0x5616fcc59dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:03.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.877+0000 7f82259a1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5616fcc5fc10 0x5616fcc59dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53576/0 (socket says 172.21.15.12:53576) 2024-05-17T22:52:03.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.877+0000 7f82259a1700 1 -- 172.21.15.12:0/611758703 learned_addr learned my addr 172.21.15.12:0/611758703 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:52:03.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.877+0000 7f82269a3700 1 --2- 172.21.15.12:0/611758703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616fcc5a310 0x5616fcc5a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:03.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.877+0000 7f82261a2700 1 --2- 172.21.15.12:0/611758703 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616fcad39f0 0x5616fcc5f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:03.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.878+0000 7f82261a2700 1 -- 172.21.15.12:0/611758703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616fcc5a310 msgr2=0x5616fcc5a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:03.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.878+0000 7f82261a2700 1 --2- 172.21.15.12:0/611758703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616fcc5a310 0x5616fcc5a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:03.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.878+0000 7f82261a2700 1 -- 172.21.15.12:0/611758703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5616fcc5fc10 msgr2=0x5616fcc59dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:03.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.878+0000 7f82261a2700 1 --2- 172.21.15.12:0/611758703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5616fcc5fc10 0x5616fcc59dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:03.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.878+0000 7f82261a2700 1 -- 172.21.15.12:0/611758703 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616fcb095c0 con 0x5616fcad39f0 2024-05-17T22:52:03.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.878+0000 7f82261a2700 1 --2- 172.21.15.12:0/611758703 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616fcad39f0 0x5616fcc5f6d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f821c011ab0 tx=0x7f821c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f47879f1513da136 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:52:03.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.878+0000 7f8214d46700 1 -- 172.21.15.12:0/611758703 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c00ee20 con 0x5616fcad39f0 2024-05-17T22:52:03.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.878+0000 7f8214d46700 1 -- 172.21.15.12:0/611758703 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f821c01c8c0 con 0x5616fcad39f0 2024-05-17T22:52:03.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.878+0000 7f8214d46700 1 -- 172.21.15.12:0/611758703 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c01da10 con 0x5616fcad39f0 2024-05-17T22:52:03.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.879+0000 7f8236de1b80 1 -- 172.21.15.12:0/611758703 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616fcad39f0 msgr2=0x5616fcc5f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:03.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.879+0000 7f8236de1b80 1 --2- 172.21.15.12:0/611758703 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616fcad39f0 0x5616fcc5f6d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f821c011ab0 tx=0x7f821c014000 comp rx=0 tx=0).stop 2024-05-17T22:52:03.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.879+0000 7f8236de1b80 1 -- 172.21.15.12:0/611758703 shutdown_connections 2024-05-17T22:52:03.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.879+0000 7f8236de1b80 1 --2- 172.21.15.12:0/611758703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616fcc5a310 0x5616fcc5a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:03.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.879+0000 7f8236de1b80 1 --2- 172.21.15.12:0/611758703 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616fcad39f0 0x5616fcc5f6d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:03.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.879+0000 7f8236de1b80 1 --2- 172.21.15.12:0/611758703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5616fcc5fc10 0x5616fcc59dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:03.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.879+0000 7f8236de1b80 1 -- 172.21.15.12:0/611758703 >> 172.21.15.12:0/611758703 conn(0x5616fcb337d0 msgr2=0x5616fcc4c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:52:03.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.879+0000 7f8236de1b80 1 -- 172.21.15.12:0/611758703 shutdown_connections 2024-05-17T22:52:03.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.880+0000 7f8236de1b80 1 -- 172.21.15.12:0/611758703 wait complete. 2024-05-17T22:52:03.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.880+0000 7f8236de1b80 1 -- start start 2024-05-17T22:52:03.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.880+0000 7f8236de1b80 1 Processor -- start 2024-05-17T22:52:03.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.881+0000 7f8236de1b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616fcad39f0 0x5616fcc632e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:03.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.881+0000 7f8236de1b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616fcc5a310 0x5616fcc63820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:03.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.881+0000 7f8236de1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5616fcc5fc10 0x5616fcc63d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:03.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.881+0000 7f8236de1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5616fca77150 con 0x5616fcc5fc10 2024-05-17T22:52:03.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.881+0000 7f82261a2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616fcad39f0 0x5616fcc632e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:03.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.881+0000 7f82261a2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616fcad39f0 0x5616fcc632e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:47866/0 (socket says 172.21.15.12:47866) 2024-05-17T22:52:03.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.881+0000 7f82261a2700 1 -- 172.21.15.12:0/708406366 learned_addr learned my addr 172.21.15.12:0/708406366 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:52:03.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.881+0000 7f82259a1700 1 --2- 172.21.15.12:0/708406366 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616fcc5a310 0x5616fcc63820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:03.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.881+0000 7f8236de1b80 1 -- 172.21.15.12:0/708406366 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5616fcb11940 con 0x5616fcad39f0 2024-05-17T22:52:03.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.881+0000 7f8236de1b80 1 -- 172.21.15.12:0/708406366 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5616fcc665e0 con 0x5616fcc5a310 2024-05-17T22:52:03.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.882+0000 7f82269a3700 1 --2- 172.21.15.12:0/708406366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5616fcc5fc10 0x5616fcc63d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:03.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.882+0000 7f8236de1b80 10 client.0 _ll_register_callbacks cb 0x5616fcb06b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:52:03.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.882+0000 7f82261a2700 1 -- 172.21.15.12:0/708406366 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616fcc5a310 msgr2=0x5616fcc63820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:03.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.882+0000 7f82261a2700 1 --2- 172.21.15.12:0/708406366 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616fcc5a310 0x5616fcc63820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:03.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.882+0000 7f82261a2700 1 -- 172.21.15.12:0/708406366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5616fcc5fc10 msgr2=0x5616fcc63d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:03.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.882+0000 7f82261a2700 1 --2- 172.21.15.12:0/708406366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5616fcc5fc10 0x5616fcc63d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:03.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.882+0000 7f82261a2700 1 -- 172.21.15.12:0/708406366 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616fcac0480 con 0x5616fcad39f0 2024-05-17T22:52:03.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.883+0000 7f82261a2700 1 --2- 172.21.15.12:0/708406366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616fcad39f0 0x5616fcc632e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f821c009be0 tx=0x7f821c01d510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45bc16a2e7f50099 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:52:03.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.883+0000 7f8209ffb700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:52:03.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.883+0000 7f8209ffb700 1 -- 172.21.15.12:0/708406366 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c01cc80 con 0x5616fcad39f0 2024-05-17T22:52:03.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.884+0000 7f8236de1b80 1 -- 172.21.15.12:0/708406366 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f821c01c8c0 con 0x5616fcad39f0 2024-05-17T22:52:03.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.884+0000 7f8209ffb700 1 -- 172.21.15.12:0/708406366 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f821c02b040 con 0x5616fcad39f0 2024-05-17T22:52:03.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.884+0000 7f8209ffb700 1 -- 172.21.15.12:0/708406366 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f821c03b050 con 0x5616fcad39f0 2024-05-17T22:52:03.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.884+0000 7f8236de1b80 10 client.26114 fetch_fsmap learned FSMap version 41 2024-05-17T22:52:03.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.884+0000 7f8236de1b80 1 -- 172.21.15.12:0/708406366 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5616fcc71050 con 0x5616fcad39f0 2024-05-17T22:52:03.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.884+0000 7f8209ffb700 1 -- 172.21.15.12:0/708406366 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f821c01c8c0 con 0x5616fcad39f0 2024-05-17T22:52:03.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.884+0000 7f8209ffb700 1 -- 172.21.15.12:0/708406366 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f821c004710 con 0x5616fcad39f0 2024-05-17T22:52:03.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.884+0000 7f8236de1b80 10 client.26114 fetch_fsmap finished waiting for FSMap version 41 2024-05-17T22:52:03.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.884+0000 7f8236de1b80 10 client.26114 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:52:03.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.884+0000 7f8236de1b80 20 client.26114 populate_metadata read hostname 'smithi012' 2024-05-17T22:52:03.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.884+0000 7f8236de1b80 1 --2- 172.21.15.12:0/708406366 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5616fcc735d0 0x5616fcc759f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:03.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.885+0000 7f8236de1b80 4 client.26114 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:52:03.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.885+0000 7f8236de1b80 1 -- 172.21.15.12:0/708406366 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616fcb82020 con 0x5616fcc735d0 2024-05-17T22:52:03.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.885+0000 7f82259a1700 1 --2- 172.21.15.12:0/708406366 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5616fcc735d0 0x5616fcc759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:03.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.885+0000 7f8209ffb700 10 client.26114 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:52:03.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.885+0000 7f82259a1700 1 --2- 172.21.15.12:0/708406366 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5616fcc735d0 0x5616fcc759f0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be95e5d1cbf1b563 server_cookie=4174c3022b8bae51 in_seq=0 out_seq=0 2024-05-17T22:52:03.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.886+0000 7f8209ffb700 1 -- 172.21.15.12:0/708406366 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5616fcb82020 con 0x5616fcc735d0 2024-05-17T22:52:03.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.886+0000 7f8209ffb700 10 client.26114 handle_command_reply: tid=0 2024-05-17T22:52:03.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.886+0000 7f8236de1b80 1 client.26114 shutdown 2024-05-17T22:52:03.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.886+0000 7f8236de1b80 1 -- 172.21.15.12:0/708406366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616fcad39f0 msgr2=0x5616fcc632e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:03.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.886+0000 7f8236de1b80 1 --2- 172.21.15.12:0/708406366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616fcad39f0 0x5616fcc632e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f821c009be0 tx=0x7f821c01d510 comp rx=0 tx=0).stop 2024-05-17T22:52:03.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.886+0000 7f8236de1b80 1 -- 172.21.15.12:0/708406366 shutdown_connections 2024-05-17T22:52:03.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.886+0000 7f8236de1b80 1 --2- 172.21.15.12:0/708406366 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5616fcc735d0 0x5616fcc759f0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:03.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.886+0000 7f8236de1b80 1 --2- 172.21.15.12:0/708406366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5616fcc5fc10 0x5616fcc63d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:03.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.886+0000 7f8236de1b80 1 --2- 172.21.15.12:0/708406366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616fcad39f0 0x5616fcc632e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:03.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.886+0000 7f8236de1b80 1 --2- 172.21.15.12:0/708406366 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616fcc5a310 0x5616fcc63820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:03.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.886+0000 7f8236de1b80 1 -- 172.21.15.12:0/708406366 >> 172.21.15.12:0/708406366 conn(0x5616fcb337d0 msgr2=0x5616fcc4c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:52:03.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.886+0000 7f8209ffb700 0 client.26114 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:52:03.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.886+0000 7f8236de1b80 1 -- 172.21.15.12:0/708406366 shutdown_connections 2024-05-17T22:52:03.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.886+0000 7f8236de1b80 1 -- 172.21.15.12:0/708406366 wait complete. 2024-05-17T22:52:03.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.886+0000 7f8236de1b80 20 client.26114 trim_cache size 0 max 16384 2024-05-17T22:52:03.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.905+0000 7f8236de1b80 1 Processor -- start 2024-05-17T22:52:03.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.905+0000 7f8236de1b80 1 -- start start 2024-05-17T22:52:03.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.905+0000 7f8236de1b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616fcad39f0 0x5616fcc65ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:03.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.905+0000 7f8236de1b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616fcc5a310 0x5616fcc63bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:03.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.905+0000 7f8236de1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5616fcc5fc10 0x5616fcc46820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:03.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.905+0000 7f8236de1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f82201b02d0 con 0x5616fcc5fc10 2024-05-17T22:52:03.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.905+0000 7f8236de1b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f82201adb60 con 0x5616fcc5a310 2024-05-17T22:52:03.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.905+0000 7f8236de1b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f82201ab9b0 con 0x5616fcad39f0 2024-05-17T22:52:03.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.905+0000 7f82269a3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5616fcc5fc10 0x5616fcc46820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:03.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.906+0000 7f82259a1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616fcc5a310 0x5616fcc63bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:03.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.906+0000 7f82259a1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616fcc5a310 0x5616fcc63bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:47872/0 (socket says 172.21.15.12:47872) 2024-05-17T22:52:03.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.906+0000 7f82269a3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5616fcc5fc10 0x5616fcc46820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53590/0 (socket says 172.21.15.12:53590) 2024-05-17T22:52:03.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.906+0000 7f82259a1700 1 -- 172.21.15.12:0/1392462802 learned_addr learned my addr 172.21.15.12:0/1392462802 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:52:03.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.906+0000 7f82261a2700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616fcad39f0 0x5616fcc65ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:03.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.906+0000 7f82261a2700 1 -- 172.21.15.12:0/1392462802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616fcc5a310 msgr2=0x5616fcc63bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:03.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.906+0000 7f82261a2700 1 --2- 172.21.15.12:0/1392462802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616fcc5a310 0x5616fcc63bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:03.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.906+0000 7f82261a2700 1 -- 172.21.15.12:0/1392462802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5616fcc5fc10 msgr2=0x5616fcc46820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:03.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.906+0000 7f82261a2700 1 --2- 172.21.15.12:0/1392462802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5616fcc5fc10 0x5616fcc46820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:03.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.906+0000 7f82261a2700 1 -- 172.21.15.12:0/1392462802 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616fcc44580 con 0x5616fcad39f0 2024-05-17T22:52:03.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.906+0000 7f82261a2700 1 --2- 172.21.15.12:0/1392462802 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616fcad39f0 0x5616fcc65ae0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f821c016040 tx=0x7f821c009fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b59a679081aca27 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:52:03.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.907+0000 7f820bfff700 1 -- 172.21.15.12:0/1392462802 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c044070 con 0x5616fcad39f0 2024-05-17T22:52:03.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.907+0000 7f820bfff700 1 -- 172.21.15.12:0/1392462802 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f821c01c8c0 con 0x5616fcad39f0 2024-05-17T22:52:03.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.907+0000 7f820bfff700 1 -- 172.21.15.12:0/1392462802 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c004040 con 0x5616fcad39f0 2024-05-17T22:52:03.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.908+0000 7f8236de1b80 1 -- 172.21.15.12:0/1392462802 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616fcad39f0 msgr2=0x5616fcc65ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:03.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.908+0000 7f8236de1b80 1 --2- 172.21.15.12:0/1392462802 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616fcad39f0 0x5616fcc65ae0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f821c016040 tx=0x7f821c009fa0 comp rx=0 tx=0).stop 2024-05-17T22:52:03.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.909+0000 7f8236de1b80 1 -- 172.21.15.12:0/1392462802 shutdown_connections 2024-05-17T22:52:03.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.909+0000 7f8236de1b80 1 --2- 172.21.15.12:0/1392462802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5616fcc5fc10 0x5616fcc46820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:03.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.909+0000 7f8236de1b80 1 --2- 172.21.15.12:0/1392462802 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616fcad39f0 0x5616fcc65ae0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:03.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.909+0000 7f8236de1b80 1 --2- 172.21.15.12:0/1392462802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616fcc5a310 0x5616fcc63bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:03.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.909+0000 7f8236de1b80 1 -- 172.21.15.12:0/1392462802 >> 172.21.15.12:0/1392462802 conn(0x5616fcb337d0 msgr2=0x5616fcc46110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:52:03.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.910+0000 7f8236de1b80 1 -- 172.21.15.12:0/1392462802 shutdown_connections 2024-05-17T22:52:03.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.914+0000 7f8236de1b80 1 -- 172.21.15.12:0/1392462802 wait complete. 2024-05-17T22:52:03.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.914+0000 7f8236de1b80 1 -- start start 2024-05-17T22:52:03.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.914+0000 7f8236de1b80 1 Processor -- start 2024-05-17T22:52:03.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.915+0000 7f8236de1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5616fcad39f0 0x5616fcc6b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:03.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.915+0000 7f8236de1b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616fcc5a310 0x5616fcc6bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:03.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.915+0000 7f8236de1b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616fcc5fc10 0x5616fcc7a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:03.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.915+0000 7f8236de1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f82201ac020 con 0x5616fcad39f0 2024-05-17T22:52:03.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.915+0000 7f8236de1b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f82201ad490 con 0x5616fcc5a310 2024-05-17T22:52:03.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.915+0000 7f8236de1b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f82201ac6f0 con 0x5616fcc5fc10 2024-05-17T22:52:03.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.915+0000 7f8236de1b80 10 client.0 _ll_register_callbacks cb 0x5616fcb06b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:52:03.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.917+0000 7f82269a3700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616fcc5fc10 0x5616fcc7a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:03.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.917+0000 7f82259a1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616fcc5a310 0x5616fcc6bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:03.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.917+0000 7f82269a3700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616fcc5fc10 0x5616fcc7a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:34394/0 (socket says 172.21.15.12:34394) 2024-05-17T22:52:03.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.917+0000 7f82269a3700 1 -- 172.21.15.12:0/846545438 learned_addr learned my addr 172.21.15.12:0/846545438 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:52:03.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.920+0000 7f82261a2700 1 --2- 172.21.15.12:0/846545438 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5616fcad39f0 0x5616fcc6b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:03.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.920+0000 7f82259a1700 1 -- 172.21.15.12:0/846545438 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616fcc5fc10 msgr2=0x5616fcc7a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:03.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.920+0000 7f82259a1700 1 --2- 172.21.15.12:0/846545438 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616fcc5fc10 0x5616fcc7a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:03.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.920+0000 7f82259a1700 1 -- 172.21.15.12:0/846545438 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5616fcad39f0 msgr2=0x5616fcc6b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:03.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.920+0000 7f82259a1700 1 --2- 172.21.15.12:0/846545438 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5616fcad39f0 0x5616fcc6b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:03.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.920+0000 7f82259a1700 1 -- 172.21.15.12:0/846545438 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616fcc7a650 con 0x5616fcc5a310 2024-05-17T22:52:03.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.928+0000 7f82259a1700 1 --2- 172.21.15.12:0/846545438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616fcc5a310 0x5616fcc6bdb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x5616fcc75fc0 tx=0x7f8210010710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a0f4fdf1e4e3a8e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:52:03.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.928+0000 7f8209ffb700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:52:03.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.932+0000 7f8209ffb700 1 -- 172.21.15.12:0/846545438 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8210004250 con 0x5616fcc5a310 2024-05-17T22:52:03.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.932+0000 7f8236de1b80 1 -- 172.21.15.12:0/846545438 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f821c01c8c0 con 0x5616fcc5a310 2024-05-17T22:52:03.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.932+0000 7f8209ffb700 1 -- 172.21.15.12:0/846545438 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f82100043f0 con 0x5616fcc5a310 2024-05-17T22:52:03.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.934+0000 7f8209ffb700 1 -- 172.21.15.12:0/846545438 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f821002b050 con 0x5616fcc5a310 2024-05-17T22:52:03.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.935+0000 7f8236de1b80 10 client.26120 fetch_fsmap learned FSMap version 41 2024-05-17T22:52:03.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.935+0000 7f8236de1b80 1 -- 172.21.15.12:0/846545438 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5616fccdd4b0 con 0x5616fcc5a310 2024-05-17T22:52:03.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.940+0000 7f8209ffb700 1 -- 172.21.15.12:0/846545438 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f821c01c8c0 con 0x5616fcc5a310 2024-05-17T22:52:03.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.940+0000 7f8209ffb700 1 -- 172.21.15.12:0/846545438 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f8210003a20 con 0x5616fcc5a310 2024-05-17T22:52:03.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.940+0000 7f8236de1b80 10 client.26120 fetch_fsmap finished waiting for FSMap version 41 2024-05-17T22:52:03.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.940+0000 7f8236de1b80 10 client.26120 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:52:03.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.940+0000 7f8236de1b80 20 client.26120 populate_metadata read hostname 'smithi012' 2024-05-17T22:52:03.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.940+0000 7f8236de1b80 1 --2- 172.21.15.12:0/846545438 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5616fccddbe0 0x5616fcce0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:03.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.940+0000 7f8236de1b80 4 client.26120 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:52:03.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.940+0000 7f8236de1b80 1 -- 172.21.15.12:0/846545438 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5616fcb7b800 con 0x5616fccddbe0 2024-05-17T22:52:03.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.941+0000 7f82261a2700 1 --2- 172.21.15.12:0/846545438 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5616fccddbe0 0x5616fcce0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:03.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.956+0000 7f82261a2700 1 --2- 172.21.15.12:0/846545438 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5616fccddbe0 0x5616fcce0000 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38e67c9564614161 server_cookie=88517df59b0560d6 in_seq=0 out_seq=0 2024-05-17T22:52:03.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.956+0000 7f8209ffb700 10 client.26120 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:52:03.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.960+0000 7f8209ffb700 1 -- 172.21.15.12:0/846545438 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5616fcb7b800 con 0x5616fccddbe0 2024-05-17T22:52:03.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.960+0000 7f8209ffb700 10 client.26120 handle_command_reply: tid=0 2024-05-17T22:52:03.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.960+0000 7f8236de1b80 1 client.26120 shutdown 2024-05-17T22:52:03.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.966+0000 7f8236de1b80 1 -- 172.21.15.12:0/846545438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616fcc5a310 msgr2=0x5616fcc6bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:03.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.966+0000 7f8236de1b80 1 --2- 172.21.15.12:0/846545438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616fcc5a310 0x5616fcc6bdb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x5616fcc75fc0 tx=0x7f8210010710 comp rx=0 tx=0).stop 2024-05-17T22:52:03.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.966+0000 7f8236de1b80 1 -- 172.21.15.12:0/846545438 shutdown_connections 2024-05-17T22:52:03.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.966+0000 7f8236de1b80 1 --2- 172.21.15.12:0/846545438 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616fcc5fc10 0x5616fcc7a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:03.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.966+0000 7f8236de1b80 1 --2- 172.21.15.12:0/846545438 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5616fccddbe0 0x5616fcce0000 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:03.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.966+0000 7f8236de1b80 1 --2- 172.21.15.12:0/846545438 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5616fcad39f0 0x5616fcc6b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:03.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.966+0000 7f8236de1b80 1 --2- 172.21.15.12:0/846545438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616fcc5a310 0x5616fcc6bdb0 secure :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0x5616fcc75fc0 tx=0x7f8210010710 comp rx=0 tx=0).stop 2024-05-17T22:52:03.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.966+0000 7f8236de1b80 1 -- 172.21.15.12:0/846545438 >> 172.21.15.12:0/846545438 conn(0x5616fcb337d0 msgr2=0x5616fcc46110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:52:03.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.970+0000 7f8209ffb700 0 client.26120 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:52:03.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.973+0000 7f8236de1b80 1 -- 172.21.15.12:0/846545438 shutdown_connections 2024-05-17T22:52:03.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.973+0000 7f8236de1b80 1 -- 172.21.15.12:0/846545438 wait complete. 2024-05-17T22:52:03.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.974+0000 7f8236de1b80 20 client.26120 trim_cache size 0 max 16384 2024-05-17T22:52:03.978 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:52:03.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.978+0000 7f8228406700 1 -- 172.21.15.12:0/747078317 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f820c0b2880 msgr2=0x7f820c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:03.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.978+0000 7f8228406700 1 --2- 172.21.15.12:0/747078317 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f820c0b2880 0x7f820c0b4d90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f821c005950 tx=0x7f821c009260 comp rx=0 tx=0).stop 2024-05-17T22:52:03.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.978+0000 7f8228406700 1 -- 172.21.15.12:0/747078317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8220108c10 msgr2=0x7f822019a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:03.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.978+0000 7f8228406700 1 --2- 172.21.15.12:0/747078317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8220108c10 0x7f822019a670 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f821800b070 tx=0x7f8218009bf0 comp rx=0 tx=0).stop 2024-05-17T22:52:03.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.978+0000 7f8228406700 1 -- 172.21.15.12:0/747078317 shutdown_connections 2024-05-17T22:52:03.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.978+0000 7f8228406700 1 --2- 172.21.15.12:0/747078317 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f820c0b2880 0x7f820c0b4d90 secure :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f821c005950 tx=0x7f821c009260 comp rx=0 tx=0).stop 2024-05-17T22:52:03.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.978+0000 7f8228406700 1 --2- 172.21.15.12:0/747078317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8220108c10 0x7f822019a670 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:03.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.978+0000 7f8228406700 1 --2- 172.21.15.12:0/747078317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82201021b0 0x7f82201a04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:03.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.978+0000 7f8228406700 1 --2- 172.21.15.12:0/747078317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8220102b00 0x7f82201a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:03.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.978+0000 7f8228406700 1 -- 172.21.15.12:0/747078317 >> 172.21.15.12:0/747078317 conn(0x7f8220078040 msgr2=0x7f82200ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:52:03.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.985+0000 7f8228406700 1 -- 172.21.15.12:0/747078317 shutdown_connections 2024-05-17T22:52:03.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:03.985+0000 7f8228406700 1 -- 172.21.15.12:0/747078317 wait complete. 2024-05-17T22:52:04.552 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:04 smithi151 ceph-mon[36101]: pgmap v823: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.0 MiB/s wr, 391 op/s 2024-05-17T22:52:04.552 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:04 smithi151 ceph-mon[36101]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:52:04.553 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:04 smithi096 ceph-mon[36065]: pgmap v823: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.0 MiB/s wr, 391 op/s 2024-05-17T22:52:04.553 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:04 smithi096 ceph-mon[36065]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:52:04.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:04 smithi012 ceph-mon[31632]: pgmap v823: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.0 MiB/s wr, 391 op/s 2024-05-17T22:52:04.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:04 smithi012 ceph-mon[31632]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:52:05.757 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-17T22:52:05.757 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-17T22:52:06.236 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:52:06.371 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 765.0 s, 611.0 tps, lat 163.910 ms stddev 219.481 2024-05-17T22:52:06.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:06 smithi096 ceph-mon[36065]: pgmap v824: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.2 MiB/s wr, 503 op/s 2024-05-17T22:52:06.554 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:06 smithi012 ceph-mon[31632]: pgmap v824: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.2 MiB/s wr, 503 op/s 2024-05-17T22:52:06.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:06 smithi151 ceph-mon[36101]: pgmap v824: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.2 MiB/s wr, 503 op/s 2024-05-17T22:52:08.460 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:08 smithi012 ceph-mon[31632]: pgmap v825: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.3 MiB/s wr, 403 op/s 2024-05-17T22:52:08.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:08 smithi096 ceph-mon[36065]: pgmap v825: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.3 MiB/s wr, 403 op/s 2024-05-17T22:52:08.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:08 smithi151 ceph-mon[36101]: pgmap v825: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.3 MiB/s wr, 403 op/s 2024-05-17T22:52:08.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:08.984+0000 7fa8ead01700 1 -- 172.21.15.12:0/4115696973 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8e411d550 msgr2=0x7fa8e411f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:08.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:08.984+0000 7fa8ead01700 1 --2- 172.21.15.12:0/4115696973 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8e411d550 0x7fa8e411f970 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc009f90 tx=0x7fa8dc00b330 comp rx=0 tx=0).stop 2024-05-17T22:52:08.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:08.985+0000 7fa8ead01700 1 -- 172.21.15.12:0/4115696973 shutdown_connections 2024-05-17T22:52:08.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:08.985+0000 7fa8ead01700 1 --2- 172.21.15.12:0/4115696973 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8e411d550 0x7fa8e411f970 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:08.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:08.985+0000 7fa8ead01700 1 --2- 172.21.15.12:0/4115696973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8e410a830 0x7fa8e410ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:08.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:08.985+0000 7fa8ead01700 1 --2- 172.21.15.12:0/4115696973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa8e410b180 0x7fa8e411d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:08.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:08.985+0000 7fa8ead01700 1 -- 172.21.15.12:0/4115696973 >> 172.21.15.12:0/4115696973 conn(0x7fa8e406efc0 msgr2=0x7fa8e406f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:52:08.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:08.987+0000 7fa8ead01700 1 -- 172.21.15.12:0/4115696973 shutdown_connections 2024-05-17T22:52:08.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:08.988+0000 7fa8ead01700 1 -- 172.21.15.12:0/4115696973 wait complete. 2024-05-17T22:52:08.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:08.989+0000 7fa8ead01700 1 Processor -- start 2024-05-17T22:52:08.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:08.989+0000 7fa8ead01700 1 -- start start 2024-05-17T22:52:08.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:08.990+0000 7fa8ead01700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8e410a830 0x7fa8e411cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:08.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:08.990+0000 7fa8ead01700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8e410b180 0x7fa8e4116020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:08.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:08.990+0000 7fa8ead01700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa8e411d550 0x7fa8e4116620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:08.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:08.990+0000 7fa8ead01700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e4122bf0 con 0x7fa8e410a830 2024-05-17T22:52:08.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:08.990+0000 7fa8ead01700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e4122a70 con 0x7fa8e411d550 2024-05-17T22:52:08.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:08.990+0000 7fa8ead01700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e4122d70 con 0x7fa8e410b180 2024-05-17T22:52:08.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:08.990+0000 7fa8e929e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa8e411d550 0x7fa8e4116620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:08.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:08.990+0000 7fa8e8a9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8e410a830 0x7fa8e411cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:08.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:08.990+0000 7fa8e8a9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8e410a830 0x7fa8e411cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53622/0 (socket says 172.21.15.12:53622) 2024-05-17T22:52:08.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:08.990+0000 7fa8e8a9d700 1 -- 172.21.15.12:0/2686722839 learned_addr learned my addr 172.21.15.12:0/2686722839 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:52:08.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:08.991+0000 7fa8e3fff700 1 --2- 172.21.15.12:0/2686722839 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8e410b180 0x7fa8e4116020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:08.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:08.991+0000 7fa8e929e700 1 -- 172.21.15.12:0/2686722839 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8e410b180 msgr2=0x7fa8e4116020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:08.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:08.991+0000 7fa8e929e700 1 --2- 172.21.15.12:0/2686722839 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8e410b180 0x7fa8e4116020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:08.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:08.991+0000 7fa8e929e700 1 -- 172.21.15.12:0/2686722839 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8e410a830 msgr2=0x7fa8e411cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:08.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:08.991+0000 7fa8e929e700 1 --2- 172.21.15.12:0/2686722839 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8e410a830 0x7fa8e411cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:08.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:08.991+0000 7fa8e929e700 1 -- 172.21.15.12:0/2686722839 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e411aa20 con 0x7fa8e411d550 2024-05-17T22:52:08.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:08.995+0000 7fa8e929e700 1 --2- 172.21.15.12:0/2686722839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa8e411d550 0x7fa8e4116620 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc00b770 tx=0x7fa8dc009400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:52:08.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:08.995+0000 7fa8e1ffb700 1 -- 172.21.15.12:0/2686722839 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc021020 con 0x7fa8e411d550 2024-05-17T22:52:08.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:08.995+0000 7fa8ead01700 1 -- 172.21.15.12:0/2686722839 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8e411acb0 con 0x7fa8e411d550 2024-05-17T22:52:08.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:08.995+0000 7fa8ead01700 1 -- 172.21.15.12:0/2686722839 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8e411b1f0 con 0x7fa8e411d550 2024-05-17T22:52:08.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:08.995+0000 7fa8e1ffb700 1 -- 172.21.15.12:0/2686722839 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8dc007d00 con 0x7fa8e411d550 2024-05-17T22:52:09.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:08.995+0000 7fa8e1ffb700 1 -- 172.21.15.12:0/2686722839 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc0052a0 con 0x7fa8e411d550 2024-05-17T22:52:09.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.012+0000 7fa8e1ffb700 1 -- 172.21.15.12:0/2686722839 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa8dc024020 con 0x7fa8e411d550 2024-05-17T22:52:09.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.012+0000 7fa8f96dcb80 1 Processor -- start 2024-05-17T22:52:09.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.012+0000 7fa8f96dcb80 1 -- start start 2024-05-17T22:52:09.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.013+0000 7fa8f96dcb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5655423a2a70 0x5655425265e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:09.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.013+0000 7fa8f96dcb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565542526b20 0x565542520cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:09.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.013+0000 7fa8f96dcb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5655425211f0 0x565542521600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:09.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.013+0000 7fa8f96dcb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56554244f310 con 0x5655425211f0 2024-05-17T22:52:09.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.013+0000 7fa8f96dcb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56554243f050 con 0x565542526b20 2024-05-17T22:52:09.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.013+0000 7fa8f96dcb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x565542442ef0 con 0x5655423a2a70 2024-05-17T22:52:09.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.013+0000 7fa8e3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5655425211f0 0x565542521600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:09.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.013+0000 7fa8e3fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5655423a2a70 0x5655425265e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:09.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.013+0000 7fa8e3fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5655423a2a70 0x5655425265e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:34420/0 (socket says 172.21.15.12:34420) 2024-05-17T22:52:09.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.013+0000 7fa8e3fff700 1 -- 172.21.15.12:0/2194875896 learned_addr learned my addr 172.21.15.12:0/2194875896 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:52:09.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.018+0000 7fa8e1ffb700 1 --2- 172.21.15.12:0/2686722839 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa8d40b2a50 0x7fa8d40b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:09.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.019+0000 7fa8e3fff700 1 -- 172.21.15.12:0/2194875896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565542526b20 msgr2=0x565542520cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:09.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.019+0000 7fa8e929e700 1 --2- 172.21.15.12:0/2686722839 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa8d40b2a50 0x7fa8d40b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:09.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.019+0000 7fa8e1ffb700 1 -- 172.21.15.12:0/2686722839 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fa8dc0e3600 con 0x7fa8e411d550 2024-05-17T22:52:09.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.017+0000 7fa8e8a9d700 1 --2- 172.21.15.12:0/2194875896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565542526b20 0x565542520cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:09.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.021+0000 7fa8e3fff700 1 --2- 172.21.15.12:0/2194875896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565542526b20 0x565542520cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:09.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.021+0000 7fa8e3fff700 1 -- 172.21.15.12:0/2194875896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5655425211f0 msgr2=0x565542521600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:09.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.021+0000 7fa8e3fff700 1 --2- 172.21.15.12:0/2194875896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5655425211f0 0x565542521600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:09.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.021+0000 7fa8e3fff700 1 -- 172.21.15.12:0/2194875896 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5655423e8610 con 0x5655423a2a70 2024-05-17T22:52:09.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.024+0000 7fa8e929e700 1 --2- 172.21.15.12:0/2686722839 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa8d40b2a50 0x7fa8d40b4f60 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc009660 tx=0x7fa8dc0a6040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:52:09.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.026+0000 7fa8e8a9d700 1 --2- 172.21.15.12:0/2194875896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565542526b20 0x565542520cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:52:09.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.028+0000 7fa8e3fff700 1 --2- 172.21.15.12:0/2194875896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5655423a2a70 0x5655425265e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8005ed0 tx=0x7fa8d80058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec9b0cd23c64ef89 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:52:09.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.029+0000 7fa8ceffd700 1 -- 172.21.15.12:0/2194875896 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d80350f0 con 0x5655423a2a70 2024-05-17T22:52:09.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.029+0000 7fa8ceffd700 1 -- 172.21.15.12:0/2194875896 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8d8004ce0 con 0x5655423a2a70 2024-05-17T22:52:09.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.030+0000 7fa8f96dcb80 1 -- 172.21.15.12:0/2194875896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5655423a2a70 msgr2=0x5655425265e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:09.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.030+0000 7fa8f96dcb80 1 --2- 172.21.15.12:0/2194875896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5655423a2a70 0x5655425265e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8005ed0 tx=0x7fa8d80058e0 comp rx=0 tx=0).stop 2024-05-17T22:52:09.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.031+0000 7fa8f96dcb80 1 -- 172.21.15.12:0/2194875896 shutdown_connections 2024-05-17T22:52:09.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.031+0000 7fa8f96dcb80 1 --2- 172.21.15.12:0/2194875896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5655425211f0 0x565542521600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:09.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.031+0000 7fa8f96dcb80 1 --2- 172.21.15.12:0/2194875896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5655423a2a70 0x5655425265e0 secure :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8005ed0 tx=0x7fa8d80058e0 comp rx=0 tx=0).stop 2024-05-17T22:52:09.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.031+0000 7fa8f96dcb80 1 --2- 172.21.15.12:0/2194875896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565542526b20 0x565542520cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:09.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.031+0000 7fa8f96dcb80 1 -- 172.21.15.12:0/2194875896 >> 172.21.15.12:0/2194875896 conn(0x5655423e6f80 msgr2=0x56554251bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:52:09.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.031+0000 7fa8f96dcb80 1 -- 172.21.15.12:0/2194875896 shutdown_connections 2024-05-17T22:52:09.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.034+0000 7fa8f96dcb80 1 -- 172.21.15.12:0/2194875896 wait complete. 2024-05-17T22:52:09.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.034+0000 7fa8f96dcb80 1 -- start start 2024-05-17T22:52:09.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.034+0000 7fa8f96dcb80 1 Processor -- start 2024-05-17T22:52:09.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.035+0000 7fa8f96dcb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5655425211f0 0x565542529ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:09.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.035+0000 7fa8f96dcb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565542526b20 0x56554252a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:09.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.035+0000 7fa8f96dcb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56554252a970 0x5655425367e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:09.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.035+0000 7fa8f96dcb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5655423461d0 con 0x5655425211f0 2024-05-17T22:52:09.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.035+0000 7fa8f96dcb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5655423dff00 con 0x565542526b20 2024-05-17T22:52:09.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.035+0000 7fa8f96dcb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56554252d460 con 0x56554252a970 2024-05-17T22:52:09.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.035+0000 7fa8f96dcb80 10 client.0 _ll_register_callbacks cb 0x5655423d9f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:52:09.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.036+0000 7fa8e8a9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565542526b20 0x56554252a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:09.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.036+0000 7fa8e8a9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565542526b20 0x56554252a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:47930/0 (socket says 172.21.15.12:47930) 2024-05-17T22:52:09.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.036+0000 7fa8e8a9d700 1 -- 172.21.15.12:0/2809716661 learned_addr learned my addr 172.21.15.12:0/2809716661 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:52:09.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.037+0000 7fa8e3fff700 1 --2- 172.21.15.12:0/2809716661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56554252a970 0x5655425367e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:09.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.037+0000 7fa8e3fff700 1 --2- 172.21.15.12:0/2809716661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5655425211f0 0x565542529ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:09.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.040+0000 7fa8e3fff700 1 -- 172.21.15.12:0/2809716661 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565542526b20 msgr2=0x56554252a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:09.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.040+0000 7fa8e3fff700 1 --2- 172.21.15.12:0/2809716661 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565542526b20 0x56554252a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:09.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.040+0000 7fa8e3fff700 1 -- 172.21.15.12:0/2809716661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5655425211f0 msgr2=0x565542529ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:09.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.040+0000 7fa8e3fff700 1 --2- 172.21.15.12:0/2809716661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5655425211f0 0x565542529ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:09.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.041+0000 7fa8e3fff700 1 -- 172.21.15.12:0/2809716661 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56554238f500 con 0x56554252a970 2024-05-17T22:52:09.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.042+0000 7fa8e3fff700 1 --2- 172.21.15.12:0/2809716661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56554252a970 0x5655425367e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8035c10 tx=0x7fa8d800dc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8071b3924fc3ed7f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:52:09.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.044+0000 7fa8c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:52:09.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.044+0000 7fa8c3fff700 1 -- 172.21.15.12:0/2809716661 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d80048b0 con 0x56554252a970 2024-05-17T22:52:09.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.044+0000 7fa8c3fff700 1 -- 172.21.15.12:0/2809716661 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fa8d8004a50 con 0x56554252a970 2024-05-17T22:52:09.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.044+0000 7fa8f96dcb80 1 -- 172.21.15.12:0/2809716661 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x565542451260 con 0x56554252a970 2024-05-17T22:52:09.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.048+0000 7fa8c3fff700 1 -- 172.21.15.12:0/2809716661 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fa8d8004c50 con 0x56554252a970 2024-05-17T22:52:09.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.051+0000 7fa8f96dcb80 10 client.26071 fetch_fsmap learned FSMap version 41 2024-05-17T22:52:09.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.051+0000 7fa8f96dcb80 1 -- 172.21.15.12:0/2809716661 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x565542538060 con 0x56554252a970 2024-05-17T22:52:09.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.053+0000 7fa8c3fff700 1 -- 172.21.15.12:0/2809716661 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x565542451260 con 0x56554252a970 2024-05-17T22:52:09.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.053+0000 7fa8c3fff700 1 -- 172.21.15.12:0/2809716661 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7fa8d803f460 con 0x56554252a970 2024-05-17T22:52:09.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.053+0000 7fa8f96dcb80 10 client.26071 fetch_fsmap finished waiting for FSMap version 41 2024-05-17T22:52:09.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.053+0000 7fa8f96dcb80 10 client.26071 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:52:09.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.053+0000 7fa8f96dcb80 20 client.26071 populate_metadata read hostname 'smithi012' 2024-05-17T22:52:09.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.053+0000 7fa8f96dcb80 1 --2- 172.21.15.12:0/2809716661 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x56554253a530 0x56554253c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:09.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.053+0000 7fa8f96dcb80 4 client.26071 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:52:09.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.053+0000 7fa8f96dcb80 1 -- 172.21.15.12:0/2809716661 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56554244aa40 con 0x56554253a530 2024-05-17T22:52:09.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.056+0000 7fa8e8a9d700 1 --2- 172.21.15.12:0/2809716661 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x56554253a530 0x56554253c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:09.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.057+0000 7fa8e8a9d700 1 --2- 172.21.15.12:0/2809716661 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x56554253a530 0x56554253c950 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a9037c65a3d0c8e server_cookie=c3b834f9d65e55de in_seq=0 out_seq=0 2024-05-17T22:52:09.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.057+0000 7fa8c3fff700 10 client.26071 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:52:09.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.060+0000 7fa8c3fff700 1 -- 172.21.15.12:0/2809716661 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56554244aa40 con 0x56554253a530 2024-05-17T22:52:09.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.061+0000 7fa8c3fff700 10 client.26071 handle_command_reply: tid=0 2024-05-17T22:52:09.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.061+0000 7fa8f96dcb80 1 client.26071 shutdown 2024-05-17T22:52:09.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.063+0000 7fa8f96dcb80 1 -- 172.21.15.12:0/2809716661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56554252a970 msgr2=0x5655425367e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:09.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.063+0000 7fa8f96dcb80 1 --2- 172.21.15.12:0/2809716661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56554252a970 0x5655425367e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8035c10 tx=0x7fa8d800dc40 comp rx=0 tx=0).stop 2024-05-17T22:52:09.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.064+0000 7fa8f96dcb80 1 -- 172.21.15.12:0/2809716661 shutdown_connections 2024-05-17T22:52:09.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.064+0000 7fa8f96dcb80 1 --2- 172.21.15.12:0/2809716661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56554252a970 0x5655425367e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:09.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.064+0000 7fa8f96dcb80 1 --2- 172.21.15.12:0/2809716661 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x56554253a530 0x56554253c950 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:09.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.064+0000 7fa8f96dcb80 1 --2- 172.21.15.12:0/2809716661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5655425211f0 0x565542529ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:09.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.064+0000 7fa8f96dcb80 1 --2- 172.21.15.12:0/2809716661 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565542526b20 0x56554252a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:09.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.064+0000 7fa8f96dcb80 1 -- 172.21.15.12:0/2809716661 >> 172.21.15.12:0/2809716661 conn(0x5655423e6f80 msgr2=0x565542515f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:52:09.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.064+0000 7fa8c3fff700 0 client.26071 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:52:09.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.065+0000 7fa8f96dcb80 1 -- 172.21.15.12:0/2809716661 shutdown_connections 2024-05-17T22:52:09.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.066+0000 7fa8f96dcb80 1 -- 172.21.15.12:0/2809716661 wait complete. 2024-05-17T22:52:09.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.066+0000 7fa8f96dcb80 20 client.26071 trim_cache size 0 max 16384 2024-05-17T22:52:09.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.108+0000 7fa8f96dcb80 1 Processor -- start 2024-05-17T22:52:09.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.111+0000 7fa8f96dcb80 1 -- start start 2024-05-17T22:52:09.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.112+0000 7fa8f96dcb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5655425211f0 0x565542520cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:09.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.112+0000 7fa8f96dcb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565542526b20 0x565542515f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:09.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.112+0000 7fa8f96dcb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56554252a970 0x5655425258c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:09.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.112+0000 7fa8f96dcb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c80114f0 con 0x565542526b20 2024-05-17T22:52:09.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.112+0000 7fa8f96dcb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c800ed80 con 0x5655425211f0 2024-05-17T22:52:09.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.112+0000 7fa8f96dcb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c800cbd0 con 0x56554252a970 2024-05-17T22:52:09.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.113+0000 7fa8e3fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5655425211f0 0x565542520cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:09.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.113+0000 7fa8e3fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5655425211f0 0x565542520cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:47934/0 (socket says 172.21.15.12:47934) 2024-05-17T22:52:09.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.113+0000 7fa8e3fff700 1 -- 172.21.15.12:0/3484121922 learned_addr learned my addr 172.21.15.12:0/3484121922 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:52:09.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.113+0000 7fa8e3fff700 1 --2- 172.21.15.12:0/3484121922 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56554252a970 0x5655425258c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:09.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.115+0000 7fa8e3fff700 1 -- 172.21.15.12:0/3484121922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5655425211f0 msgr2=0x565542520cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:09.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.115+0000 7fa8e3fff700 1 --2- 172.21.15.12:0/3484121922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5655425211f0 0x565542520cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:09.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.115+0000 7fa8e3fff700 1 -- 172.21.15.12:0/3484121922 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565542526b20 msgr2=0x565542515f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:09.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.115+0000 7fa8e3fff700 1 --2- 172.21.15.12:0/3484121922 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565542526b20 0x565542515f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:09.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.115+0000 7fa8e3fff700 1 -- 172.21.15.12:0/3484121922 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56554251cb60 con 0x56554252a970 2024-05-17T22:52:09.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.115+0000 7fa8e3fff700 1 --2- 172.21.15.12:0/3484121922 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56554252a970 0x5655425258c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8005e10 tx=0x7fa8d8000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b55a4eada269cae server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:52:09.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.118+0000 7fa8ce7fc700 1 -- 172.21.15.12:0/3484121922 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8053080 con 0x56554252a970 2024-05-17T22:52:09.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.118+0000 7fa8ce7fc700 1 -- 172.21.15.12:0/3484121922 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8d803fbd0 con 0x56554252a970 2024-05-17T22:52:09.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.118+0000 7fa8ce7fc700 1 -- 172.21.15.12:0/3484121922 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d80315a0 con 0x56554252a970 2024-05-17T22:52:09.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.121+0000 7fa8f96dcb80 1 -- 172.21.15.12:0/3484121922 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56554252a970 msgr2=0x5655425258c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:09.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.121+0000 7fa8f96dcb80 1 --2- 172.21.15.12:0/3484121922 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56554252a970 0x5655425258c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8005e10 tx=0x7fa8d8000bc0 comp rx=0 tx=0).stop 2024-05-17T22:52:09.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.122+0000 7fa8f96dcb80 1 -- 172.21.15.12:0/3484121922 shutdown_connections 2024-05-17T22:52:09.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.122+0000 7fa8f96dcb80 1 --2- 172.21.15.12:0/3484121922 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56554252a970 0x5655425258c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:09.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.122+0000 7fa8f96dcb80 1 --2- 172.21.15.12:0/3484121922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5655425211f0 0x565542520cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:09.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.122+0000 7fa8f96dcb80 1 --2- 172.21.15.12:0/3484121922 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565542526b20 0x565542515f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:09.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.122+0000 7fa8f96dcb80 1 -- 172.21.15.12:0/3484121922 >> 172.21.15.12:0/3484121922 conn(0x5655423e6f80 msgr2=0x5655425278f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:52:09.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.125+0000 7fa8f96dcb80 1 -- 172.21.15.12:0/3484121922 shutdown_connections 2024-05-17T22:52:09.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.125+0000 7fa8f96dcb80 1 -- 172.21.15.12:0/3484121922 wait complete. 2024-05-17T22:52:09.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.125+0000 7fa8f96dcb80 1 -- start start 2024-05-17T22:52:09.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.126+0000 7fa8f96dcb80 1 Processor -- start 2024-05-17T22:52:09.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.126+0000 7fa8f96dcb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5655425211f0 0x56554253dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:09.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.127+0000 7fa8f96dcb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565542526b20 0x56554253e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:09.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.127+0000 7fa8e3fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5655425211f0 0x56554253dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:09.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.127+0000 7fa8e3fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5655425211f0 0x56554253dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:47938/0 (socket says 172.21.15.12:47938) 2024-05-17T22:52:09.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.127+0000 7fa8e3fff700 1 -- 172.21.15.12:0/799560907 learned_addr learned my addr 172.21.15.12:0/799560907 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:52:09.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.127+0000 7fa8f96dcb80 1 --2- 172.21.15.12:0/799560907 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56554252a970 0x5655425a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:09.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.127+0000 7fa8f96dcb80 1 -- 172.21.15.12:0/799560907 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c800d240 con 0x565542526b20 2024-05-17T22:52:09.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.127+0000 7fa8f96dcb80 1 -- 172.21.15.12:0/799560907 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c800e6b0 con 0x5655425211f0 2024-05-17T22:52:09.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.127+0000 7fa8f96dcb80 1 -- 172.21.15.12:0/799560907 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c800d910 con 0x56554252a970 2024-05-17T22:52:09.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.127+0000 7fa8f96dcb80 10 client.0 _ll_register_callbacks cb 0x5655423d9f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:52:09.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.128+0000 7fa8e3fff700 1 --2- 172.21.15.12:0/799560907 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56554252a970 0x5655425a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:09.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.128+0000 7fa8e3fff700 1 -- 172.21.15.12:0/799560907 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56554252a970 msgr2=0x5655425a3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:09.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.128+0000 7fa8e3fff700 1 --2- 172.21.15.12:0/799560907 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56554252a970 0x5655425a3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:09.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.128+0000 7fa8e3fff700 1 -- 172.21.15.12:0/799560907 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565542526b20 msgr2=0x56554253e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:09.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.128+0000 7fa8e3fff700 1 --2- 172.21.15.12:0/799560907 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565542526b20 0x56554253e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:09.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.128+0000 7fa8e3fff700 1 -- 172.21.15.12:0/799560907 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5655425a4040 con 0x5655425211f0 2024-05-17T22:52:09.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.128+0000 7fa8f96dcb80 1 -- 172.21.15.12:0/799560907 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8d803fbd0 con 0x5655425211f0 2024-05-17T22:52:09.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.130+0000 7fa8e3fff700 1 --2- 172.21.15.12:0/799560907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5655425211f0 0x56554253dd20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa8d803f860 tx=0x7fa8d8035040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=405ede86a74c6f01 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:52:09.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.130+0000 7fa8c3fff700 10 client.26135 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:52:09.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.131+0000 7fa8c3fff700 1 -- 172.21.15.12:0/799560907 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8053040 con 0x5655425211f0 2024-05-17T22:52:09.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.131+0000 7fa8c3fff700 1 -- 172.21.15.12:0/799560907 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fa8d8004770 con 0x5655425211f0 2024-05-17T22:52:09.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.131+0000 7fa8c3fff700 1 -- 172.21.15.12:0/799560907 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fa8d80049b0 con 0x5655425211f0 2024-05-17T22:52:09.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.132+0000 7fa8f96dcb80 10 client.26135 fetch_fsmap learned FSMap version 41 2024-05-17T22:52:09.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.132+0000 7fa8f96dcb80 1 -- 172.21.15.12:0/799560907 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({fsmap=41}) -- 0x5655425a5200 con 0x5655425211f0 2024-05-17T22:52:09.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.139+0000 7fa8c3fff700 1 -- 172.21.15.12:0/799560907 <== mon.2 v2:172.21.15.96:3300/0 4 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7fa8d80315a0 con 0x5655425211f0 2024-05-17T22:52:09.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.139+0000 7fa8f96dcb80 10 client.26135 fetch_fsmap finished waiting for FSMap version 41 2024-05-17T22:52:09.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.139+0000 7fa8f96dcb80 10 client.26135 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:52:09.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.139+0000 7fa8f96dcb80 20 client.26135 populate_metadata read hostname 'smithi012' 2024-05-17T22:52:09.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.139+0000 7fa8f96dcb80 1 --2- 172.21.15.12:0/799560907 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5655425a5690 0x5655425a7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:09.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.140+0000 7fa8f96dcb80 4 client.26135 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-17T22:52:09.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.140+0000 7fa8f96dcb80 1 -- 172.21.15.12:0/799560907 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56554231b3e0 con 0x5655425a5690 2024-05-17T22:52:09.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.141+0000 7fa8e8a9d700 1 --2- 172.21.15.12:0/799560907 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5655425a5690 0x5655425a7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:09.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.150+0000 7fa8e8a9d700 1 --2- 172.21.15.12:0/799560907 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5655425a5690 0x5655425a7ab0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4217e54f5541003a server_cookie=10906b2130912d2e in_seq=0 out_seq=0 2024-05-17T22:52:09.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.152+0000 7fa8c3fff700 10 client.26135 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:52:09.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.157+0000 7fa8c3fff700 1 -- 172.21.15.12:0/799560907 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56554231b3e0 con 0x5655425a5690 2024-05-17T22:52:09.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.157+0000 7fa8c3fff700 10 client.26135 handle_command_reply: tid=0 2024-05-17T22:52:09.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.159+0000 7fa8f96dcb80 1 client.26135 shutdown 2024-05-17T22:52:09.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.162+0000 7fa8f96dcb80 1 -- 172.21.15.12:0/799560907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5655425211f0 msgr2=0x56554253dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:09.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.162+0000 7fa8f96dcb80 1 --2- 172.21.15.12:0/799560907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5655425211f0 0x56554253dd20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa8d803f860 tx=0x7fa8d8035040 comp rx=0 tx=0).stop 2024-05-17T22:52:09.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.163+0000 7fa8f96dcb80 1 -- 172.21.15.12:0/799560907 shutdown_connections 2024-05-17T22:52:09.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.163+0000 7fa8f96dcb80 1 --2- 172.21.15.12:0/799560907 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56554252a970 0x5655425a3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:09.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.163+0000 7fa8f96dcb80 1 --2- 172.21.15.12:0/799560907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5655425211f0 0x56554253dd20 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:09.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.163+0000 7fa8f96dcb80 1 --2- 172.21.15.12:0/799560907 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5655425a5690 0x5655425a7ab0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:09.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.163+0000 7fa8f96dcb80 1 --2- 172.21.15.12:0/799560907 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565542526b20 0x56554253e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:09.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.163+0000 7fa8f96dcb80 1 -- 172.21.15.12:0/799560907 >> 172.21.15.12:0/799560907 conn(0x5655423e6f80 msgr2=0x5655425278f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:52:09.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.164+0000 7fa8c3fff700 0 client.26135 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:52:09.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.166+0000 7fa8f96dcb80 1 -- 172.21.15.12:0/799560907 shutdown_connections 2024-05-17T22:52:09.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.167+0000 7fa8f96dcb80 1 -- 172.21.15.12:0/799560907 wait complete. 2024-05-17T22:52:09.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.167+0000 7fa8f96dcb80 20 client.26135 trim_cache size 0 max 16384 2024-05-17T22:52:09.170 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:52:09.170 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-17T22:52:09.171 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "1b94f082-ae8d-4aec-9c8a-37f5b88b1835", 2024-05-17T22:52:09.171 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-17T22:52:09.171 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:52:09.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.167+0000 7fa8ead01700 1 -- 172.21.15.12:0/2686722839 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa8d40b2a50 msgr2=0x7fa8d40b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:09.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.167+0000 7fa8ead01700 1 --2- 172.21.15.12:0/2686722839 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa8d40b2a50 0x7fa8d40b4f60 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc009660 tx=0x7fa8dc0a6040 comp rx=0 tx=0).stop 2024-05-17T22:52:09.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.168+0000 7fa8ead01700 1 -- 172.21.15.12:0/2686722839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa8e411d550 msgr2=0x7fa8e4116620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:09.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.168+0000 7fa8ead01700 1 --2- 172.21.15.12:0/2686722839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa8e411d550 0x7fa8e4116620 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc00b770 tx=0x7fa8dc009400 comp rx=0 tx=0).stop 2024-05-17T22:52:09.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.170+0000 7fa8ead01700 1 -- 172.21.15.12:0/2686722839 shutdown_connections 2024-05-17T22:52:09.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.170+0000 7fa8ead01700 1 --2- 172.21.15.12:0/2686722839 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fa8d40b2a50 0x7fa8d40b4f60 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:09.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.170+0000 7fa8ead01700 1 --2- 172.21.15.12:0/2686722839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa8e411d550 0x7fa8e4116620 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:09.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.170+0000 7fa8ead01700 1 --2- 172.21.15.12:0/2686722839 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8e410a830 0x7fa8e411cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:09.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.170+0000 7fa8ead01700 1 --2- 172.21.15.12:0/2686722839 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8e410b180 0x7fa8e4116020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:09.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.170+0000 7fa8ead01700 1 -- 172.21.15.12:0/2686722839 >> 172.21.15.12:0/2686722839 conn(0x7fa8e406efc0 msgr2=0x7fa8e411db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:52:09.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.172+0000 7fa8ead01700 1 -- 172.21.15.12:0/2686722839 shutdown_connections 2024-05-17T22:52:09.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:09.172+0000 7fa8ead01700 1 -- 172.21.15.12:0/2686722839 wait complete. 2024-05-17T22:52:09.951 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-17T22:52:10.301 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:10 smithi012 ceph-mon[31632]: pgmap v826: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.5 MiB/s wr, 434 op/s 2024-05-17T22:52:10.301 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:10 smithi012 ceph-mon[31632]: scrub queued for path: / 2024-05-17T22:52:10.301 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:10 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:52:10.301 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:10 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:52:10.302 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:10 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:52:10.302 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:10 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:52:10.302 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:10 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:52:10.302 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:10 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:52:10.302 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:10 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:52:10.302 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:10 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:52:10.474 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:52:10.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:10 smithi096 ceph-mon[36065]: pgmap v826: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.5 MiB/s wr, 434 op/s 2024-05-17T22:52:10.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:10 smithi096 ceph-mon[36065]: scrub queued for path: / 2024-05-17T22:52:10.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:10 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:52:10.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:10 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:52:10.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:10 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:52:10.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:10 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:52:10.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:10 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:52:10.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:10 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:52:10.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:10 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:52:10.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:10 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:52:10.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:10 smithi151 ceph-mon[36101]: pgmap v826: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.5 MiB/s wr, 434 op/s 2024-05-17T22:52:10.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:10 smithi151 ceph-mon[36101]: scrub queued for path: / 2024-05-17T22:52:10.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:10 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:52:10.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:10 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:52:10.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:10 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:52:10.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:10 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:52:10.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:10 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:52:10.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:10 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:52:10.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:10 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:52:10.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:10 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:52:11.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 770.0 s, 449.0 tps, lat 222.213 ms stddev 258.693 2024-05-17T22:52:11.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:11 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:52:11.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:11 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:52:11.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:11 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:52:12.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.483+0000 7f5a8e254700 1 -- 172.21.15.12:0/292101227 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a8810a830 msgr2=0x7f5a8810ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:12.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.483+0000 7f5a867fc700 1 -- 172.21.15.12:0/292101227 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7800b700 con 0x7f5a8810a830 2024-05-17T22:52:12.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.483+0000 7f5a8e254700 1 --2- 172.21.15.12:0/292101227 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a8810a830 0x7f5a8810ac40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5a780098f0 tx=0x7f5a78009c00 comp rx=0 tx=0).stop 2024-05-17T22:52:12.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.484+0000 7f5a87fff700 1 --2- 172.21.15.12:0/292101227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a8811d550 0x7f5a8811f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:52:12.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.484+0000 7f5a8e254700 1 -- 172.21.15.12:0/292101227 shutdown_connections 2024-05-17T22:52:12.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.484+0000 7f5a8e254700 1 --2- 172.21.15.12:0/292101227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a8811d550 0x7f5a8811f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:12.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.484+0000 7f5a8e254700 1 --2- 172.21.15.12:0/292101227 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a8810a830 0x7f5a8810ac40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:12.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.484+0000 7f5a8e254700 1 --2- 172.21.15.12:0/292101227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a8810b180 0x7f5a8811d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:12.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.484+0000 7f5a8e254700 1 -- 172.21.15.12:0/292101227 >> 172.21.15.12:0/292101227 conn(0x7f5a8806efc0 msgr2=0x7f5a8806f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:52:12.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.489+0000 7f5a8e254700 1 -- 172.21.15.12:0/292101227 shutdown_connections 2024-05-17T22:52:12.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.492+0000 7f5a8e254700 1 -- 172.21.15.12:0/292101227 wait complete. 2024-05-17T22:52:12.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.493+0000 7f5a8e254700 1 Processor -- start 2024-05-17T22:52:12.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.501+0000 7f5a8e254700 1 -- start start 2024-05-17T22:52:12.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.501+0000 7f5a8e254700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a8810a830 0x7f5a8811cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:12.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.501+0000 7f5a8e254700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a8810b180 0x7f5a88116c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:12.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.501+0000 7f5a8e254700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a8811d550 0x7f5a88117140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:12.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.501+0000 7f5a8e254700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5a88112cc0 con 0x7f5a8810a830 2024-05-17T22:52:12.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.501+0000 7f5a8e254700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5a88112b40 con 0x7f5a8811d550 2024-05-17T22:52:12.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.501+0000 7f5a8e254700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5a88112e40 con 0x7f5a8810b180 2024-05-17T22:52:12.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.502+0000 7f5a86ffd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a8810b180 0x7f5a88116c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:12.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.502+0000 7f5a87fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a8811d550 0x7f5a88117140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:12.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.503+0000 7f5a877fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a8810a830 0x7f5a8811cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:12.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.503+0000 7f5a877fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a8810a830 0x7f5a8811cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58842/0 (socket says 172.21.15.12:58842) 2024-05-17T22:52:12.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.503+0000 7f5a877fe700 1 -- 172.21.15.12:0/4118938828 learned_addr learned my addr 172.21.15.12:0/4118938828 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:52:12.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.507+0000 7f5a877fe700 1 -- 172.21.15.12:0/4118938828 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a8810b180 msgr2=0x7f5a88116c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:12.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.507+0000 7f5a877fe700 1 --2- 172.21.15.12:0/4118938828 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a8810b180 0x7f5a88116c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:12.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.507+0000 7f5a877fe700 1 -- 172.21.15.12:0/4118938828 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a8811d550 msgr2=0x7f5a88117140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:12.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.507+0000 7f5a877fe700 1 --2- 172.21.15.12:0/4118938828 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a8811d550 0x7f5a88117140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:12.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.507+0000 7f5a877fe700 1 -- 172.21.15.12:0/4118938828 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a88117680 con 0x7f5a8810a830 2024-05-17T22:52:12.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.508+0000 7f5a877fe700 1 --2- 172.21.15.12:0/4118938828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a8810a830 0x7f5a8811cae0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5a78005c50 tx=0x7f5a7800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:52:12.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.510+0000 7f5a84ff9700 1 -- 172.21.15.12:0/4118938828 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a78025070 con 0x7f5a8810a830 2024-05-17T22:52:12.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.510+0000 7f5a84ff9700 1 -- 172.21.15.12:0/4118938828 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a7800b9b0 con 0x7f5a8810a830 2024-05-17T22:52:12.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.510+0000 7f5a84ff9700 1 -- 172.21.15.12:0/4118938828 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a78017400 con 0x7f5a8810a830 2024-05-17T22:52:12.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.513+0000 7f5a8e254700 1 -- 172.21.15.12:0/4118938828 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a88117910 con 0x7f5a8810a830 2024-05-17T22:52:12.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.513+0000 7f5a8e254700 1 -- 172.21.15.12:0/4118938828 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a881be780 con 0x7f5a8810a830 2024-05-17T22:52:12.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.519+0000 7f5a9cc2fb80 1 Processor -- start 2024-05-17T22:52:12.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.520+0000 7f5a9cc2fb80 1 -- start start 2024-05-17T22:52:12.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.520+0000 7f5a9cc2fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566c1b6e9f0 0x5566c1cf2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:12.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.520+0000 7f5a9cc2fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566c1cf2aa0 0x5566c1cecc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:12.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.520+0000 7f5a9cc2fb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566c1ced170 0x5566c1ced580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:12.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.520+0000 7f5a9cc2fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5566c1c1b290 con 0x5566c1cf2aa0 2024-05-17T22:52:12.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.520+0000 7f5a9cc2fb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5566c1c0afd0 con 0x5566c1ced170 2024-05-17T22:52:12.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.520+0000 7f5a9cc2fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5566c1c0ee70 con 0x5566c1b6e9f0 2024-05-17T22:52:12.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.525+0000 7f5a87fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566c1b6e9f0 0x5566c1cf2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:12.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.526+0000 7f5a86ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566c1ced170 0x5566c1ced580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:12.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.526+0000 7f5a86ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566c1ced170 0x5566c1ced580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:55942/0 (socket says 172.21.15.12:55942) 2024-05-17T22:52:12.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.526+0000 7f5a86ffd700 1 -- 172.21.15.12:0/3458535706 learned_addr learned my addr 172.21.15.12:0/3458535706 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:52:12.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.528+0000 7f5a84ff9700 1 -- 172.21.15.12:0/4118938828 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5a78003680 con 0x7f5a8810a830 2024-05-17T22:52:12.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.529+0000 7f5a84ff9700 1 --2- 172.21.15.12:0/4118938828 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5a700b2a00 0x7f5a700b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:12.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.529+0000 7f5a84ff9700 1 -- 172.21.15.12:0/4118938828 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f5a780a6a10 con 0x7f5a8810a830 2024-05-17T22:52:12.534 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:12 smithi012 ceph-mon[31632]: pgmap v827: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.2 MiB/s wr, 491 op/s 2024-05-17T22:52:12.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.531+0000 7f5a86ffd700 1 -- 172.21.15.12:0/3458535706 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566c1b6e9f0 msgr2=0x5566c1cf2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:12.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.531+0000 7f5a86ffd700 1 --2- 172.21.15.12:0/3458535706 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566c1b6e9f0 0x5566c1cf2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:12.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.531+0000 7f5a86ffd700 1 -- 172.21.15.12:0/3458535706 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566c1cf2aa0 msgr2=0x5566c1cecc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:12.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.531+0000 7f5a86ffd700 1 --2- 172.21.15.12:0/3458535706 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566c1cf2aa0 0x5566c1cecc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:12.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.531+0000 7f5a86ffd700 1 -- 172.21.15.12:0/3458535706 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566c1bb4590 con 0x5566c1ced170 2024-05-17T22:52:12.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.532+0000 7f5a86ffd700 1 --2- 172.21.15.12:0/3458535706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566c1ced170 0x5566c1ced580 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c009cb0 tx=0x7f5a7c0113e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67a61ec977c7b54f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:52:12.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.535+0000 7f5a87fff700 1 --2- 172.21.15.12:0/4118938828 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5a700b2a00 0x7f5a700b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:12.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.535+0000 7f5a6dffb700 1 -- 172.21.15.12:0/3458535706 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c011d10 con 0x5566c1ced170 2024-05-17T22:52:12.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.535+0000 7f5a6dffb700 1 -- 172.21.15.12:0/3458535706 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a7c012900 con 0x5566c1ced170 2024-05-17T22:52:12.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.536+0000 7f5a9cc2fb80 1 -- 172.21.15.12:0/3458535706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566c1ced170 msgr2=0x5566c1ced580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:12.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.536+0000 7f5a9cc2fb80 1 --2- 172.21.15.12:0/3458535706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566c1ced170 0x5566c1ced580 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c009cb0 tx=0x7f5a7c0113e0 comp rx=0 tx=0).stop 2024-05-17T22:52:12.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.541+0000 7f5a9cc2fb80 1 -- 172.21.15.12:0/3458535706 shutdown_connections 2024-05-17T22:52:12.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.541+0000 7f5a9cc2fb80 1 --2- 172.21.15.12:0/3458535706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566c1ced170 0x5566c1ced580 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:12.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.541+0000 7f5a9cc2fb80 1 --2- 172.21.15.12:0/3458535706 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566c1b6e9f0 0x5566c1cf2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:12.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.541+0000 7f5a9cc2fb80 1 --2- 172.21.15.12:0/3458535706 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566c1cf2aa0 0x5566c1cecc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:12.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.541+0000 7f5a9cc2fb80 1 -- 172.21.15.12:0/3458535706 >> 172.21.15.12:0/3458535706 conn(0x5566c1bb2f00 msgr2=0x5566c1ce7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:52:12.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.544+0000 7f5a87fff700 1 --2- 172.21.15.12:0/4118938828 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5a700b2a00 0x7f5a700b4f10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x5566c1ce8ae0 tx=0x7f5a80004ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:52:12.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.548+0000 7f5a9cc2fb80 1 -- 172.21.15.12:0/3458535706 shutdown_connections 2024-05-17T22:52:12.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.552+0000 7f5a9cc2fb80 1 -- 172.21.15.12:0/3458535706 wait complete. 2024-05-17T22:52:12.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.552+0000 7f5a9cc2fb80 1 -- start start 2024-05-17T22:52:12.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.556+0000 7f5a9cc2fb80 1 Processor -- start 2024-05-17T22:52:12.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.563+0000 7f5a9cc2fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566c1b6e9f0 0x5566c1cf6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:12.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.563+0000 7f5a9cc2fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566c1ced170 0x5566c1cf6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:12.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.563+0000 7f5a9cc2fb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566c1cf2aa0 0x5566c1cf6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:12.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.563+0000 7f5a9cc2fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5566c1b12150 con 0x5566c1ced170 2024-05-17T22:52:12.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.563+0000 7f5a9cc2fb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5566c1babe80 con 0x5566c1cf2aa0 2024-05-17T22:52:12.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.563+0000 7f5a9cc2fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5566c1cf93e0 con 0x5566c1b6e9f0 2024-05-17T22:52:12.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.563+0000 7f5a9cc2fb80 10 client.0 _ll_register_callbacks cb 0x5566c1ba5f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:52:12.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.564+0000 7f5a87fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566c1cf2aa0 0x5566c1cf6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:12.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.564+0000 7f5a87fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566c1cf2aa0 0x5566c1cf6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:55946/0 (socket says 172.21.15.12:55946) 2024-05-17T22:52:12.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.564+0000 7f5a87fff700 1 -- 172.21.15.12:0/2980477206 learned_addr learned my addr 172.21.15.12:0/2980477206 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:52:12.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.564+0000 7f5a86ffd700 1 --2- 172.21.15.12:0/2980477206 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566c1ced170 0x5566c1cf6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:12.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.565+0000 7f5a877fe700 1 --2- 172.21.15.12:0/2980477206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566c1b6e9f0 0x5566c1cf6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:12.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.566+0000 7f5a87fff700 1 -- 172.21.15.12:0/2980477206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566c1b6e9f0 msgr2=0x5566c1cf6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:12.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.566+0000 7f5a87fff700 1 --2- 172.21.15.12:0/2980477206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566c1b6e9f0 0x5566c1cf6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:12.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.566+0000 7f5a87fff700 1 -- 172.21.15.12:0/2980477206 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566c1ced170 msgr2=0x5566c1cf6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:12.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.566+0000 7f5a87fff700 1 --2- 172.21.15.12:0/2980477206 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566c1ced170 0x5566c1cf6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:12.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.566+0000 7f5a87fff700 1 -- 172.21.15.12:0/2980477206 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566c1b5b480 con 0x5566c1cf2aa0 2024-05-17T22:52:12.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.570+0000 7f5a87fff700 1 --2- 172.21.15.12:0/2980477206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566c1cf2aa0 0x5566c1cf6cf0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5a80015a90 tx=0x7f5a80009040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0d53e303c794c73 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:52:12.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.570+0000 7f5a66ffd700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:52:12.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.574+0000 7f5a66ffd700 1 -- 172.21.15.12:0/2980477206 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a80014c20 con 0x5566c1cf2aa0 2024-05-17T22:52:12.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.574+0000 7f5a66ffd700 1 -- 172.21.15.12:0/2980477206 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f5a80014dc0 con 0x5566c1cf2aa0 2024-05-17T22:52:12.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.575+0000 7f5a9cc2fb80 1 -- 172.21.15.12:0/2980477206 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5566c1c1d1e0 con 0x5566c1cf2aa0 2024-05-17T22:52:12.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.576+0000 7f5a66ffd700 1 -- 172.21.15.12:0/2980477206 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f5a8002b450 con 0x5566c1cf2aa0 2024-05-17T22:52:12.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.581+0000 7f5a9cc2fb80 10 client.26144 fetch_fsmap learned FSMap version 41 2024-05-17T22:52:12.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.581+0000 7f5a9cc2fb80 1 -- 172.21.15.12:0/2980477206 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5566c1d04000 con 0x5566c1cf2aa0 2024-05-17T22:52:12.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.583+0000 7f5a66ffd700 1 -- 172.21.15.12:0/2980477206 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5566c1c1d1e0 con 0x5566c1cf2aa0 2024-05-17T22:52:12.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.583+0000 7f5a66ffd700 1 -- 172.21.15.12:0/2980477206 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f5a80020480 con 0x5566c1cf2aa0 2024-05-17T22:52:12.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.583+0000 7f5a9cc2fb80 10 client.26144 fetch_fsmap finished waiting for FSMap version 41 2024-05-17T22:52:12.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.583+0000 7f5a9cc2fb80 10 client.26144 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:52:12.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.583+0000 7f5a9cc2fb80 20 client.26144 populate_metadata read hostname 'smithi012' 2024-05-17T22:52:12.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.583+0000 7f5a9cc2fb80 1 --2- 172.21.15.12:0/2980477206 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5566c1d06aa0 0x5566c1d08f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:12.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.583+0000 7f5a9cc2fb80 4 client.26144 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:52:12.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.583+0000 7f5a9cc2fb80 1 -- 172.21.15.12:0/2980477206 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566c1c169c0 con 0x5566c1d06aa0 2024-05-17T22:52:12.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.588+0000 7f5a877fe700 1 --2- 172.21.15.12:0/2980477206 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5566c1d06aa0 0x5566c1d08f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:12.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.594+0000 7f5a877fe700 1 --2- 172.21.15.12:0/2980477206 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5566c1d06aa0 0x5566c1d08f70 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f019115a3b241c2 server_cookie=96280170604f6196 in_seq=0 out_seq=0 2024-05-17T22:52:12.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.594+0000 7f5a66ffd700 10 client.26144 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:52:12.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.595+0000 7f5a66ffd700 1 -- 172.21.15.12:0/2980477206 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5566c1c169c0 con 0x5566c1d06aa0 2024-05-17T22:52:12.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.595+0000 7f5a66ffd700 10 client.26144 handle_command_reply: tid=0 2024-05-17T22:52:12.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.595+0000 7f5a9cc2fb80 1 client.26144 shutdown 2024-05-17T22:52:12.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.599+0000 7f5a9cc2fb80 1 -- 172.21.15.12:0/2980477206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566c1cf2aa0 msgr2=0x5566c1cf6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:12.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.599+0000 7f5a9cc2fb80 1 --2- 172.21.15.12:0/2980477206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566c1cf2aa0 0x5566c1cf6cf0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5a80015a90 tx=0x7f5a80009040 comp rx=0 tx=0).stop 2024-05-17T22:52:12.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.600+0000 7f5a9cc2fb80 1 -- 172.21.15.12:0/2980477206 shutdown_connections 2024-05-17T22:52:12.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.600+0000 7f5a9cc2fb80 1 --2- 172.21.15.12:0/2980477206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566c1cf2aa0 0x5566c1cf6cf0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:12.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.600+0000 7f5a9cc2fb80 1 --2- 172.21.15.12:0/2980477206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566c1b6e9f0 0x5566c1cf6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:12.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.600+0000 7f5a9cc2fb80 1 --2- 172.21.15.12:0/2980477206 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5566c1d06aa0 0x5566c1d08f70 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:12.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.600+0000 7f5a9cc2fb80 1 --2- 172.21.15.12:0/2980477206 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566c1ced170 0x5566c1cf6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:12.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.600+0000 7f5a9cc2fb80 1 -- 172.21.15.12:0/2980477206 >> 172.21.15.12:0/2980477206 conn(0x5566c1bb2f00 msgr2=0x5566c1ce1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:52:12.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.600+0000 7f5a66ffd700 0 client.26144 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:52:12.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.604+0000 7f5a9cc2fb80 1 -- 172.21.15.12:0/2980477206 shutdown_connections 2024-05-17T22:52:12.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.604+0000 7f5a9cc2fb80 1 -- 172.21.15.12:0/2980477206 wait complete. 2024-05-17T22:52:12.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.604+0000 7f5a9cc2fb80 20 client.26144 trim_cache size 0 max 16384 2024-05-17T22:52:12.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.650+0000 7f5a9cc2fb80 1 Processor -- start 2024-05-17T22:52:12.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.651+0000 7f5a9cc2fb80 1 -- start start 2024-05-17T22:52:12.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.651+0000 7f5a9cc2fb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566c1b6e9f0 0x5566c1ce1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:12.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.651+0000 7f5a9cc2fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566c1ced170 0x5566c1d01f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:12.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.651+0000 7f5a9cc2fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566c1cf2aa0 0x5566c1cf6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:12.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.652+0000 7f5a9cc2fb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5a881c9b90 con 0x5566c1cf2aa0 2024-05-17T22:52:12.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.652+0000 7f5a87fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566c1cf2aa0 0x5566c1cf6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:12.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.652+0000 7f5a9cc2fb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5a881c7420 con 0x5566c1b6e9f0 2024-05-17T22:52:12.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.652+0000 7f5a9cc2fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5a881c5270 con 0x5566c1ced170 2024-05-17T22:52:12.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.655+0000 7f5a877fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566c1b6e9f0 0x5566c1ce1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:12.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.655+0000 7f5a877fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566c1b6e9f0 0x5566c1ce1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:55956/0 (socket says 172.21.15.12:55956) 2024-05-17T22:52:12.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.655+0000 7f5a877fe700 1 -- 172.21.15.12:0/955409611 learned_addr learned my addr 172.21.15.12:0/955409611 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:52:12.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.657+0000 7f5a86ffd700 1 --2- 172.21.15.12:0/955409611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566c1ced170 0x5566c1d01f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:12.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.657+0000 7f5a87fff700 1 -- 172.21.15.12:0/955409611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566c1ced170 msgr2=0x5566c1d01f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:12.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.657+0000 7f5a87fff700 1 --2- 172.21.15.12:0/955409611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566c1ced170 0x5566c1d01f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:12.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.657+0000 7f5a87fff700 1 -- 172.21.15.12:0/955409611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566c1b6e9f0 msgr2=0x5566c1ce1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:12.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.657+0000 7f5a87fff700 1 --2- 172.21.15.12:0/955409611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566c1b6e9f0 0x5566c1ce1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:12.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.657+0000 7f5a87fff700 1 -- 172.21.15.12:0/955409611 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566c1ce9130 con 0x5566c1cf2aa0 2024-05-17T22:52:12.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.657+0000 7f5a877fe700 1 --2- 172.21.15.12:0/955409611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566c1b6e9f0 0x5566c1ce1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:52:12.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.657+0000 7f5a87fff700 1 --2- 172.21.15.12:0/955409611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566c1cf2aa0 0x5566c1cf6b00 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f5a80014660 tx=0x7f5a8000f0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4980fe77c1820709 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:52:12.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.660+0000 7f5a6d7fa700 1 -- 172.21.15.12:0/955409611 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a80038070 con 0x5566c1cf2aa0 2024-05-17T22:52:12.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.660+0000 7f5a6d7fa700 1 -- 172.21.15.12:0/955409611 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a80026070 con 0x5566c1cf2aa0 2024-05-17T22:52:12.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.660+0000 7f5a6d7fa700 1 -- 172.21.15.12:0/955409611 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8002b5f0 con 0x5566c1cf2aa0 2024-05-17T22:52:12.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.663+0000 7f5a9cc2fb80 1 -- 172.21.15.12:0/955409611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566c1cf2aa0 msgr2=0x5566c1cf6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:12.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.663+0000 7f5a9cc2fb80 1 --2- 172.21.15.12:0/955409611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566c1cf2aa0 0x5566c1cf6b00 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f5a80014660 tx=0x7f5a8000f0a0 comp rx=0 tx=0).stop 2024-05-17T22:52:12.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.663+0000 7f5a9cc2fb80 1 -- 172.21.15.12:0/955409611 shutdown_connections 2024-05-17T22:52:12.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.663+0000 7f5a9cc2fb80 1 --2- 172.21.15.12:0/955409611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566c1cf2aa0 0x5566c1cf6b00 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:12.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.663+0000 7f5a9cc2fb80 1 --2- 172.21.15.12:0/955409611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566c1b6e9f0 0x5566c1ce1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:12.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.663+0000 7f5a9cc2fb80 1 --2- 172.21.15.12:0/955409611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566c1ced170 0x5566c1d01f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:12.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.663+0000 7f5a9cc2fb80 1 -- 172.21.15.12:0/955409611 >> 172.21.15.12:0/955409611 conn(0x5566c1bb2f00 msgr2=0x5566c1cf7a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:52:12.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.667+0000 7f5a9cc2fb80 1 -- 172.21.15.12:0/955409611 shutdown_connections 2024-05-17T22:52:12.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.668+0000 7f5a9cc2fb80 1 -- 172.21.15.12:0/955409611 wait complete. 2024-05-17T22:52:12.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.669+0000 7f5a9cc2fb80 1 -- start start 2024-05-17T22:52:12.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.669+0000 7f5a9cc2fb80 1 Processor -- start 2024-05-17T22:52:12.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.669+0000 7f5a9cc2fb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566c1b6e9f0 0x5566c1cfe900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:12.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.670+0000 7f5a9cc2fb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566c1ced170 0x5566c1d0de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:12.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.670+0000 7f5a877fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566c1b6e9f0 0x5566c1cfe900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:12.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.670+0000 7f5a877fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566c1b6e9f0 0x5566c1cfe900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:55964/0 (socket says 172.21.15.12:55964) 2024-05-17T22:52:12.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.670+0000 7f5a877fe700 1 -- 172.21.15.12:0/2988938914 learned_addr learned my addr 172.21.15.12:0/2988938914 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:52:12.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.670+0000 7f5a9cc2fb80 1 --2- 172.21.15.12:0/2988938914 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566c1cf2aa0 0x5566c1d0e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:12.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.670+0000 7f5a9cc2fb80 1 -- 172.21.15.12:0/2988938914 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5a881c58e0 con 0x5566c1ced170 2024-05-17T22:52:12.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.670+0000 7f5a9cc2fb80 1 -- 172.21.15.12:0/2988938914 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5a881c6d50 con 0x5566c1b6e9f0 2024-05-17T22:52:12.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.670+0000 7f5a9cc2fb80 1 -- 172.21.15.12:0/2988938914 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5a881c5fb0 con 0x5566c1cf2aa0 2024-05-17T22:52:12.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.671+0000 7f5a9cc2fb80 10 client.0 _ll_register_callbacks cb 0x5566c1ba5f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:52:12.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.671+0000 7f5a877fe700 1 -- 172.21.15.12:0/2988938914 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566c1cf2aa0 msgr2=0x5566c1d0e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:12.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.671+0000 7f5a877fe700 1 --2- 172.21.15.12:0/2988938914 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566c1cf2aa0 0x5566c1d0e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:12.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.671+0000 7f5a877fe700 1 -- 172.21.15.12:0/2988938914 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566c1ced170 msgr2=0x5566c1d0de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:12.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.671+0000 7f5a877fe700 1 --2- 172.21.15.12:0/2988938914 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566c1ced170 0x5566c1d0de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:12.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.671+0000 7f5a877fe700 1 -- 172.21.15.12:0/2988938914 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566c1d0ebf0 con 0x5566c1b6e9f0 2024-05-17T22:52:12.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.671+0000 7f5a9cc2fb80 1 -- 172.21.15.12:0/2988938914 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a80026070 con 0x5566c1b6e9f0 2024-05-17T22:52:12.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.671+0000 7f5a877fe700 1 --2- 172.21.15.12:0/2988938914 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566c1b6e9f0 0x5566c1cfe900 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5a780e07e0 tx=0x7f5a780a4ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd1f92b993390b13 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:52:12.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.674+0000 7f5a66ffd700 10 client.26150 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:52:12.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.674+0000 7f5a66ffd700 1 -- 172.21.15.12:0/2988938914 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a780a6c10 con 0x5566c1b6e9f0 2024-05-17T22:52:12.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.674+0000 7f5a66ffd700 1 -- 172.21.15.12:0/2988938914 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f5a780a66b0 con 0x5566c1b6e9f0 2024-05-17T22:52:12.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.674+0000 7f5a66ffd700 1 -- 172.21.15.12:0/2988938914 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f5a780a6db0 con 0x5566c1b6e9f0 2024-05-17T22:52:12.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.675+0000 7f5a9cc2fb80 10 client.26150 fetch_fsmap learned FSMap version 41 2024-05-17T22:52:12.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.675+0000 7f5a9cc2fb80 1 -- 172.21.15.12:0/2988938914 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({fsmap=41}) -- 0x5566c1d71210 con 0x5566c1b6e9f0 2024-05-17T22:52:12.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.679+0000 7f5a66ffd700 1 -- 172.21.15.12:0/2988938914 <== mon.2 v2:172.21.15.96:3300/0 4 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f5a7801f030 con 0x5566c1b6e9f0 2024-05-17T22:52:12.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.679+0000 7f5a9cc2fb80 10 client.26150 fetch_fsmap finished waiting for FSMap version 41 2024-05-17T22:52:12.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.679+0000 7f5a9cc2fb80 10 client.26150 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:52:12.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.679+0000 7f5a9cc2fb80 20 client.26150 populate_metadata read hostname 'smithi012' 2024-05-17T22:52:12.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.679+0000 7f5a9cc2fb80 1 --2- 172.21.15.12:0/2988938914 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5566c1d71910 0x5566c1d73db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:12.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.679+0000 7f5a9cc2fb80 4 client.26150 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:52:12.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.679+0000 7f5a9cc2fb80 1 -- 172.21.15.12:0/2988938914 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5566c1ae7360 con 0x5566c1d71910 2024-05-17T22:52:12.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.680+0000 7f5a86ffd700 1 --2- 172.21.15.12:0/2988938914 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5566c1d71910 0x5566c1d73db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:12.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.696+0000 7f5a86ffd700 1 --2- 172.21.15.12:0/2988938914 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5566c1d71910 0x5566c1d73db0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa67d7c4657066b server_cookie=408e3c4271f479e4 in_seq=0 out_seq=0 2024-05-17T22:52:12.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.697+0000 7f5a66ffd700 10 client.26150 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:52:12.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.699+0000 7f5a66ffd700 1 -- 172.21.15.12:0/2988938914 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5566c1ae7360 con 0x5566c1d71910 2024-05-17T22:52:12.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.699+0000 7f5a66ffd700 10 client.26150 handle_command_reply: tid=0 2024-05-17T22:52:12.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.700+0000 7f5a9cc2fb80 1 client.26150 shutdown 2024-05-17T22:52:12.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.702+0000 7f5a9cc2fb80 1 -- 172.21.15.12:0/2988938914 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566c1b6e9f0 msgr2=0x5566c1cfe900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:12.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.702+0000 7f5a9cc2fb80 1 --2- 172.21.15.12:0/2988938914 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566c1b6e9f0 0x5566c1cfe900 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5a780e07e0 tx=0x7f5a780a4ad0 comp rx=0 tx=0).stop 2024-05-17T22:52:12.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.708+0000 7f5a9cc2fb80 1 -- 172.21.15.12:0/2988938914 shutdown_connections 2024-05-17T22:52:12.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.708+0000 7f5a9cc2fb80 1 --2- 172.21.15.12:0/2988938914 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566c1cf2aa0 0x5566c1d0e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:12.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.708+0000 7f5a9cc2fb80 1 --2- 172.21.15.12:0/2988938914 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566c1b6e9f0 0x5566c1cfe900 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:12.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.708+0000 7f5a9cc2fb80 1 --2- 172.21.15.12:0/2988938914 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5566c1d71910 0x5566c1d73db0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:12.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.708+0000 7f5a9cc2fb80 1 --2- 172.21.15.12:0/2988938914 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566c1ced170 0x5566c1d0de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:12.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.708+0000 7f5a9cc2fb80 1 -- 172.21.15.12:0/2988938914 >> 172.21.15.12:0/2988938914 conn(0x5566c1bb2f00 msgr2=0x5566c1cf83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:52:12.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.708+0000 7f5a66ffd700 0 client.26150 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:52:12.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.708+0000 7f5a9cc2fb80 1 -- 172.21.15.12:0/2988938914 shutdown_connections 2024-05-17T22:52:12.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.709+0000 7f5a9cc2fb80 1 -- 172.21.15.12:0/2988938914 wait complete. 2024-05-17T22:52:12.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.710+0000 7f5a9cc2fb80 20 client.26150 trim_cache size 0 max 16384 2024-05-17T22:52:12.713 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:52:12.713 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-17T22:52:12.713 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-17T22:52:12.713 INFO:teuthology.orchestra.run.smithi012.stdout: "1b94f082-ae8d-4aec-9c8a-37f5b88b1835": { 2024-05-17T22:52:12.714 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-17T22:52:12.714 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "1b94f082-ae8d-4aec-9c8a-37f5b88b1835", 2024-05-17T22:52:12.714 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-17T22:52:12.714 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:52:12.714 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:52:12.714 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:52:12.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.713+0000 7f5a8e254700 1 -- 172.21.15.12:0/4118938828 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5a700b2a00 msgr2=0x7f5a700b4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:12.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.713+0000 7f5a8e254700 1 --2- 172.21.15.12:0/4118938828 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5a700b2a00 0x7f5a700b4f10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x5566c1ce8ae0 tx=0x7f5a80004ad0 comp rx=0 tx=0).stop 2024-05-17T22:52:12.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.713+0000 7f5a8e254700 1 -- 172.21.15.12:0/4118938828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a8810a830 msgr2=0x7f5a8811cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:12.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.713+0000 7f5a8e254700 1 --2- 172.21.15.12:0/4118938828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a8810a830 0x7f5a8811cae0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5a78005c50 tx=0x7f5a7800fa00 comp rx=0 tx=0).stop 2024-05-17T22:52:12.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.713+0000 7f5a8e254700 1 -- 172.21.15.12:0/4118938828 shutdown_connections 2024-05-17T22:52:12.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.713+0000 7f5a8e254700 1 --2- 172.21.15.12:0/4118938828 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f5a700b2a00 0x7f5a700b4f10 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:12.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.713+0000 7f5a8e254700 1 --2- 172.21.15.12:0/4118938828 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a8811d550 0x7f5a88117140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:12.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.713+0000 7f5a8e254700 1 --2- 172.21.15.12:0/4118938828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a8810a830 0x7f5a8811cae0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:12.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.713+0000 7f5a8e254700 1 --2- 172.21.15.12:0/4118938828 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a8810b180 0x7f5a88116c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:12.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.713+0000 7f5a8e254700 1 -- 172.21.15.12:0/4118938828 >> 172.21.15.12:0/4118938828 conn(0x7f5a8806efc0 msgr2=0x7f5a8811daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:52:12.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.713+0000 7f5a8e254700 1 -- 172.21.15.12:0/4118938828 shutdown_connections 2024-05-17T22:52:12.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:12.713+0000 7f5a8e254700 1 -- 172.21.15.12:0/4118938828 wait complete. 2024-05-17T22:52:12.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:12 smithi151 ceph-mon[36101]: pgmap v827: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.2 MiB/s wr, 491 op/s 2024-05-17T22:52:12.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:12 smithi096 ceph-mon[36065]: pgmap v827: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.2 MiB/s wr, 491 op/s 2024-05-17T22:52:13.468 INFO:tasks.cephfs.filesystem:scrub status for tag:1b94f082-ae8d-4aec-9c8a-37f5b88b1835 - {'path': '/', 'tag': '1b94f082-ae8d-4aec-9c8a-37f5b88b1835', 'options': 'recursive,force'} 2024-05-17T22:52:14.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:14 smithi012 ceph-mon[31632]: pgmap v828: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.6 MiB/s wr, 431 op/s 2024-05-17T22:52:14.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:14 smithi151 ceph-mon[36101]: pgmap v828: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.6 MiB/s wr, 431 op/s 2024-05-17T22:52:14.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:14 smithi096 ceph-mon[36065]: pgmap v828: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.6 MiB/s wr, 431 op/s 2024-05-17T22:52:16.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 775.0 s, 476.8 tps, lat 207.617 ms stddev 278.947 2024-05-17T22:52:16.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:16 smithi012 ceph-mon[31632]: pgmap v829: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.4 MiB/s wr, 526 op/s 2024-05-17T22:52:16.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:16 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:52:16.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:16 smithi151 ceph-mon[36101]: pgmap v829: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.4 MiB/s wr, 526 op/s 2024-05-17T22:52:16.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:16 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:52:16.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:16 smithi096 ceph-mon[36065]: pgmap v829: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.4 MiB/s wr, 526 op/s 2024-05-17T22:52:16.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:16 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:52:17.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:17 smithi151 ceph-mon[36101]: pgmap v830: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.1 MiB/s wr, 415 op/s 2024-05-17T22:52:17.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:17 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:52:17.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:17 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:52:17.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:17 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:52:17.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:17 smithi096 ceph-mon[36065]: pgmap v830: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.1 MiB/s wr, 415 op/s 2024-05-17T22:52:17.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:17 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:52:17.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:17 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:52:17.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:17 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:52:17.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:17 smithi012 ceph-mon[31632]: pgmap v830: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.1 MiB/s wr, 415 op/s 2024-05-17T22:52:17.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:17 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:52:17.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:17 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:52:17.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:17 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:52:18.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:18 smithi151 ceph-mon[36101]: scrub summary: idle 2024-05-17T22:52:18.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:18 smithi096 ceph-mon[36065]: scrub summary: idle 2024-05-17T22:52:18.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:18 smithi012 ceph-mon[31632]: scrub summary: idle 2024-05-17T22:52:19.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:19 smithi151 ceph-mon[36101]: scrub summary: idle 2024-05-17T22:52:19.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:19 smithi151 ceph-mon[36101]: scrub summary: idle 2024-05-17T22:52:19.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:19 smithi151 ceph-mon[36101]: pgmap v831: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.3 MiB/s wr, 453 op/s 2024-05-17T22:52:19.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:19 smithi096 ceph-mon[36065]: scrub summary: idle 2024-05-17T22:52:19.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:19 smithi096 ceph-mon[36065]: scrub summary: idle 2024-05-17T22:52:19.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:19 smithi096 ceph-mon[36065]: pgmap v831: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.3 MiB/s wr, 453 op/s 2024-05-17T22:52:19.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:19 smithi012 ceph-mon[31632]: scrub summary: idle 2024-05-17T22:52:19.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:19 smithi012 ceph-mon[31632]: scrub summary: idle 2024-05-17T22:52:19.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:19 smithi012 ceph-mon[31632]: pgmap v831: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.3 MiB/s wr, 453 op/s 2024-05-17T22:52:21.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 780.0 s, 473.2 tps, lat 215.694 ms stddev 261.057 2024-05-17T22:52:22.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:21 smithi151 ceph-mon[36101]: pgmap v832: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.2 MiB/s wr, 516 op/s 2024-05-17T22:52:22.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:21 smithi096 ceph-mon[36065]: pgmap v832: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.2 MiB/s wr, 516 op/s 2024-05-17T22:52:22.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:21 smithi012 ceph-mon[31632]: pgmap v832: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.2 MiB/s wr, 516 op/s 2024-05-17T22:52:24.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:23 smithi151 ceph-mon[36101]: pgmap v833: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.2 MiB/s wr, 418 op/s 2024-05-17T22:52:24.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:23 smithi096 ceph-mon[36065]: pgmap v833: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.2 MiB/s wr, 418 op/s 2024-05-17T22:52:24.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:23 smithi012 ceph-mon[31632]: pgmap v833: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.2 MiB/s wr, 418 op/s 2024-05-17T22:52:26.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:25 smithi096 ceph-mon[36065]: pgmap v834: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.0 MiB/s wr, 527 op/s 2024-05-17T22:52:26.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:25 smithi012 ceph-mon[31632]: pgmap v834: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.0 MiB/s wr, 527 op/s 2024-05-17T22:52:26.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 785.0 s, 507.0 tps, lat 195.526 ms stddev 219.971 2024-05-17T22:52:26.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:25 smithi151 ceph-mon[36101]: pgmap v834: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.0 MiB/s wr, 527 op/s 2024-05-17T22:52:28.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:28 smithi096 ceph-mon[36065]: pgmap v835: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.2 MiB/s wr, 430 op/s 2024-05-17T22:52:28.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:28 smithi012 ceph-mon[31632]: pgmap v835: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.2 MiB/s wr, 430 op/s 2024-05-17T22:52:28.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:28 smithi151 ceph-mon[36101]: pgmap v835: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.2 MiB/s wr, 430 op/s 2024-05-17T22:52:30.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:30 smithi096 ceph-mon[36065]: pgmap v836: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.9 MiB/s wr, 484 op/s 2024-05-17T22:52:30.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:30 smithi012 ceph-mon[31632]: pgmap v836: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.9 MiB/s wr, 484 op/s 2024-05-17T22:52:30.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:30 smithi151 ceph-mon[36101]: pgmap v836: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.9 MiB/s wr, 484 op/s 2024-05-17T22:52:31.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 790.0 s, 472.8 tps, lat 214.753 ms stddev 261.493 2024-05-17T22:52:32.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:32 smithi096 ceph-mon[36065]: pgmap v837: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.2 MiB/s wr, 535 op/s 2024-05-17T22:52:32.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:32 smithi012 ceph-mon[31632]: pgmap v837: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.2 MiB/s wr, 535 op/s 2024-05-17T22:52:32.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:32 smithi151 ceph-mon[36101]: pgmap v837: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.2 MiB/s wr, 535 op/s 2024-05-17T22:52:34.292 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:34 smithi151 ceph-mon[36101]: pgmap v838: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.5 MiB/s wr, 448 op/s 2024-05-17T22:52:34.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:34 smithi096 ceph-mon[36065]: pgmap v838: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.5 MiB/s wr, 448 op/s 2024-05-17T22:52:34.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:34 smithi012 ceph-mon[31632]: pgmap v838: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.5 MiB/s wr, 448 op/s 2024-05-17T22:52:36.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:36 smithi096 ceph-mon[36065]: pgmap v839: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.0 MiB/s wr, 555 op/s 2024-05-17T22:52:36.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:36 smithi012 ceph-mon[31632]: pgmap v839: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.0 MiB/s wr, 555 op/s 2024-05-17T22:52:36.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 795.0 s, 433.4 tps, lat 232.872 ms stddev 323.297 2024-05-17T22:52:36.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:36 smithi151 ceph-mon[36101]: pgmap v839: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.0 MiB/s wr, 555 op/s 2024-05-17T22:52:38.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:38 smithi096 ceph-mon[36065]: pgmap v840: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.1 MiB/s wr, 442 op/s 2024-05-17T22:52:38.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:38 smithi012 ceph-mon[31632]: pgmap v840: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.1 MiB/s wr, 442 op/s 2024-05-17T22:52:38.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:38 smithi151 ceph-mon[36101]: pgmap v840: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.1 MiB/s wr, 442 op/s 2024-05-17T22:52:40.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:40 smithi096 ceph-mon[36065]: pgmap v841: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.7 MiB/s wr, 480 op/s 2024-05-17T22:52:40.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:40 smithi012 ceph-mon[31632]: pgmap v841: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.7 MiB/s wr, 480 op/s 2024-05-17T22:52:40.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:40 smithi151 ceph-mon[36101]: pgmap v841: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.7 MiB/s wr, 480 op/s 2024-05-17T22:52:41.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 800.0 s, 503.2 tps, lat 194.166 ms stddev 206.411 2024-05-17T22:52:42.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:42 smithi096 ceph-mon[36065]: pgmap v842: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 5.0 MiB/s wr, 537 op/s 2024-05-17T22:52:42.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:42 smithi012 ceph-mon[31632]: pgmap v842: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 5.0 MiB/s wr, 537 op/s 2024-05-17T22:52:42.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:42 smithi151 ceph-mon[36101]: pgmap v842: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 5.0 MiB/s wr, 537 op/s 2024-05-17T22:52:43.469 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-17T22:52:43.947 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:52:44.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:44 smithi012 ceph-mon[31632]: pgmap v843: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.5 MiB/s wr, 453 op/s 2024-05-17T22:52:44.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:44 smithi151 ceph-mon[36101]: pgmap v843: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.5 MiB/s wr, 453 op/s 2024-05-17T22:52:44.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:44 smithi096 ceph-mon[36065]: pgmap v843: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.5 MiB/s wr, 453 op/s 2024-05-17T22:52:46.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:46 smithi012 ceph-mon[31632]: pgmap v844: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.1 MiB/s wr, 545 op/s 2024-05-17T22:52:46.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 805.0 s, 474.8 tps, lat 216.341 ms stddev 259.061 2024-05-17T22:52:46.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:46 smithi151 ceph-mon[36101]: pgmap v844: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.1 MiB/s wr, 545 op/s 2024-05-17T22:52:46.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:46 smithi096 ceph-mon[36065]: pgmap v844: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.1 MiB/s wr, 545 op/s 2024-05-17T22:52:47.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.074+0000 7fba17a16700 1 -- 172.21.15.12:0/1213475566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba1010f7b0 msgr2=0x7fba10117110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:47.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.074+0000 7fba17a16700 1 --2- 172.21.15.12:0/1213475566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba1010f7b0 0x7fba10117110 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fba0c009990 tx=0x7fba0c009ca0 comp rx=0 tx=0).stop 2024-05-17T22:52:47.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.076+0000 7fba17a16700 1 -- 172.21.15.12:0/1213475566 shutdown_connections 2024-05-17T22:52:47.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.076+0000 7fba17a16700 1 --2- 172.21.15.12:0/1213475566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba1010f7b0 0x7fba10117110 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:47.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.076+0000 7fba17a16700 1 --2- 172.21.15.12:0/1213475566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba1010a530 0x7fba1010a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:47.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.076+0000 7fba17a16700 1 --2- 172.21.15.12:0/1213475566 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba1010ee60 0x7fba1010f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:47.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.076+0000 7fba17a16700 1 -- 172.21.15.12:0/1213475566 >> 172.21.15.12:0/1213475566 conn(0x7fba10078570 msgr2=0x7fba10108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:52:47.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.082+0000 7fba17a16700 1 -- 172.21.15.12:0/1213475566 shutdown_connections 2024-05-17T22:52:47.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.084+0000 7fba17a16700 1 -- 172.21.15.12:0/1213475566 wait complete. 2024-05-17T22:52:47.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.085+0000 7fba17a16700 1 Processor -- start 2024-05-17T22:52:47.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.087+0000 7fba17a16700 1 -- start start 2024-05-17T22:52:47.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.087+0000 7fba17a16700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba1010a530 0x7fba100840e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:47.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.087+0000 7fba17a16700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba1010ee60 0x7fba1007c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:47.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.087+0000 7fba17a16700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba1010f7b0 0x7fba1007c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:47.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.087+0000 7fba17a16700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fba1011a470 con 0x7fba1010a530 2024-05-17T22:52:47.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.087+0000 7fba17a16700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fba1011a2f0 con 0x7fba1010f7b0 2024-05-17T22:52:47.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.087+0000 7fba17a16700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fba1011a5f0 con 0x7fba1010ee60 2024-05-17T22:52:47.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.088+0000 7fba157b2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba1010a530 0x7fba100840e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:47.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.088+0000 7fba14fb1700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba1010ee60 0x7fba1007c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:47.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.088+0000 7fba157b2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba1010a530 0x7fba100840e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46560/0 (socket says 172.21.15.12:46560) 2024-05-17T22:52:47.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.088+0000 7fba157b2700 1 -- 172.21.15.12:0/1051404517 learned_addr learned my addr 172.21.15.12:0/1051404517 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:52:47.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.088+0000 7fba15fb3700 1 --2- 172.21.15.12:0/1051404517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba1010f7b0 0x7fba1007c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:47.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.089+0000 7fba15fb3700 1 -- 172.21.15.12:0/1051404517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba1010ee60 msgr2=0x7fba1007c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:47.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.089+0000 7fba15fb3700 1 --2- 172.21.15.12:0/1051404517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba1010ee60 0x7fba1007c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:47.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.089+0000 7fba15fb3700 1 -- 172.21.15.12:0/1051404517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba1010a530 msgr2=0x7fba100840e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:47.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.089+0000 7fba15fb3700 1 --2- 172.21.15.12:0/1051404517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba1010a530 0x7fba100840e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:47.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.089+0000 7fba15fb3700 1 -- 172.21.15.12:0/1051404517 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba1007cd90 con 0x7fba1010f7b0 2024-05-17T22:52:47.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.089+0000 7fba157b2700 1 --2- 172.21.15.12:0/1051404517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba1010a530 0x7fba100840e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:52:47.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.093+0000 7fba15fb3700 1 --2- 172.21.15.12:0/1051404517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba1010f7b0 0x7fba1007c690 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fba0c005b40 tx=0x7fba0c009ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:52:47.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.093+0000 7fba067fc700 1 -- 172.21.15.12:0/1051404517 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0c027070 con 0x7fba1010f7b0 2024-05-17T22:52:47.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.094+0000 7fba067fc700 1 -- 172.21.15.12:0/1051404517 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba0c003f20 con 0x7fba1010f7b0 2024-05-17T22:52:47.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.094+0000 7fba067fc700 1 -- 172.21.15.12:0/1051404517 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0c022c40 con 0x7fba1010f7b0 2024-05-17T22:52:47.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.096+0000 7fba17a16700 1 -- 172.21.15.12:0/1051404517 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba101b5b40 con 0x7fba1010f7b0 2024-05-17T22:52:47.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.096+0000 7fba17a16700 1 -- 172.21.15.12:0/1051404517 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba101b5ee0 con 0x7fba1010f7b0 2024-05-17T22:52:47.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.103+0000 7fba263f1b80 1 Processor -- start 2024-05-17T22:52:47.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.103+0000 7fba263f1b80 1 -- start start 2024-05-17T22:52:47.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.104+0000 7fba263f1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5654785ed9f0 0x565478771560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:47.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.104+0000 7fba263f1b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565478771aa0 0x56547876bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:47.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.104+0000 7fba263f1b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56547876c170 0x56547876c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:47.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.104+0000 7fba263f1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56547869a290 con 0x5654785ed9f0 2024-05-17T22:52:47.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.104+0000 7fba263f1b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x565478689fd0 con 0x56547876c170 2024-05-17T22:52:47.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.104+0000 7fba263f1b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56547868de70 con 0x565478771aa0 2024-05-17T22:52:47.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.104+0000 7fba14fb1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56547876c170 0x56547876c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:47.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.104+0000 7fba14fb1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5654785ed9f0 0x565478771560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:47.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.104+0000 7fba14fb1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5654785ed9f0 0x565478771560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46566/0 (socket says 172.21.15.12:46566) 2024-05-17T22:52:47.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.104+0000 7fba14fb1700 1 -- 172.21.15.12:0/4126361446 learned_addr learned my addr 172.21.15.12:0/4126361446 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:52:47.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.105+0000 7fba157b2700 1 --2- 172.21.15.12:0/4126361446 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565478771aa0 0x56547876bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:47.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.109+0000 7fba067fc700 1 -- 172.21.15.12:0/1051404517 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fba0c022470 con 0x7fba1010f7b0 2024-05-17T22:52:47.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.110+0000 7fba067fc700 1 --2- 172.21.15.12:0/1051404517 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb9fc0b2980 0x7fb9fc0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:47.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.110+0000 7fba067fc700 1 -- 172.21.15.12:0/1051404517 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fba0c0e3810 con 0x7fba1010f7b0 2024-05-17T22:52:47.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.112+0000 7fba15fb3700 1 --2- 172.21.15.12:0/1051404517 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb9fc0b2980 0x7fb9fc0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:47.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.115+0000 7fba14fb1700 1 -- 172.21.15.12:0/4126361446 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565478771aa0 msgr2=0x56547876bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:47.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.115+0000 7fba14fb1700 1 --2- 172.21.15.12:0/4126361446 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565478771aa0 0x56547876bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:47.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.115+0000 7fba14fb1700 1 -- 172.21.15.12:0/4126361446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56547876c170 msgr2=0x56547876c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:47.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.115+0000 7fba14fb1700 1 --2- 172.21.15.12:0/4126361446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56547876c170 0x56547876c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:47.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.115+0000 7fba14fb1700 1 -- 172.21.15.12:0/4126361446 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565478633590 con 0x5654785ed9f0 2024-05-17T22:52:47.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.115+0000 7fba14fb1700 1 --2- 172.21.15.12:0/4126361446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5654785ed9f0 0x565478771560 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fba000059d0 tx=0x7fba00005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=387e2024dee8f6f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:52:47.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.115+0000 7fb9fb7fe700 1 -- 172.21.15.12:0/4126361446 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba00004a30 con 0x5654785ed9f0 2024-05-17T22:52:47.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.116+0000 7fb9fb7fe700 1 -- 172.21.15.12:0/4126361446 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba000054d0 con 0x5654785ed9f0 2024-05-17T22:52:47.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.117+0000 7fba263f1b80 1 -- 172.21.15.12:0/4126361446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5654785ed9f0 msgr2=0x565478771560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:47.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.117+0000 7fba263f1b80 1 --2- 172.21.15.12:0/4126361446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5654785ed9f0 0x565478771560 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fba000059d0 tx=0x7fba00005db0 comp rx=0 tx=0).stop 2024-05-17T22:52:47.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.122+0000 7fba263f1b80 1 -- 172.21.15.12:0/4126361446 shutdown_connections 2024-05-17T22:52:47.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.122+0000 7fba263f1b80 1 --2- 172.21.15.12:0/4126361446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56547876c170 0x56547876c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:47.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.122+0000 7fba263f1b80 1 --2- 172.21.15.12:0/4126361446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5654785ed9f0 0x565478771560 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:47.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.122+0000 7fba263f1b80 1 --2- 172.21.15.12:0/4126361446 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565478771aa0 0x56547876bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:47.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.122+0000 7fba263f1b80 1 -- 172.21.15.12:0/4126361446 >> 172.21.15.12:0/4126361446 conn(0x565478631f00 msgr2=0x565478766b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:52:47.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.124+0000 7fba15fb3700 1 --2- 172.21.15.12:0/1051404517 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb9fc0b2980 0x7fb9fc0b4e90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fba0c0aa7d0 tx=0x7fba0c092040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:52:47.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.129+0000 7fba263f1b80 1 -- 172.21.15.12:0/4126361446 shutdown_connections 2024-05-17T22:52:47.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.130+0000 7fba263f1b80 1 -- 172.21.15.12:0/4126361446 wait complete. 2024-05-17T22:52:47.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.131+0000 7fba263f1b80 1 -- start start 2024-05-17T22:52:47.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.131+0000 7fba263f1b80 1 Processor -- start 2024-05-17T22:52:47.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.132+0000 7fba263f1b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654785ed9f0 0x5654787751d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:47.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.132+0000 7fba263f1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56547876c170 0x565478775710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:47.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.132+0000 7fba263f1b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565478771aa0 0x565478775c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:47.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.132+0000 7fba263f1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x565478591150 con 0x56547876c170 2024-05-17T22:52:47.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.132+0000 7fba263f1b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56547862ae80 con 0x5654785ed9f0 2024-05-17T22:52:47.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.132+0000 7fba263f1b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5654787893f0 con 0x565478771aa0 2024-05-17T22:52:47.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.132+0000 7fba263f1b80 10 client.0 _ll_register_callbacks cb 0x565478624f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:52:47.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.133+0000 7fba14fb1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654785ed9f0 0x5654787751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:47.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.133+0000 7fba14fb1700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565478771aa0 0x565478775c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:47.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.133+0000 7fba14fb1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654785ed9f0 0x5654787751d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:48402/0 (socket says 172.21.15.12:48402) 2024-05-17T22:52:47.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.133+0000 7fba14fb1700 1 -- 172.21.15.12:0/1307293452 learned_addr learned my addr 172.21.15.12:0/1307293452 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:52:47.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.134+0000 7fba157b2700 1 --2- 172.21.15.12:0/1307293452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56547876c170 0x565478775710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:47.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.134+0000 7fba14fb1700 1 -- 172.21.15.12:0/1307293452 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565478771aa0 msgr2=0x565478775c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:47.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.134+0000 7fba14fb1700 1 --2- 172.21.15.12:0/1307293452 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565478771aa0 0x565478775c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:47.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.134+0000 7fba14fb1700 1 -- 172.21.15.12:0/1307293452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56547876c170 msgr2=0x565478775710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:47.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.134+0000 7fba14fb1700 1 --2- 172.21.15.12:0/1307293452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56547876c170 0x565478775710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:47.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.134+0000 7fba14fb1700 1 -- 172.21.15.12:0/1307293452 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654785da480 con 0x5654785ed9f0 2024-05-17T22:52:47.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.135+0000 7fba14fb1700 1 --2- 172.21.15.12:0/1307293452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654785ed9f0 0x5654787751d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fba00005e20 tx=0x7fba00004ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cce7e7de66b3447d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:52:47.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.136+0000 7fba157b2700 1 --2- 172.21.15.12:0/1307293452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56547876c170 0x565478775710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:52:47.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.138+0000 7fb9f8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:52:47.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.138+0000 7fb9f8ff9700 1 -- 172.21.15.12:0/1307293452 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba00004750 con 0x5654785ed9f0 2024-05-17T22:52:47.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.138+0000 7fba263f1b80 1 -- 172.21.15.12:0/1307293452 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba000054d0 con 0x5654785ed9f0 2024-05-17T22:52:47.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.138+0000 7fb9f8ff9700 1 -- 172.21.15.12:0/1307293452 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fba000048f0 con 0x5654785ed9f0 2024-05-17T22:52:47.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.138+0000 7fb9f8ff9700 1 -- 172.21.15.12:0/1307293452 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fba00030050 con 0x5654785ed9f0 2024-05-17T22:52:47.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.138+0000 7fba263f1b80 10 client.26162 fetch_fsmap learned FSMap version 41 2024-05-17T22:52:47.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.138+0000 7fba263f1b80 1 -- 172.21.15.12:0/1307293452 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x565478793e60 con 0x5654785ed9f0 2024-05-17T22:52:47.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.139+0000 7fb9f8ff9700 1 -- 172.21.15.12:0/1307293452 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba000054d0 con 0x5654785ed9f0 2024-05-17T22:52:47.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.139+0000 7fb9f8ff9700 1 -- 172.21.15.12:0/1307293452 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7fba0001f7a0 con 0x5654785ed9f0 2024-05-17T22:52:47.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.139+0000 7fba263f1b80 10 client.26162 fetch_fsmap finished waiting for FSMap version 41 2024-05-17T22:52:47.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.139+0000 7fba263f1b80 10 client.26162 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:52:47.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.139+0000 7fba263f1b80 20 client.26162 populate_metadata read hostname 'smithi012' 2024-05-17T22:52:47.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.139+0000 7fba263f1b80 1 --2- 172.21.15.12:0/1307293452 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x565478796790 0x565478798c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:47.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.139+0000 7fba263f1b80 4 client.26162 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:52:47.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.139+0000 7fba263f1b80 1 -- 172.21.15.12:0/1307293452 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56547869c1e0 con 0x565478796790 2024-05-17T22:52:47.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.140+0000 7fba157b2700 1 --2- 172.21.15.12:0/1307293452 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x565478796790 0x565478798c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:47.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.152+0000 7fba157b2700 1 --2- 172.21.15.12:0/1307293452 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x565478796790 0x565478798c00 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b70501be777e714 server_cookie=ce7215578de9560b in_seq=0 out_seq=0 2024-05-17T22:52:47.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.153+0000 7fb9f8ff9700 10 client.26162 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:52:47.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.155+0000 7fb9f8ff9700 1 -- 172.21.15.12:0/1307293452 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56547869c1e0 con 0x565478796790 2024-05-17T22:52:47.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.155+0000 7fb9f8ff9700 10 client.26162 handle_command_reply: tid=0 2024-05-17T22:52:47.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.155+0000 7fba263f1b80 1 client.26162 shutdown 2024-05-17T22:52:47.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.159+0000 7fba263f1b80 1 -- 172.21.15.12:0/1307293452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654785ed9f0 msgr2=0x5654787751d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:47.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.159+0000 7fba263f1b80 1 --2- 172.21.15.12:0/1307293452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654785ed9f0 0x5654787751d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fba00005e20 tx=0x7fba00004ce0 comp rx=0 tx=0).stop 2024-05-17T22:52:47.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.161+0000 7fba263f1b80 1 -- 172.21.15.12:0/1307293452 shutdown_connections 2024-05-17T22:52:47.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.161+0000 7fba263f1b80 1 --2- 172.21.15.12:0/1307293452 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565478771aa0 0x565478775c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:47.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.161+0000 7fba263f1b80 1 --2- 172.21.15.12:0/1307293452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654785ed9f0 0x5654787751d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:47.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.161+0000 7fba263f1b80 1 --2- 172.21.15.12:0/1307293452 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x565478796790 0x565478798c00 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:47.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.161+0000 7fba263f1b80 1 --2- 172.21.15.12:0/1307293452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56547876c170 0x565478775710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:47.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.161+0000 7fb9f8ff9700 0 client.26162 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:52:47.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.161+0000 7fba263f1b80 1 -- 172.21.15.12:0/1307293452 >> 172.21.15.12:0/1307293452 conn(0x565478631f00 msgr2=0x565478760ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:52:47.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.162+0000 7fba263f1b80 1 -- 172.21.15.12:0/1307293452 shutdown_connections 2024-05-17T22:52:47.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.162+0000 7fba263f1b80 1 -- 172.21.15.12:0/1307293452 wait complete. 2024-05-17T22:52:47.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.163+0000 7fba263f1b80 20 client.26162 trim_cache size 0 max 16384 2024-05-17T22:52:47.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.198+0000 7fba263f1b80 1 Processor -- start 2024-05-17T22:52:47.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.199+0000 7fba263f1b80 1 -- start start 2024-05-17T22:52:47.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.199+0000 7fba263f1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565478631f00 0x5654787922c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:47.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.199+0000 7fba263f1b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654785ed9f0 0x565478775aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:47.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.199+0000 7fba263f1b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56547876c170 0x565478760eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:47.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.199+0000 7fba263f1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fba101b8b30 con 0x565478631f00 2024-05-17T22:52:47.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.199+0000 7fba263f1b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fba101b63c0 con 0x5654785ed9f0 2024-05-17T22:52:47.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.199+0000 7fba263f1b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fba10073b50 con 0x56547876c170 2024-05-17T22:52:47.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.199+0000 7fba157b2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654785ed9f0 0x565478775aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:47.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.199+0000 7fba14fb1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565478631f00 0x5654787922c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:47.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.199+0000 7fba14fb1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565478631f00 0x5654787922c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46584/0 (socket says 172.21.15.12:46584) 2024-05-17T22:52:47.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.199+0000 7fba14fb1700 1 -- 172.21.15.12:0/384502693 learned_addr learned my addr 172.21.15.12:0/384502693 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:52:47.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.200+0000 7fba14fb1700 1 --2- 172.21.15.12:0/384502693 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56547876c170 0x565478760eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:47.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.200+0000 7fba14fb1700 1 -- 172.21.15.12:0/384502693 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56547876c170 msgr2=0x565478760eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:47.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.200+0000 7fba14fb1700 1 --2- 172.21.15.12:0/384502693 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56547876c170 0x565478760eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:47.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.200+0000 7fba14fb1700 1 -- 172.21.15.12:0/384502693 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654785ed9f0 msgr2=0x565478775aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:47.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.200+0000 7fba14fb1700 1 --2- 172.21.15.12:0/384502693 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654785ed9f0 0x565478775aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:47.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.200+0000 7fba14fb1700 1 -- 172.21.15.12:0/384502693 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565478767720 con 0x565478631f00 2024-05-17T22:52:47.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.200+0000 7fba14fb1700 1 --2- 172.21.15.12:0/384502693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565478631f00 0x5654787922c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fba00013fd0 tx=0x7fba0000de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=896810be78a8ab0e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:52:47.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.200+0000 7fb9faffd700 1 -- 172.21.15.12:0/384502693 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0000d930 con 0x565478631f00 2024-05-17T22:52:47.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.201+0000 7fb9faffd700 1 -- 172.21.15.12:0/384502693 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba00024070 con 0x565478631f00 2024-05-17T22:52:47.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.201+0000 7fb9faffd700 1 -- 172.21.15.12:0/384502693 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba00004020 con 0x565478631f00 2024-05-17T22:52:47.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.202+0000 7fba263f1b80 1 -- 172.21.15.12:0/384502693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565478631f00 msgr2=0x5654787922c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:47.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.202+0000 7fba263f1b80 1 --2- 172.21.15.12:0/384502693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565478631f00 0x5654787922c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fba00013fd0 tx=0x7fba0000de20 comp rx=0 tx=0).stop 2024-05-17T22:52:47.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.202+0000 7fba263f1b80 1 -- 172.21.15.12:0/384502693 shutdown_connections 2024-05-17T22:52:47.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.202+0000 7fba263f1b80 1 --2- 172.21.15.12:0/384502693 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56547876c170 0x565478760eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:47.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.202+0000 7fba263f1b80 1 --2- 172.21.15.12:0/384502693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565478631f00 0x5654787922c0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:47.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.202+0000 7fba263f1b80 1 --2- 172.21.15.12:0/384502693 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654785ed9f0 0x565478775aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:47.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.202+0000 7fba263f1b80 1 -- 172.21.15.12:0/384502693 >> 172.21.15.12:0/384502693 conn(0x565478767c10 msgr2=0x565478772bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:52:47.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.202+0000 7fba263f1b80 1 -- 172.21.15.12:0/384502693 shutdown_connections 2024-05-17T22:52:47.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.202+0000 7fba263f1b80 1 -- 172.21.15.12:0/384502693 wait complete. 2024-05-17T22:52:47.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.203+0000 7fba263f1b80 1 -- start start 2024-05-17T22:52:47.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.203+0000 7fba263f1b80 1 Processor -- start 2024-05-17T22:52:47.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.203+0000 7fba263f1b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654785ed9f0 0x56547878ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:47.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.203+0000 7fba263f1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565478631f00 0x56547879d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:47.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.203+0000 7fba263f1b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56547876c170 0x56547879dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:47.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.203+0000 7fba263f1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fba100741c0 con 0x565478631f00 2024-05-17T22:52:47.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.203+0000 7fba263f1b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fba10075630 con 0x5654785ed9f0 2024-05-17T22:52:47.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.203+0000 7fba263f1b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fba10074890 con 0x56547876c170 2024-05-17T22:52:47.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.203+0000 7fba263f1b80 10 client.0 _ll_register_callbacks cb 0x565478624f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:52:47.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.203+0000 7fba157b2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565478631f00 0x56547879d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:47.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.203+0000 7fba157b2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565478631f00 0x56547879d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46588/0 (socket says 172.21.15.12:46588) 2024-05-17T22:52:47.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.204+0000 7fba157b2700 1 -- 172.21.15.12:0/3641039228 learned_addr learned my addr 172.21.15.12:0/3641039228 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:52:47.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.204+0000 7fba157b2700 1 -- 172.21.15.12:0/3641039228 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56547876c170 msgr2=0x56547879dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:47.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.204+0000 7fba14fb1700 1 --2- 172.21.15.12:0/3641039228 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56547876c170 0x56547879dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:47.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.204+0000 7fba157b2700 1 --2- 172.21.15.12:0/3641039228 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56547876c170 0x56547879dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:47.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.204+0000 7fba157b2700 1 -- 172.21.15.12:0/3641039228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654785ed9f0 msgr2=0x56547878ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:47.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.204+0000 7fba157b2700 1 --2- 172.21.15.12:0/3641039228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654785ed9f0 0x56547878ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:47.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.204+0000 7fba157b2700 1 -- 172.21.15.12:0/3641039228 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654787710d0 con 0x565478631f00 2024-05-17T22:52:47.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.204+0000 7fba14fb1700 1 --2- 172.21.15.12:0/3641039228 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56547876c170 0x56547879dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:52:47.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.204+0000 7fba157b2700 1 --2- 172.21.15.12:0/3641039228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565478631f00 0x56547879d9f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fba08009ec0 tx=0x7fba08007cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2013b5fb5bd2afc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:52:47.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.204+0000 7fb9f8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:52:47.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.205+0000 7fb9f8ff9700 1 -- 172.21.15.12:0/3641039228 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba08017c00 con 0x565478631f00 2024-05-17T22:52:47.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.205+0000 7fba263f1b80 1 -- 172.21.15.12:0/3641039228 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba00024070 con 0x565478631f00 2024-05-17T22:52:47.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.205+0000 7fb9f8ff9700 1 -- 172.21.15.12:0/3641039228 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fba08017da0 con 0x565478631f00 2024-05-17T22:52:47.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.205+0000 7fb9f8ff9700 1 -- 172.21.15.12:0/3641039228 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fba0802e050 con 0x565478631f00 2024-05-17T22:52:47.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.205+0000 7fba263f1b80 10 client.16455 fetch_fsmap learned FSMap version 41 2024-05-17T22:52:47.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.205+0000 7fba263f1b80 1 -- 172.21.15.12:0/3641039228 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x565478800e50 con 0x565478631f00 2024-05-17T22:52:47.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.205+0000 7fb9f8ff9700 1 -- 172.21.15.12:0/3641039228 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba00024070 con 0x565478631f00 2024-05-17T22:52:47.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.206+0000 7fb9f8ff9700 1 -- 172.21.15.12:0/3641039228 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7fba0801ece0 con 0x565478631f00 2024-05-17T22:52:47.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.206+0000 7fba263f1b80 10 client.16455 fetch_fsmap finished waiting for FSMap version 41 2024-05-17T22:52:47.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.206+0000 7fba263f1b80 10 client.16455 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:52:47.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.206+0000 7fba263f1b80 20 client.16455 populate_metadata read hostname 'smithi012' 2024-05-17T22:52:47.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.206+0000 7fba263f1b80 1 --2- 172.21.15.12:0/3641039228 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x565478771aa0 0x565478803630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:47.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.206+0000 7fba263f1b80 4 client.16455 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:52:47.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.206+0000 7fba263f1b80 1 -- 172.21.15.12:0/3641039228 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5654786959c0 con 0x565478771aa0 2024-05-17T22:52:47.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.206+0000 7fba14fb1700 1 --2- 172.21.15.12:0/3641039228 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x565478771aa0 0x565478803630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:47.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.207+0000 7fb9f8ff9700 10 client.16455 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:52:47.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.207+0000 7fba14fb1700 1 --2- 172.21.15.12:0/3641039228 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x565478771aa0 0x565478803630 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86f9c554d42f9deb server_cookie=f62b8f4972edd664 in_seq=0 out_seq=0 2024-05-17T22:52:47.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.207+0000 7fb9f8ff9700 1 -- 172.21.15.12:0/3641039228 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5654786959c0 con 0x565478771aa0 2024-05-17T22:52:47.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.207+0000 7fb9f8ff9700 10 client.16455 handle_command_reply: tid=0 2024-05-17T22:52:47.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.207+0000 7fba263f1b80 1 client.16455 shutdown 2024-05-17T22:52:47.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.207+0000 7fba263f1b80 1 -- 172.21.15.12:0/3641039228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565478631f00 msgr2=0x56547879d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:47.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.207+0000 7fba263f1b80 1 --2- 172.21.15.12:0/3641039228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565478631f00 0x56547879d9f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fba08009ec0 tx=0x7fba08007cc0 comp rx=0 tx=0).stop 2024-05-17T22:52:47.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.207+0000 7fba263f1b80 1 -- 172.21.15.12:0/3641039228 shutdown_connections 2024-05-17T22:52:47.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.207+0000 7fba263f1b80 1 --2- 172.21.15.12:0/3641039228 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56547876c170 0x56547879dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:47.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.207+0000 7fba263f1b80 1 --2- 172.21.15.12:0/3641039228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654785ed9f0 0x56547878ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:47.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.207+0000 7fba263f1b80 1 --2- 172.21.15.12:0/3641039228 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x565478771aa0 0x565478803630 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:47.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.207+0000 7fba263f1b80 1 --2- 172.21.15.12:0/3641039228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565478631f00 0x56547879d9f0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:47.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.207+0000 7fb9f8ff9700 0 client.16455 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:52:47.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.208+0000 7fba263f1b80 1 -- 172.21.15.12:0/3641039228 >> 172.21.15.12:0/3641039228 conn(0x565478767c10 msgr2=0x565478772b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:52:47.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.208+0000 7fba263f1b80 1 -- 172.21.15.12:0/3641039228 shutdown_connections 2024-05-17T22:52:47.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.208+0000 7fba263f1b80 1 -- 172.21.15.12:0/3641039228 wait complete. 2024-05-17T22:52:47.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.208+0000 7fba263f1b80 20 client.16455 trim_cache size 0 max 16384 2024-05-17T22:52:47.213 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:52:47.214 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-17T22:52:47.214 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-17T22:52:47.214 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:52:47.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.209+0000 7fba17a16700 1 -- 172.21.15.12:0/1051404517 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb9fc0b2980 msgr2=0x7fb9fc0b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:47.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.209+0000 7fba17a16700 1 --2- 172.21.15.12:0/1051404517 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb9fc0b2980 0x7fb9fc0b4e90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fba0c0aa7d0 tx=0x7fba0c092040 comp rx=0 tx=0).stop 2024-05-17T22:52:47.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.209+0000 7fba17a16700 1 -- 172.21.15.12:0/1051404517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba1010f7b0 msgr2=0x7fba1007c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:47.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.209+0000 7fba17a16700 1 --2- 172.21.15.12:0/1051404517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba1010f7b0 0x7fba1007c690 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fba0c005b40 tx=0x7fba0c009ee0 comp rx=0 tx=0).stop 2024-05-17T22:52:47.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.210+0000 7fba17a16700 1 -- 172.21.15.12:0/1051404517 shutdown_connections 2024-05-17T22:52:47.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.210+0000 7fba17a16700 1 --2- 172.21.15.12:0/1051404517 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb9fc0b2980 0x7fb9fc0b4e90 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:47.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.210+0000 7fba17a16700 1 --2- 172.21.15.12:0/1051404517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba1010f7b0 0x7fba1007c690 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:47.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.210+0000 7fba17a16700 1 --2- 172.21.15.12:0/1051404517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba1010a530 0x7fba100840e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:47.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.210+0000 7fba17a16700 1 --2- 172.21.15.12:0/1051404517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba1010ee60 0x7fba1007c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:47.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.210+0000 7fba17a16700 1 -- 172.21.15.12:0/1051404517 >> 172.21.15.12:0/1051404517 conn(0x7fba10078570 msgr2=0x7fba10078b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:52:47.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.210+0000 7fba17a16700 1 -- 172.21.15.12:0/1051404517 shutdown_connections 2024-05-17T22:52:47.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:47.211+0000 7fba17a16700 1 -- 172.21.15.12:0/1051404517 wait complete. 2024-05-17T22:52:47.987 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-17T22:52:47.987 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T22:52:48.353 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:48 smithi012 ceph-mon[31632]: pgmap v845: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.2 MiB/s wr, 424 op/s 2024-05-17T22:52:48.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:48 smithi151 ceph-mon[36101]: pgmap v845: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.2 MiB/s wr, 424 op/s 2024-05-17T22:52:48.517 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:52:48.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:48 smithi096 ceph-mon[36065]: pgmap v845: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.2 MiB/s wr, 424 op/s 2024-05-17T22:52:50.276 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:50 smithi012 ceph-mon[31632]: pgmap v846: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.4 MiB/s wr, 442 op/s 2024-05-17T22:52:50.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:50 smithi151 ceph-mon[36101]: pgmap v846: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.4 MiB/s wr, 442 op/s 2024-05-17T22:52:50.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:50 smithi096 ceph-mon[36065]: pgmap v846: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.4 MiB/s wr, 442 op/s 2024-05-17T22:52:50.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:50.902+0000 7fd3d9a7f700 1 -- 172.21.15.12:0/1776686026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd3d4102b90 msgr2=0x7fd3d4102fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:50.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:50.904+0000 7fd3d9a7f700 1 --2- 172.21.15.12:0/1776686026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd3d4102b90 0x7fd3d4102fa0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd3c40098f0 tx=0x7fd3c4009c00 comp rx=0 tx=0).stop 2024-05-17T22:52:50.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:50.905+0000 7fd3d9a7f700 1 -- 172.21.15.12:0/1776686026 shutdown_connections 2024-05-17T22:52:50.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:50.905+0000 7fd3d9a7f700 1 --2- 172.21.15.12:0/1776686026 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3d4107fd0 0x7fd3d41084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:50.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:50.905+0000 7fd3d9a7f700 1 --2- 172.21.15.12:0/1776686026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd3d4102b90 0x7fd3d4102fa0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:50.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:50.905+0000 7fd3d9a7f700 1 --2- 172.21.15.12:0/1776686026 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd3d41034e0 0x7fd3d4107a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:50.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:50.905+0000 7fd3d9a7f700 1 -- 172.21.15.12:0/1776686026 >> 172.21.15.12:0/1776686026 conn(0x7fd3d4079bd0 msgr2=0x7fd3d4079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:52:50.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:50.906+0000 7fd3d9a7f700 1 -- 172.21.15.12:0/1776686026 shutdown_connections 2024-05-17T22:52:50.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:50.907+0000 7fd3d9a7f700 1 -- 172.21.15.12:0/1776686026 wait complete. 2024-05-17T22:52:50.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:50.908+0000 7fd3d9a7f700 1 Processor -- start 2024-05-17T22:52:50.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:50.908+0000 7fd3d9a7f700 1 -- start start 2024-05-17T22:52:50.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:50.909+0000 7fd3d9a7f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3d4102b90 0x7fd3d41a6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:50.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:50.909+0000 7fd3d9a7f700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd3d41034e0 0x7fd3d41a6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:50.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:50.909+0000 7fd3d9a7f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd3d4107fd0 0x7fd3d41a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:50.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:50.909+0000 7fd3d9a7f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d411a420 con 0x7fd3d4102b90 2024-05-17T22:52:50.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:50.909+0000 7fd3d9a7f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d411a2a0 con 0x7fd3d4107fd0 2024-05-17T22:52:50.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:50.909+0000 7fd3d9a7f700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d411a5a0 con 0x7fd3d41034e0 2024-05-17T22:52:50.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:50.910+0000 7fd3d37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3d4102b90 0x7fd3d41a6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:50.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:50.910+0000 7fd3d37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3d4102b90 0x7fd3d41a6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43598/0 (socket says 172.21.15.12:43598) 2024-05-17T22:52:50.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:50.910+0000 7fd3d37fe700 1 -- 172.21.15.12:0/1725470122 learned_addr learned my addr 172.21.15.12:0/1725470122 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:52:50.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:50.910+0000 7fd3d2ffd700 1 --2- 172.21.15.12:0/1725470122 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd3d41034e0 0x7fd3d41a6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:50.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:50.911+0000 7fd3d3fff700 1 --2- 172.21.15.12:0/1725470122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd3d4107fd0 0x7fd3d41a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:50.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:50.911+0000 7fd3d2ffd700 1 -- 172.21.15.12:0/1725470122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd3d4107fd0 msgr2=0x7fd3d41a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:50.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:50.911+0000 7fd3d2ffd700 1 --2- 172.21.15.12:0/1725470122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd3d4107fd0 0x7fd3d41a0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:50.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:50.911+0000 7fd3d2ffd700 1 -- 172.21.15.12:0/1725470122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3d4102b90 msgr2=0x7fd3d41a6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:50.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:50.911+0000 7fd3d2ffd700 1 --2- 172.21.15.12:0/1725470122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3d4102b90 0x7fd3d41a6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:50.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:50.911+0000 7fd3d2ffd700 1 -- 172.21.15.12:0/1725470122 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3d41a1100 con 0x7fd3d41034e0 2024-05-17T22:52:50.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:50.911+0000 7fd3d3fff700 1 --2- 172.21.15.12:0/1725470122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd3d4107fd0 0x7fd3d41a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:52:50.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:50.912+0000 7fd3d2ffd700 1 --2- 172.21.15.12:0/1725470122 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd3d41034e0 0x7fd3d41a6d70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd3c800d950 tx=0x7fd3c800dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:52:50.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:50.914+0000 7fd3d0ff9700 1 -- 172.21.15.12:0/1725470122 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c8017070 con 0x7fd3d41034e0 2024-05-17T22:52:50.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:50.914+0000 7fd3d0ff9700 1 -- 172.21.15.12:0/1725470122 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3c8009de0 con 0x7fd3d41034e0 2024-05-17T22:52:50.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:50.914+0000 7fd3d0ff9700 1 -- 172.21.15.12:0/1725470122 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c8010e20 con 0x7fd3d41034e0 2024-05-17T22:52:50.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:50.916+0000 7fd3d9a7f700 1 -- 172.21.15.12:0/1725470122 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3d41a13f0 con 0x7fd3d41034e0 2024-05-17T22:52:50.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:50.916+0000 7fd3d9a7f700 1 -- 172.21.15.12:0/1725470122 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3d41ad6c0 con 0x7fd3d41034e0 2024-05-17T22:52:50.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:50.917+0000 7fd3d9a7f700 1 -- 172.21.15.12:0/1725470122 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3d406bf60 con 0x7fd3d41034e0 2024-05-17T22:52:50.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:50.920+0000 7fd3d0ff9700 1 -- 172.21.15.12:0/1725470122 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd3c801a070 con 0x7fd3d41034e0 2024-05-17T22:52:50.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:50.921+0000 7fd3d0ff9700 1 --2- 172.21.15.12:0/1725470122 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd3bc0b29c0 0x7fd3bc0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:50.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:50.921+0000 7fd3d0ff9700 1 -- 172.21.15.12:0/1725470122 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fd3c80e1c70 con 0x7fd3d41034e0 2024-05-17T22:52:50.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:50.923+0000 7fd3d37fe700 1 --2- 172.21.15.12:0/1725470122 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd3bc0b29c0 0x7fd3bc0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:50.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:50.924+0000 7fd3d37fe700 1 --2- 172.21.15.12:0/1725470122 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd3bc0b29c0 0x7fd3bc0b4ed0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd3c40098c0 tx=0x7fd3c4019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:52:50.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:50.925+0000 7fd3d0ff9700 1 -- 172.21.15.12:0/1725470122 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd3c8021070 con 0x7fd3d41034e0 2024-05-17T22:52:51.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 810.0 s, 406.8 tps, lat 238.165 ms stddev 276.577 2024-05-17T22:52:51.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:51.393+0000 7fd3d9a7f700 1 -- 172.21.15.12:0/1725470122 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd3d41add40 con 0x7fd3d41034e0 2024-05-17T22:52:51.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:51.394+0000 7fd3d0ff9700 1 -- 172.21.15.12:0/1725470122 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11873 (secure 0 0 0) 0x7fd3c80a95a0 con 0x7fd3d41034e0 2024-05-17T22:52:51.397 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:52:51.397 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1407219319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1407219319},{"type":"v1","addr":"172.21.15.12:6835","nonce":1407219319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/630977886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":630977886},{"type":"v1","addr":"172.21.15.12:6837","nonce":630977886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3731914742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3731914742},{"type":"v1","addr":"172.21.15.12:6839","nonce":3731914742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/3642142501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3642142501},{"type":"v1","addr":"172.21.15.12:6841","nonce":3642142501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3554053059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3554053059},{"type":"v1","addr":"172.21.15.151:6833","nonce":3554053059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/4228325569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":4228325569},{"type":"v1","addr":"172.21.15.96:6833","nonce":4228325569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T22:34:42.642111+0000","modified":"2024-05-17T22:52:03.169093+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24461,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/542323754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":542323754},{"type":"v1","addr":"172.21.15.151:6835","nonce":542323754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1758442232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1758442232},{"type":"v1","addr":"172.21.15.96:6835","nonce":1758442232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3403324038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3403324038},{"type":"v1","addr":"172.21.15.151:6837","nonce":3403324038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.151:6839/2338353849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2338353849},{"type":"v1","addr":"172.21.15.151:6839","nonce":2338353849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.96:6837/4222574338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":4222574338},{"type":"v1","addr":"172.21.15.96:6837","nonce":4222574338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/2986295519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2986295519},{"type":"v1","addr":"172.21.15.96:6839","nonce":2986295519}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24457,24461,24482]},"id":1}]} 2024-05-17T22:52:51.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:51.397+0000 7fd3d9a7f700 1 -- 172.21.15.12:0/1725470122 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd3bc0b29c0 msgr2=0x7fd3bc0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:51.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:51.397+0000 7fd3d9a7f700 1 --2- 172.21.15.12:0/1725470122 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd3bc0b29c0 0x7fd3bc0b4ed0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd3c40098c0 tx=0x7fd3c4019040 comp rx=0 tx=0).stop 2024-05-17T22:52:51.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:51.397+0000 7fd3d9a7f700 1 -- 172.21.15.12:0/1725470122 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd3d41034e0 msgr2=0x7fd3d41a6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:51.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:51.397+0000 7fd3d9a7f700 1 --2- 172.21.15.12:0/1725470122 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd3d41034e0 0x7fd3d41a6d70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd3c800d950 tx=0x7fd3c800dc60 comp rx=0 tx=0).stop 2024-05-17T22:52:51.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:51.397+0000 7fd3d9a7f700 1 -- 172.21.15.12:0/1725470122 shutdown_connections 2024-05-17T22:52:51.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:51.397+0000 7fd3d9a7f700 1 --2- 172.21.15.12:0/1725470122 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd3bc0b29c0 0x7fd3bc0b4ed0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:51.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:51.397+0000 7fd3d9a7f700 1 --2- 172.21.15.12:0/1725470122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd3d4107fd0 0x7fd3d41a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:51.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:51.397+0000 7fd3d9a7f700 1 --2- 172.21.15.12:0/1725470122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3d4102b90 0x7fd3d41a6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:51.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:51.397+0000 7fd3d9a7f700 1 --2- 172.21.15.12:0/1725470122 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd3d41034e0 0x7fd3d41a6d70 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:51.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:51.397+0000 7fd3d9a7f700 1 -- 172.21.15.12:0/1725470122 >> 172.21.15.12:0/1725470122 conn(0x7fd3d4079bd0 msgr2=0x7fd3d41059e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:52:51.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:51.397+0000 7fd3d9a7f700 1 -- 172.21.15.12:0/1725470122 shutdown_connections 2024-05-17T22:52:51.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:51.397+0000 7fd3d9a7f700 1 -- 172.21.15.12:0/1725470122 wait complete. 2024-05-17T22:52:51.401 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 41 2024-05-17T22:52:52.202 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-17T22:52:52.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:52 smithi012 ceph-mon[31632]: pgmap v847: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.6 MiB/s wr, 498 op/s 2024-05-17T22:52:52.343 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:52 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/1725470122' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:52:52.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:52 smithi151 ceph-mon[36101]: pgmap v847: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.6 MiB/s wr, 498 op/s 2024-05-17T22:52:52.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:52 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/1725470122' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:52:52.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:52 smithi096 ceph-mon[36065]: pgmap v847: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.6 MiB/s wr, 498 op/s 2024-05-17T22:52:52.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:52 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/1725470122' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:52:52.693 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:52:54.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:54 smithi012 ceph-mon[31632]: pgmap v848: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.0 MiB/s wr, 406 op/s 2024-05-17T22:52:54.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:54 smithi151 ceph-mon[36101]: pgmap v848: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.0 MiB/s wr, 406 op/s 2024-05-17T22:52:54.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:54 smithi096 ceph-mon[36065]: pgmap v848: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.0 MiB/s wr, 406 op/s 2024-05-17T22:52:54.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.852+0000 7f99ac507700 1 -- 172.21.15.12:0/1952583097 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99a410a830 msgr2=0x7f99a410ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:54.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.852+0000 7f99ac507700 1 --2- 172.21.15.12:0/1952583097 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99a410a830 0x7f99a410ac40 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f999c00b0a0 tx=0x7f999c00b3b0 comp rx=0 tx=0).stop 2024-05-17T22:52:54.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.854+0000 7f99ac507700 1 -- 172.21.15.12:0/1952583097 shutdown_connections 2024-05-17T22:52:54.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.854+0000 7f99ac507700 1 --2- 172.21.15.12:0/1952583097 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99a411d550 0x7f99a411f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:54.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.854+0000 7f99ac507700 1 --2- 172.21.15.12:0/1952583097 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99a410a830 0x7f99a410ac40 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:54.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.854+0000 7f99ac507700 1 --2- 172.21.15.12:0/1952583097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99a410b180 0x7f99a411d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:54.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.854+0000 7f99ac507700 1 -- 172.21.15.12:0/1952583097 >> 172.21.15.12:0/1952583097 conn(0x7f99a406efc0 msgr2=0x7f99a406f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:52:54.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.855+0000 7f99ac507700 1 -- 172.21.15.12:0/1952583097 shutdown_connections 2024-05-17T22:52:54.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.855+0000 7f99ac507700 1 -- 172.21.15.12:0/1952583097 wait complete. 2024-05-17T22:52:54.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.856+0000 7f99ac507700 1 Processor -- start 2024-05-17T22:52:54.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.856+0000 7f99ac507700 1 -- start start 2024-05-17T22:52:54.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.857+0000 7f99ac507700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99a410a830 0x7f99a41a9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:54.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.857+0000 7f99ac507700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99a410b180 0x7f99a41a9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:54.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.857+0000 7f99ac507700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99a411d550 0x7f99a41ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:54.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.857+0000 7f99ac507700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f99a4122c90 con 0x7f99a410b180 2024-05-17T22:52:54.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.857+0000 7f99ac507700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f99a4122b10 con 0x7f99a410a830 2024-05-17T22:52:54.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.857+0000 7f99ac507700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f99a4122e10 con 0x7f99a411d550 2024-05-17T22:52:54.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.857+0000 7f99a9aa2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99a410b180 0x7f99a41a9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:54.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.857+0000 7f99a9aa2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99a410b180 0x7f99a41a9bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43602/0 (socket says 172.21.15.12:43602) 2024-05-17T22:52:54.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.857+0000 7f99a9aa2700 1 -- 172.21.15.12:0/1690458296 learned_addr learned my addr 172.21.15.12:0/1690458296 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:52:54.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.858+0000 7f99a9aa2700 1 -- 172.21.15.12:0/1690458296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99a411d550 msgr2=0x7f99a41ade60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:54.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.858+0000 7f99a9aa2700 1 --2- 172.21.15.12:0/1690458296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99a411d550 0x7f99a41ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:54.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.858+0000 7f99a9aa2700 1 -- 172.21.15.12:0/1690458296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99a410a830 msgr2=0x7f99a41a9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:54.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.858+0000 7f99a9aa2700 1 --2- 172.21.15.12:0/1690458296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99a410a830 0x7f99a41a9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:54.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.858+0000 7f99a9aa2700 1 -- 172.21.15.12:0/1690458296 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99a41ae500 con 0x7f99a410b180 2024-05-17T22:52:54.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.867+0000 7f99a9aa2700 1 --2- 172.21.15.12:0/1690458296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99a410b180 0x7f99a41a9bd0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f99a000c920 tx=0x7f99a000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:52:54.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.867+0000 7f999b7fe700 1 -- 172.21.15.12:0/1690458296 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a0007b00 con 0x7f99a410b180 2024-05-17T22:52:54.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.868+0000 7f99ac507700 1 -- 172.21.15.12:0/1690458296 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99a41ae7f0 con 0x7f99a410b180 2024-05-17T22:52:54.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.868+0000 7f99ac507700 1 -- 172.21.15.12:0/1690458296 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99a41127f0 con 0x7f99a410b180 2024-05-17T22:52:54.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.873+0000 7f999b7fe700 1 -- 172.21.15.12:0/1690458296 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99a0007ca0 con 0x7f99a410b180 2024-05-17T22:52:54.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.873+0000 7f999b7fe700 1 -- 172.21.15.12:0/1690458296 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a001d060 con 0x7f99a410b180 2024-05-17T22:52:54.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.874+0000 7f999b7fe700 1 -- 172.21.15.12:0/1690458296 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f99a00176d0 con 0x7f99a410b180 2024-05-17T22:52:54.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.876+0000 7f999b7fe700 1 --2- 172.21.15.12:0/1690458296 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f99900b29f0 0x7f99900b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:54.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.876+0000 7f999b7fe700 1 -- 172.21.15.12:0/1690458296 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f99a00e3270 con 0x7f99a410b180 2024-05-17T22:52:54.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.880+0000 7f99baee2b80 1 Processor -- start 2024-05-17T22:52:54.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.882+0000 7f99aa2a3700 1 --2- 172.21.15.12:0/1690458296 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f99900b29f0 0x7f99900b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:54.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.883+0000 7f99baee2b80 1 -- start start 2024-05-17T22:52:54.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.883+0000 7f99baee2b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55beb469e9f0 0x55beb4821ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:54.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.883+0000 7f99baee2b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55beb4822400 0x55beb481c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:54.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.883+0000 7f99baee2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55beb481cb00 0x55beb481cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:54.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.884+0000 7f99a9aa2700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55beb4822400 0x55beb481c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:54.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.884+0000 7f99a9aa2700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55beb4822400 0x55beb481c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:55470/0 (socket says 172.21.15.12:55470) 2024-05-17T22:52:54.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.884+0000 7f99a9aa2700 1 -- 172.21.15.12:0/260997006 learned_addr learned my addr 172.21.15.12:0/260997006 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:52:54.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.884+0000 7f99aaaa4700 1 --2- 172.21.15.12:0/260997006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55beb481cb00 0x55beb481cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:54.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.883+0000 7f99baee2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55beb474b0d0 con 0x55beb481cb00 2024-05-17T22:52:54.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.884+0000 7f99baee2b80 1 -- 172.21.15.12:0/260997006 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55beb473ac70 con 0x55beb469e9f0 2024-05-17T22:52:54.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.884+0000 7f99baee2b80 1 -- 172.21.15.12:0/260997006 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55beb473ecb0 con 0x55beb4822400 2024-05-17T22:52:54.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.884+0000 7f99aa2a3700 1 --2- 172.21.15.12:0/260997006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55beb469e9f0 0x55beb4821ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:54.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.885+0000 7f99aa2a3700 1 --2- 172.21.15.12:0/1690458296 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f99900b29f0 0x7f99900b4f00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f999c009e60 tx=0x7f999c0093c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:52:54.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.885+0000 7f99aa2a3700 1 -- 172.21.15.12:0/260997006 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55beb4822400 msgr2=0x55beb481c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:54.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.885+0000 7f99aa2a3700 1 --2- 172.21.15.12:0/260997006 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55beb4822400 0x55beb481c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:54.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.885+0000 7f99aa2a3700 1 -- 172.21.15.12:0/260997006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55beb481cb00 msgr2=0x55beb481cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:54.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.885+0000 7f99aa2a3700 1 --2- 172.21.15.12:0/260997006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55beb481cb00 0x55beb481cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:54.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.885+0000 7f99aa2a3700 1 -- 172.21.15.12:0/260997006 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55beb46d45c0 con 0x55beb469e9f0 2024-05-17T22:52:54.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.885+0000 7f99aa2a3700 1 --2- 172.21.15.12:0/260997006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55beb469e9f0 0x55beb4821ec0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f999c009750 tx=0x7f999c023000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5d3aa39ff4e3e2d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:52:54.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.886+0000 7f99a9aa2700 1 --2- 172.21.15.12:0/260997006 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55beb4822400 0x55beb481c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:52:54.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.886+0000 7f99aaaa4700 1 --2- 172.21.15.12:0/260997006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55beb481cb00 0x55beb481cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:52:54.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.887+0000 7f9998d46700 1 -- 172.21.15.12:0/260997006 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c02d070 con 0x55beb469e9f0 2024-05-17T22:52:54.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.887+0000 7f9998d46700 1 -- 172.21.15.12:0/260997006 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f999c008e60 con 0x55beb469e9f0 2024-05-17T22:52:54.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.887+0000 7f9998d46700 1 -- 172.21.15.12:0/260997006 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c01e430 con 0x55beb469e9f0 2024-05-17T22:52:54.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.889+0000 7f99baee2b80 1 -- 172.21.15.12:0/260997006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55beb469e9f0 msgr2=0x55beb4821ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:54.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.889+0000 7f99baee2b80 1 --2- 172.21.15.12:0/260997006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55beb469e9f0 0x55beb4821ec0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f999c009750 tx=0x7f999c023000 comp rx=0 tx=0).stop 2024-05-17T22:52:54.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.889+0000 7f99baee2b80 1 -- 172.21.15.12:0/260997006 shutdown_connections 2024-05-17T22:52:54.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.889+0000 7f99baee2b80 1 --2- 172.21.15.12:0/260997006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55beb481cb00 0x55beb481cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:54.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.889+0000 7f99baee2b80 1 --2- 172.21.15.12:0/260997006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55beb469e9f0 0x55beb4821ec0 secure :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f999c009750 tx=0x7f999c023000 comp rx=0 tx=0).stop 2024-05-17T22:52:54.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.889+0000 7f99baee2b80 1 --2- 172.21.15.12:0/260997006 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55beb4822400 0x55beb481c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:54.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.889+0000 7f99baee2b80 1 -- 172.21.15.12:0/260997006 >> 172.21.15.12:0/260997006 conn(0x55beb46fe7d0 msgr2=0x55beb48171c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:52:54.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.893+0000 7f99baee2b80 1 -- 172.21.15.12:0/260997006 shutdown_connections 2024-05-17T22:52:54.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.898+0000 7f99baee2b80 1 -- 172.21.15.12:0/260997006 wait complete. 2024-05-17T22:52:54.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.899+0000 7f99baee2b80 1 -- start start 2024-05-17T22:52:54.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.899+0000 7f99baee2b80 1 Processor -- start 2024-05-17T22:52:54.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.899+0000 7f99baee2b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55beb481cb00 0x55beb4825e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:54.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.899+0000 7f99baee2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55beb4822400 0x55beb48263d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:54.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.899+0000 7f99baee2b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55beb4830120 0x55beb4832540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:54.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.900+0000 7f99baee2b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55beb4642150 con 0x55beb4822400 2024-05-17T22:52:54.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.900+0000 7f99baee2b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55beb46dc940 con 0x55beb481cb00 2024-05-17T22:52:54.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.900+0000 7f99baee2b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55beb4828d70 con 0x55beb4830120 2024-05-17T22:52:54.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.900+0000 7f99baee2b80 10 client.0 _ll_register_callbacks cb 0x55beb46d1b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:52:54.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.903+0000 7f99aa2a3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55beb481cb00 0x55beb4825e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:54.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.904+0000 7f99aaaa4700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55beb4830120 0x55beb4832540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:54.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.904+0000 7f99aa2a3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55beb481cb00 0x55beb4825e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:59642/0 (socket says 172.21.15.12:59642) 2024-05-17T22:52:54.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.904+0000 7f99aa2a3700 1 -- 172.21.15.12:0/2742201515 learned_addr learned my addr 172.21.15.12:0/2742201515 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:52:54.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.907+0000 7f99aa2a3700 1 -- 172.21.15.12:0/2742201515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55beb4830120 msgr2=0x55beb4832540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:54.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.907+0000 7f99aa2a3700 1 --2- 172.21.15.12:0/2742201515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55beb4830120 0x55beb4832540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:54.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.907+0000 7f99aa2a3700 1 -- 172.21.15.12:0/2742201515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55beb4822400 msgr2=0x55beb48263d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:54.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.907+0000 7f99aa2a3700 1 --2- 172.21.15.12:0/2742201515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55beb4822400 0x55beb48263d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:54.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.907+0000 7f99aa2a3700 1 -- 172.21.15.12:0/2742201515 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55beb468b480 con 0x55beb481cb00 2024-05-17T22:52:54.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.910+0000 7f99aaaa4700 1 --2- 172.21.15.12:0/2742201515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55beb4830120 0x55beb4832540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:52:54.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.914+0000 7f99aa2a3700 1 --2- 172.21.15.12:0/2742201515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55beb481cb00 0x55beb4825e90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f999c02b040 tx=0x7f999c008be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3a653fe0b130d7b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:52:54.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.915+0000 7f998dffb700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:52:54.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.917+0000 7f998dffb700 1 -- 172.21.15.12:0/2742201515 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c02d040 con 0x55beb481cb00 2024-05-17T22:52:54.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.917+0000 7f998dffb700 1 -- 172.21.15.12:0/2742201515 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f999c028680 con 0x55beb481cb00 2024-05-17T22:52:54.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.917+0000 7f99baee2b80 1 -- 172.21.15.12:0/2742201515 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f999c008e60 con 0x55beb481cb00 2024-05-17T22:52:54.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.923+0000 7f998dffb700 1 -- 172.21.15.12:0/2742201515 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f999c003e20 con 0x55beb481cb00 2024-05-17T22:52:54.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.924+0000 7f99baee2b80 10 client.26177 fetch_fsmap learned FSMap version 41 2024-05-17T22:52:54.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.924+0000 7f99baee2b80 1 -- 172.21.15.12:0/2742201515 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55beb4833e10 con 0x55beb481cb00 2024-05-17T22:52:54.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.926+0000 7f998dffb700 1 -- 172.21.15.12:0/2742201515 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f999c008e60 con 0x55beb481cb00 2024-05-17T22:52:54.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.926+0000 7f998dffb700 1 -- 172.21.15.12:0/2742201515 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f999c03b030 con 0x55beb481cb00 2024-05-17T22:52:54.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.931+0000 7f99baee2b80 10 client.26177 fetch_fsmap finished waiting for FSMap version 41 2024-05-17T22:52:54.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.931+0000 7f99baee2b80 10 client.26177 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-17T22:52:54.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.931+0000 7f99baee2b80 20 client.26177 populate_metadata read hostname 'smithi012' 2024-05-17T22:52:54.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.931+0000 7f99baee2b80 1 --2- 172.21.15.12:0/2742201515 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55beb48362e0 0x55beb4838700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:54.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.931+0000 7f99baee2b80 4 client.26177 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:52:54.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.931+0000 7f99baee2b80 1 -- 172.21.15.12:0/2742201515 --> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55beb474d020 con 0x55beb48362e0 2024-05-17T22:52:54.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.933+0000 7f99a9aa2700 1 --2- 172.21.15.12:0/2742201515 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55beb48362e0 0x55beb4838700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:54.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.943+0000 7f99a9aa2700 1 --2- 172.21.15.12:0/2742201515 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55beb48362e0 0x55beb4838700 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a125049a0610d500 server_cookie=8feddcecd2d182b0 in_seq=0 out_seq=0 2024-05-17T22:52:54.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.943+0000 7f998dffb700 10 client.26177 ms_handle_connect on v2:172.21.15.151:6838/2338353849 2024-05-17T22:52:54.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.947+0000 7f998dffb700 1 -- 172.21.15.12:0/2742201515 <== mds.2 v2:172.21.15.151:6838/2338353849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55beb474d020 con 0x55beb48362e0 2024-05-17T22:52:54.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.947+0000 7f998dffb700 10 client.26177 handle_command_reply: tid=0 2024-05-17T22:52:54.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.949+0000 7f99baee2b80 1 client.26177 shutdown 2024-05-17T22:52:54.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.960+0000 7f99baee2b80 1 -- 172.21.15.12:0/2742201515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55beb481cb00 msgr2=0x55beb4825e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:54.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.960+0000 7f99baee2b80 1 --2- 172.21.15.12:0/2742201515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55beb481cb00 0x55beb4825e90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f999c02b040 tx=0x7f999c008be0 comp rx=0 tx=0).stop 2024-05-17T22:52:54.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.965+0000 7f99baee2b80 1 -- 172.21.15.12:0/2742201515 shutdown_connections 2024-05-17T22:52:54.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.965+0000 7f99baee2b80 1 --2- 172.21.15.12:0/2742201515 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55beb48362e0 0x55beb4838700 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:54.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.965+0000 7f99baee2b80 1 --2- 172.21.15.12:0/2742201515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55beb4830120 0x55beb4832540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:54.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.965+0000 7f99baee2b80 1 --2- 172.21.15.12:0/2742201515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55beb481cb00 0x55beb4825e90 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:54.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.965+0000 7f99baee2b80 1 --2- 172.21.15.12:0/2742201515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55beb4822400 0x55beb48263d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:54.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.965+0000 7f99baee2b80 1 -- 172.21.15.12:0/2742201515 >> 172.21.15.12:0/2742201515 conn(0x55beb46fe7d0 msgr2=0x55beb4817160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:52:54.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.966+0000 7f998dffb700 0 client.26177 ms_handle_reset on v2:172.21.15.151:6838/2338353849 2024-05-17T22:52:54.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.970+0000 7f99baee2b80 1 -- 172.21.15.12:0/2742201515 shutdown_connections 2024-05-17T22:52:54.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.970+0000 7f99baee2b80 1 -- 172.21.15.12:0/2742201515 wait complete. 2024-05-17T22:52:54.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:54.970+0000 7f99baee2b80 20 client.26177 trim_cache size 0 max 16384 2024-05-17T22:52:55.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.004+0000 7f99baee2b80 1 Processor -- start 2024-05-17T22:52:55.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.007+0000 7f99baee2b80 1 -- start start 2024-05-17T22:52:55.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.007+0000 7f99baee2b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55beb481cb00 0x55beb4824560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:55.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.007+0000 7f99baee2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55beb4822400 0x55beb4824aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:55.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.007+0000 7f99aa2a3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55beb481cb00 0x55beb4824560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:55.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.007+0000 7f99aa2a3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55beb481cb00 0x55beb4824560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:59658/0 (socket says 172.21.15.12:59658) 2024-05-17T22:52:55.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.007+0000 7f99a9aa2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55beb4822400 0x55beb4824aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:55.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.007+0000 7f99a9aa2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55beb4822400 0x55beb4824aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43634/0 (socket says 172.21.15.12:43634) 2024-05-17T22:52:55.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.007+0000 7f99aa2a3700 1 -- 172.21.15.12:0/3766842221 learned_addr learned my addr 172.21.15.12:0/3766842221 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:52:55.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.007+0000 7f99baee2b80 1 --2- 172.21.15.12:0/3766842221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55beb4830120 0x55beb4811110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:55.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.008+0000 7f99baee2b80 1 -- 172.21.15.12:0/3766842221 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9988011480 con 0x55beb4822400 2024-05-17T22:52:55.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.008+0000 7f99baee2b80 1 -- 172.21.15.12:0/3766842221 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f998800ed10 con 0x55beb481cb00 2024-05-17T22:52:55.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.008+0000 7f99baee2b80 1 -- 172.21.15.12:0/3766842221 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f998800cb60 con 0x55beb4830120 2024-05-17T22:52:55.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.010+0000 7f99aaaa4700 1 --2- 172.21.15.12:0/3766842221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55beb4830120 0x55beb4811110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:55.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.010+0000 7f99a9aa2700 1 -- 172.21.15.12:0/3766842221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55beb4830120 msgr2=0x55beb4811110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:55.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.010+0000 7f99a9aa2700 1 --2- 172.21.15.12:0/3766842221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55beb4830120 0x55beb4811110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:55.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.010+0000 7f99a9aa2700 1 -- 172.21.15.12:0/3766842221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55beb481cb00 msgr2=0x55beb4824560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:55.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.010+0000 7f99a9aa2700 1 --2- 172.21.15.12:0/3766842221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55beb481cb00 0x55beb4824560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:55.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.010+0000 7f99a9aa2700 1 -- 172.21.15.12:0/3766842221 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55beb4813c20 con 0x55beb4822400 2024-05-17T22:52:55.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.010+0000 7f99aa2a3700 1 --2- 172.21.15.12:0/3766842221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55beb481cb00 0x55beb4824560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:52:55.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.011+0000 7f99a9aa2700 1 --2- 172.21.15.12:0/3766842221 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55beb4822400 0x55beb4824aa0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x55beb4813880 tx=0x7f99a0017610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=529f5aaaed7711c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:52:55.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.011+0000 7f99aaaa4700 1 --2- 172.21.15.12:0/3766842221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55beb4830120 0x55beb4811110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:52:55.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.018+0000 7f998ffff700 1 -- 172.21.15.12:0/3766842221 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a00ae8b0 con 0x55beb4822400 2024-05-17T22:52:55.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.019+0000 7f998ffff700 1 -- 172.21.15.12:0/3766842221 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99a00b3360 con 0x55beb4822400 2024-05-17T22:52:55.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.019+0000 7f998ffff700 1 -- 172.21.15.12:0/3766842221 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a00b2fc0 con 0x55beb4822400 2024-05-17T22:52:55.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.020+0000 7f99baee2b80 1 -- 172.21.15.12:0/3766842221 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55beb4822400 msgr2=0x55beb4824aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:55.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.020+0000 7f99baee2b80 1 --2- 172.21.15.12:0/3766842221 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55beb4822400 0x55beb4824aa0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x55beb4813880 tx=0x7f99a0017610 comp rx=0 tx=0).stop 2024-05-17T22:52:55.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.025+0000 7f99baee2b80 1 -- 172.21.15.12:0/3766842221 shutdown_connections 2024-05-17T22:52:55.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.025+0000 7f99baee2b80 1 --2- 172.21.15.12:0/3766842221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55beb4830120 0x55beb4811110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:55.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.025+0000 7f99baee2b80 1 --2- 172.21.15.12:0/3766842221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55beb481cb00 0x55beb4824560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:55.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.025+0000 7f99baee2b80 1 --2- 172.21.15.12:0/3766842221 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55beb4822400 0x55beb4824aa0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:55.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.025+0000 7f99baee2b80 1 -- 172.21.15.12:0/3766842221 >> 172.21.15.12:0/3766842221 conn(0x55beb46fe7d0 msgr2=0x55beb4820cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:52:55.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.027+0000 7f99baee2b80 1 -- 172.21.15.12:0/3766842221 shutdown_connections 2024-05-17T22:52:55.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.027+0000 7f99baee2b80 1 -- 172.21.15.12:0/3766842221 wait complete. 2024-05-17T22:52:55.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.027+0000 7f99baee2b80 1 -- start start 2024-05-17T22:52:55.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.028+0000 7f99baee2b80 1 Processor -- start 2024-05-17T22:52:55.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.028+0000 7f99baee2b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55beb481cb00 0x55beb482c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:55.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.029+0000 7f99baee2b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55beb4822400 0x55beb482cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:55.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.029+0000 7f99aa2a3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55beb481cb00 0x55beb482c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:55.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.029+0000 7f99aa2a3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55beb481cb00 0x55beb482c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:59672/0 (socket says 172.21.15.12:59672) 2024-05-17T22:52:55.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.029+0000 7f99aa2a3700 1 -- 172.21.15.12:0/943366097 learned_addr learned my addr 172.21.15.12:0/943366097 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:52:55.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.029+0000 7f99a9aa2700 1 --2- 172.21.15.12:0/943366097 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55beb4822400 0x55beb482cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:55.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.029+0000 7f99baee2b80 1 --2- 172.21.15.12:0/943366097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55beb4830120 0x55beb482d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:55.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.030+0000 7f99baee2b80 1 -- 172.21.15.12:0/943366097 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f998800d1d0 con 0x55beb4822400 2024-05-17T22:52:55.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.030+0000 7f99aaaa4700 1 --2- 172.21.15.12:0/943366097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55beb4830120 0x55beb482d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:55.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.030+0000 7f99baee2b80 1 -- 172.21.15.12:0/943366097 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f998800e640 con 0x55beb481cb00 2024-05-17T22:52:55.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.030+0000 7f99baee2b80 1 -- 172.21.15.12:0/943366097 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f998800d8a0 con 0x55beb4830120 2024-05-17T22:52:55.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.030+0000 7f99aa2a3700 1 -- 172.21.15.12:0/943366097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55beb4830120 msgr2=0x55beb482d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:55.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.030+0000 7f99aa2a3700 1 --2- 172.21.15.12:0/943366097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55beb4830120 0x55beb482d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:55.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.030+0000 7f99aa2a3700 1 -- 172.21.15.12:0/943366097 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55beb4822400 msgr2=0x55beb482cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:52:55.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.030+0000 7f99aa2a3700 1 --2- 172.21.15.12:0/943366097 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55beb4822400 0x55beb482cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:55.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.030+0000 7f99aa2a3700 1 -- 172.21.15.12:0/943366097 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55beb480f4f0 con 0x55beb481cb00 2024-05-17T22:52:55.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.031+0000 7f99aa2a3700 1 --2- 172.21.15.12:0/943366097 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55beb481cb00 0x55beb482c640 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f999c009750 tx=0x7f999c01e9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fc9c995727d2e91 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:52:55.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.031+0000 7f998dffb700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:52:55.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.031+0000 7f998dffb700 1 -- 172.21.15.12:0/943366097 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c044070 con 0x55beb481cb00 2024-05-17T22:52:55.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.031+0000 7f998dffb700 1 -- 172.21.15.12:0/943366097 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f999c028480 con 0x55beb481cb00 2024-05-17T22:52:55.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.032+0000 7f998dffb700 10 client.0 inactive, discarding osd_map(89..89 src has 1..89) 2024-05-17T22:52:55.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.032+0000 7f99baee2b80 10 client.0 _ll_register_callbacks cb 0x55beb46d1b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:52:55.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.032+0000 7f99baee2b80 1 -- 172.21.15.12:0/943366097 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99a00b3360 con 0x55beb481cb00 2024-05-17T22:52:55.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.036+0000 7f998dffb700 1 -- 172.21.15.12:0/943366097 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f999c02d070 con 0x55beb481cb00 2024-05-17T22:52:55.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.038+0000 7f99baee2b80 10 client.26183 fetch_fsmap learned FSMap version 41 2024-05-17T22:52:55.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.038+0000 7f99baee2b80 1 -- 172.21.15.12:0/943366097 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({fsmap=41}) -- 0x55beb489f7d0 con 0x55beb481cb00 2024-05-17T22:52:55.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.042+0000 7f998dffb700 1 -- 172.21.15.12:0/943366097 <== mon.2 v2:172.21.15.96:3300/0 4 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f999c047020 con 0x55beb481cb00 2024-05-17T22:52:55.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.042+0000 7f99baee2b80 10 client.26183 fetch_fsmap finished waiting for FSMap version 41 2024-05-17T22:52:55.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.042+0000 7f99baee2b80 10 client.26183 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-17T22:52:55.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.042+0000 7f99baee2b80 20 client.26183 populate_metadata read hostname 'smithi012' 2024-05-17T22:52:55.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.042+0000 7f99baee2b80 1 --2- 172.21.15.12:0/943366097 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55beb489fcd0 0x55beb48a2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:55.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.043+0000 7f99baee2b80 4 client.26183 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:52:55.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.043+0000 7f99baee2b80 1 -- 172.21.15.12:0/943366097 --> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f99a00b3360 con 0x55beb489fcd0 2024-05-17T22:52:55.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.043+0000 7f99a9aa2700 1 --2- 172.21.15.12:0/943366097 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55beb489fcd0 0x55beb48a2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:55.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.049+0000 7f99a9aa2700 1 --2- 172.21.15.12:0/943366097 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55beb489fcd0 0x55beb48a2180 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9ddb71a9ec9b469b server_cookie=ddbe4bf879e59346 in_seq=0 out_seq=0 2024-05-17T22:52:55.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.049+0000 7f998dffb700 10 client.26183 ms_handle_connect on v2:172.21.15.151:6838/2338353849 2024-05-17T22:52:55.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.049+0000 7f998dffb700 1 -- 172.21.15.12:0/943366097 <== mds.2 v2:172.21.15.151:6838/2338353849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f99a00b3360 con 0x55beb489fcd0 2024-05-17T22:52:55.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.049+0000 7f998dffb700 10 client.26183 handle_command_reply: tid=0 2024-05-17T22:52:55.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.049+0000 7f99baee2b80 1 client.26183 shutdown 2024-05-17T22:52:55.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.051+0000 7f99baee2b80 1 -- 172.21.15.12:0/943366097 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55beb481cb00 msgr2=0x55beb482c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:55.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.051+0000 7f99baee2b80 1 --2- 172.21.15.12:0/943366097 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55beb481cb00 0x55beb482c640 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f999c009750 tx=0x7f999c01e9d0 comp rx=0 tx=0).stop 2024-05-17T22:52:55.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.052+0000 7f99baee2b80 1 -- 172.21.15.12:0/943366097 shutdown_connections 2024-05-17T22:52:55.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.052+0000 7f99baee2b80 1 --2- 172.21.15.12:0/943366097 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x55beb489fcd0 0x55beb48a2180 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:55.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.053+0000 7f99baee2b80 1 --2- 172.21.15.12:0/943366097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55beb4830120 0x55beb482d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:55.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.053+0000 7f998dffb700 0 client.26183 ms_handle_reset on v2:172.21.15.151:6838/2338353849 2024-05-17T22:52:55.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.053+0000 7f99baee2b80 1 --2- 172.21.15.12:0/943366097 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55beb481cb00 0x55beb482c640 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:55.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.053+0000 7f99baee2b80 1 --2- 172.21.15.12:0/943366097 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55beb4822400 0x55beb482cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:55.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.053+0000 7f99baee2b80 1 -- 172.21.15.12:0/943366097 >> 172.21.15.12:0/943366097 conn(0x55beb46fe7d0 msgr2=0x55beb4824470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:52:55.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.053+0000 7f99baee2b80 1 -- 172.21.15.12:0/943366097 shutdown_connections 2024-05-17T22:52:55.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.053+0000 7f99baee2b80 1 -- 172.21.15.12:0/943366097 wait complete. 2024-05-17T22:52:55.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.056+0000 7f99baee2b80 20 client.26183 trim_cache size 0 max 16384 2024-05-17T22:52:55.063 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:52:55.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.061+0000 7f99ac507700 1 -- 172.21.15.12:0/1690458296 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f99900b29f0 msgr2=0x7f99900b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:55.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.061+0000 7f99ac507700 1 --2- 172.21.15.12:0/1690458296 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f99900b29f0 0x7f99900b4f00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f999c009e60 tx=0x7f999c0093c0 comp rx=0 tx=0).stop 2024-05-17T22:52:55.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.061+0000 7f99ac507700 1 -- 172.21.15.12:0/1690458296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99a410b180 msgr2=0x7f99a41a9bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:55.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.061+0000 7f99ac507700 1 --2- 172.21.15.12:0/1690458296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99a410b180 0x7f99a41a9bd0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f99a000c920 tx=0x7f99a000cce0 comp rx=0 tx=0).stop 2024-05-17T22:52:55.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.061+0000 7f99ac507700 1 -- 172.21.15.12:0/1690458296 shutdown_connections 2024-05-17T22:52:55.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.061+0000 7f99ac507700 1 --2- 172.21.15.12:0/1690458296 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f99900b29f0 0x7f99900b4f00 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:55.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.061+0000 7f99ac507700 1 --2- 172.21.15.12:0/1690458296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99a411d550 0x7f99a41ade60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:55.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.061+0000 7f99ac507700 1 --2- 172.21.15.12:0/1690458296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99a410a830 0x7f99a41a9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:55.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.061+0000 7f99ac507700 1 --2- 172.21.15.12:0/1690458296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99a410b180 0x7f99a41a9bd0 secure :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f99a000c920 tx=0x7f99a000cce0 comp rx=0 tx=0).stop 2024-05-17T22:52:55.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.061+0000 7f99ac507700 1 -- 172.21.15.12:0/1690458296 >> 172.21.15.12:0/1690458296 conn(0x7f99a406efc0 msgr2=0x7f99a411b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:52:55.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.064+0000 7f99ac507700 1 -- 172.21.15.12:0/1690458296 shutdown_connections 2024-05-17T22:52:55.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:55.064+0000 7f99ac507700 1 -- 172.21.15.12:0/1690458296 wait complete. 2024-05-17T22:52:55.933 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-17T22:52:56.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:56 smithi012 ceph-mon[31632]: pgmap v849: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.5 MiB/s wr, 490 op/s 2024-05-17T22:52:56.375 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 815.0 s, 468.8 tps, lat 216.412 ms stddev 265.581 2024-05-17T22:52:56.417 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:52:56.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:56 smithi151 ceph-mon[36101]: pgmap v849: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.5 MiB/s wr, 490 op/s 2024-05-17T22:52:56.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:56 smithi096 ceph-mon[36065]: pgmap v849: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.5 MiB/s wr, 490 op/s 2024-05-17T22:52:58.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:52:58 smithi012 ceph-mon[31632]: pgmap v850: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.8 MiB/s wr, 399 op/s 2024-05-17T22:52:58.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:52:58 smithi151 ceph-mon[36101]: pgmap v850: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.8 MiB/s wr, 399 op/s 2024-05-17T22:52:58.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:52:58 smithi096 ceph-mon[36065]: pgmap v850: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.8 MiB/s wr, 399 op/s 2024-05-17T22:52:59.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:59.991+0000 7f564bda0700 1 -- 172.21.15.12:0/4135722828 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f564410b180 msgr2=0x7f564411d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:59.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:59.991+0000 7f564bda0700 1 --2- 172.21.15.12:0/4135722828 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f564410b180 0x7f564411d010 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f563c00b0a0 tx=0x7f563c00b3b0 comp rx=0 tx=0).stop 2024-05-17T22:52:59.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:59.992+0000 7f564bda0700 1 -- 172.21.15.12:0/4135722828 shutdown_connections 2024-05-17T22:52:59.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:59.992+0000 7f564bda0700 1 --2- 172.21.15.12:0/4135722828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f564411d550 0x7f564411f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:59.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:59.992+0000 7f564bda0700 1 --2- 172.21.15.12:0/4135722828 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f564410a830 0x7f564410ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:59.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:59.992+0000 7f564bda0700 1 --2- 172.21.15.12:0/4135722828 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f564410b180 0x7f564411d010 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:59.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:59.992+0000 7f564bda0700 1 -- 172.21.15.12:0/4135722828 >> 172.21.15.12:0/4135722828 conn(0x7f564406efc0 msgr2=0x7f564406f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:52:59.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:59.992+0000 7f564bda0700 1 -- 172.21.15.12:0/4135722828 shutdown_connections 2024-05-17T22:52:59.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:59.992+0000 7f564bda0700 1 -- 172.21.15.12:0/4135722828 wait complete. 2024-05-17T22:52:59.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:59.993+0000 7f564bda0700 1 Processor -- start 2024-05-17T22:52:59.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:59.993+0000 7f564bda0700 1 -- start start 2024-05-17T22:52:59.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:59.994+0000 7f564bda0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f564410a830 0x7f56441a96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:59.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:59.994+0000 7f564bda0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f564410b180 0x7f56441a9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:59.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:59.994+0000 7f564bda0700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f564411d550 0x7f56441adf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:52:59.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:59.994+0000 7f564bda0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5644122d40 con 0x7f564410a830 2024-05-17T22:52:59.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:59.994+0000 7f564bda0700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5644122bc0 con 0x7f564410b180 2024-05-17T22:52:59.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:59.994+0000 7f564bda0700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5644122ec0 con 0x7f564411d550 2024-05-17T22:52:59.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:59.994+0000 7f5649b3c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f564410a830 0x7f56441a96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:59.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:59.994+0000 7f564a33d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f564411d550 0x7f56441adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:59.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:59.994+0000 7f564a33d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f564411d550 0x7f56441adf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:47992/0 (socket says 172.21.15.12:47992) 2024-05-17T22:52:59.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:59.994+0000 7f564a33d700 1 -- 172.21.15.12:0/3540417655 learned_addr learned my addr 172.21.15.12:0/3540417655 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:52:59.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:59.994+0000 7f564933b700 1 --2- 172.21.15.12:0/3540417655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f564410b180 0x7f56441a9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:52:59.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:59.994+0000 7f564a33d700 1 -- 172.21.15.12:0/3540417655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f564410b180 msgr2=0x7f56441a9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:59.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:59.994+0000 7f564a33d700 1 --2- 172.21.15.12:0/3540417655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f564410b180 0x7f56441a9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:59.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:59.994+0000 7f564a33d700 1 -- 172.21.15.12:0/3540417655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f564410a830 msgr2=0x7f56441a96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:52:59.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:59.994+0000 7f564a33d700 1 --2- 172.21.15.12:0/3540417655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f564410a830 0x7f56441a96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:52:59.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:59.994+0000 7f564a33d700 1 -- 172.21.15.12:0/3540417655 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56441ae5b0 con 0x7f564411d550 2024-05-17T22:52:59.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:59.994+0000 7f564a33d700 1 --2- 172.21.15.12:0/3540417655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f564411d550 0x7f56441adf10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f563400b4a0 tx=0x7f563400b7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:52:59.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:59.995+0000 7f563affd700 1 -- 172.21.15.12:0/3540417655 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5634015020 con 0x7f564411d550 2024-05-17T22:52:59.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:59.995+0000 7f563affd700 1 -- 172.21.15.12:0/3540417655 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5634011ce0 con 0x7f564411d550 2024-05-17T22:52:59.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:59.995+0000 7f564bda0700 1 -- 172.21.15.12:0/3540417655 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56441ae8a0 con 0x7f564411d550 2024-05-17T22:52:59.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:59.995+0000 7f563affd700 1 -- 172.21.15.12:0/3540417655 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5634010c80 con 0x7f564411d550 2024-05-17T22:52:59.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:52:59.995+0000 7f564bda0700 1 -- 172.21.15.12:0/3540417655 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56441b61c0 con 0x7f564411d550 2024-05-17T22:53:00.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.000+0000 7f563affd700 1 -- 172.21.15.12:0/3540417655 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5634018020 con 0x7f564411d550 2024-05-17T22:53:00.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.001+0000 7f563affd700 1 --2- 172.21.15.12:0/3540417655 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f56300b29c0 0x7f56300b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:00.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.001+0000 7f5649b3c700 1 --2- 172.21.15.12:0/3540417655 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f56300b29c0 0x7f56300b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:00.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.001+0000 7f563affd700 1 -- 172.21.15.12:0/3540417655 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f56340e2ba0 con 0x7f564411d550 2024-05-17T22:53:00.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.002+0000 7f5649b3c700 1 --2- 172.21.15.12:0/3540417655 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f56300b29c0 0x7f56300b4ed0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5640009c60 tx=0x7f5640009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:53:00.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.003+0000 7f565a77bb80 1 Processor -- start 2024-05-17T22:53:00.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.003+0000 7f565a77bb80 1 -- start start 2024-05-17T22:53:00.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.004+0000 7f565a77bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e522fc89f0 0x55e52314c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:00.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.004+0000 7f565a77bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e52314caa0 0x55e523146c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:00.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.004+0000 7f565a77bb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e523147170 0x55e523147580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:00.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.004+0000 7f5649b3c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e522fc89f0 0x55e52314c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:00.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.004+0000 7f5649b3c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e522fc89f0 0x55e52314c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57508/0 (socket says 172.21.15.12:57508) 2024-05-17T22:53:00.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.004+0000 7f5649b3c700 1 -- 172.21.15.12:0/1079756642 learned_addr learned my addr 172.21.15.12:0/1079756642 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:53:00.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.004+0000 7f565a77bb80 1 -- 172.21.15.12:0/1079756642 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e523075290 con 0x55e522fc89f0 2024-05-17T22:53:00.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.004+0000 7f564a33d700 1 --2- 172.21.15.12:0/1079756642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e523147170 0x55e523147580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:00.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.004+0000 7f565a77bb80 1 -- 172.21.15.12:0/1079756642 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e523064fd0 con 0x55e523147170 2024-05-17T22:53:00.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.004+0000 7f565a77bb80 1 -- 172.21.15.12:0/1079756642 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e523068e70 con 0x55e52314caa0 2024-05-17T22:53:00.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.004+0000 7f564933b700 1 --2- 172.21.15.12:0/1079756642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e52314caa0 0x55e523146c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:00.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.005+0000 7f564a33d700 1 -- 172.21.15.12:0/1079756642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e52314caa0 msgr2=0x55e523146c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:53:00.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.005+0000 7f564a33d700 1 --2- 172.21.15.12:0/1079756642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e52314caa0 0x55e523146c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:00.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.005+0000 7f564a33d700 1 -- 172.21.15.12:0/1079756642 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e522fc89f0 msgr2=0x55e52314c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:53:00.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.005+0000 7f564a33d700 1 --2- 172.21.15.12:0/1079756642 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e522fc89f0 0x55e52314c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:00.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.005+0000 7f564a33d700 1 -- 172.21.15.12:0/1079756642 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e52300e590 con 0x55e523147170 2024-05-17T22:53:00.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.005+0000 7f5649b3c700 1 --2- 172.21.15.12:0/1079756642 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e522fc89f0 0x55e52314c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:53:00.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.005+0000 7f564a33d700 1 --2- 172.21.15.12:0/1079756642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e523147170 0x55e523147580 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f56340a5e50 tx=0x7f56340a9c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e09cff9f4433f97e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:53:00.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.005+0000 7f562ffff700 1 -- 172.21.15.12:0/1079756642 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56340a98e0 con 0x55e523147170 2024-05-17T22:53:00.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.005+0000 7f562ffff700 1 -- 172.21.15.12:0/1079756642 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56340aa4d0 con 0x55e523147170 2024-05-17T22:53:00.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.005+0000 7f562ffff700 1 -- 172.21.15.12:0/1079756642 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56340b2a00 con 0x55e523147170 2024-05-17T22:53:00.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.006+0000 7f565a77bb80 1 -- 172.21.15.12:0/1079756642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e523147170 msgr2=0x55e523147580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:00.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.006+0000 7f565a77bb80 1 --2- 172.21.15.12:0/1079756642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e523147170 0x55e523147580 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f56340a5e50 tx=0x7f56340a9c20 comp rx=0 tx=0).stop 2024-05-17T22:53:00.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.006+0000 7f565a77bb80 1 -- 172.21.15.12:0/1079756642 shutdown_connections 2024-05-17T22:53:00.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.006+0000 7f565a77bb80 1 --2- 172.21.15.12:0/1079756642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e523147170 0x55e523147580 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:00.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.006+0000 7f565a77bb80 1 --2- 172.21.15.12:0/1079756642 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e522fc89f0 0x55e52314c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:00.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.006+0000 7f565a77bb80 1 --2- 172.21.15.12:0/1079756642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e52314caa0 0x55e523146c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:00.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.006+0000 7f565a77bb80 1 -- 172.21.15.12:0/1079756642 >> 172.21.15.12:0/1079756642 conn(0x55e52300cf00 msgr2=0x55e523141b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:53:00.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.007+0000 7f565a77bb80 1 -- 172.21.15.12:0/1079756642 shutdown_connections 2024-05-17T22:53:00.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.007+0000 7f565a77bb80 1 -- 172.21.15.12:0/1079756642 wait complete. 2024-05-17T22:53:00.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.007+0000 7f565a77bb80 1 -- start start 2024-05-17T22:53:00.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.007+0000 7f565a77bb80 1 Processor -- start 2024-05-17T22:53:00.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.007+0000 7f565a77bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e522fc89f0 0x55e523150240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:00.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.007+0000 7f565a77bb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e523147170 0x55e523150780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:00.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.007+0000 7f565a77bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e52314caa0 0x55e523150d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:00.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.008+0000 7f565a77bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e522f6c150 con 0x55e522fc89f0 2024-05-17T22:53:00.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.008+0000 7f565a77bb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e523005e80 con 0x55e523147170 2024-05-17T22:53:00.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.008+0000 7f565a77bb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e523153390 con 0x55e52314caa0 2024-05-17T22:53:00.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.008+0000 7f564933b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e523147170 0x55e523150780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:00.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.008+0000 7f5649b3c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e522fc89f0 0x55e523150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:00.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.008+0000 7f564933b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e523147170 0x55e523150780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:50398/0 (socket says 172.21.15.12:50398) 2024-05-17T22:53:00.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.008+0000 7f564933b700 1 -- 172.21.15.12:0/3993088021 learned_addr learned my addr 172.21.15.12:0/3993088021 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:53:00.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.008+0000 7f564a33d700 1 --2- 172.21.15.12:0/3993088021 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e52314caa0 0x55e523150d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:00.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.008+0000 7f565a77bb80 10 client.0 _ll_register_callbacks cb 0x55e522ffff10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:53:00.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.008+0000 7f564933b700 1 -- 172.21.15.12:0/3993088021 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e52314caa0 msgr2=0x55e523150d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:53:00.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.008+0000 7f564933b700 1 --2- 172.21.15.12:0/3993088021 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e52314caa0 0x55e523150d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:00.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.008+0000 7f564933b700 1 -- 172.21.15.12:0/3993088021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e522fc89f0 msgr2=0x55e523150240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:53:00.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.008+0000 7f564933b700 1 --2- 172.21.15.12:0/3993088021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e522fc89f0 0x55e523150240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:00.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.008+0000 7f564933b700 1 -- 172.21.15.12:0/3993088021 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e522fb5480 con 0x55e523147170 2024-05-17T22:53:00.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.008+0000 7f5649b3c700 1 --2- 172.21.15.12:0/3993088021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e522fc89f0 0x55e523150240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:53:00.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.008+0000 7f564933b700 1 --2- 172.21.15.12:0/3993088021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e523147170 0x55e523150780 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f563c00b0a0 tx=0x7f563c00bf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b20620969899de9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:53:00.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.008+0000 7f562d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:53:00.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.008+0000 7f562d7fa700 1 -- 172.21.15.12:0/3993088021 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f563c0075c0 con 0x55e523147170 2024-05-17T22:53:00.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.008+0000 7f562d7fa700 1 -- 172.21.15.12:0/3993088021 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f563c007760 con 0x55e523147170 2024-05-17T22:53:00.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.009+0000 7f565a77bb80 1 -- 172.21.15.12:0/3993088021 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56340aa4d0 con 0x55e523147170 2024-05-17T22:53:00.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.009+0000 7f562d7fa700 1 -- 172.21.15.12:0/3993088021 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f563c007960 con 0x55e523147170 2024-05-17T22:53:00.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.009+0000 7f565a77bb80 10 client.26195 fetch_fsmap learned FSMap version 41 2024-05-17T22:53:00.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.009+0000 7f565a77bb80 1 -- 172.21.15.12:0/3993088021 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e52315de50 con 0x55e523147170 2024-05-17T22:53:00.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.009+0000 7f562d7fa700 1 -- 172.21.15.12:0/3993088021 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f563c028480 con 0x55e523147170 2024-05-17T22:53:00.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.009+0000 7f562d7fa700 1 -- 172.21.15.12:0/3993088021 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f563c011070 con 0x55e523147170 2024-05-17T22:53:00.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.009+0000 7f565a77bb80 10 client.26195 fetch_fsmap finished waiting for FSMap version 41 2024-05-17T22:53:00.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.009+0000 7f565a77bb80 10 client.26195 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-17T22:53:00.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.010+0000 7f565a77bb80 20 client.26195 populate_metadata read hostname 'smithi012' 2024-05-17T22:53:00.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.010+0000 7f565a77bb80 1 --2- 172.21.15.12:0/3993088021 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55e523160490 0x55e523162930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:00.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.010+0000 7f565a77bb80 4 client.26195 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:53:00.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.010+0000 7f565a77bb80 1 -- 172.21.15.12:0/3993088021 --> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f56340aa4d0 con 0x55e523160490 2024-05-17T22:53:00.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.010+0000 7f5649b3c700 1 --2- 172.21.15.12:0/3993088021 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55e523160490 0x55e523162930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:00.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.010+0000 7f562d7fa700 10 client.26195 ms_handle_connect on v2:172.21.15.96:6836/4222574338 2024-05-17T22:53:00.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.010+0000 7f5649b3c700 1 --2- 172.21.15.12:0/3993088021 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55e523160490 0x55e523162930 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=69864b2fb94cbeb7 server_cookie=d31ab746b18576e8 in_seq=0 out_seq=0 2024-05-17T22:53:00.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.011+0000 7f562d7fa700 1 -- 172.21.15.12:0/3993088021 <== mds.1 v2:172.21.15.96:6836/4222574338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f56340aa4d0 con 0x55e523160490 2024-05-17T22:53:00.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.011+0000 7f562d7fa700 10 client.26195 handle_command_reply: tid=0 2024-05-17T22:53:00.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.011+0000 7f565a77bb80 1 client.26195 shutdown 2024-05-17T22:53:00.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.011+0000 7f565a77bb80 1 -- 172.21.15.12:0/3993088021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e523147170 msgr2=0x55e523150780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:00.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.011+0000 7f565a77bb80 1 --2- 172.21.15.12:0/3993088021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e523147170 0x55e523150780 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f563c00b0a0 tx=0x7f563c00bf00 comp rx=0 tx=0).stop 2024-05-17T22:53:00.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.011+0000 7f565a77bb80 1 -- 172.21.15.12:0/3993088021 shutdown_connections 2024-05-17T22:53:00.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.011+0000 7f565a77bb80 1 --2- 172.21.15.12:0/3993088021 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e52314caa0 0x55e523150d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:00.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.011+0000 7f565a77bb80 1 --2- 172.21.15.12:0/3993088021 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55e523160490 0x55e523162930 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:00.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.011+0000 7f565a77bb80 1 --2- 172.21.15.12:0/3993088021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e522fc89f0 0x55e523150240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:00.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.011+0000 7f562d7fa700 0 client.26195 ms_handle_reset on v2:172.21.15.96:6836/4222574338 2024-05-17T22:53:00.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.011+0000 7f565a77bb80 1 --2- 172.21.15.12:0/3993088021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e523147170 0x55e523150780 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:00.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.011+0000 7f565a77bb80 1 -- 172.21.15.12:0/3993088021 >> 172.21.15.12:0/3993088021 conn(0x55e52300cf00 msgr2=0x55e52313bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:53:00.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.012+0000 7f565a77bb80 1 -- 172.21.15.12:0/3993088021 shutdown_connections 2024-05-17T22:53:00.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.012+0000 7f565a77bb80 1 -- 172.21.15.12:0/3993088021 wait complete. 2024-05-17T22:53:00.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.012+0000 7f565a77bb80 20 client.26195 trim_cache size 0 max 16384 2024-05-17T22:53:00.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.036+0000 7f565a77bb80 1 Processor -- start 2024-05-17T22:53:00.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.036+0000 7f565a77bb80 1 -- start start 2024-05-17T22:53:00.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.036+0000 7f565a77bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e522fc89f0 0x55e523147580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:00.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.036+0000 7f565a77bb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e523147170 0x55e52313beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:00.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.036+0000 7f565a77bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e52314caa0 0x55e523145c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:00.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.036+0000 7f565a77bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f56441c1410 con 0x55e522fc89f0 2024-05-17T22:53:00.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.036+0000 7f565a77bb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f56441beca0 con 0x55e523147170 2024-05-17T22:53:00.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.036+0000 7f565a77bb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f56441bcaf0 con 0x55e52314caa0 2024-05-17T22:53:00.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.037+0000 7f564a33d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e52314caa0 0x55e523145c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:00.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.037+0000 7f564a33d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e52314caa0 0x55e523145c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:48018/0 (socket says 172.21.15.12:48018) 2024-05-17T22:53:00.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.037+0000 7f564a33d700 1 -- 172.21.15.12:0/1503855726 learned_addr learned my addr 172.21.15.12:0/1503855726 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:53:00.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.041+0000 7f564a33d700 1 -- 172.21.15.12:0/1503855726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e523147170 msgr2=0x55e52313beb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T22:53:00.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.041+0000 7f564a33d700 1 --2- 172.21.15.12:0/1503855726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e523147170 0x55e52313beb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:00.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.041+0000 7f564a33d700 1 -- 172.21.15.12:0/1503855726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e522fc89f0 msgr2=0x55e523147580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:53:00.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.041+0000 7f564a33d700 1 --2- 172.21.15.12:0/1503855726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e522fc89f0 0x55e523147580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:00.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.041+0000 7f564a33d700 1 -- 172.21.15.12:0/1503855726 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e52314e7b0 con 0x55e52314caa0 2024-05-17T22:53:00.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.041+0000 7f564a33d700 1 --2- 172.21.15.12:0/1503855726 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e52314caa0 0x55e523145c20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f56340a9d10 tx=0x7f5634010f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0bfd446a117288d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:53:00.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.041+0000 7f562f7fe700 1 -- 172.21.15.12:0/1503855726 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5634089070 con 0x55e52314caa0 2024-05-17T22:53:00.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.041+0000 7f562f7fe700 1 -- 172.21.15.12:0/1503855726 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56340aa4d0 con 0x55e52314caa0 2024-05-17T22:53:00.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.041+0000 7f562f7fe700 1 -- 172.21.15.12:0/1503855726 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56340b29e0 con 0x55e52314caa0 2024-05-17T22:53:00.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.047+0000 7f565a77bb80 1 -- 172.21.15.12:0/1503855726 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e52314caa0 msgr2=0x55e523145c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:00.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.047+0000 7f565a77bb80 1 --2- 172.21.15.12:0/1503855726 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e52314caa0 0x55e523145c20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f56340a9d10 tx=0x7f5634010f00 comp rx=0 tx=0).stop 2024-05-17T22:53:00.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.047+0000 7f565a77bb80 1 -- 172.21.15.12:0/1503855726 shutdown_connections 2024-05-17T22:53:00.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.047+0000 7f565a77bb80 1 --2- 172.21.15.12:0/1503855726 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e52314caa0 0x55e523145c20 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:00.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.047+0000 7f565a77bb80 1 --2- 172.21.15.12:0/1503855726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e522fc89f0 0x55e523147580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:00.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.047+0000 7f565a77bb80 1 --2- 172.21.15.12:0/1503855726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e523147170 0x55e52313beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:00.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.047+0000 7f565a77bb80 1 -- 172.21.15.12:0/1503855726 >> 172.21.15.12:0/1503855726 conn(0x55e52300cf00 msgr2=0x55e523143090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:53:00.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.060+0000 7f565a77bb80 1 -- 172.21.15.12:0/1503855726 shutdown_connections 2024-05-17T22:53:00.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.063+0000 7f565a77bb80 1 -- 172.21.15.12:0/1503855726 wait complete. 2024-05-17T22:53:00.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.063+0000 7f565a77bb80 1 -- start start 2024-05-17T22:53:00.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.064+0000 7f565a77bb80 1 Processor -- start 2024-05-17T22:53:00.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.064+0000 7f565a77bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e522fc89f0 0x55e523158250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:00.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.064+0000 7f565a77bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e523147170 0x55e523158790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:00.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.064+0000 7f565a77bb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e52314caa0 0x55e5231670e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:00.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.065+0000 7f565a77bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f56441bd160 con 0x55e522fc89f0 2024-05-17T22:53:00.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.065+0000 7f565a77bb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f56441be5d0 con 0x55e52314caa0 2024-05-17T22:53:00.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.065+0000 7f565a77bb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f56441bd830 con 0x55e523147170 2024-05-17T22:53:00.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.065+0000 7f565a77bb80 10 client.0 _ll_register_callbacks cb 0x55e522ffff10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:53:00.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.067+0000 7f564a33d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e52314caa0 0x55e5231670e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:00.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.074+0000 7f5649b3c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e522fc89f0 0x55e523158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:00.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.074+0000 7f5649b3c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e522fc89f0 0x55e523158250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57540/0 (socket says 172.21.15.12:57540) 2024-05-17T22:53:00.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.074+0000 7f5649b3c700 1 -- 172.21.15.12:0/3486375903 learned_addr learned my addr 172.21.15.12:0/3486375903 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:53:00.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.074+0000 7f564a33d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e52314caa0 0x55e5231670e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:50416/0 (socket says 172.21.15.12:50416) 2024-05-17T22:53:00.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.076+0000 7f564933b700 1 --2- 172.21.15.12:0/3486375903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e523147170 0x55e523158790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:00.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.077+0000 7f564933b700 1 -- 172.21.15.12:0/3486375903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e52314caa0 msgr2=0x55e5231670e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:53:00.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.077+0000 7f564933b700 1 --2- 172.21.15.12:0/3486375903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e52314caa0 0x55e5231670e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:00.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.077+0000 7f564933b700 1 -- 172.21.15.12:0/3486375903 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e522fc89f0 msgr2=0x55e523158250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:53:00.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.077+0000 7f564933b700 1 --2- 172.21.15.12:0/3486375903 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e522fc89f0 0x55e523158250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:00.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.077+0000 7f564933b700 1 -- 172.21.15.12:0/3486375903 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e523158cd0 con 0x55e523147170 2024-05-17T22:53:00.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.078+0000 7f564933b700 1 --2- 172.21.15.12:0/3486375903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e523147170 0x55e523158790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x55e522ed9720 tx=0x7f563c0078f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a8413d0a052becd server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:53:00.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.078+0000 7f562d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:53:00.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.079+0000 7f562d7fa700 1 -- 172.21.15.12:0/3486375903 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f563c007b00 con 0x55e523147170 2024-05-17T22:53:00.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.079+0000 7f565a77bb80 1 -- 172.21.15.12:0/3486375903 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56340aa4d0 con 0x55e523147170 2024-05-17T22:53:00.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.079+0000 7f562d7fa700 1 -- 172.21.15.12:0/3486375903 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f563c003a00 con 0x55e523147170 2024-05-17T22:53:00.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.080+0000 7f562d7fa700 1 -- 172.21.15.12:0/3486375903 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f563c003c00 con 0x55e523147170 2024-05-17T22:53:00.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.082+0000 7f565a77bb80 10 client.26128 fetch_fsmap learned FSMap version 41 2024-05-17T22:53:00.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.082+0000 7f565a77bb80 1 -- 172.21.15.12:0/3486375903 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e5231c9fa0 con 0x55e523147170 2024-05-17T22:53:00.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.084+0000 7f562d7fa700 1 -- 172.21.15.12:0/3486375903 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56340aa4d0 con 0x55e523147170 2024-05-17T22:53:00.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.084+0000 7f562d7fa700 1 -- 172.21.15.12:0/3486375903 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f563c034060 con 0x55e523147170 2024-05-17T22:53:00.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.084+0000 7f565a77bb80 10 client.26128 fetch_fsmap finished waiting for FSMap version 41 2024-05-17T22:53:00.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.084+0000 7f565a77bb80 10 client.26128 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-17T22:53:00.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.084+0000 7f565a77bb80 20 client.26128 populate_metadata read hostname 'smithi012' 2024-05-17T22:53:00.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.084+0000 7f565a77bb80 1 --2- 172.21.15.12:0/3486375903 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55e523160490 0x55e5231cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:00.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.084+0000 7f565a77bb80 4 client.26128 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:53:00.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.084+0000 7f565a77bb80 1 -- 172.21.15.12:0/3486375903 --> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e5230771e0 con 0x55e523160490 2024-05-17T22:53:00.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.092+0000 7f5649b3c700 1 --2- 172.21.15.12:0/3486375903 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55e523160490 0x55e5231cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:00.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.097+0000 7f562d7fa700 10 client.26128 ms_handle_connect on v2:172.21.15.96:6836/4222574338 2024-05-17T22:53:00.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.097+0000 7f5649b3c700 1 --2- 172.21.15.12:0/3486375903 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55e523160490 0x55e5231cc720 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7dd57a431d47ab55 server_cookie=5b478fba813d5fa3 in_seq=0 out_seq=0 2024-05-17T22:53:00.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.100+0000 7f562d7fa700 1 -- 172.21.15.12:0/3486375903 <== mds.1 v2:172.21.15.96:6836/4222574338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e5230771e0 con 0x55e523160490 2024-05-17T22:53:00.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.100+0000 7f562d7fa700 10 client.26128 handle_command_reply: tid=0 2024-05-17T22:53:00.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.100+0000 7f565a77bb80 1 client.26128 shutdown 2024-05-17T22:53:00.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.106+0000 7f565a77bb80 1 -- 172.21.15.12:0/3486375903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e523147170 msgr2=0x55e523158790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:00.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.106+0000 7f565a77bb80 1 --2- 172.21.15.12:0/3486375903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e523147170 0x55e523158790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x55e522ed9720 tx=0x7f563c0078f0 comp rx=0 tx=0).stop 2024-05-17T22:53:00.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.112+0000 7f565a77bb80 1 -- 172.21.15.12:0/3486375903 shutdown_connections 2024-05-17T22:53:00.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.112+0000 7f565a77bb80 1 --2- 172.21.15.12:0/3486375903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e52314caa0 0x55e5231670e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:00.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.112+0000 7f565a77bb80 1 --2- 172.21.15.12:0/3486375903 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x55e523160490 0x55e5231cc720 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:00.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.112+0000 7f565a77bb80 1 --2- 172.21.15.12:0/3486375903 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e522fc89f0 0x55e523158250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:00.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.112+0000 7f565a77bb80 1 --2- 172.21.15.12:0/3486375903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e523147170 0x55e523158790 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:00.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.112+0000 7f562d7fa700 0 client.26128 ms_handle_reset on v2:172.21.15.96:6836/4222574338 2024-05-17T22:53:00.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.112+0000 7f565a77bb80 1 -- 172.21.15.12:0/3486375903 >> 172.21.15.12:0/3486375903 conn(0x55e52300cf00 msgr2=0x55e523149d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:53:00.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.114+0000 7f565a77bb80 1 -- 172.21.15.12:0/3486375903 shutdown_connections 2024-05-17T22:53:00.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.114+0000 7f565a77bb80 1 -- 172.21.15.12:0/3486375903 wait complete. 2024-05-17T22:53:00.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.114+0000 7f565a77bb80 20 client.26128 trim_cache size 0 max 16384 2024-05-17T22:53:00.122 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:53:00.123 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:00 smithi012 ceph-mon[31632]: pgmap v851: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.2 MiB/s wr, 426 op/s 2024-05-17T22:53:00.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.126+0000 7f564bda0700 1 -- 172.21.15.12:0/3540417655 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f56300b29c0 msgr2=0x7f56300b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:00.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.126+0000 7f564bda0700 1 --2- 172.21.15.12:0/3540417655 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f56300b29c0 0x7f56300b4ed0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5640009c60 tx=0x7f5640009260 comp rx=0 tx=0).stop 2024-05-17T22:53:00.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.126+0000 7f564bda0700 1 -- 172.21.15.12:0/3540417655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f564411d550 msgr2=0x7f56441adf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:00.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.126+0000 7f564bda0700 1 --2- 172.21.15.12:0/3540417655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f564411d550 0x7f56441adf10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f563400b4a0 tx=0x7f563400b7b0 comp rx=0 tx=0).stop 2024-05-17T22:53:00.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.126+0000 7f564bda0700 1 -- 172.21.15.12:0/3540417655 shutdown_connections 2024-05-17T22:53:00.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.126+0000 7f564bda0700 1 --2- 172.21.15.12:0/3540417655 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f56300b29c0 0x7f56300b4ed0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:00.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.126+0000 7f564bda0700 1 --2- 172.21.15.12:0/3540417655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f564411d550 0x7f56441adf10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:00.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.126+0000 7f564bda0700 1 --2- 172.21.15.12:0/3540417655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f564410a830 0x7f56441a96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:00.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.126+0000 7f564bda0700 1 --2- 172.21.15.12:0/3540417655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f564410b180 0x7f56441a9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:00.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.126+0000 7f564bda0700 1 -- 172.21.15.12:0/3540417655 >> 172.21.15.12:0/3540417655 conn(0x7f564406efc0 msgr2=0x7f564411b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:53:00.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.127+0000 7f564bda0700 1 -- 172.21.15.12:0/3540417655 shutdown_connections 2024-05-17T22:53:00.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:00.128+0000 7f564bda0700 1 -- 172.21.15.12:0/3540417655 wait complete. 2024-05-17T22:53:00.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:00 smithi151 ceph-mon[36101]: pgmap v851: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.2 MiB/s wr, 426 op/s 2024-05-17T22:53:00.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:00 smithi096 ceph-mon[36065]: pgmap v851: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.2 MiB/s wr, 426 op/s 2024-05-17T22:53:00.835 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-17T22:53:01.368 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:53:01.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 820.0 s, 491.2 tps, lat 200.784 ms stddev 239.444 2024-05-17T22:53:02.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:02 smithi012 ceph-mon[31632]: pgmap v852: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.7 MiB/s wr, 499 op/s 2024-05-17T22:53:02.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:02 smithi151 ceph-mon[36101]: pgmap v852: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.7 MiB/s wr, 499 op/s 2024-05-17T22:53:02.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:02 smithi096 ceph-mon[36065]: pgmap v852: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.7 MiB/s wr, 499 op/s 2024-05-17T22:53:04.368 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:04 smithi012 ceph-mon[31632]: pgmap v853: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.1 MiB/s wr, 421 op/s 2024-05-17T22:53:04.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:04 smithi151 ceph-mon[36101]: pgmap v853: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.1 MiB/s wr, 421 op/s 2024-05-17T22:53:04.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:04 smithi096 ceph-mon[36065]: pgmap v853: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.1 MiB/s wr, 421 op/s 2024-05-17T22:53:05.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.364+0000 7fd2aea57700 1 -- 172.21.15.12:0/1805881910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2a81025b0 msgr2=0x7fd2a81029c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:05.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.364+0000 7fd2aea57700 1 --2- 172.21.15.12:0/1805881910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2a81025b0 0x7fd2a81029c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd298009990 tx=0x7fd298009ca0 comp rx=0 tx=0).stop 2024-05-17T22:53:05.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.364+0000 7fd2aea57700 1 -- 172.21.15.12:0/1805881910 shutdown_connections 2024-05-17T22:53:05.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.364+0000 7fd2aea57700 1 --2- 172.21.15.12:0/1805881910 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2a8107830 0x7fd2a810f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:05.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.364+0000 7fd2aea57700 1 --2- 172.21.15.12:0/1805881910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2a81025b0 0x7fd2a81029c0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:05.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.364+0000 7fd2aea57700 1 --2- 172.21.15.12:0/1805881910 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd2a8106ee0 0x7fd2a81072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:05.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.364+0000 7fd2aea57700 1 -- 172.21.15.12:0/1805881910 >> 172.21.15.12:0/1805881910 conn(0x7fd2a80780d0 msgr2=0x7fd2a8078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:53:05.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.364+0000 7fd2aea57700 1 -- 172.21.15.12:0/1805881910 shutdown_connections 2024-05-17T22:53:05.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.364+0000 7fd2aea57700 1 -- 172.21.15.12:0/1805881910 wait complete. 2024-05-17T22:53:05.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.365+0000 7fd2aea57700 1 Processor -- start 2024-05-17T22:53:05.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.365+0000 7fd2aea57700 1 -- start start 2024-05-17T22:53:05.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.366+0000 7fd2aea57700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2a81025b0 0x7fd2a8198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:05.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.366+0000 7fd2aea57700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd2a8106ee0 0x7fd2a8198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:05.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.366+0000 7fd2aea57700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2a8107830 0x7fd2a819cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:05.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.366+0000 7fd2aea57700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a8111c50 con 0x7fd2a8107830 2024-05-17T22:53:05.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.366+0000 7fd2aea57700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a8111ad0 con 0x7fd2a81025b0 2024-05-17T22:53:05.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.366+0000 7fd2aea57700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a8111dd0 con 0x7fd2a8106ee0 2024-05-17T22:53:05.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.366+0000 7fd2acff4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2a8107830 0x7fd2a819cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:05.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.366+0000 7fd2acff4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2a8107830 0x7fd2a819cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57558/0 (socket says 172.21.15.12:57558) 2024-05-17T22:53:05.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.366+0000 7fd2acff4700 1 -- 172.21.15.12:0/3337062776 learned_addr learned my addr 172.21.15.12:0/3337062776 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:53:05.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.367+0000 7fd2acff4700 1 -- 172.21.15.12:0/3337062776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd2a8106ee0 msgr2=0x7fd2a8198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:05.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.367+0000 7fd2acff4700 1 --2- 172.21.15.12:0/3337062776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd2a8106ee0 0x7fd2a8198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:05.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.367+0000 7fd2acff4700 1 -- 172.21.15.12:0/3337062776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2a81025b0 msgr2=0x7fd2a8198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:05.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.367+0000 7fd2acff4700 1 --2- 172.21.15.12:0/3337062776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2a81025b0 0x7fd2a8198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:05.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.367+0000 7fd2acff4700 1 -- 172.21.15.12:0/3337062776 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a819d540 con 0x7fd2a8107830 2024-05-17T22:53:05.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.367+0000 7fd2acff4700 1 --2- 172.21.15.12:0/3337062776 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2a8107830 0x7fd2a819cea0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0003f10 tx=0x7fd2a000e410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:53:05.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.367+0000 7fd2a57fa700 1 -- 172.21.15.12:0/3337062776 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0004bb0 con 0x7fd2a8107830 2024-05-17T22:53:05.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.368+0000 7fd2a57fa700 1 -- 172.21.15.12:0/3337062776 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2a0004d50 con 0x7fd2a8107830 2024-05-17T22:53:05.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.368+0000 7fd2a57fa700 1 -- 172.21.15.12:0/3337062776 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a00135b0 con 0x7fd2a8107830 2024-05-17T22:53:05.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.368+0000 7fd2aea57700 1 -- 172.21.15.12:0/3337062776 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2a819d830 con 0x7fd2a8107830 2024-05-17T22:53:05.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.368+0000 7fd2aea57700 1 -- 172.21.15.12:0/3337062776 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2a81a5150 con 0x7fd2a8107830 2024-05-17T22:53:05.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.371+0000 7fd2a57fa700 1 -- 172.21.15.12:0/3337062776 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd2a0013750 con 0x7fd2a8107830 2024-05-17T22:53:05.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.372+0000 7fd2a57fa700 1 --2- 172.21.15.12:0/3337062776 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd2900b2980 0x7fd2900b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:05.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.372+0000 7fd2a7fff700 1 --2- 172.21.15.12:0/3337062776 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd2900b2980 0x7fd2900b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:05.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.373+0000 7fd2a57fa700 1 -- 172.21.15.12:0/3337062776 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fd2a00e64b0 con 0x7fd2a8107830 2024-05-17T22:53:05.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.373+0000 7fd2a7fff700 1 --2- 172.21.15.12:0/3337062776 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd2900b2980 0x7fd2900b4e90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd2980098e0 tx=0x7fd298005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:53:05.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.373+0000 7fd2bd432b80 1 Processor -- start 2024-05-17T22:53:05.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.373+0000 7fd2bd432b80 1 -- start start 2024-05-17T22:53:05.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.373+0000 7fd2bd432b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556feec099f0 0x556feed8cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:05.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.374+0000 7fd2bd432b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556feed8d400 0x556feed875c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:05.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.374+0000 7fd2bd432b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556feed87b00 0x556feed87f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:05.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.374+0000 7fd2bd432b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x556feecb60d0 con 0x556feec099f0 2024-05-17T22:53:05.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.374+0000 7fd2bd432b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556feeca5c70 con 0x556feed8d400 2024-05-17T22:53:05.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.374+0000 7fd2bd432b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556feeca9cb0 con 0x556feed87b00 2024-05-17T22:53:05.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.374+0000 7fd2a7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556feec099f0 0x556feed8cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:05.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.374+0000 7fd2a77fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556feed8d400 0x556feed875c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:05.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.374+0000 7fd2a7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556feec099f0 0x556feed8cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57564/0 (socket says 172.21.15.12:57564) 2024-05-17T22:53:05.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.374+0000 7fd2a77fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556feed8d400 0x556feed875c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:50448/0 (socket says 172.21.15.12:50448) 2024-05-17T22:53:05.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.374+0000 7fd2a7fff700 1 -- 172.21.15.12:0/928020099 learned_addr learned my addr 172.21.15.12:0/928020099 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:53:05.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.374+0000 7fd2a7fff700 1 -- 172.21.15.12:0/928020099 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556feed87b00 msgr2=0x556feed87f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:53:05.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.374+0000 7fd2acff4700 1 --2- 172.21.15.12:0/928020099 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556feed87b00 0x556feed87f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:05.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.374+0000 7fd2a7fff700 1 --2- 172.21.15.12:0/928020099 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556feed87b00 0x556feed87f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:05.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.374+0000 7fd2a7fff700 1 -- 172.21.15.12:0/928020099 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556feed8d400 msgr2=0x556feed875c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:53:05.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.374+0000 7fd2a7fff700 1 --2- 172.21.15.12:0/928020099 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556feed8d400 0x556feed875c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:05.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.374+0000 7fd2a7fff700 1 -- 172.21.15.12:0/928020099 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556feec3f5c0 con 0x556feec099f0 2024-05-17T22:53:05.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.374+0000 7fd2a77fe700 1 --2- 172.21.15.12:0/928020099 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556feed8d400 0x556feed875c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:53:05.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.375+0000 7fd2a7fff700 1 --2- 172.21.15.12:0/928020099 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556feec099f0 0x556feed8cec0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd29800bd80 tx=0x7fd29801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a1866618772dfb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:53:05.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.375+0000 7fd28e7fc700 1 -- 172.21.15.12:0/928020099 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd298009510 con 0x556feec099f0 2024-05-17T22:53:05.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.375+0000 7fd28e7fc700 1 -- 172.21.15.12:0/928020099 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd29800bdb0 con 0x556feec099f0 2024-05-17T22:53:05.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.375+0000 7fd28e7fc700 1 -- 172.21.15.12:0/928020099 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29802d7c0 con 0x556feec099f0 2024-05-17T22:53:05.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.376+0000 7fd2bd432b80 1 -- 172.21.15.12:0/928020099 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556feec099f0 msgr2=0x556feed8cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:05.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.376+0000 7fd2bd432b80 1 --2- 172.21.15.12:0/928020099 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556feec099f0 0x556feed8cec0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd29800bd80 tx=0x7fd29801c000 comp rx=0 tx=0).stop 2024-05-17T22:53:05.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.376+0000 7fd2bd432b80 1 -- 172.21.15.12:0/928020099 shutdown_connections 2024-05-17T22:53:05.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.376+0000 7fd2bd432b80 1 --2- 172.21.15.12:0/928020099 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556feed87b00 0x556feed87f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:05.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.376+0000 7fd2bd432b80 1 --2- 172.21.15.12:0/928020099 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556feec099f0 0x556feed8cec0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:05.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.376+0000 7fd2bd432b80 1 --2- 172.21.15.12:0/928020099 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556feed8d400 0x556feed875c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:05.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.376+0000 7fd2bd432b80 1 -- 172.21.15.12:0/928020099 >> 172.21.15.12:0/928020099 conn(0x556feec697d0 msgr2=0x556feed821c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:53:05.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.376+0000 7fd2bd432b80 1 -- 172.21.15.12:0/928020099 shutdown_connections 2024-05-17T22:53:05.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.376+0000 7fd2bd432b80 1 -- 172.21.15.12:0/928020099 wait complete. 2024-05-17T22:53:05.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.377+0000 7fd2bd432b80 1 -- start start 2024-05-17T22:53:05.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.377+0000 7fd2bd432b80 1 Processor -- start 2024-05-17T22:53:05.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.378+0000 7fd2bd432b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556feed87b00 0x556feed90e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:05.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.378+0000 7fd2bd432b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556feed8d400 0x556feed91340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:05.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.378+0000 7fd2bd432b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556feeda3900 0x556feeda5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:05.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.378+0000 7fd2bd432b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x556feebad150 con 0x556feed87b00 2024-05-17T22:53:05.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.378+0000 7fd2bd432b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556feec47940 con 0x556feed8d400 2024-05-17T22:53:05.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.378+0000 7fd2bd432b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556feed93d70 con 0x556feeda3900 2024-05-17T22:53:05.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.378+0000 7fd2bd432b80 10 client.0 _ll_register_callbacks cb 0x556feec3cb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:53:05.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.379+0000 7fd2a77fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556feed8d400 0x556feed91340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:05.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.380+0000 7fd2a7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556feed87b00 0x556feed90e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:05.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.381+0000 7fd2a7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556feed87b00 0x556feed90e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57580/0 (socket says 172.21.15.12:57580) 2024-05-17T22:53:05.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.381+0000 7fd2a7fff700 1 -- 172.21.15.12:0/852875259 learned_addr learned my addr 172.21.15.12:0/852875259 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:53:05.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.381+0000 7fd2a7fff700 1 -- 172.21.15.12:0/852875259 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556feeda3900 msgr2=0x556feeda5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:53:05.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.381+0000 7fd2a7fff700 1 --2- 172.21.15.12:0/852875259 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556feeda3900 0x556feeda5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:05.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.381+0000 7fd2a7fff700 1 -- 172.21.15.12:0/852875259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556feed8d400 msgr2=0x556feed91340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:53:05.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.381+0000 7fd2a7fff700 1 --2- 172.21.15.12:0/852875259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556feed8d400 0x556feed91340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:05.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.381+0000 7fd2a7fff700 1 -- 172.21.15.12:0/852875259 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556feebf6480 con 0x556feed87b00 2024-05-17T22:53:05.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.381+0000 7fd2a7fff700 1 --2- 172.21.15.12:0/852875259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556feed87b00 0x556feed90e00 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd29801e040 tx=0x7fd298004fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c64fe23eb3e0bd79 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:53:05.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.381+0000 7fd2877fe700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:53:05.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.382+0000 7fd2877fe700 1 -- 172.21.15.12:0/852875259 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2980047b0 con 0x556feed87b00 2024-05-17T22:53:05.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.382+0000 7fd2bd432b80 1 -- 172.21.15.12:0/852875259 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd29800bdb0 con 0x556feed87b00 2024-05-17T22:53:05.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.382+0000 7fd2877fe700 1 -- 172.21.15.12:0/852875259 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fd298004950 con 0x556feed87b00 2024-05-17T22:53:05.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.382+0000 7fd2877fe700 1 -- 172.21.15.12:0/852875259 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fd29803e050 con 0x556feed87b00 2024-05-17T22:53:05.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.382+0000 7fd2bd432b80 10 client.16497 fetch_fsmap learned FSMap version 41 2024-05-17T22:53:05.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.382+0000 7fd2bd432b80 1 -- 172.21.15.12:0/852875259 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x556feeda76d0 con 0x556feed87b00 2024-05-17T22:53:05.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.382+0000 7fd2877fe700 1 -- 172.21.15.12:0/852875259 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd298032070 con 0x556feed87b00 2024-05-17T22:53:05.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.383+0000 7fd2877fe700 1 -- 172.21.15.12:0/852875259 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7fd29800fc80 con 0x556feed87b00 2024-05-17T22:53:05.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.383+0000 7fd2bd432b80 10 client.16497 fetch_fsmap finished waiting for FSMap version 41 2024-05-17T22:53:05.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.383+0000 7fd2bd432b80 10 client.16497 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:53:05.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.383+0000 7fd2bd432b80 20 client.16497 populate_metadata read hostname 'smithi012' 2024-05-17T22:53:05.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.383+0000 7fd2bd432b80 1 --2- 172.21.15.12:0/852875259 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x556feeda9ba0 0x556feedabfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:05.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.383+0000 7fd2bd432b80 4 client.16497 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:53:05.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.383+0000 7fd2bd432b80 1 -- 172.21.15.12:0/852875259 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd29800bdb0 con 0x556feeda9ba0 2024-05-17T22:53:05.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.383+0000 7fd2a77fe700 1 --2- 172.21.15.12:0/852875259 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x556feeda9ba0 0x556feedabfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:05.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.384+0000 7fd2877fe700 10 client.16497 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:53:05.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.384+0000 7fd2a77fe700 1 --2- 172.21.15.12:0/852875259 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x556feeda9ba0 0x556feedabfc0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=463de6197aa4de85 server_cookie=52d600d7744be05c in_seq=0 out_seq=0 2024-05-17T22:53:05.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.384+0000 7fd2877fe700 1 -- 172.21.15.12:0/852875259 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fd29800bdb0 con 0x556feeda9ba0 2024-05-17T22:53:05.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.384+0000 7fd2877fe700 10 client.16497 handle_command_reply: tid=0 2024-05-17T22:53:05.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.384+0000 7fd2bd432b80 1 client.16497 shutdown 2024-05-17T22:53:05.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.385+0000 7fd2bd432b80 1 -- 172.21.15.12:0/852875259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556feed87b00 msgr2=0x556feed90e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:05.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.385+0000 7fd2bd432b80 1 --2- 172.21.15.12:0/852875259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556feed87b00 0x556feed90e00 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd29801e040 tx=0x7fd298004fe0 comp rx=0 tx=0).stop 2024-05-17T22:53:05.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.385+0000 7fd2bd432b80 1 -- 172.21.15.12:0/852875259 shutdown_connections 2024-05-17T22:53:05.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.385+0000 7fd2bd432b80 1 --2- 172.21.15.12:0/852875259 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556feeda3900 0x556feeda5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:05.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.385+0000 7fd2bd432b80 1 --2- 172.21.15.12:0/852875259 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x556feeda9ba0 0x556feedabfc0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:05.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.385+0000 7fd2bd432b80 1 --2- 172.21.15.12:0/852875259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556feed87b00 0x556feed90e00 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:05.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.385+0000 7fd2bd432b80 1 --2- 172.21.15.12:0/852875259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556feed8d400 0x556feed91340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:05.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.385+0000 7fd2bd432b80 1 -- 172.21.15.12:0/852875259 >> 172.21.15.12:0/852875259 conn(0x556feec697d0 msgr2=0x556feed82160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:53:05.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.385+0000 7fd2877fe700 0 client.16497 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:53:05.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.385+0000 7fd2bd432b80 1 -- 172.21.15.12:0/852875259 shutdown_connections 2024-05-17T22:53:05.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.385+0000 7fd2bd432b80 1 -- 172.21.15.12:0/852875259 wait complete. 2024-05-17T22:53:05.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.385+0000 7fd2bd432b80 20 client.16497 trim_cache size 0 max 16384 2024-05-17T22:53:05.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.408+0000 7fd2bd432b80 1 Processor -- start 2024-05-17T22:53:05.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.409+0000 7fd2bd432b80 1 -- start start 2024-05-17T22:53:05.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.410+0000 7fd2bd432b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556feed87b00 0x556feed875c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:05.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.410+0000 7fd2bd432b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556feed8d400 0x556feed8c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:05.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.410+0000 7fd2bd432b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556feeda3900 0x556feed8fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:05.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.410+0000 7fd2bd432b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a81b0420 con 0x556feeda3900 2024-05-17T22:53:05.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.410+0000 7fd2bd432b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a81adcb0 con 0x556feed8d400 2024-05-17T22:53:05.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.410+0000 7fd2bd432b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a81abb00 con 0x556feed87b00 2024-05-17T22:53:05.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.410+0000 7fd2acff4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556feeda3900 0x556feed8fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:05.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.410+0000 7fd2acff4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556feeda3900 0x556feed8fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57594/0 (socket says 172.21.15.12:57594) 2024-05-17T22:53:05.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.410+0000 7fd2acff4700 1 -- 172.21.15.12:0/2867396362 learned_addr learned my addr 172.21.15.12:0/2867396362 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:53:05.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.410+0000 7fd2a7fff700 1 --2- 172.21.15.12:0/2867396362 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556feed87b00 0x556feed875c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:05.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.410+0000 7fd2a77fe700 1 --2- 172.21.15.12:0/2867396362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556feed8d400 0x556feed8c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:05.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.410+0000 7fd2acff4700 1 -- 172.21.15.12:0/2867396362 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556feed87b00 msgr2=0x556feed875c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:53:05.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.410+0000 7fd2acff4700 1 --2- 172.21.15.12:0/2867396362 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556feed87b00 0x556feed875c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:05.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.410+0000 7fd2acff4700 1 -- 172.21.15.12:0/2867396362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556feed8d400 msgr2=0x556feed8c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:53:05.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.410+0000 7fd2acff4700 1 --2- 172.21.15.12:0/2867396362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556feed8d400 0x556feed8c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:05.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.410+0000 7fd2acff4700 1 -- 172.21.15.12:0/2867396362 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556feed7ec20 con 0x556feeda3900 2024-05-17T22:53:05.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.411+0000 7fd2acff4700 1 --2- 172.21.15.12:0/2867396362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556feeda3900 0x556feed8fb40 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd2a001cff0 tx=0x7fd2a00adf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375b5cbdc0dec501 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:53:05.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.411+0000 7fd28dffb700 1 -- 172.21.15.12:0/2867396362 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a00aecb0 con 0x556feeda3900 2024-05-17T22:53:05.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.411+0000 7fd28dffb700 1 -- 172.21.15.12:0/2867396362 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2a00ae5b0 con 0x556feeda3900 2024-05-17T22:53:05.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.412+0000 7fd28dffb700 1 -- 172.21.15.12:0/2867396362 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0008610 con 0x556feeda3900 2024-05-17T22:53:05.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.412+0000 7fd2bd432b80 1 -- 172.21.15.12:0/2867396362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556feeda3900 msgr2=0x556feed8fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:05.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.412+0000 7fd2bd432b80 1 --2- 172.21.15.12:0/2867396362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556feeda3900 0x556feed8fb40 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd2a001cff0 tx=0x7fd2a00adf00 comp rx=0 tx=0).stop 2024-05-17T22:53:05.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.412+0000 7fd2bd432b80 1 -- 172.21.15.12:0/2867396362 shutdown_connections 2024-05-17T22:53:05.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.412+0000 7fd2bd432b80 1 --2- 172.21.15.12:0/2867396362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556feeda3900 0x556feed8fb40 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:05.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.412+0000 7fd2bd432b80 1 --2- 172.21.15.12:0/2867396362 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556feed87b00 0x556feed875c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:05.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.412+0000 7fd2bd432b80 1 --2- 172.21.15.12:0/2867396362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556feed8d400 0x556feed8c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:05.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.412+0000 7fd2bd432b80 1 -- 172.21.15.12:0/2867396362 >> 172.21.15.12:0/2867396362 conn(0x556feec697d0 msgr2=0x556feed7c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:53:05.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.412+0000 7fd2bd432b80 1 -- 172.21.15.12:0/2867396362 shutdown_connections 2024-05-17T22:53:05.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.412+0000 7fd2bd432b80 1 -- 172.21.15.12:0/2867396362 wait complete. 2024-05-17T22:53:05.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.413+0000 7fd2bd432b80 1 -- start start 2024-05-17T22:53:05.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.413+0000 7fd2bd432b80 1 Processor -- start 2024-05-17T22:53:05.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.413+0000 7fd2bd432b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556feed87b00 0x556feeda0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:05.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.413+0000 7fd2bd432b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556feed8d400 0x556feeda0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:05.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.413+0000 7fd2bd432b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556feeda3900 0x556feee0c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:05.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.413+0000 7fd2bd432b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a81ac170 con 0x556feeda3900 2024-05-17T22:53:05.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.413+0000 7fd2bd432b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a81ad5e0 con 0x556feed8d400 2024-05-17T22:53:05.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.414+0000 7fd2bd432b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a81ac840 con 0x556feed87b00 2024-05-17T22:53:05.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.414+0000 7fd2bd432b80 10 client.0 _ll_register_callbacks cb 0x556feec3cb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:53:05.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.414+0000 7fd2a77fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556feed8d400 0x556feeda0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:05.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.414+0000 7fd2acff4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556feeda3900 0x556feee0c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:05.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.414+0000 7fd2a77fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556feed8d400 0x556feeda0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:50474/0 (socket says 172.21.15.12:50474) 2024-05-17T22:53:05.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.414+0000 7fd2a77fe700 1 -- 172.21.15.12:0/936827281 learned_addr learned my addr 172.21.15.12:0/936827281 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:53:05.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.414+0000 7fd2a77fe700 1 -- 172.21.15.12:0/936827281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556feed87b00 msgr2=0x556feeda0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:53:05.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.414+0000 7fd2a77fe700 1 --2- 172.21.15.12:0/936827281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556feed87b00 0x556feeda0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:05.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.414+0000 7fd2a77fe700 1 -- 172.21.15.12:0/936827281 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556feeda3900 msgr2=0x556feee0c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:53:05.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.414+0000 7fd2a77fe700 1 --2- 172.21.15.12:0/936827281 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556feeda3900 0x556feee0c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:05.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.414+0000 7fd2a77fe700 1 -- 172.21.15.12:0/936827281 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556feed7a4f0 con 0x556feed8d400 2024-05-17T22:53:05.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.414+0000 7fd2a77fe700 1 --2- 172.21.15.12:0/936827281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556feed8d400 0x556feeda0640 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x556feed836b0 tx=0x7fd29c00bba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9843cfbab6e924c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:53:05.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.414+0000 7fd2877fe700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:53:05.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.415+0000 7fd2877fe700 1 -- 172.21.15.12:0/936827281 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29c003c30 con 0x556feed8d400 2024-05-17T22:53:05.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.415+0000 7fd2877fe700 1 -- 172.21.15.12:0/936827281 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fd29c003dd0 con 0x556feed8d400 2024-05-17T22:53:05.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.415+0000 7fd2bd432b80 1 -- 172.21.15.12:0/936827281 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2a00ae5b0 con 0x556feed8d400 2024-05-17T22:53:05.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.415+0000 7fd2877fe700 1 -- 172.21.15.12:0/936827281 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fd29c02e050 con 0x556feed8d400 2024-05-17T22:53:05.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.415+0000 7fd2bd432b80 10 client.26210 fetch_fsmap learned FSMap version 41 2024-05-17T22:53:05.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.415+0000 7fd2bd432b80 1 -- 172.21.15.12:0/936827281 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x556feed91af0 con 0x556feed8d400 2024-05-17T22:53:05.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.415+0000 7fd2877fe700 1 -- 172.21.15.12:0/936827281 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2a00ae5b0 con 0x556feed8d400 2024-05-17T22:53:05.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.415+0000 7fd2877fe700 1 -- 172.21.15.12:0/936827281 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7fd29c01c150 con 0x556feed8d400 2024-05-17T22:53:05.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.415+0000 7fd2bd432b80 10 client.26210 fetch_fsmap finished waiting for FSMap version 41 2024-05-17T22:53:05.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.415+0000 7fd2bd432b80 10 client.26210 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:53:05.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.415+0000 7fd2bd432b80 20 client.26210 populate_metadata read hostname 'smithi012' 2024-05-17T22:53:05.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.415+0000 7fd2bd432b80 1 --2- 172.21.15.12:0/936827281 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x556feee13e00 0x556feee16220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:05.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.415+0000 7fd2bd432b80 4 client.26210 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:53:05.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.415+0000 7fd2bd432b80 1 -- 172.21.15.12:0/936827281 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556feecb8020 con 0x556feee13e00 2024-05-17T22:53:05.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.416+0000 7fd2a7fff700 1 --2- 172.21.15.12:0/936827281 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x556feee13e00 0x556feee16220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:05.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.416+0000 7fd2877fe700 10 client.26210 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:53:05.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.417+0000 7fd2a7fff700 1 --2- 172.21.15.12:0/936827281 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x556feee13e00 0x556feee16220 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec38e4dde593869d server_cookie=6a0ab8cf23ad0dda in_seq=0 out_seq=0 2024-05-17T22:53:05.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.417+0000 7fd2877fe700 1 -- 172.21.15.12:0/936827281 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556feecb8020 con 0x556feee13e00 2024-05-17T22:53:05.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.417+0000 7fd2877fe700 10 client.26210 handle_command_reply: tid=0 2024-05-17T22:53:05.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.417+0000 7fd2bd432b80 1 client.26210 shutdown 2024-05-17T22:53:05.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.417+0000 7fd2bd432b80 1 -- 172.21.15.12:0/936827281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556feed8d400 msgr2=0x556feeda0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:05.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.417+0000 7fd2bd432b80 1 --2- 172.21.15.12:0/936827281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556feed8d400 0x556feeda0640 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x556feed836b0 tx=0x7fd29c00bba0 comp rx=0 tx=0).stop 2024-05-17T22:53:05.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.417+0000 7fd2bd432b80 1 -- 172.21.15.12:0/936827281 shutdown_connections 2024-05-17T22:53:05.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.417+0000 7fd2bd432b80 1 --2- 172.21.15.12:0/936827281 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x556feee13e00 0x556feee16220 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:05.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.417+0000 7fd2bd432b80 1 --2- 172.21.15.12:0/936827281 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556feeda3900 0x556feee0c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:05.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.417+0000 7fd2bd432b80 1 --2- 172.21.15.12:0/936827281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556feed87b00 0x556feeda0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:05.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.417+0000 7fd2bd432b80 1 --2- 172.21.15.12:0/936827281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556feed8d400 0x556feeda0640 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:05.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.417+0000 7fd2bd432b80 1 -- 172.21.15.12:0/936827281 >> 172.21.15.12:0/936827281 conn(0x556feec697d0 msgr2=0x556feed8c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:53:05.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.417+0000 7fd2877fe700 0 client.26210 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:53:05.233 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:53:05.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.417+0000 7fd2bd432b80 1 -- 172.21.15.12:0/936827281 shutdown_connections 2024-05-17T22:53:05.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.417+0000 7fd2bd432b80 1 -- 172.21.15.12:0/936827281 wait complete. 2024-05-17T22:53:05.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.418+0000 7fd2bd432b80 20 client.26210 trim_cache size 0 max 16384 2024-05-17T22:53:05.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.418+0000 7fd2aea57700 1 -- 172.21.15.12:0/3337062776 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd2900b2980 msgr2=0x7fd2900b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:05.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.418+0000 7fd2aea57700 1 --2- 172.21.15.12:0/3337062776 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd2900b2980 0x7fd2900b4e90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd2980098e0 tx=0x7fd298005fb0 comp rx=0 tx=0).stop 2024-05-17T22:53:05.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.418+0000 7fd2aea57700 1 -- 172.21.15.12:0/3337062776 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2a8107830 msgr2=0x7fd2a819cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:05.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.418+0000 7fd2aea57700 1 --2- 172.21.15.12:0/3337062776 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2a8107830 0x7fd2a819cea0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0003f10 tx=0x7fd2a000e410 comp rx=0 tx=0).stop 2024-05-17T22:53:05.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.418+0000 7fd2aea57700 1 -- 172.21.15.12:0/3337062776 shutdown_connections 2024-05-17T22:53:05.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.418+0000 7fd2aea57700 1 --2- 172.21.15.12:0/3337062776 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd2900b2980 0x7fd2900b4e90 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:05.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.418+0000 7fd2aea57700 1 --2- 172.21.15.12:0/3337062776 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2a8107830 0x7fd2a819cea0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:05.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.418+0000 7fd2aea57700 1 --2- 172.21.15.12:0/3337062776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2a81025b0 0x7fd2a8198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:05.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.418+0000 7fd2aea57700 1 --2- 172.21.15.12:0/3337062776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd2a8106ee0 0x7fd2a8198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:05.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.418+0000 7fd2aea57700 1 -- 172.21.15.12:0/3337062776 >> 172.21.15.12:0/3337062776 conn(0x7fd2a80780d0 msgr2=0x7fd2a8104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:53:05.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.419+0000 7fd2aea57700 1 -- 172.21.15.12:0/3337062776 shutdown_connections 2024-05-17T22:53:05.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:04.419+0000 7fd2aea57700 1 -- 172.21.15.12:0/3337062776 wait complete. 2024-05-17T22:53:06.368 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 825.0 s, 408.8 tps, lat 251.437 ms stddev 285.032 2024-05-17T22:53:06.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:06 smithi151 ceph-mon[36101]: pgmap v854: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.6 MiB/s wr, 501 op/s 2024-05-17T22:53:06.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:06 smithi096 ceph-mon[36065]: pgmap v854: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.6 MiB/s wr, 501 op/s 2024-05-17T22:53:06.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:06 smithi012 ceph-mon[31632]: pgmap v854: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.6 MiB/s wr, 501 op/s 2024-05-17T22:53:08.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:08 smithi151 ceph-mon[36101]: pgmap v855: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 3.9 MiB/s wr, 404 op/s 2024-05-17T22:53:08.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:08 smithi096 ceph-mon[36065]: pgmap v855: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 3.9 MiB/s wr, 404 op/s 2024-05-17T22:53:08.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:08 smithi012 ceph-mon[31632]: pgmap v855: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 3.9 MiB/s wr, 404 op/s 2024-05-17T22:53:08.793 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-17T22:53:08.793 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-17T22:53:09.233 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:53:09.712 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:09 smithi012 ceph-mon[31632]: pgmap v856: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.9 MiB/s wr, 413 op/s 2024-05-17T22:53:09.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:09 smithi151 ceph-mon[36101]: pgmap v856: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.9 MiB/s wr, 413 op/s 2024-05-17T22:53:09.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:09 smithi096 ceph-mon[36065]: pgmap v856: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.9 MiB/s wr, 413 op/s 2024-05-17T22:53:11.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 830.0 s, 446.2 tps, lat 217.821 ms stddev 249.493 2024-05-17T22:53:11.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.450+0000 7f2a9dab8700 1 -- 172.21.15.12:0/2706767569 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a9810b180 msgr2=0x7f2a9811d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:11.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.450+0000 7f2a9dab8700 1 --2- 172.21.15.12:0/2706767569 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a9810b180 0x7f2a9811d010 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f2a88009990 tx=0x7f2a88009ca0 comp rx=0 tx=0).stop 2024-05-17T22:53:11.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.451+0000 7f2a9dab8700 1 -- 172.21.15.12:0/2706767569 shutdown_connections 2024-05-17T22:53:11.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.451+0000 7f2a9dab8700 1 --2- 172.21.15.12:0/2706767569 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a9811d550 0x7f2a9811f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:11.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.451+0000 7f2a9dab8700 1 --2- 172.21.15.12:0/2706767569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a9810a830 0x7f2a9810ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:11.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.451+0000 7f2a9dab8700 1 --2- 172.21.15.12:0/2706767569 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a9810b180 0x7f2a9811d010 secure :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f2a88009990 tx=0x7f2a88009ca0 comp rx=0 tx=0).stop 2024-05-17T22:53:11.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.451+0000 7f2a9dab8700 1 -- 172.21.15.12:0/2706767569 >> 172.21.15.12:0/2706767569 conn(0x7f2a9806efc0 msgr2=0x7f2a9806f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:53:11.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.454+0000 7f2a9dab8700 1 -- 172.21.15.12:0/2706767569 shutdown_connections 2024-05-17T22:53:11.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.456+0000 7f2a9dab8700 1 -- 172.21.15.12:0/2706767569 wait complete. 2024-05-17T22:53:11.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.457+0000 7f2a9dab8700 1 Processor -- start 2024-05-17T22:53:11.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.457+0000 7f2a9dab8700 1 -- start start 2024-05-17T22:53:11.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.457+0000 7f2a9dab8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a9810a830 0x7f2a98114c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:11.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.457+0000 7f2a9dab8700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a9811d550 0x7f2a98115190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:11.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.457+0000 7f2a9dab8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a981156d0 0x7f2a98119650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:11.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.457+0000 7f2a9dab8700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2a98122d40 con 0x7f2a9810a830 2024-05-17T22:53:11.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.457+0000 7f2a9dab8700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2a98122bc0 con 0x7f2a981156d0 2024-05-17T22:53:11.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.457+0000 7f2a9dab8700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2a98122ec0 con 0x7f2a9811d550 2024-05-17T22:53:11.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.458+0000 7f2a977fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a9810a830 0x7f2a98114c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:11.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.458+0000 7f2a97fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a981156d0 0x7f2a98119650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:11.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.458+0000 7f2a97fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a981156d0 0x7f2a98119650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:40174/0 (socket says 172.21.15.12:40174) 2024-05-17T22:53:11.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.458+0000 7f2a97fff700 1 -- 172.21.15.12:0/3848450856 learned_addr learned my addr 172.21.15.12:0/3848450856 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:53:11.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.458+0000 7f2a96ffd700 1 --2- 172.21.15.12:0/3848450856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a9811d550 0x7f2a98115190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:11.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.461+0000 7f2a97fff700 1 -- 172.21.15.12:0/3848450856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a9811d550 msgr2=0x7f2a98115190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:11.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.461+0000 7f2a97fff700 1 --2- 172.21.15.12:0/3848450856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a9811d550 0x7f2a98115190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:11.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.461+0000 7f2a97fff700 1 -- 172.21.15.12:0/3848450856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a9810a830 msgr2=0x7f2a98114c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:11.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.461+0000 7f2a97fff700 1 --2- 172.21.15.12:0/3848450856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a9810a830 0x7f2a98114c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:11.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.461+0000 7f2a97fff700 1 -- 172.21.15.12:0/3848450856 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a98119b90 con 0x7f2a981156d0 2024-05-17T22:53:11.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.461+0000 7f2a977fe700 1 --2- 172.21.15.12:0/3848450856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a9810a830 0x7f2a98114c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:53:11.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.461+0000 7f2a97fff700 1 --2- 172.21.15.12:0/3848450856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a981156d0 0x7f2a98119650 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2a9000b970 tx=0x7f2a9000bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:53:11.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.464+0000 7f2a94ff9700 1 -- 172.21.15.12:0/3848450856 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a90019070 con 0x7f2a981156d0 2024-05-17T22:53:11.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.464+0000 7f2a94ff9700 1 -- 172.21.15.12:0/3848450856 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a9000cde0 con 0x7f2a981156d0 2024-05-17T22:53:11.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.464+0000 7f2a94ff9700 1 -- 172.21.15.12:0/3848450856 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a90007350 con 0x7f2a981156d0 2024-05-17T22:53:11.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.465+0000 7f2a9dab8700 1 -- 172.21.15.12:0/3848450856 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a98119e80 con 0x7f2a981156d0 2024-05-17T22:53:11.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.465+0000 7f2a9dab8700 1 -- 172.21.15.12:0/3848450856 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a9811a340 con 0x7f2a981156d0 2024-05-17T22:53:11.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.473+0000 7f2aac493b80 1 Processor -- start 2024-05-17T22:53:11.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.473+0000 7f2aac493b80 1 -- start start 2024-05-17T22:53:11.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.473+0000 7f2aac493b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598fe770a70 0x5598fe8f3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:11.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.473+0000 7f2aac493b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598fe8f4480 0x5598fe8ee640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:11.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.473+0000 7f2aac493b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5598fe8eeb80 0x5598fe8eef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:11.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.473+0000 7f2aac493b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5598fe81d150 con 0x5598fe8f4480 2024-05-17T22:53:11.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.473+0000 7f2aac493b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5598fe80ccf0 con 0x5598fe770a70 2024-05-17T22:53:11.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.473+0000 7f2aac493b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5598fe810d30 con 0x5598fe8eeb80 2024-05-17T22:53:11.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.474+0000 7f2a977fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598fe8f4480 0x5598fe8ee640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:11.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.474+0000 7f2a977fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598fe8f4480 0x5598fe8ee640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45548/0 (socket says 172.21.15.12:45548) 2024-05-17T22:53:11.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.474+0000 7f2a977fe700 1 -- 172.21.15.12:0/1763178022 learned_addr learned my addr 172.21.15.12:0/1763178022 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:53:11.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.475+0000 7f2a96ffd700 1 --2- 172.21.15.12:0/1763178022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598fe770a70 0x5598fe8f3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:11.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.477+0000 7f2a96ffd700 1 --2- 172.21.15.12:0/1763178022 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5598fe8eeb80 0x5598fe8eef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:11.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.480+0000 7f2a96ffd700 1 -- 172.21.15.12:0/1763178022 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5598fe8eeb80 msgr2=0x5598fe8eef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:53:11.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.480+0000 7f2a96ffd700 1 --2- 172.21.15.12:0/1763178022 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5598fe8eeb80 0x5598fe8eef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:11.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.480+0000 7f2a96ffd700 1 -- 172.21.15.12:0/1763178022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598fe8f4480 msgr2=0x5598fe8ee640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:53:11.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.480+0000 7f2a96ffd700 1 --2- 172.21.15.12:0/1763178022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598fe8f4480 0x5598fe8ee640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:11.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.480+0000 7f2a96ffd700 1 -- 172.21.15.12:0/1763178022 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598fe7a6640 con 0x5598fe770a70 2024-05-17T22:53:11.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.480+0000 7f2a94ff9700 1 -- 172.21.15.12:0/3848450856 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2a9000c930 con 0x7f2a981156d0 2024-05-17T22:53:11.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.480+0000 7f2a96ffd700 1 --2- 172.21.15.12:0/1763178022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598fe770a70 0x5598fe8f3f40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2a88008000 tx=0x7f2a88003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8616b4c7e114ad40 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:53:11.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.481+0000 7f2a94ff9700 1 --2- 172.21.15.12:0/3848450856 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f2a800b2a90 0x7f2a800b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:11.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.481+0000 7f2a94ff9700 1 -- 172.21.15.12:0/3848450856 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f2a900e1ba0 con 0x7f2a981156d0 2024-05-17T22:53:11.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.485+0000 7f2a977fe700 1 --2- 172.21.15.12:0/3848450856 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f2a800b2a90 0x7f2a800b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:11.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.485+0000 7f2a7dffb700 1 -- 172.21.15.12:0/1763178022 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a88003960 con 0x5598fe770a70 2024-05-17T22:53:11.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.485+0000 7f2a7dffb700 1 -- 172.21.15.12:0/1763178022 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a880050d0 con 0x5598fe770a70 2024-05-17T22:53:11.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.487+0000 7f2aac493b80 1 -- 172.21.15.12:0/1763178022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598fe770a70 msgr2=0x5598fe8f3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:11.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.487+0000 7f2aac493b80 1 --2- 172.21.15.12:0/1763178022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598fe770a70 0x5598fe8f3f40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2a88008000 tx=0x7f2a88003680 comp rx=0 tx=0).stop 2024-05-17T22:53:11.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.490+0000 7f2aac493b80 1 -- 172.21.15.12:0/1763178022 shutdown_connections 2024-05-17T22:53:11.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.490+0000 7f2aac493b80 1 --2- 172.21.15.12:0/1763178022 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5598fe8eeb80 0x5598fe8eef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:11.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.490+0000 7f2aac493b80 1 --2- 172.21.15.12:0/1763178022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598fe770a70 0x5598fe8f3f40 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:11.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.490+0000 7f2aac493b80 1 --2- 172.21.15.12:0/1763178022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598fe8f4480 0x5598fe8ee640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:11.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.490+0000 7f2aac493b80 1 -- 172.21.15.12:0/1763178022 >> 172.21.15.12:0/1763178022 conn(0x5598fe7d0850 msgr2=0x5598fe8e9240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:53:11.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.491+0000 7f2a977fe700 1 --2- 172.21.15.12:0/3848450856 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f2a800b2a90 0x7f2a800b4fa0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c005fd0 tx=0x7f2a8c005f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:53:11.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.498+0000 7f2aac493b80 1 -- 172.21.15.12:0/1763178022 shutdown_connections 2024-05-17T22:53:11.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.499+0000 7f2aac493b80 1 -- 172.21.15.12:0/1763178022 wait complete. 2024-05-17T22:53:11.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.500+0000 7f2aac493b80 1 -- start start 2024-05-17T22:53:11.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.500+0000 7f2aac493b80 1 Processor -- start 2024-05-17T22:53:11.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.502+0000 7f2aac493b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598fe770a70 0x5598fe8f7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:11.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.502+0000 7f2aac493b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5598fe8eeb80 0x5598fe8f7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:11.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.502+0000 7f2aac493b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598fe8f4480 0x5598fe8f8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:11.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.502+0000 7f2aac493b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5598fe7141d0 con 0x5598fe8f4480 2024-05-17T22:53:11.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.502+0000 7f2aac493b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5598fe7ae9c0 con 0x5598fe770a70 2024-05-17T22:53:11.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.502+0000 7f2aac493b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5598fe8fae40 con 0x5598fe8eeb80 2024-05-17T22:53:11.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.502+0000 7f2aac493b80 10 client.0 _ll_register_callbacks cb 0x5598fe7a3bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:53:11.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.503+0000 7f2a97fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598fe8f4480 0x5598fe8f8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:11.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.503+0000 7f2a97fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598fe8f4480 0x5598fe8f8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45558/0 (socket says 172.21.15.12:45558) 2024-05-17T22:53:11.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.503+0000 7f2a97fff700 1 -- 172.21.15.12:0/1396271333 learned_addr learned my addr 172.21.15.12:0/1396271333 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:53:11.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.504+0000 7f2a977fe700 1 --2- 172.21.15.12:0/1396271333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598fe770a70 0x5598fe8f7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:11.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.508+0000 7f2a96ffd700 1 --2- 172.21.15.12:0/1396271333 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5598fe8eeb80 0x5598fe8f7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:11.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.509+0000 7f2a97fff700 1 -- 172.21.15.12:0/1396271333 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5598fe8eeb80 msgr2=0x5598fe8f7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:53:11.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.509+0000 7f2a97fff700 1 --2- 172.21.15.12:0/1396271333 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5598fe8eeb80 0x5598fe8f7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:11.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.509+0000 7f2a97fff700 1 -- 172.21.15.12:0/1396271333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598fe770a70 msgr2=0x5598fe8f7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:53:11.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.509+0000 7f2a97fff700 1 --2- 172.21.15.12:0/1396271333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598fe770a70 0x5598fe8f7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:11.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.509+0000 7f2a97fff700 1 -- 172.21.15.12:0/1396271333 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598fe75d500 con 0x5598fe8f4480 2024-05-17T22:53:11.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.513+0000 7f2a97fff700 1 --2- 172.21.15.12:0/1396271333 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598fe8f4480 0x5598fe8f8410 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f2a900a6040 tx=0x7f2a900a93e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91425566eee2c551 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:53:11.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.514+0000 7f2a76ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:53:11.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.518+0000 7f2a76ffd700 1 -- 172.21.15.12:0/1396271333 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a900a8dd0 con 0x5598fe8f4480 2024-05-17T22:53:11.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.518+0000 7f2aac493b80 1 -- 172.21.15.12:0/1396271333 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a880050d0 con 0x5598fe8f4480 2024-05-17T22:53:11.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.518+0000 7f2a76ffd700 1 -- 172.21.15.12:0/1396271333 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f2a900a8240 con 0x5598fe8f4480 2024-05-17T22:53:11.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.520+0000 7f2a76ffd700 1 -- 172.21.15.12:0/1396271333 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f2a900ad180 con 0x5598fe8f4480 2024-05-17T22:53:11.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.521+0000 7f2aac493b80 10 client.16512 fetch_fsmap learned FSMap version 41 2024-05-17T22:53:11.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.521+0000 7f2aac493b80 1 -- 172.21.15.12:0/1396271333 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5598fe9058c0 con 0x5598fe8f4480 2024-05-17T22:53:11.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.521+0000 7f2a76ffd700 1 -- 172.21.15.12:0/1396271333 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a880050d0 con 0x5598fe8f4480 2024-05-17T22:53:11.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.521+0000 7f2a76ffd700 1 -- 172.21.15.12:0/1396271333 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f2a900ae310 con 0x5598fe8f4480 2024-05-17T22:53:11.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.522+0000 7f2aac493b80 10 client.16512 fetch_fsmap finished waiting for FSMap version 41 2024-05-17T22:53:11.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.522+0000 7f2aac493b80 10 client.16512 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:53:11.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.522+0000 7f2aac493b80 20 client.16512 populate_metadata read hostname 'smithi012' 2024-05-17T22:53:11.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.522+0000 7f2aac493b80 1 --2- 172.21.15.12:0/1396271333 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5598fe907f00 0x5598fe90a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:11.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.522+0000 7f2aac493b80 4 client.16512 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:53:11.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.522+0000 7f2aac493b80 1 -- 172.21.15.12:0/1396271333 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598fe81f0a0 con 0x5598fe907f00 2024-05-17T22:53:11.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.528+0000 7f2a977fe700 1 --2- 172.21.15.12:0/1396271333 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5598fe907f00 0x5598fe90a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:11.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.530+0000 7f2a977fe700 1 --2- 172.21.15.12:0/1396271333 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5598fe907f00 0x5598fe90a380 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41a5d5e45664c9d7 server_cookie=d52620b31fc477de in_seq=0 out_seq=0 2024-05-17T22:53:11.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.532+0000 7f2a76ffd700 10 client.16512 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:53:11.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.532+0000 7f2a76ffd700 1 -- 172.21.15.12:0/1396271333 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5598fe81f0a0 con 0x5598fe907f00 2024-05-17T22:53:11.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.532+0000 7f2a76ffd700 10 client.16512 handle_command_reply: tid=0 2024-05-17T22:53:11.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.532+0000 7f2aac493b80 1 client.16512 shutdown 2024-05-17T22:53:11.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.536+0000 7f2aac493b80 1 -- 172.21.15.12:0/1396271333 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598fe8f4480 msgr2=0x5598fe8f8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:11.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.536+0000 7f2aac493b80 1 --2- 172.21.15.12:0/1396271333 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598fe8f4480 0x5598fe8f8410 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f2a900a6040 tx=0x7f2a900a93e0 comp rx=0 tx=0).stop 2024-05-17T22:53:11.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.542+0000 7f2aac493b80 1 -- 172.21.15.12:0/1396271333 shutdown_connections 2024-05-17T22:53:11.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.542+0000 7f2aac493b80 1 --2- 172.21.15.12:0/1396271333 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5598fe907f00 0x5598fe90a380 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:11.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.542+0000 7f2aac493b80 1 --2- 172.21.15.12:0/1396271333 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598fe8f4480 0x5598fe8f8410 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:11.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.542+0000 7f2aac493b80 1 --2- 172.21.15.12:0/1396271333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598fe770a70 0x5598fe8f7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:11.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.542+0000 7f2aac493b80 1 --2- 172.21.15.12:0/1396271333 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5598fe8eeb80 0x5598fe8f7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:11.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.542+0000 7f2a76ffd700 0 client.16512 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:53:11.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.542+0000 7f2aac493b80 1 -- 172.21.15.12:0/1396271333 >> 172.21.15.12:0/1396271333 conn(0x5598fe7d0850 msgr2=0x5598fe8e91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:53:11.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.543+0000 7f2aac493b80 1 -- 172.21.15.12:0/1396271333 shutdown_connections 2024-05-17T22:53:11.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.544+0000 7f2aac493b80 1 -- 172.21.15.12:0/1396271333 wait complete. 2024-05-17T22:53:11.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.545+0000 7f2aac493b80 20 client.16512 trim_cache size 0 max 16384 2024-05-17T22:53:11.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.587+0000 7f2aac493b80 1 Processor -- start 2024-05-17T22:53:11.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.590+0000 7f2aac493b80 1 -- start start 2024-05-17T22:53:11.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.590+0000 7f2aac493b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598fe770a70 0x5598fe8ee640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:11.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.590+0000 7f2aac493b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5598fe8eeb80 0x5598fe8e3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:11.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.590+0000 7f2aac493b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598fe8f4480 0x5598fe8f8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:11.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.590+0000 7f2aac493b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2a84011400 con 0x5598fe8f4480 2024-05-17T22:53:11.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.590+0000 7f2aac493b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8400ec90 con 0x5598fe770a70 2024-05-17T22:53:11.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.590+0000 7f2aac493b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8400cae0 con 0x5598fe8eeb80 2024-05-17T22:53:11.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.592+0000 7f2a977fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598fe770a70 0x5598fe8ee640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:11.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.592+0000 7f2a96ffd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5598fe8eeb80 0x5598fe8e3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:11.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.592+0000 7f2a97fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598fe8f4480 0x5598fe8f8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:11.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.592+0000 7f2a97fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598fe8f4480 0x5598fe8f8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45566/0 (socket says 172.21.15.12:45566) 2024-05-17T22:53:11.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.592+0000 7f2a97fff700 1 -- 172.21.15.12:0/1332050487 learned_addr learned my addr 172.21.15.12:0/1332050487 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:53:11.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.593+0000 7f2a97fff700 1 -- 172.21.15.12:0/1332050487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5598fe8eeb80 msgr2=0x5598fe8e3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:53:11.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.593+0000 7f2a97fff700 1 --2- 172.21.15.12:0/1332050487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5598fe8eeb80 0x5598fe8e3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:11.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.593+0000 7f2a97fff700 1 -- 172.21.15.12:0/1332050487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598fe770a70 msgr2=0x5598fe8ee640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:53:11.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.593+0000 7f2a97fff700 1 --2- 172.21.15.12:0/1332050487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598fe770a70 0x5598fe8ee640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:11.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.593+0000 7f2a97fff700 1 -- 172.21.15.12:0/1332050487 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598fe8e5ca0 con 0x5598fe8f4480 2024-05-17T22:53:11.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.593+0000 7f2a977fe700 1 --2- 172.21.15.12:0/1332050487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598fe770a70 0x5598fe8ee640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:53:11.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.593+0000 7f2a97fff700 1 --2- 172.21.15.12:0/1332050487 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598fe8f4480 0x5598fe8f8260 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f2a900a4e50 tx=0x7f2a9000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bedcb6419d13460c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:53:11.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.594+0000 7f2a7d7fa700 1 -- 172.21.15.12:0/1332050487 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a900a8dd0 con 0x5598fe8f4480 2024-05-17T22:53:11.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.594+0000 7f2a7d7fa700 1 -- 172.21.15.12:0/1332050487 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a900b1c30 con 0x5598fe8f4480 2024-05-17T22:53:11.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.594+0000 7f2a7d7fa700 1 -- 172.21.15.12:0/1332050487 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a900a8360 con 0x5598fe8f4480 2024-05-17T22:53:11.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.595+0000 7f2aac493b80 1 -- 172.21.15.12:0/1332050487 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598fe8f4480 msgr2=0x5598fe8f8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:11.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.595+0000 7f2aac493b80 1 --2- 172.21.15.12:0/1332050487 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598fe8f4480 0x5598fe8f8260 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f2a900a4e50 tx=0x7f2a9000cc20 comp rx=0 tx=0).stop 2024-05-17T22:53:11.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.595+0000 7f2aac493b80 1 -- 172.21.15.12:0/1332050487 shutdown_connections 2024-05-17T22:53:11.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.595+0000 7f2aac493b80 1 --2- 172.21.15.12:0/1332050487 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598fe8f4480 0x5598fe8f8260 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:11.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.595+0000 7f2aac493b80 1 --2- 172.21.15.12:0/1332050487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598fe770a70 0x5598fe8ee640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:11.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.595+0000 7f2aac493b80 1 --2- 172.21.15.12:0/1332050487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5598fe8eeb80 0x5598fe8e3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:11.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.595+0000 7f2aac493b80 1 -- 172.21.15.12:0/1332050487 >> 172.21.15.12:0/1332050487 conn(0x5598fe7d0850 msgr2=0x5598fe8f3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:53:11.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.595+0000 7f2aac493b80 1 -- 172.21.15.12:0/1332050487 shutdown_connections 2024-05-17T22:53:11.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.596+0000 7f2aac493b80 1 -- 172.21.15.12:0/1332050487 wait complete. 2024-05-17T22:53:11.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.596+0000 7f2aac493b80 1 -- start start 2024-05-17T22:53:11.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.597+0000 7f2aac493b80 1 Processor -- start 2024-05-17T22:53:11.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.598+0000 7f2aac493b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598fe770a70 0x5598fe90f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:11.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.598+0000 7f2aac493b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598fe8eeb80 0x5598fe90f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:11.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.598+0000 7f2aac493b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5598fe8f4480 0x5598fe90fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:11.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.598+0000 7f2aac493b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8400d150 con 0x5598fe8eeb80 2024-05-17T22:53:11.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.598+0000 7f2aac493b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8400e5c0 con 0x5598fe770a70 2024-05-17T22:53:11.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.599+0000 7f2aac493b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8400d820 con 0x5598fe8f4480 2024-05-17T22:53:11.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.599+0000 7f2aac493b80 10 client.0 _ll_register_callbacks cb 0x5598fe7a3bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:53:11.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.600+0000 7f2a97fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5598fe8f4480 0x5598fe90fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:11.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.600+0000 7f2a977fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598fe770a70 0x5598fe90f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:11.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.600+0000 7f2a97fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5598fe8f4480 0x5598fe90fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:60736/0 (socket says 172.21.15.12:60736) 2024-05-17T22:53:11.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.600+0000 7f2a97fff700 1 -- 172.21.15.12:0/2841451517 learned_addr learned my addr 172.21.15.12:0/2841451517 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:53:11.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.601+0000 7f2a96ffd700 1 --2- 172.21.15.12:0/2841451517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598fe8eeb80 0x5598fe90f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:11.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.601+0000 7f2a96ffd700 1 -- 172.21.15.12:0/2841451517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5598fe8f4480 msgr2=0x5598fe90fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:53:11.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.601+0000 7f2a96ffd700 1 --2- 172.21.15.12:0/2841451517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5598fe8f4480 0x5598fe90fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:11.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.601+0000 7f2a96ffd700 1 -- 172.21.15.12:0/2841451517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598fe770a70 msgr2=0x5598fe90f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:53:11.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.601+0000 7f2a96ffd700 1 --2- 172.21.15.12:0/2841451517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598fe770a70 0x5598fe90f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:11.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.601+0000 7f2a96ffd700 1 -- 172.21.15.12:0/2841451517 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598fe971780 con 0x5598fe8eeb80 2024-05-17T22:53:11.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.602+0000 7f2a96ffd700 1 --2- 172.21.15.12:0/2841451517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598fe8eeb80 0x5598fe90f950 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x5598fe8e1740 tx=0x7f2a88009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5d04260002d0706 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:53:11.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.603+0000 7f2a76ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:53:11.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.603+0000 7f2a76ffd700 1 -- 172.21.15.12:0/2841451517 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a880050d0 con 0x5598fe8eeb80 2024-05-17T22:53:11.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.603+0000 7f2a76ffd700 1 -- 172.21.15.12:0/2841451517 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f2a88021050 con 0x5598fe8eeb80 2024-05-17T22:53:11.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.603+0000 7f2aac493b80 1 -- 172.21.15.12:0/2841451517 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a900b1c30 con 0x5598fe8eeb80 2024-05-17T22:53:11.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.604+0000 7f2a76ffd700 1 -- 172.21.15.12:0/2841451517 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f2a88005270 con 0x5598fe8eeb80 2024-05-17T22:53:11.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.604+0000 7f2aac493b80 10 client.16518 fetch_fsmap learned FSMap version 41 2024-05-17T22:53:11.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.604+0000 7f2aac493b80 1 -- 172.21.15.12:0/2841451517 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5598fe972a20 con 0x5598fe8eeb80 2024-05-17T22:53:11.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.605+0000 7f2a76ffd700 1 -- 172.21.15.12:0/2841451517 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a880054d0 con 0x5598fe8eeb80 2024-05-17T22:53:11.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.605+0000 7f2a76ffd700 1 -- 172.21.15.12:0/2841451517 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f2a88025430 con 0x5598fe8eeb80 2024-05-17T22:53:11.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.606+0000 7f2aac493b80 10 client.16518 fetch_fsmap finished waiting for FSMap version 41 2024-05-17T22:53:11.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.606+0000 7f2aac493b80 10 client.16518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:53:11.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.606+0000 7f2aac493b80 20 client.16518 populate_metadata read hostname 'smithi012' 2024-05-17T22:53:11.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.606+0000 7f2aac493b80 1 --2- 172.21.15.12:0/2841451517 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5598fe972bf0 0x5598fe975010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:11.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.606+0000 7f2aac493b80 4 client.16518 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-17T22:53:11.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.606+0000 7f2aac493b80 1 -- 172.21.15.12:0/2841451517 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f2a900b1c30 con 0x5598fe972bf0 2024-05-17T22:53:11.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.607+0000 7f2a977fe700 1 --2- 172.21.15.12:0/2841451517 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5598fe972bf0 0x5598fe975010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:11.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.612+0000 7f2a977fe700 1 --2- 172.21.15.12:0/2841451517 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5598fe972bf0 0x5598fe975010 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b78c271fc6a51d9 server_cookie=fe3d0da2bba3ef7 in_seq=0 out_seq=0 2024-05-17T22:53:11.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.612+0000 7f2a76ffd700 10 client.16518 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:53:11.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.614+0000 7f2a76ffd700 1 -- 172.21.15.12:0/2841451517 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f2a900b1c30 con 0x5598fe972bf0 2024-05-17T22:53:11.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.614+0000 7f2a76ffd700 10 client.16518 handle_command_reply: tid=0 2024-05-17T22:53:11.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.614+0000 7f2aac493b80 1 client.16518 shutdown 2024-05-17T22:53:11.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.616+0000 7f2aac493b80 1 -- 172.21.15.12:0/2841451517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598fe8eeb80 msgr2=0x5598fe90f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:11.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.616+0000 7f2aac493b80 1 --2- 172.21.15.12:0/2841451517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598fe8eeb80 0x5598fe90f950 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x5598fe8e1740 tx=0x7f2a88009310 comp rx=0 tx=0).stop 2024-05-17T22:53:11.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.616+0000 7f2aac493b80 1 -- 172.21.15.12:0/2841451517 shutdown_connections 2024-05-17T22:53:11.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.616+0000 7f2aac493b80 1 --2- 172.21.15.12:0/2841451517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5598fe8f4480 0x5598fe90fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:11.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.616+0000 7f2aac493b80 1 --2- 172.21.15.12:0/2841451517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598fe770a70 0x5598fe90f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:11.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.616+0000 7f2aac493b80 1 --2- 172.21.15.12:0/2841451517 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5598fe972bf0 0x5598fe975010 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:11.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.616+0000 7f2aac493b80 1 --2- 172.21.15.12:0/2841451517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598fe8eeb80 0x5598fe90f950 secure :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0x5598fe8e1740 tx=0x7f2a88009310 comp rx=0 tx=0).stop 2024-05-17T22:53:11.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.616+0000 7f2a76ffd700 0 client.16518 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:53:11.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.616+0000 7f2aac493b80 1 -- 172.21.15.12:0/2841451517 >> 172.21.15.12:0/2841451517 conn(0x5598fe7d0850 msgr2=0x5598fe8f1870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:53:11.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.617+0000 7f2aac493b80 1 -- 172.21.15.12:0/2841451517 shutdown_connections 2024-05-17T22:53:11.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.618+0000 7f2aac493b80 1 -- 172.21.15.12:0/2841451517 wait complete. 2024-05-17T22:53:11.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.619+0000 7f2aac493b80 20 client.16518 trim_cache size 0 max 16384 2024-05-17T22:53:11.622 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:53:11.622 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-17T22:53:11.623 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "1b32803c-b836-419f-a084-9cb1afff5492", 2024-05-17T22:53:11.623 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-17T22:53:11.623 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:53:11.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.622+0000 7f2a9dab8700 1 -- 172.21.15.12:0/3848450856 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f2a800b2a90 msgr2=0x7f2a800b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:11.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.622+0000 7f2a9dab8700 1 --2- 172.21.15.12:0/3848450856 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f2a800b2a90 0x7f2a800b4fa0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c005fd0 tx=0x7f2a8c005f40 comp rx=0 tx=0).stop 2024-05-17T22:53:11.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.622+0000 7f2a9dab8700 1 -- 172.21.15.12:0/3848450856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a981156d0 msgr2=0x7f2a98119650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:11.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.622+0000 7f2a9dab8700 1 --2- 172.21.15.12:0/3848450856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a981156d0 0x7f2a98119650 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2a9000b970 tx=0x7f2a9000bc80 comp rx=0 tx=0).stop 2024-05-17T22:53:11.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.622+0000 7f2a9dab8700 1 -- 172.21.15.12:0/3848450856 shutdown_connections 2024-05-17T22:53:11.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.622+0000 7f2a9dab8700 1 --2- 172.21.15.12:0/3848450856 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f2a800b2a90 0x7f2a800b4fa0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:11.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.622+0000 7f2a9dab8700 1 --2- 172.21.15.12:0/3848450856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a981156d0 0x7f2a98119650 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:11.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.622+0000 7f2a9dab8700 1 --2- 172.21.15.12:0/3848450856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a9810a830 0x7f2a98114c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:11.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.622+0000 7f2a9dab8700 1 --2- 172.21.15.12:0/3848450856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a9811d550 0x7f2a98115190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:11.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.622+0000 7f2a9dab8700 1 -- 172.21.15.12:0/3848450856 >> 172.21.15.12:0/3848450856 conn(0x7f2a9806efc0 msgr2=0x7f2a98072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:53:11.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.622+0000 7f2a9dab8700 1 -- 172.21.15.12:0/3848450856 shutdown_connections 2024-05-17T22:53:11.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:11.622+0000 7f2a9dab8700 1 -- 172.21.15.12:0/3848450856 wait complete. 2024-05-17T22:53:12.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:12 smithi096 ceph-mon[36065]: pgmap v857: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.2 MiB/s wr, 477 op/s 2024-05-17T22:53:12.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:11 smithi012 ceph-mon[31632]: pgmap v857: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.2 MiB/s wr, 477 op/s 2024-05-17T22:53:12.395 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-17T22:53:12.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:11 smithi151 ceph-mon[36101]: pgmap v857: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.2 MiB/s wr, 477 op/s 2024-05-17T22:53:12.851 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:53:13.249 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:13 smithi012 ceph-mon[31632]: scrub queued for path: / 2024-05-17T22:53:13.249 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:13 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:53:13.250 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:13 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:53:13.250 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:13 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:53:13.250 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:13 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:53:13.250 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:13 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:53:13.250 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:13 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:53:13.250 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:13 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:53:13.250 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:13 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:53:13.250 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:13 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:53:13.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:13 smithi096 ceph-mon[36065]: scrub queued for path: / 2024-05-17T22:53:13.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:13 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:53:13.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:13 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:53:13.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:13 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:53:13.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:13 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:53:13.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:13 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:53:13.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:13 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:53:13.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:13 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:53:13.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:13 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:53:13.300 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:13 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:53:13.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:13 smithi151 ceph-mon[36101]: scrub queued for path: / 2024-05-17T22:53:13.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:13 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:53:13.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:13 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:53:13.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:13 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:53:13.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:13 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:53:13.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:13 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:53:13.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:13 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:53:13.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:13 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:53:13.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:13 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:53:13.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:13 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:53:14.296 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:14 smithi012 ceph-mon[31632]: pgmap v858: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.6 MiB/s wr, 390 op/s 2024-05-17T22:53:14.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:14 smithi096 ceph-mon[36065]: pgmap v858: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.6 MiB/s wr, 390 op/s 2024-05-17T22:53:14.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:14 smithi151 ceph-mon[36101]: pgmap v858: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.6 MiB/s wr, 390 op/s 2024-05-17T22:53:15.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.296+0000 7ff13875f700 1 -- 172.21.15.12:0/3264507835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff13010aa60 msgr2=0x7ff13010ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:15.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.296+0000 7ff13875f700 1 --2- 172.21.15.12:0/3264507835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff13010aa60 0x7ff13010ae70 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ff12c009990 tx=0x7ff12c009ca0 comp rx=0 tx=0).stop 2024-05-17T22:53:15.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.296+0000 7ff13875f700 1 -- 172.21.15.12:0/3264507835 shutdown_connections 2024-05-17T22:53:15.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.296+0000 7ff13875f700 1 --2- 172.21.15.12:0/3264507835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff13010b3b0 0x7ff130112d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:15.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.296+0000 7ff13875f700 1 --2- 172.21.15.12:0/3264507835 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff130106130 0x7ff130106540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:15.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.297+0000 7ff13875f700 1 --2- 172.21.15.12:0/3264507835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff13010aa60 0x7ff13010ae70 secure :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ff12c009990 tx=0x7ff12c009ca0 comp rx=0 tx=0).stop 2024-05-17T22:53:15.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.297+0000 7ff13875f700 1 -- 172.21.15.12:0/3264507835 >> 172.21.15.12:0/3264507835 conn(0x7ff130101bb0 msgr2=0x7ff130104000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:53:15.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.298+0000 7ff13875f700 1 -- 172.21.15.12:0/3264507835 shutdown_connections 2024-05-17T22:53:15.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.298+0000 7ff13875f700 1 -- 172.21.15.12:0/3264507835 wait complete. 2024-05-17T22:53:15.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.299+0000 7ff13875f700 1 Processor -- start 2024-05-17T22:53:15.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.299+0000 7ff13875f700 1 -- start start 2024-05-17T22:53:15.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.300+0000 7ff13875f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff130106130 0x7ff1301a2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:15.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.300+0000 7ff13875f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff13010b3b0 0x7ff1301a2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:15.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.300+0000 7ff13875f700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff13019ca80 0x7ff13019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:15.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.300+0000 7ff13875f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff130116090 con 0x7ff130106130 2024-05-17T22:53:15.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.300+0000 7ff135cfa700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff13010b3b0 0x7ff1301a2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:15.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.300+0000 7ff135cfa700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff13010b3b0 0x7ff1301a2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:40220/0 (socket says 172.21.15.12:40220) 2024-05-17T22:53:15.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.300+0000 7ff135cfa700 1 -- 172.21.15.12:0/920288645 learned_addr learned my addr 172.21.15.12:0/920288645 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:53:15.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.300+0000 7ff1364fb700 1 --2- 172.21.15.12:0/920288645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff130106130 0x7ff1301a2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:15.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.300+0000 7ff13875f700 1 -- 172.21.15.12:0/920288645 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff130115f10 con 0x7ff13010b3b0 2024-05-17T22:53:15.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.300+0000 7ff13875f700 1 -- 172.21.15.12:0/920288645 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff130116210 con 0x7ff13019ca80 2024-05-17T22:53:15.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.301+0000 7ff136cfc700 1 --2- 172.21.15.12:0/920288645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff13019ca80 0x7ff13019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:15.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.301+0000 7ff135cfa700 1 -- 172.21.15.12:0/920288645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff13019ca80 msgr2=0x7ff13019ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:15.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.301+0000 7ff135cfa700 1 --2- 172.21.15.12:0/920288645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff13019ca80 0x7ff13019ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:15.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.301+0000 7ff135cfa700 1 -- 172.21.15.12:0/920288645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff130106130 msgr2=0x7ff1301a2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:15.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.301+0000 7ff135cfa700 1 --2- 172.21.15.12:0/920288645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff130106130 0x7ff1301a2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:15.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.301+0000 7ff135cfa700 1 -- 172.21.15.12:0/920288645 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff13019d620 con 0x7ff13010b3b0 2024-05-17T22:53:15.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.301+0000 7ff136cfc700 1 --2- 172.21.15.12:0/920288645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff13019ca80 0x7ff13019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:53:15.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.302+0000 7ff135cfa700 1 --2- 172.21.15.12:0/920288645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff13010b3b0 0x7ff1301a2e60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff12c000c00 tx=0x7ff12c0057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:53:15.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.302+0000 7ff1277fe700 1 -- 172.21.15.12:0/920288645 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c003a90 con 0x7ff13010b3b0 2024-05-17T22:53:15.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.302+0000 7ff1277fe700 1 -- 172.21.15.12:0/920288645 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff12c0040e0 con 0x7ff13010b3b0 2024-05-17T22:53:15.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.302+0000 7ff1277fe700 1 -- 172.21.15.12:0/920288645 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c00fe20 con 0x7ff13010b3b0 2024-05-17T22:53:15.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.302+0000 7ff13875f700 1 -- 172.21.15.12:0/920288645 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff13019d8b0 con 0x7ff13010b3b0 2024-05-17T22:53:15.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.303+0000 7ff13875f700 1 -- 172.21.15.12:0/920288645 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1301a9920 con 0x7ff13010b3b0 2024-05-17T22:53:15.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.309+0000 7ff1277fe700 1 -- 172.21.15.12:0/920288645 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff12c00f580 con 0x7ff13010b3b0 2024-05-17T22:53:15.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.309+0000 7ff14713ab80 1 Processor -- start 2024-05-17T22:53:15.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.309+0000 7ff14713ab80 1 -- start start 2024-05-17T22:53:15.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.310+0000 7ff14713ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556bc54629f0 0x556bc55e6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:15.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.310+0000 7ff14713ab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556bc55e6aa0 0x556bc55e0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:15.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.310+0000 7ff14713ab80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc55e1170 0x556bc55e1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:15.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.310+0000 7ff14713ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x556bc550f290 con 0x556bc54629f0 2024-05-17T22:53:15.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.310+0000 7ff14713ab80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556bc54fefd0 con 0x556bc55e1170 2024-05-17T22:53:15.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.310+0000 7ff14713ab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556bc5502e70 con 0x556bc55e6aa0 2024-05-17T22:53:15.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.310+0000 7ff135cfa700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc55e1170 0x556bc55e1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:15.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.310+0000 7ff135cfa700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc55e1170 0x556bc55e1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:40236/0 (socket says 172.21.15.12:40236) 2024-05-17T22:53:15.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.310+0000 7ff135cfa700 1 -- 172.21.15.12:0/3714933080 learned_addr learned my addr 172.21.15.12:0/3714933080 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:53:15.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.310+0000 7ff136cfc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556bc54629f0 0x556bc55e6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:15.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.310+0000 7ff1364fb700 1 --2- 172.21.15.12:0/3714933080 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556bc55e6aa0 0x556bc55e0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:15.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.311+0000 7ff135cfa700 1 -- 172.21.15.12:0/3714933080 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556bc55e6aa0 msgr2=0x556bc55e0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:53:15.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.311+0000 7ff1277fe700 1 --2- 172.21.15.12:0/920288645 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff11c0b29d0 0x7ff11c0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:15.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.311+0000 7ff135cfa700 1 --2- 172.21.15.12:0/3714933080 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556bc55e6aa0 0x556bc55e0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:15.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.311+0000 7ff135cfa700 1 -- 172.21.15.12:0/3714933080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556bc54629f0 msgr2=0x556bc55e6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:53:15.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.311+0000 7ff135cfa700 1 --2- 172.21.15.12:0/3714933080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556bc54629f0 0x556bc55e6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:15.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.311+0000 7ff135cfa700 1 -- 172.21.15.12:0/3714933080 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bc54a8590 con 0x556bc55e1170 2024-05-17T22:53:15.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.311+0000 7ff136cfc700 1 --2- 172.21.15.12:0/3714933080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556bc54629f0 0x556bc55e6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:53:15.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.311+0000 7ff1277fe700 1 -- 172.21.15.12:0/920288645 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7ff12c0e3c70 con 0x7ff13010b3b0 2024-05-17T22:53:15.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.311+0000 7ff136cfc700 1 --2- 172.21.15.12:0/920288645 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff11c0b29d0 0x7ff11c0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:15.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.311+0000 7ff135cfa700 1 --2- 172.21.15.12:0/3714933080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc55e1170 0x556bc55e1580 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff12c0afd60 tx=0x7ff12c0a7600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e79ceb483fe7b5d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:53:15.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.311+0000 7ff124ff9700 1 -- 172.21.15.12:0/3714933080 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c0aad00 con 0x556bc55e1170 2024-05-17T22:53:15.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.311+0000 7ff124ff9700 1 -- 172.21.15.12:0/3714933080 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff12c0af760 con 0x556bc55e1170 2024-05-17T22:53:15.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.311+0000 7ff124ff9700 1 -- 172.21.15.12:0/3714933080 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c0be530 con 0x556bc55e1170 2024-05-17T22:53:15.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.313+0000 7ff14713ab80 1 -- 172.21.15.12:0/3714933080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc55e1170 msgr2=0x556bc55e1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:15.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.313+0000 7ff136cfc700 1 --2- 172.21.15.12:0/920288645 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff11c0b29d0 0x7ff11c0b4ee0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x556bc55dcae0 tx=0x7ff12800e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:53:15.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.313+0000 7ff14713ab80 1 --2- 172.21.15.12:0/3714933080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc55e1170 0x556bc55e1580 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff12c0afd60 tx=0x7ff12c0a7600 comp rx=0 tx=0).stop 2024-05-17T22:53:15.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.313+0000 7ff14713ab80 1 -- 172.21.15.12:0/3714933080 shutdown_connections 2024-05-17T22:53:15.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.313+0000 7ff14713ab80 1 --2- 172.21.15.12:0/3714933080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc55e1170 0x556bc55e1580 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:15.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.313+0000 7ff14713ab80 1 --2- 172.21.15.12:0/3714933080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556bc54629f0 0x556bc55e6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:15.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.313+0000 7ff14713ab80 1 --2- 172.21.15.12:0/3714933080 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556bc55e6aa0 0x556bc55e0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:15.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.313+0000 7ff14713ab80 1 -- 172.21.15.12:0/3714933080 >> 172.21.15.12:0/3714933080 conn(0x556bc54a6f00 msgr2=0x556bc55dbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:53:15.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.313+0000 7ff14713ab80 1 -- 172.21.15.12:0/3714933080 shutdown_connections 2024-05-17T22:53:15.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.313+0000 7ff14713ab80 1 -- 172.21.15.12:0/3714933080 wait complete. 2024-05-17T22:53:15.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.313+0000 7ff14713ab80 1 -- start start 2024-05-17T22:53:15.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.314+0000 7ff14713ab80 1 Processor -- start 2024-05-17T22:53:15.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.314+0000 7ff14713ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556bc54629f0 0x556bc55ea2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:15.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.314+0000 7ff14713ab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556bc55e1170 0x556bc55ea830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:15.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.314+0000 7ff14713ab80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc55e6aa0 0x556bc55feb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:15.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.314+0000 7ff14713ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x556bc5406150 con 0x556bc54629f0 2024-05-17T22:53:15.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.314+0000 7ff14713ab80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556bc549fe80 con 0x556bc55e6aa0 2024-05-17T22:53:15.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.314+0000 7ff14713ab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556bc54fefd0 con 0x556bc55e1170 2024-05-17T22:53:15.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.314+0000 7ff1364fb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556bc54629f0 0x556bc55ea2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:15.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.314+0000 7ff1364fb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556bc54629f0 0x556bc55ea2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45616/0 (socket says 172.21.15.12:45616) 2024-05-17T22:53:15.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.314+0000 7ff1364fb700 1 -- 172.21.15.12:0/3745082012 learned_addr learned my addr 172.21.15.12:0/3745082012 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:53:15.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.314+0000 7ff14713ab80 10 client.0 _ll_register_callbacks cb 0x556bc5499f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:53:15.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.314+0000 7ff136cfc700 1 --2- 172.21.15.12:0/3745082012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc55e6aa0 0x556bc55feb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:15.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.314+0000 7ff135cfa700 1 --2- 172.21.15.12:0/3745082012 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556bc55e1170 0x556bc55ea830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:15.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.315+0000 7ff136cfc700 1 -- 172.21.15.12:0/3745082012 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556bc55e1170 msgr2=0x556bc55ea830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:53:15.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.315+0000 7ff136cfc700 1 --2- 172.21.15.12:0/3745082012 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556bc55e1170 0x556bc55ea830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:15.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.315+0000 7ff136cfc700 1 -- 172.21.15.12:0/3745082012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556bc54629f0 msgr2=0x556bc55ea2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:53:15.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.315+0000 7ff136cfc700 1 --2- 172.21.15.12:0/3745082012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556bc54629f0 0x556bc55ea2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:15.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.315+0000 7ff136cfc700 1 -- 172.21.15.12:0/3745082012 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bc544f480 con 0x556bc55e6aa0 2024-05-17T22:53:15.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.315+0000 7ff136cfc700 1 --2- 172.21.15.12:0/3745082012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc55e6aa0 0x556bc55feb60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff128018780 tx=0x7ff128009850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f8447ee2d0df38d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:53:15.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.315+0000 7ff119ffb700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:53:15.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.315+0000 7ff119ffb700 1 -- 172.21.15.12:0/3745082012 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff128028060 con 0x556bc55e6aa0 2024-05-17T22:53:15.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.315+0000 7ff119ffb700 1 -- 172.21.15.12:0/3745082012 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7ff128016790 con 0x556bc55e6aa0 2024-05-17T22:53:15.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.315+0000 7ff14713ab80 1 -- 172.21.15.12:0/3745082012 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556bc55111e0 con 0x556bc55e6aa0 2024-05-17T22:53:15.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.316+0000 7ff119ffb700 1 -- 172.21.15.12:0/3745082012 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff12802f050 con 0x556bc55e6aa0 2024-05-17T22:53:15.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.316+0000 7ff14713ab80 10 client.26234 fetch_fsmap learned FSMap version 41 2024-05-17T22:53:15.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.316+0000 7ff14713ab80 1 -- 172.21.15.12:0/3745082012 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x556bc56003c0 con 0x556bc55e6aa0 2024-05-17T22:53:15.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.316+0000 7ff119ffb700 1 -- 172.21.15.12:0/3745082012 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556bc55111e0 con 0x556bc55e6aa0 2024-05-17T22:53:15.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.316+0000 7ff119ffb700 1 -- 172.21.15.12:0/3745082012 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7ff128023a90 con 0x556bc55e6aa0 2024-05-17T22:53:15.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.316+0000 7ff14713ab80 10 client.26234 fetch_fsmap finished waiting for FSMap version 41 2024-05-17T22:53:15.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.316+0000 7ff14713ab80 10 client.26234 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:53:15.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.316+0000 7ff14713ab80 20 client.26234 populate_metadata read hostname 'smithi012' 2024-05-17T22:53:15.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.316+0000 7ff14713ab80 1 --2- 172.21.15.12:0/3745082012 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x556bc5602a00 0x556bc5604e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:15.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.316+0000 7ff14713ab80 4 client.26234 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:53:15.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.316+0000 7ff14713ab80 1 -- 172.21.15.12:0/3745082012 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556bc550a9c0 con 0x556bc5602a00 2024-05-17T22:53:15.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.317+0000 7ff1364fb700 1 --2- 172.21.15.12:0/3745082012 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x556bc5602a00 0x556bc5604e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:15.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.318+0000 7ff1364fb700 1 --2- 172.21.15.12:0/3745082012 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x556bc5602a00 0x556bc5604e40 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca2fb290da2e377 server_cookie=50a20101b559683d in_seq=0 out_seq=0 2024-05-17T22:53:15.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.318+0000 7ff119ffb700 10 client.26234 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:53:15.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.319+0000 7ff119ffb700 1 -- 172.21.15.12:0/3745082012 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556bc550a9c0 con 0x556bc5602a00 2024-05-17T22:53:15.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.319+0000 7ff119ffb700 10 client.26234 handle_command_reply: tid=0 2024-05-17T22:53:15.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.319+0000 7ff14713ab80 1 client.26234 shutdown 2024-05-17T22:53:15.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.319+0000 7ff14713ab80 1 -- 172.21.15.12:0/3745082012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc55e6aa0 msgr2=0x556bc55feb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:15.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.319+0000 7ff14713ab80 1 --2- 172.21.15.12:0/3745082012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc55e6aa0 0x556bc55feb60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff128018780 tx=0x7ff128009850 comp rx=0 tx=0).stop 2024-05-17T22:53:15.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.319+0000 7ff14713ab80 1 -- 172.21.15.12:0/3745082012 shutdown_connections 2024-05-17T22:53:15.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.319+0000 7ff14713ab80 1 --2- 172.21.15.12:0/3745082012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc55e6aa0 0x556bc55feb60 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:15.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.319+0000 7ff14713ab80 1 --2- 172.21.15.12:0/3745082012 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x556bc5602a00 0x556bc5604e40 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:15.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.319+0000 7ff14713ab80 1 --2- 172.21.15.12:0/3745082012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556bc54629f0 0x556bc55ea2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:15.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.319+0000 7ff14713ab80 1 --2- 172.21.15.12:0/3745082012 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556bc55e1170 0x556bc55ea830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:15.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.319+0000 7ff14713ab80 1 -- 172.21.15.12:0/3745082012 >> 172.21.15.12:0/3745082012 conn(0x556bc54a6f00 msgr2=0x556bc55d5ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:53:15.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.319+0000 7ff119ffb700 0 client.26234 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:53:15.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.320+0000 7ff14713ab80 1 -- 172.21.15.12:0/3745082012 shutdown_connections 2024-05-17T22:53:15.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.320+0000 7ff14713ab80 1 -- 172.21.15.12:0/3745082012 wait complete. 2024-05-17T22:53:15.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.320+0000 7ff14713ab80 20 client.26234 trim_cache size 0 max 16384 2024-05-17T22:53:15.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.356+0000 7ff14713ab80 1 Processor -- start 2024-05-17T22:53:15.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.360+0000 7ff14713ab80 1 -- start start 2024-05-17T22:53:15.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.360+0000 7ff14713ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556bc54629f0 0x556bc55e0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:15.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.360+0000 7ff14713ab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556bc55e1170 0x556bc55d5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:15.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.360+0000 7ff14713ab80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc55e6aa0 0x556bc560a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:15.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.360+0000 7ff14713ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff1301b0960 con 0x556bc54629f0 2024-05-17T22:53:15.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.360+0000 7ff135cfa700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556bc55e1170 0x556bc55d5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:15.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.360+0000 7ff135cfa700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556bc55e1170 0x556bc55d5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:60776/0 (socket says 172.21.15.12:60776) 2024-05-17T22:53:15.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.360+0000 7ff135cfa700 1 -- 172.21.15.12:0/74831309 learned_addr learned my addr 172.21.15.12:0/74831309 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:53:15.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.360+0000 7ff14713ab80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff1301ae1f0 con 0x556bc55e6aa0 2024-05-17T22:53:15.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.361+0000 7ff14713ab80 1 -- 172.21.15.12:0/74831309 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff1301ac040 con 0x556bc55e1170 2024-05-17T22:53:15.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.361+0000 7ff136cfc700 1 --2- 172.21.15.12:0/74831309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc55e6aa0 0x556bc560a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:15.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.363+0000 7ff135cfa700 1 -- 172.21.15.12:0/74831309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc55e6aa0 msgr2=0x556bc560a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:53:15.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.363+0000 7ff135cfa700 1 --2- 172.21.15.12:0/74831309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc55e6aa0 0x556bc560a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:15.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.363+0000 7ff135cfa700 1 -- 172.21.15.12:0/74831309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556bc54629f0 msgr2=0x556bc55e0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:53:15.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.363+0000 7ff135cfa700 1 --2- 172.21.15.12:0/74831309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556bc54629f0 0x556bc55e0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:15.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.363+0000 7ff135cfa700 1 -- 172.21.15.12:0/74831309 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bc55dd710 con 0x556bc55e1170 2024-05-17T22:53:15.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.363+0000 7ff135cfa700 1 --2- 172.21.15.12:0/74831309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556bc55e1170 0x556bc55d5eb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff12c0040b0 tx=0x7ff12c009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=104568e5b8be152e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:53:15.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.369+0000 7ff11bfff700 1 -- 172.21.15.12:0/74831309 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c0be710 con 0x556bc55e1170 2024-05-17T22:53:15.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.372+0000 7ff11bfff700 1 -- 172.21.15.12:0/74831309 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff12c0e7480 con 0x556bc55e1170 2024-05-17T22:53:15.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.372+0000 7ff11bfff700 1 -- 172.21.15.12:0/74831309 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c0aa6a0 con 0x556bc55e1170 2024-05-17T22:53:15.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.374+0000 7ff14713ab80 1 -- 172.21.15.12:0/74831309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556bc55e1170 msgr2=0x556bc55d5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:15.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.374+0000 7ff14713ab80 1 --2- 172.21.15.12:0/74831309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556bc55e1170 0x556bc55d5eb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff12c0040b0 tx=0x7ff12c009ca0 comp rx=0 tx=0).stop 2024-05-17T22:53:15.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.378+0000 7ff14713ab80 1 -- 172.21.15.12:0/74831309 shutdown_connections 2024-05-17T22:53:15.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.378+0000 7ff14713ab80 1 --2- 172.21.15.12:0/74831309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc55e6aa0 0x556bc560a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:15.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.378+0000 7ff14713ab80 1 --2- 172.21.15.12:0/74831309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556bc54629f0 0x556bc55e0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:15.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.378+0000 7ff14713ab80 1 --2- 172.21.15.12:0/74831309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556bc55e1170 0x556bc55d5eb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:15.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.378+0000 7ff14713ab80 1 -- 172.21.15.12:0/74831309 >> 172.21.15.12:0/74831309 conn(0x556bc54a6f00 msgr2=0x556bc5604a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:53:15.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.386+0000 7ff14713ab80 1 -- 172.21.15.12:0/74831309 shutdown_connections 2024-05-17T22:53:15.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.387+0000 7ff14713ab80 1 -- 172.21.15.12:0/74831309 wait complete. 2024-05-17T22:53:15.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.387+0000 7ff14713ab80 1 -- start start 2024-05-17T22:53:15.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.387+0000 7ff14713ab80 1 Processor -- start 2024-05-17T22:53:15.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.388+0000 7ff14713ab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556bc54629f0 0x556bc55fab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:15.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.388+0000 7ff14713ab80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc55e1170 0x556bc55fb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:15.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.389+0000 7ff14713ab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556bc55e6aa0 0x556bc55fd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:15.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.389+0000 7ff14713ab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff1301ac6b0 con 0x556bc54629f0 2024-05-17T22:53:15.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.389+0000 7ff14713ab80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff1301adb20 con 0x556bc55e1170 2024-05-17T22:53:15.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.389+0000 7ff14713ab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff1301acd80 con 0x556bc55e6aa0 2024-05-17T22:53:15.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.389+0000 7ff14713ab80 10 client.0 _ll_register_callbacks cb 0x556bc5499f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:53:15.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.391+0000 7ff135cfa700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc55e1170 0x556bc55fb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:15.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.391+0000 7ff136cfc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556bc55e6aa0 0x556bc55fd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:15.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.393+0000 7ff1364fb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556bc54629f0 0x556bc55fab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:15.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.393+0000 7ff1364fb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556bc54629f0 0x556bc55fab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45630/0 (socket says 172.21.15.12:45630) 2024-05-17T22:53:15.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.393+0000 7ff1364fb700 1 -- 172.21.15.12:0/2089418398 learned_addr learned my addr 172.21.15.12:0/2089418398 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:53:15.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.395+0000 7ff1364fb700 1 -- 172.21.15.12:0/2089418398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556bc55e6aa0 msgr2=0x556bc55fd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:53:15.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.395+0000 7ff1364fb700 1 --2- 172.21.15.12:0/2089418398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556bc55e6aa0 0x556bc55fd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:15.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.395+0000 7ff1364fb700 1 -- 172.21.15.12:0/2089418398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc55e1170 msgr2=0x556bc55fb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:53:15.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.395+0000 7ff1364fb700 1 --2- 172.21.15.12:0/2089418398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc55e1170 0x556bc55fb080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:15.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.395+0000 7ff1364fb700 1 -- 172.21.15.12:0/2089418398 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bc55fdf80 con 0x556bc54629f0 2024-05-17T22:53:15.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.396+0000 7ff1364fb700 1 --2- 172.21.15.12:0/2089418398 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556bc54629f0 0x556bc55fab40 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff120005950 tx=0x7ff12000bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f2e39980adb21d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:53:15.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.397+0000 7ff119ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:53:15.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.397+0000 7ff119ffb700 1 -- 172.21.15.12:0/2089418398 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff120004830 con 0x556bc54629f0 2024-05-17T22:53:15.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.397+0000 7ff14713ab80 1 -- 172.21.15.12:0/2089418398 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff12c0e7480 con 0x556bc54629f0 2024-05-17T22:53:15.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.397+0000 7ff119ffb700 1 -- 172.21.15.12:0/2089418398 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7ff1200049d0 con 0x556bc54629f0 2024-05-17T22:53:15.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.397+0000 7ff119ffb700 1 -- 172.21.15.12:0/2089418398 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff120029050 con 0x556bc54629f0 2024-05-17T22:53:15.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.399+0000 7ff14713ab80 10 client.16533 fetch_fsmap learned FSMap version 41 2024-05-17T22:53:15.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.399+0000 7ff14713ab80 1 -- 172.21.15.12:0/2089418398 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x556bc566cbb0 con 0x556bc54629f0 2024-05-17T22:53:15.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.401+0000 7ff119ffb700 1 -- 172.21.15.12:0/2089418398 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff12c0e7480 con 0x556bc54629f0 2024-05-17T22:53:15.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.401+0000 7ff119ffb700 1 -- 172.21.15.12:0/2089418398 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7ff120011440 con 0x556bc54629f0 2024-05-17T22:53:15.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.403+0000 7ff14713ab80 10 client.16533 fetch_fsmap finished waiting for FSMap version 41 2024-05-17T22:53:15.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.403+0000 7ff14713ab80 10 client.16533 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:53:15.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.403+0000 7ff14713ab80 20 client.16533 populate_metadata read hostname 'smithi012' 2024-05-17T22:53:15.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.403+0000 7ff14713ab80 1 --2- 172.21.15.12:0/2089418398 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x556bc566d380 0x556bc566f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:15.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.403+0000 7ff14713ab80 4 client.16533 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:53:15.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.403+0000 7ff14713ab80 1 -- 172.21.15.12:0/2089418398 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556bc53db360 con 0x556bc566d380 2024-05-17T22:53:15.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.409+0000 7ff135cfa700 1 --2- 172.21.15.12:0/2089418398 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x556bc566d380 0x556bc566f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:15.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.423+0000 7ff135cfa700 1 --2- 172.21.15.12:0/2089418398 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x556bc566d380 0x556bc566f7a0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b605dad21cad7e9a server_cookie=c31d6449f0d3f25f in_seq=0 out_seq=0 2024-05-17T22:53:15.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.424+0000 7ff119ffb700 10 client.16533 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:53:15.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.428+0000 7ff119ffb700 1 -- 172.21.15.12:0/2089418398 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556bc53db360 con 0x556bc566d380 2024-05-17T22:53:15.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.428+0000 7ff119ffb700 10 client.16533 handle_command_reply: tid=0 2024-05-17T22:53:15.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.428+0000 7ff14713ab80 1 client.16533 shutdown 2024-05-17T22:53:15.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.432+0000 7ff14713ab80 1 -- 172.21.15.12:0/2089418398 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556bc54629f0 msgr2=0x556bc55fab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:15.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.432+0000 7ff14713ab80 1 --2- 172.21.15.12:0/2089418398 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556bc54629f0 0x556bc55fab40 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff120005950 tx=0x7ff12000bc50 comp rx=0 tx=0).stop 2024-05-17T22:53:15.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.433+0000 7ff14713ab80 1 -- 172.21.15.12:0/2089418398 shutdown_connections 2024-05-17T22:53:15.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.433+0000 7ff14713ab80 1 --2- 172.21.15.12:0/2089418398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556bc55e6aa0 0x556bc55fd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:15.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.433+0000 7ff14713ab80 1 --2- 172.21.15.12:0/2089418398 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x556bc566d380 0x556bc566f7a0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:15.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.433+0000 7ff14713ab80 1 --2- 172.21.15.12:0/2089418398 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556bc54629f0 0x556bc55fab40 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:15.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.433+0000 7ff14713ab80 1 --2- 172.21.15.12:0/2089418398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc55e1170 0x556bc55fb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:15.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.433+0000 7ff14713ab80 1 -- 172.21.15.12:0/2089418398 >> 172.21.15.12:0/2089418398 conn(0x556bc54a6f00 msgr2=0x556bc55e3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:53:15.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.433+0000 7ff119ffb700 0 client.16533 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:53:15.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.434+0000 7ff14713ab80 1 -- 172.21.15.12:0/2089418398 shutdown_connections 2024-05-17T22:53:15.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.437+0000 7ff14713ab80 1 -- 172.21.15.12:0/2089418398 wait complete. 2024-05-17T22:53:15.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.437+0000 7ff14713ab80 20 client.16533 trim_cache size 0 max 16384 2024-05-17T22:53:15.439 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:53:15.440 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-17T22:53:15.440 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-17T22:53:15.440 INFO:teuthology.orchestra.run.smithi012.stdout: "1b32803c-b836-419f-a084-9cb1afff5492": { 2024-05-17T22:53:15.440 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-17T22:53:15.440 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "1b32803c-b836-419f-a084-9cb1afff5492", 2024-05-17T22:53:15.440 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-17T22:53:15.440 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:53:15.440 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:53:15.440 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:53:15.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.439+0000 7ff13875f700 1 -- 172.21.15.12:0/920288645 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff11c0b29d0 msgr2=0x7ff11c0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:15.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.439+0000 7ff13875f700 1 --2- 172.21.15.12:0/920288645 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff11c0b29d0 0x7ff11c0b4ee0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x556bc55dcae0 tx=0x7ff12800e040 comp rx=0 tx=0).stop 2024-05-17T22:53:15.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.439+0000 7ff13875f700 1 -- 172.21.15.12:0/920288645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff13010b3b0 msgr2=0x7ff1301a2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:15.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.439+0000 7ff13875f700 1 --2- 172.21.15.12:0/920288645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff13010b3b0 0x7ff1301a2e60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff12c000c00 tx=0x7ff12c0057c0 comp rx=0 tx=0).stop 2024-05-17T22:53:15.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.439+0000 7ff13875f700 1 -- 172.21.15.12:0/920288645 shutdown_connections 2024-05-17T22:53:15.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.439+0000 7ff13875f700 1 --2- 172.21.15.12:0/920288645 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff11c0b29d0 0x7ff11c0b4ee0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:15.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.439+0000 7ff13875f700 1 --2- 172.21.15.12:0/920288645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff13019ca80 0x7ff13019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:15.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.439+0000 7ff13875f700 1 --2- 172.21.15.12:0/920288645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff130106130 0x7ff1301a2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:15.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.439+0000 7ff13875f700 1 --2- 172.21.15.12:0/920288645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff13010b3b0 0x7ff1301a2e60 secure :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff12c000c00 tx=0x7ff12c0057c0 comp rx=0 tx=0).stop 2024-05-17T22:53:15.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.439+0000 7ff13875f700 1 -- 172.21.15.12:0/920288645 >> 172.21.15.12:0/920288645 conn(0x7ff130101bb0 msgr2=0x7ff130103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:53:15.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.442+0000 7ff13875f700 1 -- 172.21.15.12:0/920288645 shutdown_connections 2024-05-17T22:53:15.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:15.445+0000 7ff13875f700 1 -- 172.21.15.12:0/920288645 wait complete. 2024-05-17T22:53:16.175 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:16 smithi012 ceph-mon[31632]: pgmap v859: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.5 MiB/s wr, 475 op/s 2024-05-17T22:53:16.233 INFO:tasks.cephfs.filesystem:scrub status for tag:1b32803c-b836-419f-a084-9cb1afff5492 - {'path': '/', 'tag': '1b32803c-b836-419f-a084-9cb1afff5492', 'options': 'recursive,force'} 2024-05-17T22:53:16.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:16 smithi096 ceph-mon[36065]: pgmap v859: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.5 MiB/s wr, 475 op/s 2024-05-17T22:53:16.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 835.0 s, 439.6 tps, lat 231.176 ms stddev 260.676 2024-05-17T22:53:16.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:16 smithi151 ceph-mon[36101]: pgmap v859: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.5 MiB/s wr, 475 op/s 2024-05-17T22:53:18.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:18 smithi096 ceph-mon[36065]: pgmap v860: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 3.6 MiB/s wr, 380 op/s 2024-05-17T22:53:18.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:18 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:53:18.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:18 smithi012 ceph-mon[31632]: pgmap v860: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 3.6 MiB/s wr, 380 op/s 2024-05-17T22:53:18.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:18 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:53:18.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:18 smithi151 ceph-mon[36101]: pgmap v860: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 3.6 MiB/s wr, 380 op/s 2024-05-17T22:53:18.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:18 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:53:19.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:19 smithi096 ceph-mon[36065]: scrub summary: idle 2024-05-17T22:53:19.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:19 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:53:19.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:19 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:53:19.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:19 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:53:19.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:19 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:53:19.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:19 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:53:19.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:19 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:53:19.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:19 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:53:19.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:19 smithi012 ceph-mon[31632]: scrub summary: idle 2024-05-17T22:53:19.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:19 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:53:19.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:19 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:53:19.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:19 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:53:19.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:19 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:53:19.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:19 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:53:19.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:19 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:53:19.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:19 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:53:19.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:19 smithi151 ceph-mon[36101]: scrub summary: idle 2024-05-17T22:53:19.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:19 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:53:19.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:19 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:53:19.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:19 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:53:19.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:19 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:53:19.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:19 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:53:19.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:19 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:53:19.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:19 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:53:20.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:20 smithi096 ceph-mon[36065]: scrub summary: idle 2024-05-17T22:53:20.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:20 smithi096 ceph-mon[36065]: scrub summary: idle 2024-05-17T22:53:20.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:20 smithi096 ceph-mon[36065]: Adjusting osd_memory_target on smithi012 to 177.2M 2024-05-17T22:53:20.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:20 smithi096 ceph-mon[36065]: Unable to set osd_memory_target on smithi012 to 185879142: error parsing value: Value '185879142' is below minimum 939524096 2024-05-17T22:53:20.299 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:20 smithi096 ceph-mon[36065]: pgmap v861: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.8 MiB/s wr, 403 op/s 2024-05-17T22:53:20.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:20 smithi012 ceph-mon[31632]: scrub summary: idle 2024-05-17T22:53:20.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:20 smithi012 ceph-mon[31632]: scrub summary: idle 2024-05-17T22:53:20.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:20 smithi012 ceph-mon[31632]: Adjusting osd_memory_target on smithi012 to 177.2M 2024-05-17T22:53:20.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:20 smithi012 ceph-mon[31632]: Unable to set osd_memory_target on smithi012 to 185879142: error parsing value: Value '185879142' is below minimum 939524096 2024-05-17T22:53:20.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:20 smithi012 ceph-mon[31632]: pgmap v861: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.8 MiB/s wr, 403 op/s 2024-05-17T22:53:20.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:20 smithi151 ceph-mon[36101]: scrub summary: idle 2024-05-17T22:53:20.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:20 smithi151 ceph-mon[36101]: scrub summary: idle 2024-05-17T22:53:20.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:20 smithi151 ceph-mon[36101]: Adjusting osd_memory_target on smithi012 to 177.2M 2024-05-17T22:53:20.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:20 smithi151 ceph-mon[36101]: Unable to set osd_memory_target on smithi012 to 185879142: error parsing value: Value '185879142' is below minimum 939524096 2024-05-17T22:53:20.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:20 smithi151 ceph-mon[36101]: pgmap v861: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.8 MiB/s wr, 403 op/s 2024-05-17T22:53:21.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 840.0 s, 449.2 tps, lat 220.276 ms stddev 238.377 2024-05-17T22:53:22.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:22 smithi096 ceph-mon[36065]: pgmap v862: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.5 MiB/s wr, 485 op/s 2024-05-17T22:53:22.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:22 smithi012 ceph-mon[31632]: pgmap v862: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.5 MiB/s wr, 485 op/s 2024-05-17T22:53:22.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:22 smithi151 ceph-mon[36101]: pgmap v862: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.5 MiB/s wr, 485 op/s 2024-05-17T22:53:24.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:24 smithi096 ceph-mon[36065]: pgmap v863: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.0 MiB/s wr, 409 op/s 2024-05-17T22:53:24.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:24 smithi012 ceph-mon[31632]: pgmap v863: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.0 MiB/s wr, 409 op/s 2024-05-17T22:53:24.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:24 smithi151 ceph-mon[36101]: pgmap v863: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.0 MiB/s wr, 409 op/s 2024-05-17T22:53:26.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:26 smithi096 ceph-mon[36065]: pgmap v864: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.6 MiB/s wr, 493 op/s 2024-05-17T22:53:26.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:26 smithi012 ceph-mon[31632]: pgmap v864: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.6 MiB/s wr, 493 op/s 2024-05-17T22:53:26.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 845.0 s, 458.2 tps, lat 221.773 ms stddev 265.847 2024-05-17T22:53:26.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:26 smithi151 ceph-mon[36101]: pgmap v864: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.6 MiB/s wr, 493 op/s 2024-05-17T22:53:28.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:28 smithi012 ceph-mon[31632]: pgmap v865: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.6 MiB/s wr, 396 op/s 2024-05-17T22:53:28.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:28 smithi151 ceph-mon[36101]: pgmap v865: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.6 MiB/s wr, 396 op/s 2024-05-17T22:53:28.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:28 smithi096 ceph-mon[36065]: pgmap v865: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.6 MiB/s wr, 396 op/s 2024-05-17T22:53:30.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:30 smithi012 ceph-mon[31632]: pgmap v866: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.8 MiB/s wr, 411 op/s 2024-05-17T22:53:30.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:30 smithi151 ceph-mon[36101]: pgmap v866: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.8 MiB/s wr, 411 op/s 2024-05-17T22:53:30.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:30 smithi096 ceph-mon[36065]: pgmap v866: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.8 MiB/s wr, 411 op/s 2024-05-17T22:53:31.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 850.0 s, 540.2 tps, lat 188.830 ms stddev 251.910 2024-05-17T22:53:32.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:32 smithi012 ceph-mon[31632]: pgmap v867: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.0 MiB/s wr, 526 op/s 2024-05-17T22:53:32.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:32 smithi151 ceph-mon[36101]: pgmap v867: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.0 MiB/s wr, 526 op/s 2024-05-17T22:53:32.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:32 smithi096 ceph-mon[36065]: pgmap v867: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.0 MiB/s wr, 526 op/s 2024-05-17T22:53:34.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:34 smithi012 ceph-mon[31632]: pgmap v868: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.9 MiB/s wr, 428 op/s 2024-05-17T22:53:34.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:34 smithi151 ceph-mon[36101]: pgmap v868: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.9 MiB/s wr, 428 op/s 2024-05-17T22:53:34.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:34 smithi096 ceph-mon[36065]: pgmap v868: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.9 MiB/s wr, 428 op/s 2024-05-17T22:53:36.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:36 smithi012 ceph-mon[31632]: pgmap v869: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 4.5 MiB/s wr, 512 op/s 2024-05-17T22:53:36.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 855.0 s, 421.0 tps, lat 231.822 ms stddev 261.082 2024-05-17T22:53:36.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:36 smithi151 ceph-mon[36101]: pgmap v869: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 4.5 MiB/s wr, 512 op/s 2024-05-17T22:53:36.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:36 smithi096 ceph-mon[36065]: pgmap v869: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 4.5 MiB/s wr, 512 op/s 2024-05-17T22:53:38.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:38 smithi012 ceph-mon[31632]: pgmap v870: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.9 MiB/s wr, 427 op/s 2024-05-17T22:53:38.350 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:38 smithi096 ceph-mon[36065]: pgmap v870: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.9 MiB/s wr, 427 op/s 2024-05-17T22:53:38.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:38 smithi151 ceph-mon[36101]: pgmap v870: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.9 MiB/s wr, 427 op/s 2024-05-17T22:53:40.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:40 smithi012 ceph-mon[31632]: pgmap v871: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.9 MiB/s wr, 428 op/s 2024-05-17T22:53:40.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:40 smithi151 ceph-mon[36101]: pgmap v871: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.9 MiB/s wr, 428 op/s 2024-05-17T22:53:40.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:40 smithi096 ceph-mon[36065]: pgmap v871: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.9 MiB/s wr, 428 op/s 2024-05-17T22:53:41.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 860.0 s, 524.8 tps, lat 188.395 ms stddev 208.643 2024-05-17T22:53:42.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:42 smithi012 ceph-mon[31632]: pgmap v872: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.0 MiB/s wr, 546 op/s 2024-05-17T22:53:42.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:42 smithi151 ceph-mon[36101]: pgmap v872: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.0 MiB/s wr, 546 op/s 2024-05-17T22:53:42.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:42 smithi096 ceph-mon[36065]: pgmap v872: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.0 MiB/s wr, 546 op/s 2024-05-17T22:53:44.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:44 smithi151 ceph-mon[36101]: pgmap v873: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.6 MiB/s wr, 407 op/s 2024-05-17T22:53:44.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:44 smithi096 ceph-mon[36065]: pgmap v873: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.6 MiB/s wr, 407 op/s 2024-05-17T22:53:44.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:44 smithi012 ceph-mon[31632]: pgmap v873: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.6 MiB/s wr, 407 op/s 2024-05-17T22:53:46.234 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-17T22:53:46.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 865.0 s, 487.6 tps, lat 199.327 ms stddev 230.067 2024-05-17T22:53:46.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:46 smithi151 ceph-mon[36101]: pgmap v874: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.7 MiB/s wr, 515 op/s 2024-05-17T22:53:46.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:46 smithi096 ceph-mon[36065]: pgmap v874: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.7 MiB/s wr, 515 op/s 2024-05-17T22:53:46.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:46 smithi012 ceph-mon[31632]: pgmap v874: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.7 MiB/s wr, 515 op/s 2024-05-17T22:53:46.663 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:53:48.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:48 smithi151 ceph-mon[36101]: pgmap v875: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.1 MiB/s wr, 445 op/s 2024-05-17T22:53:48.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:48 smithi096 ceph-mon[36065]: pgmap v875: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.1 MiB/s wr, 445 op/s 2024-05-17T22:53:48.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:48 smithi012 ceph-mon[31632]: pgmap v875: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.1 MiB/s wr, 445 op/s 2024-05-17T22:53:49.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.960+0000 7f399d119700 1 -- 172.21.15.12:0/2266400002 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f399810aa00 msgr2=0x7f399810ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:49.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.960+0000 7f399d119700 1 --2- 172.21.15.12:0/2266400002 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f399810aa00 0x7f399810ae10 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f39880098f0 tx=0x7f3988009c00 comp rx=0 tx=0).stop 2024-05-17T22:53:49.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.962+0000 7f399d119700 1 -- 172.21.15.12:0/2266400002 shutdown_connections 2024-05-17T22:53:49.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.962+0000 7f399d119700 1 --2- 172.21.15.12:0/2266400002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3998116a60 0x7f3998118f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:49.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.962+0000 7f399d119700 1 --2- 172.21.15.12:0/2266400002 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f399810aa00 0x7f399810ae10 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:49.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.962+0000 7f399d119700 1 --2- 172.21.15.12:0/2266400002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f399810b350 0x7f3998116440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:49.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.962+0000 7f399d119700 1 -- 172.21.15.12:0/2266400002 >> 172.21.15.12:0/2266400002 conn(0x7f399806efc0 msgr2=0x7f399806f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:53:49.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.963+0000 7f399d119700 1 -- 172.21.15.12:0/2266400002 shutdown_connections 2024-05-17T22:53:49.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.963+0000 7f399d119700 1 -- 172.21.15.12:0/2266400002 wait complete. 2024-05-17T22:53:49.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.964+0000 7f399d119700 1 Processor -- start 2024-05-17T22:53:49.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.967+0000 7f399d119700 1 -- start start 2024-05-17T22:53:49.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.967+0000 7f399d119700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f399810aa00 0x7f3998120e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:49.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.968+0000 7f399d119700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f399810b350 0x7f3998121370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:49.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.968+0000 7f399d119700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3998116a60 0x7f399811bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:49.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.968+0000 7f399d119700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f399811a440 con 0x7f3998116a60 2024-05-17T22:53:49.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.968+0000 7f399d119700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f399811a2c0 con 0x7f399810b350 2024-05-17T22:53:49.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.968+0000 7f399d119700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f399811a5c0 con 0x7f399810aa00 2024-05-17T22:53:49.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.968+0000 7f3996d9d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f399810aa00 0x7f3998120e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:49.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.968+0000 7f3996d9d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f399810aa00 0x7f3998120e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:49038/0 (socket says 172.21.15.12:49038) 2024-05-17T22:53:49.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.968+0000 7f3996d9d700 1 -- 172.21.15.12:0/334061363 learned_addr learned my addr 172.21.15.12:0/334061363 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:53:49.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.968+0000 7f399759e700 1 --2- 172.21.15.12:0/334061363 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3998116a60 0x7f399811bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:49.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.969+0000 7f399659c700 1 --2- 172.21.15.12:0/334061363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f399810b350 0x7f3998121370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:49.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.969+0000 7f3996d9d700 1 -- 172.21.15.12:0/334061363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f399810b350 msgr2=0x7f3998121370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:49.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.969+0000 7f3996d9d700 1 --2- 172.21.15.12:0/334061363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f399810b350 0x7f3998121370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:49.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.970+0000 7f3996d9d700 1 -- 172.21.15.12:0/334061363 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3998116a60 msgr2=0x7f399811bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:49.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.970+0000 7f3996d9d700 1 --2- 172.21.15.12:0/334061363 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3998116a60 0x7f399811bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:49.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.970+0000 7f3996d9d700 1 -- 172.21.15.12:0/334061363 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f399811c800 con 0x7f399810aa00 2024-05-17T22:53:49.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.970+0000 7f399759e700 1 --2- 172.21.15.12:0/334061363 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3998116a60 0x7f399811bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:53:49.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.977+0000 7f3996d9d700 1 --2- 172.21.15.12:0/334061363 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f399810aa00 0x7f3998120e10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3988009230 tx=0x7f398800fa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:53:49.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.979+0000 7f397ffff700 1 -- 172.21.15.12:0/334061363 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3988005140 con 0x7f399810aa00 2024-05-17T22:53:49.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.979+0000 7f397ffff700 1 -- 172.21.15.12:0/334061363 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f398800b560 con 0x7f399810aa00 2024-05-17T22:53:49.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.979+0000 7f397ffff700 1 -- 172.21.15.12:0/334061363 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3988003ae0 con 0x7f399810aa00 2024-05-17T22:53:49.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.979+0000 7f399d119700 1 -- 172.21.15.12:0/334061363 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f399811ca90 con 0x7f399810aa00 2024-05-17T22:53:49.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.980+0000 7f399d119700 1 -- 172.21.15.12:0/334061363 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3998124b20 con 0x7f399810aa00 2024-05-17T22:53:49.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.986+0000 7f397ffff700 1 -- 172.21.15.12:0/334061363 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3988003c80 con 0x7f399810aa00 2024-05-17T22:53:49.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.987+0000 7f397ffff700 1 --2- 172.21.15.12:0/334061363 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f39800b29d0 0x7f39800b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:49.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.987+0000 7f399659c700 1 --2- 172.21.15.12:0/334061363 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f39800b29d0 0x7f39800b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:49.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.987+0000 7f397ffff700 1 -- 172.21.15.12:0/334061363 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f39880e3480 con 0x7f399810aa00 2024-05-17T22:53:49.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.988+0000 7f399659c700 1 --2- 172.21.15.12:0/334061363 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f39800b29d0 0x7f39800b4ee0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3998074f10 tx=0x7f398c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:53:49.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.989+0000 7f39abaf4b80 1 Processor -- start 2024-05-17T22:53:49.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.990+0000 7f39abaf4b80 1 -- start start 2024-05-17T22:53:49.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.990+0000 7f39abaf4b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592c7d5d9f0 0x5592c7ee0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:49.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.990+0000 7f39abaf4b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592c7ee1400 0x5592c7edb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:49.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.990+0000 7f3996d9d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592c7d5d9f0 0x5592c7ee0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:49.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.991+0000 7f3996d9d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592c7d5d9f0 0x5592c7ee0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:49042/0 (socket says 172.21.15.12:49042) 2024-05-17T22:53:49.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.991+0000 7f399659c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592c7ee1400 0x5592c7edb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:49.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.991+0000 7f399659c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592c7ee1400 0x5592c7edb5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:50248/0 (socket says 172.21.15.12:50248) 2024-05-17T22:53:49.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.991+0000 7f39abaf4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5592c7edbb00 0x5592c7edbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:49.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.991+0000 7f39abaf4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5592c7e0a0d0 con 0x5592c7edbb00 2024-05-17T22:53:49.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.991+0000 7f39abaf4b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5592c7df9c70 con 0x5592c7ee1400 2024-05-17T22:53:49.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.991+0000 7f3996d9d700 1 -- 172.21.15.12:0/3568052088 learned_addr learned my addr 172.21.15.12:0/3568052088 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:53:49.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.991+0000 7f39abaf4b80 1 -- 172.21.15.12:0/3568052088 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5592c7dfdcb0 con 0x5592c7d5d9f0 2024-05-17T22:53:49.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.991+0000 7f399759e700 1 --2- 172.21.15.12:0/3568052088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5592c7edbb00 0x5592c7edbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:49.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.991+0000 7f3996d9d700 1 -- 172.21.15.12:0/3568052088 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592c7ee1400 msgr2=0x5592c7edb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:53:49.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.991+0000 7f3996d9d700 1 --2- 172.21.15.12:0/3568052088 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592c7ee1400 0x5592c7edb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:49.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.991+0000 7f3996d9d700 1 -- 172.21.15.12:0/3568052088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5592c7edbb00 msgr2=0x5592c7edbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:53:49.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.991+0000 7f3996d9d700 1 --2- 172.21.15.12:0/3568052088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5592c7edbb00 0x5592c7edbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:49.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.991+0000 7f3996d9d700 1 -- 172.21.15.12:0/3568052088 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592c7d935c0 con 0x5592c7d5d9f0 2024-05-17T22:53:49.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.992+0000 7f3996d9d700 1 --2- 172.21.15.12:0/3568052088 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592c7d5d9f0 0x5592c7ee0ec0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f39880a6820 tx=0x7f3988009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70aff2c69c197cb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:53:49.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.993+0000 7f397d7fa700 1 -- 172.21.15.12:0/3568052088 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39880b10b0 con 0x5592c7d5d9f0 2024-05-17T22:53:49.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.993+0000 7f397d7fa700 1 -- 172.21.15.12:0/3568052088 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39880aa2b0 con 0x5592c7d5d9f0 2024-05-17T22:53:49.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.993+0000 7f397d7fa700 1 -- 172.21.15.12:0/3568052088 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39880aaa00 con 0x5592c7d5d9f0 2024-05-17T22:53:49.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.994+0000 7f39abaf4b80 1 -- 172.21.15.12:0/3568052088 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592c7d5d9f0 msgr2=0x5592c7ee0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:49.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.994+0000 7f39abaf4b80 1 --2- 172.21.15.12:0/3568052088 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592c7d5d9f0 0x5592c7ee0ec0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f39880a6820 tx=0x7f3988009c00 comp rx=0 tx=0).stop 2024-05-17T22:53:49.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.994+0000 7f39abaf4b80 1 -- 172.21.15.12:0/3568052088 shutdown_connections 2024-05-17T22:53:49.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.994+0000 7f39abaf4b80 1 --2- 172.21.15.12:0/3568052088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5592c7edbb00 0x5592c7edbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:49.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.994+0000 7f39abaf4b80 1 --2- 172.21.15.12:0/3568052088 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592c7d5d9f0 0x5592c7ee0ec0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:49.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.994+0000 7f39abaf4b80 1 --2- 172.21.15.12:0/3568052088 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592c7ee1400 0x5592c7edb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:49.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.994+0000 7f39abaf4b80 1 -- 172.21.15.12:0/3568052088 >> 172.21.15.12:0/3568052088 conn(0x5592c7dbd7d0 msgr2=0x5592c7ed61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:53:49.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.994+0000 7f39abaf4b80 1 -- 172.21.15.12:0/3568052088 shutdown_connections 2024-05-17T22:53:49.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.994+0000 7f39abaf4b80 1 -- 172.21.15.12:0/3568052088 wait complete. 2024-05-17T22:53:49.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.995+0000 7f39abaf4b80 1 -- start start 2024-05-17T22:53:49.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.995+0000 7f39abaf4b80 1 Processor -- start 2024-05-17T22:53:49.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.996+0000 7f39abaf4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5592c7d5d9f0 0x5592c7ee4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:49.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.996+0000 7f39abaf4b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592c7edbb00 0x5592c7ee5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:49.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.996+0000 7f39abaf4b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592c7ee1400 0x5592c7ee5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:49.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.996+0000 7f39abaf4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5592c7d01150 con 0x5592c7d5d9f0 2024-05-17T22:53:50.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.996+0000 7f399759e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592c7ee1400 0x5592c7ee5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:50.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.996+0000 7f399759e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592c7ee1400 0x5592c7ee5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:49052/0 (socket says 172.21.15.12:49052) 2024-05-17T22:53:50.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.996+0000 7f399759e700 1 -- 172.21.15.12:0/108856830 learned_addr learned my addr 172.21.15.12:0/108856830 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:53:50.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.996+0000 7f39abaf4b80 1 -- 172.21.15.12:0/108856830 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5592c7d9b940 con 0x5592c7edbb00 2024-05-17T22:53:50.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.996+0000 7f39abaf4b80 1 -- 172.21.15.12:0/108856830 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5592c7ee7c20 con 0x5592c7ee1400 2024-05-17T22:53:50.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.997+0000 7f3996d9d700 1 --2- 172.21.15.12:0/108856830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5592c7d5d9f0 0x5592c7ee4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:50.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.997+0000 7f399759e700 1 -- 172.21.15.12:0/108856830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592c7edbb00 msgr2=0x5592c7ee5000 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T22:53:50.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.997+0000 7f399759e700 1 --2- 172.21.15.12:0/108856830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592c7edbb00 0x5592c7ee5000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:50.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.997+0000 7f399759e700 1 -- 172.21.15.12:0/108856830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5592c7d5d9f0 msgr2=0x5592c7ee4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:53:50.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.997+0000 7f399759e700 1 --2- 172.21.15.12:0/108856830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5592c7d5d9f0 0x5592c7ee4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:50.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.997+0000 7f399759e700 1 -- 172.21.15.12:0/108856830 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592c7d4a480 con 0x5592c7ee1400 2024-05-17T22:53:50.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.997+0000 7f39abaf4b80 10 client.0 _ll_register_callbacks cb 0x5592c7d90b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:53:50.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.998+0000 7f39abaf4b80 1 -- 172.21.15.12:0/108856830 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39880aa2b0 con 0x5592c7ee1400 2024-05-17T22:53:50.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.998+0000 7f399759e700 1 --2- 172.21.15.12:0/108856830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592c7ee1400 0x5592c7ee5560 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f39900042c0 tx=0x7f39900043a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3eeb93d2bce8863 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:53:50.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:49.999+0000 7f397aff5700 10 client.26173 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:53:50.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.001+0000 7f397aff5700 1 -- 172.21.15.12:0/108856830 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399001a030 con 0x5592c7ee1400 2024-05-17T22:53:50.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.001+0000 7f397aff5700 1 -- 172.21.15.12:0/108856830 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f3990004bb0 con 0x5592c7ee1400 2024-05-17T22:53:50.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.001+0000 7f397aff5700 1 -- 172.21.15.12:0/108856830 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f3990004df0 con 0x5592c7ee1400 2024-05-17T22:53:50.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.001+0000 7f39abaf4b80 10 client.26173 fetch_fsmap learned FSMap version 42 2024-05-17T22:53:50.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.001+0000 7f39abaf4b80 1 -- 172.21.15.12:0/108856830 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({fsmap=42}) -- 0x5592c7ef24f0 con 0x5592c7ee1400 2024-05-17T22:53:50.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.001+0000 7f397aff5700 1 -- 172.21.15.12:0/108856830 <== mon.1 v2:172.21.15.151:3300/0 4 ==== fsmap(e 42) ==== 8350+0+0 (secure 0 0 0) 0x7f3990027020 con 0x5592c7ee1400 2024-05-17T22:53:50.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.002+0000 7f39abaf4b80 10 client.26173 fetch_fsmap finished waiting for FSMap version 42 2024-05-17T22:53:50.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.002+0000 7f39abaf4b80 10 client.26173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:53:50.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.002+0000 7f39abaf4b80 20 client.26173 populate_metadata read hostname 'smithi012' 2024-05-17T22:53:50.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.002+0000 7f39abaf4b80 1 --2- 172.21.15.12:0/108856830 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5592c7ef4a70 0x5592c7ef6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:50.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.002+0000 7f39abaf4b80 4 client.26173 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:53:50.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.002+0000 7f39abaf4b80 1 -- 172.21.15.12:0/108856830 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592c7e0c020 con 0x5592c7ef4a70 2024-05-17T22:53:50.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.002+0000 7f3996d9d700 1 --2- 172.21.15.12:0/108856830 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5592c7ef4a70 0x5592c7ef6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:50.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.003+0000 7f3996d9d700 1 --2- 172.21.15.12:0/108856830 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5592c7ef4a70 0x5592c7ef6f60 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f18822f47b4b440 server_cookie=dada4b3f779eafd5 in_seq=0 out_seq=0 2024-05-17T22:53:50.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.003+0000 7f397aff5700 10 client.26173 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:53:50.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.003+0000 7f397aff5700 1 -- 172.21.15.12:0/108856830 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5592c7e0c020 con 0x5592c7ef4a70 2024-05-17T22:53:50.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.003+0000 7f397aff5700 10 client.26173 handle_command_reply: tid=0 2024-05-17T22:53:50.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.003+0000 7f39abaf4b80 1 client.26173 shutdown 2024-05-17T22:53:50.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.004+0000 7f39abaf4b80 1 -- 172.21.15.12:0/108856830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592c7ee1400 msgr2=0x5592c7ee5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:50.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.004+0000 7f39abaf4b80 1 --2- 172.21.15.12:0/108856830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592c7ee1400 0x5592c7ee5560 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f39900042c0 tx=0x7f39900043a0 comp rx=0 tx=0).stop 2024-05-17T22:53:50.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.004+0000 7f39abaf4b80 1 -- 172.21.15.12:0/108856830 shutdown_connections 2024-05-17T22:53:50.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.004+0000 7f39abaf4b80 1 --2- 172.21.15.12:0/108856830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592c7ee1400 0x5592c7ee5560 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:50.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.004+0000 7f39abaf4b80 1 --2- 172.21.15.12:0/108856830 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5592c7ef4a70 0x5592c7ef6f60 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:50.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.004+0000 7f39abaf4b80 1 --2- 172.21.15.12:0/108856830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5592c7d5d9f0 0x5592c7ee4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:50.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.004+0000 7f39abaf4b80 1 --2- 172.21.15.12:0/108856830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592c7edbb00 0x5592c7ee5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:50.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.004+0000 7f39abaf4b80 1 -- 172.21.15.12:0/108856830 >> 172.21.15.12:0/108856830 conn(0x5592c7dbd7d0 msgr2=0x5592c7ed6160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:53:50.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.004+0000 7f397aff5700 0 client.26173 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:53:50.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.004+0000 7f39abaf4b80 1 -- 172.21.15.12:0/108856830 shutdown_connections 2024-05-17T22:53:50.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.004+0000 7f39abaf4b80 1 -- 172.21.15.12:0/108856830 wait complete. 2024-05-17T22:53:50.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.004+0000 7f39abaf4b80 20 client.26173 trim_cache size 0 max 16384 2024-05-17T22:53:50.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.031+0000 7f39abaf4b80 1 Processor -- start 2024-05-17T22:53:50.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.031+0000 7f39abaf4b80 1 -- start start 2024-05-17T22:53:50.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.031+0000 7f39abaf4b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592c7d5d9f0 0x5592c7ef09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:50.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.031+0000 7f39abaf4b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592c7edbb00 0x5592c7ee5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:50.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.031+0000 7f39abaf4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5592c7ee1400 0x5592c7ee5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:50.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.031+0000 7f3996d9d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592c7d5d9f0 0x5592c7ef09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:50.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.031+0000 7f3996d9d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592c7d5d9f0 0x5592c7ef09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:49066/0 (socket says 172.21.15.12:49066) 2024-05-17T22:53:50.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.031+0000 7f3996d9d700 1 -- 172.21.15.12:0/3278789739 learned_addr learned my addr 172.21.15.12:0/3278789739 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:53:50.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.031+0000 7f39abaf4b80 1 -- 172.21.15.12:0/3278789739 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f39981c9c40 con 0x5592c7ee1400 2024-05-17T22:53:50.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.031+0000 7f399659c700 1 --2- 172.21.15.12:0/3278789739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592c7edbb00 0x5592c7ee5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:50.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.031+0000 7f39abaf4b80 1 -- 172.21.15.12:0/3278789739 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f39981c74d0 con 0x5592c7edbb00 2024-05-17T22:53:50.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.031+0000 7f39abaf4b80 1 -- 172.21.15.12:0/3278789739 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f39981c5320 con 0x5592c7d5d9f0 2024-05-17T22:53:50.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.032+0000 7f399759e700 1 --2- 172.21.15.12:0/3278789739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5592c7ee1400 0x5592c7ee5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:50.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.032+0000 7f399659c700 1 -- 172.21.15.12:0/3278789739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592c7d5d9f0 msgr2=0x5592c7ef09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:53:50.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.032+0000 7f399659c700 1 --2- 172.21.15.12:0/3278789739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592c7d5d9f0 0x5592c7ef09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:50.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.032+0000 7f399659c700 1 -- 172.21.15.12:0/3278789739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5592c7ee1400 msgr2=0x5592c7ee5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:53:50.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.032+0000 7f399659c700 1 --2- 172.21.15.12:0/3278789739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5592c7ee1400 0x5592c7ee5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:50.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.032+0000 7f399659c700 1 -- 172.21.15.12:0/3278789739 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592c7ed2c20 con 0x5592c7edbb00 2024-05-17T22:53:50.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.032+0000 7f399659c700 1 --2- 172.21.15.12:0/3278789739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592c7edbb00 0x5592c7ee5010 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x5592c7ee6280 tx=0x7f398c013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a9b4f09ea58bce6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:53:50.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.032+0000 7f397cff9700 1 -- 172.21.15.12:0/3278789739 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f398c012a30 con 0x5592c7edbb00 2024-05-17T22:53:50.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.032+0000 7f397cff9700 1 -- 172.21.15.12:0/3278789739 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f398c024070 con 0x5592c7edbb00 2024-05-17T22:53:50.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.032+0000 7f397cff9700 1 -- 172.21.15.12:0/3278789739 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f398c020430 con 0x5592c7edbb00 2024-05-17T22:53:50.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.033+0000 7f39abaf4b80 1 -- 172.21.15.12:0/3278789739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592c7edbb00 msgr2=0x5592c7ee5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:50.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.033+0000 7f39abaf4b80 1 --2- 172.21.15.12:0/3278789739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592c7edbb00 0x5592c7ee5010 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x5592c7ee6280 tx=0x7f398c013000 comp rx=0 tx=0).stop 2024-05-17T22:53:50.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.034+0000 7f39abaf4b80 1 -- 172.21.15.12:0/3278789739 shutdown_connections 2024-05-17T22:53:50.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.034+0000 7f39abaf4b80 1 --2- 172.21.15.12:0/3278789739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5592c7ee1400 0x5592c7ee5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:50.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.034+0000 7f39abaf4b80 1 --2- 172.21.15.12:0/3278789739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592c7d5d9f0 0x5592c7ef09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:50.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.034+0000 7f39abaf4b80 1 --2- 172.21.15.12:0/3278789739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592c7edbb00 0x5592c7ee5010 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:50.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.034+0000 7f39abaf4b80 1 -- 172.21.15.12:0/3278789739 >> 172.21.15.12:0/3278789739 conn(0x5592c7dbd7d0 msgr2=0x5592c7edfd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:53:50.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.034+0000 7f39abaf4b80 1 -- 172.21.15.12:0/3278789739 shutdown_connections 2024-05-17T22:53:50.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.034+0000 7f39abaf4b80 1 -- 172.21.15.12:0/3278789739 wait complete. 2024-05-17T22:53:50.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.034+0000 7f39abaf4b80 1 -- start start 2024-05-17T22:53:50.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.034+0000 7f39abaf4b80 1 Processor -- start 2024-05-17T22:53:50.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.035+0000 7f39abaf4b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5592c7d5d9f0 0x5592c7eed010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:50.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.035+0000 7f39abaf4b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592c7edbb00 0x5592c7ee9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:50.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.035+0000 7f39abaf4b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592c7ee1400 0x5592c7eea170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:50.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.035+0000 7f39abaf4b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f39981c5990 con 0x5592c7d5d9f0 2024-05-17T22:53:50.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.035+0000 7f39abaf4b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f39981c6e00 con 0x5592c7ee1400 2024-05-17T22:53:50.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.035+0000 7f39abaf4b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f39981c6060 con 0x5592c7edbb00 2024-05-17T22:53:50.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.035+0000 7f39abaf4b80 10 client.0 _ll_register_callbacks cb 0x5592c7d90b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:53:50.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.035+0000 7f399659c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592c7edbb00 0x5592c7ee9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:50.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.035+0000 7f399659c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592c7edbb00 0x5592c7ee9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:49082/0 (socket says 172.21.15.12:49082) 2024-05-17T22:53:50.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.035+0000 7f399659c700 1 -- 172.21.15.12:0/1841739206 learned_addr learned my addr 172.21.15.12:0/1841739206 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:53:50.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.035+0000 7f399759e700 1 --2- 172.21.15.12:0/1841739206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592c7ee1400 0x5592c7eea170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:50.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.036+0000 7f3996d9d700 1 --2- 172.21.15.12:0/1841739206 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5592c7d5d9f0 0x5592c7eed010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:50.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.036+0000 7f399759e700 1 -- 172.21.15.12:0/1841739206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592c7edbb00 msgr2=0x5592c7ee9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:53:50.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.036+0000 7f399759e700 1 --2- 172.21.15.12:0/1841739206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592c7edbb00 0x5592c7ee9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:50.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.036+0000 7f399759e700 1 -- 172.21.15.12:0/1841739206 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5592c7d5d9f0 msgr2=0x5592c7eed010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:53:50.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.036+0000 7f399759e700 1 --2- 172.21.15.12:0/1841739206 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5592c7d5d9f0 0x5592c7eed010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:50.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.036+0000 7f399759e700 1 -- 172.21.15.12:0/1841739206 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592c7eea6b0 con 0x5592c7ee1400 2024-05-17T22:53:50.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.036+0000 7f3996d9d700 1 --2- 172.21.15.12:0/1841739206 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5592c7d5d9f0 0x5592c7eed010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:53:50.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.037+0000 7f399759e700 1 --2- 172.21.15.12:0/1841739206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592c7ee1400 0x5592c7eea170 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3990009ec0 tx=0x7f3990004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2aaacdf5bcfd019 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:53:50.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.037+0000 7f397aff5700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:53:50.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.037+0000 7f397aff5700 1 -- 172.21.15.12:0/1841739206 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399002d020 con 0x5592c7ee1400 2024-05-17T22:53:50.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.037+0000 7f39abaf4b80 1 -- 172.21.15.12:0/1841739206 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f398c024070 con 0x5592c7ee1400 2024-05-17T22:53:50.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.037+0000 7f397aff5700 1 -- 172.21.15.12:0/1841739206 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f399000e980 con 0x5592c7ee1400 2024-05-17T22:53:50.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.037+0000 7f397aff5700 1 -- 172.21.15.12:0/1841739206 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f399001a040 con 0x5592c7ee1400 2024-05-17T22:53:50.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.037+0000 7f39abaf4b80 10 client.26249 fetch_fsmap learned FSMap version 42 2024-05-17T22:53:50.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.037+0000 7f39abaf4b80 1 -- 172.21.15.12:0/1841739206 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5592c7f5ff80 con 0x5592c7ee1400 2024-05-17T22:53:50.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.038+0000 7f397aff5700 1 -- 172.21.15.12:0/1841739206 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f398c024070 con 0x5592c7ee1400 2024-05-17T22:53:50.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.038+0000 7f397aff5700 1 -- 172.21.15.12:0/1841739206 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 42) ==== 8350+0+0 (secure 0 0 0) 0x7f3990027040 con 0x5592c7ee1400 2024-05-17T22:53:50.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.038+0000 7f39abaf4b80 10 client.26249 fetch_fsmap finished waiting for FSMap version 42 2024-05-17T22:53:50.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.038+0000 7f39abaf4b80 10 client.26249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:53:50.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.038+0000 7f39abaf4b80 20 client.26249 populate_metadata read hostname 'smithi012' 2024-05-17T22:53:50.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.038+0000 7f39abaf4b80 1 --2- 172.21.15.12:0/1841739206 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5592c7f60680 0x5592c7f62aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:50.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.038+0000 7f39abaf4b80 4 client.26249 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:53:50.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.038+0000 7f39abaf4b80 1 -- 172.21.15.12:0/1841739206 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5592c7e05800 con 0x5592c7f60680 2024-05-17T22:53:50.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.039+0000 7f3996d9d700 1 --2- 172.21.15.12:0/1841739206 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5592c7f60680 0x5592c7f62aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:50.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.039+0000 7f397aff5700 10 client.26249 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:53:50.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.039+0000 7f3996d9d700 1 --2- 172.21.15.12:0/1841739206 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5592c7f60680 0x5592c7f62aa0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=221cbedc91faa787 server_cookie=10f117244488b115 in_seq=0 out_seq=0 2024-05-17T22:53:50.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.040+0000 7f397aff5700 1 -- 172.21.15.12:0/1841739206 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5592c7e05800 con 0x5592c7f60680 2024-05-17T22:53:50.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.040+0000 7f397aff5700 10 client.26249 handle_command_reply: tid=0 2024-05-17T22:53:50.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.040+0000 7f39abaf4b80 1 client.26249 shutdown 2024-05-17T22:53:50.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.040+0000 7f39abaf4b80 1 -- 172.21.15.12:0/1841739206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592c7ee1400 msgr2=0x5592c7eea170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:50.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.040+0000 7f39abaf4b80 1 --2- 172.21.15.12:0/1841739206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592c7ee1400 0x5592c7eea170 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3990009ec0 tx=0x7f3990004ad0 comp rx=0 tx=0).stop 2024-05-17T22:53:50.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.040+0000 7f39abaf4b80 1 -- 172.21.15.12:0/1841739206 shutdown_connections 2024-05-17T22:53:50.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.040+0000 7f39abaf4b80 1 --2- 172.21.15.12:0/1841739206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592c7ee1400 0x5592c7eea170 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:50.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.040+0000 7f39abaf4b80 1 --2- 172.21.15.12:0/1841739206 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5592c7f60680 0x5592c7f62aa0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:50.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.040+0000 7f397aff5700 0 client.26249 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:53:50.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.040+0000 7f39abaf4b80 1 --2- 172.21.15.12:0/1841739206 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5592c7d5d9f0 0x5592c7eed010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:50.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.041+0000 7f39abaf4b80 1 --2- 172.21.15.12:0/1841739206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592c7edbb00 0x5592c7ee9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:50.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.041+0000 7f39abaf4b80 1 -- 172.21.15.12:0/1841739206 >> 172.21.15.12:0/1841739206 conn(0x5592c7dbd7d0 msgr2=0x5592c7edfd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:53:50.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.042+0000 7f39abaf4b80 1 -- 172.21.15.12:0/1841739206 shutdown_connections 2024-05-17T22:53:50.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.042+0000 7f39abaf4b80 1 -- 172.21.15.12:0/1841739206 wait complete. 2024-05-17T22:53:50.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.042+0000 7f39abaf4b80 20 client.26249 trim_cache size 0 max 16384 2024-05-17T22:53:50.048 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:53:50.049 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-17T22:53:50.049 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-17T22:53:50.049 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:53:50.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.044+0000 7f399d119700 1 -- 172.21.15.12:0/334061363 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f39800b29d0 msgr2=0x7f39800b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:50.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.044+0000 7f399d119700 1 --2- 172.21.15.12:0/334061363 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f39800b29d0 0x7f39800b4ee0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3998074f10 tx=0x7f398c009310 comp rx=0 tx=0).stop 2024-05-17T22:53:50.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.044+0000 7f399d119700 1 -- 172.21.15.12:0/334061363 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f399810aa00 msgr2=0x7f3998120e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:50.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.044+0000 7f399d119700 1 --2- 172.21.15.12:0/334061363 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f399810aa00 0x7f3998120e10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3988009230 tx=0x7f398800fa70 comp rx=0 tx=0).stop 2024-05-17T22:53:50.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.044+0000 7f399d119700 1 -- 172.21.15.12:0/334061363 shutdown_connections 2024-05-17T22:53:50.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.044+0000 7f399d119700 1 --2- 172.21.15.12:0/334061363 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f39800b29d0 0x7f39800b4ee0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:50.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.044+0000 7f399d119700 1 --2- 172.21.15.12:0/334061363 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3998116a60 0x7f399811bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:50.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.044+0000 7f399d119700 1 --2- 172.21.15.12:0/334061363 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f399810aa00 0x7f3998120e10 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:50.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.044+0000 7f399d119700 1 --2- 172.21.15.12:0/334061363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f399810b350 0x7f3998121370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:50.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.044+0000 7f399d119700 1 -- 172.21.15.12:0/334061363 >> 172.21.15.12:0/334061363 conn(0x7f399806efc0 msgr2=0x7f3998072560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:53:50.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.044+0000 7f399d119700 1 -- 172.21.15.12:0/334061363 shutdown_connections 2024-05-17T22:53:50.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:50.045+0000 7f399d119700 1 -- 172.21.15.12:0/334061363 wait complete. 2024-05-17T22:53:50.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:50 smithi012 ceph-mon[31632]: pgmap v876: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.0 MiB/s wr, 445 op/s 2024-05-17T22:53:50.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:50 smithi012 ceph-mon[31632]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:53:50.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:50 smithi151 ceph-mon[36101]: pgmap v876: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.0 MiB/s wr, 445 op/s 2024-05-17T22:53:50.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:50 smithi151 ceph-mon[36101]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:53:50.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:50 smithi096 ceph-mon[36065]: pgmap v876: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.0 MiB/s wr, 445 op/s 2024-05-17T22:53:50.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:50 smithi096 ceph-mon[36065]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:53:51.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 870.0 s, 420.4 tps, lat 244.677 ms stddev 284.833 2024-05-17T22:53:52.339 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-17T22:53:52.340 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T22:53:52.374 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:52 smithi012 ceph-mon[31632]: pgmap v877: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.0 MiB/s wr, 552 op/s 2024-05-17T22:53:52.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:52 smithi151 ceph-mon[36101]: pgmap v877: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.0 MiB/s wr, 552 op/s 2024-05-17T22:53:52.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:52 smithi096 ceph-mon[36065]: pgmap v877: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.0 MiB/s wr, 552 op/s 2024-05-17T22:53:52.763 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:53:54.444 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:54 smithi012 ceph-mon[31632]: pgmap v878: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.7 MiB/s wr, 412 op/s 2024-05-17T22:53:54.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:54 smithi151 ceph-mon[36101]: pgmap v878: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.7 MiB/s wr, 412 op/s 2024-05-17T22:53:54.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:54 smithi096 ceph-mon[36065]: pgmap v878: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.7 MiB/s wr, 412 op/s 2024-05-17T22:53:56.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.300+0000 7f948b197700 1 -- 172.21.15.12:0/251876799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f948410b180 msgr2=0x7f9484117540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:56.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.300+0000 7f948b197700 1 --2- 172.21.15.12:0/251876799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f948410b180 0x7f9484117540 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f94740099c0 tx=0x7f9474009cd0 comp rx=0 tx=0).stop 2024-05-17T22:53:56.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.301+0000 7f948b197700 1 -- 172.21.15.12:0/251876799 shutdown_connections 2024-05-17T22:53:56.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.301+0000 7f948b197700 1 --2- 172.21.15.12:0/251876799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f948410b180 0x7f9484117540 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:56.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.301+0000 7f948b197700 1 --2- 172.21.15.12:0/251876799 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9484110710 0x7f9484110b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:56.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.301+0000 7f948b197700 1 --2- 172.21.15.12:0/251876799 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f948410a830 0x7f948410ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:56.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.301+0000 7f948b197700 1 -- 172.21.15.12:0/251876799 >> 172.21.15.12:0/251876799 conn(0x7f948406f420 msgr2=0x7f948406f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:53:56.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.312+0000 7f948b197700 1 -- 172.21.15.12:0/251876799 shutdown_connections 2024-05-17T22:53:56.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.315+0000 7f948b197700 1 -- 172.21.15.12:0/251876799 wait complete. 2024-05-17T22:53:56.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.316+0000 7f948b197700 1 Processor -- start 2024-05-17T22:53:56.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.324+0000 7f948b197700 1 -- start start 2024-05-17T22:53:56.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.325+0000 7f948b197700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f948410b180 0x7f94841a0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:56.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.325+0000 7f948b197700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9484110710 0x7f94841a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:56.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.325+0000 7f948b197700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94841a18f0 0x7f94841a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:56.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.325+0000 7f948b197700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f948411a530 con 0x7f948410b180 2024-05-17T22:53:56.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.325+0000 7f948b197700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f948411a3b0 con 0x7f94841a18f0 2024-05-17T22:53:56.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.325+0000 7f948b197700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f948411a6b0 con 0x7f9484110710 2024-05-17T22:53:56.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.327+0000 7f9483fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9484110710 0x7f94841a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:56.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.327+0000 7f9483fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9484110710 0x7f94841a13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:49092/0 (socket says 172.21.15.12:49092) 2024-05-17T22:53:56.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.327+0000 7f9483fff700 1 -- 172.21.15.12:0/3385604804 learned_addr learned my addr 172.21.15.12:0/3385604804 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:53:56.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.327+0000 7f9489734700 1 --2- 172.21.15.12:0/3385604804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94841a18f0 0x7f94841a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:56.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.328+0000 7f9489734700 1 -- 172.21.15.12:0/3385604804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9484110710 msgr2=0x7f94841a13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:56.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.328+0000 7f9489734700 1 --2- 172.21.15.12:0/3385604804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9484110710 0x7f94841a13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:56.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.328+0000 7f9489734700 1 -- 172.21.15.12:0/3385604804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f948410b180 msgr2=0x7f94841a0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:56.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.328+0000 7f9489734700 1 --2- 172.21.15.12:0/3385604804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f948410b180 0x7f94841a0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:56.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.328+0000 7f9489734700 1 -- 172.21.15.12:0/3385604804 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94841a5db0 con 0x7f94841a18f0 2024-05-17T22:53:56.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.336+0000 7f9489734700 1 --2- 172.21.15.12:0/3385604804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94841a18f0 0x7f94841a5870 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f947400b580 tx=0x7f94740057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:53:56.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.338+0000 7f9481ffb700 1 -- 172.21.15.12:0/3385604804 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9474027070 con 0x7f94841a18f0 2024-05-17T22:53:56.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.338+0000 7f948b197700 1 -- 172.21.15.12:0/3385604804 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94841a6040 con 0x7f94841a18f0 2024-05-17T22:53:56.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.338+0000 7f948b197700 1 -- 172.21.15.12:0/3385604804 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94841a6570 con 0x7f94841a18f0 2024-05-17T22:53:56.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.338+0000 7f9481ffb700 1 -- 172.21.15.12:0/3385604804 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9474003f90 con 0x7f94841a18f0 2024-05-17T22:53:56.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.338+0000 7f9481ffb700 1 -- 172.21.15.12:0/3385604804 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9474022ca0 con 0x7f94841a18f0 2024-05-17T22:53:56.346 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:56 smithi012 ceph-mon[31632]: pgmap v879: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.7 MiB/s wr, 509 op/s 2024-05-17T22:53:56.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.344+0000 7f946b7fe700 1 -- 172.21.15.12:0/3385604804 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9464005330 con 0x7f94841a18f0 2024-05-17T22:53:56.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.346+0000 7f9481ffb700 1 -- 172.21.15.12:0/3385604804 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9474022470 con 0x7f94841a18f0 2024-05-17T22:53:56.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.348+0000 7f9481ffb700 1 --2- 172.21.15.12:0/3385604804 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f946c0b2960 0x7f946c0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:53:56.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.348+0000 7f9481ffb700 1 -- 172.21.15.12:0/3385604804 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f94740e3e70 con 0x7f94841a18f0 2024-05-17T22:53:56.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.350+0000 7f9488f33700 1 --2- 172.21.15.12:0/3385604804 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f946c0b2960 0x7f946c0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:53:56.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.354+0000 7f9481ffb700 1 -- 172.21.15.12:0/3385604804 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9474013070 con 0x7f94841a18f0 2024-05-17T22:53:56.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.364+0000 7f9488f33700 1 --2- 172.21.15.12:0/3385604804 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f946c0b2960 0x7f946c0b4e70 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f94841a26c0 tx=0x7f9478006cd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:53:56.368 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 875.0 s, 499.6 tps, lat 201.335 ms stddev 237.834 2024-05-17T22:53:56.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:56 smithi151 ceph-mon[36101]: pgmap v879: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.7 MiB/s wr, 509 op/s 2024-05-17T22:53:56.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:56 smithi096 ceph-mon[36065]: pgmap v879: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.7 MiB/s wr, 509 op/s 2024-05-17T22:53:56.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.848+0000 7f946b7fe700 1 -- 172.21.15.12:0/3385604804 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9464005a40 con 0x7f94841a18f0 2024-05-17T22:53:56.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.849+0000 7f9481ffb700 1 -- 172.21.15.12:0/3385604804 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11871 (secure 0 0 0) 0x7f94740ab830 con 0x7f94841a18f0 2024-05-17T22:53:56.851 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:53:56.852 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1407219319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1407219319},{"type":"v1","addr":"172.21.15.12:6835","nonce":1407219319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/630977886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":630977886},{"type":"v1","addr":"172.21.15.12:6837","nonce":630977886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3731914742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3731914742},{"type":"v1","addr":"172.21.15.12:6839","nonce":3731914742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/3642142501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3642142501},{"type":"v1","addr":"172.21.15.12:6841","nonce":3642142501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3554053059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3554053059},{"type":"v1","addr":"172.21.15.151:6833","nonce":3554053059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/4228325569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":4228325569},{"type":"v1","addr":"172.21.15.96:6833","nonce":4228325569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T22:34:42.642111+0000","modified":"2024-05-17T22:53:48.170663+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24461,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/542323754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":542323754},{"type":"v1","addr":"172.21.15.151:6835","nonce":542323754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1758442232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1758442232},{"type":"v1","addr":"172.21.15.96:6835","nonce":1758442232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3403324038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3403324038},{"type":"v1","addr":"172.21.15.151:6837","nonce":3403324038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.151:6839/2338353849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2338353849},{"type":"v1","addr":"172.21.15.151:6839","nonce":2338353849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.96:6837/4222574338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":4222574338},{"type":"v1","addr":"172.21.15.96:6837","nonce":4222574338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/2986295519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2986295519},{"type":"v1","addr":"172.21.15.96:6839","nonce":2986295519}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24457,24482,24461]},"id":1}]} 2024-05-17T22:53:56.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.855+0000 7f946b7fe700 1 -- 172.21.15.12:0/3385604804 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f946c0b2960 msgr2=0x7f946c0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:56.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.855+0000 7f946b7fe700 1 --2- 172.21.15.12:0/3385604804 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f946c0b2960 0x7f946c0b4e70 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f94841a26c0 tx=0x7f9478006cd0 comp rx=0 tx=0).stop 2024-05-17T22:53:56.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.855+0000 7f946b7fe700 1 -- 172.21.15.12:0/3385604804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94841a18f0 msgr2=0x7f94841a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:53:56.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.855+0000 7f946b7fe700 1 --2- 172.21.15.12:0/3385604804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94841a18f0 0x7f94841a5870 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f947400b580 tx=0x7f94740057c0 comp rx=0 tx=0).stop 2024-05-17T22:53:56.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.856+0000 7f946b7fe700 1 -- 172.21.15.12:0/3385604804 shutdown_connections 2024-05-17T22:53:56.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.856+0000 7f946b7fe700 1 --2- 172.21.15.12:0/3385604804 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f946c0b2960 0x7f946c0b4e70 secure :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f94841a26c0 tx=0x7f9478006cd0 comp rx=0 tx=0).stop 2024-05-17T22:53:56.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.856+0000 7f946b7fe700 1 --2- 172.21.15.12:0/3385604804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94841a18f0 0x7f94841a5870 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:56.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.856+0000 7f946b7fe700 1 --2- 172.21.15.12:0/3385604804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f948410b180 0x7f94841a0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:56.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.856+0000 7f946b7fe700 1 --2- 172.21.15.12:0/3385604804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9484110710 0x7f94841a13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:53:56.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.856+0000 7f946b7fe700 1 -- 172.21.15.12:0/3385604804 >> 172.21.15.12:0/3385604804 conn(0x7f948406f420 msgr2=0x7f9484073f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:53:56.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.860+0000 7f946b7fe700 1 -- 172.21.15.12:0/3385604804 shutdown_connections 2024-05-17T22:53:56.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:53:56.860+0000 7f946b7fe700 1 -- 172.21.15.12:0/3385604804 wait complete. 2024-05-17T22:53:56.868 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 42 2024-05-17T22:53:57.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:57 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/3385604804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:53:57.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:57 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/3385604804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:53:57.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:57 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/3385604804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:53:57.778 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-17T22:53:58.351 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:53:58.481 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:53:58 smithi012 ceph-mon[31632]: pgmap v880: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.7 MiB/s wr, 417 op/s 2024-05-17T22:53:58.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:53:58 smithi151 ceph-mon[36101]: pgmap v880: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.7 MiB/s wr, 417 op/s 2024-05-17T22:53:58.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:53:58 smithi096 ceph-mon[36065]: pgmap v880: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.7 MiB/s wr, 417 op/s 2024-05-17T22:54:00.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.329+0000 7ff017fff700 1 -- 172.21.15.12:0/2678070602 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff01810aac0 msgr2=0x7ff01810aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:00.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.329+0000 7ff017fff700 1 --2- 172.21.15.12:0/2678070602 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff01810aac0 0x7ff01810aed0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff01000c340 tx=0x7ff01000c650 comp rx=0 tx=0).stop 2024-05-17T22:54:00.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.333+0000 7ff017fff700 1 -- 172.21.15.12:0/2678070602 shutdown_connections 2024-05-17T22:54:00.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.333+0000 7ff017fff700 1 --2- 172.21.15.12:0/2678070602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff01810f9f0 0x7ff018117350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:00.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.333+0000 7ff017fff700 1 --2- 172.21.15.12:0/2678070602 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff01810aac0 0x7ff01810aed0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:00.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.333+0000 7ff017fff700 1 --2- 172.21.15.12:0/2678070602 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff01810f0a0 0x7ff01810f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:00.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.333+0000 7ff017fff700 1 -- 172.21.15.12:0/2678070602 >> 172.21.15.12:0/2678070602 conn(0x7ff01806eda0 msgr2=0x7ff01806f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:00.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.333+0000 7ff017fff700 1 -- 172.21.15.12:0/2678070602 shutdown_connections 2024-05-17T22:54:00.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.334+0000 7ff017fff700 1 -- 172.21.15.12:0/2678070602 wait complete. 2024-05-17T22:54:00.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.335+0000 7ff017fff700 1 Processor -- start 2024-05-17T22:54:00.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.335+0000 7ff017fff700 1 -- start start 2024-05-17T22:54:00.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.336+0000 7ff017fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff01810aac0 0x7ff0181a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:00.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.336+0000 7ff017fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff01810f0a0 0x7ff0181a6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:00.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.336+0000 7ff017fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff01810f9f0 0x7ff0181a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:00.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.336+0000 7ff017fff700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff01811a360 con 0x7ff01810f0a0 2024-05-17T22:54:00.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.336+0000 7ff0177fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff01810f9f0 0x7ff0181a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:00.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.336+0000 7ff0177fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff01810f9f0 0x7ff0181a0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:33840/0 (socket says 172.21.15.12:33840) 2024-05-17T22:54:00.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.336+0000 7ff0177fe700 1 -- 172.21.15.12:0/3760623469 learned_addr learned my addr 172.21.15.12:0/3760623469 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:54:00.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.337+0000 7ff016ffd700 1 --2- 172.21.15.12:0/3760623469 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff01810aac0 0x7ff0181a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:00.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.337+0000 7ff017fff700 1 -- 172.21.15.12:0/3760623469 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff01811a1e0 con 0x7ff01810f9f0 2024-05-17T22:54:00.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.337+0000 7ff017fff700 1 -- 172.21.15.12:0/3760623469 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff01811a4e0 con 0x7ff01810aac0 2024-05-17T22:54:00.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.337+0000 7ff0167fc700 1 --2- 172.21.15.12:0/3760623469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff01810f0a0 0x7ff0181a6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:00.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.338+0000 7ff0167fc700 1 -- 172.21.15.12:0/3760623469 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff01810aac0 msgr2=0x7ff0181a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:00.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.338+0000 7ff0167fc700 1 --2- 172.21.15.12:0/3760623469 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff01810aac0 0x7ff0181a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:00.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.338+0000 7ff0167fc700 1 -- 172.21.15.12:0/3760623469 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff01810f9f0 msgr2=0x7ff0181a0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:00.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.338+0000 7ff0167fc700 1 --2- 172.21.15.12:0/3760623469 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff01810f9f0 0x7ff0181a0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:00.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.338+0000 7ff0167fc700 1 -- 172.21.15.12:0/3760623469 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0181a10c0 con 0x7ff01810f0a0 2024-05-17T22:54:00.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.339+0000 7ff0167fc700 1 --2- 172.21.15.12:0/3760623469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff01810f0a0 0x7ff0181a6d30 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff01806cbf0 tx=0x7ff00800b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:00.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.343+0000 7ff01c9fa700 1 -- 172.21.15.12:0/3760623469 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff008015020 con 0x7ff01810f0a0 2024-05-17T22:54:00.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.343+0000 7ff01c9fa700 1 -- 172.21.15.12:0/3760623469 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff008011ce0 con 0x7ff01810f0a0 2024-05-17T22:54:00.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.343+0000 7ff01c9fa700 1 -- 172.21.15.12:0/3760623469 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff008010d40 con 0x7ff01810f0a0 2024-05-17T22:54:00.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.343+0000 7ff017fff700 1 -- 172.21.15.12:0/3760623469 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0181a13b0 con 0x7ff01810f0a0 2024-05-17T22:54:00.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.343+0000 7ff017fff700 1 -- 172.21.15.12:0/3760623469 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0181ad680 con 0x7ff01810f0a0 2024-05-17T22:54:00.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.349+0000 7ff02c3d7b80 1 Processor -- start 2024-05-17T22:54:00.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.349+0000 7ff02c3d7b80 1 -- start start 2024-05-17T22:54:00.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.349+0000 7ff02c3d7b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563e347a79f0 0x563e3492adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:00.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.349+0000 7ff02c3d7b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e3492b330 0x563e349255c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:00.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.349+0000 7ff02c3d7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e3492b740 0x563e34925b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:00.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.349+0000 7ff02c3d7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563e348540d0 con 0x563e3492b740 2024-05-17T22:54:00.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.349+0000 7ff0177fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563e347a79f0 0x563e3492adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:00.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.350+0000 7ff0177fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563e347a79f0 0x563e3492adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:52278/0 (socket says 172.21.15.12:52278) 2024-05-17T22:54:00.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.350+0000 7ff0177fe700 1 -- 172.21.15.12:0/1017092650 learned_addr learned my addr 172.21.15.12:0/1017092650 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:54:00.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.350+0000 7ff016ffd700 1 --2- 172.21.15.12:0/1017092650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e3492b330 0x563e349255c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:00.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.349+0000 7ff02c3d7b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563e34843c70 con 0x563e3492b330 2024-05-17T22:54:00.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.350+0000 7ff02c3d7b80 1 -- 172.21.15.12:0/1017092650 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563e34847cb0 con 0x563e347a79f0 2024-05-17T22:54:00.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.351+0000 7ff016ffd700 1 -- 172.21.15.12:0/1017092650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563e347a79f0 msgr2=0x563e3492adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:00.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.351+0000 7ff016ffd700 1 --2- 172.21.15.12:0/1017092650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563e347a79f0 0x563e3492adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:00.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.351+0000 7ff016ffd700 1 -- 172.21.15.12:0/1017092650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e3492b740 msgr2=0x563e34925b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T22:54:00.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.351+0000 7ff016ffd700 1 --2- 172.21.15.12:0/1017092650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e3492b740 0x563e34925b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:00.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.351+0000 7ff016ffd700 1 -- 172.21.15.12:0/1017092650 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e347dd5c0 con 0x563e3492b330 2024-05-17T22:54:00.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.354+0000 7ff01c9fa700 1 -- 172.21.15.12:0/3760623469 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff008018020 con 0x7ff01810f0a0 2024-05-17T22:54:00.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.355+0000 7ff01c9fa700 1 --2- 172.21.15.12:0/3760623469 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff0000b2a80 0x7ff0000b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:00.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.356+0000 7ff01c9fa700 1 -- 172.21.15.12:0/3760623469 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7ff0080e1af0 con 0x7ff01810f0a0 2024-05-17T22:54:00.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.356+0000 7ff0177fe700 1 --2- 172.21.15.12:0/3760623469 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff0000b2a80 0x7ff0000b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:00.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.360+0000 7ff0177fe700 1 --2- 172.21.15.12:0/3760623469 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff0000b2a80 0x7ff0000b4f90 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x563e34921b60 tx=0x7ff00c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:00.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.364+0000 7ff016ffd700 1 --2- 172.21.15.12:0/1017092650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e3492b330 0x563e349255c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff010000f80 tx=0x7ff0100077e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22e6396865ed7728 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:00.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.368+0000 7feffdffb700 1 -- 172.21.15.12:0/1017092650 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff010023020 con 0x563e3492b330 2024-05-17T22:54:00.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.368+0000 7feffdffb700 1 -- 172.21.15.12:0/1017092650 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff010012390 con 0x563e3492b330 2024-05-17T22:54:00.371 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:00 smithi012 ceph-mon[31632]: pgmap v881: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 3.5 MiB/s wr, 392 op/s 2024-05-17T22:54:00.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.369+0000 7ff02c3d7b80 1 -- 172.21.15.12:0/1017092650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e3492b330 msgr2=0x563e349255c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:00.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.369+0000 7ff02c3d7b80 1 --2- 172.21.15.12:0/1017092650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e3492b330 0x563e349255c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff010000f80 tx=0x7ff0100077e0 comp rx=0 tx=0).stop 2024-05-17T22:54:00.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.372+0000 7ff02c3d7b80 1 -- 172.21.15.12:0/1017092650 shutdown_connections 2024-05-17T22:54:00.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.372+0000 7ff02c3d7b80 1 --2- 172.21.15.12:0/1017092650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e3492b740 0x563e34925b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:00.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.372+0000 7ff02c3d7b80 1 --2- 172.21.15.12:0/1017092650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563e347a79f0 0x563e3492adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:00.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.372+0000 7ff02c3d7b80 1 --2- 172.21.15.12:0/1017092650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e3492b330 0x563e349255c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:00.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.372+0000 7ff02c3d7b80 1 -- 172.21.15.12:0/1017092650 >> 172.21.15.12:0/1017092650 conn(0x563e348077d0 msgr2=0x563e34920080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:00.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.375+0000 7ff02c3d7b80 1 -- 172.21.15.12:0/1017092650 shutdown_connections 2024-05-17T22:54:00.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.376+0000 7ff02c3d7b80 1 -- 172.21.15.12:0/1017092650 wait complete. 2024-05-17T22:54:00.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.376+0000 7ff02c3d7b80 1 -- start start 2024-05-17T22:54:00.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.376+0000 7ff02c3d7b80 1 Processor -- start 2024-05-17T22:54:00.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.377+0000 7ff02c3d7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e347a79f0 0x563e3492f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:00.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.377+0000 7ff02c3d7b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563e3492b330 0x563e3492fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:00.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.377+0000 7ff02c3d7b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e3492b740 0x563e34933cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:00.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.377+0000 7ff02c3d7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563e3474b150 con 0x563e347a79f0 2024-05-17T22:54:00.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.377+0000 7ff02c3d7b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563e347e5940 con 0x563e3492b740 2024-05-17T22:54:00.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.377+0000 7ff02c3d7b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563e349299d0 con 0x563e3492b330 2024-05-17T22:54:00.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.377+0000 7ff02c3d7b80 10 client.0 _ll_register_callbacks cb 0x563e347dab50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:54:00.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.378+0000 7ff0167fc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563e3492b330 0x563e3492fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:00.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.378+0000 7ff0167fc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563e3492b330 0x563e3492fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:52280/0 (socket says 172.21.15.12:52280) 2024-05-17T22:54:00.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.378+0000 7ff0167fc700 1 -- 172.21.15.12:0/2038814405 learned_addr learned my addr 172.21.15.12:0/2038814405 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:54:00.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.379+0000 7ff0167fc700 1 -- 172.21.15.12:0/2038814405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e3492b740 msgr2=0x563e34933cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T22:54:00.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.379+0000 7ff0167fc700 1 --2- 172.21.15.12:0/2038814405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e3492b740 0x563e34933cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:00.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.379+0000 7ff0167fc700 1 -- 172.21.15.12:0/2038814405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e347a79f0 msgr2=0x563e3492f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:00.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.379+0000 7ff0167fc700 1 --2- 172.21.15.12:0/2038814405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e347a79f0 0x563e3492f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:00.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.379+0000 7ff0167fc700 1 -- 172.21.15.12:0/2038814405 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e34794480 con 0x563e3492b330 2024-05-17T22:54:00.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.382+0000 7ff0167fc700 1 --2- 172.21.15.12:0/2038814405 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563e3492b330 0x563e3492fa20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff0080a8cb0 tx=0x7ff0080a4d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3fd4812431b64ba server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:00.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.382+0000 7feff6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:54:00.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.382+0000 7feff6ffd700 1 -- 172.21.15.12:0/2038814405 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0080bf070 con 0x563e3492b330 2024-05-17T22:54:00.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.382+0000 7feff6ffd700 1 -- 172.21.15.12:0/2038814405 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7ff008010410 con 0x563e3492b330 2024-05-17T22:54:00.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.382+0000 7ff02c3d7b80 1 -- 172.21.15.12:0/2038814405 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff010012390 con 0x563e3492b330 2024-05-17T22:54:00.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.382+0000 7feff6ffd700 1 -- 172.21.15.12:0/2038814405 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ff0080a9420 con 0x563e3492b330 2024-05-17T22:54:00.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.383+0000 7ff02c3d7b80 10 client.26191 fetch_fsmap learned FSMap version 42 2024-05-17T22:54:00.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.383+0000 7ff02c3d7b80 1 -- 172.21.15.12:0/2038814405 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x563e3493cb30 con 0x563e3492b330 2024-05-17T22:54:00.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.384+0000 7feff6ffd700 1 -- 172.21.15.12:0/2038814405 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff010012390 con 0x563e3492b330 2024-05-17T22:54:00.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.385+0000 7feff6ffd700 1 -- 172.21.15.12:0/2038814405 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 42) ==== 8350+0+0 (secure 0 0 0) 0x7ff008022060 con 0x563e3492b330 2024-05-17T22:54:00.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.385+0000 7ff02c3d7b80 10 client.26191 fetch_fsmap finished waiting for FSMap version 42 2024-05-17T22:54:00.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.385+0000 7ff02c3d7b80 10 client.26191 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-17T22:54:00.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.385+0000 7ff02c3d7b80 20 client.26191 populate_metadata read hostname 'smithi012' 2024-05-17T22:54:00.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.385+0000 7ff02c3d7b80 1 --2- 172.21.15.12:0/2038814405 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x563e3493f360 0x563e349417d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:00.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.385+0000 7ff02c3d7b80 4 client.26191 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:54:00.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.385+0000 7ff02c3d7b80 1 -- 172.21.15.12:0/2038814405 --> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e34856020 con 0x563e3493f360 2024-05-17T22:54:00.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.385+0000 7ff016ffd700 1 --2- 172.21.15.12:0/2038814405 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x563e3493f360 0x563e349417d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:00.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.386+0000 7ff016ffd700 1 --2- 172.21.15.12:0/2038814405 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x563e3493f360 0x563e349417d0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4826f5544ace36e3 server_cookie=5b20265e4aeed6e3 in_seq=0 out_seq=0 2024-05-17T22:54:00.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.386+0000 7feff6ffd700 10 client.26191 ms_handle_connect on v2:172.21.15.151:6838/2338353849 2024-05-17T22:54:00.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.387+0000 7feff6ffd700 1 -- 172.21.15.12:0/2038814405 <== mds.2 v2:172.21.15.151:6838/2338353849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563e34856020 con 0x563e3493f360 2024-05-17T22:54:00.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.387+0000 7feff6ffd700 10 client.26191 handle_command_reply: tid=0 2024-05-17T22:54:00.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.387+0000 7ff02c3d7b80 1 client.26191 shutdown 2024-05-17T22:54:00.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.387+0000 7ff02c3d7b80 1 -- 172.21.15.12:0/2038814405 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563e3492b330 msgr2=0x563e3492fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:00.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.387+0000 7ff02c3d7b80 1 --2- 172.21.15.12:0/2038814405 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563e3492b330 0x563e3492fa20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff0080a8cb0 tx=0x7ff0080a4d10 comp rx=0 tx=0).stop 2024-05-17T22:54:00.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.388+0000 7ff02c3d7b80 1 -- 172.21.15.12:0/2038814405 shutdown_connections 2024-05-17T22:54:00.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.388+0000 7ff02c3d7b80 1 --2- 172.21.15.12:0/2038814405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e3492b740 0x563e34933cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:00.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.388+0000 7ff02c3d7b80 1 --2- 172.21.15.12:0/2038814405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e347a79f0 0x563e3492f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:00.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.388+0000 7ff02c3d7b80 1 --2- 172.21.15.12:0/2038814405 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x563e3493f360 0x563e349417d0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:00.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.388+0000 7ff02c3d7b80 1 --2- 172.21.15.12:0/2038814405 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563e3492b330 0x563e3492fa20 secure :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff0080a8cb0 tx=0x7ff0080a4d10 comp rx=0 tx=0).stop 2024-05-17T22:54:00.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.388+0000 7ff02c3d7b80 1 -- 172.21.15.12:0/2038814405 >> 172.21.15.12:0/2038814405 conn(0x563e348077d0 msgr2=0x563e3492a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:00.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.388+0000 7feff6ffd700 0 client.26191 ms_handle_reset on v2:172.21.15.151:6838/2338353849 2024-05-17T22:54:00.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.390+0000 7ff02c3d7b80 1 -- 172.21.15.12:0/2038814405 shutdown_connections 2024-05-17T22:54:00.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.390+0000 7ff02c3d7b80 1 -- 172.21.15.12:0/2038814405 wait complete. 2024-05-17T22:54:00.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.390+0000 7ff02c3d7b80 20 client.26191 trim_cache size 0 max 16384 2024-05-17T22:54:00.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.417+0000 7ff02c3d7b80 1 Processor -- start 2024-05-17T22:54:00.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.417+0000 7ff02c3d7b80 1 -- start start 2024-05-17T22:54:00.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.418+0000 7ff02c3d7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e347a79f0 0x563e34946fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:00.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.418+0000 7ff02c3d7b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e3492b740 0x563e34947510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:00.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.418+0000 7ff02c3d7b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563e34929620 0x563e349235b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:00.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.418+0000 7ff02c3d7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff0181b8830 con 0x563e347a79f0 2024-05-17T22:54:00.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.418+0000 7ff02c3d7b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff0181b60c0 con 0x563e3492b740 2024-05-17T22:54:00.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.418+0000 7ff02c3d7b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff0181b3fc0 con 0x563e34929620 2024-05-17T22:54:00.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.418+0000 7ff016ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e347a79f0 0x563e34946fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:00.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.418+0000 7ff016ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e347a79f0 0x563e34946fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60604/0 (socket says 172.21.15.12:60604) 2024-05-17T22:54:00.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.418+0000 7ff016ffd700 1 -- 172.21.15.12:0/4025864670 learned_addr learned my addr 172.21.15.12:0/4025864670 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:54:00.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.418+0000 7ff0167fc700 1 --2- 172.21.15.12:0/4025864670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e3492b740 0x563e34947510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:00.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.419+0000 7ff0177fe700 1 --2- 172.21.15.12:0/4025864670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563e34929620 0x563e349235b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:00.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.419+0000 7ff016ffd700 1 -- 172.21.15.12:0/4025864670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563e34929620 msgr2=0x563e349235b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:00.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.419+0000 7ff016ffd700 1 --2- 172.21.15.12:0/4025864670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563e34929620 0x563e349235b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:00.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.419+0000 7ff016ffd700 1 -- 172.21.15.12:0/4025864670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e3492b740 msgr2=0x563e34947510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:00.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.419+0000 7ff016ffd700 1 --2- 172.21.15.12:0/4025864670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e3492b740 0x563e34947510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:00.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.419+0000 7ff016ffd700 1 -- 172.21.15.12:0/4025864670 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e34925f30 con 0x563e347a79f0 2024-05-17T22:54:00.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.419+0000 7ff0167fc700 1 --2- 172.21.15.12:0/4025864670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e3492b740 0x563e34947510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:54:00.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.419+0000 7ff016ffd700 1 --2- 172.21.15.12:0/4025864670 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e347a79f0 0x563e34946fd0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ff01000c010 tx=0x7ff01000c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd91bbc8df3af2be server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:00.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.419+0000 7feffd7fa700 1 -- 172.21.15.12:0/4025864670 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff010035070 con 0x563e347a79f0 2024-05-17T22:54:00.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.420+0000 7feffd7fa700 1 -- 172.21.15.12:0/4025864670 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff010012cf0 con 0x563e347a79f0 2024-05-17T22:54:00.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.420+0000 7feffd7fa700 1 -- 172.21.15.12:0/4025864670 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff010023040 con 0x563e347a79f0 2024-05-17T22:54:00.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.421+0000 7ff02c3d7b80 1 -- 172.21.15.12:0/4025864670 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e347a79f0 msgr2=0x563e34946fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:00.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.421+0000 7ff02c3d7b80 1 --2- 172.21.15.12:0/4025864670 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e347a79f0 0x563e34946fd0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ff01000c010 tx=0x7ff01000c040 comp rx=0 tx=0).stop 2024-05-17T22:54:00.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.421+0000 7ff02c3d7b80 1 -- 172.21.15.12:0/4025864670 shutdown_connections 2024-05-17T22:54:00.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.421+0000 7ff02c3d7b80 1 --2- 172.21.15.12:0/4025864670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563e34929620 0x563e349235b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:00.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.421+0000 7ff02c3d7b80 1 --2- 172.21.15.12:0/4025864670 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e347a79f0 0x563e34946fd0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:00.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.421+0000 7ff02c3d7b80 1 --2- 172.21.15.12:0/4025864670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e3492b740 0x563e34947510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:00.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.421+0000 7ff02c3d7b80 1 -- 172.21.15.12:0/4025864670 >> 172.21.15.12:0/4025864670 conn(0x563e348077d0 msgr2=0x563e34941370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:00.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.421+0000 7ff02c3d7b80 1 -- 172.21.15.12:0/4025864670 shutdown_connections 2024-05-17T22:54:00.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.421+0000 7ff02c3d7b80 1 -- 172.21.15.12:0/4025864670 wait complete. 2024-05-17T22:54:00.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.422+0000 7ff02c3d7b80 1 -- start start 2024-05-17T22:54:00.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.422+0000 7ff02c3d7b80 1 Processor -- start 2024-05-17T22:54:00.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.422+0000 7ff02c3d7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e347a79f0 0x563e34936ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:00.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.423+0000 7ff02c3d7b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563e34929620 0x563e34937020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:00.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.423+0000 7ff02c3d7b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e3492b740 0x563e349a1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:00.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.423+0000 7ff016ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e347a79f0 0x563e34936ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:00.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.423+0000 7ff02c3d7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff0181b4580 con 0x563e347a79f0 2024-05-17T22:54:00.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.423+0000 7ff02c3d7b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff0181b59f0 con 0x563e3492b740 2024-05-17T22:54:00.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.423+0000 7ff02c3d7b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff0181b4c50 con 0x563e34929620 2024-05-17T22:54:00.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.423+0000 7ff016ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e347a79f0 0x563e34936ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60614/0 (socket says 172.21.15.12:60614) 2024-05-17T22:54:00.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.423+0000 7ff016ffd700 1 -- 172.21.15.12:0/3768557405 learned_addr learned my addr 172.21.15.12:0/3768557405 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:54:00.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.423+0000 7ff02c3d7b80 10 client.0 _ll_register_callbacks cb 0x563e347dab50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:54:00.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.423+0000 7ff0167fc700 1 --2- 172.21.15.12:0/3768557405 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563e34929620 0x563e34937020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:00.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.423+0000 7ff0177fe700 1 --2- 172.21.15.12:0/3768557405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e3492b740 0x563e349a1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:00.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.423+0000 7ff016ffd700 1 -- 172.21.15.12:0/3768557405 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563e34929620 msgr2=0x563e34937020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:00.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.423+0000 7ff016ffd700 1 --2- 172.21.15.12:0/3768557405 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563e34929620 0x563e34937020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:00.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.423+0000 7ff016ffd700 1 -- 172.21.15.12:0/3768557405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e3492b740 msgr2=0x563e349a1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:00.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.423+0000 7ff016ffd700 1 --2- 172.21.15.12:0/3768557405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e3492b740 0x563e349a1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:00.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.424+0000 7ff016ffd700 1 -- 172.21.15.12:0/3768557405 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e349a20f0 con 0x563e347a79f0 2024-05-17T22:54:00.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.424+0000 7ff0177fe700 1 --2- 172.21.15.12:0/3768557405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e3492b740 0x563e349a1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:54:00.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.424+0000 7ff016ffd700 1 --2- 172.21.15.12:0/3768557405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e347a79f0 0x563e34936ae0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff01000c340 tx=0x7ff01000c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1344aa1a34e51094 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:00.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.424+0000 7feff6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:54:00.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.424+0000 7feff6ffd700 1 -- 172.21.15.12:0/3768557405 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff010035030 con 0x563e347a79f0 2024-05-17T22:54:00.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.424+0000 7feff6ffd700 1 -- 172.21.15.12:0/3768557405 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7ff010010040 con 0x563e347a79f0 2024-05-17T22:54:00.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.424+0000 7ff02c3d7b80 1 -- 172.21.15.12:0/3768557405 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff010012cf0 con 0x563e347a79f0 2024-05-17T22:54:00.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.424+0000 7feff6ffd700 1 -- 172.21.15.12:0/3768557405 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ff010023070 con 0x563e347a79f0 2024-05-17T22:54:00.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.424+0000 7ff02c3d7b80 10 client.16560 fetch_fsmap learned FSMap version 42 2024-05-17T22:54:00.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.424+0000 7ff02c3d7b80 1 -- 172.21.15.12:0/3768557405 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x563e349a93c0 con 0x563e347a79f0 2024-05-17T22:54:00.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.425+0000 7feff6ffd700 1 -- 172.21.15.12:0/3768557405 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0100129a0 con 0x563e347a79f0 2024-05-17T22:54:00.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.425+0000 7feff6ffd700 1 -- 172.21.15.12:0/3768557405 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 42) ==== 8350+0+0 (secure 0 0 0) 0x7ff01002fce0 con 0x563e347a79f0 2024-05-17T22:54:00.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.425+0000 7ff02c3d7b80 10 client.16560 fetch_fsmap finished waiting for FSMap version 42 2024-05-17T22:54:00.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.425+0000 7ff02c3d7b80 10 client.16560 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-17T22:54:00.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.425+0000 7ff02c3d7b80 20 client.16560 populate_metadata read hostname 'smithi012' 2024-05-17T22:54:00.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.425+0000 7ff02c3d7b80 1 --2- 172.21.15.12:0/3768557405 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x563e349a9590 0x563e349ab9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:00.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.425+0000 7ff02c3d7b80 4 client.16560 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:54:00.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.425+0000 7ff02c3d7b80 1 -- 172.21.15.12:0/3768557405 --> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ff010012cf0 con 0x563e349a9590 2024-05-17T22:54:00.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.425+0000 7ff0167fc700 1 --2- 172.21.15.12:0/3768557405 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x563e349a9590 0x563e349ab9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:00.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.426+0000 7feff6ffd700 10 client.16560 ms_handle_connect on v2:172.21.15.151:6838/2338353849 2024-05-17T22:54:00.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.426+0000 7ff0167fc700 1 --2- 172.21.15.12:0/3768557405 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x563e349a9590 0x563e349ab9b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=787d9ea69c80d377 server_cookie=acc771d280e85cc4 in_seq=0 out_seq=0 2024-05-17T22:54:00.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.426+0000 7feff6ffd700 1 -- 172.21.15.12:0/3768557405 <== mds.2 v2:172.21.15.151:6838/2338353849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ff010012cf0 con 0x563e349a9590 2024-05-17T22:54:00.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.426+0000 7feff6ffd700 10 client.16560 handle_command_reply: tid=0 2024-05-17T22:54:00.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.426+0000 7ff02c3d7b80 1 client.16560 shutdown 2024-05-17T22:54:00.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.426+0000 7ff02c3d7b80 1 -- 172.21.15.12:0/3768557405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e347a79f0 msgr2=0x563e34936ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:00.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.426+0000 7ff02c3d7b80 1 --2- 172.21.15.12:0/3768557405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e347a79f0 0x563e34936ae0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff01000c340 tx=0x7ff01000c6c0 comp rx=0 tx=0).stop 2024-05-17T22:54:00.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.426+0000 7ff02c3d7b80 1 -- 172.21.15.12:0/3768557405 shutdown_connections 2024-05-17T22:54:00.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.426+0000 7ff02c3d7b80 1 --2- 172.21.15.12:0/3768557405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e3492b740 0x563e349a1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:00.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.426+0000 7ff02c3d7b80 1 --2- 172.21.15.12:0/3768557405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e347a79f0 0x563e34936ae0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:00.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.426+0000 7ff02c3d7b80 1 --2- 172.21.15.12:0/3768557405 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x563e349a9590 0x563e349ab9b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:00.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.426+0000 7ff02c3d7b80 1 --2- 172.21.15.12:0/3768557405 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563e34929620 0x563e34937020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:00.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.426+0000 7feff6ffd700 0 client.16560 ms_handle_reset on v2:172.21.15.151:6838/2338353849 2024-05-17T22:54:00.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.427+0000 7ff02c3d7b80 1 -- 172.21.15.12:0/3768557405 >> 172.21.15.12:0/3768557405 conn(0x563e348077d0 msgr2=0x563e34924620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:00.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.427+0000 7ff02c3d7b80 1 -- 172.21.15.12:0/3768557405 shutdown_connections 2024-05-17T22:54:00.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.427+0000 7ff02c3d7b80 1 -- 172.21.15.12:0/3768557405 wait complete. 2024-05-17T22:54:00.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.427+0000 7ff02c3d7b80 20 client.16560 trim_cache size 0 max 16384 2024-05-17T22:54:00.436 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:54:00.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.427+0000 7ff017fff700 1 -- 172.21.15.12:0/3760623469 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff0000b2a80 msgr2=0x7ff0000b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:00.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.427+0000 7ff017fff700 1 --2- 172.21.15.12:0/3760623469 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff0000b2a80 0x7ff0000b4f90 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x563e34921b60 tx=0x7ff00c008040 comp rx=0 tx=0).stop 2024-05-17T22:54:00.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.427+0000 7ff017fff700 1 -- 172.21.15.12:0/3760623469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff01810f0a0 msgr2=0x7ff0181a6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:00.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.427+0000 7ff017fff700 1 --2- 172.21.15.12:0/3760623469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff01810f0a0 0x7ff0181a6d30 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff01806cbf0 tx=0x7ff00800b7e0 comp rx=0 tx=0).stop 2024-05-17T22:54:00.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.428+0000 7ff017fff700 1 -- 172.21.15.12:0/3760623469 shutdown_connections 2024-05-17T22:54:00.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.428+0000 7ff017fff700 1 --2- 172.21.15.12:0/3760623469 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff0000b2a80 0x7ff0000b4f90 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:00.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.428+0000 7ff017fff700 1 --2- 172.21.15.12:0/3760623469 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff01810f9f0 0x7ff0181a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:00.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.428+0000 7ff017fff700 1 --2- 172.21.15.12:0/3760623469 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff01810aac0 0x7ff0181a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:00.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.428+0000 7ff017fff700 1 --2- 172.21.15.12:0/3760623469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff01810f0a0 0x7ff0181a6d30 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:00.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.428+0000 7ff017fff700 1 -- 172.21.15.12:0/3760623469 >> 172.21.15.12:0/3760623469 conn(0x7ff01806eda0 msgr2=0x7ff018073050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:00.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.428+0000 7ff017fff700 1 -- 172.21.15.12:0/3760623469 shutdown_connections 2024-05-17T22:54:00.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:00.428+0000 7ff017fff700 1 -- 172.21.15.12:0/3760623469 wait complete. 2024-05-17T22:54:00.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:00 smithi151 ceph-mon[36101]: pgmap v881: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 3.5 MiB/s wr, 392 op/s 2024-05-17T22:54:00.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:00 smithi096 ceph-mon[36065]: pgmap v881: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 3.5 MiB/s wr, 392 op/s 2024-05-17T22:54:01.227 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-17T22:54:01.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 880.0 s, 502.8 tps, lat 204.258 ms stddev 255.539 2024-05-17T22:54:01.741 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:54:02.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:02 smithi151 ceph-mon[36101]: pgmap v882: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.8 MiB/s wr, 524 op/s 2024-05-17T22:54:02.550 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:02 smithi096 ceph-mon[36065]: pgmap v882: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.8 MiB/s wr, 524 op/s 2024-05-17T22:54:02.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:02 smithi012 ceph-mon[31632]: pgmap v882: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.8 MiB/s wr, 524 op/s 2024-05-17T22:54:04.457 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:04 smithi012 ceph-mon[31632]: pgmap v883: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.7 MiB/s wr, 410 op/s 2024-05-17T22:54:04.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:04 smithi151 ceph-mon[36101]: pgmap v883: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.7 MiB/s wr, 410 op/s 2024-05-17T22:54:04.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:04 smithi096 ceph-mon[36065]: pgmap v883: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.7 MiB/s wr, 410 op/s 2024-05-17T22:54:04.881 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:54:04.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.640+0000 7f9f8f59e700 1 -- 172.21.15.12:0/31877718 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f880b7c10 msgr2=0x7f9f880ba030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:04.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.640+0000 7f9f8f59e700 1 --2- 172.21.15.12:0/31877718 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f880b7c10 0x7f9f880ba030 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9f84009990 tx=0x7f9f84009ca0 comp rx=0 tx=0).stop 2024-05-17T22:54:04.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.640+0000 7f9f8f59e700 1 -- 172.21.15.12:0/31877718 shutdown_connections 2024-05-17T22:54:04.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.640+0000 7f9f8f59e700 1 --2- 172.21.15.12:0/31877718 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f880b7c10 0x7f9f880ba030 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:04.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.640+0000 7f9f8f59e700 1 --2- 172.21.15.12:0/31877718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f880a4cb0 0x7f9f880a50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:04.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.640+0000 7f9f8f59e700 1 --2- 172.21.15.12:0/31877718 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f880a5600 0x7f9f880b76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:04.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.640+0000 7f9f8f59e700 1 -- 172.21.15.12:0/31877718 >> 172.21.15.12:0/31877718 conn(0x7f9f8801aa00 msgr2=0x7f9f8801ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:04.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.643+0000 7f9f8f59e700 1 -- 172.21.15.12:0/31877718 shutdown_connections 2024-05-17T22:54:04.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.650+0000 7f9f8f59e700 1 -- 172.21.15.12:0/31877718 wait complete. 2024-05-17T22:54:04.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.652+0000 7f9f8f59e700 1 Processor -- start 2024-05-17T22:54:04.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.653+0000 7f9f8f59e700 1 -- start start 2024-05-17T22:54:04.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.653+0000 7f9f8f59e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f880a5600 0x7f9f88144210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:04.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.653+0000 7f9f8f59e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f880b7c10 0x7f9f881447b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:04.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.653+0000 7f9f8f59e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f88148a40 0x7f9f88148e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:04.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.653+0000 7f9f8f59e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9f880bd790 con 0x7f9f880b7c10 2024-05-17T22:54:04.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.653+0000 7f9f8f59e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9f880bd610 con 0x7f9f88148a40 2024-05-17T22:54:04.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.653+0000 7f9f8f59e700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9f880bd910 con 0x7f9f880a5600 2024-05-17T22:54:04.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.655+0000 7f9f8ed9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f88148a40 0x7f9f88148e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:04.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.656+0000 7f9f8e59c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f880a5600 0x7f9f88144210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:04.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.656+0000 7f9f8e59c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f880a5600 0x7f9f88144210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:52308/0 (socket says 172.21.15.12:52308) 2024-05-17T22:54:04.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.656+0000 7f9f8e59c700 1 -- 172.21.15.12:0/3917394992 learned_addr learned my addr 172.21.15.12:0/3917394992 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:54:04.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.657+0000 7f9f8e59c700 1 -- 172.21.15.12:0/3917394992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f88148a40 msgr2=0x7f9f88148e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:04.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.657+0000 7f9f8e59c700 1 --2- 172.21.15.12:0/3917394992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f88148a40 0x7f9f88148e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:04.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.657+0000 7f9f8e59c700 1 -- 172.21.15.12:0/3917394992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f880b7c10 msgr2=0x7f9f881447b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:04.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.657+0000 7f9f8e59c700 1 --2- 172.21.15.12:0/3917394992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f880b7c10 0x7f9f881447b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:04.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.657+0000 7f9f8e59c700 1 -- 172.21.15.12:0/3917394992 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f881494f0 con 0x7f9f880a5600 2024-05-17T22:54:04.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.657+0000 7f9f8e59c700 1 --2- 172.21.15.12:0/3917394992 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f880a5600 0x7f9f88144210 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9f8000c920 tx=0x7f9f8000cce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:04.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.661+0000 7f9f7f7fe700 1 -- 172.21.15.12:0/3917394992 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f80007b00 con 0x7f9f880a5600 2024-05-17T22:54:04.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.661+0000 7f9f7f7fe700 1 -- 172.21.15.12:0/3917394992 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f80007ca0 con 0x7f9f880a5600 2024-05-17T22:54:04.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.661+0000 7f9f7f7fe700 1 -- 172.21.15.12:0/3917394992 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f80018660 con 0x7f9f880a5600 2024-05-17T22:54:04.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.662+0000 7f9f8f59e700 1 -- 172.21.15.12:0/3917394992 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f881497e0 con 0x7f9f880a5600 2024-05-17T22:54:04.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.662+0000 7f9f8f59e700 1 -- 172.21.15.12:0/3917394992 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f880ad140 con 0x7f9f880a5600 2024-05-17T22:54:04.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.667+0000 7f9fa3386b80 1 Processor -- start 2024-05-17T22:54:04.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.668+0000 7f9fa3386b80 1 -- start start 2024-05-17T22:54:04.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.668+0000 7f9fa3386b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560141da99f0 0x560141f2d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:04.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.668+0000 7f9fa3386b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560141f2dcb0 0x560141f27c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:04.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.668+0000 7f9fa3386b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560141f28170 0x560141f28580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:04.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.668+0000 7f9fa3386b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x560141e56290 con 0x560141da99f0 2024-05-17T22:54:04.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.668+0000 7f9fa3386b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560141e45fd0 con 0x560141f2dcb0 2024-05-17T22:54:04.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.668+0000 7f9fa3386b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560141e49e70 con 0x560141f28170 2024-05-17T22:54:04.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.668+0000 7f9f8ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560141da99f0 0x560141f2d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:04.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.668+0000 7f9f8dd9b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560141f28170 0x560141f28580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:04.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.668+0000 7f9f8dd9b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560141f28170 0x560141f28580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:52318/0 (socket says 172.21.15.12:52318) 2024-05-17T22:54:04.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.668+0000 7f9f8dd9b700 1 -- 172.21.15.12:0/2425847689 learned_addr learned my addr 172.21.15.12:0/2425847689 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:54:04.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.668+0000 7f9f8ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560141da99f0 0x560141f2d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60634/0 (socket says 172.21.15.12:60634) 2024-05-17T22:54:04.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.669+0000 7f9f8dd9b700 1 -- 172.21.15.12:0/2425847689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560141f2dcb0 msgr2=0x560141f27c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T22:54:04.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.669+0000 7f9f8dd9b700 1 --2- 172.21.15.12:0/2425847689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560141f2dcb0 0x560141f27c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:04.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.669+0000 7f9f8dd9b700 1 -- 172.21.15.12:0/2425847689 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560141da99f0 msgr2=0x560141f2d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:04.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.669+0000 7f9f7f7fe700 1 -- 172.21.15.12:0/3917394992 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9f80018800 con 0x7f9f880a5600 2024-05-17T22:54:04.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.669+0000 7f9f8dd9b700 1 --2- 172.21.15.12:0/2425847689 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560141da99f0 0x560141f2d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:04.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.669+0000 7f9f8dd9b700 1 -- 172.21.15.12:0/2425847689 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560141def590 con 0x560141f28170 2024-05-17T22:54:04.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.669+0000 7f9f8dd9b700 1 --2- 172.21.15.12:0/2425847689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560141f28170 0x560141f28580 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9f90069a80 tx=0x7f9f90069fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4fa383f1b1fea29 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:04.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.670+0000 7f9f7f7fe700 1 --2- 172.21.15.12:0/3917394992 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9f780b2a10 0x7f9f780b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:04.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.671+0000 7f9f7f7fe700 1 -- 172.21.15.12:0/3917394992 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f9f800e23c0 con 0x7f9f880a5600 2024-05-17T22:54:04.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.675+0000 7f9f8ed9d700 1 --2- 172.21.15.12:0/3917394992 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9f780b2a10 0x7f9f780b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:04.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.679+0000 7f9f7cff9700 1 -- 172.21.15.12:0/2425847689 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9007d070 con 0x560141f28170 2024-05-17T22:54:04.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.679+0000 7f9f7cff9700 1 -- 172.21.15.12:0/2425847689 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f90078ab0 con 0x560141f28170 2024-05-17T22:54:04.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.679+0000 7f9f7cff9700 1 -- 172.21.15.12:0/2425847689 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f90077970 con 0x560141f28170 2024-05-17T22:54:04.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.682+0000 7f9fa3386b80 1 -- 172.21.15.12:0/2425847689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560141f28170 msgr2=0x560141f28580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:04.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.682+0000 7f9fa3386b80 1 --2- 172.21.15.12:0/2425847689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560141f28170 0x560141f28580 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9f90069a80 tx=0x7f9f90069fb0 comp rx=0 tx=0).stop 2024-05-17T22:54:04.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.688+0000 7f9fa3386b80 1 -- 172.21.15.12:0/2425847689 shutdown_connections 2024-05-17T22:54:04.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.688+0000 7f9fa3386b80 1 --2- 172.21.15.12:0/2425847689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560141f28170 0x560141f28580 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:04.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.688+0000 7f9fa3386b80 1 --2- 172.21.15.12:0/2425847689 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560141da99f0 0x560141f2d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:04.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.688+0000 7f9fa3386b80 1 --2- 172.21.15.12:0/2425847689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560141f2dcb0 0x560141f27c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:04.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.688+0000 7f9fa3386b80 1 -- 172.21.15.12:0/2425847689 >> 172.21.15.12:0/2425847689 conn(0x560141dedf00 msgr2=0x560141f22b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:04.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.692+0000 7f9f8ed9d700 1 --2- 172.21.15.12:0/3917394992 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9f780b2a10 0x7f9f780b4f20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9f840098e0 tx=0x7f9f8400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:04.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.693+0000 7f9fa3386b80 1 -- 172.21.15.12:0/2425847689 shutdown_connections 2024-05-17T22:54:04.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.694+0000 7f9fa3386b80 1 -- 172.21.15.12:0/2425847689 wait complete. 2024-05-17T22:54:04.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.694+0000 7f9fa3386b80 1 -- start start 2024-05-17T22:54:04.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.694+0000 7f9fa3386b80 1 Processor -- start 2024-05-17T22:54:04.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.701+0000 7f9fa3386b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560141da99f0 0x560141f31360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:04.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.701+0000 7f9fa3386b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560141f28170 0x560141f318a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:04.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.701+0000 7f9fa3386b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560141f2dcb0 0x560141f31de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:04.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.701+0000 7f9fa3386b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x560141d4d150 con 0x560141f28170 2024-05-17T22:54:04.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.701+0000 7f9fa3386b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560141de6e80 con 0x560141f2dcb0 2024-05-17T22:54:04.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.701+0000 7f9fa3386b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560141f345a0 con 0x560141da99f0 2024-05-17T22:54:04.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.701+0000 7f9fa3386b80 10 client.0 _ll_register_callbacks cb 0x560141de0f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:54:04.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.702+0000 7f9f8dd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560141f28170 0x560141f318a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:04.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.702+0000 7f9f8dd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560141f28170 0x560141f318a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60638/0 (socket says 172.21.15.12:60638) 2024-05-17T22:54:04.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.702+0000 7f9f8dd9b700 1 -- 172.21.15.12:0/663233005 learned_addr learned my addr 172.21.15.12:0/663233005 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:54:04.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.704+0000 7f9f8e59c700 1 --2- 172.21.15.12:0/663233005 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560141da99f0 0x560141f31360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:04.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.705+0000 7f9f8dd9b700 1 -- 172.21.15.12:0/663233005 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560141da99f0 msgr2=0x560141f31360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:04.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.705+0000 7f9f8dd9b700 1 --2- 172.21.15.12:0/663233005 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560141da99f0 0x560141f31360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:04.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.705+0000 7f9f8dd9b700 1 -- 172.21.15.12:0/663233005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560141f2dcb0 msgr2=0x560141f31de0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T22:54:04.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.705+0000 7f9f8dd9b700 1 --2- 172.21.15.12:0/663233005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560141f2dcb0 0x560141f31de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:04.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.705+0000 7f9f8dd9b700 1 -- 172.21.15.12:0/663233005 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560141d96480 con 0x560141f28170 2024-05-17T22:54:04.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.710+0000 7f9f8dd9b700 1 --2- 172.21.15.12:0/663233005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560141f28170 0x560141f318a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f9f900694c0 tx=0x7f9f900786e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27a0012fd4dc522c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:04.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.710+0000 7f9f75ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:54:04.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.711+0000 7f9f75ffb700 1 -- 172.21.15.12:0/663233005 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f90088060 con 0x560141f28170 2024-05-17T22:54:04.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.711+0000 7f9fa3386b80 1 -- 172.21.15.12:0/663233005 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f90078ab0 con 0x560141f28170 2024-05-17T22:54:04.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.711+0000 7f9f75ffb700 1 -- 172.21.15.12:0/663233005 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f9f9006aae0 con 0x560141f28170 2024-05-17T22:54:04.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.712+0000 7f9f75ffb700 1 -- 172.21.15.12:0/663233005 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f9f9008f050 con 0x560141f28170 2024-05-17T22:54:04.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.716+0000 7f9fa3386b80 10 client.16566 fetch_fsmap learned FSMap version 42 2024-05-17T22:54:04.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.716+0000 7f9fa3386b80 1 -- 172.21.15.12:0/663233005 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x560141f3ef60 con 0x560141f28170 2024-05-17T22:54:04.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.718+0000 7f9f75ffb700 1 -- 172.21.15.12:0/663233005 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f90078ab0 con 0x560141f28170 2024-05-17T22:54:04.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.718+0000 7f9f75ffb700 1 -- 172.21.15.12:0/663233005 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 42) ==== 8350+0+0 (secure 0 0 0) 0x7f9f90083470 con 0x560141f28170 2024-05-17T22:54:04.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.718+0000 7f9fa3386b80 10 client.16566 fetch_fsmap finished waiting for FSMap version 42 2024-05-17T22:54:04.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.718+0000 7f9fa3386b80 10 client.16566 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-17T22:54:04.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.718+0000 7f9fa3386b80 20 client.16566 populate_metadata read hostname 'smithi012' 2024-05-17T22:54:04.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.718+0000 7f9fa3386b80 1 --2- 172.21.15.12:0/663233005 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x560141f41870 0x560141f43ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:04.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.718+0000 7f9fa3386b80 4 client.16566 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:54:04.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.718+0000 7f9fa3386b80 1 -- 172.21.15.12:0/663233005 --> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560141e581e0 con 0x560141f41870 2024-05-17T22:54:04.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.725+0000 7f9f8e59c700 1 --2- 172.21.15.12:0/663233005 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x560141f41870 0x560141f43ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:04.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.729+0000 7f9f8e59c700 1 --2- 172.21.15.12:0/663233005 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x560141f41870 0x560141f43ce0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2f6bf861f550a645 server_cookie=a8f3a423836e5741 in_seq=0 out_seq=0 2024-05-17T22:54:04.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.729+0000 7f9f75ffb700 10 client.16566 ms_handle_connect on v2:172.21.15.96:6836/4222574338 2024-05-17T22:54:04.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.731+0000 7f9f75ffb700 1 -- 172.21.15.12:0/663233005 <== mds.1 v2:172.21.15.96:6836/4222574338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560141e581e0 con 0x560141f41870 2024-05-17T22:54:04.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.731+0000 7f9f75ffb700 10 client.16566 handle_command_reply: tid=0 2024-05-17T22:54:04.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.731+0000 7f9fa3386b80 1 client.16566 shutdown 2024-05-17T22:54:04.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.738+0000 7f9fa3386b80 1 -- 172.21.15.12:0/663233005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560141f28170 msgr2=0x560141f318a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:04.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.738+0000 7f9fa3386b80 1 --2- 172.21.15.12:0/663233005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560141f28170 0x560141f318a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f9f900694c0 tx=0x7f9f900786e0 comp rx=0 tx=0).stop 2024-05-17T22:54:04.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.739+0000 7f9fa3386b80 1 -- 172.21.15.12:0/663233005 shutdown_connections 2024-05-17T22:54:04.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.739+0000 7f9fa3386b80 1 --2- 172.21.15.12:0/663233005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560141f2dcb0 0x560141f31de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:04.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.739+0000 7f9fa3386b80 1 --2- 172.21.15.12:0/663233005 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560141da99f0 0x560141f31360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:04.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.739+0000 7f9fa3386b80 1 --2- 172.21.15.12:0/663233005 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x560141f41870 0x560141f43ce0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:04.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.739+0000 7f9fa3386b80 1 --2- 172.21.15.12:0/663233005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560141f28170 0x560141f318a0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:04.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.739+0000 7f9fa3386b80 1 -- 172.21.15.12:0/663233005 >> 172.21.15.12:0/663233005 conn(0x560141dedf00 msgr2=0x560141f22110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:04.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.739+0000 7f9f75ffb700 0 client.16566 ms_handle_reset on v2:172.21.15.96:6836/4222574338 2024-05-17T22:54:04.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.743+0000 7f9fa3386b80 1 -- 172.21.15.12:0/663233005 shutdown_connections 2024-05-17T22:54:04.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.745+0000 7f9fa3386b80 1 -- 172.21.15.12:0/663233005 wait complete. 2024-05-17T22:54:04.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.745+0000 7f9fa3386b80 20 client.16566 trim_cache size 0 max 16384 2024-05-17T22:54:04.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.777+0000 7f9fa3386b80 1 Processor -- start 2024-05-17T22:54:04.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.779+0000 7f9fa3386b80 1 -- start start 2024-05-17T22:54:04.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.780+0000 7f9fa3386b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560141da99f0 0x560141f27c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:04.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.780+0000 7f9fa3386b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560141f28170 0x560141f3d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:04.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.780+0000 7f9fa3386b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560141f2dcb0 0x560141f2ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:04.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.780+0000 7f9fa3386b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9f88164e10 con 0x560141da99f0 2024-05-17T22:54:04.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.780+0000 7f9fa3386b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9f881626a0 con 0x560141f2dcb0 2024-05-17T22:54:04.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.780+0000 7f9fa3386b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9f880b3ea0 con 0x560141f28170 2024-05-17T22:54:04.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.781+0000 7f9f8dd9b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560141f28170 0x560141f3d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:04.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.781+0000 7f9f8dd9b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560141f28170 0x560141f3d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:52336/0 (socket says 172.21.15.12:52336) 2024-05-17T22:54:04.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.781+0000 7f9f8dd9b700 1 -- 172.21.15.12:0/1835949434 learned_addr learned my addr 172.21.15.12:0/1835949434 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:54:04.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.783+0000 7f9f8e59c700 1 --2- 172.21.15.12:0/1835949434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560141da99f0 0x560141f27c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:04.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.784+0000 7f9f8ed9d700 1 --2- 172.21.15.12:0/1835949434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560141f2dcb0 0x560141f2ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:04.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.784+0000 7f9f8e59c700 1 -- 172.21.15.12:0/1835949434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560141f28170 msgr2=0x560141f3d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:04.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.784+0000 7f9f8e59c700 1 --2- 172.21.15.12:0/1835949434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560141f28170 0x560141f3d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:04.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.784+0000 7f9f8e59c700 1 -- 172.21.15.12:0/1835949434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560141f2dcb0 msgr2=0x560141f2ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:04.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.784+0000 7f9f8e59c700 1 --2- 172.21.15.12:0/1835949434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560141f2dcb0 0x560141f2ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:04.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.785+0000 7f9f8e59c700 1 -- 172.21.15.12:0/1835949434 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560141f2d2e0 con 0x560141da99f0 2024-05-17T22:54:04.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.785+0000 7f9f8e59c700 1 --2- 172.21.15.12:0/1835949434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560141da99f0 0x560141f27c30 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x560141f2d240 tx=0x7f9f80005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=841a669d3226536e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:04.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.785+0000 7f9f77fff700 1 -- 172.21.15.12:0/1835949434 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f800c7020 con 0x560141da99f0 2024-05-17T22:54:04.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.785+0000 7f9f77fff700 1 -- 172.21.15.12:0/1835949434 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f800c2070 con 0x560141da99f0 2024-05-17T22:54:04.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.785+0000 7f9f77fff700 1 -- 172.21.15.12:0/1835949434 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f800a98b0 con 0x560141da99f0 2024-05-17T22:54:04.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.788+0000 7f9fa3386b80 1 -- 172.21.15.12:0/1835949434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560141da99f0 msgr2=0x560141f27c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:04.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.788+0000 7f9fa3386b80 1 --2- 172.21.15.12:0/1835949434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560141da99f0 0x560141f27c30 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x560141f2d240 tx=0x7f9f80005df0 comp rx=0 tx=0).stop 2024-05-17T22:54:04.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.789+0000 7f9fa3386b80 1 -- 172.21.15.12:0/1835949434 shutdown_connections 2024-05-17T22:54:04.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.789+0000 7f9fa3386b80 1 --2- 172.21.15.12:0/1835949434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560141f2dcb0 0x560141f2ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:04.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.789+0000 7f9fa3386b80 1 --2- 172.21.15.12:0/1835949434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560141da99f0 0x560141f27c30 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:04.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.789+0000 7f9fa3386b80 1 --2- 172.21.15.12:0/1835949434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560141f28170 0x560141f3d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:04.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.789+0000 7f9fa3386b80 1 -- 172.21.15.12:0/1835949434 >> 172.21.15.12:0/1835949434 conn(0x560141dedf00 msgr2=0x560141f2c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:04.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.794+0000 7f9fa3386b80 1 -- 172.21.15.12:0/1835949434 shutdown_connections 2024-05-17T22:54:04.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.794+0000 7f9fa3386b80 1 -- 172.21.15.12:0/1835949434 wait complete. 2024-05-17T22:54:04.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.794+0000 7f9fa3386b80 1 -- start start 2024-05-17T22:54:04.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.795+0000 7f9fa3386b80 1 Processor -- start 2024-05-17T22:54:04.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.795+0000 7f9fa3386b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560141da99f0 0x560141f393d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:04.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.796+0000 7f9fa3386b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560141f28170 0x560141f39910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:04.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.796+0000 7f9fa3386b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560141f2dcb0 0x560141f308c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:04.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.796+0000 7f9fa3386b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9f880b4510 con 0x560141f28170 2024-05-17T22:54:04.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.796+0000 7f9fa3386b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9f88161fd0 con 0x560141da99f0 2024-05-17T22:54:04.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.796+0000 7f9fa3386b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9f880b4be0 con 0x560141f2dcb0 2024-05-17T22:54:04.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.796+0000 7f9fa3386b80 10 client.0 _ll_register_callbacks cb 0x560141de0f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:54:04.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.797+0000 7f9f8dd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560141f28170 0x560141f39910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:04.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.797+0000 7f9f8dd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560141f28170 0x560141f39910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60666/0 (socket says 172.21.15.12:60666) 2024-05-17T22:54:04.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.797+0000 7f9f8dd9b700 1 -- 172.21.15.12:0/118768029 learned_addr learned my addr 172.21.15.12:0/118768029 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:54:04.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.798+0000 7f9f8e59c700 1 --2- 172.21.15.12:0/118768029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560141da99f0 0x560141f393d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:04.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.798+0000 7f9f8ed9d700 1 --2- 172.21.15.12:0/118768029 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560141f2dcb0 0x560141f308c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:04.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.799+0000 7f9f8dd9b700 1 -- 172.21.15.12:0/118768029 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560141f2dcb0 msgr2=0x560141f308c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:04.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.799+0000 7f9f8dd9b700 1 --2- 172.21.15.12:0/118768029 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560141f2dcb0 0x560141f308c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:04.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.799+0000 7f9f8dd9b700 1 -- 172.21.15.12:0/118768029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560141da99f0 msgr2=0x560141f393d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:04.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.799+0000 7f9f8dd9b700 1 --2- 172.21.15.12:0/118768029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560141da99f0 0x560141f393d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:04.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.799+0000 7f9f8dd9b700 1 -- 172.21.15.12:0/118768029 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560141f32390 con 0x560141f28170 2024-05-17T22:54:04.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.799+0000 7f9f8dd9b700 1 --2- 172.21.15.12:0/118768029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560141f28170 0x560141f39910 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9f900694c0 tx=0x7f9f90078b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c60c805ba54f4896 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:04.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.799+0000 7f9f75ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:54:04.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.800+0000 7f9f75ffb700 1 -- 172.21.15.12:0/118768029 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f90097070 con 0x560141f28170 2024-05-17T22:54:04.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.800+0000 7f9fa3386b80 1 -- 172.21.15.12:0/118768029 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f800c2070 con 0x560141f28170 2024-05-17T22:54:04.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.800+0000 7f9f75ffb700 1 -- 172.21.15.12:0/118768029 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f9f90093430 con 0x560141f28170 2024-05-17T22:54:04.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.800+0000 7f9f75ffb700 1 -- 172.21.15.12:0/118768029 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f9f9009f050 con 0x560141f28170 2024-05-17T22:54:04.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.801+0000 7f9fa3386b80 10 client.16572 fetch_fsmap learned FSMap version 42 2024-05-17T22:54:04.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.801+0000 7f9fa3386b80 1 -- 172.21.15.12:0/118768029 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x560141fab570 con 0x560141f28170 2024-05-17T22:54:04.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.801+0000 7f9f75ffb700 1 -- 172.21.15.12:0/118768029 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f800c2070 con 0x560141f28170 2024-05-17T22:54:04.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.801+0000 7f9f75ffb700 1 -- 172.21.15.12:0/118768029 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 42) ==== 8350+0+0 (secure 0 0 0) 0x7f9f9009a030 con 0x560141f28170 2024-05-17T22:54:04.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.801+0000 7f9fa3386b80 10 client.16572 fetch_fsmap finished waiting for FSMap version 42 2024-05-17T22:54:04.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.801+0000 7f9fa3386b80 10 client.16572 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-17T22:54:04.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.801+0000 7f9fa3386b80 20 client.16572 populate_metadata read hostname 'smithi012' 2024-05-17T22:54:04.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.801+0000 7f9fa3386b80 1 --2- 172.21.15.12:0/118768029 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x560141fabb80 0x560141fae030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:04.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.801+0000 7f9fa3386b80 4 client.16572 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:54:04.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.801+0000 7f9fa3386b80 1 -- 172.21.15.12:0/118768029 --> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560141e519c0 con 0x560141fabb80 2024-05-17T22:54:04.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.802+0000 7f9f8e59c700 1 --2- 172.21.15.12:0/118768029 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x560141fabb80 0x560141fae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:04.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.813+0000 7f9f8e59c700 1 --2- 172.21.15.12:0/118768029 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x560141fabb80 0x560141fae030 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=41deaf037deeabad server_cookie=10b65b142551e293 in_seq=0 out_seq=0 2024-05-17T22:54:04.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.813+0000 7f9f75ffb700 10 client.16572 ms_handle_connect on v2:172.21.15.96:6836/4222574338 2024-05-17T22:54:04.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.817+0000 7f9f75ffb700 1 -- 172.21.15.12:0/118768029 <== mds.1 v2:172.21.15.96:6836/4222574338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560141e519c0 con 0x560141fabb80 2024-05-17T22:54:04.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.817+0000 7f9f75ffb700 10 client.16572 handle_command_reply: tid=0 2024-05-17T22:54:04.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.817+0000 7f9fa3386b80 1 client.16572 shutdown 2024-05-17T22:54:04.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.823+0000 7f9fa3386b80 1 -- 172.21.15.12:0/118768029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560141f28170 msgr2=0x560141f39910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:04.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.823+0000 7f9fa3386b80 1 --2- 172.21.15.12:0/118768029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560141f28170 0x560141f39910 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9f900694c0 tx=0x7f9f90078b20 comp rx=0 tx=0).stop 2024-05-17T22:54:04.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.826+0000 7f9fa3386b80 1 -- 172.21.15.12:0/118768029 shutdown_connections 2024-05-17T22:54:04.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.826+0000 7f9fa3386b80 1 --2- 172.21.15.12:0/118768029 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560141f2dcb0 0x560141f308c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:04.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.826+0000 7f9fa3386b80 1 --2- 172.21.15.12:0/118768029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560141da99f0 0x560141f393d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:04.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.826+0000 7f9fa3386b80 1 --2- 172.21.15.12:0/118768029 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x560141fabb80 0x560141fae030 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:04.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.826+0000 7f9fa3386b80 1 --2- 172.21.15.12:0/118768029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560141f28170 0x560141f39910 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:04.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.826+0000 7f9fa3386b80 1 -- 172.21.15.12:0/118768029 >> 172.21.15.12:0/118768029 conn(0x560141dedf00 msgr2=0x560141f2cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:04.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.826+0000 7f9f75ffb700 0 client.16572 ms_handle_reset on v2:172.21.15.96:6836/4222574338 2024-05-17T22:54:04.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.828+0000 7f9fa3386b80 1 -- 172.21.15.12:0/118768029 shutdown_connections 2024-05-17T22:54:04.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.828+0000 7f9fa3386b80 1 -- 172.21.15.12:0/118768029 wait complete. 2024-05-17T22:54:04.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.832+0000 7f9fa3386b80 20 client.16572 trim_cache size 0 max 16384 2024-05-17T22:54:04.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.833+0000 7f9f8f59e700 1 -- 172.21.15.12:0/3917394992 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9f780b2a10 msgr2=0x7f9f780b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:04.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.833+0000 7f9f8f59e700 1 --2- 172.21.15.12:0/3917394992 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9f780b2a10 0x7f9f780b4f20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9f840098e0 tx=0x7f9f8400b540 comp rx=0 tx=0).stop 2024-05-17T22:54:04.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.833+0000 7f9f8f59e700 1 -- 172.21.15.12:0/3917394992 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f880a5600 msgr2=0x7f9f88144210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:04.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.833+0000 7f9f8f59e700 1 --2- 172.21.15.12:0/3917394992 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f880a5600 0x7f9f88144210 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9f8000c920 tx=0x7f9f8000cce0 comp rx=0 tx=0).stop 2024-05-17T22:54:04.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.833+0000 7f9f8f59e700 1 -- 172.21.15.12:0/3917394992 shutdown_connections 2024-05-17T22:54:04.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.833+0000 7f9f8f59e700 1 --2- 172.21.15.12:0/3917394992 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9f780b2a10 0x7f9f780b4f20 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:04.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.833+0000 7f9f8f59e700 1 --2- 172.21.15.12:0/3917394992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f88148a40 0x7f9f88148e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:04.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.833+0000 7f9f8f59e700 1 --2- 172.21.15.12:0/3917394992 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f880a5600 0x7f9f88144210 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:04.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.833+0000 7f9f8f59e700 1 --2- 172.21.15.12:0/3917394992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f880b7c10 0x7f9f881447b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:04.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.833+0000 7f9f8f59e700 1 -- 172.21.15.12:0/3917394992 >> 172.21.15.12:0/3917394992 conn(0x7f9f8801aa00 msgr2=0x7f9f880b5740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:04.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.834+0000 7f9f8f59e700 1 -- 172.21.15.12:0/3917394992 shutdown_connections 2024-05-17T22:54:04.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:04.834+0000 7f9f8f59e700 1 -- 172.21.15.12:0/3917394992 wait complete. 2024-05-17T22:54:05.633 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-17T22:54:06.191 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:54:06.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 885.0 s, 410.2 tps, lat 239.690 ms stddev 295.166 2024-05-17T22:54:06.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:06 smithi151 ceph-mon[36101]: pgmap v884: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.7 MiB/s wr, 510 op/s 2024-05-17T22:54:06.502 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:06 smithi012 ceph-mon[31632]: pgmap v884: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.7 MiB/s wr, 510 op/s 2024-05-17T22:54:06.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:06 smithi096 ceph-mon[36065]: pgmap v884: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.7 MiB/s wr, 510 op/s 2024-05-17T22:54:08.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.234+0000 7f475d801700 1 -- 172.21.15.12:0/777276375 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f475811d550 msgr2=0x7f475811f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:08.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.234+0000 7f475d801700 1 --2- 172.21.15.12:0/777276375 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f475811d550 0x7f475811f970 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4750009f90 tx=0x7f475000b330 comp rx=0 tx=0).stop 2024-05-17T22:54:08.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.238+0000 7f475d801700 1 -- 172.21.15.12:0/777276375 shutdown_connections 2024-05-17T22:54:08.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.238+0000 7f475d801700 1 --2- 172.21.15.12:0/777276375 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f475811d550 0x7f475811f970 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:08.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.238+0000 7f475d801700 1 --2- 172.21.15.12:0/777276375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f475810a830 0x7f475810ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:08.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.239+0000 7f475d801700 1 --2- 172.21.15.12:0/777276375 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f475810b180 0x7f475811d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:08.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.239+0000 7f475d801700 1 -- 172.21.15.12:0/777276375 >> 172.21.15.12:0/777276375 conn(0x7f475806efc0 msgr2=0x7f475806f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:08.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.245+0000 7f475d801700 1 -- 172.21.15.12:0/777276375 shutdown_connections 2024-05-17T22:54:08.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.248+0000 7f475d801700 1 -- 172.21.15.12:0/777276375 wait complete. 2024-05-17T22:54:08.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.250+0000 7f475d801700 1 Processor -- start 2024-05-17T22:54:08.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.252+0000 7f475d801700 1 -- start start 2024-05-17T22:54:08.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.253+0000 7f475d801700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f475810a830 0x7f47581a9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:08.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.253+0000 7f475d801700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f475810b180 0x7f47581a9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:08.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.253+0000 7f475d801700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f475811d550 0x7f47581add40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:08.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.253+0000 7f475d801700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4758122c90 con 0x7f475811d550 2024-05-17T22:54:08.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.253+0000 7f475d801700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4758122b10 con 0x7f475810b180 2024-05-17T22:54:08.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.254+0000 7f475d801700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4758122e10 con 0x7f475810a830 2024-05-17T22:54:08.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.255+0000 7f47567fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f475810b180 0x7f47581a9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:08.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.255+0000 7f47567fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f475810b180 0x7f47581a9ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:33950/0 (socket says 172.21.15.12:33950) 2024-05-17T22:54:08.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.255+0000 7f47567fc700 1 -- 172.21.15.12:0/548149131 learned_addr learned my addr 172.21.15.12:0/548149131 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:54:08.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.257+0000 7f47567fc700 1 -- 172.21.15.12:0/548149131 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f475810a830 msgr2=0x7f47581a9510 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-17T22:54:08.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.257+0000 7f47567fc700 1 --2- 172.21.15.12:0/548149131 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f475810a830 0x7f47581a9510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:08.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.257+0000 7f47567fc700 1 -- 172.21.15.12:0/548149131 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f475811d550 msgr2=0x7f47581add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:08.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.257+0000 7f47567fc700 1 --2- 172.21.15.12:0/548149131 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f475811d550 0x7f47581add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:08.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.257+0000 7f47567fc700 1 -- 172.21.15.12:0/548149131 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47581ae3e0 con 0x7f475810b180 2024-05-17T22:54:08.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.262+0000 7f47567fc700 1 --2- 172.21.15.12:0/548149131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f475810b180 0x7f47581a9ab0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f474c00d980 tx=0x7f474c00dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:08.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.264+0000 7f473ffff700 1 -- 172.21.15.12:0/548149131 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c016070 con 0x7f475810b180 2024-05-17T22:54:08.270 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:08 smithi012 ceph-mon[31632]: pgmap v885: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.9 MiB/s wr, 437 op/s 2024-05-17T22:54:08.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.264+0000 7f473ffff700 1 -- 172.21.15.12:0/548149131 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f474c00de60 con 0x7f475810b180 2024-05-17T22:54:08.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.264+0000 7f473ffff700 1 -- 172.21.15.12:0/548149131 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c00f3c0 con 0x7f475810b180 2024-05-17T22:54:08.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.267+0000 7f475d801700 1 -- 172.21.15.12:0/548149131 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47581ae6d0 con 0x7f475810b180 2024-05-17T22:54:08.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.267+0000 7f475d801700 1 -- 172.21.15.12:0/548149131 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47581b5ff0 con 0x7f475810b180 2024-05-17T22:54:08.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.278+0000 7f476c1dcb80 1 Processor -- start 2024-05-17T22:54:08.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.279+0000 7f473ffff700 1 -- 172.21.15.12:0/548149131 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f474c019070 con 0x7f475810b180 2024-05-17T22:54:08.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.280+0000 7f473ffff700 1 --2- 172.21.15.12:0/548149131 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f47400b2a50 0x7f47400b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:08.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.280+0000 7f473ffff700 1 -- 172.21.15.12:0/548149131 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f474c0e1f20 con 0x7f475810b180 2024-05-17T22:54:08.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.281+0000 7f476c1dcb80 1 -- start start 2024-05-17T22:54:08.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.281+0000 7f476c1dcb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5615545159f0 0x561554698ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:08.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.282+0000 7f476c1dcb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561554699400 0x5615546935c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:08.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.282+0000 7f476c1dcb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561554693b00 0x561554693f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:08.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.282+0000 7f476c1dcb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5615545c20d0 con 0x561554693b00 2024-05-17T22:54:08.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.282+0000 7f476c1dcb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5615545b1c70 con 0x561554699400 2024-05-17T22:54:08.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.282+0000 7f476c1dcb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5615545b5cb0 con 0x5615545159f0 2024-05-17T22:54:08.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.282+0000 7f47577fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561554693b00 0x561554693f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:08.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.282+0000 7f47577fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561554693b00 0x561554693f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60698/0 (socket says 172.21.15.12:60698) 2024-05-17T22:54:08.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.282+0000 7f47577fe700 1 -- 172.21.15.12:0/1351600529 learned_addr learned my addr 172.21.15.12:0/1351600529 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:54:08.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.282+0000 7f47577fe700 1 --2- 172.21.15.12:0/548149131 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f47400b2a50 0x7f47400b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:08.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.283+0000 7f47577fe700 1 -- 172.21.15.12:0/1351600529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5615545159f0 msgr2=0x561554698ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T22:54:08.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.283+0000 7f47577fe700 1 --2- 172.21.15.12:0/1351600529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5615545159f0 0x561554698ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:08.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.283+0000 7f47577fe700 1 -- 172.21.15.12:0/1351600529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561554699400 msgr2=0x5615546935c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:08.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.283+0000 7f47577fe700 1 --2- 172.21.15.12:0/1351600529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561554699400 0x5615546935c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:08.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.283+0000 7f47577fe700 1 -- 172.21.15.12:0/1351600529 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56155454b5c0 con 0x561554693b00 2024-05-17T22:54:08.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.283+0000 7f47577fe700 1 --2- 172.21.15.12:0/548149131 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f47400b2a50 0x7f47400b4f60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4750007620 tx=0x7f4750026000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:08.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.284+0000 7f47577fe700 1 --2- 172.21.15.12:0/1351600529 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561554693b00 0x561554693f10 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f475000cfd0 tx=0x7f4750007500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58644a78df1ad33e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:08.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.285+0000 7f473d7fa700 1 -- 172.21.15.12:0/1351600529 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475000cac0 con 0x561554693b00 2024-05-17T22:54:08.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.285+0000 7f473d7fa700 1 -- 172.21.15.12:0/1351600529 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4750020810 con 0x561554693b00 2024-05-17T22:54:08.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.285+0000 7f473d7fa700 1 -- 172.21.15.12:0/1351600529 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4750005330 con 0x561554693b00 2024-05-17T22:54:08.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.287+0000 7f476c1dcb80 1 -- 172.21.15.12:0/1351600529 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561554693b00 msgr2=0x561554693f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:08.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.287+0000 7f476c1dcb80 1 --2- 172.21.15.12:0/1351600529 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561554693b00 0x561554693f10 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f475000cfd0 tx=0x7f4750007500 comp rx=0 tx=0).stop 2024-05-17T22:54:08.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.288+0000 7f476c1dcb80 1 -- 172.21.15.12:0/1351600529 shutdown_connections 2024-05-17T22:54:08.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.288+0000 7f476c1dcb80 1 --2- 172.21.15.12:0/1351600529 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561554693b00 0x561554693f10 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:08.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.288+0000 7f476c1dcb80 1 --2- 172.21.15.12:0/1351600529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5615545159f0 0x561554698ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:08.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.288+0000 7f476c1dcb80 1 --2- 172.21.15.12:0/1351600529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561554699400 0x5615546935c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:08.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.288+0000 7f476c1dcb80 1 -- 172.21.15.12:0/1351600529 >> 172.21.15.12:0/1351600529 conn(0x5615545757d0 msgr2=0x56155468e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:08.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.296+0000 7f476c1dcb80 1 -- 172.21.15.12:0/1351600529 shutdown_connections 2024-05-17T22:54:08.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.299+0000 7f476c1dcb80 1 -- 172.21.15.12:0/1351600529 wait complete. 2024-05-17T22:54:08.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.299+0000 7f476c1dcb80 1 -- start start 2024-05-17T22:54:08.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.300+0000 7f476c1dcb80 1 Processor -- start 2024-05-17T22:54:08.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.300+0000 7f476c1dcb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615545159f0 0x56155469cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:08.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.300+0000 7f476c1dcb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561554693b00 0x56155469d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:08.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.301+0000 7f476c1dcb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561554699400 0x56155469d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:08.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.301+0000 7f47567fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561554693b00 0x56155469d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:08.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.301+0000 7f47567fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561554693b00 0x56155469d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:33958/0 (socket says 172.21.15.12:33958) 2024-05-17T22:54:08.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.301+0000 7f47567fc700 1 -- 172.21.15.12:0/3613810625 learned_addr learned my addr 172.21.15.12:0/3613810625 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:54:08.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.301+0000 7f47577fe700 1 --2- 172.21.15.12:0/3613810625 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561554699400 0x56155469d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:08.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.301+0000 7f476c1dcb80 1 -- 172.21.15.12:0/3613810625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5615544b9150 con 0x5615545159f0 2024-05-17T22:54:08.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.301+0000 7f476c1dcb80 1 -- 172.21.15.12:0/3613810625 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561554553940 con 0x561554693b00 2024-05-17T22:54:08.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.301+0000 7f476c1dcb80 1 -- 172.21.15.12:0/3613810625 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5615546b0d20 con 0x561554699400 2024-05-17T22:54:08.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.301+0000 7f476c1dcb80 10 client.0 _ll_register_callbacks cb 0x561554548b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:54:08.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.302+0000 7f4756ffd700 1 --2- 172.21.15.12:0/3613810625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615545159f0 0x56155469cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:08.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.307+0000 7f47577fe700 1 -- 172.21.15.12:0/3613810625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561554693b00 msgr2=0x56155469d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:08.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.307+0000 7f47577fe700 1 --2- 172.21.15.12:0/3613810625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561554693b00 0x56155469d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:08.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.307+0000 7f47577fe700 1 -- 172.21.15.12:0/3613810625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615545159f0 msgr2=0x56155469cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:08.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.307+0000 7f47577fe700 1 --2- 172.21.15.12:0/3613810625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615545159f0 0x56155469cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:08.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.307+0000 7f47577fe700 1 -- 172.21.15.12:0/3613810625 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561554502480 con 0x561554699400 2024-05-17T22:54:08.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.311+0000 7f47577fe700 1 --2- 172.21.15.12:0/3613810625 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561554699400 0x56155469d640 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f475000cfd0 tx=0x7f4750020f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc039a23b0eecf0c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:08.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.313+0000 7f473aff5700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:54:08.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.313+0000 7f473aff5700 1 -- 172.21.15.12:0/3613810625 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4750038020 con 0x561554699400 2024-05-17T22:54:08.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.313+0000 7f473aff5700 1 -- 172.21.15.12:0/3613810625 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f475002a440 con 0x561554699400 2024-05-17T22:54:08.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.314+0000 7f476c1dcb80 1 -- 172.21.15.12:0/3613810625 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4750020810 con 0x561554699400 2024-05-17T22:54:08.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.319+0000 7f473aff5700 1 -- 172.21.15.12:0/3613810625 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f475000cac0 con 0x561554699400 2024-05-17T22:54:08.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.319+0000 7f476c1dcb80 10 client.26218 fetch_fsmap learned FSMap version 42 2024-05-17T22:54:08.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.319+0000 7f476c1dcb80 1 -- 172.21.15.12:0/3613810625 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5615546bb700 con 0x561554699400 2024-05-17T22:54:08.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.319+0000 7f473aff5700 1 -- 172.21.15.12:0/3613810625 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4750020810 con 0x561554699400 2024-05-17T22:54:08.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.321+0000 7f473aff5700 1 -- 172.21.15.12:0/3613810625 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 42) ==== 8350+0+0 (secure 0 0 0) 0x7f4750011070 con 0x561554699400 2024-05-17T22:54:08.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.321+0000 7f476c1dcb80 10 client.26218 fetch_fsmap finished waiting for FSMap version 42 2024-05-17T22:54:08.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.321+0000 7f476c1dcb80 10 client.26218 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:54:08.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.321+0000 7f476c1dcb80 20 client.26218 populate_metadata read hostname 'smithi012' 2024-05-17T22:54:08.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.321+0000 7f476c1dcb80 1 --2- 172.21.15.12:0/3613810625 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5615546bdd40 0x5615546c0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:08.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.321+0000 7f476c1dcb80 4 client.26218 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:54:08.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.321+0000 7f476c1dcb80 1 -- 172.21.15.12:0/3613810625 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615545c4020 con 0x5615546bdd40 2024-05-17T22:54:08.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.321+0000 7f4756ffd700 1 --2- 172.21.15.12:0/3613810625 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5615546bdd40 0x5615546c0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:08.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.325+0000 7f4756ffd700 1 --2- 172.21.15.12:0/3613810625 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5615546bdd40 0x5615546c0190 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5064a2c6bc8ece server_cookie=bd14dca4d4f305d6 in_seq=0 out_seq=0 2024-05-17T22:54:08.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.325+0000 7f473aff5700 10 client.26218 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:54:08.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.325+0000 7f473aff5700 1 -- 172.21.15.12:0/3613810625 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5615545c4020 con 0x5615546bdd40 2024-05-17T22:54:08.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.325+0000 7f473aff5700 10 client.26218 handle_command_reply: tid=0 2024-05-17T22:54:08.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.325+0000 7f476c1dcb80 1 client.26218 shutdown 2024-05-17T22:54:08.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.333+0000 7f476c1dcb80 1 -- 172.21.15.12:0/3613810625 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561554699400 msgr2=0x56155469d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:08.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.333+0000 7f476c1dcb80 1 --2- 172.21.15.12:0/3613810625 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561554699400 0x56155469d640 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f475000cfd0 tx=0x7f4750020f90 comp rx=0 tx=0).stop 2024-05-17T22:54:08.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.333+0000 7f476c1dcb80 1 -- 172.21.15.12:0/3613810625 shutdown_connections 2024-05-17T22:54:08.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.333+0000 7f476c1dcb80 1 --2- 172.21.15.12:0/3613810625 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561554699400 0x56155469d640 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:08.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.333+0000 7f476c1dcb80 1 --2- 172.21.15.12:0/3613810625 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5615546bdd40 0x5615546c0190 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:08.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.333+0000 7f476c1dcb80 1 --2- 172.21.15.12:0/3613810625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615545159f0 0x56155469cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:08.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.333+0000 7f476c1dcb80 1 --2- 172.21.15.12:0/3613810625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561554693b00 0x56155469d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:08.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.333+0000 7f476c1dcb80 1 -- 172.21.15.12:0/3613810625 >> 172.21.15.12:0/3613810625 conn(0x5615545757d0 msgr2=0x56155468e160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:08.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.333+0000 7f473aff5700 0 client.26218 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:54:08.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.334+0000 7f476c1dcb80 1 -- 172.21.15.12:0/3613810625 shutdown_connections 2024-05-17T22:54:08.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.334+0000 7f476c1dcb80 1 -- 172.21.15.12:0/3613810625 wait complete. 2024-05-17T22:54:08.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.334+0000 7f476c1dcb80 20 client.26218 trim_cache size 0 max 16384 2024-05-17T22:54:08.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.367+0000 7f476c1dcb80 1 Processor -- start 2024-05-17T22:54:08.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.367+0000 7f476c1dcb80 1 -- start start 2024-05-17T22:54:08.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.368+0000 7f476c1dcb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5615545159f0 0x5615546935c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:08.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.368+0000 7f476c1dcb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561554693b00 0x5615546971b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:08.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.368+0000 7f476c1dcb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561554699400 0x56155469bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:08.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.368+0000 7f476c1dcb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f47581b8b40 con 0x561554693b00 2024-05-17T22:54:08.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.368+0000 7f476c1dcb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f47581b63d0 con 0x561554699400 2024-05-17T22:54:08.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.368+0000 7f476c1dcb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4758119110 con 0x5615545159f0 2024-05-17T22:54:08.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.368+0000 7f47577fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561554699400 0x56155469bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:08.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.368+0000 7f47567fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561554693b00 0x5615546971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:08.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.369+0000 7f4756ffd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5615545159f0 0x5615546935c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:08.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.369+0000 7f47567fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561554693b00 0x5615546971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60712/0 (socket says 172.21.15.12:60712) 2024-05-17T22:54:08.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.369+0000 7f47567fc700 1 -- 172.21.15.12:0/173786589 learned_addr learned my addr 172.21.15.12:0/173786589 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:54:08.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.375+0000 7f47567fc700 1 -- 172.21.15.12:0/173786589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5615545159f0 msgr2=0x5615546935c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:08.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.375+0000 7f47567fc700 1 --2- 172.21.15.12:0/173786589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5615545159f0 0x5615546935c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:08.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.375+0000 7f47567fc700 1 -- 172.21.15.12:0/173786589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561554699400 msgr2=0x56155469bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:08.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.375+0000 7f47567fc700 1 --2- 172.21.15.12:0/173786589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561554699400 0x56155469bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:08.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.375+0000 7f47567fc700 1 -- 172.21.15.12:0/173786589 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56155468ac20 con 0x561554693b00 2024-05-17T22:54:08.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.384+0000 7f47567fc700 1 --2- 172.21.15.12:0/173786589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561554693b00 0x5615546971b0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f474c00b2d0 tx=0x7f474c0a91d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80af5105e9258ba1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:08.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.386+0000 7f473cff9700 1 -- 172.21.15.12:0/173786589 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c0c2020 con 0x561554693b00 2024-05-17T22:54:08.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.387+0000 7f473cff9700 1 -- 172.21.15.12:0/173786589 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f474c0ad160 con 0x561554693b00 2024-05-17T22:54:08.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.387+0000 7f473cff9700 1 -- 172.21.15.12:0/173786589 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c0b1b20 con 0x561554693b00 2024-05-17T22:54:08.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.389+0000 7f476c1dcb80 1 -- 172.21.15.12:0/173786589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561554693b00 msgr2=0x5615546971b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:08.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.389+0000 7f476c1dcb80 1 --2- 172.21.15.12:0/173786589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561554693b00 0x5615546971b0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f474c00b2d0 tx=0x7f474c0a91d0 comp rx=0 tx=0).stop 2024-05-17T22:54:08.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.393+0000 7f476c1dcb80 1 -- 172.21.15.12:0/173786589 shutdown_connections 2024-05-17T22:54:08.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.393+0000 7f476c1dcb80 1 --2- 172.21.15.12:0/173786589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561554699400 0x56155469bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:08.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.393+0000 7f476c1dcb80 1 --2- 172.21.15.12:0/173786589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5615545159f0 0x5615546935c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:08.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.393+0000 7f476c1dcb80 1 --2- 172.21.15.12:0/173786589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561554693b00 0x5615546971b0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:08.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.393+0000 7f476c1dcb80 1 -- 172.21.15.12:0/173786589 >> 172.21.15.12:0/173786589 conn(0x5615545757d0 msgr2=0x56155469e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:08.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.397+0000 7f476c1dcb80 1 -- 172.21.15.12:0/173786589 shutdown_connections 2024-05-17T22:54:08.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.398+0000 7f476c1dcb80 1 -- 172.21.15.12:0/173786589 wait complete. 2024-05-17T22:54:08.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.399+0000 7f476c1dcb80 1 -- start start 2024-05-17T22:54:08.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.399+0000 7f476c1dcb80 1 Processor -- start 2024-05-17T22:54:08.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.402+0000 7f476c1dcb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5615545159f0 0x5615546c5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:08.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.402+0000 7f476c1dcb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561554693b00 0x5615546b5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:08.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.403+0000 7f476c1dcb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561554699400 0x5615546b65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:08.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.403+0000 7f4756ffd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5615545159f0 0x5615546c5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:08.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.403+0000 7f4756ffd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5615545159f0 0x5615546c5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:52380/0 (socket says 172.21.15.12:52380) 2024-05-17T22:54:08.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.403+0000 7f4756ffd700 1 -- 172.21.15.12:0/87808220 learned_addr learned my addr 172.21.15.12:0/87808220 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:54:08.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.403+0000 7f476c1dcb80 1 -- 172.21.15.12:0/87808220 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f47581196d0 con 0x561554699400 2024-05-17T22:54:08.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.403+0000 7f476c1dcb80 1 -- 172.21.15.12:0/87808220 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f475811ab40 con 0x561554693b00 2024-05-17T22:54:08.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.403+0000 7f476c1dcb80 1 -- 172.21.15.12:0/87808220 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4758119da0 con 0x5615545159f0 2024-05-17T22:54:08.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.403+0000 7f47567fc700 1 --2- 172.21.15.12:0/87808220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561554693b00 0x5615546b5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:08.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.403+0000 7f476c1dcb80 10 client.0 _ll_register_callbacks cb 0x561554548b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:54:08.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.404+0000 7f47567fc700 1 -- 172.21.15.12:0/87808220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5615545159f0 msgr2=0x5615546c5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:08.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.404+0000 7f47567fc700 1 --2- 172.21.15.12:0/87808220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5615545159f0 0x5615546c5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:08.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.404+0000 7f47567fc700 1 -- 172.21.15.12:0/87808220 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561554699400 msgr2=0x5615546b65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:08.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.404+0000 7f47567fc700 1 --2- 172.21.15.12:0/87808220 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561554699400 0x5615546b65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:08.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.404+0000 7f47567fc700 1 -- 172.21.15.12:0/87808220 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615546c4a80 con 0x561554693b00 2024-05-17T22:54:08.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.409+0000 7f47567fc700 1 --2- 172.21.15.12:0/87808220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561554693b00 0x5615546b5fe0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f474c0a51d0 tx=0x7f474c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b234c60e049d2e08 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:08.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.409+0000 7f473aff5700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:54:08.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.413+0000 7f473aff5700 1 -- 172.21.15.12:0/87808220 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c0b1b20 con 0x561554693b00 2024-05-17T22:54:08.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.413+0000 7f476c1dcb80 1 -- 172.21.15.12:0/87808220 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f474c0ad160 con 0x561554693b00 2024-05-17T22:54:08.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.413+0000 7f473aff5700 1 -- 172.21.15.12:0/87808220 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f474c0b1cc0 con 0x561554693b00 2024-05-17T22:54:08.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.417+0000 7f473aff5700 1 -- 172.21.15.12:0/87808220 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f474c0ba050 con 0x561554693b00 2024-05-17T22:54:08.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.418+0000 7f476c1dcb80 10 client.26282 fetch_fsmap learned FSMap version 42 2024-05-17T22:54:08.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.418+0000 7f476c1dcb80 1 -- 172.21.15.12:0/87808220 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x561554727860 con 0x561554693b00 2024-05-17T22:54:08.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.422+0000 7f473aff5700 1 -- 172.21.15.12:0/87808220 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f474c0ad160 con 0x561554693b00 2024-05-17T22:54:08.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.422+0000 7f473aff5700 1 -- 172.21.15.12:0/87808220 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 42) ==== 8350+0+0 (secure 0 0 0) 0x7f474c0dfaf0 con 0x561554693b00 2024-05-17T22:54:08.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.422+0000 7f476c1dcb80 10 client.26282 fetch_fsmap finished waiting for FSMap version 42 2024-05-17T22:54:08.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.422+0000 7f476c1dcb80 10 client.26282 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:54:08.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.422+0000 7f476c1dcb80 20 client.26282 populate_metadata read hostname 'smithi012' 2024-05-17T22:54:08.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.422+0000 7f476c1dcb80 1 --2- 172.21.15.12:0/87808220 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x561554727f30 0x56155472a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:08.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.422+0000 7f476c1dcb80 4 client.26282 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:54:08.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.422+0000 7f476c1dcb80 1 -- 172.21.15.12:0/87808220 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5615545bd800 con 0x561554727f30 2024-05-17T22:54:08.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.422+0000 7f4756ffd700 1 --2- 172.21.15.12:0/87808220 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x561554727f30 0x56155472a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:08.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.423+0000 7f473aff5700 10 client.26282 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:54:08.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.423+0000 7f4756ffd700 1 --2- 172.21.15.12:0/87808220 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x561554727f30 0x56155472a350 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=118a7e96f0d0720 server_cookie=769e8fe56d79cac2 in_seq=0 out_seq=0 2024-05-17T22:54:08.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.424+0000 7f473aff5700 1 -- 172.21.15.12:0/87808220 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5615545bd800 con 0x561554727f30 2024-05-17T22:54:08.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.424+0000 7f473aff5700 10 client.26282 handle_command_reply: tid=0 2024-05-17T22:54:08.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.427+0000 7f476c1dcb80 1 client.26282 shutdown 2024-05-17T22:54:08.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.430+0000 7f476c1dcb80 1 -- 172.21.15.12:0/87808220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561554693b00 msgr2=0x5615546b5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:08.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.430+0000 7f476c1dcb80 1 --2- 172.21.15.12:0/87808220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561554693b00 0x5615546b5fe0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f474c0a51d0 tx=0x7f474c000bc0 comp rx=0 tx=0).stop 2024-05-17T22:54:08.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.430+0000 7f476c1dcb80 1 -- 172.21.15.12:0/87808220 shutdown_connections 2024-05-17T22:54:08.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.430+0000 7f476c1dcb80 1 --2- 172.21.15.12:0/87808220 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x561554727f30 0x56155472a350 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:08.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.430+0000 7f476c1dcb80 1 --2- 172.21.15.12:0/87808220 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561554699400 0x5615546b65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:08.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.430+0000 7f476c1dcb80 1 --2- 172.21.15.12:0/87808220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5615545159f0 0x5615546c5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:08.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.430+0000 7f476c1dcb80 1 --2- 172.21.15.12:0/87808220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561554693b00 0x5615546b5fe0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:08.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.430+0000 7f476c1dcb80 1 -- 172.21.15.12:0/87808220 >> 172.21.15.12:0/87808220 conn(0x5615545757d0 msgr2=0x561554696f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:08.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.430+0000 7f473aff5700 0 client.26282 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:54:08.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.431+0000 7f476c1dcb80 1 -- 172.21.15.12:0/87808220 shutdown_connections 2024-05-17T22:54:08.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.431+0000 7f476c1dcb80 1 -- 172.21.15.12:0/87808220 wait complete. 2024-05-17T22:54:08.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.432+0000 7f476c1dcb80 20 client.26282 trim_cache size 0 max 16384 2024-05-17T22:54:08.436 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:54:08.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.434+0000 7f475d801700 1 -- 172.21.15.12:0/548149131 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f47400b2a50 msgr2=0x7f47400b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:08.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.434+0000 7f475d801700 1 --2- 172.21.15.12:0/548149131 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f47400b2a50 0x7f47400b4f60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4750007620 tx=0x7f4750026000 comp rx=0 tx=0).stop 2024-05-17T22:54:08.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.434+0000 7f475d801700 1 -- 172.21.15.12:0/548149131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f475810b180 msgr2=0x7f47581a9ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:08.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.434+0000 7f475d801700 1 --2- 172.21.15.12:0/548149131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f475810b180 0x7f47581a9ab0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f474c00d980 tx=0x7f474c00dd40 comp rx=0 tx=0).stop 2024-05-17T22:54:08.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.435+0000 7f475d801700 1 -- 172.21.15.12:0/548149131 shutdown_connections 2024-05-17T22:54:08.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.435+0000 7f475d801700 1 --2- 172.21.15.12:0/548149131 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f47400b2a50 0x7f47400b4f60 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:08.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.435+0000 7f475d801700 1 --2- 172.21.15.12:0/548149131 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f475811d550 0x7f47581add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:08.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.435+0000 7f475d801700 1 --2- 172.21.15.12:0/548149131 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f475810a830 0x7f47581a9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:08.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.435+0000 7f475d801700 1 --2- 172.21.15.12:0/548149131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f475810b180 0x7f47581a9ab0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:08.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.435+0000 7f475d801700 1 -- 172.21.15.12:0/548149131 >> 172.21.15.12:0/548149131 conn(0x7f475806efc0 msgr2=0x7f475811b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:08.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.439+0000 7f475d801700 1 -- 172.21.15.12:0/548149131 shutdown_connections 2024-05-17T22:54:08.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:08.440+0000 7f475d801700 1 -- 172.21.15.12:0/548149131 wait complete. 2024-05-17T22:54:08.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:08 smithi151 ceph-mon[36101]: pgmap v885: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.9 MiB/s wr, 437 op/s 2024-05-17T22:54:08.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:08 smithi096 ceph-mon[36065]: pgmap v885: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.9 MiB/s wr, 437 op/s 2024-05-17T22:54:10.278 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-17T22:54:10.278 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-17T22:54:10.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:10 smithi151 ceph-mon[36101]: pgmap v886: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 3.7 MiB/s wr, 416 op/s 2024-05-17T22:54:10.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:10 smithi096 ceph-mon[36065]: pgmap v886: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 3.7 MiB/s wr, 416 op/s 2024-05-17T22:54:10.593 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:10 smithi012 ceph-mon[31632]: pgmap v886: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 3.7 MiB/s wr, 416 op/s 2024-05-17T22:54:10.691 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:54:11.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 890.0 s, 450.0 tps, lat 216.090 ms stddev 240.441 2024-05-17T22:54:12.276 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:12 smithi012 ceph-mon[31632]: pgmap v887: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.5 MiB/s wr, 508 op/s 2024-05-17T22:54:12.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:12 smithi151 ceph-mon[36101]: pgmap v887: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.5 MiB/s wr, 508 op/s 2024-05-17T22:54:12.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:12 smithi096 ceph-mon[36065]: pgmap v887: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.5 MiB/s wr, 508 op/s 2024-05-17T22:54:14.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.003+0000 7fd9243cc700 1 -- 172.21.15.12:0/3062362689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd91c10a830 msgr2=0x7fd91c10ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:14.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.003+0000 7fd9243cc700 1 --2- 172.21.15.12:0/3062362689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd91c10a830 0x7fd91c10ac40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd918009990 tx=0x7fd918009ca0 comp rx=0 tx=0).stop 2024-05-17T22:54:14.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.004+0000 7fd9243cc700 1 -- 172.21.15.12:0/3062362689 shutdown_connections 2024-05-17T22:54:14.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.004+0000 7fd9243cc700 1 --2- 172.21.15.12:0/3062362689 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd91c11d550 0x7fd91c11f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:14.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.004+0000 7fd9243cc700 1 --2- 172.21.15.12:0/3062362689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd91c10a830 0x7fd91c10ac40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:14.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.004+0000 7fd9243cc700 1 --2- 172.21.15.12:0/3062362689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd91c10b180 0x7fd91c11d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:14.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.004+0000 7fd9243cc700 1 -- 172.21.15.12:0/3062362689 >> 172.21.15.12:0/3062362689 conn(0x7fd91c06efc0 msgr2=0x7fd91c06f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:14.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.007+0000 7fd9243cc700 1 -- 172.21.15.12:0/3062362689 shutdown_connections 2024-05-17T22:54:14.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.008+0000 7fd9243cc700 1 -- 172.21.15.12:0/3062362689 wait complete. 2024-05-17T22:54:14.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.010+0000 7fd9243cc700 1 Processor -- start 2024-05-17T22:54:14.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.010+0000 7fd9243cc700 1 -- start start 2024-05-17T22:54:14.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.010+0000 7fd9243cc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd91c10a830 0x7fd91c11ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:14.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.010+0000 7fd9243cc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd91c10b180 0x7fd91c114bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:14.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.010+0000 7fd9243cc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd91c11d550 0x7fd91c1152b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:14.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.010+0000 7fd9243cc700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd91c122db0 con 0x7fd91c10b180 2024-05-17T22:54:14.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.010+0000 7fd9243cc700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd91c122c30 con 0x7fd91c10a830 2024-05-17T22:54:14.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.010+0000 7fd9243cc700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd91c122f30 con 0x7fd91c11d550 2024-05-17T22:54:14.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.012+0000 7fd922969700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd91c11d550 0x7fd91c1152b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:14.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.012+0000 7fd922969700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd91c11d550 0x7fd91c1152b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:37794/0 (socket says 172.21.15.12:37794) 2024-05-17T22:54:14.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.012+0000 7fd922969700 1 -- 172.21.15.12:0/4221552365 learned_addr learned my addr 172.21.15.12:0/4221552365 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:54:14.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.012+0000 7fd922168700 1 --2- 172.21.15.12:0/4221552365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd91c10a830 0x7fd91c11ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:14.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.014+0000 7fd922969700 1 -- 172.21.15.12:0/4221552365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd91c10a830 msgr2=0x7fd91c11ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:14.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.014+0000 7fd922969700 1 --2- 172.21.15.12:0/4221552365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd91c10a830 0x7fd91c11ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:14.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.014+0000 7fd922969700 1 -- 172.21.15.12:0/4221552365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd91c10b180 msgr2=0x7fd91c114bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:14.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.014+0000 7fd922969700 1 --2- 172.21.15.12:0/4221552365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd91c10b180 0x7fd91c114bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:14.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.014+0000 7fd922969700 1 -- 172.21.15.12:0/4221552365 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd91c1195c0 con 0x7fd91c11d550 2024-05-17T22:54:14.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.016+0000 7fd922969700 1 --2- 172.21.15.12:0/4221552365 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd91c11d550 0x7fd91c1152b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd91400b970 tx=0x7fd91400bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:14.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.016+0000 7fd9137fe700 1 -- 172.21.15.12:0/4221552365 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd914019070 con 0x7fd91c11d550 2024-05-17T22:54:14.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.016+0000 7fd9243cc700 1 -- 172.21.15.12:0/4221552365 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd91c1198b0 con 0x7fd91c11d550 2024-05-17T22:54:14.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.016+0000 7fd9243cc700 1 -- 172.21.15.12:0/4221552365 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd91c119df0 con 0x7fd91c11d550 2024-05-17T22:54:14.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.016+0000 7fd9137fe700 1 -- 172.21.15.12:0/4221552365 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd91400cde0 con 0x7fd91c11d550 2024-05-17T22:54:14.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.016+0000 7fd9137fe700 1 -- 172.21.15.12:0/4221552365 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd914007370 con 0x7fd91c11d550 2024-05-17T22:54:14.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.023+0000 7fd9137fe700 1 -- 172.21.15.12:0/4221552365 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd91401c070 con 0x7fd91c11d550 2024-05-17T22:54:14.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.024+0000 7fd9137fe700 1 --2- 172.21.15.12:0/4221552365 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd9080b2a90 0x7fd9080b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:14.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.025+0000 7fd9137fe700 1 -- 172.21.15.12:0/4221552365 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fd9140e2850 con 0x7fd91c11d550 2024-05-17T22:54:14.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.026+0000 7fd932da7b80 1 Processor -- start 2024-05-17T22:54:14.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.026+0000 7fd932da7b80 1 -- start start 2024-05-17T22:54:14.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.026+0000 7fd922168700 1 --2- 172.21.15.12:0/4221552365 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd9080b2a90 0x7fd9080b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:14.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.026+0000 7fd932da7b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556265235a70 0x5562653b8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:14.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.026+0000 7fd932da7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562653b9480 0x5562653b3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:14.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.026+0000 7fd932da7b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562653b3b80 0x5562653b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:14.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.026+0000 7fd932da7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5562652e2150 con 0x5562653b9480 2024-05-17T22:54:14.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.027+0000 7fd932da7b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5562652d1cf0 con 0x556265235a70 2024-05-17T22:54:14.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.027+0000 7fd932da7b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5562652d5d30 con 0x5562653b3b80 2024-05-17T22:54:14.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.027+0000 7fd922969700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562653b3b80 0x5562653b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:14.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.027+0000 7fd922969700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562653b3b80 0x5562653b3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:37800/0 (socket says 172.21.15.12:37800) 2024-05-17T22:54:14.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.027+0000 7fd922969700 1 -- 172.21.15.12:0/3364621454 learned_addr learned my addr 172.21.15.12:0/3364621454 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:54:14.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.027+0000 7fd921967700 1 --2- 172.21.15.12:0/3364621454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562653b9480 0x5562653b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:14.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.027+0000 7fd922168700 1 --2- 172.21.15.12:0/3364621454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556265235a70 0x5562653b8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:14.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.028+0000 7fd922969700 1 -- 172.21.15.12:0/3364621454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556265235a70 msgr2=0x5562653b8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:14.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.028+0000 7fd922969700 1 --2- 172.21.15.12:0/3364621454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556265235a70 0x5562653b8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:14.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.028+0000 7fd922969700 1 -- 172.21.15.12:0/3364621454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562653b9480 msgr2=0x5562653b3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:14.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.028+0000 7fd922969700 1 --2- 172.21.15.12:0/3364621454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562653b9480 0x5562653b3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:14.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.028+0000 7fd922969700 1 -- 172.21.15.12:0/3364621454 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55626526b640 con 0x5562653b3b80 2024-05-17T22:54:14.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.028+0000 7fd921967700 1 --2- 172.21.15.12:0/3364621454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562653b9480 0x5562653b3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:54:14.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.028+0000 7fd922969700 1 --2- 172.21.15.12:0/3364621454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562653b3b80 0x5562653b3f90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd91400b940 tx=0x7fd914004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9993467041a05718 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:14.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.028+0000 7fd910d46700 1 -- 172.21.15.12:0/3364621454 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9140aa180 con 0x5562653b3b80 2024-05-17T22:54:14.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.029+0000 7fd910d46700 1 -- 172.21.15.12:0/3364621454 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9140a9a80 con 0x5562653b3b80 2024-05-17T22:54:14.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.030+0000 7fd932da7b80 1 -- 172.21.15.12:0/3364621454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562653b3b80 msgr2=0x5562653b3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:14.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.030+0000 7fd922168700 1 --2- 172.21.15.12:0/4221552365 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd9080b2a90 0x7fd9080b4fa0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd918003800 tx=0x7fd9180036f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:14.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.030+0000 7fd932da7b80 1 --2- 172.21.15.12:0/3364621454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562653b3b80 0x5562653b3f90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd91400b940 tx=0x7fd914004ad0 comp rx=0 tx=0).stop 2024-05-17T22:54:14.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.030+0000 7fd932da7b80 1 -- 172.21.15.12:0/3364621454 shutdown_connections 2024-05-17T22:54:14.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.030+0000 7fd932da7b80 1 --2- 172.21.15.12:0/3364621454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562653b3b80 0x5562653b3f90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:14.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.030+0000 7fd932da7b80 1 --2- 172.21.15.12:0/3364621454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556265235a70 0x5562653b8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:14.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.030+0000 7fd932da7b80 1 --2- 172.21.15.12:0/3364621454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562653b9480 0x5562653b3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:14.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.030+0000 7fd932da7b80 1 -- 172.21.15.12:0/3364621454 >> 172.21.15.12:0/3364621454 conn(0x556265295850 msgr2=0x5562653ae240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:14.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.031+0000 7fd932da7b80 1 -- 172.21.15.12:0/3364621454 shutdown_connections 2024-05-17T22:54:14.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.031+0000 7fd932da7b80 1 -- 172.21.15.12:0/3364621454 wait complete. 2024-05-17T22:54:14.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.032+0000 7fd932da7b80 1 -- start start 2024-05-17T22:54:14.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.032+0000 7fd932da7b80 1 Processor -- start 2024-05-17T22:54:14.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.033+0000 7fd932da7b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556265235a70 0x5562653bcb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:14.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.033+0000 7fd932da7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562653b3b80 0x5562653bd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:14.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.033+0000 7fd932da7b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562653b9480 0x5562653bd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:14.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.033+0000 7fd922168700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556265235a70 0x5562653bcb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:14.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.034+0000 7fd922168700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556265235a70 0x5562653bcb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:37812/0 (socket says 172.21.15.12:37812) 2024-05-17T22:54:14.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.034+0000 7fd932da7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5562651d91d0 con 0x5562653b3b80 2024-05-17T22:54:14.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.034+0000 7fd921967700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562653b3b80 0x5562653bd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:14.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.034+0000 7fd922168700 1 -- 172.21.15.12:0/650216237 learned_addr learned my addr 172.21.15.12:0/650216237 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:54:14.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.034+0000 7fd932da7b80 1 -- 172.21.15.12:0/650216237 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5562652739c0 con 0x5562653b9480 2024-05-17T22:54:14.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.034+0000 7fd932da7b80 1 -- 172.21.15.12:0/650216237 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5562653bfea0 con 0x556265235a70 2024-05-17T22:54:14.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.035+0000 7fd932da7b80 10 client.0 _ll_register_callbacks cb 0x556265268bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:54:14.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.035+0000 7fd922969700 1 --2- 172.21.15.12:0/650216237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562653b9480 0x5562653bd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:14.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.035+0000 7fd922168700 1 -- 172.21.15.12:0/650216237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562653b9480 msgr2=0x5562653bd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:14.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.035+0000 7fd922168700 1 --2- 172.21.15.12:0/650216237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562653b9480 0x5562653bd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:14.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.035+0000 7fd922168700 1 -- 172.21.15.12:0/650216237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562653b3b80 msgr2=0x5562653bd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:14.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.035+0000 7fd922168700 1 --2- 172.21.15.12:0/650216237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562653b3b80 0x5562653bd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:14.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.035+0000 7fd922168700 1 -- 172.21.15.12:0/650216237 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556265222500 con 0x556265235a70 2024-05-17T22:54:14.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.035+0000 7fd922969700 1 --2- 172.21.15.12:0/650216237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562653b9480 0x5562653bd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:54:14.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.035+0000 7fd922168700 1 --2- 172.21.15.12:0/650216237 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556265235a70 0x5562653bcb60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd918009990 tx=0x7fd918009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a7c5590c7ba4bbc server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:14.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.036+0000 7fd905ffb700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:54:14.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.036+0000 7fd905ffb700 1 -- 172.21.15.12:0/650216237 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91800f460 con 0x556265235a70 2024-05-17T22:54:14.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.036+0000 7fd905ffb700 1 -- 172.21.15.12:0/650216237 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fd91800f600 con 0x556265235a70 2024-05-17T22:54:14.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.036+0000 7fd932da7b80 1 -- 172.21.15.12:0/650216237 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9140a9a80 con 0x556265235a70 2024-05-17T22:54:14.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.036+0000 7fd905ffb700 1 -- 172.21.15.12:0/650216237 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd918039050 con 0x556265235a70 2024-05-17T22:54:14.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.036+0000 7fd932da7b80 10 client.26236 fetch_fsmap learned FSMap version 43 2024-05-17T22:54:14.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.036+0000 7fd932da7b80 1 -- 172.21.15.12:0/650216237 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5562653ca890 con 0x556265235a70 2024-05-17T22:54:14.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.036+0000 7fd905ffb700 1 -- 172.21.15.12:0/650216237 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd91800fcb0 con 0x556265235a70 2024-05-17T22:54:14.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.038+0000 7fd905ffb700 1 -- 172.21.15.12:0/650216237 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7fd91800f800 con 0x556265235a70 2024-05-17T22:54:14.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.038+0000 7fd932da7b80 10 client.26236 fetch_fsmap finished waiting for FSMap version 43 2024-05-17T22:54:14.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.038+0000 7fd932da7b80 10 client.26236 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:54:14.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.038+0000 7fd932da7b80 20 client.26236 populate_metadata read hostname 'smithi012' 2024-05-17T22:54:14.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.038+0000 7fd932da7b80 1 --2- 172.21.15.12:0/650216237 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5562653cced0 0x5562653cf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:14.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.038+0000 7fd932da7b80 4 client.26236 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:54:14.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.038+0000 7fd932da7b80 1 -- 172.21.15.12:0/650216237 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd9140a9a80 con 0x5562653cced0 2024-05-17T22:54:14.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.038+0000 7fd921967700 1 --2- 172.21.15.12:0/650216237 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5562653cced0 0x5562653cf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:14.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.039+0000 7fd921967700 1 --2- 172.21.15.12:0/650216237 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5562653cced0 0x5562653cf310 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b135e59dce4d5b server_cookie=fffe3afe17b121ca in_seq=0 out_seq=0 2024-05-17T22:54:14.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.039+0000 7fd905ffb700 10 client.26236 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:54:14.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.040+0000 7fd905ffb700 1 -- 172.21.15.12:0/650216237 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fd9140a9a80 con 0x5562653cced0 2024-05-17T22:54:14.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.040+0000 7fd905ffb700 10 client.26236 handle_command_reply: tid=0 2024-05-17T22:54:14.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.040+0000 7fd932da7b80 1 client.26236 shutdown 2024-05-17T22:54:14.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.040+0000 7fd932da7b80 1 -- 172.21.15.12:0/650216237 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556265235a70 msgr2=0x5562653bcb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:14.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.040+0000 7fd932da7b80 1 --2- 172.21.15.12:0/650216237 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556265235a70 0x5562653bcb60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd918009990 tx=0x7fd918009ca0 comp rx=0 tx=0).stop 2024-05-17T22:54:14.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.040+0000 7fd932da7b80 1 -- 172.21.15.12:0/650216237 shutdown_connections 2024-05-17T22:54:14.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.040+0000 7fd932da7b80 1 --2- 172.21.15.12:0/650216237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562653b9480 0x5562653bd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:14.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.040+0000 7fd932da7b80 1 --2- 172.21.15.12:0/650216237 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556265235a70 0x5562653bcb60 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:14.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.040+0000 7fd932da7b80 1 --2- 172.21.15.12:0/650216237 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5562653cced0 0x5562653cf310 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:14.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.040+0000 7fd932da7b80 1 --2- 172.21.15.12:0/650216237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562653b3b80 0x5562653bd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:14.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.040+0000 7fd905ffb700 0 client.26236 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:54:14.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.040+0000 7fd932da7b80 1 -- 172.21.15.12:0/650216237 >> 172.21.15.12:0/650216237 conn(0x556265295850 msgr2=0x5562653ae1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:14.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.041+0000 7fd932da7b80 1 -- 172.21.15.12:0/650216237 shutdown_connections 2024-05-17T22:54:14.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.041+0000 7fd932da7b80 1 -- 172.21.15.12:0/650216237 wait complete. 2024-05-17T22:54:14.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.041+0000 7fd932da7b80 20 client.26236 trim_cache size 0 max 16384 2024-05-17T22:54:14.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.068+0000 7fd932da7b80 1 Processor -- start 2024-05-17T22:54:14.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.068+0000 7fd932da7b80 1 -- start start 2024-05-17T22:54:14.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.069+0000 7fd932da7b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556265295850 0x5562653b3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:14.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.069+0000 7fd932da7b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556265235a70 0x5562653a8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:14.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.069+0000 7fd932da7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562653b3b80 0x5562653bd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:14.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.069+0000 7fd932da7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd900011360 con 0x5562653b3b80 2024-05-17T22:54:14.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.069+0000 7fd932da7b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd90000ebf0 con 0x556265295850 2024-05-17T22:54:14.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.069+0000 7fd932da7b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd90000caf0 con 0x556265235a70 2024-05-17T22:54:14.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.069+0000 7fd922168700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556265295850 0x5562653b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:14.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.069+0000 7fd922168700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556265295850 0x5562653b3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:48048/0 (socket says 172.21.15.12:48048) 2024-05-17T22:54:14.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.069+0000 7fd922168700 1 -- 172.21.15.12:0/818303387 learned_addr learned my addr 172.21.15.12:0/818303387 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:54:14.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.069+0000 7fd922969700 1 --2- 172.21.15.12:0/818303387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562653b3b80 0x5562653bd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:14.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.069+0000 7fd921967700 1 --2- 172.21.15.12:0/818303387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556265235a70 0x5562653a8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:14.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.070+0000 7fd922168700 1 -- 172.21.15.12:0/818303387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556265235a70 msgr2=0x5562653a8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:14.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.070+0000 7fd922168700 1 --2- 172.21.15.12:0/818303387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556265235a70 0x5562653a8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:14.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.070+0000 7fd922168700 1 -- 172.21.15.12:0/818303387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562653b3b80 msgr2=0x5562653bd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:14.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.070+0000 7fd922168700 1 --2- 172.21.15.12:0/818303387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562653b3b80 0x5562653bd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:14.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.070+0000 7fd922168700 1 -- 172.21.15.12:0/818303387 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562653bfd30 con 0x556265295850 2024-05-17T22:54:14.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.070+0000 7fd922969700 1 --2- 172.21.15.12:0/818303387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562653b3b80 0x5562653bd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:54:14.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.070+0000 7fd922168700 1 --2- 172.21.15.12:0/818303387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556265295850 0x5562653b3640 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd9180093c0 tx=0x7fd918005ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e48171d64aaf2662 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:14.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.070+0000 7fd907fff700 1 -- 172.21.15.12:0/818303387 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd918018990 con 0x556265295850 2024-05-17T22:54:14.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.070+0000 7fd907fff700 1 -- 172.21.15.12:0/818303387 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd91800f460 con 0x556265295850 2024-05-17T22:54:14.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.070+0000 7fd907fff700 1 -- 172.21.15.12:0/818303387 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd918028880 con 0x556265295850 2024-05-17T22:54:14.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.071+0000 7fd932da7b80 1 -- 172.21.15.12:0/818303387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556265295850 msgr2=0x5562653b3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:14.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.071+0000 7fd932da7b80 1 --2- 172.21.15.12:0/818303387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556265295850 0x5562653b3640 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd9180093c0 tx=0x7fd918005ce0 comp rx=0 tx=0).stop 2024-05-17T22:54:14.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.071+0000 7fd932da7b80 1 -- 172.21.15.12:0/818303387 shutdown_connections 2024-05-17T22:54:14.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.071+0000 7fd932da7b80 1 --2- 172.21.15.12:0/818303387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5562653b3b80 0x5562653bd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:14.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.072+0000 7fd932da7b80 1 --2- 172.21.15.12:0/818303387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556265295850 0x5562653b3640 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:14.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.072+0000 7fd932da7b80 1 --2- 172.21.15.12:0/818303387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556265235a70 0x5562653a8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:14.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.072+0000 7fd932da7b80 1 -- 172.21.15.12:0/818303387 >> 172.21.15.12:0/818303387 conn(0x5562653ae540 msgr2=0x5562653b6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:14.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.072+0000 7fd932da7b80 1 -- 172.21.15.12:0/818303387 shutdown_connections 2024-05-17T22:54:14.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.072+0000 7fd932da7b80 1 -- 172.21.15.12:0/818303387 wait complete. 2024-05-17T22:54:14.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.072+0000 7fd932da7b80 1 -- start start 2024-05-17T22:54:14.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.072+0000 7fd932da7b80 1 Processor -- start 2024-05-17T22:54:14.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.073+0000 7fd932da7b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556265235a70 0x5562653c21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:14.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.073+0000 7fd932da7b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556265295850 0x5562653c26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:14.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.073+0000 7fd932da7b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562653b3b80 0x5562653c2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:14.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.073+0000 7fd932da7b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd90000d0b0 con 0x556265295850 2024-05-17T22:54:14.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.073+0000 7fd932da7b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd90000e520 con 0x5562653b3b80 2024-05-17T22:54:14.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.073+0000 7fd932da7b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd90000d780 con 0x556265235a70 2024-05-17T22:54:14.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.073+0000 7fd932da7b80 10 client.0 _ll_register_callbacks cb 0x556265268bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:54:14.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.073+0000 7fd921967700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556265295850 0x5562653c26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:14.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.073+0000 7fd922168700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556265235a70 0x5562653c21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:14.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.073+0000 7fd921967700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556265295850 0x5562653c26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58956/0 (socket says 172.21.15.12:58956) 2024-05-17T22:54:14.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.073+0000 7fd921967700 1 -- 172.21.15.12:0/1737201439 learned_addr learned my addr 172.21.15.12:0/1737201439 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:54:14.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.073+0000 7fd922969700 1 --2- 172.21.15.12:0/1737201439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562653b3b80 0x5562653c2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:14.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.073+0000 7fd921967700 1 -- 172.21.15.12:0/1737201439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556265235a70 msgr2=0x5562653c21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:14.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.073+0000 7fd921967700 1 --2- 172.21.15.12:0/1737201439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556265235a70 0x5562653c21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:14.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.073+0000 7fd921967700 1 -- 172.21.15.12:0/1737201439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562653b3b80 msgr2=0x5562653c2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:14.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.073+0000 7fd921967700 1 --2- 172.21.15.12:0/1737201439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562653b3b80 0x5562653c2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:14.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.073+0000 7fd921967700 1 -- 172.21.15.12:0/1737201439 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556265437960 con 0x556265295850 2024-05-17T22:54:14.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.073+0000 7fd922969700 1 --2- 172.21.15.12:0/1737201439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562653b3b80 0x5562653c2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:54:14.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.073+0000 7fd922168700 1 --2- 172.21.15.12:0/1737201439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556265235a70 0x5562653c21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:54:14.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.074+0000 7fd921967700 1 --2- 172.21.15.12:0/1737201439 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556265295850 0x5562653c26f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fd90c000c00 tx=0x7fd90c01a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f33173889a6f113 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:14.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.074+0000 7fd905ffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:54:14.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.074+0000 7fd905ffb700 1 -- 172.21.15.12:0/1737201439 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90c01aa60 con 0x556265295850 2024-05-17T22:54:14.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.074+0000 7fd932da7b80 1 -- 172.21.15.12:0/1737201439 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd91800f460 con 0x556265295850 2024-05-17T22:54:14.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.074+0000 7fd905ffb700 1 -- 172.21.15.12:0/1737201439 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fd90c023480 con 0x556265295850 2024-05-17T22:54:14.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.074+0000 7fd905ffb700 1 -- 172.21.15.12:0/1737201439 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd90c02c050 con 0x556265295850 2024-05-17T22:54:14.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.074+0000 7fd932da7b80 10 client.16596 fetch_fsmap learned FSMap version 43 2024-05-17T22:54:14.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.074+0000 7fd932da7b80 1 -- 172.21.15.12:0/1737201439 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x556265438c60 con 0x556265295850 2024-05-17T22:54:14.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.074+0000 7fd905ffb700 1 -- 172.21.15.12:0/1737201439 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd91800f460 con 0x556265295850 2024-05-17T22:54:14.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.075+0000 7fd905ffb700 1 -- 172.21.15.12:0/1737201439 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7fd90c0147f0 con 0x556265295850 2024-05-17T22:54:14.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.075+0000 7fd932da7b80 10 client.16596 fetch_fsmap finished waiting for FSMap version 43 2024-05-17T22:54:14.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.075+0000 7fd932da7b80 10 client.16596 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:54:14.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.075+0000 7fd932da7b80 20 client.16596 populate_metadata read hostname 'smithi012' 2024-05-17T22:54:14.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.075+0000 7fd932da7b80 1 --2- 172.21.15.12:0/1737201439 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5562653b9480 0x55626543b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:14.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.075+0000 7fd932da7b80 4 client.16596 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-17T22:54:14.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.075+0000 7fd932da7b80 1 -- 172.21.15.12:0/1737201439 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5562652e40a0 con 0x5562653b9480 2024-05-17T22:54:14.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.075+0000 7fd922168700 1 --2- 172.21.15.12:0/1737201439 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5562653b9480 0x55626543b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:14.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.076+0000 7fd922168700 1 --2- 172.21.15.12:0/1737201439 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5562653b9480 0x55626543b360 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36e909349f13424c server_cookie=7f70b9cc85ab673a in_seq=0 out_seq=0 2024-05-17T22:54:14.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.076+0000 7fd905ffb700 10 client.16596 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:54:14.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.077+0000 7fd905ffb700 1 -- 172.21.15.12:0/1737201439 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5562652e40a0 con 0x5562653b9480 2024-05-17T22:54:14.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.077+0000 7fd905ffb700 10 client.16596 handle_command_reply: tid=0 2024-05-17T22:54:14.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.077+0000 7fd932da7b80 1 client.16596 shutdown 2024-05-17T22:54:14.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.077+0000 7fd932da7b80 1 -- 172.21.15.12:0/1737201439 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556265295850 msgr2=0x5562653c26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:14.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.077+0000 7fd932da7b80 1 --2- 172.21.15.12:0/1737201439 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556265295850 0x5562653c26f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fd90c000c00 tx=0x7fd90c01a420 comp rx=0 tx=0).stop 2024-05-17T22:54:14.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.077+0000 7fd932da7b80 1 -- 172.21.15.12:0/1737201439 shutdown_connections 2024-05-17T22:54:14.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.077+0000 7fd932da7b80 1 --2- 172.21.15.12:0/1737201439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562653b3b80 0x5562653c2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:14.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.077+0000 7fd932da7b80 1 --2- 172.21.15.12:0/1737201439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556265235a70 0x5562653c21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:14.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.077+0000 7fd932da7b80 1 --2- 172.21.15.12:0/1737201439 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x5562653b9480 0x55626543b360 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:14.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.077+0000 7fd932da7b80 1 --2- 172.21.15.12:0/1737201439 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556265295850 0x5562653c26f0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:14.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.077+0000 7fd932da7b80 1 -- 172.21.15.12:0/1737201439 >> 172.21.15.12:0/1737201439 conn(0x5562653ae540 msgr2=0x5562653be140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:14.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.077+0000 7fd905ffb700 0 client.16596 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:54:14.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.077+0000 7fd932da7b80 1 -- 172.21.15.12:0/1737201439 shutdown_connections 2024-05-17T22:54:14.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.077+0000 7fd932da7b80 1 -- 172.21.15.12:0/1737201439 wait complete. 2024-05-17T22:54:14.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.077+0000 7fd932da7b80 20 client.16596 trim_cache size 0 max 16384 2024-05-17T22:54:14.085 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:54:14.085 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-05-17T22:54:14.085 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "581decdf-3470-4b25-953e-76e92b49291c", 2024-05-17T22:54:14.085 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-05-17T22:54:14.085 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:54:14.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.078+0000 7fd9243cc700 1 -- 172.21.15.12:0/4221552365 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd9080b2a90 msgr2=0x7fd9080b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:14.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.078+0000 7fd9243cc700 1 --2- 172.21.15.12:0/4221552365 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd9080b2a90 0x7fd9080b4fa0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd918003800 tx=0x7fd9180036f0 comp rx=0 tx=0).stop 2024-05-17T22:54:14.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.078+0000 7fd9243cc700 1 -- 172.21.15.12:0/4221552365 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd91c11d550 msgr2=0x7fd91c1152b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:14.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.078+0000 7fd9243cc700 1 --2- 172.21.15.12:0/4221552365 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd91c11d550 0x7fd91c1152b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd91400b970 tx=0x7fd91400bc80 comp rx=0 tx=0).stop 2024-05-17T22:54:14.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.078+0000 7fd9243cc700 1 -- 172.21.15.12:0/4221552365 shutdown_connections 2024-05-17T22:54:14.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.078+0000 7fd9243cc700 1 --2- 172.21.15.12:0/4221552365 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd9080b2a90 0x7fd9080b4fa0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:14.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.078+0000 7fd9243cc700 1 --2- 172.21.15.12:0/4221552365 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd91c11d550 0x7fd91c1152b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:14.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.078+0000 7fd9243cc700 1 --2- 172.21.15.12:0/4221552365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd91c10a830 0x7fd91c11ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:14.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.078+0000 7fd9243cc700 1 --2- 172.21.15.12:0/4221552365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd91c10b180 0x7fd91c114bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:14.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.078+0000 7fd9243cc700 1 -- 172.21.15.12:0/4221552365 >> 172.21.15.12:0/4221552365 conn(0x7fd91c06efc0 msgr2=0x7fd91c072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:14.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.078+0000 7fd9243cc700 1 -- 172.21.15.12:0/4221552365 shutdown_connections 2024-05-17T22:54:14.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:14.078+0000 7fd9243cc700 1 -- 172.21.15.12:0/4221552365 wait complete. 2024-05-17T22:54:14.345 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:14 smithi012 ceph-mon[31632]: pgmap v888: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.6 MiB/s wr, 399 op/s 2024-05-17T22:54:14.345 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:14 smithi012 ceph-mon[31632]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:54:14.345 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:14 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:54:14.345 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:14 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:54:14.345 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:14 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:54:14.345 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:14 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:54:14.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:14 smithi151 ceph-mon[36101]: pgmap v888: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.6 MiB/s wr, 399 op/s 2024-05-17T22:54:14.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:14 smithi151 ceph-mon[36101]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:54:14.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:14 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:54:14.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:14 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:54:14.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:14 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:54:14.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:14 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:54:14.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:14 smithi096 ceph-mon[36065]: pgmap v888: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.6 MiB/s wr, 399 op/s 2024-05-17T22:54:14.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:14 smithi096 ceph-mon[36065]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:54:14.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:14 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:54:14.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:14 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:54:14.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:14 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:54:14.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:14 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:54:14.777 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-17T22:54:15.371 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:54:15.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:15 smithi151 ceph-mon[36101]: scrub queued for path: / 2024-05-17T22:54:15.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:15 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:54:15.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:15 smithi151 ceph-mon[36101]: scrub summary: active paths [/] 2024-05-17T22:54:15.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:15 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:54:15.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:15 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:54:15.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:15 smithi151 ceph-mon[36101]: scrub summary: idle+waiting paths [/] 2024-05-17T22:54:15.514 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:15 smithi012 ceph-mon[31632]: scrub queued for path: / 2024-05-17T22:54:15.514 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:15 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:54:15.514 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:15 smithi012 ceph-mon[31632]: scrub summary: active paths [/] 2024-05-17T22:54:15.514 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:15 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:54:15.514 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:15 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:54:15.514 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:15 smithi012 ceph-mon[31632]: scrub summary: idle+waiting paths [/] 2024-05-17T22:54:15.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:15 smithi096 ceph-mon[36065]: scrub queued for path: / 2024-05-17T22:54:15.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:15 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:54:15.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:15 smithi096 ceph-mon[36065]: scrub summary: active paths [/] 2024-05-17T22:54:15.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:15 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:54:15.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:15 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:54:15.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:15 smithi096 ceph-mon[36065]: scrub summary: idle+waiting paths [/] 2024-05-17T22:54:16.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 895.0 s, 445.2 tps, lat 224.153 ms stddev 278.226 2024-05-17T22:54:16.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:16 smithi151 ceph-mon[36101]: pgmap v889: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.2 MiB/s wr, 485 op/s 2024-05-17T22:54:16.495 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:16 smithi012 ceph-mon[31632]: pgmap v889: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.2 MiB/s wr, 485 op/s 2024-05-17T22:54:16.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:16 smithi096 ceph-mon[36065]: pgmap v889: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.2 MiB/s wr, 485 op/s 2024-05-17T22:54:18.522 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:18 smithi012 ceph-mon[31632]: pgmap v890: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 3.7 MiB/s wr, 424 op/s 2024-05-17T22:54:18.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:18 smithi096 ceph-mon[36065]: pgmap v890: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 3.7 MiB/s wr, 424 op/s 2024-05-17T22:54:18.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.727+0000 7fb338440700 1 -- 172.21.15.12:0/3708451130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb330107730 msgr2=0x7fb330107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:18.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.727+0000 7fb338440700 1 --2- 172.21.15.12:0/3708451130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb330107730 0x7fb330107b40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb32801c0d0 tx=0x7fb32801c3e0 comp rx=0 tx=0).stop 2024-05-17T22:54:18.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.727+0000 7fb338440700 1 -- 172.21.15.12:0/3708451130 shutdown_connections 2024-05-17T22:54:18.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.727+0000 7fb338440700 1 --2- 172.21.15.12:0/3708451130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb330108080 0x7fb33010f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:18.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.727+0000 7fb338440700 1 --2- 172.21.15.12:0/3708451130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb330102e00 0x7fb330103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:18.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.727+0000 7fb338440700 1 --2- 172.21.15.12:0/3708451130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb330107730 0x7fb330107b40 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:18.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.727+0000 7fb338440700 1 -- 172.21.15.12:0/3708451130 >> 172.21.15.12:0/3708451130 conn(0x7fb330079bd0 msgr2=0x7fb330079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:18.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.727+0000 7fb338440700 1 -- 172.21.15.12:0/3708451130 shutdown_connections 2024-05-17T22:54:18.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.727+0000 7fb338440700 1 -- 172.21.15.12:0/3708451130 wait complete. 2024-05-17T22:54:18.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.728+0000 7fb338440700 1 Processor -- start 2024-05-17T22:54:18.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.728+0000 7fb338440700 1 -- start start 2024-05-17T22:54:18.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.729+0000 7fb338440700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb330102e00 0x7fb330075250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:18.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.729+0000 7fb338440700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb330107730 0x7fb330070380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:18.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.729+0000 7fb338440700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb330108080 0x7fb3300708c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:18.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.729+0000 7fb338440700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb33006e8c0 con 0x7fb330102e00 2024-05-17T22:54:18.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.729+0000 7fb338440700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb33006e740 con 0x7fb330107730 2024-05-17T22:54:18.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.729+0000 7fb338440700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb33006ea40 con 0x7fb330108080 2024-05-17T22:54:18.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.729+0000 7fb3359db700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb330107730 0x7fb330070380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:18.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.729+0000 7fb3361dc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb330102e00 0x7fb330075250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:18.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.730+0000 7fb3361dc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb330102e00 0x7fb330075250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58974/0 (socket says 172.21.15.12:58974) 2024-05-17T22:54:18.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.730+0000 7fb3361dc700 1 -- 172.21.15.12:0/1372129954 learned_addr learned my addr 172.21.15.12:0/1372129954 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:54:18.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.730+0000 7fb3369dd700 1 --2- 172.21.15.12:0/1372129954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb330108080 0x7fb3300708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:18.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.731+0000 7fb3369dd700 1 -- 172.21.15.12:0/1372129954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb330107730 msgr2=0x7fb330070380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:18.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.731+0000 7fb3369dd700 1 --2- 172.21.15.12:0/1372129954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb330107730 0x7fb330070380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:18.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.731+0000 7fb3369dd700 1 -- 172.21.15.12:0/1372129954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb330102e00 msgr2=0x7fb330075250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:18.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.731+0000 7fb3369dd700 1 --2- 172.21.15.12:0/1372129954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb330102e00 0x7fb330075250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:18.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.731+0000 7fb3369dd700 1 -- 172.21.15.12:0/1372129954 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb330070fc0 con 0x7fb330108080 2024-05-17T22:54:18.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.731+0000 7fb3359db700 1 --2- 172.21.15.12:0/1372129954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb330107730 0x7fb330070380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:54:18.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.733+0000 7fb3369dd700 1 --2- 172.21.15.12:0/1372129954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb330108080 0x7fb3300708c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb32000d720 tx=0x7fb32000da30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:18.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.734+0000 7fb3277fe700 1 -- 172.21.15.12:0/1372129954 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb320016070 con 0x7fb330108080 2024-05-17T22:54:18.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.734+0000 7fb338440700 1 -- 172.21.15.12:0/1372129954 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3301175f0 con 0x7fb330108080 2024-05-17T22:54:18.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.734+0000 7fb338440700 1 -- 172.21.15.12:0/1372129954 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb330117b30 con 0x7fb330108080 2024-05-17T22:54:18.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.734+0000 7fb3277fe700 1 -- 172.21.15.12:0/1372129954 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb32000f690 con 0x7fb330108080 2024-05-17T22:54:18.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.734+0000 7fb3277fe700 1 -- 172.21.15.12:0/1372129954 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb320016070 con 0x7fb330108080 2024-05-17T22:54:18.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:18 smithi151 ceph-mon[36101]: pgmap v890: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 3.7 MiB/s wr, 424 op/s 2024-05-17T22:54:18.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.739+0000 7fb3277fe700 1 -- 172.21.15.12:0/1372129954 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb3200037d0 con 0x7fb330108080 2024-05-17T22:54:18.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.740+0000 7fb3277fe700 1 --2- 172.21.15.12:0/1372129954 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb31c0b2d40 0x7fb31c0b5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:18.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.741+0000 7fb3277fe700 1 -- 172.21.15.12:0/1372129954 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fb320004020 con 0x7fb330108080 2024-05-17T22:54:18.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.742+0000 7fb346e1bb80 1 Processor -- start 2024-05-17T22:54:18.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.742+0000 7fb346e1bb80 1 -- start start 2024-05-17T22:54:18.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.742+0000 7fb346e1bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55926ab529f0 0x55926acd5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:18.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.742+0000 7fb346e1bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55926acd6400 0x55926acd05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:18.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.742+0000 7fb346e1bb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55926acd0b00 0x55926acd0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:18.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.742+0000 7fb346e1bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55926abff0d0 con 0x55926acd6400 2024-05-17T22:54:18.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.742+0000 7fb346e1bb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55926abeec70 con 0x55926acd0b00 2024-05-17T22:54:18.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.742+0000 7fb346e1bb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55926abf2cb0 con 0x55926ab529f0 2024-05-17T22:54:18.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.743+0000 7fb3369dd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55926acd0b00 0x55926acd0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:18.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.743+0000 7fb3369dd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55926acd0b00 0x55926acd0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:48088/0 (socket says 172.21.15.12:48088) 2024-05-17T22:54:18.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.743+0000 7fb3369dd700 1 -- 172.21.15.12:0/661781418 learned_addr learned my addr 172.21.15.12:0/661781418 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:54:18.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.744+0000 7fb3361dc700 1 --2- 172.21.15.12:0/1372129954 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb31c0b2d40 0x7fb31c0b5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:18.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.744+0000 7fb3369dd700 1 -- 172.21.15.12:0/661781418 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55926ab529f0 msgr2=0x55926acd5ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T22:54:18.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.744+0000 7fb3369dd700 1 --2- 172.21.15.12:0/661781418 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55926ab529f0 0x55926acd5ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:18.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.744+0000 7fb3369dd700 1 -- 172.21.15.12:0/661781418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55926acd6400 msgr2=0x55926acd05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:18.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.744+0000 7fb3369dd700 1 --2- 172.21.15.12:0/661781418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55926acd6400 0x55926acd05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:18.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.744+0000 7fb3369dd700 1 -- 172.21.15.12:0/661781418 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55926ab885c0 con 0x55926acd0b00 2024-05-17T22:54:18.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.744+0000 7fb3361dc700 1 --2- 172.21.15.12:0/1372129954 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb31c0b2d40 0x7fb31c0b5250 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x55926accc1d0 tx=0x7fb32c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:18.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.745+0000 7fb3369dd700 1 --2- 172.21.15.12:0/661781418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55926acd0b00 0x55926acd0f10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb3200a5ef0 tx=0x7fb3200a9380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72107a22e744f45d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:18.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.745+0000 7fb324d46700 1 -- 172.21.15.12:0/661781418 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb320016070 con 0x55926acd0b00 2024-05-17T22:54:18.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.745+0000 7fb324d46700 1 -- 172.21.15.12:0/661781418 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3200ad030 con 0x55926acd0b00 2024-05-17T22:54:18.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.745+0000 7fb324d46700 1 -- 172.21.15.12:0/661781418 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3200b18b0 con 0x55926acd0b00 2024-05-17T22:54:18.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.746+0000 7fb346e1bb80 1 -- 172.21.15.12:0/661781418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55926acd0b00 msgr2=0x55926acd0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:18.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.746+0000 7fb346e1bb80 1 --2- 172.21.15.12:0/661781418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55926acd0b00 0x55926acd0f10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb3200a5ef0 tx=0x7fb3200a9380 comp rx=0 tx=0).stop 2024-05-17T22:54:18.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.746+0000 7fb346e1bb80 1 -- 172.21.15.12:0/661781418 shutdown_connections 2024-05-17T22:54:18.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.746+0000 7fb346e1bb80 1 --2- 172.21.15.12:0/661781418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55926acd0b00 0x55926acd0f10 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:18.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.746+0000 7fb346e1bb80 1 --2- 172.21.15.12:0/661781418 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55926ab529f0 0x55926acd5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:18.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.746+0000 7fb346e1bb80 1 --2- 172.21.15.12:0/661781418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55926acd6400 0x55926acd05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:18.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.746+0000 7fb346e1bb80 1 -- 172.21.15.12:0/661781418 >> 172.21.15.12:0/661781418 conn(0x55926abb27d0 msgr2=0x55926accb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:18.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.746+0000 7fb346e1bb80 1 -- 172.21.15.12:0/661781418 shutdown_connections 2024-05-17T22:54:18.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.746+0000 7fb346e1bb80 1 -- 172.21.15.12:0/661781418 wait complete. 2024-05-17T22:54:18.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.747+0000 7fb346e1bb80 1 -- start start 2024-05-17T22:54:18.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.747+0000 7fb346e1bb80 1 Processor -- start 2024-05-17T22:54:18.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.747+0000 7fb346e1bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55926ab529f0 0x55926acd9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:18.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.748+0000 7fb346e1bb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55926acd0b00 0x55926acd9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:18.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.748+0000 7fb346e1bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55926acd6400 0x55926acda4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:18.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.748+0000 7fb346e1bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55926aaf6150 con 0x55926acd6400 2024-05-17T22:54:18.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.748+0000 7fb346e1bb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55926ab90940 con 0x55926acd0b00 2024-05-17T22:54:18.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.748+0000 7fb346e1bb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55926acdcd70 con 0x55926ab529f0 2024-05-17T22:54:18.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.748+0000 7fb346e1bb80 10 client.0 _ll_register_callbacks cb 0x55926ab85b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:54:18.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.748+0000 7fb3359db700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55926acd0b00 0x55926acd9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:18.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.748+0000 7fb3359db700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55926acd0b00 0x55926acd9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:48090/0 (socket says 172.21.15.12:48090) 2024-05-17T22:54:18.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.748+0000 7fb3359db700 1 -- 172.21.15.12:0/2963155232 learned_addr learned my addr 172.21.15.12:0/2963155232 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:54:18.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.748+0000 7fb3361dc700 1 --2- 172.21.15.12:0/2963155232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55926ab529f0 0x55926acd9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:18.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.748+0000 7fb3369dd700 1 --2- 172.21.15.12:0/2963155232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55926acd6400 0x55926acda4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:18.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.748+0000 7fb3359db700 1 -- 172.21.15.12:0/2963155232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55926ab529f0 msgr2=0x55926acd9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:18.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.748+0000 7fb3359db700 1 --2- 172.21.15.12:0/2963155232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55926ab529f0 0x55926acd9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:18.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.748+0000 7fb3359db700 1 -- 172.21.15.12:0/2963155232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55926acd6400 msgr2=0x55926acda4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:18.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.748+0000 7fb3359db700 1 --2- 172.21.15.12:0/2963155232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55926acd6400 0x55926acda4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:18.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.748+0000 7fb3359db700 1 -- 172.21.15.12:0/2963155232 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55926ab3f480 con 0x55926acd0b00 2024-05-17T22:54:18.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.749+0000 7fb3359db700 1 --2- 172.21.15.12:0/2963155232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55926acd0b00 0x55926acd9fa0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb32801c0d0 tx=0x7fb328004cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60041f535d3a31b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:18.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.749+0000 7fb319ffb700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:54:18.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.749+0000 7fb319ffb700 1 -- 172.21.15.12:0/2963155232 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32801dea0 con 0x55926acd0b00 2024-05-17T22:54:18.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.749+0000 7fb319ffb700 1 -- 172.21.15.12:0/2963155232 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fb3280189e0 con 0x55926acd0b00 2024-05-17T22:54:18.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.749+0000 7fb346e1bb80 1 -- 172.21.15.12:0/2963155232 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3200ad030 con 0x55926acd0b00 2024-05-17T22:54:18.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.749+0000 7fb319ffb700 1 -- 172.21.15.12:0/2963155232 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fb328040050 con 0x55926acd0b00 2024-05-17T22:54:18.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.749+0000 7fb346e1bb80 10 client.26309 fetch_fsmap learned FSMap version 43 2024-05-17T22:54:18.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.749+0000 7fb346e1bb80 1 -- 172.21.15.12:0/2963155232 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55926ace7890 con 0x55926acd0b00 2024-05-17T22:54:18.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.749+0000 7fb319ffb700 1 -- 172.21.15.12:0/2963155232 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3200ad030 con 0x55926acd0b00 2024-05-17T22:54:18.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.749+0000 7fb319ffb700 1 -- 172.21.15.12:0/2963155232 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7fb328022070 con 0x55926acd0b00 2024-05-17T22:54:18.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.750+0000 7fb346e1bb80 10 client.26309 fetch_fsmap finished waiting for FSMap version 43 2024-05-17T22:54:18.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.750+0000 7fb346e1bb80 10 client.26309 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:54:18.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.750+0000 7fb346e1bb80 20 client.26309 populate_metadata read hostname 'smithi012' 2024-05-17T22:54:18.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.750+0000 7fb346e1bb80 1 --2- 172.21.15.12:0/2963155232 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55926ace9f70 0x55926acec3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:18.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.750+0000 7fb346e1bb80 4 client.26309 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:54:18.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.750+0000 7fb346e1bb80 1 -- 172.21.15.12:0/2963155232 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55926ac01020 con 0x55926ace9f70 2024-05-17T22:54:18.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.750+0000 7fb3361dc700 1 --2- 172.21.15.12:0/2963155232 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55926ace9f70 0x55926acec3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:18.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.750+0000 7fb319ffb700 10 client.26309 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:54:18.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.750+0000 7fb3361dc700 1 --2- 172.21.15.12:0/2963155232 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55926ace9f70 0x55926acec3d0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=146793e7e6a0cfa5 server_cookie=8704d57c80a6c187 in_seq=0 out_seq=0 2024-05-17T22:54:18.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.751+0000 7fb319ffb700 1 -- 172.21.15.12:0/2963155232 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55926ac01020 con 0x55926ace9f70 2024-05-17T22:54:18.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.751+0000 7fb319ffb700 10 client.26309 handle_command_reply: tid=0 2024-05-17T22:54:18.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.751+0000 7fb346e1bb80 1 client.26309 shutdown 2024-05-17T22:54:18.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.751+0000 7fb346e1bb80 1 -- 172.21.15.12:0/2963155232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55926acd0b00 msgr2=0x55926acd9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:18.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.751+0000 7fb346e1bb80 1 --2- 172.21.15.12:0/2963155232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55926acd0b00 0x55926acd9fa0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb32801c0d0 tx=0x7fb328004cf0 comp rx=0 tx=0).stop 2024-05-17T22:54:18.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.751+0000 7fb346e1bb80 1 -- 172.21.15.12:0/2963155232 shutdown_connections 2024-05-17T22:54:18.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.751+0000 7fb346e1bb80 1 --2- 172.21.15.12:0/2963155232 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55926ace9f70 0x55926acec3d0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:18.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.752+0000 7fb319ffb700 0 client.26309 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:54:18.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.752+0000 7fb346e1bb80 1 --2- 172.21.15.12:0/2963155232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55926acd6400 0x55926acda4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:18.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.752+0000 7fb346e1bb80 1 --2- 172.21.15.12:0/2963155232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55926ab529f0 0x55926acd9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:18.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.752+0000 7fb346e1bb80 1 --2- 172.21.15.12:0/2963155232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55926acd0b00 0x55926acd9fa0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:18.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.752+0000 7fb346e1bb80 1 -- 172.21.15.12:0/2963155232 >> 172.21.15.12:0/2963155232 conn(0x55926abb27d0 msgr2=0x55926accb160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:18.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.752+0000 7fb346e1bb80 1 -- 172.21.15.12:0/2963155232 shutdown_connections 2024-05-17T22:54:18.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.752+0000 7fb346e1bb80 1 -- 172.21.15.12:0/2963155232 wait complete. 2024-05-17T22:54:18.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.752+0000 7fb346e1bb80 20 client.26309 trim_cache size 0 max 16384 2024-05-17T22:54:18.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.776+0000 7fb346e1bb80 1 Processor -- start 2024-05-17T22:54:18.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.777+0000 7fb346e1bb80 1 -- start start 2024-05-17T22:54:18.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.777+0000 7fb346e1bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55926ab529f0 0x55926acd05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:18.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.777+0000 7fb346e1bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55926acd0b00 0x55926acda330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:18.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.777+0000 7fb346e1bb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55926acd6400 0x55926acd51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:18.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.777+0000 7fb346e1bb80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb314011330 con 0x55926ab529f0 2024-05-17T22:54:18.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.777+0000 7fb346e1bb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb31400ebc0 con 0x55926acd6400 2024-05-17T22:54:18.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.777+0000 7fb346e1bb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb31400ca10 con 0x55926acd0b00 2024-05-17T22:54:18.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.777+0000 7fb3361dc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55926ab529f0 0x55926acd05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:18.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.777+0000 7fb3369dd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55926acd6400 0x55926acd51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:18.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.777+0000 7fb3369dd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55926acd6400 0x55926acd51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:48102/0 (socket says 172.21.15.12:48102) 2024-05-17T22:54:18.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.777+0000 7fb3369dd700 1 -- 172.21.15.12:0/4261781827 learned_addr learned my addr 172.21.15.12:0/4261781827 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:54:18.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.778+0000 7fb3359db700 1 --2- 172.21.15.12:0/4261781827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55926acd0b00 0x55926acda330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:18.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.778+0000 7fb3369dd700 1 -- 172.21.15.12:0/4261781827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55926acd0b00 msgr2=0x55926acda330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:18.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.778+0000 7fb3369dd700 1 --2- 172.21.15.12:0/4261781827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55926acd0b00 0x55926acda330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:18.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.778+0000 7fb3369dd700 1 -- 172.21.15.12:0/4261781827 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55926ab529f0 msgr2=0x55926acd05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:18.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.778+0000 7fb3369dd700 1 --2- 172.21.15.12:0/4261781827 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55926ab529f0 0x55926acd05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:18.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.778+0000 7fb3369dd700 1 -- 172.21.15.12:0/4261781827 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55926acc35a0 con 0x55926acd6400 2024-05-17T22:54:18.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.778+0000 7fb3361dc700 1 --2- 172.21.15.12:0/4261781827 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55926ab529f0 0x55926acd05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:54:18.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.778+0000 7fb3369dd700 1 --2- 172.21.15.12:0/4261781827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55926acd6400 0x55926acd51b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb3200186b0 tx=0x7fb3200a4b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34be12bf0289e19c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:18.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.778+0000 7fb31bfff700 1 -- 172.21.15.12:0/4261781827 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32006c030 con 0x55926acd6400 2024-05-17T22:54:18.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.778+0000 7fb31bfff700 1 -- 172.21.15.12:0/4261781827 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3200c3350 con 0x55926acd6400 2024-05-17T22:54:18.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.778+0000 7fb31bfff700 1 -- 172.21.15.12:0/4261781827 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb320016040 con 0x55926acd6400 2024-05-17T22:54:18.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.780+0000 7fb346e1bb80 1 -- 172.21.15.12:0/4261781827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55926acd6400 msgr2=0x55926acd51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.780+0000 7fb346e1bb80 1 --2- 172.21.15.12:0/4261781827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55926acd6400 0x55926acd51b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb3200186b0 tx=0x7fb3200a4b40 comp rx=0 tx=0).stop 2024-05-17T22:54:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.780+0000 7fb346e1bb80 1 -- 172.21.15.12:0/4261781827 shutdown_connections 2024-05-17T22:54:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.780+0000 7fb346e1bb80 1 --2- 172.21.15.12:0/4261781827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55926acd6400 0x55926acd51b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.780+0000 7fb346e1bb80 1 --2- 172.21.15.12:0/4261781827 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55926ab529f0 0x55926acd05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.780+0000 7fb346e1bb80 1 --2- 172.21.15.12:0/4261781827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55926acd0b00 0x55926acda330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.780+0000 7fb346e1bb80 1 -- 172.21.15.12:0/4261781827 >> 172.21.15.12:0/4261781827 conn(0x55926abb27d0 msgr2=0x55926acd4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.781+0000 7fb346e1bb80 1 -- 172.21.15.12:0/4261781827 shutdown_connections 2024-05-17T22:54:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.781+0000 7fb346e1bb80 1 -- 172.21.15.12:0/4261781827 wait complete. 2024-05-17T22:54:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.781+0000 7fb346e1bb80 1 -- start start 2024-05-17T22:54:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.781+0000 7fb346e1bb80 1 Processor -- start 2024-05-17T22:54:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.782+0000 7fb346e1bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55926ab529f0 0x55926ace2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.782+0000 7fb346e1bb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55926acd0b00 0x55926acdf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.782+0000 7fb346e1bb80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55926acd6400 0x55926acdf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:18.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.782+0000 7fb3359db700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55926acd0b00 0x55926acdf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:18.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.782+0000 7fb3361dc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55926ab529f0 0x55926ace2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:18.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.782+0000 7fb3359db700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55926acd0b00 0x55926acdf060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:48110/0 (socket says 172.21.15.12:48110) 2024-05-17T22:54:18.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.782+0000 7fb3361dc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55926ab529f0 0x55926ace2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.12:37874/0 (socket says 172.21.15.12:37874) 2024-05-17T22:54:18.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.782+0000 7fb3359db700 1 -- 172.21.15.12:0/913941563 learned_addr learned my addr 172.21.15.12:0/913941563 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:54:18.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.782+0000 7fb346e1bb80 1 -- 172.21.15.12:0/913941563 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb31400d080 con 0x55926acd6400 2024-05-17T22:54:18.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.782+0000 7fb346e1bb80 1 -- 172.21.15.12:0/913941563 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb31400e4f0 con 0x55926acd0b00 2024-05-17T22:54:18.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.782+0000 7fb346e1bb80 1 -- 172.21.15.12:0/913941563 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb31400d750 con 0x55926ab529f0 2024-05-17T22:54:18.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.782+0000 7fb346e1bb80 10 client.0 _ll_register_callbacks cb 0x55926ab85b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:54:18.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.782+0000 7fb3359db700 1 -- 172.21.15.12:0/913941563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55926ab529f0 msgr2=0x55926ace2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:18.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.782+0000 7fb3359db700 1 --2- 172.21.15.12:0/913941563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55926ab529f0 0x55926ace2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:18.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.782+0000 7fb3359db700 1 -- 172.21.15.12:0/913941563 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55926acd6400 msgr2=0x55926acdf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:18.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.782+0000 7fb3359db700 1 --2- 172.21.15.12:0/913941563 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55926acd6400 0x55926acdf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:18.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.782+0000 7fb3359db700 1 -- 172.21.15.12:0/913941563 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55926ace2a10 con 0x55926acd0b00 2024-05-17T22:54:18.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.783+0000 7fb3359db700 1 --2- 172.21.15.12:0/913941563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55926acd0b00 0x55926acdf060 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x55926acdb220 tx=0x7fb328018a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db36962ecca4e7ef server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:18.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.783+0000 7fb319ffb700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-17T22:54:18.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.783+0000 7fb319ffb700 1 -- 172.21.15.12:0/913941563 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32801dea0 con 0x55926acd0b00 2024-05-17T22:54:18.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.783+0000 7fb319ffb700 1 -- 172.21.15.12:0/913941563 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fb3280378d0 con 0x55926acd0b00 2024-05-17T22:54:18.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.784+0000 7fb346e1bb80 1 -- 172.21.15.12:0/913941563 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55926abfa800 con 0x55926acd0b00 2024-05-17T22:54:18.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.784+0000 7fb319ffb700 1 -- 172.21.15.12:0/913941563 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fb328037ad0 con 0x55926acd0b00 2024-05-17T22:54:18.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.784+0000 7fb346e1bb80 10 client.26315 fetch_fsmap learned FSMap version 43 2024-05-17T22:54:18.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.784+0000 7fb346e1bb80 1 -- 172.21.15.12:0/913941563 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55926ad550f0 con 0x55926acd0b00 2024-05-17T22:54:18.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.785+0000 7fb319ffb700 1 -- 172.21.15.12:0/913941563 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb328037c70 con 0x55926acd0b00 2024-05-17T22:54:18.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.785+0000 7fb319ffb700 1 -- 172.21.15.12:0/913941563 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7fb328022040 con 0x55926acd0b00 2024-05-17T22:54:18.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.785+0000 7fb346e1bb80 10 client.26315 fetch_fsmap finished waiting for FSMap version 43 2024-05-17T22:54:18.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.785+0000 7fb346e1bb80 10 client.26315 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:54:18.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.785+0000 7fb346e1bb80 20 client.26315 populate_metadata read hostname 'smithi012' 2024-05-17T22:54:18.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.785+0000 7fb346e1bb80 1 --2- 172.21.15.12:0/913941563 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55926ad55740 0x55926ad57b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:18.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.785+0000 7fb346e1bb80 4 client.26315 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:54:18.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.785+0000 7fb346e1bb80 1 -- 172.21.15.12:0/913941563 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55926abfa800 con 0x55926ad55740 2024-05-17T22:54:18.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.785+0000 7fb3361dc700 1 --2- 172.21.15.12:0/913941563 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55926ad55740 0x55926ad57b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:18.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.786+0000 7fb319ffb700 10 client.26315 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:54:18.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.787+0000 7fb3361dc700 1 --2- 172.21.15.12:0/913941563 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55926ad55740 0x55926ad57b60 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eca4ab94ab6d6f2 server_cookie=6a81edd546447410 in_seq=0 out_seq=0 2024-05-17T22:54:18.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.787+0000 7fb319ffb700 1 -- 172.21.15.12:0/913941563 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55926abfa800 con 0x55926ad55740 2024-05-17T22:54:18.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.787+0000 7fb319ffb700 10 client.26315 handle_command_reply: tid=0 2024-05-17T22:54:18.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.787+0000 7fb346e1bb80 1 client.26315 shutdown 2024-05-17T22:54:18.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.787+0000 7fb346e1bb80 1 -- 172.21.15.12:0/913941563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55926acd0b00 msgr2=0x55926acdf060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:18.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.787+0000 7fb346e1bb80 1 --2- 172.21.15.12:0/913941563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55926acd0b00 0x55926acdf060 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x55926acdb220 tx=0x7fb328018a10 comp rx=0 tx=0).stop 2024-05-17T22:54:18.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.787+0000 7fb346e1bb80 1 -- 172.21.15.12:0/913941563 shutdown_connections 2024-05-17T22:54:18.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.787+0000 7fb346e1bb80 1 --2- 172.21.15.12:0/913941563 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55926ad55740 0x55926ad57b60 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:18.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.787+0000 7fb346e1bb80 1 --2- 172.21.15.12:0/913941563 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55926acd6400 0x55926acdf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:18.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.787+0000 7fb346e1bb80 1 --2- 172.21.15.12:0/913941563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55926ab529f0 0x55926ace2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:18.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.787+0000 7fb346e1bb80 1 --2- 172.21.15.12:0/913941563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55926acd0b00 0x55926acdf060 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:18.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.787+0000 7fb346e1bb80 1 -- 172.21.15.12:0/913941563 >> 172.21.15.12:0/913941563 conn(0x55926abb27d0 msgr2=0x55926acd4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:18.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.787+0000 7fb319ffb700 0 client.26315 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:54:18.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.787+0000 7fb346e1bb80 1 -- 172.21.15.12:0/913941563 shutdown_connections 2024-05-17T22:54:18.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.787+0000 7fb346e1bb80 1 -- 172.21.15.12:0/913941563 wait complete. 2024-05-17T22:54:18.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.788+0000 7fb346e1bb80 20 client.26315 trim_cache size 0 max 16384 2024-05-17T22:54:18.793 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:54:18.794 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-17T22:54:18.794 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-05-17T22:54:18.794 INFO:teuthology.orchestra.run.smithi012.stdout: "581decdf-3470-4b25-953e-76e92b49291c": { 2024-05-17T22:54:18.794 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-05-17T22:54:18.794 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "581decdf-3470-4b25-953e-76e92b49291c", 2024-05-17T22:54:18.794 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-05-17T22:54:18.794 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:54:18.794 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:54:18.794 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:54:18.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.788+0000 7fb338440700 1 -- 172.21.15.12:0/1372129954 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb31c0b2d40 msgr2=0x7fb31c0b5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:18.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.788+0000 7fb338440700 1 --2- 172.21.15.12:0/1372129954 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb31c0b2d40 0x7fb31c0b5250 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x55926accc1d0 tx=0x7fb32c009260 comp rx=0 tx=0).stop 2024-05-17T22:54:18.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.789+0000 7fb338440700 1 -- 172.21.15.12:0/1372129954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb330108080 msgr2=0x7fb3300708c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:18.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.789+0000 7fb338440700 1 --2- 172.21.15.12:0/1372129954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb330108080 0x7fb3300708c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb32000d720 tx=0x7fb32000da30 comp rx=0 tx=0).stop 2024-05-17T22:54:18.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.789+0000 7fb338440700 1 -- 172.21.15.12:0/1372129954 shutdown_connections 2024-05-17T22:54:18.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.789+0000 7fb338440700 1 --2- 172.21.15.12:0/1372129954 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb31c0b2d40 0x7fb31c0b5250 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:18.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.789+0000 7fb338440700 1 --2- 172.21.15.12:0/1372129954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb330108080 0x7fb3300708c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:18.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.789+0000 7fb338440700 1 --2- 172.21.15.12:0/1372129954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb330102e00 0x7fb330075250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:18.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.789+0000 7fb338440700 1 --2- 172.21.15.12:0/1372129954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb330107730 0x7fb330070380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:18.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.789+0000 7fb338440700 1 -- 172.21.15.12:0/1372129954 >> 172.21.15.12:0/1372129954 conn(0x7fb330079bd0 msgr2=0x7fb330105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:18.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.789+0000 7fb338440700 1 -- 172.21.15.12:0/1372129954 shutdown_connections 2024-05-17T22:54:18.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:18.789+0000 7fb338440700 1 -- 172.21.15.12:0/1372129954 wait complete. 2024-05-17T22:54:19.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:19 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:54:19.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:19 smithi012 ceph-mon[31632]: pgmap v891: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.0 MiB/s wr, 415 op/s 2024-05-17T22:54:19.687 INFO:tasks.cephfs.filesystem:scrub status for tag:581decdf-3470-4b25-953e-76e92b49291c - {'path': '/', 'tag': '581decdf-3470-4b25-953e-76e92b49291c', 'options': 'recursive,force'} 2024-05-17T22:54:19.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:19 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:54:19.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:19 smithi151 ceph-mon[36101]: pgmap v891: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.0 MiB/s wr, 415 op/s 2024-05-17T22:54:19.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:19 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:54:19.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:19 smithi096 ceph-mon[36065]: pgmap v891: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.0 MiB/s wr, 415 op/s 2024-05-17T22:54:20.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:20 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:54:20.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:20 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:54:20.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:20 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:54:20.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:20 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:54:20.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:20 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:54:20.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:20 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:54:20.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:20 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:54:20.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:20 smithi151 ceph-mon[36101]: Adjusting osd_memory_target on smithi096 to 176.2M 2024-05-17T22:54:20.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:20 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:54:20.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:20 smithi151 ceph-mon[36101]: Adjusting osd_memory_target on smithi151 to 176.2M 2024-05-17T22:54:20.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:20 smithi151 ceph-mon[36101]: Unable to set osd_memory_target on smithi096 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-05-17T22:54:20.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:20 smithi151 ceph-mon[36101]: Unable to set osd_memory_target on smithi151 to 184786739: error parsing value: Value '184786739' is below minimum 939524096 2024-05-17T22:54:20.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:20 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:54:20.742 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:20 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:54:20.742 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:20 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:54:20.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:20 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:54:20.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:20 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:54:20.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:20 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:54:20.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:20 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:54:20.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:20 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:54:20.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:20 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:54:20.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:20 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:54:20.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:20 smithi096 ceph-mon[36065]: Adjusting osd_memory_target on smithi096 to 176.2M 2024-05-17T22:54:20.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:20 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:54:20.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:20 smithi096 ceph-mon[36065]: Adjusting osd_memory_target on smithi151 to 176.2M 2024-05-17T22:54:20.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:20 smithi096 ceph-mon[36065]: Unable to set osd_memory_target on smithi096 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-05-17T22:54:20.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:20 smithi096 ceph-mon[36065]: Unable to set osd_memory_target on smithi151 to 184786739: error parsing value: Value '184786739' is below minimum 939524096 2024-05-17T22:54:20.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:20 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:54:20.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:20 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:54:20.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:20 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:54:20.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:20 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:54:20.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:20 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:54:20.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:20 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:54:20.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:20 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:54:20.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:20 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:54:20.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:20 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:54:20.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:20 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:54:20.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:20 smithi012 ceph-mon[31632]: Adjusting osd_memory_target on smithi096 to 176.2M 2024-05-17T22:54:20.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:20 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-17T22:54:20.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:20 smithi012 ceph-mon[31632]: Adjusting osd_memory_target on smithi151 to 176.2M 2024-05-17T22:54:20.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:20 smithi012 ceph-mon[31632]: Unable to set osd_memory_target on smithi096 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-05-17T22:54:20.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:20 smithi012 ceph-mon[31632]: Unable to set osd_memory_target on smithi151 to 184786739: error parsing value: Value '184786739' is below minimum 939524096 2024-05-17T22:54:20.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:20 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:54:20.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:20 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:54:20.843 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:20 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:54:21.369 INFO:teuthology.orchestra.run.smithi012.stderr:progress: 900.0 s, 423.4 tps, lat 239.181 ms stddev 282.451 2024-05-17T22:54:21.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:21 smithi151 ceph-mon[36101]: pgmap v892: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.4 MiB/s wr, 512 op/s 2024-05-17T22:54:21.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:21 smithi096 ceph-mon[36065]: pgmap v892: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.4 MiB/s wr, 512 op/s 2024-05-17T22:54:21.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:21 smithi012 ceph-mon[31632]: pgmap v892: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.4 MiB/s wr, 512 op/s 2024-05-17T22:54:22.066 INFO:teuthology.orchestra.run.smithi012.stdout:transaction type: 2024-05-17T22:54:22.066 INFO:teuthology.orchestra.run.smithi012.stdout:scaling factor: 32 2024-05-17T22:54:22.066 INFO:teuthology.orchestra.run.smithi012.stdout:query mode: simple 2024-05-17T22:54:22.066 INFO:teuthology.orchestra.run.smithi012.stdout:number of clients: 100 2024-05-17T22:54:22.067 INFO:teuthology.orchestra.run.smithi012.stdout:number of threads: 4 2024-05-17T22:54:22.067 INFO:teuthology.orchestra.run.smithi012.stdout:duration: 900 s 2024-05-17T22:54:22.067 INFO:teuthology.orchestra.run.smithi012.stdout:number of transactions actually processed: 838752 2024-05-17T22:54:22.067 INFO:teuthology.orchestra.run.smithi012.stdout:latency average = 107.339 ms 2024-05-17T22:54:22.067 INFO:teuthology.orchestra.run.smithi012.stdout:latency stddev = 176.540 ms 2024-05-17T22:54:22.067 INFO:teuthology.orchestra.run.smithi012.stdout:tps = 931.225332 (including connections establishing) 2024-05-17T22:54:22.067 INFO:teuthology.orchestra.run.smithi012.stdout:tps = 931.227886 (excluding connections establishing) 2024-05-17T22:54:22.069 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2024-05-17T22:54:24.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:24 smithi096 ceph-mon[36065]: pgmap v893: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.1 MiB/s wr, 440 op/s 2024-05-17T22:54:24.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:24 smithi096 ceph-mon[36065]: scrub summary: idle 2024-05-17T22:54:24.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:24 smithi012 ceph-mon[31632]: pgmap v893: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.1 MiB/s wr, 440 op/s 2024-05-17T22:54:24.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:24 smithi012 ceph-mon[31632]: scrub summary: idle 2024-05-17T22:54:24.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:24 smithi151 ceph-mon[36101]: pgmap v893: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.1 MiB/s wr, 440 op/s 2024-05-17T22:54:24.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:24 smithi151 ceph-mon[36101]: scrub summary: idle 2024-05-17T22:54:25.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:25 smithi096 ceph-mon[36065]: scrub summary: idle 2024-05-17T22:54:25.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:25 smithi096 ceph-mon[36065]: scrub summary: idle 2024-05-17T22:54:25.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:25 smithi012 ceph-mon[31632]: scrub summary: idle 2024-05-17T22:54:25.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:25 smithi012 ceph-mon[31632]: scrub summary: idle 2024-05-17T22:54:25.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:25 smithi151 ceph-mon[36101]: scrub summary: idle 2024-05-17T22:54:25.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:25 smithi151 ceph-mon[36101]: scrub summary: idle 2024-05-17T22:54:26.298 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:26 smithi096 ceph-mon[36065]: pgmap v894: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 6.4 MiB/s wr, 539 op/s 2024-05-17T22:54:26.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:26 smithi012 ceph-mon[31632]: pgmap v894: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 6.4 MiB/s wr, 539 op/s 2024-05-17T22:54:26.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:26 smithi151 ceph-mon[36101]: pgmap v894: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 6.4 MiB/s wr, 539 op/s 2024-05-17T22:54:28.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:28 smithi012 ceph-mon[31632]: pgmap v895: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 6.5 MiB/s wr, 536 op/s 2024-05-17T22:54:28.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:28 smithi151 ceph-mon[36101]: pgmap v895: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 6.5 MiB/s wr, 536 op/s 2024-05-17T22:54:28.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:28 smithi096 ceph-mon[36065]: pgmap v895: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 6.5 MiB/s wr, 536 op/s 2024-05-17T22:54:29.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:29 smithi012 ceph-mon[31632]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:54:29.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:29 smithi151 ceph-mon[36101]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:54:29.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:29 smithi096 ceph-mon[36065]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:54:30.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:30 smithi012 ceph-mon[31632]: pgmap v896: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 6.5 MiB/s wr, 543 op/s 2024-05-17T22:54:30.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:30 smithi151 ceph-mon[36101]: pgmap v896: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 6.5 MiB/s wr, 543 op/s 2024-05-17T22:54:30.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:30 smithi096 ceph-mon[36065]: pgmap v896: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 6.5 MiB/s wr, 543 op/s 2024-05-17T22:54:32.343 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:32 smithi012 ceph-mon[31632]: pgmap v897: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 7.6 MiB/s wr, 685 op/s 2024-05-17T22:54:32.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:32 smithi151 ceph-mon[36101]: pgmap v897: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 7.6 MiB/s wr, 685 op/s 2024-05-17T22:54:32.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:32 smithi096 ceph-mon[36065]: pgmap v897: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 7.6 MiB/s wr, 685 op/s 2024-05-17T22:54:34.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:34 smithi012 ceph-mon[31632]: pgmap v898: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 6.7 MiB/s wr, 649 op/s 2024-05-17T22:54:34.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:34 smithi151 ceph-mon[36101]: pgmap v898: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 6.7 MiB/s wr, 649 op/s 2024-05-17T22:54:34.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:34 smithi096 ceph-mon[36065]: pgmap v898: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 6.7 MiB/s wr, 649 op/s 2024-05-17T22:54:35.724 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-05-17T22:54:35.798 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/: 2024-05-17T22:54:35.798 INFO:teuthology.orchestra.run.smithi012.stdout:total 0 2024-05-17T22:54:35.799 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 17 22:39 . 2024-05-17T22:54:35.799 INFO:teuthology.orchestra.run.smithi012.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 May 17 22:39 .. 2024-05-17T22:54:35.799 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 17 22:54 data 2024-05-17T22:54:35.799 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:54:35.799 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data: 2024-05-17T22:54:35.799 INFO:teuthology.orchestra.run.smithi012.stdout:total 36 2024-05-17T22:54:35.799 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 17 22:54 . 2024-05-17T22:54:35.799 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 17 22:39 .. 2024-05-17T22:54:35.799 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 May 17 22:39 base 2024-05-17T22:54:35.799 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 May 17 22:39 current_logfiles 2024-05-17T22:54:35.799 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 May 17 22:39 global 2024-05-17T22:54:35.799 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 17 22:39 log 2024-05-17T22:54:35.799 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 pg_commit_ts 2024-05-17T22:54:35.799 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 pg_dynshmem 2024-05-17T22:54:35.800 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 May 17 22:39 pg_hba.conf 2024-05-17T22:54:35.800 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 May 17 22:39 pg_ident.conf 2024-05-17T22:54:35.800 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 17 22:54 pg_logical 2024-05-17T22:54:35.800 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 17 22:39 pg_multixact 2024-05-17T22:54:35.800 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 pg_notify 2024-05-17T22:54:35.800 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 pg_replslot 2024-05-17T22:54:35.800 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 pg_serial 2024-05-17T22:54:35.800 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 pg_snapshots 2024-05-17T22:54:35.800 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 17 22:54 pg_stat 2024-05-17T22:54:35.800 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:54 pg_stat_tmp 2024-05-17T22:54:35.800 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 17 22:54 pg_subtrans 2024-05-17T22:54:35.800 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 pg_tblspc 2024-05-17T22:54:35.800 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 pg_twophase 2024-05-17T22:54:35.801 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 17 22:39 PG_VERSION 2024-05-17T22:54:35.801 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 62 May 17 22:54 pg_wal 2024-05-17T22:54:35.801 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 17 22:39 pg_xact 2024-05-17T22:54:35.801 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 May 17 22:39 postgresql.auto.conf 2024-05-17T22:54:35.801 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 May 17 22:39 postgresql.conf 2024-05-17T22:54:35.801 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 May 17 22:39 postmaster.opts 2024-05-17T22:54:35.801 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:54:35.801 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/base: 2024-05-17T22:54:35.801 INFO:teuthology.orchestra.run.smithi012.stdout:total 0 2024-05-17T22:54:35.801 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 May 17 22:39 . 2024-05-17T22:54:35.801 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 17 22:54 .. 2024-05-17T22:54:35.801 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 May 17 22:39 1 2024-05-17T22:54:35.801 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 May 17 22:39 13493 2024-05-17T22:54:35.801 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 May 17 22:46 13494 2024-05-17T22:54:35.802 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 pgsql_tmp 2024-05-17T22:54:35.802 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:54:35.802 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/base/1: 2024-05-17T22:54:35.802 INFO:teuthology.orchestra.run.smithi012.stdout:total 7777 2024-05-17T22:54:35.802 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 May 17 22:39 . 2024-05-17T22:54:35.802 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 May 17 22:39 .. 2024-05-17T22:54:35.802 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 112 2024-05-17T22:54:35.802 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 113 2024-05-17T22:54:35.802 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 17 22:39 1247 2024-05-17T22:54:35.802 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 1247_fsm 2024-05-17T22:54:35.802 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1247_vm 2024-05-17T22:54:35.802 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 17 22:39 1249 2024-05-17T22:54:35.802 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 1249_fsm 2024-05-17T22:54:35.803 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1249_vm 2024-05-17T22:54:35.803 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 17 22:39 1255 2024-05-17T22:54:35.803 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 1255_fsm 2024-05-17T22:54:35.803 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1255_vm 2024-05-17T22:54:35.803 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 17 22:39 1259 2024-05-17T22:54:35.803 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 1259_fsm 2024-05-17T22:54:35.803 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1259_vm 2024-05-17T22:54:35.803 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 17 22:39 13344 2024-05-17T22:54:35.803 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 13344_fsm 2024-05-17T22:54:35.803 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13344_vm 2024-05-17T22:54:35.803 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 13346 2024-05-17T22:54:35.803 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13348 2024-05-17T22:54:35.803 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13349 2024-05-17T22:54:35.804 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 13349_fsm 2024-05-17T22:54:35.804 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13349_vm 2024-05-17T22:54:35.804 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 13351 2024-05-17T22:54:35.804 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13353 2024-05-17T22:54:35.804 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13354 2024-05-17T22:54:35.804 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 13354_fsm 2024-05-17T22:54:35.804 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13354_vm 2024-05-17T22:54:35.804 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 13356 2024-05-17T22:54:35.804 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13358 2024-05-17T22:54:35.804 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13359 2024-05-17T22:54:35.804 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 13359_fsm 2024-05-17T22:54:35.804 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13359_vm 2024-05-17T22:54:35.804 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 13361 2024-05-17T22:54:35.804 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13363 2024-05-17T22:54:35.805 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 1417 2024-05-17T22:54:35.805 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 1418 2024-05-17T22:54:35.805 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 174 2024-05-17T22:54:35.805 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 175 2024-05-17T22:54:35.805 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2187 2024-05-17T22:54:35.805 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2224 2024-05-17T22:54:35.805 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2328 2024-05-17T22:54:35.805 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2336 2024-05-17T22:54:35.805 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2337 2024-05-17T22:54:35.805 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2579 2024-05-17T22:54:35.805 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2600 2024-05-17T22:54:35.805 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2600_fsm 2024-05-17T22:54:35.805 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2600_vm 2024-05-17T22:54:35.805 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2601 2024-05-17T22:54:35.806 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2601_fsm 2024-05-17T22:54:35.806 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2601_vm 2024-05-17T22:54:35.806 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 17 22:39 2602 2024-05-17T22:54:35.806 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2602_fsm 2024-05-17T22:54:35.806 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2602_vm 2024-05-17T22:54:35.806 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 2603 2024-05-17T22:54:35.806 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2603_fsm 2024-05-17T22:54:35.806 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2603_vm 2024-05-17T22:54:35.806 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2604 2024-05-17T22:54:35.806 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2605 2024-05-17T22:54:35.806 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2605_fsm 2024-05-17T22:54:35.806 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2605_vm 2024-05-17T22:54:35.806 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2606 2024-05-17T22:54:35.807 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2606_fsm 2024-05-17T22:54:35.807 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2606_vm 2024-05-17T22:54:35.807 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2607 2024-05-17T22:54:35.807 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2607_fsm 2024-05-17T22:54:35.807 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2607_vm 2024-05-17T22:54:35.807 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 17 22:39 2608 2024-05-17T22:54:35.807 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2608_fsm 2024-05-17T22:54:35.807 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2608_vm 2024-05-17T22:54:35.807 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 17 22:39 2609 2024-05-17T22:54:35.807 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2609_fsm 2024-05-17T22:54:35.807 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2609_vm 2024-05-17T22:54:35.807 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 2610 2024-05-17T22:54:35.807 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2610_fsm 2024-05-17T22:54:35.808 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2610_vm 2024-05-17T22:54:35.808 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2611 2024-05-17T22:54:35.808 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2612 2024-05-17T22:54:35.808 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2612_fsm 2024-05-17T22:54:35.808 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2612_vm 2024-05-17T22:54:35.808 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2613 2024-05-17T22:54:35.808 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2615 2024-05-17T22:54:35.808 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2615_fsm 2024-05-17T22:54:35.808 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2615_vm 2024-05-17T22:54:35.808 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2616 2024-05-17T22:54:35.808 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2616_fsm 2024-05-17T22:54:35.808 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2616_vm 2024-05-17T22:54:35.808 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 17 22:39 2617 2024-05-17T22:54:35.808 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2617_fsm 2024-05-17T22:54:35.809 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2617_vm 2024-05-17T22:54:35.809 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 17 22:39 2618 2024-05-17T22:54:35.809 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2618_fsm 2024-05-17T22:54:35.809 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2618_vm 2024-05-17T22:54:35.809 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 17 22:39 2619 2024-05-17T22:54:35.809 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2619_fsm 2024-05-17T22:54:35.809 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2619_vm 2024-05-17T22:54:35.809 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2620 2024-05-17T22:54:35.809 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2650 2024-05-17T22:54:35.809 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2651 2024-05-17T22:54:35.809 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2652 2024-05-17T22:54:35.809 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 22:39 2653 2024-05-17T22:54:35.810 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 22:39 2654 2024-05-17T22:54:35.810 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 2655 2024-05-17T22:54:35.810 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2656 2024-05-17T22:54:35.810 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2657 2024-05-17T22:54:35.810 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 17 22:39 2658 2024-05-17T22:54:35.810 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 17 22:39 2659 2024-05-17T22:54:35.810 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2660 2024-05-17T22:54:35.810 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2661 2024-05-17T22:54:35.810 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 2662 2024-05-17T22:54:35.810 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 17 22:39 2663 2024-05-17T22:54:35.810 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2664 2024-05-17T22:54:35.810 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2665 2024-05-17T22:54:35.810 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2666 2024-05-17T22:54:35.810 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2667 2024-05-17T22:54:35.811 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2668 2024-05-17T22:54:35.811 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2669 2024-05-17T22:54:35.811 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2670 2024-05-17T22:54:35.811 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 17 22:39 2673 2024-05-17T22:54:35.811 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 17 22:39 2674 2024-05-17T22:54:35.811 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 17 22:39 2675 2024-05-17T22:54:35.811 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2678 2024-05-17T22:54:35.811 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2679 2024-05-17T22:54:35.811 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2680 2024-05-17T22:54:35.811 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2681 2024-05-17T22:54:35.811 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2682 2024-05-17T22:54:35.811 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2683 2024-05-17T22:54:35.811 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2684 2024-05-17T22:54:35.811 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2685 2024-05-17T22:54:35.812 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2686 2024-05-17T22:54:35.812 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2687 2024-05-17T22:54:35.812 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 22:39 2688 2024-05-17T22:54:35.812 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 17 22:39 2689 2024-05-17T22:54:35.812 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 17 22:39 2690 2024-05-17T22:54:35.812 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 17 22:39 2691 2024-05-17T22:54:35.812 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2692 2024-05-17T22:54:35.812 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2693 2024-05-17T22:54:35.812 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2696 2024-05-17T22:54:35.812 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2699 2024-05-17T22:54:35.812 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2701 2024-05-17T22:54:35.812 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2702 2024-05-17T22:54:35.812 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 2703 2024-05-17T22:54:35.813 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 22:39 2704 2024-05-17T22:54:35.813 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2753 2024-05-17T22:54:35.813 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2753_fsm 2024-05-17T22:54:35.813 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2753_vm 2024-05-17T22:54:35.813 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2754 2024-05-17T22:54:35.813 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2755 2024-05-17T22:54:35.813 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 2756 2024-05-17T22:54:35.813 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 2757 2024-05-17T22:54:35.813 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2830 2024-05-17T22:54:35.813 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2831 2024-05-17T22:54:35.813 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2832 2024-05-17T22:54:35.813 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2833 2024-05-17T22:54:35.813 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2834 2024-05-17T22:54:35.814 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2835 2024-05-17T22:54:35.814 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2836 2024-05-17T22:54:35.814 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2837 2024-05-17T22:54:35.814 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 17 22:39 2838 2024-05-17T22:54:35.814 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2838_fsm 2024-05-17T22:54:35.814 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2838_vm 2024-05-17T22:54:35.814 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2839 2024-05-17T22:54:35.814 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2840 2024-05-17T22:54:35.814 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2840_fsm 2024-05-17T22:54:35.814 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2840_vm 2024-05-17T22:54:35.814 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2841 2024-05-17T22:54:35.814 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2995 2024-05-17T22:54:35.814 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2996 2024-05-17T22:54:35.815 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3079 2024-05-17T22:54:35.815 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3079_fsm 2024-05-17T22:54:35.815 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3079_vm 2024-05-17T22:54:35.815 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3080 2024-05-17T22:54:35.815 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3081 2024-05-17T22:54:35.815 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 22:39 3085 2024-05-17T22:54:35.815 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3118 2024-05-17T22:54:35.815 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3119 2024-05-17T22:54:35.815 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 17 22:39 3164 2024-05-17T22:54:35.815 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3256 2024-05-17T22:54:35.815 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3257 2024-05-17T22:54:35.815 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3258 2024-05-17T22:54:35.815 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3350 2024-05-17T22:54:35.815 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3351 2024-05-17T22:54:35.816 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3379 2024-05-17T22:54:35.816 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3380 2024-05-17T22:54:35.816 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3381 2024-05-17T22:54:35.816 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3394 2024-05-17T22:54:35.816 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3394_fsm 2024-05-17T22:54:35.816 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3394_vm 2024-05-17T22:54:35.816 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3395 2024-05-17T22:54:35.816 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3429 2024-05-17T22:54:35.816 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3430 2024-05-17T22:54:35.816 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3431 2024-05-17T22:54:35.816 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3433 2024-05-17T22:54:35.816 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3439 2024-05-17T22:54:35.816 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3440 2024-05-17T22:54:35.816 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 22:39 3455 2024-05-17T22:54:35.817 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 17 22:39 3456 2024-05-17T22:54:35.817 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3456_fsm 2024-05-17T22:54:35.817 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3456_vm 2024-05-17T22:54:35.817 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3466 2024-05-17T22:54:35.817 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3467 2024-05-17T22:54:35.817 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3468 2024-05-17T22:54:35.817 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3501 2024-05-17T22:54:35.817 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3502 2024-05-17T22:54:35.817 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3503 2024-05-17T22:54:35.817 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3534 2024-05-17T22:54:35.817 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3541 2024-05-17T22:54:35.817 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3541_fsm 2024-05-17T22:54:35.817 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3541_vm 2024-05-17T22:54:35.817 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3542 2024-05-17T22:54:35.818 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3574 2024-05-17T22:54:35.818 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3575 2024-05-17T22:54:35.818 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3576 2024-05-17T22:54:35.818 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3596 2024-05-17T22:54:35.818 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3597 2024-05-17T22:54:35.818 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3598 2024-05-17T22:54:35.818 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3599 2024-05-17T22:54:35.818 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3600 2024-05-17T22:54:35.818 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3600_fsm 2024-05-17T22:54:35.818 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3600_vm 2024-05-17T22:54:35.818 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3601 2024-05-17T22:54:35.818 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3601_fsm 2024-05-17T22:54:35.818 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3601_vm 2024-05-17T22:54:35.819 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3602 2024-05-17T22:54:35.819 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3602_fsm 2024-05-17T22:54:35.819 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3602_vm 2024-05-17T22:54:35.819 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3603 2024-05-17T22:54:35.819 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3603_fsm 2024-05-17T22:54:35.819 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3603_vm 2024-05-17T22:54:35.819 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3604 2024-05-17T22:54:35.819 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3605 2024-05-17T22:54:35.819 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3606 2024-05-17T22:54:35.819 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3607 2024-05-17T22:54:35.819 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3608 2024-05-17T22:54:35.819 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 3609 2024-05-17T22:54:35.819 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3712 2024-05-17T22:54:35.819 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3764 2024-05-17T22:54:35.820 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3764_fsm 2024-05-17T22:54:35.820 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3764_vm 2024-05-17T22:54:35.820 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3766 2024-05-17T22:54:35.820 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3767 2024-05-17T22:54:35.820 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3997 2024-05-17T22:54:35.820 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4143 2024-05-17T22:54:35.820 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4144 2024-05-17T22:54:35.820 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4145 2024-05-17T22:54:35.820 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4146 2024-05-17T22:54:35.820 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4147 2024-05-17T22:54:35.820 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4148 2024-05-17T22:54:35.820 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4149 2024-05-17T22:54:35.820 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4150 2024-05-17T22:54:35.820 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4151 2024-05-17T22:54:35.821 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4152 2024-05-17T22:54:35.821 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4153 2024-05-17T22:54:35.821 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4154 2024-05-17T22:54:35.821 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4155 2024-05-17T22:54:35.821 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4156 2024-05-17T22:54:35.821 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4157 2024-05-17T22:54:35.821 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4158 2024-05-17T22:54:35.821 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4159 2024-05-17T22:54:35.821 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4160 2024-05-17T22:54:35.821 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4161 2024-05-17T22:54:35.821 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4162 2024-05-17T22:54:35.822 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4163 2024-05-17T22:54:35.823 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4164 2024-05-17T22:54:35.823 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4165 2024-05-17T22:54:35.823 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4166 2024-05-17T22:54:35.823 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4167 2024-05-17T22:54:35.823 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4168 2024-05-17T22:54:35.823 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4169 2024-05-17T22:54:35.823 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4170 2024-05-17T22:54:35.823 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4171 2024-05-17T22:54:35.823 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4172 2024-05-17T22:54:35.823 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4173 2024-05-17T22:54:35.823 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4174 2024-05-17T22:54:35.823 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 5002 2024-05-17T22:54:35.824 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 548 2024-05-17T22:54:35.824 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 549 2024-05-17T22:54:35.824 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 6102 2024-05-17T22:54:35.824 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 6104 2024-05-17T22:54:35.824 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 6106 2024-05-17T22:54:35.824 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 6110 2024-05-17T22:54:35.824 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 6111 2024-05-17T22:54:35.824 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 6112 2024-05-17T22:54:35.824 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 6113 2024-05-17T22:54:35.824 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 6117 2024-05-17T22:54:35.824 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 826 2024-05-17T22:54:35.824 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 827 2024-05-17T22:54:35.824 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 828 2024-05-17T22:54:35.825 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 17 22:39 pg_filenode.map 2024-05-17T22:54:35.825 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 17 22:39 PG_VERSION 2024-05-17T22:54:35.825 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:54:35.825 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-05-17T22:54:35.825 INFO:teuthology.orchestra.run.smithi012.stdout:total 7777 2024-05-17T22:54:35.825 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 May 17 22:39 . 2024-05-17T22:54:35.825 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 May 17 22:39 .. 2024-05-17T22:54:35.825 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 112 2024-05-17T22:54:35.825 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 113 2024-05-17T22:54:35.825 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 17 22:39 1247 2024-05-17T22:54:35.825 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 1247_fsm 2024-05-17T22:54:35.825 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1247_vm 2024-05-17T22:54:35.825 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 17 22:39 1249 2024-05-17T22:54:35.826 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 1249_fsm 2024-05-17T22:54:35.826 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1249_vm 2024-05-17T22:54:35.826 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 17 22:39 1255 2024-05-17T22:54:35.826 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 1255_fsm 2024-05-17T22:54:35.826 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1255_vm 2024-05-17T22:54:35.826 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 17 22:39 1259 2024-05-17T22:54:35.826 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 1259_fsm 2024-05-17T22:54:35.826 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1259_vm 2024-05-17T22:54:35.826 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 17 22:39 13344 2024-05-17T22:54:35.826 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 13344_fsm 2024-05-17T22:54:35.826 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13344_vm 2024-05-17T22:54:35.826 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 13346 2024-05-17T22:54:35.826 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13348 2024-05-17T22:54:35.826 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13349 2024-05-17T22:54:35.827 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 13349_fsm 2024-05-17T22:54:35.827 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13349_vm 2024-05-17T22:54:35.827 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 13351 2024-05-17T22:54:35.827 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13353 2024-05-17T22:54:35.827 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13354 2024-05-17T22:54:35.827 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 13354_fsm 2024-05-17T22:54:35.827 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13354_vm 2024-05-17T22:54:35.827 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 13356 2024-05-17T22:54:35.827 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13358 2024-05-17T22:54:35.827 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13359 2024-05-17T22:54:35.827 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 13359_fsm 2024-05-17T22:54:35.827 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13359_vm 2024-05-17T22:54:35.827 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 13361 2024-05-17T22:54:35.827 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13363 2024-05-17T22:54:35.828 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 1417 2024-05-17T22:54:35.828 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 1418 2024-05-17T22:54:35.828 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 174 2024-05-17T22:54:35.828 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 175 2024-05-17T22:54:35.828 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2187 2024-05-17T22:54:35.828 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2224 2024-05-17T22:54:35.828 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2328 2024-05-17T22:54:35.828 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2336 2024-05-17T22:54:35.828 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2337 2024-05-17T22:54:35.828 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2579 2024-05-17T22:54:35.828 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2600 2024-05-17T22:54:35.828 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2600_fsm 2024-05-17T22:54:35.828 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2600_vm 2024-05-17T22:54:35.828 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2601 2024-05-17T22:54:35.829 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2601_fsm 2024-05-17T22:54:35.829 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2601_vm 2024-05-17T22:54:35.829 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 17 22:39 2602 2024-05-17T22:54:35.829 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2602_fsm 2024-05-17T22:54:35.829 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2602_vm 2024-05-17T22:54:35.829 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 2603 2024-05-17T22:54:35.829 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2603_fsm 2024-05-17T22:54:35.829 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2603_vm 2024-05-17T22:54:35.829 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2604 2024-05-17T22:54:35.829 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2605 2024-05-17T22:54:35.829 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2605_fsm 2024-05-17T22:54:35.829 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2605_vm 2024-05-17T22:54:35.829 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2606 2024-05-17T22:54:35.829 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2606_fsm 2024-05-17T22:54:35.830 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2606_vm 2024-05-17T22:54:35.830 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2607 2024-05-17T22:54:35.830 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2607_fsm 2024-05-17T22:54:35.830 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2607_vm 2024-05-17T22:54:35.830 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 17 22:39 2608 2024-05-17T22:54:35.830 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2608_fsm 2024-05-17T22:54:35.830 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2608_vm 2024-05-17T22:54:35.830 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 17 22:39 2609 2024-05-17T22:54:35.830 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2609_fsm 2024-05-17T22:54:35.830 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2609_vm 2024-05-17T22:54:35.830 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 2610 2024-05-17T22:54:35.830 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2610_fsm 2024-05-17T22:54:35.830 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2610_vm 2024-05-17T22:54:35.831 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2611 2024-05-17T22:54:35.831 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2612 2024-05-17T22:54:35.831 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2612_fsm 2024-05-17T22:54:35.831 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2612_vm 2024-05-17T22:54:35.831 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2613 2024-05-17T22:54:35.831 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2615 2024-05-17T22:54:35.831 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2615_fsm 2024-05-17T22:54:35.831 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2615_vm 2024-05-17T22:54:35.831 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2616 2024-05-17T22:54:35.831 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2616_fsm 2024-05-17T22:54:35.831 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2616_vm 2024-05-17T22:54:35.831 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 17 22:39 2617 2024-05-17T22:54:35.831 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2617_fsm 2024-05-17T22:54:35.832 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2617_vm 2024-05-17T22:54:35.832 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 17 22:39 2618 2024-05-17T22:54:35.832 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2618_fsm 2024-05-17T22:54:35.832 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2618_vm 2024-05-17T22:54:35.832 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 17 22:39 2619 2024-05-17T22:54:35.832 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2619_fsm 2024-05-17T22:54:35.832 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2619_vm 2024-05-17T22:54:35.832 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2620 2024-05-17T22:54:35.832 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2650 2024-05-17T22:54:35.832 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2651 2024-05-17T22:54:35.832 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2652 2024-05-17T22:54:35.832 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 22:39 2653 2024-05-17T22:54:35.832 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 22:39 2654 2024-05-17T22:54:35.832 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 2655 2024-05-17T22:54:35.833 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2656 2024-05-17T22:54:35.833 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2657 2024-05-17T22:54:35.833 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 17 22:39 2658 2024-05-17T22:54:35.833 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 17 22:39 2659 2024-05-17T22:54:35.833 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2660 2024-05-17T22:54:35.833 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2661 2024-05-17T22:54:35.833 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 2662 2024-05-17T22:54:35.833 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 17 22:39 2663 2024-05-17T22:54:35.833 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2664 2024-05-17T22:54:35.833 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2665 2024-05-17T22:54:35.833 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2666 2024-05-17T22:54:35.833 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2667 2024-05-17T22:54:35.833 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2668 2024-05-17T22:54:35.833 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2669 2024-05-17T22:54:35.834 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2670 2024-05-17T22:54:35.834 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 17 22:39 2673 2024-05-17T22:54:35.834 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 17 22:39 2674 2024-05-17T22:54:35.834 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 17 22:39 2675 2024-05-17T22:54:35.834 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2678 2024-05-17T22:54:35.834 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2679 2024-05-17T22:54:35.834 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2680 2024-05-17T22:54:35.834 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2681 2024-05-17T22:54:35.834 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2682 2024-05-17T22:54:35.834 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2683 2024-05-17T22:54:35.834 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2684 2024-05-17T22:54:35.834 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2685 2024-05-17T22:54:35.834 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2686 2024-05-17T22:54:35.834 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2687 2024-05-17T22:54:35.835 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 22:39 2688 2024-05-17T22:54:35.835 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 17 22:39 2689 2024-05-17T22:54:35.835 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 17 22:39 2690 2024-05-17T22:54:35.835 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 17 22:39 2691 2024-05-17T22:54:35.835 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2692 2024-05-17T22:54:35.835 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2693 2024-05-17T22:54:35.835 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2696 2024-05-17T22:54:35.835 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2699 2024-05-17T22:54:35.835 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2701 2024-05-17T22:54:35.835 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2702 2024-05-17T22:54:35.835 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 2703 2024-05-17T22:54:35.835 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 22:39 2704 2024-05-17T22:54:35.835 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2753 2024-05-17T22:54:35.835 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2753_fsm 2024-05-17T22:54:35.836 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2753_vm 2024-05-17T22:54:35.836 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2754 2024-05-17T22:54:35.836 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2755 2024-05-17T22:54:35.836 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 2756 2024-05-17T22:54:35.836 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 2757 2024-05-17T22:54:35.836 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2830 2024-05-17T22:54:35.836 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2831 2024-05-17T22:54:35.836 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2832 2024-05-17T22:54:35.836 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2833 2024-05-17T22:54:35.836 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2834 2024-05-17T22:54:35.836 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2835 2024-05-17T22:54:35.836 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2836 2024-05-17T22:54:35.836 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2837 2024-05-17T22:54:35.837 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 17 22:39 2838 2024-05-17T22:54:35.837 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2838_fsm 2024-05-17T22:54:35.837 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2838_vm 2024-05-17T22:54:35.837 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2839 2024-05-17T22:54:35.837 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2840 2024-05-17T22:54:35.837 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2840_fsm 2024-05-17T22:54:35.837 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2840_vm 2024-05-17T22:54:35.837 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2841 2024-05-17T22:54:35.837 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2995 2024-05-17T22:54:35.837 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2996 2024-05-17T22:54:35.837 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3079 2024-05-17T22:54:35.837 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3079_fsm 2024-05-17T22:54:35.837 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3079_vm 2024-05-17T22:54:35.837 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3080 2024-05-17T22:54:35.838 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3081 2024-05-17T22:54:35.838 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 22:39 3085 2024-05-17T22:54:35.838 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3118 2024-05-17T22:54:35.838 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3119 2024-05-17T22:54:35.838 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 17 22:39 3164 2024-05-17T22:54:35.838 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3256 2024-05-17T22:54:35.838 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3257 2024-05-17T22:54:35.838 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3258 2024-05-17T22:54:35.838 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3350 2024-05-17T22:54:35.838 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3351 2024-05-17T22:54:35.838 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3379 2024-05-17T22:54:35.838 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3380 2024-05-17T22:54:35.838 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3381 2024-05-17T22:54:35.838 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3394 2024-05-17T22:54:35.839 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3394_fsm 2024-05-17T22:54:35.839 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3394_vm 2024-05-17T22:54:35.839 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3395 2024-05-17T22:54:35.839 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3429 2024-05-17T22:54:35.839 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3430 2024-05-17T22:54:35.839 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3431 2024-05-17T22:54:35.839 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3433 2024-05-17T22:54:35.839 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3439 2024-05-17T22:54:35.839 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3440 2024-05-17T22:54:35.839 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 22:39 3455 2024-05-17T22:54:35.839 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 17 22:39 3456 2024-05-17T22:54:35.839 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3456_fsm 2024-05-17T22:54:35.839 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3456_vm 2024-05-17T22:54:35.839 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3466 2024-05-17T22:54:35.839 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3467 2024-05-17T22:54:35.840 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3468 2024-05-17T22:54:35.840 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3501 2024-05-17T22:54:35.840 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3502 2024-05-17T22:54:35.840 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3503 2024-05-17T22:54:35.840 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3534 2024-05-17T22:54:35.840 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3541 2024-05-17T22:54:35.840 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3541_fsm 2024-05-17T22:54:35.840 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3541_vm 2024-05-17T22:54:35.840 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3542 2024-05-17T22:54:35.840 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3574 2024-05-17T22:54:35.840 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3575 2024-05-17T22:54:35.840 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3576 2024-05-17T22:54:35.840 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3596 2024-05-17T22:54:35.841 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3597 2024-05-17T22:54:35.841 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3598 2024-05-17T22:54:35.841 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3599 2024-05-17T22:54:35.841 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3600 2024-05-17T22:54:35.841 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3600_fsm 2024-05-17T22:54:35.841 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3600_vm 2024-05-17T22:54:35.841 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3601 2024-05-17T22:54:35.841 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3601_fsm 2024-05-17T22:54:35.841 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3601_vm 2024-05-17T22:54:35.841 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3602 2024-05-17T22:54:35.841 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3602_fsm 2024-05-17T22:54:35.841 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3602_vm 2024-05-17T22:54:35.841 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3603 2024-05-17T22:54:35.841 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3603_fsm 2024-05-17T22:54:35.842 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3603_vm 2024-05-17T22:54:35.842 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3604 2024-05-17T22:54:35.842 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3605 2024-05-17T22:54:35.842 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3606 2024-05-17T22:54:35.842 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3607 2024-05-17T22:54:35.842 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3608 2024-05-17T22:54:35.842 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 3609 2024-05-17T22:54:35.842 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3712 2024-05-17T22:54:35.842 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3764 2024-05-17T22:54:35.842 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3764_fsm 2024-05-17T22:54:35.842 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3764_vm 2024-05-17T22:54:35.842 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3766 2024-05-17T22:54:35.842 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3767 2024-05-17T22:54:35.843 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3997 2024-05-17T22:54:35.843 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4143 2024-05-17T22:54:35.843 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4144 2024-05-17T22:54:35.843 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4145 2024-05-17T22:54:35.843 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4146 2024-05-17T22:54:35.843 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4147 2024-05-17T22:54:35.843 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4148 2024-05-17T22:54:35.843 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4149 2024-05-17T22:54:35.843 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4150 2024-05-17T22:54:35.843 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4151 2024-05-17T22:54:35.843 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4152 2024-05-17T22:54:35.843 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4153 2024-05-17T22:54:35.843 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4154 2024-05-17T22:54:35.843 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4155 2024-05-17T22:54:35.844 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4156 2024-05-17T22:54:35.844 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4157 2024-05-17T22:54:35.844 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4158 2024-05-17T22:54:35.844 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4159 2024-05-17T22:54:35.844 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4160 2024-05-17T22:54:35.844 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4161 2024-05-17T22:54:35.844 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4162 2024-05-17T22:54:35.844 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4163 2024-05-17T22:54:35.844 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4164 2024-05-17T22:54:35.844 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4165 2024-05-17T22:54:35.844 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4166 2024-05-17T22:54:35.844 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4167 2024-05-17T22:54:35.844 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4168 2024-05-17T22:54:35.844 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4169 2024-05-17T22:54:35.845 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4170 2024-05-17T22:54:35.845 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4171 2024-05-17T22:54:35.845 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4172 2024-05-17T22:54:35.847 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4173 2024-05-17T22:54:35.847 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4174 2024-05-17T22:54:35.847 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 5002 2024-05-17T22:54:35.847 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 548 2024-05-17T22:54:35.847 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 549 2024-05-17T22:54:35.847 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 6102 2024-05-17T22:54:35.847 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 6104 2024-05-17T22:54:35.847 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 6106 2024-05-17T22:54:35.848 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 6110 2024-05-17T22:54:35.848 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 6111 2024-05-17T22:54:35.848 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 6112 2024-05-17T22:54:35.848 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 6113 2024-05-17T22:54:35.848 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 6117 2024-05-17T22:54:35.848 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 826 2024-05-17T22:54:35.848 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 827 2024-05-17T22:54:35.848 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 828 2024-05-17T22:54:35.848 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 17 22:39 pg_filenode.map 2024-05-17T22:54:35.848 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 17 22:39 PG_VERSION 2024-05-17T22:54:35.848 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:54:35.848 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-05-17T22:54:35.848 INFO:teuthology.orchestra.run.smithi012.stdout:total 552294 2024-05-17T22:54:35.848 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 May 17 22:46 . 2024-05-17T22:54:35.849 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 May 17 22:39 .. 2024-05-17T22:54:35.849 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 112 2024-05-17T22:54:35.849 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 113 2024-05-17T22:54:35.849 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 17 22:39 1247 2024-05-17T22:54:35.849 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 1247_fsm 2024-05-17T22:54:35.849 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1247_vm 2024-05-17T22:54:35.849 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 17 22:39 1249 2024-05-17T22:54:35.849 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 1249_fsm 2024-05-17T22:54:35.849 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1249_vm 2024-05-17T22:54:35.849 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 17 22:39 1255 2024-05-17T22:54:35.849 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 1255_fsm 2024-05-17T22:54:35.849 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1255_vm 2024-05-17T22:54:35.849 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 17 22:54 1259 2024-05-17T22:54:35.849 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 1259_fsm 2024-05-17T22:54:35.850 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1259_vm 2024-05-17T22:54:35.850 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 17 22:39 13344 2024-05-17T22:54:35.850 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 13344_fsm 2024-05-17T22:54:35.850 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13344_vm 2024-05-17T22:54:35.850 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 13346 2024-05-17T22:54:35.850 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13348 2024-05-17T22:54:35.850 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13349 2024-05-17T22:54:35.850 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 13349_fsm 2024-05-17T22:54:35.850 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13349_vm 2024-05-17T22:54:35.850 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 13351 2024-05-17T22:54:35.850 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13353 2024-05-17T22:54:35.850 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13354 2024-05-17T22:54:35.850 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 13354_fsm 2024-05-17T22:54:35.850 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13354_vm 2024-05-17T22:54:35.851 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 13356 2024-05-17T22:54:35.851 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13358 2024-05-17T22:54:35.851 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13359 2024-05-17T22:54:35.851 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 13359_fsm 2024-05-17T22:54:35.851 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13359_vm 2024-05-17T22:54:35.851 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 13361 2024-05-17T22:54:35.851 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 13363 2024-05-17T22:54:35.851 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 1417 2024-05-17T22:54:35.851 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 1418 2024-05-17T22:54:35.851 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 439894016 May 17 22:54 16396 2024-05-17T22:54:35.851 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 May 17 22:54 16396_fsm 2024-05-17T22:54:35.851 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:45 16396_vm 2024-05-17T22:54:35.851 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 17 22:54 16397 2024-05-17T22:54:35.851 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:54 16397_fsm 2024-05-17T22:54:35.852 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:54 16397_vm 2024-05-17T22:54:35.852 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 737280 May 17 22:54 16399 2024-05-17T22:54:35.852 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:54 16399_fsm 2024-05-17T22:54:35.852 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:53 16399_vm 2024-05-17T22:54:35.852 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:54 16400 2024-05-17T22:54:35.852 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 22:54 16402 2024-05-17T22:54:35.852 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 71901184 May 17 22:54 16404 2024-05-17T22:54:35.852 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 44105728 May 17 22:54 16406 2024-05-17T22:54:35.852 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:54 16406_fsm 2024-05-17T22:54:35.852 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:54 16406_vm 2024-05-17T22:54:35.852 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 174 2024-05-17T22:54:35.852 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 175 2024-05-17T22:54:35.852 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2187 2024-05-17T22:54:35.852 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2224 2024-05-17T22:54:35.853 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2328 2024-05-17T22:54:35.853 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2336 2024-05-17T22:54:35.853 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2337 2024-05-17T22:54:35.853 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2579 2024-05-17T22:54:35.853 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2600 2024-05-17T22:54:35.853 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2600_fsm 2024-05-17T22:54:35.853 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2600_vm 2024-05-17T22:54:35.853 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2601 2024-05-17T22:54:35.853 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2601_fsm 2024-05-17T22:54:35.853 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2601_vm 2024-05-17T22:54:35.853 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 17 22:39 2602 2024-05-17T22:54:35.853 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2602_fsm 2024-05-17T22:54:35.853 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2602_vm 2024-05-17T22:54:35.853 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 2603 2024-05-17T22:54:35.854 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2603_fsm 2024-05-17T22:54:35.854 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2603_vm 2024-05-17T22:54:35.854 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2604 2024-05-17T22:54:35.854 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2605 2024-05-17T22:54:35.854 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2605_fsm 2024-05-17T22:54:35.854 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2605_vm 2024-05-17T22:54:35.854 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2606 2024-05-17T22:54:35.854 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2606_fsm 2024-05-17T22:54:35.854 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2606_vm 2024-05-17T22:54:35.854 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2607 2024-05-17T22:54:35.854 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2607_fsm 2024-05-17T22:54:35.854 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2607_vm 2024-05-17T22:54:35.854 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 17 22:39 2608 2024-05-17T22:54:35.855 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2608_fsm 2024-05-17T22:54:35.855 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2608_vm 2024-05-17T22:54:35.855 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 17 22:39 2609 2024-05-17T22:54:35.855 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2609_fsm 2024-05-17T22:54:35.855 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2609_vm 2024-05-17T22:54:35.855 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 2610 2024-05-17T22:54:35.855 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2610_fsm 2024-05-17T22:54:35.855 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2610_vm 2024-05-17T22:54:35.855 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2611 2024-05-17T22:54:35.855 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2612 2024-05-17T22:54:35.855 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2612_fsm 2024-05-17T22:54:35.855 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2612_vm 2024-05-17T22:54:35.855 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2613 2024-05-17T22:54:35.856 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2615 2024-05-17T22:54:35.856 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2615_fsm 2024-05-17T22:54:35.856 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2615_vm 2024-05-17T22:54:35.856 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2616 2024-05-17T22:54:35.856 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2616_fsm 2024-05-17T22:54:35.856 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2616_vm 2024-05-17T22:54:35.856 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 17 22:39 2617 2024-05-17T22:54:35.856 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2617_fsm 2024-05-17T22:54:35.856 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2617_vm 2024-05-17T22:54:35.856 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 17 22:39 2618 2024-05-17T22:54:35.856 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2618_fsm 2024-05-17T22:54:35.856 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2618_vm 2024-05-17T22:54:35.856 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 May 17 22:54 2619 2024-05-17T22:54:35.856 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:53 2619_fsm 2024-05-17T22:54:35.856 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:48 2619_vm 2024-05-17T22:54:35.857 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2620 2024-05-17T22:54:35.857 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2650 2024-05-17T22:54:35.857 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2651 2024-05-17T22:54:35.857 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2652 2024-05-17T22:54:35.857 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 22:39 2653 2024-05-17T22:54:35.857 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 22:39 2654 2024-05-17T22:54:35.857 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 2655 2024-05-17T22:54:35.857 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2656 2024-05-17T22:54:35.857 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2657 2024-05-17T22:54:35.857 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 17 22:39 2658 2024-05-17T22:54:35.857 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 17 22:39 2659 2024-05-17T22:54:35.857 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2660 2024-05-17T22:54:35.857 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2661 2024-05-17T22:54:35.857 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 2662 2024-05-17T22:54:35.858 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 17 22:39 2663 2024-05-17T22:54:35.858 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2664 2024-05-17T22:54:35.858 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2665 2024-05-17T22:54:35.858 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2666 2024-05-17T22:54:35.858 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2667 2024-05-17T22:54:35.858 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2668 2024-05-17T22:54:35.858 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2669 2024-05-17T22:54:35.858 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2670 2024-05-17T22:54:35.858 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 17 22:39 2673 2024-05-17T22:54:35.858 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 17 22:39 2674 2024-05-17T22:54:35.858 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 17 22:39 2675 2024-05-17T22:54:35.858 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2678 2024-05-17T22:54:35.858 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2679 2024-05-17T22:54:35.859 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2680 2024-05-17T22:54:35.859 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2681 2024-05-17T22:54:35.859 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2682 2024-05-17T22:54:35.859 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2683 2024-05-17T22:54:35.859 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2684 2024-05-17T22:54:35.859 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2685 2024-05-17T22:54:35.859 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2686 2024-05-17T22:54:35.859 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2687 2024-05-17T22:54:35.859 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 22:39 2688 2024-05-17T22:54:35.859 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 17 22:39 2689 2024-05-17T22:54:35.859 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 17 22:39 2690 2024-05-17T22:54:35.859 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 17 22:39 2691 2024-05-17T22:54:35.859 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2692 2024-05-17T22:54:35.859 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2693 2024-05-17T22:54:35.860 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:54 2696 2024-05-17T22:54:35.860 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2699 2024-05-17T22:54:35.860 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2701 2024-05-17T22:54:35.860 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2702 2024-05-17T22:54:35.860 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 2703 2024-05-17T22:54:35.860 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 22:39 2704 2024-05-17T22:54:35.860 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2753 2024-05-17T22:54:35.860 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2753_fsm 2024-05-17T22:54:35.860 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2753_vm 2024-05-17T22:54:35.860 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2754 2024-05-17T22:54:35.860 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2755 2024-05-17T22:54:35.860 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 2756 2024-05-17T22:54:35.860 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 2757 2024-05-17T22:54:35.860 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2830 2024-05-17T22:54:35.861 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2831 2024-05-17T22:54:35.861 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2832 2024-05-17T22:54:35.861 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2833 2024-05-17T22:54:35.861 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2834 2024-05-17T22:54:35.861 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2835 2024-05-17T22:54:35.861 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2836 2024-05-17T22:54:35.861 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2837 2024-05-17T22:54:35.861 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 17 22:39 2838 2024-05-17T22:54:35.861 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2838_fsm 2024-05-17T22:54:35.861 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2838_vm 2024-05-17T22:54:35.861 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2839 2024-05-17T22:54:35.861 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2840 2024-05-17T22:54:35.861 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2840_fsm 2024-05-17T22:54:35.861 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2840_vm 2024-05-17T22:54:35.862 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2841 2024-05-17T22:54:35.862 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2995 2024-05-17T22:54:35.862 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2996 2024-05-17T22:54:35.862 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3079 2024-05-17T22:54:35.862 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3079_fsm 2024-05-17T22:54:35.862 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3079_vm 2024-05-17T22:54:35.862 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3080 2024-05-17T22:54:35.862 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3081 2024-05-17T22:54:35.862 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 22:39 3085 2024-05-17T22:54:35.862 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3118 2024-05-17T22:54:35.862 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3119 2024-05-17T22:54:35.862 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 17 22:39 3164 2024-05-17T22:54:35.862 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3256 2024-05-17T22:54:35.863 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3257 2024-05-17T22:54:35.863 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3258 2024-05-17T22:54:35.863 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3350 2024-05-17T22:54:35.863 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3351 2024-05-17T22:54:35.863 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3379 2024-05-17T22:54:35.863 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3380 2024-05-17T22:54:35.863 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3381 2024-05-17T22:54:35.863 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3394 2024-05-17T22:54:35.863 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3394_fsm 2024-05-17T22:54:35.863 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3394_vm 2024-05-17T22:54:35.863 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3395 2024-05-17T22:54:35.863 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3429 2024-05-17T22:54:35.863 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3430 2024-05-17T22:54:35.863 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3431 2024-05-17T22:54:35.864 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3433 2024-05-17T22:54:35.864 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3439 2024-05-17T22:54:35.864 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3440 2024-05-17T22:54:35.864 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 22:39 3455 2024-05-17T22:54:35.864 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 17 22:39 3456 2024-05-17T22:54:35.864 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3456_fsm 2024-05-17T22:54:35.864 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3456_vm 2024-05-17T22:54:35.864 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3466 2024-05-17T22:54:35.864 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3467 2024-05-17T22:54:35.864 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3468 2024-05-17T22:54:35.864 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3501 2024-05-17T22:54:35.864 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3502 2024-05-17T22:54:35.864 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3503 2024-05-17T22:54:35.864 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3534 2024-05-17T22:54:35.865 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3541 2024-05-17T22:54:35.865 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3541_fsm 2024-05-17T22:54:35.865 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3541_vm 2024-05-17T22:54:35.865 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3542 2024-05-17T22:54:35.865 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3574 2024-05-17T22:54:35.865 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3575 2024-05-17T22:54:35.865 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3576 2024-05-17T22:54:35.865 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3596 2024-05-17T22:54:35.865 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3597 2024-05-17T22:54:35.865 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3598 2024-05-17T22:54:35.865 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3599 2024-05-17T22:54:35.865 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3600 2024-05-17T22:54:35.865 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3600_fsm 2024-05-17T22:54:35.865 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3600_vm 2024-05-17T22:54:35.866 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3601 2024-05-17T22:54:35.866 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3601_fsm 2024-05-17T22:54:35.866 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3601_vm 2024-05-17T22:54:35.866 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3602 2024-05-17T22:54:35.866 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3602_fsm 2024-05-17T22:54:35.866 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3602_vm 2024-05-17T22:54:35.866 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3603 2024-05-17T22:54:35.866 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3603_fsm 2024-05-17T22:54:35.866 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3603_vm 2024-05-17T22:54:35.866 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3604 2024-05-17T22:54:35.866 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3605 2024-05-17T22:54:35.866 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3606 2024-05-17T22:54:35.866 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3607 2024-05-17T22:54:35.867 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3608 2024-05-17T22:54:35.867 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 22:39 3609 2024-05-17T22:54:35.867 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3712 2024-05-17T22:54:35.867 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3764 2024-05-17T22:54:35.867 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 3764_fsm 2024-05-17T22:54:35.867 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3764_vm 2024-05-17T22:54:35.867 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3766 2024-05-17T22:54:35.867 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 3767 2024-05-17T22:54:35.867 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3997 2024-05-17T22:54:35.867 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4143 2024-05-17T22:54:35.867 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4144 2024-05-17T22:54:35.867 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4145 2024-05-17T22:54:35.867 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4146 2024-05-17T22:54:35.867 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4147 2024-05-17T22:54:35.868 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4148 2024-05-17T22:54:35.868 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4149 2024-05-17T22:54:35.868 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4150 2024-05-17T22:54:35.868 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4151 2024-05-17T22:54:35.868 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4152 2024-05-17T22:54:35.868 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4153 2024-05-17T22:54:35.868 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4154 2024-05-17T22:54:35.868 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4155 2024-05-17T22:54:35.868 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4156 2024-05-17T22:54:35.868 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4157 2024-05-17T22:54:35.868 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4158 2024-05-17T22:54:35.868 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4159 2024-05-17T22:54:35.868 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4160 2024-05-17T22:54:35.868 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4161 2024-05-17T22:54:35.869 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4162 2024-05-17T22:54:35.869 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4163 2024-05-17T22:54:35.869 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4164 2024-05-17T22:54:35.869 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4165 2024-05-17T22:54:35.869 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4166 2024-05-17T22:54:35.869 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4167 2024-05-17T22:54:35.869 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4168 2024-05-17T22:54:35.869 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4169 2024-05-17T22:54:35.869 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4170 2024-05-17T22:54:35.869 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4171 2024-05-17T22:54:35.869 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4172 2024-05-17T22:54:35.869 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4173 2024-05-17T22:54:35.869 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4174 2024-05-17T22:54:35.869 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 5002 2024-05-17T22:54:35.870 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 548 2024-05-17T22:54:35.870 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 549 2024-05-17T22:54:35.870 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 6102 2024-05-17T22:54:35.870 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 6104 2024-05-17T22:54:35.870 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 6106 2024-05-17T22:54:35.870 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 6110 2024-05-17T22:54:35.870 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 6111 2024-05-17T22:54:35.870 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 6112 2024-05-17T22:54:35.870 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 6113 2024-05-17T22:54:35.870 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 6117 2024-05-17T22:54:35.870 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 826 2024-05-17T22:54:35.870 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 827 2024-05-17T22:54:35.870 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 828 2024-05-17T22:54:35.870 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 17 22:39 pg_filenode.map 2024-05-17T22:54:35.871 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 151596 May 17 22:39 pg_internal.init 2024-05-17T22:54:35.871 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 17 22:39 PG_VERSION 2024-05-17T22:54:35.871 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:54:35.871 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2024-05-17T22:54:35.871 INFO:teuthology.orchestra.run.smithi012.stdout:total 0 2024-05-17T22:54:35.871 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 . 2024-05-17T22:54:35.871 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 May 17 22:39 .. 2024-05-17T22:54:35.871 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:54:35.871 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/global: 2024-05-17T22:54:35.871 INFO:teuthology.orchestra.run.smithi012.stdout:total 560 2024-05-17T22:54:35.871 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 May 17 22:39 . 2024-05-17T22:54:35.871 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 17 22:54 .. 2024-05-17T22:54:35.871 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1213 2024-05-17T22:54:35.871 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 1213_fsm 2024-05-17T22:54:35.872 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1213_vm 2024-05-17T22:54:35.872 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1214 2024-05-17T22:54:35.872 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 1214_fsm 2024-05-17T22:54:35.872 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1214_vm 2024-05-17T22:54:35.872 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 1232 2024-05-17T22:54:35.872 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 1233 2024-05-17T22:54:35.872 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1260 2024-05-17T22:54:35.872 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 1260_fsm 2024-05-17T22:54:35.872 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1260_vm 2024-05-17T22:54:35.872 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1261 2024-05-17T22:54:35.872 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 1261_fsm 2024-05-17T22:54:35.872 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1261_vm 2024-05-17T22:54:35.872 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1262 2024-05-17T22:54:35.873 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 1262_fsm 2024-05-17T22:54:35.873 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 1262_vm 2024-05-17T22:54:35.873 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2396 2024-05-17T22:54:35.873 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 22:39 2396_fsm 2024-05-17T22:54:35.873 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2396_vm 2024-05-17T22:54:35.873 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2397 2024-05-17T22:54:35.873 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2671 2024-05-17T22:54:35.873 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2672 2024-05-17T22:54:35.873 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2676 2024-05-17T22:54:35.873 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2677 2024-05-17T22:54:35.873 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2694 2024-05-17T22:54:35.873 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2695 2024-05-17T22:54:35.873 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2697 2024-05-17T22:54:35.873 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 22:39 2698 2024-05-17T22:54:35.874 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2846 2024-05-17T22:54:35.874 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2847 2024-05-17T22:54:35.874 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2964 2024-05-17T22:54:35.874 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2965 2024-05-17T22:54:35.874 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 2966 2024-05-17T22:54:35.874 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 2967 2024-05-17T22:54:35.874 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 3592 2024-05-17T22:54:35.874 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 3593 2024-05-17T22:54:35.874 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4060 2024-05-17T22:54:35.874 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4061 2024-05-17T22:54:35.874 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4175 2024-05-17T22:54:35.874 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4176 2024-05-17T22:54:35.874 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4177 2024-05-17T22:54:35.875 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4178 2024-05-17T22:54:35.875 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4181 2024-05-17T22:54:35.875 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4182 2024-05-17T22:54:35.875 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4183 2024-05-17T22:54:35.875 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4184 2024-05-17T22:54:35.875 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 4185 2024-05-17T22:54:35.875 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 4186 2024-05-17T22:54:35.875 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 6000 2024-05-17T22:54:35.875 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 6001 2024-05-17T22:54:35.875 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 6002 2024-05-17T22:54:35.875 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 6100 2024-05-17T22:54:35.875 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 6114 2024-05-17T22:54:35.875 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 6115 2024-05-17T22:54:35.875 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:54 pg_control 2024-05-17T22:54:35.876 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 17 22:39 pg_filenode.map 2024-05-17T22:54:35.876 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 23144 May 17 22:39 pg_internal.init 2024-05-17T22:54:35.876 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:54:35.876 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/log: 2024-05-17T22:54:35.876 INFO:teuthology.orchestra.run.smithi012.stdout:total 2 2024-05-17T22:54:35.876 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 17 22:39 . 2024-05-17T22:54:35.876 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 17 22:54 .. 2024-05-17T22:54:35.876 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1333 May 17 22:54 postgresql-Fri.log 2024-05-17T22:54:35.876 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:54:35.876 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-05-17T22:54:35.876 INFO:teuthology.orchestra.run.smithi012.stdout:total 0 2024-05-17T22:54:35.876 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 . 2024-05-17T22:54:35.876 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 17 22:54 .. 2024-05-17T22:54:35.876 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:54:35.877 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-05-17T22:54:35.877 INFO:teuthology.orchestra.run.smithi012.stdout:total 0 2024-05-17T22:54:35.877 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 . 2024-05-17T22:54:35.877 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 17 22:54 .. 2024-05-17T22:54:35.877 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:54:35.877 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-05-17T22:54:35.877 INFO:teuthology.orchestra.run.smithi012.stdout:total 1 2024-05-17T22:54:35.877 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 17 22:54 . 2024-05-17T22:54:35.877 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 17 22:54 .. 2024-05-17T22:54:35.877 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 mappings 2024-05-17T22:54:35.877 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 17 22:54 replorigin_checkpoint 2024-05-17T22:54:35.877 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 snapshots 2024-05-17T22:54:35.877 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:54:35.878 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-05-17T22:54:35.878 INFO:teuthology.orchestra.run.smithi012.stdout:total 0 2024-05-17T22:54:35.878 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 . 2024-05-17T22:54:35.878 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 17 22:54 .. 2024-05-17T22:54:35.878 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:54:35.878 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-05-17T22:54:35.878 INFO:teuthology.orchestra.run.smithi012.stdout:total 0 2024-05-17T22:54:35.878 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 . 2024-05-17T22:54:35.878 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 17 22:54 .. 2024-05-17T22:54:35.878 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:54:35.878 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-05-17T22:54:35.878 INFO:teuthology.orchestra.run.smithi012.stdout:total 0 2024-05-17T22:54:35.878 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 17 22:39 . 2024-05-17T22:54:35.879 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 17 22:54 .. 2024-05-17T22:54:35.879 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 17 22:39 members 2024-05-17T22:54:35.879 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 17 22:39 offsets 2024-05-17T22:54:35.879 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:54:35.879 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-05-17T22:54:35.879 INFO:teuthology.orchestra.run.smithi012.stdout:total 8 2024-05-17T22:54:35.879 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 17 22:39 . 2024-05-17T22:54:35.879 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 17 22:39 .. 2024-05-17T22:54:35.879 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:39 0000 2024-05-17T22:54:35.879 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:54:35.879 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-05-17T22:54:35.879 INFO:teuthology.orchestra.run.smithi012.stdout:total 8 2024-05-17T22:54:35.879 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 17 22:39 . 2024-05-17T22:54:35.879 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 17 22:39 .. 2024-05-17T22:54:35.880 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 22:44 0000 2024-05-17T22:54:35.880 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:54:35.880 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-05-17T22:54:35.880 INFO:teuthology.orchestra.run.smithi012.stdout:total 0 2024-05-17T22:54:35.880 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 . 2024-05-17T22:54:35.880 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 17 22:54 .. 2024-05-17T22:54:35.880 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:54:35.880 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-05-17T22:54:35.880 INFO:teuthology.orchestra.run.smithi012.stdout:total 0 2024-05-17T22:54:35.880 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 . 2024-05-17T22:54:35.880 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 17 22:54 .. 2024-05-17T22:54:35.880 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:54:35.880 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-05-17T22:54:35.881 INFO:teuthology.orchestra.run.smithi012.stdout:total 0 2024-05-17T22:54:35.881 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 . 2024-05-17T22:54:35.881 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 17 22:54 .. 2024-05-17T22:54:35.881 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:54:35.881 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-05-17T22:54:35.881 INFO:teuthology.orchestra.run.smithi012.stdout:total 0 2024-05-17T22:54:35.881 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 . 2024-05-17T22:54:35.881 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 17 22:54 .. 2024-05-17T22:54:35.881 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:54:35.881 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-05-17T22:54:35.881 INFO:teuthology.orchestra.run.smithi012.stdout:total 15 2024-05-17T22:54:35.881 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 17 22:54 . 2024-05-17T22:54:35.881 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 17 22:54 .. 2024-05-17T22:54:35.881 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1952 May 17 22:54 db_0.stat 2024-05-17T22:54:35.882 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 10979 May 17 22:54 db_13494.stat 2024-05-17T22:54:35.882 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1151 May 17 22:54 global.stat 2024-05-17T22:54:35.882 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:54:35.882 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-05-17T22:54:35.882 INFO:teuthology.orchestra.run.smithi012.stdout:total 0 2024-05-17T22:54:35.882 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:54 . 2024-05-17T22:54:35.882 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 17 22:54 .. 2024-05-17T22:54:35.882 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:54:35.882 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-05-17T22:54:35.882 INFO:teuthology.orchestra.run.smithi012.stdout:total 208 2024-05-17T22:54:35.882 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 17 22:54 . 2024-05-17T22:54:35.882 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 17 22:54 .. 2024-05-17T22:54:35.882 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 212992 May 17 22:54 000C 2024-05-17T22:54:35.883 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:54:35.883 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-05-17T22:54:35.883 INFO:teuthology.orchestra.run.smithi012.stdout:total 0 2024-05-17T22:54:35.883 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 . 2024-05-17T22:54:35.883 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 17 22:54 .. 2024-05-17T22:54:35.883 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:54:35.883 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-05-17T22:54:35.883 INFO:teuthology.orchestra.run.smithi012.stdout:total 0 2024-05-17T22:54:35.883 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 . 2024-05-17T22:54:35.883 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 17 22:54 .. 2024-05-17T22:54:35.883 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:54:35.883 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-05-17T22:54:35.883 INFO:teuthology.orchestra.run.smithi012.stdout:total 999424 2024-05-17T22:54:35.883 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 62 May 17 22:54 . 2024-05-17T22:54:35.883 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 17 22:54 .. 2024-05-17T22:54:35.884 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:54 000000010000000000000065 2024-05-17T22:54:35.884 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:48 000000010000000000000066 2024-05-17T22:54:35.884 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:44 000000010000000000000067 2024-05-17T22:54:35.884 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:46 000000010000000000000068 2024-05-17T22:54:35.884 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:44 000000010000000000000069 2024-05-17T22:54:35.884 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:44 00000001000000000000006A 2024-05-17T22:54:35.884 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:46 00000001000000000000006B 2024-05-17T22:54:35.884 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:44 00000001000000000000006C 2024-05-17T22:54:35.884 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:45 00000001000000000000006D 2024-05-17T22:54:35.884 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:45 00000001000000000000006E 2024-05-17T22:54:35.884 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:44 00000001000000000000006F 2024-05-17T22:54:35.884 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:47 000000010000000000000070 2024-05-17T22:54:35.884 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:47 000000010000000000000071 2024-05-17T22:54:35.885 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:45 000000010000000000000072 2024-05-17T22:54:35.885 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:45 000000010000000000000073 2024-05-17T22:54:35.885 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:46 000000010000000000000074 2024-05-17T22:54:35.885 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:45 000000010000000000000075 2024-05-17T22:54:35.885 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:46 000000010000000000000076 2024-05-17T22:54:35.885 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:44 000000010000000000000077 2024-05-17T22:54:35.885 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:44 000000010000000000000078 2024-05-17T22:54:35.885 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:45 000000010000000000000079 2024-05-17T22:54:35.885 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:44 00000001000000000000007A 2024-05-17T22:54:35.885 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:47 00000001000000000000007B 2024-05-17T22:54:35.885 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:44 00000001000000000000007C 2024-05-17T22:54:35.885 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:45 00000001000000000000007D 2024-05-17T22:54:35.885 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:45 00000001000000000000007E 2024-05-17T22:54:35.885 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:44 00000001000000000000007F 2024-05-17T22:54:35.886 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:46 000000010000000000000080 2024-05-17T22:54:35.886 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:48 000000010000000000000081 2024-05-17T22:54:35.886 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:45 000000010000000000000082 2024-05-17T22:54:35.886 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:44 000000010000000000000083 2024-05-17T22:54:35.886 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:44 000000010000000000000084 2024-05-17T22:54:35.886 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:49 000000010000000000000085 2024-05-17T22:54:35.886 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:50 000000010000000000000086 2024-05-17T22:54:35.886 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:50 000000010000000000000087 2024-05-17T22:54:35.886 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:51 000000010000000000000088 2024-05-17T22:54:35.886 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:51 000000010000000000000089 2024-05-17T22:54:35.886 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:53 00000001000000000000008A 2024-05-17T22:54:35.886 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:49 00000001000000000000008B 2024-05-17T22:54:35.886 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:50 00000001000000000000008C 2024-05-17T22:54:35.887 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:53 00000001000000000000008D 2024-05-17T22:54:35.887 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:51 00000001000000000000008E 2024-05-17T22:54:35.887 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:50 00000001000000000000008F 2024-05-17T22:54:35.887 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:52 000000010000000000000090 2024-05-17T22:54:35.887 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:50 000000010000000000000091 2024-05-17T22:54:35.887 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:51 000000010000000000000092 2024-05-17T22:54:35.887 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:49 000000010000000000000093 2024-05-17T22:54:35.887 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:50 000000010000000000000094 2024-05-17T22:54:35.887 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:49 000000010000000000000095 2024-05-17T22:54:35.887 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:50 000000010000000000000096 2024-05-17T22:54:35.887 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:54 000000010000000000000097 2024-05-17T22:54:35.887 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:51 000000010000000000000098 2024-05-17T22:54:35.887 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:49 000000010000000000000099 2024-05-17T22:54:35.887 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:49 00000001000000000000009A 2024-05-17T22:54:35.888 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:50 00000001000000000000009B 2024-05-17T22:54:35.888 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:49 00000001000000000000009C 2024-05-17T22:54:35.888 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:52 00000001000000000000009D 2024-05-17T22:54:35.888 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:50 00000001000000000000009E 2024-05-17T22:54:35.888 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:49 00000001000000000000009F 2024-05-17T22:54:35.888 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:49 0000000100000000000000A0 2024-05-17T22:54:35.888 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 22:52 0000000100000000000000A1 2024-05-17T22:54:35.888 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 archive_status 2024-05-17T22:54:35.888 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:54:35.888 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-05-17T22:54:35.888 INFO:teuthology.orchestra.run.smithi012.stdout:total 0 2024-05-17T22:54:35.888 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 22:39 . 2024-05-17T22:54:35.888 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 62 May 17 22:54 .. 2024-05-17T22:54:35.888 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:54:35.889 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-05-17T22:54:35.889 INFO:teuthology.orchestra.run.smithi012.stdout:total 208 2024-05-17T22:54:35.889 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 17 22:39 . 2024-05-17T22:54:35.889 INFO:teuthology.orchestra.run.smithi012.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 17 22:54 .. 2024-05-17T22:54:35.889 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 212992 May 17 22:54 0000 2024-05-17T22:54:35.889 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-05-17T22:54:35.899 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-05-17T22:54:36.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:36 smithi012 ceph-mon[31632]: pgmap v899: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 7.4 MiB/s wr, 718 op/s 2024-05-17T22:54:36.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:36 smithi151 ceph-mon[36101]: pgmap v899: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 7.4 MiB/s wr, 718 op/s 2024-05-17T22:54:36.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:36 smithi096 ceph-mon[36065]: pgmap v899: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 7.4 MiB/s wr, 718 op/s 2024-05-17T22:54:38.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:38 smithi012 ceph-mon[31632]: pgmap v900: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.1 MiB/s wr, 636 op/s 2024-05-17T22:54:38.350 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:38 smithi096 ceph-mon[36065]: pgmap v900: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.1 MiB/s wr, 636 op/s 2024-05-17T22:54:38.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:38 smithi151 ceph-mon[36101]: pgmap v900: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.1 MiB/s wr, 636 op/s 2024-05-17T22:54:40.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:40 smithi151 ceph-mon[36101]: pgmap v901: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.3 MiB/s wr, 547 op/s 2024-05-17T22:54:40.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:40 smithi096 ceph-mon[36065]: pgmap v901: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.3 MiB/s wr, 547 op/s 2024-05-17T22:54:40.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:40 smithi012 ceph-mon[31632]: pgmap v901: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.3 MiB/s wr, 547 op/s 2024-05-17T22:54:41.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:41 smithi151 ceph-mon[36101]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:54:41.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:41 smithi096 ceph-mon[36065]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:54:41.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:41 smithi012 ceph-mon[31632]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:54:42.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:42 smithi151 ceph-mon[36101]: pgmap v902: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 4.7 MiB/s wr, 491 op/s 2024-05-17T22:54:42.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:42 smithi096 ceph-mon[36065]: pgmap v902: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 4.7 MiB/s wr, 491 op/s 2024-05-17T22:54:42.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:42 smithi012 ceph-mon[31632]: pgmap v902: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 4.7 MiB/s wr, 491 op/s 2024-05-17T22:54:44.366 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:44 smithi012 ceph-mon[31632]: pgmap v903: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 3.0 MiB/s wr, 313 op/s 2024-05-17T22:54:44.367 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:44 smithi012 ceph-mon[31632]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:54:44.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:44 smithi151 ceph-mon[36101]: pgmap v903: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 3.0 MiB/s wr, 313 op/s 2024-05-17T22:54:44.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:44 smithi151 ceph-mon[36101]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:54:44.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:44 smithi096 ceph-mon[36065]: pgmap v903: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 3.0 MiB/s wr, 313 op/s 2024-05-17T22:54:44.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:44 smithi096 ceph-mon[36065]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T22:54:46.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:46 smithi151 ceph-mon[36101]: pgmap v904: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 2.2 MiB/s wr, 225 op/s 2024-05-17T22:54:46.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:46 smithi096 ceph-mon[36065]: pgmap v904: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 2.2 MiB/s wr, 225 op/s 2024-05-17T22:54:46.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:46 smithi012 ceph-mon[31632]: pgmap v904: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 2.2 MiB/s wr, 225 op/s 2024-05-17T22:54:48.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:48 smithi151 ceph-mon[36101]: pgmap v905: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 812 KiB/s wr, 107 op/s 2024-05-17T22:54:48.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:48 smithi096 ceph-mon[36065]: pgmap v905: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 812 KiB/s wr, 107 op/s 2024-05-17T22:54:48.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:48 smithi012 ceph-mon[31632]: pgmap v905: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 812 KiB/s wr, 107 op/s 2024-05-17T22:54:49.688 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-17T22:54:49.956 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:54:50.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:50 smithi012 ceph-mon[31632]: pgmap v906: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 223 KiB/s wr, 34 op/s 2024-05-17T22:54:50.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:50 smithi151 ceph-mon[36101]: pgmap v906: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 223 KiB/s wr, 34 op/s 2024-05-17T22:54:50.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:50 smithi096 ceph-mon[36065]: pgmap v906: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 223 KiB/s wr, 34 op/s 2024-05-17T22:54:51.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.904+0000 7f2da77b2700 1 -- 172.21.15.12:0/1955549947 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2da0102e00 msgr2=0x7f2da0103210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:51.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.904+0000 7f2da77b2700 1 --2- 172.21.15.12:0/1955549947 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2da0102e00 0x7f2da0103210 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c009990 tx=0x7f2d9c009ca0 comp rx=0 tx=0).stop 2024-05-17T22:54:51.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.904+0000 7f2da77b2700 1 -- 172.21.15.12:0/1955549947 shutdown_connections 2024-05-17T22:54:51.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.904+0000 7f2da77b2700 1 --2- 172.21.15.12:0/1955549947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2da0108080 0x7f2da010f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:51.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.904+0000 7f2da77b2700 1 --2- 172.21.15.12:0/1955549947 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2da0102e00 0x7f2da0103210 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:51.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.904+0000 7f2da77b2700 1 --2- 172.21.15.12:0/1955549947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2da0107730 0x7f2da0107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:51.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.904+0000 7f2da77b2700 1 -- 172.21.15.12:0/1955549947 >> 172.21.15.12:0/1955549947 conn(0x7f2da0079bd0 msgr2=0x7f2da0079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:51.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.904+0000 7f2da77b2700 1 -- 172.21.15.12:0/1955549947 shutdown_connections 2024-05-17T22:54:51.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.904+0000 7f2da77b2700 1 -- 172.21.15.12:0/1955549947 wait complete. 2024-05-17T22:54:51.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.905+0000 7f2da77b2700 1 Processor -- start 2024-05-17T22:54:51.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.905+0000 7f2da77b2700 1 -- start start 2024-05-17T22:54:51.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.905+0000 7f2da77b2700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2da0102e00 0x7f2da019a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:51.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.905+0000 7f2da77b2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2da0107730 0x7f2da019ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:51.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.905+0000 7f2da77b2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2da0108080 0x7f2da019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:51.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.905+0000 7f2da77b2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2da0112cb0 con 0x7f2da0108080 2024-05-17T22:54:51.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.905+0000 7f2da77b2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2da0112b30 con 0x7f2da0107730 2024-05-17T22:54:51.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.905+0000 7f2da77b2700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2da0112e30 con 0x7f2da0102e00 2024-05-17T22:54:51.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.905+0000 7f2da5d4f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2da0108080 0x7f2da019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:51.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.905+0000 7f2da5d4f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2da0108080 0x7f2da019efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52716/0 (socket says 172.21.15.12:52716) 2024-05-17T22:54:51.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.905+0000 7f2da5d4f700 1 -- 172.21.15.12:0/987319805 learned_addr learned my addr 172.21.15.12:0/987319805 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:54:51.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.905+0000 7f2da4d4d700 1 --2- 172.21.15.12:0/987319805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2da0107730 0x7f2da019ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:51.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.905+0000 7f2da554e700 1 --2- 172.21.15.12:0/987319805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2da0102e00 0x7f2da019a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:51.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.905+0000 7f2da5d4f700 1 -- 172.21.15.12:0/987319805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2da0102e00 msgr2=0x7f2da019a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:51.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.905+0000 7f2da5d4f700 1 --2- 172.21.15.12:0/987319805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2da0102e00 0x7f2da019a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:51.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.906+0000 7f2da5d4f700 1 -- 172.21.15.12:0/987319805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2da0107730 msgr2=0x7f2da019ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:51.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.906+0000 7f2da5d4f700 1 --2- 172.21.15.12:0/987319805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2da0107730 0x7f2da019ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:51.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.906+0000 7f2da5d4f700 1 -- 172.21.15.12:0/987319805 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2da019f5c0 con 0x7f2da0108080 2024-05-17T22:54:51.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.906+0000 7f2da5d4f700 1 --2- 172.21.15.12:0/987319805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2da0108080 0x7f2da019efd0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2d98007ee0 tx=0x7f2d9800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:51.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.906+0000 7f2d967fc700 1 -- 172.21.15.12:0/987319805 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9800dbd0 con 0x7f2da0108080 2024-05-17T22:54:51.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.906+0000 7f2d967fc700 1 -- 172.21.15.12:0/987319805 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d9800dd70 con 0x7f2da0108080 2024-05-17T22:54:51.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.906+0000 7f2da77b2700 1 -- 172.21.15.12:0/987319805 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2da019f8b0 con 0x7f2da0108080 2024-05-17T22:54:51.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.906+0000 7f2d967fc700 1 -- 172.21.15.12:0/987319805 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d98005990 con 0x7f2da0108080 2024-05-17T22:54:51.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.906+0000 7f2da77b2700 1 -- 172.21.15.12:0/987319805 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2da019fdf0 con 0x7f2da0108080 2024-05-17T22:54:51.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.909+0000 7f2d967fc700 1 -- 172.21.15.12:0/987319805 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2d9801a020 con 0x7f2da0108080 2024-05-17T22:54:51.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.910+0000 7f2d967fc700 1 --2- 172.21.15.12:0/987319805 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f2d8c0b2900 0x7f2d8c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:51.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.910+0000 7f2da554e700 1 --2- 172.21.15.12:0/987319805 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f2d8c0b2900 0x7f2d8c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:51.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.910+0000 7f2d967fc700 1 -- 172.21.15.12:0/987319805 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f2d980e1c80 con 0x7f2da0108080 2024-05-17T22:54:51.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.910+0000 7f2db618db80 1 Processor -- start 2024-05-17T22:54:51.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.910+0000 7f2db618db80 1 -- start start 2024-05-17T22:54:51.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.910+0000 7f2da554e700 1 --2- 172.21.15.12:0/987319805 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f2d8c0b2900 0x7f2d8c0b4e10 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c003800 tx=0x7f2d9c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:51.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.910+0000 7f2db618db80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55de83c179f0 0x55de83d9f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:51.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.910+0000 7f2db618db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de83d9f810 0x55de83d999d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:51.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.910+0000 7f2db618db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de83d99f10 0x55de83d9a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:51.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.910+0000 7f2db618db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55de83cc40d0 con 0x55de83d99f10 2024-05-17T22:54:51.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.910+0000 7f2db618db80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55de83cb3c70 con 0x55de83c179f0 2024-05-17T22:54:51.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.910+0000 7f2db618db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55de83cb7cb0 con 0x55de83d9f810 2024-05-17T22:54:51.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.911+0000 7f2da5d4f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de83d99f10 0x55de83d9a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:51.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.911+0000 7f2da554e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55de83c179f0 0x55de83d9f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:51.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.911+0000 7f2da5d4f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de83d99f10 0x55de83d9a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52728/0 (socket says 172.21.15.12:52728) 2024-05-17T22:54:51.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.911+0000 7f2da5d4f700 1 -- 172.21.15.12:0/225459417 learned_addr learned my addr 172.21.15.12:0/225459417 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:54:51.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.911+0000 7f2da4d4d700 1 --2- 172.21.15.12:0/225459417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de83d9f810 0x55de83d999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:51.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.911+0000 7f2da5d4f700 1 -- 172.21.15.12:0/225459417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de83d9f810 msgr2=0x55de83d999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:51.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.911+0000 7f2da5d4f700 1 --2- 172.21.15.12:0/225459417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de83d9f810 0x55de83d999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:51.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.911+0000 7f2da5d4f700 1 -- 172.21.15.12:0/225459417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55de83c179f0 msgr2=0x55de83d9f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:51.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.911+0000 7f2da5d4f700 1 --2- 172.21.15.12:0/225459417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55de83c179f0 0x55de83d9f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:51.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.911+0000 7f2da5d4f700 1 -- 172.21.15.12:0/225459417 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de83c4d5c0 con 0x55de83d99f10 2024-05-17T22:54:51.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.911+0000 7f2da554e700 1 --2- 172.21.15.12:0/225459417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55de83c179f0 0x55de83d9f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:54:51.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.911+0000 7f2da5d4f700 1 --2- 172.21.15.12:0/225459417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de83d99f10 0x55de83d9a320 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2d980095b0 tx=0x7f2d980a8930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=107d2520ce2cf11b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:51.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.911+0000 7f2d8b7fe700 1 -- 172.21.15.12:0/225459417 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d980a8f10 con 0x55de83d99f10 2024-05-17T22:54:51.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.911+0000 7f2d8b7fe700 1 -- 172.21.15.12:0/225459417 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d980ad2c0 con 0x55de83d99f10 2024-05-17T22:54:51.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.911+0000 7f2d8b7fe700 1 -- 172.21.15.12:0/225459417 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d980b1b00 con 0x55de83d99f10 2024-05-17T22:54:51.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.912+0000 7f2db618db80 1 -- 172.21.15.12:0/225459417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de83d99f10 msgr2=0x55de83d9a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:51.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.912+0000 7f2db618db80 1 --2- 172.21.15.12:0/225459417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de83d99f10 0x55de83d9a320 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2d980095b0 tx=0x7f2d980a8930 comp rx=0 tx=0).stop 2024-05-17T22:54:51.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.912+0000 7f2db618db80 1 -- 172.21.15.12:0/225459417 shutdown_connections 2024-05-17T22:54:51.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.912+0000 7f2db618db80 1 --2- 172.21.15.12:0/225459417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de83d99f10 0x55de83d9a320 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:51.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.912+0000 7f2db618db80 1 --2- 172.21.15.12:0/225459417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55de83c179f0 0x55de83d9f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:51.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.912+0000 7f2db618db80 1 --2- 172.21.15.12:0/225459417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de83d9f810 0x55de83d999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:51.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.912+0000 7f2db618db80 1 -- 172.21.15.12:0/225459417 >> 172.21.15.12:0/225459417 conn(0x55de83c777d0 msgr2=0x55de83d901c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:51.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.912+0000 7f2db618db80 1 -- 172.21.15.12:0/225459417 shutdown_connections 2024-05-17T22:54:51.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.912+0000 7f2db618db80 1 -- 172.21.15.12:0/225459417 wait complete. 2024-05-17T22:54:51.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.913+0000 7f2db618db80 1 -- start start 2024-05-17T22:54:51.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.913+0000 7f2db618db80 1 Processor -- start 2024-05-17T22:54:51.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.913+0000 7f2db618db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de83c179f0 0x55de83da2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:51.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.913+0000 7f2db618db80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55de83d99f10 0x55de83da33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:51.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.913+0000 7f2db618db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de83d9f810 0x55de83da3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:51.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.913+0000 7f2db618db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55de83bbb150 con 0x55de83c179f0 2024-05-17T22:54:51.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.913+0000 7f2db618db80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55de83c55940 con 0x55de83d99f10 2024-05-17T22:54:51.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.913+0000 7f2db618db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55de83da61e0 con 0x55de83d9f810 2024-05-17T22:54:51.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.914+0000 7f2db618db80 10 client.0 _ll_register_callbacks cb 0x55de83c4ab50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:54:51.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.914+0000 7f2da554e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de83c179f0 0x55de83da2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:51.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.914+0000 7f2da554e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de83c179f0 0x55de83da2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52732/0 (socket says 172.21.15.12:52732) 2024-05-17T22:54:51.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.914+0000 7f2da554e700 1 -- 172.21.15.12:0/893301426 learned_addr learned my addr 172.21.15.12:0/893301426 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:54:51.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.914+0000 7f2da4d4d700 1 --2- 172.21.15.12:0/893301426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55de83d99f10 0x55de83da33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:51.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.914+0000 7f2da5d4f700 1 --2- 172.21.15.12:0/893301426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de83d9f810 0x55de83da3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:51.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.914+0000 7f2da554e700 1 -- 172.21.15.12:0/893301426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de83d9f810 msgr2=0x55de83da3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:51.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.914+0000 7f2da554e700 1 --2- 172.21.15.12:0/893301426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de83d9f810 0x55de83da3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:51.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.914+0000 7f2da554e700 1 -- 172.21.15.12:0/893301426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55de83d99f10 msgr2=0x55de83da33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:51.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.914+0000 7f2da554e700 1 --2- 172.21.15.12:0/893301426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55de83d99f10 0x55de83da33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:51.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.914+0000 7f2da554e700 1 -- 172.21.15.12:0/893301426 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de83c04480 con 0x55de83c179f0 2024-05-17T22:54:51.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.914+0000 7f2da5d4f700 1 --2- 172.21.15.12:0/893301426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de83d9f810 0x55de83da3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:54:51.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.914+0000 7f2da554e700 1 --2- 172.21.15.12:0/893301426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de83c179f0 0x55de83da2e90 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c009990 tx=0x7f2d9c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb4575584d23546e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:51.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.914+0000 7f2d88ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:54:51.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.914+0000 7f2d88ff9700 1 -- 172.21.15.12:0/893301426 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c02e850 con 0x55de83c179f0 2024-05-17T22:54:51.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.914+0000 7f2d88ff9700 1 -- 172.21.15.12:0/893301426 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f2d9c02e9f0 con 0x55de83c179f0 2024-05-17T22:54:51.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.914+0000 7f2db618db80 1 -- 172.21.15.12:0/893301426 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d980ad2c0 con 0x55de83c179f0 2024-05-17T22:54:51.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.915+0000 7f2d88ff9700 1 -- 172.21.15.12:0/893301426 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f2d9c03c050 con 0x55de83c179f0 2024-05-17T22:54:51.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.915+0000 7f2db618db80 10 client.16617 fetch_fsmap learned FSMap version 46 2024-05-17T22:54:51.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.915+0000 7f2db618db80 1 -- 172.21.15.12:0/893301426 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55de83db0d10 con 0x55de83c179f0 2024-05-17T22:54:51.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.915+0000 7f2d88ff9700 1 -- 172.21.15.12:0/893301426 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d980ad2c0 con 0x55de83c179f0 2024-05-17T22:54:51.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.915+0000 7f2d88ff9700 1 -- 172.21.15.12:0/893301426 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7f2d9c022c00 con 0x55de83c179f0 2024-05-17T22:54:51.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.915+0000 7f2db618db80 10 client.16617 fetch_fsmap finished waiting for FSMap version 46 2024-05-17T22:54:51.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.915+0000 7f2db618db80 10 client.16617 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:54:51.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.915+0000 7f2db618db80 20 client.16617 populate_metadata read hostname 'smithi012' 2024-05-17T22:54:51.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.915+0000 7f2db618db80 1 --2- 172.21.15.12:0/893301426 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55de83db33f0 0x55de83db5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:51.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.915+0000 7f2db618db80 4 client.16617 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:54:51.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.915+0000 7f2db618db80 1 -- 172.21.15.12:0/893301426 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de83cc6020 con 0x55de83db33f0 2024-05-17T22:54:51.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.916+0000 7f2da4d4d700 1 --2- 172.21.15.12:0/893301426 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55de83db33f0 0x55de83db5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:51.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.916+0000 7f2d88ff9700 10 client.16617 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:54:51.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.916+0000 7f2da4d4d700 1 --2- 172.21.15.12:0/893301426 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55de83db33f0 0x55de83db5810 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17252149b0c4d6c9 server_cookie=39762da19583936c in_seq=0 out_seq=0 2024-05-17T22:54:51.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.917+0000 7f2d88ff9700 1 -- 172.21.15.12:0/893301426 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55de83cc6020 con 0x55de83db33f0 2024-05-17T22:54:51.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.917+0000 7f2d88ff9700 10 client.16617 handle_command_reply: tid=0 2024-05-17T22:54:51.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.917+0000 7f2db618db80 1 client.16617 shutdown 2024-05-17T22:54:51.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.917+0000 7f2db618db80 1 -- 172.21.15.12:0/893301426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de83c179f0 msgr2=0x55de83da2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:51.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.917+0000 7f2db618db80 1 --2- 172.21.15.12:0/893301426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de83c179f0 0x55de83da2e90 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c009990 tx=0x7f2d9c009310 comp rx=0 tx=0).stop 2024-05-17T22:54:51.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.917+0000 7f2db618db80 1 -- 172.21.15.12:0/893301426 shutdown_connections 2024-05-17T22:54:51.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.917+0000 7f2db618db80 1 --2- 172.21.15.12:0/893301426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de83d9f810 0x55de83da3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:51.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.917+0000 7f2db618db80 1 --2- 172.21.15.12:0/893301426 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55de83db33f0 0x55de83db5810 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:51.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.917+0000 7f2db618db80 1 --2- 172.21.15.12:0/893301426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de83c179f0 0x55de83da2e90 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:51.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.917+0000 7f2db618db80 1 --2- 172.21.15.12:0/893301426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55de83d99f10 0x55de83da33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:51.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.917+0000 7f2d88ff9700 0 client.16617 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:54:51.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.917+0000 7f2db618db80 1 -- 172.21.15.12:0/893301426 >> 172.21.15.12:0/893301426 conn(0x55de83c777d0 msgr2=0x55de83d90160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:51.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.917+0000 7f2db618db80 1 -- 172.21.15.12:0/893301426 shutdown_connections 2024-05-17T22:54:51.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.917+0000 7f2db618db80 1 -- 172.21.15.12:0/893301426 wait complete. 2024-05-17T22:54:51.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.917+0000 7f2db618db80 20 client.16617 trim_cache size 0 max 16384 2024-05-17T22:54:51.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.938+0000 7f2db618db80 1 Processor -- start 2024-05-17T22:54:51.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.938+0000 7f2db618db80 1 -- start start 2024-05-17T22:54:51.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.938+0000 7f2db618db80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55de83c179f0 0x55de83d999d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:51.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.938+0000 7f2db618db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de83d99f10 0x55de83daf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:51.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.938+0000 7f2db618db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de83d9f810 0x55de83d9e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:51.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.938+0000 7f2db618db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2da01b0290 con 0x55de83d9f810 2024-05-17T22:54:51.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.938+0000 7f2db618db80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2da01ada90 con 0x55de83c179f0 2024-05-17T22:54:51.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.938+0000 7f2db618db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2da01ab900 con 0x55de83d99f10 2024-05-17T22:54:51.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.938+0000 7f2da5d4f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de83d9f810 0x55de83d9e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:51.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.938+0000 7f2da5d4f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de83d9f810 0x55de83d9e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52740/0 (socket says 172.21.15.12:52740) 2024-05-17T22:54:51.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.938+0000 7f2da5d4f700 1 -- 172.21.15.12:0/3217690917 learned_addr learned my addr 172.21.15.12:0/3217690917 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:54:51.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.938+0000 7f2da554e700 1 --2- 172.21.15.12:0/3217690917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55de83c179f0 0x55de83d999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:51.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.939+0000 7f2da4d4d700 1 --2- 172.21.15.12:0/3217690917 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de83d99f10 0x55de83daf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:51.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.939+0000 7f2da5d4f700 1 -- 172.21.15.12:0/3217690917 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de83d99f10 msgr2=0x55de83daf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:51.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.939+0000 7f2da5d4f700 1 --2- 172.21.15.12:0/3217690917 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de83d99f10 0x55de83daf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:51.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.939+0000 7f2da5d4f700 1 -- 172.21.15.12:0/3217690917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55de83c179f0 msgr2=0x55de83d999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:51.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.939+0000 7f2da5d4f700 1 --2- 172.21.15.12:0/3217690917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55de83c179f0 0x55de83d999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:51.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.939+0000 7f2da5d4f700 1 -- 172.21.15.12:0/3217690917 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de83d9f640 con 0x55de83d9f810 2024-05-17T22:54:51.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.939+0000 7f2da554e700 1 --2- 172.21.15.12:0/3217690917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55de83c179f0 0x55de83d999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:54:51.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.939+0000 7f2da5d4f700 1 --2- 172.21.15.12:0/3217690917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de83d9f810 0x55de83d9e870 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2d9800d7b0 tx=0x7f2d9800df20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfc8cb523018cf43 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:51.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.939+0000 7f2d8affd700 1 -- 172.21.15.12:0/3217690917 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d980a8f10 con 0x55de83d9f810 2024-05-17T22:54:51.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.939+0000 7f2d8affd700 1 -- 172.21.15.12:0/3217690917 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d980ad2c0 con 0x55de83d9f810 2024-05-17T22:54:51.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.939+0000 7f2d8affd700 1 -- 172.21.15.12:0/3217690917 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d980a4880 con 0x55de83d9f810 2024-05-17T22:54:51.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.940+0000 7f2db618db80 1 -- 172.21.15.12:0/3217690917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de83d9f810 msgr2=0x55de83d9e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:51.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.940+0000 7f2db618db80 1 --2- 172.21.15.12:0/3217690917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de83d9f810 0x55de83d9e870 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2d9800d7b0 tx=0x7f2d9800df20 comp rx=0 tx=0).stop 2024-05-17T22:54:51.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.940+0000 7f2db618db80 1 -- 172.21.15.12:0/3217690917 shutdown_connections 2024-05-17T22:54:51.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.940+0000 7f2db618db80 1 --2- 172.21.15.12:0/3217690917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de83d9f810 0x55de83d9e870 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:51.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.940+0000 7f2db618db80 1 --2- 172.21.15.12:0/3217690917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55de83c179f0 0x55de83d999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:51.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.940+0000 7f2db618db80 1 --2- 172.21.15.12:0/3217690917 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de83d99f10 0x55de83daf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:51.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.940+0000 7f2db618db80 1 -- 172.21.15.12:0/3217690917 >> 172.21.15.12:0/3217690917 conn(0x55de83c777d0 msgr2=0x55de83d9d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:51.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.940+0000 7f2db618db80 1 -- 172.21.15.12:0/3217690917 shutdown_connections 2024-05-17T22:54:51.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.940+0000 7f2db618db80 1 -- 172.21.15.12:0/3217690917 wait complete. 2024-05-17T22:54:51.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.940+0000 7f2db618db80 1 -- start start 2024-05-17T22:54:51.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.940+0000 7f2db618db80 1 Processor -- start 2024-05-17T22:54:51.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.941+0000 7f2db618db80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de83c179f0 0x55de83dabb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:51.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.941+0000 7f2db618db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de83d99f10 0x55de83dba660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:51.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.941+0000 7f2db618db80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55de83d9f810 0x55de83dbac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:51.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.941+0000 7f2db618db80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2da01abf50 con 0x55de83c179f0 2024-05-17T22:54:51.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.941+0000 7f2db618db80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2da01ad3c0 con 0x55de83d9f810 2024-05-17T22:54:51.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.941+0000 7f2db618db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2da01ac620 con 0x55de83d99f10 2024-05-17T22:54:51.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.941+0000 7f2db618db80 10 client.0 _ll_register_callbacks cb 0x55de83c4ab50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:54:51.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.941+0000 7f2da554e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de83c179f0 0x55de83dabb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:51.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.941+0000 7f2da554e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de83c179f0 0x55de83dabb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52756/0 (socket says 172.21.15.12:52756) 2024-05-17T22:54:51.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.941+0000 7f2da554e700 1 -- 172.21.15.12:0/369461015 learned_addr learned my addr 172.21.15.12:0/369461015 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:54:51.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.941+0000 7f2da4d4d700 1 --2- 172.21.15.12:0/369461015 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de83d99f10 0x55de83dba660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:51.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.941+0000 7f2da5d4f700 1 --2- 172.21.15.12:0/369461015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55de83d9f810 0x55de83dbac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:51.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.941+0000 7f2da554e700 1 -- 172.21.15.12:0/369461015 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de83d99f10 msgr2=0x55de83dba660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:51.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.941+0000 7f2da554e700 1 --2- 172.21.15.12:0/369461015 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de83d99f10 0x55de83dba660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:51.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.941+0000 7f2da554e700 1 -- 172.21.15.12:0/369461015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55de83d9f810 msgr2=0x55de83dbac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:51.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.941+0000 7f2da554e700 1 --2- 172.21.15.12:0/369461015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55de83d9f810 0x55de83dbac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:51.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.941+0000 7f2da554e700 1 -- 172.21.15.12:0/369461015 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de83dbb430 con 0x55de83c179f0 2024-05-17T22:54:51.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.942+0000 7f2da554e700 1 --2- 172.21.15.12:0/369461015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de83c179f0 0x55de83dabb70 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c000c00 tx=0x7f2d9c018520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=243a1c42b542db2e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:51.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.942+0000 7f2d88ff9700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:54:51.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.942+0000 7f2d88ff9700 1 -- 172.21.15.12:0/369461015 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c046070 con 0x55de83c179f0 2024-05-17T22:54:51.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.942+0000 7f2d88ff9700 1 -- 172.21.15.12:0/369461015 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f2d9c018dd0 con 0x55de83c179f0 2024-05-17T22:54:51.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.942+0000 7f2db618db80 1 -- 172.21.15.12:0/369461015 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d980ad2c0 con 0x55de83c179f0 2024-05-17T22:54:51.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.942+0000 7f2d88ff9700 1 -- 172.21.15.12:0/369461015 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f2d9c009570 con 0x55de83c179f0 2024-05-17T22:54:51.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.942+0000 7f2db618db80 10 client.16623 fetch_fsmap learned FSMap version 46 2024-05-17T22:54:51.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.942+0000 7f2db618db80 1 -- 172.21.15.12:0/369461015 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55de83e1dc10 con 0x55de83c179f0 2024-05-17T22:54:51.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.942+0000 7f2d88ff9700 1 -- 172.21.15.12:0/369461015 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d980ad2c0 con 0x55de83c179f0 2024-05-17T22:54:51.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.942+0000 7f2d88ff9700 1 -- 172.21.15.12:0/369461015 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7f2d9c0227e0 con 0x55de83c179f0 2024-05-17T22:54:51.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.942+0000 7f2db618db80 10 client.16623 fetch_fsmap finished waiting for FSMap version 46 2024-05-17T22:54:51.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.942+0000 7f2db618db80 10 client.16623 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:54:51.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.942+0000 7f2db618db80 20 client.16623 populate_metadata read hostname 'smithi012' 2024-05-17T22:54:51.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.942+0000 7f2db618db80 1 --2- 172.21.15.12:0/369461015 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55de83e1e3a0 0x55de83e207c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:51.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.942+0000 7f2db618db80 4 client.16623 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:54:51.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.942+0000 7f2db618db80 1 -- 172.21.15.12:0/369461015 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55de83cbf800 con 0x55de83e1e3a0 2024-05-17T22:54:51.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.943+0000 7f2da4d4d700 1 --2- 172.21.15.12:0/369461015 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55de83e1e3a0 0x55de83e207c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:51.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.943+0000 7f2d88ff9700 10 client.16623 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:54:51.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.943+0000 7f2da4d4d700 1 --2- 172.21.15.12:0/369461015 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55de83e1e3a0 0x55de83e207c0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de95d56e838ab23f server_cookie=ed59efdb21c02be7 in_seq=0 out_seq=0 2024-05-17T22:54:51.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.943+0000 7f2d88ff9700 1 -- 172.21.15.12:0/369461015 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55de83cbf800 con 0x55de83e1e3a0 2024-05-17T22:54:51.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.943+0000 7f2d88ff9700 10 client.16623 handle_command_reply: tid=0 2024-05-17T22:54:51.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.943+0000 7f2db618db80 1 client.16623 shutdown 2024-05-17T22:54:51.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.943+0000 7f2db618db80 1 -- 172.21.15.12:0/369461015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de83c179f0 msgr2=0x55de83dabb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:51.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.943+0000 7f2db618db80 1 --2- 172.21.15.12:0/369461015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de83c179f0 0x55de83dabb70 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c000c00 tx=0x7f2d9c018520 comp rx=0 tx=0).stop 2024-05-17T22:54:51.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.943+0000 7f2db618db80 1 -- 172.21.15.12:0/369461015 shutdown_connections 2024-05-17T22:54:51.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.943+0000 7f2db618db80 1 --2- 172.21.15.12:0/369461015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55de83d9f810 0x55de83dbac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:51.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.943+0000 7f2db618db80 1 --2- 172.21.15.12:0/369461015 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55de83e1e3a0 0x55de83e207c0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:51.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.943+0000 7f2db618db80 1 --2- 172.21.15.12:0/369461015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de83c179f0 0x55de83dabb70 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:51.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.943+0000 7f2d88ff9700 0 client.16623 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:54:51.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.943+0000 7f2db618db80 1 --2- 172.21.15.12:0/369461015 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de83d99f10 0x55de83dba660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:51.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.943+0000 7f2db618db80 1 -- 172.21.15.12:0/369461015 >> 172.21.15.12:0/369461015 conn(0x55de83c777d0 msgr2=0x55de83d9d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:51.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.944+0000 7f2db618db80 1 -- 172.21.15.12:0/369461015 shutdown_connections 2024-05-17T22:54:51.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.944+0000 7f2db618db80 1 -- 172.21.15.12:0/369461015 wait complete. 2024-05-17T22:54:51.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.944+0000 7f2db618db80 20 client.16623 trim_cache size 0 max 16384 2024-05-17T22:54:51.951 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:54:51.951 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-05-17T22:54:51.951 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-05-17T22:54:51.951 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:54:51.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.944+0000 7f2da77b2700 1 -- 172.21.15.12:0/987319805 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f2d8c0b2900 msgr2=0x7f2d8c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:51.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.944+0000 7f2da77b2700 1 --2- 172.21.15.12:0/987319805 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f2d8c0b2900 0x7f2d8c0b4e10 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c003800 tx=0x7f2d9c01a040 comp rx=0 tx=0).stop 2024-05-17T22:54:51.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.944+0000 7f2da77b2700 1 -- 172.21.15.12:0/987319805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2da0108080 msgr2=0x7f2da019efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:51.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.944+0000 7f2da77b2700 1 --2- 172.21.15.12:0/987319805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2da0108080 0x7f2da019efd0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2d98007ee0 tx=0x7f2d9800d3b0 comp rx=0 tx=0).stop 2024-05-17T22:54:51.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.944+0000 7f2da77b2700 1 -- 172.21.15.12:0/987319805 shutdown_connections 2024-05-17T22:54:51.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.944+0000 7f2da77b2700 1 --2- 172.21.15.12:0/987319805 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f2d8c0b2900 0x7f2d8c0b4e10 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:51.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.944+0000 7f2da77b2700 1 --2- 172.21.15.12:0/987319805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2da0108080 0x7f2da019efd0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:51.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.944+0000 7f2da77b2700 1 --2- 172.21.15.12:0/987319805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2da0102e00 0x7f2da019a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:51.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.944+0000 7f2da77b2700 1 --2- 172.21.15.12:0/987319805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2da0107730 0x7f2da019ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:51.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.944+0000 7f2da77b2700 1 -- 172.21.15.12:0/987319805 >> 172.21.15.12:0/987319805 conn(0x7f2da0079bd0 msgr2=0x7f2da0106110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:51.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.944+0000 7f2da77b2700 1 -- 172.21.15.12:0/987319805 shutdown_connections 2024-05-17T22:54:51.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:51.944+0000 7f2da77b2700 1 -- 172.21.15.12:0/987319805 wait complete. 2024-05-17T22:54:52.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:52 smithi151 ceph-mon[36101]: pgmap v907: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 105 KiB/s wr, 20 op/s 2024-05-17T22:54:52.512 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:52 smithi012 ceph-mon[31632]: pgmap v907: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 105 KiB/s wr, 20 op/s 2024-05-17T22:54:52.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:52 smithi096 ceph-mon[36065]: pgmap v907: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 105 KiB/s wr, 20 op/s 2024-05-17T22:54:53.295 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-17T22:54:53.296 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T22:54:53.555 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:54:54.292 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:54 smithi012 ceph-mon[31632]: pgmap v908: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 50 KiB/s wr, 11 op/s 2024-05-17T22:54:54.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:54 smithi151 ceph-mon[36101]: pgmap v908: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 50 KiB/s wr, 11 op/s 2024-05-17T22:54:54.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:54 smithi096 ceph-mon[36065]: pgmap v908: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 50 KiB/s wr, 11 op/s 2024-05-17T22:54:56.206 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:56 smithi012 ceph-mon[31632]: pgmap v909: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 53 KiB/s wr, 12 op/s 2024-05-17T22:54:56.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.325+0000 7f7fa5486700 1 -- 172.21.15.12:0/4020540260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7fa0108840 msgr2=0x7f7fa0108c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:56.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.325+0000 7f7fa5486700 1 --2- 172.21.15.12:0/4020540260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7fa0108840 0x7f7fa0108c50 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7f88009990 tx=0x7f7f88009ca0 comp rx=0 tx=0).stop 2024-05-17T22:54:56.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.325+0000 7f7fa5486700 1 -- 172.21.15.12:0/4020540260 shutdown_connections 2024-05-17T22:54:56.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.325+0000 7f7fa5486700 1 --2- 172.21.15.12:0/4020540260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fa0109190 0x7f7fa0110af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:56.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.325+0000 7f7fa5486700 1 --2- 172.21.15.12:0/4020540260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fa0103f10 0x7f7fa0104320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:56.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.325+0000 7f7fa5486700 1 --2- 172.21.15.12:0/4020540260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7fa0108840 0x7f7fa0108c50 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:56.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.325+0000 7f7fa5486700 1 -- 172.21.15.12:0/4020540260 >> 172.21.15.12:0/4020540260 conn(0x7f7fa00ff9b0 msgr2=0x7f7fa0101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:56.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.325+0000 7f7fa5486700 1 -- 172.21.15.12:0/4020540260 shutdown_connections 2024-05-17T22:54:56.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.325+0000 7f7fa5486700 1 -- 172.21.15.12:0/4020540260 wait complete. 2024-05-17T22:54:56.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.326+0000 7f7fa5486700 1 Processor -- start 2024-05-17T22:54:56.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.326+0000 7f7fa5486700 1 -- start start 2024-05-17T22:54:56.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.326+0000 7f7fa5486700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fa0103f10 0x7f7fa0075b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:56.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.326+0000 7f7fa5486700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fa0108840 0x7f7fa0076070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:56.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.326+0000 7f7fa5486700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7fa0109190 0x7f7fa00765b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:56.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.326+0000 7f7fa5486700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa0113dc0 con 0x7f7fa0109190 2024-05-17T22:54:56.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.326+0000 7f7fa5486700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa0113c40 con 0x7f7fa0103f10 2024-05-17T22:54:56.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.326+0000 7f7fa5486700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa0113f40 con 0x7f7fa0108840 2024-05-17T22:54:56.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.326+0000 7f7f9f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7fa0109190 0x7f7fa00765b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:56.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.326+0000 7f7f9f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7fa0109190 0x7f7fa00765b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52774/0 (socket says 172.21.15.12:52774) 2024-05-17T22:54:56.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.326+0000 7f7f9f7fe700 1 -- 172.21.15.12:0/2146086583 learned_addr learned my addr 172.21.15.12:0/2146086583 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:54:56.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.326+0000 7f7f97fff700 1 --2- 172.21.15.12:0/2146086583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fa0108840 0x7f7fa0076070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:56.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.326+0000 7f7f9effd700 1 --2- 172.21.15.12:0/2146086583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fa0103f10 0x7f7fa0075b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:56.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.326+0000 7f7f9f7fe700 1 -- 172.21.15.12:0/2146086583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fa0108840 msgr2=0x7f7fa0076070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:56.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.326+0000 7f7f9f7fe700 1 --2- 172.21.15.12:0/2146086583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fa0108840 0x7f7fa0076070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:56.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.326+0000 7f7f9f7fe700 1 -- 172.21.15.12:0/2146086583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fa0103f10 msgr2=0x7f7fa0075b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:56.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.326+0000 7f7f9f7fe700 1 --2- 172.21.15.12:0/2146086583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fa0103f10 0x7f7fa0075b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:56.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.326+0000 7f7f9f7fe700 1 -- 172.21.15.12:0/2146086583 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fa01a8ad0 con 0x7f7fa0109190 2024-05-17T22:54:56.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.326+0000 7f7f9effd700 1 --2- 172.21.15.12:0/2146086583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fa0103f10 0x7f7fa0075b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:54:56.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.327+0000 7f7f9f7fe700 1 --2- 172.21.15.12:0/2146086583 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7fa0109190 0x7f7fa00765b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f7f98007ee0 tx=0x7f7f9800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:56.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.327+0000 7f7f9cff9700 1 -- 172.21.15.12:0/2146086583 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9800dbd0 con 0x7f7fa0109190 2024-05-17T22:54:56.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.327+0000 7f7f9cff9700 1 -- 172.21.15.12:0/2146086583 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f9800dd70 con 0x7f7fa0109190 2024-05-17T22:54:56.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.327+0000 7f7fa5486700 1 -- 172.21.15.12:0/2146086583 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fa01a8dc0 con 0x7f7fa0109190 2024-05-17T22:54:56.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.327+0000 7f7f9cff9700 1 -- 172.21.15.12:0/2146086583 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f98005990 con 0x7f7fa0109190 2024-05-17T22:54:56.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.327+0000 7f7fa5486700 1 -- 172.21.15.12:0/2146086583 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fa01a9270 con 0x7f7fa0109190 2024-05-17T22:54:56.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.328+0000 7f7fa5486700 1 -- 172.21.15.12:0/2146086583 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fa0104320 con 0x7f7fa0109190 2024-05-17T22:54:56.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.330+0000 7f7f9cff9700 1 -- 172.21.15.12:0/2146086583 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7f9801a020 con 0x7f7fa0109190 2024-05-17T22:54:56.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.330+0000 7f7f9cff9700 1 --2- 172.21.15.12:0/2146086583 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f7f8c0b2880 0x7f7f8c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:56.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.330+0000 7f7f9effd700 1 --2- 172.21.15.12:0/2146086583 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f7f8c0b2880 0x7f7f8c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:56.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.331+0000 7f7f9cff9700 1 -- 172.21.15.12:0/2146086583 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f7f980e2360 con 0x7f7fa0109190 2024-05-17T22:54:56.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.331+0000 7f7f9effd700 1 --2- 172.21.15.12:0/2146086583 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f7f8c0b2880 0x7f7f8c0b4d90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7f90005fd0 tx=0x7f7f90005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:56.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.334+0000 7f7f9cff9700 1 -- 172.21.15.12:0/2146086583 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7f98010070 con 0x7f7fa0109190 2024-05-17T22:54:56.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:56 smithi151 ceph-mon[36101]: pgmap v909: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 53 KiB/s wr, 12 op/s 2024-05-17T22:54:56.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:56 smithi096 ceph-mon[36065]: pgmap v909: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 53 KiB/s wr, 12 op/s 2024-05-17T22:54:56.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.599+0000 7f7fa5486700 1 -- 172.21.15.12:0/2146086583 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7fa0064440 con 0x7f7fa0109190 2024-05-17T22:54:56.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.599+0000 7f7f9cff9700 1 -- 172.21.15.12:0/2146086583 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11874 (secure 0 0 0) 0x7f7f980a9d90 con 0x7f7fa0109190 2024-05-17T22:54:56.602 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:54:56.602 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1407219319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1407219319},{"type":"v1","addr":"172.21.15.12:6835","nonce":1407219319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/630977886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":630977886},{"type":"v1","addr":"172.21.15.12:6837","nonce":630977886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3731914742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3731914742},{"type":"v1","addr":"172.21.15.12:6839","nonce":3731914742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/3642142501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3642142501},{"type":"v1","addr":"172.21.15.12:6841","nonce":3642142501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3554053059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3554053059},{"type":"v1","addr":"172.21.15.151:6833","nonce":3554053059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/4228325569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":4228325569},{"type":"v1","addr":"172.21.15.96:6833","nonce":4228325569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T22:34:42.642111+0000","modified":"2024-05-17T22:54:42.952771+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24461,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/542323754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":542323754},{"type":"v1","addr":"172.21.15.151:6835","nonce":542323754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1758442232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1758442232},{"type":"v1","addr":"172.21.15.96:6835","nonce":1758442232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3403324038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3403324038},{"type":"v1","addr":"172.21.15.151:6837","nonce":3403324038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.151:6839/2338353849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2338353849},{"type":"v1","addr":"172.21.15.151:6839","nonce":2338353849}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.96:6837/4222574338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":4222574338},{"type":"v1","addr":"172.21.15.96:6837","nonce":4222574338}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/2986295519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2986295519},{"type":"v1","addr":"172.21.15.96:6839","nonce":2986295519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24457,24482,24461]},"id":1}]} 2024-05-17T22:54:56.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.601+0000 7f7fa5486700 1 -- 172.21.15.12:0/2146086583 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f7f8c0b2880 msgr2=0x7f7f8c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:56.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.601+0000 7f7fa5486700 1 --2- 172.21.15.12:0/2146086583 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f7f8c0b2880 0x7f7f8c0b4d90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7f90005fd0 tx=0x7f7f90005e20 comp rx=0 tx=0).stop 2024-05-17T22:54:56.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.601+0000 7f7fa5486700 1 -- 172.21.15.12:0/2146086583 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7fa0109190 msgr2=0x7f7fa00765b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:56.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.601+0000 7f7fa5486700 1 --2- 172.21.15.12:0/2146086583 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7fa0109190 0x7f7fa00765b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f7f98007ee0 tx=0x7f7f9800d3b0 comp rx=0 tx=0).stop 2024-05-17T22:54:56.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.602+0000 7f7fa5486700 1 -- 172.21.15.12:0/2146086583 shutdown_connections 2024-05-17T22:54:56.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.602+0000 7f7fa5486700 1 --2- 172.21.15.12:0/2146086583 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f7f8c0b2880 0x7f7f8c0b4d90 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:56.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.602+0000 7f7fa5486700 1 --2- 172.21.15.12:0/2146086583 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7fa0109190 0x7f7fa00765b0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:56.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.602+0000 7f7fa5486700 1 --2- 172.21.15.12:0/2146086583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fa0103f10 0x7f7fa0075b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:56.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.602+0000 7f7fa5486700 1 --2- 172.21.15.12:0/2146086583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fa0108840 0x7f7fa0076070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:56.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.602+0000 7f7fa5486700 1 -- 172.21.15.12:0/2146086583 >> 172.21.15.12:0/2146086583 conn(0x7f7fa00ff9b0 msgr2=0x7f7fa0101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:56.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.602+0000 7f7fa5486700 1 -- 172.21.15.12:0/2146086583 shutdown_connections 2024-05-17T22:54:56.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:56.602+0000 7f7fa5486700 1 -- 172.21.15.12:0/2146086583 wait complete. 2024-05-17T22:54:56.605 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 46 2024-05-17T22:54:57.229 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:57 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/2146086583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:54:57.257 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-17T22:54:57.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:57 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/2146086583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:54:57.509 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:54:57.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:57 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/2146086583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:54:58.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:54:58 smithi012 ceph-mon[31632]: pgmap v910: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 60 KiB/s wr, 12 op/s 2024-05-17T22:54:58.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:54:58 smithi151 ceph-mon[36101]: pgmap v910: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 60 KiB/s wr, 12 op/s 2024-05-17T22:54:58.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:54:58 smithi096 ceph-mon[36065]: pgmap v910: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 60 KiB/s wr, 12 op/s 2024-05-17T22:54:59.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.246+0000 7f0589545700 1 -- 172.21.15.12:0/1374151246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05841080a0 msgr2=0x7f058410fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:59.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.246+0000 7f0589545700 1 --2- 172.21.15.12:0/1374151246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05841080a0 0x7f058410fa00 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f057c00b0a0 tx=0x7f057c00b3b0 comp rx=0 tx=0).stop 2024-05-17T22:54:59.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.246+0000 7f0589545700 1 -- 172.21.15.12:0/1374151246 shutdown_connections 2024-05-17T22:54:59.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.246+0000 7f0589545700 1 --2- 172.21.15.12:0/1374151246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05841080a0 0x7f058410fa00 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:59.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.246+0000 7f0589545700 1 --2- 172.21.15.12:0/1374151246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0584102e70 0x7f0584103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:59.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.246+0000 7f0589545700 1 --2- 172.21.15.12:0/1374151246 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0584107750 0x7f0584107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:59.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.246+0000 7f0589545700 1 -- 172.21.15.12:0/1374151246 >> 172.21.15.12:0/1374151246 conn(0x7f0584079ae0 msgr2=0x7f0584079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:59.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.246+0000 7f0589545700 1 -- 172.21.15.12:0/1374151246 shutdown_connections 2024-05-17T22:54:59.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.246+0000 7f0589545700 1 -- 172.21.15.12:0/1374151246 wait complete. 2024-05-17T22:54:59.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.247+0000 7f0589545700 1 Processor -- start 2024-05-17T22:54:59.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.247+0000 7f0589545700 1 -- start start 2024-05-17T22:54:59.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.248+0000 7f0589545700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0584102e70 0x7f058419b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:59.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.248+0000 7f0589545700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0584107750 0x7f058419bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:59.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.248+0000 7f0589545700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05841080a0 0x7f058419c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:59.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.248+0000 7f0589545700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0584112d50 con 0x7f0584107750 2024-05-17T22:54:59.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.248+0000 7f0589545700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0584112bd0 con 0x7f05841080a0 2024-05-17T22:54:59.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.248+0000 7f0589545700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0584112ed0 con 0x7f0584102e70 2024-05-17T22:54:59.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.248+0000 7f05827fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0584107750 0x7f058419bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:59.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.248+0000 7f05827fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0584107750 0x7f058419bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52800/0 (socket says 172.21.15.12:52800) 2024-05-17T22:54:59.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.248+0000 7f05827fc700 1 -- 172.21.15.12:0/3102908641 learned_addr learned my addr 172.21.15.12:0/3102908641 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:54:59.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.248+0000 7f0582ffd700 1 --2- 172.21.15.12:0/3102908641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0584102e70 0x7f058419b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:59.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.248+0000 7f05837fe700 1 --2- 172.21.15.12:0/3102908641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05841080a0 0x7f058419c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:59.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.248+0000 7f05827fc700 1 -- 172.21.15.12:0/3102908641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0584102e70 msgr2=0x7f058419b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:59.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.248+0000 7f05827fc700 1 --2- 172.21.15.12:0/3102908641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0584102e70 0x7f058419b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:59.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.248+0000 7f05827fc700 1 -- 172.21.15.12:0/3102908641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05841080a0 msgr2=0x7f058419c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:59.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.248+0000 7f05827fc700 1 --2- 172.21.15.12:0/3102908641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05841080a0 0x7f058419c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:59.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.248+0000 7f05827fc700 1 -- 172.21.15.12:0/3102908641 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05841a6950 con 0x7f0584107750 2024-05-17T22:54:59.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.249+0000 7f05827fc700 1 --2- 172.21.15.12:0/3102908641 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0584107750 0x7f058419bb40 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f057800d980 tx=0x7f057800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:59.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.249+0000 7f056bfff700 1 -- 172.21.15.12:0/3102908641 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0578017070 con 0x7f0584107750 2024-05-17T22:54:59.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.249+0000 7f0589545700 1 -- 172.21.15.12:0/3102908641 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05841a6c40 con 0x7f0584107750 2024-05-17T22:54:59.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.249+0000 7f056bfff700 1 -- 172.21.15.12:0/3102908641 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f057800fd40 con 0x7f0584107750 2024-05-17T22:54:59.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.249+0000 7f0589545700 1 -- 172.21.15.12:0/3102908641 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05841a7180 con 0x7f0584107750 2024-05-17T22:54:59.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.249+0000 7f056bfff700 1 -- 172.21.15.12:0/3102908641 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057800e3b0 con 0x7f0584107750 2024-05-17T22:54:59.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.253+0000 7f056bfff700 1 -- 172.21.15.12:0/3102908641 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f057800e550 con 0x7f0584107750 2024-05-17T22:54:59.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.253+0000 7f0597f20b80 1 Processor -- start 2024-05-17T22:54:59.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.254+0000 7f0597f20b80 1 -- start start 2024-05-17T22:54:59.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.254+0000 7f056bfff700 1 --2- 172.21.15.12:0/3102908641 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f056c0b2900 0x7f056c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:59.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.254+0000 7f05837fe700 1 --2- 172.21.15.12:0/3102908641 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f056c0b2900 0x7f056c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:59.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.254+0000 7f0597f20b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560263add9f0 0x560263c61560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:59.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.254+0000 7f056bfff700 1 -- 172.21.15.12:0/3102908641 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f05780e23a0 con 0x7f0584107750 2024-05-17T22:54:59.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.254+0000 7f0597f20b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560263c61aa0 0x560263c5bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:59.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.254+0000 7f0597f20b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560263c5c170 0x560263c5c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:59.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.254+0000 7f0597f20b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x560263b8a290 con 0x560263c5c170 2024-05-17T22:54:59.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.254+0000 7f0597f20b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560263b79fd0 con 0x560263add9f0 2024-05-17T22:54:59.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.254+0000 7f0597f20b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560263b7de70 con 0x560263c61aa0 2024-05-17T22:54:59.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.254+0000 7f05837fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560263c5c170 0x560263c5c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:59.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.254+0000 7f05837fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560263c5c170 0x560263c5c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52816/0 (socket says 172.21.15.12:52816) 2024-05-17T22:54:59.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.254+0000 7f05837fe700 1 -- 172.21.15.12:0/1535768463 learned_addr learned my addr 172.21.15.12:0/1535768463 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:54:59.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.254+0000 7f0582ffd700 1 --2- 172.21.15.12:0/1535768463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560263add9f0 0x560263c61560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:59.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.254+0000 7f05827fc700 1 --2- 172.21.15.12:0/1535768463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560263c61aa0 0x560263c5bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:59.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.254+0000 7f05837fe700 1 --2- 172.21.15.12:0/3102908641 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f056c0b2900 0x7f056c0b4e10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f057c00b070 tx=0x7f057c009540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:59.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.254+0000 7f05837fe700 1 -- 172.21.15.12:0/1535768463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560263c61aa0 msgr2=0x560263c5bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:59.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.255+0000 7f05837fe700 1 --2- 172.21.15.12:0/1535768463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560263c61aa0 0x560263c5bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:59.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.255+0000 7f05837fe700 1 -- 172.21.15.12:0/1535768463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560263add9f0 msgr2=0x560263c61560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:59.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.255+0000 7f05837fe700 1 --2- 172.21.15.12:0/1535768463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560263add9f0 0x560263c61560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:59.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.255+0000 7f05837fe700 1 -- 172.21.15.12:0/1535768463 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560263b23590 con 0x560263c5c170 2024-05-17T22:54:59.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.255+0000 7f05827fc700 1 --2- 172.21.15.12:0/1535768463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560263c61aa0 0x560263c5bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:54:59.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.255+0000 7f05837fe700 1 --2- 172.21.15.12:0/1535768463 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560263c5c170 0x560263c5c580 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f057c015000 tx=0x7f057c029000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9931b6b5b8e05b4b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:59.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.255+0000 7f05697fa700 1 -- 172.21.15.12:0/1535768463 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c007920 con 0x560263c5c170 2024-05-17T22:54:59.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.255+0000 7f05697fa700 1 -- 172.21.15.12:0/1535768463 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f057c01e440 con 0x560263c5c170 2024-05-17T22:54:59.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.255+0000 7f05697fa700 1 -- 172.21.15.12:0/1535768463 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c01f6a0 con 0x560263c5c170 2024-05-17T22:54:59.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.256+0000 7f0597f20b80 1 -- 172.21.15.12:0/1535768463 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560263c5c170 msgr2=0x560263c5c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:59.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.256+0000 7f0597f20b80 1 --2- 172.21.15.12:0/1535768463 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560263c5c170 0x560263c5c580 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f057c015000 tx=0x7f057c029000 comp rx=0 tx=0).stop 2024-05-17T22:54:59.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.256+0000 7f0597f20b80 1 -- 172.21.15.12:0/1535768463 shutdown_connections 2024-05-17T22:54:59.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.256+0000 7f0597f20b80 1 --2- 172.21.15.12:0/1535768463 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560263c5c170 0x560263c5c580 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:59.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.256+0000 7f0597f20b80 1 --2- 172.21.15.12:0/1535768463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560263add9f0 0x560263c61560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:59.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.256+0000 7f0597f20b80 1 --2- 172.21.15.12:0/1535768463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560263c61aa0 0x560263c5bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:59.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.256+0000 7f0597f20b80 1 -- 172.21.15.12:0/1535768463 >> 172.21.15.12:0/1535768463 conn(0x560263b21f00 msgr2=0x560263c56b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:59.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.256+0000 7f0597f20b80 1 -- 172.21.15.12:0/1535768463 shutdown_connections 2024-05-17T22:54:59.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.256+0000 7f0597f20b80 1 -- 172.21.15.12:0/1535768463 wait complete. 2024-05-17T22:54:59.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.256+0000 7f0597f20b80 1 -- start start 2024-05-17T22:54:59.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.257+0000 7f0597f20b80 1 Processor -- start 2024-05-17T22:54:59.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.257+0000 7f0597f20b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560263add9f0 0x560263c651d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:59.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.257+0000 7f0597f20b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560263c5c170 0x560263c65710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:59.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.257+0000 7f0597f20b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560263c61aa0 0x560263c65c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:59.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.257+0000 7f0597f20b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x560263a81150 con 0x560263c5c170 2024-05-17T22:54:59.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.257+0000 7f0597f20b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560263b1ae80 con 0x560263c61aa0 2024-05-17T22:54:59.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.257+0000 7f0597f20b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560263c68390 con 0x560263add9f0 2024-05-17T22:54:59.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.257+0000 7f0597f20b80 10 client.0 _ll_register_callbacks cb 0x560263b14f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:54:59.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.257+0000 7f05827fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560263c5c170 0x560263c65710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:59.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.257+0000 7f05827fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560263c5c170 0x560263c65710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52828/0 (socket says 172.21.15.12:52828) 2024-05-17T22:54:59.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.257+0000 7f05827fc700 1 -- 172.21.15.12:0/4226260994 learned_addr learned my addr 172.21.15.12:0/4226260994 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:54:59.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.257+0000 7f05837fe700 1 --2- 172.21.15.12:0/4226260994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560263c61aa0 0x560263c65c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:59.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.257+0000 7f0582ffd700 1 --2- 172.21.15.12:0/4226260994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560263add9f0 0x560263c651d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:59.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.257+0000 7f05827fc700 1 -- 172.21.15.12:0/4226260994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560263add9f0 msgr2=0x560263c651d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:59.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.257+0000 7f05827fc700 1 --2- 172.21.15.12:0/4226260994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560263add9f0 0x560263c651d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:59.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.257+0000 7f05827fc700 1 -- 172.21.15.12:0/4226260994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560263c61aa0 msgr2=0x560263c65c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:59.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.257+0000 7f05827fc700 1 --2- 172.21.15.12:0/4226260994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560263c61aa0 0x560263c65c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:59.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.257+0000 7f05827fc700 1 -- 172.21.15.12:0/4226260994 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560263aca480 con 0x560263c5c170 2024-05-17T22:54:59.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.258+0000 7f05827fc700 1 --2- 172.21.15.12:0/4226260994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560263c5c170 0x560263c65710 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f0578000c00 tx=0x7f05780a9190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f77f3185a11f7118 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:59.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.258+0000 7f0566ff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:54:59.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.258+0000 7f0566ff5700 1 -- 172.21.15.12:0/4226260994 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0578020040 con 0x560263c5c170 2024-05-17T22:54:59.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.258+0000 7f0566ff5700 1 -- 172.21.15.12:0/4226260994 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f05780a9830 con 0x560263c5c170 2024-05-17T22:54:59.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.258+0000 7f0597f20b80 1 -- 172.21.15.12:0/4226260994 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f057c01e440 con 0x560263c5c170 2024-05-17T22:54:59.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.258+0000 7f0566ff5700 1 -- 172.21.15.12:0/4226260994 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f05780e2160 con 0x560263c5c170 2024-05-17T22:54:59.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.258+0000 7f0597f20b80 10 client.16641 fetch_fsmap learned FSMap version 46 2024-05-17T22:54:59.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.258+0000 7f0597f20b80 1 -- 172.21.15.12:0/4226260994 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x560263c72de0 con 0x560263c5c170 2024-05-17T22:54:59.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.258+0000 7f0566ff5700 1 -- 172.21.15.12:0/4226260994 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f057c01e440 con 0x560263c5c170 2024-05-17T22:54:59.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.258+0000 7f0566ff5700 1 -- 172.21.15.12:0/4226260994 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7f05780b2030 con 0x560263c5c170 2024-05-17T22:54:59.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.258+0000 7f0597f20b80 10 client.16641 fetch_fsmap finished waiting for FSMap version 46 2024-05-17T22:54:59.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.258+0000 7f0597f20b80 10 client.16641 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-17T22:54:59.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.258+0000 7f0597f20b80 20 client.16641 populate_metadata read hostname 'smithi012' 2024-05-17T22:54:59.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.258+0000 7f0597f20b80 1 --2- 172.21.15.12:0/4226260994 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x560263c75410 0x560263c77890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:59.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.259+0000 7f0597f20b80 4 client.16641 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:54:59.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.259+0000 7f0597f20b80 1 -- 172.21.15.12:0/4226260994 --> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560263b8c1e0 con 0x560263c75410 2024-05-17T22:54:59.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.259+0000 7f0582ffd700 1 --2- 172.21.15.12:0/4226260994 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x560263c75410 0x560263c77890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:59.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.259+0000 7f0566ff5700 10 client.16641 ms_handle_connect on v2:172.21.15.151:6838/2338353849 2024-05-17T22:54:59.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.259+0000 7f0582ffd700 1 --2- 172.21.15.12:0/4226260994 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x560263c75410 0x560263c77890 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d94ba9b007b78c41 server_cookie=6fc3b4f2742c207c in_seq=0 out_seq=0 2024-05-17T22:54:59.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.260+0000 7f0566ff5700 1 -- 172.21.15.12:0/4226260994 <== mds.2 v2:172.21.15.151:6838/2338353849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560263b8c1e0 con 0x560263c75410 2024-05-17T22:54:59.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.260+0000 7f0566ff5700 10 client.16641 handle_command_reply: tid=0 2024-05-17T22:54:59.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.260+0000 7f0597f20b80 1 client.16641 shutdown 2024-05-17T22:54:59.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.260+0000 7f0597f20b80 1 -- 172.21.15.12:0/4226260994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560263c5c170 msgr2=0x560263c65710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:59.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.260+0000 7f0597f20b80 1 --2- 172.21.15.12:0/4226260994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560263c5c170 0x560263c65710 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f0578000c00 tx=0x7f05780a9190 comp rx=0 tx=0).stop 2024-05-17T22:54:59.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.260+0000 7f0597f20b80 1 -- 172.21.15.12:0/4226260994 shutdown_connections 2024-05-17T22:54:59.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.260+0000 7f0597f20b80 1 --2- 172.21.15.12:0/4226260994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560263c61aa0 0x560263c65c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:59.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.260+0000 7f0597f20b80 1 --2- 172.21.15.12:0/4226260994 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x560263c75410 0x560263c77890 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:59.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.260+0000 7f0566ff5700 0 client.16641 ms_handle_reset on v2:172.21.15.151:6838/2338353849 2024-05-17T22:54:59.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.260+0000 7f0597f20b80 1 --2- 172.21.15.12:0/4226260994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560263add9f0 0x560263c651d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:59.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.260+0000 7f0597f20b80 1 --2- 172.21.15.12:0/4226260994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560263c5c170 0x560263c65710 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:59.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.260+0000 7f0597f20b80 1 -- 172.21.15.12:0/4226260994 >> 172.21.15.12:0/4226260994 conn(0x560263b21f00 msgr2=0x560263c50ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:59.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.260+0000 7f0597f20b80 1 -- 172.21.15.12:0/4226260994 shutdown_connections 2024-05-17T22:54:59.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.260+0000 7f0597f20b80 1 -- 172.21.15.12:0/4226260994 wait complete. 2024-05-17T22:54:59.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.260+0000 7f0597f20b80 20 client.16641 trim_cache size 0 max 16384 2024-05-17T22:54:59.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.276+0000 7f0597f20b80 1 Processor -- start 2024-05-17T22:54:59.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.276+0000 7f0597f20b80 1 -- start start 2024-05-17T22:54:59.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.277+0000 7f0597f20b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560263add9f0 0x560263c5bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:59.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.277+0000 7f0597f20b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560263c5c170 0x560263c5c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:59.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.277+0000 7f0597f20b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560263c61aa0 0x560263c71190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:59.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.277+0000 7f0597f20b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f05841b2670 con 0x560263add9f0 2024-05-17T22:54:59.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.277+0000 7f0597f20b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f05841afe70 con 0x560263c61aa0 2024-05-17T22:54:59.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.277+0000 7f0597f20b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f05841adcc0 con 0x560263c5c170 2024-05-17T22:54:59.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.277+0000 7f0582ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560263add9f0 0x560263c5bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:59.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.277+0000 7f0582ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560263add9f0 0x560263c5bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52832/0 (socket says 172.21.15.12:52832) 2024-05-17T22:54:59.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.277+0000 7f0582ffd700 1 -- 172.21.15.12:0/3784798486 learned_addr learned my addr 172.21.15.12:0/3784798486 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:54:59.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.277+0000 7f05837fe700 1 --2- 172.21.15.12:0/3784798486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560263c61aa0 0x560263c71190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:59.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.277+0000 7f05827fc700 1 --2- 172.21.15.12:0/3784798486 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560263c5c170 0x560263c5c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:59.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.277+0000 7f0582ffd700 1 -- 172.21.15.12:0/3784798486 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560263c5c170 msgr2=0x560263c5c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:59.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.277+0000 7f0582ffd700 1 --2- 172.21.15.12:0/3784798486 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560263c5c170 0x560263c5c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:59.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.277+0000 7f0582ffd700 1 -- 172.21.15.12:0/3784798486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560263c61aa0 msgr2=0x560263c71190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:59.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.277+0000 7f0582ffd700 1 --2- 172.21.15.12:0/3784798486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560263c61aa0 0x560263c71190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:59.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.277+0000 7f0582ffd700 1 -- 172.21.15.12:0/3784798486 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560263c56ad0 con 0x560263add9f0 2024-05-17T22:54:59.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.277+0000 7f05827fc700 1 --2- 172.21.15.12:0/3784798486 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560263c5c170 0x560263c5c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:54:59.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.277+0000 7f0582ffd700 1 --2- 172.21.15.12:0/3784798486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560263add9f0 0x560263c5bc30 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f0574009f90 tx=0x7f0574008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f27666e9ace706d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:59.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.278+0000 7f0568ff9700 1 -- 172.21.15.12:0/3784798486 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0574017a30 con 0x560263add9f0 2024-05-17T22:54:59.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.278+0000 7f0568ff9700 1 -- 172.21.15.12:0/3784798486 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0574021030 con 0x560263add9f0 2024-05-17T22:54:59.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.278+0000 7f0568ff9700 1 -- 172.21.15.12:0/3784798486 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05740253d0 con 0x560263add9f0 2024-05-17T22:54:59.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.278+0000 7f0597f20b80 1 -- 172.21.15.12:0/3784798486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560263add9f0 msgr2=0x560263c5bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:59.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.278+0000 7f0597f20b80 1 --2- 172.21.15.12:0/3784798486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560263add9f0 0x560263c5bc30 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f0574009f90 tx=0x7f0574008040 comp rx=0 tx=0).stop 2024-05-17T22:54:59.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.279+0000 7f0597f20b80 1 -- 172.21.15.12:0/3784798486 shutdown_connections 2024-05-17T22:54:59.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.279+0000 7f0597f20b80 1 --2- 172.21.15.12:0/3784798486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560263c61aa0 0x560263c71190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:59.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.279+0000 7f0597f20b80 1 --2- 172.21.15.12:0/3784798486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560263add9f0 0x560263c5bc30 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:59.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.279+0000 7f0597f20b80 1 --2- 172.21.15.12:0/3784798486 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560263c5c170 0x560263c5c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:59.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.279+0000 7f0597f20b80 1 -- 172.21.15.12:0/3784798486 >> 172.21.15.12:0/3784798486 conn(0x560263b21f00 msgr2=0x560263c58050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:59.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.279+0000 7f0597f20b80 1 -- 172.21.15.12:0/3784798486 shutdown_connections 2024-05-17T22:54:59.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.279+0000 7f0597f20b80 1 -- 172.21.15.12:0/3784798486 wait complete. 2024-05-17T22:54:59.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.279+0000 7f0597f20b80 1 -- start start 2024-05-17T22:54:59.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.279+0000 7f0597f20b80 1 Processor -- start 2024-05-17T22:54:59.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.279+0000 7f0597f20b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560263add9f0 0x560263c6a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:59.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.279+0000 7f0597f20b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560263c5c170 0x560263c6aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:59.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.279+0000 7f0597f20b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560263c61aa0 0x560263c6af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:59.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.279+0000 7f0597f20b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f05841ae330 con 0x560263c5c170 2024-05-17T22:54:59.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.279+0000 7f0597f20b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f05841af7a0 con 0x560263c61aa0 2024-05-17T22:54:59.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.279+0000 7f0597f20b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f05841aea00 con 0x560263add9f0 2024-05-17T22:54:59.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.279+0000 7f0597f20b80 10 client.0 _ll_register_callbacks cb 0x560263b14f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:54:59.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.280+0000 7f05837fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560263c61aa0 0x560263c6af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:59.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.280+0000 7f05827fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560263c5c170 0x560263c6aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:59.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.280+0000 7f0582ffd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560263add9f0 0x560263c6a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:59.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.280+0000 7f05837fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560263c61aa0 0x560263c6af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.12:59992/0 (socket says 172.21.15.12:59992) 2024-05-17T22:54:59.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.280+0000 7f05837fe700 1 -- 172.21.15.12:0/2902060452 learned_addr learned my addr 172.21.15.12:0/2902060452 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:54:59.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.280+0000 7f05827fc700 1 -- 172.21.15.12:0/2902060452 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560263add9f0 msgr2=0x560263c6a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:59.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.280+0000 7f05827fc700 1 --2- 172.21.15.12:0/2902060452 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560263add9f0 0x560263c6a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:59.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.280+0000 7f05827fc700 1 -- 172.21.15.12:0/2902060452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560263c61aa0 msgr2=0x560263c6af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:54:59.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.280+0000 7f05827fc700 1 --2- 172.21.15.12:0/2902060452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560263c61aa0 0x560263c6af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:59.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.280+0000 7f05827fc700 1 -- 172.21.15.12:0/2902060452 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560263cddc70 con 0x560263c5c170 2024-05-17T22:54:59.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.280+0000 7f05827fc700 1 --2- 172.21.15.12:0/2902060452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560263c5c170 0x560263c6aa20 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x560263b18350 tx=0x7f057808fc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f02266ad0489fdc server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:54:59.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.280+0000 7f0566ff5700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:54:59.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.280+0000 7f0566ff5700 1 -- 172.21.15.12:0/2902060452 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0578020030 con 0x560263c5c170 2024-05-17T22:54:59.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.280+0000 7f0566ff5700 1 -- 172.21.15.12:0/2902060452 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f05780b1fa0 con 0x560263c5c170 2024-05-17T22:54:59.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.280+0000 7f0597f20b80 1 -- 172.21.15.12:0/2902060452 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0574021030 con 0x560263c5c170 2024-05-17T22:54:59.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.280+0000 7f0566ff5700 1 -- 172.21.15.12:0/2902060452 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f057808c050 con 0x560263c5c170 2024-05-17T22:54:59.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.281+0000 7f0597f20b80 10 client.16647 fetch_fsmap learned FSMap version 46 2024-05-17T22:54:59.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.281+0000 7f0597f20b80 1 -- 172.21.15.12:0/2902060452 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x560263cdee20 con 0x560263c5c170 2024-05-17T22:54:59.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.281+0000 7f0566ff5700 1 -- 172.21.15.12:0/2902060452 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0574021030 con 0x560263c5c170 2024-05-17T22:54:59.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.281+0000 7f0566ff5700 1 -- 172.21.15.12:0/2902060452 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7f0578083ce0 con 0x560263c5c170 2024-05-17T22:54:59.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.281+0000 7f0597f20b80 10 client.16647 fetch_fsmap finished waiting for FSMap version 46 2024-05-17T22:54:59.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.281+0000 7f0597f20b80 10 client.16647 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-17T22:54:59.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.281+0000 7f0597f20b80 20 client.16647 populate_metadata read hostname 'smithi012' 2024-05-17T22:54:59.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.281+0000 7f0597f20b80 1 --2- 172.21.15.12:0/2902060452 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x560263c75410 0x560263ce1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:54:59.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.281+0000 7f0597f20b80 4 client.16647 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:54:59.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.281+0000 7f0597f20b80 1 -- 172.21.15.12:0/2902060452 --> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560263b859c0 con 0x560263c75410 2024-05-17T22:54:59.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.281+0000 7f0582ffd700 1 --2- 172.21.15.12:0/2902060452 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x560263c75410 0x560263ce1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:54:59.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.282+0000 7f0582ffd700 1 --2- 172.21.15.12:0/2902060452 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x560263c75410 0x560263ce1420 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fd7cf005b4f449e3 server_cookie=ca0f281829e93eb0 in_seq=0 out_seq=0 2024-05-17T22:54:59.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.282+0000 7f0566ff5700 10 client.16647 ms_handle_connect on v2:172.21.15.151:6838/2338353849 2024-05-17T22:54:59.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.282+0000 7f0566ff5700 1 -- 172.21.15.12:0/2902060452 <== mds.2 v2:172.21.15.151:6838/2338353849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560263b859c0 con 0x560263c75410 2024-05-17T22:54:59.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.282+0000 7f0566ff5700 10 client.16647 handle_command_reply: tid=0 2024-05-17T22:54:59.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.282+0000 7f0597f20b80 1 client.16647 shutdown 2024-05-17T22:54:59.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.282+0000 7f0597f20b80 1 -- 172.21.15.12:0/2902060452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560263c5c170 msgr2=0x560263c6aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:59.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.282+0000 7f0597f20b80 1 --2- 172.21.15.12:0/2902060452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560263c5c170 0x560263c6aa20 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x560263b18350 tx=0x7f057808fc40 comp rx=0 tx=0).stop 2024-05-17T22:54:59.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.282+0000 7f0597f20b80 1 -- 172.21.15.12:0/2902060452 shutdown_connections 2024-05-17T22:54:59.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.282+0000 7f0597f20b80 1 --2- 172.21.15.12:0/2902060452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560263c61aa0 0x560263c6af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:59.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.282+0000 7f0597f20b80 1 --2- 172.21.15.12:0/2902060452 >> [v2:172.21.15.151:6838/2338353849,v1:172.21.15.151:6839/2338353849] conn(0x560263c75410 0x560263ce1420 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:59.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.282+0000 7f0597f20b80 1 --2- 172.21.15.12:0/2902060452 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560263add9f0 0x560263c6a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:59.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.282+0000 7f0597f20b80 1 --2- 172.21.15.12:0/2902060452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560263c5c170 0x560263c6aa20 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:59.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.282+0000 7f0566ff5700 0 client.16647 ms_handle_reset on v2:172.21.15.151:6838/2338353849 2024-05-17T22:54:59.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.282+0000 7f0597f20b80 1 -- 172.21.15.12:0/2902060452 >> 172.21.15.12:0/2902060452 conn(0x560263b21f00 msgr2=0x560263c669c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:59.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.282+0000 7f0597f20b80 1 -- 172.21.15.12:0/2902060452 shutdown_connections 2024-05-17T22:54:59.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.282+0000 7f0597f20b80 1 -- 172.21.15.12:0/2902060452 wait complete. 2024-05-17T22:54:59.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.282+0000 7f0597f20b80 20 client.16647 trim_cache size 0 max 16384 2024-05-17T22:54:59.291 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:54:59.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.283+0000 7f0589545700 1 -- 172.21.15.12:0/3102908641 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f056c0b2900 msgr2=0x7f056c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:59.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.283+0000 7f0589545700 1 --2- 172.21.15.12:0/3102908641 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f056c0b2900 0x7f056c0b4e10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f057c00b070 tx=0x7f057c009540 comp rx=0 tx=0).stop 2024-05-17T22:54:59.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.283+0000 7f0589545700 1 -- 172.21.15.12:0/3102908641 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0584107750 msgr2=0x7f058419bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:54:59.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.283+0000 7f0589545700 1 --2- 172.21.15.12:0/3102908641 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0584107750 0x7f058419bb40 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f057800d980 tx=0x7f057800dc90 comp rx=0 tx=0).stop 2024-05-17T22:54:59.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.283+0000 7f0589545700 1 -- 172.21.15.12:0/3102908641 shutdown_connections 2024-05-17T22:54:59.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.283+0000 7f0589545700 1 --2- 172.21.15.12:0/3102908641 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f056c0b2900 0x7f056c0b4e10 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:59.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.283+0000 7f0589545700 1 --2- 172.21.15.12:0/3102908641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05841080a0 0x7f058419c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:59.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.283+0000 7f0589545700 1 --2- 172.21.15.12:0/3102908641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0584102e70 0x7f058419b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:59.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.283+0000 7f0589545700 1 --2- 172.21.15.12:0/3102908641 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0584107750 0x7f058419bb40 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:54:59.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.283+0000 7f0589545700 1 -- 172.21.15.12:0/3102908641 >> 172.21.15.12:0/3102908641 conn(0x7f0584079ae0 msgr2=0x7f0584105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:54:59.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.283+0000 7f0589545700 1 -- 172.21.15.12:0/3102908641 shutdown_connections 2024-05-17T22:54:59.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:54:59.283+0000 7f0589545700 1 -- 172.21.15.12:0/3102908641 wait complete. 2024-05-17T22:55:00.072 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-17T22:55:00.327 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:55:00.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:00 smithi012 ceph-mon[31632]: pgmap v911: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 40 KiB/s wr, 10 op/s 2024-05-17T22:55:00.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:00 smithi151 ceph-mon[36101]: pgmap v911: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 40 KiB/s wr, 10 op/s 2024-05-17T22:55:00.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:00 smithi096 ceph-mon[36065]: pgmap v911: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 40 KiB/s wr, 10 op/s 2024-05-17T22:55:02.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.236+0000 7f49bca3e700 1 -- 172.21.15.12:0/2723365123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49b8104e10 msgr2=0x7f49b8105220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:02.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.236+0000 7f49bca3e700 1 --2- 172.21.15.12:0/2723365123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49b8104e10 0x7f49b8105220 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f49a8009990 tx=0x7f49a8009ca0 comp rx=0 tx=0).stop 2024-05-17T22:55:02.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.237+0000 7f49bca3e700 1 -- 172.21.15.12:0/2723365123 shutdown_connections 2024-05-17T22:55:02.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.237+0000 7f49bca3e700 1 --2- 172.21.15.12:0/2723365123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49b810e240 0x7f49b8110750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:02.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.237+0000 7f49bca3e700 1 --2- 172.21.15.12:0/2723365123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49b8104e10 0x7f49b8105220 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:02.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.237+0000 7f49bca3e700 1 --2- 172.21.15.12:0/2723365123 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49b8105760 0x7f49b810dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:02.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.237+0000 7f49bca3e700 1 -- 172.21.15.12:0/2723365123 >> 172.21.15.12:0/2723365123 conn(0x7f49b8079ae0 msgr2=0x7f49b8079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:55:02.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.237+0000 7f49bca3e700 1 -- 172.21.15.12:0/2723365123 shutdown_connections 2024-05-17T22:55:02.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.237+0000 7f49bca3e700 1 -- 172.21.15.12:0/2723365123 wait complete. 2024-05-17T22:55:02.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.238+0000 7f49bca3e700 1 Processor -- start 2024-05-17T22:55:02.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.238+0000 7f49bca3e700 1 -- start start 2024-05-17T22:55:02.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.238+0000 7f49bca3e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49b8104e10 0x7f49b81a04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:02.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.238+0000 7f49bca3e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49b8105760 0x7f49b81a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:02.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.238+0000 7f49bca3e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49b810e240 0x7f49b819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:02.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.238+0000 7f49bca3e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f49b8113e40 con 0x7f49b810e240 2024-05-17T22:55:02.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.238+0000 7f49bca3e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f49b8113cc0 con 0x7f49b8105760 2024-05-17T22:55:02.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.238+0000 7f49bca3e700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f49b8113fc0 con 0x7f49b8104e10 2024-05-17T22:55:02.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.238+0000 7f49b6d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49b810e240 0x7f49b819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:02.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.238+0000 7f49b6d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49b810e240 0x7f49b819a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47418/0 (socket says 172.21.15.12:47418) 2024-05-17T22:55:02.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.238+0000 7f49b5d9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49b8105760 0x7f49b81a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:02.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.238+0000 7f49b6d9d700 1 -- 172.21.15.12:0/1948782899 learned_addr learned my addr 172.21.15.12:0/1948782899 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:55:02.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.238+0000 7f49b659c700 1 --2- 172.21.15.12:0/1948782899 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49b8104e10 0x7f49b81a04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:02.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.239+0000 7f49b6d9d700 1 -- 172.21.15.12:0/1948782899 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49b8104e10 msgr2=0x7f49b81a04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:02.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.239+0000 7f49b6d9d700 1 --2- 172.21.15.12:0/1948782899 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49b8104e10 0x7f49b81a04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:02.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.239+0000 7f49b6d9d700 1 -- 172.21.15.12:0/1948782899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49b8105760 msgr2=0x7f49b81a0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:02.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.239+0000 7f49b6d9d700 1 --2- 172.21.15.12:0/1948782899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49b8105760 0x7f49b81a0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:02.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.239+0000 7f49b6d9d700 1 -- 172.21.15.12:0/1948782899 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49b819ad80 con 0x7f49b810e240 2024-05-17T22:55:02.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.239+0000 7f49b5d9b700 1 --2- 172.21.15.12:0/1948782899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49b8105760 0x7f49b81a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:55:02.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.239+0000 7f49b6d9d700 1 --2- 172.21.15.12:0/1948782899 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49b810e240 0x7f49b819a650 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f49b0007ee0 tx=0x7f49b000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:55:02.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.239+0000 7f49a77fe700 1 -- 172.21.15.12:0/1948782899 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b0004bb0 con 0x7f49b810e240 2024-05-17T22:55:02.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.239+0000 7f49bca3e700 1 -- 172.21.15.12:0/1948782899 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49b819b070 con 0x7f49b810e240 2024-05-17T22:55:02.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.239+0000 7f49a77fe700 1 -- 172.21.15.12:0/1948782899 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49b0004d50 con 0x7f49b810e240 2024-05-17T22:55:02.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.239+0000 7f49a77fe700 1 -- 172.21.15.12:0/1948782899 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b0005920 con 0x7f49b810e240 2024-05-17T22:55:02.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.239+0000 7f49bca3e700 1 -- 172.21.15.12:0/1948782899 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49b81a7270 con 0x7f49b810e240 2024-05-17T22:55:02.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.242+0000 7f49a77fe700 1 -- 172.21.15.12:0/1948782899 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f49b001a020 con 0x7f49b810e240 2024-05-17T22:55:02.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.242+0000 7f49a77fe700 1 --2- 172.21.15.12:0/1948782899 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f49a00b2880 0x7f49a00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:02.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.243+0000 7f49b659c700 1 --2- 172.21.15.12:0/1948782899 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f49a00b2880 0x7f49a00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:02.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.243+0000 7f49a77fe700 1 -- 172.21.15.12:0/1948782899 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f49b00e1c80 con 0x7f49b810e240 2024-05-17T22:55:02.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.243+0000 7f49cb419b80 1 Processor -- start 2024-05-17T22:55:02.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.243+0000 7f49cb419b80 1 -- start start 2024-05-17T22:55:02.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.243+0000 7f49b659c700 1 --2- 172.21.15.12:0/1948782899 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f49a00b2880 0x7f49a00b4d90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f49a800b5c0 tx=0x7f49a8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:55:02.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.243+0000 7f49cb419b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590b19a29f0 0x5590b1b26560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:02.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.243+0000 7f49cb419b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b1b26aa0 0x5590b1b20c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:02.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.243+0000 7f49cb419b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590b1b21170 0x5590b1b21580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:02.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.243+0000 7f49cb419b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5590b1a4f290 con 0x5590b1b21170 2024-05-17T22:55:02.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.243+0000 7f49cb419b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5590b1a3efd0 con 0x5590b19a29f0 2024-05-17T22:55:02.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.243+0000 7f49cb419b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5590b1a42e70 con 0x5590b1b26aa0 2024-05-17T22:55:02.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.244+0000 7f49b6d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590b1b21170 0x5590b1b21580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:02.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.244+0000 7f49b6d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590b1b21170 0x5590b1b21580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47420/0 (socket says 172.21.15.12:47420) 2024-05-17T22:55:02.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.244+0000 7f49b6d9d700 1 -- 172.21.15.12:0/2917748246 learned_addr learned my addr 172.21.15.12:0/2917748246 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:55:02.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.244+0000 7f49b659c700 1 --2- 172.21.15.12:0/2917748246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590b19a29f0 0x5590b1b26560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:02.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.244+0000 7f49b5d9b700 1 --2- 172.21.15.12:0/2917748246 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b1b26aa0 0x5590b1b20c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:02.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.244+0000 7f49b6d9d700 1 -- 172.21.15.12:0/2917748246 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b1b26aa0 msgr2=0x5590b1b20c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:55:02.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.244+0000 7f49b6d9d700 1 --2- 172.21.15.12:0/2917748246 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b1b26aa0 0x5590b1b20c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:02.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.244+0000 7f49b6d9d700 1 -- 172.21.15.12:0/2917748246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590b19a29f0 msgr2=0x5590b1b26560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:55:02.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.244+0000 7f49b6d9d700 1 --2- 172.21.15.12:0/2917748246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590b19a29f0 0x5590b1b26560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:02.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.244+0000 7f49b6d9d700 1 -- 172.21.15.12:0/2917748246 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590b19e8590 con 0x5590b1b21170 2024-05-17T22:55:02.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.244+0000 7f49b5d9b700 1 --2- 172.21.15.12:0/2917748246 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b1b26aa0 0x5590b1b20c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:55:02.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.244+0000 7f49b6d9d700 1 --2- 172.21.15.12:0/2917748246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590b1b21170 0x5590b1b21580 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f49b0009510 tx=0x7f49b00a88e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85a59f4fe268101c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:55:02.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.244+0000 7f49a4ff9700 1 -- 172.21.15.12:0/2917748246 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b0005020 con 0x5590b1b21170 2024-05-17T22:55:02.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.244+0000 7f49a4ff9700 1 -- 172.21.15.12:0/2917748246 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49b00051c0 con 0x5590b1b21170 2024-05-17T22:55:02.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.244+0000 7f49a4ff9700 1 -- 172.21.15.12:0/2917748246 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b00a4c80 con 0x5590b1b21170 2024-05-17T22:55:02.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.245+0000 7f49cb419b80 1 -- 172.21.15.12:0/2917748246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590b1b21170 msgr2=0x5590b1b21580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:02.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.245+0000 7f49cb419b80 1 --2- 172.21.15.12:0/2917748246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590b1b21170 0x5590b1b21580 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f49b0009510 tx=0x7f49b00a88e0 comp rx=0 tx=0).stop 2024-05-17T22:55:02.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.245+0000 7f49cb419b80 1 -- 172.21.15.12:0/2917748246 shutdown_connections 2024-05-17T22:55:02.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.245+0000 7f49cb419b80 1 --2- 172.21.15.12:0/2917748246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590b1b21170 0x5590b1b21580 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:02.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.245+0000 7f49cb419b80 1 --2- 172.21.15.12:0/2917748246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590b19a29f0 0x5590b1b26560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:02.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.245+0000 7f49cb419b80 1 --2- 172.21.15.12:0/2917748246 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b1b26aa0 0x5590b1b20c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:02.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.245+0000 7f49cb419b80 1 -- 172.21.15.12:0/2917748246 >> 172.21.15.12:0/2917748246 conn(0x5590b19e6f00 msgr2=0x5590b1b1bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:55:02.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.246+0000 7f49cb419b80 1 -- 172.21.15.12:0/2917748246 shutdown_connections 2024-05-17T22:55:02.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.246+0000 7f49cb419b80 1 -- 172.21.15.12:0/2917748246 wait complete. 2024-05-17T22:55:02.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.246+0000 7f49cb419b80 1 -- start start 2024-05-17T22:55:02.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.246+0000 7f49cb419b80 1 Processor -- start 2024-05-17T22:55:02.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.246+0000 7f49cb419b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b19a29f0 0x5590b1b2a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:02.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.246+0000 7f49cb419b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590b1b21170 0x5590b1b2a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:02.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.246+0000 7f49cb419b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590b1b26aa0 0x5590b1b2acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:02.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.246+0000 7f49cb419b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5590b1946150 con 0x5590b1b21170 2024-05-17T22:55:02.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.246+0000 7f49cb419b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5590b19dfe80 con 0x5590b1b26aa0 2024-05-17T22:55:02.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.246+0000 7f49cb419b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5590b1b2d390 con 0x5590b19a29f0 2024-05-17T22:55:02.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.246+0000 7f49cb419b80 10 client.0 _ll_register_callbacks cb 0x5590b19d9f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:55:02.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.246+0000 7f49b5d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590b1b21170 0x5590b1b2a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:02.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.247+0000 7f49b5d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590b1b21170 0x5590b1b2a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47434/0 (socket says 172.21.15.12:47434) 2024-05-17T22:55:02.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.247+0000 7f49b5d9b700 1 -- 172.21.15.12:0/2606155702 learned_addr learned my addr 172.21.15.12:0/2606155702 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:55:02.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.247+0000 7f49b659c700 1 --2- 172.21.15.12:0/2606155702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b19a29f0 0x5590b1b2a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:02.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.247+0000 7f49b6d9d700 1 --2- 172.21.15.12:0/2606155702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590b1b26aa0 0x5590b1b2acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:02.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.247+0000 7f49b5d9b700 1 -- 172.21.15.12:0/2606155702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b19a29f0 msgr2=0x5590b1b2a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:55:02.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.247+0000 7f49b5d9b700 1 --2- 172.21.15.12:0/2606155702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b19a29f0 0x5590b1b2a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:02.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.247+0000 7f49b5d9b700 1 -- 172.21.15.12:0/2606155702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590b1b26aa0 msgr2=0x5590b1b2acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:55:02.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.247+0000 7f49b5d9b700 1 --2- 172.21.15.12:0/2606155702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590b1b26aa0 0x5590b1b2acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:02.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.247+0000 7f49b5d9b700 1 -- 172.21.15.12:0/2606155702 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590b198f480 con 0x5590b1b21170 2024-05-17T22:55:02.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.247+0000 7f49b5d9b700 1 --2- 172.21.15.12:0/2606155702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590b1b21170 0x5590b1b2a770 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f49ac011620 tx=0x7f49ac011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c4b24b8e184099f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:55:02.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.247+0000 7f499dffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:55:02.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.247+0000 7f499dffb700 1 -- 172.21.15.12:0/2606155702 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac019070 con 0x5590b1b21170 2024-05-17T22:55:02.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.247+0000 7f499dffb700 1 -- 172.21.15.12:0/2606155702 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f49ac015870 con 0x5590b1b21170 2024-05-17T22:55:02.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.247+0000 7f49cb419b80 1 -- 172.21.15.12:0/2606155702 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49b00051c0 con 0x5590b1b21170 2024-05-17T22:55:02.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.248+0000 7f499dffb700 1 -- 172.21.15.12:0/2606155702 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f49ac025050 con 0x5590b1b21170 2024-05-17T22:55:02.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.248+0000 7f49cb419b80 10 client.16659 fetch_fsmap learned FSMap version 46 2024-05-17T22:55:02.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.248+0000 7f49cb419b80 1 -- 172.21.15.12:0/2606155702 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5590b1b37eb0 con 0x5590b1b21170 2024-05-17T22:55:02.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.248+0000 7f499dffb700 1 -- 172.21.15.12:0/2606155702 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49b00051c0 con 0x5590b1b21170 2024-05-17T22:55:02.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.248+0000 7f499dffb700 1 -- 172.21.15.12:0/2606155702 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7f49ac013b80 con 0x5590b1b21170 2024-05-17T22:55:02.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.248+0000 7f49cb419b80 10 client.16659 fetch_fsmap finished waiting for FSMap version 46 2024-05-17T22:55:02.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.248+0000 7f49cb419b80 10 client.16659 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-17T22:55:02.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.248+0000 7f49cb419b80 20 client.16659 populate_metadata read hostname 'smithi012' 2024-05-17T22:55:02.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.248+0000 7f49cb419b80 1 --2- 172.21.15.12:0/2606155702 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5590b1b3a490 0x5590b1b3c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:02.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.248+0000 7f49cb419b80 4 client.16659 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:55:02.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.248+0000 7f49cb419b80 1 -- 172.21.15.12:0/2606155702 --> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590b1a511e0 con 0x5590b1b3a490 2024-05-17T22:55:02.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.248+0000 7f49b659c700 1 --2- 172.21.15.12:0/2606155702 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5590b1b3a490 0x5590b1b3c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:02.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.249+0000 7f499dffb700 10 client.16659 ms_handle_connect on v2:172.21.15.96:6836/4222574338 2024-05-17T22:55:02.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.249+0000 7f49b659c700 1 --2- 172.21.15.12:0/2606155702 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5590b1b3a490 0x5590b1b3c8d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=69c7d2e9d4b228be server_cookie=8189c60c722db3e2 in_seq=0 out_seq=0 2024-05-17T22:55:02.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.250+0000 7f499dffb700 1 -- 172.21.15.12:0/2606155702 <== mds.1 v2:172.21.15.96:6836/4222574338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5590b1a511e0 con 0x5590b1b3a490 2024-05-17T22:55:02.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.250+0000 7f499dffb700 10 client.16659 handle_command_reply: tid=0 2024-05-17T22:55:02.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.250+0000 7f49cb419b80 1 client.16659 shutdown 2024-05-17T22:55:02.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.250+0000 7f49cb419b80 1 -- 172.21.15.12:0/2606155702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590b1b21170 msgr2=0x5590b1b2a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:02.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.250+0000 7f49cb419b80 1 --2- 172.21.15.12:0/2606155702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590b1b21170 0x5590b1b2a770 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f49ac011620 tx=0x7f49ac011930 comp rx=0 tx=0).stop 2024-05-17T22:55:02.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.250+0000 7f49cb419b80 1 -- 172.21.15.12:0/2606155702 shutdown_connections 2024-05-17T22:55:02.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.250+0000 7f49cb419b80 1 --2- 172.21.15.12:0/2606155702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590b1b26aa0 0x5590b1b2acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:02.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.250+0000 7f49cb419b80 1 --2- 172.21.15.12:0/2606155702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b19a29f0 0x5590b1b2a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:02.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.250+0000 7f49cb419b80 1 --2- 172.21.15.12:0/2606155702 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5590b1b3a490 0x5590b1b3c8d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:02.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.250+0000 7f49cb419b80 1 --2- 172.21.15.12:0/2606155702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590b1b21170 0x5590b1b2a770 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:02.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.250+0000 7f49cb419b80 1 -- 172.21.15.12:0/2606155702 >> 172.21.15.12:0/2606155702 conn(0x5590b19e6f00 msgr2=0x5590b1b15ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:55:02.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.250+0000 7f499dffb700 0 client.16659 ms_handle_reset on v2:172.21.15.96:6836/4222574338 2024-05-17T22:55:02.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.250+0000 7f49cb419b80 1 -- 172.21.15.12:0/2606155702 shutdown_connections 2024-05-17T22:55:02.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.250+0000 7f49cb419b80 1 -- 172.21.15.12:0/2606155702 wait complete. 2024-05-17T22:55:02.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.250+0000 7f49cb419b80 20 client.16659 trim_cache size 0 max 16384 2024-05-17T22:55:02.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.266+0000 7f49cb419b80 1 Processor -- start 2024-05-17T22:55:02.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.266+0000 7f49cb419b80 1 -- start start 2024-05-17T22:55:02.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.266+0000 7f49cb419b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590b19a29f0 0x5590b1b20c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:02.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.266+0000 7f49cb419b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b1b21170 0x5590b1b25c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:02.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.266+0000 7f49cb419b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590b1b26aa0 0x5590b1b2acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:02.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.266+0000 7f49cb419b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f49b81b2700 con 0x5590b1b26aa0 2024-05-17T22:55:02.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.266+0000 7f49cb419b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f49b81aff00 con 0x5590b19a29f0 2024-05-17T22:55:02.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.266+0000 7f49cb419b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f49b81add50 con 0x5590b1b21170 2024-05-17T22:55:02.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.266+0000 7f49b6d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590b1b26aa0 0x5590b1b2acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:02.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.266+0000 7f49b6d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590b1b26aa0 0x5590b1b2acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47448/0 (socket says 172.21.15.12:47448) 2024-05-17T22:55:02.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.266+0000 7f49b6d9d700 1 -- 172.21.15.12:0/690756353 learned_addr learned my addr 172.21.15.12:0/690756353 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:55:02.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.266+0000 7f49b659c700 1 --2- 172.21.15.12:0/690756353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590b19a29f0 0x5590b1b20c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:02.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.266+0000 7f49b5d9b700 1 --2- 172.21.15.12:0/690756353 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b1b21170 0x5590b1b25c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:02.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.266+0000 7f49b6d9d700 1 -- 172.21.15.12:0/690756353 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b1b21170 msgr2=0x5590b1b25c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:55:02.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.266+0000 7f49b6d9d700 1 --2- 172.21.15.12:0/690756353 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b1b21170 0x5590b1b25c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:02.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.266+0000 7f49b6d9d700 1 -- 172.21.15.12:0/690756353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590b19a29f0 msgr2=0x5590b1b20c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:55:02.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.266+0000 7f49b6d9d700 1 --2- 172.21.15.12:0/690756353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590b19a29f0 0x5590b1b20c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:02.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.266+0000 7f49b6d9d700 1 -- 172.21.15.12:0/690756353 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590b1b28780 con 0x5590b1b26aa0 2024-05-17T22:55:02.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.267+0000 7f49b6d9d700 1 --2- 172.21.15.12:0/690756353 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590b1b26aa0 0x5590b1b2acf0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f49b000d700 tx=0x7f49b00a8fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=604689e1ecf3eaae server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:55:02.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.267+0000 7f499ffff700 1 -- 172.21.15.12:0/690756353 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b00c8070 con 0x5590b1b26aa0 2024-05-17T22:55:02.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.267+0000 7f499ffff700 1 -- 172.21.15.12:0/690756353 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49b00b1ae0 con 0x5590b1b26aa0 2024-05-17T22:55:02.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.267+0000 7f499ffff700 1 -- 172.21.15.12:0/690756353 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b006f070 con 0x5590b1b26aa0 2024-05-17T22:55:02.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.267+0000 7f49cb419b80 1 -- 172.21.15.12:0/690756353 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590b1b26aa0 msgr2=0x5590b1b2acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:02.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.267+0000 7f49cb419b80 1 --2- 172.21.15.12:0/690756353 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590b1b26aa0 0x5590b1b2acf0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f49b000d700 tx=0x7f49b00a8fd0 comp rx=0 tx=0).stop 2024-05-17T22:55:02.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.267+0000 7f49cb419b80 1 -- 172.21.15.12:0/690756353 shutdown_connections 2024-05-17T22:55:02.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.267+0000 7f49cb419b80 1 --2- 172.21.15.12:0/690756353 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590b1b26aa0 0x5590b1b2acf0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:02.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.267+0000 7f49cb419b80 1 --2- 172.21.15.12:0/690756353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590b19a29f0 0x5590b1b20c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:02.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.267+0000 7f49cb419b80 1 --2- 172.21.15.12:0/690756353 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b1b21170 0x5590b1b25c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:02.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.268+0000 7f49cb419b80 1 -- 172.21.15.12:0/690756353 >> 172.21.15.12:0/690756353 conn(0x5590b19e6f00 msgr2=0x5590b1b29170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:55:02.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.268+0000 7f49cb419b80 1 -- 172.21.15.12:0/690756353 shutdown_connections 2024-05-17T22:55:02.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.268+0000 7f49cb419b80 1 -- 172.21.15.12:0/690756353 wait complete. 2024-05-17T22:55:02.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.268+0000 7f49cb419b80 1 -- start start 2024-05-17T22:55:02.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.268+0000 7f49cb419b80 1 Processor -- start 2024-05-17T22:55:02.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.268+0000 7f49cb419b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b19a29f0 0x5590b1b32c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:02.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.268+0000 7f49cb419b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590b1b21170 0x5590b1b2d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:02.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.268+0000 7f49cb419b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590b1b26aa0 0x5590b1b2dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:02.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.268+0000 7f49cb419b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f49b81ae3c0 con 0x5590b1b26aa0 2024-05-17T22:55:02.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.268+0000 7f49cb419b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f49b81af830 con 0x5590b1b21170 2024-05-17T22:55:02.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.268+0000 7f49cb419b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f49b81aea90 con 0x5590b19a29f0 2024-05-17T22:55:02.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.268+0000 7f49cb419b80 10 client.0 _ll_register_callbacks cb 0x5590b19d9f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:55:02.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.269+0000 7f49b6d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590b1b26aa0 0x5590b1b2dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:02.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.269+0000 7f49b5d9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590b1b21170 0x5590b1b2d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:02.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.269+0000 7f49b6d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590b1b26aa0 0x5590b1b2dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47450/0 (socket says 172.21.15.12:47450) 2024-05-17T22:55:02.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.269+0000 7f49b659c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b19a29f0 0x5590b1b32c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:02.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.269+0000 7f49b6d9d700 1 -- 172.21.15.12:0/4056176778 learned_addr learned my addr 172.21.15.12:0/4056176778 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:55:02.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.269+0000 7f49b659c700 1 -- 172.21.15.12:0/4056176778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590b1b21170 msgr2=0x5590b1b2d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:55:02.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.269+0000 7f49b659c700 1 --2- 172.21.15.12:0/4056176778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590b1b21170 0x5590b1b2d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:02.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.269+0000 7f49b659c700 1 -- 172.21.15.12:0/4056176778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590b1b26aa0 msgr2=0x5590b1b2dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:55:02.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.269+0000 7f49b659c700 1 --2- 172.21.15.12:0/4056176778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590b1b26aa0 0x5590b1b2dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:02.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.269+0000 7f49b659c700 1 -- 172.21.15.12:0/4056176778 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590b1ba2c30 con 0x5590b19a29f0 2024-05-17T22:55:02.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.269+0000 7f49b6d9d700 1 --2- 172.21.15.12:0/4056176778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590b1b26aa0 0x5590b1b2dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:55:02.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.269+0000 7f49b659c700 1 --2- 172.21.15.12:0/4056176778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b19a29f0 0x5590b1b32c20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f49a80059d0 tx=0x7f49a8009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37991b9eeb65154a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:55:02.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.269+0000 7f499dffb700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-17T22:55:02.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.269+0000 7f499dffb700 1 -- 172.21.15.12:0/4056176778 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a8039070 con 0x5590b19a29f0 2024-05-17T22:55:02.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.270+0000 7f499dffb700 1 -- 172.21.15.12:0/4056176778 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f49a800bbe0 con 0x5590b19a29f0 2024-05-17T22:55:02.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.270+0000 7f49cb419b80 1 -- 172.21.15.12:0/4056176778 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49b00b1ae0 con 0x5590b19a29f0 2024-05-17T22:55:02.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.270+0000 7f499dffb700 1 -- 172.21.15.12:0/4056176778 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f49a8042050 con 0x5590b19a29f0 2024-05-17T22:55:02.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.270+0000 7f49cb419b80 10 client.26305 fetch_fsmap learned FSMap version 46 2024-05-17T22:55:02.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.270+0000 7f49cb419b80 1 -- 172.21.15.12:0/4056176778 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5590b1ba3d70 con 0x5590b19a29f0 2024-05-17T22:55:02.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.270+0000 7f499dffb700 1 -- 172.21.15.12:0/4056176778 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49b00b1ae0 con 0x5590b19a29f0 2024-05-17T22:55:02.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.270+0000 7f499dffb700 1 -- 172.21.15.12:0/4056176778 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7f49a803c020 con 0x5590b19a29f0 2024-05-17T22:55:02.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.270+0000 7f49cb419b80 10 client.26305 fetch_fsmap finished waiting for FSMap version 46 2024-05-17T22:55:02.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.270+0000 7f49cb419b80 10 client.26305 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-17T22:55:02.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.270+0000 7f49cb419b80 20 client.26305 populate_metadata read hostname 'smithi012' 2024-05-17T22:55:02.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.270+0000 7f49cb419b80 1 --2- 172.21.15.12:0/4056176778 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5590b1ba4460 0x5590b1ba6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:02.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.270+0000 7f49cb419b80 4 client.26305 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:55:02.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.270+0000 7f49cb419b80 1 -- 172.21.15.12:0/4056176778 --> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5590b1a4a9c0 con 0x5590b1ba4460 2024-05-17T22:55:02.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.271+0000 7f49b5d9b700 1 --2- 172.21.15.12:0/4056176778 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5590b1ba4460 0x5590b1ba6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:02.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.271+0000 7f49b5d9b700 1 --2- 172.21.15.12:0/4056176778 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5590b1ba4460 0x5590b1ba6880 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=998abfe7e2519a1a server_cookie=25d10394e340b083 in_seq=0 out_seq=0 2024-05-17T22:55:02.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.271+0000 7f499dffb700 10 client.26305 ms_handle_connect on v2:172.21.15.96:6836/4222574338 2024-05-17T22:55:02.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.271+0000 7f499dffb700 1 -- 172.21.15.12:0/4056176778 <== mds.1 v2:172.21.15.96:6836/4222574338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5590b1a4a9c0 con 0x5590b1ba4460 2024-05-17T22:55:02.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.271+0000 7f499dffb700 10 client.26305 handle_command_reply: tid=0 2024-05-17T22:55:02.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.271+0000 7f49cb419b80 1 client.26305 shutdown 2024-05-17T22:55:02.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.271+0000 7f49cb419b80 1 -- 172.21.15.12:0/4056176778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b19a29f0 msgr2=0x5590b1b32c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:02.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.271+0000 7f49cb419b80 1 --2- 172.21.15.12:0/4056176778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b19a29f0 0x5590b1b32c20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f49a80059d0 tx=0x7f49a8009ca0 comp rx=0 tx=0).stop 2024-05-17T22:55:02.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.271+0000 7f49cb419b80 1 -- 172.21.15.12:0/4056176778 shutdown_connections 2024-05-17T22:55:02.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.271+0000 7f49cb419b80 1 --2- 172.21.15.12:0/4056176778 >> [v2:172.21.15.96:6836/4222574338,v1:172.21.15.96:6837/4222574338] conn(0x5590b1ba4460 0x5590b1ba6880 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:02.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.271+0000 7f49cb419b80 1 --2- 172.21.15.12:0/4056176778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590b1b26aa0 0x5590b1b2dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:02.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.271+0000 7f49cb419b80 1 --2- 172.21.15.12:0/4056176778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b19a29f0 0x5590b1b32c20 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:02.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.271+0000 7f499dffb700 0 client.26305 ms_handle_reset on v2:172.21.15.96:6836/4222574338 2024-05-17T22:55:02.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.271+0000 7f49cb419b80 1 --2- 172.21.15.12:0/4056176778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590b1b21170 0x5590b1b2d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:02.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.271+0000 7f49cb419b80 1 -- 172.21.15.12:0/4056176778 >> 172.21.15.12:0/4056176778 conn(0x5590b19e6f00 msgr2=0x5590b1b25a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:55:02.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.271+0000 7f49cb419b80 1 -- 172.21.15.12:0/4056176778 shutdown_connections 2024-05-17T22:55:02.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.272+0000 7f49cb419b80 1 -- 172.21.15.12:0/4056176778 wait complete. 2024-05-17T22:55:02.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.272+0000 7f49cb419b80 20 client.26305 trim_cache size 0 max 16384 2024-05-17T22:55:02.281 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:55:02.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.272+0000 7f49bca3e700 1 -- 172.21.15.12:0/1948782899 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f49a00b2880 msgr2=0x7f49a00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:02.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.272+0000 7f49bca3e700 1 --2- 172.21.15.12:0/1948782899 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f49a00b2880 0x7f49a00b4d90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f49a800b5c0 tx=0x7f49a8005960 comp rx=0 tx=0).stop 2024-05-17T22:55:02.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.272+0000 7f49bca3e700 1 -- 172.21.15.12:0/1948782899 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49b810e240 msgr2=0x7f49b819a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:02.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.272+0000 7f49bca3e700 1 --2- 172.21.15.12:0/1948782899 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49b810e240 0x7f49b819a650 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f49b0007ee0 tx=0x7f49b000d3b0 comp rx=0 tx=0).stop 2024-05-17T22:55:02.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.272+0000 7f49bca3e700 1 -- 172.21.15.12:0/1948782899 shutdown_connections 2024-05-17T22:55:02.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.272+0000 7f49bca3e700 1 --2- 172.21.15.12:0/1948782899 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f49a00b2880 0x7f49a00b4d90 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:02.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.272+0000 7f49bca3e700 1 --2- 172.21.15.12:0/1948782899 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49b810e240 0x7f49b819a650 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:02.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.272+0000 7f49bca3e700 1 --2- 172.21.15.12:0/1948782899 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49b8104e10 0x7f49b81a04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:02.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.272+0000 7f49bca3e700 1 --2- 172.21.15.12:0/1948782899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49b8105760 0x7f49b81a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:02.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.272+0000 7f49bca3e700 1 -- 172.21.15.12:0/1948782899 >> 172.21.15.12:0/1948782899 conn(0x7f49b8079ae0 msgr2=0x7f49b80ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:55:02.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.272+0000 7f49bca3e700 1 -- 172.21.15.12:0/1948782899 shutdown_connections 2024-05-17T22:55:02.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:02.272+0000 7f49bca3e700 1 -- 172.21.15.12:0/1948782899 wait complete. 2024-05-17T22:55:02.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:02 smithi151 ceph-mon[36101]: pgmap v912: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 43 KiB/s wr, 13 op/s 2024-05-17T22:55:02.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:02 smithi096 ceph-mon[36065]: pgmap v912: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 43 KiB/s wr, 13 op/s 2024-05-17T22:55:02.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:02 smithi012 ceph-mon[31632]: pgmap v912: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 43 KiB/s wr, 13 op/s 2024-05-17T22:55:03.132 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-17T22:55:03.391 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:55:04.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:04 smithi151 ceph-mon[36101]: pgmap v913: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 24 KiB/s wr, 10 op/s 2024-05-17T22:55:04.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:04 smithi096 ceph-mon[36065]: pgmap v913: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 24 KiB/s wr, 10 op/s 2024-05-17T22:55:04.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:04 smithi012 ceph-mon[31632]: pgmap v913: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 24 KiB/s wr, 10 op/s 2024-05-17T22:55:05.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.243+0000 7f4a2cac6700 1 -- 172.21.15.12:0/3195171214 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a281080a0 msgr2=0x7f4a2810fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:05.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.243+0000 7f4a2cac6700 1 --2- 172.21.15.12:0/3195171214 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a281080a0 0x7f4a2810fa00 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f4a20009990 tx=0x7f4a20009ca0 comp rx=0 tx=0).stop 2024-05-17T22:55:05.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.244+0000 7f4a2cac6700 1 -- 172.21.15.12:0/3195171214 shutdown_connections 2024-05-17T22:55:05.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.244+0000 7f4a2cac6700 1 --2- 172.21.15.12:0/3195171214 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a281080a0 0x7f4a2810fa00 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:05.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.244+0000 7f4a2cac6700 1 --2- 172.21.15.12:0/3195171214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a28102e20 0x7f4a28103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:05.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.244+0000 7f4a2cac6700 1 --2- 172.21.15.12:0/3195171214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a28107750 0x7f4a28107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:05.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.244+0000 7f4a2cac6700 1 -- 172.21.15.12:0/3195171214 >> 172.21.15.12:0/3195171214 conn(0x7f4a28079bd0 msgr2=0x7f4a28079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:55:05.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.244+0000 7f4a2cac6700 1 -- 172.21.15.12:0/3195171214 shutdown_connections 2024-05-17T22:55:05.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.244+0000 7f4a2cac6700 1 -- 172.21.15.12:0/3195171214 wait complete. 2024-05-17T22:55:05.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.244+0000 7f4a2cac6700 1 Processor -- start 2024-05-17T22:55:05.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.245+0000 7f4a2cac6700 1 -- start start 2024-05-17T22:55:05.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.245+0000 7f4a2cac6700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a28102e20 0x7f4a281051b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:05.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.245+0000 7f4a2cac6700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a28107750 0x7f4a281056f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:05.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.245+0000 7f4a2cac6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a281080a0 0x7f4a28105c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:05.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.245+0000 7f4a2cac6700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4a28112d10 con 0x7f4a281080a0 2024-05-17T22:55:05.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.245+0000 7f4a2cac6700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4a28112b90 con 0x7f4a28102e20 2024-05-17T22:55:05.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.245+0000 7f4a2cac6700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4a28112e90 con 0x7f4a28107750 2024-05-17T22:55:05.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.245+0000 7f4a26d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a281080a0 0x7f4a28105c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:05.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.245+0000 7f4a26d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a281080a0 0x7f4a28105c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47474/0 (socket says 172.21.15.12:47474) 2024-05-17T22:55:05.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.245+0000 7f4a26d9d700 1 -- 172.21.15.12:0/764395720 learned_addr learned my addr 172.21.15.12:0/764395720 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:55:05.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.245+0000 7f4a2659c700 1 --2- 172.21.15.12:0/764395720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a28102e20 0x7f4a281051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:05.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.245+0000 7f4a26d9d700 1 -- 172.21.15.12:0/764395720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a28107750 msgr2=0x7f4a281056f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:05.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.245+0000 7f4a25d9b700 1 --2- 172.21.15.12:0/764395720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a28107750 0x7f4a281056f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:05.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.245+0000 7f4a26d9d700 1 --2- 172.21.15.12:0/764395720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a28107750 0x7f4a281056f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:05.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.245+0000 7f4a26d9d700 1 -- 172.21.15.12:0/764395720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a28102e20 msgr2=0x7f4a281051b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:05.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.245+0000 7f4a26d9d700 1 --2- 172.21.15.12:0/764395720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a28102e20 0x7f4a281051b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:05.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.245+0000 7f4a26d9d700 1 -- 172.21.15.12:0/764395720 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a281a8b80 con 0x7f4a281080a0 2024-05-17T22:55:05.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.245+0000 7f4a2659c700 1 --2- 172.21.15.12:0/764395720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a28102e20 0x7f4a281051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:55:05.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.245+0000 7f4a25d9b700 1 --2- 172.21.15.12:0/764395720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a28107750 0x7f4a281056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:55:05.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.246+0000 7f4a26d9d700 1 --2- 172.21.15.12:0/764395720 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a281080a0 0x7f4a28105c30 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f4a2000c010 tx=0x7f4a200047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:55:05.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.246+0000 7f4a177fe700 1 -- 172.21.15.12:0/764395720 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a20027070 con 0x7f4a281080a0 2024-05-17T22:55:05.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.246+0000 7f4a2cac6700 1 -- 172.21.15.12:0/764395720 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a281a8e10 con 0x7f4a281080a0 2024-05-17T22:55:05.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.246+0000 7f4a177fe700 1 -- 172.21.15.12:0/764395720 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a20022470 con 0x7f4a281080a0 2024-05-17T22:55:05.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.246+0000 7f4a177fe700 1 -- 172.21.15.12:0/764395720 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a20017650 con 0x7f4a281080a0 2024-05-17T22:55:05.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.246+0000 7f4a2cac6700 1 -- 172.21.15.12:0/764395720 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a281a9200 con 0x7f4a281080a0 2024-05-17T22:55:05.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.249+0000 7f4a177fe700 1 -- 172.21.15.12:0/764395720 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4a2002a070 con 0x7f4a281080a0 2024-05-17T22:55:05.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.250+0000 7f4a177fe700 1 --2- 172.21.15.12:0/764395720 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f4a100b2900 0x7f4a100b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:05.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.250+0000 7f4a2659c700 1 --2- 172.21.15.12:0/764395720 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f4a100b2900 0x7f4a100b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:05.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.250+0000 7f4a177fe700 1 -- 172.21.15.12:0/764395720 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f4a200e3680 con 0x7f4a281080a0 2024-05-17T22:55:05.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.250+0000 7f4a3b4a1b80 1 Processor -- start 2024-05-17T22:55:05.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.250+0000 7f4a2659c700 1 --2- 172.21.15.12:0/764395720 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f4a100b2900 0x7f4a100b4e10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4a18009dd0 tx=0x7f4a18009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:55:05.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.250+0000 7f4a3b4a1b80 1 -- start start 2024-05-17T22:55:05.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.250+0000 7f4a3b4a1b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8070b99f0 0x55e80723d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:05.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.251+0000 7f4a3b4a1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80723daa0 0x55e807237c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:05.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.251+0000 7f4a3b4a1b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e807238170 0x55e807238580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:05.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.251+0000 7f4a3b4a1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e807166290 con 0x55e80723daa0 2024-05-17T22:55:05.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.251+0000 7f4a3b4a1b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e807155fd0 con 0x55e8070b99f0 2024-05-17T22:55:05.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.251+0000 7f4a3b4a1b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e807159e70 con 0x55e807238170 2024-05-17T22:55:05.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.251+0000 7f4a25d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80723daa0 0x55e807237c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:05.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.251+0000 7f4a25d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80723daa0 0x55e807237c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47484/0 (socket says 172.21.15.12:47484) 2024-05-17T22:55:05.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.251+0000 7f4a25d9b700 1 -- 172.21.15.12:0/3274689552 learned_addr learned my addr 172.21.15.12:0/3274689552 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:55:05.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.251+0000 7f4a2659c700 1 --2- 172.21.15.12:0/3274689552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8070b99f0 0x55e80723d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:05.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.251+0000 7f4a26d9d700 1 --2- 172.21.15.12:0/3274689552 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e807238170 0x55e807238580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:05.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.251+0000 7f4a25d9b700 1 -- 172.21.15.12:0/3274689552 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e807238170 msgr2=0x55e807238580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:55:05.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.251+0000 7f4a25d9b700 1 --2- 172.21.15.12:0/3274689552 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e807238170 0x55e807238580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:05.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.251+0000 7f4a25d9b700 1 -- 172.21.15.12:0/3274689552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8070b99f0 msgr2=0x55e80723d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:55:05.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.251+0000 7f4a25d9b700 1 --2- 172.21.15.12:0/3274689552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8070b99f0 0x55e80723d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:05.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.251+0000 7f4a25d9b700 1 -- 172.21.15.12:0/3274689552 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8070ff590 con 0x55e80723daa0 2024-05-17T22:55:05.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.251+0000 7f4a26d9d700 1 --2- 172.21.15.12:0/3274689552 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e807238170 0x55e807238580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:55:05.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.251+0000 7f4a2659c700 1 --2- 172.21.15.12:0/3274689552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8070b99f0 0x55e80723d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:55:05.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.251+0000 7f4a25d9b700 1 --2- 172.21.15.12:0/3274689552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80723daa0 0x55e807237c30 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c009d10 tx=0x7f4a1c00f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bff87abfa4aec26 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:55:05.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.251+0000 7f4a14ff9700 1 -- 172.21.15.12:0/3274689552 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c009e60 con 0x55e80723daa0 2024-05-17T22:55:05.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.252+0000 7f4a14ff9700 1 -- 172.21.15.12:0/3274689552 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a1c0044e0 con 0x55e80723daa0 2024-05-17T22:55:05.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.252+0000 7f4a14ff9700 1 -- 172.21.15.12:0/3274689552 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c011440 con 0x55e80723daa0 2024-05-17T22:55:05.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.252+0000 7f4a3b4a1b80 1 -- 172.21.15.12:0/3274689552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80723daa0 msgr2=0x55e807237c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:05.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.252+0000 7f4a3b4a1b80 1 --2- 172.21.15.12:0/3274689552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80723daa0 0x55e807237c30 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c009d10 tx=0x7f4a1c00f640 comp rx=0 tx=0).stop 2024-05-17T22:55:05.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.252+0000 7f4a3b4a1b80 1 -- 172.21.15.12:0/3274689552 shutdown_connections 2024-05-17T22:55:05.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.252+0000 7f4a3b4a1b80 1 --2- 172.21.15.12:0/3274689552 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e807238170 0x55e807238580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:05.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.252+0000 7f4a3b4a1b80 1 --2- 172.21.15.12:0/3274689552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8070b99f0 0x55e80723d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:05.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.252+0000 7f4a3b4a1b80 1 --2- 172.21.15.12:0/3274689552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80723daa0 0x55e807237c30 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:05.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.252+0000 7f4a3b4a1b80 1 -- 172.21.15.12:0/3274689552 >> 172.21.15.12:0/3274689552 conn(0x55e8070fdf00 msgr2=0x55e807232b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:55:05.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.253+0000 7f4a3b4a1b80 1 -- 172.21.15.12:0/3274689552 shutdown_connections 2024-05-17T22:55:05.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.253+0000 7f4a3b4a1b80 1 -- 172.21.15.12:0/3274689552 wait complete. 2024-05-17T22:55:05.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.253+0000 7f4a3b4a1b80 1 -- start start 2024-05-17T22:55:05.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.253+0000 7f4a3b4a1b80 1 Processor -- start 2024-05-17T22:55:05.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.253+0000 7f4a3b4a1b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8070b99f0 0x55e8072411d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:05.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.253+0000 7f4a3b4a1b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e807238170 0x55e807241710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:05.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.253+0000 7f4a3b4a1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80723daa0 0x55e807241c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:05.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.253+0000 7f4a3b4a1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e80705d150 con 0x55e80723daa0 2024-05-17T22:55:05.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.253+0000 7f4a3b4a1b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e8070f6e80 con 0x55e807238170 2024-05-17T22:55:05.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.253+0000 7f4a3b4a1b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e807244390 con 0x55e8070b99f0 2024-05-17T22:55:05.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.254+0000 7f4a3b4a1b80 10 client.0 _ll_register_callbacks cb 0x55e8070f0f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:55:05.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.254+0000 7f4a26d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80723daa0 0x55e807241c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:05.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.254+0000 7f4a26d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80723daa0 0x55e807241c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47496/0 (socket says 172.21.15.12:47496) 2024-05-17T22:55:05.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.254+0000 7f4a26d9d700 1 -- 172.21.15.12:0/3112121507 learned_addr learned my addr 172.21.15.12:0/3112121507 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:55:05.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.254+0000 7f4a25d9b700 1 --2- 172.21.15.12:0/3112121507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e807238170 0x55e807241710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:05.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.254+0000 7f4a2659c700 1 --2- 172.21.15.12:0/3112121507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8070b99f0 0x55e8072411d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:05.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.254+0000 7f4a26d9d700 1 -- 172.21.15.12:0/3112121507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8070b99f0 msgr2=0x55e8072411d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:55:05.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.254+0000 7f4a26d9d700 1 --2- 172.21.15.12:0/3112121507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8070b99f0 0x55e8072411d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:05.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.254+0000 7f4a26d9d700 1 -- 172.21.15.12:0/3112121507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e807238170 msgr2=0x55e807241710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:55:05.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.254+0000 7f4a26d9d700 1 --2- 172.21.15.12:0/3112121507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e807238170 0x55e807241710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:05.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.254+0000 7f4a26d9d700 1 -- 172.21.15.12:0/3112121507 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8070a6480 con 0x55e80723daa0 2024-05-17T22:55:05.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.254+0000 7f4a26d9d700 1 --2- 172.21.15.12:0/3112121507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80723daa0 0x55e807241c50 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f4a200e0d90 tx=0x7f4a20022c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dfa65a38c66dbe server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:55:05.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.254+0000 7f4a0dffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:55:05.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.254+0000 7f4a0dffb700 1 -- 172.21.15.12:0/3112121507 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a200aa450 con 0x55e80723daa0 2024-05-17T22:55:05.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.254+0000 7f4a3b4a1b80 1 -- 172.21.15.12:0/3112121507 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a1c0044e0 con 0x55e80723daa0 2024-05-17T22:55:05.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.254+0000 7f4a0dffb700 1 -- 172.21.15.12:0/3112121507 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f4a200cd0b0 con 0x55e80723daa0 2024-05-17T22:55:05.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.255+0000 7f4a0dffb700 1 -- 172.21.15.12:0/3112121507 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f4a200ab040 con 0x55e80723daa0 2024-05-17T22:55:05.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.255+0000 7f4a3b4a1b80 10 client.16677 fetch_fsmap learned FSMap version 46 2024-05-17T22:55:05.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.255+0000 7f4a3b4a1b80 1 -- 172.21.15.12:0/3112121507 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55e80724ede0 con 0x55e80723daa0 2024-05-17T22:55:05.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.255+0000 7f4a0dffb700 1 -- 172.21.15.12:0/3112121507 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a1c0044e0 con 0x55e80723daa0 2024-05-17T22:55:05.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.255+0000 7f4a0dffb700 1 -- 172.21.15.12:0/3112121507 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7f4a200e3070 con 0x55e80723daa0 2024-05-17T22:55:05.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.255+0000 7f4a3b4a1b80 10 client.16677 fetch_fsmap finished waiting for FSMap version 46 2024-05-17T22:55:05.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.255+0000 7f4a3b4a1b80 10 client.16677 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:55:05.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.255+0000 7f4a3b4a1b80 20 client.16677 populate_metadata read hostname 'smithi012' 2024-05-17T22:55:05.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.255+0000 7f4a3b4a1b80 1 --2- 172.21.15.12:0/3112121507 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55e807251350 0x55e8072537b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:05.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.255+0000 7f4a3b4a1b80 4 client.16677 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:55:05.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.255+0000 7f4a3b4a1b80 1 -- 172.21.15.12:0/3112121507 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8071681e0 con 0x55e807251350 2024-05-17T22:55:05.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.255+0000 7f4a2659c700 1 --2- 172.21.15.12:0/3112121507 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55e807251350 0x55e8072537b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:05.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.256+0000 7f4a0dffb700 10 client.16677 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:55:05.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.256+0000 7f4a2659c700 1 --2- 172.21.15.12:0/3112121507 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55e807251350 0x55e8072537b0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8e74a12edc6810f server_cookie=1c91c3a5d940def1 in_seq=0 out_seq=0 2024-05-17T22:55:05.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.257+0000 7f4a0dffb700 1 -- 172.21.15.12:0/3112121507 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e8071681e0 con 0x55e807251350 2024-05-17T22:55:05.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.257+0000 7f4a0dffb700 10 client.16677 handle_command_reply: tid=0 2024-05-17T22:55:05.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.257+0000 7f4a3b4a1b80 1 client.16677 shutdown 2024-05-17T22:55:05.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.257+0000 7f4a3b4a1b80 1 -- 172.21.15.12:0/3112121507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80723daa0 msgr2=0x55e807241c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:05.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.257+0000 7f4a3b4a1b80 1 --2- 172.21.15.12:0/3112121507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80723daa0 0x55e807241c50 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f4a200e0d90 tx=0x7f4a20022c00 comp rx=0 tx=0).stop 2024-05-17T22:55:05.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.257+0000 7f4a3b4a1b80 1 -- 172.21.15.12:0/3112121507 shutdown_connections 2024-05-17T22:55:05.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.257+0000 7f4a3b4a1b80 1 --2- 172.21.15.12:0/3112121507 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55e807251350 0x55e8072537b0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:05.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.257+0000 7f4a3b4a1b80 1 --2- 172.21.15.12:0/3112121507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80723daa0 0x55e807241c50 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:05.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.257+0000 7f4a3b4a1b80 1 --2- 172.21.15.12:0/3112121507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8070b99f0 0x55e8072411d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:05.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.257+0000 7f4a3b4a1b80 1 --2- 172.21.15.12:0/3112121507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e807238170 0x55e807241710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:05.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.257+0000 7f4a3b4a1b80 1 -- 172.21.15.12:0/3112121507 >> 172.21.15.12:0/3112121507 conn(0x55e8070fdf00 msgr2=0x55e80722ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:55:05.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.257+0000 7f4a0dffb700 0 client.16677 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:55:05.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.257+0000 7f4a3b4a1b80 1 -- 172.21.15.12:0/3112121507 shutdown_connections 2024-05-17T22:55:05.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.257+0000 7f4a3b4a1b80 1 -- 172.21.15.12:0/3112121507 wait complete. 2024-05-17T22:55:05.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.257+0000 7f4a3b4a1b80 20 client.16677 trim_cache size 0 max 16384 2024-05-17T22:55:05.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.272+0000 7f4a3b4a1b80 1 Processor -- start 2024-05-17T22:55:05.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.272+0000 7f4a3b4a1b80 1 -- start start 2024-05-17T22:55:05.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.273+0000 7f4a3b4a1b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8070b99f0 0x55e80724d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:05.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.273+0000 7f4a3b4a1b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e807238170 0x55e807232ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:05.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.273+0000 7f4a3b4a1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80723daa0 0x55e8072439d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:05.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.273+0000 7f4a3b4a1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4a281b4720 con 0x55e80723daa0 2024-05-17T22:55:05.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.273+0000 7f4a3b4a1b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4a281b1f20 con 0x55e8070b99f0 2024-05-17T22:55:05.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.273+0000 7f4a3b4a1b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4a281afd70 con 0x55e807238170 2024-05-17T22:55:05.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.273+0000 7f4a26d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80723daa0 0x55e8072439d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:05.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.273+0000 7f4a26d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80723daa0 0x55e8072439d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47506/0 (socket says 172.21.15.12:47506) 2024-05-17T22:55:05.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.273+0000 7f4a26d9d700 1 -- 172.21.15.12:0/3688496655 learned_addr learned my addr 172.21.15.12:0/3688496655 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:55:05.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.273+0000 7f4a2659c700 1 --2- 172.21.15.12:0/3688496655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8070b99f0 0x55e80724d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:05.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.273+0000 7f4a26d9d700 1 -- 172.21.15.12:0/3688496655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e807238170 msgr2=0x55e807232ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:55:05.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.273+0000 7f4a26d9d700 1 --2- 172.21.15.12:0/3688496655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e807238170 0x55e807232ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:05.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.273+0000 7f4a26d9d700 1 -- 172.21.15.12:0/3688496655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8070b99f0 msgr2=0x55e80724d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:55:05.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.273+0000 7f4a26d9d700 1 --2- 172.21.15.12:0/3688496655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8070b99f0 0x55e80724d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:05.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.273+0000 7f4a26d9d700 1 -- 172.21.15.12:0/3688496655 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e80723d0d0 con 0x55e80723daa0 2024-05-17T22:55:05.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.273+0000 7f4a26d9d700 1 --2- 172.21.15.12:0/3688496655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80723daa0 0x55e8072439d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f4a200b31e0 tx=0x7f4a200c50d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283b6007834e453c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:55:05.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.273+0000 7f4a0ffff700 1 -- 172.21.15.12:0/3688496655 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a200aa290 con 0x55e80723daa0 2024-05-17T22:55:05.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.273+0000 7f4a0ffff700 1 -- 172.21.15.12:0/3688496655 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a200aa450 con 0x55e80723daa0 2024-05-17T22:55:05.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.274+0000 7f4a0ffff700 1 -- 172.21.15.12:0/3688496655 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a200cd070 con 0x55e80723daa0 2024-05-17T22:55:05.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.274+0000 7f4a3b4a1b80 1 -- 172.21.15.12:0/3688496655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80723daa0 msgr2=0x55e8072439d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:05.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.274+0000 7f4a3b4a1b80 1 --2- 172.21.15.12:0/3688496655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80723daa0 0x55e8072439d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f4a200b31e0 tx=0x7f4a200c50d0 comp rx=0 tx=0).stop 2024-05-17T22:55:05.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.274+0000 7f4a3b4a1b80 1 -- 172.21.15.12:0/3688496655 shutdown_connections 2024-05-17T22:55:05.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.274+0000 7f4a3b4a1b80 1 --2- 172.21.15.12:0/3688496655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80723daa0 0x55e8072439d0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:05.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.274+0000 7f4a3b4a1b80 1 --2- 172.21.15.12:0/3688496655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8070b99f0 0x55e80724d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:05.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.274+0000 7f4a3b4a1b80 1 --2- 172.21.15.12:0/3688496655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e807238170 0x55e807232ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:05.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.274+0000 7f4a3b4a1b80 1 -- 172.21.15.12:0/3688496655 >> 172.21.15.12:0/3688496655 conn(0x55e8070fdf00 msgr2=0x55e80723c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:55:05.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.274+0000 7f4a3b4a1b80 1 -- 172.21.15.12:0/3688496655 shutdown_connections 2024-05-17T22:55:05.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.274+0000 7f4a3b4a1b80 1 -- 172.21.15.12:0/3688496655 wait complete. 2024-05-17T22:55:05.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.275+0000 7f4a3b4a1b80 1 -- start start 2024-05-17T22:55:05.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.275+0000 7f4a3b4a1b80 1 Processor -- start 2024-05-17T22:55:05.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.275+0000 7f4a3b4a1b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e8070b99f0 0x55e807249b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:05.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.275+0000 7f4a3b4a1b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e807238170 0x55e8072464a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:05.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.275+0000 7f4a3b4a1b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e80723daa0 0x55e807246a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:05.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.275+0000 7f4a3b4a1b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4a281b03e0 con 0x55e8070b99f0 2024-05-17T22:55:05.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.275+0000 7f4a3b4a1b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4a281b1850 con 0x55e807238170 2024-05-17T22:55:05.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.275+0000 7f4a3b4a1b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4a281b0ab0 con 0x55e80723daa0 2024-05-17T22:55:05.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.275+0000 7f4a3b4a1b80 10 client.0 _ll_register_callbacks cb 0x55e8070f0f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:55:05.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.275+0000 7f4a2659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e8070b99f0 0x55e807249b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:05.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.275+0000 7f4a2659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e8070b99f0 0x55e807249b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47512/0 (socket says 172.21.15.12:47512) 2024-05-17T22:55:05.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.275+0000 7f4a2659c700 1 -- 172.21.15.12:0/2993927285 learned_addr learned my addr 172.21.15.12:0/2993927285 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:55:05.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.275+0000 7f4a25d9b700 1 --2- 172.21.15.12:0/2993927285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e807238170 0x55e8072464a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:05.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.275+0000 7f4a26d9d700 1 --2- 172.21.15.12:0/2993927285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e80723daa0 0x55e807246a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:05.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.276+0000 7f4a2659c700 1 -- 172.21.15.12:0/2993927285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e80723daa0 msgr2=0x55e807246a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:55:05.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.276+0000 7f4a2659c700 1 --2- 172.21.15.12:0/2993927285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e80723daa0 0x55e807246a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:05.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.276+0000 7f4a2659c700 1 -- 172.21.15.12:0/2993927285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e807238170 msgr2=0x55e8072464a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:55:05.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.276+0000 7f4a2659c700 1 --2- 172.21.15.12:0/2993927285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e807238170 0x55e8072464a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:05.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.276+0000 7f4a2659c700 1 -- 172.21.15.12:0/2993927285 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e807247200 con 0x55e8070b99f0 2024-05-17T22:55:05.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.276+0000 7f4a25d9b700 1 --2- 172.21.15.12:0/2993927285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e807238170 0x55e8072464a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:55:05.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.276+0000 7f4a2659c700 1 --2- 172.21.15.12:0/2993927285 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e8070b99f0 0x55e807249b50 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f4a18009bb0 tx=0x7f4a18012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7863e3526a34b115 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:55:05.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.276+0000 7f4a0dffb700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-05-17T22:55:05.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.276+0000 7f4a0dffb700 1 -- 172.21.15.12:0/2993927285 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a18031060 con 0x55e8070b99f0 2024-05-17T22:55:05.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.276+0000 7f4a0dffb700 1 -- 172.21.15.12:0/2993927285 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f4a18016a70 con 0x55e8070b99f0 2024-05-17T22:55:05.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.276+0000 7f4a3b4a1b80 1 -- 172.21.15.12:0/2993927285 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a200aa450 con 0x55e8070b99f0 2024-05-17T22:55:05.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.276+0000 7f4a0dffb700 1 -- 172.21.15.12:0/2993927285 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f4a18038050 con 0x55e8070b99f0 2024-05-17T22:55:05.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.276+0000 7f4a3b4a1b80 10 client.16683 fetch_fsmap learned FSMap version 46 2024-05-17T22:55:05.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.276+0000 7f4a3b4a1b80 1 -- 172.21.15.12:0/2993927285 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55e8072bab30 con 0x55e8070b99f0 2024-05-17T22:55:05.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.277+0000 7f4a0dffb700 1 -- 172.21.15.12:0/2993927285 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a200aa450 con 0x55e8070b99f0 2024-05-17T22:55:05.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.277+0000 7f4a0dffb700 1 -- 172.21.15.12:0/2993927285 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7f4a1802bce0 con 0x55e8070b99f0 2024-05-17T22:55:05.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.277+0000 7f4a3b4a1b80 10 client.16683 fetch_fsmap finished waiting for FSMap version 46 2024-05-17T22:55:05.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.277+0000 7f4a3b4a1b80 10 client.16683 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T22:55:05.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.277+0000 7f4a3b4a1b80 20 client.16683 populate_metadata read hostname 'smithi012' 2024-05-17T22:55:05.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.277+0000 7f4a3b4a1b80 1 --2- 172.21.15.12:0/2993927285 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55e8072bb220 0x55e8072bd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:05.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.277+0000 7f4a3b4a1b80 4 client.16683 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T22:55:05.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.277+0000 7f4a3b4a1b80 1 -- 172.21.15.12:0/2993927285 --> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e8071619c0 con 0x55e8072bb220 2024-05-17T22:55:05.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.277+0000 7f4a25d9b700 1 --2- 172.21.15.12:0/2993927285 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55e8072bb220 0x55e8072bd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:05.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.278+0000 7f4a0dffb700 10 client.16683 ms_handle_connect on v2:172.21.15.96:6838/2986295519 2024-05-17T22:55:05.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.278+0000 7f4a25d9b700 1 --2- 172.21.15.12:0/2993927285 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55e8072bb220 0x55e8072bd640 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7b0f6f9678c7e5b server_cookie=889e87d51d579a40 in_seq=0 out_seq=0 2024-05-17T22:55:05.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.278+0000 7f4a0dffb700 1 -- 172.21.15.12:0/2993927285 <== mds.0 v2:172.21.15.96:6838/2986295519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e8071619c0 con 0x55e8072bb220 2024-05-17T22:55:05.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.278+0000 7f4a0dffb700 10 client.16683 handle_command_reply: tid=0 2024-05-17T22:55:05.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.278+0000 7f4a3b4a1b80 1 client.16683 shutdown 2024-05-17T22:55:05.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.278+0000 7f4a3b4a1b80 1 -- 172.21.15.12:0/2993927285 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e8070b99f0 msgr2=0x55e807249b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:05.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.278+0000 7f4a3b4a1b80 1 --2- 172.21.15.12:0/2993927285 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e8070b99f0 0x55e807249b50 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f4a18009bb0 tx=0x7f4a18012000 comp rx=0 tx=0).stop 2024-05-17T22:55:05.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.278+0000 7f4a3b4a1b80 1 -- 172.21.15.12:0/2993927285 shutdown_connections 2024-05-17T22:55:05.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.278+0000 7f4a3b4a1b80 1 --2- 172.21.15.12:0/2993927285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e80723daa0 0x55e807246a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:05.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.278+0000 7f4a3b4a1b80 1 --2- 172.21.15.12:0/2993927285 >> [v2:172.21.15.96:6838/2986295519,v1:172.21.15.96:6839/2986295519] conn(0x55e8072bb220 0x55e8072bd640 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:05.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.278+0000 7f4a3b4a1b80 1 --2- 172.21.15.12:0/2993927285 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e8070b99f0 0x55e807249b50 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:05.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.278+0000 7f4a3b4a1b80 1 --2- 172.21.15.12:0/2993927285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e807238170 0x55e8072464a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:05.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.278+0000 7f4a3b4a1b80 1 -- 172.21.15.12:0/2993927285 >> 172.21.15.12:0/2993927285 conn(0x55e8070fdf00 msgr2=0x55e80723ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:55:05.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.278+0000 7f4a0dffb700 0 client.16683 ms_handle_reset on v2:172.21.15.96:6838/2986295519 2024-05-17T22:55:05.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.278+0000 7f4a3b4a1b80 1 -- 172.21.15.12:0/2993927285 shutdown_connections 2024-05-17T22:55:05.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.278+0000 7f4a3b4a1b80 1 -- 172.21.15.12:0/2993927285 wait complete. 2024-05-17T22:55:05.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.278+0000 7f4a3b4a1b80 20 client.16683 trim_cache size 0 max 16384 2024-05-17T22:55:05.292 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-05-17T22:55:05.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.279+0000 7f4a2cac6700 1 -- 172.21.15.12:0/764395720 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f4a100b2900 msgr2=0x7f4a100b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:05.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.279+0000 7f4a2cac6700 1 --2- 172.21.15.12:0/764395720 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f4a100b2900 0x7f4a100b4e10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4a18009dd0 tx=0x7f4a18009040 comp rx=0 tx=0).stop 2024-05-17T22:55:05.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.279+0000 7f4a2cac6700 1 -- 172.21.15.12:0/764395720 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a281080a0 msgr2=0x7f4a28105c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:05.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.279+0000 7f4a2cac6700 1 --2- 172.21.15.12:0/764395720 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a281080a0 0x7f4a28105c30 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f4a2000c010 tx=0x7f4a200047b0 comp rx=0 tx=0).stop 2024-05-17T22:55:05.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.279+0000 7f4a2cac6700 1 -- 172.21.15.12:0/764395720 shutdown_connections 2024-05-17T22:55:05.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.279+0000 7f4a2cac6700 1 --2- 172.21.15.12:0/764395720 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f4a100b2900 0x7f4a100b4e10 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:05.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.279+0000 7f4a2cac6700 1 --2- 172.21.15.12:0/764395720 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a281080a0 0x7f4a28105c30 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:05.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.279+0000 7f4a2cac6700 1 --2- 172.21.15.12:0/764395720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a28102e20 0x7f4a281051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:05.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.279+0000 7f4a2cac6700 1 --2- 172.21.15.12:0/764395720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a28107750 0x7f4a281056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:05.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.279+0000 7f4a2cac6700 1 -- 172.21.15.12:0/764395720 >> 172.21.15.12:0/764395720 conn(0x7f4a28079bd0 msgr2=0x7f4a2806c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:55:05.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.279+0000 7f4a2cac6700 1 -- 172.21.15.12:0/764395720 shutdown_connections 2024-05-17T22:55:05.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:05.279+0000 7f4a2cac6700 1 -- 172.21.15.12:0/764395720 wait complete. 2024-05-17T22:55:06.184 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_931809a9e5d295bd61984693cac44fe131ded4bc/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_931809a9e5d295bd61984693cac44fe131ded4bc/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_931809a9e5d295bd61984693cac44fe131ded4bc/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-05-17T22:55:06.185 INFO:tasks.fwd_scrub:done joining 2024-05-17T22:55:06.186 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-17T22:55:06.196 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T22:55:06.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.404+0000 7f8e29247640 1 -- 172.21.15.12:0/2600224144 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e24108df0 msgr2=0x7f8e2410b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:06.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.404+0000 7f8e29247640 1 --2- 172.21.15.12:0/2600224144 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e24108df0 0x7f8e2410b300 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c009900 tx=0x7f8e1c02a410 comp rx=0 tx=0).stop 2024-05-17T22:55:06.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.405+0000 7f8e29247640 1 -- 172.21.15.12:0/2600224144 shutdown_connections 2024-05-17T22:55:06.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.405+0000 7f8e29247640 1 --2- 172.21.15.12:0/2600224144 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e24108df0 0x7f8e2410b300 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:06.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.405+0000 7f8e29247640 1 --2- 172.21.15.12:0/2600224144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e241062b0 0x7f8e24108730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:06.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.405+0000 7f8e29247640 1 --2- 172.21.15.12:0/2600224144 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e2406acc0 0x7f8e24105ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:06.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.405+0000 7f8e29247640 1 -- 172.21.15.12:0/2600224144 >> 172.21.15.12:0/2600224144 conn(0x7f8e240fd860 msgr2=0x7f8e240ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:55:06.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.405+0000 7f8e29247640 1 -- 172.21.15.12:0/2600224144 shutdown_connections 2024-05-17T22:55:06.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.405+0000 7f8e29247640 1 -- 172.21.15.12:0/2600224144 wait complete. 2024-05-17T22:55:06.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.405+0000 7f8e29247640 1 Processor -- start 2024-05-17T22:55:06.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.406+0000 7f8e29247640 1 -- start start 2024-05-17T22:55:06.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.406+0000 7f8e29247640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e2406acc0 0x7f8e24199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:06.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.406+0000 7f8e29247640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e241062b0 0x7f8e24199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:06.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.406+0000 7f8e29247640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e24108df0 0x7f8e241a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:06.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.406+0000 7f8e29247640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2410ec70 con 0x7f8e24108df0 2024-05-17T22:55:06.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.406+0000 7f8e29247640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2410eaf0 con 0x7f8e2406acc0 2024-05-17T22:55:06.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.406+0000 7f8e29247640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2410edf0 con 0x7f8e241062b0 2024-05-17T22:55:06.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.406+0000 7f8e2359e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e24108df0 0x7f8e241a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:06.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.406+0000 7f8e2359e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e24108df0 0x7f8e241a0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47540/0 (socket says 172.21.15.12:47540) 2024-05-17T22:55:06.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.406+0000 7f8e2359e640 1 -- 172.21.15.12:0/1754896200 learned_addr learned my addr 172.21.15.12:0/1754896200 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:55:06.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.406+0000 7f8e22d9d640 1 --2- 172.21.15.12:0/1754896200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e2406acc0 0x7f8e24199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:06.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.406+0000 7f8e2259c640 1 --2- 172.21.15.12:0/1754896200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e241062b0 0x7f8e24199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:06.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.407+0000 7f8e2359e640 1 -- 172.21.15.12:0/1754896200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e241062b0 msgr2=0x7f8e24199d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:06.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.407+0000 7f8e2359e640 1 --2- 172.21.15.12:0/1754896200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e241062b0 0x7f8e24199d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:06.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.407+0000 7f8e2359e640 1 -- 172.21.15.12:0/1754896200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e2406acc0 msgr2=0x7f8e24199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:06.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.407+0000 7f8e2359e640 1 --2- 172.21.15.12:0/1754896200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e2406acc0 0x7f8e24199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:06.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.407+0000 7f8e2359e640 1 -- 172.21.15.12:0/1754896200 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e241a12e0 con 0x7f8e24108df0 2024-05-17T22:55:06.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.407+0000 7f8e2359e640 1 --2- 172.21.15.12:0/1754896200 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e24108df0 0x7f8e241a0d20 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c009230 tx=0x7f8e1c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:55:06.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.407+0000 7f8e1bfff640 1 -- 172.21.15.12:0/1754896200 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c03f030 con 0x7f8e24108df0 2024-05-17T22:55:06.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.407+0000 7f8e29247640 1 -- 172.21.15.12:0/1754896200 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e241a1570 con 0x7f8e24108df0 2024-05-17T22:55:06.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.407+0000 7f8e1bfff640 1 -- 172.21.15.12:0/1754896200 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e1c003aa0 con 0x7f8e24108df0 2024-05-17T22:55:06.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.407+0000 7f8e1bfff640 1 -- 172.21.15.12:0/1754896200 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c035070 con 0x7f8e24108df0 2024-05-17T22:55:06.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.407+0000 7f8e29247640 1 -- 172.21.15.12:0/1754896200 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e241a1a60 con 0x7f8e24108df0 2024-05-17T22:55:06.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.408+0000 7f8e29247640 1 -- 172.21.15.12:0/1754896200 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e240ff380 con 0x7f8e24108df0 2024-05-17T22:55:06.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.410+0000 7f8e1bfff640 1 -- 172.21.15.12:0/1754896200 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8e1c042030 con 0x7f8e24108df0 2024-05-17T22:55:06.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.411+0000 7f8e1bfff640 1 --2- 172.21.15.12:0/1754896200 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8e100b27f0 0x7f8e100b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:06.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.411+0000 7f8e22d9d640 1 --2- 172.21.15.12:0/1754896200 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8e100b27f0 0x7f8e100b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:06.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.411+0000 7f8e1bfff640 1 -- 172.21.15.12:0/1754896200 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f8e1c102f30 con 0x7f8e24108df0 2024-05-17T22:55:06.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.412+0000 7f8e22d9d640 1 --2- 172.21.15.12:0/1754896200 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8e100b27f0 0x7f8e100b4d00 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8e2406a9b0 tx=0x7f8e14009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:55:06.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.414+0000 7f8e1bfff640 1 -- 172.21.15.12:0/1754896200 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8e1c0ca860 con 0x7f8e24108df0 2024-05-17T22:55:06.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:06 smithi151 ceph-mon[36101]: pgmap v914: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-05-17T22:55:06.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:06 smithi096 ceph-mon[36065]: pgmap v914: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-05-17T22:55:06.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:06 smithi012 ceph-mon[31632]: pgmap v914: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-05-17T22:55:06.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.610+0000 7f8e29247640 1 -- 172.21.15.12:0/1754896200 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8e24063940 con 0x7f8e24108df0 2024-05-17T22:55:06.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.611+0000 7f8e1bfff640 1 -- 172.21.15.12:0/1754896200 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24715 (secure 0 0 0) 0x7f8e1c0ca160 con 0x7f8e24108df0 2024-05-17T22:55:06.614 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:55:06.614 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":89,"fsid":"4d593780-149c-11ef-bc9a-c7b262605968","created":"2024-05-17T22:26:16.955626+0000","modified":"2024-05-17T22:36:15.618992+0000","last_up_change":"2024-05-17T22:33:36.551093+0000","last_in_change":"2024-05-17T22:33:15.032677+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T22:29:42.475513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T22:34:35.504697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T22:34:39.377097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-17T22:34:50.467684+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5d42798f-dc5f-4e66-bce2-ec33dc789e4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6803","nonce":2015605593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6805","nonce":2015605593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6809","nonce":2015605593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6807","nonce":2015605593}]},"public_addr":"172.21.15.12:6803/2015605593","cluster_addr":"172.21.15.12:6805/2015605593","heartbeat_back_addr":"172.21.15.12:6809/2015605593","heartbeat_front_addr":"172.21.15.12:6807/2015605593","state":["exists","up"]},{"osd":1,"uuid":"58afecc7-119b-4c16-a401-640fb77d4714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6801","nonce":1238316358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6803","nonce":1238316358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6807","nonce":1238316358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6805","nonce":1238316358}]},"public_addr":"172.21.15.96:6801/1238316358","cluster_addr":"172.21.15.96:6803/1238316358","heartbeat_back_addr":"172.21.15.96:6807/1238316358","heartbeat_front_addr":"172.21.15.96:6805/1238316358","state":["exists","up"]},{"osd":2,"uuid":"4bef4049-482f-49fb-82fb-38052400eb2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6801","nonce":148729815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6803","nonce":148729815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6807","nonce":148729815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6805","nonce":148729815}]},"public_addr":"172.21.15.151:6801/148729815","cluster_addr":"172.21.15.151:6803/148729815","heartbeat_back_addr":"172.21.15.151:6807/148729815","heartbeat_front_addr":"172.21.15.151:6805/148729815","state":["exists","up"]},{"osd":3,"uuid":"6fed8e36-82a2-49e4-82a3-16e6d9de87f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6811","nonce":3563309320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6813","nonce":3563309320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6817","nonce":3563309320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6815","nonce":3563309320}]},"public_addr":"172.21.15.12:6811/3563309320","cluster_addr":"172.21.15.12:6813/3563309320","heartbeat_back_addr":"172.21.15.12:6817/3563309320","heartbeat_front_addr":"172.21.15.12:6815/3563309320","state":["exists","up"]},{"osd":4,"uuid":"c0e8689a-2816-4089-ade3-931960bae4f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6809","nonce":2219103166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6811","nonce":2219103166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6815","nonce":2219103166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6813","nonce":2219103166}]},"public_addr":"172.21.15.96:6809/2219103166","cluster_addr":"172.21.15.96:6811/2219103166","heartbeat_back_addr":"172.21.15.96:6815/2219103166","heartbeat_front_addr":"172.21.15.96:6813/2219103166","state":["exists","up"]},{"osd":5,"uuid":"e76ba432-94ff-4998-aaba-2645d9e6055d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6809","nonce":2589835078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6811","nonce":2589835078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6815","nonce":2589835078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6813","nonce":2589835078}]},"public_addr":"172.21.15.151:6809/2589835078","cluster_addr":"172.21.15.151:6811/2589835078","heartbeat_back_addr":"172.21.15.151:6815/2589835078","heartbeat_front_addr":"172.21.15.151:6813/2589835078","state":["exists","up"]},{"osd":6,"uuid":"b7ece35c-efd1-40df-95da-05a0cc147372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6819","nonce":3112661724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6821","nonce":3112661724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6825","nonce":3112661724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6823","nonce":3112661724}]},"public_addr":"172.21.15.12:6819/3112661724","cluster_addr":"172.21.15.12:6821/3112661724","heartbeat_back_addr":"172.21.15.12:6825/3112661724","heartbeat_front_addr":"172.21.15.12:6823/3112661724","state":["exists","up"]},{"osd":7,"uuid":"6a65bf30-daf5-4fba-b67e-8e513d88a971","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6817","nonce":2449989110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6819","nonce":2449989110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6823","nonce":2449989110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6821","nonce":2449989110}]},"public_addr":"172.21.15.96:6817/2449989110","cluster_addr":"172.21.15.96:6819/2449989110","heartbeat_back_addr":"172.21.15.96:6823/2449989110","heartbeat_front_addr":"172.21.15.96:6821/2449989110","state":["exists","up"]},{"osd":8,"uuid":"ad62ea44-e416-4a3e-bbbe-93f2a710dbc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6817","nonce":2545224628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6819","nonce":2545224628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6823","nonce":2545224628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6821","nonce":2545224628}]},"public_addr":"172.21.15.151:6817/2545224628","cluster_addr":"172.21.15.151:6819/2545224628","heartbeat_back_addr":"172.21.15.151:6823/2545224628","heartbeat_front_addr":"172.21.15.151:6821/2545224628","state":["exists","up"]},{"osd":9,"uuid":"1275de08-2a3c-48f3-920b-dd68b4024d42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6827","nonce":2100661273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6829","nonce":2100661273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6833","nonce":2100661273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6831","nonce":2100661273}]},"public_addr":"172.21.15.12:6827/2100661273","cluster_addr":"172.21.15.12:6829/2100661273","heartbeat_back_addr":"172.21.15.12:6833/2100661273","heartbeat_front_addr":"172.21.15.12:6831/2100661273","state":["exists","up"]},{"osd":10,"uuid":"347e4e40-9a9d-43fe-8f52-fdd3199a2a5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6825","nonce":304016309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6827","nonce":304016309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6831","nonce":304016309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6829","nonce":304016309}]},"public_addr":"172.21.15.96:6825/304016309","cluster_addr":"172.21.15.96:6827/304016309","heartbeat_back_addr":"172.21.15.96:6831/304016309","heartbeat_front_addr":"172.21.15.96:6829/304016309","state":["exists","up"]},{"osd":11,"uuid":"ee8d3502-8b3c-4e6f-824f-3a97e19111ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6825","nonce":3822307314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6827","nonce":3822307314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6831","nonce":3822307314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6829","nonce":3822307314}]},"public_addr":"172.21.15.151:6825/3822307314","cluster_addr":"172.21.15.151:6827/3822307314","heartbeat_back_addr":"172.21.15.151:6831/3822307314","heartbeat_front_addr":"172.21.15.151:6829/3822307314","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:28:51.542055+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:29:15.566424+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:29:39.500981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:04.797907+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:29.655650+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:55.211880+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:31:21.559462+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:31:47.154852+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:32:13.114892+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:32:40.529708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:33:06.930942+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:33:34.061783+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":10}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/2682700253":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6801/2250046521":"2024-05-18T22:27:14.372972+0000","172.21.15.12:0/4241094239":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/4172568611":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/3512529242":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6800/1039789215":"2024-05-18T22:26:45.466719+0000","172.21.15.12:6800/2250046521":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6801/1039789215":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/2043758453":"2024-05-18T22:27:14.372972+0000","172.21.15.12:0/2694593732":"2024-05-18T22:26:45.466719+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T22:55:06.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.614+0000 7f8e29247640 1 -- 172.21.15.12:0/1754896200 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8e100b27f0 msgr2=0x7f8e100b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:06.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.614+0000 7f8e29247640 1 --2- 172.21.15.12:0/1754896200 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8e100b27f0 0x7f8e100b4d00 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8e2406a9b0 tx=0x7f8e14009680 comp rx=0 tx=0).stop 2024-05-17T22:55:06.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.614+0000 7f8e29247640 1 -- 172.21.15.12:0/1754896200 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e24108df0 msgr2=0x7f8e241a0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:06.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.614+0000 7f8e29247640 1 --2- 172.21.15.12:0/1754896200 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e24108df0 0x7f8e241a0d20 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c009230 tx=0x7f8e1c031a50 comp rx=0 tx=0).stop 2024-05-17T22:55:06.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.614+0000 7f8e29247640 1 -- 172.21.15.12:0/1754896200 shutdown_connections 2024-05-17T22:55:06.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.614+0000 7f8e29247640 1 --2- 172.21.15.12:0/1754896200 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f8e100b27f0 0x7f8e100b4d00 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:06.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.614+0000 7f8e29247640 1 --2- 172.21.15.12:0/1754896200 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e24108df0 0x7f8e241a0d20 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:06.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.614+0000 7f8e29247640 1 --2- 172.21.15.12:0/1754896200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e241062b0 0x7f8e24199d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:06.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.614+0000 7f8e29247640 1 --2- 172.21.15.12:0/1754896200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e2406acc0 0x7f8e24199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:06.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.614+0000 7f8e29247640 1 -- 172.21.15.12:0/1754896200 >> 172.21.15.12:0/1754896200 conn(0x7f8e240fd860 msgr2=0x7f8e241044f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:55:06.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.614+0000 7f8e29247640 1 -- 172.21.15.12:0/1754896200 shutdown_connections 2024-05-17T22:55:06.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.614+0000 7f8e29247640 1 -- 172.21.15.12:0/1754896200 wait complete. 2024-05-17T22:55:06.629 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-17T22:29:42.475513+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-17T22:34:35.504697+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-17T22:34:39.377097+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-17T22:34:50.467684+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-17T22:55:06.630 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-17T22:55:06.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.746+0000 7f9bb8791640 1 -- 172.21.15.12:0/4052481725 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bb0103160 msgr2=0x7f9bb0103570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:06.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.746+0000 7f9bb8791640 1 --2- 172.21.15.12:0/4052481725 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bb0103160 0x7f9bb0103570 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0009900 tx=0x7f9ba002a410 comp rx=0 tx=0).stop 2024-05-17T22:55:06.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.746+0000 7f9bb8791640 1 -- 172.21.15.12:0/4052481725 shutdown_connections 2024-05-17T22:55:06.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.746+0000 7f9bb8791640 1 --2- 172.21.15.12:0/4052481725 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bb0103ab0 0x7f9bb010a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:06.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.746+0000 7f9bb8791640 1 --2- 172.21.15.12:0/4052481725 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bb0103160 0x7f9bb0103570 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:06.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.746+0000 7f9bb8791640 1 --2- 172.21.15.12:0/4052481725 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bb0102040 0x7f9bb0102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:06.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.746+0000 7f9bb8791640 1 -- 172.21.15.12:0/4052481725 >> 172.21.15.12:0/4052481725 conn(0x7f9bb00fd860 msgr2=0x7f9bb00ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:55:06.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.746+0000 7f9bb8791640 1 -- 172.21.15.12:0/4052481725 shutdown_connections 2024-05-17T22:55:06.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.747+0000 7f9bb8791640 1 -- 172.21.15.12:0/4052481725 wait complete. 2024-05-17T22:55:06.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.747+0000 7f9bb8791640 1 Processor -- start 2024-05-17T22:55:06.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.747+0000 7f9bb8791640 1 -- start start 2024-05-17T22:55:06.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.747+0000 7f9bb8791640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bb0102040 0x7f9bb006bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:06.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.747+0000 7f9bb8791640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bb0103160 0x7f9bb019e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:06.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.747+0000 7f9bb8791640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bb0103ab0 0x7f9bb01a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:06.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.747+0000 7f9bb8791640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb010dbc0 con 0x7f9bb0102040 2024-05-17T22:55:06.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.747+0000 7f9bb8791640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb010da40 con 0x7f9bb0103ab0 2024-05-17T22:55:06.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.747+0000 7f9bb8791640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb010dd40 con 0x7f9bb0103160 2024-05-17T22:55:06.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.747+0000 7f9bb652d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bb0102040 0x7f9bb006bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:06.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.748+0000 7f9bb652d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bb0102040 0x7f9bb006bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47564/0 (socket says 172.21.15.12:47564) 2024-05-17T22:55:06.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.748+0000 7f9bb652d640 1 -- 172.21.15.12:0/946707805 learned_addr learned my addr 172.21.15.12:0/946707805 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:55:06.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.748+0000 7f9bb6d2e640 1 --2- 172.21.15.12:0/946707805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bb0103ab0 0x7f9bb01a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:06.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.748+0000 7f9bb5d2c640 1 --2- 172.21.15.12:0/946707805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bb0103160 0x7f9bb019e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:06.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.748+0000 7f9bb652d640 1 -- 172.21.15.12:0/946707805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bb0103160 msgr2=0x7f9bb019e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:06.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.748+0000 7f9bb652d640 1 --2- 172.21.15.12:0/946707805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bb0103160 0x7f9bb019e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:06.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.748+0000 7f9bb652d640 1 -- 172.21.15.12:0/946707805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bb0103ab0 msgr2=0x7f9bb01a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:06.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.748+0000 7f9bb652d640 1 --2- 172.21.15.12:0/946707805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bb0103ab0 0x7f9bb01a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:06.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.748+0000 7f9bb652d640 1 -- 172.21.15.12:0/946707805 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bb01a1400 con 0x7f9bb0102040 2024-05-17T22:55:06.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.748+0000 7f9bb5d2c640 1 --2- 172.21.15.12:0/946707805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bb0103160 0x7f9bb019e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:55:06.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.748+0000 7f9bb652d640 1 --2- 172.21.15.12:0/946707805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bb0102040 0x7f9bb006bcd0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f9bac005fd0 tx=0x7f9bac00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:55:06.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.748+0000 7f9ba77fe640 1 -- 172.21.15.12:0/946707805 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bac017070 con 0x7f9bb0102040 2024-05-17T22:55:06.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.748+0000 7f9bb8791640 1 -- 172.21.15.12:0/946707805 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bb01a16f0 con 0x7f9bb0102040 2024-05-17T22:55:06.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.748+0000 7f9ba77fe640 1 -- 172.21.15.12:0/946707805 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bac009c30 con 0x7f9bb0102040 2024-05-17T22:55:06.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.748+0000 7f9bb8791640 1 -- 172.21.15.12:0/946707805 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bb01a1c90 con 0x7f9bb0102040 2024-05-17T22:55:06.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.748+0000 7f9ba77fe640 1 -- 172.21.15.12:0/946707805 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bac003d30 con 0x7f9bb0102040 2024-05-17T22:55:06.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.749+0000 7f9bb8791640 1 -- 172.21.15.12:0/946707805 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bb0109230 con 0x7f9bb0102040 2024-05-17T22:55:06.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.751+0000 7f9ba77fe640 1 -- 172.21.15.12:0/946707805 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9bac003ed0 con 0x7f9bb0102040 2024-05-17T22:55:06.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.752+0000 7f9ba77fe640 1 --2- 172.21.15.12:0/946707805 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9b9c0b27f0 0x7f9b9c0b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:06.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.752+0000 7f9bb5d2c640 1 --2- 172.21.15.12:0/946707805 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9b9c0b27f0 0x7f9b9c0b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:06.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.752+0000 7f9ba77fe640 1 -- 172.21.15.12:0/946707805 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f9bac0e2a60 con 0x7f9bb0102040 2024-05-17T22:55:06.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.752+0000 7f9bb5d2c640 1 --2- 172.21.15.12:0/946707805 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9b9c0b27f0 0x7f9b9c0b4d00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9ba002a930 tx=0x7f9ba0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:55:06.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.755+0000 7f9ba77fe640 1 -- 172.21.15.12:0/946707805 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9bac020030 con 0x7f9bb0102040 2024-05-17T22:55:06.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.947+0000 7f9bb8791640 1 -- 172.21.15.12:0/946707805 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9bb002e060 con 0x7f9bb0102040 2024-05-17T22:55:06.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.948+0000 7f9ba77fe640 1 -- 172.21.15.12:0/946707805 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f9bac009de0 con 0x7f9bb0102040 2024-05-17T22:55:06.950 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-05-17T22:55:06.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.950+0000 7f9bb8791640 1 -- 172.21.15.12:0/946707805 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9b9c0b27f0 msgr2=0x7f9b9c0b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:06.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.950+0000 7f9bb8791640 1 --2- 172.21.15.12:0/946707805 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9b9c0b27f0 0x7f9b9c0b4d00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9ba002a930 tx=0x7f9ba0005fd0 comp rx=0 tx=0).stop 2024-05-17T22:55:06.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.950+0000 7f9bb8791640 1 -- 172.21.15.12:0/946707805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bb0102040 msgr2=0x7f9bb006bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:06.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.950+0000 7f9bb8791640 1 --2- 172.21.15.12:0/946707805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bb0102040 0x7f9bb006bcd0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f9bac005fd0 tx=0x7f9bac00dde0 comp rx=0 tx=0).stop 2024-05-17T22:55:06.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.950+0000 7f9bb8791640 1 -- 172.21.15.12:0/946707805 shutdown_connections 2024-05-17T22:55:06.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.950+0000 7f9bb8791640 1 --2- 172.21.15.12:0/946707805 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f9b9c0b27f0 0x7f9b9c0b4d00 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:06.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.950+0000 7f9bb8791640 1 --2- 172.21.15.12:0/946707805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bb0103ab0 0x7f9bb01a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:06.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.950+0000 7f9bb8791640 1 --2- 172.21.15.12:0/946707805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bb0103160 0x7f9bb019e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:06.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.950+0000 7f9bb8791640 1 --2- 172.21.15.12:0/946707805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bb0102040 0x7f9bb006bcd0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:06.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.950+0000 7f9bb8791640 1 -- 172.21.15.12:0/946707805 >> 172.21.15.12:0/946707805 conn(0x7f9bb00fd860 msgr2=0x7f9bb0106400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:55:06.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.950+0000 7f9bb8791640 1 -- 172.21.15.12:0/946707805 shutdown_connections 2024-05-17T22:55:06.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:06.950+0000 7f9bb8791640 1 -- 172.21.15.12:0/946707805 wait complete. 2024-05-17T22:55:06.964 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-17T22:55:07.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.080+0000 7f6b73a3c640 1 -- 172.21.15.12:0/4104838439 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b6c1065a0 msgr2=0x7f6b6c1089c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:07.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.080+0000 7f6b73a3c640 1 --2- 172.21.15.12:0/4104838439 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b6c1065a0 0x7f6b6c1089c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c009900 tx=0x7f6b5c02a410 comp rx=0 tx=0).stop 2024-05-17T22:55:07.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.080+0000 7f6b73a3c640 1 -- 172.21.15.12:0/4104838439 shutdown_connections 2024-05-17T22:55:07.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.080+0000 7f6b73a3c640 1 --2- 172.21.15.12:0/4104838439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b6c108f00 0x7f6b6c10b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:07.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.080+0000 7f6b73a3c640 1 --2- 172.21.15.12:0/4104838439 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b6c1065a0 0x7f6b6c1089c0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:07.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.080+0000 7f6b73a3c640 1 --2- 172.21.15.12:0/4104838439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b6c1014c0 0x7f6b6c1018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:07.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.080+0000 7f6b73a3c640 1 -- 172.21.15.12:0/4104838439 >> 172.21.15.12:0/4104838439 conn(0x7f6b6c0fcfe0 msgr2=0x7f6b6c0ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:55:07.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.080+0000 7f6b73a3c640 1 -- 172.21.15.12:0/4104838439 shutdown_connections 2024-05-17T22:55:07.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.080+0000 7f6b73a3c640 1 -- 172.21.15.12:0/4104838439 wait complete. 2024-05-17T22:55:07.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.081+0000 7f6b73a3c640 1 Processor -- start 2024-05-17T22:55:07.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.081+0000 7f6b73a3c640 1 -- start start 2024-05-17T22:55:07.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.081+0000 7f6b73a3c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b6c1014c0 0x7f6b6c199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:07.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.081+0000 7f6b73a3c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b6c1065a0 0x7f6b6c199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:07.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.081+0000 7f6b73a3c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b6c108f00 0x7f6b6c1a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:07.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.081+0000 7f6b73a3c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6b6c10ecc0 con 0x7f6b6c1014c0 2024-05-17T22:55:07.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.081+0000 7f6b73a3c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6b6c10eb40 con 0x7f6b6c108f00 2024-05-17T22:55:07.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.081+0000 7f6b73a3c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6b6c10ee40 con 0x7f6b6c1065a0 2024-05-17T22:55:07.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.081+0000 7f6b717d8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b6c1014c0 0x7f6b6c199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:07.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.081+0000 7f6b717d8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b6c1014c0 0x7f6b6c199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47582/0 (socket says 172.21.15.12:47582) 2024-05-17T22:55:07.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.081+0000 7f6b717d8640 1 -- 172.21.15.12:0/2607710169 learned_addr learned my addr 172.21.15.12:0/2607710169 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:55:07.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.082+0000 7f6b71fd9640 1 --2- 172.21.15.12:0/2607710169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b6c108f00 0x7f6b6c1a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:07.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.082+0000 7f6b70fd7640 1 --2- 172.21.15.12:0/2607710169 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b6c1065a0 0x7f6b6c199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:07.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.082+0000 7f6b717d8640 1 -- 172.21.15.12:0/2607710169 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b6c1065a0 msgr2=0x7f6b6c199d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:07.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.082+0000 7f6b717d8640 1 --2- 172.21.15.12:0/2607710169 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b6c1065a0 0x7f6b6c199d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:07.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.082+0000 7f6b717d8640 1 -- 172.21.15.12:0/2607710169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b6c108f00 msgr2=0x7f6b6c1a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:07.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.082+0000 7f6b717d8640 1 --2- 172.21.15.12:0/2607710169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b6c108f00 0x7f6b6c1a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:07.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.082+0000 7f6b717d8640 1 -- 172.21.15.12:0/2607710169 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b6c1a1370 con 0x7f6b6c1014c0 2024-05-17T22:55:07.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.082+0000 7f6b717d8640 1 --2- 172.21.15.12:0/2607710169 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b6c1014c0 0x7f6b6c199830 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6b6800e9f0 tx=0x7f6b6800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:55:07.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.082+0000 7f6b627fc640 1 -- 172.21.15.12:0/2607710169 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6800cd80 con 0x7f6b6c1014c0 2024-05-17T22:55:07.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.082+0000 7f6b627fc640 1 -- 172.21.15.12:0/2607710169 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b680108e0 con 0x7f6b6c1014c0 2024-05-17T22:55:07.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.082+0000 7f6b627fc640 1 -- 172.21.15.12:0/2607710169 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b68018980 con 0x7f6b6c1014c0 2024-05-17T22:55:07.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.082+0000 7f6b73a3c640 1 -- 172.21.15.12:0/2607710169 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b6c1a1660 con 0x7f6b6c1014c0 2024-05-17T22:55:07.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.082+0000 7f6b73a3c640 1 -- 172.21.15.12:0/2607710169 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b6c1a1bb0 con 0x7f6b6c1014c0 2024-05-17T22:55:07.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.083+0000 7f6b73a3c640 1 -- 172.21.15.12:0/2607710169 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b6c0feb00 con 0x7f6b6c1014c0 2024-05-17T22:55:07.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.085+0000 7f6b627fc640 1 -- 172.21.15.12:0/2607710169 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6b68010430 con 0x7f6b6c1014c0 2024-05-17T22:55:07.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.085+0000 7f6b627fc640 1 --2- 172.21.15.12:0/2607710169 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f6b580b27f0 0x7f6b580b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:07.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.086+0000 7f6b70fd7640 1 --2- 172.21.15.12:0/2607710169 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f6b580b27f0 0x7f6b580b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:07.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.086+0000 7f6b627fc640 1 -- 172.21.15.12:0/2607710169 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f6b680e2a80 con 0x7f6b6c1014c0 2024-05-17T22:55:07.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.086+0000 7f6b70fd7640 1 --2- 172.21.15.12:0/2607710169 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f6b580b27f0 0x7f6b580b4d00 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c006010 tx=0x7f6b5c031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:55:07.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.090+0000 7f6b627fc640 1 -- 172.21.15.12:0/2607710169 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6b680e4e20 con 0x7f6b6c1014c0 2024-05-17T22:55:07.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.282+0000 7f6b73a3c640 1 -- 172.21.15.12:0/2607710169 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6b6c063940 con 0x7f6b6c1014c0 2024-05-17T22:55:07.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.282+0000 7f6b627fc640 1 -- 172.21.15.12:0/2607710169 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f6b680aa3b0 con 0x7f6b6c1014c0 2024-05-17T22:55:07.285 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-05-17T22:55:07.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.284+0000 7f6b73a3c640 1 -- 172.21.15.12:0/2607710169 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f6b580b27f0 msgr2=0x7f6b580b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:07.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.284+0000 7f6b73a3c640 1 --2- 172.21.15.12:0/2607710169 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f6b580b27f0 0x7f6b580b4d00 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c006010 tx=0x7f6b5c031400 comp rx=0 tx=0).stop 2024-05-17T22:55:07.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.284+0000 7f6b73a3c640 1 -- 172.21.15.12:0/2607710169 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b6c1014c0 msgr2=0x7f6b6c199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:07.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.284+0000 7f6b73a3c640 1 --2- 172.21.15.12:0/2607710169 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b6c1014c0 0x7f6b6c199830 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6b6800e9f0 tx=0x7f6b6800eeb0 comp rx=0 tx=0).stop 2024-05-17T22:55:07.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.284+0000 7f6b73a3c640 1 -- 172.21.15.12:0/2607710169 shutdown_connections 2024-05-17T22:55:07.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.284+0000 7f6b73a3c640 1 --2- 172.21.15.12:0/2607710169 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f6b580b27f0 0x7f6b580b4d00 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:07.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.284+0000 7f6b73a3c640 1 --2- 172.21.15.12:0/2607710169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b6c108f00 0x7f6b6c1a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:07.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.284+0000 7f6b73a3c640 1 --2- 172.21.15.12:0/2607710169 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b6c1065a0 0x7f6b6c199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:07.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.284+0000 7f6b73a3c640 1 --2- 172.21.15.12:0/2607710169 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b6c1014c0 0x7f6b6c199830 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:07.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.284+0000 7f6b73a3c640 1 -- 172.21.15.12:0/2607710169 >> 172.21.15.12:0/2607710169 conn(0x7f6b6c0fcfe0 msgr2=0x7f6b6c107270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:55:07.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.284+0000 7f6b73a3c640 1 -- 172.21.15.12:0/2607710169 shutdown_connections 2024-05-17T22:55:07.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.284+0000 7f6b73a3c640 1 -- 172.21.15.12:0/2607710169 wait complete. 2024-05-17T22:55:07.297 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:07 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/1754896200' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:55:07.298 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:07 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/946707805' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-17T22:55:07.299 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-17T22:55:07.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.415+0000 7ff0e026d640 1 -- 172.21.15.12:0/1489010360 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0d8108ef0 msgr2=0x7ff0d810b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:07.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.415+0000 7ff0e026d640 1 --2- 172.21.15.12:0/1489010360 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0d8108ef0 0x7ff0d810b310 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0009900 tx=0x7ff0d002a3f0 comp rx=0 tx=0).stop 2024-05-17T22:55:07.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.415+0000 7ff0e026d640 1 -- 172.21.15.12:0/1489010360 shutdown_connections 2024-05-17T22:55:07.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.415+0000 7ff0e026d640 1 --2- 172.21.15.12:0/1489010360 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0d8108ef0 0x7ff0d810b310 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:07.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.415+0000 7ff0e026d640 1 --2- 172.21.15.12:0/1489010360 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0d8106590 0x7ff0d81089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:07.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.415+0000 7ff0e026d640 1 --2- 172.21.15.12:0/1489010360 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0d8103c30 0x7ff0d8106050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:07.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.415+0000 7ff0e026d640 1 -- 172.21.15.12:0/1489010360 >> 172.21.15.12:0/1489010360 conn(0x7ff0d80fd860 msgr2=0x7ff0d80ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:55:07.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.415+0000 7ff0e026d640 1 -- 172.21.15.12:0/1489010360 shutdown_connections 2024-05-17T22:55:07.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.415+0000 7ff0e026d640 1 -- 172.21.15.12:0/1489010360 wait complete. 2024-05-17T22:55:07.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.416+0000 7ff0e026d640 1 Processor -- start 2024-05-17T22:55:07.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.416+0000 7ff0e026d640 1 -- start start 2024-05-17T22:55:07.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.416+0000 7ff0e026d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0d8103c30 0x7ff0d8199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:07.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.416+0000 7ff0e026d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0d8106590 0x7ff0d8199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:07.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.416+0000 7ff0e026d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0d8108ef0 0x7ff0d81a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:07.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.416+0000 7ff0e026d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d810ec60 con 0x7ff0d8106590 2024-05-17T22:55:07.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.416+0000 7ff0e026d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d810eae0 con 0x7ff0d8103c30 2024-05-17T22:55:07.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.416+0000 7ff0e026d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d810ede0 con 0x7ff0d8108ef0 2024-05-17T22:55:07.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.416+0000 7ff0dd808640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0d8106590 0x7ff0d8199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:07.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.416+0000 7ff0dd808640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0d8106590 0x7ff0d8199d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47602/0 (socket says 172.21.15.12:47602) 2024-05-17T22:55:07.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.416+0000 7ff0dd808640 1 -- 172.21.15.12:0/613988101 learned_addr learned my addr 172.21.15.12:0/613988101 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:55:07.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.416+0000 7ff0de80a640 1 --2- 172.21.15.12:0/613988101 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0d8108ef0 0x7ff0d81a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:07.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.416+0000 7ff0de009640 1 --2- 172.21.15.12:0/613988101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0d8103c30 0x7ff0d8199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:07.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.416+0000 7ff0dd808640 1 -- 172.21.15.12:0/613988101 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0d8108ef0 msgr2=0x7ff0d81a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:07.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.416+0000 7ff0dd808640 1 --2- 172.21.15.12:0/613988101 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0d8108ef0 0x7ff0d81a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:07.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.416+0000 7ff0dd808640 1 -- 172.21.15.12:0/613988101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0d8103c30 msgr2=0x7ff0d8199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:07.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.416+0000 7ff0dd808640 1 --2- 172.21.15.12:0/613988101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0d8103c30 0x7ff0d8199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:07.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.416+0000 7ff0dd808640 1 -- 172.21.15.12:0/613988101 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d81a12c0 con 0x7ff0d8106590 2024-05-17T22:55:07.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.416+0000 7ff0dd808640 1 --2- 172.21.15.12:0/613988101 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0d8106590 0x7ff0d8199d70 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ff0c800c920 tx=0x7ff0c800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:55:07.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.417+0000 7ff0cf7fe640 1 -- 172.21.15.12:0/613988101 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8007c70 con 0x7ff0d8106590 2024-05-17T22:55:07.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.417+0000 7ff0cf7fe640 1 -- 172.21.15.12:0/613988101 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0c8007e10 con 0x7ff0d8106590 2024-05-17T22:55:07.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.417+0000 7ff0e026d640 1 -- 172.21.15.12:0/613988101 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0d81a15b0 con 0x7ff0d8106590 2024-05-17T22:55:07.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.417+0000 7ff0cf7fe640 1 -- 172.21.15.12:0/613988101 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8018820 con 0x7ff0d8106590 2024-05-17T22:55:07.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.417+0000 7ff0e026d640 1 -- 172.21.15.12:0/613988101 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0d81a1b50 con 0x7ff0d8106590 2024-05-17T22:55:07.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.418+0000 7ff0e026d640 1 -- 172.21.15.12:0/613988101 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0d80ff380 con 0x7ff0d8106590 2024-05-17T22:55:07.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.420+0000 7ff0cf7fe640 1 -- 172.21.15.12:0/613988101 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff0c800f450 con 0x7ff0d8106590 2024-05-17T22:55:07.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.420+0000 7ff0cf7fe640 1 --2- 172.21.15.12:0/613988101 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff0c40b2870 0x7ff0c40b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:07.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.420+0000 7ff0de009640 1 --2- 172.21.15.12:0/613988101 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff0c40b2870 0x7ff0c40b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:07.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.420+0000 7ff0cf7fe640 1 -- 172.21.15.12:0/613988101 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7ff0c80e29f0 con 0x7ff0d8106590 2024-05-17T22:55:07.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.421+0000 7ff0de009640 1 --2- 172.21.15.12:0/613988101 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff0c40b2870 0x7ff0c40b4d80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4005c40 tx=0x7ff0d400a780 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:55:07.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.423+0000 7ff0cf7fe640 1 -- 172.21.15.12:0/613988101 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff0c8014030 con 0x7ff0d8106590 2024-05-17T22:55:07.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:07 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/1754896200' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:55:07.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:07 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/946707805' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-17T22:55:07.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:07 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/1754896200' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:55:07.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:07 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/946707805' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-17T22:55:07.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.615+0000 7ff0e026d640 1 -- 172.21.15.12:0/613988101 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff0d8063940 con 0x7ff0d8106590 2024-05-17T22:55:07.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.616+0000 7ff0cf7fe640 1 -- 172.21.15.12:0/613988101 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7ff0c80aa490 con 0x7ff0d8106590 2024-05-17T22:55:07.618 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-05-17T22:55:07.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.618+0000 7ff0e026d640 1 -- 172.21.15.12:0/613988101 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff0c40b2870 msgr2=0x7ff0c40b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:07.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.618+0000 7ff0e026d640 1 --2- 172.21.15.12:0/613988101 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff0c40b2870 0x7ff0c40b4d80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4005c40 tx=0x7ff0d400a780 comp rx=0 tx=0).stop 2024-05-17T22:55:07.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.618+0000 7ff0e026d640 1 -- 172.21.15.12:0/613988101 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0d8106590 msgr2=0x7ff0d8199d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:07.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.618+0000 7ff0e026d640 1 --2- 172.21.15.12:0/613988101 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0d8106590 0x7ff0d8199d70 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ff0c800c920 tx=0x7ff0c800cde0 comp rx=0 tx=0).stop 2024-05-17T22:55:07.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.618+0000 7ff0e026d640 1 -- 172.21.15.12:0/613988101 shutdown_connections 2024-05-17T22:55:07.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.618+0000 7ff0e026d640 1 --2- 172.21.15.12:0/613988101 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff0c40b2870 0x7ff0c40b4d80 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:07.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.618+0000 7ff0e026d640 1 --2- 172.21.15.12:0/613988101 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0d8108ef0 0x7ff0d81a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:07.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.618+0000 7ff0e026d640 1 --2- 172.21.15.12:0/613988101 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0d8106590 0x7ff0d8199d70 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:07.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.618+0000 7ff0e026d640 1 --2- 172.21.15.12:0/613988101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0d8103c30 0x7ff0d8199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:07.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.618+0000 7ff0e026d640 1 -- 172.21.15.12:0/613988101 >> 172.21.15.12:0/613988101 conn(0x7ff0d80fd860 msgr2=0x7ff0d81048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:55:07.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.618+0000 7ff0e026d640 1 -- 172.21.15.12:0/613988101 shutdown_connections 2024-05-17T22:55:07.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.618+0000 7ff0e026d640 1 -- 172.21.15.12:0/613988101 wait complete. 2024-05-17T22:55:07.632 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-17T22:55:07.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.748+0000 7fca7bc97640 1 -- 172.21.15.12:0/107721822 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca74102040 msgr2=0x7fca74102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:07.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.748+0000 7fca7bc97640 1 --2- 172.21.15.12:0/107721822 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca74102040 0x7fca74102450 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fca70009900 tx=0x7fca7002a410 comp rx=0 tx=0).stop 2024-05-17T22:55:07.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.748+0000 7fca7bc97640 1 -- 172.21.15.12:0/107721822 shutdown_connections 2024-05-17T22:55:07.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.748+0000 7fca7bc97640 1 --2- 172.21.15.12:0/107721822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca74103ab0 0x7fca7410a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:07.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.748+0000 7fca7bc97640 1 --2- 172.21.15.12:0/107721822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca74103160 0x7fca74103570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:07.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.748+0000 7fca7bc97640 1 --2- 172.21.15.12:0/107721822 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca74102040 0x7fca74102450 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:07.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.748+0000 7fca7bc97640 1 -- 172.21.15.12:0/107721822 >> 172.21.15.12:0/107721822 conn(0x7fca740fd860 msgr2=0x7fca740ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:55:07.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.748+0000 7fca7bc97640 1 -- 172.21.15.12:0/107721822 shutdown_connections 2024-05-17T22:55:07.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.748+0000 7fca7bc97640 1 -- 172.21.15.12:0/107721822 wait complete. 2024-05-17T22:55:07.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.749+0000 7fca7bc97640 1 Processor -- start 2024-05-17T22:55:07.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.749+0000 7fca7bc97640 1 -- start start 2024-05-17T22:55:07.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.749+0000 7fca7bc97640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca74102040 0x7fca741998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:07.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.749+0000 7fca7bc97640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca74103160 0x7fca74199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:07.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.749+0000 7fca7bc97640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca74103ab0 0x7fca741a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:07.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.749+0000 7fca7bc97640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fca7410ec80 con 0x7fca74102040 2024-05-17T22:55:07.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.749+0000 7fca7bc97640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fca7410eb00 con 0x7fca74103160 2024-05-17T22:55:07.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.749+0000 7fca7bc97640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fca7410ee00 con 0x7fca74103ab0 2024-05-17T22:55:07.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.749+0000 7fca79a33640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca74102040 0x7fca741998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:07.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.749+0000 7fca79a33640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca74102040 0x7fca741998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47614/0 (socket says 172.21.15.12:47614) 2024-05-17T22:55:07.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.749+0000 7fca79a33640 1 -- 172.21.15.12:0/519088477 learned_addr learned my addr 172.21.15.12:0/519088477 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:55:07.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.749+0000 7fca79232640 1 --2- 172.21.15.12:0/519088477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca74103160 0x7fca74199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:07.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.749+0000 7fca7a234640 1 --2- 172.21.15.12:0/519088477 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca74103ab0 0x7fca741a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:07.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.750+0000 7fca79a33640 1 -- 172.21.15.12:0/519088477 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca74103ab0 msgr2=0x7fca741a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:07.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.750+0000 7fca79a33640 1 --2- 172.21.15.12:0/519088477 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca74103ab0 0x7fca741a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:07.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.750+0000 7fca79a33640 1 -- 172.21.15.12:0/519088477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca74103160 msgr2=0x7fca74199df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:07.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.750+0000 7fca79a33640 1 --2- 172.21.15.12:0/519088477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca74103160 0x7fca74199df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:07.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.750+0000 7fca79a33640 1 -- 172.21.15.12:0/519088477 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca741a13f0 con 0x7fca74102040 2024-05-17T22:55:07.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.750+0000 7fca7a234640 1 --2- 172.21.15.12:0/519088477 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca74103ab0 0x7fca741a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:55:07.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.750+0000 7fca79232640 1 --2- 172.21.15.12:0/519088477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca74103160 0x7fca74199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:55:07.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.750+0000 7fca79a33640 1 --2- 172.21.15.12:0/519088477 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca74102040 0x7fca741998b0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fca70006010 tx=0x7fca70004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:55:07.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.750+0000 7fca6affd640 1 -- 172.21.15.12:0/519088477 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7003f030 con 0x7fca74102040 2024-05-17T22:55:07.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.750+0000 7fca6affd640 1 -- 172.21.15.12:0/519088477 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca70003aa0 con 0x7fca74102040 2024-05-17T22:55:07.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.750+0000 7fca7bc97640 1 -- 172.21.15.12:0/519088477 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca741a1680 con 0x7fca74102040 2024-05-17T22:55:07.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.750+0000 7fca6affd640 1 -- 172.21.15.12:0/519088477 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca70035070 con 0x7fca74102040 2024-05-17T22:55:07.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.750+0000 7fca7bc97640 1 -- 172.21.15.12:0/519088477 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca741a1b70 con 0x7fca74102040 2024-05-17T22:55:07.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.751+0000 7fca7bc97640 1 -- 172.21.15.12:0/519088477 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca74109290 con 0x7fca74102040 2024-05-17T22:55:07.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.753+0000 7fca6affd640 1 -- 172.21.15.12:0/519088477 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fca70042030 con 0x7fca74102040 2024-05-17T22:55:07.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.754+0000 7fca6affd640 1 --2- 172.21.15.12:0/519088477 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fca600b27f0 0x7fca600b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:07.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.754+0000 7fca79232640 1 --2- 172.21.15.12:0/519088477 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fca600b27f0 0x7fca600b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:07.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.754+0000 7fca6affd640 1 -- 172.21.15.12:0/519088477 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fca701031d0 con 0x7fca74102040 2024-05-17T22:55:07.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.755+0000 7fca79232640 1 --2- 172.21.15.12:0/519088477 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fca600b27f0 0x7fca600b4d00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fca7419a950 tx=0x7fca64005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:55:07.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.757+0000 7fca6affd640 1 -- 172.21.15.12:0/519088477 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fca700cab60 con 0x7fca74102040 2024-05-17T22:55:07.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.948+0000 7fca7bc97640 1 -- 172.21.15.12:0/519088477 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fca74063940 con 0x7fca74102040 2024-05-17T22:55:07.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.949+0000 7fca6affd640 1 -- 172.21.15.12:0/519088477 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) ==== 103+0+11 (secure 0 0 0) 0x7fca700ca460 con 0x7fca74102040 2024-05-17T22:55:07.951 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-05-17T22:55:07.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.951+0000 7fca7bc97640 1 -- 172.21.15.12:0/519088477 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fca600b27f0 msgr2=0x7fca600b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:07.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.951+0000 7fca7bc97640 1 --2- 172.21.15.12:0/519088477 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fca600b27f0 0x7fca600b4d00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fca7419a950 tx=0x7fca64005f90 comp rx=0 tx=0).stop 2024-05-17T22:55:07.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.951+0000 7fca7bc97640 1 -- 172.21.15.12:0/519088477 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca74102040 msgr2=0x7fca741998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:07.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.951+0000 7fca7bc97640 1 --2- 172.21.15.12:0/519088477 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca74102040 0x7fca741998b0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fca70006010 tx=0x7fca70004270 comp rx=0 tx=0).stop 2024-05-17T22:55:07.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.951+0000 7fca7bc97640 1 -- 172.21.15.12:0/519088477 shutdown_connections 2024-05-17T22:55:07.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.951+0000 7fca7bc97640 1 --2- 172.21.15.12:0/519088477 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fca600b27f0 0x7fca600b4d00 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:07.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.951+0000 7fca7bc97640 1 --2- 172.21.15.12:0/519088477 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca74103ab0 0x7fca741a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:07.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.951+0000 7fca7bc97640 1 --2- 172.21.15.12:0/519088477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca74103160 0x7fca74199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:07.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.951+0000 7fca7bc97640 1 --2- 172.21.15.12:0/519088477 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca74102040 0x7fca741998b0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:07.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.951+0000 7fca7bc97640 1 -- 172.21.15.12:0/519088477 >> 172.21.15.12:0/519088477 conn(0x7fca740fd860 msgr2=0x7fca74106460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:55:07.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.951+0000 7fca7bc97640 1 -- 172.21.15.12:0/519088477 shutdown_connections 2024-05-17T22:55:07.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:07.951+0000 7fca7bc97640 1 -- 172.21.15.12:0/519088477 wait complete. 2024-05-17T22:55:07.965 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-17T22:55:08.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.077+0000 7fd989ef6640 1 -- 172.21.15.12:0/2487036784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd984104270 msgr2=0x7fd984104680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:08.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.077+0000 7fd989ef6640 1 --2- 172.21.15.12:0/2487036784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd984104270 0x7fd984104680 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fd978009900 tx=0x7fd97802a410 comp rx=0 tx=0).stop 2024-05-17T22:55:08.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.077+0000 7fd989ef6640 1 -- 172.21.15.12:0/2487036784 shutdown_connections 2024-05-17T22:55:08.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.077+0000 7fd989ef6640 1 --2- 172.21.15.12:0/2487036784 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd984104bc0 0x7fd98410b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:08.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.077+0000 7fd989ef6640 1 --2- 172.21.15.12:0/2487036784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd984104270 0x7fd984104680 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:08.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.077+0000 7fd989ef6640 1 --2- 172.21.15.12:0/2487036784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd984103150 0x7fd984103560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:08.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.077+0000 7fd989ef6640 1 -- 172.21.15.12:0/2487036784 >> 172.21.15.12:0/2487036784 conn(0x7fd9840fe950 msgr2=0x7fd984100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:55:08.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.077+0000 7fd989ef6640 1 -- 172.21.15.12:0/2487036784 shutdown_connections 2024-05-17T22:55:08.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.077+0000 7fd989ef6640 1 -- 172.21.15.12:0/2487036784 wait complete. 2024-05-17T22:55:08.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.078+0000 7fd989ef6640 1 Processor -- start 2024-05-17T22:55:08.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.078+0000 7fd989ef6640 1 -- start start 2024-05-17T22:55:08.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.078+0000 7fd989ef6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd984103150 0x7fd9841998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:08.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.078+0000 7fd989ef6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd984104270 0x7fd984199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:08.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.078+0000 7fd989ef6640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd984104bc0 0x7fd9841a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:08.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.078+0000 7fd989ef6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd98410ec80 con 0x7fd984104270 2024-05-17T22:55:08.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.078+0000 7fd989ef6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd98410eb00 con 0x7fd984103150 2024-05-17T22:55:08.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.078+0000 7fd989ef6640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd98410ee00 con 0x7fd984104bc0 2024-05-17T22:55:08.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.079+0000 7fd982ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd984104270 0x7fd984199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:08.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.079+0000 7fd982ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd984104270 0x7fd984199e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47632/0 (socket says 172.21.15.12:47632) 2024-05-17T22:55:08.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.079+0000 7fd982ffd640 1 -- 172.21.15.12:0/2043518230 learned_addr learned my addr 172.21.15.12:0/2043518230 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:55:08.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.079+0000 7fd9837fe640 1 --2- 172.21.15.12:0/2043518230 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd984103150 0x7fd9841998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:08.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.079+0000 7fd983fff640 1 --2- 172.21.15.12:0/2043518230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd984104bc0 0x7fd9841a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:08.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.079+0000 7fd982ffd640 1 -- 172.21.15.12:0/2043518230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd984104bc0 msgr2=0x7fd9841a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:08.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.079+0000 7fd982ffd640 1 --2- 172.21.15.12:0/2043518230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd984104bc0 0x7fd9841a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:08.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.079+0000 7fd982ffd640 1 -- 172.21.15.12:0/2043518230 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd984103150 msgr2=0x7fd9841998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:08.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.079+0000 7fd982ffd640 1 --2- 172.21.15.12:0/2043518230 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd984103150 0x7fd9841998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:08.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.079+0000 7fd982ffd640 1 -- 172.21.15.12:0/2043518230 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9841a1410 con 0x7fd984104270 2024-05-17T22:55:08.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.079+0000 7fd9837fe640 1 --2- 172.21.15.12:0/2043518230 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd984103150 0x7fd9841998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:55:08.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.079+0000 7fd982ffd640 1 --2- 172.21.15.12:0/2043518230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd984104270 0x7fd984199e10 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fd978006010 tx=0x7fd978004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:55:08.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.079+0000 7fd980ff9640 1 -- 172.21.15.12:0/2043518230 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97803f030 con 0x7fd984104270 2024-05-17T22:55:08.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.079+0000 7fd980ff9640 1 -- 172.21.15.12:0/2043518230 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd978003cb0 con 0x7fd984104270 2024-05-17T22:55:08.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.079+0000 7fd989ef6640 1 -- 172.21.15.12:0/2043518230 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9841a16a0 con 0x7fd984104270 2024-05-17T22:55:08.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.079+0000 7fd980ff9640 1 -- 172.21.15.12:0/2043518230 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd978035070 con 0x7fd984104270 2024-05-17T22:55:08.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.079+0000 7fd989ef6640 1 -- 172.21.15.12:0/2043518230 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9841a1b90 con 0x7fd984104270 2024-05-17T22:55:08.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.081+0000 7fd989ef6640 1 -- 172.21.15.12:0/2043518230 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd98410a3a0 con 0x7fd984104270 2024-05-17T22:55:08.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.082+0000 7fd980ff9640 1 -- 172.21.15.12:0/2043518230 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd978042030 con 0x7fd984104270 2024-05-17T22:55:08.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.083+0000 7fd980ff9640 1 --2- 172.21.15.12:0/2043518230 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd96c0b27f0 0x7fd96c0b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:08.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.083+0000 7fd9837fe640 1 --2- 172.21.15.12:0/2043518230 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd96c0b27f0 0x7fd96c0b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:08.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.083+0000 7fd980ff9640 1 -- 172.21.15.12:0/2043518230 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fd978102d60 con 0x7fd984104270 2024-05-17T22:55:08.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.083+0000 7fd9837fe640 1 --2- 172.21.15.12:0/2043518230 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd96c0b27f0 0x7fd96c0b4d00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd9740058c0 tx=0x7fd974005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:55:08.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.087+0000 7fd980ff9640 1 -- 172.21.15.12:0/2043518230 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd978003e60 con 0x7fd984104270 2024-05-17T22:55:08.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.328+0000 7fd989ef6640 1 -- 172.21.15.12:0/2043518230 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fd984063940 con 0x7fd984104270 2024-05-17T22:55:08.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.337+0000 7fd980ff9640 1 -- 172.21.15.12:0/2043518230 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+815642 (secure 0 0 0) 0x7fd9780ca690 con 0x7fd984104270 2024-05-17T22:55:08.343 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:55:08.343 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:55:08.343 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 18, 2024-05-17T22:55:08.343 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.343 INFO:teuthology.orchestra.run.smithi012.stdout: "active_gid": 14152, 2024-05-17T22:55:08.343 INFO:teuthology.orchestra.run.smithi012.stdout: "active_name": "x", 2024-05-17T22:55:08.344 INFO:teuthology.orchestra.run.smithi012.stdout: "active_addrs": { 2024-05-17T22:55:08.344 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-05-17T22:55:08.344 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.344 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-05-17T22:55:08.344 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6800", 2024-05-17T22:55:08.344 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1388514047 2024-05-17T22:55:08.344 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.344 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.344 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-05-17T22:55:08.344 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6801", 2024-05-17T22:55:08.344 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1388514047 2024-05-17T22:55:08.344 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.344 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-05-17T22:55:08.344 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.345 INFO:teuthology.orchestra.run.smithi012.stdout: "active_addr": "172.21.15.12:6801/1388514047", 2024-05-17T22:55:08.345 INFO:teuthology.orchestra.run.smithi012.stdout: "active_change": "2024-05-17T22:27:14.373097+0000", 2024-05-17T22:55:08.345 INFO:teuthology.orchestra.run.smithi012.stdout: "active_mgr_features": 4540701547738038271, 2024-05-17T22:55:08.345 INFO:teuthology.orchestra.run.smithi012.stdout: "available": true, 2024-05-17T22:55:08.345 INFO:teuthology.orchestra.run.smithi012.stdout: "standbys": [ 2024-05-17T22:55:08.345 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.345 INFO:teuthology.orchestra.run.smithi012.stdout: "gid": 24109, 2024-05-17T22:55:08.345 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "z", 2024-05-17T22:55:08.345 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_features": 4540701547738038271, 2024-05-17T22:55:08.345 INFO:teuthology.orchestra.run.smithi012.stdout: "available_modules": [ 2024-05-17T22:55:08.345 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.345 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "alerts", 2024-05-17T22:55:08.345 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:08.345 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:08.346 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:08.346 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-17T22:55:08.346 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-17T22:55:08.346 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:08.346 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.346 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.346 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-17T22:55:08.346 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.346 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.346 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.346 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How frequently to reexamine health status", 2024-05-17T22:55:08.346 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.347 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.347 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.347 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.347 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:08.347 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:08.347 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.347 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.347 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.347 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.347 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.347 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.347 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.347 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.348 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.348 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.348 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.348 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.348 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.348 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.348 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.348 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.348 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.348 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.348 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.348 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:08.348 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:08.349 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.349 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.349 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.349 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.349 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.349 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.349 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.349 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.349 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.349 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.349 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.349 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.349 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:08.349 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:08.350 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.350 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.350 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.350 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:08.350 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.350 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.350 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.350 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.350 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.350 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.350 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.350 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.350 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.350 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.351 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.351 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.351 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.351 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.351 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.351 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:08.351 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:08.351 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.351 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.351 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.351 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.351 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.351 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.351 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.352 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.352 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.352 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.352 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.352 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.352 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_destination": { 2024-05-17T22:55:08.352 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_destination", 2024-05-17T22:55:08.352 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.352 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.352 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.352 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.352 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.352 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.352 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.353 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Email address to send alerts to", 2024-05-17T22:55:08.353 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.353 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.353 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.353 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.353 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_from_name": { 2024-05-17T22:55:08.353 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_from_name", 2024-05-17T22:55:08.353 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.353 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.353 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.353 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "Ceph", 2024-05-17T22:55:08.353 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.353 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.353 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.354 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Email From: name", 2024-05-17T22:55:08.354 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.354 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.354 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.354 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.354 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_host": { 2024-05-17T22:55:08.354 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_host", 2024-05-17T22:55:08.354 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.354 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.354 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.354 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.354 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.354 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.355 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.355 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP server", 2024-05-17T22:55:08.355 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.355 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.355 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.355 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.355 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_password": { 2024-05-17T22:55:08.355 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_password", 2024-05-17T22:55:08.355 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.355 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.355 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.355 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.355 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.356 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.356 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.356 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Password to authenticate with", 2024-05-17T22:55:08.356 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.356 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.356 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.356 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.356 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_port": { 2024-05-17T22:55:08.356 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_port", 2024-05-17T22:55:08.356 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.357 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.357 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.357 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "465", 2024-05-17T22:55:08.357 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.357 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.357 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.357 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP port", 2024-05-17T22:55:08.357 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.357 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.357 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.357 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.357 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_sender": { 2024-05-17T22:55:08.357 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_sender", 2024-05-17T22:55:08.357 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.358 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.358 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.358 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.358 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.358 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.358 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.358 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP envelope sender", 2024-05-17T22:55:08.358 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.358 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.358 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.358 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.358 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_ssl": { 2024-05-17T22:55:08.358 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_ssl", 2024-05-17T22:55:08.359 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.359 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.359 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.359 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.359 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.359 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.359 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.359 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-17T22:55:08.359 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.359 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.359 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.359 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.359 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_user": { 2024-05-17T22:55:08.359 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_user", 2024-05-17T22:55:08.360 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.360 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.360 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.360 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.360 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.360 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.360 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.360 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "User to authenticate as", 2024-05-17T22:55:08.360 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.360 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.360 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.360 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.360 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:08.360 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:08.361 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.361 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:08.361 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.361 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.361 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.361 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.361 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.361 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.361 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.361 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.361 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.361 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.361 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.361 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.362 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.362 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "balancer", 2024-05-17T22:55:08.362 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:08.362 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:08.362 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:08.362 INFO:teuthology.orchestra.run.smithi012.stdout: "active": { 2024-05-17T22:55:08.362 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "active", 2024-05-17T22:55:08.362 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.362 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.362 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.362 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.362 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.362 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.362 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.363 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically balance PGs across cluster", 2024-05-17T22:55:08.363 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.363 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.363 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.363 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.363 INFO:teuthology.orchestra.run.smithi012.stdout: "begin_time": { 2024-05-17T22:55:08.363 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "begin_time", 2024-05-17T22:55:08.363 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.363 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.363 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.363 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0000", 2024-05-17T22:55:08.363 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.363 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.364 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.364 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "beginning time of day to automatically balance", 2024-05-17T22:55:08.364 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-17T22:55:08.364 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.364 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.364 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.364 INFO:teuthology.orchestra.run.smithi012.stdout: "begin_weekday": { 2024-05-17T22:55:08.364 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "begin_weekday", 2024-05-17T22:55:08.364 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-17T22:55:08.364 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.364 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.364 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.364 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0", 2024-05-17T22:55:08.364 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "6", 2024-05-17T22:55:08.365 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.365 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-17T22:55:08.365 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-17T22:55:08.365 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.365 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.365 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.365 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_max_iterations": { 2024-05-17T22:55:08.365 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_max_iterations", 2024-05-17T22:55:08.365 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-17T22:55:08.365 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.365 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.365 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "25", 2024-05-17T22:55:08.365 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-05-17T22:55:08.365 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "250", 2024-05-17T22:55:08.366 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.366 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-17T22:55:08.366 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.366 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.366 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.366 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.366 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_metrics": { 2024-05-17T22:55:08.366 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_metrics", 2024-05-17T22:55:08.366 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.366 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.366 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.366 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "pgs,objects,bytes", 2024-05-17T22:55:08.366 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.366 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.367 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.367 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-17T22:55:08.367 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-17T22:55:08.367 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.367 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.367 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.367 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_step": { 2024-05-17T22:55:08.367 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_step", 2024-05-17T22:55:08.367 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-17T22:55:08.367 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.367 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.367 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0.5", 2024-05-17T22:55:08.367 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0.001", 2024-05-17T22:55:08.368 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "0.999", 2024-05-17T22:55:08.368 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.368 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "aggressiveness of optimization", 2024-05-17T22:55:08.368 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-17T22:55:08.368 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.368 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.368 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.368 INFO:teuthology.orchestra.run.smithi012.stdout: "end_time": { 2024-05-17T22:55:08.368 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "end_time", 2024-05-17T22:55:08.368 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.368 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.368 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.368 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "2359", 2024-05-17T22:55:08.368 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.369 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.369 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.369 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "ending time of day to automatically balance", 2024-05-17T22:55:08.369 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-17T22:55:08.369 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.369 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.369 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.369 INFO:teuthology.orchestra.run.smithi012.stdout: "end_weekday": { 2024-05-17T22:55:08.369 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "end_weekday", 2024-05-17T22:55:08.369 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-17T22:55:08.369 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.369 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.369 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.370 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0", 2024-05-17T22:55:08.370 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "6", 2024-05-17T22:55:08.370 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.370 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-17T22:55:08.370 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-17T22:55:08.370 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.370 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.370 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.370 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:08.370 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:08.370 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.370 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.370 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.371 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.371 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.371 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.371 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.371 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.371 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.371 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.371 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.371 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.371 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.371 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.371 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.371 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.371 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.372 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.372 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.372 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:08.372 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:08.372 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.372 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.372 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.372 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.372 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.372 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.372 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.372 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.372 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.372 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.373 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.373 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.373 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:08.373 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:08.373 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.373 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.373 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.373 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:08.373 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.373 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.373 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.373 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.373 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.374 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.374 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.374 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.374 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.374 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.374 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.374 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.374 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.374 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.374 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.374 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:08.374 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:08.374 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.375 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.375 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.375 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.375 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.375 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.375 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.375 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.375 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.375 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.375 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.375 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.375 INFO:teuthology.orchestra.run.smithi012.stdout: "min_score": { 2024-05-17T22:55:08.375 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "min_score", 2024-05-17T22:55:08.375 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-17T22:55:08.375 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.376 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.376 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.376 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.376 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.376 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.376 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-17T22:55:08.376 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.376 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.376 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.376 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.376 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": { 2024-05-17T22:55:08.376 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mode", 2024-05-17T22:55:08.376 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.376 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.377 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.377 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "upmap", 2024-05-17T22:55:08.377 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.377 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.377 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.377 INFO:teuthology.orchestra.run.smithi012.stdout: "crush-compat", 2024-05-17T22:55:08.377 INFO:teuthology.orchestra.run.smithi012.stdout: "none", 2024-05-17T22:55:08.377 INFO:teuthology.orchestra.run.smithi012.stdout: "read", 2024-05-17T22:55:08.377 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap", 2024-05-17T22:55:08.377 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap-read" 2024-05-17T22:55:08.377 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.377 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Balancer mode", 2024-05-17T22:55:08.377 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.377 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.377 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.378 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.378 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_ids": { 2024-05-17T22:55:08.378 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pool_ids", 2024-05-17T22:55:08.378 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.378 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.378 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.378 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.378 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.378 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.378 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.378 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-17T22:55:08.379 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.379 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.379 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.379 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.379 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-17T22:55:08.379 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-17T22:55:08.379 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:08.379 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.379 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.379 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-17T22:55:08.380 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.380 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.380 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.380 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-17T22:55:08.380 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.380 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.380 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.380 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.380 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:08.380 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:08.380 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.380 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:08.380 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.380 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.381 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.381 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.381 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.381 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.381 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.381 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.381 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.381 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.381 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap_max_deviation": { 2024-05-17T22:55:08.381 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "upmap_max_deviation", 2024-05-17T22:55:08.381 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.381 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.381 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.381 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-05-17T22:55:08.382 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-05-17T22:55:08.382 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.382 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.382 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-17T22:55:08.382 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-17T22:55:08.382 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.382 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.382 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.382 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap_max_optimizations": { 2024-05-17T22:55:08.382 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "upmap_max_optimizations", 2024-05-17T22:55:08.382 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-17T22:55:08.382 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.382 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.383 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-17T22:55:08.383 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.383 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.383 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.383 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-17T22:55:08.383 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.383 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.383 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.383 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.383 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.383 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.383 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.383 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cephadm", 2024-05-17T22:55:08.384 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:08.384 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:08.384 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:08.384 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_down_multiplier": { 2024-05-17T22:55:08.384 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_down_multiplier", 2024-05-17T22:55:08.384 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-17T22:55:08.384 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.384 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.384 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3.0", 2024-05-17T22:55:08.384 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.384 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.384 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.384 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-17T22:55:08.385 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.385 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.385 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.385 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.385 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_refresh_rate": { 2024-05-17T22:55:08.385 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_refresh_rate", 2024-05-17T22:55:08.385 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:08.385 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.385 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.385 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "20", 2024-05-17T22:55:08.385 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.385 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.385 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.386 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-17T22:55:08.386 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.386 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.386 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.386 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.386 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_starting_port": { 2024-05-17T22:55:08.386 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_starting_port", 2024-05-17T22:55:08.386 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.386 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.386 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.386 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "4721", 2024-05-17T22:55:08.386 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.386 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.387 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.387 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-17T22:55:08.387 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.387 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.387 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.387 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.387 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_ptrace": { 2024-05-17T22:55:08.387 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_ptrace", 2024-05-17T22:55:08.387 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.387 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.387 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.387 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.387 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.387 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.388 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.388 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-17T22:55:08.388 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-17T22:55:08.388 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.388 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.388 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.388 INFO:teuthology.orchestra.run.smithi012.stdout: "autotune_interval": { 2024-05-17T22:55:08.388 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "autotune_interval", 2024-05-17T22:55:08.388 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:08.388 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.388 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.388 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-17T22:55:08.388 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.388 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.389 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.389 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-17T22:55:08.389 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.389 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.389 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.389 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.389 INFO:teuthology.orchestra.run.smithi012.stdout: "autotune_memory_target_ratio": { 2024-05-17T22:55:08.389 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "autotune_memory_target_ratio", 2024-05-17T22:55:08.389 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-17T22:55:08.389 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.389 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.389 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0.7", 2024-05-17T22:55:08.389 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.389 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.390 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.390 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-17T22:55:08.390 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.390 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.390 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.390 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.390 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm_log_destination": { 2024-05-17T22:55:08.390 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cephadm_log_destination", 2024-05-17T22:55:08.390 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.390 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.390 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.390 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.390 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.391 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.391 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.391 INFO:teuthology.orchestra.run.smithi012.stdout: "file", 2024-05-17T22:55:08.391 INFO:teuthology.orchestra.run.smithi012.stdout: "file,syslog", 2024-05-17T22:55:08.391 INFO:teuthology.orchestra.run.smithi012.stdout: "syslog" 2024-05-17T22:55:08.391 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.391 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-17T22:55:08.391 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.391 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.391 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.391 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.391 INFO:teuthology.orchestra.run.smithi012.stdout: "cgroups_split": { 2024-05-17T22:55:08.391 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cgroups_split", 2024-05-17T22:55:08.391 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.392 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.392 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.392 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.392 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.392 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.392 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.392 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-17T22:55:08.392 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.392 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.392 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.392 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.392 INFO:teuthology.orchestra.run.smithi012.stdout: "config_checks_enabled": { 2024-05-17T22:55:08.392 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_checks_enabled", 2024-05-17T22:55:08.393 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.393 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.393 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.393 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.393 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.393 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.393 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.393 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-17T22:55:08.393 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.393 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.393 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.393 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.393 INFO:teuthology.orchestra.run.smithi012.stdout: "config_dashboard": { 2024-05-17T22:55:08.393 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_dashboard", 2024-05-17T22:55:08.394 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.394 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.394 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.394 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.394 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.394 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.394 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.394 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-17T22:55:08.394 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.394 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.394 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.394 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.394 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_alertmanager": { 2024-05-17T22:55:08.395 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_alertmanager", 2024-05-17T22:55:08.395 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.395 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.395 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.395 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-17T22:55:08.395 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.395 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.395 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.395 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-05-17T22:55:08.395 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.395 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.395 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.395 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.396 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_base": { 2024-05-17T22:55:08.396 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_base", 2024-05-17T22:55:08.396 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.396 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.396 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.396 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-17T22:55:08.396 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.396 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.396 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.396 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Container image name, without the tag", 2024-05-17T22:55:08.396 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.396 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.396 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.396 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.397 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_elasticsearch": { 2024-05-17T22:55:08.397 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_elasticsearch", 2024-05-17T22:55:08.397 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.397 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.397 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.397 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-17T22:55:08.397 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.397 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.397 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.397 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "elasticsearch container image", 2024-05-17T22:55:08.397 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.397 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.397 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.398 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.398 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_grafana": { 2024-05-17T22:55:08.398 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_grafana", 2024-05-17T22:55:08.398 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.398 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.398 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.398 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-17T22:55:08.398 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.398 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.398 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.398 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-05-17T22:55:08.398 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.398 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.399 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.399 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.399 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_haproxy": { 2024-05-17T22:55:08.399 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_haproxy", 2024-05-17T22:55:08.399 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.399 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.399 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.399 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-17T22:55:08.399 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.399 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.399 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.399 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "HAproxy container image", 2024-05-17T22:55:08.399 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.400 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.400 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.400 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.400 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_agent": { 2024-05-17T22:55:08.400 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_agent", 2024-05-17T22:55:08.400 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.400 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.400 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.400 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-17T22:55:08.400 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.400 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.400 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.400 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger agent container image", 2024-05-17T22:55:08.400 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.401 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.401 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.401 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.401 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_collector": { 2024-05-17T22:55:08.401 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_collector", 2024-05-17T22:55:08.401 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.401 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.401 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.401 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-17T22:55:08.401 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.401 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.401 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.401 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger collector container image", 2024-05-17T22:55:08.401 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.402 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.402 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.402 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.402 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_query": { 2024-05-17T22:55:08.402 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_query", 2024-05-17T22:55:08.402 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.402 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.402 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.402 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-17T22:55:08.402 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.402 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.402 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.402 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger query container image", 2024-05-17T22:55:08.403 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.403 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.403 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.403 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.403 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_keepalived": { 2024-05-17T22:55:08.403 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_keepalived", 2024-05-17T22:55:08.403 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.403 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.403 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.403 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-17T22:55:08.403 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.403 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.403 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.404 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Keepalived container image", 2024-05-17T22:55:08.404 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.404 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.404 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.404 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.404 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_loki": { 2024-05-17T22:55:08.404 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_loki", 2024-05-17T22:55:08.404 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.404 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.404 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.404 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-05-17T22:55:08.404 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.404 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.404 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.405 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Loki container image", 2024-05-17T22:55:08.405 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.405 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.405 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.405 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.405 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_node_exporter": { 2024-05-17T22:55:08.405 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_node_exporter", 2024-05-17T22:55:08.405 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.405 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.405 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.405 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-17T22:55:08.405 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.405 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.406 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.406 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-05-17T22:55:08.406 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.406 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.406 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.406 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.406 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_nvmeof": { 2024-05-17T22:55:08.406 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_nvmeof", 2024-05-17T22:55:08.406 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.406 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.406 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.406 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-17T22:55:08.406 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.407 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.407 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.407 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Nvme-of container image", 2024-05-17T22:55:08.407 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.407 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.407 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.407 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.407 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_prometheus": { 2024-05-17T22:55:08.407 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_prometheus", 2024-05-17T22:55:08.407 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.407 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.407 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.407 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-17T22:55:08.407 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.408 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.408 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.408 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-05-17T22:55:08.408 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.408 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.408 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.408 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.408 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_promtail": { 2024-05-17T22:55:08.408 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_promtail", 2024-05-17T22:55:08.408 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.408 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.408 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.408 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-05-17T22:55:08.408 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.408 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.409 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.409 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Promtail container image", 2024-05-17T22:55:08.409 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.409 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.409 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.409 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.409 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_samba": { 2024-05-17T22:55:08.409 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_samba", 2024-05-17T22:55:08.409 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.409 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.409 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.409 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-17T22:55:08.409 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.409 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.410 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.410 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Samba/SMB container image", 2024-05-17T22:55:08.410 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.410 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.410 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.410 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.410 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_snmp_gateway": { 2024-05-17T22:55:08.410 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_snmp_gateway", 2024-05-17T22:55:08.410 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.410 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.410 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.410 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-17T22:55:08.410 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.411 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.411 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.411 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SNMP Gateway container image", 2024-05-17T22:55:08.411 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.411 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.411 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.411 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.411 INFO:teuthology.orchestra.run.smithi012.stdout: "container_init": { 2024-05-17T22:55:08.411 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_init", 2024-05-17T22:55:08.411 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.411 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.411 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.411 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.411 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.412 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.412 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.412 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Run podman/docker with `--init`", 2024-05-17T22:55:08.412 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.412 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.412 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.412 INFO:teuthology.orchestra.run.smithi012.stdout: "daemon_cache_timeout": { 2024-05-17T22:55:08.412 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "daemon_cache_timeout", 2024-05-17T22:55:08.412 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:08.412 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.412 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.412 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-17T22:55:08.413 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.413 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.413 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.413 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-17T22:55:08.413 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.413 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.413 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.413 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.413 INFO:teuthology.orchestra.run.smithi012.stdout: "default_cephadm_command_timeout": { 2024-05-17T22:55:08.413 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default_cephadm_command_timeout", 2024-05-17T22:55:08.413 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.413 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.413 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.413 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "900", 2024-05-17T22:55:08.414 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.414 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.414 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.414 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-17T22:55:08.414 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.414 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.414 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.414 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.414 INFO:teuthology.orchestra.run.smithi012.stdout: "default_registry": { 2024-05-17T22:55:08.414 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default_registry", 2024-05-17T22:55:08.414 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.414 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.414 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.415 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io", 2024-05-17T22:55:08.415 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.415 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.415 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.415 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-17T22:55:08.415 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.415 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.415 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.415 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.415 INFO:teuthology.orchestra.run.smithi012.stdout: "device_cache_timeout": { 2024-05-17T22:55:08.415 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_cache_timeout", 2024-05-17T22:55:08.415 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:08.415 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.416 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.416 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1800", 2024-05-17T22:55:08.416 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.416 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.416 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.416 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache device inventory", 2024-05-17T22:55:08.416 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.416 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.416 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.416 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.417 INFO:teuthology.orchestra.run.smithi012.stdout: "device_enhanced_scan": { 2024-05-17T22:55:08.417 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_enhanced_scan", 2024-05-17T22:55:08.417 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.417 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.417 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.417 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.417 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.417 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.417 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.417 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-17T22:55:08.417 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.417 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.417 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.418 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.418 INFO:teuthology.orchestra.run.smithi012.stdout: "facts_cache_timeout": { 2024-05-17T22:55:08.418 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "facts_cache_timeout", 2024-05-17T22:55:08.418 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:08.418 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.418 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.418 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-17T22:55:08.418 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.418 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.418 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.418 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache host facts data", 2024-05-17T22:55:08.418 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.418 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.419 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.419 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.419 INFO:teuthology.orchestra.run.smithi012.stdout: "grafana_dashboards_path": { 2024-05-17T22:55:08.419 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "grafana_dashboards_path", 2024-05-17T22:55:08.419 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.419 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.419 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.419 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-17T22:55:08.419 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.419 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.419 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.419 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-17T22:55:08.419 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.419 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.419 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.420 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.420 INFO:teuthology.orchestra.run.smithi012.stdout: "host_check_interval": { 2024-05-17T22:55:08.420 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "host_check_interval", 2024-05-17T22:55:08.420 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:08.420 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.420 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.420 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-17T22:55:08.420 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.420 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.420 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.420 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to perform a host check", 2024-05-17T22:55:08.420 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.420 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.420 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.421 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.421 INFO:teuthology.orchestra.run.smithi012.stdout: "hw_monitoring": { 2024-05-17T22:55:08.421 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "hw_monitoring", 2024-05-17T22:55:08.421 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.421 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.421 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.421 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.421 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.421 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.421 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.421 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-17T22:55:08.421 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.421 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.422 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.422 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.422 INFO:teuthology.orchestra.run.smithi012.stdout: "inventory_list_all": { 2024-05-17T22:55:08.422 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "inventory_list_all", 2024-05-17T22:55:08.422 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.422 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.422 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.422 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.422 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.422 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.422 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.422 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-17T22:55:08.422 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.423 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.423 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.423 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.423 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:08.423 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:08.423 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.423 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.423 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.423 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.423 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.423 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.423 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.423 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.424 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.424 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.424 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.424 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.424 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.424 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.424 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.424 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.424 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.424 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.424 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.424 INFO:teuthology.orchestra.run.smithi012.stdout: "log_refresh_metadata": { 2024-05-17T22:55:08.424 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_refresh_metadata", 2024-05-17T22:55:08.424 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.425 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.425 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.425 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.425 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.425 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.425 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.425 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-17T22:55:08.425 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.425 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.425 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.425 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.425 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:08.425 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:08.425 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.426 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.426 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.426 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.426 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.426 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.426 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.426 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-17T22:55:08.426 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.426 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.426 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.426 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.426 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:08.426 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:08.427 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.427 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.427 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.427 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:08.427 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.427 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.427 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.427 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.427 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.427 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.427 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.427 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.427 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.427 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.428 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.428 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.428 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.428 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.428 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.428 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:08.428 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:08.428 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.428 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.428 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.428 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.428 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.428 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.428 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.429 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.429 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.429 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.429 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.429 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.429 INFO:teuthology.orchestra.run.smithi012.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-17T22:55:08.429 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-17T22:55:08.429 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.429 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.429 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.429 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.429 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.429 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.429 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.430 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-17T22:55:08.430 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.430 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.430 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.430 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.430 INFO:teuthology.orchestra.run.smithi012.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-17T22:55:08.430 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-17T22:55:08.430 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.430 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.430 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.430 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "*", 2024-05-17T22:55:08.430 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.430 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.431 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.431 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-17T22:55:08.431 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.431 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.431 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.431 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.431 INFO:teuthology.orchestra.run.smithi012.stdout: "max_count_per_host": { 2024-05-17T22:55:08.431 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_count_per_host", 2024-05-17T22:55:08.431 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.431 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.431 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.431 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-17T22:55:08.431 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.431 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.432 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.432 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "max number of daemons per service per host", 2024-05-17T22:55:08.432 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.432 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.432 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.432 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.432 INFO:teuthology.orchestra.run.smithi012.stdout: "max_osd_draining_count": { 2024-05-17T22:55:08.432 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_osd_draining_count", 2024-05-17T22:55:08.432 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.432 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.432 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.432 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-17T22:55:08.432 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.433 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.433 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.433 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-17T22:55:08.433 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.433 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.433 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.433 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.433 INFO:teuthology.orchestra.run.smithi012.stdout: "migration_current": { 2024-05-17T22:55:08.433 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "migration_current", 2024-05-17T22:55:08.433 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.433 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.433 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.433 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.433 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.433 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.434 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.434 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "internal - do not modify", 2024-05-17T22:55:08.434 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.434 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.434 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.434 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.434 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": { 2024-05-17T22:55:08.434 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mode", 2024-05-17T22:55:08.434 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.434 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.434 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.434 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "root", 2024-05-17T22:55:08.434 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.434 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.435 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.435 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm-package", 2024-05-17T22:55:08.435 INFO:teuthology.orchestra.run.smithi012.stdout: "root" 2024-05-17T22:55:08.435 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.435 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "mode for remote execution of cephadm", 2024-05-17T22:55:08.435 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.435 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.435 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.435 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.435 INFO:teuthology.orchestra.run.smithi012.stdout: "oob_default_addr": { 2024-05-17T22:55:08.435 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "oob_default_addr", 2024-05-17T22:55:08.435 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.435 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.435 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.436 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "169.254.1.1", 2024-05-17T22:55:08.436 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.436 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.436 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.436 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-17T22:55:08.436 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.436 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.436 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.436 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.436 INFO:teuthology.orchestra.run.smithi012.stdout: "prometheus_alerts_path": { 2024-05-17T22:55:08.436 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prometheus_alerts_path", 2024-05-17T22:55:08.436 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.436 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.437 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.437 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-17T22:55:08.437 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.437 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.437 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.437 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-17T22:55:08.437 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.437 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.437 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.437 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.437 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_insecure": { 2024-05-17T22:55:08.437 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_insecure", 2024-05-17T22:55:08.437 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.437 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.438 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.438 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.438 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.438 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.438 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.438 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-17T22:55:08.438 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.438 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.438 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.438 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_password": { 2024-05-17T22:55:08.438 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_password", 2024-05-17T22:55:08.438 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.438 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.439 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.439 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.439 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.439 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.439 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.439 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-17T22:55:08.439 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.439 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.439 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.439 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_url": { 2024-05-17T22:55:08.439 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_url", 2024-05-17T22:55:08.439 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.439 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.440 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.440 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.440 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.440 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.440 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.440 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-17T22:55:08.440 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.440 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.440 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.440 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_username": { 2024-05-17T22:55:08.440 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_username", 2024-05-17T22:55:08.440 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.441 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.441 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.441 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.441 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.441 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.441 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.441 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-17T22:55:08.441 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.441 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.441 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.441 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.441 INFO:teuthology.orchestra.run.smithi012.stdout: "secure_monitoring_stack": { 2024-05-17T22:55:08.441 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "secure_monitoring_stack", 2024-05-17T22:55:08.441 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.442 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.442 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.442 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.442 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.442 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.442 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.442 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-17T22:55:08.442 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.442 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.442 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.442 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.442 INFO:teuthology.orchestra.run.smithi012.stdout: "service_discovery_port": { 2024-05-17T22:55:08.442 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "service_discovery_port", 2024-05-17T22:55:08.443 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.443 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.443 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.443 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8765", 2024-05-17T22:55:08.443 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.443 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.443 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.443 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "cephadm service discovery port", 2024-05-17T22:55:08.443 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.443 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.443 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.443 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.443 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:08.443 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:08.444 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.444 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:08.444 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.444 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.444 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.444 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.444 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.444 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.444 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.444 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.444 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.444 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.444 INFO:teuthology.orchestra.run.smithi012.stdout: "ssh_config_file": { 2024-05-17T22:55:08.444 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssh_config_file", 2024-05-17T22:55:08.444 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.445 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.445 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.445 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.445 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.445 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.445 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.445 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-17T22:55:08.445 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.445 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.445 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.445 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.445 INFO:teuthology.orchestra.run.smithi012.stdout: "use_agent": { 2024-05-17T22:55:08.445 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "use_agent", 2024-05-17T22:55:08.445 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.446 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.446 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.446 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.446 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.446 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.446 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.446 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-17T22:55:08.446 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.446 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.446 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.446 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.446 INFO:teuthology.orchestra.run.smithi012.stdout: "use_repo_digest": { 2024-05-17T22:55:08.446 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "use_repo_digest", 2024-05-17T22:55:08.446 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.446 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.447 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.447 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.447 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.447 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.447 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.447 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-17T22:55:08.447 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.447 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.447 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.447 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.447 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_failed_host_check": { 2024-05-17T22:55:08.447 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_failed_host_check", 2024-05-17T22:55:08.447 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.448 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.448 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.448 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.448 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.448 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.448 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.448 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if the host check fails", 2024-05-17T22:55:08.448 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.448 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.448 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.448 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.448 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_stray_daemons": { 2024-05-17T22:55:08.448 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_stray_daemons", 2024-05-17T22:55:08.448 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.449 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.449 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.449 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.449 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.449 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.449 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.449 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-17T22:55:08.449 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.449 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.449 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.449 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.449 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_stray_hosts": { 2024-05-17T22:55:08.449 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_stray_hosts", 2024-05-17T22:55:08.449 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.449 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.450 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.450 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.450 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.450 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.450 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.450 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-17T22:55:08.450 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.450 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.450 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.450 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.450 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.450 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.450 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.451 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crash", 2024-05-17T22:55:08.451 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:08.451 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:08.451 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:08.451 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:08.451 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:08.451 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.451 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.451 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.451 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.451 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.451 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.451 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.452 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.452 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.452 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.452 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.452 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.452 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.452 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.452 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.452 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.452 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.452 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.452 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.452 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:08.452 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:08.453 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.453 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.453 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.453 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.453 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.453 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.453 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.453 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.453 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.453 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.453 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.453 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.453 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:08.453 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:08.453 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.454 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.454 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.454 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:08.454 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.454 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.454 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.454 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.454 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.454 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.454 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.454 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.454 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.454 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.454 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.455 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.455 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.455 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.455 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.455 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:08.455 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:08.455 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.455 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.455 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.455 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.455 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.455 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.455 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.455 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.456 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.456 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.456 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.456 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.456 INFO:teuthology.orchestra.run.smithi012.stdout: "retain_interval": { 2024-05-17T22:55:08.456 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "retain_interval", 2024-05-17T22:55:08.456 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:08.456 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.456 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.456 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "31536000", 2024-05-17T22:55:08.456 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.456 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.456 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.457 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-17T22:55:08.457 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.457 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.457 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.457 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.457 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:08.457 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:08.457 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.457 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:08.457 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.457 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.457 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.457 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.458 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.458 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.458 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.458 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.458 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.458 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.458 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_recent_interval": { 2024-05-17T22:55:08.458 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_recent_interval", 2024-05-17T22:55:08.458 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:08.458 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.458 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.458 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1209600", 2024-05-17T22:55:08.458 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.458 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.459 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.459 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-17T22:55:08.459 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.459 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.459 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.459 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.459 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.459 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.459 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "dashboard", 2024-05-17T22:55:08.459 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:08.459 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:08.459 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:08.459 INFO:teuthology.orchestra.run.smithi012.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-17T22:55:08.460 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-17T22:55:08.460 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.460 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.460 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.460 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-17T22:55:08.460 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.460 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.460 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.460 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.460 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.460 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.460 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.460 INFO:teuthology.orchestra.run.smithi012.stdout: "ALERTMANAGER_API_HOST": { 2024-05-17T22:55:08.460 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-17T22:55:08.461 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.461 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.461 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.461 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.461 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.461 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.461 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.461 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.461 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.461 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.461 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.461 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.461 INFO:teuthology.orchestra.run.smithi012.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-17T22:55:08.462 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-17T22:55:08.462 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.462 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.462 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.462 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.462 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.462 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.462 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.462 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.462 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.462 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.462 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.463 INFO:teuthology.orchestra.run.smithi012.stdout: "AUDIT_API_ENABLED": { 2024-05-17T22:55:08.463 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "AUDIT_API_ENABLED", 2024-05-17T22:55:08.463 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.463 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.463 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.463 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.463 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.463 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.463 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.463 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.463 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.463 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.463 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.463 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.464 INFO:teuthology.orchestra.run.smithi012.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-17T22:55:08.464 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-17T22:55:08.464 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.464 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.464 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.464 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.464 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.464 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.464 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.464 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.464 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.464 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.464 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.464 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.465 INFO:teuthology.orchestra.run.smithi012.stdout: "ENABLE_BROWSABLE_API": { 2024-05-17T22:55:08.465 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-17T22:55:08.465 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.465 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.465 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.465 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.465 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.465 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.465 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.465 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.465 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.465 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.465 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.465 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.466 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-17T22:55:08.466 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-17T22:55:08.466 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.466 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.466 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.466 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.466 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.466 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.466 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.466 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.466 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.466 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.466 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.466 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.466 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-17T22:55:08.467 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-17T22:55:08.467 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.467 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.467 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.467 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.467 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.467 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.467 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.467 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.467 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.467 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.467 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.467 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.467 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-17T22:55:08.468 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-17T22:55:08.468 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.468 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.468 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.468 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.468 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.468 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.468 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.468 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.468 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.468 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.468 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.468 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-17T22:55:08.469 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-17T22:55:08.469 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.469 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.469 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.469 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.469 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.469 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.469 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.469 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.469 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.469 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.469 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.469 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.469 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-17T22:55:08.470 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-17T22:55:08.470 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.470 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.470 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.470 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.470 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.470 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.470 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.470 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.470 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.470 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.470 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.470 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.471 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-17T22:55:08.471 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-17T22:55:08.471 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.471 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.471 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.471 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.471 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.471 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.471 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.471 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.471 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.471 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.471 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.471 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.472 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-17T22:55:08.472 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-17T22:55:08.472 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.472 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.472 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.472 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.472 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.472 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.472 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.472 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.472 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.472 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.472 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.473 INFO:teuthology.orchestra.run.smithi012.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-17T22:55:08.473 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-17T22:55:08.473 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.473 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.473 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.473 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.473 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.473 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.473 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.473 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.473 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.473 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.473 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.474 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_PASSWORD": { 2024-05-17T22:55:08.474 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-17T22:55:08.474 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.474 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.474 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.474 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-05-17T22:55:08.474 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.474 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.474 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.474 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.474 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.474 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.474 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.475 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-17T22:55:08.475 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-17T22:55:08.475 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.475 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.475 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.475 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.475 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.475 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.475 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.475 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.475 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.475 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.476 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.476 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.476 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_URL": { 2024-05-17T22:55:08.476 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_URL", 2024-05-17T22:55:08.476 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.476 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.476 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.476 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.476 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.476 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.476 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.476 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.476 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.477 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.477 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.477 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_USERNAME": { 2024-05-17T22:55:08.477 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-17T22:55:08.477 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.477 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.477 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.477 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-05-17T22:55:08.477 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.477 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.477 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.477 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.477 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.478 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.478 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.478 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.478 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-17T22:55:08.478 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-17T22:55:08.478 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.478 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.478 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.478 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.478 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.478 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.478 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.478 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.479 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.479 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.479 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.479 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.479 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-17T22:55:08.479 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-17T22:55:08.479 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.479 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.479 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.479 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.479 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.480 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.480 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.480 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.480 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.480 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.480 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.480 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.480 INFO:teuthology.orchestra.run.smithi012.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-17T22:55:08.480 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-17T22:55:08.480 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.480 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.480 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.480 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.480 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.481 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.481 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.481 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.481 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.481 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.481 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.481 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.481 INFO:teuthology.orchestra.run.smithi012.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-17T22:55:08.481 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-17T22:55:08.481 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.481 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.481 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.481 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.482 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.482 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.482 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.482 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.482 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.482 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.482 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.482 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.482 INFO:teuthology.orchestra.run.smithi012.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-17T22:55:08.482 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-17T22:55:08.482 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.482 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.482 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.483 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "[]", 2024-05-17T22:55:08.483 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.483 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.483 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.483 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.483 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.483 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.483 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.483 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.483 INFO:teuthology.orchestra.run.smithi012.stdout: "MULTICLUSTER_CONFIG": { 2024-05-17T22:55:08.483 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-17T22:55:08.483 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.483 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.483 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.484 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "{}", 2024-05-17T22:55:08.484 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.484 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.484 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.484 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.484 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.484 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.484 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.484 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.484 INFO:teuthology.orchestra.run.smithi012.stdout: "PROMETHEUS_API_HOST": { 2024-05-17T22:55:08.484 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-17T22:55:08.484 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.484 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.485 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.485 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.485 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.485 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.485 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.485 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.485 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.485 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.485 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.485 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.485 INFO:teuthology.orchestra.run.smithi012.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-17T22:55:08.485 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-17T22:55:08.485 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.486 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.486 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.486 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.486 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.486 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.486 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.486 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.486 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.486 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.486 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.486 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.486 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-17T22:55:08.486 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-17T22:55:08.486 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.487 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.487 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.487 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.487 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.487 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.487 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.487 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.487 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.487 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.487 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.487 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.487 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-17T22:55:08.487 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-17T22:55:08.487 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.488 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.488 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.488 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.488 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.488 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.488 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.488 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.488 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.488 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.488 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.488 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.488 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-17T22:55:08.488 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-17T22:55:08.488 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.489 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.489 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.489 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.489 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.489 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.489 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.489 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.489 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.489 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.489 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.489 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.489 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-17T22:55:08.489 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-17T22:55:08.489 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.490 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.490 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.490 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.490 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.490 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.490 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.490 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.490 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.490 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.490 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.490 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.490 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-17T22:55:08.491 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-17T22:55:08.491 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.491 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.491 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.491 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.491 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.491 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.491 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.491 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.491 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.491 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.491 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.491 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.491 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-17T22:55:08.492 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-17T22:55:08.492 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.492 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.492 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.492 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.492 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.492 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.492 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.492 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.492 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.492 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.492 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.492 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.492 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-17T22:55:08.493 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-17T22:55:08.493 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.493 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.493 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.493 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.493 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.493 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.493 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.493 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.493 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.493 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.493 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.493 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.493 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_ENABLED": { 2024-05-17T22:55:08.494 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-17T22:55:08.494 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.494 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.494 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.494 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.494 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.494 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.494 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.494 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.494 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.494 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.494 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.494 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.494 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-17T22:55:08.495 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-17T22:55:08.495 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.495 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.495 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.495 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-17T22:55:08.495 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.495 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.495 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.495 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.495 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.495 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.495 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.495 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.495 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-17T22:55:08.495 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-17T22:55:08.496 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.496 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.496 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.496 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-17T22:55:08.496 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.496 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.496 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.496 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.496 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.496 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.496 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.496 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.496 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-17T22:55:08.496 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-17T22:55:08.496 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.497 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.497 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.497 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8", 2024-05-17T22:55:08.497 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.497 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.497 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.497 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.497 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.497 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.497 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.497 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.497 INFO:teuthology.orchestra.run.smithi012.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-17T22:55:08.497 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-17T22:55:08.497 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.497 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.498 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.498 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "45", 2024-05-17T22:55:08.498 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.498 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.498 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.498 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.498 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.498 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.498 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.498 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.498 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_ACCESS_KEY": { 2024-05-17T22:55:08.498 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-17T22:55:08.498 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.498 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.499 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.499 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.499 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.499 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.499 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.499 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.499 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.499 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.499 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.499 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.499 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-17T22:55:08.499 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-17T22:55:08.499 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.500 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.500 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.500 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-05-17T22:55:08.500 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.500 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.500 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.500 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.500 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.500 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.500 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.500 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.500 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_SECRET_KEY": { 2024-05-17T22:55:08.500 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-17T22:55:08.500 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.501 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.501 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.501 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.501 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.501 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.501 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.501 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.501 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.501 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.501 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.501 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.501 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_SSL_VERIFY": { 2024-05-17T22:55:08.501 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-17T22:55:08.501 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.502 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.502 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.502 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.502 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.502 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.502 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.502 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.502 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.502 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.502 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.502 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.502 INFO:teuthology.orchestra.run.smithi012.stdout: "UNSAFE_TLS_v1_2": { 2024-05-17T22:55:08.502 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-17T22:55:08.503 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.503 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.503 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.503 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.503 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.503 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.503 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.503 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.503 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.503 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.503 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.503 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.503 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-17T22:55:08.503 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-17T22:55:08.504 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.504 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.504 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.504 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.504 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.504 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.504 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.504 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.504 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.504 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.504 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.504 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.504 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-17T22:55:08.505 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-17T22:55:08.505 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.505 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.505 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.505 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-17T22:55:08.505 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.505 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.505 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.505 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.505 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.505 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.505 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.505 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.505 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-17T22:55:08.506 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-17T22:55:08.506 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.506 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.506 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.506 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-05-17T22:55:08.506 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.506 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.506 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.506 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.506 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.506 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.506 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.506 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.506 INFO:teuthology.orchestra.run.smithi012.stdout: "cross_origin_url": { 2024-05-17T22:55:08.507 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cross_origin_url", 2024-05-17T22:55:08.507 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.507 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.507 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.507 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.507 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.507 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.507 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.507 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.507 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.507 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.507 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.507 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.508 INFO:teuthology.orchestra.run.smithi012.stdout: "crt_file": { 2024-05-17T22:55:08.508 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crt_file", 2024-05-17T22:55:08.508 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.508 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.508 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.508 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.508 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.508 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.508 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.508 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.508 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.508 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.508 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.508 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.509 INFO:teuthology.orchestra.run.smithi012.stdout: "debug": { 2024-05-17T22:55:08.509 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "debug", 2024-05-17T22:55:08.509 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.509 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.509 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.509 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.509 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.509 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.509 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.509 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable/disable debug options", 2024-05-17T22:55:08.509 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.509 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.509 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.509 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.510 INFO:teuthology.orchestra.run.smithi012.stdout: "jwt_token_ttl": { 2024-05-17T22:55:08.510 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "jwt_token_ttl", 2024-05-17T22:55:08.510 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.510 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.510 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.510 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "28800", 2024-05-17T22:55:08.510 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.510 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.510 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.510 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.510 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.510 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.510 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.510 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.511 INFO:teuthology.orchestra.run.smithi012.stdout: "key_file": { 2024-05-17T22:55:08.511 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "key_file", 2024-05-17T22:55:08.511 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.511 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.511 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.511 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.511 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.511 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.511 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.511 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.511 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.511 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.511 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.511 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.512 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:08.512 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:08.512 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.512 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.512 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.512 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.512 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.512 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.512 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.512 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.512 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.512 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.512 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.512 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.513 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.513 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.513 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.513 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.513 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.513 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.513 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.513 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:08.513 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:08.513 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.513 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.513 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.513 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.513 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.514 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.514 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.514 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.514 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.514 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.514 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.514 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.514 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:08.514 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:08.514 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.514 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.514 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.514 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:08.514 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.515 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.515 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.515 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.515 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.515 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.515 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.515 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.515 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.515 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.515 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.515 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.515 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.515 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.515 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.516 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:08.516 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:08.516 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.516 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.516 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.516 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.516 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.516 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.516 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.516 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.516 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.516 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.516 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.516 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.517 INFO:teuthology.orchestra.run.smithi012.stdout: "motd": { 2024-05-17T22:55:08.517 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "motd", 2024-05-17T22:55:08.517 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.517 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.517 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.517 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.517 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.517 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.517 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.517 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "The message of the day", 2024-05-17T22:55:08.517 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.517 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.517 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.517 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.518 INFO:teuthology.orchestra.run.smithi012.stdout: "redirect_resolve_ip_addr": { 2024-05-17T22:55:08.518 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "redirect_resolve_ip_addr", 2024-05-17T22:55:08.518 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.518 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.518 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.518 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.518 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.518 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.518 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.518 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.518 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.518 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.518 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.519 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.519 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-05-17T22:55:08.519 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-05-17T22:55:08.519 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.519 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.519 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.519 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "::", 2024-05-17T22:55:08.519 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.519 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.519 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.519 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.519 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.519 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.519 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.520 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-05-17T22:55:08.520 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-05-17T22:55:08.520 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.520 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.520 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.520 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8080", 2024-05-17T22:55:08.520 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.520 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.520 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.520 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.520 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.520 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.520 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.521 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:08.521 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:08.521 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.521 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:08.521 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.521 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.521 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.521 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.521 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.521 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.521 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.521 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.521 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.522 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.522 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl": { 2024-05-17T22:55:08.522 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl", 2024-05-17T22:55:08.522 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.522 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.522 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.522 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.522 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.522 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.522 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.522 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.522 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.522 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.522 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.523 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.523 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl_server_port": { 2024-05-17T22:55:08.523 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl_server_port", 2024-05-17T22:55:08.523 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.523 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.523 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.523 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8443", 2024-05-17T22:55:08.523 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.523 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.523 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.523 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.523 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.523 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.523 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.524 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.524 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_behaviour": { 2024-05-17T22:55:08.524 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_behaviour", 2024-05-17T22:55:08.524 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.524 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.524 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.524 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "redirect", 2024-05-17T22:55:08.524 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.524 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.524 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.524 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.524 INFO:teuthology.orchestra.run.smithi012.stdout: "redirect" 2024-05-17T22:55:08.524 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.525 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.525 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.525 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.525 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.525 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.525 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_error_status_code": { 2024-05-17T22:55:08.525 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_error_status_code", 2024-05-17T22:55:08.525 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.525 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.525 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.525 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-05-17T22:55:08.525 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "400", 2024-05-17T22:55:08.525 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "599", 2024-05-17T22:55:08.525 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.526 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.526 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.526 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.526 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.526 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.526 INFO:teuthology.orchestra.run.smithi012.stdout: "url_prefix": { 2024-05-17T22:55:08.526 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "url_prefix", 2024-05-17T22:55:08.526 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.526 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.526 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.526 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.526 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.527 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.527 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.527 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.527 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.527 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.527 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.527 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.527 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.527 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.527 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.527 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "devicehealth", 2024-05-17T22:55:08.527 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:08.527 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:08.527 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:08.527 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_monitoring": { 2024-05-17T22:55:08.528 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enable_monitoring", 2024-05-17T22:55:08.528 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.528 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.528 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.528 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.528 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.528 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.528 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.528 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "monitor device health metrics", 2024-05-17T22:55:08.528 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.528 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.528 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.528 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.528 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:08.528 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:08.529 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.529 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.529 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.529 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.529 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.529 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.529 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.529 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.529 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.529 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.529 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.529 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.529 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.530 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.530 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.530 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.530 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.530 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.530 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.530 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:08.530 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:08.530 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.530 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.530 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.530 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.530 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.531 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.531 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.531 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.531 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.531 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.531 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.531 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.531 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:08.531 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:08.531 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.531 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.531 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.531 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:08.532 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.532 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.532 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.532 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.532 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.532 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.532 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.532 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.532 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.532 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.532 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.532 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.532 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.532 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.533 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.533 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:08.533 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:08.533 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.533 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.533 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.533 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.533 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.533 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.533 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.533 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.533 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.533 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.533 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.534 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.534 INFO:teuthology.orchestra.run.smithi012.stdout: "mark_out_threshold": { 2024-05-17T22:55:08.534 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mark_out_threshold", 2024-05-17T22:55:08.534 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:08.534 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.534 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.534 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "2419200", 2024-05-17T22:55:08.534 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.534 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.534 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.534 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-17T22:55:08.534 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.534 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.535 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.535 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.535 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": { 2024-05-17T22:55:08.535 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pool_name", 2024-05-17T22:55:08.535 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.535 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.535 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.535 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "device_health_metrics", 2024-05-17T22:55:08.535 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.535 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.535 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.535 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-17T22:55:08.535 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.535 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.536 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.536 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.536 INFO:teuthology.orchestra.run.smithi012.stdout: "retention_period": { 2024-05-17T22:55:08.536 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "retention_period", 2024-05-17T22:55:08.536 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:08.536 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.536 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.536 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15552000", 2024-05-17T22:55:08.536 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.536 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.536 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.536 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long to retain device health metrics", 2024-05-17T22:55:08.536 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.536 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.537 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.537 INFO:teuthology.orchestra.run.smithi012.stdout: "scrape_frequency": { 2024-05-17T22:55:08.537 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "scrape_frequency", 2024-05-17T22:55:08.537 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:08.537 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.537 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.537 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "86400", 2024-05-17T22:55:08.537 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.537 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.537 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.537 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-17T22:55:08.537 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.537 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.537 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.538 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.538 INFO:teuthology.orchestra.run.smithi012.stdout: "self_heal": { 2024-05-17T22:55:08.538 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "self_heal", 2024-05-17T22:55:08.538 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.538 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.538 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.538 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.538 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.538 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.538 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.538 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-17T22:55:08.538 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.538 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.539 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.539 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.539 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-17T22:55:08.539 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-17T22:55:08.539 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:08.539 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.539 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.539 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-17T22:55:08.539 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.539 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.539 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.539 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to wake up and check device health", 2024-05-17T22:55:08.539 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.540 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.540 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.540 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:08.540 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:08.540 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.540 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:08.540 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.540 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.540 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.540 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.540 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.540 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.540 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.541 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.541 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.541 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_threshold": { 2024-05-17T22:55:08.541 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_threshold", 2024-05-17T22:55:08.541 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:08.541 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.541 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.541 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "7257600", 2024-05-17T22:55:08.541 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.541 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.541 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.541 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-17T22:55:08.542 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.542 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.542 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.542 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.542 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.542 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.542 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.542 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "diskprediction_local", 2024-05-17T22:55:08.542 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:08.542 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:08.542 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:08.542 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:08.542 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:08.543 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.543 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.543 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.543 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.543 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.543 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.543 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.543 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.543 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.543 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.543 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.543 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.543 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.543 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.544 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.544 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.544 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.544 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.544 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.544 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:08.544 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:08.544 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.544 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.544 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.544 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.544 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.544 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.544 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.545 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.545 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.545 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.545 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.545 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:08.545 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:08.545 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.545 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.545 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.545 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:08.545 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.545 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.545 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.546 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.546 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.546 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.546 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.546 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.546 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.546 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.546 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.546 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.546 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.546 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.546 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:08.547 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:08.547 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.547 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.547 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.547 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.547 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.547 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.547 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.547 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.547 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.547 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.547 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.547 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.547 INFO:teuthology.orchestra.run.smithi012.stdout: "predict_interval": { 2024-05-17T22:55:08.548 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "predict_interval", 2024-05-17T22:55:08.548 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.548 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.548 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.548 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "86400", 2024-05-17T22:55:08.548 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.550 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.550 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.550 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.550 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.550 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.551 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.551 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.551 INFO:teuthology.orchestra.run.smithi012.stdout: "predictor_model": { 2024-05-17T22:55:08.551 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "predictor_model", 2024-05-17T22:55:08.551 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.551 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.551 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.551 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "prophetstor", 2024-05-17T22:55:08.551 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.551 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.551 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.551 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.551 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.552 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.552 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.552 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-17T22:55:08.552 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-17T22:55:08.552 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.552 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.552 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.552 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-17T22:55:08.552 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.552 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.552 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.552 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.552 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.553 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.553 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.553 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:08.553 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:08.553 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.553 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:08.553 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.553 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.553 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.553 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.553 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.553 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.553 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.554 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.554 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.554 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.554 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.554 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.554 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "influx", 2024-05-17T22:55:08.554 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": false, 2024-05-17T22:55:08.554 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "influxdb python module not found", 2024-05-17T22:55:08.554 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:08.554 INFO:teuthology.orchestra.run.smithi012.stdout: "batch_size": { 2024-05-17T22:55:08.554 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "batch_size", 2024-05-17T22:55:08.554 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.555 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.555 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.555 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5000", 2024-05-17T22:55:08.555 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.555 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.555 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.555 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-17T22:55:08.555 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.555 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.555 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.555 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.555 INFO:teuthology.orchestra.run.smithi012.stdout: "database": { 2024-05-17T22:55:08.555 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "database", 2024-05-17T22:55:08.556 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.556 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.556 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.556 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "ceph", 2024-05-17T22:55:08.556 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.556 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.556 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.556 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-17T22:55:08.556 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.556 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.556 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.556 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.556 INFO:teuthology.orchestra.run.smithi012.stdout: "hostname": { 2024-05-17T22:55:08.556 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "hostname", 2024-05-17T22:55:08.557 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.557 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.557 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.557 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.557 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.557 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.557 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.557 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB server hostname", 2024-05-17T22:55:08.557 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.557 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.557 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.557 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.557 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-17T22:55:08.557 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-17T22:55:08.558 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:08.558 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.558 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.558 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "30", 2024-05-17T22:55:08.558 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "5", 2024-05-17T22:55:08.558 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.558 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.558 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-17T22:55:08.558 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.558 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.558 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.558 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.558 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:08.559 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:08.559 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.559 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.559 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.559 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.559 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.559 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.559 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.559 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.559 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.559 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.559 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.559 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.559 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.560 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.560 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.560 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.560 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.560 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.560 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.560 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:08.560 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:08.560 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.560 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.560 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.560 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.560 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.560 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.561 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.561 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.561 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.561 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.561 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.561 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.561 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:08.561 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:08.561 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.561 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.561 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.561 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:08.561 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.562 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.562 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.562 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.562 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.562 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.562 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.562 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.562 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.562 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.562 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.562 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.562 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.562 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.563 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.563 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:08.563 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:08.563 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.563 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.563 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.563 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.563 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.563 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.563 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.563 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.563 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.563 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.563 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.564 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.564 INFO:teuthology.orchestra.run.smithi012.stdout: "password": { 2024-05-17T22:55:08.564 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "password", 2024-05-17T22:55:08.564 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.564 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.564 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.564 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.564 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.564 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.564 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.564 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "password of InfluxDB server user", 2024-05-17T22:55:08.564 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.564 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.564 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.565 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.565 INFO:teuthology.orchestra.run.smithi012.stdout: "port": { 2024-05-17T22:55:08.565 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "port", 2024-05-17T22:55:08.565 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.565 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.565 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.565 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8086", 2024-05-17T22:55:08.565 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.565 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.565 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.565 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB server port", 2024-05-17T22:55:08.565 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.565 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.565 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.566 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.566 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:08.566 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:08.566 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.566 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:08.566 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.566 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.566 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.566 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.566 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.566 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.566 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.566 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.567 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.567 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.567 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl": { 2024-05-17T22:55:08.567 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl", 2024-05-17T22:55:08.567 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.567 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.567 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.567 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "false", 2024-05-17T22:55:08.567 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.567 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.567 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.567 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-17T22:55:08.567 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.567 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.568 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.568 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.568 INFO:teuthology.orchestra.run.smithi012.stdout: "threads": { 2024-05-17T22:55:08.568 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "threads", 2024-05-17T22:55:08.568 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.568 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.568 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.568 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-05-17T22:55:08.568 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-05-17T22:55:08.568 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "32", 2024-05-17T22:55:08.568 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.568 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-17T22:55:08.568 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.568 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.568 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.569 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.569 INFO:teuthology.orchestra.run.smithi012.stdout: "username": { 2024-05-17T22:55:08.569 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "username", 2024-05-17T22:55:08.569 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.569 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.569 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.569 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.569 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.569 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.569 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.569 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "username of InfluxDB server user", 2024-05-17T22:55:08.569 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.569 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.570 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.570 INFO:teuthology.orchestra.run.smithi012.stdout: "verify_ssl": { 2024-05-17T22:55:08.570 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "verify_ssl", 2024-05-17T22:55:08.570 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.570 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.570 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.570 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "true", 2024-05-17T22:55:08.570 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.570 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.570 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.570 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-17T22:55:08.570 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.570 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.571 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.571 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.571 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.571 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.571 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.571 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "insights", 2024-05-17T22:55:08.571 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:08.571 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:08.571 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:08.571 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:08.571 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:08.571 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.571 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.571 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.571 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.572 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.572 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.572 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.572 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.572 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.572 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.572 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.572 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.572 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.572 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.572 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.572 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.572 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.573 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.573 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.573 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:08.573 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:08.573 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.573 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.573 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.573 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.573 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.573 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.573 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.573 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.573 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.574 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.574 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.574 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:08.574 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:08.574 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.574 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.574 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.574 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:08.574 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.574 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.574 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.574 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.575 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.575 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.575 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.575 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.575 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.575 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.575 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.575 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.575 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.575 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.575 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.575 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:08.575 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:08.576 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.576 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.576 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.576 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.576 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.576 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.576 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.576 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.576 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.576 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.576 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.576 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.576 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:08.576 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:08.576 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.577 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:08.577 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.577 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.577 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.577 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.577 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.577 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.577 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.577 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.577 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.577 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.577 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.577 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.577 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.578 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "iostat", 2024-05-17T22:55:08.578 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:08.578 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:08.578 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:08.578 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:08.578 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:08.578 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.578 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.578 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.578 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.578 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.578 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.578 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.578 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.579 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.579 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.579 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.579 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.579 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.579 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.579 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.579 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.579 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.579 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.579 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.579 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:08.579 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:08.580 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.580 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.580 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.580 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.580 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.580 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.580 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.580 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.580 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.580 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.580 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.580 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.580 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:08.581 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:08.581 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.581 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.581 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.581 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:08.581 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.581 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.581 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.581 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.581 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.581 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.581 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.581 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.581 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.581 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.582 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.582 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.582 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.582 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.582 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.582 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:08.582 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:08.582 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.582 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.582 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.582 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.582 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.583 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.583 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.583 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.583 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.583 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.583 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.583 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:08.583 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:08.583 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.583 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:08.583 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.583 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.583 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.584 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.584 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.584 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.584 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.584 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.584 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.584 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.584 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.584 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.584 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.584 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "k8sevents", 2024-05-17T22:55:08.584 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:08.584 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:08.584 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:08.585 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_event_retention_days": { 2024-05-17T22:55:08.585 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ceph_event_retention_days", 2024-05-17T22:55:08.585 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.585 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.585 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.585 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "7", 2024-05-17T22:55:08.585 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.585 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.585 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.585 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-17T22:55:08.585 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.585 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.585 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.586 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.586 INFO:teuthology.orchestra.run.smithi012.stdout: "config_check_secs": { 2024-05-17T22:55:08.586 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_check_secs", 2024-05-17T22:55:08.586 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.586 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.586 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.586 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-17T22:55:08.586 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "10", 2024-05-17T22:55:08.586 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.586 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.586 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-17T22:55:08.586 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.586 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.587 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.587 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.587 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:08.587 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:08.587 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.587 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.587 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.587 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.587 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.587 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.587 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.587 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.587 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.587 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.587 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.588 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.588 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.588 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.588 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.588 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.588 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.588 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.588 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:08.588 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:08.588 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.588 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.588 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.588 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.589 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.589 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.589 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.589 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.589 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.589 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.589 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.589 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.589 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:08.589 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:08.589 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.589 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.589 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.589 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:08.590 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.590 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.590 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.590 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.590 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.590 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.590 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.590 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.590 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.590 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.590 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.590 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.591 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.591 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.591 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.591 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:08.591 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:08.591 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.591 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.591 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.591 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.591 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.591 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.591 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.591 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.591 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.592 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.592 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.592 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.592 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:08.592 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:08.592 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.592 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:08.592 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.592 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.592 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.592 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.592 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.592 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.592 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.593 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.593 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.593 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.593 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.593 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.593 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "localpool", 2024-05-17T22:55:08.593 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:08.593 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:08.593 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:08.593 INFO:teuthology.orchestra.run.smithi012.stdout: "failure_domain": { 2024-05-17T22:55:08.593 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "failure_domain", 2024-05-17T22:55:08.593 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.593 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.594 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.594 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "host", 2024-05-17T22:55:08.594 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.594 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.594 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.594 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "failure domain for any created local pool", 2024-05-17T22:55:08.594 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-17T22:55:08.594 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.594 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.594 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.594 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:08.594 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:08.594 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.594 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.595 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.595 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.595 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.595 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.595 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.595 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.595 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.595 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.595 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.595 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.595 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.595 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.595 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.596 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.596 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.596 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.596 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.596 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:08.596 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:08.596 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.596 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.596 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.596 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.596 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.596 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.596 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.596 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.597 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.597 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.597 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.597 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.597 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:08.597 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:08.597 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.597 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.597 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.597 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:08.597 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.597 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.597 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.597 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.598 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.598 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.598 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.598 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.598 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.598 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.598 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.598 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.598 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.598 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.598 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.598 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:08.598 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:08.599 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.599 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.599 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.599 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.599 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.599 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.599 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.599 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.599 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.599 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.599 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.599 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.599 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": { 2024-05-17T22:55:08.599 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "min_size", 2024-05-17T22:55:08.600 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.600 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.600 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.600 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.600 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.600 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.600 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.600 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default min_size for any created local pool", 2024-05-17T22:55:08.600 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-17T22:55:08.600 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.600 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.600 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.600 INFO:teuthology.orchestra.run.smithi012.stdout: "num_rep": { 2024-05-17T22:55:08.601 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "num_rep", 2024-05-17T22:55:08.601 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.601 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.601 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.601 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3", 2024-05-17T22:55:08.601 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.601 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.601 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.601 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default replica count for any created local pool", 2024-05-17T22:55:08.601 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.601 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.601 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.601 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.601 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": { 2024-05-17T22:55:08.602 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pg_num", 2024-05-17T22:55:08.602 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.602 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.602 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.602 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "128", 2024-05-17T22:55:08.602 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.602 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.602 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.602 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default pg_num for any created local pool", 2024-05-17T22:55:08.602 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.602 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.602 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.602 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.603 INFO:teuthology.orchestra.run.smithi012.stdout: "prefix": { 2024-05-17T22:55:08.603 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prefix", 2024-05-17T22:55:08.603 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.603 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.603 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.603 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.603 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.603 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.603 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.603 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "name prefix for any created local pool", 2024-05-17T22:55:08.603 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.603 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.603 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.603 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.604 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:08.604 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:08.604 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.604 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:08.604 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.604 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.604 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.604 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.604 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.604 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.604 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.604 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.604 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.605 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.605 INFO:teuthology.orchestra.run.smithi012.stdout: "subtree": { 2024-05-17T22:55:08.605 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "subtree", 2024-05-17T22:55:08.605 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.605 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.605 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.605 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "rack", 2024-05-17T22:55:08.605 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.605 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.605 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.605 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-17T22:55:08.605 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-17T22:55:08.605 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.605 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.605 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.606 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.606 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.606 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.606 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mds_autoscaler", 2024-05-17T22:55:08.606 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:08.606 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:08.606 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:08.606 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:08.606 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:08.606 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.606 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.606 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.606 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.607 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.607 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.607 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.607 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.607 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.607 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.607 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.607 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.607 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.607 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.607 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.607 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.607 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.607 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.608 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.608 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:08.608 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:08.608 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.608 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.608 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.608 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.608 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.608 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.608 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.608 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.608 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.608 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.608 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.609 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.609 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:08.609 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:08.609 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.609 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.609 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.609 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:08.609 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.609 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.609 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.609 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.609 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.609 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.609 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.610 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.610 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.610 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.610 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.610 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.610 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.610 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.610 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.610 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:08.610 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:08.610 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.610 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.610 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.611 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.611 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.611 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.611 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.611 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.611 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.611 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.611 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.611 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.611 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:08.611 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:08.611 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.611 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:08.611 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.612 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.612 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.612 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.612 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.612 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.612 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.612 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.612 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.612 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.612 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.612 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.612 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.613 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mirroring", 2024-05-17T22:55:08.613 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:08.613 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:08.613 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:08.613 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:08.613 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:08.613 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.613 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.613 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.613 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.613 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.613 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.613 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.613 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.614 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.614 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.614 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.614 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.614 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.614 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.614 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.614 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.614 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.614 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.614 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.614 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:08.614 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:08.615 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.615 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.615 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.615 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.615 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.615 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.615 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.615 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.615 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.615 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.615 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.615 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.615 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:08.616 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:08.616 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.616 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.616 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.616 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:08.616 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.616 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.616 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.616 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.616 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.616 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.616 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.616 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.617 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.617 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.617 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.617 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.617 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.617 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.617 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.617 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:08.617 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:08.617 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.617 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.617 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.617 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.617 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.618 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.618 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.618 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.618 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.618 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.618 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.618 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.618 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:08.618 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:08.618 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.618 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:08.618 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.618 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.619 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.619 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.619 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.619 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.619 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.619 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.619 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.619 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.619 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.619 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.619 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.619 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "nfs", 2024-05-17T22:55:08.619 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:08.619 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:08.620 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:08.620 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:08.620 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:08.620 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.620 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.620 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.620 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.620 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.620 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.620 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.620 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.620 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.620 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.620 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.621 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.621 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.621 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.621 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.621 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.621 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.621 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.621 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.621 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:08.621 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:08.621 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.621 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.621 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.622 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.622 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.622 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.622 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.622 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.622 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.622 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.622 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.622 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.622 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:08.622 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:08.622 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.622 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.623 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.623 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:08.623 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.623 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.623 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.623 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.623 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.623 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.623 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.623 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.623 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.623 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.623 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.623 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.624 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.624 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.624 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.624 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:08.624 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:08.624 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.624 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.624 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.624 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.624 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.624 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.624 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.624 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.625 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.625 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.625 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.625 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.625 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:08.625 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:08.625 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.625 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:08.625 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.625 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.625 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.625 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.625 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.625 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.626 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.626 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.626 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.626 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.626 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.626 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.626 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.626 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "orchestrator", 2024-05-17T22:55:08.626 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:08.626 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:08.626 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:08.626 INFO:teuthology.orchestra.run.smithi012.stdout: "fail_fs": { 2024-05-17T22:55:08.626 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "fail_fs", 2024-05-17T22:55:08.627 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.627 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.627 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.627 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.627 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.627 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.627 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.627 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-17T22:55:08.627 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.627 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.627 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.627 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.627 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:08.627 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:08.628 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.628 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.628 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.628 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.628 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.628 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.628 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.628 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.628 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.628 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.628 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.628 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.628 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.628 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.629 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.629 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.629 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.629 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.629 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.629 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:08.629 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:08.629 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.629 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.629 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.629 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.629 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.629 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.629 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.630 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.630 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.630 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.630 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.630 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.630 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:08.630 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:08.630 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.630 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.630 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.630 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:08.630 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.630 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.631 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.631 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.631 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.631 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.631 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.631 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.631 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.631 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.631 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.631 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.631 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.631 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.631 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:08.632 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:08.632 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.632 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.632 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.632 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.632 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.632 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.632 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.632 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.632 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.632 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.632 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.632 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.633 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator": { 2024-05-17T22:55:08.633 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "orchestrator", 2024-05-17T22:55:08.633 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.633 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.633 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.633 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.633 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.633 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.633 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.633 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm", 2024-05-17T22:55:08.633 INFO:teuthology.orchestra.run.smithi012.stdout: "rook", 2024-05-17T22:55:08.633 INFO:teuthology.orchestra.run.smithi012.stdout: "test_orchestrator" 2024-05-17T22:55:08.633 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.633 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Orchestrator backend", 2024-05-17T22:55:08.634 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.634 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.634 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.634 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.634 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:08.634 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:08.634 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.634 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:08.634 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.634 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.634 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.634 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.634 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.635 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.635 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.635 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.635 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.635 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.635 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.635 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.635 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.635 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "osd_perf_query", 2024-05-17T22:55:08.635 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:08.635 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:08.635 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:08.635 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:08.635 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:08.635 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.636 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.636 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.636 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.636 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.636 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.636 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.636 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.636 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.636 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.636 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.636 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.636 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.636 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.636 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.636 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.637 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.637 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.637 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.637 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:08.637 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:08.637 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.637 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.637 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.637 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.637 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.637 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.637 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.637 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.637 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.638 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.638 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.638 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.638 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:08.638 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:08.638 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.638 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.638 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.638 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:08.638 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.638 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.638 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.638 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.639 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.639 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.639 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.639 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.639 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.639 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.639 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.639 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.639 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.639 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.639 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.639 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:08.639 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:08.639 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.640 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.640 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.640 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.640 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.640 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.640 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.640 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.640 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.640 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.640 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.640 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.640 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:08.640 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:08.641 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.641 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:08.641 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.641 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.641 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.641 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.641 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.641 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.641 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.641 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.641 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.641 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.641 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.641 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.642 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.642 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "osd_support", 2024-05-17T22:55:08.642 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:08.642 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:08.642 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:08.642 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:08.642 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:08.642 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.642 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.642 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.642 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.642 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.642 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.643 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.643 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.643 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.643 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.643 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.643 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.643 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.643 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.643 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.643 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.643 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.643 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.643 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.643 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:08.643 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:08.644 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.644 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.644 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.644 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.644 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.644 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.644 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.644 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.644 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.644 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.644 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.644 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.644 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:08.644 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:08.645 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.645 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.645 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.645 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:08.645 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.645 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.645 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.645 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.645 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.645 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.645 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.645 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.645 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.645 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.645 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.646 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.646 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.646 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.646 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.646 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:08.646 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:08.646 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.646 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.646 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.646 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.646 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.646 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.646 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.647 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.647 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.647 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.647 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.647 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.647 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:08.647 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:08.647 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.647 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:08.647 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.647 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.647 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.647 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.648 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.648 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.648 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.648 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.648 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.648 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.648 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.648 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.648 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.648 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pg_autoscaler", 2024-05-17T22:55:08.648 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:08.648 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:08.648 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:08.648 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:08.649 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:08.649 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.649 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.649 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.649 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.649 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.649 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.649 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.649 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.649 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.649 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.649 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.649 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.649 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.650 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.650 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.650 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.650 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.650 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.650 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.650 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:08.650 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:08.650 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.650 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.650 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.650 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.650 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.650 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.651 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.651 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.651 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.651 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.651 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.651 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.651 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:08.651 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:08.651 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.651 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.651 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.651 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:08.651 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.651 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.652 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.652 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.652 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.652 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.652 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.652 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.652 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.652 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.652 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.652 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.652 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.652 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.652 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.652 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:08.653 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:08.653 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.653 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.653 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.653 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.653 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.653 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.653 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.653 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.653 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.653 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.653 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.653 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.653 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-17T22:55:08.654 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-17T22:55:08.654 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:08.654 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.654 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.654 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-17T22:55:08.654 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.654 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.654 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.654 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.654 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.654 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.654 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.654 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.655 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:08.655 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:08.655 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.655 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:08.655 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.655 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.655 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.655 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.655 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.655 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.655 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.655 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.655 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.655 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.656 INFO:teuthology.orchestra.run.smithi012.stdout: "threshold": { 2024-05-17T22:55:08.656 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "threshold", 2024-05-17T22:55:08.656 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-17T22:55:08.656 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.656 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.656 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3.0", 2024-05-17T22:55:08.656 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1.0", 2024-05-17T22:55:08.656 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.656 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.656 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "scaling threshold", 2024-05-17T22:55:08.656 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-17T22:55:08.656 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.656 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.656 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.657 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.657 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.657 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.657 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "progress", 2024-05-17T22:55:08.657 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:08.657 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:08.657 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:08.657 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_pg_recovery_event": { 2024-05-17T22:55:08.657 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_pg_recovery_event", 2024-05-17T22:55:08.657 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.657 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.657 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.657 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.657 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.658 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.658 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.658 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-17T22:55:08.658 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.658 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.658 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.658 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.658 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": { 2024-05-17T22:55:08.658 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enabled", 2024-05-17T22:55:08.658 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.658 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.658 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.658 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.659 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.659 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.659 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.659 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.659 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.659 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.659 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.659 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.659 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:08.659 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:08.659 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.659 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.659 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.659 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.660 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.660 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.660 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.660 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.660 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.660 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.660 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.660 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.660 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.660 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.660 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.660 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.660 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.660 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.660 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.661 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:08.661 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:08.661 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.661 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.661 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.661 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.661 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.661 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.661 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.661 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.661 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.661 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.661 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.661 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.662 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:08.662 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:08.662 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.662 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.662 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.662 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:08.662 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.662 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.662 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.662 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.662 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.662 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.662 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.662 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.663 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.663 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.663 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.663 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.663 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.663 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.663 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.663 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:08.663 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:08.663 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.663 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.663 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.663 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.663 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.664 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.664 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.664 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.664 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.664 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.664 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.664 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.664 INFO:teuthology.orchestra.run.smithi012.stdout: "max_completed_events": { 2024-05-17T22:55:08.664 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_completed_events", 2024-05-17T22:55:08.664 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.664 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.664 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.664 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "50", 2024-05-17T22:55:08.664 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.664 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.665 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.665 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "number of past completed events to remember", 2024-05-17T22:55:08.665 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.665 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.665 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.665 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.665 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-17T22:55:08.665 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-17T22:55:08.665 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:08.665 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.665 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.665 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-05-17T22:55:08.665 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.666 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.666 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.666 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long the module is going to sleep", 2024-05-17T22:55:08.666 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.666 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.666 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.666 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.666 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:08.666 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:08.666 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.666 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:08.666 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.666 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.666 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.667 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.667 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.667 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.667 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.667 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.667 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.667 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.667 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.667 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.667 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.667 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prometheus", 2024-05-17T22:55:08.667 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:08.667 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:08.667 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:08.668 INFO:teuthology.orchestra.run.smithi012.stdout: "cache": { 2024-05-17T22:55:08.668 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cache", 2024-05-17T22:55:08.668 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.668 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.668 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.668 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.668 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.668 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.668 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.668 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.668 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.668 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.668 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.668 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.669 INFO:teuthology.orchestra.run.smithi012.stdout: "exclude_perf_counters": { 2024-05-17T22:55:08.669 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "exclude_perf_counters", 2024-05-17T22:55:08.669 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.669 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.669 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.669 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.669 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.669 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.669 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.669 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-17T22:55:08.669 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-17T22:55:08.669 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.669 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.669 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.670 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:08.670 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:08.670 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.670 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.670 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.670 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.670 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.670 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.670 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.670 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.670 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.670 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.670 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.671 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:08 smithi012 ceph-mon[31632]: pgmap v915: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-17T22:55:08.671 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:08 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/2607710169' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-17T22:55:08.671 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:08 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/613988101' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-17T22:55:08.671 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:08 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/519088477' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-17T22:55:08.674 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:08 smithi151 ceph-mon[36101]: pgmap v915: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-17T22:55:08.674 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:08 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/2607710169' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-17T22:55:08.674 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:08 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/613988101' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-17T22:55:08.674 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:08 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/519088477' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-17T22:55:08.674 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:08 smithi096 ceph-mon[36065]: pgmap v915: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-17T22:55:08.674 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:08 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/2607710169' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-17T22:55:08.675 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:08 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/613988101' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-17T22:55:08.675 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:08 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/519088477' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-17T22:55:08.679 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.679 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.679 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.679 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.679 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.679 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.679 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.679 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.680 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:08.680 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:08.680 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.680 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.680 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.680 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.680 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.680 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.680 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.680 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.680 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.680 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.680 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.681 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.681 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:08.681 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:08.681 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.681 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.681 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.681 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:08.681 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.681 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.681 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.681 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.681 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.681 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.682 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.682 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.682 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.682 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.682 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.682 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.682 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.682 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.682 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.682 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:08.682 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:08.682 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.682 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.683 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.683 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.683 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.683 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.683 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.683 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.683 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.683 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.683 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.683 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.683 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_stats_pools": { 2024-05-17T22:55:08.683 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_stats_pools", 2024-05-17T22:55:08.683 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.684 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.684 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.684 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.684 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.684 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.684 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.684 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.684 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.684 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.684 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.684 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.684 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-17T22:55:08.684 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-17T22:55:08.684 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.685 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.685 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.685 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "300", 2024-05-17T22:55:08.685 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.685 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.685 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.685 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.685 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.685 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.685 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.685 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.685 INFO:teuthology.orchestra.run.smithi012.stdout: "scrape_interval": { 2024-05-17T22:55:08.685 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "scrape_interval", 2024-05-17T22:55:08.686 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-17T22:55:08.686 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.686 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.686 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15.0", 2024-05-17T22:55:08.686 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.686 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.686 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.686 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.686 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.686 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.686 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.686 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.686 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-05-17T22:55:08.687 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-05-17T22:55:08.687 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.687 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.687 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.687 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "::", 2024-05-17T22:55:08.687 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.687 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.687 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.687 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-17T22:55:08.687 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.687 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.687 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.687 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.687 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-05-17T22:55:08.688 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-05-17T22:55:08.688 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.688 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.688 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.688 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "9283", 2024-05-17T22:55:08.688 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.688 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.688 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.688 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-17T22:55:08.688 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.688 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.688 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.688 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.688 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:08.689 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:08.689 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.689 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:08.689 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.689 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.689 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.689 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.689 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.689 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.689 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.689 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.689 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.689 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.689 INFO:teuthology.orchestra.run.smithi012.stdout: "stale_cache_strategy": { 2024-05-17T22:55:08.690 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "stale_cache_strategy", 2024-05-17T22:55:08.690 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.690 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.690 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.690 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "log", 2024-05-17T22:55:08.690 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.690 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.690 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.690 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.690 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.690 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.690 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.690 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.691 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_behaviour": { 2024-05-17T22:55:08.691 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_behaviour", 2024-05-17T22:55:08.691 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.691 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.691 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.691 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "default", 2024-05-17T22:55:08.691 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.691 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.691 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.691 INFO:teuthology.orchestra.run.smithi012.stdout: "default", 2024-05-17T22:55:08.691 INFO:teuthology.orchestra.run.smithi012.stdout: "error" 2024-05-17T22:55:08.691 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.691 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.692 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.692 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.692 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.692 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.692 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_error_status_code": { 2024-05-17T22:55:08.692 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_error_status_code", 2024-05-17T22:55:08.692 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.692 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.692 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.692 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-05-17T22:55:08.692 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "400", 2024-05-17T22:55:08.692 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "599", 2024-05-17T22:55:08.692 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.692 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.692 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.693 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.693 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.693 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.693 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.693 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.693 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.693 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_support", 2024-05-17T22:55:08.693 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:08.693 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:08.693 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:08.693 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:08.693 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:08.693 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.693 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.694 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.694 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.694 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.694 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.694 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.694 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.694 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.694 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.694 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.694 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.694 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.694 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.695 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.695 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.695 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.695 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.695 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.695 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:08.695 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:08.695 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.695 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.695 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.695 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.695 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.695 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.695 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.695 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.696 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.696 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.696 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.696 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.696 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:08.696 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:08.696 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.696 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.696 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.696 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:08.696 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.696 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.696 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.697 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.697 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.697 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.697 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.697 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.697 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.697 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.697 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.697 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.697 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.697 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.697 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.697 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:08.697 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:08.697 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.698 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.698 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.698 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.698 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.698 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.698 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.698 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.698 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.698 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.698 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.698 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.698 INFO:teuthology.orchestra.run.smithi012.stdout: "max_concurrent_snap_create": { 2024-05-17T22:55:08.698 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_concurrent_snap_create", 2024-05-17T22:55:08.699 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.699 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.699 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.699 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-17T22:55:08.699 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.699 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.699 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.699 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.699 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.699 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.699 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.699 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.699 INFO:teuthology.orchestra.run.smithi012.stdout: "mirror_snapshot_schedule": { 2024-05-17T22:55:08.700 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mirror_snapshot_schedule", 2024-05-17T22:55:08.700 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.700 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.700 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.700 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.700 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.700 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.700 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.700 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.700 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.700 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.700 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.700 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.700 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:08.700 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:08.701 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.701 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:08.701 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.701 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.701 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.701 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.701 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.701 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.701 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.701 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.701 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.701 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.701 INFO:teuthology.orchestra.run.smithi012.stdout: "trash_purge_schedule": { 2024-05-17T22:55:08.702 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "trash_purge_schedule", 2024-05-17T22:55:08.702 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.702 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.702 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.702 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.702 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.702 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.702 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.702 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.702 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.702 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.702 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.702 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.703 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.703 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.703 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.703 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "restful", 2024-05-17T22:55:08.703 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:08.703 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:08.703 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:08.703 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_auth": { 2024-05-17T22:55:08.703 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enable_auth", 2024-05-17T22:55:08.703 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.703 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.703 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.703 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.703 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.704 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.704 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.704 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.704 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.704 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.704 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.704 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.704 INFO:teuthology.orchestra.run.smithi012.stdout: "key_file": { 2024-05-17T22:55:08.704 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "key_file", 2024-05-17T22:55:08.704 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.704 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.704 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.704 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.704 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.705 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.705 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.705 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.705 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.705 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.705 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.705 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.705 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:08.705 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:08.705 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.705 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.705 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.705 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.705 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.705 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.706 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.706 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.706 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.706 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.706 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.706 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.706 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.706 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.706 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.706 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.706 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.706 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.706 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.707 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:08.707 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:08.707 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.707 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.707 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.707 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.707 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.707 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.707 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.707 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.707 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.707 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.707 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.708 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.708 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:08.708 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:08.708 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.708 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.708 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.708 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:08.708 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.708 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.708 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.708 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.708 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.708 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.708 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.709 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.709 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.709 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.709 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.709 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.709 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.709 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.709 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.709 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:08.709 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:08.709 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.709 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.709 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.710 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.710 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.710 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.710 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.710 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.710 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.710 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.710 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.710 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.710 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-05-17T22:55:08.710 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-05-17T22:55:08.710 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.710 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.711 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.711 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.711 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.711 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.711 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.711 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.711 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.711 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.711 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.711 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-05-17T22:55:08.711 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-05-17T22:55:08.711 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.711 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.712 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.712 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.712 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.712 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.712 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.712 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.712 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.712 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.712 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.712 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:08.712 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:08.712 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.712 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:08.713 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.713 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.713 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.713 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.713 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.713 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.713 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.713 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.713 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.713 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.713 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.713 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.713 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.713 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rgw", 2024-05-17T22:55:08.714 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:08.714 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:08.714 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:08.714 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:08.714 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:08.714 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.714 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.714 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.714 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.714 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.714 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.714 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.714 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.715 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.715 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.715 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.715 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.715 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.715 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.715 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.715 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.715 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.715 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.715 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.715 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:08.715 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:08.715 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.715 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.716 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.716 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.716 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.716 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.716 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.716 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.716 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.716 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.716 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.716 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.716 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:08.716 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:08.716 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.716 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.717 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.717 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:08.717 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.717 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.717 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.717 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.717 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.717 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.717 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.717 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.717 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.717 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.717 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.717 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.717 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.718 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.718 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.718 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:08.718 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:08.718 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.718 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.718 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.718 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.718 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.718 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.718 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.718 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.718 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.719 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.719 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.719 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.719 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:08.719 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:08.719 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.719 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:08.719 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.719 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.719 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.719 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.719 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.719 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.720 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.720 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.720 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.720 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.720 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.720 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.720 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.720 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rook", 2024-05-17T22:55:08.720 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:08.720 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:08.720 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:08.720 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:08.720 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:08.721 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.721 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.721 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.721 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.721 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.721 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.721 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.721 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.721 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.721 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.721 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.721 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.721 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.721 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.722 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.722 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.722 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.722 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.722 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:08.722 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:08.722 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.722 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.722 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.722 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.722 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.723 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.723 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.723 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.723 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.723 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.723 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.723 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.723 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:08.723 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:08.723 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.723 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.723 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.723 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:08.723 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.724 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.724 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.724 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.724 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.724 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.724 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.724 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.724 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.724 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.724 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.724 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.724 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.724 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.725 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.725 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:08.725 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:08.725 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.725 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.725 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.725 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.725 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.725 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.725 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.725 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.725 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.725 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.725 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.726 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:08.726 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:08.726 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.726 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:08.726 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.726 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.726 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.726 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.726 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.726 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.726 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.726 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.726 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.727 INFO:teuthology.orchestra.run.smithi012.stdout: "storage_class": { 2024-05-17T22:55:08.727 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "storage_class", 2024-05-17T22:55:08.727 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.727 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.727 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.727 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "local", 2024-05-17T22:55:08.727 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.727 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.727 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.727 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-17T22:55:08.727 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.727 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.727 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.727 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.728 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.728 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.728 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.728 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "selftest", 2024-05-17T22:55:08.728 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:08.728 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:08.728 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:08.728 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:08.728 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:08.728 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.728 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.728 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.728 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.728 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.728 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.729 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.729 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.729 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.729 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.729 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.729 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.729 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.729 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.729 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.729 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.729 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.729 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.730 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:08.730 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:08.730 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.730 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.730 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.730 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.730 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.730 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.730 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.730 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.730 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.730 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.730 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.731 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:08.731 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:08.731 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.731 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.731 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.731 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:08.731 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.731 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.731 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.731 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.731 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.731 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.731 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.732 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.732 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.732 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.732 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.732 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.732 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.732 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.732 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.732 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:08.732 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:08.732 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.732 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.732 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.733 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.733 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.733 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.733 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.733 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.733 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.733 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.733 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.733 INFO:teuthology.orchestra.run.smithi012.stdout: "roption1": { 2024-05-17T22:55:08.733 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "roption1", 2024-05-17T22:55:08.733 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.733 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.734 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.734 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.734 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.734 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.734 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.734 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.734 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.734 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.734 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.734 INFO:teuthology.orchestra.run.smithi012.stdout: "roption2": { 2024-05-17T22:55:08.734 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "roption2", 2024-05-17T22:55:08.734 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.734 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.735 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.735 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "xyz", 2024-05-17T22:55:08.735 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.735 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.735 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.735 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.735 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.735 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.735 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.735 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption1": { 2024-05-17T22:55:08.735 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption1", 2024-05-17T22:55:08.735 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.735 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.736 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.736 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.736 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.736 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.736 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.736 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.736 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.736 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.736 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.736 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption2": { 2024-05-17T22:55:08.736 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption2", 2024-05-17T22:55:08.736 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.737 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.737 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.737 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.737 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.737 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.737 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.737 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.737 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.737 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.737 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.737 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption3": { 2024-05-17T22:55:08.737 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption3", 2024-05-17T22:55:08.737 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-17T22:55:08.738 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.738 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.738 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.738 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.738 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.738 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.738 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.738 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.738 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.738 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.738 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption4": { 2024-05-17T22:55:08.739 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption4", 2024-05-17T22:55:08.739 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.739 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.739 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.739 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.739 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.739 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.739 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.739 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.739 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.739 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.739 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.740 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption5": { 2024-05-17T22:55:08.740 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption5", 2024-05-17T22:55:08.740 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.740 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.740 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.740 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.740 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.740 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.740 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.740 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.740 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.740 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.740 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.741 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption6": { 2024-05-17T22:55:08.741 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption6", 2024-05-17T22:55:08.741 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.741 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.741 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.741 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.741 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.741 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.741 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.741 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.741 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.741 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.741 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.742 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption7": { 2024-05-17T22:55:08.742 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption7", 2024-05-17T22:55:08.742 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.742 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.742 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.742 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.742 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-05-17T22:55:08.742 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "42", 2024-05-17T22:55:08.742 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.742 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.742 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.742 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.743 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.743 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:08.743 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:08.743 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.743 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:08.743 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.743 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.743 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.743 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.743 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.743 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.743 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.743 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.744 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.744 INFO:teuthology.orchestra.run.smithi012.stdout: "testkey": { 2024-05-17T22:55:08.744 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testkey", 2024-05-17T22:55:08.744 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.744 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.744 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.744 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.744 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.744 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.744 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.744 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.744 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.744 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.745 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.745 INFO:teuthology.orchestra.run.smithi012.stdout: "testlkey": { 2024-05-17T22:55:08.745 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testlkey", 2024-05-17T22:55:08.745 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.745 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.745 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.745 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.745 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.745 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.745 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.745 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.745 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.745 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.746 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.746 INFO:teuthology.orchestra.run.smithi012.stdout: "testnewline": { 2024-05-17T22:55:08.746 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testnewline", 2024-05-17T22:55:08.746 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.746 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.746 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.746 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.746 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.746 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.746 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.746 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.746 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.746 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.747 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.747 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.747 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.747 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smb", 2024-05-17T22:55:08.747 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:08.747 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:08.747 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:08.747 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:08.747 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:08.747 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.747 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.747 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.748 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.748 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.748 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.748 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.748 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.748 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.748 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.748 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.748 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.748 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.748 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.748 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.748 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.748 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.748 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.749 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:08.749 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:08.749 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.749 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.749 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.749 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.749 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.749 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.749 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.749 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.749 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.749 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.749 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.750 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:08.750 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:08.750 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.750 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.750 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.750 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:08.750 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.750 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.750 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.750 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.750 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.751 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.751 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.751 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.751 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.751 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.751 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.751 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.751 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.751 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.751 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:08.751 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:08.751 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.751 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.752 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.752 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.752 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.752 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.752 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.752 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.752 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.752 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.752 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.752 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:08.752 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:08.752 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.753 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:08.753 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.753 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.753 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.753 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.753 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.753 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.753 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.753 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.753 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.753 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.753 INFO:teuthology.orchestra.run.smithi012.stdout: "update_orchestration": { 2024-05-17T22:55:08.753 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "update_orchestration", 2024-05-17T22:55:08.753 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.754 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.754 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.754 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.754 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.754 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.754 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.754 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-17T22:55:08.754 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.754 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.754 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.754 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.754 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.755 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.755 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snap_schedule", 2024-05-17T22:55:08.755 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:08.755 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:08.755 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:08.755 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_m_granularity": { 2024-05-17T22:55:08.755 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_m_granularity", 2024-05-17T22:55:08.755 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.755 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.755 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.755 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.755 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.755 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.755 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.756 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow minute scheduled snapshots", 2024-05-17T22:55:08.756 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.756 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.756 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.756 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.756 INFO:teuthology.orchestra.run.smithi012.stdout: "dump_on_update": { 2024-05-17T22:55:08.756 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "dump_on_update", 2024-05-17T22:55:08.756 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.756 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.756 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.756 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.756 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.756 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.757 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.757 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "dump database to debug log on update", 2024-05-17T22:55:08.757 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.757 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.757 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.757 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.757 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:08.757 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:08.757 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.757 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.757 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.757 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.757 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.758 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.758 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.758 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.758 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.758 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.758 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.758 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.758 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.758 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.758 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.758 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.758 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.758 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.759 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:08.759 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:08.759 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.759 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.759 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.759 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.759 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.759 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.759 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.759 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.759 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.759 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.759 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.760 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:08.760 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:08.760 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.760 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.760 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.760 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:08.760 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.760 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.760 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.760 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.760 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.760 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.760 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.761 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.761 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.761 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.761 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.761 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.761 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.761 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.761 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.761 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:08.761 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:08.761 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.761 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.761 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.761 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.761 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.762 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.762 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.762 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.762 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.762 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.762 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.762 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:08.762 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:08.762 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.762 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:08.762 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.762 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.763 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.763 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.763 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.763 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.763 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.763 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.763 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.763 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.763 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.763 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.763 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "stats", 2024-05-17T22:55:08.763 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:08.763 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:08.764 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:08.764 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:08.764 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:08.764 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.764 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.764 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.764 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.764 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.764 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.764 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.764 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.764 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.764 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.764 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.765 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.765 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.765 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.765 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.765 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.765 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.765 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.765 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.765 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:08.765 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:08.765 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.765 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.765 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.765 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.766 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.766 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.766 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.766 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.766 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.766 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.766 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.766 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.766 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:08.766 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:08.766 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.766 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.766 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.767 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:08.767 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.767 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.767 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.767 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.767 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.767 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.767 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.767 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.767 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.767 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.767 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.767 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.767 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.768 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.768 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:08.768 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:08.768 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.768 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.768 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.768 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.768 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.768 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.768 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.768 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.768 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.768 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.769 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.769 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:08.769 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:08.769 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.769 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:08.769 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.769 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.769 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.769 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.769 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.769 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.769 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.769 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.770 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.770 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.770 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.770 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.770 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.770 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "status", 2024-05-17T22:55:08.770 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:08.770 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:08.770 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:08.770 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:08.770 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:08.770 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.770 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.771 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.771 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.771 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.771 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.771 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.771 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.771 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.771 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.771 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.771 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.771 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.771 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.771 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.772 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.772 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.772 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.772 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.772 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:08.772 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:08.772 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.772 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.772 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.772 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.772 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.772 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.772 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.772 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.773 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.773 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.773 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.773 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:08.773 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:08.773 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.773 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.773 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.773 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:08.773 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.773 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.773 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.773 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.774 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.774 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.774 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.774 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.774 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.774 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.774 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.774 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.774 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.774 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.774 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.774 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:08.774 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:08.775 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.775 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.775 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.775 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.775 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.775 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.775 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.775 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.775 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.775 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.775 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.775 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.775 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:08.775 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:08.775 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.776 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:08.776 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.776 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.776 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.776 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.776 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.776 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.776 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.776 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.776 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.776 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.776 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.776 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.776 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.777 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "telegraf", 2024-05-17T22:55:08.777 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:08.777 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:08.777 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:08.777 INFO:teuthology.orchestra.run.smithi012.stdout: "address": { 2024-05-17T22:55:08.777 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "address", 2024-05-17T22:55:08.777 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.777 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.777 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.777 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-17T22:55:08.777 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.777 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.777 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.777 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.778 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.778 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.778 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.778 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-17T22:55:08.778 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-17T22:55:08.778 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:08.778 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.778 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.778 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15", 2024-05-17T22:55:08.778 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.778 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.778 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.778 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.779 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.779 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.779 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.779 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.779 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:08.779 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:08.779 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.779 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.779 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.779 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.779 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.779 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.779 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.780 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.780 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.780 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.780 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.780 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.780 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.780 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.780 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.780 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.780 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.780 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.780 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.780 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:08.781 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:08.781 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.781 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.781 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.781 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.781 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.781 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.781 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.781 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.781 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.781 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.781 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.781 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.781 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:08.782 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:08.782 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.782 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.782 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.782 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:08.782 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.782 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.782 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.782 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.782 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.782 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.782 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.782 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.783 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.783 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.783 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.783 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.783 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.783 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.783 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.783 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:08.783 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:08.783 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.783 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.783 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.783 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.783 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.784 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.784 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.784 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.784 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.784 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.784 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.784 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.784 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:08.784 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:08.784 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.784 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:08.784 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.784 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.785 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.785 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.785 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.785 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.785 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.785 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.785 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.785 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.785 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.785 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.785 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "telemetry", 2024-05-17T22:55:08.786 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:08.786 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:08.786 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:08.786 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_basic": { 2024-05-17T22:55:08.786 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_basic", 2024-05-17T22:55:08.786 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.786 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.786 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.786 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.786 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.786 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.786 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.786 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-17T22:55:08.787 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.787 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.787 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.787 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.787 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_crash": { 2024-05-17T22:55:08.787 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_crash", 2024-05-17T22:55:08.787 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.787 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.787 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.787 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.787 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.787 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.787 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.787 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-17T22:55:08.787 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.788 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.788 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.788 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.788 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_device": { 2024-05-17T22:55:08.788 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_device", 2024-05-17T22:55:08.788 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.788 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.788 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.788 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.788 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.788 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.788 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.788 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-17T22:55:08.789 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.789 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.789 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.789 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.789 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_ident": { 2024-05-17T22:55:08.789 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_ident", 2024-05-17T22:55:08.789 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.789 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.789 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.789 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.789 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.789 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.789 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.789 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-17T22:55:08.789 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.790 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.790 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.790 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.790 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_perf": { 2024-05-17T22:55:08.790 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_perf", 2024-05-17T22:55:08.790 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.790 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.790 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.790 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.790 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.790 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.790 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.790 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-17T22:55:08.790 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.791 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.791 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.791 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.791 INFO:teuthology.orchestra.run.smithi012.stdout: "contact": { 2024-05-17T22:55:08.791 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "contact", 2024-05-17T22:55:08.791 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.791 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.791 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.791 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.791 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.791 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.791 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.791 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.792 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.792 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.792 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.792 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.792 INFO:teuthology.orchestra.run.smithi012.stdout: "description": { 2024-05-17T22:55:08.792 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "description", 2024-05-17T22:55:08.792 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.792 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.792 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.792 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.792 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.792 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.792 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.793 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.793 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.793 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.793 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.793 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.793 INFO:teuthology.orchestra.run.smithi012.stdout: "device_url": { 2024-05-17T22:55:08.793 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_url", 2024-05-17T22:55:08.793 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.793 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.793 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.793 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-17T22:55:08.793 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.793 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.793 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.794 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.794 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.794 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.794 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.794 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.794 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": { 2024-05-17T22:55:08.794 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enabled", 2024-05-17T22:55:08.794 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.794 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.794 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.794 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.794 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.794 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.795 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.795 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.795 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.795 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.795 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.795 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.795 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-17T22:55:08.795 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-17T22:55:08.795 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.795 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.795 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.795 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "24", 2024-05-17T22:55:08.795 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "8", 2024-05-17T22:55:08.796 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.796 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.796 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.796 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.796 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.796 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.796 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.796 INFO:teuthology.orchestra.run.smithi012.stdout: "last_opt_revision": { 2024-05-17T22:55:08.796 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "last_opt_revision", 2024-05-17T22:55:08.796 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.796 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.796 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.796 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1", 2024-05-17T22:55:08.796 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.797 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.797 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.797 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.797 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.797 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.797 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.797 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.797 INFO:teuthology.orchestra.run.smithi012.stdout: "leaderboard": { 2024-05-17T22:55:08.797 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "leaderboard", 2024-05-17T22:55:08.797 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.797 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.797 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.797 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.798 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.798 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.798 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.798 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.798 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.798 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.798 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.798 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.798 INFO:teuthology.orchestra.run.smithi012.stdout: "leaderboard_description": { 2024-05-17T22:55:08.798 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "leaderboard_description", 2024-05-17T22:55:08.798 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.798 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.798 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.798 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.799 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.799 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.799 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.799 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.799 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.799 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.799 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.799 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.799 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:08.799 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:08.799 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.799 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.799 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.799 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.799 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.800 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.800 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.800 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.800 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.800 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.800 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.800 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.800 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.800 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.800 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.800 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.800 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.800 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.800 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.801 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:08.801 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:08.801 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.801 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.801 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.801 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.801 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.801 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.801 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.801 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.801 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.801 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.801 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.801 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.801 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:08.802 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:08.802 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.802 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.802 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.802 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:08.802 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.802 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.802 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.802 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.802 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.802 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.802 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.802 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.803 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.803 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.803 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.803 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.803 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.803 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.803 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.803 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:08.803 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:08.803 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.803 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.803 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.803 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.803 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.803 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.804 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.804 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.804 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.804 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.804 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.804 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.804 INFO:teuthology.orchestra.run.smithi012.stdout: "organization": { 2024-05-17T22:55:08.804 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "organization", 2024-05-17T22:55:08.804 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.804 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.804 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.804 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.804 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.805 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.805 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.805 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.805 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.805 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.805 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.805 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.805 INFO:teuthology.orchestra.run.smithi012.stdout: "proxy": { 2024-05-17T22:55:08.805 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "proxy", 2024-05-17T22:55:08.805 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.805 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.805 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.805 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.805 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.806 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.806 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.806 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.806 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.806 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.806 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.806 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.806 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:08.806 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:08.806 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.806 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:08.806 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.806 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.807 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.807 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.807 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.807 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.807 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.807 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.807 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.807 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.807 INFO:teuthology.orchestra.run.smithi012.stdout: "url": { 2024-05-17T22:55:08.807 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "url", 2024-05-17T22:55:08.807 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.807 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.807 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.808 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-17T22:55:08.808 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.808 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.808 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.808 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.808 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.808 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.808 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.808 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.808 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.808 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.808 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.808 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "test_orchestrator", 2024-05-17T22:55:08.808 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:08.809 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:08.809 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:08.809 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:08.809 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:08.809 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.809 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.809 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.809 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.809 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.809 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.809 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.809 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.809 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.809 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.809 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.810 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.810 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.810 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.810 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.810 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.810 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.810 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.810 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.810 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:08.810 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:08.810 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.810 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.810 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.811 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.811 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.811 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.811 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.811 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.811 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.811 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.811 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.811 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.811 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:08.811 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:08.811 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.811 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.812 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.812 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:08.812 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.812 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.812 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.812 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.812 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.812 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.812 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.812 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.812 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.812 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.812 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.812 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.813 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.813 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.813 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.813 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:08.813 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:08.813 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.813 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.813 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.813 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.813 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.813 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.813 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.813 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.813 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.814 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.814 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.814 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.814 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:08.814 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:08.814 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.814 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:08.814 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.814 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.814 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.814 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.814 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.814 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.814 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.815 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.815 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.815 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.815 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.815 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.815 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.815 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "volumes", 2024-05-17T22:55:08.815 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:08.815 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:08.815 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:08.815 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:08.815 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:08.815 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.815 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.815 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.816 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.816 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.816 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.816 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.816 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.816 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.816 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.816 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.816 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.816 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.816 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.816 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.816 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.817 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.817 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.817 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.817 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:08.817 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:08.817 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.817 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.817 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.817 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.817 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.817 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.817 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.817 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.817 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.817 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.818 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.818 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.818 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:08.818 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:08.818 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.818 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.818 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.818 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:08.818 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.818 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.818 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.818 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.818 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.819 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.819 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.819 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.819 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.819 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.819 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.819 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.819 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.819 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.819 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.819 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:08.819 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:08.819 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.819 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.820 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.820 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.820 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.820 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.820 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.820 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.820 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.820 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.820 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.820 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.820 INFO:teuthology.orchestra.run.smithi012.stdout: "max_concurrent_clones": { 2024-05-17T22:55:08.820 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_concurrent_clones", 2024-05-17T22:55:08.820 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.820 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.821 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.821 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "4", 2024-05-17T22:55:08.821 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.821 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.821 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.821 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-17T22:55:08.821 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.821 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.821 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.821 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.821 INFO:teuthology.orchestra.run.smithi012.stdout: "periodic_async_work": { 2024-05-17T22:55:08.821 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "periodic_async_work", 2024-05-17T22:55:08.821 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.822 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.822 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.822 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.822 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.822 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.822 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.822 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Periodically check for async work", 2024-05-17T22:55:08.822 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.822 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.822 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.822 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.822 INFO:teuthology.orchestra.run.smithi012.stdout: "snapshot_clone_delay": { 2024-05-17T22:55:08.822 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snapshot_clone_delay", 2024-05-17T22:55:08.823 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.823 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.823 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.823 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.823 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.823 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.823 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.823 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-17T22:55:08.823 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.823 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.823 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.823 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.823 INFO:teuthology.orchestra.run.smithi012.stdout: "snapshot_clone_no_wait": { 2024-05-17T22:55:08.824 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snapshot_clone_no_wait", 2024-05-17T22:55:08.824 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.824 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.824 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.824 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.824 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.824 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.824 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.824 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-17T22:55:08.824 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.824 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.824 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.824 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.824 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:08.825 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:08.825 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.825 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:08.825 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.825 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.825 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.825 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.825 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.825 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.825 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.825 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.825 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.825 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.825 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.826 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.826 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.826 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix", 2024-05-17T22:55:08.826 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:08.826 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:08.826 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:08.826 INFO:teuthology.orchestra.run.smithi012.stdout: "discovery_interval": { 2024-05-17T22:55:08.826 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "discovery_interval", 2024-05-17T22:55:08.826 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-17T22:55:08.826 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.826 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.826 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "100", 2024-05-17T22:55:08.827 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.827 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.827 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.827 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.827 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.827 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.827 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.827 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.827 INFO:teuthology.orchestra.run.smithi012.stdout: "identifier": { 2024-05-17T22:55:08.827 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "identifier", 2024-05-17T22:55:08.827 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.827 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.827 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.828 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.828 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.828 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.828 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.828 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.828 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.828 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.828 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.828 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.828 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-17T22:55:08.828 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-17T22:55:08.828 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:08.828 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.829 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.829 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-17T22:55:08.829 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.829 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.829 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.829 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.829 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.829 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.829 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.829 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.829 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:08.829 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:08.829 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.830 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.830 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.830 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.830 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.830 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.830 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.830 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.830 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.830 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.830 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.830 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.830 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.830 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.831 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.831 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.831 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.831 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.831 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.831 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:08.831 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:08.831 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.831 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.831 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.831 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.831 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.831 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.832 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.832 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.832 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.832 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.832 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.832 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.832 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:08.832 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:08.832 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.832 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.832 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.832 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:08.832 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.832 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.833 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.833 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.833 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.833 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.833 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.833 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.833 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.833 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.833 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.833 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.833 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.833 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.833 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.833 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:08.834 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:08.834 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.834 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.834 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.834 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.834 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.834 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.834 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.834 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.834 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.834 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.834 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.834 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.835 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:08.835 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:08.835 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.835 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:08.835 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.835 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.835 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.835 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.835 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.835 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.835 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.835 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.835 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.836 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.836 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_host": { 2024-05-17T22:55:08.836 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_host", 2024-05-17T22:55:08.836 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.836 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.836 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.836 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.836 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.836 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.836 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.836 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.836 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.836 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.836 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.837 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.837 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_port": { 2024-05-17T22:55:08.837 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_port", 2024-05-17T22:55:08.837 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.837 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.837 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.837 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10051", 2024-05-17T22:55:08.837 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.837 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.837 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.837 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.837 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.837 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.838 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.838 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.838 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_sender": { 2024-05-17T22:55:08.838 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_sender", 2024-05-17T22:55:08.838 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.838 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.838 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.838 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-17T22:55:08.838 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.838 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.838 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.838 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.839 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.839 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.839 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.839 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.839 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.839 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.839 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-05-17T22:55:08.839 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.839 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.839 INFO:teuthology.orchestra.run.smithi012.stdout: "gid": 24116, 2024-05-17T22:55:08.839 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "y", 2024-05-17T22:55:08.839 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_features": 4540701547738038271, 2024-05-17T22:55:08.839 INFO:teuthology.orchestra.run.smithi012.stdout: "available_modules": [ 2024-05-17T22:55:08.839 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.840 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "alerts", 2024-05-17T22:55:08.840 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:08.840 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:08.840 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:08.840 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-17T22:55:08.840 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-17T22:55:08.840 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:08.840 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.840 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.840 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-17T22:55:08.840 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.840 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.840 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.840 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How frequently to reexamine health status", 2024-05-17T22:55:08.841 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.841 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.841 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.841 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.841 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:08.841 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:08.841 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.841 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.841 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.841 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.841 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.841 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.841 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.842 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.842 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.842 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.842 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.842 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.842 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.842 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.842 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.842 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.842 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.842 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.842 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.843 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:08.843 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:08.843 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.843 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.843 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.843 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.843 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.843 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.843 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.843 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.843 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.843 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.843 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.843 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.844 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:08.844 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:08.844 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.844 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.844 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.844 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:08.844 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.844 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.844 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.844 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.844 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.844 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.844 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.844 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.844 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.845 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.845 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.845 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.845 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.845 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.845 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.845 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:08.845 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:08.845 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.845 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.845 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.845 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.845 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.845 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.846 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.846 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.846 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.846 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.846 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.846 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.846 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_destination": { 2024-05-17T22:55:08.846 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_destination", 2024-05-17T22:55:08.846 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.846 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.846 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.846 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.846 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.846 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.847 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.847 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Email address to send alerts to", 2024-05-17T22:55:08.847 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.847 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.847 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.847 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.847 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_from_name": { 2024-05-17T22:55:08.847 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_from_name", 2024-05-17T22:55:08.847 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.847 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.847 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.847 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "Ceph", 2024-05-17T22:55:08.847 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.847 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.847 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.848 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Email From: name", 2024-05-17T22:55:08.848 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.848 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.848 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.848 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.848 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_host": { 2024-05-17T22:55:08.848 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_host", 2024-05-17T22:55:08.848 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.848 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.848 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.848 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.848 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.848 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.848 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.848 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP server", 2024-05-17T22:55:08.849 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.849 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.849 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.849 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.849 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_password": { 2024-05-17T22:55:08.849 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_password", 2024-05-17T22:55:08.849 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.849 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.849 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.849 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.849 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.849 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.849 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.849 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Password to authenticate with", 2024-05-17T22:55:08.849 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.850 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.850 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.850 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.850 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_port": { 2024-05-17T22:55:08.850 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_port", 2024-05-17T22:55:08.850 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.850 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.850 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.850 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "465", 2024-05-17T22:55:08.850 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.850 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.850 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.850 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP port", 2024-05-17T22:55:08.851 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.851 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.851 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.851 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.851 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_sender": { 2024-05-17T22:55:08.851 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_sender", 2024-05-17T22:55:08.851 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.851 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.851 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.851 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.851 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.851 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.851 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.851 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP envelope sender", 2024-05-17T22:55:08.851 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.852 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.852 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.852 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.852 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_ssl": { 2024-05-17T22:55:08.852 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_ssl", 2024-05-17T22:55:08.852 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.852 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.852 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.852 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.852 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.852 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.852 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.852 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-17T22:55:08.853 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.853 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.853 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.853 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.853 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_user": { 2024-05-17T22:55:08.853 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_user", 2024-05-17T22:55:08.853 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.853 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.853 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.853 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.853 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.853 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.853 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.854 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "User to authenticate as", 2024-05-17T22:55:08.854 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.854 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.854 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.854 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.854 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:08.854 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:08.854 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.854 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:08.854 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.854 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.854 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.854 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.855 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.855 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.855 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.855 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.855 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.855 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.855 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.855 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.855 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.855 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "balancer", 2024-05-17T22:55:08.855 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:08.855 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:08.855 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:08.856 INFO:teuthology.orchestra.run.smithi012.stdout: "active": { 2024-05-17T22:55:08.856 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "active", 2024-05-17T22:55:08.856 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.856 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.856 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.856 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.856 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.856 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.856 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.856 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically balance PGs across cluster", 2024-05-17T22:55:08.856 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.856 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.856 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.856 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.857 INFO:teuthology.orchestra.run.smithi012.stdout: "begin_time": { 2024-05-17T22:55:08.857 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "begin_time", 2024-05-17T22:55:08.857 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.857 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.857 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.857 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0000", 2024-05-17T22:55:08.857 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.857 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.857 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.857 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "beginning time of day to automatically balance", 2024-05-17T22:55:08.857 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-17T22:55:08.857 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.857 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.858 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.858 INFO:teuthology.orchestra.run.smithi012.stdout: "begin_weekday": { 2024-05-17T22:55:08.858 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "begin_weekday", 2024-05-17T22:55:08.858 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-17T22:55:08.858 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.858 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.858 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.858 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0", 2024-05-17T22:55:08.858 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "6", 2024-05-17T22:55:08.858 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.858 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-17T22:55:08.858 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-17T22:55:08.858 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.859 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.859 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.859 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_max_iterations": { 2024-05-17T22:55:08.859 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_max_iterations", 2024-05-17T22:55:08.859 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-17T22:55:08.859 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.859 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.859 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "25", 2024-05-17T22:55:08.859 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-05-17T22:55:08.859 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "250", 2024-05-17T22:55:08.859 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.859 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-17T22:55:08.859 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.859 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.860 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.860 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.860 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_metrics": { 2024-05-17T22:55:08.860 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_metrics", 2024-05-17T22:55:08.860 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.860 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.860 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.860 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "pgs,objects,bytes", 2024-05-17T22:55:08.860 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.860 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.860 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.860 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-17T22:55:08.860 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-17T22:55:08.861 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.861 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.861 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.861 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_step": { 2024-05-17T22:55:08.861 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_step", 2024-05-17T22:55:08.861 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-17T22:55:08.861 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.861 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.861 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0.5", 2024-05-17T22:55:08.861 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0.001", 2024-05-17T22:55:08.861 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "0.999", 2024-05-17T22:55:08.861 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.861 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "aggressiveness of optimization", 2024-05-17T22:55:08.861 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-17T22:55:08.862 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.862 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.862 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.862 INFO:teuthology.orchestra.run.smithi012.stdout: "end_time": { 2024-05-17T22:55:08.862 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "end_time", 2024-05-17T22:55:08.862 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.862 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.862 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.862 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "2359", 2024-05-17T22:55:08.862 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.862 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.862 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.862 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "ending time of day to automatically balance", 2024-05-17T22:55:08.863 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-17T22:55:08.863 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.863 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.863 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.863 INFO:teuthology.orchestra.run.smithi012.stdout: "end_weekday": { 2024-05-17T22:55:08.863 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "end_weekday", 2024-05-17T22:55:08.863 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-17T22:55:08.863 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.863 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.863 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.863 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0", 2024-05-17T22:55:08.863 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "6", 2024-05-17T22:55:08.863 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.864 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-17T22:55:08.864 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-17T22:55:08.864 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.864 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.864 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.864 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:08.864 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:08.864 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.864 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.864 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.864 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.864 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.864 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.864 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.865 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.865 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.865 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.865 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.865 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.865 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.865 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.865 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.865 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.865 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.865 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.865 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.865 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:08.865 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:08.865 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.866 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.866 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.866 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.866 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.866 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.866 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.866 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.866 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.866 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.866 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.866 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.866 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:08.866 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:08.867 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.867 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.867 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.867 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:08.867 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.867 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.867 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.867 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.867 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.867 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.867 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.867 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.867 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.868 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.868 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.868 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.868 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.868 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.868 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.868 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:08.868 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:08.868 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.868 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.868 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.868 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.868 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.868 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.869 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.869 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.869 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.869 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.869 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.869 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.869 INFO:teuthology.orchestra.run.smithi012.stdout: "min_score": { 2024-05-17T22:55:08.869 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "min_score", 2024-05-17T22:55:08.869 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-17T22:55:08.869 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.869 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.869 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.869 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.869 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.869 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.870 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-17T22:55:08.870 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.870 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.870 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.870 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.870 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": { 2024-05-17T22:55:08.870 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mode", 2024-05-17T22:55:08.870 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.870 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.870 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.870 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "upmap", 2024-05-17T22:55:08.870 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.870 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.870 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.871 INFO:teuthology.orchestra.run.smithi012.stdout: "crush-compat", 2024-05-17T22:55:08.871 INFO:teuthology.orchestra.run.smithi012.stdout: "none", 2024-05-17T22:55:08.871 INFO:teuthology.orchestra.run.smithi012.stdout: "read", 2024-05-17T22:55:08.871 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap", 2024-05-17T22:55:08.871 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap-read" 2024-05-17T22:55:08.871 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.871 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Balancer mode", 2024-05-17T22:55:08.871 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.871 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.871 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.871 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.871 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_ids": { 2024-05-17T22:55:08.871 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pool_ids", 2024-05-17T22:55:08.871 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.872 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.872 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.872 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.872 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.872 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.872 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.872 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-17T22:55:08.872 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.872 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.872 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.872 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.872 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-17T22:55:08.872 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-17T22:55:08.872 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:08.873 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.873 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.873 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-17T22:55:08.873 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.873 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.873 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.873 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-17T22:55:08.873 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.873 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.873 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.873 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.873 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:08.873 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:08.873 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.874 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:08.874 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.874 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.874 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.874 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.874 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.874 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.874 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.874 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.874 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.874 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.874 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap_max_deviation": { 2024-05-17T22:55:08.874 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "upmap_max_deviation", 2024-05-17T22:55:08.874 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.875 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.875 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.875 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-05-17T22:55:08.875 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-05-17T22:55:08.875 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.875 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.875 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-17T22:55:08.875 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-17T22:55:08.875 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.875 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.875 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.875 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap_max_optimizations": { 2024-05-17T22:55:08.875 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "upmap_max_optimizations", 2024-05-17T22:55:08.876 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-17T22:55:08.876 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.876 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.876 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-17T22:55:08.876 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.876 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.876 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.876 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-17T22:55:08.876 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.876 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.876 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.876 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.876 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.877 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.877 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.877 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cephadm", 2024-05-17T22:55:08.877 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:08.877 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:08.877 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:08.877 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_down_multiplier": { 2024-05-17T22:55:08.877 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_down_multiplier", 2024-05-17T22:55:08.877 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-17T22:55:08.877 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.877 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.877 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3.0", 2024-05-17T22:55:08.877 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.877 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.878 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.878 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-17T22:55:08.878 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.878 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.878 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.878 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.878 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_refresh_rate": { 2024-05-17T22:55:08.878 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_refresh_rate", 2024-05-17T22:55:08.878 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:08.878 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.878 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.878 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "20", 2024-05-17T22:55:08.878 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.878 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.879 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.879 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-17T22:55:08.879 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.879 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.879 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.879 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.879 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_starting_port": { 2024-05-17T22:55:08.879 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_starting_port", 2024-05-17T22:55:08.879 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.879 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.879 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.879 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "4721", 2024-05-17T22:55:08.879 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.879 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.880 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.880 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-17T22:55:08.880 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.880 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.880 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.880 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.880 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_ptrace": { 2024-05-17T22:55:08.880 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_ptrace", 2024-05-17T22:55:08.880 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.880 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.880 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.880 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.880 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.880 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.881 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.881 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-17T22:55:08.881 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-17T22:55:08.881 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.881 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.881 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.881 INFO:teuthology.orchestra.run.smithi012.stdout: "autotune_interval": { 2024-05-17T22:55:08.881 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "autotune_interval", 2024-05-17T22:55:08.881 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:08.881 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.881 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.881 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-17T22:55:08.881 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.881 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.881 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.882 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-17T22:55:08.882 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.882 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.882 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.882 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.882 INFO:teuthology.orchestra.run.smithi012.stdout: "autotune_memory_target_ratio": { 2024-05-17T22:55:08.882 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "autotune_memory_target_ratio", 2024-05-17T22:55:08.882 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-17T22:55:08.882 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.882 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.882 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0.7", 2024-05-17T22:55:08.882 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.882 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.882 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.883 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-17T22:55:08.883 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.883 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.883 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.883 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.883 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm_log_destination": { 2024-05-17T22:55:08.883 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cephadm_log_destination", 2024-05-17T22:55:08.883 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.883 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.883 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.883 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.883 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.883 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.883 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.883 INFO:teuthology.orchestra.run.smithi012.stdout: "file", 2024-05-17T22:55:08.883 INFO:teuthology.orchestra.run.smithi012.stdout: "file,syslog", 2024-05-17T22:55:08.884 INFO:teuthology.orchestra.run.smithi012.stdout: "syslog" 2024-05-17T22:55:08.884 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.884 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-17T22:55:08.884 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.884 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.884 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.884 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.884 INFO:teuthology.orchestra.run.smithi012.stdout: "cgroups_split": { 2024-05-17T22:55:08.884 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cgroups_split", 2024-05-17T22:55:08.884 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.884 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.884 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.884 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.884 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.885 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.885 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.885 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-17T22:55:08.885 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.885 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.885 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.885 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.885 INFO:teuthology.orchestra.run.smithi012.stdout: "config_checks_enabled": { 2024-05-17T22:55:08.885 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_checks_enabled", 2024-05-17T22:55:08.885 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.885 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.885 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.885 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.886 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.886 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.886 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.886 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-17T22:55:08.886 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.886 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.886 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.886 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.886 INFO:teuthology.orchestra.run.smithi012.stdout: "config_dashboard": { 2024-05-17T22:55:08.886 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_dashboard", 2024-05-17T22:55:08.886 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.886 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.886 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.887 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.887 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.887 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.887 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.887 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-17T22:55:08.887 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.887 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.887 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.887 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.887 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_alertmanager": { 2024-05-17T22:55:08.887 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_alertmanager", 2024-05-17T22:55:08.887 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.887 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.887 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.888 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-17T22:55:08.888 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.888 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.888 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.888 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-05-17T22:55:08.888 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.888 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.888 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.888 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.888 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_base": { 2024-05-17T22:55:08.888 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_base", 2024-05-17T22:55:08.888 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.888 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.888 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.889 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-17T22:55:08.889 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.889 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.889 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.889 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Container image name, without the tag", 2024-05-17T22:55:08.889 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.889 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.889 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.889 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.889 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_elasticsearch": { 2024-05-17T22:55:08.889 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_elasticsearch", 2024-05-17T22:55:08.889 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.889 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.889 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.890 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-17T22:55:08.890 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.890 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.890 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.890 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "elasticsearch container image", 2024-05-17T22:55:08.890 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.890 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.890 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.890 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.890 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_grafana": { 2024-05-17T22:55:08.890 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_grafana", 2024-05-17T22:55:08.890 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.890 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.890 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.891 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-17T22:55:08.891 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.891 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.891 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.891 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-05-17T22:55:08.891 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.891 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.891 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.891 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.891 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_haproxy": { 2024-05-17T22:55:08.891 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_haproxy", 2024-05-17T22:55:08.891 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.891 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.891 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.892 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-17T22:55:08.892 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.892 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.892 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.892 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "HAproxy container image", 2024-05-17T22:55:08.892 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.892 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.892 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.892 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.892 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_agent": { 2024-05-17T22:55:08.892 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_agent", 2024-05-17T22:55:08.892 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.892 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.892 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.893 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-17T22:55:08.893 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.893 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.893 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.893 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger agent container image", 2024-05-17T22:55:08.893 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.893 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.893 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.893 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.893 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_collector": { 2024-05-17T22:55:08.893 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_collector", 2024-05-17T22:55:08.893 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.893 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.893 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.893 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-17T22:55:08.894 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.894 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.894 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.894 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger collector container image", 2024-05-17T22:55:08.894 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.894 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.894 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.894 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.894 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_query": { 2024-05-17T22:55:08.894 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_query", 2024-05-17T22:55:08.894 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.894 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.894 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.894 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-17T22:55:08.895 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.895 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.895 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.895 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger query container image", 2024-05-17T22:55:08.895 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.895 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.895 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.895 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.895 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_keepalived": { 2024-05-17T22:55:08.895 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_keepalived", 2024-05-17T22:55:08.895 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.895 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.895 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.895 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-17T22:55:08.896 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.896 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.896 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.896 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Keepalived container image", 2024-05-17T22:55:08.896 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.896 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.896 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.896 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.896 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_loki": { 2024-05-17T22:55:08.896 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_loki", 2024-05-17T22:55:08.896 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.896 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.896 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.896 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-05-17T22:55:08.897 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.897 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.897 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.897 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Loki container image", 2024-05-17T22:55:08.897 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.897 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.897 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.897 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.897 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_node_exporter": { 2024-05-17T22:55:08.897 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_node_exporter", 2024-05-17T22:55:08.897 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.897 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.897 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.898 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-17T22:55:08.898 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.898 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.898 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.898 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-05-17T22:55:08.898 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.898 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.898 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.898 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.898 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_nvmeof": { 2024-05-17T22:55:08.898 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_nvmeof", 2024-05-17T22:55:08.898 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.898 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.898 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.898 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-17T22:55:08.899 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.899 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.899 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.899 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Nvme-of container image", 2024-05-17T22:55:08.899 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.899 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.899 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.899 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.899 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_prometheus": { 2024-05-17T22:55:08.899 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_prometheus", 2024-05-17T22:55:08.899 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.899 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.899 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.900 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-17T22:55:08.900 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.900 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.900 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.900 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-05-17T22:55:08.900 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.900 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.900 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.900 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.900 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_promtail": { 2024-05-17T22:55:08.900 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_promtail", 2024-05-17T22:55:08.900 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.900 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.900 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.901 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-05-17T22:55:08.901 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.901 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.901 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.901 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Promtail container image", 2024-05-17T22:55:08.901 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.901 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.901 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.901 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.901 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_samba": { 2024-05-17T22:55:08.901 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_samba", 2024-05-17T22:55:08.901 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.901 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.901 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.901 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-17T22:55:08.902 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.902 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.902 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.902 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Samba/SMB container image", 2024-05-17T22:55:08.902 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.902 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.902 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.902 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.902 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_snmp_gateway": { 2024-05-17T22:55:08.902 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_snmp_gateway", 2024-05-17T22:55:08.902 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.902 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.902 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.902 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-17T22:55:08.903 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.903 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.903 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.903 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SNMP Gateway container image", 2024-05-17T22:55:08.903 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.903 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.903 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.903 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.903 INFO:teuthology.orchestra.run.smithi012.stdout: "container_init": { 2024-05-17T22:55:08.903 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_init", 2024-05-17T22:55:08.903 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.903 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.904 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.904 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.904 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.904 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.904 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.904 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Run podman/docker with `--init`", 2024-05-17T22:55:08.904 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.904 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.904 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.904 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.904 INFO:teuthology.orchestra.run.smithi012.stdout: "daemon_cache_timeout": { 2024-05-17T22:55:08.904 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "daemon_cache_timeout", 2024-05-17T22:55:08.904 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:08.905 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.905 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.905 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-17T22:55:08.905 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.905 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.905 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.905 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-17T22:55:08.905 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.905 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.905 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.905 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.905 INFO:teuthology.orchestra.run.smithi012.stdout: "default_cephadm_command_timeout": { 2024-05-17T22:55:08.905 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default_cephadm_command_timeout", 2024-05-17T22:55:08.906 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.906 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.906 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.906 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "900", 2024-05-17T22:55:08.906 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.906 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.906 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.906 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-17T22:55:08.906 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.906 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.906 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.906 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.906 INFO:teuthology.orchestra.run.smithi012.stdout: "default_registry": { 2024-05-17T22:55:08.907 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default_registry", 2024-05-17T22:55:08.907 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.907 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.907 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.907 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io", 2024-05-17T22:55:08.907 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.907 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.907 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.907 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-17T22:55:08.907 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.907 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.907 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.907 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.907 INFO:teuthology.orchestra.run.smithi012.stdout: "device_cache_timeout": { 2024-05-17T22:55:08.908 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_cache_timeout", 2024-05-17T22:55:08.908 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:08.908 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.908 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.908 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1800", 2024-05-17T22:55:08.908 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.908 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.908 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.908 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache device inventory", 2024-05-17T22:55:08.908 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.908 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.908 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.908 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.908 INFO:teuthology.orchestra.run.smithi012.stdout: "device_enhanced_scan": { 2024-05-17T22:55:08.909 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_enhanced_scan", 2024-05-17T22:55:08.909 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.909 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.909 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.909 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.909 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.909 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.909 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.909 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-17T22:55:08.909 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.909 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.909 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.909 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.909 INFO:teuthology.orchestra.run.smithi012.stdout: "facts_cache_timeout": { 2024-05-17T22:55:08.910 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "facts_cache_timeout", 2024-05-17T22:55:08.910 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:08.910 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.910 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.910 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-17T22:55:08.910 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.910 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.910 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.910 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache host facts data", 2024-05-17T22:55:08.910 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.910 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.910 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.910 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.910 INFO:teuthology.orchestra.run.smithi012.stdout: "grafana_dashboards_path": { 2024-05-17T22:55:08.911 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "grafana_dashboards_path", 2024-05-17T22:55:08.911 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.911 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.911 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.911 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-17T22:55:08.911 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.911 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.911 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.911 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-17T22:55:08.911 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.911 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.911 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.911 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.911 INFO:teuthology.orchestra.run.smithi012.stdout: "host_check_interval": { 2024-05-17T22:55:08.911 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "host_check_interval", 2024-05-17T22:55:08.912 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:08.912 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.912 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.912 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-17T22:55:08.912 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.912 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.912 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.912 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to perform a host check", 2024-05-17T22:55:08.912 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.912 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.912 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.912 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.912 INFO:teuthology.orchestra.run.smithi012.stdout: "hw_monitoring": { 2024-05-17T22:55:08.913 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "hw_monitoring", 2024-05-17T22:55:08.913 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.913 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.913 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.913 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.913 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.913 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.913 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.913 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-17T22:55:08.913 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.913 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.913 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.914 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.914 INFO:teuthology.orchestra.run.smithi012.stdout: "inventory_list_all": { 2024-05-17T22:55:08.914 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "inventory_list_all", 2024-05-17T22:55:08.914 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.914 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.914 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.914 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.914 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.914 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.914 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.914 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-17T22:55:08.914 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.914 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.914 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.915 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.915 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:08.915 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:08.915 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.915 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.915 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.915 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.915 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.915 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.915 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.915 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.915 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.915 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.915 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.916 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.916 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.916 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.916 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.916 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.916 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.916 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.916 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.916 INFO:teuthology.orchestra.run.smithi012.stdout: "log_refresh_metadata": { 2024-05-17T22:55:08.916 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_refresh_metadata", 2024-05-17T22:55:08.916 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.916 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.916 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.916 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.917 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.917 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.917 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.917 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-17T22:55:08.917 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.917 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.917 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.917 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.917 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:08.917 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:08.917 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.917 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.917 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.917 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.918 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.918 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.918 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.918 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-17T22:55:08.918 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.918 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.918 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.918 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.918 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:08.918 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:08.918 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.918 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.918 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.918 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:08.919 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.919 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.919 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.919 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.919 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.919 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.919 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.919 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.919 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.919 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.919 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.919 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.919 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.919 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.920 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.920 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:08.920 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:08.920 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.920 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.920 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.920 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.920 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.920 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.920 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.920 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.920 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.920 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.920 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.921 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.921 INFO:teuthology.orchestra.run.smithi012.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-17T22:55:08.921 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-17T22:55:08.921 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.921 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.921 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.921 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.921 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.921 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.921 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.921 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-17T22:55:08.921 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.921 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.921 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.922 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.922 INFO:teuthology.orchestra.run.smithi012.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-17T22:55:08.922 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-17T22:55:08.922 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.922 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.922 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.922 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "*", 2024-05-17T22:55:08.922 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.922 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.922 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.922 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-17T22:55:08.922 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.922 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.923 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.923 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.923 INFO:teuthology.orchestra.run.smithi012.stdout: "max_count_per_host": { 2024-05-17T22:55:08.923 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_count_per_host", 2024-05-17T22:55:08.923 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.923 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.923 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.923 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-17T22:55:08.923 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.923 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.923 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.923 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "max number of daemons per service per host", 2024-05-17T22:55:08.923 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.923 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.923 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.924 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.924 INFO:teuthology.orchestra.run.smithi012.stdout: "max_osd_draining_count": { 2024-05-17T22:55:08.924 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_osd_draining_count", 2024-05-17T22:55:08.924 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.924 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.924 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.924 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-17T22:55:08.924 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.924 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.924 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.924 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-17T22:55:08.924 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.924 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.925 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.925 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.925 INFO:teuthology.orchestra.run.smithi012.stdout: "migration_current": { 2024-05-17T22:55:08.925 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "migration_current", 2024-05-17T22:55:08.925 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.925 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.925 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.925 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.925 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.925 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.925 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.925 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "internal - do not modify", 2024-05-17T22:55:08.925 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.925 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.926 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.926 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.926 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": { 2024-05-17T22:55:08.926 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mode", 2024-05-17T22:55:08.926 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.926 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.926 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.926 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "root", 2024-05-17T22:55:08.926 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.926 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.926 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.926 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm-package", 2024-05-17T22:55:08.926 INFO:teuthology.orchestra.run.smithi012.stdout: "root" 2024-05-17T22:55:08.927 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.927 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "mode for remote execution of cephadm", 2024-05-17T22:55:08.927 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.927 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.927 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.927 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.927 INFO:teuthology.orchestra.run.smithi012.stdout: "oob_default_addr": { 2024-05-17T22:55:08.927 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "oob_default_addr", 2024-05-17T22:55:08.927 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.927 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.927 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.927 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "169.254.1.1", 2024-05-17T22:55:08.927 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.927 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.927 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.928 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-17T22:55:08.928 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.928 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.928 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.928 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.928 INFO:teuthology.orchestra.run.smithi012.stdout: "prometheus_alerts_path": { 2024-05-17T22:55:08.928 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prometheus_alerts_path", 2024-05-17T22:55:08.928 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.928 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.928 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.928 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-17T22:55:08.928 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.928 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.928 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.929 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-17T22:55:08.929 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.929 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.929 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.929 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.929 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_insecure": { 2024-05-17T22:55:08.929 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_insecure", 2024-05-17T22:55:08.929 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.929 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.929 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.929 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.929 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.929 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.929 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.929 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-17T22:55:08.930 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.930 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.930 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.930 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.930 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_password": { 2024-05-17T22:55:08.930 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_password", 2024-05-17T22:55:08.930 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.930 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.930 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.930 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.930 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.930 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.930 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.931 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-17T22:55:08.931 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.931 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.931 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.931 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.931 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_url": { 2024-05-17T22:55:08.931 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_url", 2024-05-17T22:55:08.931 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.931 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.931 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.931 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.931 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.931 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.932 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.932 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-17T22:55:08.932 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.932 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.932 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.932 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.932 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_username": { 2024-05-17T22:55:08.932 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_username", 2024-05-17T22:55:08.932 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.932 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.932 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.932 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.932 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.932 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.932 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.933 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-17T22:55:08.933 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.933 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.933 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.933 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.933 INFO:teuthology.orchestra.run.smithi012.stdout: "secure_monitoring_stack": { 2024-05-17T22:55:08.933 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "secure_monitoring_stack", 2024-05-17T22:55:08.933 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.933 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.933 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.933 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.933 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.933 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.933 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.934 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-17T22:55:08.934 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.934 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.934 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.934 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.934 INFO:teuthology.orchestra.run.smithi012.stdout: "service_discovery_port": { 2024-05-17T22:55:08.934 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "service_discovery_port", 2024-05-17T22:55:08.934 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.934 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.934 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.934 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8765", 2024-05-17T22:55:08.934 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.934 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.935 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.935 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "cephadm service discovery port", 2024-05-17T22:55:08.935 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.935 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.935 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.935 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.935 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:08.935 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:08.935 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.935 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:08.935 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.935 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.935 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.935 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.936 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.936 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.936 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.936 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.936 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.936 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.936 INFO:teuthology.orchestra.run.smithi012.stdout: "ssh_config_file": { 2024-05-17T22:55:08.936 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssh_config_file", 2024-05-17T22:55:08.936 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.936 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.936 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.936 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.936 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.936 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.936 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.937 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-17T22:55:08.937 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.937 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.937 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.937 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.937 INFO:teuthology.orchestra.run.smithi012.stdout: "use_agent": { 2024-05-17T22:55:08.937 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "use_agent", 2024-05-17T22:55:08.937 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.937 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.937 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.937 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.937 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.937 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.938 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.938 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-17T22:55:08.938 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.938 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.938 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.938 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.938 INFO:teuthology.orchestra.run.smithi012.stdout: "use_repo_digest": { 2024-05-17T22:55:08.938 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "use_repo_digest", 2024-05-17T22:55:08.938 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.938 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.938 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.938 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.938 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.938 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.939 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.939 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-17T22:55:08.939 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.939 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.939 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.939 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.939 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_failed_host_check": { 2024-05-17T22:55:08.939 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_failed_host_check", 2024-05-17T22:55:08.939 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.939 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.939 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.939 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.939 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.939 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.939 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.940 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if the host check fails", 2024-05-17T22:55:08.940 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.940 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.940 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.940 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.940 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_stray_daemons": { 2024-05-17T22:55:08.940 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_stray_daemons", 2024-05-17T22:55:08.940 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.940 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.940 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.940 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.940 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.940 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.940 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.941 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-17T22:55:08.941 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.941 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.941 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.941 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.941 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_stray_hosts": { 2024-05-17T22:55:08.941 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_stray_hosts", 2024-05-17T22:55:08.941 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.941 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.941 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.941 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.941 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.941 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.941 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.941 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-17T22:55:08.942 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.942 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.942 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.942 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.942 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.942 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.942 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.942 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crash", 2024-05-17T22:55:08.942 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:08.942 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:08.942 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:08.942 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:08.942 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:08.943 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.943 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.943 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.943 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.943 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.943 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.943 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.943 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.943 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.943 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.943 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.943 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.943 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.943 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.944 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.944 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.944 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.944 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.944 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.944 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:08.944 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:08.944 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.944 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.944 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.944 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.944 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.944 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.944 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.945 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.945 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.945 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.945 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.945 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.945 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:08.945 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:08.945 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.945 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.945 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.945 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:08.945 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.945 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.946 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:08.946 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:08.946 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:08.946 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:08.946 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:08.946 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:08.946 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:08.946 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:08.946 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.946 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.946 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.946 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.946 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.946 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:08.947 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:08.947 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.947 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.947 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.947 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.947 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.947 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.947 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.947 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.947 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.947 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.947 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.947 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.947 INFO:teuthology.orchestra.run.smithi012.stdout: "retain_interval": { 2024-05-17T22:55:08.948 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "retain_interval", 2024-05-17T22:55:08.948 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:08.948 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.948 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.948 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "31536000", 2024-05-17T22:55:08.948 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.948 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.948 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.948 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-17T22:55:08.948 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.948 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.948 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.948 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.948 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:08.948 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:08.949 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.949 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:08.949 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.949 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.949 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.949 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.949 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.949 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.949 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.949 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.949 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.949 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.949 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_recent_interval": { 2024-05-17T22:55:08.949 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_recent_interval", 2024-05-17T22:55:08.949 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:08.950 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.950 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:08.950 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1209600", 2024-05-17T22:55:08.950 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.950 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.950 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.950 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-17T22:55:08.950 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.950 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.950 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.950 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.950 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:08.950 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.950 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:08.951 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "dashboard", 2024-05-17T22:55:08.951 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:08.951 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:08.951 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:08.951 INFO:teuthology.orchestra.run.smithi012.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-17T22:55:08.951 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-17T22:55:08.951 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.951 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.951 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.951 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-17T22:55:08.951 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.951 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.951 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.951 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.951 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.952 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.952 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.952 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.952 INFO:teuthology.orchestra.run.smithi012.stdout: "ALERTMANAGER_API_HOST": { 2024-05-17T22:55:08.952 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-17T22:55:08.952 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.952 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.952 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.952 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.952 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.952 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.952 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.952 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.952 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.953 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.953 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.953 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.953 INFO:teuthology.orchestra.run.smithi012.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-17T22:55:08.953 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-17T22:55:08.953 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.953 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.953 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.953 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.953 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.953 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.953 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.953 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.954 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.954 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.954 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.954 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.954 INFO:teuthology.orchestra.run.smithi012.stdout: "AUDIT_API_ENABLED": { 2024-05-17T22:55:08.954 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "AUDIT_API_ENABLED", 2024-05-17T22:55:08.954 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.954 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.954 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.954 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.954 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.954 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.955 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.955 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.955 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.955 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.955 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.955 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.955 INFO:teuthology.orchestra.run.smithi012.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-17T22:55:08.955 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-17T22:55:08.955 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.955 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.955 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.955 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.955 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.956 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.956 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.956 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.956 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.956 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.956 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.956 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.956 INFO:teuthology.orchestra.run.smithi012.stdout: "ENABLE_BROWSABLE_API": { 2024-05-17T22:55:08.956 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-17T22:55:08.956 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.956 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.956 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.956 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.957 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.957 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.957 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.957 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.957 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.957 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.957 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.957 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.957 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-17T22:55:08.957 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-17T22:55:08.957 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.957 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.957 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.957 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.958 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.958 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.958 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.958 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.958 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.958 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.958 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.958 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.958 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-17T22:55:08.958 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-17T22:55:08.958 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.962 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.962 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.962 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.962 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.962 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.962 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.962 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.962 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.962 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.962 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.963 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.963 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-17T22:55:08.963 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-17T22:55:08.963 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.963 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.963 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.963 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.963 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.963 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.963 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.963 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.963 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.963 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.963 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.964 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.964 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-17T22:55:08.964 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-17T22:55:08.964 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.964 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.964 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.964 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.964 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.964 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.964 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.964 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.964 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.964 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.964 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.965 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.965 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-17T22:55:08.965 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-17T22:55:08.965 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.965 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.965 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.965 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.965 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.965 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.965 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.965 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.965 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.965 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.965 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.966 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.966 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-17T22:55:08.966 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-17T22:55:08.966 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.966 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.966 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.966 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.966 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.966 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.966 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.966 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.966 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.966 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.966 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.967 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.967 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-17T22:55:08.967 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-17T22:55:08.967 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.967 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.967 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.967 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.967 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.967 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.967 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.967 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.967 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.968 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.968 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.968 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.968 INFO:teuthology.orchestra.run.smithi012.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-17T22:55:08.968 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-17T22:55:08.968 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.968 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.968 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.968 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.968 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.968 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.968 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.968 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.968 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.969 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.969 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.969 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.969 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_PASSWORD": { 2024-05-17T22:55:08.969 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-17T22:55:08.969 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.969 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.969 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.969 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-05-17T22:55:08.969 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.969 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.969 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.969 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.970 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.970 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.970 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.970 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.970 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-17T22:55:08.970 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-17T22:55:08.970 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.970 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.970 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.970 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.970 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.970 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.970 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.971 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.971 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.971 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.971 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.971 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.971 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_URL": { 2024-05-17T22:55:08.971 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_URL", 2024-05-17T22:55:08.971 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.971 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.971 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.971 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.971 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.971 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.972 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.972 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.972 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.972 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.972 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.972 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.972 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_USERNAME": { 2024-05-17T22:55:08.972 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-17T22:55:08.972 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.972 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.972 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.972 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-05-17T22:55:08.972 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.973 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.973 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.973 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.973 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.973 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.973 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.973 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.973 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-17T22:55:08.973 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-17T22:55:08.973 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.973 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.973 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.973 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.973 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.974 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.974 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.974 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.974 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.974 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.974 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.974 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.974 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-17T22:55:08.974 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-17T22:55:08.974 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.974 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.974 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.974 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.975 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.975 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.975 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.975 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.975 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.975 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.975 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.975 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.975 INFO:teuthology.orchestra.run.smithi012.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-17T22:55:08.975 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-17T22:55:08.975 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.975 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.975 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.975 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.976 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.976 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.976 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.976 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.976 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.976 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.976 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.976 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.976 INFO:teuthology.orchestra.run.smithi012.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-17T22:55:08.976 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-17T22:55:08.976 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.976 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.976 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.976 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.977 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.977 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.977 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.977 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.977 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.977 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.977 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.977 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.977 INFO:teuthology.orchestra.run.smithi012.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-17T22:55:08.977 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-17T22:55:08.977 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.977 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.977 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.977 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "[]", 2024-05-17T22:55:08.977 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.978 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.978 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.978 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.978 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.978 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.978 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.978 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.978 INFO:teuthology.orchestra.run.smithi012.stdout: "MULTICLUSTER_CONFIG": { 2024-05-17T22:55:08.978 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-17T22:55:08.978 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.978 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.978 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.978 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "{}", 2024-05-17T22:55:08.978 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.979 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.979 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.979 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.979 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.979 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.979 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.979 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.979 INFO:teuthology.orchestra.run.smithi012.stdout: "PROMETHEUS_API_HOST": { 2024-05-17T22:55:08.979 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-17T22:55:08.979 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.979 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.979 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.979 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.979 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.979 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.980 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.980 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.980 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.980 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.980 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.980 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.980 INFO:teuthology.orchestra.run.smithi012.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-17T22:55:08.980 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-17T22:55:08.980 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.980 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.980 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.980 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.980 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.980 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.980 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.981 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.981 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.981 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.981 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.981 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.981 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-17T22:55:08.981 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-17T22:55:08.981 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.981 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.981 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.981 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.981 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.982 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.982 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.982 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.982 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.982 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.982 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.982 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.982 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-17T22:55:08.982 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-17T22:55:08.982 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.982 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.982 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.982 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.982 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.983 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.983 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.983 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.983 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.983 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.983 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.983 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.983 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-17T22:55:08.983 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-17T22:55:08.983 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.983 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.983 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.983 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.983 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.984 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.984 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.984 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.984 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.984 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.984 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.984 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.984 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-17T22:55:08.984 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-17T22:55:08.984 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.984 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.984 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.984 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.985 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.985 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.985 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.985 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.985 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.985 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.985 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.985 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.985 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-17T22:55:08.985 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-17T22:55:08.985 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.985 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.986 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.986 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.986 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.986 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.986 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.986 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.986 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.986 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.986 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.986 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.986 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-17T22:55:08.986 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-17T22:55:08.986 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.986 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.987 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.987 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.987 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.987 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.987 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.987 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.987 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.987 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.987 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.987 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.987 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-17T22:55:08.987 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-17T22:55:08.987 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.987 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.988 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.988 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.988 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.988 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.988 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.988 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.988 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.988 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.988 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.988 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.988 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_ENABLED": { 2024-05-17T22:55:08.988 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-17T22:55:08.988 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.988 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.988 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.989 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.989 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.989 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.989 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.989 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.989 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.989 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.989 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.989 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.989 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-17T22:55:08.989 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-17T22:55:08.989 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.989 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.989 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.989 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-17T22:55:08.990 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.990 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.990 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.990 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.990 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.990 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.990 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.990 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.990 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-17T22:55:08.990 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-17T22:55:08.990 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.990 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.990 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.990 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-17T22:55:08.991 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.991 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.991 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.991 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.991 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.991 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.991 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.991 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.991 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-17T22:55:08.991 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-17T22:55:08.991 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.991 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.991 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.991 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8", 2024-05-17T22:55:08.992 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.992 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.992 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.992 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.992 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.992 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.992 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.992 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.992 INFO:teuthology.orchestra.run.smithi012.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-17T22:55:08.992 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-17T22:55:08.992 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.992 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.992 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.992 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "45", 2024-05-17T22:55:08.993 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.993 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.993 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.993 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.993 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.993 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.993 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.993 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.993 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_ACCESS_KEY": { 2024-05-17T22:55:08.993 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-17T22:55:08.993 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.993 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.993 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.993 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.994 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.994 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.994 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.994 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.994 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.994 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.994 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.994 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-17T22:55:08.994 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-17T22:55:08.994 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.994 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.994 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.995 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-05-17T22:55:08.995 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.995 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.995 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.995 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.995 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.995 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.995 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.995 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_SECRET_KEY": { 2024-05-17T22:55:08.995 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-17T22:55:08.995 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:08.995 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.996 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.996 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:08.996 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.996 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.996 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.996 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.996 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.996 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.996 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.996 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.996 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_SSL_VERIFY": { 2024-05-17T22:55:08.996 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-17T22:55:08.996 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.996 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.997 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.997 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:08.997 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.997 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.997 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.997 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.997 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.997 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.997 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.997 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.997 INFO:teuthology.orchestra.run.smithi012.stdout: "UNSAFE_TLS_v1_2": { 2024-05-17T22:55:08.997 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-17T22:55:08.997 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:08.997 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.998 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.998 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:08.998 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.998 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.998 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.998 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.998 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.998 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.998 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.998 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.998 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-17T22:55:08.998 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-17T22:55:08.998 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.999 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:08.999 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:08.999 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:08.999 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:08.999 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:08.999 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:08.999 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:08.999 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:08.999 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:08.999 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:08.999 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:08.999 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-17T22:55:08.999 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-17T22:55:08.999 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:08.999 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.000 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.000 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-17T22:55:09.000 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.000 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.000 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.000 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.000 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.000 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.000 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.000 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.000 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-17T22:55:09.000 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-17T22:55:09.000 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.000 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.001 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.001 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-05-17T22:55:09.001 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.001 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.001 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.001 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.001 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.001 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.001 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.001 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.001 INFO:teuthology.orchestra.run.smithi012.stdout: "cross_origin_url": { 2024-05-17T22:55:09.001 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cross_origin_url", 2024-05-17T22:55:09.001 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.001 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.002 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.002 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.002 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.002 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.002 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.002 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.002 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.002 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.002 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.002 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.002 INFO:teuthology.orchestra.run.smithi012.stdout: "crt_file": { 2024-05-17T22:55:09.002 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crt_file", 2024-05-17T22:55:09.002 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.003 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.003 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.003 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.003 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.003 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.003 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.003 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.003 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.003 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.003 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.003 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.003 INFO:teuthology.orchestra.run.smithi012.stdout: "debug": { 2024-05-17T22:55:09.003 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "debug", 2024-05-17T22:55:09.003 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.004 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.004 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.004 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.004 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.004 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.004 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.004 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable/disable debug options", 2024-05-17T22:55:09.004 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.004 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.004 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.004 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.004 INFO:teuthology.orchestra.run.smithi012.stdout: "jwt_token_ttl": { 2024-05-17T22:55:09.004 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "jwt_token_ttl", 2024-05-17T22:55:09.005 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.005 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.005 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.005 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "28800", 2024-05-17T22:55:09.005 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.005 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.005 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.005 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.005 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.005 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.005 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.005 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.005 INFO:teuthology.orchestra.run.smithi012.stdout: "key_file": { 2024-05-17T22:55:09.005 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "key_file", 2024-05-17T22:55:09.006 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.006 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.006 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.006 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.006 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.006 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.006 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.006 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.006 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.006 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.006 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.006 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.006 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.006 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.007 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.007 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.007 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.007 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.007 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.007 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.007 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.007 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.007 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.007 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.007 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.007 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.007 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.008 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.008 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.008 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.008 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.008 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.008 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.008 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.008 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.008 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.008 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.008 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.008 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.008 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.008 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.009 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.009 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.009 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.009 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.009 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.009 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.009 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.009 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.009 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.009 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.009 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.009 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.009 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.009 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.010 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.010 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.010 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.010 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.010 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.010 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.010 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.010 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.010 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.010 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.010 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.010 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.010 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.011 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.011 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.011 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.011 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.011 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.011 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.011 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.011 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.011 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.011 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.011 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.011 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.011 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.012 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.012 INFO:teuthology.orchestra.run.smithi012.stdout: "motd": { 2024-05-17T22:55:09.012 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "motd", 2024-05-17T22:55:09.012 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.012 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.012 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.012 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.012 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.012 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.012 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.012 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "The message of the day", 2024-05-17T22:55:09.012 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.012 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.012 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.013 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.013 INFO:teuthology.orchestra.run.smithi012.stdout: "redirect_resolve_ip_addr": { 2024-05-17T22:55:09.013 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "redirect_resolve_ip_addr", 2024-05-17T22:55:09.013 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.013 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.013 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.013 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.013 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.013 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.013 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.013 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.013 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.013 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.014 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.014 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.014 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-05-17T22:55:09.014 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-05-17T22:55:09.014 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.014 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.014 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.014 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "::", 2024-05-17T22:55:09.014 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.014 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.014 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.014 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.014 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.014 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.015 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.015 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.015 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-05-17T22:55:09.015 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-05-17T22:55:09.015 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.015 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.015 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.015 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8080", 2024-05-17T22:55:09.015 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.015 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.015 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.015 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.015 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.015 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.016 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.016 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.016 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.016 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.016 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.016 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.016 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.016 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.016 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.016 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.016 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.016 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.016 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.016 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.017 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.017 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.017 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl": { 2024-05-17T22:55:09.017 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl", 2024-05-17T22:55:09.017 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.017 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.017 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.017 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.017 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.017 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.017 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.017 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.017 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.017 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.018 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.018 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.018 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl_server_port": { 2024-05-17T22:55:09.018 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl_server_port", 2024-05-17T22:55:09.018 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.018 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.018 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.018 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8443", 2024-05-17T22:55:09.018 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.018 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.018 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.018 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.018 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.019 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.019 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.019 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.019 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_behaviour": { 2024-05-17T22:55:09.019 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_behaviour", 2024-05-17T22:55:09.019 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.019 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.019 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.019 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "redirect", 2024-05-17T22:55:09.019 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.019 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.019 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.019 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.019 INFO:teuthology.orchestra.run.smithi012.stdout: "redirect" 2024-05-17T22:55:09.019 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.020 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.020 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.020 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.020 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.020 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.020 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_error_status_code": { 2024-05-17T22:55:09.020 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_error_status_code", 2024-05-17T22:55:09.020 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.020 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.020 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.020 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-05-17T22:55:09.020 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "400", 2024-05-17T22:55:09.020 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "599", 2024-05-17T22:55:09.020 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.021 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.021 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.021 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.021 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.021 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.021 INFO:teuthology.orchestra.run.smithi012.stdout: "url_prefix": { 2024-05-17T22:55:09.021 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "url_prefix", 2024-05-17T22:55:09.021 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.021 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.021 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.021 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.021 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.021 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.021 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.022 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.022 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.022 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.022 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.022 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.022 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.022 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.022 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.022 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "devicehealth", 2024-05-17T22:55:09.022 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.022 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.022 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.022 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_monitoring": { 2024-05-17T22:55:09.022 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enable_monitoring", 2024-05-17T22:55:09.023 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.023 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.023 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.023 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.023 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.023 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.023 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.023 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "monitor device health metrics", 2024-05-17T22:55:09.023 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.023 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.023 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.023 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.023 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.024 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.024 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.024 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.024 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.024 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.024 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.024 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.024 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.024 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.024 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.024 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.024 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.024 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.025 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.025 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.025 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.025 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.025 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.025 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.025 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.025 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.025 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.025 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.025 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.025 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.025 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.026 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.026 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.026 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.026 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.026 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.026 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.026 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.026 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.026 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.026 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.026 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.026 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.027 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.027 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.027 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.027 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.027 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.027 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.027 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.027 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.027 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.027 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.027 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.027 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.027 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.027 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.028 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.028 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.028 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.028 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.028 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.028 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.028 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.028 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.028 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.028 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.028 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.028 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.028 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.028 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.029 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.029 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.029 INFO:teuthology.orchestra.run.smithi012.stdout: "mark_out_threshold": { 2024-05-17T22:55:09.029 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mark_out_threshold", 2024-05-17T22:55:09.029 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:09.029 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.029 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.029 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "2419200", 2024-05-17T22:55:09.029 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.029 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.029 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.029 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-17T22:55:09.029 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.030 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.030 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.030 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.030 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": { 2024-05-17T22:55:09.030 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pool_name", 2024-05-17T22:55:09.030 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.030 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.030 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.030 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "device_health_metrics", 2024-05-17T22:55:09.030 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.030 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.030 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.030 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-17T22:55:09.030 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.031 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.031 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.031 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.031 INFO:teuthology.orchestra.run.smithi012.stdout: "retention_period": { 2024-05-17T22:55:09.031 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "retention_period", 2024-05-17T22:55:09.031 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:09.031 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.031 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.031 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15552000", 2024-05-17T22:55:09.031 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.031 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.031 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.031 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long to retain device health metrics", 2024-05-17T22:55:09.031 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.032 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.032 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.032 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.032 INFO:teuthology.orchestra.run.smithi012.stdout: "scrape_frequency": { 2024-05-17T22:55:09.032 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "scrape_frequency", 2024-05-17T22:55:09.032 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:09.032 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.032 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.032 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "86400", 2024-05-17T22:55:09.032 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.032 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.032 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.032 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-17T22:55:09.032 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.033 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.033 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.033 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.033 INFO:teuthology.orchestra.run.smithi012.stdout: "self_heal": { 2024-05-17T22:55:09.033 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "self_heal", 2024-05-17T22:55:09.033 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.033 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.033 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.033 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.033 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.033 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.033 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.033 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-17T22:55:09.033 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.033 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.034 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.034 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.034 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-17T22:55:09.034 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-17T22:55:09.034 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:09.034 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.034 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.034 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-17T22:55:09.034 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.034 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.034 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.034 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to wake up and check device health", 2024-05-17T22:55:09.034 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.035 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.035 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.035 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.035 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.035 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.035 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.035 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.035 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.035 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.035 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.035 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.036 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.036 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.036 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.036 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.036 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.036 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.036 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_threshold": { 2024-05-17T22:55:09.036 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_threshold", 2024-05-17T22:55:09.036 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:09.036 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.036 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.036 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "7257600", 2024-05-17T22:55:09.036 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.037 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.037 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.037 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-17T22:55:09.037 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.037 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.037 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.037 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.037 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.037 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.037 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.037 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "diskprediction_local", 2024-05-17T22:55:09.037 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.037 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.037 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.038 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.038 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.038 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.038 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.038 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.038 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.038 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.038 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.038 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.038 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.038 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.038 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.038 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.039 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.039 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.039 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.039 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.039 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.039 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.039 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.039 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.039 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.039 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.039 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.039 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.039 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.039 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.040 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.040 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.040 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.040 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.040 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.040 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.040 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.040 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.040 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.040 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.040 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.040 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.040 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.040 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.041 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.041 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.041 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.041 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.041 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.041 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.041 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.041 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.041 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.041 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.041 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.041 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.041 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.041 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.042 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.042 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.042 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.042 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.042 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.042 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.042 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.042 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.042 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.042 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.042 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.042 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.042 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.042 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.043 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.043 INFO:teuthology.orchestra.run.smithi012.stdout: "predict_interval": { 2024-05-17T22:55:09.043 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "predict_interval", 2024-05-17T22:55:09.043 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.043 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.043 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.043 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "86400", 2024-05-17T22:55:09.043 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.043 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.043 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.043 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.043 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.043 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.043 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.044 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.044 INFO:teuthology.orchestra.run.smithi012.stdout: "predictor_model": { 2024-05-17T22:55:09.044 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "predictor_model", 2024-05-17T22:55:09.044 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.044 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.044 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.044 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "prophetstor", 2024-05-17T22:55:09.044 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.044 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.044 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.044 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.044 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.044 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.045 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.045 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.045 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-17T22:55:09.045 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-17T22:55:09.045 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.045 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.045 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.045 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-17T22:55:09.045 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.045 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.045 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.045 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.045 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.046 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.046 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.046 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.046 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.046 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.046 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.046 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.046 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.046 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.046 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.046 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.046 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.046 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.046 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.047 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.047 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.047 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.047 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.047 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.047 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.047 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "influx", 2024-05-17T22:55:09.047 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": false, 2024-05-17T22:55:09.047 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "influxdb python module not found", 2024-05-17T22:55:09.047 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.047 INFO:teuthology.orchestra.run.smithi012.stdout: "batch_size": { 2024-05-17T22:55:09.047 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "batch_size", 2024-05-17T22:55:09.047 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.047 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.047 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.048 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5000", 2024-05-17T22:55:09.048 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.048 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.048 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.048 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-17T22:55:09.048 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.048 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.048 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.048 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.048 INFO:teuthology.orchestra.run.smithi012.stdout: "database": { 2024-05-17T22:55:09.048 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "database", 2024-05-17T22:55:09.048 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.048 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.048 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.049 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "ceph", 2024-05-17T22:55:09.049 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.049 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.049 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.049 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-17T22:55:09.049 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.049 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.049 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.049 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.049 INFO:teuthology.orchestra.run.smithi012.stdout: "hostname": { 2024-05-17T22:55:09.049 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "hostname", 2024-05-17T22:55:09.049 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.049 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.049 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.050 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.050 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.050 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.050 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.050 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB server hostname", 2024-05-17T22:55:09.050 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.050 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.050 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.050 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.050 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-17T22:55:09.050 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-17T22:55:09.050 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:09.050 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.050 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.051 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "30", 2024-05-17T22:55:09.051 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "5", 2024-05-17T22:55:09.051 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.051 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.051 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-17T22:55:09.051 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.051 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.051 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.051 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.051 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.051 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.051 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.051 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.051 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.052 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.052 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.052 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.052 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.052 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.052 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.052 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.052 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.052 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.052 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.052 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.052 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.052 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.052 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.052 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.053 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.053 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.053 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.053 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.053 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.053 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.053 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.053 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.053 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.053 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.053 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.053 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.053 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.053 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.054 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.054 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.054 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.054 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.054 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.054 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.054 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.054 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.054 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.054 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.054 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.054 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.055 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.055 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.055 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.055 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.055 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.055 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.055 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.055 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.055 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.055 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.055 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.055 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.055 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.055 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.056 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.056 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.056 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.056 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.056 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.056 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.056 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.056 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.056 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.056 INFO:teuthology.orchestra.run.smithi012.stdout: "password": { 2024-05-17T22:55:09.056 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "password", 2024-05-17T22:55:09.056 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.056 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.057 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.057 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.057 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.057 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.057 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.057 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "password of InfluxDB server user", 2024-05-17T22:55:09.057 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.057 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.057 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.057 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.057 INFO:teuthology.orchestra.run.smithi012.stdout: "port": { 2024-05-17T22:55:09.057 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "port", 2024-05-17T22:55:09.057 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.057 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.058 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.058 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8086", 2024-05-17T22:55:09.058 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.058 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.058 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.058 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB server port", 2024-05-17T22:55:09.058 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.058 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.058 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.058 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.058 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.058 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.058 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.059 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.059 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.059 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.059 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.059 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.059 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.059 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.059 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.059 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.059 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.059 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.059 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl": { 2024-05-17T22:55:09.059 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl", 2024-05-17T22:55:09.059 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.059 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.060 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.060 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "false", 2024-05-17T22:55:09.060 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.060 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.060 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.060 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-17T22:55:09.060 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.060 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.060 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.060 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.060 INFO:teuthology.orchestra.run.smithi012.stdout: "threads": { 2024-05-17T22:55:09.060 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "threads", 2024-05-17T22:55:09.060 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.060 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.061 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.061 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-05-17T22:55:09.061 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-05-17T22:55:09.061 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "32", 2024-05-17T22:55:09.061 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.061 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-17T22:55:09.061 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.061 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.061 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.061 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.061 INFO:teuthology.orchestra.run.smithi012.stdout: "username": { 2024-05-17T22:55:09.061 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "username", 2024-05-17T22:55:09.061 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.061 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.061 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.062 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.062 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.062 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.062 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.062 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "username of InfluxDB server user", 2024-05-17T22:55:09.062 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.062 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.062 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.062 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.062 INFO:teuthology.orchestra.run.smithi012.stdout: "verify_ssl": { 2024-05-17T22:55:09.062 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "verify_ssl", 2024-05-17T22:55:09.062 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.062 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.063 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.063 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "true", 2024-05-17T22:55:09.063 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.063 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.063 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.063 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-17T22:55:09.063 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.063 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.063 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.063 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.063 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.063 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.063 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.063 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "insights", 2024-05-17T22:55:09.064 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.064 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.064 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.064 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.064 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.064 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.064 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.064 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.064 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.064 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.064 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.064 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.064 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.064 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.065 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.065 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.065 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.065 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.065 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.065 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.065 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.065 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.065 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.065 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.065 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.065 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.065 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.065 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.065 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.066 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.066 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.066 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.066 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.066 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.066 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.066 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.066 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.066 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.066 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.066 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.066 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.066 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.066 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.067 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.067 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.067 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.067 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.067 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.067 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.067 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.067 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.067 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.067 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.067 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.067 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.067 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.067 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.067 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.068 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.068 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.068 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.068 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.068 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.068 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.068 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.068 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.068 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.068 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.068 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.068 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.068 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.069 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.069 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.069 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.069 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.069 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.069 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.069 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.069 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.069 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.069 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.069 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.069 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.069 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.069 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.069 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.070 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.070 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.070 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.070 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.070 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "iostat", 2024-05-17T22:55:09.070 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.070 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.070 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.070 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.070 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.070 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.070 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.070 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.070 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.071 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.071 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.071 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.071 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.071 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.071 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.071 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.071 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.071 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.071 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.071 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.071 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.071 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.071 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.072 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.072 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.072 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.072 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.072 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.072 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.072 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.072 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.072 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.072 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.072 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.072 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.072 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.073 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.073 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.073 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.073 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.073 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.073 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.073 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.073 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.073 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.073 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.073 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.073 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.073 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.073 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.074 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.074 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.074 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.074 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.074 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.074 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.074 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.074 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.074 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.074 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.074 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.074 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.074 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.074 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.075 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.075 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.075 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.075 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.075 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.075 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.075 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.075 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.075 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.075 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.075 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.075 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.075 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.075 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.075 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.076 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.076 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.076 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.076 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.076 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.076 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.076 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.076 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.076 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.076 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.076 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.076 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "k8sevents", 2024-05-17T22:55:09.076 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.077 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.077 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.077 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_event_retention_days": { 2024-05-17T22:55:09.077 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ceph_event_retention_days", 2024-05-17T22:55:09.077 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.077 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.077 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.077 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "7", 2024-05-17T22:55:09.077 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.077 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.077 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.077 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-17T22:55:09.077 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.077 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.077 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.078 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.078 INFO:teuthology.orchestra.run.smithi012.stdout: "config_check_secs": { 2024-05-17T22:55:09.078 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_check_secs", 2024-05-17T22:55:09.078 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.078 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.078 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.078 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-17T22:55:09.078 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "10", 2024-05-17T22:55:09.078 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.078 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.078 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-17T22:55:09.078 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.078 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.078 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.079 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.079 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.079 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.079 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.079 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.079 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.079 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.079 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.079 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.079 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.079 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.079 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.079 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.079 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.079 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.080 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.080 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.080 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.080 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.080 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.080 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.080 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.080 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.080 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.080 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.080 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.080 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.080 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.081 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.081 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.081 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.081 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.081 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.081 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.081 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.081 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.081 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.081 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.081 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.081 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.081 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.082 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.082 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.082 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.082 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.082 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.082 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.082 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.082 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.082 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.082 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.082 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.082 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.082 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.082 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.082 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.083 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.083 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.083 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.083 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.083 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.083 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.083 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.083 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.083 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.083 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.083 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.083 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.083 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.083 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.084 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.084 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.084 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.084 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.084 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.084 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.084 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.084 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.084 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.084 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.084 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.084 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.084 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.084 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.085 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.085 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.085 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.085 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.085 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "localpool", 2024-05-17T22:55:09.085 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.085 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.085 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.085 INFO:teuthology.orchestra.run.smithi012.stdout: "failure_domain": { 2024-05-17T22:55:09.085 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "failure_domain", 2024-05-17T22:55:09.085 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.085 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.085 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.086 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "host", 2024-05-17T22:55:09.086 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.086 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.086 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.086 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "failure domain for any created local pool", 2024-05-17T22:55:09.086 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-17T22:55:09.086 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.086 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.086 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.086 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.086 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.086 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.086 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.087 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.087 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.087 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.087 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.087 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.087 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.087 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.087 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.087 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.087 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.087 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.087 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.087 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.087 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.088 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.088 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.088 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.088 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.088 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.088 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.088 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.088 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.088 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.088 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.088 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.088 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.088 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.088 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.089 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.089 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.089 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.089 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.089 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.089 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.089 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.089 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.089 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.089 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.089 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.089 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.089 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.089 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.090 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.090 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.090 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.090 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.090 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.090 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.090 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.090 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.090 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.090 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.090 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.090 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.091 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.091 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.091 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.091 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.091 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.091 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.091 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.091 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.091 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.091 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.091 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.091 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.091 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": { 2024-05-17T22:55:09.092 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "min_size", 2024-05-17T22:55:09.092 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.092 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.092 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.092 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.092 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.092 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.092 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.092 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default min_size for any created local pool", 2024-05-17T22:55:09.092 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-17T22:55:09.092 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.092 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.092 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.092 INFO:teuthology.orchestra.run.smithi012.stdout: "num_rep": { 2024-05-17T22:55:09.093 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "num_rep", 2024-05-17T22:55:09.093 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.093 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.093 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.093 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3", 2024-05-17T22:55:09.093 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.093 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.093 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.093 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default replica count for any created local pool", 2024-05-17T22:55:09.093 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.093 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.093 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.093 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.093 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": { 2024-05-17T22:55:09.094 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pg_num", 2024-05-17T22:55:09.094 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.094 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.094 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.094 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "128", 2024-05-17T22:55:09.094 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.094 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.094 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.094 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default pg_num for any created local pool", 2024-05-17T22:55:09.094 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.094 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.094 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.094 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.094 INFO:teuthology.orchestra.run.smithi012.stdout: "prefix": { 2024-05-17T22:55:09.095 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prefix", 2024-05-17T22:55:09.095 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.095 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.095 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.095 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.095 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.095 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.095 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.095 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "name prefix for any created local pool", 2024-05-17T22:55:09.095 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.095 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.095 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.095 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.095 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.096 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.096 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.096 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.096 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.096 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.096 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.096 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.096 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.096 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.096 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.096 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.096 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.096 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.096 INFO:teuthology.orchestra.run.smithi012.stdout: "subtree": { 2024-05-17T22:55:09.097 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "subtree", 2024-05-17T22:55:09.097 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.097 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.097 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.097 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "rack", 2024-05-17T22:55:09.097 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.097 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.097 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.097 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-17T22:55:09.097 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-17T22:55:09.097 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.097 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.097 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.098 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.098 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.098 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.098 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mds_autoscaler", 2024-05-17T22:55:09.098 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.098 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.098 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.098 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.098 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.098 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.098 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.098 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.098 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.099 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.099 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.099 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.099 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.099 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.099 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.099 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.099 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.099 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.099 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.099 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.099 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.099 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.099 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.099 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.100 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.100 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.100 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.100 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.100 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.100 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.100 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.100 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.100 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.100 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.100 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.100 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.100 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.100 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.101 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.101 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.101 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.101 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.101 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.101 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.101 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.101 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.101 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.101 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.101 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.101 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.101 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.101 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.102 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.102 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.102 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.102 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.102 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.102 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.102 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.102 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.102 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.102 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.102 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.102 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.102 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.103 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.103 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.103 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.103 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.103 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.103 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.103 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.103 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.103 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.103 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.103 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.103 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.103 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.103 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.104 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.104 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.104 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.104 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.104 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.104 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.104 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.104 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.104 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.104 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.104 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.104 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mirroring", 2024-05-17T22:55:09.104 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.104 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.104 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.105 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.105 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.105 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.105 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.105 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.105 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.105 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.105 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.105 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.105 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.105 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.105 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.105 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.105 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.106 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.106 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.106 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.106 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.106 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.106 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.106 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.106 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.106 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.106 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.106 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.106 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.106 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.107 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.107 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.107 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.107 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.107 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.107 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.107 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.107 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.107 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.107 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.107 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.107 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.107 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.108 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.108 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.108 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.108 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.108 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.108 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.108 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.108 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.108 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.108 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.108 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.108 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.108 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.108 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.109 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.109 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.109 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.109 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.109 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.109 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.109 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.109 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.109 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.109 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.109 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.109 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.109 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.109 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.110 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.110 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.110 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.110 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.110 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.110 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.110 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.110 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.110 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.110 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.110 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.110 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.110 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.111 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.111 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.111 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.111 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.111 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.111 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.111 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "nfs", 2024-05-17T22:55:09.111 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.111 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.111 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.111 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.111 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.111 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.111 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.111 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.112 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.112 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.112 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.112 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.112 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.112 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.112 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.112 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.112 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.112 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.112 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.112 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.112 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.112 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.112 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.113 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.113 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.113 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.113 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.113 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.113 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.113 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.113 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.113 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.113 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.113 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.113 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.113 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.113 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.114 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.114 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.114 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.114 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.114 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.114 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.114 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.114 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.114 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.114 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.114 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.114 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.114 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.114 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.115 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.115 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.115 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.115 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.115 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.115 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.115 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.115 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.115 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.115 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.115 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.115 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.115 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.115 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.116 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.116 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.116 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.116 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.116 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.116 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.116 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.116 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.116 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.116 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.116 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.116 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.116 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.116 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.116 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.117 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.117 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.117 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.117 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.117 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.117 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.117 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.117 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.117 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.117 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.117 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "orchestrator", 2024-05-17T22:55:09.117 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.117 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.118 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.118 INFO:teuthology.orchestra.run.smithi012.stdout: "fail_fs": { 2024-05-17T22:55:09.118 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "fail_fs", 2024-05-17T22:55:09.118 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.118 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.118 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.118 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.118 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.118 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.118 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.118 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-17T22:55:09.118 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.119 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.119 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.119 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.119 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.119 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.119 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.119 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.119 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.119 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.119 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.119 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.119 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.119 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.119 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.119 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.120 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.120 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.120 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.120 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.120 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.120 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.120 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.120 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.120 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.120 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.120 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.120 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.120 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.120 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.121 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.121 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.121 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.121 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.121 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.121 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.121 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.121 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.121 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.121 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.121 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.121 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.121 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.121 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.122 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.122 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.122 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.122 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.122 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.122 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.122 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.122 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.122 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.122 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.122 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.122 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.122 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.122 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.123 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.123 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.123 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.123 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.123 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.123 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.123 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.123 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.123 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.123 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.123 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.123 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.123 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.123 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.124 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.124 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.124 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator": { 2024-05-17T22:55:09.124 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "orchestrator", 2024-05-17T22:55:09.124 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.124 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.124 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.124 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.124 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.124 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.124 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.124 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm", 2024-05-17T22:55:09.124 INFO:teuthology.orchestra.run.smithi012.stdout: "rook", 2024-05-17T22:55:09.125 INFO:teuthology.orchestra.run.smithi012.stdout: "test_orchestrator" 2024-05-17T22:55:09.125 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.125 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Orchestrator backend", 2024-05-17T22:55:09.125 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.125 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.125 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.125 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.125 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.125 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.125 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.125 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.125 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.125 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.125 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.126 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.126 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.126 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.126 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.126 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.126 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.126 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.126 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.126 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.126 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.126 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "osd_perf_query", 2024-05-17T22:55:09.126 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.126 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.126 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.127 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.127 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.127 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.127 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.127 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.127 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.127 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.127 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.127 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.127 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.127 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.127 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.127 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.127 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.127 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.127 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.128 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.128 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.128 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.128 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.128 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.128 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.128 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.128 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.128 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.128 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.128 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.128 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.128 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.128 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.129 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.129 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.129 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.129 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.129 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.129 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.129 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.129 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.129 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.129 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.129 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.129 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.129 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.129 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.129 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.130 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.130 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.130 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.130 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.130 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.130 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.130 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.130 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.130 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.130 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.130 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.130 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.130 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.130 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.131 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.131 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.131 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.131 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.131 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.131 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.131 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.131 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.131 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.131 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.131 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.131 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.131 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.131 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.132 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.132 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.132 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.132 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.132 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.132 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.132 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.132 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.132 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.132 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.132 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.132 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.132 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.133 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.133 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "osd_support", 2024-05-17T22:55:09.133 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.133 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.133 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.133 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.133 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.133 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.133 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.133 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.133 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.133 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.133 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.133 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.134 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.134 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.134 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.134 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.134 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.134 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.134 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.134 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.134 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.134 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.134 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.134 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.134 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.134 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.135 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.135 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.135 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.135 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.135 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.135 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.135 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.135 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.135 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.135 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.135 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.135 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.135 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.135 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.136 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.136 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.136 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.136 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.136 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.136 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.136 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.136 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.136 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.136 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.136 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.136 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.136 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.136 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.137 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.137 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.137 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.137 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.137 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.137 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.137 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.137 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.137 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.137 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.137 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.137 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.137 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.137 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.138 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.138 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.138 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.138 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.138 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.138 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.138 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.138 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.138 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.138 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.138 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.138 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.138 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.139 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.139 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.139 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.139 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.139 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.139 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.139 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.139 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.139 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.139 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pg_autoscaler", 2024-05-17T22:55:09.139 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.139 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.139 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.139 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.139 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.140 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.140 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.140 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.140 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.140 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.140 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.140 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.140 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.140 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.140 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.140 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.140 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.140 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.140 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.141 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.141 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.141 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.141 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.141 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.141 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.141 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.141 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.141 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.141 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.141 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.141 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.141 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.141 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.142 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.142 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.142 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.142 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.142 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.142 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.142 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.142 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.142 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.142 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.142 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.142 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.142 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.143 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.143 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.143 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.143 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.143 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.143 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.143 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.143 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.143 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.143 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.143 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.143 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.143 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.143 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.144 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.144 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.144 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.144 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.144 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.144 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.144 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.144 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.144 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.144 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.144 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.144 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.144 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.144 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-17T22:55:09.144 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-17T22:55:09.145 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:09.145 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.145 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.145 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-17T22:55:09.145 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.145 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.145 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.145 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.145 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.145 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.145 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.145 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.145 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.145 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.146 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.146 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.146 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.146 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.146 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.146 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.146 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.146 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.146 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.146 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.146 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.146 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.146 INFO:teuthology.orchestra.run.smithi012.stdout: "threshold": { 2024-05-17T22:55:09.147 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "threshold", 2024-05-17T22:55:09.147 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-17T22:55:09.147 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.147 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.147 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3.0", 2024-05-17T22:55:09.147 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1.0", 2024-05-17T22:55:09.147 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.147 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.147 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "scaling threshold", 2024-05-17T22:55:09.147 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-17T22:55:09.147 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.147 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.147 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.147 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.147 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.148 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.148 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "progress", 2024-05-17T22:55:09.148 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.148 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.148 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.148 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_pg_recovery_event": { 2024-05-17T22:55:09.148 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_pg_recovery_event", 2024-05-17T22:55:09.148 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.148 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.148 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.148 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.148 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.148 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.148 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.149 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-17T22:55:09.149 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.149 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.149 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.149 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.149 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": { 2024-05-17T22:55:09.149 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enabled", 2024-05-17T22:55:09.149 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.149 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.149 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.149 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.149 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.149 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.149 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.149 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.150 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.150 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.150 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.150 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.150 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.150 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.150 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.150 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.150 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.150 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.150 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.150 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.150 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.150 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.151 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.151 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.151 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.151 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.151 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.151 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.151 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.151 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.151 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.151 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.151 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.151 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.151 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.152 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.152 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.152 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.152 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.152 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.152 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.152 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.152 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.152 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.152 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.152 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.152 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.152 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.152 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.152 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.153 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.153 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.153 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.153 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.153 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.153 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.153 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.153 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.153 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.153 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.153 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.153 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.153 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.153 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.154 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.154 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.154 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.154 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.154 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.154 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.154 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.154 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.154 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.154 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.154 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.154 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.154 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.154 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.155 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.155 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.155 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.155 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.155 INFO:teuthology.orchestra.run.smithi012.stdout: "max_completed_events": { 2024-05-17T22:55:09.155 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_completed_events", 2024-05-17T22:55:09.155 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.155 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.155 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.155 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "50", 2024-05-17T22:55:09.155 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.155 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.155 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.155 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "number of past completed events to remember", 2024-05-17T22:55:09.156 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.156 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.156 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.156 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.156 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-17T22:55:09.156 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-17T22:55:09.156 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:09.156 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.156 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.156 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-05-17T22:55:09.156 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.156 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.156 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.156 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long the module is going to sleep", 2024-05-17T22:55:09.157 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.157 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.157 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.157 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.157 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.157 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.157 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.157 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.157 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.157 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.157 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.157 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.157 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.158 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.158 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.158 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.158 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.158 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.158 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.158 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.158 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.158 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prometheus", 2024-05-17T22:55:09.158 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.158 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.158 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.158 INFO:teuthology.orchestra.run.smithi012.stdout: "cache": { 2024-05-17T22:55:09.158 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cache", 2024-05-17T22:55:09.159 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.159 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.159 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.159 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.159 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.159 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.159 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.159 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.159 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.159 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.159 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.159 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.159 INFO:teuthology.orchestra.run.smithi012.stdout: "exclude_perf_counters": { 2024-05-17T22:55:09.160 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "exclude_perf_counters", 2024-05-17T22:55:09.160 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.160 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.160 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.160 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.160 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.160 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.160 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.160 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-17T22:55:09.160 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-17T22:55:09.160 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.160 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.160 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.160 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.161 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.161 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.161 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.161 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.161 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.161 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.161 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.161 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.161 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.161 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.161 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.161 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.161 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.162 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.162 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.162 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.162 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.162 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.162 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.162 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.162 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.166 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.166 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.166 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.166 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.166 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.166 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.166 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.167 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.167 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.167 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.167 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.167 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.167 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.167 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.167 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.167 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.167 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.167 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.167 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.167 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.167 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.168 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.168 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.168 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.168 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.168 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.168 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.168 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.168 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.168 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.168 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.168 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.168 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.168 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.168 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.169 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.169 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.169 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.169 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.169 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.169 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.169 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.169 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.169 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.169 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.169 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.169 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.169 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.169 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_stats_pools": { 2024-05-17T22:55:09.170 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_stats_pools", 2024-05-17T22:55:09.170 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.170 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.170 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.170 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.170 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.170 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.170 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.170 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.170 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.170 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.170 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.170 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.170 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-17T22:55:09.171 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-17T22:55:09.171 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.171 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.171 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.171 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "300", 2024-05-17T22:55:09.171 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.171 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.171 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.171 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.171 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.171 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.171 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.171 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.171 INFO:teuthology.orchestra.run.smithi012.stdout: "scrape_interval": { 2024-05-17T22:55:09.171 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "scrape_interval", 2024-05-17T22:55:09.171 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-17T22:55:09.172 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.172 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.172 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15.0", 2024-05-17T22:55:09.172 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.172 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.172 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.172 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.172 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.172 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.172 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.172 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.172 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-05-17T22:55:09.172 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-05-17T22:55:09.173 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.173 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.173 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.173 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "::", 2024-05-17T22:55:09.173 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.173 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.173 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.173 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-17T22:55:09.173 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.173 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.173 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.173 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.173 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-05-17T22:55:09.173 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-05-17T22:55:09.173 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.174 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.174 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.174 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "9283", 2024-05-17T22:55:09.174 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.174 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.174 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.174 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-17T22:55:09.174 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.174 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.174 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.174 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.174 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.174 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.175 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.175 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.175 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.175 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.175 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.175 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.175 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.175 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.175 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.175 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.175 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.175 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.175 INFO:teuthology.orchestra.run.smithi012.stdout: "stale_cache_strategy": { 2024-05-17T22:55:09.175 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "stale_cache_strategy", 2024-05-17T22:55:09.176 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.176 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.176 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.176 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "log", 2024-05-17T22:55:09.176 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.176 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.176 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.176 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.176 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.176 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.176 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.176 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.176 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_behaviour": { 2024-05-17T22:55:09.176 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_behaviour", 2024-05-17T22:55:09.177 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.177 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.177 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.177 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "default", 2024-05-17T22:55:09.177 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.177 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.177 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.177 INFO:teuthology.orchestra.run.smithi012.stdout: "default", 2024-05-17T22:55:09.177 INFO:teuthology.orchestra.run.smithi012.stdout: "error" 2024-05-17T22:55:09.177 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.177 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.177 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.177 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.177 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.178 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.178 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_error_status_code": { 2024-05-17T22:55:09.178 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_error_status_code", 2024-05-17T22:55:09.178 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.178 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.178 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.178 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-05-17T22:55:09.178 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "400", 2024-05-17T22:55:09.178 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "599", 2024-05-17T22:55:09.178 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.178 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.178 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.178 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.179 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.179 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.179 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.179 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.179 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.179 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_support", 2024-05-17T22:55:09.179 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.179 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.179 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.179 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.179 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.179 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.179 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.179 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.180 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.180 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.180 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.180 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.180 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.180 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.180 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.180 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.180 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.180 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.180 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.180 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.180 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.180 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.181 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.181 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.181 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.181 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.181 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.181 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.181 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.181 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.181 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.181 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.181 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.181 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.181 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.181 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.182 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.182 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.182 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.182 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.182 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.182 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.182 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.182 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.182 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.182 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.182 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.182 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.182 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.183 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.183 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.183 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.183 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.183 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.183 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.183 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.183 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.183 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.183 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.183 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.183 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.183 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.183 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.184 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.184 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.184 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.184 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.184 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.184 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.184 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.184 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.184 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.184 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.184 INFO:teuthology.orchestra.run.smithi012.stdout: "max_concurrent_snap_create": { 2024-05-17T22:55:09.184 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_concurrent_snap_create", 2024-05-17T22:55:09.184 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.184 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.184 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.185 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-17T22:55:09.185 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.185 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.185 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.185 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.185 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.185 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.185 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.185 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.185 INFO:teuthology.orchestra.run.smithi012.stdout: "mirror_snapshot_schedule": { 2024-05-17T22:55:09.185 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mirror_snapshot_schedule", 2024-05-17T22:55:09.185 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.186 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.186 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.186 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.186 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.186 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.186 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.186 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.186 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.186 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.186 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.186 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.186 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.186 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.186 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.187 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.187 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.187 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.187 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.187 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.187 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.187 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.187 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.187 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.187 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.187 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.187 INFO:teuthology.orchestra.run.smithi012.stdout: "trash_purge_schedule": { 2024-05-17T22:55:09.187 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "trash_purge_schedule", 2024-05-17T22:55:09.187 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.187 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.188 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.188 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.188 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.188 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.188 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.188 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.188 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.188 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.188 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.188 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.188 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.188 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.188 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.188 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "restful", 2024-05-17T22:55:09.188 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.189 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.189 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.189 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_auth": { 2024-05-17T22:55:09.189 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enable_auth", 2024-05-17T22:55:09.189 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.189 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.189 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.189 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.189 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.189 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.189 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.189 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.189 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.189 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.190 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.190 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.190 INFO:teuthology.orchestra.run.smithi012.stdout: "key_file": { 2024-05-17T22:55:09.190 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "key_file", 2024-05-17T22:55:09.190 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.190 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.190 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.190 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.190 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.190 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.190 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.190 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.191 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.191 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.191 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.191 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.191 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.191 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.191 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.191 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.191 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.191 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.191 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.191 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.191 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.191 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.191 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.192 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.192 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.192 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.192 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.192 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.192 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.192 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.192 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.192 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.192 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.192 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.192 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.192 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.192 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.193 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.193 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.193 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.193 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.193 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.193 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.193 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.193 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.193 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.193 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.193 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.193 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.193 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.193 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.194 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.194 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.194 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.194 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.194 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.194 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.194 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.194 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.194 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.194 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.194 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.194 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.194 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.194 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.195 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.195 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.195 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.195 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.195 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.195 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.195 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.195 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.195 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.195 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.195 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.195 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.195 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.195 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.195 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.196 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.196 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.196 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-05-17T22:55:09.196 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-05-17T22:55:09.196 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.196 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.196 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.196 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.196 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.196 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.196 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.196 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.196 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.196 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.196 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.197 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.197 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-05-17T22:55:09.197 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-05-17T22:55:09.197 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.197 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.197 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.197 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.197 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.197 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.197 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.197 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.197 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.197 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.197 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.198 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.198 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.198 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.198 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.198 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.198 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.198 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.198 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.198 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.198 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.198 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.198 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.198 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.199 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.199 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.199 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.199 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.199 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.199 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rgw", 2024-05-17T22:55:09.199 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.199 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.199 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.199 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.199 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.199 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.199 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.199 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.199 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.200 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.200 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.200 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.200 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.200 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.200 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.200 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.200 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.200 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.200 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.200 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.200 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.200 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.200 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.200 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.201 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.201 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.201 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.201 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.201 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.201 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.201 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.201 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.201 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.201 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.201 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.201 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.201 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.201 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.202 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.202 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.202 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.202 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.202 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.202 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.202 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.202 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.202 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.202 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.202 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.202 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.202 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.202 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.203 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.203 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.203 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.203 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.203 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.203 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.203 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.203 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.203 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.203 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.203 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.203 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.203 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.203 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.204 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.204 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.204 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.204 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.204 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.204 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.204 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.204 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.204 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.204 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.204 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.204 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.204 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.204 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.205 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.205 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.205 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.205 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.205 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.205 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.205 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.205 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.205 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.205 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.205 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rook", 2024-05-17T22:55:09.205 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.205 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.205 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.206 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.206 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.206 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.206 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.206 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.206 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.206 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.206 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.206 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.206 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.206 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.206 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.206 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.207 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.207 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.207 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.207 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.207 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.207 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.207 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.207 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.207 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.207 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.207 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.207 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.207 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.207 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.208 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.208 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.208 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.208 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.208 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.208 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.208 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.208 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.208 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.208 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.208 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.208 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.208 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.208 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.209 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.209 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.209 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.209 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.209 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.209 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.209 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.209 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.209 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.209 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.209 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.209 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.209 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.209 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.210 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.210 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.210 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.210 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.210 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.210 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.210 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.210 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.210 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.210 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.210 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.210 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.210 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.211 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.211 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.211 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.211 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.211 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.211 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.211 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.211 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.211 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.211 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.211 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.211 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.211 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.211 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.212 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.212 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.212 INFO:teuthology.orchestra.run.smithi012.stdout: "storage_class": { 2024-05-17T22:55:09.212 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "storage_class", 2024-05-17T22:55:09.212 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.212 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.212 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.212 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "local", 2024-05-17T22:55:09.212 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.212 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.212 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.212 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-17T22:55:09.212 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.212 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.213 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.213 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.213 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.213 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.213 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.213 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "selftest", 2024-05-17T22:55:09.213 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.213 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.213 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.213 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.213 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.213 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.213 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.213 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.214 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.214 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.214 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.214 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.214 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.214 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.214 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.214 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.214 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.214 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.214 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.214 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.214 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.215 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.215 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.215 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.215 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.215 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.215 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.215 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.215 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.215 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.215 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.215 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.215 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.215 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.215 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.216 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.216 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.216 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.216 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.216 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.216 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.216 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.216 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.216 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.216 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.216 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.216 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.216 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.216 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.217 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.217 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.217 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.217 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.217 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.217 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.217 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.217 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.217 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.217 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.217 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.217 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.217 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.217 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.217 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.218 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.218 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.218 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.218 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.218 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.218 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.218 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.218 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.218 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.218 INFO:teuthology.orchestra.run.smithi012.stdout: "roption1": { 2024-05-17T22:55:09.218 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "roption1", 2024-05-17T22:55:09.218 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.218 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.219 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.219 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.219 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.219 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.219 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.219 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.219 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.219 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.219 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.219 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.219 INFO:teuthology.orchestra.run.smithi012.stdout: "roption2": { 2024-05-17T22:55:09.219 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "roption2", 2024-05-17T22:55:09.219 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.220 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.220 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.220 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "xyz", 2024-05-17T22:55:09.220 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.220 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.220 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.220 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.220 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.220 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.220 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.220 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.220 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption1": { 2024-05-17T22:55:09.220 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption1", 2024-05-17T22:55:09.220 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.220 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.221 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.221 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.221 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.221 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.221 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.221 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.221 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.221 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.221 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.221 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.221 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption2": { 2024-05-17T22:55:09.221 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption2", 2024-05-17T22:55:09.221 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.221 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.222 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.222 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.222 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.222 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.222 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.222 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.222 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.222 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.222 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.222 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.222 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption3": { 2024-05-17T22:55:09.222 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption3", 2024-05-17T22:55:09.222 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-17T22:55:09.222 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.223 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.223 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.223 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.223 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.223 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.223 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.223 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.223 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.223 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.223 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.223 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption4": { 2024-05-17T22:55:09.223 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption4", 2024-05-17T22:55:09.223 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.223 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.224 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.224 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.224 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.224 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.224 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.224 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.224 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.224 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.224 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.224 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.224 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption5": { 2024-05-17T22:55:09.224 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption5", 2024-05-17T22:55:09.224 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.224 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.224 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.225 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.225 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.225 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.225 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.225 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.225 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.225 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.225 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.225 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.225 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption6": { 2024-05-17T22:55:09.225 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption6", 2024-05-17T22:55:09.225 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.225 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.226 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.226 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.226 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.226 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.226 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.226 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.226 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.226 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.226 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.226 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.226 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption7": { 2024-05-17T22:55:09.226 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption7", 2024-05-17T22:55:09.226 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.226 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.227 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.227 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.227 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-05-17T22:55:09.227 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "42", 2024-05-17T22:55:09.227 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.227 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.227 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.227 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.227 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.227 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.227 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.227 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.227 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.227 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.228 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.228 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.228 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.228 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.228 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.228 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.228 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.228 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.228 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.228 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.228 INFO:teuthology.orchestra.run.smithi012.stdout: "testkey": { 2024-05-17T22:55:09.228 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testkey", 2024-05-17T22:55:09.228 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.228 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.229 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.229 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.229 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.229 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.229 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.229 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.229 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.229 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.229 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.229 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.229 INFO:teuthology.orchestra.run.smithi012.stdout: "testlkey": { 2024-05-17T22:55:09.229 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testlkey", 2024-05-17T22:55:09.229 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.229 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.229 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.230 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.230 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.230 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.230 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.230 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.230 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.230 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.230 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.230 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.230 INFO:teuthology.orchestra.run.smithi012.stdout: "testnewline": { 2024-05-17T22:55:09.230 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testnewline", 2024-05-17T22:55:09.230 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.230 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.231 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.231 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.231 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.231 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.231 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.231 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.231 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.231 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.231 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.231 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.231 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.231 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.231 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.231 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smb", 2024-05-17T22:55:09.231 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.232 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.232 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.232 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.232 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.232 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.232 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.232 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.232 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.232 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.232 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.232 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.232 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.232 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.233 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.233 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.233 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.233 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.233 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.233 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.233 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.233 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.233 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.233 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.233 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.233 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.233 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.233 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.234 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.234 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.234 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.234 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.234 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.234 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.234 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.234 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.234 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.234 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.234 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.234 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.234 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.235 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.235 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.235 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.235 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.235 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.235 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.235 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.235 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.235 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.235 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.235 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.235 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.235 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.235 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.236 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.236 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.236 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.236 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.236 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.236 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.236 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.236 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.236 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.236 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.236 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.236 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.236 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.236 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.237 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.237 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.237 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.237 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.237 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.237 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.237 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.237 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.237 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.237 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.237 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.237 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.237 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.237 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.238 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.238 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.238 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.238 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.238 INFO:teuthology.orchestra.run.smithi012.stdout: "update_orchestration": { 2024-05-17T22:55:09.238 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "update_orchestration", 2024-05-17T22:55:09.238 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.238 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.238 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.238 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.238 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.238 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.238 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.239 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-17T22:55:09.239 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.239 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.239 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.239 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.239 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.239 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.239 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snap_schedule", 2024-05-17T22:55:09.239 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.239 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.239 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.239 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_m_granularity": { 2024-05-17T22:55:09.239 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_m_granularity", 2024-05-17T22:55:09.240 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.240 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.240 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.240 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.240 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.240 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.240 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.240 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow minute scheduled snapshots", 2024-05-17T22:55:09.240 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.240 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.240 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.240 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.240 INFO:teuthology.orchestra.run.smithi012.stdout: "dump_on_update": { 2024-05-17T22:55:09.240 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "dump_on_update", 2024-05-17T22:55:09.241 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.241 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.241 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.241 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.241 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.241 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.241 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.241 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "dump database to debug log on update", 2024-05-17T22:55:09.241 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.241 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.241 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.241 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.241 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.241 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.242 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.242 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.242 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.242 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.242 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.242 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.242 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.242 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.242 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.242 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.242 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.242 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.242 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.243 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.243 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.243 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.243 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.243 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.243 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.243 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.243 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.243 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.243 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.243 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.243 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.243 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.244 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.244 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.244 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.244 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.244 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.244 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.244 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.244 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.244 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.244 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.244 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.244 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.245 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.245 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.245 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.245 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.245 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.245 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.245 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.245 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.245 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.245 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.245 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.245 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.245 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.246 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.246 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.246 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.246 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.246 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.246 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.246 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.246 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.246 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.246 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.246 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.246 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.247 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.247 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.247 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.247 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.247 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.247 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.247 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.247 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.247 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.247 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.247 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.247 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.247 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.248 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.248 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.248 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.248 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "stats", 2024-05-17T22:55:09.248 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.248 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.248 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.248 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.248 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.248 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.248 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.248 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.249 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.249 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.249 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.249 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.249 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.249 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.249 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.249 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.249 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.249 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.249 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.249 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.249 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.249 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.249 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.250 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.250 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.250 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.250 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.250 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.250 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.250 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.250 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.250 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.250 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.250 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.250 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.251 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.251 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.251 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.251 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.251 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.251 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.251 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.251 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.251 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.251 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.251 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.251 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.251 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.251 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.252 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.252 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.252 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.252 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.252 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.252 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.252 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.252 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.252 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.252 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.252 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.252 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.252 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.253 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.253 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.253 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.253 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.253 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.253 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.253 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.253 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.253 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.253 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.253 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.253 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.253 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.253 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.254 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.254 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.254 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.254 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.254 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.254 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.254 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.254 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.254 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.254 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "status", 2024-05-17T22:55:09.254 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.254 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.254 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.255 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.255 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.255 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.255 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.255 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.255 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.255 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.255 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.255 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.255 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.255 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.255 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.255 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.255 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.256 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.256 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.256 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.256 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.256 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.256 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.256 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.256 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.256 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.256 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.256 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.256 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.256 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.256 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.257 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.257 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.257 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.257 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.257 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.257 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.257 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.257 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.257 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.257 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.257 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.257 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.257 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.258 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.258 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.258 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.258 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.258 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.258 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.258 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.258 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.258 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.258 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.258 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.258 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.258 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.259 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.259 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.259 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.259 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.259 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.259 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.259 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.259 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.259 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.259 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.259 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.259 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.259 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.260 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.260 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.260 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.260 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.260 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.260 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.260 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.260 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.260 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.260 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.260 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.260 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.260 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.261 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.261 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.261 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "telegraf", 2024-05-17T22:55:09.261 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.261 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.261 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.261 INFO:teuthology.orchestra.run.smithi012.stdout: "address": { 2024-05-17T22:55:09.261 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "address", 2024-05-17T22:55:09.261 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.261 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.261 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.261 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-17T22:55:09.261 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.262 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.262 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.262 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.262 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.262 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.262 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.262 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-17T22:55:09.262 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-17T22:55:09.262 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:09.262 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.262 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.262 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15", 2024-05-17T22:55:09.262 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.263 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.263 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.263 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.263 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.263 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.263 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.263 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.263 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.263 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.263 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.263 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.263 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.263 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.263 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.263 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.264 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.264 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.264 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.264 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.264 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.264 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.264 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.264 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.264 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.264 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.264 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.264 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.264 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.265 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.265 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.265 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.265 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.265 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.265 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.265 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.265 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.265 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.265 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.265 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.265 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.265 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.266 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.266 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.266 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.266 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.266 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.266 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.266 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.266 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.266 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.266 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.266 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.266 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.266 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.267 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.267 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.267 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.267 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.267 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.267 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.267 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.267 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.267 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.267 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.267 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.267 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.267 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.267 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.268 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.268 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.268 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.268 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.268 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.268 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.268 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.268 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.268 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.268 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.268 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.268 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.268 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.269 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.269 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.269 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.269 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.269 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.269 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.269 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.269 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "telemetry", 2024-05-17T22:55:09.269 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.269 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.269 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.269 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_basic": { 2024-05-17T22:55:09.269 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_basic", 2024-05-17T22:55:09.270 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.270 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.270 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.270 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.270 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.270 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.270 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.270 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-17T22:55:09.270 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.270 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.270 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.270 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_crash": { 2024-05-17T22:55:09.270 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_crash", 2024-05-17T22:55:09.271 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.271 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.271 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.271 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.271 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.271 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.271 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.271 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-17T22:55:09.271 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.271 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.271 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.271 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_device": { 2024-05-17T22:55:09.272 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_device", 2024-05-17T22:55:09.272 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.272 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.272 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.272 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.272 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.272 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.272 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.272 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-17T22:55:09.272 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.272 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.272 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.272 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_ident": { 2024-05-17T22:55:09.273 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_ident", 2024-05-17T22:55:09.273 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.273 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.273 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.273 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.273 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.273 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.273 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.273 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-17T22:55:09.273 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.273 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.273 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.273 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.273 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_perf": { 2024-05-17T22:55:09.274 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_perf", 2024-05-17T22:55:09.274 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.274 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.274 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.274 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.274 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.274 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.274 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.274 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-17T22:55:09.274 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.274 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.274 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.274 INFO:teuthology.orchestra.run.smithi012.stdout: "contact": { 2024-05-17T22:55:09.275 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "contact", 2024-05-17T22:55:09.275 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.275 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.275 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.275 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.275 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.275 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.275 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.275 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.275 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.275 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.275 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.275 INFO:teuthology.orchestra.run.smithi012.stdout: "description": { 2024-05-17T22:55:09.276 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "description", 2024-05-17T22:55:09.276 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.276 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.276 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.276 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.276 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.276 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.276 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.276 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.276 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.276 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.276 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.277 INFO:teuthology.orchestra.run.smithi012.stdout: "device_url": { 2024-05-17T22:55:09.277 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_url", 2024-05-17T22:55:09.277 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.277 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.277 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.277 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-17T22:55:09.277 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.277 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.277 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.277 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.277 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.277 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.277 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.278 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": { 2024-05-17T22:55:09.278 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enabled", 2024-05-17T22:55:09.278 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.278 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.278 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.278 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.278 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.278 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.278 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.278 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.278 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.278 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.278 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.279 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-17T22:55:09.279 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-17T22:55:09.279 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.279 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.279 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.279 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "24", 2024-05-17T22:55:09.279 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "8", 2024-05-17T22:55:09.279 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.279 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.279 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.279 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.279 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.279 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.280 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.280 INFO:teuthology.orchestra.run.smithi012.stdout: "last_opt_revision": { 2024-05-17T22:55:09.280 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "last_opt_revision", 2024-05-17T22:55:09.280 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.280 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.280 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.280 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1", 2024-05-17T22:55:09.280 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.280 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.280 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.280 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.280 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.280 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.280 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.281 INFO:teuthology.orchestra.run.smithi012.stdout: "leaderboard": { 2024-05-17T22:55:09.281 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "leaderboard", 2024-05-17T22:55:09.281 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.281 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.281 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.281 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.281 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.281 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.281 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.281 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.281 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.281 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.282 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.282 INFO:teuthology.orchestra.run.smithi012.stdout: "leaderboard_description": { 2024-05-17T22:55:09.282 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "leaderboard_description", 2024-05-17T22:55:09.282 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.282 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.282 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.282 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.282 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.282 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.282 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.282 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.283 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.283 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.283 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.283 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.283 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.283 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.283 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.283 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.283 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.283 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.283 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.283 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.283 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.283 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.284 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.284 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.284 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.284 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.284 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.284 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.284 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.284 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.284 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.284 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.284 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.284 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.284 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.284 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.285 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.285 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.285 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.285 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.285 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.285 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.285 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.285 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.285 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.285 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.285 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.285 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.285 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.286 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.286 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.286 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.286 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.286 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.286 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.286 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.286 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.286 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.286 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.286 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.286 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.286 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.286 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.287 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.287 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.287 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.287 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.287 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.287 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.287 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.287 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.287 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.287 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.287 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.287 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.288 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.288 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.288 INFO:teuthology.orchestra.run.smithi012.stdout: "organization": { 2024-05-17T22:55:09.288 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "organization", 2024-05-17T22:55:09.288 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.288 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.288 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.288 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.288 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.288 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.288 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.288 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.288 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.288 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.289 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.289 INFO:teuthology.orchestra.run.smithi012.stdout: "proxy": { 2024-05-17T22:55:09.289 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "proxy", 2024-05-17T22:55:09.289 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.289 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.289 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.289 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.289 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.289 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.289 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.289 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.289 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.289 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.290 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.290 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.290 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.290 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.290 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.290 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.290 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.290 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.290 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.290 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.290 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.290 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.290 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.291 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.291 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.291 INFO:teuthology.orchestra.run.smithi012.stdout: "url": { 2024-05-17T22:55:09.291 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "url", 2024-05-17T22:55:09.291 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.291 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.291 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.291 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-17T22:55:09.291 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.291 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.291 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.291 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.291 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.292 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.292 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.292 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.292 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.292 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.292 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "test_orchestrator", 2024-05-17T22:55:09.292 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.292 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.292 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.292 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.292 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.292 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.293 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.293 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.293 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.293 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.293 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.293 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.293 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.293 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.293 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.293 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.293 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.293 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.293 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.293 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.294 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.294 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.294 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.294 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.294 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.294 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.294 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.294 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.294 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.294 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.294 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.294 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.294 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.295 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.295 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.295 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.295 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.295 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.295 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.295 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.295 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.295 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.295 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.295 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.295 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.295 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.295 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.295 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.295 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.296 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.296 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.296 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.296 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.296 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.296 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.296 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.296 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.296 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.296 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.296 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.296 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.297 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.297 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.297 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.297 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.297 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.297 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.297 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.297 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.297 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.297 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.297 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.297 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.297 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.298 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.298 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.298 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.298 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.298 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.298 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.298 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.298 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.298 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.298 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.298 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.298 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.298 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.299 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.299 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "volumes", 2024-05-17T22:55:09.299 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.299 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.299 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.299 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.299 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.299 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.299 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.299 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.299 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.299 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.299 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.299 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.299 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.300 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.300 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.300 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.300 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.300 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.300 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.300 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.300 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.300 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.300 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.300 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.300 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.300 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.300 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.301 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.301 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.301 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.301 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.301 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.301 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.301 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.301 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.301 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.301 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.301 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.301 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.301 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.301 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.302 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.302 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.302 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.302 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.302 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.302 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.302 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.302 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.302 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.302 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.302 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.302 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.302 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.302 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.303 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.303 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.303 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.303 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.303 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.303 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.303 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.303 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.303 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.303 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.303 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.303 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.304 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.304 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.304 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.304 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.304 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.304 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.304 INFO:teuthology.orchestra.run.smithi012.stdout: "max_concurrent_clones": { 2024-05-17T22:55:09.304 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_concurrent_clones", 2024-05-17T22:55:09.304 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.304 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.304 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.304 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "4", 2024-05-17T22:55:09.304 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.304 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.305 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.305 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-17T22:55:09.305 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.305 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.305 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.305 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.305 INFO:teuthology.orchestra.run.smithi012.stdout: "periodic_async_work": { 2024-05-17T22:55:09.305 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "periodic_async_work", 2024-05-17T22:55:09.305 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.305 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.305 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.305 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.305 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.305 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.306 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.306 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Periodically check for async work", 2024-05-17T22:55:09.306 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.306 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.306 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.306 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.306 INFO:teuthology.orchestra.run.smithi012.stdout: "snapshot_clone_delay": { 2024-05-17T22:55:09.306 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snapshot_clone_delay", 2024-05-17T22:55:09.306 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.306 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.306 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.306 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.307 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.307 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.307 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.307 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-17T22:55:09.307 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.307 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.307 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.307 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.307 INFO:teuthology.orchestra.run.smithi012.stdout: "snapshot_clone_no_wait": { 2024-05-17T22:55:09.307 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snapshot_clone_no_wait", 2024-05-17T22:55:09.307 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.307 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.307 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.308 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.308 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.308 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.308 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.308 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-17T22:55:09.308 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.308 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.308 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.308 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.308 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.308 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.308 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.308 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.308 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.309 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.309 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.309 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.309 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.309 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.309 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.309 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.309 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.309 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.309 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.309 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.309 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.309 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix", 2024-05-17T22:55:09.309 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.310 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.310 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.310 INFO:teuthology.orchestra.run.smithi012.stdout: "discovery_interval": { 2024-05-17T22:55:09.310 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "discovery_interval", 2024-05-17T22:55:09.310 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-17T22:55:09.310 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.310 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.310 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "100", 2024-05-17T22:55:09.310 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.310 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.310 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.310 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.310 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.311 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.311 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.311 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.311 INFO:teuthology.orchestra.run.smithi012.stdout: "identifier": { 2024-05-17T22:55:09.311 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "identifier", 2024-05-17T22:55:09.311 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.311 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.311 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.311 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.311 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.311 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.311 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.311 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.311 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.312 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.312 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.312 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.312 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-17T22:55:09.312 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-17T22:55:09.312 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:09.312 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.312 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.312 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-17T22:55:09.312 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.312 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.312 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.312 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.312 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.312 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.313 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.313 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.313 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.313 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.313 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.313 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.313 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.313 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.313 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.313 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.313 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.313 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.313 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.313 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.314 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.314 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.314 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.314 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.314 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.314 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.314 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.314 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.314 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.314 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.314 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.314 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.314 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.314 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.315 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.315 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.315 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.315 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.315 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.315 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.315 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.315 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.315 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.315 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.315 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.315 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.315 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.315 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.315 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.316 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.316 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.316 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.316 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.316 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.316 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.316 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.316 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.316 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.316 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.316 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.316 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.316 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.316 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.317 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.317 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.317 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.317 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.317 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.317 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.317 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.317 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.317 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.317 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.317 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.317 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.317 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.317 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.318 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.318 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.318 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.318 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.318 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.318 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.318 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.318 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.318 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.318 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.318 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.318 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.318 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.319 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.319 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.319 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_host": { 2024-05-17T22:55:09.319 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_host", 2024-05-17T22:55:09.319 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.319 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.319 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.319 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.319 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.319 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.319 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.319 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.319 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.320 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.320 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.320 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.320 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_port": { 2024-05-17T22:55:09.320 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_port", 2024-05-17T22:55:09.320 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.320 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.320 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.320 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10051", 2024-05-17T22:55:09.320 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.320 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.320 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.320 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.320 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.321 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.321 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.321 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.321 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_sender": { 2024-05-17T22:55:09.321 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_sender", 2024-05-17T22:55:09.321 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.321 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.321 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.321 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-17T22:55:09.321 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.321 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.321 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.321 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.321 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.322 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.322 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.322 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.322 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.322 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.322 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-05-17T22:55:09.322 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.322 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.322 INFO:teuthology.orchestra.run.smithi012.stdout: "modules": [ 2024-05-17T22:55:09.322 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm", 2024-05-17T22:55:09.322 INFO:teuthology.orchestra.run.smithi012.stdout: "dashboard", 2024-05-17T22:55:09.322 INFO:teuthology.orchestra.run.smithi012.stdout: "iostat", 2024-05-17T22:55:09.322 INFO:teuthology.orchestra.run.smithi012.stdout: "nfs", 2024-05-17T22:55:09.323 INFO:teuthology.orchestra.run.smithi012.stdout: "restful" 2024-05-17T22:55:09.323 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.323 INFO:teuthology.orchestra.run.smithi012.stdout: "available_modules": [ 2024-05-17T22:55:09.323 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.323 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "alerts", 2024-05-17T22:55:09.323 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.323 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.323 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.323 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-17T22:55:09.323 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-17T22:55:09.323 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:09.323 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.323 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.323 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-17T22:55:09.324 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.324 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.324 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.324 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How frequently to reexamine health status", 2024-05-17T22:55:09.324 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.324 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.324 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.324 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.324 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.324 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.324 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.324 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.324 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.324 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.324 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.325 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.325 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.325 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.325 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.325 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.325 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.325 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.325 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.325 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.325 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.325 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.325 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.325 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.325 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.326 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.326 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.326 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.326 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.326 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.326 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.326 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.326 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.326 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.326 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.326 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.326 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.326 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.327 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.327 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.327 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.327 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.327 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.327 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.327 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.327 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.327 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.327 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.327 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.327 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.327 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.328 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.328 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.328 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.328 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.328 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.328 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.328 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.328 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.328 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.328 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.328 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.328 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.328 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.328 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.328 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.329 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.329 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.329 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.329 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.329 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.329 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.329 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.329 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.329 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_destination": { 2024-05-17T22:55:09.329 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_destination", 2024-05-17T22:55:09.329 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.329 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.329 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.329 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.330 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.330 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.330 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.330 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Email address to send alerts to", 2024-05-17T22:55:09.330 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.330 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.330 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.330 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.330 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_from_name": { 2024-05-17T22:55:09.330 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_from_name", 2024-05-17T22:55:09.330 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.330 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.330 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.331 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "Ceph", 2024-05-17T22:55:09.331 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.331 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.331 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.331 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Email From: name", 2024-05-17T22:55:09.331 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.331 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.331 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.331 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.331 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_host": { 2024-05-17T22:55:09.331 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_host", 2024-05-17T22:55:09.331 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.332 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.332 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.332 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.332 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.332 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.332 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.332 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP server", 2024-05-17T22:55:09.332 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.332 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.332 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.332 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.332 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_password": { 2024-05-17T22:55:09.332 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_password", 2024-05-17T22:55:09.332 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.333 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.333 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.333 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.333 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.333 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.333 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.333 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Password to authenticate with", 2024-05-17T22:55:09.333 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.333 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.333 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.333 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.333 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_port": { 2024-05-17T22:55:09.333 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_port", 2024-05-17T22:55:09.333 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.333 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.334 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.334 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "465", 2024-05-17T22:55:09.334 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.334 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.334 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.334 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP port", 2024-05-17T22:55:09.334 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.334 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.334 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.334 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.334 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_sender": { 2024-05-17T22:55:09.334 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_sender", 2024-05-17T22:55:09.334 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.335 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.335 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.335 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.335 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.335 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.335 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.335 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP envelope sender", 2024-05-17T22:55:09.335 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.335 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.335 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.335 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.335 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_ssl": { 2024-05-17T22:55:09.335 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_ssl", 2024-05-17T22:55:09.335 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.336 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.336 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.336 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.336 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.336 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.336 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.336 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-17T22:55:09.336 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.336 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.336 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.336 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.336 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_user": { 2024-05-17T22:55:09.336 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_user", 2024-05-17T22:55:09.337 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.337 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.337 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.337 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.337 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.337 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.337 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.337 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "User to authenticate as", 2024-05-17T22:55:09.337 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.337 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.337 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.337 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.337 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.338 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.338 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.338 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.338 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.338 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.338 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.338 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.338 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.338 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.338 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.338 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.338 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.338 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.338 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.339 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.339 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.339 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "balancer", 2024-05-17T22:55:09.339 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.339 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.339 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.339 INFO:teuthology.orchestra.run.smithi012.stdout: "active": { 2024-05-17T22:55:09.339 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "active", 2024-05-17T22:55:09.339 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.339 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.339 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.339 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.339 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.339 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.340 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.340 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically balance PGs across cluster", 2024-05-17T22:55:09.340 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.340 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.340 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.340 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.340 INFO:teuthology.orchestra.run.smithi012.stdout: "begin_time": { 2024-05-17T22:55:09.340 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "begin_time", 2024-05-17T22:55:09.340 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.340 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.340 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.340 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0000", 2024-05-17T22:55:09.340 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.340 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.341 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.341 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "beginning time of day to automatically balance", 2024-05-17T22:55:09.341 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-17T22:55:09.341 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.341 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.341 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.341 INFO:teuthology.orchestra.run.smithi012.stdout: "begin_weekday": { 2024-05-17T22:55:09.341 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "begin_weekday", 2024-05-17T22:55:09.341 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-17T22:55:09.341 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.341 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.341 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.341 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0", 2024-05-17T22:55:09.341 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "6", 2024-05-17T22:55:09.342 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.342 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-17T22:55:09.342 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-17T22:55:09.342 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.342 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.342 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.342 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_max_iterations": { 2024-05-17T22:55:09.342 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_max_iterations", 2024-05-17T22:55:09.342 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-17T22:55:09.342 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.342 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.342 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "25", 2024-05-17T22:55:09.342 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-05-17T22:55:09.343 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "250", 2024-05-17T22:55:09.343 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.343 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-17T22:55:09.343 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.343 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.343 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.343 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.343 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_metrics": { 2024-05-17T22:55:09.343 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_metrics", 2024-05-17T22:55:09.343 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.343 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.343 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.343 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "pgs,objects,bytes", 2024-05-17T22:55:09.344 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.344 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.344 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.344 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-17T22:55:09.344 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-17T22:55:09.344 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.344 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.344 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.344 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_step": { 2024-05-17T22:55:09.344 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_step", 2024-05-17T22:55:09.344 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-17T22:55:09.344 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.344 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.345 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0.5", 2024-05-17T22:55:09.345 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0.001", 2024-05-17T22:55:09.345 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "0.999", 2024-05-17T22:55:09.345 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.345 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "aggressiveness of optimization", 2024-05-17T22:55:09.345 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-17T22:55:09.345 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.345 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.345 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.345 INFO:teuthology.orchestra.run.smithi012.stdout: "end_time": { 2024-05-17T22:55:09.345 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "end_time", 2024-05-17T22:55:09.345 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.345 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.345 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.346 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "2359", 2024-05-17T22:55:09.346 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.346 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.346 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.346 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "ending time of day to automatically balance", 2024-05-17T22:55:09.346 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-17T22:55:09.346 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.346 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.346 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.346 INFO:teuthology.orchestra.run.smithi012.stdout: "end_weekday": { 2024-05-17T22:55:09.346 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "end_weekday", 2024-05-17T22:55:09.346 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-17T22:55:09.346 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.347 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.347 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.347 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0", 2024-05-17T22:55:09.347 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "6", 2024-05-17T22:55:09.347 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.347 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-17T22:55:09.347 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-17T22:55:09.347 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.347 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.347 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.347 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.347 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.347 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.348 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.348 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.348 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.348 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.348 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.348 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.348 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.348 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.348 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.348 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.348 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.348 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.348 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.348 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.349 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.349 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.349 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.349 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.349 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.349 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.349 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.349 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.349 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.349 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.349 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.349 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.349 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.349 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.350 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.350 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.350 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.350 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.350 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.350 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.350 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.350 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.350 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.350 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.350 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.350 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.350 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.350 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.351 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.351 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.351 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.351 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.351 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.351 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.351 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.351 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.351 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.351 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.351 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.351 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.351 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.351 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.352 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.352 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.352 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.352 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.352 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.352 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.352 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.352 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.352 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.352 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.352 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.352 INFO:teuthology.orchestra.run.smithi012.stdout: "min_score": { 2024-05-17T22:55:09.352 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "min_score", 2024-05-17T22:55:09.352 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-17T22:55:09.353 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.353 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.353 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.353 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.353 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.353 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.353 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-17T22:55:09.353 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.353 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.353 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.353 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.353 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": { 2024-05-17T22:55:09.353 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mode", 2024-05-17T22:55:09.353 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.354 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.354 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.354 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "upmap", 2024-05-17T22:55:09.354 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.354 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.354 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.354 INFO:teuthology.orchestra.run.smithi012.stdout: "crush-compat", 2024-05-17T22:55:09.354 INFO:teuthology.orchestra.run.smithi012.stdout: "none", 2024-05-17T22:55:09.354 INFO:teuthology.orchestra.run.smithi012.stdout: "read", 2024-05-17T22:55:09.354 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap", 2024-05-17T22:55:09.354 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap-read" 2024-05-17T22:55:09.354 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.354 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Balancer mode", 2024-05-17T22:55:09.354 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.354 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.355 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.355 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.355 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_ids": { 2024-05-17T22:55:09.355 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pool_ids", 2024-05-17T22:55:09.355 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.355 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.355 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.355 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.355 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.355 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.355 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.355 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-17T22:55:09.355 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.355 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.356 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.356 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.356 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-17T22:55:09.356 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-17T22:55:09.356 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:09.356 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.356 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.356 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-17T22:55:09.356 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.356 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.356 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.356 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-17T22:55:09.356 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.356 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.356 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.357 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.357 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.357 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.357 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.357 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.357 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.357 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.357 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.357 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.357 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.357 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.357 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.357 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.357 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.357 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.358 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap_max_deviation": { 2024-05-17T22:55:09.358 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "upmap_max_deviation", 2024-05-17T22:55:09.358 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.358 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.358 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.358 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-05-17T22:55:09.358 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-05-17T22:55:09.358 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.358 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.358 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-17T22:55:09.358 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-17T22:55:09.358 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.358 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.359 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.359 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap_max_optimizations": { 2024-05-17T22:55:09.359 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "upmap_max_optimizations", 2024-05-17T22:55:09.359 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-17T22:55:09.359 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.359 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.359 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-17T22:55:09.359 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.359 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.359 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.359 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-17T22:55:09.359 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.359 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.359 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.360 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.360 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.360 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.360 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.360 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cephadm", 2024-05-17T22:55:09.360 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.360 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.360 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.360 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_down_multiplier": { 2024-05-17T22:55:09.360 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_down_multiplier", 2024-05-17T22:55:09.360 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-17T22:55:09.360 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.360 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.360 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3.0", 2024-05-17T22:55:09.361 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.361 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.361 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.361 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-17T22:55:09.361 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.361 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.361 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.361 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.361 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_refresh_rate": { 2024-05-17T22:55:09.361 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_refresh_rate", 2024-05-17T22:55:09.361 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:09.361 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.361 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.361 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "20", 2024-05-17T22:55:09.361 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.362 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.362 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.362 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-17T22:55:09.362 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.362 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.362 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.362 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.362 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_starting_port": { 2024-05-17T22:55:09.362 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_starting_port", 2024-05-17T22:55:09.362 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.362 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.362 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.362 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "4721", 2024-05-17T22:55:09.362 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.363 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.363 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.363 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-17T22:55:09.363 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.363 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.363 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.363 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.363 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_ptrace": { 2024-05-17T22:55:09.363 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_ptrace", 2024-05-17T22:55:09.363 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.363 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.363 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.363 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.364 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.364 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.364 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.364 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-17T22:55:09.364 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-17T22:55:09.364 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.364 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.364 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.364 INFO:teuthology.orchestra.run.smithi012.stdout: "autotune_interval": { 2024-05-17T22:55:09.364 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "autotune_interval", 2024-05-17T22:55:09.364 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:09.364 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.364 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.364 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-17T22:55:09.364 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.364 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.365 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.365 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-17T22:55:09.365 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.365 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.365 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.365 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.365 INFO:teuthology.orchestra.run.smithi012.stdout: "autotune_memory_target_ratio": { 2024-05-17T22:55:09.365 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "autotune_memory_target_ratio", 2024-05-17T22:55:09.365 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-17T22:55:09.365 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.365 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.365 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0.7", 2024-05-17T22:55:09.365 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.365 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.366 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.366 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-17T22:55:09.366 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.366 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.366 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.366 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.366 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm_log_destination": { 2024-05-17T22:55:09.366 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cephadm_log_destination", 2024-05-17T22:55:09.366 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.366 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.366 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.366 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.366 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.367 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.367 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.367 INFO:teuthology.orchestra.run.smithi012.stdout: "file", 2024-05-17T22:55:09.367 INFO:teuthology.orchestra.run.smithi012.stdout: "file,syslog", 2024-05-17T22:55:09.367 INFO:teuthology.orchestra.run.smithi012.stdout: "syslog" 2024-05-17T22:55:09.367 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.367 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-17T22:55:09.367 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.367 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.367 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.367 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.367 INFO:teuthology.orchestra.run.smithi012.stdout: "cgroups_split": { 2024-05-17T22:55:09.367 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cgroups_split", 2024-05-17T22:55:09.367 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.368 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.368 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.368 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.368 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.368 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.368 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.368 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-17T22:55:09.368 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.368 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.368 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.368 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.368 INFO:teuthology.orchestra.run.smithi012.stdout: "config_checks_enabled": { 2024-05-17T22:55:09.368 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_checks_enabled", 2024-05-17T22:55:09.369 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.369 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.369 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.369 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.369 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.369 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.369 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.369 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-17T22:55:09.369 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.369 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.369 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.369 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.369 INFO:teuthology.orchestra.run.smithi012.stdout: "config_dashboard": { 2024-05-17T22:55:09.369 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_dashboard", 2024-05-17T22:55:09.369 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.370 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.370 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.370 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.370 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.370 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.370 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.370 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-17T22:55:09.370 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.370 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.370 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.370 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.370 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_alertmanager": { 2024-05-17T22:55:09.371 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_alertmanager", 2024-05-17T22:55:09.371 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.371 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.371 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.371 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-17T22:55:09.371 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.371 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.371 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.371 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-05-17T22:55:09.371 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.371 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.371 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.371 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.371 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_base": { 2024-05-17T22:55:09.371 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_base", 2024-05-17T22:55:09.372 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.372 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.372 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.372 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-17T22:55:09.372 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.372 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.372 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.372 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Container image name, without the tag", 2024-05-17T22:55:09.372 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.372 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.372 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.372 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.372 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_elasticsearch": { 2024-05-17T22:55:09.372 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_elasticsearch", 2024-05-17T22:55:09.372 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.373 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.373 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.373 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-17T22:55:09.373 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.373 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.373 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.373 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "elasticsearch container image", 2024-05-17T22:55:09.373 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.373 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.373 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.373 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.373 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_grafana": { 2024-05-17T22:55:09.374 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_grafana", 2024-05-17T22:55:09.374 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.374 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.374 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.374 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-17T22:55:09.374 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.374 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.374 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.374 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-05-17T22:55:09.374 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.374 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.374 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.374 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.374 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_haproxy": { 2024-05-17T22:55:09.375 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_haproxy", 2024-05-17T22:55:09.375 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.375 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.375 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.375 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-17T22:55:09.375 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.375 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.375 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.375 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "HAproxy container image", 2024-05-17T22:55:09.375 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.375 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.375 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.375 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.376 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_agent": { 2024-05-17T22:55:09.376 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_agent", 2024-05-17T22:55:09.376 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.376 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.376 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.376 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-17T22:55:09.376 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.376 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.376 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.376 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger agent container image", 2024-05-17T22:55:09.376 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.376 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.376 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.376 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.377 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_collector": { 2024-05-17T22:55:09.377 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_collector", 2024-05-17T22:55:09.377 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.377 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.377 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.377 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-17T22:55:09.377 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.377 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.377 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.377 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger collector container image", 2024-05-17T22:55:09.377 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.377 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.377 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.377 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.378 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_query": { 2024-05-17T22:55:09.378 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_query", 2024-05-17T22:55:09.378 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.378 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.378 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.378 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-17T22:55:09.378 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.378 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.378 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.378 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger query container image", 2024-05-17T22:55:09.378 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.378 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.378 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.378 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.379 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_keepalived": { 2024-05-17T22:55:09.379 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_keepalived", 2024-05-17T22:55:09.379 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.379 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.379 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.379 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-17T22:55:09.379 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.379 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.379 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.379 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Keepalived container image", 2024-05-17T22:55:09.379 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.379 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.379 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.379 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.380 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_loki": { 2024-05-17T22:55:09.380 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_loki", 2024-05-17T22:55:09.380 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.380 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.380 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.380 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-05-17T22:55:09.380 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.380 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.380 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.380 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Loki container image", 2024-05-17T22:55:09.380 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.380 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.380 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.380 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.381 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_node_exporter": { 2024-05-17T22:55:09.381 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_node_exporter", 2024-05-17T22:55:09.381 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.381 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.381 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.381 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-17T22:55:09.381 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.381 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.381 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.381 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-05-17T22:55:09.381 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.381 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.381 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.382 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.382 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_nvmeof": { 2024-05-17T22:55:09.382 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_nvmeof", 2024-05-17T22:55:09.382 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.382 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.382 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.382 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-17T22:55:09.382 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.382 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.382 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.382 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Nvme-of container image", 2024-05-17T22:55:09.382 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.382 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.383 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.383 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.383 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_prometheus": { 2024-05-17T22:55:09.383 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_prometheus", 2024-05-17T22:55:09.383 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.383 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.383 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.383 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-17T22:55:09.383 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.383 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.383 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.383 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-05-17T22:55:09.383 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.383 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.384 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.384 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.384 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_promtail": { 2024-05-17T22:55:09.384 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_promtail", 2024-05-17T22:55:09.384 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.384 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.384 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.384 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-05-17T22:55:09.384 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.384 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.384 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.384 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Promtail container image", 2024-05-17T22:55:09.384 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.384 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.385 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.385 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.385 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_samba": { 2024-05-17T22:55:09.385 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_samba", 2024-05-17T22:55:09.385 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.385 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.385 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.385 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-17T22:55:09.385 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.385 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.385 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.385 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Samba/SMB container image", 2024-05-17T22:55:09.385 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.385 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.385 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.386 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.386 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_snmp_gateway": { 2024-05-17T22:55:09.386 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_snmp_gateway", 2024-05-17T22:55:09.386 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.386 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.386 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.386 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-17T22:55:09.386 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.386 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.386 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.386 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SNMP Gateway container image", 2024-05-17T22:55:09.386 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.386 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.387 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.387 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.387 INFO:teuthology.orchestra.run.smithi012.stdout: "container_init": { 2024-05-17T22:55:09.387 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_init", 2024-05-17T22:55:09.387 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.387 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.387 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.387 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.387 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.387 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.387 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.387 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Run podman/docker with `--init`", 2024-05-17T22:55:09.387 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.387 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.388 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.388 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.388 INFO:teuthology.orchestra.run.smithi012.stdout: "daemon_cache_timeout": { 2024-05-17T22:55:09.388 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "daemon_cache_timeout", 2024-05-17T22:55:09.388 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:09.388 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.388 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.388 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-17T22:55:09.388 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.388 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.388 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.388 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-17T22:55:09.388 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.388 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.388 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.389 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.389 INFO:teuthology.orchestra.run.smithi012.stdout: "default_cephadm_command_timeout": { 2024-05-17T22:55:09.389 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default_cephadm_command_timeout", 2024-05-17T22:55:09.389 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.389 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.389 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.389 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "900", 2024-05-17T22:55:09.389 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.389 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.389 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.389 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-17T22:55:09.389 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.389 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.390 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.390 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.390 INFO:teuthology.orchestra.run.smithi012.stdout: "default_registry": { 2024-05-17T22:55:09.390 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default_registry", 2024-05-17T22:55:09.390 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.390 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.390 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.390 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io", 2024-05-17T22:55:09.390 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.390 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.390 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.390 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-17T22:55:09.390 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.390 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.390 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.391 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.391 INFO:teuthology.orchestra.run.smithi012.stdout: "device_cache_timeout": { 2024-05-17T22:55:09.391 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_cache_timeout", 2024-05-17T22:55:09.391 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:09.391 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.391 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.391 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1800", 2024-05-17T22:55:09.391 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.391 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.391 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.391 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache device inventory", 2024-05-17T22:55:09.391 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.391 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.391 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.391 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.392 INFO:teuthology.orchestra.run.smithi012.stdout: "device_enhanced_scan": { 2024-05-17T22:55:09.392 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_enhanced_scan", 2024-05-17T22:55:09.392 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.392 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.392 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.392 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.392 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.392 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.392 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.392 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-17T22:55:09.392 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.392 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.392 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.392 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.393 INFO:teuthology.orchestra.run.smithi012.stdout: "facts_cache_timeout": { 2024-05-17T22:55:09.393 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "facts_cache_timeout", 2024-05-17T22:55:09.393 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:09.393 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.393 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.393 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-17T22:55:09.393 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.393 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.393 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.393 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache host facts data", 2024-05-17T22:55:09.393 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.393 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.393 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.393 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.393 INFO:teuthology.orchestra.run.smithi012.stdout: "grafana_dashboards_path": { 2024-05-17T22:55:09.394 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "grafana_dashboards_path", 2024-05-17T22:55:09.394 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.394 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.394 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.394 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-17T22:55:09.394 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.394 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.394 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.394 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-17T22:55:09.394 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.394 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.394 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.394 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.395 INFO:teuthology.orchestra.run.smithi012.stdout: "host_check_interval": { 2024-05-17T22:55:09.395 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "host_check_interval", 2024-05-17T22:55:09.395 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:09.395 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.395 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.395 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-17T22:55:09.395 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.395 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.395 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.395 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to perform a host check", 2024-05-17T22:55:09.395 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.395 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.395 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.395 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.396 INFO:teuthology.orchestra.run.smithi012.stdout: "hw_monitoring": { 2024-05-17T22:55:09.396 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "hw_monitoring", 2024-05-17T22:55:09.396 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.396 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.396 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.396 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.396 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.396 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.396 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.396 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-17T22:55:09.396 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.396 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.396 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.396 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.396 INFO:teuthology.orchestra.run.smithi012.stdout: "inventory_list_all": { 2024-05-17T22:55:09.397 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "inventory_list_all", 2024-05-17T22:55:09.397 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.397 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.397 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.397 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.397 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.397 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.397 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.397 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-17T22:55:09.397 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.397 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.397 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.397 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.397 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.397 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.398 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.398 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.398 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.398 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.398 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.398 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.398 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.398 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.398 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.398 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.398 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.398 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.398 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.398 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.399 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.399 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.399 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.399 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.399 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.399 INFO:teuthology.orchestra.run.smithi012.stdout: "log_refresh_metadata": { 2024-05-17T22:55:09.399 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_refresh_metadata", 2024-05-17T22:55:09.399 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.399 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.399 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.399 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.399 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.399 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.400 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.400 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-17T22:55:09.400 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.400 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.400 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.400 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.400 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.400 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.400 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.400 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.400 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.400 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.400 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.400 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.400 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.401 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-17T22:55:09.401 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.401 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.401 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.401 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.401 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.401 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.401 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.401 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.401 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.401 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.401 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.401 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.401 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.401 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.402 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.402 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.402 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.402 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.402 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.402 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.402 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.402 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.402 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.402 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.402 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.402 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.402 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.402 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.403 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.403 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.403 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.403 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.403 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.403 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.403 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.403 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.403 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.403 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.403 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.403 INFO:teuthology.orchestra.run.smithi012.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-17T22:55:09.403 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-17T22:55:09.403 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.403 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.403 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.404 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.404 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.404 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.404 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.404 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-17T22:55:09.404 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.404 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.404 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.404 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.404 INFO:teuthology.orchestra.run.smithi012.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-17T22:55:09.404 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-17T22:55:09.404 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.404 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.405 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.405 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "*", 2024-05-17T22:55:09.405 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.405 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.405 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.405 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-17T22:55:09.405 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.405 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.405 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.405 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.405 INFO:teuthology.orchestra.run.smithi012.stdout: "max_count_per_host": { 2024-05-17T22:55:09.405 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_count_per_host", 2024-05-17T22:55:09.405 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.405 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.405 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.405 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-17T22:55:09.406 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.406 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.406 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.406 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "max number of daemons per service per host", 2024-05-17T22:55:09.406 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.406 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.406 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.406 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.406 INFO:teuthology.orchestra.run.smithi012.stdout: "max_osd_draining_count": { 2024-05-17T22:55:09.406 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_osd_draining_count", 2024-05-17T22:55:09.407 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.407 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.407 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.407 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-17T22:55:09.407 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.407 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.407 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.407 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-17T22:55:09.407 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.407 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.407 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.407 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.407 INFO:teuthology.orchestra.run.smithi012.stdout: "migration_current": { 2024-05-17T22:55:09.408 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "migration_current", 2024-05-17T22:55:09.408 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.408 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.408 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.408 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.408 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.408 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.408 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.408 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "internal - do not modify", 2024-05-17T22:55:09.408 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.408 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.408 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.408 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.409 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": { 2024-05-17T22:55:09.409 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mode", 2024-05-17T22:55:09.409 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.409 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.409 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.409 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "root", 2024-05-17T22:55:09.409 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.409 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.409 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.409 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm-package", 2024-05-17T22:55:09.409 INFO:teuthology.orchestra.run.smithi012.stdout: "root" 2024-05-17T22:55:09.409 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.409 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "mode for remote execution of cephadm", 2024-05-17T22:55:09.410 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.410 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.410 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.410 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.410 INFO:teuthology.orchestra.run.smithi012.stdout: "oob_default_addr": { 2024-05-17T22:55:09.410 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "oob_default_addr", 2024-05-17T22:55:09.410 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.410 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.410 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.410 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "169.254.1.1", 2024-05-17T22:55:09.410 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.410 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.410 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.410 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-17T22:55:09.411 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.411 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.411 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.411 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.411 INFO:teuthology.orchestra.run.smithi012.stdout: "prometheus_alerts_path": { 2024-05-17T22:55:09.411 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prometheus_alerts_path", 2024-05-17T22:55:09.411 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.411 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.411 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.411 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-17T22:55:09.411 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.411 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.411 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.411 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-17T22:55:09.412 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.412 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.412 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.412 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_insecure": { 2024-05-17T22:55:09.412 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_insecure", 2024-05-17T22:55:09.412 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.412 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.412 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.412 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.412 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.412 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.412 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.413 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-17T22:55:09.413 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.413 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.413 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.413 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.413 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_password": { 2024-05-17T22:55:09.413 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_password", 2024-05-17T22:55:09.413 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.413 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.413 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.413 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.413 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.413 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.413 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.413 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-17T22:55:09.414 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.414 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.414 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.414 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.414 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_url": { 2024-05-17T22:55:09.414 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_url", 2024-05-17T22:55:09.414 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.414 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.414 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.414 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.414 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.414 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.414 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.415 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-17T22:55:09.415 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.415 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.415 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.415 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.415 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_username": { 2024-05-17T22:55:09.415 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_username", 2024-05-17T22:55:09.415 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.415 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.415 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.415 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.415 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.415 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.415 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.416 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-17T22:55:09.416 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.416 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.416 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.416 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.416 INFO:teuthology.orchestra.run.smithi012.stdout: "secure_monitoring_stack": { 2024-05-17T22:55:09.416 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "secure_monitoring_stack", 2024-05-17T22:55:09.416 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.416 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.416 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.416 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.416 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.416 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.416 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.417 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-17T22:55:09.417 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.417 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.417 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.417 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.417 INFO:teuthology.orchestra.run.smithi012.stdout: "service_discovery_port": { 2024-05-17T22:55:09.417 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "service_discovery_port", 2024-05-17T22:55:09.417 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.417 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.417 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.417 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8765", 2024-05-17T22:55:09.417 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.417 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.417 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.418 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "cephadm service discovery port", 2024-05-17T22:55:09.418 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.418 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.418 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.418 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.418 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.418 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.418 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.418 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.418 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.418 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.418 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.418 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.418 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.419 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.419 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.419 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.419 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.419 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.419 INFO:teuthology.orchestra.run.smithi012.stdout: "ssh_config_file": { 2024-05-17T22:55:09.419 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssh_config_file", 2024-05-17T22:55:09.419 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.419 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.419 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.419 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.419 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.419 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.419 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.420 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-17T22:55:09.420 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.420 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.420 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.420 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.420 INFO:teuthology.orchestra.run.smithi012.stdout: "use_agent": { 2024-05-17T22:55:09.420 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "use_agent", 2024-05-17T22:55:09.420 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.420 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.420 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.420 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.420 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.420 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.421 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.421 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-17T22:55:09.421 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.421 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.421 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.421 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.421 INFO:teuthology.orchestra.run.smithi012.stdout: "use_repo_digest": { 2024-05-17T22:55:09.421 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "use_repo_digest", 2024-05-17T22:55:09.421 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.421 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.421 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.421 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.421 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.421 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.422 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.422 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-17T22:55:09.422 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.422 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.422 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.422 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.422 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_failed_host_check": { 2024-05-17T22:55:09.422 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_failed_host_check", 2024-05-17T22:55:09.422 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.422 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.422 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.422 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.423 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.423 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.423 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.423 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if the host check fails", 2024-05-17T22:55:09.423 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.423 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.423 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.423 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.423 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_stray_daemons": { 2024-05-17T22:55:09.423 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_stray_daemons", 2024-05-17T22:55:09.423 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.423 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.423 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.423 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.424 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.424 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.424 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.424 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-17T22:55:09.424 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.424 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.424 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.424 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.424 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_stray_hosts": { 2024-05-17T22:55:09.424 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_stray_hosts", 2024-05-17T22:55:09.424 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.424 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.424 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.424 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.424 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.425 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.425 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.425 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-17T22:55:09.425 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.425 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.425 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.425 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.425 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.425 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.425 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.425 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crash", 2024-05-17T22:55:09.425 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.425 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.425 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.426 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.426 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.426 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.426 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.426 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.426 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.426 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.426 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.426 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.426 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.426 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.426 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.426 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.426 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.427 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.427 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.427 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.427 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.427 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.427 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.427 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.427 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.427 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.427 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.427 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.427 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.427 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.427 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.428 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.428 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.428 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.428 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.428 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.428 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.428 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.428 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.428 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.428 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.428 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.428 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.428 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.428 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.428 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.429 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.429 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.429 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.429 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.429 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.429 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.429 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.429 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.429 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.429 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.429 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.429 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.429 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.429 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.430 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.430 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.430 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.430 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.430 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.430 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.430 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.430 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.430 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.430 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.430 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.430 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.430 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.431 INFO:teuthology.orchestra.run.smithi012.stdout: "retain_interval": { 2024-05-17T22:55:09.431 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "retain_interval", 2024-05-17T22:55:09.431 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:09.431 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.431 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.431 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "31536000", 2024-05-17T22:55:09.431 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.431 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.431 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.431 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-17T22:55:09.431 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.431 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.431 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.431 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.431 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.432 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.432 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.432 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.432 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.432 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.432 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.432 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.432 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.432 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.432 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.432 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.432 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.432 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.432 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_recent_interval": { 2024-05-17T22:55:09.433 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_recent_interval", 2024-05-17T22:55:09.433 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:09.433 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.433 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.433 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1209600", 2024-05-17T22:55:09.433 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.433 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.433 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.433 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-17T22:55:09.433 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.433 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.433 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.433 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.433 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.433 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.434 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.434 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "dashboard", 2024-05-17T22:55:09.434 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.434 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.434 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.434 INFO:teuthology.orchestra.run.smithi012.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-17T22:55:09.434 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-17T22:55:09.434 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.434 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.434 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.434 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-17T22:55:09.434 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.434 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.435 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.435 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.435 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.435 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.435 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.435 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.435 INFO:teuthology.orchestra.run.smithi012.stdout: "ALERTMANAGER_API_HOST": { 2024-05-17T22:55:09.435 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-17T22:55:09.435 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.435 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.435 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.435 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.435 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.435 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.435 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.436 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.436 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.436 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.436 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.436 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.436 INFO:teuthology.orchestra.run.smithi012.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-17T22:55:09.436 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-17T22:55:09.436 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.436 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.436 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.436 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.436 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.436 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.436 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.437 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.437 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.437 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.437 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.437 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.437 INFO:teuthology.orchestra.run.smithi012.stdout: "AUDIT_API_ENABLED": { 2024-05-17T22:55:09.437 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "AUDIT_API_ENABLED", 2024-05-17T22:55:09.437 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.437 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.437 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.437 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.437 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.437 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.437 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.437 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.438 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.438 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.438 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.438 INFO:teuthology.orchestra.run.smithi012.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-17T22:55:09.438 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-17T22:55:09.438 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.438 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.438 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.438 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.438 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.438 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.438 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.438 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.439 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.439 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.439 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.439 INFO:teuthology.orchestra.run.smithi012.stdout: "ENABLE_BROWSABLE_API": { 2024-05-17T22:55:09.439 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-17T22:55:09.439 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.439 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.439 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.439 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.439 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.439 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.439 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.439 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.439 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.440 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.440 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.440 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-17T22:55:09.440 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-17T22:55:09.440 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.440 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.440 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.440 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.440 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.440 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.440 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.440 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.440 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.441 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.441 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.441 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.441 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-17T22:55:09.441 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-17T22:55:09.441 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.441 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.441 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.441 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.441 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.441 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.441 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.441 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.442 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.442 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.442 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.442 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.442 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-17T22:55:09.442 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-17T22:55:09.442 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.442 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.442 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.442 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.442 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.442 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.442 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.442 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.442 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.443 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.443 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.443 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.443 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-17T22:55:09.443 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-17T22:55:09.443 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.443 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.443 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.443 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.443 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.443 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.443 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.443 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.444 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.444 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.444 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.444 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.444 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-17T22:55:09.444 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-17T22:55:09.444 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.444 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.444 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.444 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.444 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.444 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.444 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.444 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.445 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.445 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.445 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.445 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.445 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-17T22:55:09.445 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-17T22:55:09.445 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.445 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.445 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.445 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.445 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.445 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.445 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.446 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.446 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.446 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.446 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.446 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.446 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-17T22:55:09.446 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-17T22:55:09.446 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.446 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.446 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.446 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.446 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.446 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.446 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.447 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.447 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.447 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.447 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.447 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.447 INFO:teuthology.orchestra.run.smithi012.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-17T22:55:09.447 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-17T22:55:09.447 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.447 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.447 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.447 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.447 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.447 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.448 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.448 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.448 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.448 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.448 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.448 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.448 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_PASSWORD": { 2024-05-17T22:55:09.448 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-17T22:55:09.448 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.448 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.448 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.448 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-05-17T22:55:09.448 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.448 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.449 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.449 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.449 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.449 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.449 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.449 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.449 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-17T22:55:09.449 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-17T22:55:09.449 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.449 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.449 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.449 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.449 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.449 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.450 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.450 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.450 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.450 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.450 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.450 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.450 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_URL": { 2024-05-17T22:55:09.450 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_URL", 2024-05-17T22:55:09.450 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.450 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.450 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.450 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.450 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.451 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.451 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.451 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.451 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.451 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.451 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.451 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.451 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_USERNAME": { 2024-05-17T22:55:09.451 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-17T22:55:09.451 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.451 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.451 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.451 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-05-17T22:55:09.451 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.452 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.452 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.452 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.452 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.452 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.452 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.452 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.452 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-17T22:55:09.452 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-17T22:55:09.452 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.452 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.452 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.452 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.452 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.452 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.453 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.453 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.453 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.453 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.453 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.453 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.453 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-17T22:55:09.453 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-17T22:55:09.453 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.453 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.453 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.453 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.453 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.453 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.454 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.454 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.454 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.454 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.454 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.454 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.454 INFO:teuthology.orchestra.run.smithi012.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-17T22:55:09.454 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-17T22:55:09.454 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.454 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.454 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.454 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.455 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.455 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.455 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.455 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.455 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.455 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.455 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.455 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.455 INFO:teuthology.orchestra.run.smithi012.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-17T22:55:09.455 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-17T22:55:09.455 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.455 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.455 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.455 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.455 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.456 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.456 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.456 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.456 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.456 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.456 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.456 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.456 INFO:teuthology.orchestra.run.smithi012.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-17T22:55:09.456 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-17T22:55:09.456 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.456 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.456 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.456 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "[]", 2024-05-17T22:55:09.456 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.456 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.457 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.457 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.457 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.457 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.457 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.457 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.457 INFO:teuthology.orchestra.run.smithi012.stdout: "MULTICLUSTER_CONFIG": { 2024-05-17T22:55:09.457 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-17T22:55:09.457 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.457 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.457 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.457 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "{}", 2024-05-17T22:55:09.457 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.457 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.457 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.458 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.458 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.458 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.458 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.458 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.458 INFO:teuthology.orchestra.run.smithi012.stdout: "PROMETHEUS_API_HOST": { 2024-05-17T22:55:09.458 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-17T22:55:09.458 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.458 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.458 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.458 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.458 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.458 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.458 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.459 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.459 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.459 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.459 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.459 INFO:teuthology.orchestra.run.smithi012.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-17T22:55:09.459 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-17T22:55:09.459 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.459 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.459 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.459 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.459 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.459 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.460 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.460 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.460 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.460 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.460 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.460 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-17T22:55:09.460 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-17T22:55:09.460 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.460 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.460 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.460 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.460 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.460 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.461 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.461 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.461 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.461 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.461 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.461 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.461 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-17T22:55:09.461 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-17T22:55:09.461 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.461 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.461 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.461 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.461 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.461 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.462 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.462 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.462 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.462 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.462 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.462 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-17T22:55:09.462 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-17T22:55:09.462 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.462 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.462 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.462 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.462 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.462 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.463 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.463 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.463 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.463 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.463 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.463 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.463 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-17T22:55:09.463 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-17T22:55:09.463 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.463 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.463 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.463 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.463 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.463 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.463 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.463 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.464 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.464 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.464 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.464 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.464 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-17T22:55:09.464 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-17T22:55:09.464 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.464 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.464 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.464 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.464 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.464 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.464 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.465 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.465 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.465 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.465 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.465 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.465 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-17T22:55:09.465 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-17T22:55:09.465 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.465 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.465 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.465 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.465 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.465 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.465 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.465 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.466 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.466 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.466 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.466 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.466 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-17T22:55:09.466 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-17T22:55:09.466 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.466 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.466 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.466 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.466 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.466 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.466 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.466 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.467 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.467 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.467 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.467 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.467 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_ENABLED": { 2024-05-17T22:55:09.467 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-17T22:55:09.467 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.467 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.467 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.467 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.467 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.467 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.467 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.467 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.468 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.468 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.468 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.468 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-17T22:55:09.468 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-17T22:55:09.468 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.468 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.468 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.468 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-17T22:55:09.468 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.468 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.468 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.468 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.469 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.469 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.469 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.469 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.469 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-17T22:55:09.469 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-17T22:55:09.469 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.469 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.469 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.469 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-17T22:55:09.469 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.469 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.469 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.469 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.470 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.470 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.470 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.470 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.470 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-17T22:55:09.470 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-17T22:55:09.470 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.470 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.470 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.470 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8", 2024-05-17T22:55:09.470 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.470 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.470 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.470 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.471 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.471 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.471 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.471 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.471 INFO:teuthology.orchestra.run.smithi012.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-17T22:55:09.471 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-17T22:55:09.471 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.471 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.471 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.471 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "45", 2024-05-17T22:55:09.471 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.471 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.471 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.472 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.472 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.472 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.472 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.472 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_ACCESS_KEY": { 2024-05-17T22:55:09.472 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-17T22:55:09.472 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.472 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.472 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.472 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.472 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.472 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.472 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.472 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.473 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.473 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.473 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.473 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-17T22:55:09.473 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-17T22:55:09.473 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.473 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.473 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.473 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-05-17T22:55:09.473 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.473 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.473 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.473 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.474 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.474 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.474 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.474 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.474 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_SECRET_KEY": { 2024-05-17T22:55:09.474 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-17T22:55:09.474 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.474 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.474 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.474 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.474 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.474 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.474 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.475 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.475 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.475 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.475 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.475 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_SSL_VERIFY": { 2024-05-17T22:55:09.475 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-17T22:55:09.475 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.475 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.475 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.475 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.475 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.475 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.475 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.476 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.476 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.476 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.476 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.476 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.476 INFO:teuthology.orchestra.run.smithi012.stdout: "UNSAFE_TLS_v1_2": { 2024-05-17T22:55:09.476 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-17T22:55:09.476 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.476 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.476 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.476 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.476 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.476 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.476 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.477 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.477 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.477 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.477 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.477 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-17T22:55:09.477 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-17T22:55:09.477 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.477 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.477 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.477 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.477 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.477 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.477 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.477 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.478 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.478 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.478 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.478 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.478 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-17T22:55:09.478 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-17T22:55:09.478 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.478 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.478 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.478 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-17T22:55:09.478 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.478 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.478 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.479 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.479 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.479 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.479 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.479 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.479 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-17T22:55:09.479 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-17T22:55:09.479 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.479 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.479 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.479 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-05-17T22:55:09.479 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.479 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.479 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.480 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.480 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.480 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.480 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.480 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.480 INFO:teuthology.orchestra.run.smithi012.stdout: "cross_origin_url": { 2024-05-17T22:55:09.480 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cross_origin_url", 2024-05-17T22:55:09.480 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.480 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.480 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.480 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.480 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.480 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.480 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.481 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.481 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.481 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.481 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.481 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.481 INFO:teuthology.orchestra.run.smithi012.stdout: "crt_file": { 2024-05-17T22:55:09.481 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crt_file", 2024-05-17T22:55:09.481 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.481 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.481 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.481 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.481 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.481 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.482 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.482 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.482 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.482 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.482 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.482 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.482 INFO:teuthology.orchestra.run.smithi012.stdout: "debug": { 2024-05-17T22:55:09.482 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "debug", 2024-05-17T22:55:09.482 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.482 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.482 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.482 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.482 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.483 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.483 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.483 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable/disable debug options", 2024-05-17T22:55:09.483 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.483 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.483 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.483 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.483 INFO:teuthology.orchestra.run.smithi012.stdout: "jwt_token_ttl": { 2024-05-17T22:55:09.483 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "jwt_token_ttl", 2024-05-17T22:55:09.483 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.483 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.483 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.483 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "28800", 2024-05-17T22:55:09.483 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.483 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.484 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.484 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.484 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.484 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.484 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.484 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.484 INFO:teuthology.orchestra.run.smithi012.stdout: "key_file": { 2024-05-17T22:55:09.484 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "key_file", 2024-05-17T22:55:09.484 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.484 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.484 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.484 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.484 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.484 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.485 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.485 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.485 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.485 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.485 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.485 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.485 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.485 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.485 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.485 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.485 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.485 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.485 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.485 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.486 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.486 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.486 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.486 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.486 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.486 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.486 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.486 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.486 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.486 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.486 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.486 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.486 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.487 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.487 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.487 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.487 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.487 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.487 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.487 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.487 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.487 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.487 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.487 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.487 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.487 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.487 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.488 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.488 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.488 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.488 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.488 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.488 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.488 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.488 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.488 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.488 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.488 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.488 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.488 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.488 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.489 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.489 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.489 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.489 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.489 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.489 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.489 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.489 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.489 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.489 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.489 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.489 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.489 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.489 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.490 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.490 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.490 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.490 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.490 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.490 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.490 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.490 INFO:teuthology.orchestra.run.smithi012.stdout: "motd": { 2024-05-17T22:55:09.490 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "motd", 2024-05-17T22:55:09.490 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.490 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.490 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.490 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.491 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.491 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.491 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.491 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "The message of the day", 2024-05-17T22:55:09.491 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.491 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.491 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.491 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.491 INFO:teuthology.orchestra.run.smithi012.stdout: "redirect_resolve_ip_addr": { 2024-05-17T22:55:09.491 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "redirect_resolve_ip_addr", 2024-05-17T22:55:09.491 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.491 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.491 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.491 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.492 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.492 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.492 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.492 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.492 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.492 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.492 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.492 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.492 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-05-17T22:55:09.492 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-05-17T22:55:09.492 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.492 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.492 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.492 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "::", 2024-05-17T22:55:09.492 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.493 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.493 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.493 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.493 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.493 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.493 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.493 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.493 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-05-17T22:55:09.493 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-05-17T22:55:09.493 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.493 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.493 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.493 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8080", 2024-05-17T22:55:09.493 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.494 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.494 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.494 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.494 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.494 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.494 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.494 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.494 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.494 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.494 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.494 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.494 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.494 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.494 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.495 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.495 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.495 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.495 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.495 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.495 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.495 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.495 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl": { 2024-05-17T22:55:09.495 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl", 2024-05-17T22:55:09.495 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.495 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.495 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.495 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.495 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.496 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.496 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.496 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.496 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.496 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.496 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.496 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.496 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl_server_port": { 2024-05-17T22:55:09.496 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl_server_port", 2024-05-17T22:55:09.496 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.496 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.496 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.496 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8443", 2024-05-17T22:55:09.496 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.497 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.497 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.497 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.497 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.497 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.497 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.497 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.497 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_behaviour": { 2024-05-17T22:55:09.497 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_behaviour", 2024-05-17T22:55:09.497 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.497 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.497 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.497 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "redirect", 2024-05-17T22:55:09.497 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.498 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.498 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.498 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.498 INFO:teuthology.orchestra.run.smithi012.stdout: "redirect" 2024-05-17T22:55:09.498 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.498 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.498 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.498 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.498 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.498 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.498 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_error_status_code": { 2024-05-17T22:55:09.498 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_error_status_code", 2024-05-17T22:55:09.498 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.498 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.499 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.499 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-05-17T22:55:09.499 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "400", 2024-05-17T22:55:09.499 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "599", 2024-05-17T22:55:09.499 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.499 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.499 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.499 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.499 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.499 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.499 INFO:teuthology.orchestra.run.smithi012.stdout: "url_prefix": { 2024-05-17T22:55:09.499 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "url_prefix", 2024-05-17T22:55:09.499 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.499 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.500 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.500 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.500 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.500 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.500 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.500 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.500 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.500 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.500 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.500 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.500 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.500 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.500 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.500 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "devicehealth", 2024-05-17T22:55:09.501 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.501 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.501 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.501 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_monitoring": { 2024-05-17T22:55:09.501 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enable_monitoring", 2024-05-17T22:55:09.501 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.501 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.501 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.501 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.501 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.501 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.501 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.501 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "monitor device health metrics", 2024-05-17T22:55:09.501 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.502 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.502 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.502 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.502 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.502 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.502 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.502 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.502 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.502 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.502 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.502 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.502 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.502 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.502 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.503 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.503 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.503 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.503 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.503 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.503 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.503 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.503 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.503 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.503 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.503 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.503 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.503 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.503 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.504 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.504 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.504 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.504 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.504 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.504 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.504 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.504 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.504 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.504 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.504 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.504 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.504 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.504 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.505 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.505 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.505 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.505 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.505 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.505 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.505 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.505 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.505 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.505 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.505 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.505 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.505 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.505 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.506 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.506 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.506 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.506 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.506 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.506 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.506 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.506 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.506 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.506 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.506 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.506 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.506 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.507 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.507 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.507 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.507 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.507 INFO:teuthology.orchestra.run.smithi012.stdout: "mark_out_threshold": { 2024-05-17T22:55:09.507 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mark_out_threshold", 2024-05-17T22:55:09.507 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:09.507 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.507 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.507 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "2419200", 2024-05-17T22:55:09.507 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.507 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.507 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.507 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-17T22:55:09.507 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.508 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.508 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.508 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.508 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": { 2024-05-17T22:55:09.508 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pool_name", 2024-05-17T22:55:09.508 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.508 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.508 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.508 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "device_health_metrics", 2024-05-17T22:55:09.508 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.508 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.508 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.508 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-17T22:55:09.509 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.509 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.509 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.509 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.509 INFO:teuthology.orchestra.run.smithi012.stdout: "retention_period": { 2024-05-17T22:55:09.509 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "retention_period", 2024-05-17T22:55:09.509 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:09.509 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.509 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.509 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15552000", 2024-05-17T22:55:09.509 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.509 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.509 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.510 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long to retain device health metrics", 2024-05-17T22:55:09.510 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.510 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.510 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.510 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.510 INFO:teuthology.orchestra.run.smithi012.stdout: "scrape_frequency": { 2024-05-17T22:55:09.510 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "scrape_frequency", 2024-05-17T22:55:09.510 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:09.510 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.510 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.510 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "86400", 2024-05-17T22:55:09.510 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.510 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.510 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.511 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-17T22:55:09.511 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.511 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.511 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.511 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.511 INFO:teuthology.orchestra.run.smithi012.stdout: "self_heal": { 2024-05-17T22:55:09.511 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "self_heal", 2024-05-17T22:55:09.511 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.511 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.511 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.511 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.511 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.511 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.511 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.511 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-17T22:55:09.512 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.512 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.512 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.512 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.512 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-17T22:55:09.512 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-17T22:55:09.512 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:09.512 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.512 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.512 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-17T22:55:09.512 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.512 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.512 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.512 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to wake up and check device health", 2024-05-17T22:55:09.513 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.513 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.513 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.513 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.513 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.513 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.513 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.513 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.513 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.513 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.513 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.513 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.513 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.513 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.514 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.514 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.514 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.514 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.514 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_threshold": { 2024-05-17T22:55:09.514 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_threshold", 2024-05-17T22:55:09.514 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:09.514 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.514 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.514 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "7257600", 2024-05-17T22:55:09.514 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.514 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.514 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.515 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-17T22:55:09.515 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.515 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.515 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.515 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.515 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.515 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.515 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.515 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "diskprediction_local", 2024-05-17T22:55:09.515 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.515 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.515 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.515 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.515 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.516 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.516 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.516 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.516 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.516 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.516 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.516 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.516 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.516 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.516 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.516 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.516 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.516 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.516 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.517 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.517 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.517 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.517 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.517 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.517 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.517 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.517 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.517 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.517 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.517 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.517 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.517 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.517 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.518 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.518 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.518 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.518 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.518 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.518 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.518 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.518 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.518 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.518 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.518 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.518 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.518 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.518 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.519 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.519 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.519 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.519 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.519 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.519 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.519 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.519 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.519 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.519 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.519 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.519 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.519 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.519 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.520 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.520 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.520 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.520 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.520 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.520 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.520 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.520 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.520 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.520 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.520 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.520 INFO:teuthology.orchestra.run.smithi012.stdout: "predict_interval": { 2024-05-17T22:55:09.520 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "predict_interval", 2024-05-17T22:55:09.521 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.521 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.521 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.521 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "86400", 2024-05-17T22:55:09.521 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.521 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.521 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.521 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.521 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.521 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.521 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.521 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.521 INFO:teuthology.orchestra.run.smithi012.stdout: "predictor_model": { 2024-05-17T22:55:09.521 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "predictor_model", 2024-05-17T22:55:09.522 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.522 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.522 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.522 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "prophetstor", 2024-05-17T22:55:09.522 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.522 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.522 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.522 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.522 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.522 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.522 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.522 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.522 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-17T22:55:09.523 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-17T22:55:09.523 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.523 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.523 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.523 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-05-17T22:55:09.523 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.523 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.523 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.523 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.523 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.523 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.523 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.523 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.524 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.524 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.524 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.524 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.524 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.524 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.524 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.524 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.524 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.524 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.524 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.524 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.524 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.524 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.524 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.525 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.525 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.525 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "influx", 2024-05-17T22:55:09.525 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": false, 2024-05-17T22:55:09.525 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "influxdb python module not found", 2024-05-17T22:55:09.525 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.525 INFO:teuthology.orchestra.run.smithi012.stdout: "batch_size": { 2024-05-17T22:55:09.525 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "batch_size", 2024-05-17T22:55:09.525 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.525 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.525 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.525 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5000", 2024-05-17T22:55:09.525 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.525 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.525 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.526 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-17T22:55:09.526 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.526 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.526 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.526 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.526 INFO:teuthology.orchestra.run.smithi012.stdout: "database": { 2024-05-17T22:55:09.526 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "database", 2024-05-17T22:55:09.526 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.526 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.526 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.526 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "ceph", 2024-05-17T22:55:09.526 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.526 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.527 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.527 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-17T22:55:09.527 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.527 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.527 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.527 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.527 INFO:teuthology.orchestra.run.smithi012.stdout: "hostname": { 2024-05-17T22:55:09.527 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "hostname", 2024-05-17T22:55:09.527 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.527 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.527 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.527 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.527 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.527 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.528 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.528 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB server hostname", 2024-05-17T22:55:09.528 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.528 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.528 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.528 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.528 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-17T22:55:09.528 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-17T22:55:09.528 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:09.528 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.528 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.528 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "30", 2024-05-17T22:55:09.528 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "5", 2024-05-17T22:55:09.528 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.528 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.529 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-17T22:55:09.529 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.529 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.529 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.529 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.529 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.529 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.529 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.529 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.529 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.529 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.529 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.529 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.529 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.529 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.530 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.530 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.530 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.530 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.530 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.530 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.530 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.530 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.530 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.530 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.530 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.530 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.530 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.530 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.531 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.531 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.531 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.531 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.531 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.531 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.531 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.531 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.531 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.531 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.531 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.531 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.531 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.531 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.532 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.532 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.532 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.532 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.532 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.532 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.532 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.532 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.532 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.532 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.532 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.532 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.532 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.532 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.532 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.533 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.533 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.533 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.533 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.533 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.533 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.533 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.533 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.533 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.533 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.533 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.533 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.533 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.533 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.534 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.534 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.534 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.534 INFO:teuthology.orchestra.run.smithi012.stdout: "password": { 2024-05-17T22:55:09.534 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "password", 2024-05-17T22:55:09.534 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.534 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.534 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.534 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.534 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.534 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.534 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.534 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "password of InfluxDB server user", 2024-05-17T22:55:09.534 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.535 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.535 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.535 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.535 INFO:teuthology.orchestra.run.smithi012.stdout: "port": { 2024-05-17T22:55:09.535 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "port", 2024-05-17T22:55:09.535 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.535 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.535 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.535 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8086", 2024-05-17T22:55:09.535 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.535 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.535 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.535 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB server port", 2024-05-17T22:55:09.535 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.535 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.536 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.536 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.536 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.536 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.536 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.536 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.536 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.536 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.536 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.536 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.536 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.536 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.536 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.537 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.537 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.537 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl": { 2024-05-17T22:55:09.537 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl", 2024-05-17T22:55:09.537 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.537 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.537 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.537 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "false", 2024-05-17T22:55:09.537 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.537 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.537 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.537 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-17T22:55:09.537 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.537 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.538 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.538 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.538 INFO:teuthology.orchestra.run.smithi012.stdout: "threads": { 2024-05-17T22:55:09.538 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "threads", 2024-05-17T22:55:09.538 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.538 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.538 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.538 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-05-17T22:55:09.538 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-05-17T22:55:09.538 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "32", 2024-05-17T22:55:09.538 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.538 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-17T22:55:09.538 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.538 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.539 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.539 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.539 INFO:teuthology.orchestra.run.smithi012.stdout: "username": { 2024-05-17T22:55:09.539 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "username", 2024-05-17T22:55:09.539 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.539 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.539 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.539 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.539 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.539 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.539 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.539 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "username of InfluxDB server user", 2024-05-17T22:55:09.539 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.539 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.540 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.540 INFO:teuthology.orchestra.run.smithi012.stdout: "verify_ssl": { 2024-05-17T22:55:09.540 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "verify_ssl", 2024-05-17T22:55:09.540 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.540 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.540 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.540 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "true", 2024-05-17T22:55:09.540 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.540 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.540 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.540 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-17T22:55:09.540 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.541 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.541 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.541 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.541 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.541 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.541 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "insights", 2024-05-17T22:55:09.541 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.541 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.541 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.541 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.541 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.541 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.541 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.541 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.542 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.542 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.542 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.542 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.542 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.542 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.542 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.542 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.542 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.542 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.542 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.542 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.542 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.543 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.543 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.543 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.543 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.543 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.543 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.543 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.543 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.543 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.543 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.543 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.543 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.543 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.543 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.543 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.544 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.544 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.544 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.544 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.544 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.544 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.544 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.544 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.544 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.544 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.544 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.544 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.544 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.544 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.545 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.545 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.545 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.545 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.545 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.545 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.545 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.545 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.545 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.545 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.545 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.545 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.546 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.546 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.546 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.546 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.546 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.546 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.546 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.546 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.546 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.546 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.546 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.546 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.546 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.547 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.547 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.547 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.547 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.547 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.547 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.547 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.547 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.547 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.547 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.547 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.547 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.547 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.547 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "iostat", 2024-05-17T22:55:09.548 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.548 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.548 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.548 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.548 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.548 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.548 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.548 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.548 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.548 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.548 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.548 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.548 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.548 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.548 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.549 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.549 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.549 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.549 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.549 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.549 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.549 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.549 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.549 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.549 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.549 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.549 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.550 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.550 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.550 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.550 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.550 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.550 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.550 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.550 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.550 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.550 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.550 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.550 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.550 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.550 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.551 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.551 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.551 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.551 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.551 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.551 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.551 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.551 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.551 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.551 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.551 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.551 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.551 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.551 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.552 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.552 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.552 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.552 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.552 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.552 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.552 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.552 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.552 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.552 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.552 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.552 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.552 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.552 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.553 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.553 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.553 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.553 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.553 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.553 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.553 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.553 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.553 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.553 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.553 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.553 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.554 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.554 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.554 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.554 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.554 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.554 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "k8sevents", 2024-05-17T22:55:09.554 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.554 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.554 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.554 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_event_retention_days": { 2024-05-17T22:55:09.554 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ceph_event_retention_days", 2024-05-17T22:55:09.554 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.554 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.555 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.555 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "7", 2024-05-17T22:55:09.555 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.555 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.555 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.555 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-17T22:55:09.555 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.555 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.555 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.555 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.555 INFO:teuthology.orchestra.run.smithi012.stdout: "config_check_secs": { 2024-05-17T22:55:09.555 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_check_secs", 2024-05-17T22:55:09.555 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.555 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.556 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.556 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-17T22:55:09.556 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "10", 2024-05-17T22:55:09.556 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.556 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.556 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-17T22:55:09.556 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.556 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.556 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.556 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.556 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.556 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.556 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.557 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.557 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.557 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.557 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.557 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.557 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.557 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.557 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.557 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.557 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.557 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.557 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.557 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.557 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.558 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.558 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.558 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.558 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.558 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.558 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.558 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.558 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.558 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.558 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.558 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.558 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.558 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.559 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.559 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.559 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.559 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.559 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.559 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.559 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.559 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.559 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.559 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.559 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.559 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.559 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.559 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.560 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.560 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.560 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.560 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.560 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.560 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.560 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.560 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.560 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.560 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.560 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.560 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.560 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.561 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.561 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.561 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.561 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.561 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.561 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.561 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.561 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.561 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.561 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.561 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.561 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.561 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.561 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.562 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.562 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.562 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.562 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.562 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.562 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.562 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.562 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.562 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.562 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.562 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.562 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.562 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.562 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.563 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.563 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.563 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "localpool", 2024-05-17T22:55:09.563 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.563 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.563 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.563 INFO:teuthology.orchestra.run.smithi012.stdout: "failure_domain": { 2024-05-17T22:55:09.563 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "failure_domain", 2024-05-17T22:55:09.563 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.563 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.563 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.563 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "host", 2024-05-17T22:55:09.563 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.563 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.564 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.564 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "failure domain for any created local pool", 2024-05-17T22:55:09.564 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-17T22:55:09.564 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.564 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.564 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.564 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.564 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.564 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.564 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.564 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.564 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.564 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.564 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.565 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.565 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.565 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.565 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.565 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.565 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.565 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.565 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.565 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.565 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.565 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.565 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.565 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.565 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.565 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.566 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.566 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.566 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.566 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.566 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.566 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.566 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.566 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.566 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.566 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.566 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.566 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.566 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.567 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.567 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.567 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.567 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.567 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.567 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.567 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.567 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.567 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.567 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.567 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.567 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.567 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.567 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.567 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.568 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.568 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.568 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.568 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.568 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.568 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.568 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.568 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.568 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.568 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.568 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.568 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.568 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.568 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.569 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.569 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.569 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.569 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.569 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.569 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": { 2024-05-17T22:55:09.569 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "min_size", 2024-05-17T22:55:09.569 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.569 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.569 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.569 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.569 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.569 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.569 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.570 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default min_size for any created local pool", 2024-05-17T22:55:09.570 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-17T22:55:09.570 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.570 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.570 INFO:teuthology.orchestra.run.smithi012.stdout: "num_rep": { 2024-05-17T22:55:09.570 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "num_rep", 2024-05-17T22:55:09.570 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.570 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.570 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.570 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3", 2024-05-17T22:55:09.570 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.571 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.571 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.571 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default replica count for any created local pool", 2024-05-17T22:55:09.571 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.571 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.571 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.571 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.571 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": { 2024-05-17T22:55:09.571 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pg_num", 2024-05-17T22:55:09.571 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.571 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.571 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.571 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "128", 2024-05-17T22:55:09.571 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.572 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.572 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.572 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default pg_num for any created local pool", 2024-05-17T22:55:09.572 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.572 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.572 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.572 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.572 INFO:teuthology.orchestra.run.smithi012.stdout: "prefix": { 2024-05-17T22:55:09.572 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prefix", 2024-05-17T22:55:09.572 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.572 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.572 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.572 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.573 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.573 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.573 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.573 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "name prefix for any created local pool", 2024-05-17T22:55:09.573 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.573 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.573 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.573 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.573 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.573 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.573 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.573 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.573 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.573 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.574 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.574 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.574 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.574 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.574 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.574 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.574 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.574 INFO:teuthology.orchestra.run.smithi012.stdout: "subtree": { 2024-05-17T22:55:09.574 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "subtree", 2024-05-17T22:55:09.574 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.574 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.574 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.575 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "rack", 2024-05-17T22:55:09.575 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.575 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.575 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.575 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-17T22:55:09.575 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-17T22:55:09.575 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.575 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.575 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.575 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.575 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.575 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.575 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mds_autoscaler", 2024-05-17T22:55:09.575 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.576 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.576 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.576 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.576 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.576 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.576 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.576 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.576 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.576 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.576 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.576 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.576 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.576 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.576 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.576 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.577 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.577 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.577 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.577 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.577 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.577 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.577 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.577 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.577 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.577 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.577 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.577 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.577 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.577 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.578 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.578 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.578 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.578 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.578 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.578 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.578 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.578 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.578 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.578 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.578 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.578 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.579 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.579 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.579 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.579 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.579 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.579 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.579 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.579 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.579 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.579 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.579 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.579 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.579 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.579 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.580 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.580 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.580 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.580 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.580 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.580 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.580 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.580 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.580 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.580 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.580 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.580 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.580 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.580 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.581 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.581 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.581 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.581 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.581 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.581 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.581 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.581 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.581 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.581 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.581 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.581 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.581 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.582 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.582 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.582 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.582 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.582 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.582 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mirroring", 2024-05-17T22:55:09.582 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.582 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.582 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.582 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.582 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.582 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.582 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.583 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.583 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.583 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.583 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.583 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.583 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.583 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.583 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.583 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.583 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.583 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.583 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.583 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.583 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.584 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.584 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.584 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.584 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.584 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.584 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.584 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.584 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.584 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.584 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.584 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.584 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.584 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.584 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.584 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.585 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.585 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.585 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.585 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.585 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.585 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.585 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.585 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.585 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.585 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.585 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.585 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.585 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.585 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.586 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.586 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.586 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.586 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.586 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.586 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.586 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.586 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.586 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.586 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.586 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.586 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.586 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.587 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.587 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.587 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.587 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.587 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.587 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.587 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.587 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.587 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.587 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.587 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.587 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.587 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.587 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.588 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.588 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.588 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.588 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.588 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.588 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.588 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.588 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.588 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.588 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.588 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.588 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "nfs", 2024-05-17T22:55:09.588 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.589 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.589 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.589 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.589 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.589 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.589 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.589 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.589 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.589 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.589 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.589 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.589 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.589 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.590 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.590 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.590 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.590 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.590 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.590 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.590 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.590 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.590 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.590 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.590 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.590 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.590 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.590 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.591 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.591 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.591 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.591 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.591 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.591 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.591 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.591 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.591 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.591 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.591 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.591 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.591 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.591 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.592 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.592 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.592 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.592 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.592 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.592 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.592 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.592 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.592 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.592 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.592 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.592 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.592 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.592 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.593 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.593 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.593 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.593 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.593 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.593 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.593 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.593 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.593 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.593 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.593 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.593 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.593 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.593 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.594 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.594 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.594 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.594 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.594 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.594 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.594 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.594 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.594 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.594 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.594 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.594 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.594 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.594 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.595 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.595 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.595 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.595 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.595 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.595 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "orchestrator", 2024-05-17T22:55:09.595 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.595 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.595 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.595 INFO:teuthology.orchestra.run.smithi012.stdout: "fail_fs": { 2024-05-17T22:55:09.595 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "fail_fs", 2024-05-17T22:55:09.595 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.595 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.595 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.595 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.596 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.596 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.596 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.596 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-17T22:55:09.596 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.596 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.596 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.596 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.596 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.596 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.596 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.596 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.596 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.596 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.596 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.597 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.597 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.597 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.597 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.597 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.597 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.597 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.597 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.597 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.597 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.597 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.597 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.597 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.598 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.598 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.598 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.598 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.598 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.598 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.598 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.598 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.598 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.598 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.598 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.598 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.598 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.598 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.599 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.599 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.599 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.599 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.599 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.599 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.599 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.599 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.599 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.599 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.599 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.599 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.599 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.599 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.600 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.600 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.600 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.600 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.600 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.600 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.600 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.600 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.600 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.600 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.600 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.600 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.600 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.600 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.601 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.601 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.601 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.601 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.601 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.601 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.601 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.601 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.601 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator": { 2024-05-17T22:55:09.601 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "orchestrator", 2024-05-17T22:55:09.601 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.601 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.601 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.601 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.602 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.602 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.602 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.602 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm", 2024-05-17T22:55:09.602 INFO:teuthology.orchestra.run.smithi012.stdout: "rook", 2024-05-17T22:55:09.602 INFO:teuthology.orchestra.run.smithi012.stdout: "test_orchestrator" 2024-05-17T22:55:09.602 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.602 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Orchestrator backend", 2024-05-17T22:55:09.602 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.602 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.602 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.602 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.602 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.602 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.603 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.603 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.603 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.603 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.603 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.603 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.603 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.603 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.603 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.603 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.603 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.603 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.603 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.603 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.604 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.604 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "osd_perf_query", 2024-05-17T22:55:09.604 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.604 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.604 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.604 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.604 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.604 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.604 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.604 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.604 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.604 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.604 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.605 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.605 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.605 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.605 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.605 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.605 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.605 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.605 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.605 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.605 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.605 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.605 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.605 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.605 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.606 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.606 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.606 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.606 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.606 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.606 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.606 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.606 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.606 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.606 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.606 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.606 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.606 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.607 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.607 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.607 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.607 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.607 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.607 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.607 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.607 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.607 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.607 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.607 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.607 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.607 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.607 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.607 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.608 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.608 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.608 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.608 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.608 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.608 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.608 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.608 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.608 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.608 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.608 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.608 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.608 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.609 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.609 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.609 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.609 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.609 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.609 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.609 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.609 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.609 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.609 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.609 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.609 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.609 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.609 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.610 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.610 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.610 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.610 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.610 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.610 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.610 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.610 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.610 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.610 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.610 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "osd_support", 2024-05-17T22:55:09.610 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.610 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.610 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.611 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.611 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.611 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.611 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.611 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.611 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.611 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.611 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.611 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.611 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.611 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.611 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.611 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.611 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.612 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.612 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.612 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.612 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.612 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.612 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.612 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.612 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.612 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.612 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.612 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.612 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.612 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.612 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.613 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.613 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.613 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.613 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.613 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.613 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.613 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.613 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.613 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.613 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.613 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.613 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.613 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.613 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.614 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.614 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.614 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.614 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.614 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.614 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.614 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.614 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.614 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.614 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.614 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.614 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.614 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.615 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.615 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.615 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.615 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.615 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.615 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.615 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.615 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.615 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.615 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.615 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.615 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.615 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.615 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.616 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.616 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.616 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.616 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.616 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.616 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.616 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.616 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.616 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.616 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.616 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.616 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.616 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.617 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.617 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.617 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.617 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.617 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.617 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pg_autoscaler", 2024-05-17T22:55:09.617 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.617 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.617 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.617 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.617 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.617 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.617 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.617 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.618 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.618 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.618 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.618 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.618 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.618 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.618 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.618 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.618 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.618 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.618 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.618 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.618 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.619 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.619 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.619 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.619 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.619 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.619 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.619 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.619 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.619 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.619 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.619 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.619 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.619 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.619 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.620 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.620 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.620 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.620 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.620 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.620 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.620 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.620 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.620 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.620 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.620 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.620 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.620 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.620 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.621 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.621 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.621 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.621 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.621 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.621 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.621 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.621 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.621 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.621 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.621 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.621 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.621 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.622 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.622 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.622 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.622 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.622 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.622 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.622 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.622 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.622 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.622 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.622 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.622 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-17T22:55:09.622 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-17T22:55:09.623 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:09.623 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.623 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.623 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-17T22:55:09.623 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.623 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.623 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.623 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.623 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.623 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.623 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.623 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.623 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.624 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.624 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.624 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.624 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.624 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.624 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.624 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.624 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.624 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.624 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.624 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.624 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.624 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.624 INFO:teuthology.orchestra.run.smithi012.stdout: "threshold": { 2024-05-17T22:55:09.625 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "threshold", 2024-05-17T22:55:09.625 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-17T22:55:09.625 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.625 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.625 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3.0", 2024-05-17T22:55:09.625 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1.0", 2024-05-17T22:55:09.625 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.625 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.625 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "scaling threshold", 2024-05-17T22:55:09.625 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-17T22:55:09.625 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.625 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.625 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.626 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.626 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.626 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.626 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "progress", 2024-05-17T22:55:09.626 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.626 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.626 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.626 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_pg_recovery_event": { 2024-05-17T22:55:09.626 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_pg_recovery_event", 2024-05-17T22:55:09.626 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.626 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.626 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.626 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.627 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.627 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.627 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.627 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-17T22:55:09.627 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.627 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.627 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.627 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.627 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": { 2024-05-17T22:55:09.627 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enabled", 2024-05-17T22:55:09.627 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.627 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.627 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.627 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.628 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.628 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.628 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.628 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.628 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.628 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.628 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.628 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.628 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.628 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.628 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.628 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.628 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.628 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.629 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.629 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.629 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.629 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.629 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.629 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.629 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.629 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.629 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.629 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.629 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.629 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.629 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.629 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.630 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.630 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.630 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.630 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.630 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.630 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.630 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.630 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.630 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.630 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.630 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.630 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.630 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.630 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.631 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.631 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.631 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.631 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.631 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.631 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.631 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.631 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.631 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.631 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.631 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.631 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.631 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.632 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.632 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.632 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.632 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.632 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.632 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.632 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.632 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.632 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.632 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.632 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.632 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.632 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.632 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.632 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.633 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.633 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.633 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.633 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.633 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.633 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.633 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.633 INFO:teuthology.orchestra.run.smithi012.stdout: "max_completed_events": { 2024-05-17T22:55:09.633 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_completed_events", 2024-05-17T22:55:09.633 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.633 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.633 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.633 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "50", 2024-05-17T22:55:09.633 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.634 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.634 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.634 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "number of past completed events to remember", 2024-05-17T22:55:09.634 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.634 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.634 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.634 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.634 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-05-17T22:55:09.634 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-05-17T22:55:09.634 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:09.634 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.634 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.634 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-05-17T22:55:09.635 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.635 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.635 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.635 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long the module is going to sleep", 2024-05-17T22:55:09.635 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.635 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.635 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.635 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.635 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.635 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.635 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.635 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.635 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.635 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.635 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.636 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.636 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.636 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.636 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.636 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.636 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.636 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.636 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.636 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.636 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.636 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prometheus", 2024-05-17T22:55:09.636 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.636 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.636 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.637 INFO:teuthology.orchestra.run.smithi012.stdout: "cache": { 2024-05-17T22:55:09.637 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cache", 2024-05-17T22:55:09.637 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.637 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.637 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.637 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.637 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.637 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.637 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.637 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.637 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.637 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.637 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.637 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.638 INFO:teuthology.orchestra.run.smithi012.stdout: "exclude_perf_counters": { 2024-05-17T22:55:09.638 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "exclude_perf_counters", 2024-05-17T22:55:09.638 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.638 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.638 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.638 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.638 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.638 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.638 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.638 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-17T22:55:09.638 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-17T22:55:09.638 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.638 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.639 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.639 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.639 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.639 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.639 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.639 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.639 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.639 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.639 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.639 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.639 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.639 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.639 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.639 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.640 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.640 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.640 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.640 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.640 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.640 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.640 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.640 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.640 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.640 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.640 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.640 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.640 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.641 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.641 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.641 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.641 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.641 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.641 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.641 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.641 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.641 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.641 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.641 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.641 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.641 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.641 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.641 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.642 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.642 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.642 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.642 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.642 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.642 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.642 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.642 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.642 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.642 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.642 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.642 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.642 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.643 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.643 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.643 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.643 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.643 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.643 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.643 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.643 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.643 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.643 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.643 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.643 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.643 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.643 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.644 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.644 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.644 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_stats_pools": { 2024-05-17T22:55:09.644 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_stats_pools", 2024-05-17T22:55:09.644 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.644 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.644 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.644 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.644 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.644 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.644 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.644 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.644 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.645 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.645 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.645 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.645 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-17T22:55:09.645 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-17T22:55:09.645 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.645 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.645 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.645 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "300", 2024-05-17T22:55:09.645 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.645 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.645 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.645 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.645 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.646 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.646 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.646 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.646 INFO:teuthology.orchestra.run.smithi012.stdout: "scrape_interval": { 2024-05-17T22:55:09.646 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "scrape_interval", 2024-05-17T22:55:09.646 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-17T22:55:09.646 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.646 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.646 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15.0", 2024-05-17T22:55:09.646 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.646 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.646 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.646 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.647 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.647 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.647 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.647 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.647 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-05-17T22:55:09.647 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-05-17T22:55:09.647 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.647 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.647 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.647 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "::", 2024-05-17T22:55:09.647 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.647 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.647 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.647 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-17T22:55:09.648 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.648 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.648 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.648 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.648 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-05-17T22:55:09.648 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-05-17T22:55:09.648 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.648 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.648 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.648 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "9283", 2024-05-17T22:55:09.648 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.648 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.648 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.649 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-17T22:55:09.649 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.649 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.649 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.649 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.649 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.649 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.649 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.649 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.649 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.649 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.649 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.649 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.650 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.650 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.650 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.650 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.650 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.650 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.650 INFO:teuthology.orchestra.run.smithi012.stdout: "stale_cache_strategy": { 2024-05-17T22:55:09.650 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "stale_cache_strategy", 2024-05-17T22:55:09.650 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.650 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.650 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.650 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "log", 2024-05-17T22:55:09.650 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.650 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.651 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.651 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.651 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.651 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.651 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.651 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.651 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_behaviour": { 2024-05-17T22:55:09.651 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_behaviour", 2024-05-17T22:55:09.651 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.651 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.651 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.651 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "default", 2024-05-17T22:55:09.651 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.652 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.652 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.652 INFO:teuthology.orchestra.run.smithi012.stdout: "default", 2024-05-17T22:55:09.652 INFO:teuthology.orchestra.run.smithi012.stdout: "error" 2024-05-17T22:55:09.652 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.652 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.652 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.652 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.652 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.652 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.652 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_error_status_code": { 2024-05-17T22:55:09.652 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_error_status_code", 2024-05-17T22:55:09.652 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.652 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.653 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.653 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-05-17T22:55:09.653 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "400", 2024-05-17T22:55:09.653 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "599", 2024-05-17T22:55:09.653 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.653 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.653 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.653 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.653 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.653 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.653 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.653 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.653 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.654 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_support", 2024-05-17T22:55:09.654 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.654 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.654 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.654 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.654 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.654 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.654 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.654 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.654 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.654 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.654 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.654 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.655 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.655 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.655 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.655 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.655 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.655 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.655 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.655 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.655 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.655 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.655 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.655 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.655 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.656 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.656 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.656 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.656 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.656 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.656 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.656 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.656 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.656 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.656 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.656 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.656 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.656 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.656 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.657 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.657 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.657 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.657 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.657 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.657 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.657 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.657 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.657 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.657 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.657 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.657 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.657 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.657 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.658 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.658 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.658 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.658 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.658 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.658 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.658 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.658 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.658 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.658 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.658 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.658 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.658 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.659 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.659 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.659 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.659 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.659 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.659 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.659 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.659 INFO:teuthology.orchestra.run.smithi012.stdout: "max_concurrent_snap_create": { 2024-05-17T22:55:09.659 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_concurrent_snap_create", 2024-05-17T22:55:09.659 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.659 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.659 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.659 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-05-17T22:55:09.660 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.660 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.660 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.660 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.660 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.660 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.660 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.660 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.660 INFO:teuthology.orchestra.run.smithi012.stdout: "mirror_snapshot_schedule": { 2024-05-17T22:55:09.660 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mirror_snapshot_schedule", 2024-05-17T22:55:09.660 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.660 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.660 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.661 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.661 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.661 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.661 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.661 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.661 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.661 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.661 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.661 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.661 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.661 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.661 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.661 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.662 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.662 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.662 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.662 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.662 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.662 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.662 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.662 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.662 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.662 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.662 INFO:teuthology.orchestra.run.smithi012.stdout: "trash_purge_schedule": { 2024-05-17T22:55:09.662 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "trash_purge_schedule", 2024-05-17T22:55:09.662 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.663 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.663 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.663 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.663 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.663 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.663 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.663 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.663 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.663 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.663 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.663 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.663 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.663 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.663 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.663 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "restful", 2024-05-17T22:55:09.664 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.664 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.664 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.664 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_auth": { 2024-05-17T22:55:09.664 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enable_auth", 2024-05-17T22:55:09.664 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.664 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.664 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.664 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.664 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.664 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.664 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.664 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.664 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.665 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.665 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.665 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.665 INFO:teuthology.orchestra.run.smithi012.stdout: "key_file": { 2024-05-17T22:55:09.665 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "key_file", 2024-05-17T22:55:09.665 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.665 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.665 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.665 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.665 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.665 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.665 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.665 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.665 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.665 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.666 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.666 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.666 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.666 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.666 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.666 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.666 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.666 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.666 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.666 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.666 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.666 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.666 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.666 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.667 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.667 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.667 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.667 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.667 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.667 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.667 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.667 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.667 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.667 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.667 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.667 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.667 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.668 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.668 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.668 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.668 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.668 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.668 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.668 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.668 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.668 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.668 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.668 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.668 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.668 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.668 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.668 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.669 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.669 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.669 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.669 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.669 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.669 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.669 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.669 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.669 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.669 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.669 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.669 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.669 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.669 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.670 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.670 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.670 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.670 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.670 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.670 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.670 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.670 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.670 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.670 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.670 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.670 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.670 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.671 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.671 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.671 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.671 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-05-17T22:55:09.671 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-05-17T22:55:09.671 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.671 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.671 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.671 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.671 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.671 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.671 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.671 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.672 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.672 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.672 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.672 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.672 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-05-17T22:55:09.672 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-05-17T22:55:09.672 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.672 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.672 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.672 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.672 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.672 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.672 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.672 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.673 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.673 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.673 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.673 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.673 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.673 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.673 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.673 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.673 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.673 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.673 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.673 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.673 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.674 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.674 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.674 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.674 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.674 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.674 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.674 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.674 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.674 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rgw", 2024-05-17T22:55:09.674 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.674 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.674 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.674 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.674 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.675 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.675 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.675 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.675 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.675 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.675 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.675 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.675 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.675 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.675 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.675 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.675 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.675 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.675 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.676 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.676 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.676 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.676 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.676 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.676 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.676 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.676 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.676 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.676 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.676 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.676 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.676 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.676 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.677 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.677 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.677 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.677 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.677 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.677 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.677 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.677 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.677 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.677 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.677 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.677 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.677 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.677 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.677 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.678 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.678 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.678 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.678 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.678 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.678 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.678 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.678 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.678 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.678 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.678 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.678 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.678 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.679 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.679 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.679 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.679 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.679 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.679 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.679 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.679 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.679 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.679 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.679 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.679 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.679 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.679 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.679 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.680 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.680 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.680 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.680 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.680 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.680 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.680 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.680 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.680 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.680 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.680 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.680 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.680 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.680 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.681 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rook", 2024-05-17T22:55:09.681 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.681 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.681 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.681 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.681 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.681 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.681 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.681 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.681 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.681 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.681 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.681 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.681 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.682 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.682 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.682 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.682 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.682 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.682 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.682 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.682 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.682 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.682 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.682 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.682 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.682 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.682 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.683 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.683 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.683 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.683 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.683 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.683 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.683 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.683 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.683 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.683 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.683 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.683 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.683 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.684 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.684 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.684 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.684 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.684 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.684 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.684 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.684 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.684 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.684 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.684 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.684 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.684 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.684 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.684 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.685 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.685 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.685 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.685 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.685 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.685 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.685 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.685 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.685 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.685 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.685 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.685 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.685 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.685 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.685 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.686 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.686 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.686 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.686 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.686 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.686 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.686 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.686 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.686 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.686 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.686 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.686 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.686 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.687 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.687 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.687 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.687 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.687 INFO:teuthology.orchestra.run.smithi012.stdout: "storage_class": { 2024-05-17T22:55:09.687 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "storage_class", 2024-05-17T22:55:09.687 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.687 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.687 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.687 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "local", 2024-05-17T22:55:09.687 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.687 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.687 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.688 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-17T22:55:09.688 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.688 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.688 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.688 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.688 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.688 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.688 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.688 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "selftest", 2024-05-17T22:55:09.688 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.688 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.688 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.688 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.689 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.689 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.689 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.689 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.689 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.689 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.689 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.689 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.689 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.689 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.689 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.689 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.689 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.689 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.689 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.689 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.690 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.690 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.690 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.690 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.690 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.690 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.690 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.690 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.690 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.690 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.690 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.690 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.690 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.691 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.691 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.691 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.691 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.691 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.691 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.691 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.691 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.691 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.691 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.691 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.691 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.691 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.691 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.692 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.692 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.692 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.692 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.692 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.692 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.692 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.692 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.692 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.692 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.692 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.692 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.692 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.692 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.693 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.693 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.693 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.693 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.693 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.693 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.693 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.693 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.693 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.693 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.693 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.693 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.693 INFO:teuthology.orchestra.run.smithi012.stdout: "roption1": { 2024-05-17T22:55:09.693 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "roption1", 2024-05-17T22:55:09.694 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.694 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.694 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.694 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.694 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.694 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.694 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.694 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.694 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.694 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.694 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.694 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.694 INFO:teuthology.orchestra.run.smithi012.stdout: "roption2": { 2024-05-17T22:55:09.695 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "roption2", 2024-05-17T22:55:09.695 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.695 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.695 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.695 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "xyz", 2024-05-17T22:55:09.695 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.695 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.695 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.695 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.695 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.695 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.695 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.695 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.695 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption1": { 2024-05-17T22:55:09.696 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption1", 2024-05-17T22:55:09.696 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.696 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.696 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.696 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.696 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.696 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.696 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.696 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.696 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.696 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.696 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.696 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.697 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption2": { 2024-05-17T22:55:09.697 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption2", 2024-05-17T22:55:09.697 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.697 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.697 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.697 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.697 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.697 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.697 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.697 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.697 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.697 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.697 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.697 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.698 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption3": { 2024-05-17T22:55:09.698 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption3", 2024-05-17T22:55:09.698 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-05-17T22:55:09.698 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.698 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.698 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.698 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.698 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.698 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.698 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.698 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.698 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.698 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.699 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.699 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption4": { 2024-05-17T22:55:09.699 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption4", 2024-05-17T22:55:09.699 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.699 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.699 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.699 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.699 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.699 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.699 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.699 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.699 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.699 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.700 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.700 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.700 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption5": { 2024-05-17T22:55:09.700 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption5", 2024-05-17T22:55:09.700 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.700 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.700 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.700 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.700 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.700 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.700 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.700 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.700 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.700 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.701 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.701 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.701 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption6": { 2024-05-17T22:55:09.701 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption6", 2024-05-17T22:55:09.701 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.701 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.701 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.701 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.701 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.701 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.701 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.701 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.701 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.702 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.702 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.702 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.702 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption7": { 2024-05-17T22:55:09.702 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption7", 2024-05-17T22:55:09.702 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.702 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.702 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.702 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.702 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-05-17T22:55:09.702 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "42", 2024-05-17T22:55:09.702 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.702 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.702 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.703 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.703 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.703 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.703 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.703 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.703 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.703 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.703 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.703 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.703 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.703 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.703 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.703 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.703 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.704 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.704 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.704 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.704 INFO:teuthology.orchestra.run.smithi012.stdout: "testkey": { 2024-05-17T22:55:09.704 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testkey", 2024-05-17T22:55:09.704 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.704 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.704 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.704 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.704 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.704 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.704 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.704 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.704 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.704 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.705 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.705 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.705 INFO:teuthology.orchestra.run.smithi012.stdout: "testlkey": { 2024-05-17T22:55:09.705 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testlkey", 2024-05-17T22:55:09.705 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.705 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.705 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.705 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.705 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.705 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.705 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.705 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.705 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.705 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.706 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.706 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.706 INFO:teuthology.orchestra.run.smithi012.stdout: "testnewline": { 2024-05-17T22:55:09.706 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testnewline", 2024-05-17T22:55:09.706 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.706 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.706 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.706 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.706 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.706 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.706 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.706 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.706 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.706 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.707 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.707 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.707 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.707 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.707 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.707 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smb", 2024-05-17T22:55:09.707 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.707 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.707 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.707 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.707 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.707 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.707 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.707 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.708 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.708 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.708 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.708 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.708 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.708 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.708 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.708 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.708 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.708 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.708 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.708 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.708 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.709 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.709 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.709 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.709 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.709 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.709 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.709 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.709 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.709 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.709 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.709 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.709 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.709 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.709 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.710 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.710 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.710 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.710 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.710 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.710 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.710 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.710 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.710 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.710 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.710 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.710 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.710 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.711 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.711 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.711 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.711 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.711 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.711 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.711 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.711 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.711 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.711 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.711 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.711 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.711 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.711 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.712 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.712 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.712 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.712 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.712 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.712 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.712 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.712 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.712 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.712 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.712 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.712 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.712 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.712 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.713 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.713 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.713 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.713 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.713 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.713 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.713 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.713 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.713 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.713 INFO:teuthology.orchestra.run.smithi012.stdout: "update_orchestration": { 2024-05-17T22:55:09.713 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "update_orchestration", 2024-05-17T22:55:09.713 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.713 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.713 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.714 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.714 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.714 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.714 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.714 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-17T22:55:09.714 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.714 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.714 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.714 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.714 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.714 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.714 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.714 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snap_schedule", 2024-05-17T22:55:09.714 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.715 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.715 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.715 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_m_granularity": { 2024-05-17T22:55:09.715 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_m_granularity", 2024-05-17T22:55:09.715 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.715 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.715 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.715 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.715 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.715 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.715 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.715 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow minute scheduled snapshots", 2024-05-17T22:55:09.715 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.716 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.716 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.716 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.716 INFO:teuthology.orchestra.run.smithi012.stdout: "dump_on_update": { 2024-05-17T22:55:09.716 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "dump_on_update", 2024-05-17T22:55:09.716 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.716 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.716 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.716 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.716 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.716 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.716 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.716 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "dump database to debug log on update", 2024-05-17T22:55:09.716 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.717 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.717 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.717 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.717 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.717 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.717 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.717 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.717 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.717 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.717 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.717 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.717 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.717 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.717 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.717 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.718 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.718 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.718 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.718 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.718 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.718 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.718 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.718 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.718 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.718 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.718 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.718 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.718 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.719 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.719 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.719 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.719 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.719 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.719 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.719 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.719 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.719 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.719 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.719 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.719 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.719 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.719 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.720 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.720 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.720 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.720 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.720 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.720 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.720 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.720 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.720 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.720 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.720 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.720 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.720 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.720 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.721 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.721 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.721 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.721 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.721 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.721 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.721 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.721 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.721 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.721 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.721 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.721 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.721 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.721 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.722 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.722 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.722 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.722 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.722 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.722 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.722 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.722 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.722 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.722 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.722 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.722 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.723 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.723 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.723 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.723 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.723 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.723 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.723 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.723 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "stats", 2024-05-17T22:55:09.723 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.723 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.723 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.723 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.723 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.723 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.724 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.724 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.724 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.724 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.724 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.724 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.724 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.724 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.724 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.724 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.724 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.724 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.724 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.724 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.725 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.725 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.725 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.725 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.725 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.725 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.725 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.725 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.725 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.725 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.725 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.725 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.725 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.726 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.726 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.726 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.726 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.726 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.726 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.726 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.726 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.726 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.726 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.726 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.726 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.727 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.727 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.727 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.727 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.727 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.727 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.727 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.727 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.727 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.727 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.727 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.727 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.728 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.728 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.728 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.728 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.728 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.728 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.728 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.728 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.728 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.728 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.728 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.728 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.728 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.728 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.729 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.729 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.729 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.729 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.729 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.729 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.729 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.729 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.729 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.729 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.729 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.729 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.729 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.729 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.730 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.730 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.730 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "status", 2024-05-17T22:55:09.730 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.730 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.730 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.730 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.730 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.730 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.730 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.730 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.730 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.731 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.731 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.731 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.731 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.731 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.731 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.731 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.731 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.731 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.731 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.731 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.731 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.731 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.732 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.732 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.732 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.732 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.732 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.732 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.732 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.732 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.732 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.732 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.732 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.732 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.732 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.732 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.733 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.733 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.733 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.733 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.733 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.733 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.733 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.733 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.733 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.733 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.733 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.733 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.733 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.733 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.734 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.734 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.734 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.734 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.734 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.734 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.734 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.734 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.734 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.734 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.734 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.734 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.734 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.735 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.735 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.735 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.735 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.735 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.735 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.735 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.735 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.735 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.735 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.735 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.735 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.735 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.736 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.736 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.736 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.736 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.736 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.736 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.736 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.736 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.736 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.736 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "telegraf", 2024-05-17T22:55:09.736 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.736 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.737 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.737 INFO:teuthology.orchestra.run.smithi012.stdout: "address": { 2024-05-17T22:55:09.737 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "address", 2024-05-17T22:55:09.737 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.737 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.737 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.737 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-17T22:55:09.737 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.737 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.737 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.737 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.737 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.737 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.737 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.738 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-17T22:55:09.738 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-17T22:55:09.738 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:09.738 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.738 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.738 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15", 2024-05-17T22:55:09.738 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.738 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.738 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.738 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.738 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.738 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.739 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.739 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.739 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.739 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.739 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.739 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.739 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.739 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.739 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.739 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.739 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.739 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.739 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.740 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.740 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.740 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.740 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.740 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.740 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.740 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.740 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.740 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.740 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.740 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.740 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.740 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.741 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.741 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.741 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.741 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.741 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.741 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.741 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.741 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.741 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.741 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.741 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.741 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.741 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.741 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.742 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.742 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.742 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.742 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.742 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.742 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.742 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.742 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.742 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.742 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.742 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.742 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.742 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.742 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.743 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.743 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.743 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.743 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.743 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.743 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.743 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.743 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.743 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.743 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.743 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.743 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.743 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.744 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.744 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.744 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.744 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.744 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.744 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.744 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.744 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.744 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.744 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.744 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.744 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.745 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.745 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.745 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.745 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "telemetry", 2024-05-17T22:55:09.745 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.745 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.745 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.745 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_basic": { 2024-05-17T22:55:09.745 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_basic", 2024-05-17T22:55:09.745 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.745 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.745 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.746 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.746 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.746 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.746 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.746 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-17T22:55:09.746 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.746 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.746 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.746 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_crash": { 2024-05-17T22:55:09.746 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_crash", 2024-05-17T22:55:09.746 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.747 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.747 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.747 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.747 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.747 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.747 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.747 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-17T22:55:09.747 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.747 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.747 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.747 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_device": { 2024-05-17T22:55:09.747 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_device", 2024-05-17T22:55:09.748 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.748 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.748 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.748 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.748 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.748 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.748 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.748 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-17T22:55:09.748 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.748 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.748 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.748 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_ident": { 2024-05-17T22:55:09.748 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_ident", 2024-05-17T22:55:09.749 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.749 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.749 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.749 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.749 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.749 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.749 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.749 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-17T22:55:09.749 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.749 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.749 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.749 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_perf": { 2024-05-17T22:55:09.749 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_perf", 2024-05-17T22:55:09.750 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.750 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.750 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.750 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.750 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.750 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.750 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.750 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-17T22:55:09.750 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.750 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.750 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.750 INFO:teuthology.orchestra.run.smithi012.stdout: "contact": { 2024-05-17T22:55:09.750 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "contact", 2024-05-17T22:55:09.751 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.751 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.751 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.751 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.751 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.751 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.751 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.751 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.751 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.751 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.751 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.751 INFO:teuthology.orchestra.run.smithi012.stdout: "description": { 2024-05-17T22:55:09.751 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "description", 2024-05-17T22:55:09.751 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.752 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.752 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.752 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.752 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.752 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.752 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.752 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.752 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.752 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.752 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.752 INFO:teuthology.orchestra.run.smithi012.stdout: "device_url": { 2024-05-17T22:55:09.752 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_url", 2024-05-17T22:55:09.752 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.753 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.753 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.753 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-17T22:55:09.753 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.753 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.753 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.753 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.753 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.753 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.753 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.753 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.753 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": { 2024-05-17T22:55:09.753 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enabled", 2024-05-17T22:55:09.753 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.754 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.754 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.754 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.754 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.754 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.754 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.754 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.754 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.754 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.754 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.754 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-17T22:55:09.754 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-17T22:55:09.754 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.755 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.755 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.755 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "24", 2024-05-17T22:55:09.755 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "8", 2024-05-17T22:55:09.755 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.755 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.755 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.755 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.755 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.755 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.755 INFO:teuthology.orchestra.run.smithi012.stdout: "last_opt_revision": { 2024-05-17T22:55:09.755 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "last_opt_revision", 2024-05-17T22:55:09.755 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.756 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.756 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.756 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1", 2024-05-17T22:55:09.756 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.756 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.756 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.756 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.756 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.756 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.756 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.756 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.756 INFO:teuthology.orchestra.run.smithi012.stdout: "leaderboard": { 2024-05-17T22:55:09.756 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "leaderboard", 2024-05-17T22:55:09.756 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.757 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.757 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.757 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.757 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.757 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.757 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.757 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.757 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.757 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.757 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.757 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.757 INFO:teuthology.orchestra.run.smithi012.stdout: "leaderboard_description": { 2024-05-17T22:55:09.757 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "leaderboard_description", 2024-05-17T22:55:09.757 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.758 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.758 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.758 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.758 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.758 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.758 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.758 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.758 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.758 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.758 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.758 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.758 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.758 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.759 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.759 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.759 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.759 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.759 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.759 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.759 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.759 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.759 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.759 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.759 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.759 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.759 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.759 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.760 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.760 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.760 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.760 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.760 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.760 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.760 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.760 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.760 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.760 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.760 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.760 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.761 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.761 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.761 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.761 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.761 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.761 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.761 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.761 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.761 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.761 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.761 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.761 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.761 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.762 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.762 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.762 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.762 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.762 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.762 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.762 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.762 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.762 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.762 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.762 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.762 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.763 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.763 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.763 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.763 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.763 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.763 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.763 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.763 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.763 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.763 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.763 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.763 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.764 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.764 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.764 INFO:teuthology.orchestra.run.smithi012.stdout: "organization": { 2024-05-17T22:55:09.764 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "organization", 2024-05-17T22:55:09.764 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.764 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.764 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.764 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.764 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.764 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.764 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.764 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.764 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.764 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.765 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.765 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.765 INFO:teuthology.orchestra.run.smithi012.stdout: "proxy": { 2024-05-17T22:55:09.765 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "proxy", 2024-05-17T22:55:09.765 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.765 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.765 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.765 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.765 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.765 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.765 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.765 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.765 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.765 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.765 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.766 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.766 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.766 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.766 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.766 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.766 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.766 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.766 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.766 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.766 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.766 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.766 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.766 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.766 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.767 INFO:teuthology.orchestra.run.smithi012.stdout: "url": { 2024-05-17T22:55:09.767 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "url", 2024-05-17T22:55:09.767 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.767 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.767 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.767 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-17T22:55:09.767 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.767 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.767 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.767 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.767 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.767 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.767 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.768 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.768 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.768 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.768 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "test_orchestrator", 2024-05-17T22:55:09.768 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.768 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.768 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.768 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.768 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.768 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.768 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.768 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.768 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.769 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.769 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.769 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.769 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.769 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.769 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.769 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.769 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.769 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.769 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.769 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.769 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.769 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.769 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.770 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.770 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.770 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.770 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.770 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.770 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.770 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.770 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.770 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.770 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.770 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.770 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.770 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.771 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.771 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.771 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.771 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.771 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.771 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.771 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.771 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.771 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.771 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.771 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.771 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.771 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.772 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.772 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.772 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.772 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.772 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.772 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.772 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.772 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.772 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.772 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.772 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.772 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.772 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.773 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.773 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.773 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.773 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.773 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.773 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.773 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.773 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.773 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.773 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.773 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.773 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.773 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.774 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.774 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.774 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.774 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.774 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.774 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.774 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.774 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.774 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.774 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.774 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.774 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.774 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.775 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "volumes", 2024-05-17T22:55:09.775 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.775 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.775 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.775 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.775 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.775 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.775 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.775 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.775 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.775 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.775 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.775 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.775 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.776 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.776 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.776 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.776 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.776 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.776 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.776 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.776 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.776 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.776 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.776 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.776 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.776 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.776 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.776 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.778 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.778 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.778 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.778 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.778 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.779 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.779 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.779 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.779 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.779 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.779 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.779 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.779 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.779 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.779 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.779 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.779 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.779 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.780 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.780 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.780 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.780 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.780 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.780 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.780 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.780 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.780 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.780 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.780 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.780 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.780 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.780 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.781 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.781 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.781 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.781 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.781 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.781 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.781 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.781 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.781 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.781 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.781 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.781 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.781 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.781 INFO:teuthology.orchestra.run.smithi012.stdout: "max_concurrent_clones": { 2024-05-17T22:55:09.782 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_concurrent_clones", 2024-05-17T22:55:09.782 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.782 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.782 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.782 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "4", 2024-05-17T22:55:09.782 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.782 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.782 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.782 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-17T22:55:09.782 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.782 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.782 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.782 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.783 INFO:teuthology.orchestra.run.smithi012.stdout: "periodic_async_work": { 2024-05-17T22:55:09.783 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "periodic_async_work", 2024-05-17T22:55:09.783 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.783 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.783 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.783 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.783 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.783 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.783 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.783 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Periodically check for async work", 2024-05-17T22:55:09.783 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.783 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.783 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.783 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.784 INFO:teuthology.orchestra.run.smithi012.stdout: "snapshot_clone_delay": { 2024-05-17T22:55:09.784 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snapshot_clone_delay", 2024-05-17T22:55:09.784 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.784 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.784 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.784 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.784 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.784 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.784 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.784 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-17T22:55:09.784 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.784 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.784 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.784 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.784 INFO:teuthology.orchestra.run.smithi012.stdout: "snapshot_clone_no_wait": { 2024-05-17T22:55:09.785 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snapshot_clone_no_wait", 2024-05-17T22:55:09.785 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.785 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.785 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.785 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-05-17T22:55:09.785 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.785 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.785 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.785 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-17T22:55:09.785 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.785 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.785 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.785 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.785 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.786 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.786 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.786 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.786 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.786 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.786 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.786 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.786 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.786 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.786 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.786 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.786 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.786 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.786 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.787 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.787 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix", 2024-05-17T22:55:09.787 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-05-17T22:55:09.787 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-05-17T22:55:09.787 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-05-17T22:55:09.787 INFO:teuthology.orchestra.run.smithi012.stdout: "discovery_interval": { 2024-05-17T22:55:09.787 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "discovery_interval", 2024-05-17T22:55:09.787 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-05-17T22:55:09.787 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.787 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.787 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "100", 2024-05-17T22:55:09.787 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.787 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.788 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.788 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.788 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.788 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.788 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.788 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.788 INFO:teuthology.orchestra.run.smithi012.stdout: "identifier": { 2024-05-17T22:55:09.788 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "identifier", 2024-05-17T22:55:09.788 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.788 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.788 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.788 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.788 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.789 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.789 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.789 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.789 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.789 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.789 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.789 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.789 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-05-17T22:55:09.789 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-05-17T22:55:09.789 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-05-17T22:55:09.789 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.789 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.789 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-05-17T22:55:09.789 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.789 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.790 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.790 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.790 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.790 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.790 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.790 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.790 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-05-17T22:55:09.790 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-05-17T22:55:09.790 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.790 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.790 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.790 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.790 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.791 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.791 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.791 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.791 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.791 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.791 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.791 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.791 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.791 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.791 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.791 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.791 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.791 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.792 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.792 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-05-17T22:55:09.792 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-05-17T22:55:09.792 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.792 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.792 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.792 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.792 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.792 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.792 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.792 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.792 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.792 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.792 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.793 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.793 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-05-17T22:55:09.793 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-05-17T22:55:09.793 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.793 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.793 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.793 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-05-17T22:55:09.793 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.793 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.793 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-05-17T22:55:09.793 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-05-17T22:55:09.793 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-05-17T22:55:09.793 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-05-17T22:55:09.793 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-05-17T22:55:09.793 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-05-17T22:55:09.794 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-05-17T22:55:09.794 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.794 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.794 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.794 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.794 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.794 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.794 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-05-17T22:55:09.794 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-05-17T22:55:09.794 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-05-17T22:55:09.794 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.794 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.794 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-05-17T22:55:09.795 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.795 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.795 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.795 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.795 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.795 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.795 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.795 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.795 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-05-17T22:55:09.795 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-05-17T22:55:09.795 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.795 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-05-17T22:55:09.795 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-05-17T22:55:09.796 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-05-17T22:55:09.796 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.796 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.796 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.796 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.796 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.796 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.796 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.796 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.796 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_host": { 2024-05-17T22:55:09.796 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_host", 2024-05-17T22:55:09.796 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.796 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.796 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.797 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-05-17T22:55:09.797 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.797 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.797 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.797 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.797 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.797 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.797 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.797 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.797 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_port": { 2024-05-17T22:55:09.797 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_port", 2024-05-17T22:55:09.797 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-05-17T22:55:09.797 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.797 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.797 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10051", 2024-05-17T22:55:09.798 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.798 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.798 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.798 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.798 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.798 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.798 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.798 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.798 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_sender": { 2024-05-17T22:55:09.798 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_sender", 2024-05-17T22:55:09.798 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-05-17T22:55:09.798 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-05-17T22:55:09.798 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-05-17T22:55:09.798 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-17T22:55:09.799 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-05-17T22:55:09.799 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-05-17T22:55:09.799 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-05-17T22:55:09.799 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-05-17T22:55:09.799 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-05-17T22:55:09.799 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-05-17T22:55:09.799 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-05-17T22:55:09.799 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.799 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.799 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.799 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.799 INFO:teuthology.orchestra.run.smithi012.stdout: "services": { 2024-05-17T22:55:09.799 INFO:teuthology.orchestra.run.smithi012.stdout: "dashboard": "https://172.21.15.12:8443/" 2024-05-17T22:55:09.800 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.800 INFO:teuthology.orchestra.run.smithi012.stdout: "always_on_modules": { 2024-05-17T22:55:09.800 INFO:teuthology.orchestra.run.smithi012.stdout: "octopus": [ 2024-05-17T22:55:09.800 INFO:teuthology.orchestra.run.smithi012.stdout: "balancer", 2024-05-17T22:55:09.800 INFO:teuthology.orchestra.run.smithi012.stdout: "crash", 2024-05-17T22:55:09.800 INFO:teuthology.orchestra.run.smithi012.stdout: "devicehealth", 2024-05-17T22:55:09.800 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator", 2024-05-17T22:55:09.800 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscaler", 2024-05-17T22:55:09.800 INFO:teuthology.orchestra.run.smithi012.stdout: "progress", 2024-05-17T22:55:09.800 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_support", 2024-05-17T22:55:09.800 INFO:teuthology.orchestra.run.smithi012.stdout: "status", 2024-05-17T22:55:09.800 INFO:teuthology.orchestra.run.smithi012.stdout: "telemetry", 2024-05-17T22:55:09.800 INFO:teuthology.orchestra.run.smithi012.stdout: "volumes" 2024-05-17T22:55:09.800 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.800 INFO:teuthology.orchestra.run.smithi012.stdout: "pacific": [ 2024-05-17T22:55:09.801 INFO:teuthology.orchestra.run.smithi012.stdout: "balancer", 2024-05-17T22:55:09.801 INFO:teuthology.orchestra.run.smithi012.stdout: "crash", 2024-05-17T22:55:09.801 INFO:teuthology.orchestra.run.smithi012.stdout: "devicehealth", 2024-05-17T22:55:09.801 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator", 2024-05-17T22:55:09.801 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscaler", 2024-05-17T22:55:09.801 INFO:teuthology.orchestra.run.smithi012.stdout: "progress", 2024-05-17T22:55:09.801 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_support", 2024-05-17T22:55:09.801 INFO:teuthology.orchestra.run.smithi012.stdout: "status", 2024-05-17T22:55:09.801 INFO:teuthology.orchestra.run.smithi012.stdout: "telemetry", 2024-05-17T22:55:09.801 INFO:teuthology.orchestra.run.smithi012.stdout: "volumes" 2024-05-17T22:55:09.801 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.801 INFO:teuthology.orchestra.run.smithi012.stdout: "quincy": [ 2024-05-17T22:55:09.801 INFO:teuthology.orchestra.run.smithi012.stdout: "balancer", 2024-05-17T22:55:09.802 INFO:teuthology.orchestra.run.smithi012.stdout: "crash", 2024-05-17T22:55:09.802 INFO:teuthology.orchestra.run.smithi012.stdout: "devicehealth", 2024-05-17T22:55:09.802 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator", 2024-05-17T22:55:09.802 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscaler", 2024-05-17T22:55:09.802 INFO:teuthology.orchestra.run.smithi012.stdout: "progress", 2024-05-17T22:55:09.802 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_support", 2024-05-17T22:55:09.802 INFO:teuthology.orchestra.run.smithi012.stdout: "status", 2024-05-17T22:55:09.802 INFO:teuthology.orchestra.run.smithi012.stdout: "telemetry", 2024-05-17T22:55:09.802 INFO:teuthology.orchestra.run.smithi012.stdout: "volumes" 2024-05-17T22:55:09.802 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.802 INFO:teuthology.orchestra.run.smithi012.stdout: "reef": [ 2024-05-17T22:55:09.802 INFO:teuthology.orchestra.run.smithi012.stdout: "balancer", 2024-05-17T22:55:09.802 INFO:teuthology.orchestra.run.smithi012.stdout: "crash", 2024-05-17T22:55:09.803 INFO:teuthology.orchestra.run.smithi012.stdout: "devicehealth", 2024-05-17T22:55:09.803 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator", 2024-05-17T22:55:09.803 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscaler", 2024-05-17T22:55:09.803 INFO:teuthology.orchestra.run.smithi012.stdout: "progress", 2024-05-17T22:55:09.803 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_support", 2024-05-17T22:55:09.803 INFO:teuthology.orchestra.run.smithi012.stdout: "status", 2024-05-17T22:55:09.803 INFO:teuthology.orchestra.run.smithi012.stdout: "telemetry", 2024-05-17T22:55:09.803 INFO:teuthology.orchestra.run.smithi012.stdout: "volumes" 2024-05-17T22:55:09.803 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-05-17T22:55:09.803 INFO:teuthology.orchestra.run.smithi012.stdout: "squid": [ 2024-05-17T22:55:09.803 INFO:teuthology.orchestra.run.smithi012.stdout: "balancer", 2024-05-17T22:55:09.803 INFO:teuthology.orchestra.run.smithi012.stdout: "crash", 2024-05-17T22:55:09.803 INFO:teuthology.orchestra.run.smithi012.stdout: "devicehealth", 2024-05-17T22:55:09.804 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator", 2024-05-17T22:55:09.804 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscaler", 2024-05-17T22:55:09.804 INFO:teuthology.orchestra.run.smithi012.stdout: "progress", 2024-05-17T22:55:09.804 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_support", 2024-05-17T22:55:09.804 INFO:teuthology.orchestra.run.smithi012.stdout: "status", 2024-05-17T22:55:09.804 INFO:teuthology.orchestra.run.smithi012.stdout: "telemetry", 2024-05-17T22:55:09.804 INFO:teuthology.orchestra.run.smithi012.stdout: "volumes" 2024-05-17T22:55:09.804 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-05-17T22:55:09.804 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.804 INFO:teuthology.orchestra.run.smithi012.stdout: "last_failure_osd_epoch": 3, 2024-05-17T22:55:09.804 INFO:teuthology.orchestra.run.smithi012.stdout: "active_clients": [ 2024-05-17T22:55:09.804 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.804 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "devicehealth", 2024-05-17T22:55:09.804 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-05-17T22:55:09.804 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.805 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-05-17T22:55:09.805 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:0", 2024-05-17T22:55:09.805 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3460962211 2024-05-17T22:55:09.805 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.805 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-05-17T22:55:09.805 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.805 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.805 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "libcephsqlite", 2024-05-17T22:55:09.805 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-05-17T22:55:09.805 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.805 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-05-17T22:55:09.805 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:0", 2024-05-17T22:55:09.805 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2647229663 2024-05-17T22:55:09.805 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.805 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-05-17T22:55:09.806 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.806 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.806 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_support", 2024-05-17T22:55:09.806 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-05-17T22:55:09.806 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.806 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-05-17T22:55:09.806 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:0", 2024-05-17T22:55:09.806 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2957487127 2024-05-17T22:55:09.806 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.806 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-05-17T22:55:09.806 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:09.806 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.806 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "volumes", 2024-05-17T22:55:09.806 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-05-17T22:55:09.807 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-05-17T22:55:09.807 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-05-17T22:55:09.807 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:0", 2024-05-17T22:55:09.807 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 852753273 2024-05-17T22:55:09.807 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.807 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-05-17T22:55:09.807 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:09.807 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-05-17T22:55:09.807 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:55:09.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.343+0000 7fd989ef6640 1 -- 172.21.15.12:0/2043518230 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd96c0b27f0 msgr2=0x7fd96c0b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:09.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.343+0000 7fd989ef6640 1 --2- 172.21.15.12:0/2043518230 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd96c0b27f0 0x7fd96c0b4d00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd9740058c0 tx=0x7fd974005850 comp rx=0 tx=0).stop 2024-05-17T22:55:09.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.343+0000 7fd989ef6640 1 -- 172.21.15.12:0/2043518230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd984104270 msgr2=0x7fd984199e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:09.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.343+0000 7fd989ef6640 1 --2- 172.21.15.12:0/2043518230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd984104270 0x7fd984199e10 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fd978006010 tx=0x7fd978004340 comp rx=0 tx=0).stop 2024-05-17T22:55:09.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.343+0000 7fd989ef6640 1 -- 172.21.15.12:0/2043518230 shutdown_connections 2024-05-17T22:55:09.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.343+0000 7fd989ef6640 1 --2- 172.21.15.12:0/2043518230 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fd96c0b27f0 0x7fd96c0b4d00 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:09.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.343+0000 7fd989ef6640 1 --2- 172.21.15.12:0/2043518230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd984104bc0 0x7fd9841a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:09.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.343+0000 7fd989ef6640 1 --2- 172.21.15.12:0/2043518230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd984104270 0x7fd984199e10 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:09.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.343+0000 7fd989ef6640 1 --2- 172.21.15.12:0/2043518230 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd984103150 0x7fd9841998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:09.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.343+0000 7fd989ef6640 1 -- 172.21.15.12:0/2043518230 >> 172.21.15.12:0/2043518230 conn(0x7fd9840fe950 msgr2=0x7fd984107570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:55:09.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.343+0000 7fd989ef6640 1 -- 172.21.15.12:0/2043518230 shutdown_connections 2024-05-17T22:55:09.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:08.343+0000 7fd989ef6640 1 -- 172.21.15.12:0/2043518230 wait complete. 2024-05-17T22:55:09.857 DEBUG:tasks.check_counter:Getting stats from a 2024-05-17T22:55:09.857 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-17T22:55:09.858 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:09 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/2043518230' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-17T22:55:09.859 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:09 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/2043518230' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-17T22:55:09.859 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:09 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/2043518230' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-17T22:55:10.121 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:55:10.476 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:10 smithi012 ceph-mon[31632]: pgmap v916: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.8 KiB/s wr, 9 op/s 2024-05-17T22:55:10.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:10 smithi151 ceph-mon[36101]: pgmap v916: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.8 KiB/s wr, 9 op/s 2024-05-17T22:55:10.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:10 smithi096 ceph-mon[36065]: pgmap v916: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.8 KiB/s wr, 9 op/s 2024-05-17T22:55:12.096 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:55:12.096 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-0": { 2024-05-17T22:55:12.096 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 4, 2024-05-17T22:55:12.096 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 4, 2024-05-17T22:55:12.097 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 2992, 2024-05-17T22:55:12.097 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 448, 2024-05-17T22:55:12.097 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 4, 2024-05-17T22:55:12.097 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 0, 2024-05-17T22:55:12.097 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.001822223, 2024-05-17T22:55:12.097 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.000039527, 2024-05-17T22:55:12.097 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.000649874, 2024-05-17T22:55:12.097 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T22:55:12.097 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T22:55:12.097 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 3, 2024-05-17T22:55:12.097 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000811890, 2024-05-17T22:55:12.097 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000270630 2024-05-17T22:55:12.098 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:12.098 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-05-17T22:55:12.098 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-17T22:55:12.098 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-17T22:55:12.098 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-17T22:55:12.098 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:12.098 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-17T22:55:12.098 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 448 2024-05-17T22:55:12.098 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:12.098 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-1": { 2024-05-17T22:55:12.098 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 2, 2024-05-17T22:55:12.098 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 254, 2024-05-17T22:55:12.098 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 320, 2024-05-17T22:55:12.099 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 202400, 2024-05-17T22:55:12.099 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 4, 2024-05-17T22:55:12.099 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 2, 2024-05-17T22:55:12.099 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.019113984, 2024-05-17T22:55:12.099 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.014371971, 2024-05-17T22:55:12.099 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.001882569, 2024-05-17T22:55:12.099 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T22:55:12.099 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T22:55:12.099 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 252, 2024-05-17T22:55:12.099 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.024992212, 2024-05-17T22:55:12.099 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000099175 2024-05-17T22:55:12.099 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:12.099 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-05-17T22:55:12.100 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-17T22:55:12.100 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-17T22:55:12.100 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-17T22:55:12.100 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:12.100 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-17T22:55:12.100 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 202400 2024-05-17T22:55:12.100 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:12.100 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-2": { 2024-05-17T22:55:12.100 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 334, 2024-05-17T22:55:12.100 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 331, 2024-05-17T22:55:12.100 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 546400, 2024-05-17T22:55:12.100 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 204720, 2024-05-17T22:55:12.100 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 2, 2024-05-17T22:55:12.100 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 1, 2024-05-17T22:55:12.101 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.052480720, 2024-05-17T22:55:12.101 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.022223438, 2024-05-17T22:55:12.101 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.036221715, 2024-05-17T22:55:12.101 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T22:55:12.101 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T22:55:12.101 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 330, 2024-05-17T22:55:12.101 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.024680862, 2024-05-17T22:55:12.101 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000074790 2024-05-17T22:55:12.101 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:12.101 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-05-17T22:55:12.101 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-17T22:55:12.101 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-17T22:55:12.101 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-17T22:55:12.101 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:12.101 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 546400, 2024-05-17T22:55:12.102 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 204720 2024-05-17T22:55:12.102 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:12.102 INFO:teuthology.orchestra.run.smithi012.stdout: "cct": { 2024-05-17T22:55:12.102 INFO:teuthology.orchestra.run.smithi012.stdout: "total_workers": 0, 2024-05-17T22:55:12.102 INFO:teuthology.orchestra.run.smithi012.stdout: "unhealthy_workers": 0 2024-05-17T22:55:12.102 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:12.102 INFO:teuthology.orchestra.run.smithi012.stdout: "mempool": { 2024-05-17T22:55:12.102 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_bytes": 0, 2024-05-17T22:55:12.102 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_items": 0, 2024-05-17T22:55:12.102 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_bytes": 0, 2024-05-17T22:55:12.102 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_items": 0, 2024-05-17T22:55:12.102 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_bytes": 0, 2024-05-17T22:55:12.103 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_items": 0, 2024-05-17T22:55:12.103 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-17T22:55:12.103 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_items": 0, 2024-05-17T22:55:12.103 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-17T22:55:12.103 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_items": 0, 2024-05-17T22:55:12.103 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_bytes": 0, 2024-05-17T22:55:12.103 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_items": 0, 2024-05-17T22:55:12.103 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-17T22:55:12.103 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_items": 0, 2024-05-17T22:55:12.103 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_bytes": 0, 2024-05-17T22:55:12.103 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_items": 0, 2024-05-17T22:55:12.104 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_bytes": 0, 2024-05-17T22:55:12.104 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_items": 0, 2024-05-17T22:55:12.104 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-17T22:55:12.104 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_items": 0, 2024-05-17T22:55:12.104 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-17T22:55:12.104 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_items": 0, 2024-05-17T22:55:12.104 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_bytes": 0, 2024-05-17T22:55:12.104 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_items": 0, 2024-05-17T22:55:12.104 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_bytes": 0, 2024-05-17T22:55:12.104 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_items": 0, 2024-05-17T22:55:12.104 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-17T22:55:12.104 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_items": 0, 2024-05-17T22:55:12.104 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_bytes": 0, 2024-05-17T22:55:12.104 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_items": 0, 2024-05-17T22:55:12.105 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_bytes": 0, 2024-05-17T22:55:12.105 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_items": 0, 2024-05-17T22:55:12.105 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_bytes": 0, 2024-05-17T22:55:12.105 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_items": 0, 2024-05-17T22:55:12.105 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_bytes": 0, 2024-05-17T22:55:12.105 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_items": 0, 2024-05-17T22:55:12.105 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_bytes": 148744, 2024-05-17T22:55:12.105 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_items": 30, 2024-05-17T22:55:12.105 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_bytes": 0, 2024-05-17T22:55:12.105 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_items": 0, 2024-05-17T22:55:12.105 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_bytes": 0, 2024-05-17T22:55:12.105 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_items": 0, 2024-05-17T22:55:12.105 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_bytes": 0, 2024-05-17T22:55:12.105 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_items": 0, 2024-05-17T22:55:12.106 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_bytes": 0, 2024-05-17T22:55:12.106 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_items": 0, 2024-05-17T22:55:12.106 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_bytes": 0, 2024-05-17T22:55:12.106 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_items": 0, 2024-05-17T22:55:12.106 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_bytes": 0, 2024-05-17T22:55:12.106 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_items": 0, 2024-05-17T22:55:12.106 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_bytes": 0, 2024-05-17T22:55:12.106 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_items": 0, 2024-05-17T22:55:12.106 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_bytes": 928, 2024-05-17T22:55:12.106 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_items": 2, 2024-05-17T22:55:12.106 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_bytes": 0, 2024-05-17T22:55:12.106 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_items": 0, 2024-05-17T22:55:12.106 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_bytes": 0, 2024-05-17T22:55:12.107 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_items": 0 2024-05-17T22:55:12.107 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:12.107 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-17T22:55:12.107 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-05-17T22:55:12.107 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 104857600, 2024-05-17T22:55:12.107 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-05-17T22:55:12.107 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 336, 2024-05-17T22:55:12.107 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 499167, 2024-05-17T22:55:12.107 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-05-17T22:55:12.107 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 336, 2024-05-17T22:55:12.107 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 0, 2024-05-17T22:55:12.107 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 0, 2024-05-17T22:55:12.107 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 336, 2024-05-17T22:55:12.107 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 499167, 2024-05-17T22:55:12.108 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-05-17T22:55:12.108 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-17T22:55:12.108 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-17T22:55:12.108 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-17T22:55:12.108 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:12.108 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:12.108 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:55:12.336 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:12 smithi012 ceph-mon[31632]: pgmap v917: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 9.5 KiB/s wr, 10 op/s 2024-05-17T22:55:12.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:12 smithi151 ceph-mon[36101]: pgmap v917: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 9.5 KiB/s wr, 10 op/s 2024-05-17T22:55:12.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:12 smithi096 ceph-mon[36065]: pgmap v917: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 9.5 KiB/s wr, 10 op/s 2024-05-17T22:55:12.809 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-05-17T22:55:12.809 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-05-17T22:55:12.809 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-05-17T22:55:12.809 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-05-17T22:55:12.809 DEBUG:tasks.check_counter:Getting stats from d 2024-05-17T22:55:12.810 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-17T22:55:13.062 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:55:14.475 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:14 smithi012 ceph-mon[31632]: pgmap v918: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.2 KiB/s wr, 8 op/s 2024-05-17T22:55:14.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:14 smithi151 ceph-mon[36101]: pgmap v918: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.2 KiB/s wr, 8 op/s 2024-05-17T22:55:14.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:14 smithi096 ceph-mon[36065]: pgmap v918: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.2 KiB/s wr, 8 op/s 2024-05-17T22:55:14.783 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:55:14.783 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-0": { 2024-05-17T22:55:14.783 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 0, 2024-05-17T22:55:14.784 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 2, 2024-05-17T22:55:14.784 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 0, 2024-05-17T22:55:14.784 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 0, 2024-05-17T22:55:14.784 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 4, 2024-05-17T22:55:14.784 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 0, 2024-05-17T22:55:14.784 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.001427874, 2024-05-17T22:55:14.784 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.000000000, 2024-05-17T22:55:14.784 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.000358859, 2024-05-17T22:55:14.784 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T22:55:14.784 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T22:55:14.784 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-17T22:55:14.784 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-17T22:55:14.784 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-17T22:55:14.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:14.785 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-05-17T22:55:14.785 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-17T22:55:14.785 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-17T22:55:14.785 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-17T22:55:14.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:14.785 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-17T22:55:14.785 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 0 2024-05-17T22:55:14.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:14.785 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-1": { 2024-05-17T22:55:14.785 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 2, 2024-05-17T22:55:14.785 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 259, 2024-05-17T22:55:14.785 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 320, 2024-05-17T22:55:14.786 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 203280, 2024-05-17T22:55:14.786 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 4, 2024-05-17T22:55:14.786 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 2, 2024-05-17T22:55:14.786 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.021474703, 2024-05-17T22:55:14.786 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.016346864, 2024-05-17T22:55:14.786 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.001864708, 2024-05-17T22:55:14.786 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T22:55:14.786 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T22:55:14.786 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 257, 2024-05-17T22:55:14.786 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.015871359, 2024-05-17T22:55:14.786 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000061756 2024-05-17T22:55:14.786 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:14.786 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-05-17T22:55:14.786 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-17T22:55:14.787 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-17T22:55:14.787 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-17T22:55:14.787 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:14.787 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-17T22:55:14.787 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 203280 2024-05-17T22:55:14.787 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:14.787 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-2": { 2024-05-17T22:55:14.787 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 346, 2024-05-17T22:55:14.787 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 341, 2024-05-17T22:55:14.787 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 552256, 2024-05-17T22:55:14.787 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 209504, 2024-05-17T22:55:14.787 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 2, 2024-05-17T22:55:14.787 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 1, 2024-05-17T22:55:14.788 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.053610989, 2024-05-17T22:55:14.788 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.022157966, 2024-05-17T22:55:14.788 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.037041169, 2024-05-17T22:55:14.788 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T22:55:14.788 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T22:55:14.788 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 341, 2024-05-17T22:55:14.788 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.028834912, 2024-05-17T22:55:14.788 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000084559 2024-05-17T22:55:14.788 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:14.788 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-05-17T22:55:14.788 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-17T22:55:14.788 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-17T22:55:14.788 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-17T22:55:14.789 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:14.789 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 552256, 2024-05-17T22:55:14.789 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 209504 2024-05-17T22:55:14.789 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:14.789 INFO:teuthology.orchestra.run.smithi012.stdout: "cct": { 2024-05-17T22:55:14.789 INFO:teuthology.orchestra.run.smithi012.stdout: "total_workers": 0, 2024-05-17T22:55:14.789 INFO:teuthology.orchestra.run.smithi012.stdout: "unhealthy_workers": 0 2024-05-17T22:55:14.789 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:14.789 INFO:teuthology.orchestra.run.smithi012.stdout: "mempool": { 2024-05-17T22:55:14.789 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_bytes": 0, 2024-05-17T22:55:14.789 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_items": 0, 2024-05-17T22:55:14.789 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_bytes": 0, 2024-05-17T22:55:14.789 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_items": 0, 2024-05-17T22:55:14.789 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_bytes": 0, 2024-05-17T22:55:14.790 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_items": 0, 2024-05-17T22:55:14.790 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-17T22:55:14.790 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_items": 0, 2024-05-17T22:55:14.790 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-17T22:55:14.790 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_items": 0, 2024-05-17T22:55:14.790 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_bytes": 0, 2024-05-17T22:55:14.790 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_items": 0, 2024-05-17T22:55:14.790 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-17T22:55:14.790 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_items": 0, 2024-05-17T22:55:14.790 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_bytes": 0, 2024-05-17T22:55:14.790 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_items": 0, 2024-05-17T22:55:14.790 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_bytes": 0, 2024-05-17T22:55:14.790 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_items": 0, 2024-05-17T22:55:14.791 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-17T22:55:14.791 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_items": 0, 2024-05-17T22:55:14.791 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-17T22:55:14.791 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_items": 0, 2024-05-17T22:55:14.791 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_bytes": 0, 2024-05-17T22:55:14.791 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_items": 0, 2024-05-17T22:55:14.791 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_bytes": 0, 2024-05-17T22:55:14.791 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_items": 0, 2024-05-17T22:55:14.791 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-17T22:55:14.791 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_items": 0, 2024-05-17T22:55:14.791 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_bytes": 0, 2024-05-17T22:55:14.791 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_items": 0, 2024-05-17T22:55:14.791 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_bytes": 0, 2024-05-17T22:55:14.792 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_items": 0, 2024-05-17T22:55:14.792 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_bytes": 0, 2024-05-17T22:55:14.792 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_items": 0, 2024-05-17T22:55:14.792 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_bytes": 0, 2024-05-17T22:55:14.792 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_items": 0, 2024-05-17T22:55:14.792 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_bytes": 148744, 2024-05-17T22:55:14.792 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_items": 30, 2024-05-17T22:55:14.792 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_bytes": 0, 2024-05-17T22:55:14.792 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_items": 0, 2024-05-17T22:55:14.792 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_bytes": 0, 2024-05-17T22:55:14.792 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_items": 0, 2024-05-17T22:55:14.792 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_bytes": 0, 2024-05-17T22:55:14.792 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_items": 0, 2024-05-17T22:55:14.792 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_bytes": 0, 2024-05-17T22:55:14.793 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_items": 0, 2024-05-17T22:55:14.793 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_bytes": 0, 2024-05-17T22:55:14.793 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_items": 0, 2024-05-17T22:55:14.793 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_bytes": 0, 2024-05-17T22:55:14.793 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_items": 0, 2024-05-17T22:55:14.793 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_bytes": 0, 2024-05-17T22:55:14.793 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_items": 0, 2024-05-17T22:55:14.793 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_bytes": 928, 2024-05-17T22:55:14.793 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_items": 2, 2024-05-17T22:55:14.793 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_bytes": 0, 2024-05-17T22:55:14.793 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_items": 0, 2024-05-17T22:55:14.793 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_bytes": 0, 2024-05-17T22:55:14.793 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_items": 0 2024-05-17T22:55:14.794 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:14.794 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-17T22:55:14.794 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-05-17T22:55:14.794 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 104857600, 2024-05-17T22:55:14.794 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-05-17T22:55:14.794 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 344, 2024-05-17T22:55:14.794 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 500908, 2024-05-17T22:55:14.794 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-05-17T22:55:14.794 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 344, 2024-05-17T22:55:14.794 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 0, 2024-05-17T22:55:14.794 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 0, 2024-05-17T22:55:14.794 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 344, 2024-05-17T22:55:14.794 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 500908, 2024-05-17T22:55:14.795 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-05-17T22:55:14.795 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-17T22:55:14.795 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-17T22:55:14.795 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-17T22:55:14.795 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:14.795 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:14.795 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:55:15.519 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-05-17T22:55:15.519 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-05-17T22:55:15.519 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-05-17T22:55:15.520 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-05-17T22:55:15.520 DEBUG:tasks.check_counter:Getting stats from g 2024-05-17T22:55:15.520 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-17T22:55:15.774 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:55:16.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:16 smithi151 ceph-mon[36101]: pgmap v919: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.3 KiB/s wr, 9 op/s 2024-05-17T22:55:16.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:16 smithi096 ceph-mon[36065]: pgmap v919: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.3 KiB/s wr, 9 op/s 2024-05-17T22:55:16.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:16 smithi012 ceph-mon[31632]: pgmap v919: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.3 KiB/s wr, 9 op/s 2024-05-17T22:55:17.743 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:55:17.743 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-0": { 2024-05-17T22:55:17.743 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 4, 2024-05-17T22:55:17.743 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 4, 2024-05-17T22:55:17.744 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 2992, 2024-05-17T22:55:17.744 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 448, 2024-05-17T22:55:17.744 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 4, 2024-05-17T22:55:17.744 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 0, 2024-05-17T22:55:17.744 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.001922812, 2024-05-17T22:55:17.744 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.000038208, 2024-05-17T22:55:17.744 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.000672533, 2024-05-17T22:55:17.744 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T22:55:17.744 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T22:55:17.745 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 3, 2024-05-17T22:55:17.745 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000825427, 2024-05-17T22:55:17.745 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000275142 2024-05-17T22:55:17.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:17.745 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-05-17T22:55:17.745 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-17T22:55:17.745 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-17T22:55:17.745 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-17T22:55:17.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:17.746 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-17T22:55:17.746 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 448 2024-05-17T22:55:17.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:17.746 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-1": { 2024-05-17T22:55:17.746 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 337, 2024-05-17T22:55:17.746 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 334, 2024-05-17T22:55:17.747 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 547216, 2024-05-17T22:55:17.747 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 206640, 2024-05-17T22:55:17.747 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 2, 2024-05-17T22:55:17.747 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 1, 2024-05-17T22:55:17.747 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.059758964, 2024-05-17T22:55:17.747 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.021526732, 2024-05-17T22:55:17.747 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.050915592, 2024-05-17T22:55:17.747 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T22:55:17.747 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T22:55:17.747 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 333, 2024-05-17T22:55:17.747 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.019749085, 2024-05-17T22:55:17.748 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000059306 2024-05-17T22:55:17.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:17.748 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-05-17T22:55:17.748 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-17T22:55:17.748 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-17T22:55:17.748 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-17T22:55:17.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:17.748 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 547216, 2024-05-17T22:55:17.748 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 206640 2024-05-17T22:55:17.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:17.748 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-2": { 2024-05-17T22:55:17.749 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 2, 2024-05-17T22:55:17.749 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 256, 2024-05-17T22:55:17.749 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 320, 2024-05-17T22:55:17.749 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 202752, 2024-05-17T22:55:17.749 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 4, 2024-05-17T22:55:17.749 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 2, 2024-05-17T22:55:17.749 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.020651307, 2024-05-17T22:55:17.749 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.015065698, 2024-05-17T22:55:17.750 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.002673873, 2024-05-17T22:55:17.750 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T22:55:17.750 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T22:55:17.750 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 254, 2024-05-17T22:55:17.750 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.015833377, 2024-05-17T22:55:17.750 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000062336 2024-05-17T22:55:17.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:17.750 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-05-17T22:55:17.750 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-17T22:55:17.751 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-17T22:55:17.751 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-17T22:55:17.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:17.751 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-17T22:55:17.751 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 202752 2024-05-17T22:55:17.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:17.751 INFO:teuthology.orchestra.run.smithi012.stdout: "cct": { 2024-05-17T22:55:17.751 INFO:teuthology.orchestra.run.smithi012.stdout: "total_workers": 0, 2024-05-17T22:55:17.751 INFO:teuthology.orchestra.run.smithi012.stdout: "unhealthy_workers": 0 2024-05-17T22:55:17.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:17.752 INFO:teuthology.orchestra.run.smithi012.stdout: "mempool": { 2024-05-17T22:55:17.752 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_bytes": 0, 2024-05-17T22:55:17.752 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_items": 0, 2024-05-17T22:55:17.752 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_bytes": 0, 2024-05-17T22:55:17.752 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_items": 0, 2024-05-17T22:55:17.752 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_bytes": 0, 2024-05-17T22:55:17.752 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_items": 0, 2024-05-17T22:55:17.752 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-17T22:55:17.752 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_items": 0, 2024-05-17T22:55:17.753 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-17T22:55:17.753 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_items": 0, 2024-05-17T22:55:17.753 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_bytes": 0, 2024-05-17T22:55:17.753 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_items": 0, 2024-05-17T22:55:17.753 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-17T22:55:17.753 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_items": 0, 2024-05-17T22:55:17.753 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_bytes": 0, 2024-05-17T22:55:17.753 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_items": 0, 2024-05-17T22:55:17.753 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_bytes": 0, 2024-05-17T22:55:17.753 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_items": 0, 2024-05-17T22:55:17.754 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-17T22:55:17.754 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_items": 0, 2024-05-17T22:55:17.754 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-17T22:55:17.754 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_items": 0, 2024-05-17T22:55:17.754 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_bytes": 0, 2024-05-17T22:55:17.754 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_items": 0, 2024-05-17T22:55:17.754 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_bytes": 0, 2024-05-17T22:55:17.755 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_items": 0, 2024-05-17T22:55:17.755 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-17T22:55:17.755 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_items": 0, 2024-05-17T22:55:17.755 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_bytes": 0, 2024-05-17T22:55:17.755 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_items": 0, 2024-05-17T22:55:17.755 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_bytes": 0, 2024-05-17T22:55:17.755 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_items": 0, 2024-05-17T22:55:17.756 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_bytes": 0, 2024-05-17T22:55:17.756 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_items": 0, 2024-05-17T22:55:17.756 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_bytes": 0, 2024-05-17T22:55:17.756 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_items": 0, 2024-05-17T22:55:17.756 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_bytes": 148761, 2024-05-17T22:55:17.756 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_items": 31, 2024-05-17T22:55:17.756 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_bytes": 0, 2024-05-17T22:55:17.756 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_items": 0, 2024-05-17T22:55:17.756 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_bytes": 0, 2024-05-17T22:55:17.756 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_items": 0, 2024-05-17T22:55:17.757 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_bytes": 0, 2024-05-17T22:55:17.757 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_items": 0, 2024-05-17T22:55:17.757 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_bytes": 0, 2024-05-17T22:55:17.757 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_items": 0, 2024-05-17T22:55:17.757 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_bytes": 0, 2024-05-17T22:55:17.757 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_items": 0, 2024-05-17T22:55:17.757 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_bytes": 0, 2024-05-17T22:55:17.757 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_items": 0, 2024-05-17T22:55:17.757 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_bytes": 0, 2024-05-17T22:55:17.757 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_items": 0, 2024-05-17T22:55:17.757 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_bytes": 928, 2024-05-17T22:55:17.757 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_items": 2, 2024-05-17T22:55:17.758 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_bytes": 0, 2024-05-17T22:55:17.758 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_items": 0, 2024-05-17T22:55:17.758 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_bytes": 0, 2024-05-17T22:55:17.758 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_items": 0 2024-05-17T22:55:17.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:17.758 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-17T22:55:17.758 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-05-17T22:55:17.758 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 104857600, 2024-05-17T22:55:17.758 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-05-17T22:55:17.758 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 339, 2024-05-17T22:55:17.758 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 499557, 2024-05-17T22:55:17.758 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-05-17T22:55:17.759 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 339, 2024-05-17T22:55:17.759 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 0, 2024-05-17T22:55:17.759 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 0, 2024-05-17T22:55:17.759 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 339, 2024-05-17T22:55:17.759 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 499557, 2024-05-17T22:55:17.759 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-05-17T22:55:17.759 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-17T22:55:17.759 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-17T22:55:17.759 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-17T22:55:17.759 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:17.759 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:17.759 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:55:18.398 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:18 smithi012 ceph-mon[31632]: pgmap v920: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.9 KiB/s wr, 7 op/s 2024-05-17T22:55:18.427 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-05-17T22:55:18.427 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-05-17T22:55:18.427 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-05-17T22:55:18.427 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-05-17T22:55:18.427 DEBUG:tasks.check_counter:Getting stats from j 2024-05-17T22:55:18.428 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-17T22:55:18.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:18 smithi151 ceph-mon[36101]: pgmap v920: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.9 KiB/s wr, 7 op/s 2024-05-17T22:55:18.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:18 smithi096 ceph-mon[36065]: pgmap v920: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.9 KiB/s wr, 7 op/s 2024-05-17T22:55:18.680 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:55:20.424 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-05-17T22:55:20.424 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-0": { 2024-05-17T22:55:20.424 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 4, 2024-05-17T22:55:20.424 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 4, 2024-05-17T22:55:20.424 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 2992, 2024-05-17T22:55:20.424 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 448, 2024-05-17T22:55:20.424 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 4, 2024-05-17T22:55:20.424 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 0, 2024-05-17T22:55:20.424 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.001649716, 2024-05-17T22:55:20.424 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.000041082, 2024-05-17T22:55:20.425 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.000629616, 2024-05-17T22:55:20.425 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T22:55:20.425 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T22:55:20.425 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 3, 2024-05-17T22:55:20.425 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000785253, 2024-05-17T22:55:20.425 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000261751 2024-05-17T22:55:20.425 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:20.425 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-05-17T22:55:20.425 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-17T22:55:20.425 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-17T22:55:20.425 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-17T22:55:20.426 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:20.426 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-17T22:55:20.426 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 448 2024-05-17T22:55:20.426 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:20.426 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-1": { 2024-05-17T22:55:20.426 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 2, 2024-05-17T22:55:20.426 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 261, 2024-05-17T22:55:20.426 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 320, 2024-05-17T22:55:20.426 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 203632, 2024-05-17T22:55:20.426 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 4, 2024-05-17T22:55:20.426 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 2, 2024-05-17T22:55:20.426 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.021944781, 2024-05-17T22:55:20.426 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.016973819, 2024-05-17T22:55:20.427 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.001955495, 2024-05-17T22:55:20.427 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T22:55:20.427 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T22:55:20.427 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 259, 2024-05-17T22:55:20.427 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.022070956, 2024-05-17T22:55:20.427 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000085216 2024-05-17T22:55:20.427 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:20.427 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-05-17T22:55:20.427 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-17T22:55:20.427 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-17T22:55:20.427 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-17T22:55:20.427 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:20.427 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-17T22:55:20.427 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 203632 2024-05-17T22:55:20.428 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:20.428 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-2": { 2024-05-17T22:55:20.428 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 345, 2024-05-17T22:55:20.428 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 342, 2024-05-17T22:55:20.428 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 550080, 2024-05-17T22:55:20.428 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 210976, 2024-05-17T22:55:20.428 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 2, 2024-05-17T22:55:20.428 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 1, 2024-05-17T22:55:20.428 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.054715068, 2024-05-17T22:55:20.428 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.023613140, 2024-05-17T22:55:20.428 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.036843147, 2024-05-17T22:55:20.428 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T22:55:20.428 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T22:55:20.428 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 341, 2024-05-17T22:55:20.429 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.025169544, 2024-05-17T22:55:20.429 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000073810 2024-05-17T22:55:20.429 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:20.429 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-05-17T22:55:20.429 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-17T22:55:20.429 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-17T22:55:20.429 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-17T22:55:20.429 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:20.429 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 550080, 2024-05-17T22:55:20.429 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 210976 2024-05-17T22:55:20.429 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:20.430 INFO:teuthology.orchestra.run.smithi012.stdout: "cct": { 2024-05-17T22:55:20.430 INFO:teuthology.orchestra.run.smithi012.stdout: "total_workers": 0, 2024-05-17T22:55:20.430 INFO:teuthology.orchestra.run.smithi012.stdout: "unhealthy_workers": 0 2024-05-17T22:55:20.430 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:20.430 INFO:teuthology.orchestra.run.smithi012.stdout: "mempool": { 2024-05-17T22:55:20.430 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_bytes": 0, 2024-05-17T22:55:20.430 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_items": 0, 2024-05-17T22:55:20.430 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_bytes": 0, 2024-05-17T22:55:20.430 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_items": 0, 2024-05-17T22:55:20.430 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_bytes": 0, 2024-05-17T22:55:20.430 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_items": 0, 2024-05-17T22:55:20.430 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-17T22:55:20.430 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_items": 0, 2024-05-17T22:55:20.431 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-17T22:55:20.431 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_items": 0, 2024-05-17T22:55:20.431 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_bytes": 0, 2024-05-17T22:55:20.431 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_items": 0, 2024-05-17T22:55:20.431 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-17T22:55:20.431 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_items": 0, 2024-05-17T22:55:20.431 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_bytes": 0, 2024-05-17T22:55:20.431 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_items": 0, 2024-05-17T22:55:20.431 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_bytes": 0, 2024-05-17T22:55:20.431 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_items": 0, 2024-05-17T22:55:20.431 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-17T22:55:20.431 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_items": 0, 2024-05-17T22:55:20.431 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-17T22:55:20.431 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_items": 0, 2024-05-17T22:55:20.431 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_bytes": 0, 2024-05-17T22:55:20.432 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_items": 0, 2024-05-17T22:55:20.432 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_bytes": 0, 2024-05-17T22:55:20.432 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_items": 0, 2024-05-17T22:55:20.432 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-17T22:55:20.432 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_items": 0, 2024-05-17T22:55:20.432 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_bytes": 0, 2024-05-17T22:55:20.432 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_items": 0, 2024-05-17T22:55:20.432 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_bytes": 0, 2024-05-17T22:55:20.432 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_items": 0, 2024-05-17T22:55:20.432 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_bytes": 0, 2024-05-17T22:55:20.432 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_items": 0, 2024-05-17T22:55:20.432 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_bytes": 0, 2024-05-17T22:55:20.432 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_items": 0, 2024-05-17T22:55:20.433 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_bytes": 148704, 2024-05-17T22:55:20.433 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_items": 29, 2024-05-17T22:55:20.433 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_bytes": 0, 2024-05-17T22:55:20.433 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_items": 0, 2024-05-17T22:55:20.433 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_bytes": 0, 2024-05-17T22:55:20.433 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_items": 0, 2024-05-17T22:55:20.433 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_bytes": 0, 2024-05-17T22:55:20.433 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_items": 0, 2024-05-17T22:55:20.433 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_bytes": 0, 2024-05-17T22:55:20.433 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_items": 0, 2024-05-17T22:55:20.434 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_bytes": 0, 2024-05-17T22:55:20.434 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_items": 0, 2024-05-17T22:55:20.434 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_bytes": 0, 2024-05-17T22:55:20.434 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_items": 0, 2024-05-17T22:55:20.434 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_bytes": 0, 2024-05-17T22:55:20.434 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_items": 0, 2024-05-17T22:55:20.434 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_bytes": 928, 2024-05-17T22:55:20.434 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_items": 2, 2024-05-17T22:55:20.434 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_bytes": 0, 2024-05-17T22:55:20.434 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_items": 0, 2024-05-17T22:55:20.434 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_bytes": 0, 2024-05-17T22:55:20.434 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_items": 0 2024-05-17T22:55:20.435 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-05-17T22:55:20.435 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-17T22:55:20.435 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-05-17T22:55:20.435 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 104857600, 2024-05-17T22:55:20.435 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-05-17T22:55:20.435 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 347, 2024-05-17T22:55:20.435 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 501298, 2024-05-17T22:55:20.435 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-05-17T22:55:20.435 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 347, 2024-05-17T22:55:20.435 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 0, 2024-05-17T22:55:20.435 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 0, 2024-05-17T22:55:20.436 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 347, 2024-05-17T22:55:20.436 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 501298, 2024-05-17T22:55:20.436 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-05-17T22:55:20.436 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-05-17T22:55:20.436 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-05-17T22:55:20.436 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-05-17T22:55:20.436 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:20.436 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-05-17T22:55:20.436 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-05-17T22:55:20.436 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:20 smithi012 ceph-mon[31632]: pgmap v921: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-05-17T22:55:20.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:20 smithi151 ceph-mon[36101]: pgmap v921: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-05-17T22:55:20.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:20 smithi096 ceph-mon[36065]: pgmap v921: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-05-17T22:55:21.235 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-17T22:55:21.235 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-17T22:55:21.235 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-05-17T22:55:21.235 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-05-17T22:55:21.235 DEBUG:tasks.check_counter:Getting stats from b 2024-05-17T22:55:21.236 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-17T22:55:21.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:21 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:55:21.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:21 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:55:21.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:21 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T22:55:21.577 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.b/config 2024-05-17T22:55:22.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:22 smithi151 ceph-mon[36101]: pgmap v922: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-05-17T22:55:22.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:22 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:55:22.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:22 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:55:22.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:22 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:55:22.537 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:22 smithi096 ceph-mon[36065]: pgmap v922: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-05-17T22:55:22.537 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:22 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:55:22.537 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:22 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:55:22.537 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:22 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:55:22.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:22 smithi012 ceph-mon[31632]: pgmap v922: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-05-17T22:55:22.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:22 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T22:55:22.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:22 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T22:55:22.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:22 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' 2024-05-17T22:55:23.547 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-05-17T22:55:23.547 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-05-17T22:55:23.547 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 5999, 2024-05-17T22:55:23.547 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 6914, 2024-05-17T22:55:23.547 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 1759528, 2024-05-17T22:55:23.547 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 11895848, 2024-05-17T22:55:23.547 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 129, 2024-05-17T22:55:23.547 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 5, 2024-05-17T22:55:23.547 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.543718379, 2024-05-17T22:55:23.547 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.212855234, 2024-05-17T22:55:23.548 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.263142948, 2024-05-17T22:55:23.548 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.061968059, 2024-05-17T22:55:23.548 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T22:55:23.548 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 6916, 2024-05-17T22:55:23.548 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.299408503, 2024-05-17T22:55:23.548 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000043292 2024-05-17T22:55:23.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.548 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-05-17T22:55:23.548 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 7347, 2024-05-17T22:55:23.549 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.001607383, 2024-05-17T22:55:23.549 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000218 2024-05-17T22:55:23.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.549 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-17T22:55:23.549 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 448 2024-05-17T22:55:23.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.549 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-05-17T22:55:23.549 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 5113, 2024-05-17T22:55:23.549 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 6164, 2024-05-17T22:55:23.549 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 2038182, 2024-05-17T22:55:23.549 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 5656983, 2024-05-17T22:55:23.550 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 7, 2024-05-17T22:55:23.550 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 5, 2024-05-17T22:55:23.550 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.389921272, 2024-05-17T22:55:23.550 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.140378553, 2024-05-17T22:55:23.550 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.209275770, 2024-05-17T22:55:23.550 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.056157840, 2024-05-17T22:55:23.550 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T22:55:23.550 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 6166, 2024-05-17T22:55:23.550 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.529766098, 2024-05-17T22:55:23.550 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000085917 2024-05-17T22:55:23.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.550 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-05-17T22:55:23.550 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 5557, 2024-05-17T22:55:23.551 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.001231278, 2024-05-17T22:55:23.551 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000221 2024-05-17T22:55:23.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.551 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 689424, 2024-05-17T22:55:23.551 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 234672 2024-05-17T22:55:23.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.551 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-05-17T22:55:23.551 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 8710, 2024-05-17T22:55:23.551 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 11158, 2024-05-17T22:55:23.551 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 2559240, 2024-05-17T22:55:23.551 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 6050288, 2024-05-17T22:55:23.551 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 18, 2024-05-17T22:55:23.551 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 8, 2024-05-17T22:55:23.552 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.569771392, 2024-05-17T22:55:23.552 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.203184916, 2024-05-17T22:55:23.552 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.288765483, 2024-05-17T22:55:23.552 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.110696739, 2024-05-17T22:55:23.552 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T22:55:23.552 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 11150, 2024-05-17T22:55:23.552 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.618553803, 2024-05-17T22:55:23.552 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000055475 2024-05-17T22:55:23.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.552 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-05-17T22:55:23.552 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 8221, 2024-05-17T22:55:23.552 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.002021223, 2024-05-17T22:55:23.552 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000245 2024-05-17T22:55:23.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.553 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 52288, 2024-05-17T22:55:23.553 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 568368 2024-05-17T22:55:23.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.553 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-05-17T22:55:23.553 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 1, 2024-05-17T22:55:23.553 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-05-17T22:55:23.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.553 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-MDSRank": { 2024-05-17T22:55:23.553 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-05-17T22:55:23.553 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-05-17T22:55:23.553 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 6934, 2024-05-17T22:55:23.553 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 3.199963873, 2024-05-17T22:55:23.553 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000461488 2024-05-17T22:55:23.553 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-17T22:55:23.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.554 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-PurgeQueue": { 2024-05-17T22:55:23.554 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-05-17T22:55:23.554 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-05-17T22:55:23.554 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 115, 2024-05-17T22:55:23.554 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.004680584, 2024-05-17T22:55:23.554 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000040700 2024-05-17T22:55:23.554 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-17T22:55:23.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.554 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": { 2024-05-17T22:55:23.554 INFO:teuthology.orchestra.run.smithi096.stdout: "request": 2992, 2024-05-17T22:55:23.554 INFO:teuthology.orchestra.run.smithi096.stdout: "reply": 2933, 2024-05-17T22:55:23.554 INFO:teuthology.orchestra.run.smithi096.stdout: "reply_latency": { 2024-05-17T22:55:23.555 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 2933, 2024-05-17T22:55:23.555 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 6.261910779, 2024-05-17T22:55:23.555 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.002134984 2024-05-17T22:55:23.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.555 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_reply": 0, 2024-05-17T22:55:23.555 INFO:teuthology.orchestra.run.smithi096.stdout: "forward": 59, 2024-05-17T22:55:23.555 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_complete": 0, 2024-05-17T22:55:23.555 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_keys": 0, 2024-05-17T22:55:23.555 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_commit": 39, 2024-05-17T22:55:23.555 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_split": 3, 2024-05-17T22:55:23.555 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_merge": 0, 2024-05-17T22:55:23.555 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes": 389, 2024-05-17T22:55:23.555 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_top": 2, 2024-05-17T22:55:23.555 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_bottom": 299, 2024-05-17T22:55:23.556 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pin_tail": 88, 2024-05-17T22:55:23.556 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pinned": 386, 2024-05-17T22:55:23.556 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_expired": 1159, 2024-05-17T22:55:23.556 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_with_caps": 102, 2024-05-17T22:55:23.556 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": 102, 2024-05-17T22:55:23.556 INFO:teuthology.orchestra.run.smithi096.stdout: "subtrees": 10, 2024-05-17T22:55:23.556 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse": 5740, 2024-05-17T22:55:23.556 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_hit": 4496, 2024-05-17T22:55:23.556 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_forward": 59, 2024-05-17T22:55:23.556 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_discover": 2, 2024-05-17T22:55:23.556 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_dir_fetch": 0, 2024-05-17T22:55:23.556 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_remote_ino": 0, 2024-05-17T22:55:23.556 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_lock": 260, 2024-05-17T22:55:23.556 INFO:teuthology.orchestra.run.smithi096.stdout: "load_cent": 235, 2024-05-17T22:55:23.557 INFO:teuthology.orchestra.run.smithi096.stdout: "q": 0, 2024-05-17T22:55:23.557 INFO:teuthology.orchestra.run.smithi096.stdout: "exported": 23, 2024-05-17T22:55:23.557 INFO:teuthology.orchestra.run.smithi096.stdout: "exported_inodes": 1142, 2024-05-17T22:55:23.557 INFO:teuthology.orchestra.run.smithi096.stdout: "imported": 12, 2024-05-17T22:55:23.557 INFO:teuthology.orchestra.run.smithi096.stdout: "imported_inodes": 188, 2024-05-17T22:55:23.557 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_dir_fetch": 0, 2024-05-17T22:55:23.557 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_backtrace_fetch": 4684, 2024-05-17T22:55:23.557 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_peer_discover": 0, 2024-05-17T22:55:23.557 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rfiles": 1022, 2024-05-17T22:55:23.557 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rbytes": 1605948972, 2024-05-17T22:55:23.557 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rsnaps": 0, 2024-05-17T22:55:23.557 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_fetch": 4684, 2024-05-17T22:55:23.557 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_set_tag": 0, 2024-05-17T22:55:23.558 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_repaired": 0, 2024-05-17T22:55:23.558 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_inotable_repaired": 0, 2024-05-17T22:55:23.558 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_inodes": 451, 2024-05-17T22:55:23.558 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_base_inodes": 15, 2024-05-17T22:55:23.558 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dirfrag_rstats": 436, 2024-05-17T22:55:23.558 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_file_inodes": 4233, 2024-05-17T22:55:23.558 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_inode_file_caps": 15, 2024-05-17T22:55:23.558 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_revoke": 286, 2024-05-17T22:55:23.558 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_grant": 1837, 2024-05-17T22:55:23.558 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_trunc": 0, 2024-05-17T22:55:23.558 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-17T22:55:23.558 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-17T22:55:23.558 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps": 2018, 2024-05-17T22:55:23.559 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps_dirty": 1479, 2024-05-17T22:55:23.559 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_cap_release": 61, 2024-05-17T22:55:23.559 INFO:teuthology.orchestra.run.smithi096.stdout: "process_request_cap_release": 737 2024-05-17T22:55:23.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.559 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache": { 2024-05-17T22:55:23.559 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update": 91, 2024-05-17T22:55:23.559 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update_receipt": 34, 2024-05-17T22:55:23.559 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_try_discover": 18, 2024-05-17T22:55:23.559 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_send_discover": 18, 2024-05-17T22:55:23.559 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_handle_discover": 195, 2024-05-17T22:55:23.559 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays": 0, 2024-05-17T22:55:23.559 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_delayed": 0, 2024-05-17T22:55:23.559 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_enqueuing": 0, 2024-05-17T22:55:23.559 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_created": 181, 2024-05-17T22:55:23.560 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_enqueued": 28, 2024-05-17T22:55:23.560 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_reintegrated": 153, 2024-05-17T22:55:23.560 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_migrated": 0, 2024-05-17T22:55:23.560 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_processing": 0, 2024-05-17T22:55:23.560 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_enqueued": 0, 2024-05-17T22:55:23.560 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_prioritized": 0, 2024-05-17T22:55:23.560 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_started": 0, 2024-05-17T22:55:23.560 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_completed": 0, 2024-05-17T22:55:23.560 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_path": 0, 2024-05-17T22:55:23.560 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_inode": 0, 2024-05-17T22:55:23.560 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_enqueue_scrub": 15, 2024-05-17T22:55:23.560 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_exportdir": 23, 2024-05-17T22:55:23.560 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_flush": 0, 2024-05-17T22:55:23.561 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragmentdir": 3, 2024-05-17T22:55:23.561 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragstats": 0, 2024-05-17T22:55:23.561 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_inodestats": 0 2024-05-17T22:55:23.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.561 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log": { 2024-05-17T22:55:23.561 INFO:teuthology.orchestra.run.smithi096.stdout: "evlrg": 0, 2024-05-17T22:55:23.561 INFO:teuthology.orchestra.run.smithi096.stdout: "evadd": 4213, 2024-05-17T22:55:23.561 INFO:teuthology.orchestra.run.smithi096.stdout: "evex": 0, 2024-05-17T22:55:23.561 INFO:teuthology.orchestra.run.smithi096.stdout: "evtrm": 0, 2024-05-17T22:55:23.561 INFO:teuthology.orchestra.run.smithi096.stdout: "ev": 4213, 2024-05-17T22:55:23.561 INFO:teuthology.orchestra.run.smithi096.stdout: "evexg": 0, 2024-05-17T22:55:23.561 INFO:teuthology.orchestra.run.smithi096.stdout: "evexd": 0, 2024-05-17T22:55:23.561 INFO:teuthology.orchestra.run.smithi096.stdout: "segadd": 6, 2024-05-17T22:55:23.561 INFO:teuthology.orchestra.run.smithi096.stdout: "segex": 0, 2024-05-17T22:55:23.562 INFO:teuthology.orchestra.run.smithi096.stdout: "segtrm": 0, 2024-05-17T22:55:23.562 INFO:teuthology.orchestra.run.smithi096.stdout: "seg": 6, 2024-05-17T22:55:23.562 INFO:teuthology.orchestra.run.smithi096.stdout: "segmjr": 2, 2024-05-17T22:55:23.562 INFO:teuthology.orchestra.run.smithi096.stdout: "segexg": 0, 2024-05-17T22:55:23.562 INFO:teuthology.orchestra.run.smithi096.stdout: "segexd": 0, 2024-05-17T22:55:23.562 INFO:teuthology.orchestra.run.smithi096.stdout: "expos": 4194304, 2024-05-17T22:55:23.562 INFO:teuthology.orchestra.run.smithi096.stdout: "wrpos": 15610515, 2024-05-17T22:55:23.562 INFO:teuthology.orchestra.run.smithi096.stdout: "rdpos": 0, 2024-05-17T22:55:23.562 INFO:teuthology.orchestra.run.smithi096.stdout: "jlat": { 2024-05-17T22:55:23.562 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1587, 2024-05-17T22:55:23.562 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 9.991366050, 2024-05-17T22:55:23.562 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.006295756 2024-05-17T22:55:23.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.562 INFO:teuthology.orchestra.run.smithi096.stdout: "replayed": 0 2024-05-17T22:55:23.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.563 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_mem": { 2024-05-17T22:55:23.563 INFO:teuthology.orchestra.run.smithi096.stdout: "ino": 128, 2024-05-17T22:55:23.563 INFO:teuthology.orchestra.run.smithi096.stdout: "ino+": 1284, 2024-05-17T22:55:23.563 INFO:teuthology.orchestra.run.smithi096.stdout: "ino-": 1156, 2024-05-17T22:55:23.563 INFO:teuthology.orchestra.run.smithi096.stdout: "dir": 62, 2024-05-17T22:55:23.563 INFO:teuthology.orchestra.run.smithi096.stdout: "dir+": 98, 2024-05-17T22:55:23.563 INFO:teuthology.orchestra.run.smithi096.stdout: "dir-": 36, 2024-05-17T22:55:23.563 INFO:teuthology.orchestra.run.smithi096.stdout: "dn": 389, 2024-05-17T22:55:23.563 INFO:teuthology.orchestra.run.smithi096.stdout: "dn+": 1830, 2024-05-17T22:55:23.563 INFO:teuthology.orchestra.run.smithi096.stdout: "dn-": 1441, 2024-05-17T22:55:23.563 INFO:teuthology.orchestra.run.smithi096.stdout: "cap": 102, 2024-05-17T22:55:23.563 INFO:teuthology.orchestra.run.smithi096.stdout: "cap+": 1244, 2024-05-17T22:55:23.563 INFO:teuthology.orchestra.run.smithi096.stdout: "cap-": 1142, 2024-05-17T22:55:23.564 INFO:teuthology.orchestra.run.smithi096.stdout: "rss": 140316, 2024-05-17T22:55:23.564 INFO:teuthology.orchestra.run.smithi096.stdout: "heap": 231824 2024-05-17T22:55:23.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.564 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_server": { 2024-05-17T22:55:23.564 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_client_request": 4189, 2024-05-17T22:55:23.564 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_server_request": 470, 2024-05-17T22:55:23.564 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_request": 2992, 2024-05-17T22:55:23.564 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_session": 355, 2024-05-17T22:55:23.564 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_peer_request": 952, 2024-05-17T22:55:23.564 INFO:teuthology.orchestra.run.smithi096.stdout: "req_create_latency": { 2024-05-17T22:55:23.564 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1052, 2024-05-17T22:55:23.564 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 2.207224140, 2024-05-17T22:55:23.564 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.002098121 2024-05-17T22:55:23.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.564 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getattr_latency": { 2024-05-17T22:55:23.565 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 38, 2024-05-17T22:55:23.565 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.028628840, 2024-05-17T22:55:23.565 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000753390 2024-05-17T22:55:23.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.565 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getfilelock_latency": { 2024-05-17T22:55:23.565 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:23.565 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:23.565 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:23.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.565 INFO:teuthology.orchestra.run.smithi096.stdout: "req_link_latency": { 2024-05-17T22:55:23.565 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 153, 2024-05-17T22:55:23.565 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.449519366, 2024-05-17T22:55:23.565 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.002938035 2024-05-17T22:55:23.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.566 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookup_latency": { 2024-05-17T22:55:23.566 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 805, 2024-05-17T22:55:23.566 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.943104152, 2024-05-17T22:55:23.566 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.001171557 2024-05-17T22:55:23.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.566 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookuphash_latency": { 2024-05-17T22:55:23.566 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:23.566 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:23.566 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:23.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.566 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupino_latency": { 2024-05-17T22:55:23.566 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:23.566 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:23.567 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:23.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.567 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupname_latency": { 2024-05-17T22:55:23.567 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:23.567 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:23.567 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:23.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.567 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupparent_latency": { 2024-05-17T22:55:23.567 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:23.567 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:23.567 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:23.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.567 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupsnap_latency": { 2024-05-17T22:55:23.567 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:23.567 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:23.568 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:23.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.568 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lssnap_latency": { 2024-05-17T22:55:23.568 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:23.568 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:23.568 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:23.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.569 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mkdir_latency": { 2024-05-17T22:55:23.569 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 37, 2024-05-17T22:55:23.569 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.142881325, 2024-05-17T22:55:23.569 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.003861657 2024-05-17T22:55:23.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.569 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mknod_latency": { 2024-05-17T22:55:23.569 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:23.569 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:23.569 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:23.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.569 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mksnap_latency": { 2024-05-17T22:55:23.569 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:23.569 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:23.569 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:23.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.570 INFO:teuthology.orchestra.run.smithi096.stdout: "req_open_latency": { 2024-05-17T22:55:23.570 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 437, 2024-05-17T22:55:23.570 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.133469957, 2024-05-17T22:55:23.570 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000305423 2024-05-17T22:55:23.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.570 INFO:teuthology.orchestra.run.smithi096.stdout: "req_readdir_latency": { 2024-05-17T22:55:23.570 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 53, 2024-05-17T22:55:23.570 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.107412498, 2024-05-17T22:55:23.570 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.002026650 2024-05-17T22:55:23.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.570 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rename_latency": { 2024-05-17T22:55:23.570 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 181, 2024-05-17T22:55:23.571 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 1.777725866, 2024-05-17T22:55:23.571 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.009821689 2024-05-17T22:55:23.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.571 INFO:teuthology.orchestra.run.smithi096.stdout: "req_renamesnap_latency": { 2024-05-17T22:55:23.571 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:23.571 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:23.571 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:23.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.571 INFO:teuthology.orchestra.run.smithi096.stdout: "req_snapdiff_latency": { 2024-05-17T22:55:23.571 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:23.571 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:23.571 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:23.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.572 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmdir_latency": { 2024-05-17T22:55:23.572 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 2, 2024-05-17T22:55:23.572 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.024745643, 2024-05-17T22:55:23.572 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.012372821 2024-05-17T22:55:23.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.572 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmsnap_latency": { 2024-05-17T22:55:23.572 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:23.572 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:23.572 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:23.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.572 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmxattr_latency": { 2024-05-17T22:55:23.572 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:23.572 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:23.573 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:23.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.573 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setattr_latency": { 2024-05-17T22:55:23.573 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 2, 2024-05-17T22:55:23.573 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.001431138, 2024-05-17T22:55:23.573 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000715569 2024-05-17T22:55:23.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.573 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setdirlayout_latency": { 2024-05-17T22:55:23.573 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:23.573 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:23.573 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:23.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.573 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setfilelock_latency": { 2024-05-17T22:55:23.574 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:23.574 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:23.574 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:23.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.574 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setlayout_latency": { 2024-05-17T22:55:23.574 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:23.574 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:23.574 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:23.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.574 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setxattr_latency": { 2024-05-17T22:55:23.574 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 8, 2024-05-17T22:55:23.574 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.022329164, 2024-05-17T22:55:23.574 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.002791145 2024-05-17T22:55:23.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.575 INFO:teuthology.orchestra.run.smithi096.stdout: "req_symlink_latency": { 2024-05-17T22:55:23.575 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:23.575 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:23.575 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:23.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.575 INFO:teuthology.orchestra.run.smithi096.stdout: "req_unlink_latency": { 2024-05-17T22:55:23.575 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 165, 2024-05-17T22:55:23.575 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.423438690, 2024-05-17T22:55:23.575 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.002566295 2024-05-17T22:55:23.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.575 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_revoke_eviction": 0, 2024-05-17T22:55:23.575 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_acquisition_throttle": 0, 2024-05-17T22:55:23.575 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getvxattr_latency": { 2024-05-17T22:55:23.576 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:23.576 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:23.576 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:23.576 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-17T22:55:23.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.576 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_sessions": { 2024-05-17T22:55:23.576 INFO:teuthology.orchestra.run.smithi096.stdout: "session_count": 1, 2024-05-17T22:55:23.576 INFO:teuthology.orchestra.run.smithi096.stdout: "session_add": 3, 2024-05-17T22:55:23.576 INFO:teuthology.orchestra.run.smithi096.stdout: "session_remove": 2, 2024-05-17T22:55:23.576 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_open": 1, 2024-05-17T22:55:23.576 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_stale": 0, 2024-05-17T22:55:23.576 INFO:teuthology.orchestra.run.smithi096.stdout: "total_load": 367, 2024-05-17T22:55:23.576 INFO:teuthology.orchestra.run.smithi096.stdout: "average_load": 367, 2024-05-17T22:55:23.576 INFO:teuthology.orchestra.run.smithi096.stdout: "avg_session_uptime": 987, 2024-05-17T22:55:23.577 INFO:teuthology.orchestra.run.smithi096.stdout: "mdthresh_evicted": 0 2024-05-17T22:55:23.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.577 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-05-17T22:55:23.577 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 0, 2024-05-17T22:55:23.577 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 0, 2024-05-17T22:55:23.577 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-05-17T22:55:23.577 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-05-17T22:55:23.577 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-05-17T22:55:23.577 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-05-17T22:55:23.577 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-17T22:55:23.577 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-05-17T22:55:23.577 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-17T22:55:23.577 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-05-17T22:55:23.577 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-05-17T22:55:23.578 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-05-17T22:55:23.578 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-17T22:55:23.578 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-05-17T22:55:23.578 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-05-17T22:55:23.578 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-05-17T22:55:23.578 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-05-17T22:55:23.578 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-05-17T22:55:23.578 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-17T22:55:23.578 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-05-17T22:55:23.578 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-17T22:55:23.578 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-05-17T22:55:23.578 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-05-17T22:55:23.578 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-05-17T22:55:23.578 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-05-17T22:55:23.579 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-05-17T22:55:23.579 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-17T22:55:23.579 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-05-17T22:55:23.579 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-05-17T22:55:23.579 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-05-17T22:55:23.579 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-05-17T22:55:23.579 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-05-17T22:55:23.579 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-05-17T22:55:23.579 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-05-17T22:55:23.579 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-05-17T22:55:23.579 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-05-17T22:55:23.579 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 290498, 2024-05-17T22:55:23.579 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 200, 2024-05-17T22:55:23.579 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 0, 2024-05-17T22:55:23.580 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 0, 2024-05-17T22:55:23.580 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-05-17T22:55:23.580 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-05-17T22:55:23.580 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-05-17T22:55:23.580 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-05-17T22:55:23.580 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-05-17T22:55:23.580 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-05-17T22:55:23.580 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 8888, 2024-05-17T22:55:23.580 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 170, 2024-05-17T22:55:23.580 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-05-17T22:55:23.580 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-05-17T22:55:23.580 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-05-17T22:55:23.580 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-05-17T22:55:23.580 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 737118, 2024-05-17T22:55:23.581 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 18808, 2024-05-17T22:55:23.581 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-05-17T22:55:23.581 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-05-17T22:55:23.581 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-05-17T22:55:23.581 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-05-17T22:55:23.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.581 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter": { 2024-05-17T22:55:23.581 INFO:teuthology.orchestra.run.smithi096.stdout: "op_active": 0, 2024-05-17T22:55:23.581 INFO:teuthology.orchestra.run.smithi096.stdout: "op_laggy": 0, 2024-05-17T22:55:23.581 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send": 7215, 2024-05-17T22:55:23.581 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send_bytes": 11926687, 2024-05-17T22:55:23.581 INFO:teuthology.orchestra.run.smithi096.stdout: "op_resend": 37, 2024-05-17T22:55:23.581 INFO:teuthology.orchestra.run.smithi096.stdout: "op_reply": 7215, 2024-05-17T22:55:23.581 INFO:teuthology.orchestra.run.smithi096.stdout: "op_latency": { 2024-05-17T22:55:23.582 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 7215, 2024-05-17T22:55:23.582 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 43.071049774, 2024-05-17T22:55:23.582 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.005969653 2024-05-17T22:55:23.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.582 INFO:teuthology.orchestra.run.smithi096.stdout: "op_inflight": 0, 2024-05-17T22:55:23.582 INFO:teuthology.orchestra.run.smithi096.stdout: "oplen_avg": { 2024-05-17T22:55:23.582 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 7215, 2024-05-17T22:55:23.582 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 7462 2024-05-17T22:55:23.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.582 INFO:teuthology.orchestra.run.smithi096.stdout: "op": 7215, 2024-05-17T22:55:23.582 INFO:teuthology.orchestra.run.smithi096.stdout: "op_r": 4724, 2024-05-17T22:55:23.582 INFO:teuthology.orchestra.run.smithi096.stdout: "op_w": 2491, 2024-05-17T22:55:23.582 INFO:teuthology.orchestra.run.smithi096.stdout: "op_rmw": 0, 2024-05-17T22:55:23.583 INFO:teuthology.orchestra.run.smithi096.stdout: "op_pg": 0, 2024-05-17T22:55:23.583 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_stat": 0, 2024-05-17T22:55:23.583 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_create": 11, 2024-05-17T22:55:23.583 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_read": 25, 2024-05-17T22:55:23.583 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_write": 1597, 2024-05-17T22:55:23.583 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writefull": 79, 2024-05-17T22:55:23.583 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writesame": 0, 2024-05-17T22:55:23.583 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_append": 0, 2024-05-17T22:55:23.583 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_zero": 0, 2024-05-17T22:55:23.583 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_truncate": 3, 2024-05-17T22:55:23.583 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_delete": 585, 2024-05-17T22:55:23.583 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_mapext": 0, 2024-05-17T22:55:23.583 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_sparse_read": 0, 2024-05-17T22:55:23.583 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_clonerange": 0, 2024-05-17T22:55:23.584 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_getxattr": 4699, 2024-05-17T22:55:23.584 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_setxattr": 22, 2024-05-17T22:55:23.584 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_cmpxattr": 0, 2024-05-17T22:55:23.584 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_rmxattr": 0, 2024-05-17T22:55:23.584 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_resetxattrs": 0, 2024-05-17T22:55:23.584 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_call": 0, 2024-05-17T22:55:23.584 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_watch": 0, 2024-05-17T22:55:23.584 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_notify": 0, 2024-05-17T22:55:23.584 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_src_cmpxattr": 0, 2024-05-17T22:55:23.584 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls": 0, 2024-05-17T22:55:23.584 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls_filter": 0, 2024-05-17T22:55:23.584 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_other": 0, 2024-05-17T22:55:23.584 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_active": 0, 2024-05-17T22:55:23.584 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_send": 0, 2024-05-17T22:55:23.584 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_resend": 0, 2024-05-17T22:55:23.585 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_ping": 0, 2024-05-17T22:55:23.585 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_active": 0, 2024-05-17T22:55:23.585 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_send": 0, 2024-05-17T22:55:23.585 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_resend": 0, 2024-05-17T22:55:23.585 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_active": 0, 2024-05-17T22:55:23.585 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_send": 0, 2024-05-17T22:55:23.585 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_resend": 0, 2024-05-17T22:55:23.585 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_active": 0, 2024-05-17T22:55:23.585 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_send": 0, 2024-05-17T22:55:23.585 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_resend": 0, 2024-05-17T22:55:23.585 INFO:teuthology.orchestra.run.smithi096.stdout: "command_active": 0, 2024-05-17T22:55:23.585 INFO:teuthology.orchestra.run.smithi096.stdout: "command_send": 0, 2024-05-17T22:55:23.585 INFO:teuthology.orchestra.run.smithi096.stdout: "command_resend": 0, 2024-05-17T22:55:23.586 INFO:teuthology.orchestra.run.smithi096.stdout: "map_epoch": 89, 2024-05-17T22:55:23.586 INFO:teuthology.orchestra.run.smithi096.stdout: "map_full": 0, 2024-05-17T22:55:23.586 INFO:teuthology.orchestra.run.smithi096.stdout: "map_inc": 11, 2024-05-17T22:55:23.586 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_sessions": 12, 2024-05-17T22:55:23.586 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_open": 12, 2024-05-17T22:55:23.586 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_close": 0, 2024-05-17T22:55:23.586 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_laggy": 0, 2024-05-17T22:55:23.586 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_wr": 353, 2024-05-17T22:55:23.586 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_rd": 0, 2024-05-17T22:55:23.586 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_del": 88 2024-05-17T22:55:23.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.586 INFO:teuthology.orchestra.run.smithi096.stdout: "oft": { 2024-05-17T22:55:23.586 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_objs": 1, 2024-05-17T22:55:23.587 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_kv_pairs": 0, 2024-05-17T22:55:23.587 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_updates": 0, 2024-05-17T22:55:23.587 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_removes": 0 2024-05-17T22:55:23.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.587 INFO:teuthology.orchestra.run.smithi096.stdout: "osd-slow-ops": { 2024-05-17T22:55:23.587 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_ops_count": 0 2024-05-17T22:55:23.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.587 INFO:teuthology.orchestra.run.smithi096.stdout: "purge_queue": { 2024-05-17T22:55:23.587 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops": 0, 2024-05-17T22:55:23.587 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops_high_water": 18, 2024-05-17T22:55:23.587 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing": 0, 2024-05-17T22:55:23.587 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_high_water": 8, 2024-05-17T22:55:23.587 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed_ops": 71, 2024-05-17T22:55:23.587 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed": 28, 2024-05-17T22:55:23.587 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_item_in_journal": 0 2024-05-17T22:55:23.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.588 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-17T22:55:23.588 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-17T22:55:23.588 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-05-17T22:55:23.588 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-17T22:55:23.588 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 19818, 2024-05-17T22:55:23.588 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 4641574, 2024-05-17T22:55:23.588 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-17T22:55:23.588 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 19818, 2024-05-17T22:55:23.588 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-05-17T22:55:23.588 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-05-17T22:55:23.588 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 19818, 2024-05-17T22:55:23.588 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 4641574, 2024-05-17T22:55:23.588 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-17T22:55:23.589 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:23.589 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:23.589 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:23.589 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-17T22:55:23.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.589 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_bytes": { 2024-05-17T22:55:23.589 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-17T22:55:23.589 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-05-17T22:55:23.589 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-17T22:55:23.589 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-05-17T22:55:23.589 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-05-17T22:55:23.589 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-17T22:55:23.589 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-05-17T22:55:23.589 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 7215, 2024-05-17T22:55:23.590 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 11929723, 2024-05-17T22:55:23.590 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 7215, 2024-05-17T22:55:23.590 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 11929723, 2024-05-17T22:55:23.590 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-17T22:55:23.590 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:23.590 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:23.590 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:23.590 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-17T22:55:23.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.590 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_ops": { 2024-05-17T22:55:23.590 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-17T22:55:23.590 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 1024, 2024-05-17T22:55:23.590 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-17T22:55:23.590 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-05-17T22:55:23.591 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-05-17T22:55:23.591 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-17T22:55:23.591 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-05-17T22:55:23.591 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 7215, 2024-05-17T22:55:23.591 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 7215, 2024-05-17T22:55:23.591 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 7215, 2024-05-17T22:55:23.591 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 7215, 2024-05-17T22:55:23.591 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-17T22:55:23.591 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:23.591 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:23.591 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:23.591 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-17T22:55:23.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.592 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle": { 2024-05-17T22:55:23.592 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-17T22:55:23.592 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-05-17T22:55:23.592 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-17T22:55:23.592 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 28, 2024-05-17T22:55:23.592 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 3036, 2024-05-17T22:55:23.592 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-17T22:55:23.592 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 28, 2024-05-17T22:55:23.592 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-05-17T22:55:23.592 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-05-17T22:55:23.592 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 10, 2024-05-17T22:55:23.592 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 3036, 2024-05-17T22:55:23.592 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-17T22:55:23.592 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:23.593 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:23.593 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:23.593 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-17T22:55:23.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:23.593 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle-0x55e9845e2820": { 2024-05-17T22:55:23.593 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-17T22:55:23.593 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-05-17T22:55:23.593 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-17T22:55:23.593 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 4213, 2024-05-17T22:55:23.593 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 11416211, 2024-05-17T22:55:23.593 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-17T22:55:23.593 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 4213, 2024-05-17T22:55:23.593 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-05-17T22:55:23.593 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-05-17T22:55:23.593 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 1587, 2024-05-17T22:55:23.594 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 11416211, 2024-05-17T22:55:23.594 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-17T22:55:23.594 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:23.594 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:23.594 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:23.594 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-17T22:55:23.594 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-17T22:55:23.594 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-05-17T22:55:24.277 INFO:tasks.check_counter:Daemon mds.b mds.exported=23 2024-05-17T22:55:24.277 INFO:tasks.check_counter:Daemon mds.b mds.imported=12 2024-05-17T22:55:24.277 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=91 2024-05-17T22:55:24.277 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=34 2024-05-17T22:55:24.277 DEBUG:tasks.check_counter:Getting stats from e 2024-05-17T22:55:24.278 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-17T22:55:24.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:24 smithi151 ceph-mon[36101]: pgmap v923: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-05-17T22:55:24.523 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.b/config 2024-05-17T22:55:24.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:24 smithi096 ceph-mon[36065]: pgmap v923: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-05-17T22:55:24.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:24 smithi012 ceph-mon[31632]: pgmap v923: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-05-17T22:55:26.398 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:26 smithi096 ceph-mon[36065]: pgmap v924: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-05-17T22:55:26.461 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-05-17T22:55:26.461 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-05-17T22:55:26.461 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 3590, 2024-05-17T22:55:26.461 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 3592, 2024-05-17T22:55:26.461 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 3629442, 2024-05-17T22:55:26.461 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 1141627, 2024-05-17T22:55:26.461 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 6, 2024-05-17T22:55:26.461 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 2, 2024-05-17T22:55:26.461 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.286576237, 2024-05-17T22:55:26.461 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.077263709, 2024-05-17T22:55:26.461 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.071707814, 2024-05-17T22:55:26.462 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.118835727, 2024-05-17T22:55:26.462 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T22:55:26.462 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 3590, 2024-05-17T22:55:26.462 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.141914593, 2024-05-17T22:55:26.462 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000039530 2024-05-17T22:55:26.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.462 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-05-17T22:55:26.462 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:26.462 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:26.462 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:26.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.462 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-17T22:55:26.462 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 0 2024-05-17T22:55:26.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.463 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-05-17T22:55:26.463 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 8, 2024-05-17T22:55:26.463 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 264, 2024-05-17T22:55:26.463 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 3868, 2024-05-17T22:55:26.463 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 602922, 2024-05-17T22:55:26.463 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 6, 2024-05-17T22:55:26.463 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 3, 2024-05-17T22:55:26.463 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.015617874, 2024-05-17T22:55:26.463 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.010550825, 2024-05-17T22:55:26.463 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.002699836, 2024-05-17T22:55:26.463 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000058672, 2024-05-17T22:55:26.464 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T22:55:26.464 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 263, 2024-05-17T22:55:26.464 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.019813083, 2024-05-17T22:55:26.464 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000075334 2024-05-17T22:55:26.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.464 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-05-17T22:55:26.464 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:26.464 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:26.464 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:26.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.464 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-05-17T22:55:26.464 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 602224 2024-05-17T22:55:26.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.465 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-05-17T22:55:26.465 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 374, 2024-05-17T22:55:26.465 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 350, 2024-05-17T22:55:26.465 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 656876, 2024-05-17T22:55:26.465 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 212478, 2024-05-17T22:55:26.465 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 4, 2024-05-17T22:55:26.465 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 2, 2024-05-17T22:55:26.465 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.046039558, 2024-05-17T22:55:26.465 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.020689754, 2024-05-17T22:55:26.465 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.032333151, 2024-05-17T22:55:26.465 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000018014, 2024-05-17T22:55:26.465 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T22:55:26.465 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 349, 2024-05-17T22:55:26.465 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.016347358, 2024-05-17T22:55:26.465 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000046840 2024-05-17T22:55:26.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.466 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-05-17T22:55:26.466 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:26.466 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:26.466 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:26.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.466 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 656608, 2024-05-17T22:55:26.466 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 212160 2024-05-17T22:55:26.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.466 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-05-17T22:55:26.466 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 1, 2024-05-17T22:55:26.466 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-05-17T22:55:26.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.467 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-MDSRank": { 2024-05-17T22:55:26.467 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-05-17T22:55:26.467 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-05-17T22:55:26.467 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 5976, 2024-05-17T22:55:26.467 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.149519825, 2024-05-17T22:55:26.467 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000025020 2024-05-17T22:55:26.467 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-17T22:55:26.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.467 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-PurgeQueue": { 2024-05-17T22:55:26.467 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-05-17T22:55:26.467 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-05-17T22:55:26.467 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:26.467 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:26.467 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:26.468 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-17T22:55:26.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.468 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": { 2024-05-17T22:55:26.468 INFO:teuthology.orchestra.run.smithi096.stdout: "request": 0, 2024-05-17T22:55:26.468 INFO:teuthology.orchestra.run.smithi096.stdout: "reply": 0, 2024-05-17T22:55:26.468 INFO:teuthology.orchestra.run.smithi096.stdout: "reply_latency": { 2024-05-17T22:55:26.468 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:26.468 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:26.468 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:26.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.468 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_reply": 0, 2024-05-17T22:55:26.469 INFO:teuthology.orchestra.run.smithi096.stdout: "forward": 0, 2024-05-17T22:55:26.469 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_complete": 0, 2024-05-17T22:55:26.469 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_keys": 0, 2024-05-17T22:55:26.469 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_commit": 0, 2024-05-17T22:55:26.469 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_split": 0, 2024-05-17T22:55:26.469 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_merge": 0, 2024-05-17T22:55:26.469 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes": 724, 2024-05-17T22:55:26.469 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_top": 0, 2024-05-17T22:55:26.469 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_bottom": 724, 2024-05-17T22:55:26.469 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pin_tail": 0, 2024-05-17T22:55:26.469 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pinned": 724, 2024-05-17T22:55:26.469 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_expired": 0, 2024-05-17T22:55:26.469 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_with_caps": 0, 2024-05-17T22:55:26.469 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": 0, 2024-05-17T22:55:26.470 INFO:teuthology.orchestra.run.smithi096.stdout: "subtrees": 13, 2024-05-17T22:55:26.470 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse": 0, 2024-05-17T22:55:26.470 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_hit": 0, 2024-05-17T22:55:26.470 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_forward": 0, 2024-05-17T22:55:26.470 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_discover": 0, 2024-05-17T22:55:26.470 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_dir_fetch": 0, 2024-05-17T22:55:26.470 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_remote_ino": 0, 2024-05-17T22:55:26.470 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_lock": 0, 2024-05-17T22:55:26.470 INFO:teuthology.orchestra.run.smithi096.stdout: "load_cent": 0, 2024-05-17T22:55:26.470 INFO:teuthology.orchestra.run.smithi096.stdout: "q": 0, 2024-05-17T22:55:26.470 INFO:teuthology.orchestra.run.smithi096.stdout: "exported": 0, 2024-05-17T22:55:26.471 INFO:teuthology.orchestra.run.smithi096.stdout: "exported_inodes": 0, 2024-05-17T22:55:26.471 INFO:teuthology.orchestra.run.smithi096.stdout: "imported": 0, 2024-05-17T22:55:26.471 INFO:teuthology.orchestra.run.smithi096.stdout: "imported_inodes": 0, 2024-05-17T22:55:26.471 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_dir_fetch": 0, 2024-05-17T22:55:26.471 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_backtrace_fetch": 0, 2024-05-17T22:55:26.471 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_peer_discover": 0, 2024-05-17T22:55:26.471 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rfiles": 0, 2024-05-17T22:55:26.471 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rbytes": 0, 2024-05-17T22:55:26.471 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rsnaps": 0, 2024-05-17T22:55:26.471 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_fetch": 0, 2024-05-17T22:55:26.471 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_set_tag": 0, 2024-05-17T22:55:26.471 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_repaired": 0, 2024-05-17T22:55:26.471 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_inotable_repaired": 0, 2024-05-17T22:55:26.471 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_inodes": 0, 2024-05-17T22:55:26.472 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_base_inodes": 0, 2024-05-17T22:55:26.472 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dirfrag_rstats": 0, 2024-05-17T22:55:26.472 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_file_inodes": 0, 2024-05-17T22:55:26.472 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_inode_file_caps": 0, 2024-05-17T22:55:26.472 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_revoke": 0, 2024-05-17T22:55:26.472 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_grant": 0, 2024-05-17T22:55:26.472 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_trunc": 0, 2024-05-17T22:55:26.472 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-17T22:55:26.472 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-17T22:55:26.472 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps": 0, 2024-05-17T22:55:26.472 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps_dirty": 0, 2024-05-17T22:55:26.472 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_cap_release": 0, 2024-05-17T22:55:26.472 INFO:teuthology.orchestra.run.smithi096.stdout: "process_request_cap_release": 0 2024-05-17T22:55:26.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.473 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache": { 2024-05-17T22:55:26.473 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update": 0, 2024-05-17T22:55:26.473 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update_receipt": 0, 2024-05-17T22:55:26.473 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_try_discover": 0, 2024-05-17T22:55:26.473 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_send_discover": 0, 2024-05-17T22:55:26.473 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_handle_discover": 0, 2024-05-17T22:55:26.473 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays": 0, 2024-05-17T22:55:26.473 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_delayed": 0, 2024-05-17T22:55:26.473 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_enqueuing": 0, 2024-05-17T22:55:26.473 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_created": 0, 2024-05-17T22:55:26.473 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_enqueued": 0, 2024-05-17T22:55:26.473 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_reintegrated": 0, 2024-05-17T22:55:26.473 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_migrated": 0, 2024-05-17T22:55:26.473 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_processing": 0, 2024-05-17T22:55:26.474 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_enqueued": 0, 2024-05-17T22:55:26.474 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_prioritized": 0, 2024-05-17T22:55:26.474 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_started": 0, 2024-05-17T22:55:26.474 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_completed": 0, 2024-05-17T22:55:26.474 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_path": 0, 2024-05-17T22:55:26.474 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_inode": 0, 2024-05-17T22:55:26.474 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_enqueue_scrub": 0, 2024-05-17T22:55:26.474 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_exportdir": 0, 2024-05-17T22:55:26.474 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_flush": 0, 2024-05-17T22:55:26.474 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragmentdir": 0, 2024-05-17T22:55:26.474 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragstats": 0, 2024-05-17T22:55:26.474 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_inodestats": 0 2024-05-17T22:55:26.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.475 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log": { 2024-05-17T22:55:26.475 INFO:teuthology.orchestra.run.smithi096.stdout: "evlrg": 0, 2024-05-17T22:55:26.475 INFO:teuthology.orchestra.run.smithi096.stdout: "evadd": 0, 2024-05-17T22:55:26.475 INFO:teuthology.orchestra.run.smithi096.stdout: "evex": 0, 2024-05-17T22:55:26.475 INFO:teuthology.orchestra.run.smithi096.stdout: "evtrm": 0, 2024-05-17T22:55:26.475 INFO:teuthology.orchestra.run.smithi096.stdout: "ev": 1393, 2024-05-17T22:55:26.475 INFO:teuthology.orchestra.run.smithi096.stdout: "evexg": 0, 2024-05-17T22:55:26.475 INFO:teuthology.orchestra.run.smithi096.stdout: "evexd": 0, 2024-05-17T22:55:26.475 INFO:teuthology.orchestra.run.smithi096.stdout: "segadd": 0, 2024-05-17T22:55:26.475 INFO:teuthology.orchestra.run.smithi096.stdout: "segex": 0, 2024-05-17T22:55:26.475 INFO:teuthology.orchestra.run.smithi096.stdout: "segtrm": 0, 2024-05-17T22:55:26.475 INFO:teuthology.orchestra.run.smithi096.stdout: "seg": 3, 2024-05-17T22:55:26.475 INFO:teuthology.orchestra.run.smithi096.stdout: "segmjr": 2, 2024-05-17T22:55:26.475 INFO:teuthology.orchestra.run.smithi096.stdout: "segexg": 0, 2024-05-17T22:55:26.475 INFO:teuthology.orchestra.run.smithi096.stdout: "segexd": 0, 2024-05-17T22:55:26.476 INFO:teuthology.orchestra.run.smithi096.stdout: "expos": 4194304, 2024-05-17T22:55:26.476 INFO:teuthology.orchestra.run.smithi096.stdout: "wrpos": 6815857, 2024-05-17T22:55:26.476 INFO:teuthology.orchestra.run.smithi096.stdout: "rdpos": 6814149, 2024-05-17T22:55:26.476 INFO:teuthology.orchestra.run.smithi096.stdout: "jlat": { 2024-05-17T22:55:26.476 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:26.476 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:26.476 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:26.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.476 INFO:teuthology.orchestra.run.smithi096.stdout: "replayed": 1393 2024-05-17T22:55:26.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.476 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_mem": { 2024-05-17T22:55:26.476 INFO:teuthology.orchestra.run.smithi096.stdout: "ino": 511, 2024-05-17T22:55:26.476 INFO:teuthology.orchestra.run.smithi096.stdout: "ino+": 880, 2024-05-17T22:55:26.476 INFO:teuthology.orchestra.run.smithi096.stdout: "ino-": 369, 2024-05-17T22:55:26.477 INFO:teuthology.orchestra.run.smithi096.stdout: "dir": 22, 2024-05-17T22:55:26.477 INFO:teuthology.orchestra.run.smithi096.stdout: "dir+": 37, 2024-05-17T22:55:26.477 INFO:teuthology.orchestra.run.smithi096.stdout: "dir-": 15, 2024-05-17T22:55:26.477 INFO:teuthology.orchestra.run.smithi096.stdout: "dn": 724, 2024-05-17T22:55:26.477 INFO:teuthology.orchestra.run.smithi096.stdout: "dn+": 918, 2024-05-17T22:55:26.477 INFO:teuthology.orchestra.run.smithi096.stdout: "dn-": 194, 2024-05-17T22:55:26.477 INFO:teuthology.orchestra.run.smithi096.stdout: "cap": 0, 2024-05-17T22:55:26.477 INFO:teuthology.orchestra.run.smithi096.stdout: "cap+": 0, 2024-05-17T22:55:26.477 INFO:teuthology.orchestra.run.smithi096.stdout: "cap-": 0, 2024-05-17T22:55:26.477 INFO:teuthology.orchestra.run.smithi096.stdout: "rss": 122836, 2024-05-17T22:55:26.477 INFO:teuthology.orchestra.run.smithi096.stdout: "heap": 215440 2024-05-17T22:55:26.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.477 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_server": { 2024-05-17T22:55:26.477 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_client_request": 0, 2024-05-17T22:55:26.478 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_server_request": 0, 2024-05-17T22:55:26.478 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_request": 0, 2024-05-17T22:55:26.478 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_session": 0, 2024-05-17T22:55:26.478 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_peer_request": 0, 2024-05-17T22:55:26.478 INFO:teuthology.orchestra.run.smithi096.stdout: "req_create_latency": { 2024-05-17T22:55:26.478 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:26.478 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:26.478 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:26.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.478 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getattr_latency": { 2024-05-17T22:55:26.478 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:26.478 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:26.478 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:26.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.479 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getfilelock_latency": { 2024-05-17T22:55:26.479 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:26.479 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:26.479 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:26.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.479 INFO:teuthology.orchestra.run.smithi096.stdout: "req_link_latency": { 2024-05-17T22:55:26.479 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:26.479 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:26.479 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:26.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.479 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookup_latency": { 2024-05-17T22:55:26.479 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:26.479 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:26.480 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:26.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.480 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookuphash_latency": { 2024-05-17T22:55:26.480 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:26.480 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:26.480 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:26.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.480 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupino_latency": { 2024-05-17T22:55:26.480 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:26.480 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:26.480 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:26.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.481 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupname_latency": { 2024-05-17T22:55:26.481 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:26.481 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:26.481 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:26.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.481 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupparent_latency": { 2024-05-17T22:55:26.481 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:26.481 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:26.481 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:26.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.481 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupsnap_latency": { 2024-05-17T22:55:26.481 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:26.481 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:26.482 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:26.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.482 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lssnap_latency": { 2024-05-17T22:55:26.482 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:26.482 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:26.482 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:26.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.482 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mkdir_latency": { 2024-05-17T22:55:26.483 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:26.483 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:26.483 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:26.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.483 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mknod_latency": { 2024-05-17T22:55:26.483 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:26.483 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:26.483 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:26.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.483 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mksnap_latency": { 2024-05-17T22:55:26.483 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:26.484 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:26.484 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:26.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.484 INFO:teuthology.orchestra.run.smithi096.stdout: "req_open_latency": { 2024-05-17T22:55:26.484 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:26.484 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:26.484 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:26.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.484 INFO:teuthology.orchestra.run.smithi096.stdout: "req_readdir_latency": { 2024-05-17T22:55:26.484 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:26.484 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:26.485 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:26.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.485 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rename_latency": { 2024-05-17T22:55:26.485 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:26.485 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:26.485 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:26.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.485 INFO:teuthology.orchestra.run.smithi096.stdout: "req_renamesnap_latency": { 2024-05-17T22:55:26.485 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:26.485 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:26.485 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:26.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.486 INFO:teuthology.orchestra.run.smithi096.stdout: "req_snapdiff_latency": { 2024-05-17T22:55:26.486 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:26.486 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:26.486 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:26.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.486 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmdir_latency": { 2024-05-17T22:55:26.486 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:26.486 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:26.486 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:26.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.486 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmsnap_latency": { 2024-05-17T22:55:26.486 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:26.486 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:26.487 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:26.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.487 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmxattr_latency": { 2024-05-17T22:55:26.487 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:26.487 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:26.487 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:26.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.487 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setattr_latency": { 2024-05-17T22:55:26.487 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:26.487 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:26.487 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:26.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.488 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setdirlayout_latency": { 2024-05-17T22:55:26.488 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:26.488 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:26.488 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:26.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.488 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setfilelock_latency": { 2024-05-17T22:55:26.488 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:26.488 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:26.488 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:26.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.488 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setlayout_latency": { 2024-05-17T22:55:26.488 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:26.489 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:26.489 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:26.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.489 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setxattr_latency": { 2024-05-17T22:55:26.489 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:26.489 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:26.489 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:26.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.489 INFO:teuthology.orchestra.run.smithi096.stdout: "req_symlink_latency": { 2024-05-17T22:55:26.489 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:26.489 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:26.489 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:26.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.489 INFO:teuthology.orchestra.run.smithi096.stdout: "req_unlink_latency": { 2024-05-17T22:55:26.490 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:26.490 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:26.490 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:26.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.490 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_revoke_eviction": 0, 2024-05-17T22:55:26.490 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_acquisition_throttle": 0, 2024-05-17T22:55:26.490 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getvxattr_latency": { 2024-05-17T22:55:26.490 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:26.490 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:26.490 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:26.490 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-17T22:55:26.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.490 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_sessions": { 2024-05-17T22:55:26.491 INFO:teuthology.orchestra.run.smithi096.stdout: "session_count": 1, 2024-05-17T22:55:26.491 INFO:teuthology.orchestra.run.smithi096.stdout: "session_add": 2, 2024-05-17T22:55:26.491 INFO:teuthology.orchestra.run.smithi096.stdout: "session_remove": 1, 2024-05-17T22:55:26.491 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_open": 1, 2024-05-17T22:55:26.491 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_stale": 0, 2024-05-17T22:55:26.491 INFO:teuthology.orchestra.run.smithi096.stdout: "total_load": 0, 2024-05-17T22:55:26.491 INFO:teuthology.orchestra.run.smithi096.stdout: "average_load": 0, 2024-05-17T22:55:26.491 INFO:teuthology.orchestra.run.smithi096.stdout: "avg_session_uptime": 2449, 2024-05-17T22:55:26.491 INFO:teuthology.orchestra.run.smithi096.stdout: "mdthresh_evicted": 0 2024-05-17T22:55:26.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.491 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-05-17T22:55:26.492 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 0, 2024-05-17T22:55:26.492 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 0, 2024-05-17T22:55:26.492 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-05-17T22:55:26.492 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-05-17T22:55:26.492 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-05-17T22:55:26.492 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-05-17T22:55:26.492 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-17T22:55:26.492 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-05-17T22:55:26.492 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-17T22:55:26.492 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-05-17T22:55:26.492 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-05-17T22:55:26.492 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-05-17T22:55:26.492 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-17T22:55:26.492 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-05-17T22:55:26.493 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-05-17T22:55:26.493 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-05-17T22:55:26.493 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-05-17T22:55:26.493 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-05-17T22:55:26.493 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-17T22:55:26.493 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-05-17T22:55:26.493 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-17T22:55:26.493 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-05-17T22:55:26.493 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-05-17T22:55:26.493 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-05-17T22:55:26.493 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-05-17T22:55:26.493 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-05-17T22:55:26.493 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-17T22:55:26.493 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-05-17T22:55:26.494 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-05-17T22:55:26.494 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-05-17T22:55:26.494 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-05-17T22:55:26.494 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-05-17T22:55:26.494 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-05-17T22:55:26.494 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-05-17T22:55:26.494 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-05-17T22:55:26.494 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-05-17T22:55:26.494 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 200661, 2024-05-17T22:55:26.494 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 550, 2024-05-17T22:55:26.494 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 0, 2024-05-17T22:55:26.494 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 0, 2024-05-17T22:55:26.494 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-05-17T22:55:26.495 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-05-17T22:55:26.495 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-05-17T22:55:26.495 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-05-17T22:55:26.495 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-05-17T22:55:26.495 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-05-17T22:55:26.495 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 8888, 2024-05-17T22:55:26.495 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 170, 2024-05-17T22:55:26.495 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-05-17T22:55:26.495 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-05-17T22:55:26.495 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-05-17T22:55:26.495 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-05-17T22:55:26.495 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 1726299, 2024-05-17T22:55:26.495 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 15237, 2024-05-17T22:55:26.495 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-05-17T22:55:26.495 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-05-17T22:55:26.496 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-05-17T22:55:26.496 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-05-17T22:55:26.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.496 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter": { 2024-05-17T22:55:26.496 INFO:teuthology.orchestra.run.smithi096.stdout: "op_active": 0, 2024-05-17T22:55:26.496 INFO:teuthology.orchestra.run.smithi096.stdout: "op_laggy": 0, 2024-05-17T22:55:26.496 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send": 3593, 2024-05-17T22:55:26.496 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send_bytes": 21, 2024-05-17T22:55:26.496 INFO:teuthology.orchestra.run.smithi096.stdout: "op_resend": 3, 2024-05-17T22:55:26.496 INFO:teuthology.orchestra.run.smithi096.stdout: "op_reply": 3593, 2024-05-17T22:55:26.496 INFO:teuthology.orchestra.run.smithi096.stdout: "op_latency": { 2024-05-17T22:55:26.496 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 3593, 2024-05-17T22:55:26.496 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 8.754807114, 2024-05-17T22:55:26.496 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.002436628 2024-05-17T22:55:26.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.497 INFO:teuthology.orchestra.run.smithi096.stdout: "op_inflight": 0, 2024-05-17T22:55:26.497 INFO:teuthology.orchestra.run.smithi096.stdout: "oplen_avg": { 2024-05-17T22:55:26.497 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 3593, 2024-05-17T22:55:26.497 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 3594 2024-05-17T22:55:26.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.497 INFO:teuthology.orchestra.run.smithi096.stdout: "op": 3593, 2024-05-17T22:55:26.497 INFO:teuthology.orchestra.run.smithi096.stdout: "op_r": 3593, 2024-05-17T22:55:26.497 INFO:teuthology.orchestra.run.smithi096.stdout: "op_w": 0, 2024-05-17T22:55:26.497 INFO:teuthology.orchestra.run.smithi096.stdout: "op_rmw": 0, 2024-05-17T22:55:26.497 INFO:teuthology.orchestra.run.smithi096.stdout: "op_pg": 0, 2024-05-17T22:55:26.497 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_stat": 2257, 2024-05-17T22:55:26.497 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_create": 0, 2024-05-17T22:55:26.497 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_read": 1334, 2024-05-17T22:55:26.498 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_write": 0, 2024-05-17T22:55:26.498 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writefull": 0, 2024-05-17T22:55:26.498 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writesame": 0, 2024-05-17T22:55:26.498 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_append": 0, 2024-05-17T22:55:26.498 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_zero": 0, 2024-05-17T22:55:26.498 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_truncate": 0, 2024-05-17T22:55:26.498 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_delete": 0, 2024-05-17T22:55:26.498 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_mapext": 0, 2024-05-17T22:55:26.498 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_sparse_read": 0, 2024-05-17T22:55:26.498 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_clonerange": 0, 2024-05-17T22:55:26.498 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_getxattr": 1, 2024-05-17T22:55:26.498 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_setxattr": 0, 2024-05-17T22:55:26.498 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_cmpxattr": 0, 2024-05-17T22:55:26.498 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_rmxattr": 0, 2024-05-17T22:55:26.499 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_resetxattrs": 0, 2024-05-17T22:55:26.499 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_call": 0, 2024-05-17T22:55:26.499 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_watch": 0, 2024-05-17T22:55:26.499 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_notify": 0, 2024-05-17T22:55:26.499 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_src_cmpxattr": 0, 2024-05-17T22:55:26.499 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls": 0, 2024-05-17T22:55:26.499 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls_filter": 0, 2024-05-17T22:55:26.499 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_other": 0, 2024-05-17T22:55:26.499 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_active": 0, 2024-05-17T22:55:26.499 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_send": 0, 2024-05-17T22:55:26.499 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_resend": 0, 2024-05-17T22:55:26.499 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_ping": 0, 2024-05-17T22:55:26.499 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_active": 0, 2024-05-17T22:55:26.499 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_send": 0, 2024-05-17T22:55:26.499 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_resend": 0, 2024-05-17T22:55:26.500 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_active": 0, 2024-05-17T22:55:26.500 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_send": 0, 2024-05-17T22:55:26.500 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_resend": 0, 2024-05-17T22:55:26.500 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_active": 0, 2024-05-17T22:55:26.500 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_send": 0, 2024-05-17T22:55:26.500 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_resend": 0, 2024-05-17T22:55:26.500 INFO:teuthology.orchestra.run.smithi096.stdout: "command_active": 0, 2024-05-17T22:55:26.500 INFO:teuthology.orchestra.run.smithi096.stdout: "command_send": 0, 2024-05-17T22:55:26.500 INFO:teuthology.orchestra.run.smithi096.stdout: "command_resend": 0, 2024-05-17T22:55:26.500 INFO:teuthology.orchestra.run.smithi096.stdout: "map_epoch": 89, 2024-05-17T22:55:26.500 INFO:teuthology.orchestra.run.smithi096.stdout: "map_full": 0, 2024-05-17T22:55:26.501 INFO:teuthology.orchestra.run.smithi096.stdout: "map_inc": 2, 2024-05-17T22:55:26.501 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_sessions": 4, 2024-05-17T22:55:26.501 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_open": 6, 2024-05-17T22:55:26.501 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_close": 2, 2024-05-17T22:55:26.501 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_laggy": 0, 2024-05-17T22:55:26.501 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_wr": 0, 2024-05-17T22:55:26.501 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_rd": 2, 2024-05-17T22:55:26.501 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_del": 0 2024-05-17T22:55:26.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.501 INFO:teuthology.orchestra.run.smithi096.stdout: "oft": { 2024-05-17T22:55:26.501 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_objs": 0, 2024-05-17T22:55:26.502 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_kv_pairs": 0, 2024-05-17T22:55:26.502 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_updates": 0, 2024-05-17T22:55:26.502 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_removes": 0 2024-05-17T22:55:26.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.502 INFO:teuthology.orchestra.run.smithi096.stdout: "osd-slow-ops": { 2024-05-17T22:55:26.502 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_ops_count": 0 2024-05-17T22:55:26.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.502 INFO:teuthology.orchestra.run.smithi096.stdout: "purge_queue": { 2024-05-17T22:55:26.502 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops": 0, 2024-05-17T22:55:26.502 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops_high_water": 0, 2024-05-17T22:55:26.502 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing": 0, 2024-05-17T22:55:26.502 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_high_water": 0, 2024-05-17T22:55:26.502 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed_ops": 0, 2024-05-17T22:55:26.503 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed": 0, 2024-05-17T22:55:26.503 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_item_in_journal": 0 2024-05-17T22:55:26.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.503 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-17T22:55:26.503 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-17T22:55:26.503 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-05-17T22:55:26.503 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-17T22:55:26.503 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 3968, 2024-05-17T22:55:26.503 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 3910902, 2024-05-17T22:55:26.503 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-17T22:55:26.503 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 3968, 2024-05-17T22:55:26.503 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-05-17T22:55:26.503 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-05-17T22:55:26.503 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 3968, 2024-05-17T22:55:26.503 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 3910902, 2024-05-17T22:55:26.504 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-17T22:55:26.504 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:26.504 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:26.504 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:26.504 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-17T22:55:26.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.504 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_bytes": { 2024-05-17T22:55:26.504 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-17T22:55:26.504 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-05-17T22:55:26.504 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-17T22:55:26.504 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-05-17T22:55:26.504 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-05-17T22:55:26.504 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-17T22:55:26.504 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-05-17T22:55:26.505 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 3593, 2024-05-17T22:55:26.505 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 2621558, 2024-05-17T22:55:26.505 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 3593, 2024-05-17T22:55:26.505 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 2621558, 2024-05-17T22:55:26.505 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-17T22:55:26.505 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:26.505 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:26.505 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:26.505 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-17T22:55:26.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.505 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_ops": { 2024-05-17T22:55:26.505 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-17T22:55:26.505 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 1024, 2024-05-17T22:55:26.505 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-17T22:55:26.505 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-05-17T22:55:26.506 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-05-17T22:55:26.506 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-17T22:55:26.506 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-05-17T22:55:26.506 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 3593, 2024-05-17T22:55:26.506 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 3593, 2024-05-17T22:55:26.506 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 3593, 2024-05-17T22:55:26.506 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 3593, 2024-05-17T22:55:26.506 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-17T22:55:26.506 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:26.506 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:26.506 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:26.506 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-17T22:55:26.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.507 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle": { 2024-05-17T22:55:26.507 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-17T22:55:26.507 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-05-17T22:55:26.507 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-17T22:55:26.507 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-05-17T22:55:26.507 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-05-17T22:55:26.507 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-17T22:55:26.507 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-05-17T22:55:26.507 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-05-17T22:55:26.507 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-05-17T22:55:26.507 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 0, 2024-05-17T22:55:26.507 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 0, 2024-05-17T22:55:26.507 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-17T22:55:26.508 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:26.508 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:26.508 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:26.508 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-17T22:55:26.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:26.508 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle-0x555f25bd6c30": { 2024-05-17T22:55:26.508 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-17T22:55:26.508 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-05-17T22:55:26.508 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-17T22:55:26.508 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-05-17T22:55:26.508 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-05-17T22:55:26.508 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-17T22:55:26.509 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-05-17T22:55:26.509 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-05-17T22:55:26.509 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-05-17T22:55:26.509 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 0, 2024-05-17T22:55:26.509 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 0, 2024-05-17T22:55:26.509 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-17T22:55:26.509 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:26.509 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:26.509 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:26.509 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-17T22:55:26.509 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-17T22:55:26.509 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-05-17T22:55:26.510 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:26 smithi151 ceph-mon[36101]: pgmap v924: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-05-17T22:55:26.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:26 smithi012 ceph-mon[31632]: pgmap v924: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-05-17T22:55:27.512 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-05-17T22:55:27.512 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-05-17T22:55:27.512 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-05-17T22:55:27.512 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-05-17T22:55:27.512 DEBUG:tasks.check_counter:Getting stats from h 2024-05-17T22:55:27.513 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-17T22:55:27.765 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.b/config 2024-05-17T22:55:28.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:28 smithi151 ceph-mon[36101]: pgmap v925: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1023 B/s wr, 6 op/s 2024-05-17T22:55:28.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:28 smithi096 ceph-mon[36065]: pgmap v925: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1023 B/s wr, 6 op/s 2024-05-17T22:55:28.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:28 smithi012 ceph-mon[31632]: pgmap v925: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1023 B/s wr, 6 op/s 2024-05-17T22:55:29.499 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-05-17T22:55:29.500 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-05-17T22:55:29.500 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 4957, 2024-05-17T22:55:29.500 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 4042, 2024-05-17T22:55:29.500 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 1613987, 2024-05-17T22:55:29.500 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 4304883, 2024-05-17T22:55:29.500 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 40, 2024-05-17T22:55:29.500 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 7, 2024-05-17T22:55:29.500 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.377001378, 2024-05-17T22:55:29.500 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.158784188, 2024-05-17T22:55:29.500 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.218928785, 2024-05-17T22:55:29.500 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.030049453, 2024-05-17T22:55:29.500 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T22:55:29.501 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 4040, 2024-05-17T22:55:29.501 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.247958770, 2024-05-17T22:55:29.501 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000061375 2024-05-17T22:55:29.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.501 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-05-17T22:55:29.501 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 7305, 2024-05-17T22:55:29.501 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.001440921, 2024-05-17T22:55:29.501 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000197 2024-05-17T22:55:29.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.501 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-17T22:55:29.501 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 0 2024-05-17T22:55:29.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.501 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-05-17T22:55:29.502 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 2558, 2024-05-17T22:55:29.502 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 4191, 2024-05-17T22:55:29.502 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 1310043, 2024-05-17T22:55:29.502 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 2041351, 2024-05-17T22:55:29.502 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 13, 2024-05-17T22:55:29.502 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 5, 2024-05-17T22:55:29.502 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.217706098, 2024-05-17T22:55:29.502 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.090762707, 2024-05-17T22:55:29.502 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.128040124, 2024-05-17T22:55:29.502 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.006059930, 2024-05-17T22:55:29.502 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T22:55:29.502 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 4191, 2024-05-17T22:55:29.502 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.256068981, 2024-05-17T22:55:29.503 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000061099 2024-05-17T22:55:29.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.503 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-05-17T22:55:29.503 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 3197, 2024-05-17T22:55:29.503 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.001026142, 2024-05-17T22:55:29.503 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000320 2024-05-17T22:55:29.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.503 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 681664, 2024-05-17T22:55:29.503 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 233632 2024-05-17T22:55:29.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.503 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-05-17T22:55:29.503 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 1048, 2024-05-17T22:55:29.503 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 1236, 2024-05-17T22:55:29.503 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 807574, 2024-05-17T22:55:29.504 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 1068701, 2024-05-17T22:55:29.504 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 9, 2024-05-17T22:55:29.504 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 6, 2024-05-17T22:55:29.504 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.085254079, 2024-05-17T22:55:29.504 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.031528835, 2024-05-17T22:55:29.504 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.031840010, 2024-05-17T22:55:29.504 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.019342964, 2024-05-17T22:55:29.504 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T22:55:29.504 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1234, 2024-05-17T22:55:29.504 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.074259096, 2024-05-17T22:55:29.504 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000060177 2024-05-17T22:55:29.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.504 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-05-17T22:55:29.504 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 581, 2024-05-17T22:55:29.505 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000104011, 2024-05-17T22:55:29.505 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000179 2024-05-17T22:55:29.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.505 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-17T22:55:29.505 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 603792 2024-05-17T22:55:29.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.505 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-05-17T22:55:29.505 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 1, 2024-05-17T22:55:29.505 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-05-17T22:55:29.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.505 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-MDSRank": { 2024-05-17T22:55:29.505 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-05-17T22:55:29.505 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-05-17T22:55:29.506 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1176, 2024-05-17T22:55:29.506 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.623255414, 2024-05-17T22:55:29.506 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000529979 2024-05-17T22:55:29.506 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-17T22:55:29.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.506 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-PurgeQueue": { 2024-05-17T22:55:29.506 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-05-17T22:55:29.506 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-05-17T22:55:29.506 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 661, 2024-05-17T22:55:29.506 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.034631514, 2024-05-17T22:55:29.506 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000052392 2024-05-17T22:55:29.506 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-17T22:55:29.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.507 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": { 2024-05-17T22:55:29.507 INFO:teuthology.orchestra.run.smithi096.stdout: "request": 910, 2024-05-17T22:55:29.507 INFO:teuthology.orchestra.run.smithi096.stdout: "reply": 818, 2024-05-17T22:55:29.507 INFO:teuthology.orchestra.run.smithi096.stdout: "reply_latency": { 2024-05-17T22:55:29.507 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 818, 2024-05-17T22:55:29.507 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 5.672636716, 2024-05-17T22:55:29.507 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.006934763 2024-05-17T22:55:29.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.507 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_reply": 0, 2024-05-17T22:55:29.507 INFO:teuthology.orchestra.run.smithi096.stdout: "forward": 92, 2024-05-17T22:55:29.507 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_complete": 0, 2024-05-17T22:55:29.507 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_keys": 0, 2024-05-17T22:55:29.507 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_commit": 11, 2024-05-17T22:55:29.507 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_split": 0, 2024-05-17T22:55:29.507 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_merge": 0, 2024-05-17T22:55:29.508 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes": 539, 2024-05-17T22:55:29.508 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_top": 2, 2024-05-17T22:55:29.508 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_bottom": 511, 2024-05-17T22:55:29.508 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pin_tail": 26, 2024-05-17T22:55:29.508 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pinned": 536, 2024-05-17T22:55:29.508 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_expired": 202, 2024-05-17T22:55:29.508 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_with_caps": 479, 2024-05-17T22:55:29.508 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": 479, 2024-05-17T22:55:29.508 INFO:teuthology.orchestra.run.smithi096.stdout: "subtrees": 15, 2024-05-17T22:55:29.508 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse": 2309, 2024-05-17T22:55:29.508 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_hit": 1784, 2024-05-17T22:55:29.508 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_forward": 91, 2024-05-17T22:55:29.508 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_discover": 38, 2024-05-17T22:55:29.508 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_dir_fetch": 0, 2024-05-17T22:55:29.509 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_remote_ino": 0, 2024-05-17T22:55:29.509 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_lock": 85, 2024-05-17T22:55:29.509 INFO:teuthology.orchestra.run.smithi096.stdout: "load_cent": 141, 2024-05-17T22:55:29.509 INFO:teuthology.orchestra.run.smithi096.stdout: "q": 0, 2024-05-17T22:55:29.509 INFO:teuthology.orchestra.run.smithi096.stdout: "exported": 9, 2024-05-17T22:55:29.509 INFO:teuthology.orchestra.run.smithi096.stdout: "exported_inodes": 179, 2024-05-17T22:55:29.509 INFO:teuthology.orchestra.run.smithi096.stdout: "imported": 20, 2024-05-17T22:55:29.509 INFO:teuthology.orchestra.run.smithi096.stdout: "imported_inodes": 667, 2024-05-17T22:55:29.509 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_dir_fetch": 0, 2024-05-17T22:55:29.509 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_backtrace_fetch": 486, 2024-05-17T22:55:29.509 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_peer_discover": 0, 2024-05-17T22:55:29.509 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rfiles": 1024, 2024-05-17T22:55:29.509 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rbytes": 1621914670, 2024-05-17T22:55:29.509 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rsnaps": 0, 2024-05-17T22:55:29.509 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_fetch": 486, 2024-05-17T22:55:29.510 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_set_tag": 0, 2024-05-17T22:55:29.510 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_repaired": 0, 2024-05-17T22:55:29.510 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_inotable_repaired": 0, 2024-05-17T22:55:29.510 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_inodes": 13, 2024-05-17T22:55:29.510 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_base_inodes": 0, 2024-05-17T22:55:29.510 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dirfrag_rstats": 13, 2024-05-17T22:55:29.510 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_file_inodes": 473, 2024-05-17T22:55:29.510 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_inode_file_caps": 0, 2024-05-17T22:55:29.510 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_revoke": 324, 2024-05-17T22:55:29.510 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_grant": 881, 2024-05-17T22:55:29.510 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_trunc": 0, 2024-05-17T22:55:29.510 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-17T22:55:29.510 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-17T22:55:29.510 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps": 756, 2024-05-17T22:55:29.510 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps_dirty": 279, 2024-05-17T22:55:29.511 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_cap_release": 138, 2024-05-17T22:55:29.511 INFO:teuthology.orchestra.run.smithi096.stdout: "process_request_cap_release": 855 2024-05-17T22:55:29.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.511 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache": { 2024-05-17T22:55:29.511 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update": 38, 2024-05-17T22:55:29.511 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update_receipt": 166, 2024-05-17T22:55:29.511 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_try_discover": 97, 2024-05-17T22:55:29.511 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_send_discover": 114, 2024-05-17T22:55:29.511 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_handle_discover": 39, 2024-05-17T22:55:29.511 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays": 0, 2024-05-17T22:55:29.511 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_delayed": 0, 2024-05-17T22:55:29.511 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_enqueuing": 0, 2024-05-17T22:55:29.511 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_created": 209, 2024-05-17T22:55:29.511 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_enqueued": 201, 2024-05-17T22:55:29.511 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_reintegrated": 8, 2024-05-17T22:55:29.512 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_migrated": 0, 2024-05-17T22:55:29.512 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_processing": 0, 2024-05-17T22:55:29.512 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_enqueued": 0, 2024-05-17T22:55:29.512 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_prioritized": 0, 2024-05-17T22:55:29.512 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_started": 0, 2024-05-17T22:55:29.512 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_completed": 0, 2024-05-17T22:55:29.512 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_path": 0, 2024-05-17T22:55:29.512 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_inode": 0, 2024-05-17T22:55:29.512 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_enqueue_scrub": 0, 2024-05-17T22:55:29.512 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_exportdir": 9, 2024-05-17T22:55:29.512 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_flush": 0, 2024-05-17T22:55:29.512 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragmentdir": 0, 2024-05-17T22:55:29.512 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragstats": 0, 2024-05-17T22:55:29.512 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_inodestats": 0 2024-05-17T22:55:29.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.513 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log": { 2024-05-17T22:55:29.513 INFO:teuthology.orchestra.run.smithi096.stdout: "evlrg": 0, 2024-05-17T22:55:29.513 INFO:teuthology.orchestra.run.smithi096.stdout: "evadd": 1398, 2024-05-17T22:55:29.513 INFO:teuthology.orchestra.run.smithi096.stdout: "evex": 0, 2024-05-17T22:55:29.513 INFO:teuthology.orchestra.run.smithi096.stdout: "evtrm": 0, 2024-05-17T22:55:29.513 INFO:teuthology.orchestra.run.smithi096.stdout: "ev": 1398, 2024-05-17T22:55:29.513 INFO:teuthology.orchestra.run.smithi096.stdout: "evexg": 0, 2024-05-17T22:55:29.513 INFO:teuthology.orchestra.run.smithi096.stdout: "evexd": 0, 2024-05-17T22:55:29.513 INFO:teuthology.orchestra.run.smithi096.stdout: "segadd": 3, 2024-05-17T22:55:29.513 INFO:teuthology.orchestra.run.smithi096.stdout: "segex": 0, 2024-05-17T22:55:29.513 INFO:teuthology.orchestra.run.smithi096.stdout: "segtrm": 0, 2024-05-17T22:55:29.513 INFO:teuthology.orchestra.run.smithi096.stdout: "seg": 3, 2024-05-17T22:55:29.513 INFO:teuthology.orchestra.run.smithi096.stdout: "segmjr": 2, 2024-05-17T22:55:29.513 INFO:teuthology.orchestra.run.smithi096.stdout: "segexg": 0, 2024-05-17T22:55:29.514 INFO:teuthology.orchestra.run.smithi096.stdout: "segexd": 0, 2024-05-17T22:55:29.514 INFO:teuthology.orchestra.run.smithi096.stdout: "expos": 4194304, 2024-05-17T22:55:29.514 INFO:teuthology.orchestra.run.smithi096.stdout: "wrpos": 6818727, 2024-05-17T22:55:29.514 INFO:teuthology.orchestra.run.smithi096.stdout: "rdpos": 0, 2024-05-17T22:55:29.514 INFO:teuthology.orchestra.run.smithi096.stdout: "jlat": { 2024-05-17T22:55:29.514 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 323, 2024-05-17T22:55:29.514 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 3.885835642, 2024-05-17T22:55:29.514 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.012030450 2024-05-17T22:55:29.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.514 INFO:teuthology.orchestra.run.smithi096.stdout: "replayed": 0 2024-05-17T22:55:29.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.514 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_mem": { 2024-05-17T22:55:29.514 INFO:teuthology.orchestra.run.smithi096.stdout: "ino": 528, 2024-05-17T22:55:29.514 INFO:teuthology.orchestra.run.smithi096.stdout: "ino+": 893, 2024-05-17T22:55:29.514 INFO:teuthology.orchestra.run.smithi096.stdout: "ino-": 365, 2024-05-17T22:55:29.515 INFO:teuthology.orchestra.run.smithi096.stdout: "dir": 52, 2024-05-17T22:55:29.515 INFO:teuthology.orchestra.run.smithi096.stdout: "dir+": 98, 2024-05-17T22:55:29.515 INFO:teuthology.orchestra.run.smithi096.stdout: "dir-": 46, 2024-05-17T22:55:29.515 INFO:teuthology.orchestra.run.smithi096.stdout: "dn": 539, 2024-05-17T22:55:29.515 INFO:teuthology.orchestra.run.smithi096.stdout: "dn+": 942, 2024-05-17T22:55:29.515 INFO:teuthology.orchestra.run.smithi096.stdout: "dn-": 403, 2024-05-17T22:55:29.515 INFO:teuthology.orchestra.run.smithi096.stdout: "cap": 479, 2024-05-17T22:55:29.515 INFO:teuthology.orchestra.run.smithi096.stdout: "cap+": 877, 2024-05-17T22:55:29.515 INFO:teuthology.orchestra.run.smithi096.stdout: "cap-": 398, 2024-05-17T22:55:29.515 INFO:teuthology.orchestra.run.smithi096.stdout: "rss": 128140, 2024-05-17T22:55:29.515 INFO:teuthology.orchestra.run.smithi096.stdout: "heap": 223632 2024-05-17T22:55:29.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.515 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_server": { 2024-05-17T22:55:29.515 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_client_request": 1531, 2024-05-17T22:55:29.516 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_server_request": 12, 2024-05-17T22:55:29.516 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_request": 910, 2024-05-17T22:55:29.516 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_session": 39, 2024-05-17T22:55:29.516 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_peer_request": 458, 2024-05-17T22:55:29.516 INFO:teuthology.orchestra.run.smithi096.stdout: "req_create_latency": { 2024-05-17T22:55:29.516 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 210, 2024-05-17T22:55:29.516 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.298244932, 2024-05-17T22:55:29.516 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.001420213 2024-05-17T22:55:29.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.516 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getattr_latency": { 2024-05-17T22:55:29.516 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1, 2024-05-17T22:55:29.516 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.049143634, 2024-05-17T22:55:29.516 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.049143634 2024-05-17T22:55:29.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.517 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getfilelock_latency": { 2024-05-17T22:55:29.517 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:29.517 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:29.517 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:29.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.517 INFO:teuthology.orchestra.run.smithi096.stdout: "req_link_latency": { 2024-05-17T22:55:29.517 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 8, 2024-05-17T22:55:29.517 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.012570457, 2024-05-17T22:55:29.517 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.001571307 2024-05-17T22:55:29.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.517 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookup_latency": { 2024-05-17T22:55:29.517 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 238, 2024-05-17T22:55:29.517 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 4.605706793, 2024-05-17T22:55:29.517 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.019351709 2024-05-17T22:55:29.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.518 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookuphash_latency": { 2024-05-17T22:55:29.518 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:29.518 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:29.518 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:29.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.518 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupino_latency": { 2024-05-17T22:55:29.518 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 2, 2024-05-17T22:55:29.518 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000278325, 2024-05-17T22:55:29.518 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000139162 2024-05-17T22:55:29.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.518 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupname_latency": { 2024-05-17T22:55:29.518 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:29.518 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:29.518 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:29.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.519 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupparent_latency": { 2024-05-17T22:55:29.519 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:29.519 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:29.519 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:29.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.519 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupsnap_latency": { 2024-05-17T22:55:29.519 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:29.519 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:29.519 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:29.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.519 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lssnap_latency": { 2024-05-17T22:55:29.519 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:29.520 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:29.520 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:29.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.520 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mkdir_latency": { 2024-05-17T22:55:29.520 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:29.520 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:29.520 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:29.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.520 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mknod_latency": { 2024-05-17T22:55:29.520 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:29.521 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:29.521 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:29.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.521 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mksnap_latency": { 2024-05-17T22:55:29.521 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:29.521 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:29.521 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:29.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.521 INFO:teuthology.orchestra.run.smithi096.stdout: "req_open_latency": { 2024-05-17T22:55:29.521 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 141, 2024-05-17T22:55:29.521 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.063378786, 2024-05-17T22:55:29.521 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000449494 2024-05-17T22:55:29.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.521 INFO:teuthology.orchestra.run.smithi096.stdout: "req_readdir_latency": { 2024-05-17T22:55:29.522 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1, 2024-05-17T22:55:29.522 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.001191847, 2024-05-17T22:55:29.522 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.001191847 2024-05-17T22:55:29.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.522 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rename_latency": { 2024-05-17T22:55:29.522 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 206, 2024-05-17T22:55:29.522 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.617044966, 2024-05-17T22:55:29.522 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.002995363 2024-05-17T22:55:29.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.522 INFO:teuthology.orchestra.run.smithi096.stdout: "req_renamesnap_latency": { 2024-05-17T22:55:29.522 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:29.522 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:29.522 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:29.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.523 INFO:teuthology.orchestra.run.smithi096.stdout: "req_snapdiff_latency": { 2024-05-17T22:55:29.523 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:29.523 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:29.523 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:29.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.523 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmdir_latency": { 2024-05-17T22:55:29.523 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:29.523 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:29.523 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:29.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.523 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmsnap_latency": { 2024-05-17T22:55:29.523 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:29.523 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:29.523 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:29.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.524 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmxattr_latency": { 2024-05-17T22:55:29.524 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:29.524 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:29.524 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:29.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.524 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setattr_latency": { 2024-05-17T22:55:29.524 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:29.524 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:29.524 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:29.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.524 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setdirlayout_latency": { 2024-05-17T22:55:29.524 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:29.524 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:29.525 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:29.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.525 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setfilelock_latency": { 2024-05-17T22:55:29.525 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:29.525 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:29.525 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:29.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.525 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setlayout_latency": { 2024-05-17T22:55:29.525 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:29.525 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:29.525 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:29.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.525 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setxattr_latency": { 2024-05-17T22:55:29.525 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:29.526 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:29.526 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:29.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.526 INFO:teuthology.orchestra.run.smithi096.stdout: "req_symlink_latency": { 2024-05-17T22:55:29.526 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:29.526 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:29.526 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:29.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.526 INFO:teuthology.orchestra.run.smithi096.stdout: "req_unlink_latency": { 2024-05-17T22:55:29.526 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 11, 2024-05-17T22:55:29.526 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.025076976, 2024-05-17T22:55:29.526 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.002279725 2024-05-17T22:55:29.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.526 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_revoke_eviction": 0, 2024-05-17T22:55:29.527 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_acquisition_throttle": 0, 2024-05-17T22:55:29.527 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getvxattr_latency": { 2024-05-17T22:55:29.527 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:29.527 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:29.527 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:29.527 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-17T22:55:29.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.527 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_sessions": { 2024-05-17T22:55:29.527 INFO:teuthology.orchestra.run.smithi096.stdout: "session_count": 1, 2024-05-17T22:55:29.527 INFO:teuthology.orchestra.run.smithi096.stdout: "session_add": 2, 2024-05-17T22:55:29.527 INFO:teuthology.orchestra.run.smithi096.stdout: "session_remove": 1, 2024-05-17T22:55:29.527 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_open": 1, 2024-05-17T22:55:29.527 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_stale": 0, 2024-05-17T22:55:29.528 INFO:teuthology.orchestra.run.smithi096.stdout: "total_load": 83, 2024-05-17T22:55:29.528 INFO:teuthology.orchestra.run.smithi096.stdout: "average_load": 83, 2024-05-17T22:55:29.528 INFO:teuthology.orchestra.run.smithi096.stdout: "avg_session_uptime": 989, 2024-05-17T22:55:29.528 INFO:teuthology.orchestra.run.smithi096.stdout: "mdthresh_evicted": 0 2024-05-17T22:55:29.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.528 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-05-17T22:55:29.528 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 0, 2024-05-17T22:55:29.528 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 0, 2024-05-17T22:55:29.528 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-05-17T22:55:29.528 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-05-17T22:55:29.528 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-05-17T22:55:29.528 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-05-17T22:55:29.528 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-17T22:55:29.528 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-05-17T22:55:29.528 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-17T22:55:29.529 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-05-17T22:55:29.529 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-05-17T22:55:29.529 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-05-17T22:55:29.529 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-17T22:55:29.529 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-05-17T22:55:29.529 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-05-17T22:55:29.529 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-05-17T22:55:29.529 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-05-17T22:55:29.529 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-05-17T22:55:29.529 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-17T22:55:29.529 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-05-17T22:55:29.529 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-17T22:55:29.529 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-05-17T22:55:29.530 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-05-17T22:55:29.530 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-05-17T22:55:29.530 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-05-17T22:55:29.530 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-05-17T22:55:29.530 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-17T22:55:29.530 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-05-17T22:55:29.530 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-05-17T22:55:29.530 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-05-17T22:55:29.530 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-05-17T22:55:29.530 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-05-17T22:55:29.530 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-05-17T22:55:29.530 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-05-17T22:55:29.530 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-05-17T22:55:29.531 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-05-17T22:55:29.531 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 366016, 2024-05-17T22:55:29.531 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 657, 2024-05-17T22:55:29.531 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 704, 2024-05-17T22:55:29.531 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 8, 2024-05-17T22:55:29.531 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-05-17T22:55:29.531 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-05-17T22:55:29.531 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-05-17T22:55:29.531 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-05-17T22:55:29.531 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-05-17T22:55:29.531 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-05-17T22:55:29.531 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 8888, 2024-05-17T22:55:29.531 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 170, 2024-05-17T22:55:29.531 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-05-17T22:55:29.532 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-05-17T22:55:29.532 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-05-17T22:55:29.532 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-05-17T22:55:29.532 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 1873595, 2024-05-17T22:55:29.532 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 18322, 2024-05-17T22:55:29.532 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-05-17T22:55:29.532 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-05-17T22:55:29.532 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-05-17T22:55:29.532 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-05-17T22:55:29.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.532 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter": { 2024-05-17T22:55:29.533 INFO:teuthology.orchestra.run.smithi096.stdout: "op_active": 0, 2024-05-17T22:55:29.533 INFO:teuthology.orchestra.run.smithi096.stdout: "op_laggy": 0, 2024-05-17T22:55:29.533 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send": 1639, 2024-05-17T22:55:29.533 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send_bytes": 2713786, 2024-05-17T22:55:29.533 INFO:teuthology.orchestra.run.smithi096.stdout: "op_resend": 33, 2024-05-17T22:55:29.533 INFO:teuthology.orchestra.run.smithi096.stdout: "op_reply": 1639, 2024-05-17T22:55:29.533 INFO:teuthology.orchestra.run.smithi096.stdout: "op_latency": { 2024-05-17T22:55:29.533 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1639, 2024-05-17T22:55:29.533 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 20.266553318, 2024-05-17T22:55:29.533 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.012365194 2024-05-17T22:55:29.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.533 INFO:teuthology.orchestra.run.smithi096.stdout: "op_inflight": 0, 2024-05-17T22:55:29.533 INFO:teuthology.orchestra.run.smithi096.stdout: "oplen_avg": { 2024-05-17T22:55:29.533 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1639, 2024-05-17T22:55:29.534 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 1785 2024-05-17T22:55:29.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.534 INFO:teuthology.orchestra.run.smithi096.stdout: "op": 1639, 2024-05-17T22:55:29.534 INFO:teuthology.orchestra.run.smithi096.stdout: "op_r": 565, 2024-05-17T22:55:29.534 INFO:teuthology.orchestra.run.smithi096.stdout: "op_w": 1074, 2024-05-17T22:55:29.534 INFO:teuthology.orchestra.run.smithi096.stdout: "op_rmw": 0, 2024-05-17T22:55:29.534 INFO:teuthology.orchestra.run.smithi096.stdout: "op_pg": 0, 2024-05-17T22:55:29.534 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_stat": 0, 2024-05-17T22:55:29.534 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_create": 10, 2024-05-17T22:55:29.534 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_read": 79, 2024-05-17T22:55:29.534 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_write": 402, 2024-05-17T22:55:29.534 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writefull": 119, 2024-05-17T22:55:29.534 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writesame": 0, 2024-05-17T22:55:29.534 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_append": 0, 2024-05-17T22:55:29.535 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_zero": 0, 2024-05-17T22:55:29.535 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_truncate": 0, 2024-05-17T22:55:29.535 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_delete": 414, 2024-05-17T22:55:29.535 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_mapext": 0, 2024-05-17T22:55:29.535 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_sparse_read": 0, 2024-05-17T22:55:29.535 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_clonerange": 0, 2024-05-17T22:55:29.535 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_getxattr": 486, 2024-05-17T22:55:29.535 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_setxattr": 20, 2024-05-17T22:55:29.535 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_cmpxattr": 0, 2024-05-17T22:55:29.535 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_rmxattr": 0, 2024-05-17T22:55:29.535 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_resetxattrs": 0, 2024-05-17T22:55:29.535 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_call": 0, 2024-05-17T22:55:29.535 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_watch": 0, 2024-05-17T22:55:29.535 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_notify": 0, 2024-05-17T22:55:29.536 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_src_cmpxattr": 0, 2024-05-17T22:55:29.536 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls": 0, 2024-05-17T22:55:29.536 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls_filter": 0, 2024-05-17T22:55:29.536 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_other": 0, 2024-05-17T22:55:29.536 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_active": 0, 2024-05-17T22:55:29.536 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_send": 0, 2024-05-17T22:55:29.536 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_resend": 0, 2024-05-17T22:55:29.536 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_ping": 0, 2024-05-17T22:55:29.536 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_active": 0, 2024-05-17T22:55:29.536 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_send": 0, 2024-05-17T22:55:29.536 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_resend": 0, 2024-05-17T22:55:29.536 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_active": 0, 2024-05-17T22:55:29.536 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_send": 0, 2024-05-17T22:55:29.536 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_resend": 0, 2024-05-17T22:55:29.537 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_active": 0, 2024-05-17T22:55:29.537 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_send": 0, 2024-05-17T22:55:29.537 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_resend": 0, 2024-05-17T22:55:29.537 INFO:teuthology.orchestra.run.smithi096.stdout: "command_active": 0, 2024-05-17T22:55:29.537 INFO:teuthology.orchestra.run.smithi096.stdout: "command_send": 0, 2024-05-17T22:55:29.537 INFO:teuthology.orchestra.run.smithi096.stdout: "command_resend": 0, 2024-05-17T22:55:29.537 INFO:teuthology.orchestra.run.smithi096.stdout: "map_epoch": 89, 2024-05-17T22:55:29.537 INFO:teuthology.orchestra.run.smithi096.stdout: "map_full": 0, 2024-05-17T22:55:29.537 INFO:teuthology.orchestra.run.smithi096.stdout: "map_inc": 2, 2024-05-17T22:55:29.537 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_sessions": 12, 2024-05-17T22:55:29.537 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_open": 12, 2024-05-17T22:55:29.537 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_close": 0, 2024-05-17T22:55:29.538 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_laggy": 0, 2024-05-17T22:55:29.538 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_wr": 192, 2024-05-17T22:55:29.538 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_rd": 0, 2024-05-17T22:55:29.538 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_del": 63 2024-05-17T22:55:29.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.538 INFO:teuthology.orchestra.run.smithi096.stdout: "oft": { 2024-05-17T22:55:29.538 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_objs": 1, 2024-05-17T22:55:29.538 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_kv_pairs": 0, 2024-05-17T22:55:29.538 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_updates": 0, 2024-05-17T22:55:29.538 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_removes": 0 2024-05-17T22:55:29.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.538 INFO:teuthology.orchestra.run.smithi096.stdout: "osd-slow-ops": { 2024-05-17T22:55:29.538 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_ops_count": 0 2024-05-17T22:55:29.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.539 INFO:teuthology.orchestra.run.smithi096.stdout: "purge_queue": { 2024-05-17T22:55:29.539 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops": 0, 2024-05-17T22:55:29.539 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops_high_water": 16, 2024-05-17T22:55:29.539 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing": 0, 2024-05-17T22:55:29.539 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_high_water": 8, 2024-05-17T22:55:29.539 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed_ops": 402, 2024-05-17T22:55:29.539 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed": 201, 2024-05-17T22:55:29.539 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_item_in_journal": 0 2024-05-17T22:55:29.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.539 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-17T22:55:29.539 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-17T22:55:29.539 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-05-17T22:55:29.539 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-17T22:55:29.539 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 8559, 2024-05-17T22:55:29.540 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 2966380, 2024-05-17T22:55:29.540 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-17T22:55:29.540 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 8559, 2024-05-17T22:55:29.540 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-05-17T22:55:29.540 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-05-17T22:55:29.540 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 8559, 2024-05-17T22:55:29.540 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 2966380, 2024-05-17T22:55:29.540 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-17T22:55:29.540 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:29.540 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:29.540 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:29.540 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-17T22:55:29.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.540 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_bytes": { 2024-05-17T22:55:29.540 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-17T22:55:29.541 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-05-17T22:55:29.541 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-17T22:55:29.541 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-05-17T22:55:29.541 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-05-17T22:55:29.541 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-17T22:55:29.541 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-05-17T22:55:29.541 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 1639, 2024-05-17T22:55:29.541 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 2735695, 2024-05-17T22:55:29.541 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 1639, 2024-05-17T22:55:29.541 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 2735695, 2024-05-17T22:55:29.541 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-17T22:55:29.541 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:29.541 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:29.542 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:29.542 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-17T22:55:29.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.542 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_ops": { 2024-05-17T22:55:29.542 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-17T22:55:29.542 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 1024, 2024-05-17T22:55:29.542 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-17T22:55:29.542 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-05-17T22:55:29.542 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-05-17T22:55:29.542 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-17T22:55:29.542 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-05-17T22:55:29.542 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 1639, 2024-05-17T22:55:29.542 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 1639, 2024-05-17T22:55:29.542 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 1639, 2024-05-17T22:55:29.543 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 1639, 2024-05-17T22:55:29.543 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-17T22:55:29.543 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:29.543 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:29.543 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:29.543 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-17T22:55:29.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.543 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle": { 2024-05-17T22:55:29.543 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-17T22:55:29.543 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-05-17T22:55:29.543 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-17T22:55:29.543 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 201, 2024-05-17T22:55:29.543 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 21909, 2024-05-17T22:55:29.543 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-17T22:55:29.544 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 201, 2024-05-17T22:55:29.544 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-05-17T22:55:29.544 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-05-17T22:55:29.544 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 79, 2024-05-17T22:55:29.544 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 21909, 2024-05-17T22:55:29.544 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-17T22:55:29.544 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:29.544 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:29.544 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:29.544 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-17T22:55:29.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:29.544 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle-0x557dfa622680": { 2024-05-17T22:55:29.544 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 2870, 2024-05-17T22:55:29.544 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-05-17T22:55:29.544 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-17T22:55:29.545 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 1398, 2024-05-17T22:55:29.545 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 2624423, 2024-05-17T22:55:29.545 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-17T22:55:29.545 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 1398, 2024-05-17T22:55:29.545 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-05-17T22:55:29.545 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-05-17T22:55:29.545 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 323, 2024-05-17T22:55:29.545 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 2621553, 2024-05-17T22:55:29.545 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-17T22:55:29.545 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:29.545 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:29.545 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:29.545 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-17T22:55:29.545 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-17T22:55:29.546 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-05-17T22:55:30.344 INFO:tasks.check_counter:Daemon mds.h mds.exported=9 2024-05-17T22:55:30.344 INFO:tasks.check_counter:Daemon mds.h mds.imported=20 2024-05-17T22:55:30.344 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=38 2024-05-17T22:55:30.344 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=166 2024-05-17T22:55:30.344 DEBUG:tasks.check_counter:Getting stats from k 2024-05-17T22:55:30.345 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-17T22:55:30.366 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:30 smithi096 ceph-mon[36065]: pgmap v926: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1023 B/s wr, 6 op/s 2024-05-17T22:55:30.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:30 smithi151 ceph-mon[36101]: pgmap v926: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1023 B/s wr, 6 op/s 2024-05-17T22:55:30.582 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.b/config 2024-05-17T22:55:30.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:30 smithi012 ceph-mon[31632]: pgmap v926: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1023 B/s wr, 6 op/s 2024-05-17T22:55:32.293 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:32 smithi096 ceph-mon[36065]: pgmap v927: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-05-17T22:55:32.293 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-05-17T22:55:32.293 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-05-17T22:55:32.293 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 349, 2024-05-17T22:55:32.293 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 346, 2024-05-17T22:55:32.294 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 551168, 2024-05-17T22:55:32.294 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 213536, 2024-05-17T22:55:32.294 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 4, 2024-05-17T22:55:32.294 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 0, 2024-05-17T22:55:32.294 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.046090106, 2024-05-17T22:55:32.294 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.020222356, 2024-05-17T22:55:32.294 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.031861832, 2024-05-17T22:55:32.294 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T22:55:32.294 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T22:55:32.294 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 345, 2024-05-17T22:55:32.294 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.027438004, 2024-05-17T22:55:32.294 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000079530 2024-05-17T22:55:32.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:32.295 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-05-17T22:55:32.295 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:32.295 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:32.295 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:32.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:32.295 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 551168, 2024-05-17T22:55:32.295 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 213536 2024-05-17T22:55:32.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:32.295 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-05-17T22:55:32.295 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 2, 2024-05-17T22:55:32.295 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 264, 2024-05-17T22:55:32.295 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 320, 2024-05-17T22:55:32.295 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 204160, 2024-05-17T22:55:32.295 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 4, 2024-05-17T22:55:32.296 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 2, 2024-05-17T22:55:32.296 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.014380852, 2024-05-17T22:55:32.296 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.010299852, 2024-05-17T22:55:32.296 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.001490746, 2024-05-17T22:55:32.296 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T22:55:32.296 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T22:55:32.296 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 262, 2024-05-17T22:55:32.296 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.017704368, 2024-05-17T22:55:32.296 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000067573 2024-05-17T22:55:32.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:32.296 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-05-17T22:55:32.296 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:32.296 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:32.297 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:32.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:32.297 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-17T22:55:32.297 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 204160 2024-05-17T22:55:32.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:32.297 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-05-17T22:55:32.297 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 4, 2024-05-17T22:55:32.297 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 4, 2024-05-17T22:55:32.297 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 2992, 2024-05-17T22:55:32.297 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 448, 2024-05-17T22:55:32.297 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 2, 2024-05-17T22:55:32.297 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 1, 2024-05-17T22:55:32.297 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.000661987, 2024-05-17T22:55:32.297 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.000029679, 2024-05-17T22:55:32.298 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.000613463, 2024-05-17T22:55:32.298 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T22:55:32.298 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T22:55:32.298 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 3, 2024-05-17T22:55:32.298 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000694705, 2024-05-17T22:55:32.298 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000231568 2024-05-17T22:55:32.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:32.298 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-05-17T22:55:32.298 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:32.298 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:32.298 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:32.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:32.298 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-17T22:55:32.298 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 448 2024-05-17T22:55:32.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:32.299 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-05-17T22:55:32.299 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 0, 2024-05-17T22:55:32.299 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-05-17T22:55:32.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:32.299 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-05-17T22:55:32.299 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 0, 2024-05-17T22:55:32.299 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 0, 2024-05-17T22:55:32.299 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-05-17T22:55:32.299 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-05-17T22:55:32.299 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-05-17T22:55:32.299 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-05-17T22:55:32.299 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-17T22:55:32.300 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-05-17T22:55:32.300 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-17T22:55:32.300 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-05-17T22:55:32.300 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-05-17T22:55:32.300 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-05-17T22:55:32.300 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-17T22:55:32.300 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-05-17T22:55:32.300 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-05-17T22:55:32.300 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-05-17T22:55:32.300 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-05-17T22:55:32.300 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-05-17T22:55:32.300 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-17T22:55:32.300 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-05-17T22:55:32.301 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-17T22:55:32.301 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-05-17T22:55:32.301 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-05-17T22:55:32.301 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-05-17T22:55:32.301 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-05-17T22:55:32.301 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-05-17T22:55:32.301 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-17T22:55:32.301 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-05-17T22:55:32.301 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-05-17T22:55:32.301 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-05-17T22:55:32.301 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-05-17T22:55:32.301 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-05-17T22:55:32.301 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-05-17T22:55:32.301 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-05-17T22:55:32.301 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-05-17T22:55:32.302 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-05-17T22:55:32.302 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 148704, 2024-05-17T22:55:32.302 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 29, 2024-05-17T22:55:32.302 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 0, 2024-05-17T22:55:32.302 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 0, 2024-05-17T22:55:32.302 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-05-17T22:55:32.302 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-05-17T22:55:32.302 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-05-17T22:55:32.302 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-05-17T22:55:32.302 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-05-17T22:55:32.302 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-05-17T22:55:32.302 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 0, 2024-05-17T22:55:32.302 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 0, 2024-05-17T22:55:32.303 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-05-17T22:55:32.303 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-05-17T22:55:32.303 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-05-17T22:55:32.303 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-05-17T22:55:32.303 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 928, 2024-05-17T22:55:32.303 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 2, 2024-05-17T22:55:32.303 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-05-17T22:55:32.303 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-05-17T22:55:32.303 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-05-17T22:55:32.303 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-05-17T22:55:32.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-17T22:55:32.303 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-17T22:55:32.303 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-17T22:55:32.303 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-05-17T22:55:32.303 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-17T22:55:32.304 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 351, 2024-05-17T22:55:32.304 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 501818, 2024-05-17T22:55:32.304 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-17T22:55:32.304 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 351, 2024-05-17T22:55:32.304 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-05-17T22:55:32.304 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-05-17T22:55:32.304 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 351, 2024-05-17T22:55:32.304 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 501818, 2024-05-17T22:55:32.304 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-17T22:55:32.304 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-17T22:55:32.304 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-17T22:55:32.304 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-17T22:55:32.304 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-17T22:55:32.304 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-17T22:55:32.305 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-05-17T22:55:32.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:32 smithi151 ceph-mon[36101]: pgmap v927: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-05-17T22:55:32.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:32 smithi012 ceph-mon[31632]: pgmap v927: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-05-17T22:55:32.914 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-05-17T22:55:32.915 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-05-17T22:55:32.915 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-05-17T22:55:32.915 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-05-17T22:55:32.915 DEBUG:tasks.check_counter:Getting stats from c 2024-05-17T22:55:32.915 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-17T22:55:33.213 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T22:55:34.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:34 smithi151 ceph-mon[36101]: pgmap v928: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-05-17T22:55:34.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:34 smithi096 ceph-mon[36065]: pgmap v928: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-05-17T22:55:34.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:34 smithi012 ceph-mon[31632]: pgmap v928: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-05-17T22:55:35.316 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-17T22:55:35.316 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-0": { 2024-05-17T22:55:35.316 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 1136, 2024-05-17T22:55:35.317 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 1112, 2024-05-17T22:55:35.317 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 5045219, 2024-05-17T22:55:35.317 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 454637, 2024-05-17T22:55:35.317 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 5, 2024-05-17T22:55:35.317 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 1, 2024-05-17T22:55:35.317 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.113145170, 2024-05-17T22:55:35.317 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.042785171, 2024-05-17T22:55:35.317 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.050457906, 2024-05-17T22:55:35.317 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.021249598, 2024-05-17T22:55:35.317 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T22:55:35.318 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 1111, 2024-05-17T22:55:35.318 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.054791560, 2024-05-17T22:55:35.318 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000049317 2024-05-17T22:55:35.318 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.318 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-05-17T22:55:35.318 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:35.318 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:35.318 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:35.319 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.319 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 656768, 2024-05-17T22:55:35.319 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 212320 2024-05-17T22:55:35.319 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.319 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-1": { 2024-05-17T22:55:35.319 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 2477, 2024-05-17T22:55:35.319 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 2734, 2024-05-17T22:55:35.319 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 7956112, 2024-05-17T22:55:35.319 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 1303309, 2024-05-17T22:55:35.319 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 5, 2024-05-17T22:55:35.319 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 4, 2024-05-17T22:55:35.319 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.248868008, 2024-05-17T22:55:35.319 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.081340039, 2024-05-17T22:55:35.320 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.064962212, 2024-05-17T22:55:35.320 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.081758219, 2024-05-17T22:55:35.320 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T22:55:35.320 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 2733, 2024-05-17T22:55:35.320 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.149247665, 2024-05-17T22:55:35.320 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000054609 2024-05-17T22:55:35.320 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.320 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-05-17T22:55:35.320 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:35.320 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:35.321 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:35.321 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.321 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-05-17T22:55:35.321 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 517152 2024-05-17T22:55:35.321 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.321 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-2": { 2024-05-17T22:55:35.321 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 531, 2024-05-17T22:55:35.321 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 533, 2024-05-17T22:55:35.321 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 131269, 2024-05-17T22:55:35.321 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 168873, 2024-05-17T22:55:35.321 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 5, 2024-05-17T22:55:35.321 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 4, 2024-05-17T22:55:35.321 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.054640805, 2024-05-17T22:55:35.321 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.011737237, 2024-05-17T22:55:35.322 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.015112013, 2024-05-17T22:55:35.322 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.023370639, 2024-05-17T22:55:35.322 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T22:55:35.322 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 531, 2024-05-17T22:55:35.322 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.026402978, 2024-05-17T22:55:35.322 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000049723 2024-05-17T22:55:35.322 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.322 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-05-17T22:55:35.323 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:35.323 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:35.323 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:35.323 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.323 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-17T22:55:35.323 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 0 2024-05-17T22:55:35.323 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.323 INFO:teuthology.orchestra.run.smithi151.stdout: "cct": { 2024-05-17T22:55:35.323 INFO:teuthology.orchestra.run.smithi151.stdout: "total_workers": 1, 2024-05-17T22:55:35.323 INFO:teuthology.orchestra.run.smithi151.stdout: "unhealthy_workers": 0 2024-05-17T22:55:35.323 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.323 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-MDSRank": { 2024-05-17T22:55:35.323 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-05-17T22:55:35.324 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-05-17T22:55:35.324 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 6326, 2024-05-17T22:55:35.324 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.175309058, 2024-05-17T22:55:35.324 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000027712 2024-05-17T22:55:35.324 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-17T22:55:35.324 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.324 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-PurgeQueue": { 2024-05-17T22:55:35.324 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-05-17T22:55:35.324 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-05-17T22:55:35.324 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:35.325 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:35.325 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:35.325 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-17T22:55:35.325 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.325 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": { 2024-05-17T22:55:35.325 INFO:teuthology.orchestra.run.smithi151.stdout: "request": 0, 2024-05-17T22:55:35.325 INFO:teuthology.orchestra.run.smithi151.stdout: "reply": 0, 2024-05-17T22:55:35.325 INFO:teuthology.orchestra.run.smithi151.stdout: "reply_latency": { 2024-05-17T22:55:35.325 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:35.325 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:35.325 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:35.325 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.326 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_reply": 0, 2024-05-17T22:55:35.326 INFO:teuthology.orchestra.run.smithi151.stdout: "forward": 0, 2024-05-17T22:55:35.326 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_complete": 0, 2024-05-17T22:55:35.326 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_keys": 0, 2024-05-17T22:55:35.326 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_commit": 0, 2024-05-17T22:55:35.326 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_split": 0, 2024-05-17T22:55:35.326 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_merge": 0, 2024-05-17T22:55:35.326 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes": 408, 2024-05-17T22:55:35.327 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_top": 0, 2024-05-17T22:55:35.327 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_bottom": 408, 2024-05-17T22:55:35.327 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pin_tail": 0, 2024-05-17T22:55:35.327 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pinned": 408, 2024-05-17T22:55:35.327 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_expired": 0, 2024-05-17T22:55:35.327 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_with_caps": 0, 2024-05-17T22:55:35.327 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": 0, 2024-05-17T22:55:35.327 INFO:teuthology.orchestra.run.smithi151.stdout: "subtrees": 5, 2024-05-17T22:55:35.327 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse": 0, 2024-05-17T22:55:35.327 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_hit": 0, 2024-05-17T22:55:35.327 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_forward": 0, 2024-05-17T22:55:35.327 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_discover": 0, 2024-05-17T22:55:35.327 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_dir_fetch": 0, 2024-05-17T22:55:35.327 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_remote_ino": 0, 2024-05-17T22:55:35.328 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_lock": 0, 2024-05-17T22:55:35.328 INFO:teuthology.orchestra.run.smithi151.stdout: "load_cent": 0, 2024-05-17T22:55:35.328 INFO:teuthology.orchestra.run.smithi151.stdout: "q": 0, 2024-05-17T22:55:35.328 INFO:teuthology.orchestra.run.smithi151.stdout: "exported": 0, 2024-05-17T22:55:35.328 INFO:teuthology.orchestra.run.smithi151.stdout: "exported_inodes": 0, 2024-05-17T22:55:35.328 INFO:teuthology.orchestra.run.smithi151.stdout: "imported": 0, 2024-05-17T22:55:35.328 INFO:teuthology.orchestra.run.smithi151.stdout: "imported_inodes": 0, 2024-05-17T22:55:35.328 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_dir_fetch": 0, 2024-05-17T22:55:35.328 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_backtrace_fetch": 0, 2024-05-17T22:55:35.328 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_peer_discover": 0, 2024-05-17T22:55:35.328 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rfiles": 1022, 2024-05-17T22:55:35.328 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rbytes": 1605948972, 2024-05-17T22:55:35.328 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rsnaps": 0, 2024-05-17T22:55:35.328 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_fetch": 0, 2024-05-17T22:55:35.329 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_set_tag": 0, 2024-05-17T22:55:35.329 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_repaired": 0, 2024-05-17T22:55:35.329 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_inotable_repaired": 0, 2024-05-17T22:55:35.329 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_inodes": 0, 2024-05-17T22:55:35.329 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_base_inodes": 0, 2024-05-17T22:55:35.329 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dirfrag_rstats": 0, 2024-05-17T22:55:35.329 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_file_inodes": 0, 2024-05-17T22:55:35.329 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_inode_file_caps": 0, 2024-05-17T22:55:35.329 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_revoke": 0, 2024-05-17T22:55:35.329 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_grant": 0, 2024-05-17T22:55:35.329 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_trunc": 0, 2024-05-17T22:55:35.329 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-17T22:55:35.329 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-17T22:55:35.330 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps": 0, 2024-05-17T22:55:35.330 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps_dirty": 0, 2024-05-17T22:55:35.330 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_cap_release": 0, 2024-05-17T22:55:35.330 INFO:teuthology.orchestra.run.smithi151.stdout: "process_request_cap_release": 0 2024-05-17T22:55:35.330 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.330 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_cache": { 2024-05-17T22:55:35.330 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update": 0, 2024-05-17T22:55:35.330 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update_receipt": 0, 2024-05-17T22:55:35.330 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_try_discover": 0, 2024-05-17T22:55:35.330 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_send_discover": 0, 2024-05-17T22:55:35.330 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_handle_discover": 0, 2024-05-17T22:55:35.330 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays": 0, 2024-05-17T22:55:35.330 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_delayed": 0, 2024-05-17T22:55:35.331 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_enqueuing": 0, 2024-05-17T22:55:35.331 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_created": 0, 2024-05-17T22:55:35.331 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_enqueued": 0, 2024-05-17T22:55:35.331 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_reintegrated": 0, 2024-05-17T22:55:35.331 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_migrated": 0, 2024-05-17T22:55:35.331 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_processing": 0, 2024-05-17T22:55:35.331 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_enqueued": 0, 2024-05-17T22:55:35.331 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_prioritized": 0, 2024-05-17T22:55:35.331 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_started": 0, 2024-05-17T22:55:35.331 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_completed": 0, 2024-05-17T22:55:35.331 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_path": 0, 2024-05-17T22:55:35.331 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_inode": 0, 2024-05-17T22:55:35.331 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_enqueue_scrub": 0, 2024-05-17T22:55:35.331 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_exportdir": 0, 2024-05-17T22:55:35.332 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_flush": 0, 2024-05-17T22:55:35.332 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragmentdir": 0, 2024-05-17T22:55:35.332 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragstats": 0, 2024-05-17T22:55:35.332 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_inodestats": 0 2024-05-17T22:55:35.332 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.332 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_log": { 2024-05-17T22:55:35.332 INFO:teuthology.orchestra.run.smithi151.stdout: "evlrg": 0, 2024-05-17T22:55:35.332 INFO:teuthology.orchestra.run.smithi151.stdout: "evadd": 0, 2024-05-17T22:55:35.332 INFO:teuthology.orchestra.run.smithi151.stdout: "evex": 0, 2024-05-17T22:55:35.332 INFO:teuthology.orchestra.run.smithi151.stdout: "evtrm": 0, 2024-05-17T22:55:35.332 INFO:teuthology.orchestra.run.smithi151.stdout: "ev": 4214, 2024-05-17T22:55:35.332 INFO:teuthology.orchestra.run.smithi151.stdout: "evexg": 0, 2024-05-17T22:55:35.332 INFO:teuthology.orchestra.run.smithi151.stdout: "evexd": 0, 2024-05-17T22:55:35.333 INFO:teuthology.orchestra.run.smithi151.stdout: "segadd": 0, 2024-05-17T22:55:35.333 INFO:teuthology.orchestra.run.smithi151.stdout: "segex": 0, 2024-05-17T22:55:35.333 INFO:teuthology.orchestra.run.smithi151.stdout: "segtrm": 0, 2024-05-17T22:55:35.333 INFO:teuthology.orchestra.run.smithi151.stdout: "seg": 6, 2024-05-17T22:55:35.333 INFO:teuthology.orchestra.run.smithi151.stdout: "segmjr": 2, 2024-05-17T22:55:35.333 INFO:teuthology.orchestra.run.smithi151.stdout: "segexg": 0, 2024-05-17T22:55:35.333 INFO:teuthology.orchestra.run.smithi151.stdout: "segexd": 0, 2024-05-17T22:55:35.333 INFO:teuthology.orchestra.run.smithi151.stdout: "expos": 4194304, 2024-05-17T22:55:35.333 INFO:teuthology.orchestra.run.smithi151.stdout: "wrpos": 15617544, 2024-05-17T22:55:35.333 INFO:teuthology.orchestra.run.smithi151.stdout: "rdpos": 15610515, 2024-05-17T22:55:35.333 INFO:teuthology.orchestra.run.smithi151.stdout: "jlat": { 2024-05-17T22:55:35.333 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:35.333 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:35.334 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:35.334 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.334 INFO:teuthology.orchestra.run.smithi151.stdout: "replayed": 4214 2024-05-17T22:55:35.334 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.334 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_mem": { 2024-05-17T22:55:35.334 INFO:teuthology.orchestra.run.smithi151.stdout: "ino": 119, 2024-05-17T22:55:35.334 INFO:teuthology.orchestra.run.smithi151.stdout: "ino+": 1262, 2024-05-17T22:55:35.334 INFO:teuthology.orchestra.run.smithi151.stdout: "ino-": 1143, 2024-05-17T22:55:35.334 INFO:teuthology.orchestra.run.smithi151.stdout: "dir": 38, 2024-05-17T22:55:35.334 INFO:teuthology.orchestra.run.smithi151.stdout: "dir+": 77, 2024-05-17T22:55:35.334 INFO:teuthology.orchestra.run.smithi151.stdout: "dir-": 39, 2024-05-17T22:55:35.334 INFO:teuthology.orchestra.run.smithi151.stdout: "dn": 408, 2024-05-17T22:55:35.334 INFO:teuthology.orchestra.run.smithi151.stdout: "dn+": 1546, 2024-05-17T22:55:35.334 INFO:teuthology.orchestra.run.smithi151.stdout: "dn-": 1138, 2024-05-17T22:55:35.335 INFO:teuthology.orchestra.run.smithi151.stdout: "cap": 0, 2024-05-17T22:55:35.335 INFO:teuthology.orchestra.run.smithi151.stdout: "cap+": 0, 2024-05-17T22:55:35.335 INFO:teuthology.orchestra.run.smithi151.stdout: "cap-": 0, 2024-05-17T22:55:35.335 INFO:teuthology.orchestra.run.smithi151.stdout: "rss": 123840, 2024-05-17T22:55:35.335 INFO:teuthology.orchestra.run.smithi151.stdout: "heap": 215440 2024-05-17T22:55:35.335 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.335 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_server": { 2024-05-17T22:55:35.335 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_client_request": 0, 2024-05-17T22:55:35.335 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_server_request": 0, 2024-05-17T22:55:35.335 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_request": 0, 2024-05-17T22:55:35.335 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_session": 0, 2024-05-17T22:55:35.335 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_peer_request": 0, 2024-05-17T22:55:35.335 INFO:teuthology.orchestra.run.smithi151.stdout: "req_create_latency": { 2024-05-17T22:55:35.335 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:35.336 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:35.336 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:35.336 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.336 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getattr_latency": { 2024-05-17T22:55:35.336 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:35.336 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:35.336 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:35.336 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.336 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getfilelock_latency": { 2024-05-17T22:55:35.336 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:35.336 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:35.336 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:35.336 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.336 INFO:teuthology.orchestra.run.smithi151.stdout: "req_link_latency": { 2024-05-17T22:55:35.337 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:35.337 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:35.337 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:35.337 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.337 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookup_latency": { 2024-05-17T22:55:35.337 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:35.337 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:35.337 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:35.337 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.337 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookuphash_latency": { 2024-05-17T22:55:35.337 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:35.337 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:35.337 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:35.337 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.338 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupino_latency": { 2024-05-17T22:55:35.338 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:35.338 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:35.338 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:35.338 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.338 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupname_latency": { 2024-05-17T22:55:35.338 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:35.338 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:35.338 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:35.338 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.338 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupparent_latency": { 2024-05-17T22:55:35.338 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:35.338 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:35.339 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:35.339 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.339 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupsnap_latency": { 2024-05-17T22:55:35.339 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:35.339 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:35.339 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:35.339 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.339 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lssnap_latency": { 2024-05-17T22:55:35.339 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:35.339 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:35.339 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:35.339 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.340 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mkdir_latency": { 2024-05-17T22:55:35.340 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:35.340 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:35.340 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:35.340 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.340 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mknod_latency": { 2024-05-17T22:55:35.340 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:35.340 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:35.341 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:35.341 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.341 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mksnap_latency": { 2024-05-17T22:55:35.341 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:35.341 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:35.341 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:35.341 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.341 INFO:teuthology.orchestra.run.smithi151.stdout: "req_open_latency": { 2024-05-17T22:55:35.341 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:35.341 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:35.341 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:35.341 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.341 INFO:teuthology.orchestra.run.smithi151.stdout: "req_readdir_latency": { 2024-05-17T22:55:35.342 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:35.342 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:35.342 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:35.342 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.342 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rename_latency": { 2024-05-17T22:55:35.342 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:35.342 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:35.342 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:35.342 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.342 INFO:teuthology.orchestra.run.smithi151.stdout: "req_renamesnap_latency": { 2024-05-17T22:55:35.342 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:35.342 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:35.342 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:35.342 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.343 INFO:teuthology.orchestra.run.smithi151.stdout: "req_snapdiff_latency": { 2024-05-17T22:55:35.343 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:35.343 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:35.343 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:35.343 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.343 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmdir_latency": { 2024-05-17T22:55:35.343 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:35.343 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:35.343 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:35.343 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.343 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmsnap_latency": { 2024-05-17T22:55:35.343 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:35.343 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:35.343 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:35.344 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.344 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmxattr_latency": { 2024-05-17T22:55:35.344 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:35.344 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:35.344 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:35.344 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.344 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setattr_latency": { 2024-05-17T22:55:35.344 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:35.344 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:35.344 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:35.344 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.344 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setdirlayout_latency": { 2024-05-17T22:55:35.344 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:35.344 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:35.345 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:35.345 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.345 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setfilelock_latency": { 2024-05-17T22:55:35.345 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:35.345 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:35.345 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:35.345 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.345 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setlayout_latency": { 2024-05-17T22:55:35.345 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:35.345 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:35.345 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:35.345 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.345 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setxattr_latency": { 2024-05-17T22:55:35.346 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:35.346 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:35.346 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:35.346 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.346 INFO:teuthology.orchestra.run.smithi151.stdout: "req_symlink_latency": { 2024-05-17T22:55:35.346 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:35.346 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:35.346 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:35.346 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.346 INFO:teuthology.orchestra.run.smithi151.stdout: "req_unlink_latency": { 2024-05-17T22:55:35.346 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:35.346 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:35.346 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:35.347 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.347 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_revoke_eviction": 0, 2024-05-17T22:55:35.347 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_acquisition_throttle": 0, 2024-05-17T22:55:35.347 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getvxattr_latency": { 2024-05-17T22:55:35.347 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:35.347 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:35.347 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:35.347 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-17T22:55:35.347 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.347 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_sessions": { 2024-05-17T22:55:35.347 INFO:teuthology.orchestra.run.smithi151.stdout: "session_count": 1, 2024-05-17T22:55:35.347 INFO:teuthology.orchestra.run.smithi151.stdout: "session_add": 3, 2024-05-17T22:55:35.347 INFO:teuthology.orchestra.run.smithi151.stdout: "session_remove": 2, 2024-05-17T22:55:35.347 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_open": 1, 2024-05-17T22:55:35.348 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_stale": 0, 2024-05-17T22:55:35.348 INFO:teuthology.orchestra.run.smithi151.stdout: "total_load": 0, 2024-05-17T22:55:35.348 INFO:teuthology.orchestra.run.smithi151.stdout: "average_load": 0, 2024-05-17T22:55:35.348 INFO:teuthology.orchestra.run.smithi151.stdout: "avg_session_uptime": 2449, 2024-05-17T22:55:35.348 INFO:teuthology.orchestra.run.smithi151.stdout: "mdthresh_evicted": 0 2024-05-17T22:55:35.348 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.348 INFO:teuthology.orchestra.run.smithi151.stdout: "mempool": { 2024-05-17T22:55:35.348 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_bytes": 0, 2024-05-17T22:55:35.348 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_items": 0, 2024-05-17T22:55:35.348 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_bytes": 0, 2024-05-17T22:55:35.348 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_items": 0, 2024-05-17T22:55:35.348 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_bytes": 0, 2024-05-17T22:55:35.348 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_items": 0, 2024-05-17T22:55:35.348 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-17T22:55:35.349 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_items": 0, 2024-05-17T22:55:35.349 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-17T22:55:35.349 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_items": 0, 2024-05-17T22:55:35.349 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_bytes": 0, 2024-05-17T22:55:35.349 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_items": 0, 2024-05-17T22:55:35.349 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-17T22:55:35.349 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_items": 0, 2024-05-17T22:55:35.349 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_bytes": 0, 2024-05-17T22:55:35.349 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_items": 0, 2024-05-17T22:55:35.349 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_bytes": 0, 2024-05-17T22:55:35.349 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_items": 0, 2024-05-17T22:55:35.349 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-17T22:55:35.349 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_items": 0, 2024-05-17T22:55:35.349 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-17T22:55:35.350 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_items": 0, 2024-05-17T22:55:35.350 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_bytes": 0, 2024-05-17T22:55:35.350 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_items": 0, 2024-05-17T22:55:35.350 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_bytes": 0, 2024-05-17T22:55:35.350 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_items": 0, 2024-05-17T22:55:35.350 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-17T22:55:35.350 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_items": 0, 2024-05-17T22:55:35.350 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_bytes": 0, 2024-05-17T22:55:35.350 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_items": 0, 2024-05-17T22:55:35.350 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_bytes": 0, 2024-05-17T22:55:35.350 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_items": 0, 2024-05-17T22:55:35.350 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_bytes": 0, 2024-05-17T22:55:35.350 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_items": 0, 2024-05-17T22:55:35.351 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_bytes": 0, 2024-05-17T22:55:35.351 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_items": 0, 2024-05-17T22:55:35.351 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_bytes": 201200, 2024-05-17T22:55:35.351 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_items": 155, 2024-05-17T22:55:35.351 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_bytes": 0, 2024-05-17T22:55:35.351 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_items": 0, 2024-05-17T22:55:35.351 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_bytes": 0, 2024-05-17T22:55:35.351 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_items": 0, 2024-05-17T22:55:35.351 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_bytes": 0, 2024-05-17T22:55:35.352 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_items": 0, 2024-05-17T22:55:35.352 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_bytes": 0, 2024-05-17T22:55:35.352 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_items": 0, 2024-05-17T22:55:35.352 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_bytes": 8888, 2024-05-17T22:55:35.352 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_items": 170, 2024-05-17T22:55:35.352 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_bytes": 0, 2024-05-17T22:55:35.352 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_items": 0, 2024-05-17T22:55:35.352 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_bytes": 0, 2024-05-17T22:55:35.352 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_items": 0, 2024-05-17T22:55:35.352 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_bytes": 625338, 2024-05-17T22:55:35.352 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_items": 14685, 2024-05-17T22:55:35.352 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_bytes": 0, 2024-05-17T22:55:35.352 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_items": 0, 2024-05-17T22:55:35.352 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_bytes": 0, 2024-05-17T22:55:35.353 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_items": 0 2024-05-17T22:55:35.353 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.353 INFO:teuthology.orchestra.run.smithi151.stdout: "objecter": { 2024-05-17T22:55:35.353 INFO:teuthology.orchestra.run.smithi151.stdout: "op_active": 0, 2024-05-17T22:55:35.353 INFO:teuthology.orchestra.run.smithi151.stdout: "op_laggy": 0, 2024-05-17T22:55:35.353 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send": 3765, 2024-05-17T22:55:35.353 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send_bytes": 26, 2024-05-17T22:55:35.353 INFO:teuthology.orchestra.run.smithi151.stdout: "op_resend": 4, 2024-05-17T22:55:35.353 INFO:teuthology.orchestra.run.smithi151.stdout: "op_reply": 3765, 2024-05-17T22:55:35.353 INFO:teuthology.orchestra.run.smithi151.stdout: "op_latency": { 2024-05-17T22:55:35.353 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 3765, 2024-05-17T22:55:35.353 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 38.602600041, 2024-05-17T22:55:35.353 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.010253014 2024-05-17T22:55:35.353 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.354 INFO:teuthology.orchestra.run.smithi151.stdout: "op_inflight": 0, 2024-05-17T22:55:35.354 INFO:teuthology.orchestra.run.smithi151.stdout: "oplen_avg": { 2024-05-17T22:55:35.354 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 3765, 2024-05-17T22:55:35.354 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 3766 2024-05-17T22:55:35.354 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.354 INFO:teuthology.orchestra.run.smithi151.stdout: "op": 3765, 2024-05-17T22:55:35.354 INFO:teuthology.orchestra.run.smithi151.stdout: "op_r": 3765, 2024-05-17T22:55:35.354 INFO:teuthology.orchestra.run.smithi151.stdout: "op_w": 0, 2024-05-17T22:55:35.354 INFO:teuthology.orchestra.run.smithi151.stdout: "op_rmw": 0, 2024-05-17T22:55:35.354 INFO:teuthology.orchestra.run.smithi151.stdout: "op_pg": 0, 2024-05-17T22:55:35.354 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_stat": 2294, 2024-05-17T22:55:35.354 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_create": 0, 2024-05-17T22:55:35.354 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_read": 1468, 2024-05-17T22:55:35.355 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_write": 0, 2024-05-17T22:55:35.355 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writefull": 0, 2024-05-17T22:55:35.355 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writesame": 0, 2024-05-17T22:55:35.355 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_append": 0, 2024-05-17T22:55:35.355 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_zero": 0, 2024-05-17T22:55:35.355 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_truncate": 0, 2024-05-17T22:55:35.355 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_delete": 0, 2024-05-17T22:55:35.355 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_mapext": 0, 2024-05-17T22:55:35.355 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_sparse_read": 0, 2024-05-17T22:55:35.355 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_clonerange": 0, 2024-05-17T22:55:35.355 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_getxattr": 2, 2024-05-17T22:55:35.355 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_setxattr": 0, 2024-05-17T22:55:35.355 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_cmpxattr": 0, 2024-05-17T22:55:35.355 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_rmxattr": 0, 2024-05-17T22:55:35.356 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_resetxattrs": 0, 2024-05-17T22:55:35.356 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_call": 0, 2024-05-17T22:55:35.356 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_watch": 0, 2024-05-17T22:55:35.356 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_notify": 0, 2024-05-17T22:55:35.356 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_src_cmpxattr": 0, 2024-05-17T22:55:35.356 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls": 0, 2024-05-17T22:55:35.356 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls_filter": 0, 2024-05-17T22:55:35.356 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_other": 0, 2024-05-17T22:55:35.356 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_active": 0, 2024-05-17T22:55:35.356 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_send": 0, 2024-05-17T22:55:35.356 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_resend": 0, 2024-05-17T22:55:35.356 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_ping": 0, 2024-05-17T22:55:35.356 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_active": 0, 2024-05-17T22:55:35.356 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_send": 0, 2024-05-17T22:55:35.356 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_resend": 0, 2024-05-17T22:55:35.357 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_active": 0, 2024-05-17T22:55:35.357 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_send": 0, 2024-05-17T22:55:35.357 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_resend": 0, 2024-05-17T22:55:35.357 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_active": 0, 2024-05-17T22:55:35.357 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_send": 0, 2024-05-17T22:55:35.357 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_resend": 0, 2024-05-17T22:55:35.357 INFO:teuthology.orchestra.run.smithi151.stdout: "command_active": 0, 2024-05-17T22:55:35.357 INFO:teuthology.orchestra.run.smithi151.stdout: "command_send": 0, 2024-05-17T22:55:35.357 INFO:teuthology.orchestra.run.smithi151.stdout: "command_resend": 0, 2024-05-17T22:55:35.357 INFO:teuthology.orchestra.run.smithi151.stdout: "map_epoch": 89, 2024-05-17T22:55:35.357 INFO:teuthology.orchestra.run.smithi151.stdout: "map_full": 0, 2024-05-17T22:55:35.357 INFO:teuthology.orchestra.run.smithi151.stdout: "map_inc": 2, 2024-05-17T22:55:35.357 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_sessions": 6, 2024-05-17T22:55:35.357 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_open": 6, 2024-05-17T22:55:35.358 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_close": 0, 2024-05-17T22:55:35.358 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_laggy": 0, 2024-05-17T22:55:35.358 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_wr": 0, 2024-05-17T22:55:35.358 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_rd": 2, 2024-05-17T22:55:35.358 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_del": 0 2024-05-17T22:55:35.358 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.358 INFO:teuthology.orchestra.run.smithi151.stdout: "oft": { 2024-05-17T22:55:35.358 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_objs": 0, 2024-05-17T22:55:35.358 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_kv_pairs": 0, 2024-05-17T22:55:35.358 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_updates": 0, 2024-05-17T22:55:35.358 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_removes": 0 2024-05-17T22:55:35.358 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.358 INFO:teuthology.orchestra.run.smithi151.stdout: "osd-slow-ops": { 2024-05-17T22:55:35.359 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_ops_count": 0 2024-05-17T22:55:35.359 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.359 INFO:teuthology.orchestra.run.smithi151.stdout: "purge_queue": { 2024-05-17T22:55:35.359 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops": 0, 2024-05-17T22:55:35.359 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops_high_water": 0, 2024-05-17T22:55:35.359 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing": 0, 2024-05-17T22:55:35.359 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_high_water": 0, 2024-05-17T22:55:35.359 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed_ops": 0, 2024-05-17T22:55:35.359 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed": 0, 2024-05-17T22:55:35.359 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_item_in_journal": 0 2024-05-17T22:55:35.359 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.359 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-17T22:55:35.359 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-05-17T22:55:35.359 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-05-17T22:55:35.360 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-17T22:55:35.360 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 4140, 2024-05-17T22:55:35.360 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 12737833, 2024-05-17T22:55:35.360 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-17T22:55:35.360 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 4140, 2024-05-17T22:55:35.360 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-05-17T22:55:35.360 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-05-17T22:55:35.360 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 4140, 2024-05-17T22:55:35.360 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 12737833, 2024-05-17T22:55:35.360 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-17T22:55:35.360 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:35.360 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:35.360 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:35.360 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-17T22:55:35.361 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.361 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_bytes": { 2024-05-17T22:55:35.361 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-05-17T22:55:35.361 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-05-17T22:55:35.361 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-17T22:55:35.361 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-05-17T22:55:35.361 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-05-17T22:55:35.361 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-17T22:55:35.361 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-05-17T22:55:35.361 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 3765, 2024-05-17T22:55:35.361 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 11423250, 2024-05-17T22:55:35.361 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 3765, 2024-05-17T22:55:35.361 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 11423250, 2024-05-17T22:55:35.362 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-17T22:55:35.362 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:35.362 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:35.362 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:35.362 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-17T22:55:35.362 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.362 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_ops": { 2024-05-17T22:55:35.362 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-05-17T22:55:35.362 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1024, 2024-05-17T22:55:35.362 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-17T22:55:35.362 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-05-17T22:55:35.362 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-05-17T22:55:35.362 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-17T22:55:35.363 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-05-17T22:55:35.363 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 3765, 2024-05-17T22:55:35.363 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 3765, 2024-05-17T22:55:35.363 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 3765, 2024-05-17T22:55:35.363 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 3765, 2024-05-17T22:55:35.363 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-17T22:55:35.363 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:35.363 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:35.363 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:35.363 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-17T22:55:35.363 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.363 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle": { 2024-05-17T22:55:35.363 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-05-17T22:55:35.363 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-05-17T22:55:35.363 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-17T22:55:35.364 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-05-17T22:55:35.364 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-05-17T22:55:35.364 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-17T22:55:35.364 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-05-17T22:55:35.364 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-05-17T22:55:35.364 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-05-17T22:55:35.364 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 0, 2024-05-17T22:55:35.364 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 0, 2024-05-17T22:55:35.364 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-17T22:55:35.364 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:35.364 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:35.364 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:35.364 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-17T22:55:35.364 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:35.365 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle-0x55b268c5ac30": { 2024-05-17T22:55:35.365 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-05-17T22:55:35.365 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-05-17T22:55:35.365 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-17T22:55:35.365 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-05-17T22:55:35.365 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-05-17T22:55:35.365 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-17T22:55:35.365 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-05-17T22:55:35.365 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-05-17T22:55:35.365 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-05-17T22:55:35.365 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 0, 2024-05-17T22:55:35.365 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 0, 2024-05-17T22:55:35.365 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-17T22:55:35.365 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:35.366 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:35.366 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:35.366 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-17T22:55:35.366 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-17T22:55:35.366 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-17T22:55:36.095 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-05-17T22:55:36.096 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-05-17T22:55:36.096 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-05-17T22:55:36.096 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-05-17T22:55:36.096 DEBUG:tasks.check_counter:Getting stats from f 2024-05-17T22:55:36.096 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-17T22:55:36.342 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T22:55:36.464 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:36 smithi151 ceph-mon[36101]: pgmap v929: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-17T22:55:36.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:36 smithi096 ceph-mon[36065]: pgmap v929: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-17T22:55:36.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:36 smithi012 ceph-mon[31632]: pgmap v929: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-17T22:55:38.410 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:38 smithi151 ceph-mon[36101]: pgmap v930: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-17T22:55:38.411 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-17T22:55:38.411 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-0": { 2024-05-17T22:55:38.411 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 376, 2024-05-17T22:55:38.411 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 352, 2024-05-17T22:55:38.411 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 657986, 2024-05-17T22:55:38.411 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 213764, 2024-05-17T22:55:38.411 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 6, 2024-05-17T22:55:38.411 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 1, 2024-05-17T22:55:38.411 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.048886960, 2024-05-17T22:55:38.412 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.021100977, 2024-05-17T22:55:38.412 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.036710864, 2024-05-17T22:55:38.412 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.000024128, 2024-05-17T22:55:38.412 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T22:55:38.412 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 351, 2024-05-17T22:55:38.412 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.018222299, 2024-05-17T22:55:38.412 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000051915 2024-05-17T22:55:38.412 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.412 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-05-17T22:55:38.412 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:38.412 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:38.412 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:38.412 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.412 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 657152, 2024-05-17T22:55:38.413 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 213440 2024-05-17T22:55:38.413 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.413 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-1": { 2024-05-17T22:55:38.413 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 1097, 2024-05-17T22:55:38.413 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 1355, 2024-05-17T22:55:38.413 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 271806, 2024-05-17T22:55:38.413 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 864589, 2024-05-17T22:55:38.413 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 5, 2024-05-17T22:55:38.413 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 4, 2024-05-17T22:55:38.413 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.124315420, 2024-05-17T22:55:38.413 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.034678432, 2024-05-17T22:55:38.413 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.029202942, 2024-05-17T22:55:38.413 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.047236942, 2024-05-17T22:55:38.413 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T22:55:38.414 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 1354, 2024-05-17T22:55:38.414 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.092238853, 2024-05-17T22:55:38.414 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000068123 2024-05-17T22:55:38.414 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.414 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-05-17T22:55:38.414 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:38.414 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:38.414 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:38.414 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.414 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-05-17T22:55:38.414 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 517328 2024-05-17T22:55:38.414 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.414 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-2": { 2024-05-17T22:55:38.414 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 2540, 2024-05-17T22:55:38.415 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 2542, 2024-05-17T22:55:38.415 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 3166821, 2024-05-17T22:55:38.415 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 807778, 2024-05-17T22:55:38.415 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 4, 2024-05-17T22:55:38.415 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 3, 2024-05-17T22:55:38.415 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.259575350, 2024-05-17T22:55:38.415 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.068146846, 2024-05-17T22:55:38.415 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.069578157, 2024-05-17T22:55:38.415 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.102047807, 2024-05-17T22:55:38.415 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T22:55:38.415 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 2540, 2024-05-17T22:55:38.415 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.122419584, 2024-05-17T22:55:38.415 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000048196 2024-05-17T22:55:38.415 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.416 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-05-17T22:55:38.416 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:38.416 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:38.416 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:38.416 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.416 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-17T22:55:38.416 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 0 2024-05-17T22:55:38.416 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.416 INFO:teuthology.orchestra.run.smithi151.stdout: "cct": { 2024-05-17T22:55:38.416 INFO:teuthology.orchestra.run.smithi151.stdout: "total_workers": 1, 2024-05-17T22:55:38.416 INFO:teuthology.orchestra.run.smithi151.stdout: "unhealthy_workers": 0 2024-05-17T22:55:38.416 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.416 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-MDSRank": { 2024-05-17T22:55:38.416 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-05-17T22:55:38.417 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-05-17T22:55:38.417 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 6111, 2024-05-17T22:55:38.417 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.197120593, 2024-05-17T22:55:38.417 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000032256 2024-05-17T22:55:38.417 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-17T22:55:38.417 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.417 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-PurgeQueue": { 2024-05-17T22:55:38.417 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-05-17T22:55:38.417 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-05-17T22:55:38.417 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:38.417 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:38.417 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:38.417 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-17T22:55:38.417 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.418 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": { 2024-05-17T22:55:38.418 INFO:teuthology.orchestra.run.smithi151.stdout: "request": 0, 2024-05-17T22:55:38.418 INFO:teuthology.orchestra.run.smithi151.stdout: "reply": 0, 2024-05-17T22:55:38.418 INFO:teuthology.orchestra.run.smithi151.stdout: "reply_latency": { 2024-05-17T22:55:38.418 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:38.418 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:38.418 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:38.418 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.418 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_reply": 0, 2024-05-17T22:55:38.418 INFO:teuthology.orchestra.run.smithi151.stdout: "forward": 0, 2024-05-17T22:55:38.418 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_complete": 0, 2024-05-17T22:55:38.418 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_keys": 0, 2024-05-17T22:55:38.418 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_commit": 0, 2024-05-17T22:55:38.418 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_split": 0, 2024-05-17T22:55:38.419 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_merge": 0, 2024-05-17T22:55:38.419 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes": 499, 2024-05-17T22:55:38.419 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_top": 1, 2024-05-17T22:55:38.419 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_bottom": 498, 2024-05-17T22:55:38.419 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pin_tail": 0, 2024-05-17T22:55:38.419 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pinned": 497, 2024-05-17T22:55:38.419 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_expired": 0, 2024-05-17T22:55:38.419 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_with_caps": 0, 2024-05-17T22:55:38.419 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": 0, 2024-05-17T22:55:38.419 INFO:teuthology.orchestra.run.smithi151.stdout: "subtrees": 14, 2024-05-17T22:55:38.419 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse": 0, 2024-05-17T22:55:38.419 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_hit": 0, 2024-05-17T22:55:38.419 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_forward": 0, 2024-05-17T22:55:38.419 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_discover": 0, 2024-05-17T22:55:38.420 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_dir_fetch": 0, 2024-05-17T22:55:38.420 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_remote_ino": 0, 2024-05-17T22:55:38.420 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_lock": 0, 2024-05-17T22:55:38.420 INFO:teuthology.orchestra.run.smithi151.stdout: "load_cent": 0, 2024-05-17T22:55:38.420 INFO:teuthology.orchestra.run.smithi151.stdout: "q": 0, 2024-05-17T22:55:38.420 INFO:teuthology.orchestra.run.smithi151.stdout: "exported": 0, 2024-05-17T22:55:38.420 INFO:teuthology.orchestra.run.smithi151.stdout: "exported_inodes": 0, 2024-05-17T22:55:38.420 INFO:teuthology.orchestra.run.smithi151.stdout: "imported": 0, 2024-05-17T22:55:38.420 INFO:teuthology.orchestra.run.smithi151.stdout: "imported_inodes": 0, 2024-05-17T22:55:38.420 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_dir_fetch": 0, 2024-05-17T22:55:38.420 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_backtrace_fetch": 0, 2024-05-17T22:55:38.420 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_peer_discover": 0, 2024-05-17T22:55:38.420 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rfiles": 0, 2024-05-17T22:55:38.420 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rbytes": 0, 2024-05-17T22:55:38.420 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rsnaps": 0, 2024-05-17T22:55:38.421 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_fetch": 0, 2024-05-17T22:55:38.421 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_set_tag": 0, 2024-05-17T22:55:38.421 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_repaired": 0, 2024-05-17T22:55:38.421 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_inotable_repaired": 0, 2024-05-17T22:55:38.421 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_inodes": 0, 2024-05-17T22:55:38.421 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_base_inodes": 0, 2024-05-17T22:55:38.421 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dirfrag_rstats": 0, 2024-05-17T22:55:38.421 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_file_inodes": 0, 2024-05-17T22:55:38.421 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_inode_file_caps": 0, 2024-05-17T22:55:38.421 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_revoke": 0, 2024-05-17T22:55:38.421 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_grant": 0, 2024-05-17T22:55:38.421 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_trunc": 0, 2024-05-17T22:55:38.421 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-17T22:55:38.421 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-17T22:55:38.422 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps": 0, 2024-05-17T22:55:38.422 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps_dirty": 0, 2024-05-17T22:55:38.422 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_cap_release": 0, 2024-05-17T22:55:38.422 INFO:teuthology.orchestra.run.smithi151.stdout: "process_request_cap_release": 0 2024-05-17T22:55:38.422 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.422 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_cache": { 2024-05-17T22:55:38.422 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update": 0, 2024-05-17T22:55:38.422 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update_receipt": 0, 2024-05-17T22:55:38.422 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_try_discover": 0, 2024-05-17T22:55:38.422 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_send_discover": 0, 2024-05-17T22:55:38.422 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_handle_discover": 0, 2024-05-17T22:55:38.422 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays": 0, 2024-05-17T22:55:38.422 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_delayed": 0, 2024-05-17T22:55:38.422 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_enqueuing": 0, 2024-05-17T22:55:38.423 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_created": 0, 2024-05-17T22:55:38.423 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_enqueued": 0, 2024-05-17T22:55:38.423 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_reintegrated": 0, 2024-05-17T22:55:38.423 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_migrated": 0, 2024-05-17T22:55:38.423 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_processing": 0, 2024-05-17T22:55:38.423 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_enqueued": 0, 2024-05-17T22:55:38.423 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_prioritized": 0, 2024-05-17T22:55:38.423 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_started": 0, 2024-05-17T22:55:38.423 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_completed": 0, 2024-05-17T22:55:38.423 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_path": 0, 2024-05-17T22:55:38.423 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_inode": 0, 2024-05-17T22:55:38.423 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_enqueue_scrub": 0, 2024-05-17T22:55:38.423 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_exportdir": 0, 2024-05-17T22:55:38.423 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_flush": 0, 2024-05-17T22:55:38.424 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragmentdir": 0, 2024-05-17T22:55:38.424 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragstats": 0, 2024-05-17T22:55:38.424 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_inodestats": 0 2024-05-17T22:55:38.424 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.424 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_log": { 2024-05-17T22:55:38.424 INFO:teuthology.orchestra.run.smithi151.stdout: "evlrg": 0, 2024-05-17T22:55:38.424 INFO:teuthology.orchestra.run.smithi151.stdout: "evadd": 0, 2024-05-17T22:55:38.424 INFO:teuthology.orchestra.run.smithi151.stdout: "evex": 0, 2024-05-17T22:55:38.424 INFO:teuthology.orchestra.run.smithi151.stdout: "evtrm": 0, 2024-05-17T22:55:38.424 INFO:teuthology.orchestra.run.smithi151.stdout: "ev": 1080, 2024-05-17T22:55:38.424 INFO:teuthology.orchestra.run.smithi151.stdout: "evexg": 0, 2024-05-17T22:55:38.424 INFO:teuthology.orchestra.run.smithi151.stdout: "evexd": 0, 2024-05-17T22:55:38.424 INFO:teuthology.orchestra.run.smithi151.stdout: "segadd": 0, 2024-05-17T22:55:38.424 INFO:teuthology.orchestra.run.smithi151.stdout: "segex": 0, 2024-05-17T22:55:38.424 INFO:teuthology.orchestra.run.smithi151.stdout: "segtrm": 0, 2024-05-17T22:55:38.425 INFO:teuthology.orchestra.run.smithi151.stdout: "seg": 3, 2024-05-17T22:55:38.425 INFO:teuthology.orchestra.run.smithi151.stdout: "segmjr": 2, 2024-05-17T22:55:38.425 INFO:teuthology.orchestra.run.smithi151.stdout: "segexg": 0, 2024-05-17T22:55:38.425 INFO:teuthology.orchestra.run.smithi151.stdout: "segexd": 0, 2024-05-17T22:55:38.425 INFO:teuthology.orchestra.run.smithi151.stdout: "expos": 4194304, 2024-05-17T22:55:38.425 INFO:teuthology.orchestra.run.smithi151.stdout: "wrpos": 6611636, 2024-05-17T22:55:38.425 INFO:teuthology.orchestra.run.smithi151.stdout: "rdpos": 6611062, 2024-05-17T22:55:38.425 INFO:teuthology.orchestra.run.smithi151.stdout: "jlat": { 2024-05-17T22:55:38.425 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:38.425 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:38.425 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:38.425 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.425 INFO:teuthology.orchestra.run.smithi151.stdout: "replayed": 1080 2024-05-17T22:55:38.425 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.426 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_mem": { 2024-05-17T22:55:38.426 INFO:teuthology.orchestra.run.smithi151.stdout: "ino": 464, 2024-05-17T22:55:38.426 INFO:teuthology.orchestra.run.smithi151.stdout: "ino+": 931, 2024-05-17T22:55:38.426 INFO:teuthology.orchestra.run.smithi151.stdout: "ino-": 467, 2024-05-17T22:55:38.426 INFO:teuthology.orchestra.run.smithi151.stdout: "dir": 37, 2024-05-17T22:55:38.426 INFO:teuthology.orchestra.run.smithi151.stdout: "dir+": 44, 2024-05-17T22:55:38.426 INFO:teuthology.orchestra.run.smithi151.stdout: "dir-": 7, 2024-05-17T22:55:38.426 INFO:teuthology.orchestra.run.smithi151.stdout: "dn": 499, 2024-05-17T22:55:38.426 INFO:teuthology.orchestra.run.smithi151.stdout: "dn+": 954, 2024-05-17T22:55:38.426 INFO:teuthology.orchestra.run.smithi151.stdout: "dn-": 455, 2024-05-17T22:55:38.426 INFO:teuthology.orchestra.run.smithi151.stdout: "cap": 0, 2024-05-17T22:55:38.426 INFO:teuthology.orchestra.run.smithi151.stdout: "cap+": 0, 2024-05-17T22:55:38.426 INFO:teuthology.orchestra.run.smithi151.stdout: "cap-": 0, 2024-05-17T22:55:38.426 INFO:teuthology.orchestra.run.smithi151.stdout: "rss": 122420, 2024-05-17T22:55:38.427 INFO:teuthology.orchestra.run.smithi151.stdout: "heap": 215440 2024-05-17T22:55:38.427 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.427 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_server": { 2024-05-17T22:55:38.427 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_client_request": 0, 2024-05-17T22:55:38.427 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_server_request": 0, 2024-05-17T22:55:38.427 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_request": 0, 2024-05-17T22:55:38.427 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_session": 0, 2024-05-17T22:55:38.427 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_peer_request": 0, 2024-05-17T22:55:38.427 INFO:teuthology.orchestra.run.smithi151.stdout: "req_create_latency": { 2024-05-17T22:55:38.427 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:38.427 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:38.427 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:38.427 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.427 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getattr_latency": { 2024-05-17T22:55:38.428 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:38.428 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:38.428 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:38.428 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.428 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getfilelock_latency": { 2024-05-17T22:55:38.428 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:38.428 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:38.428 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:38.428 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.428 INFO:teuthology.orchestra.run.smithi151.stdout: "req_link_latency": { 2024-05-17T22:55:38.428 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:38.428 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:38.428 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:38.428 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.428 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookup_latency": { 2024-05-17T22:55:38.429 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:38.429 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:38.429 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:38.429 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.429 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookuphash_latency": { 2024-05-17T22:55:38.429 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:38.429 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:38.429 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:38.429 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.429 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupino_latency": { 2024-05-17T22:55:38.429 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:38.429 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:38.429 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:38.429 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.429 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupname_latency": { 2024-05-17T22:55:38.430 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:38.430 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:38.430 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:38.430 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.430 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupparent_latency": { 2024-05-17T22:55:38.430 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:38.430 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:38.430 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:38.430 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.430 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupsnap_latency": { 2024-05-17T22:55:38.430 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:38.430 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:38.430 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:38.430 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.431 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lssnap_latency": { 2024-05-17T22:55:38.431 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:38.431 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:38.431 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:38.431 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.431 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mkdir_latency": { 2024-05-17T22:55:38.431 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:38.432 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:38.432 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:38.432 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.432 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mknod_latency": { 2024-05-17T22:55:38.432 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:38.432 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:38.432 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:38.432 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.432 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mksnap_latency": { 2024-05-17T22:55:38.432 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:38.432 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:38.432 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:38.432 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.433 INFO:teuthology.orchestra.run.smithi151.stdout: "req_open_latency": { 2024-05-17T22:55:38.433 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:38.433 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:38.433 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:38.433 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.433 INFO:teuthology.orchestra.run.smithi151.stdout: "req_readdir_latency": { 2024-05-17T22:55:38.433 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:38.433 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:38.433 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:38.433 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.433 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rename_latency": { 2024-05-17T22:55:38.433 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:38.433 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:38.433 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:38.434 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.434 INFO:teuthology.orchestra.run.smithi151.stdout: "req_renamesnap_latency": { 2024-05-17T22:55:38.434 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:38.434 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:38.434 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:38.434 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.434 INFO:teuthology.orchestra.run.smithi151.stdout: "req_snapdiff_latency": { 2024-05-17T22:55:38.434 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:38.434 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:38.434 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:38.434 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.434 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmdir_latency": { 2024-05-17T22:55:38.434 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:38.434 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:38.435 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:38.435 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.435 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmsnap_latency": { 2024-05-17T22:55:38.435 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:38.435 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:38.435 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:38.435 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.435 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmxattr_latency": { 2024-05-17T22:55:38.435 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:38.435 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:38.435 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:38.435 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.435 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setattr_latency": { 2024-05-17T22:55:38.435 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:38.436 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:38.436 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:38.436 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.436 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setdirlayout_latency": { 2024-05-17T22:55:38.436 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:38.436 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:38.436 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:38.436 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.436 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setfilelock_latency": { 2024-05-17T22:55:38.436 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:38.436 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:38.436 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:38.436 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.436 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setlayout_latency": { 2024-05-17T22:55:38.437 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:38.437 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:38.437 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:38.437 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.437 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setxattr_latency": { 2024-05-17T22:55:38.437 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:38.437 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:38.437 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:38.437 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.437 INFO:teuthology.orchestra.run.smithi151.stdout: "req_symlink_latency": { 2024-05-17T22:55:38.437 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:38.437 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:38.437 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:38.437 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.438 INFO:teuthology.orchestra.run.smithi151.stdout: "req_unlink_latency": { 2024-05-17T22:55:38.438 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:38.438 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:38.438 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:38.438 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.438 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_revoke_eviction": 0, 2024-05-17T22:55:38.438 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_acquisition_throttle": 0, 2024-05-17T22:55:38.438 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getvxattr_latency": { 2024-05-17T22:55:38.438 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:38.438 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:38.438 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:38.438 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-17T22:55:38.438 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.439 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_sessions": { 2024-05-17T22:55:38.439 INFO:teuthology.orchestra.run.smithi151.stdout: "session_count": 1, 2024-05-17T22:55:38.439 INFO:teuthology.orchestra.run.smithi151.stdout: "session_add": 2, 2024-05-17T22:55:38.439 INFO:teuthology.orchestra.run.smithi151.stdout: "session_remove": 1, 2024-05-17T22:55:38.439 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_open": 1, 2024-05-17T22:55:38.439 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_stale": 0, 2024-05-17T22:55:38.439 INFO:teuthology.orchestra.run.smithi151.stdout: "total_load": 0, 2024-05-17T22:55:38.439 INFO:teuthology.orchestra.run.smithi151.stdout: "average_load": 0, 2024-05-17T22:55:38.439 INFO:teuthology.orchestra.run.smithi151.stdout: "avg_session_uptime": 2451, 2024-05-17T22:55:38.439 INFO:teuthology.orchestra.run.smithi151.stdout: "mdthresh_evicted": 0 2024-05-17T22:55:38.439 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.439 INFO:teuthology.orchestra.run.smithi151.stdout: "mempool": { 2024-05-17T22:55:38.439 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_bytes": 0, 2024-05-17T22:55:38.439 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_items": 0, 2024-05-17T22:55:38.440 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_bytes": 0, 2024-05-17T22:55:38.440 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_items": 0, 2024-05-17T22:55:38.440 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_bytes": 0, 2024-05-17T22:55:38.440 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_items": 0, 2024-05-17T22:55:38.440 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-17T22:55:38.440 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_items": 0, 2024-05-17T22:55:38.440 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-17T22:55:38.440 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_items": 0, 2024-05-17T22:55:38.440 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_bytes": 0, 2024-05-17T22:55:38.440 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_items": 0, 2024-05-17T22:55:38.440 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-17T22:55:38.440 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_items": 0, 2024-05-17T22:55:38.440 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_bytes": 0, 2024-05-17T22:55:38.440 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_items": 0, 2024-05-17T22:55:38.441 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_bytes": 0, 2024-05-17T22:55:38.441 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_items": 0, 2024-05-17T22:55:38.441 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-17T22:55:38.441 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_items": 0, 2024-05-17T22:55:38.441 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-17T22:55:38.441 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_items": 0, 2024-05-17T22:55:38.441 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_bytes": 0, 2024-05-17T22:55:38.441 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_items": 0, 2024-05-17T22:55:38.441 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_bytes": 0, 2024-05-17T22:55:38.441 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_items": 0, 2024-05-17T22:55:38.441 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-17T22:55:38.441 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_items": 0, 2024-05-17T22:55:38.441 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_bytes": 0, 2024-05-17T22:55:38.441 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_items": 0, 2024-05-17T22:55:38.442 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_bytes": 0, 2024-05-17T22:55:38.442 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_items": 0, 2024-05-17T22:55:38.442 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_bytes": 0, 2024-05-17T22:55:38.442 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_items": 0, 2024-05-17T22:55:38.442 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_bytes": 0, 2024-05-17T22:55:38.442 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_items": 0, 2024-05-17T22:55:38.442 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_bytes": 207786, 2024-05-17T22:55:38.442 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_items": 509, 2024-05-17T22:55:38.442 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_bytes": 0, 2024-05-17T22:55:38.442 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_items": 0, 2024-05-17T22:55:38.442 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_bytes": 0, 2024-05-17T22:55:38.442 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_items": 0, 2024-05-17T22:55:38.442 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_bytes": 0, 2024-05-17T22:55:38.442 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_items": 0, 2024-05-17T22:55:38.442 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_bytes": 0, 2024-05-17T22:55:38.443 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_items": 0, 2024-05-17T22:55:38.443 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_bytes": 8888, 2024-05-17T22:55:38.443 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_items": 170, 2024-05-17T22:55:38.443 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_bytes": 0, 2024-05-17T22:55:38.443 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_items": 0, 2024-05-17T22:55:38.443 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_bytes": 0, 2024-05-17T22:55:38.443 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_items": 0, 2024-05-17T22:55:38.443 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_bytes": 1479153, 2024-05-17T22:55:38.443 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_items": 13250, 2024-05-17T22:55:38.443 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_bytes": 0, 2024-05-17T22:55:38.443 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_items": 0, 2024-05-17T22:55:38.443 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_bytes": 0, 2024-05-17T22:55:38.443 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_items": 0 2024-05-17T22:55:38.443 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.444 INFO:teuthology.orchestra.run.smithi151.stdout: "objecter": { 2024-05-17T22:55:38.444 INFO:teuthology.orchestra.run.smithi151.stdout: "op_active": 0, 2024-05-17T22:55:38.444 INFO:teuthology.orchestra.run.smithi151.stdout: "op_laggy": 0, 2024-05-17T22:55:38.444 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send": 3633, 2024-05-17T22:55:38.444 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send_bytes": 21, 2024-05-17T22:55:38.444 INFO:teuthology.orchestra.run.smithi151.stdout: "op_resend": 3, 2024-05-17T22:55:38.444 INFO:teuthology.orchestra.run.smithi151.stdout: "op_reply": 3633, 2024-05-17T22:55:38.444 INFO:teuthology.orchestra.run.smithi151.stdout: "op_latency": { 2024-05-17T22:55:38.444 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 3633, 2024-05-17T22:55:38.444 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 14.907842823, 2024-05-17T22:55:38.444 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.004103452 2024-05-17T22:55:38.444 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.444 INFO:teuthology.orchestra.run.smithi151.stdout: "op_inflight": 0, 2024-05-17T22:55:38.444 INFO:teuthology.orchestra.run.smithi151.stdout: "oplen_avg": { 2024-05-17T22:55:38.444 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 3633, 2024-05-17T22:55:38.445 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 3634 2024-05-17T22:55:38.445 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.445 INFO:teuthology.orchestra.run.smithi151.stdout: "op": 3633, 2024-05-17T22:55:38.445 INFO:teuthology.orchestra.run.smithi151.stdout: "op_r": 3633, 2024-05-17T22:55:38.445 INFO:teuthology.orchestra.run.smithi151.stdout: "op_w": 0, 2024-05-17T22:55:38.445 INFO:teuthology.orchestra.run.smithi151.stdout: "op_rmw": 0, 2024-05-17T22:55:38.445 INFO:teuthology.orchestra.run.smithi151.stdout: "op_pg": 0, 2024-05-17T22:55:38.445 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_stat": 2265, 2024-05-17T22:55:38.445 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_create": 0, 2024-05-17T22:55:38.445 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_read": 1366, 2024-05-17T22:55:38.445 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_write": 0, 2024-05-17T22:55:38.445 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writefull": 0, 2024-05-17T22:55:38.445 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writesame": 0, 2024-05-17T22:55:38.445 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_append": 0, 2024-05-17T22:55:38.446 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_zero": 0, 2024-05-17T22:55:38.446 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_truncate": 0, 2024-05-17T22:55:38.446 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_delete": 0, 2024-05-17T22:55:38.446 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_mapext": 0, 2024-05-17T22:55:38.446 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_sparse_read": 0, 2024-05-17T22:55:38.446 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_clonerange": 0, 2024-05-17T22:55:38.446 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_getxattr": 1, 2024-05-17T22:55:38.446 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_setxattr": 0, 2024-05-17T22:55:38.446 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_cmpxattr": 0, 2024-05-17T22:55:38.446 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_rmxattr": 0, 2024-05-17T22:55:38.446 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_resetxattrs": 0, 2024-05-17T22:55:38.446 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_call": 0, 2024-05-17T22:55:38.446 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_watch": 0, 2024-05-17T22:55:38.446 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_notify": 0, 2024-05-17T22:55:38.446 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_src_cmpxattr": 0, 2024-05-17T22:55:38.447 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls": 0, 2024-05-17T22:55:38.447 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls_filter": 0, 2024-05-17T22:55:38.447 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_other": 0, 2024-05-17T22:55:38.447 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_active": 0, 2024-05-17T22:55:38.447 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_send": 0, 2024-05-17T22:55:38.447 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_resend": 0, 2024-05-17T22:55:38.447 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_ping": 0, 2024-05-17T22:55:38.447 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_active": 0, 2024-05-17T22:55:38.447 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_send": 0, 2024-05-17T22:55:38.447 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_resend": 0, 2024-05-17T22:55:38.447 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_active": 0, 2024-05-17T22:55:38.447 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_send": 0, 2024-05-17T22:55:38.447 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_resend": 0, 2024-05-17T22:55:38.447 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_active": 0, 2024-05-17T22:55:38.448 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_send": 0, 2024-05-17T22:55:38.448 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_resend": 0, 2024-05-17T22:55:38.448 INFO:teuthology.orchestra.run.smithi151.stdout: "command_active": 0, 2024-05-17T22:55:38.448 INFO:teuthology.orchestra.run.smithi151.stdout: "command_send": 0, 2024-05-17T22:55:38.448 INFO:teuthology.orchestra.run.smithi151.stdout: "command_resend": 0, 2024-05-17T22:55:38.448 INFO:teuthology.orchestra.run.smithi151.stdout: "map_epoch": 89, 2024-05-17T22:55:38.448 INFO:teuthology.orchestra.run.smithi151.stdout: "map_full": 0, 2024-05-17T22:55:38.448 INFO:teuthology.orchestra.run.smithi151.stdout: "map_inc": 2, 2024-05-17T22:55:38.448 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_sessions": 5, 2024-05-17T22:55:38.448 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_open": 6, 2024-05-17T22:55:38.448 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_close": 1, 2024-05-17T22:55:38.448 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_laggy": 0, 2024-05-17T22:55:38.448 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_wr": 0, 2024-05-17T22:55:38.448 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_rd": 2, 2024-05-17T22:55:38.448 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_del": 0 2024-05-17T22:55:38.449 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.449 INFO:teuthology.orchestra.run.smithi151.stdout: "oft": { 2024-05-17T22:55:38.449 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_objs": 0, 2024-05-17T22:55:38.449 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_kv_pairs": 0, 2024-05-17T22:55:38.449 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_updates": 0, 2024-05-17T22:55:38.449 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_removes": 0 2024-05-17T22:55:38.449 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.449 INFO:teuthology.orchestra.run.smithi151.stdout: "osd-slow-ops": { 2024-05-17T22:55:38.449 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_ops_count": 0 2024-05-17T22:55:38.449 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.449 INFO:teuthology.orchestra.run.smithi151.stdout: "purge_queue": { 2024-05-17T22:55:38.449 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops": 0, 2024-05-17T22:55:38.449 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops_high_water": 0, 2024-05-17T22:55:38.449 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing": 0, 2024-05-17T22:55:38.450 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_high_water": 0, 2024-05-17T22:55:38.450 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed_ops": 0, 2024-05-17T22:55:38.450 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed": 0, 2024-05-17T22:55:38.450 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_item_in_journal": 0 2024-05-17T22:55:38.450 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.450 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-17T22:55:38.450 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-05-17T22:55:38.450 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-05-17T22:55:38.450 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-17T22:55:38.450 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 4009, 2024-05-17T22:55:38.450 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 3713578, 2024-05-17T22:55:38.450 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-17T22:55:38.450 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 4009, 2024-05-17T22:55:38.450 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-05-17T22:55:38.450 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-05-17T22:55:38.451 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 4009, 2024-05-17T22:55:38.451 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 3713578, 2024-05-17T22:55:38.451 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-17T22:55:38.451 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:38.451 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:38.451 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:38.451 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-17T22:55:38.451 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.451 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_bytes": { 2024-05-17T22:55:38.451 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-05-17T22:55:38.451 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-05-17T22:55:38.451 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-17T22:55:38.451 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-05-17T22:55:38.451 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-05-17T22:55:38.452 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-17T22:55:38.452 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-05-17T22:55:38.452 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 3633, 2024-05-17T22:55:38.452 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 2417337, 2024-05-17T22:55:38.452 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 3633, 2024-05-17T22:55:38.452 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 2417337, 2024-05-17T22:55:38.452 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-17T22:55:38.452 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:38.452 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:38.452 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:38.452 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-17T22:55:38.452 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.452 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_ops": { 2024-05-17T22:55:38.452 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-05-17T22:55:38.452 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1024, 2024-05-17T22:55:38.453 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-17T22:55:38.453 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-05-17T22:55:38.453 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-05-17T22:55:38.453 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-17T22:55:38.453 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-05-17T22:55:38.453 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 3633, 2024-05-17T22:55:38.453 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 3633, 2024-05-17T22:55:38.453 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 3633, 2024-05-17T22:55:38.453 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 3633, 2024-05-17T22:55:38.453 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-17T22:55:38.453 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:38.453 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:38.453 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:38.453 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-17T22:55:38.454 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.454 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle": { 2024-05-17T22:55:38.454 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-05-17T22:55:38.454 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-05-17T22:55:38.454 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-17T22:55:38.454 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-05-17T22:55:38.454 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-05-17T22:55:38.454 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-17T22:55:38.454 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-05-17T22:55:38.454 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-05-17T22:55:38.454 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-05-17T22:55:38.454 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 0, 2024-05-17T22:55:38.454 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 0, 2024-05-17T22:55:38.454 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-17T22:55:38.455 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:38.455 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:38.455 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:38.455 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-17T22:55:38.455 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:38.455 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle-0x559bf7cc48f0": { 2024-05-17T22:55:38.455 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-05-17T22:55:38.455 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-05-17T22:55:38.455 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-17T22:55:38.455 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-05-17T22:55:38.455 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-05-17T22:55:38.455 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-17T22:55:38.456 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-05-17T22:55:38.456 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-05-17T22:55:38.456 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-05-17T22:55:38.456 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 0, 2024-05-17T22:55:38.456 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 0, 2024-05-17T22:55:38.456 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-17T22:55:38.456 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:38.456 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:38.456 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:38.456 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-17T22:55:38.456 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-17T22:55:38.456 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-17T22:55:38.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:38 smithi096 ceph-mon[36065]: pgmap v930: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-17T22:55:38.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:38 smithi012 ceph-mon[31632]: pgmap v930: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-17T22:55:39.303 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-05-17T22:55:39.303 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-05-17T22:55:39.303 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-05-17T22:55:39.303 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-05-17T22:55:39.303 DEBUG:tasks.check_counter:Getting stats from i 2024-05-17T22:55:39.303 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-17T22:55:39.551 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T22:55:40.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:40 smithi151 ceph-mon[36101]: pgmap v931: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-05-17T22:55:40.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:40 smithi096 ceph-mon[36065]: pgmap v931: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-05-17T22:55:40.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:40 smithi012 ceph-mon[31632]: pgmap v931: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-05-17T22:55:41.476 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-17T22:55:41.477 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-0": { 2024-05-17T22:55:41.477 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 6310, 2024-05-17T22:55:41.477 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 6283, 2024-05-17T22:55:41.477 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 2164633, 2024-05-17T22:55:41.477 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 4633890, 2024-05-17T22:55:41.477 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 39, 2024-05-17T22:55:41.477 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 7, 2024-05-17T22:55:41.477 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.428389776, 2024-05-17T22:55:41.477 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.135771120, 2024-05-17T22:55:41.477 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.128029628, 2024-05-17T22:55:41.477 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.138770173, 2024-05-17T22:55:41.477 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T22:55:41.478 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 6283, 2024-05-17T22:55:41.478 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.224256129, 2024-05-17T22:55:41.478 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000035692 2024-05-17T22:55:41.478 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.478 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-05-17T22:55:41.478 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 30, 2024-05-17T22:55:41.478 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000007059, 2024-05-17T22:55:41.478 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000235 2024-05-17T22:55:41.478 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.478 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 677776, 2024-05-17T22:55:41.478 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 230848 2024-05-17T22:55:41.478 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.478 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-1": { 2024-05-17T22:55:41.479 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 4153, 2024-05-17T22:55:41.479 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 5171, 2024-05-17T22:55:41.479 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 1121263, 2024-05-17T22:55:41.479 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 2748586, 2024-05-17T22:55:41.479 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 14, 2024-05-17T22:55:41.479 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 7, 2024-05-17T22:55:41.479 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.296436136, 2024-05-17T22:55:41.479 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.106697574, 2024-05-17T22:55:41.479 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.124195040, 2024-05-17T22:55:41.479 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.065385565, 2024-05-17T22:55:41.479 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T22:55:41.479 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 5167, 2024-05-17T22:55:41.479 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.249364252, 2024-05-17T22:55:41.479 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000048260 2024-05-17T22:55:41.480 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.480 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-05-17T22:55:41.480 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 1683, 2024-05-17T22:55:41.480 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000865571, 2024-05-17T22:55:41.480 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000514 2024-05-17T22:55:41.480 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.480 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 1744, 2024-05-17T22:55:41.480 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 522272 2024-05-17T22:55:41.480 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.480 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-2": { 2024-05-17T22:55:41.480 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 5198, 2024-05-17T22:55:41.480 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 4198, 2024-05-17T22:55:41.480 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 2411334, 2024-05-17T22:55:41.480 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 1874565, 2024-05-17T22:55:41.481 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 8, 2024-05-17T22:55:41.481 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 5, 2024-05-17T22:55:41.481 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.322958216, 2024-05-17T22:55:41.481 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.106741367, 2024-05-17T22:55:41.481 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.206546021, 2024-05-17T22:55:41.481 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.043555505, 2024-05-17T22:55:41.481 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T22:55:41.481 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 4198, 2024-05-17T22:55:41.481 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.213747808, 2024-05-17T22:55:41.481 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000050916 2024-05-17T22:55:41.481 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.481 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-05-17T22:55:41.481 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 6245, 2024-05-17T22:55:41.481 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.001466282, 2024-05-17T22:55:41.482 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000234 2024-05-17T22:55:41.482 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.482 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-17T22:55:41.482 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 0 2024-05-17T22:55:41.482 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.482 INFO:teuthology.orchestra.run.smithi151.stdout: "cct": { 2024-05-17T22:55:41.482 INFO:teuthology.orchestra.run.smithi151.stdout: "total_workers": 1, 2024-05-17T22:55:41.482 INFO:teuthology.orchestra.run.smithi151.stdout: "unhealthy_workers": 0 2024-05-17T22:55:41.482 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.482 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-MDSRank": { 2024-05-17T22:55:41.482 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-05-17T22:55:41.482 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-05-17T22:55:41.482 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 8853, 2024-05-17T22:55:41.483 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 1.498084437, 2024-05-17T22:55:41.483 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000169217 2024-05-17T22:55:41.483 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-17T22:55:41.483 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.483 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-PurgeQueue": { 2024-05-17T22:55:41.483 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-05-17T22:55:41.483 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-05-17T22:55:41.483 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 82, 2024-05-17T22:55:41.483 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.004295091, 2024-05-17T22:55:41.483 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000052379 2024-05-17T22:55:41.483 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-17T22:55:41.483 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.483 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": { 2024-05-17T22:55:41.483 INFO:teuthology.orchestra.run.smithi151.stdout: "request": 396, 2024-05-17T22:55:41.484 INFO:teuthology.orchestra.run.smithi151.stdout: "reply": 263, 2024-05-17T22:55:41.484 INFO:teuthology.orchestra.run.smithi151.stdout: "reply_latency": { 2024-05-17T22:55:41.484 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 263, 2024-05-17T22:55:41.484 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.252810037, 2024-05-17T22:55:41.484 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000961254 2024-05-17T22:55:41.484 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.484 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_reply": 0, 2024-05-17T22:55:41.484 INFO:teuthology.orchestra.run.smithi151.stdout: "forward": 133, 2024-05-17T22:55:41.484 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_complete": 0, 2024-05-17T22:55:41.484 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_keys": 0, 2024-05-17T22:55:41.484 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_commit": 11, 2024-05-17T22:55:41.484 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_split": 0, 2024-05-17T22:55:41.484 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_merge": 0, 2024-05-17T22:55:41.484 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes": 492, 2024-05-17T22:55:41.485 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_top": 2, 2024-05-17T22:55:41.485 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_bottom": 8, 2024-05-17T22:55:41.485 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pin_tail": 482, 2024-05-17T22:55:41.485 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pinned": 489, 2024-05-17T22:55:41.485 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_expired": 461, 2024-05-17T22:55:41.485 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_with_caps": 444, 2024-05-17T22:55:41.485 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": 444, 2024-05-17T22:55:41.485 INFO:teuthology.orchestra.run.smithi151.stdout: "subtrees": 14, 2024-05-17T22:55:41.485 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse": 724, 2024-05-17T22:55:41.485 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_hit": 521, 2024-05-17T22:55:41.485 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_forward": 132, 2024-05-17T22:55:41.485 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_discover": 28, 2024-05-17T22:55:41.485 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_dir_fetch": 0, 2024-05-17T22:55:41.485 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_remote_ino": 0, 2024-05-17T22:55:41.486 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_lock": 2, 2024-05-17T22:55:41.486 INFO:teuthology.orchestra.run.smithi151.stdout: "load_cent": 1097, 2024-05-17T22:55:41.486 INFO:teuthology.orchestra.run.smithi151.stdout: "q": 1, 2024-05-17T22:55:41.486 INFO:teuthology.orchestra.run.smithi151.stdout: "exported": 13, 2024-05-17T22:55:41.486 INFO:teuthology.orchestra.run.smithi151.stdout: "exported_inodes": 451, 2024-05-17T22:55:41.486 INFO:teuthology.orchestra.run.smithi151.stdout: "imported": 13, 2024-05-17T22:55:41.486 INFO:teuthology.orchestra.run.smithi151.stdout: "imported_inodes": 917, 2024-05-17T22:55:41.486 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_dir_fetch": 0, 2024-05-17T22:55:41.486 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_backtrace_fetch": 9726, 2024-05-17T22:55:41.486 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_peer_discover": 0, 2024-05-17T22:55:41.486 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rfiles": 1024, 2024-05-17T22:55:41.486 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rbytes": 1621914670, 2024-05-17T22:55:41.486 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rsnaps": 0, 2024-05-17T22:55:41.486 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_fetch": 9726, 2024-05-17T22:55:41.487 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_set_tag": 0, 2024-05-17T22:55:41.487 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_repaired": 0, 2024-05-17T22:55:41.487 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_inotable_repaired": 0, 2024-05-17T22:55:41.487 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_inodes": 72, 2024-05-17T22:55:41.487 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_base_inodes": 0, 2024-05-17T22:55:41.487 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dirfrag_rstats": 72, 2024-05-17T22:55:41.487 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_file_inodes": 9654, 2024-05-17T22:55:41.487 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_inode_file_caps": 0, 2024-05-17T22:55:41.487 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_revoke": 12, 2024-05-17T22:55:41.487 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_grant": 257, 2024-05-17T22:55:41.487 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_trunc": 0, 2024-05-17T22:55:41.487 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-17T22:55:41.487 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-17T22:55:41.487 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps": 626, 2024-05-17T22:55:41.488 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps_dirty": 296, 2024-05-17T22:55:41.488 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_cap_release": 26, 2024-05-17T22:55:41.488 INFO:teuthology.orchestra.run.smithi151.stdout: "process_request_cap_release": 77 2024-05-17T22:55:41.488 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.488 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_cache": { 2024-05-17T22:55:41.488 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update": 40, 2024-05-17T22:55:41.488 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update_receipt": 194, 2024-05-17T22:55:41.488 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_try_discover": 110, 2024-05-17T22:55:41.488 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_send_discover": 148, 2024-05-17T22:55:41.488 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_handle_discover": 46, 2024-05-17T22:55:41.488 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays": 0, 2024-05-17T22:55:41.488 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_delayed": 0, 2024-05-17T22:55:41.488 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_enqueuing": 0, 2024-05-17T22:55:41.488 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_created": 19, 2024-05-17T22:55:41.489 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_enqueued": 19, 2024-05-17T22:55:41.489 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_reintegrated": 0, 2024-05-17T22:55:41.489 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_migrated": 0, 2024-05-17T22:55:41.489 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_processing": 0, 2024-05-17T22:55:41.489 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_enqueued": 0, 2024-05-17T22:55:41.489 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_prioritized": 0, 2024-05-17T22:55:41.489 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_started": 0, 2024-05-17T22:55:41.489 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_completed": 0, 2024-05-17T22:55:41.489 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_path": 0, 2024-05-17T22:55:41.489 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_inode": 0, 2024-05-17T22:55:41.489 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_enqueue_scrub": 0, 2024-05-17T22:55:41.489 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_exportdir": 13, 2024-05-17T22:55:41.489 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_flush": 0, 2024-05-17T22:55:41.489 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragmentdir": 0, 2024-05-17T22:55:41.490 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragstats": 0, 2024-05-17T22:55:41.490 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_inodestats": 0 2024-05-17T22:55:41.490 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.490 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_log": { 2024-05-17T22:55:41.490 INFO:teuthology.orchestra.run.smithi151.stdout: "evlrg": 0, 2024-05-17T22:55:41.490 INFO:teuthology.orchestra.run.smithi151.stdout: "evadd": 1082, 2024-05-17T22:55:41.490 INFO:teuthology.orchestra.run.smithi151.stdout: "evex": 0, 2024-05-17T22:55:41.490 INFO:teuthology.orchestra.run.smithi151.stdout: "evtrm": 0, 2024-05-17T22:55:41.490 INFO:teuthology.orchestra.run.smithi151.stdout: "ev": 1082, 2024-05-17T22:55:41.490 INFO:teuthology.orchestra.run.smithi151.stdout: "evexg": 0, 2024-05-17T22:55:41.490 INFO:teuthology.orchestra.run.smithi151.stdout: "evexd": 0, 2024-05-17T22:55:41.490 INFO:teuthology.orchestra.run.smithi151.stdout: "segadd": 3, 2024-05-17T22:55:41.490 INFO:teuthology.orchestra.run.smithi151.stdout: "segex": 0, 2024-05-17T22:55:41.490 INFO:teuthology.orchestra.run.smithi151.stdout: "segtrm": 0, 2024-05-17T22:55:41.491 INFO:teuthology.orchestra.run.smithi151.stdout: "seg": 3, 2024-05-17T22:55:41.491 INFO:teuthology.orchestra.run.smithi151.stdout: "segmjr": 2, 2024-05-17T22:55:41.491 INFO:teuthology.orchestra.run.smithi151.stdout: "segexg": 0, 2024-05-17T22:55:41.491 INFO:teuthology.orchestra.run.smithi151.stdout: "segexd": 0, 2024-05-17T22:55:41.491 INFO:teuthology.orchestra.run.smithi151.stdout: "expos": 4194304, 2024-05-17T22:55:41.491 INFO:teuthology.orchestra.run.smithi151.stdout: "wrpos": 6615352, 2024-05-17T22:55:41.491 INFO:teuthology.orchestra.run.smithi151.stdout: "rdpos": 0, 2024-05-17T22:55:41.491 INFO:teuthology.orchestra.run.smithi151.stdout: "jlat": { 2024-05-17T22:55:41.491 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 349, 2024-05-17T22:55:41.491 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 6.332224737, 2024-05-17T22:55:41.491 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.018143910 2024-05-17T22:55:41.491 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.491 INFO:teuthology.orchestra.run.smithi151.stdout: "replayed": 0 2024-05-17T22:55:41.491 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.492 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_mem": { 2024-05-17T22:55:41.492 INFO:teuthology.orchestra.run.smithi151.stdout: "ino": 479, 2024-05-17T22:55:41.492 INFO:teuthology.orchestra.run.smithi151.stdout: "ino+": 951, 2024-05-17T22:55:41.492 INFO:teuthology.orchestra.run.smithi151.stdout: "ino-": 472, 2024-05-17T22:55:41.492 INFO:teuthology.orchestra.run.smithi151.stdout: "dir": 51, 2024-05-17T22:55:41.492 INFO:teuthology.orchestra.run.smithi151.stdout: "dir+": 109, 2024-05-17T22:55:41.492 INFO:teuthology.orchestra.run.smithi151.stdout: "dir-": 58, 2024-05-17T22:55:41.492 INFO:teuthology.orchestra.run.smithi151.stdout: "dn": 492, 2024-05-17T22:55:41.492 INFO:teuthology.orchestra.run.smithi151.stdout: "dn+": 975, 2024-05-17T22:55:41.492 INFO:teuthology.orchestra.run.smithi151.stdout: "dn-": 483, 2024-05-17T22:55:41.492 INFO:teuthology.orchestra.run.smithi151.stdout: "cap": 70, 2024-05-17T22:55:41.492 INFO:teuthology.orchestra.run.smithi151.stdout: "cap+": 1259, 2024-05-17T22:55:41.492 INFO:teuthology.orchestra.run.smithi151.stdout: "cap-": 1189, 2024-05-17T22:55:41.492 INFO:teuthology.orchestra.run.smithi151.stdout: "rss": 131224, 2024-05-17T22:55:41.492 INFO:teuthology.orchestra.run.smithi151.stdout: "heap": 223632 2024-05-17T22:55:41.493 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.493 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_server": { 2024-05-17T22:55:41.493 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_client_request": 451, 2024-05-17T22:55:41.493 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_server_request": 0, 2024-05-17T22:55:41.493 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_request": 396, 2024-05-17T22:55:41.493 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_session": 35, 2024-05-17T22:55:41.493 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_peer_request": 24, 2024-05-17T22:55:41.493 INFO:teuthology.orchestra.run.smithi151.stdout: "req_create_latency": { 2024-05-17T22:55:41.493 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 9, 2024-05-17T22:55:41.493 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.052589761, 2024-05-17T22:55:41.493 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.005843306 2024-05-17T22:55:41.493 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.493 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getattr_latency": { 2024-05-17T22:55:41.493 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 1, 2024-05-17T22:55:41.494 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000309833, 2024-05-17T22:55:41.494 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000309833 2024-05-17T22:55:41.494 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.494 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getfilelock_latency": { 2024-05-17T22:55:41.494 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:41.494 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:41.494 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:41.494 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.494 INFO:teuthology.orchestra.run.smithi151.stdout: "req_link_latency": { 2024-05-17T22:55:41.494 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:41.494 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:41.494 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:41.494 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.494 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookup_latency": { 2024-05-17T22:55:41.495 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 171, 2024-05-17T22:55:41.495 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.086266272, 2024-05-17T22:55:41.495 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000504481 2024-05-17T22:55:41.495 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.495 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookuphash_latency": { 2024-05-17T22:55:41.495 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:41.495 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:41.495 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:41.495 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.495 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupino_latency": { 2024-05-17T22:55:41.495 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:41.495 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:41.495 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:41.495 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.496 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupname_latency": { 2024-05-17T22:55:41.496 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:41.496 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:41.496 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:41.496 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.496 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupparent_latency": { 2024-05-17T22:55:41.496 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:41.496 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:41.496 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:41.496 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.496 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupsnap_latency": { 2024-05-17T22:55:41.496 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:41.496 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:41.496 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:41.497 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.497 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lssnap_latency": { 2024-05-17T22:55:41.497 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:41.497 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:41.498 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:41.498 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.498 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mkdir_latency": { 2024-05-17T22:55:41.498 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:41.498 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:41.498 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:41.498 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.498 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mknod_latency": { 2024-05-17T22:55:41.498 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:41.498 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:41.498 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:41.498 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.498 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mksnap_latency": { 2024-05-17T22:55:41.499 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:41.499 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:41.499 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:41.499 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.499 INFO:teuthology.orchestra.run.smithi151.stdout: "req_open_latency": { 2024-05-17T22:55:41.499 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 34, 2024-05-17T22:55:41.499 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.032394131, 2024-05-17T22:55:41.499 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000952768 2024-05-17T22:55:41.499 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.499 INFO:teuthology.orchestra.run.smithi151.stdout: "req_readdir_latency": { 2024-05-17T22:55:41.499 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 29, 2024-05-17T22:55:41.499 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.041308746, 2024-05-17T22:55:41.499 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.001424439 2024-05-17T22:55:41.499 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.500 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rename_latency": { 2024-05-17T22:55:41.500 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 2, 2024-05-17T22:55:41.500 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.005556815, 2024-05-17T22:55:41.500 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.002778407 2024-05-17T22:55:41.500 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.500 INFO:teuthology.orchestra.run.smithi151.stdout: "req_renamesnap_latency": { 2024-05-17T22:55:41.500 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:41.500 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:41.500 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:41.500 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.500 INFO:teuthology.orchestra.run.smithi151.stdout: "req_snapdiff_latency": { 2024-05-17T22:55:41.500 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:41.500 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:41.500 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:41.501 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.501 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmdir_latency": { 2024-05-17T22:55:41.501 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:41.501 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:41.501 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:41.501 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.501 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmsnap_latency": { 2024-05-17T22:55:41.501 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:41.501 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:41.501 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:41.501 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.501 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmxattr_latency": { 2024-05-17T22:55:41.501 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:41.501 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:41.502 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:41.502 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.502 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setattr_latency": { 2024-05-17T22:55:41.502 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:41.502 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:41.502 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:41.502 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.502 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setdirlayout_latency": { 2024-05-17T22:55:41.502 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:41.502 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:41.502 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:41.502 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.502 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setfilelock_latency": { 2024-05-17T22:55:41.502 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:41.503 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:41.503 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:41.503 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.503 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setlayout_latency": { 2024-05-17T22:55:41.503 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:41.503 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:41.503 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:41.503 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.503 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setxattr_latency": { 2024-05-17T22:55:41.503 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:41.503 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:41.503 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:41.503 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.503 INFO:teuthology.orchestra.run.smithi151.stdout: "req_symlink_latency": { 2024-05-17T22:55:41.504 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:41.504 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:41.504 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:41.504 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.504 INFO:teuthology.orchestra.run.smithi151.stdout: "req_unlink_latency": { 2024-05-17T22:55:41.504 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 17, 2024-05-17T22:55:41.504 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.034384479, 2024-05-17T22:55:41.504 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.002022616 2024-05-17T22:55:41.504 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.504 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_revoke_eviction": 0, 2024-05-17T22:55:41.504 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_acquisition_throttle": 0, 2024-05-17T22:55:41.504 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getvxattr_latency": { 2024-05-17T22:55:41.504 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:41.504 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:41.505 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:41.505 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-17T22:55:41.505 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.505 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_sessions": { 2024-05-17T22:55:41.505 INFO:teuthology.orchestra.run.smithi151.stdout: "session_count": 1, 2024-05-17T22:55:41.505 INFO:teuthology.orchestra.run.smithi151.stdout: "session_add": 2, 2024-05-17T22:55:41.505 INFO:teuthology.orchestra.run.smithi151.stdout: "session_remove": 1, 2024-05-17T22:55:41.505 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_open": 1, 2024-05-17T22:55:41.505 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_stale": 0, 2024-05-17T22:55:41.505 INFO:teuthology.orchestra.run.smithi151.stdout: "total_load": 34, 2024-05-17T22:55:41.505 INFO:teuthology.orchestra.run.smithi151.stdout: "average_load": 34, 2024-05-17T22:55:41.505 INFO:teuthology.orchestra.run.smithi151.stdout: "avg_session_uptime": 981, 2024-05-17T22:55:41.505 INFO:teuthology.orchestra.run.smithi151.stdout: "mdthresh_evicted": 0 2024-05-17T22:55:41.505 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.506 INFO:teuthology.orchestra.run.smithi151.stdout: "mempool": { 2024-05-17T22:55:41.506 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_bytes": 0, 2024-05-17T22:55:41.506 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_items": 0, 2024-05-17T22:55:41.506 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_bytes": 0, 2024-05-17T22:55:41.506 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_items": 0, 2024-05-17T22:55:41.506 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_bytes": 0, 2024-05-17T22:55:41.506 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_items": 0, 2024-05-17T22:55:41.506 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-17T22:55:41.506 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_items": 0, 2024-05-17T22:55:41.506 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-17T22:55:41.506 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_items": 0, 2024-05-17T22:55:41.506 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_bytes": 0, 2024-05-17T22:55:41.506 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_items": 0, 2024-05-17T22:55:41.506 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-17T22:55:41.507 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_items": 0, 2024-05-17T22:55:41.507 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_bytes": 0, 2024-05-17T22:55:41.507 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_items": 0, 2024-05-17T22:55:41.507 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_bytes": 0, 2024-05-17T22:55:41.507 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_items": 0, 2024-05-17T22:55:41.507 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-17T22:55:41.507 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_items": 0, 2024-05-17T22:55:41.507 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-17T22:55:41.507 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_items": 0, 2024-05-17T22:55:41.507 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_bytes": 0, 2024-05-17T22:55:41.507 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_items": 0, 2024-05-17T22:55:41.507 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_bytes": 0, 2024-05-17T22:55:41.507 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_items": 0, 2024-05-17T22:55:41.508 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-17T22:55:41.508 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_items": 0, 2024-05-17T22:55:41.508 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_bytes": 0, 2024-05-17T22:55:41.508 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_items": 0, 2024-05-17T22:55:41.508 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_bytes": 0, 2024-05-17T22:55:41.508 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_items": 0, 2024-05-17T22:55:41.508 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_bytes": 0, 2024-05-17T22:55:41.508 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_items": 0, 2024-05-17T22:55:41.508 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_bytes": 0, 2024-05-17T22:55:41.508 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_items": 0, 2024-05-17T22:55:41.508 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_bytes": 308840, 2024-05-17T22:55:41.508 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_items": 581, 2024-05-17T22:55:41.508 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_bytes": 88, 2024-05-17T22:55:41.508 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_items": 1, 2024-05-17T22:55:41.509 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_bytes": 0, 2024-05-17T22:55:41.509 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_items": 0, 2024-05-17T22:55:41.509 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_bytes": 0, 2024-05-17T22:55:41.509 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_items": 0, 2024-05-17T22:55:41.509 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_bytes": 0, 2024-05-17T22:55:41.509 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_items": 0, 2024-05-17T22:55:41.509 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_bytes": 8888, 2024-05-17T22:55:41.509 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_items": 170, 2024-05-17T22:55:41.509 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_bytes": 0, 2024-05-17T22:55:41.509 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_items": 0, 2024-05-17T22:55:41.509 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_bytes": 0, 2024-05-17T22:55:41.510 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_items": 0, 2024-05-17T22:55:41.510 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_bytes": 1714223, 2024-05-17T22:55:41.510 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_items": 17108, 2024-05-17T22:55:41.510 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_bytes": 0, 2024-05-17T22:55:41.510 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_items": 0, 2024-05-17T22:55:41.510 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_bytes": 0, 2024-05-17T22:55:41.510 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_items": 0 2024-05-17T22:55:41.510 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.510 INFO:teuthology.orchestra.run.smithi151.stdout: "objecter": { 2024-05-17T22:55:41.510 INFO:teuthology.orchestra.run.smithi151.stdout: "op_active": 0, 2024-05-17T22:55:41.510 INFO:teuthology.orchestra.run.smithi151.stdout: "op_laggy": 0, 2024-05-17T22:55:41.510 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send": 10303, 2024-05-17T22:55:41.510 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send_bytes": 2535000, 2024-05-17T22:55:41.511 INFO:teuthology.orchestra.run.smithi151.stdout: "op_resend": 33, 2024-05-17T22:55:41.511 INFO:teuthology.orchestra.run.smithi151.stdout: "op_reply": 10303, 2024-05-17T22:55:41.511 INFO:teuthology.orchestra.run.smithi151.stdout: "op_latency": { 2024-05-17T22:55:41.511 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 10303, 2024-05-17T22:55:41.511 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 32.155661872, 2024-05-17T22:55:41.511 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.003120999 2024-05-17T22:55:41.511 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.511 INFO:teuthology.orchestra.run.smithi151.stdout: "op_inflight": 0, 2024-05-17T22:55:41.512 INFO:teuthology.orchestra.run.smithi151.stdout: "oplen_avg": { 2024-05-17T22:55:41.512 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 10303, 2024-05-17T22:55:41.512 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 10397 2024-05-17T22:55:41.512 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.512 INFO:teuthology.orchestra.run.smithi151.stdout: "op": 10303, 2024-05-17T22:55:41.512 INFO:teuthology.orchestra.run.smithi151.stdout: "op_r": 9735, 2024-05-17T22:55:41.512 INFO:teuthology.orchestra.run.smithi151.stdout: "op_w": 568, 2024-05-17T22:55:41.512 INFO:teuthology.orchestra.run.smithi151.stdout: "op_rmw": 0, 2024-05-17T22:55:41.512 INFO:teuthology.orchestra.run.smithi151.stdout: "op_pg": 0, 2024-05-17T22:55:41.512 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_stat": 0, 2024-05-17T22:55:41.512 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_create": 10, 2024-05-17T22:55:41.512 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_read": 9, 2024-05-17T22:55:41.512 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_write": 358, 2024-05-17T22:55:41.513 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writefull": 72, 2024-05-17T22:55:41.513 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writesame": 0, 2024-05-17T22:55:41.513 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_append": 0, 2024-05-17T22:55:41.513 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_zero": 0, 2024-05-17T22:55:41.513 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_truncate": 0, 2024-05-17T22:55:41.513 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_delete": 50, 2024-05-17T22:55:41.513 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_mapext": 0, 2024-05-17T22:55:41.513 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_sparse_read": 0, 2024-05-17T22:55:41.513 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_clonerange": 0, 2024-05-17T22:55:41.513 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_getxattr": 9726, 2024-05-17T22:55:41.513 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_setxattr": 20, 2024-05-17T22:55:41.513 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_cmpxattr": 0, 2024-05-17T22:55:41.513 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_rmxattr": 0, 2024-05-17T22:55:41.514 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_resetxattrs": 0, 2024-05-17T22:55:41.514 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_call": 0, 2024-05-17T22:55:41.514 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_watch": 0, 2024-05-17T22:55:41.514 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_notify": 0, 2024-05-17T22:55:41.514 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_src_cmpxattr": 0, 2024-05-17T22:55:41.514 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls": 0, 2024-05-17T22:55:41.514 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls_filter": 0, 2024-05-17T22:55:41.514 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_other": 0, 2024-05-17T22:55:41.514 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_active": 0, 2024-05-17T22:55:41.514 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_send": 0, 2024-05-17T22:55:41.514 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_resend": 0, 2024-05-17T22:55:41.514 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_ping": 0, 2024-05-17T22:55:41.514 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_active": 0, 2024-05-17T22:55:41.514 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_send": 0, 2024-05-17T22:55:41.515 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_resend": 0, 2024-05-17T22:55:41.515 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_active": 0, 2024-05-17T22:55:41.515 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_send": 0, 2024-05-17T22:55:41.515 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_resend": 0, 2024-05-17T22:55:41.515 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_active": 0, 2024-05-17T22:55:41.515 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_send": 0, 2024-05-17T22:55:41.515 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_resend": 0, 2024-05-17T22:55:41.515 INFO:teuthology.orchestra.run.smithi151.stdout: "command_active": 0, 2024-05-17T22:55:41.515 INFO:teuthology.orchestra.run.smithi151.stdout: "command_send": 0, 2024-05-17T22:55:41.515 INFO:teuthology.orchestra.run.smithi151.stdout: "command_resend": 0, 2024-05-17T22:55:41.515 INFO:teuthology.orchestra.run.smithi151.stdout: "map_epoch": 89, 2024-05-17T22:55:41.515 INFO:teuthology.orchestra.run.smithi151.stdout: "map_full": 0, 2024-05-17T22:55:41.515 INFO:teuthology.orchestra.run.smithi151.stdout: "map_inc": 2, 2024-05-17T22:55:41.515 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_sessions": 12, 2024-05-17T22:55:41.516 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_open": 12, 2024-05-17T22:55:41.516 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_close": 0, 2024-05-17T22:55:41.516 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_laggy": 0, 2024-05-17T22:55:41.516 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_wr": 118, 2024-05-17T22:55:41.516 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_rd": 0, 2024-05-17T22:55:41.516 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_del": 34 2024-05-17T22:55:41.516 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.516 INFO:teuthology.orchestra.run.smithi151.stdout: "oft": { 2024-05-17T22:55:41.516 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_objs": 1, 2024-05-17T22:55:41.516 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_kv_pairs": 0, 2024-05-17T22:55:41.516 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_updates": 0, 2024-05-17T22:55:41.516 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_removes": 0 2024-05-17T22:55:41.516 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.516 INFO:teuthology.orchestra.run.smithi151.stdout: "osd-slow-ops": { 2024-05-17T22:55:41.517 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_ops_count": 0 2024-05-17T22:55:41.517 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.517 INFO:teuthology.orchestra.run.smithi151.stdout: "purge_queue": { 2024-05-17T22:55:41.517 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops": 0, 2024-05-17T22:55:41.517 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops_high_water": 16, 2024-05-17T22:55:41.517 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing": 0, 2024-05-17T22:55:41.517 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_high_water": 8, 2024-05-17T22:55:41.517 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed_ops": 38, 2024-05-17T22:55:41.517 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed": 19, 2024-05-17T22:55:41.517 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_item_in_journal": 0 2024-05-17T22:55:41.517 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.517 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-17T22:55:41.517 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-05-17T22:55:41.518 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-05-17T22:55:41.518 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-17T22:55:41.518 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 15657, 2024-05-17T22:55:41.518 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 4281240, 2024-05-17T22:55:41.518 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-17T22:55:41.518 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 15657, 2024-05-17T22:55:41.518 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-05-17T22:55:41.518 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-05-17T22:55:41.518 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 15657, 2024-05-17T22:55:41.518 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 4281240, 2024-05-17T22:55:41.518 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-17T22:55:41.518 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:41.518 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:41.518 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:41.519 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-17T22:55:41.519 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.519 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_bytes": { 2024-05-17T22:55:41.519 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-05-17T22:55:41.519 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-05-17T22:55:41.519 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-17T22:55:41.519 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-05-17T22:55:41.519 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-05-17T22:55:41.519 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-17T22:55:41.519 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-05-17T22:55:41.519 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 10303, 2024-05-17T22:55:41.519 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 2537071, 2024-05-17T22:55:41.519 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 10303, 2024-05-17T22:55:41.519 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 2537071, 2024-05-17T22:55:41.520 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-17T22:55:41.520 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:41.520 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:41.520 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:41.520 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-17T22:55:41.520 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.520 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_ops": { 2024-05-17T22:55:41.520 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-05-17T22:55:41.520 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1024, 2024-05-17T22:55:41.520 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-17T22:55:41.520 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-05-17T22:55:41.520 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-05-17T22:55:41.520 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-17T22:55:41.520 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-05-17T22:55:41.521 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 10303, 2024-05-17T22:55:41.521 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 10303, 2024-05-17T22:55:41.521 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 10303, 2024-05-17T22:55:41.521 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 10303, 2024-05-17T22:55:41.521 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-17T22:55:41.521 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:41.521 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:41.521 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:41.521 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-17T22:55:41.521 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.521 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle": { 2024-05-17T22:55:41.521 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-05-17T22:55:41.521 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-05-17T22:55:41.521 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-17T22:55:41.521 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 19, 2024-05-17T22:55:41.522 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 2071, 2024-05-17T22:55:41.522 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-17T22:55:41.522 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 19, 2024-05-17T22:55:41.522 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-05-17T22:55:41.522 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-05-17T22:55:41.522 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 9, 2024-05-17T22:55:41.522 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 2071, 2024-05-17T22:55:41.522 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-17T22:55:41.523 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:41.523 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:41.523 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:41.523 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-17T22:55:41.523 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:41.523 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle-0x561a5093e820": { 2024-05-17T22:55:41.523 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 3716, 2024-05-17T22:55:41.523 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-05-17T22:55:41.523 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-17T22:55:41.523 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 1082, 2024-05-17T22:55:41.523 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 2421048, 2024-05-17T22:55:41.524 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-17T22:55:41.524 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 1082, 2024-05-17T22:55:41.524 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-05-17T22:55:41.524 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-05-17T22:55:41.524 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 349, 2024-05-17T22:55:41.524 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 2417332, 2024-05-17T22:55:41.524 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-17T22:55:41.524 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:41.524 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:41.524 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:41.524 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-17T22:55:41.524 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-17T22:55:41.524 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-17T22:55:42.284 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:42 smithi151 ceph-mon[36101]: pgmap v932: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-05-17T22:55:42.286 INFO:tasks.check_counter:Daemon mds.i mds.exported=13 2024-05-17T22:55:42.286 INFO:tasks.check_counter:Daemon mds.i mds.imported=13 2024-05-17T22:55:42.286 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=40 2024-05-17T22:55:42.286 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=194 2024-05-17T22:55:42.286 DEBUG:tasks.check_counter:Getting stats from l 2024-05-17T22:55:42.287 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-17T22:55:42.538 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T22:55:42.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:42 smithi096 ceph-mon[36065]: pgmap v932: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-05-17T22:55:42.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:42 smithi012 ceph-mon[31632]: pgmap v932: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-05-17T22:55:44.418 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:44 smithi151 ceph-mon[36101]: pgmap v933: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.8 KiB/s wr, 6 op/s 2024-05-17T22:55:44.480 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-17T22:55:44.480 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-0": { 2024-05-17T22:55:44.480 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 0, 2024-05-17T22:55:44.480 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 2, 2024-05-17T22:55:44.480 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 0, 2024-05-17T22:55:44.481 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 0, 2024-05-17T22:55:44.481 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 4, 2024-05-17T22:55:44.481 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 0, 2024-05-17T22:55:44.481 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.001153917, 2024-05-17T22:55:44.481 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.000000000, 2024-05-17T22:55:44.481 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.000306357, 2024-05-17T22:55:44.481 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T22:55:44.481 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T22:55:44.481 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:44.481 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:44.481 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:44.481 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:44.482 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-05-17T22:55:44.482 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:44.482 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:44.482 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:44.482 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:44.482 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-17T22:55:44.482 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 0 2024-05-17T22:55:44.483 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:44.483 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-1": { 2024-05-17T22:55:44.483 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 357, 2024-05-17T22:55:44.483 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 352, 2024-05-17T22:55:44.483 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 555248, 2024-05-17T22:55:44.483 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 216544, 2024-05-17T22:55:44.483 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 2, 2024-05-17T22:55:44.483 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 1, 2024-05-17T22:55:44.483 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.049405192, 2024-05-17T22:55:44.483 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.022123186, 2024-05-17T22:55:44.483 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.033672663, 2024-05-17T22:55:44.484 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T22:55:44.484 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T22:55:44.484 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 352, 2024-05-17T22:55:44.484 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.027614274, 2024-05-17T22:55:44.484 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000078449 2024-05-17T22:55:44.484 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:44.484 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-05-17T22:55:44.484 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:44.484 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:44.484 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:44.484 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:44.485 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 555248, 2024-05-17T22:55:44.485 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 216544 2024-05-17T22:55:44.485 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:44.485 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-2": { 2024-05-17T22:55:44.485 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 2, 2024-05-17T22:55:44.485 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 267, 2024-05-17T22:55:44.485 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 320, 2024-05-17T22:55:44.485 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 204688, 2024-05-17T22:55:44.485 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 4, 2024-05-17T22:55:44.485 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 2, 2024-05-17T22:55:44.485 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.014728129, 2024-05-17T22:55:44.485 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.010823532, 2024-05-17T22:55:44.485 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.001435825, 2024-05-17T22:55:44.485 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T22:55:44.486 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T22:55:44.486 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 265, 2024-05-17T22:55:44.486 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.023787451, 2024-05-17T22:55:44.486 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000089763 2024-05-17T22:55:44.486 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:44.486 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-05-17T22:55:44.486 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:44.486 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:44.486 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:44.486 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:44.487 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-17T22:55:44.487 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 204688 2024-05-17T22:55:44.487 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:44.487 INFO:teuthology.orchestra.run.smithi151.stdout: "cct": { 2024-05-17T22:55:44.487 INFO:teuthology.orchestra.run.smithi151.stdout: "total_workers": 0, 2024-05-17T22:55:44.487 INFO:teuthology.orchestra.run.smithi151.stdout: "unhealthy_workers": 0 2024-05-17T22:55:44.487 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:44.487 INFO:teuthology.orchestra.run.smithi151.stdout: "mempool": { 2024-05-17T22:55:44.487 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_bytes": 0, 2024-05-17T22:55:44.487 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_items": 0, 2024-05-17T22:55:44.487 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_bytes": 0, 2024-05-17T22:55:44.488 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_items": 0, 2024-05-17T22:55:44.488 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_bytes": 0, 2024-05-17T22:55:44.488 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_items": 0, 2024-05-17T22:55:44.488 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-17T22:55:44.488 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_items": 0, 2024-05-17T22:55:44.488 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-17T22:55:44.488 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_items": 0, 2024-05-17T22:55:44.488 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_bytes": 0, 2024-05-17T22:55:44.488 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_items": 0, 2024-05-17T22:55:44.488 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-17T22:55:44.488 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_items": 0, 2024-05-17T22:55:44.489 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_bytes": 0, 2024-05-17T22:55:44.489 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_items": 0, 2024-05-17T22:55:44.489 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_bytes": 0, 2024-05-17T22:55:44.489 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_items": 0, 2024-05-17T22:55:44.489 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-17T22:55:44.489 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_items": 0, 2024-05-17T22:55:44.490 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-17T22:55:44.490 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_items": 0, 2024-05-17T22:55:44.490 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_bytes": 0, 2024-05-17T22:55:44.490 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_items": 0, 2024-05-17T22:55:44.490 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_bytes": 0, 2024-05-17T22:55:44.490 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_items": 0, 2024-05-17T22:55:44.491 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-17T22:55:44.491 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_items": 0, 2024-05-17T22:55:44.493 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_bytes": 0, 2024-05-17T22:55:44.493 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_items": 0, 2024-05-17T22:55:44.494 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_bytes": 0, 2024-05-17T22:55:44.494 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_items": 0, 2024-05-17T22:55:44.494 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_bytes": 0, 2024-05-17T22:55:44.494 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_items": 0, 2024-05-17T22:55:44.494 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_bytes": 0, 2024-05-17T22:55:44.494 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_items": 0, 2024-05-17T22:55:44.494 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_bytes": 148704, 2024-05-17T22:55:44.494 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_items": 29, 2024-05-17T22:55:44.494 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_bytes": 0, 2024-05-17T22:55:44.495 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_items": 0, 2024-05-17T22:55:44.495 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_bytes": 0, 2024-05-17T22:55:44.495 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_items": 0, 2024-05-17T22:55:44.495 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_bytes": 0, 2024-05-17T22:55:44.495 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_items": 0, 2024-05-17T22:55:44.495 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_bytes": 0, 2024-05-17T22:55:44.495 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_items": 0, 2024-05-17T22:55:44.495 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_bytes": 0, 2024-05-17T22:55:44.495 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_items": 0, 2024-05-17T22:55:44.495 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_bytes": 0, 2024-05-17T22:55:44.495 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_items": 0, 2024-05-17T22:55:44.495 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_bytes": 0, 2024-05-17T22:55:44.495 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_items": 0, 2024-05-17T22:55:44.495 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_bytes": 928, 2024-05-17T22:55:44.495 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_items": 2, 2024-05-17T22:55:44.496 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_bytes": 0, 2024-05-17T22:55:44.496 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_items": 0, 2024-05-17T22:55:44.496 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_bytes": 0, 2024-05-17T22:55:44.496 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_items": 0 2024-05-17T22:55:44.496 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-17T22:55:44.496 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-17T22:55:44.496 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-05-17T22:55:44.496 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-05-17T22:55:44.496 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-17T22:55:44.496 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 355, 2024-05-17T22:55:44.497 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 502338, 2024-05-17T22:55:44.497 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-17T22:55:44.497 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 355, 2024-05-17T22:55:44.497 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-05-17T22:55:44.497 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-05-17T22:55:44.497 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 355, 2024-05-17T22:55:44.497 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 502338, 2024-05-17T22:55:44.497 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-17T22:55:44.497 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-17T22:55:44.497 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-17T22:55:44.497 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-17T22:55:44.498 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-17T22:55:44.498 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-17T22:55:44.498 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-17T22:55:44.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:44 smithi096 ceph-mon[36065]: pgmap v933: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.8 KiB/s wr, 6 op/s 2024-05-17T22:55:44.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:44 smithi012 ceph-mon[31632]: pgmap v933: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.8 KiB/s wr, 6 op/s 2024-05-17T22:55:45.226 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-05-17T22:55:45.226 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-05-17T22:55:45.227 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-05-17T22:55:45.227 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-05-17T22:55:45.227 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-17T22:55:45.237 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-17T22:55:45.237 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-17T22:55:45.237 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:55:45.266 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-05-17T22:55:45.266 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:55:45.326 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph osd blocklist ls 2024-05-17T22:55:45.629 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:55:46.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:46 smithi012 ceph-mon[31632]: pgmap v934: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.4 KiB/s wr, 7 op/s 2024-05-17T22:55:46.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:46 smithi151 ceph-mon[36101]: pgmap v934: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.4 KiB/s wr, 7 op/s 2024-05-17T22:55:46.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:46 smithi096 ceph-mon[36065]: pgmap v934: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.4 KiB/s wr, 7 op/s 2024-05-17T22:55:47.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.571+0000 7fbbc7029700 1 -- 172.21.15.12:0/1818581253 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbc0107750 msgr2=0x7fbbc0107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:47.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.571+0000 7fbbc7029700 1 --2- 172.21.15.12:0/1818581253 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbc0107750 0x7fbbc0107b60 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0009990 tx=0x7fbbb0009ca0 comp rx=0 tx=0).stop 2024-05-17T22:55:47.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.571+0000 7fbbc7029700 1 -- 172.21.15.12:0/1818581253 shutdown_connections 2024-05-17T22:55:47.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.571+0000 7fbbc7029700 1 --2- 172.21.15.12:0/1818581253 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbc01080a0 0x7fbbc010fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:47.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.571+0000 7fbbc7029700 1 --2- 172.21.15.12:0/1818581253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbbc0102e70 0x7fbbc0103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:47.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.571+0000 7fbbc7029700 1 --2- 172.21.15.12:0/1818581253 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbc0107750 0x7fbbc0107b60 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:47.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.571+0000 7fbbc7029700 1 -- 172.21.15.12:0/1818581253 >> 172.21.15.12:0/1818581253 conn(0x7fbbc0079ae0 msgr2=0x7fbbc0079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:55:47.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.571+0000 7fbbc7029700 1 -- 172.21.15.12:0/1818581253 shutdown_connections 2024-05-17T22:55:47.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.571+0000 7fbbc7029700 1 -- 172.21.15.12:0/1818581253 wait complete. 2024-05-17T22:55:47.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.571+0000 7fbbc7029700 1 Processor -- start 2024-05-17T22:55:47.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.572+0000 7fbbc7029700 1 -- start start 2024-05-17T22:55:47.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.572+0000 7fbbc7029700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbc0102e70 0x7fbbc01a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:47.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.572+0000 7fbbc7029700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbbc0107750 0x7fbbc01a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:47.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.572+0000 7fbbc7029700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbc01080a0 0x7fbbc019a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:47.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.572+0000 7fbbc7029700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc0112d30 con 0x7fbbc0102e70 2024-05-17T22:55:47.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.572+0000 7fbbc7029700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc0112bb0 con 0x7fbbc01080a0 2024-05-17T22:55:47.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.572+0000 7fbbc7029700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc0112eb0 con 0x7fbbc0107750 2024-05-17T22:55:47.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.572+0000 7fbbc4dc5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbc0102e70 0x7fbbc01a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:47.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.572+0000 7fbbc4dc5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbc0102e70 0x7fbbc01a05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49366/0 (socket says 172.21.15.12:49366) 2024-05-17T22:55:47.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.572+0000 7fbbc4dc5700 1 -- 172.21.15.12:0/165958198 learned_addr learned my addr 172.21.15.12:0/165958198 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:55:47.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.572+0000 7fbbc55c6700 1 --2- 172.21.15.12:0/165958198 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbc01080a0 0x7fbbc019a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:47.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.572+0000 7fbbb7fff700 1 --2- 172.21.15.12:0/165958198 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbbc0107750 0x7fbbc01a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:47.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.572+0000 7fbbc4dc5700 1 -- 172.21.15.12:0/165958198 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbbc0107750 msgr2=0x7fbbc01a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:47.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.572+0000 7fbbc4dc5700 1 --2- 172.21.15.12:0/165958198 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbbc0107750 0x7fbbc01a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:47.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.572+0000 7fbbc4dc5700 1 -- 172.21.15.12:0/165958198 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbc01080a0 msgr2=0x7fbbc019a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:47.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.572+0000 7fbbc4dc5700 1 --2- 172.21.15.12:0/165958198 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbc01080a0 0x7fbbc019a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:47.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.572+0000 7fbbc4dc5700 1 -- 172.21.15.12:0/165958198 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbc019add0 con 0x7fbbc0102e70 2024-05-17T22:55:47.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.572+0000 7fbbb7fff700 1 --2- 172.21.15.12:0/165958198 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbbc0107750 0x7fbbc01a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:55:47.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.572+0000 7fbbc4dc5700 1 --2- 172.21.15.12:0/165958198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbc0102e70 0x7fbbc01a05a0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc00b530 tx=0x7fbbbc00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:55:47.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.573+0000 7fbbb5ffb700 1 -- 172.21.15.12:0/165958198 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc015020 con 0x7fbbc0102e70 2024-05-17T22:55:47.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.573+0000 7fbbb5ffb700 1 -- 172.21.15.12:0/165958198 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbbc011ce0 con 0x7fbbc0102e70 2024-05-17T22:55:47.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.573+0000 7fbbb5ffb700 1 -- 172.21.15.12:0/165958198 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc010d20 con 0x7fbbc0102e70 2024-05-17T22:55:47.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.573+0000 7fbbc7029700 1 -- 172.21.15.12:0/165958198 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbc019b0c0 con 0x7fbbc0102e70 2024-05-17T22:55:47.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.573+0000 7fbbc7029700 1 -- 172.21.15.12:0/165958198 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbc01a7350 con 0x7fbbc0102e70 2024-05-17T22:55:47.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.574+0000 7fbbc7029700 1 -- 172.21.15.12:0/165958198 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbc01042d0 con 0x7fbbc0102e70 2024-05-17T22:55:47.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.575+0000 7fbbb5ffb700 1 -- 172.21.15.12:0/165958198 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fbbbc018020 con 0x7fbbc0102e70 2024-05-17T22:55:47.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.576+0000 7fbbb5ffb700 1 --2- 172.21.15.12:0/165958198 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fbba80b2880 0x7fbba80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:47.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.576+0000 7fbbb7fff700 1 --2- 172.21.15.12:0/165958198 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fbba80b2880 0x7fbba80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:47.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.576+0000 7fbbb5ffb700 1 -- 172.21.15.12:0/165958198 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fbbbc0e2350 con 0x7fbbc0102e70 2024-05-17T22:55:47.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.577+0000 7fbbb7fff700 1 --2- 172.21.15.12:0/165958198 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fbba80b2880 0x7fbba80b4d90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbbc019bed0 tx=0x7fbbb001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:55:47.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.579+0000 7fbbb5ffb700 1 -- 172.21.15.12:0/165958198 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbbbc01f020 con 0x7fbbc0102e70 2024-05-17T22:55:47.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.813+0000 7fbbc7029700 1 -- 172.21.15.12:0/165958198 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbbc019bc40 con 0x7fbbc0102e70 2024-05-17T22:55:47.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.813+0000 7fbbb5ffb700 1 -- 172.21.15.12:0/165958198 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+592 (secure 0 0 0) 0x7fbbbc0a9ce0 con 0x7fbbc0102e70 2024-05-17T22:55:47.817 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2682700253 2024-05-18T22:27:14.372972+0000 2024-05-17T22:55:47.817 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/2250046521 2024-05-18T22:27:14.372972+0000 2024-05-17T22:55:47.817 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/4241094239 2024-05-18T22:26:45.466719+0000 2024-05-17T22:55:47.817 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/4172568611 2024-05-18T22:26:45.466719+0000 2024-05-17T22:55:47.817 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/3512529242 2024-05-18T22:27:14.372972+0000 2024-05-17T22:55:47.817 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/1039789215 2024-05-18T22:26:45.466719+0000 2024-05-17T22:55:47.817 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/2250046521 2024-05-18T22:27:14.372972+0000 2024-05-17T22:55:47.817 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/1039789215 2024-05-18T22:26:45.466719+0000 2024-05-17T22:55:47.818 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2043758453 2024-05-18T22:27:14.372972+0000 2024-05-17T22:55:47.818 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2694593732 2024-05-18T22:26:45.466719+0000 2024-05-17T22:55:47.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.816+0000 7fbbc7029700 1 -- 172.21.15.12:0/165958198 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fbba80b2880 msgr2=0x7fbba80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:47.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.816+0000 7fbbc7029700 1 --2- 172.21.15.12:0/165958198 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fbba80b2880 0x7fbba80b4d90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbbc019bed0 tx=0x7fbbb001a040 comp rx=0 tx=0).stop 2024-05-17T22:55:47.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.816+0000 7fbbc7029700 1 -- 172.21.15.12:0/165958198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbc0102e70 msgr2=0x7fbbc01a05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:47.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.816+0000 7fbbc7029700 1 --2- 172.21.15.12:0/165958198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbc0102e70 0x7fbbc01a05a0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc00b530 tx=0x7fbbbc00b8f0 comp rx=0 tx=0).stop 2024-05-17T22:55:47.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.816+0000 7fbbc7029700 1 -- 172.21.15.12:0/165958198 shutdown_connections 2024-05-17T22:55:47.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.816+0000 7fbbc7029700 1 --2- 172.21.15.12:0/165958198 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fbba80b2880 0x7fbba80b4d90 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:47.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.816+0000 7fbbc7029700 1 --2- 172.21.15.12:0/165958198 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbc01080a0 0x7fbbc019a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:47.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.816+0000 7fbbc7029700 1 --2- 172.21.15.12:0/165958198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbc0102e70 0x7fbbc01a05a0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:47.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.816+0000 7fbbc7029700 1 --2- 172.21.15.12:0/165958198 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbbc0107750 0x7fbbc01a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:47.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.816+0000 7fbbc7029700 1 -- 172.21.15.12:0/165958198 >> 172.21.15.12:0/165958198 conn(0x7fbbc0079ae0 msgr2=0x7fbbc0101d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:55:47.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.816+0000 7fbbc7029700 1 -- 172.21.15.12:0/165958198 shutdown_connections 2024-05-17T22:55:47.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:47.816+0000 7fbbc7029700 1 -- 172.21.15.12:0/165958198 wait complete. 2024-05-17T22:55:47.828 INFO:teuthology.orchestra.run.smithi012.stderr:listed 10 entries 2024-05-17T22:55:48.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:48 smithi151 ceph-mon[36101]: pgmap v935: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-05-17T22:55:48.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:48 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/165958198' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-17T22:55:48.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:48 smithi096 ceph-mon[36065]: pgmap v935: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-05-17T22:55:48.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:48 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/165958198' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-17T22:55:48.566 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:48 smithi012 ceph-mon[31632]: pgmap v935: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-05-17T22:55:48.566 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:48 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/165958198' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-17T22:55:48.601 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-17T22:55:48.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:55:48.601 DEBUG:teuthology.orchestra.run.smithi012:> sudo umount /tmp/cephfs 2024-05-17T22:55:48.686 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi012.front.sepia.ceph.com 2024-05-17T22:55:48.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:55:48.686 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2024-05-17T22:55:48.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:55:48.730 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2024-05-17T22:55:48.824 INFO:teuthology.orchestra.run.smithi012.stdout:ceph-ns--tmp-cephfs (id: 0) 2024-05-17T22:55:48.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:55:48.825 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns delete ceph-ns--tmp-cephfs 2024-05-17T22:55:48.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:55:48.853 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link delete ceph-brx 2024-05-17T22:55:48.953 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-17T22:55:49.530 INFO:tasks.cephadm:Teardown begin 2024-05-17T22:55:49.531 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-17T22:55:49.531 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T22:55:49.788 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:55:50.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:50 smithi151 ceph-mon[36101]: pgmap v936: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 KiB/s wr, 7 op/s 2024-05-17T22:55:50.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:50 smithi096 ceph-mon[36065]: pgmap v936: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 KiB/s wr, 7 op/s 2024-05-17T22:55:50.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:50 smithi012 ceph-mon[31632]: pgmap v936: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 KiB/s wr, 7 op/s 2024-05-17T22:55:51.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.572+0000 7f57cd10a700 1 -- 172.21.15.12:0/2458964505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57c8105760 msgr2=0x7f57c810dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:51.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.572+0000 7f57cd10a700 1 --2- 172.21.15.12:0/2458964505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57c8105760 0x7f57c810dcb0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f57bc009990 tx=0x7f57bc009ca0 comp rx=0 tx=0).stop 2024-05-17T22:55:51.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.572+0000 7f57cd10a700 1 -- 172.21.15.12:0/2458964505 shutdown_connections 2024-05-17T22:55:51.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.572+0000 7f57cd10a700 1 --2- 172.21.15.12:0/2458964505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57c810e240 0x7f57c8110750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:51.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.572+0000 7f57cd10a700 1 --2- 172.21.15.12:0/2458964505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57c8104e10 0x7f57c8105220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:51.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.572+0000 7f57cd10a700 1 --2- 172.21.15.12:0/2458964505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57c8105760 0x7f57c810dcb0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:51.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.572+0000 7f57cd10a700 1 -- 172.21.15.12:0/2458964505 >> 172.21.15.12:0/2458964505 conn(0x7f57c8079ae0 msgr2=0x7f57c8079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:55:51.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.572+0000 7f57cd10a700 1 -- 172.21.15.12:0/2458964505 shutdown_connections 2024-05-17T22:55:51.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.572+0000 7f57cd10a700 1 -- 172.21.15.12:0/2458964505 wait complete. 2024-05-17T22:55:51.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.573+0000 7f57cd10a700 1 Processor -- start 2024-05-17T22:55:51.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.573+0000 7f57cd10a700 1 -- start start 2024-05-17T22:55:51.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.574+0000 7f57cd10a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57c8104e10 0x7f57c8102b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:51.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.574+0000 7f57cd10a700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57c8105760 0x7f57c8103040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:51.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.574+0000 7f57cd10a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57c810e240 0x7f57c8103580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:51.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.574+0000 7f57cd10a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f57c8113dc0 con 0x7f57c810e240 2024-05-17T22:55:51.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.574+0000 7f57cd10a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f57c8113c40 con 0x7f57c8104e10 2024-05-17T22:55:51.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.574+0000 7f57cd10a700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f57c8113f40 con 0x7f57c8105760 2024-05-17T22:55:51.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.574+0000 7f57c759e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57c810e240 0x7f57c8103580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:51.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.574+0000 7f57c759e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57c810e240 0x7f57c8103580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37540/0 (socket says 172.21.15.12:37540) 2024-05-17T22:55:51.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.574+0000 7f57c759e700 1 -- 172.21.15.12:0/3292185768 learned_addr learned my addr 172.21.15.12:0/3292185768 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:55:51.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.574+0000 7f57c6d9d700 1 --2- 172.21.15.12:0/3292185768 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57c8104e10 0x7f57c8102b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:51.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.574+0000 7f57c659c700 1 --2- 172.21.15.12:0/3292185768 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57c8105760 0x7f57c8103040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:51.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.574+0000 7f57c759e700 1 -- 172.21.15.12:0/3292185768 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57c8105760 msgr2=0x7f57c8103040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:51.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.574+0000 7f57c759e700 1 --2- 172.21.15.12:0/3292185768 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57c8105760 0x7f57c8103040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:51.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.574+0000 7f57c759e700 1 -- 172.21.15.12:0/3292185768 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57c8104e10 msgr2=0x7f57c8102b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:51.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.574+0000 7f57c759e700 1 --2- 172.21.15.12:0/3292185768 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57c8104e10 0x7f57c8102b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:51.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.574+0000 7f57c759e700 1 -- 172.21.15.12:0/3292185768 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57c81a8b60 con 0x7f57c810e240 2024-05-17T22:55:51.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.574+0000 7f57c759e700 1 --2- 172.21.15.12:0/3292185768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57c810e240 0x7f57c8103580 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f57c000d980 tx=0x7f57c000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:55:51.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.575+0000 7f57affff700 1 -- 172.21.15.12:0/3292185768 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c0017070 con 0x7f57c810e240 2024-05-17T22:55:51.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.575+0000 7f57cd10a700 1 -- 172.21.15.12:0/3292185768 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57c81a8e50 con 0x7f57c810e240 2024-05-17T22:55:51.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.575+0000 7f57affff700 1 -- 172.21.15.12:0/3292185768 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57c0009df0 con 0x7f57c810e240 2024-05-17T22:55:51.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.575+0000 7f57affff700 1 -- 172.21.15.12:0/3292185768 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c00053a0 con 0x7f57c810e240 2024-05-17T22:55:51.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.575+0000 7f57cd10a700 1 -- 172.21.15.12:0/3292185768 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57c81a92d0 con 0x7f57c810e240 2024-05-17T22:55:51.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.576+0000 7f57cd10a700 1 -- 172.21.15.12:0/3292185768 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57c8105220 con 0x7f57c810e240 2024-05-17T22:55:51.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.577+0000 7f57affff700 1 -- 172.21.15.12:0/3292185768 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f57c0004750 con 0x7f57c810e240 2024-05-17T22:55:51.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.578+0000 7f57affff700 1 --2- 172.21.15.12:0/3292185768 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f57b00b2900 0x7f57b00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:51.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.578+0000 7f57c6d9d700 1 --2- 172.21.15.12:0/3292185768 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f57b00b2900 0x7f57b00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:51.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.578+0000 7f57affff700 1 -- 172.21.15.12:0/3292185768 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f57c00e2960 con 0x7f57c810e240 2024-05-17T22:55:51.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.578+0000 7f57c6d9d700 1 --2- 172.21.15.12:0/3292185768 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f57b00b2900 0x7f57b00b4e10 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f57b8009d10 tx=0x7f57b8009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:55:51.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.581+0000 7f57affff700 1 -- 172.21.15.12:0/3292185768 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f57c0020020 con 0x7f57c810e240 2024-05-17T22:55:51.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.844+0000 7f57cd10a700 1 -- 172.21.15.12:0/3292185768 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57c8064440 con 0x7f57c810e240 2024-05-17T22:55:51.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.845+0000 7f57affff700 1 -- 172.21.15.12:0/3292185768 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11874 (secure 0 0 0) 0x7f57c00aa310 con 0x7f57c810e240 2024-05-17T22:55:51.848 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:55:51.848 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1407219319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1407219319},{"type":"v1","addr":"172.21.15.12:6835","nonce":1407219319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/630977886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":630977886},{"type":"v1","addr":"172.21.15.12:6837","nonce":630977886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3731914742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3731914742},{"type":"v1","addr":"172.21.15.12:6839","nonce":3731914742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/3642142501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3642142501},{"type":"v1","addr":"172.21.15.12:6841","nonce":3642142501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3554053059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3554053059},{"type":"v1","addr":"172.21.15.151:6833","nonce":3554053059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/4228325569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":4228325569},{"type":"v1","addr":"172.21.15.96:6833","nonce":4228325569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T22:34:42.642111+0000","modified":"2024-05-17T22:54:42.952771+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24461,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/542323754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":542323754},{"type":"v1","addr":"172.21.15.151:6835","nonce":542323754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1758442232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1758442232},{"type":"v1","addr":"172.21.15.96:6835","nonce":1758442232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3403324038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3403324038},{"type":"v1","addr":"172.21.15.151:6837","nonce":3403324038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.151:6839/2338353849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2338353849},{"type":"v1","addr":"172.21.15.151:6839","nonce":2338353849}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.96:6837/4222574338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":4222574338},{"type":"v1","addr":"172.21.15.96:6837","nonce":4222574338}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/2986295519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2986295519},{"type":"v1","addr":"172.21.15.96:6839","nonce":2986295519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24457,24482,24461]},"id":1}]} 2024-05-17T22:55:51.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.848+0000 7f57cd10a700 1 -- 172.21.15.12:0/3292185768 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f57b00b2900 msgr2=0x7f57b00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:51.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.848+0000 7f57cd10a700 1 --2- 172.21.15.12:0/3292185768 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f57b00b2900 0x7f57b00b4e10 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f57b8009d10 tx=0x7f57b8009040 comp rx=0 tx=0).stop 2024-05-17T22:55:51.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.848+0000 7f57cd10a700 1 -- 172.21.15.12:0/3292185768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57c810e240 msgr2=0x7f57c8103580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:51.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.848+0000 7f57cd10a700 1 --2- 172.21.15.12:0/3292185768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57c810e240 0x7f57c8103580 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f57c000d980 tx=0x7f57c000dd40 comp rx=0 tx=0).stop 2024-05-17T22:55:51.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.848+0000 7f57cd10a700 1 -- 172.21.15.12:0/3292185768 shutdown_connections 2024-05-17T22:55:51.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.848+0000 7f57cd10a700 1 --2- 172.21.15.12:0/3292185768 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f57b00b2900 0x7f57b00b4e10 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:51.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.848+0000 7f57cd10a700 1 --2- 172.21.15.12:0/3292185768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57c810e240 0x7f57c8103580 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:51.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.848+0000 7f57cd10a700 1 --2- 172.21.15.12:0/3292185768 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57c8104e10 0x7f57c8102b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:51.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.848+0000 7f57cd10a700 1 --2- 172.21.15.12:0/3292185768 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57c8105760 0x7f57c8103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:51.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.848+0000 7f57cd10a700 1 -- 172.21.15.12:0/3292185768 >> 172.21.15.12:0/3292185768 conn(0x7f57c8079ae0 msgr2=0x7f57c80ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:55:51.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.848+0000 7f57cd10a700 1 -- 172.21.15.12:0/3292185768 shutdown_connections 2024-05-17T22:55:51.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:51.848+0000 7f57cd10a700 1 -- 172.21.15.12:0/3292185768 wait complete. 2024-05-17T22:55:51.851 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 46 2024-05-17T22:55:52.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:52 smithi151 ceph-mon[36101]: pgmap v937: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-05-17T22:55:52.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:52 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/3292185768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:55:52.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:52 smithi096 ceph-mon[36065]: pgmap v937: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-05-17T22:55:52.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:52 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/3292185768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:55:52.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:52 smithi012 ceph-mon[31632]: pgmap v937: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-05-17T22:55:52.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:52 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/3292185768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:55:52.680 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T22:55:52.941 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:55:54.459 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:54 smithi012 ceph-mon[31632]: pgmap v938: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 KiB/s wr, 9 op/s 2024-05-17T22:55:54.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:54 smithi151 ceph-mon[36101]: pgmap v938: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 KiB/s wr, 9 op/s 2024-05-17T22:55:54.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:54 smithi096 ceph-mon[36065]: pgmap v938: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 KiB/s wr, 9 op/s 2024-05-17T22:55:54.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.702+0000 7ff979ac8700 1 -- 172.21.15.12:0/857941460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff974107750 msgr2=0x7ff974107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:54.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.702+0000 7ff979ac8700 1 --2- 172.21.15.12:0/857941460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff974107750 0x7ff974107b60 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ff968009990 tx=0x7ff968009ca0 comp rx=0 tx=0).stop 2024-05-17T22:55:54.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.702+0000 7ff979ac8700 1 -- 172.21.15.12:0/857941460 shutdown_connections 2024-05-17T22:55:54.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.702+0000 7ff979ac8700 1 --2- 172.21.15.12:0/857941460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9741080a0 0x7ff97410fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:54.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.702+0000 7ff979ac8700 1 --2- 172.21.15.12:0/857941460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff974102e20 0x7ff974103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:54.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.702+0000 7ff979ac8700 1 --2- 172.21.15.12:0/857941460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff974107750 0x7ff974107b60 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:54.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.702+0000 7ff979ac8700 1 -- 172.21.15.12:0/857941460 >> 172.21.15.12:0/857941460 conn(0x7ff974079af0 msgr2=0x7ff974079f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:55:54.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.703+0000 7ff979ac8700 1 -- 172.21.15.12:0/857941460 shutdown_connections 2024-05-17T22:55:54.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.703+0000 7ff979ac8700 1 -- 172.21.15.12:0/857941460 wait complete. 2024-05-17T22:55:54.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.703+0000 7ff979ac8700 1 Processor -- start 2024-05-17T22:55:54.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.703+0000 7ff979ac8700 1 -- start start 2024-05-17T22:55:54.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.704+0000 7ff979ac8700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff974102e20 0x7ff974199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:54.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.704+0000 7ff979ac8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff974107750 0x7ff974199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:54.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.704+0000 7ff979ac8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9741080a0 0x7ff97419df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:54.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.704+0000 7ff979ac8700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff974112cb0 con 0x7ff9741080a0 2024-05-17T22:55:54.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.704+0000 7ff979ac8700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff974112b30 con 0x7ff974107750 2024-05-17T22:55:54.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.704+0000 7ff979ac8700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff974112e30 con 0x7ff974102e20 2024-05-17T22:55:54.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.704+0000 7ff973fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9741080a0 0x7ff97419df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:54.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.704+0000 7ff973fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9741080a0 0x7ff97419df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37556/0 (socket says 172.21.15.12:37556) 2024-05-17T22:55:54.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.704+0000 7ff973fff700 1 -- 172.21.15.12:0/1662464336 learned_addr learned my addr 172.21.15.12:0/1662464336 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:55:54.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.704+0000 7ff9737fe700 1 --2- 172.21.15.12:0/1662464336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff974102e20 0x7ff974199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:54.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.704+0000 7ff972ffd700 1 --2- 172.21.15.12:0/1662464336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff974107750 0x7ff974199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:54.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.704+0000 7ff973fff700 1 -- 172.21.15.12:0/1662464336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff974102e20 msgr2=0x7ff974199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:54.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.704+0000 7ff973fff700 1 --2- 172.21.15.12:0/1662464336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff974102e20 0x7ff974199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:54.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.704+0000 7ff973fff700 1 -- 172.21.15.12:0/1662464336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff974107750 msgr2=0x7ff974199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:54.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.704+0000 7ff973fff700 1 --2- 172.21.15.12:0/1662464336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff974107750 0x7ff974199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:54.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.704+0000 7ff973fff700 1 -- 172.21.15.12:0/1662464336 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff97419e5e0 con 0x7ff9741080a0 2024-05-17T22:55:54.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.704+0000 7ff972ffd700 1 --2- 172.21.15.12:0/1662464336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff974107750 0x7ff974199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:55:54.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.705+0000 7ff973fff700 1 --2- 172.21.15.12:0/1662464336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9741080a0 0x7ff97419df40 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff96c007f10 tx=0x7ff96c00f5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:55:54.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.705+0000 7ff970ff9700 1 -- 172.21.15.12:0/1662464336 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96c016030 con 0x7ff9741080a0 2024-05-17T22:55:54.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.705+0000 7ff970ff9700 1 -- 172.21.15.12:0/1662464336 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff96c00fde0 con 0x7ff9741080a0 2024-05-17T22:55:54.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.705+0000 7ff979ac8700 1 -- 172.21.15.12:0/1662464336 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff97419e8d0 con 0x7ff9741080a0 2024-05-17T22:55:54.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.705+0000 7ff970ff9700 1 -- 172.21.15.12:0/1662464336 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96c005020 con 0x7ff9741080a0 2024-05-17T22:55:54.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.705+0000 7ff979ac8700 1 -- 172.21.15.12:0/1662464336 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff97406c040 con 0x7ff9741080a0 2024-05-17T22:55:54.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.706+0000 7ff979ac8700 1 -- 172.21.15.12:0/1662464336 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9741042d0 con 0x7ff9741080a0 2024-05-17T22:55:54.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.707+0000 7ff970ff9700 1 -- 172.21.15.12:0/1662464336 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff96c004cb0 con 0x7ff9741080a0 2024-05-17T22:55:54.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.708+0000 7ff970ff9700 1 --2- 172.21.15.12:0/1662464336 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff95c0b2880 0x7ff95c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:54.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.708+0000 7ff9737fe700 1 --2- 172.21.15.12:0/1662464336 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff95c0b2880 0x7ff95c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:54.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.708+0000 7ff970ff9700 1 -- 172.21.15.12:0/1662464336 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7ff96c0e4c00 con 0x7ff9741080a0 2024-05-17T22:55:54.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.709+0000 7ff9737fe700 1 --2- 172.21.15.12:0/1662464336 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff95c0b2880 0x7ff95c0b4d90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff964005950 tx=0x7ff96400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:55:54.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.713+0000 7ff970ff9700 1 -- 172.21.15.12:0/1662464336 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff96c0ac590 con 0x7ff9741080a0 2024-05-17T22:55:54.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.973+0000 7ff979ac8700 1 -- 172.21.15.12:0/1662464336 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff97419aa70 con 0x7ff9741080a0 2024-05-17T22:55:54.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.974+0000 7ff970ff9700 1 -- 172.21.15.12:0/1662464336 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11874 (secure 0 0 0) 0x7ff96c0abe90 con 0x7ff9741080a0 2024-05-17T22:55:54.976 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:55:54.976 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1407219319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1407219319},{"type":"v1","addr":"172.21.15.12:6835","nonce":1407219319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/630977886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":630977886},{"type":"v1","addr":"172.21.15.12:6837","nonce":630977886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3731914742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3731914742},{"type":"v1","addr":"172.21.15.12:6839","nonce":3731914742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/3642142501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3642142501},{"type":"v1","addr":"172.21.15.12:6841","nonce":3642142501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3554053059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3554053059},{"type":"v1","addr":"172.21.15.151:6833","nonce":3554053059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/4228325569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":4228325569},{"type":"v1","addr":"172.21.15.96:6833","nonce":4228325569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T22:34:42.642111+0000","modified":"2024-05-17T22:54:42.952771+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24461,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/542323754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":542323754},{"type":"v1","addr":"172.21.15.151:6835","nonce":542323754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1758442232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1758442232},{"type":"v1","addr":"172.21.15.96:6835","nonce":1758442232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3403324038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3403324038},{"type":"v1","addr":"172.21.15.151:6837","nonce":3403324038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.151:6839/2338353849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2338353849},{"type":"v1","addr":"172.21.15.151:6839","nonce":2338353849}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.96:6837/4222574338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":4222574338},{"type":"v1","addr":"172.21.15.96:6837","nonce":4222574338}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/2986295519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2986295519},{"type":"v1","addr":"172.21.15.96:6839","nonce":2986295519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24457,24482,24461]},"id":1}]} 2024-05-17T22:55:54.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.976+0000 7ff979ac8700 1 -- 172.21.15.12:0/1662464336 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff95c0b2880 msgr2=0x7ff95c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:54.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.976+0000 7ff979ac8700 1 --2- 172.21.15.12:0/1662464336 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff95c0b2880 0x7ff95c0b4d90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff964005950 tx=0x7ff96400b410 comp rx=0 tx=0).stop 2024-05-17T22:55:54.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.976+0000 7ff979ac8700 1 -- 172.21.15.12:0/1662464336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9741080a0 msgr2=0x7ff97419df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:54.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.976+0000 7ff979ac8700 1 --2- 172.21.15.12:0/1662464336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9741080a0 0x7ff97419df40 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff96c007f10 tx=0x7ff96c00f5c0 comp rx=0 tx=0).stop 2024-05-17T22:55:54.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.976+0000 7ff979ac8700 1 -- 172.21.15.12:0/1662464336 shutdown_connections 2024-05-17T22:55:54.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.976+0000 7ff979ac8700 1 --2- 172.21.15.12:0/1662464336 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff95c0b2880 0x7ff95c0b4d90 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:54.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.976+0000 7ff979ac8700 1 --2- 172.21.15.12:0/1662464336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9741080a0 0x7ff97419df40 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:54.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.976+0000 7ff979ac8700 1 --2- 172.21.15.12:0/1662464336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff974102e20 0x7ff974199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:54.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.976+0000 7ff979ac8700 1 --2- 172.21.15.12:0/1662464336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff974107750 0x7ff974199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:54.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.976+0000 7ff979ac8700 1 -- 172.21.15.12:0/1662464336 >> 172.21.15.12:0/1662464336 conn(0x7ff974079af0 msgr2=0x7ff974105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:55:54.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.977+0000 7ff979ac8700 1 -- 172.21.15.12:0/1662464336 shutdown_connections 2024-05-17T22:55:54.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:54.977+0000 7ff979ac8700 1 -- 172.21.15.12:0/1662464336 wait complete. 2024-05-17T22:55:54.980 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 46 2024-05-17T22:55:55.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:55 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/1662464336' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:55:55.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:55 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/1662464336' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:55:55.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:55 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/1662464336' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T22:55:55.803 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-17T22:55:56.060 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:55:56.514 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:56 smithi012 ceph-mon[31632]: pgmap v939: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 KiB/s wr, 9 op/s 2024-05-17T22:55:56.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:56 smithi096 ceph-mon[36065]: pgmap v939: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 KiB/s wr, 9 op/s 2024-05-17T22:55:56.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:56 smithi151 ceph-mon[36101]: pgmap v939: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 KiB/s wr, 9 op/s 2024-05-17T22:55:57.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.756+0000 7ff6a4160700 1 -- 172.21.15.12:0/2367132726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff69c1057a0 msgr2=0x7ff69c10dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:57.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.756+0000 7ff6a4160700 1 --2- 172.21.15.12:0/2367132726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff69c1057a0 0x7ff69c10dcf0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff690009990 tx=0x7ff690009ca0 comp rx=0 tx=0).stop 2024-05-17T22:55:57.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.756+0000 7ff6a4160700 1 -- 172.21.15.12:0/2367132726 shutdown_connections 2024-05-17T22:55:57.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.756+0000 7ff6a4160700 1 --2- 172.21.15.12:0/2367132726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff69c10e280 0x7ff69c110750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:57.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.756+0000 7ff6a4160700 1 --2- 172.21.15.12:0/2367132726 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff69c104e50 0x7ff69c105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:57.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.756+0000 7ff6a4160700 1 --2- 172.21.15.12:0/2367132726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff69c1057a0 0x7ff69c10dcf0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:57.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.756+0000 7ff6a4160700 1 -- 172.21.15.12:0/2367132726 >> 172.21.15.12:0/2367132726 conn(0x7ff69c079ae0 msgr2=0x7ff69c079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:55:57.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.756+0000 7ff6a4160700 1 -- 172.21.15.12:0/2367132726 shutdown_connections 2024-05-17T22:55:57.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.756+0000 7ff6a4160700 1 -- 172.21.15.12:0/2367132726 wait complete. 2024-05-17T22:55:57.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.757+0000 7ff6a4160700 1 Processor -- start 2024-05-17T22:55:57.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.757+0000 7ff6a4160700 1 -- start start 2024-05-17T22:55:57.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.757+0000 7ff6a4160700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff69c104e50 0x7ff69c102b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:57.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.757+0000 7ff6a4160700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff69c1057a0 0x7ff69c103080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:57.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.757+0000 7ff6a4160700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff69c10e280 0x7ff69c1035c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:57.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.757+0000 7ff6a4160700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c113dc0 con 0x7ff69c1057a0 2024-05-17T22:55:57.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.757+0000 7ff6a4160700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c113c40 con 0x7ff69c10e280 2024-05-17T22:55:57.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.757+0000 7ff6a4160700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c113f40 con 0x7ff69c104e50 2024-05-17T22:55:57.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.758+0000 7ff6a16fb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff69c1057a0 0x7ff69c103080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:57.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.758+0000 7ff6a16fb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff69c1057a0 0x7ff69c103080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37566/0 (socket says 172.21.15.12:37566) 2024-05-17T22:55:57.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.758+0000 7ff6a16fb700 1 -- 172.21.15.12:0/3856835030 learned_addr learned my addr 172.21.15.12:0/3856835030 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:55:57.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.758+0000 7ff6a26fd700 1 --2- 172.21.15.12:0/3856835030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff69c10e280 0x7ff69c1035c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:57.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.758+0000 7ff6a1efc700 1 --2- 172.21.15.12:0/3856835030 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff69c104e50 0x7ff69c102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:57.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.758+0000 7ff6a16fb700 1 -- 172.21.15.12:0/3856835030 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff69c104e50 msgr2=0x7ff69c102b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:57.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.758+0000 7ff6a16fb700 1 --2- 172.21.15.12:0/3856835030 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff69c104e50 0x7ff69c102b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:57.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.758+0000 7ff6a16fb700 1 -- 172.21.15.12:0/3856835030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff69c10e280 msgr2=0x7ff69c1035c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:57.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.758+0000 7ff6a16fb700 1 --2- 172.21.15.12:0/3856835030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff69c10e280 0x7ff69c1035c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:57.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.758+0000 7ff6a16fb700 1 -- 172.21.15.12:0/3856835030 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff69c1a8af0 con 0x7ff69c1057a0 2024-05-17T22:55:57.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.758+0000 7ff6a1efc700 1 --2- 172.21.15.12:0/3856835030 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff69c104e50 0x7ff69c102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:55:57.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.758+0000 7ff6a16fb700 1 --2- 172.21.15.12:0/3856835030 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff69c1057a0 0x7ff69c103080 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff690000c00 tx=0x7ff6900047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:55:57.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.758+0000 7ff68effd700 1 -- 172.21.15.12:0/3856835030 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff690027070 con 0x7ff69c1057a0 2024-05-17T22:55:57.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.759+0000 7ff68effd700 1 -- 172.21.15.12:0/3856835030 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6900040e0 con 0x7ff69c1057a0 2024-05-17T22:55:57.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.759+0000 7ff68effd700 1 -- 172.21.15.12:0/3856835030 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69000f460 con 0x7ff69c1057a0 2024-05-17T22:55:57.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.759+0000 7ff6a4160700 1 -- 172.21.15.12:0/3856835030 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff69c1a8d80 con 0x7ff69c1057a0 2024-05-17T22:55:57.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.759+0000 7ff6a4160700 1 -- 172.21.15.12:0/3856835030 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff69c1a91b0 con 0x7ff69c1057a0 2024-05-17T22:55:57.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.760+0000 7ff6a4160700 1 -- 172.21.15.12:0/3856835030 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff69c105260 con 0x7ff69c1057a0 2024-05-17T22:55:57.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.761+0000 7ff68effd700 1 -- 172.21.15.12:0/3856835030 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff69002a070 con 0x7ff69c1057a0 2024-05-17T22:55:57.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.762+0000 7ff68effd700 1 --2- 172.21.15.12:0/3856835030 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff6880b2880 0x7ff6880b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:55:57.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.762+0000 7ff6a1efc700 1 --2- 172.21.15.12:0/3856835030 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff6880b2880 0x7ff6880b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:55:57.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.762+0000 7ff68effd700 1 -- 172.21.15.12:0/3856835030 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7ff6900e3880 con 0x7ff69c1057a0 2024-05-17T22:55:57.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.763+0000 7ff6a1efc700 1 --2- 172.21.15.12:0/3856835030 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff6880b2880 0x7ff6880b4d90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff698005950 tx=0x7ff698005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:55:57.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.765+0000 7ff68effd700 1 -- 172.21.15.12:0/3856835030 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff690013070 con 0x7ff69c1057a0 2024-05-17T22:55:57.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.980+0000 7ff6a4160700 1 -- 172.21.15.12:0/3856835030 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff69c064440 con 0x7ff69c1057a0 2024-05-17T22:55:57.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.982+0000 7ff68effd700 1 -- 172.21.15.12:0/3856835030 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24715 (secure 0 0 0) 0x7ff6900ab2d0 con 0x7ff69c1057a0 2024-05-17T22:55:57.986 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-05-17T22:55:57.987 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":89,"fsid":"4d593780-149c-11ef-bc9a-c7b262605968","created":"2024-05-17T22:26:16.955626+0000","modified":"2024-05-17T22:36:15.618992+0000","last_up_change":"2024-05-17T22:33:36.551093+0000","last_in_change":"2024-05-17T22:33:15.032677+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T22:29:42.475513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T22:34:35.504697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T22:34:39.377097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-17T22:34:50.467684+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5d42798f-dc5f-4e66-bce2-ec33dc789e4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6803","nonce":2015605593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6805","nonce":2015605593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6809","nonce":2015605593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2015605593},{"type":"v1","addr":"172.21.15.12:6807","nonce":2015605593}]},"public_addr":"172.21.15.12:6803/2015605593","cluster_addr":"172.21.15.12:6805/2015605593","heartbeat_back_addr":"172.21.15.12:6809/2015605593","heartbeat_front_addr":"172.21.15.12:6807/2015605593","state":["exists","up"]},{"osd":1,"uuid":"58afecc7-119b-4c16-a401-640fb77d4714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6801","nonce":1238316358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6803","nonce":1238316358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6807","nonce":1238316358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1238316358},{"type":"v1","addr":"172.21.15.96:6805","nonce":1238316358}]},"public_addr":"172.21.15.96:6801/1238316358","cluster_addr":"172.21.15.96:6803/1238316358","heartbeat_back_addr":"172.21.15.96:6807/1238316358","heartbeat_front_addr":"172.21.15.96:6805/1238316358","state":["exists","up"]},{"osd":2,"uuid":"4bef4049-482f-49fb-82fb-38052400eb2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6801","nonce":148729815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6803","nonce":148729815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6807","nonce":148729815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":148729815},{"type":"v1","addr":"172.21.15.151:6805","nonce":148729815}]},"public_addr":"172.21.15.151:6801/148729815","cluster_addr":"172.21.15.151:6803/148729815","heartbeat_back_addr":"172.21.15.151:6807/148729815","heartbeat_front_addr":"172.21.15.151:6805/148729815","state":["exists","up"]},{"osd":3,"uuid":"6fed8e36-82a2-49e4-82a3-16e6d9de87f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6811","nonce":3563309320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6813","nonce":3563309320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6817","nonce":3563309320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3563309320},{"type":"v1","addr":"172.21.15.12:6815","nonce":3563309320}]},"public_addr":"172.21.15.12:6811/3563309320","cluster_addr":"172.21.15.12:6813/3563309320","heartbeat_back_addr":"172.21.15.12:6817/3563309320","heartbeat_front_addr":"172.21.15.12:6815/3563309320","state":["exists","up"]},{"osd":4,"uuid":"c0e8689a-2816-4089-ade3-931960bae4f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6809","nonce":2219103166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6811","nonce":2219103166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6815","nonce":2219103166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2219103166},{"type":"v1","addr":"172.21.15.96:6813","nonce":2219103166}]},"public_addr":"172.21.15.96:6809/2219103166","cluster_addr":"172.21.15.96:6811/2219103166","heartbeat_back_addr":"172.21.15.96:6815/2219103166","heartbeat_front_addr":"172.21.15.96:6813/2219103166","state":["exists","up"]},{"osd":5,"uuid":"e76ba432-94ff-4998-aaba-2645d9e6055d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6809","nonce":2589835078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6811","nonce":2589835078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6815","nonce":2589835078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":2589835078},{"type":"v1","addr":"172.21.15.151:6813","nonce":2589835078}]},"public_addr":"172.21.15.151:6809/2589835078","cluster_addr":"172.21.15.151:6811/2589835078","heartbeat_back_addr":"172.21.15.151:6815/2589835078","heartbeat_front_addr":"172.21.15.151:6813/2589835078","state":["exists","up"]},{"osd":6,"uuid":"b7ece35c-efd1-40df-95da-05a0cc147372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6819","nonce":3112661724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6821","nonce":3112661724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6825","nonce":3112661724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3112661724},{"type":"v1","addr":"172.21.15.12:6823","nonce":3112661724}]},"public_addr":"172.21.15.12:6819/3112661724","cluster_addr":"172.21.15.12:6821/3112661724","heartbeat_back_addr":"172.21.15.12:6825/3112661724","heartbeat_front_addr":"172.21.15.12:6823/3112661724","state":["exists","up"]},{"osd":7,"uuid":"6a65bf30-daf5-4fba-b67e-8e513d88a971","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6817","nonce":2449989110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6819","nonce":2449989110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6823","nonce":2449989110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2449989110},{"type":"v1","addr":"172.21.15.96:6821","nonce":2449989110}]},"public_addr":"172.21.15.96:6817/2449989110","cluster_addr":"172.21.15.96:6819/2449989110","heartbeat_back_addr":"172.21.15.96:6823/2449989110","heartbeat_front_addr":"172.21.15.96:6821/2449989110","state":["exists","up"]},{"osd":8,"uuid":"ad62ea44-e416-4a3e-bbbe-93f2a710dbc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6817","nonce":2545224628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6819","nonce":2545224628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6823","nonce":2545224628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2545224628},{"type":"v1","addr":"172.21.15.151:6821","nonce":2545224628}]},"public_addr":"172.21.15.151:6817/2545224628","cluster_addr":"172.21.15.151:6819/2545224628","heartbeat_back_addr":"172.21.15.151:6823/2545224628","heartbeat_front_addr":"172.21.15.151:6821/2545224628","state":["exists","up"]},{"osd":9,"uuid":"1275de08-2a3c-48f3-920b-dd68b4024d42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6827","nonce":2100661273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6829","nonce":2100661273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6833","nonce":2100661273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":2100661273},{"type":"v1","addr":"172.21.15.12:6831","nonce":2100661273}]},"public_addr":"172.21.15.12:6827/2100661273","cluster_addr":"172.21.15.12:6829/2100661273","heartbeat_back_addr":"172.21.15.12:6833/2100661273","heartbeat_front_addr":"172.21.15.12:6831/2100661273","state":["exists","up"]},{"osd":10,"uuid":"347e4e40-9a9d-43fe-8f52-fdd3199a2a5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6825","nonce":304016309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6827","nonce":304016309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6831","nonce":304016309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":304016309},{"type":"v1","addr":"172.21.15.96:6829","nonce":304016309}]},"public_addr":"172.21.15.96:6825/304016309","cluster_addr":"172.21.15.96:6827/304016309","heartbeat_back_addr":"172.21.15.96:6831/304016309","heartbeat_front_addr":"172.21.15.96:6829/304016309","state":["exists","up"]},{"osd":11,"uuid":"ee8d3502-8b3c-4e6f-824f-3a97e19111ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6825","nonce":3822307314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6827","nonce":3822307314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6831","nonce":3822307314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3822307314},{"type":"v1","addr":"172.21.15.151:6829","nonce":3822307314}]},"public_addr":"172.21.15.151:6825/3822307314","cluster_addr":"172.21.15.151:6827/3822307314","heartbeat_back_addr":"172.21.15.151:6831/3822307314","heartbeat_front_addr":"172.21.15.151:6829/3822307314","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:28:51.542055+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:29:15.566424+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:29:39.500981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:04.797907+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:29.655650+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:30:55.211880+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:31:21.559462+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:31:47.154852+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:32:13.114892+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:32:40.529708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:33:06.930942+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T22:33:34.061783+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":10}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/2682700253":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6801/2250046521":"2024-05-18T22:27:14.372972+0000","172.21.15.12:0/4241094239":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/4172568611":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/3512529242":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6800/1039789215":"2024-05-18T22:26:45.466719+0000","172.21.15.12:6800/2250046521":"2024-05-18T22:27:14.372972+0000","172.21.15.12:6801/1039789215":"2024-05-18T22:26:45.466719+0000","172.21.15.12:0/2043758453":"2024-05-18T22:27:14.372972+0000","172.21.15.12:0/2694593732":"2024-05-18T22:26:45.466719+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T22:55:57.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.984+0000 7ff6a4160700 1 -- 172.21.15.12:0/3856835030 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff6880b2880 msgr2=0x7ff6880b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:57.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.984+0000 7ff6a4160700 1 --2- 172.21.15.12:0/3856835030 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff6880b2880 0x7ff6880b4d90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff698005950 tx=0x7ff698005f50 comp rx=0 tx=0).stop 2024-05-17T22:55:57.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.984+0000 7ff6a4160700 1 -- 172.21.15.12:0/3856835030 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff69c1057a0 msgr2=0x7ff69c103080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:55:57.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.984+0000 7ff6a4160700 1 --2- 172.21.15.12:0/3856835030 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff69c1057a0 0x7ff69c103080 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff690000c00 tx=0x7ff6900047b0 comp rx=0 tx=0).stop 2024-05-17T22:55:57.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.984+0000 7ff6a4160700 1 -- 172.21.15.12:0/3856835030 shutdown_connections 2024-05-17T22:55:57.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.984+0000 7ff6a4160700 1 --2- 172.21.15.12:0/3856835030 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7ff6880b2880 0x7ff6880b4d90 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:57.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.984+0000 7ff6a4160700 1 --2- 172.21.15.12:0/3856835030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff69c10e280 0x7ff69c1035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:57.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.984+0000 7ff6a4160700 1 --2- 172.21.15.12:0/3856835030 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff69c104e50 0x7ff69c102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:57.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.984+0000 7ff6a4160700 1 --2- 172.21.15.12:0/3856835030 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff69c1057a0 0x7ff69c103080 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:55:57.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.984+0000 7ff6a4160700 1 -- 172.21.15.12:0/3856835030 >> 172.21.15.12:0/3856835030 conn(0x7ff69c079ae0 msgr2=0x7ff69c0ffbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:55:57.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.984+0000 7ff6a4160700 1 -- 172.21.15.12:0/3856835030 shutdown_connections 2024-05-17T22:55:57.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:55:57.985+0000 7ff6a4160700 1 -- 172.21.15.12:0/3856835030 wait complete. 2024-05-17T22:55:58.369 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:58 smithi012 ceph-mon[31632]: pgmap v940: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 8 op/s 2024-05-17T22:55:58.369 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:55:58 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/3856835030' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:55:58.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:58 smithi096 ceph-mon[36065]: pgmap v940: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 8 op/s 2024-05-17T22:55:58.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:55:58 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/3856835030' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:55:58.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:58 smithi151 ceph-mon[36101]: pgmap v940: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 8 op/s 2024-05-17T22:55:58.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:55:58 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/3856835030' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T22:55:58.798 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs fail cephfs 2024-05-17T22:55:59.059 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:56:00.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:00 smithi096 ceph-mon[36065]: pgmap v941: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.7 KiB/s wr, 8 op/s 2024-05-17T22:56:00.553 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:00 smithi012 ceph-mon[31632]: pgmap v941: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.7 KiB/s wr, 8 op/s 2024-05-17T22:56:00.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:00 smithi151 ceph-mon[36101]: pgmap v941: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.7 KiB/s wr, 8 op/s 2024-05-17T22:56:00.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:00.822+0000 7f812b2e8700 1 -- 172.21.15.12:0/2357709020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8124103f10 msgr2=0x7f8124104320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:56:00.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:00.822+0000 7f812b2e8700 1 --2- 172.21.15.12:0/2357709020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8124103f10 0x7f8124104320 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f811400b0a0 tx=0x7f811400b3b0 comp rx=0 tx=0).stop 2024-05-17T22:56:00.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:00.822+0000 7f812b2e8700 1 -- 172.21.15.12:0/2357709020 shutdown_connections 2024-05-17T22:56:00.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:00.822+0000 7f812b2e8700 1 --2- 172.21.15.12:0/2357709020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8124109190 0x7f8124110af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:00.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:00.822+0000 7f812b2e8700 1 --2- 172.21.15.12:0/2357709020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8124103f10 0x7f8124104320 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:00.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:00.822+0000 7f812b2e8700 1 --2- 172.21.15.12:0/2357709020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8124108840 0x7f8124108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:00.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:00.822+0000 7f812b2e8700 1 -- 172.21.15.12:0/2357709020 >> 172.21.15.12:0/2357709020 conn(0x7f81240ff990 msgr2=0x7f8124101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:56:00.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:00.823+0000 7f812b2e8700 1 -- 172.21.15.12:0/2357709020 shutdown_connections 2024-05-17T22:56:00.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:00.823+0000 7f812b2e8700 1 -- 172.21.15.12:0/2357709020 wait complete. 2024-05-17T22:56:00.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:00.823+0000 7f812b2e8700 1 Processor -- start 2024-05-17T22:56:00.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:00.823+0000 7f812b2e8700 1 -- start start 2024-05-17T22:56:00.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:00.824+0000 7f812b2e8700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8124103f10 0x7f81241a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:56:00.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:00.824+0000 7f812b2e8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8124108840 0x7f81241a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:56:00.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:00.824+0000 7f812b2e8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8124109190 0x7f812419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:56:00.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:00.824+0000 7f812b2e8700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8124113ec0 con 0x7f8124108840 2024-05-17T22:56:00.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:00.824+0000 7f812b2e8700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8124113d40 con 0x7f8124109190 2024-05-17T22:56:00.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:00.824+0000 7f812b2e8700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8124114040 con 0x7f8124103f10 2024-05-17T22:56:00.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:00.824+0000 7f8128883700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8124108840 0x7f81241a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:56:00.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:00.824+0000 7f8128883700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8124108840 0x7f81241a0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42884/0 (socket says 172.21.15.12:42884) 2024-05-17T22:56:00.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:00.824+0000 7f8128883700 1 -- 172.21.15.12:0/2454966420 learned_addr learned my addr 172.21.15.12:0/2454966420 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:56:00.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:00.824+0000 7f8129885700 1 --2- 172.21.15.12:0/2454966420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8124109190 0x7f812419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:56:00.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:00.824+0000 7f8129084700 1 --2- 172.21.15.12:0/2454966420 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8124103f10 0x7f81241a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:56:00.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:00.824+0000 7f8128883700 1 -- 172.21.15.12:0/2454966420 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8124103f10 msgr2=0x7f81241a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:56:00.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:00.824+0000 7f8128883700 1 --2- 172.21.15.12:0/2454966420 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8124103f10 0x7f81241a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:00.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:00.824+0000 7f8128883700 1 -- 172.21.15.12:0/2454966420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8124109190 msgr2=0x7f812419a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:56:00.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:00.824+0000 7f8128883700 1 --2- 172.21.15.12:0/2454966420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8124109190 0x7f812419a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:00.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:00.824+0000 7f8128883700 1 -- 172.21.15.12:0/2454966420 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f812419ae80 con 0x7f8124108840 2024-05-17T22:56:00.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:00.824+0000 7f8128883700 1 --2- 172.21.15.12:0/2454966420 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8124108840 0x7f81241a0b00 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f811800b500 tx=0x7f811800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:56:00.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:00.824+0000 7f81227fc700 1 -- 172.21.15.12:0/2454966420 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8118015020 con 0x7f8124108840 2024-05-17T22:56:00.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:00.825+0000 7f81227fc700 1 -- 172.21.15.12:0/2454966420 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8118011ce0 con 0x7f8124108840 2024-05-17T22:56:00.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:00.825+0000 7f812b2e8700 1 -- 172.21.15.12:0/2454966420 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f812419b170 con 0x7f8124108840 2024-05-17T22:56:00.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:00.825+0000 7f81227fc700 1 -- 172.21.15.12:0/2454966420 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8118010cb0 con 0x7f8124108840 2024-05-17T22:56:00.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:00.825+0000 7f812b2e8700 1 -- 172.21.15.12:0/2454966420 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81241a7370 con 0x7f8124108840 2024-05-17T22:56:00.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:00.826+0000 7f812b2e8700 1 -- 172.21.15.12:0/2454966420 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81241053c0 con 0x7f8124108840 2024-05-17T22:56:00.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:00.827+0000 7f81227fc700 1 -- 172.21.15.12:0/2454966420 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8118018020 con 0x7f8124108840 2024-05-17T22:56:00.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:00.828+0000 7f81227fc700 1 --2- 172.21.15.12:0/2454966420 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f810c0b2900 0x7f810c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:56:00.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:00.828+0000 7f8129084700 1 --2- 172.21.15.12:0/2454966420 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f810c0b2900 0x7f810c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:56:00.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:00.828+0000 7f81227fc700 1 -- 172.21.15.12:0/2454966420 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f81180e21a0 con 0x7f8124108840 2024-05-17T22:56:00.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:00.829+0000 7f8129084700 1 --2- 172.21.15.12:0/2454966420 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f810c0b2900 0x7f810c0b4e10 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8114009e60 tx=0x7f811400b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:56:00.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:00.831+0000 7f81227fc700 1 -- 172.21.15.12:0/2454966420 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f811801f020 con 0x7f8124108840 2024-05-17T22:56:01.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:01.093+0000 7f812b2e8700 1 -- 172.21.15.12:0/2454966420 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f812419bc60 con 0x7f8124108840 2024-05-17T22:56:01.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:01.096+0000 7f81227fc700 1 -- 172.21.15.12:0/2454966420 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v47) ==== 161+0+0 (secure 0 0 0) 0x7f81180a9ad0 con 0x7f8124108840 2024-05-17T22:56:01.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:01.099+0000 7f812b2e8700 1 -- 172.21.15.12:0/2454966420 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f810c0b2900 msgr2=0x7f810c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:56:01.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:01.099+0000 7f812b2e8700 1 --2- 172.21.15.12:0/2454966420 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f810c0b2900 0x7f810c0b4e10 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8114009e60 tx=0x7f811400b7b0 comp rx=0 tx=0).stop 2024-05-17T22:56:01.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:01.099+0000 7f812b2e8700 1 -- 172.21.15.12:0/2454966420 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8124108840 msgr2=0x7f81241a0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:56:01.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:01.099+0000 7f812b2e8700 1 --2- 172.21.15.12:0/2454966420 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8124108840 0x7f81241a0b00 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f811800b500 tx=0x7f811800b810 comp rx=0 tx=0).stop 2024-05-17T22:56:01.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:01.099+0000 7f812b2e8700 1 -- 172.21.15.12:0/2454966420 shutdown_connections 2024-05-17T22:56:01.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:01.099+0000 7f812b2e8700 1 --2- 172.21.15.12:0/2454966420 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f810c0b2900 0x7f810c0b4e10 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:01.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:01.099+0000 7f812b2e8700 1 --2- 172.21.15.12:0/2454966420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8124109190 0x7f812419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:01.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:01.099+0000 7f812b2e8700 1 --2- 172.21.15.12:0/2454966420 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8124103f10 0x7f81241a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:01.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:01.099+0000 7f812b2e8700 1 --2- 172.21.15.12:0/2454966420 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8124108840 0x7f81241a0b00 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:01.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:01.099+0000 7f812b2e8700 1 -- 172.21.15.12:0/2454966420 >> 172.21.15.12:0/2454966420 conn(0x7f81240ff990 msgr2=0x7f8124101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:56:01.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:01.099+0000 7f812b2e8700 1 -- 172.21.15.12:0/2454966420 shutdown_connections 2024-05-17T22:56:01.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:01.099+0000 7f812b2e8700 1 -- 172.21.15.12:0/2454966420 wait complete. 2024-05-17T22:56:01.103 INFO:teuthology.orchestra.run.smithi012.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-17T22:56:01.454 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:01 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mon-a[31628]: 2024-05-17T22:56:01.094+0000 7f780201f700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-17T22:56:01.454 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:01 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/2454966420' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-17T22:56:01.454 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:01 smithi012 ceph-mon[31632]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-17T22:56:01.454 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:01 smithi012 ceph-mon[31632]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-17T22:56:01.455 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:01 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/2454966420' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-17T22:56:01.455 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:01 smithi012 ceph-mon[31632]: osdmap e90: 12 total, 12 up, 12 in 2024-05-17T22:56:01.455 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:01 smithi012 ceph-mon[31632]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-17T22:56:01.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:01 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/2454966420' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-17T22:56:01.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:01 smithi151 ceph-mon[36101]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-17T22:56:01.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:01 smithi151 ceph-mon[36101]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-17T22:56:01.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:01 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/2454966420' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-17T22:56:01.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:01 smithi151 ceph-mon[36101]: osdmap e90: 12 total, 12 up, 12 in 2024-05-17T22:56:01.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:01 smithi151 ceph-mon[36101]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-17T22:56:01.491 INFO:journalctl@ceph.mds.f.smithi151.stdout:May 17 22:56:01 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-f[66703]: ignoring --setuser ceph since I am not root 2024-05-17T22:56:01.491 INFO:journalctl@ceph.mds.f.smithi151.stdout:May 17 22:56:01 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-f[66703]: ignoring --setgroup ceph since I am not root 2024-05-17T22:56:01.491 INFO:journalctl@ceph.mds.f.smithi151.stdout:May 17 22:56:01 smithi151 ceph-mds[66707]: -- 172.21.15.151:0/637944747 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561c0ecd7a40 con 0x561c0ed1da80 2024-05-17T22:56:01.491 INFO:journalctl@ceph.mds.f.smithi151.stdout:May 17 22:56:01 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-f[66703]: starting mds.f at 2024-05-17T22:56:01.491 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 17 22:56:01 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-i[67309]: ignoring --setuser ceph since I am not root 2024-05-17T22:56:01.492 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 17 22:56:01 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-i[67309]: ignoring --setgroup ceph since I am not root 2024-05-17T22:56:01.492 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 17 22:56:01 smithi151 ceph-mds[67313]: -- 172.21.15.151:0/1175289903 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d21860fa40 con 0x55d218655600 2024-05-17T22:56:01.492 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 17 22:56:01 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-i[67309]: starting mds.i at 2024-05-17T22:56:01.492 INFO:journalctl@ceph.mds.c.smithi151.stdout:May 17 22:56:01 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-c[67008]: ignoring --setuser ceph since I am not root 2024-05-17T22:56:01.492 INFO:journalctl@ceph.mds.c.smithi151.stdout:May 17 22:56:01 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-c[67008]: ignoring --setgroup ceph since I am not root 2024-05-17T22:56:01.492 INFO:journalctl@ceph.mds.c.smithi151.stdout:May 17 22:56:01 smithi151 ceph-mds[67012]: -- 172.21.15.151:0/1820598038 <== mon.0 v2:172.21.15.12:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e757d9da40 con 0x55e757de3a80 2024-05-17T22:56:01.492 INFO:journalctl@ceph.mds.c.smithi151.stdout:May 17 22:56:01 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-c[67008]: starting mds.c at 2024-05-17T22:56:01.492 INFO:journalctl@ceph.mds.c.smithi151.stdout:May 17 22:56:01 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-c[67008]: 2024-05-17T22:56:01.174+0000 7fd3c9dc5700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6835/0: (98) Address already in use 2024-05-17T22:56:01.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:01 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/2454966420' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-17T22:56:01.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:01 smithi096 ceph-mon[36065]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-17T22:56:01.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:01 smithi096 ceph-mon[36065]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-17T22:56:01.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:01 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/2454966420' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-17T22:56:01.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:01 smithi096 ceph-mon[36065]: osdmap e90: 12 total, 12 up, 12 in 2024-05-17T22:56:01.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:01 smithi096 ceph-mon[36065]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-17T22:56:01.549 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 17 22:56:01 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-h[66183]: ignoring --setuser ceph since I am not root 2024-05-17T22:56:01.549 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 17 22:56:01 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-h[66183]: ignoring --setgroup ceph since I am not root 2024-05-17T22:56:01.549 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 17 22:56:01 smithi096 ceph-mds[66187]: -- 172.21.15.96:0/202595953 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bab3b6da40 con 0x55bab3bb3180 2024-05-17T22:56:01.550 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 17 22:56:01 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-h[66183]: starting mds.h at 2024-05-17T22:56:01.550 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 17 22:56:01 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-b[66490]: ignoring --setuser ceph since I am not root 2024-05-17T22:56:01.550 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 17 22:56:01 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-b[66490]: ignoring --setgroup ceph since I am not root 2024-05-17T22:56:01.550 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 17 22:56:01 smithi096 ceph-mds[66494]: -- 172.21.15.96:0/3248665710 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5555ae47ba40 con 0x5555ae4c1180 2024-05-17T22:56:01.550 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 17 22:56:01 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-b[66490]: starting mds.b at 2024-05-17T22:56:01.550 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 17 22:56:01 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-e[65880]: ignoring --setuser ceph since I am not root 2024-05-17T22:56:01.550 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 17 22:56:01 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-e[65880]: ignoring --setgroup ceph since I am not root 2024-05-17T22:56:01.551 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 17 22:56:01 smithi096 ceph-mds[65884]: -- 172.21.15.96:0/1669182133 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562e7607ba40 con 0x562e760c1180 2024-05-17T22:56:01.551 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 17 22:56:01 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-e[65880]: starting mds.e at 2024-05-17T22:56:01.844 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-05-17T22:56:02.098 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:56:02.350 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:02 smithi012 ceph-mon[31632]: pgmap v942: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2024-05-17T22:56:02.351 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:02 smithi012 ceph-mon[31632]: mds.? [v2:172.21.15.96:6835/3095044732,v1:172.21.15.96:6838/3095044732] up:boot 2024-05-17T22:56:02.351 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:02 smithi012 ceph-mon[31632]: mds.? [v2:172.21.15.96:6836/2353640503,v1:172.21.15.96:6839/2353640503] up:boot 2024-05-17T22:56:02.351 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:02 smithi012 ceph-mon[31632]: mds.? [v2:172.21.15.96:6834/1784052459,v1:172.21.15.96:6837/1784052459] up:boot 2024-05-17T22:56:02.351 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:02 smithi012 ceph-mon[31632]: mds.? [v2:172.21.15.151:6834/2304849160,v1:172.21.15.151:6837/2304849160] up:boot 2024-05-17T22:56:02.351 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:02 smithi012 ceph-mon[31632]: mds.? [v2:172.21.15.151:6835/3314664066,v1:172.21.15.151:6838/3314664066] up:boot 2024-05-17T22:56:02.351 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:02 smithi012 ceph-mon[31632]: mds.? [v2:172.21.15.151:6836/1196155727,v1:172.21.15.151:6839/1196155727] up:boot 2024-05-17T22:56:02.351 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:02 smithi012 ceph-mon[31632]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-17T22:56:02.351 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:02 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-17T22:56:02.351 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:02 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-17T22:56:02.351 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:02 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-17T22:56:02.351 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:02 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-17T22:56:02.351 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:02 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-17T22:56:02.351 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:02 smithi012 ceph-mon[31632]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-17T22:56:02.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:02 smithi096 ceph-mon[36065]: pgmap v942: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2024-05-17T22:56:02.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:02 smithi096 ceph-mon[36065]: mds.? [v2:172.21.15.96:6835/3095044732,v1:172.21.15.96:6838/3095044732] up:boot 2024-05-17T22:56:02.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:02 smithi096 ceph-mon[36065]: mds.? [v2:172.21.15.96:6836/2353640503,v1:172.21.15.96:6839/2353640503] up:boot 2024-05-17T22:56:02.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:02 smithi096 ceph-mon[36065]: mds.? [v2:172.21.15.96:6834/1784052459,v1:172.21.15.96:6837/1784052459] up:boot 2024-05-17T22:56:02.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:02 smithi096 ceph-mon[36065]: mds.? [v2:172.21.15.151:6834/2304849160,v1:172.21.15.151:6837/2304849160] up:boot 2024-05-17T22:56:02.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:02 smithi096 ceph-mon[36065]: mds.? [v2:172.21.15.151:6835/3314664066,v1:172.21.15.151:6838/3314664066] up:boot 2024-05-17T22:56:02.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:02 smithi096 ceph-mon[36065]: mds.? [v2:172.21.15.151:6836/1196155727,v1:172.21.15.151:6839/1196155727] up:boot 2024-05-17T22:56:02.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:02 smithi096 ceph-mon[36065]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-17T22:56:02.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:02 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-17T22:56:02.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:02 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-17T22:56:02.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:02 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-17T22:56:02.550 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:02 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-17T22:56:02.550 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:02 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-17T22:56:02.550 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:02 smithi096 ceph-mon[36065]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-17T22:56:02.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:02 smithi151 ceph-mon[36101]: pgmap v942: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2024-05-17T22:56:02.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:02 smithi151 ceph-mon[36101]: mds.? [v2:172.21.15.96:6835/3095044732,v1:172.21.15.96:6838/3095044732] up:boot 2024-05-17T22:56:02.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:02 smithi151 ceph-mon[36101]: mds.? [v2:172.21.15.96:6836/2353640503,v1:172.21.15.96:6839/2353640503] up:boot 2024-05-17T22:56:02.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:02 smithi151 ceph-mon[36101]: mds.? [v2:172.21.15.96:6834/1784052459,v1:172.21.15.96:6837/1784052459] up:boot 2024-05-17T22:56:02.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:02 smithi151 ceph-mon[36101]: mds.? [v2:172.21.15.151:6834/2304849160,v1:172.21.15.151:6837/2304849160] up:boot 2024-05-17T22:56:02.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:02 smithi151 ceph-mon[36101]: mds.? [v2:172.21.15.151:6835/3314664066,v1:172.21.15.151:6838/3314664066] up:boot 2024-05-17T22:56:02.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:02 smithi151 ceph-mon[36101]: mds.? [v2:172.21.15.151:6836/1196155727,v1:172.21.15.151:6839/1196155727] up:boot 2024-05-17T22:56:02.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:02 smithi151 ceph-mon[36101]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-17T22:56:02.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:02 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-17T22:56:02.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:02 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-17T22:56:02.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:02 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-17T22:56:02.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:02 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-17T22:56:02.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:02 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-17T22:56:02.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:02 smithi151 ceph-mon[36101]: from='mgr.14152 172.21.15.12:0/2221462737' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-17T22:56:03.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:03.929+0000 7fc97fcdb700 1 -- 172.21.15.12:0/1276769431 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9781080a0 msgr2=0x7fc97810fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:56:03.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:03.929+0000 7fc97fcdb700 1 --2- 172.21.15.12:0/1276769431 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9781080a0 0x7fc97810fa00 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fc97000b0a0 tx=0x7fc97000b3b0 comp rx=0 tx=0).stop 2024-05-17T22:56:03.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:03.929+0000 7fc97fcdb700 1 -- 172.21.15.12:0/1276769431 shutdown_connections 2024-05-17T22:56:03.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:03.929+0000 7fc97fcdb700 1 --2- 172.21.15.12:0/1276769431 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9781080a0 0x7fc97810fa00 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:03.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:03.929+0000 7fc97fcdb700 1 --2- 172.21.15.12:0/1276769431 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc978102e70 0x7fc978103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:03.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:03.929+0000 7fc97fcdb700 1 --2- 172.21.15.12:0/1276769431 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc978107750 0x7fc978107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:03.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:03.929+0000 7fc97fcdb700 1 -- 172.21.15.12:0/1276769431 >> 172.21.15.12:0/1276769431 conn(0x7fc978079ae0 msgr2=0x7fc978079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:56:03.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:03.929+0000 7fc97fcdb700 1 -- 172.21.15.12:0/1276769431 shutdown_connections 2024-05-17T22:56:03.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:03.929+0000 7fc97fcdb700 1 -- 172.21.15.12:0/1276769431 wait complete. 2024-05-17T22:56:03.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:03.930+0000 7fc97fcdb700 1 Processor -- start 2024-05-17T22:56:03.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:03.930+0000 7fc97fcdb700 1 -- start start 2024-05-17T22:56:03.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:03.930+0000 7fc97fcdb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc978102e70 0x7fc97819b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:56:03.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:03.930+0000 7fc97fcdb700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc978107750 0x7fc97819ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:56:03.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:03.930+0000 7fc97fcdb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9781080a0 0x7fc9781a65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:56:03.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:03.930+0000 7fc97fcdb700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc978112ce0 con 0x7fc9781080a0 2024-05-17T22:56:03.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:03.930+0000 7fc97fcdb700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc978112b60 con 0x7fc978102e70 2024-05-17T22:56:03.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:03.930+0000 7fc97fcdb700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc978112e60 con 0x7fc978107750 2024-05-17T22:56:03.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:03.930+0000 7fc97e278700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9781080a0 0x7fc9781a65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:56:03.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:03.930+0000 7fc97e278700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9781080a0 0x7fc9781a65b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42908/0 (socket says 172.21.15.12:42908) 2024-05-17T22:56:03.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:03.930+0000 7fc97e278700 1 -- 172.21.15.12:0/3749275198 learned_addr learned my addr 172.21.15.12:0/3749275198 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:56:03.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:03.930+0000 7fc97d276700 1 --2- 172.21.15.12:0/3749275198 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc978107750 0x7fc97819ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:56:03.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:03.931+0000 7fc97da77700 1 --2- 172.21.15.12:0/3749275198 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc978102e70 0x7fc97819b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:56:03.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:03.931+0000 7fc97e278700 1 -- 172.21.15.12:0/3749275198 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc978107750 msgr2=0x7fc97819ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:56:03.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:03.931+0000 7fc97e278700 1 --2- 172.21.15.12:0/3749275198 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc978107750 0x7fc97819ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:03.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:03.931+0000 7fc97e278700 1 -- 172.21.15.12:0/3749275198 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc978102e70 msgr2=0x7fc97819b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:56:03.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:03.931+0000 7fc97e278700 1 --2- 172.21.15.12:0/3749275198 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc978102e70 0x7fc97819b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:03.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:03.931+0000 7fc97e278700 1 -- 172.21.15.12:0/3749275198 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc97819c300 con 0x7fc9781080a0 2024-05-17T22:56:03.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:03.931+0000 7fc97e278700 1 --2- 172.21.15.12:0/3749275198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9781080a0 0x7fc9781a65b0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fc97000b830 tx=0x7fc970004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:56:03.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:03.931+0000 7fc96effd700 1 -- 172.21.15.12:0/3749275198 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc970021070 con 0x7fc9781080a0 2024-05-17T22:56:03.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:03.931+0000 7fc97fcdb700 1 -- 172.21.15.12:0/3749275198 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9781a6b50 con 0x7fc9781080a0 2024-05-17T22:56:03.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:03.931+0000 7fc96effd700 1 -- 172.21.15.12:0/3749275198 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc970007e30 con 0x7fc9781080a0 2024-05-17T22:56:03.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:03.931+0000 7fc96effd700 1 -- 172.21.15.12:0/3749275198 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc970004700 con 0x7fc9781080a0 2024-05-17T22:56:03.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:03.931+0000 7fc97fcdb700 1 -- 172.21.15.12:0/3749275198 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9781a7060 con 0x7fc9781080a0 2024-05-17T22:56:03.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:03.932+0000 7fc97fcdb700 1 -- 172.21.15.12:0/3749275198 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc978103280 con 0x7fc9781080a0 2024-05-17T22:56:03.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:03.935+0000 7fc96effd700 1 -- 172.21.15.12:0/3749275198 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc970003d70 con 0x7fc9781080a0 2024-05-17T22:56:03.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:03.936+0000 7fc96effd700 1 --2- 172.21.15.12:0/3749275198 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fc9640b2880 0x7fc9640b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:56:03.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:03.936+0000 7fc97da77700 1 --2- 172.21.15.12:0/3749275198 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fc9640b2880 0x7fc9640b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:56:03.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:03.936+0000 7fc96effd700 1 -- 172.21.15.12:0/3749275198 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9408+0+0 (secure 0 0 0) 0x7fc97000f020 con 0x7fc9781080a0 2024-05-17T22:56:03.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:03.937+0000 7fc97da77700 1 --2- 172.21.15.12:0/3749275198 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fc9640b2880 0x7fc9640b4d90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc974005950 tx=0x7fc97400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:56:03.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:03.939+0000 7fc96effd700 1 -- 172.21.15.12:0/3749275198 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc97002a020 con 0x7fc9781080a0 2024-05-17T22:56:04.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:04.210+0000 7fc97fcdb700 1 -- 172.21.15.12:0/3749275198 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc97806c670 con 0x7fc9781080a0 2024-05-17T22:56:04.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:04.271+0000 7fc96effd700 1 -- 172.21.15.12:0/3749275198 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v49) ==== 104+0+0 (secure 0 0 0) 0x7fc9700ab8c0 con 0x7fc9781080a0 2024-05-17T22:56:04.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:04.274+0000 7fc97fcdb700 1 -- 172.21.15.12:0/3749275198 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fc9640b2880 msgr2=0x7fc9640b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:56:04.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:04.274+0000 7fc97fcdb700 1 --2- 172.21.15.12:0/3749275198 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fc9640b2880 0x7fc9640b4d90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc974005950 tx=0x7fc97400b410 comp rx=0 tx=0).stop 2024-05-17T22:56:04.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:04.274+0000 7fc97fcdb700 1 -- 172.21.15.12:0/3749275198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9781080a0 msgr2=0x7fc9781a65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:56:04.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:04.274+0000 7fc97fcdb700 1 --2- 172.21.15.12:0/3749275198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9781080a0 0x7fc9781a65b0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fc97000b830 tx=0x7fc970004ce0 comp rx=0 tx=0).stop 2024-05-17T22:56:04.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:04.274+0000 7fc97fcdb700 1 -- 172.21.15.12:0/3749275198 shutdown_connections 2024-05-17T22:56:04.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:04.274+0000 7fc97fcdb700 1 --2- 172.21.15.12:0/3749275198 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fc9640b2880 0x7fc9640b4d90 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:04.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:04.274+0000 7fc97fcdb700 1 --2- 172.21.15.12:0/3749275198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9781080a0 0x7fc9781a65b0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:04.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:04.274+0000 7fc97fcdb700 1 --2- 172.21.15.12:0/3749275198 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc978102e70 0x7fc97819b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:04.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:04.274+0000 7fc97fcdb700 1 --2- 172.21.15.12:0/3749275198 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc978107750 0x7fc97819ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:04.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:04.274+0000 7fc97fcdb700 1 -- 172.21.15.12:0/3749275198 >> 172.21.15.12:0/3749275198 conn(0x7fc978079ae0 msgr2=0x7fc978105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:56:04.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:04.274+0000 7fc97fcdb700 1 -- 172.21.15.12:0/3749275198 shutdown_connections 2024-05-17T22:56:04.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:04.274+0000 7fc97fcdb700 1 -- 172.21.15.12:0/3749275198 wait complete. 2024-05-17T22:56:04.543 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:04 smithi151 ceph-mon[36101]: pgmap v944: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-17T22:56:04.543 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:04 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/3749275198' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-17T22:56:04.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:04 smithi096 ceph-mon[36065]: pgmap v944: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-17T22:56:04.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:04 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/3749275198' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-17T22:56:04.580 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:04 smithi012 ceph-mon[31632]: pgmap v944: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-17T22:56:04.580 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:04 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/3749275198' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-17T22:56:04.901 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-17T22:56:05.154 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:56:05.526 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:05 smithi012 ceph-mon[31632]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-17T22:56:05.526 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:05 smithi012 ceph-mon[31632]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-17T22:56:05.526 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:05 smithi012 ceph-mon[31632]: Cluster is now healthy 2024-05-17T22:56:05.526 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:05 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/3749275198' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-17T22:56:05.526 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:05 smithi012 ceph-mon[31632]: fsmap 12 up:standby 2024-05-17T22:56:05.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:05 smithi096 ceph-mon[36065]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-17T22:56:05.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:05 smithi096 ceph-mon[36065]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-17T22:56:05.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:05 smithi096 ceph-mon[36065]: Cluster is now healthy 2024-05-17T22:56:05.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:05 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/3749275198' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-17T22:56:05.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:05 smithi096 ceph-mon[36065]: fsmap 12 up:standby 2024-05-17T22:56:05.742 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:05 smithi151 ceph-mon[36101]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-17T22:56:05.742 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:05 smithi151 ceph-mon[36101]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-17T22:56:05.742 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:05 smithi151 ceph-mon[36101]: Cluster is now healthy 2024-05-17T22:56:05.743 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:05 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/3749275198' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-17T22:56:05.743 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:05 smithi151 ceph-mon[36101]: fsmap 12 up:standby 2024-05-17T22:56:06.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:06 smithi096 ceph-mon[36065]: pgmap v945: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 4 op/s 2024-05-17T22:56:06.549 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:06 smithi012 ceph-mon[31632]: pgmap v945: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 4 op/s 2024-05-17T22:56:06.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:06 smithi151 ceph-mon[36101]: pgmap v945: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 4 op/s 2024-05-17T22:56:06.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:06.922+0000 7f3aa9025700 1 -- 172.21.15.12:0/4247670799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3aa4102e00 msgr2=0x7f3aa4103210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:56:06.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:06.922+0000 7f3aa9025700 1 --2- 172.21.15.12:0/4247670799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3aa4102e00 0x7f3aa4103210 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f3a9400b0a0 tx=0x7f3a9400b3b0 comp rx=0 tx=0).stop 2024-05-17T22:56:06.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:06.922+0000 7f3aa9025700 1 -- 172.21.15.12:0/4247670799 shutdown_connections 2024-05-17T22:56:06.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:06.922+0000 7f3aa9025700 1 --2- 172.21.15.12:0/4247670799 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3aa4108080 0x7f3aa410f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:06.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:06.922+0000 7f3aa9025700 1 --2- 172.21.15.12:0/4247670799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3aa4102e00 0x7f3aa4103210 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:06.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:06.922+0000 7f3aa9025700 1 --2- 172.21.15.12:0/4247670799 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3aa4107730 0x7f3aa4107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:06.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:06.922+0000 7f3aa9025700 1 -- 172.21.15.12:0/4247670799 >> 172.21.15.12:0/4247670799 conn(0x7f3aa4079bd0 msgr2=0x7f3aa4079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:56:06.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:06.922+0000 7f3aa9025700 1 -- 172.21.15.12:0/4247670799 shutdown_connections 2024-05-17T22:56:06.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:06.922+0000 7f3aa9025700 1 -- 172.21.15.12:0/4247670799 wait complete. 2024-05-17T22:56:06.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:06.923+0000 7f3aa9025700 1 Processor -- start 2024-05-17T22:56:06.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:06.923+0000 7f3aa9025700 1 -- start start 2024-05-17T22:56:06.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:06.924+0000 7f3aa9025700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3aa4102e00 0x7f3aa419b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:56:06.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:06.924+0000 7f3aa9025700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3aa4107730 0x7f3aa419bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:56:06.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:06.924+0000 7f3aa9025700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3aa4108080 0x7f3aa419c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:56:06.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:06.924+0000 7f3aa9025700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa4112cb0 con 0x7f3aa4108080 2024-05-17T22:56:06.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:06.924+0000 7f3aa9025700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa4112b30 con 0x7f3aa4107730 2024-05-17T22:56:06.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:06.924+0000 7f3aa9025700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa4112e30 con 0x7f3aa4102e00 2024-05-17T22:56:06.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:06.924+0000 7f3aa359e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3aa4108080 0x7f3aa419c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:56:06.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:06.924+0000 7f3aa359e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3aa4108080 0x7f3aa419c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42928/0 (socket says 172.21.15.12:42928) 2024-05-17T22:56:06.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:06.924+0000 7f3aa359e700 1 -- 172.21.15.12:0/2985030742 learned_addr learned my addr 172.21.15.12:0/2985030742 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:56:06.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:06.924+0000 7f3aa259c700 1 --2- 172.21.15.12:0/2985030742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3aa4107730 0x7f3aa419bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:56:06.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:06.924+0000 7f3aa2d9d700 1 --2- 172.21.15.12:0/2985030742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3aa4102e00 0x7f3aa419b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:56:06.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:06.924+0000 7f3aa359e700 1 -- 172.21.15.12:0/2985030742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3aa4102e00 msgr2=0x7f3aa419b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:56:06.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:06.924+0000 7f3aa359e700 1 --2- 172.21.15.12:0/2985030742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3aa4102e00 0x7f3aa419b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:06.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:06.924+0000 7f3aa359e700 1 -- 172.21.15.12:0/2985030742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3aa4107730 msgr2=0x7f3aa419bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:56:06.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:06.924+0000 7f3aa359e700 1 --2- 172.21.15.12:0/2985030742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3aa4107730 0x7f3aa419bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:06.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:06.924+0000 7f3aa359e700 1 -- 172.21.15.12:0/2985030742 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa41a68c0 con 0x7f3aa4108080 2024-05-17T22:56:06.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:06.924+0000 7f3aa359e700 1 --2- 172.21.15.12:0/2985030742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3aa4108080 0x7f3aa419c0a0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c00d980 tx=0x7f3a9c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:56:06.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:06.925+0000 7f3a8bfff700 1 -- 172.21.15.12:0/2985030742 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c017070 con 0x7f3aa4108080 2024-05-17T22:56:06.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:06.925+0000 7f3a8bfff700 1 -- 172.21.15.12:0/2985030742 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a9c005020 con 0x7f3aa4108080 2024-05-17T22:56:06.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:06.925+0000 7f3aa9025700 1 -- 172.21.15.12:0/2985030742 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aa41a6bb0 con 0x7f3aa4108080 2024-05-17T22:56:06.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:06.925+0000 7f3a8bfff700 1 -- 172.21.15.12:0/2985030742 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c00fa10 con 0x7f3aa4108080 2024-05-17T22:56:06.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:06.925+0000 7f3aa9025700 1 -- 172.21.15.12:0/2985030742 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aa41a70a0 con 0x7f3aa4108080 2024-05-17T22:56:06.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:06.926+0000 7f3aa9025700 1 -- 172.21.15.12:0/2985030742 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3aa4103210 con 0x7f3aa4108080 2024-05-17T22:56:06.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:06.928+0000 7f3a8bfff700 1 -- 172.21.15.12:0/2985030742 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3a9c004a30 con 0x7f3aa4108080 2024-05-17T22:56:06.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:06.929+0000 7f3a8bfff700 1 --2- 172.21.15.12:0/2985030742 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f3a8c0b2880 0x7f3a8c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:56:06.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:06.929+0000 7f3aa2d9d700 1 --2- 172.21.15.12:0/2985030742 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f3a8c0b2880 0x7f3a8c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:56:06.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:06.929+0000 7f3a8bfff700 1 -- 172.21.15.12:0/2985030742 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9408+0+0 (secure 0 0 0) 0x7f3a9c0e2890 con 0x7f3aa4108080 2024-05-17T22:56:06.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:06.929+0000 7f3aa2d9d700 1 --2- 172.21.15.12:0/2985030742 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f3a8c0b2880 0x7f3a8c0b4d90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3a94009900 tx=0x7f3a9400b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:56:06.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:06.932+0000 7f3a8bfff700 1 -- 172.21.15.12:0/2985030742 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3a9c020020 con 0x7f3aa4108080 2024-05-17T22:56:07.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:07.147+0000 7f3aa9025700 1 -- 172.21.15.12:0/2985030742 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3aa4064440 con 0x7f3aa4108080 2024-05-17T22:56:07.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:07.285+0000 7f3a8bfff700 1 -- 172.21.15.12:0/2985030742 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v91) ==== 181+0+0 (secure 0 0 0) 0x7f3a9c01ae40 con 0x7f3aa4108080 2024-05-17T22:56:07.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:07.287+0000 7f3aa9025700 1 -- 172.21.15.12:0/2985030742 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f3a8c0b2880 msgr2=0x7f3a8c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:56:07.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:07.287+0000 7f3aa9025700 1 --2- 172.21.15.12:0/2985030742 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f3a8c0b2880 0x7f3a8c0b4d90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3a94009900 tx=0x7f3a9400b7b0 comp rx=0 tx=0).stop 2024-05-17T22:56:07.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:07.287+0000 7f3aa9025700 1 -- 172.21.15.12:0/2985030742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3aa4108080 msgr2=0x7f3aa419c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:56:07.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:07.287+0000 7f3aa9025700 1 --2- 172.21.15.12:0/2985030742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3aa4108080 0x7f3aa419c0a0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c00d980 tx=0x7f3a9c00dc90 comp rx=0 tx=0).stop 2024-05-17T22:56:07.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:07.287+0000 7f3aa9025700 1 -- 172.21.15.12:0/2985030742 shutdown_connections 2024-05-17T22:56:07.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:07.287+0000 7f3aa9025700 1 --2- 172.21.15.12:0/2985030742 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f3a8c0b2880 0x7f3a8c0b4d90 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:07.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:07.287+0000 7f3aa9025700 1 --2- 172.21.15.12:0/2985030742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3aa4108080 0x7f3aa419c0a0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:07.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:07.287+0000 7f3aa9025700 1 --2- 172.21.15.12:0/2985030742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3aa4102e00 0x7f3aa419b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:07.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:07.287+0000 7f3aa9025700 1 --2- 172.21.15.12:0/2985030742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3aa4107730 0x7f3aa419bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:07.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:07.287+0000 7f3aa9025700 1 -- 172.21.15.12:0/2985030742 >> 172.21.15.12:0/2985030742 conn(0x7f3aa4079bd0 msgr2=0x7f3aa41064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:56:07.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:07.288+0000 7f3aa9025700 1 -- 172.21.15.12:0/2985030742 shutdown_connections 2024-05-17T22:56:07.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:07.288+0000 7f3aa9025700 1 -- 172.21.15.12:0/2985030742 wait complete. 2024-05-17T22:56:07.291 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_metadata' removed 2024-05-17T22:56:07.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:07 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/2985030742' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-17T22:56:07.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:07 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/2985030742' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-17T22:56:07.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:07 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/2985030742' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-17T22:56:08.094 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-17T22:56:08.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:08 smithi012 ceph-mon[31632]: pgmap v946: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 511 B/s wr, 4 op/s 2024-05-17T22:56:08.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:08 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/2985030742' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-17T22:56:08.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:08 smithi012 ceph-mon[31632]: osdmap e91: 12 total, 12 up, 12 in 2024-05-17T22:56:08.359 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:56:08.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:08 smithi096 ceph-mon[36065]: pgmap v946: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 511 B/s wr, 4 op/s 2024-05-17T22:56:08.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:08 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/2985030742' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-17T22:56:08.549 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:08 smithi096 ceph-mon[36065]: osdmap e91: 12 total, 12 up, 12 in 2024-05-17T22:56:08.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:08 smithi151 ceph-mon[36101]: pgmap v946: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 511 B/s wr, 4 op/s 2024-05-17T22:56:08.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:08 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/2985030742' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-17T22:56:08.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:08 smithi151 ceph-mon[36101]: osdmap e91: 12 total, 12 up, 12 in 2024-05-17T22:56:10.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.050+0000 7fb3a5f22700 1 -- 172.21.15.12:0/2321117195 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3a0102e70 msgr2=0x7fb3a0103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:56:10.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.050+0000 7fb3a5f22700 1 --2- 172.21.15.12:0/2321117195 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3a0102e70 0x7fb3a0103280 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb39000b0a0 tx=0x7fb39000b3b0 comp rx=0 tx=0).stop 2024-05-17T22:56:10.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.050+0000 7fb3a5f22700 1 -- 172.21.15.12:0/2321117195 shutdown_connections 2024-05-17T22:56:10.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.050+0000 7fb3a5f22700 1 --2- 172.21.15.12:0/2321117195 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3a01080a0 0x7fb3a010fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:10.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.050+0000 7fb3a5f22700 1 --2- 172.21.15.12:0/2321117195 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3a0102e70 0x7fb3a0103280 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:10.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.050+0000 7fb3a5f22700 1 --2- 172.21.15.12:0/2321117195 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3a0107750 0x7fb3a0107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:10.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.050+0000 7fb3a5f22700 1 -- 172.21.15.12:0/2321117195 >> 172.21.15.12:0/2321117195 conn(0x7fb3a0079ae0 msgr2=0x7fb3a0079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:56:10.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.050+0000 7fb3a5f22700 1 -- 172.21.15.12:0/2321117195 shutdown_connections 2024-05-17T22:56:10.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.050+0000 7fb3a5f22700 1 -- 172.21.15.12:0/2321117195 wait complete. 2024-05-17T22:56:10.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.051+0000 7fb3a5f22700 1 Processor -- start 2024-05-17T22:56:10.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.051+0000 7fb3a5f22700 1 -- start start 2024-05-17T22:56:10.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.052+0000 7fb3a5f22700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3a0102e70 0x7fb3a0105200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:56:10.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.052+0000 7fb3a5f22700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3a0107750 0x7fb3a0105740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:56:10.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.052+0000 7fb3a5f22700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3a01080a0 0x7fb3a0105c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:56:10.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.052+0000 7fb3a5f22700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a0112d20 con 0x7fb3a0107750 2024-05-17T22:56:10.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.052+0000 7fb3a5f22700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a0112ba0 con 0x7fb3a0102e70 2024-05-17T22:56:10.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.052+0000 7fb3a5f22700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a0112ea0 con 0x7fb3a01080a0 2024-05-17T22:56:10.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.052+0000 7fb39effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3a0107750 0x7fb3a0105740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:56:10.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.052+0000 7fb39effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3a0107750 0x7fb3a0105740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35402/0 (socket says 172.21.15.12:35402) 2024-05-17T22:56:10.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.052+0000 7fb39effd700 1 -- 172.21.15.12:0/3631758875 learned_addr learned my addr 172.21.15.12:0/3631758875 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:56:10.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.052+0000 7fb39f7fe700 1 --2- 172.21.15.12:0/3631758875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3a0102e70 0x7fb3a0105200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:56:10.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.052+0000 7fb39ffff700 1 --2- 172.21.15.12:0/3631758875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3a01080a0 0x7fb3a0105c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:56:10.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.052+0000 7fb39effd700 1 -- 172.21.15.12:0/3631758875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3a01080a0 msgr2=0x7fb3a0105c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:56:10.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.052+0000 7fb39effd700 1 --2- 172.21.15.12:0/3631758875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3a01080a0 0x7fb3a0105c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:10.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.052+0000 7fb39effd700 1 -- 172.21.15.12:0/3631758875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3a0102e70 msgr2=0x7fb3a0105200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:56:10.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.052+0000 7fb39effd700 1 --2- 172.21.15.12:0/3631758875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3a0102e70 0x7fb3a0105200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:10.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.052+0000 7fb39effd700 1 -- 172.21.15.12:0/3631758875 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3a01a8b70 con 0x7fb3a0107750 2024-05-17T22:56:10.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.052+0000 7fb39effd700 1 --2- 172.21.15.12:0/3631758875 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3a0107750 0x7fb3a0105740 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb38800d980 tx=0x7fb38800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:56:10.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.052+0000 7fb39cff9700 1 -- 172.21.15.12:0/3631758875 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb388017070 con 0x7fb3a0107750 2024-05-17T22:56:10.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.053+0000 7fb3a5f22700 1 -- 172.21.15.12:0/3631758875 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3a01a8e60 con 0x7fb3a0107750 2024-05-17T22:56:10.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.053+0000 7fb3a5f22700 1 -- 172.21.15.12:0/3631758875 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3a01a9310 con 0x7fb3a0107750 2024-05-17T22:56:10.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.053+0000 7fb39cff9700 1 -- 172.21.15.12:0/3631758875 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb388009de0 con 0x7fb3a0107750 2024-05-17T22:56:10.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.053+0000 7fb39cff9700 1 -- 172.21.15.12:0/3631758875 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38800f3c0 con 0x7fb3a0107750 2024-05-17T22:56:10.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.054+0000 7fb3a5f22700 1 -- 172.21.15.12:0/3631758875 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb380005360 con 0x7fb3a0107750 2024-05-17T22:56:10.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.055+0000 7fb39cff9700 1 -- 172.21.15.12:0/3631758875 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb388009930 con 0x7fb3a0107750 2024-05-17T22:56:10.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.056+0000 7fb39cff9700 1 --2- 172.21.15.12:0/3631758875 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb38c0b2900 0x7fb38c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:56:10.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.056+0000 7fb39f7fe700 1 --2- 172.21.15.12:0/3631758875 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb38c0b2900 0x7fb38c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:56:10.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.056+0000 7fb39cff9700 1 -- 172.21.15.12:0/3631758875 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8883+0+0 (secure 0 0 0) 0x7fb3880e21e0 con 0x7fb3a0107750 2024-05-17T22:56:10.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.057+0000 7fb39f7fe700 1 --2- 172.21.15.12:0/3631758875 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb38c0b2900 0x7fb38c0b4e10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb390009fd0 tx=0x7fb390006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:56:10.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.059+0000 7fb39cff9700 1 -- 172.21.15.12:0/3631758875 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb3880a9c90 con 0x7fb3a0107750 2024-05-17T22:56:10.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.274+0000 7fb3a5f22700 1 -- 172.21.15.12:0/3631758875 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb380005a70 con 0x7fb3a0107750 2024-05-17T22:56:10.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.294+0000 7fb39cff9700 1 -- 172.21.15.12:0/3631758875 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v92) ==== 169+0+0 (secure 0 0 0) 0x7fb3880a9590 con 0x7fb3a0107750 2024-05-17T22:56:10.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.298+0000 7fb3a5f22700 1 -- 172.21.15.12:0/3631758875 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb38c0b2900 msgr2=0x7fb38c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:56:10.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.298+0000 7fb3a5f22700 1 --2- 172.21.15.12:0/3631758875 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb38c0b2900 0x7fb38c0b4e10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb390009fd0 tx=0x7fb390006210 comp rx=0 tx=0).stop 2024-05-17T22:56:10.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.298+0000 7fb3a5f22700 1 -- 172.21.15.12:0/3631758875 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3a0107750 msgr2=0x7fb3a0105740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:56:10.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.298+0000 7fb3a5f22700 1 --2- 172.21.15.12:0/3631758875 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3a0107750 0x7fb3a0105740 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb38800d980 tx=0x7fb38800dc90 comp rx=0 tx=0).stop 2024-05-17T22:56:10.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.298+0000 7fb3a5f22700 1 -- 172.21.15.12:0/3631758875 shutdown_connections 2024-05-17T22:56:10.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.298+0000 7fb3a5f22700 1 --2- 172.21.15.12:0/3631758875 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7fb38c0b2900 0x7fb38c0b4e10 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:10.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.298+0000 7fb3a5f22700 1 --2- 172.21.15.12:0/3631758875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3a01080a0 0x7fb3a0105c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:10.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.298+0000 7fb3a5f22700 1 --2- 172.21.15.12:0/3631758875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3a0102e70 0x7fb3a0105200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:10.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.298+0000 7fb3a5f22700 1 --2- 172.21.15.12:0/3631758875 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3a0107750 0x7fb3a0105740 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:10.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.298+0000 7fb3a5f22700 1 -- 172.21.15.12:0/3631758875 >> 172.21.15.12:0/3631758875 conn(0x7fb3a0079ae0 msgr2=0x7fb3a006c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:56:10.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.299+0000 7fb3a5f22700 1 -- 172.21.15.12:0/3631758875 shutdown_connections 2024-05-17T22:56:10.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:10.299+0000 7fb3a5f22700 1 -- 172.21.15.12:0/3631758875 wait complete. 2024-05-17T22:56:10.302 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_data' removed 2024-05-17T22:56:10.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:10 smithi096 ceph-mon[36065]: pgmap v948: 97 pgs: 97 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:56:10.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:10 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/3631758875' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-17T22:56:10.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:10 smithi012 ceph-mon[31632]: pgmap v948: 97 pgs: 97 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:56:10.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:10 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/3631758875' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-17T22:56:10.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:10 smithi151 ceph-mon[36101]: pgmap v948: 97 pgs: 97 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:56:10.741 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:10 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/3631758875' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-17T22:56:10.899 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 4d593780-149c-11ef-bc9a-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-05-17T22:56:11.156 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T22:56:11.577 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:11 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/3631758875' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-17T22:56:11.578 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:11 smithi012 ceph-mon[31632]: osdmap e92: 12 total, 12 up, 12 in 2024-05-17T22:56:11.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:11 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/3631758875' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-17T22:56:11.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:11 smithi151 ceph-mon[36101]: osdmap e92: 12 total, 12 up, 12 in 2024-05-17T22:56:11.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:11 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/3631758875' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-17T22:56:11.799 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:11 smithi096 ceph-mon[36065]: osdmap e92: 12 total, 12 up, 12 in 2024-05-17T22:56:12.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:12 smithi012 ceph-mon[31632]: pgmap v950: 33 pgs: 33 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:56:12.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:12 smithi151 ceph-mon[36101]: pgmap v950: 33 pgs: 33 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:56:12.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:12 smithi096 ceph-mon[36065]: pgmap v950: 33 pgs: 33 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:56:12.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:12.918+0000 7f7df0d1e700 1 -- 172.21.15.12:0/1491500758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7dec108840 msgr2=0x7f7dec108c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:56:12.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:12.918+0000 7f7df0d1e700 1 --2- 172.21.15.12:0/1491500758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7dec108840 0x7f7dec108c50 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f7de0009990 tx=0x7f7de0009ca0 comp rx=0 tx=0).stop 2024-05-17T22:56:12.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:12.918+0000 7f7df0d1e700 1 -- 172.21.15.12:0/1491500758 shutdown_connections 2024-05-17T22:56:12.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:12.918+0000 7f7df0d1e700 1 --2- 172.21.15.12:0/1491500758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7dec109190 0x7f7dec110af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:12.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:12.918+0000 7f7df0d1e700 1 --2- 172.21.15.12:0/1491500758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7dec103f10 0x7f7dec104320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:12.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:12.918+0000 7f7df0d1e700 1 --2- 172.21.15.12:0/1491500758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7dec108840 0x7f7dec108c50 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:12.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:12.918+0000 7f7df0d1e700 1 -- 172.21.15.12:0/1491500758 >> 172.21.15.12:0/1491500758 conn(0x7f7dec0ff9b0 msgr2=0x7f7dec101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:56:12.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:12.918+0000 7f7df0d1e700 1 -- 172.21.15.12:0/1491500758 shutdown_connections 2024-05-17T22:56:12.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:12.918+0000 7f7df0d1e700 1 -- 172.21.15.12:0/1491500758 wait complete. 2024-05-17T22:56:12.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:12.919+0000 7f7df0d1e700 1 Processor -- start 2024-05-17T22:56:12.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:12.919+0000 7f7df0d1e700 1 -- start start 2024-05-17T22:56:12.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:12.919+0000 7f7df0d1e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7dec103f10 0x7f7dec19a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:56:12.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:12.919+0000 7f7df0d1e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7dec108840 0x7f7dec19ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:56:12.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:12.919+0000 7f7df0d1e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7dec109190 0x7f7dec19eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:56:12.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:12.919+0000 7f7df0d1e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7dec113da0 con 0x7f7dec103f10 2024-05-17T22:56:12.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:12.919+0000 7f7df0d1e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7dec113c20 con 0x7f7dec108840 2024-05-17T22:56:12.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:12.919+0000 7f7df0d1e700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7dec113f20 con 0x7f7dec109190 2024-05-17T22:56:12.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:12.919+0000 7f7dea59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7dec103f10 0x7f7dec19a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:56:12.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:12.919+0000 7f7dea59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7dec103f10 0x7f7dec19a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35422/0 (socket says 172.21.15.12:35422) 2024-05-17T22:56:12.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:12.919+0000 7f7dea59c700 1 -- 172.21.15.12:0/3298342296 learned_addr learned my addr 172.21.15.12:0/3298342296 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-05-17T22:56:12.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:12.920+0000 7f7de9d9b700 1 --2- 172.21.15.12:0/3298342296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7dec108840 0x7f7dec19ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:56:12.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:12.920+0000 7f7dead9d700 1 --2- 172.21.15.12:0/3298342296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7dec109190 0x7f7dec19eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:56:12.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:12.920+0000 7f7dea59c700 1 -- 172.21.15.12:0/3298342296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7dec109190 msgr2=0x7f7dec19eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:56:12.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:12.920+0000 7f7dea59c700 1 --2- 172.21.15.12:0/3298342296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7dec109190 0x7f7dec19eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:12.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:12.920+0000 7f7dea59c700 1 -- 172.21.15.12:0/3298342296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7dec108840 msgr2=0x7f7dec19ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:56:12.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:12.920+0000 7f7dea59c700 1 --2- 172.21.15.12:0/3298342296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7dec108840 0x7f7dec19ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:12.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:12.920+0000 7f7dea59c700 1 -- 172.21.15.12:0/3298342296 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dec19f690 con 0x7f7dec103f10 2024-05-17T22:56:12.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:12.920+0000 7f7de9d9b700 1 --2- 172.21.15.12:0/3298342296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7dec108840 0x7f7dec19ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:56:12.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:12.920+0000 7f7dea59c700 1 --2- 172.21.15.12:0/3298342296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7dec103f10 0x7f7dec19a7c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc00e9a0 tx=0x7f7ddc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:56:12.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:12.920+0000 7f7ddb7fe700 1 -- 172.21.15.12:0/3298342296 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc00ca10 con 0x7f7dec103f10 2024-05-17T22:56:12.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:12.920+0000 7f7df0d1e700 1 -- 172.21.15.12:0/3298342296 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dec19f980 con 0x7f7dec103f10 2024-05-17T22:56:12.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:12.920+0000 7f7ddb7fe700 1 -- 172.21.15.12:0/3298342296 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ddc00cbb0 con 0x7f7dec103f10 2024-05-17T22:56:12.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:12.920+0000 7f7ddb7fe700 1 -- 172.21.15.12:0/3298342296 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc018680 con 0x7f7dec103f10 2024-05-17T22:56:12.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:12.921+0000 7f7df0d1e700 1 -- 172.21.15.12:0/3298342296 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dec1a72a0 con 0x7f7dec103f10 2024-05-17T22:56:12.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:12.922+0000 7f7df0d1e700 1 -- 172.21.15.12:0/3298342296 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dec104320 con 0x7f7dec103f10 2024-05-17T22:56:12.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:12.924+0000 7f7ddb7fe700 1 -- 172.21.15.12:0/3298342296 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7ddc010430 con 0x7f7dec103f10 2024-05-17T22:56:12.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:12.924+0000 7f7ddb7fe700 1 --2- 172.21.15.12:0/3298342296 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f7dd40b2880 0x7f7dd40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:56:12.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:12.925+0000 7f7de9d9b700 1 --2- 172.21.15.12:0/3298342296 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f7dd40b2880 0x7f7dd40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:56:12.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:12.925+0000 7f7ddb7fe700 1 -- 172.21.15.12:0/3298342296 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8324+0+0 (secure 0 0 0) 0x7f7ddc0e2720 con 0x7f7dec103f10 2024-05-17T22:56:12.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:12.925+0000 7f7de9d9b700 1 --2- 172.21.15.12:0/3298342296 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f7dd40b2880 0x7f7dd40b4d90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7de000b2e0 tx=0x7f7de0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:56:12.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:12.928+0000 7f7ddb7fe700 1 -- 172.21.15.12:0/3298342296 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7ddc0aa3e0 con 0x7f7dec103f10 2024-05-17T22:56:13.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:13.144+0000 7f7df0d1e700 1 -- 172.21.15.12:0/3298342296 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7dec19bb20 con 0x7f7dec103f10 2024-05-17T22:56:13.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:13.308+0000 7f7ddb7fe700 1 -- 172.21.15.12:0/3298342296 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v93) ==== 178+0+0 (secure 0 0 0) 0x7f7ddc0a9ce0 con 0x7f7dec103f10 2024-05-17T22:56:13.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:13.312+0000 7f7df0d1e700 1 -- 172.21.15.12:0/3298342296 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f7dd40b2880 msgr2=0x7f7dd40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:56:13.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:13.312+0000 7f7df0d1e700 1 --2- 172.21.15.12:0/3298342296 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f7dd40b2880 0x7f7dd40b4d90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7de000b2e0 tx=0x7f7de0005960 comp rx=0 tx=0).stop 2024-05-17T22:56:13.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:13.312+0000 7f7df0d1e700 1 -- 172.21.15.12:0/3298342296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7dec103f10 msgr2=0x7f7dec19a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:56:13.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:13.312+0000 7f7df0d1e700 1 --2- 172.21.15.12:0/3298342296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7dec103f10 0x7f7dec19a7c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc00e9a0 tx=0x7f7ddc00ecb0 comp rx=0 tx=0).stop 2024-05-17T22:56:13.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:13.312+0000 7f7df0d1e700 1 -- 172.21.15.12:0/3298342296 shutdown_connections 2024-05-17T22:56:13.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:13.312+0000 7f7df0d1e700 1 --2- 172.21.15.12:0/3298342296 >> [v2:172.21.15.12:6800/1388514047,v1:172.21.15.12:6801/1388514047] conn(0x7f7dd40b2880 0x7f7dd40b4d90 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:13.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:13.312+0000 7f7df0d1e700 1 --2- 172.21.15.12:0/3298342296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7dec109190 0x7f7dec19eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:13.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:13.312+0000 7f7df0d1e700 1 --2- 172.21.15.12:0/3298342296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7dec103f10 0x7f7dec19a7c0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:13.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:13.312+0000 7f7df0d1e700 1 --2- 172.21.15.12:0/3298342296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7dec108840 0x7f7dec19ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:56:13.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:13.312+0000 7f7df0d1e700 1 -- 172.21.15.12:0/3298342296 >> 172.21.15.12:0/3298342296 conn(0x7f7dec0ff9b0 msgr2=0x7f7dec101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:56:13.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:13.313+0000 7f7df0d1e700 1 -- 172.21.15.12:0/3298342296 shutdown_connections 2024-05-17T22:56:13.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-05-17T22:56:13.313+0000 7f7df0d1e700 1 -- 172.21.15.12:0/3298342296 wait complete. 2024-05-17T22:56:13.317 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_data_ec' removed 2024-05-17T22:56:13.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:13 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/3298342296' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-17T22:56:13.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:13 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/3298342296' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-17T22:56:13.798 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:13 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/3298342296' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-17T22:56:14.052 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-17T22:56:14.094 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-17T22:56:14.150 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-17T22:56:14.192 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-17T22:56:14.192 DEBUG:teuthology.orchestra.run.smithi012:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-17T22:56:14.219 DEBUG:teuthology.orchestra.run.smithi096:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-17T22:56:14.242 DEBUG:teuthology.orchestra.run.smithi151:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-17T22:56:14.263 INFO:tasks.cephadm:Stopping all daemons... 2024-05-17T22:56:14.263 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-17T22:56:14.263 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-4d593780-149c-11ef-bc9a-c7b262605968@mon.a 2024-05-17T22:56:14.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:14 smithi151 ceph-mon[36101]: pgmap v951: 33 pgs: 33 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:56:14.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:14 smithi151 ceph-mon[36101]: from='client.? 172.21.15.12:0/3298342296' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-05-17T22:56:14.490 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:14 smithi151 ceph-mon[36101]: osdmap e93: 12 total, 12 up, 12 in 2024-05-17T22:56:14.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:14 smithi096 ceph-mon[36065]: pgmap v951: 33 pgs: 33 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:56:14.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:14 smithi096 ceph-mon[36065]: from='client.? 172.21.15.12:0/3298342296' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-05-17T22:56:14.548 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:14 smithi096 ceph-mon[36065]: osdmap e93: 12 total, 12 up, 12 in 2024-05-17T22:56:14.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:14 smithi012 ceph-mon[31632]: pgmap v951: 33 pgs: 33 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T22:56:14.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:14 smithi012 ceph-mon[31632]: from='client.? 172.21.15.12:0/3298342296' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-05-17T22:56:14.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:14 smithi012 ceph-mon[31632]: osdmap e93: 12 total, 12 up, 12 in 2024-05-17T22:56:14.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:14 smithi012 systemd[1]: Stopping Ceph mon.a for 4d593780-149c-11ef-bc9a-c7b262605968... 2024-05-17T22:56:15.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:14 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mon-a[31628]: 2024-05-17T22:56:14.774+0000 7f780782a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T22:56:15.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:14 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mon-a[31628]: 2024-05-17T22:56:14.774+0000 7f780782a700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-17T22:56:15.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:14 smithi012 podman[114829]: 2024-05-17 22:56:14.861335034 +0000 UTC m=+0.243295627 container died 094715d271e66aa5d1c22c288d8f4145e04d35b0727765f704ae06c0acb67537 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514) 2024-05-17T22:56:15.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:15 smithi012 podman[114829]: 2024-05-17 22:56:15.590610084 +0000 UTC m=+0.972570671 container cleanup 094715d271e66aa5d1c22c288d8f4145e04d35b0727765f704ae06c0acb67537 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514) 2024-05-17T22:56:15.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:May 17 22:56:15 smithi012 bash[114829]: ceph-4d593780-149c-11ef-bc9a-c7b262605968-mon-a 2024-05-17T22:56:16.005 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@mon.a.service' 2024-05-17T22:56:16.083 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T22:56:16.084 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-17T22:56:16.084 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-17T22:56:16.084 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-4d593780-149c-11ef-bc9a-c7b262605968@mon.b 2024-05-17T22:56:16.410 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:16 smithi096 systemd[1]: Stopping Ceph mon.b for 4d593780-149c-11ef-bc9a-c7b262605968... 2024-05-17T22:56:16.762 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:16 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mon-b[36060]: 2024-05-17T22:56:16.407+0000 7f21e097b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T22:56:16.762 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:16 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mon-b[36060]: 2024-05-17T22:56:16.407+0000 7f21e097b700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-17T22:56:16.762 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:16 smithi096 podman[72014]: 2024-05-17 22:56:16.489690703 +0000 UTC m=+0.177649671 container died 10fbdf09832dc9524878610c7c1ad0e86c74c45cfdada026bef7ff40d8c8f8c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mon-b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a) 2024-05-17T22:56:17.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:16 smithi096 podman[72014]: 2024-05-17 22:56:16.850979126 +0000 UTC m=+0.538938085 container cleanup 10fbdf09832dc9524878610c7c1ad0e86c74c45cfdada026bef7ff40d8c8f8c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mon-b, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-17T22:56:17.048 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 17 22:56:16 smithi096 bash[72014]: ceph-4d593780-149c-11ef-bc9a-c7b262605968-mon-b 2024-05-17T22:56:17.540 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@mon.b.service' 2024-05-17T22:56:17.582 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T22:56:17.582 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-17T22:56:17.583 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-17T22:56:17.583 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-4d593780-149c-11ef-bc9a-c7b262605968@mon.c 2024-05-17T22:56:17.910 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:17 smithi151 systemd[1]: Stopping Ceph mon.c for 4d593780-149c-11ef-bc9a-c7b262605968... 2024-05-17T22:56:18.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:17 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mon-c[36084]: 2024-05-17T22:56:17.908+0000 7ff00fbb7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T22:56:18.694 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:17 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mon-c[36084]: 2024-05-17T22:56:17.908+0000 7ff00fbb7700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-17T22:56:18.694 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:17 smithi151 podman[73164]: 2024-05-17 22:56:17.969775228 +0000 UTC m=+0.185591217 container died 4e7aba1032ccfa00d24703300e98d418673be9849b41ccf2981f57a0ac8e13ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240514) 2024-05-17T22:56:18.695 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:18 smithi151 podman[73164]: 2024-05-17 22:56:18.385572668 +0000 UTC m=+0.601388645 container cleanup 4e7aba1032ccfa00d24703300e98d418673be9849b41ccf2981f57a0ac8e13ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mon-c, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.build-date=20240514, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-17T22:56:18.695 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 17 22:56:18 smithi151 bash[73164]: ceph-4d593780-149c-11ef-bc9a-c7b262605968-mon-c 2024-05-17T22:56:18.781 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@mon.c.service' 2024-05-17T22:56:18.825 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T22:56:18.825 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-17T22:56:18.825 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-17T22:56:18.826 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-4d593780-149c-11ef-bc9a-c7b262605968@mgr.x 2024-05-17T22:56:19.157 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:56:18 smithi012 systemd[1]: Stopping Ceph mgr.x for 4d593780-149c-11ef-bc9a-c7b262605968... 2024-05-17T22:56:19.524 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:56:19 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x[31863]: 2024-05-17T22:56:19.154+0000 7f3bf8888700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T22:56:19.524 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:56:19 smithi012 podman[114945]: 2024-05-17 22:56:19.195353906 +0000 UTC m=+0.161231993 container died 16e3da619b4f49bf7fe5b2376cc0de31ff82e408d9f88026cf9d42d2bf577e87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, ceph=True) 2024-05-17T22:56:19.841 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:56:19 smithi012 podman[114945]: 2024-05-17 22:56:19.639005792 +0000 UTC m=+0.604883872 container cleanup 16e3da619b4f49bf7fe5b2376cc0de31ff82e408d9f88026cf9d42d2bf577e87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-17T22:56:19.841 INFO:journalctl@ceph.mgr.x.smithi012.stdout:May 17 22:56:19 smithi012 bash[114945]: ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-x 2024-05-17T22:56:20.079 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@mgr.x.service' 2024-05-17T22:56:20.116 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T22:56:20.116 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-17T22:56:20.116 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-17T22:56:20.116 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-4d593780-149c-11ef-bc9a-c7b262605968@mgr.y 2024-05-17T22:56:20.445 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 17 22:56:20 smithi096 systemd[1]: Stopping Ceph mgr.y for 4d593780-149c-11ef-bc9a-c7b262605968... 2024-05-17T22:56:20.744 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 17 22:56:20 smithi096 podman[72140]: 2024-05-17 22:56:20.444049797 +0000 UTC m=+0.135255022 container died b2c251c0b1058e3ac7181754684a563a14688fccbd6c1e95cbee8ed1f66db4f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-17T22:56:21.048 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 17 22:56:20 smithi096 podman[72140]: 2024-05-17 22:56:20.841251792 +0000 UTC m=+0.532457017 container cleanup b2c251c0b1058e3ac7181754684a563a14688fccbd6c1e95cbee8ed1f66db4f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-y, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T22:56:21.049 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 17 22:56:20 smithi096 bash[72140]: ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-y 2024-05-17T22:56:21.546 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@mgr.y.service' 2024-05-17T22:56:21.579 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T22:56:21.579 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-17T22:56:21.579 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-17T22:56:21.579 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-4d593780-149c-11ef-bc9a-c7b262605968@mgr.z 2024-05-17T22:56:21.924 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 17 22:56:21 smithi151 systemd[1]: Stopping Ceph mgr.z for 4d593780-149c-11ef-bc9a-c7b262605968... 2024-05-17T22:56:22.240 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 17 22:56:21 smithi151 podman[73289]: 2024-05-17 22:56:21.922869428 +0000 UTC m=+0.152962150 container died 8bc7ce2a5c7341dae379993da8738c80e651af3644a203981a85952085bcee7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-17T22:56:22.899 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 17 22:56:22 smithi151 podman[73289]: 2024-05-17 22:56:22.602931701 +0000 UTC m=+0.833024378 container cleanup 8bc7ce2a5c7341dae379993da8738c80e651af3644a203981a85952085bcee7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-z, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux ) 2024-05-17T22:56:22.899 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 17 22:56:22 smithi151 bash[73289]: ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-z 2024-05-17T22:56:23.153 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 17 22:56:22 smithi151 podman[73303]: 2024-05-17 22:56:22.897123573 +0000 UTC m=+0.864481151 container remove 8bc7ce2a5c7341dae379993da8738c80e651af3644a203981a85952085bcee7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mgr-z, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-17T22:56:23.153 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 17 22:56:22 smithi151 systemd[1]: ceph-4d593780-149c-11ef-bc9a-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-05-17T22:56:23.157 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@mgr.z.service' 2024-05-17T22:56:23.190 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T22:56:23.190 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-17T22:56:23.191 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-17T22:56:23.191 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-4d593780-149c-11ef-bc9a-c7b262605968@osd.0 2024-05-17T22:56:23.521 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 17 22:56:23 smithi012 systemd[1]: Stopping Ceph osd.0 for 4d593780-149c-11ef-bc9a-c7b262605968... 2024-05-17T22:56:23.841 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 17 22:56:23 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-0[45860]: 2024-05-17T22:56:23.518+0000 7fb4d54e7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T22:56:23.841 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 17 22:56:23 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-0[45860]: 2024-05-17T22:56:23.518+0000 7fb4d54e7700 -1 osd.0 93 *** Got signal Terminated *** 2024-05-17T22:56:23.841 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 17 22:56:23 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-0[45860]: 2024-05-17T22:56:23.518+0000 7fb4d54e7700 -1 osd.0 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T22:56:29.379 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 17 22:56:29 smithi012 podman[115067]: 2024-05-17 22:56:29.120057922 +0000 UTC m=+5.726423485 container died d0519659339b5836c7d7beffedf58d94dc5e58206683cefd919da71917d40371 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-17T22:56:29.841 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 17 22:56:29 smithi012 podman[115067]: 2024-05-17 22:56:29.568415524 +0000 UTC m=+6.174781085 container cleanup d0519659339b5836c7d7beffedf58d94dc5e58206683cefd919da71917d40371 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T22:56:29.841 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 17 22:56:29 smithi012 bash[115067]: ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-0 2024-05-17T22:56:30.198 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 17 22:56:29 smithi012 podman[115083]: 2024-05-17 22:56:29.845248478 +0000 UTC m=+0.724117847 container remove d0519659339b5836c7d7beffedf58d94dc5e58206683cefd919da71917d40371 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-17T22:56:30.450 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 17 22:56:30 smithi012 podman[115170]: 2024-05-17 22:56:30.196224116 +0000 UTC m=+0.109440584 container create 0c682e44c88c4f025d746fed3464c302a8ea59f1e9ea68625a9b714eca700753 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-0-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-17T22:56:30.450 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 17 22:56:30 smithi012 podman[115170]: 2024-05-17 22:56:30.104636596 +0000 UTC m=+0.017853084 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T22:56:30.450 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 17 22:56:30 smithi012 podman[115170]: 2024-05-17 22:56:30.448832571 +0000 UTC m=+0.362049049 container init 0c682e44c88c4f025d746fed3464c302a8ea59f1e9ea68625a9b714eca700753 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T22:56:30.841 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 17 22:56:30 smithi012 podman[115170]: 2024-05-17 22:56:30.4525377 +0000 UTC m=+0.365754182 container start 0c682e44c88c4f025d746fed3464c302a8ea59f1e9ea68625a9b714eca700753 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-0-deactivate, ceph=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-17T22:56:30.842 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 17 22:56:30 smithi012 podman[115170]: 2024-05-17 22:56:30.607738505 +0000 UTC m=+0.520954980 container attach 0c682e44c88c4f025d746fed3464c302a8ea59f1e9ea68625a9b714eca700753 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T22:56:31.310 INFO:journalctl@ceph.osd.0.smithi012.stdout:May 17 22:56:31 smithi012 podman[115170]: 2024-05-17 22:56:31.024864382 +0000 UTC m=+0.938080857 container died 0c682e44c88c4f025d746fed3464c302a8ea59f1e9ea68625a9b714eca700753 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514) 2024-05-17T22:56:31.648 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@osd.0.service' 2024-05-17T22:56:31.688 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T22:56:31.688 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-17T22:56:31.689 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-17T22:56:31.689 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-4d593780-149c-11ef-bc9a-c7b262605968@osd.1 2024-05-17T22:56:31.998 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 17 22:56:31 smithi096 systemd[1]: Stopping Ceph osd.1 for 4d593780-149c-11ef-bc9a-c7b262605968... 2024-05-17T22:56:32.298 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 17 22:56:31 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-1[40656]: 2024-05-17T22:56:31.995+0000 7f936e336700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T22:56:32.298 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 17 22:56:31 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-1[40656]: 2024-05-17T22:56:31.995+0000 7f936e336700 -1 osd.1 93 *** Got signal Terminated *** 2024-05-17T22:56:32.299 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 17 22:56:31 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-1[40656]: 2024-05-17T22:56:31.995+0000 7f936e336700 -1 osd.1 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T22:56:37.932 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 17 22:56:37 smithi096 podman[72263]: 2024-05-17 22:56:37.630287333 +0000 UTC m=+5.745738702 container died 3e5db6f9040acbaaa8dd72d3a91f6d1190059d32b7f4d37948be5cef22f63cad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-1, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-17T22:56:38.257 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 17 22:56:38 smithi096 podman[72263]: 2024-05-17 22:56:38.00381388 +0000 UTC m=+6.119265246 container cleanup 3e5db6f9040acbaaa8dd72d3a91f6d1190059d32b7f4d37948be5cef22f63cad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-1, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True) 2024-05-17T22:56:38.257 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 17 22:56:38 smithi096 bash[72263]: ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-1 2024-05-17T22:56:38.548 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 17 22:56:38 smithi096 podman[72280]: 2024-05-17 22:56:38.255555932 +0000 UTC m=+0.627174494 container remove 3e5db6f9040acbaaa8dd72d3a91f6d1190059d32b7f4d37948be5cef22f63cad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-1, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-17T22:56:38.869 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 17 22:56:38 smithi096 podman[72371]: 2024-05-17 22:56:38.490270518 +0000 UTC m=+0.017695189 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T22:56:38.870 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 17 22:56:38 smithi096 podman[72371]: 2024-05-17 22:56:38.598668547 +0000 UTC m=+0.126093198 container create 41a3885b79b1d661b5739982cc9d8234dd46c8f3a0304ce96b1a355c3541a169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-17T22:56:39.179 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 17 22:56:38 smithi096 podman[72371]: 2024-05-17 22:56:38.868142496 +0000 UTC m=+0.395567153 container init 41a3885b79b1d661b5739982cc9d8234dd46c8f3a0304ce96b1a355c3541a169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-17T22:56:39.180 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 17 22:56:38 smithi096 podman[72371]: 2024-05-17 22:56:38.871424529 +0000 UTC m=+0.398849190 container start 41a3885b79b1d661b5739982cc9d8234dd46c8f3a0304ce96b1a355c3541a169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, io.buildah.version=1.33.5) 2024-05-17T22:56:39.180 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 17 22:56:38 smithi096 podman[72371]: 2024-05-17 22:56:38.968259896 +0000 UTC m=+0.495684553 container attach 41a3885b79b1d661b5739982cc9d8234dd46c8f3a0304ce96b1a355c3541a169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T22:56:39.180 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 17 22:56:39 smithi096 podman[72371]: 2024-05-17 22:56:39.177832945 +0000 UTC m=+0.705257606 container died 41a3885b79b1d661b5739982cc9d8234dd46c8f3a0304ce96b1a355c3541a169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-1-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.vendor=CentOS) 2024-05-17T22:56:39.828 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@osd.1.service' 2024-05-17T22:56:39.867 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T22:56:39.867 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-17T22:56:39.868 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-17T22:56:39.868 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-4d593780-149c-11ef-bc9a-c7b262605968@osd.2 2024-05-17T22:56:40.241 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 17 22:56:39 smithi151 systemd[1]: Stopping Ceph osd.2 for 4d593780-149c-11ef-bc9a-c7b262605968... 2024-05-17T22:56:40.517 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 17 22:56:40 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-2[41218]: 2024-05-17T22:56:40.241+0000 7fa941ce4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T22:56:40.518 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 17 22:56:40 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-2[41218]: 2024-05-17T22:56:40.241+0000 7fa941ce4700 -1 osd.2 93 *** Got signal Terminated *** 2024-05-17T22:56:40.518 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 17 22:56:40 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-2[41218]: 2024-05-17T22:56:40.241+0000 7fa941ce4700 -1 osd.2 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T22:56:45.990 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 17 22:56:45 smithi151 podman[73415]: 2024-05-17 22:56:45.711925727 +0000 UTC m=+5.655315828 container died 5accbd07f0691af93993e47193e6fd6ae241cf76d46d090a4eeb752ecda18481 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS) 2024-05-17T22:56:46.355 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 17 22:56:46 smithi151 podman[73415]: 2024-05-17 22:56:46.110493028 +0000 UTC m=+6.053883119 container cleanup 5accbd07f0691af93993e47193e6fd6ae241cf76d46d090a4eeb752ecda18481 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_BRANCH=HEAD) 2024-05-17T22:56:46.356 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 17 22:56:46 smithi151 bash[73415]: ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-2 2024-05-17T22:56:46.690 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 17 22:56:46 smithi151 podman[73445]: 2024-05-17 22:56:46.354147901 +0000 UTC m=+0.642537879 container remove 5accbd07f0691af93993e47193e6fd6ae241cf76d46d090a4eeb752ecda18481 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-05-17T22:56:46.990 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 17 22:56:46 smithi151 podman[73532]: 2024-05-17 22:56:46.591557938 +0000 UTC m=+0.017042244 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T22:56:46.991 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 17 22:56:46 smithi151 podman[73532]: 2024-05-17 22:56:46.705730304 +0000 UTC m=+0.131214651 container create 2af980dbb08df252e494c15dcd633a1812f2c2c382663c6a091862c91cc62369 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True) 2024-05-17T22:56:47.380 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 17 22:56:47 smithi151 podman[73532]: 2024-05-17 22:56:47.001074784 +0000 UTC m=+0.426559203 container init 2af980dbb08df252e494c15dcd633a1812f2c2c382663c6a091862c91cc62369 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-17T22:56:47.380 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 17 22:56:47 smithi151 podman[73532]: 2024-05-17 22:56:47.004324499 +0000 UTC m=+0.429808809 container start 2af980dbb08df252e494c15dcd633a1812f2c2c382663c6a091862c91cc62369 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2) 2024-05-17T22:56:47.381 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 17 22:56:47 smithi151 podman[73532]: 2024-05-17 22:56:47.100915119 +0000 UTC m=+0.526399469 container attach 2af980dbb08df252e494c15dcd633a1812f2c2c382663c6a091862c91cc62369 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-17T22:56:47.715 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 17 22:56:47 smithi151 podman[73532]: 2024-05-17 22:56:47.379080438 +0000 UTC m=+0.804564740 container died 2af980dbb08df252e494c15dcd633a1812f2c2c382663c6a091862c91cc62369 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-2-deactivate, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-17T22:56:48.031 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@osd.2.service' 2024-05-17T22:56:48.072 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T22:56:48.072 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-17T22:56:48.072 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-17T22:56:48.072 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-4d593780-149c-11ef-bc9a-c7b262605968@osd.3 2024-05-17T22:56:48.399 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 17 22:56:48 smithi012 systemd[1]: Stopping Ceph osd.3 for 4d593780-149c-11ef-bc9a-c7b262605968... 2024-05-17T22:56:48.841 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 17 22:56:48 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-3[52771]: 2024-05-17T22:56:48.396+0000 7f71e9885700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T22:56:48.841 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 17 22:56:48 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-3[52771]: 2024-05-17T22:56:48.396+0000 7f71e9885700 -1 osd.3 93 *** Got signal Terminated *** 2024-05-17T22:56:48.841 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 17 22:56:48 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-3[52771]: 2024-05-17T22:56:48.396+0000 7f71e9885700 -1 osd.3 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T22:56:49.741 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:56:49 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5[48151]: 2024-05-17T22:56:49.449+0000 7f96d04e8700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.562993+0000 front 2024-05-17T22:56:24.562934+0000 (oldest deadline 2024-05-17T22:56:49.262999+0000) 2024-05-17T22:56:49.842 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:56:49 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:56:49.504+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:26.427462+0000 front 2024-05-17T22:56:26.427603+0000 (oldest deadline 2024-05-17T22:56:49.327570+0000) 2024-05-17T22:56:50.591 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:56:50 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:56:50.460+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:26.427462+0000 front 2024-05-17T22:56:26.427603+0000 (oldest deadline 2024-05-17T22:56:49.327570+0000) 2024-05-17T22:56:50.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:56:50 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:56:50.255+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:56:50.740 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:56:50 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5[48151]: 2024-05-17T22:56:50.410+0000 7f96d04e8700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.562993+0000 front 2024-05-17T22:56:24.562934+0000 (oldest deadline 2024-05-17T22:56:49.262999+0000) 2024-05-17T22:56:51.048 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:56:50 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:56:50.680+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:56:51.591 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:56:51 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:56:51.444+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:26.427462+0000 front 2024-05-17T22:56:26.427603+0000 (oldest deadline 2024-05-17T22:56:49.327570+0000) 2024-05-17T22:56:51.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:56:51 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:56:51.228+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:56:51.740 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:56:51 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5[48151]: 2024-05-17T22:56:51.426+0000 7f96d04e8700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.562993+0000 front 2024-05-17T22:56:24.562934+0000 (oldest deadline 2024-05-17T22:56:49.262999+0000) 2024-05-17T22:56:52.048 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:56:51 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:56:51.699+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:56:52.091 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 17 22:56:51 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-3[52771]: 2024-05-17T22:56:51.789+0000 7f71e1e33700 -1 osd.3 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:26.301857+0000 front 2024-05-17T22:56:26.301895+0000 (oldest deadline 2024-05-17T22:56:51.001987+0000) 2024-05-17T22:56:52.410 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:56:52 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:56:52.108+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:56:52.591 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:56:52 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:56:52.492+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:26.427462+0000 front 2024-05-17T22:56:26.427603+0000 (oldest deadline 2024-05-17T22:56:49.327570+0000) 2024-05-17T22:56:52.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:56:52 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:56:52.274+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:56:52.741 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:56:52 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5[48151]: 2024-05-17T22:56:52.408+0000 7f96d04e8700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.562993+0000 front 2024-05-17T22:56:24.562934+0000 (oldest deadline 2024-05-17T22:56:49.262999+0000) 2024-05-17T22:56:52.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:56:52 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:56:52.438+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:56:53.048 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:56:52 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:56:52.741+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:56:53.091 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 17 22:56:52 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-3[52771]: 2024-05-17T22:56:52.753+0000 7f71e1e33700 -1 osd.3 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:26.301857+0000 front 2024-05-17T22:56:26.301895+0000 (oldest deadline 2024-05-17T22:56:51.001987+0000) 2024-05-17T22:56:53.436 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:56:53 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:56:53.072+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:56:53.591 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:56:53 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:56:53.451+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:26.427462+0000 front 2024-05-17T22:56:26.427603+0000 (oldest deadline 2024-05-17T22:56:49.327570+0000) 2024-05-17T22:56:53.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:56:53 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:56:53.225+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:56:53.740 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:56:53 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:56:53.434+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:56:53.741 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:56:53 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5[48151]: 2024-05-17T22:56:53.447+0000 7f96d04e8700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.562993+0000 front 2024-05-17T22:56:24.562934+0000 (oldest deadline 2024-05-17T22:56:49.262999+0000) 2024-05-17T22:56:54.048 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:56:53 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:56:53.788+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:56:54.315 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 17 22:56:54 smithi012 podman[115287]: 2024-05-17 22:56:54.047167995 +0000 UTC m=+5.764068183 container died e4851f49c8b5fd9ac76370e565cfc3c9dd40f2f1492f823d2bd983213173adf3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-3, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T22:56:54.316 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:56:54 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:56:54.192+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:56:54.400 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:56:54 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:56:54.110+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:56:54.591 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:56:54 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:56:54.466+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:26.427462+0000 front 2024-05-17T22:56:26.427603+0000 (oldest deadline 2024-05-17T22:56:49.327570+0000) 2024-05-17T22:56:54.592 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 17 22:56:54 smithi012 podman[115287]: 2024-05-17 22:56:54.504488567 +0000 UTC m=+6.221388781 container cleanup e4851f49c8b5fd9ac76370e565cfc3c9dd40f2f1492f823d2bd983213173adf3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T22:56:54.592 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 17 22:56:54 smithi012 bash[115287]: ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-3 2024-05-17T22:56:54.740 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:56:54 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5[48151]: 2024-05-17T22:56:54.492+0000 7f96d04e8700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.562993+0000 front 2024-05-17T22:56:24.562934+0000 (oldest deadline 2024-05-17T22:56:49.262999+0000) 2024-05-17T22:56:54.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:56:54 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:56:54.397+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:56:55.048 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:56:54 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:56:54.760+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:56:55.091 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 17 22:56:54 smithi012 podman[115302]: 2024-05-17 22:56:54.823091552 +0000 UTC m=+0.776683541 container remove e4851f49c8b5fd9ac76370e565cfc3c9dd40f2f1492f823d2bd983213173adf3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-3, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS) 2024-05-17T22:56:55.398 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:56:55 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:56:55.136+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:56:55.462 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:56:55 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:56:55.228+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:56:55.462 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 17 22:56:55 smithi012 podman[115390]: 2024-05-17 22:56:55.099929094 +0000 UTC m=+0.017958067 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T22:56:55.462 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 17 22:56:55 smithi012 podman[115390]: 2024-05-17 22:56:55.215569063 +0000 UTC m=+0.133598026 container create 5647714e577535c3e4780aacf0182726413e830104781ca38b3aafced2704af7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-17T22:56:55.462 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 17 22:56:55 smithi012 podman[115390]: 2024-05-17 22:56:55.46001106 +0000 UTC m=+0.378040020 container init 5647714e577535c3e4780aacf0182726413e830104781ca38b3aafced2704af7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-17T22:56:55.740 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:56:55 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:56:55.395+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:56:55.741 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:56:55 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5[48151]: 2024-05-17T22:56:55.444+0000 7f96d04e8700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.562993+0000 front 2024-05-17T22:56:24.562934+0000 (oldest deadline 2024-05-17T22:56:49.262999+0000) 2024-05-17T22:56:55.778 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:56:55 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:56:55.481+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:26.427462+0000 front 2024-05-17T22:56:26.427603+0000 (oldest deadline 2024-05-17T22:56:49.327570+0000) 2024-05-17T22:56:55.779 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 17 22:56:55 smithi012 podman[115390]: 2024-05-17 22:56:55.463747769 +0000 UTC m=+0.381776730 container start 5647714e577535c3e4780aacf0182726413e830104781ca38b3aafced2704af7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-17T22:56:55.779 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 17 22:56:55 smithi012 podman[115390]: 2024-05-17 22:56:55.551650702 +0000 UTC m=+0.469679670 container attach 5647714e577535c3e4780aacf0182726413e830104781ca38b3aafced2704af7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-17T22:56:56.048 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:56:55 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:56:55.748+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:56:56.091 INFO:journalctl@ceph.osd.3.smithi012.stdout:May 17 22:56:55 smithi012 podman[115390]: 2024-05-17 22:56:55.776753092 +0000 UTC m=+0.694782063 container died 5647714e577535c3e4780aacf0182726413e830104781ca38b3aafced2704af7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0) 2024-05-17T22:56:56.431 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:56:56 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:56:56.107+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:56:56.496 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:56:56 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:56:56.212+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:56:56.535 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@osd.3.service' 2024-05-17T22:56:56.578 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T22:56:56.579 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-17T22:56:56.579 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-17T22:56:56.579 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-4d593780-149c-11ef-bc9a-c7b262605968@osd.4 2024-05-17T22:56:56.740 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:56:56 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:56:56.427+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:56:56.740 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:56:56 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5[48151]: 2024-05-17T22:56:56.488+0000 7f96d04e8700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.562993+0000 front 2024-05-17T22:56:24.562934+0000 (oldest deadline 2024-05-17T22:56:49.262999+0000) 2024-05-17T22:56:56.841 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:56:56 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:56:56.493+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:26.427462+0000 front 2024-05-17T22:56:26.427603+0000 (oldest deadline 2024-05-17T22:56:49.327570+0000) 2024-05-17T22:56:56.892 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 17 22:56:56 smithi096 systemd[1]: Stopping Ceph osd.4 for 4d593780-149c-11ef-bc9a-c7b262605968... 2024-05-17T22:56:56.893 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:56:56 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:56:56.788+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:56:57.298 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 17 22:56:56 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-4[47442]: 2024-05-17T22:56:56.890+0000 7ff3c49d0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T22:56:57.299 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 17 22:56:56 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-4[47442]: 2024-05-17T22:56:56.890+0000 7ff3c49d0700 -1 osd.4 93 *** Got signal Terminated *** 2024-05-17T22:56:57.299 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 17 22:56:56 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-4[47442]: 2024-05-17T22:56:56.890+0000 7ff3c49d0700 -1 osd.4 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T22:56:57.418 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:56:57 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:56:57.094+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:56:57.591 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:56:57 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:56:57.487+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:26.427462+0000 front 2024-05-17T22:56:26.427603+0000 (oldest deadline 2024-05-17T22:56:49.327570+0000) 2024-05-17T22:56:57.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:56:57 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:56:57.251+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:56:57.740 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:56:57 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5[48151]: 2024-05-17T22:56:57.524+0000 7f96d04e8700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.562993+0000 front 2024-05-17T22:56:24.562934+0000 (oldest deadline 2024-05-17T22:56:49.262999+0000) 2024-05-17T22:56:57.740 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:56:57 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:56:57.415+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:56:58.048 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:56:57 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:56:57.768+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:56:58.377 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:56:58 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:56:58.084+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:56:58.511 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:56:58 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:56:58.240+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:56:58.740 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:56:58 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5[48151]: 2024-05-17T22:56:58.513+0000 7f96d04e8700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.562993+0000 front 2024-05-17T22:56:24.562934+0000 (oldest deadline 2024-05-17T22:56:49.262999+0000) 2024-05-17T22:56:58.740 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:56:58 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:56:58.374+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:56:58.841 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:56:58 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:56:58.509+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:26.427462+0000 front 2024-05-17T22:56:26.427603+0000 (oldest deadline 2024-05-17T22:56:49.327570+0000) 2024-05-17T22:56:59.048 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:56:58 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:56:58.792+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:56:59.350 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:56:59 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:56:59.080+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:56:59.350 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:56:59 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:56:59.080+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.526326+0000 front 2024-05-17T22:56:33.526441+0000 (oldest deadline 2024-05-17T22:56:58.226157+0000) 2024-05-17T22:56:59.533 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:56:59 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:56:59.245+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:56:59.533 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:56:59 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:56:59.245+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:56:59.548 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:56:59 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:56:59.074+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.731253+0000 front 2024-05-17T22:56:33.731287+0000 (oldest deadline 2024-05-17T22:56:58.431402+0000) 2024-05-17T22:56:59.740 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:56:59 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5[48151]: 2024-05-17T22:56:59.532+0000 7f96d04e8700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.562993+0000 front 2024-05-17T22:56:24.562934+0000 (oldest deadline 2024-05-17T22:56:49.262999+0000) 2024-05-17T22:56:59.740 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:56:59 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5[48151]: 2024-05-17T22:56:59.532+0000 7f96d04e8700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:34.465624+0000 front 2024-05-17T22:56:34.465500+0000 (oldest deadline 2024-05-17T22:56:59.165495+0000) 2024-05-17T22:56:59.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:56:59 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:56:59.347+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:56:59.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:56:59 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:56:59.347+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:56:59.841 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:56:59 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:56:59.530+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:26.427462+0000 front 2024-05-17T22:56:26.427603+0000 (oldest deadline 2024-05-17T22:56:49.327570+0000) 2024-05-17T22:57:00.060 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:56:59 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:56:59.801+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:00.309 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:00 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:00.047+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:57:00.309 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:00 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:00.047+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.526326+0000 front 2024-05-17T22:56:33.526441+0000 (oldest deadline 2024-05-17T22:56:58.226157+0000) 2024-05-17T22:57:00.525 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:00 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:00.248+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:00.526 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:00 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:00.248+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:00.548 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:00 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:00.058+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.731253+0000 front 2024-05-17T22:56:33.731287+0000 (oldest deadline 2024-05-17T22:56:58.431402+0000) 2024-05-17T22:57:00.560 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:00 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:00.305+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:00.560 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:00 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:00.305+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:00.841 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:00 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:00.523+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:26.427462+0000 front 2024-05-17T22:56:26.427603+0000 (oldest deadline 2024-05-17T22:56:49.327570+0000) 2024-05-17T22:57:00.841 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:00 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:00.523+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:36.329474+0000 front 2024-05-17T22:56:36.329364+0000 (oldest deadline 2024-05-17T22:56:59.829540+0000) 2024-05-17T22:57:00.990 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:57:00 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5[48151]: 2024-05-17T22:57:00.558+0000 7f96d04e8700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.562993+0000 front 2024-05-17T22:56:24.562934+0000 (oldest deadline 2024-05-17T22:56:49.262999+0000) 2024-05-17T22:57:00.990 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:57:00 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5[48151]: 2024-05-17T22:57:00.558+0000 7f96d04e8700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:34.465624+0000 front 2024-05-17T22:56:34.465500+0000 (oldest deadline 2024-05-17T22:56:59.165495+0000) 2024-05-17T22:57:01.048 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 17 22:57:00 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-4[47442]: 2024-05-17T22:57:00.758+0000 7ff3bcf7e700 -1 osd.4 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:31.592433+0000 front 2024-05-17T22:56:31.592405+0000 (oldest deadline 2024-05-17T22:57:00.393240+0000) 2024-05-17T22:57:01.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:00 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:00.805+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:01.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:00 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:00.805+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:01.490 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:01 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:01.078+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:57:01.490 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:01 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:01.078+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.526326+0000 front 2024-05-17T22:56:33.526441+0000 (oldest deadline 2024-05-17T22:56:58.226157+0000) 2024-05-17T22:57:01.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:01 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:01.308+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:01.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:01 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:01.308+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:01.548 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:01 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:01.083+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.731253+0000 front 2024-05-17T22:56:33.731287+0000 (oldest deadline 2024-05-17T22:56:58.431402+0000) 2024-05-17T22:57:01.571 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:01 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:01.297+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:01.571 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:01 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:01.297+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:01.841 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:01 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:01.569+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:26.427462+0000 front 2024-05-17T22:56:26.427603+0000 (oldest deadline 2024-05-17T22:56:49.327570+0000) 2024-05-17T22:57:01.841 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:01 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:01.569+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:36.329474+0000 front 2024-05-17T22:56:36.329364+0000 (oldest deadline 2024-05-17T22:56:59.829540+0000) 2024-05-17T22:57:01.990 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:57:01 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5[48151]: 2024-05-17T22:57:01.540+0000 7f96d04e8700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.562993+0000 front 2024-05-17T22:56:24.562934+0000 (oldest deadline 2024-05-17T22:56:49.262999+0000) 2024-05-17T22:57:01.991 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:57:01 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5[48151]: 2024-05-17T22:57:01.540+0000 7f96d04e8700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:34.465624+0000 front 2024-05-17T22:56:34.465500+0000 (oldest deadline 2024-05-17T22:56:59.165495+0000) 2024-05-17T22:57:02.048 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 17 22:57:01 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-4[47442]: 2024-05-17T22:57:01.767+0000 7ff3bcf7e700 -1 osd.4 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:31.592433+0000 front 2024-05-17T22:56:31.592405+0000 (oldest deadline 2024-05-17T22:57:00.393240+0000) 2024-05-17T22:57:02.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:01 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:01.828+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:02.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:01 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:01.828+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:02.490 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:02 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:02.087+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:57:02.490 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:02 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:02.087+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.526326+0000 front 2024-05-17T22:56:33.526441+0000 (oldest deadline 2024-05-17T22:56:58.226157+0000) 2024-05-17T22:57:02.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:02 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:02.268+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:02.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:02 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:02.268+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:02.548 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:02 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:02.111+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.731253+0000 front 2024-05-17T22:56:33.731287+0000 (oldest deadline 2024-05-17T22:56:58.431402+0000) 2024-05-17T22:57:02.743 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:02 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:02.519+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:26.427462+0000 front 2024-05-17T22:56:26.427603+0000 (oldest deadline 2024-05-17T22:56:49.327570+0000) 2024-05-17T22:57:02.743 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:02 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:02.519+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:36.329474+0000 front 2024-05-17T22:56:36.329364+0000 (oldest deadline 2024-05-17T22:56:59.829540+0000) 2024-05-17T22:57:02.743 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:02 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:02.346+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:02.743 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:02 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:02.346+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:02.885 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:02 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:02.849+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:02.885 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:02 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:02.849+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:02.886 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 17 22:57:02 smithi096 podman[72489]: 2024-05-17 22:57:02.606258106 +0000 UTC m=+5.833749112 container died 1062eb66a2c35bd51fd2cf608ca3856cf750f4765d7eb131d03d2ab9338c5283 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-4, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T22:57:02.990 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:57:02 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5[48151]: 2024-05-17T22:57:02.553+0000 7f96d04e8700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.562993+0000 front 2024-05-17T22:56:24.562934+0000 (oldest deadline 2024-05-17T22:56:49.262999+0000) 2024-05-17T22:57:02.990 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:57:02 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5[48151]: 2024-05-17T22:57:02.553+0000 7f96d04e8700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:34.465624+0000 front 2024-05-17T22:56:34.465500+0000 (oldest deadline 2024-05-17T22:56:59.165495+0000) 2024-05-17T22:57:03.298 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:03 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:03.086+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.731253+0000 front 2024-05-17T22:56:33.731287+0000 (oldest deadline 2024-05-17T22:56:58.431402+0000) 2024-05-17T22:57:03.299 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 17 22:57:03 smithi096 podman[72489]: 2024-05-17 22:57:03.056852371 +0000 UTC m=+6.284343374 container cleanup 1062eb66a2c35bd51fd2cf608ca3856cf750f4765d7eb131d03d2ab9338c5283 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-4, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-17T22:57:03.299 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 17 22:57:03 smithi096 bash[72489]: ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-4 2024-05-17T22:57:03.490 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:03 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:03.113+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:57:03.490 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:03 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:03.114+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.526326+0000 front 2024-05-17T22:56:33.526441+0000 (oldest deadline 2024-05-17T22:56:58.226157+0000) 2024-05-17T22:57:03.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:03 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:03.293+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:03.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:03 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:03.293+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:03.591 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:03 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:03.489+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:26.427462+0000 front 2024-05-17T22:56:26.427603+0000 (oldest deadline 2024-05-17T22:56:49.327570+0000) 2024-05-17T22:57:03.591 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:03 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:03.489+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:36.329474+0000 front 2024-05-17T22:56:36.329364+0000 (oldest deadline 2024-05-17T22:56:59.829540+0000) 2024-05-17T22:57:03.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:03 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:03.303+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:03.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:03 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:03.303+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:03.798 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 17 22:57:03 smithi096 podman[72506]: 2024-05-17 22:57:03.376034373 +0000 UTC m=+0.761519642 container remove 1062eb66a2c35bd51fd2cf608ca3856cf750f4765d7eb131d03d2ab9338c5283 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5) 2024-05-17T22:57:03.990 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:57:03 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5[48151]: 2024-05-17T22:57:03.523+0000 7f96d04e8700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.562993+0000 front 2024-05-17T22:56:24.562934+0000 (oldest deadline 2024-05-17T22:56:49.262999+0000) 2024-05-17T22:57:03.991 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:57:03 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5[48151]: 2024-05-17T22:57:03.523+0000 7f96d04e8700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:34.465624+0000 front 2024-05-17T22:56:34.465500+0000 (oldest deadline 2024-05-17T22:56:59.165495+0000) 2024-05-17T22:57:04.098 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 17 22:57:03 smithi096 podman[72591]: 2024-05-17 22:57:03.730112238 +0000 UTC m=+0.017359559 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T22:57:04.099 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 17 22:57:03 smithi096 podman[72591]: 2024-05-17 22:57:03.843652084 +0000 UTC m=+0.130899404 container create 94f5c3042b00e978655908a995d2d8e835cce62c4696a4f34752c4752e553805 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-4-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0) 2024-05-17T22:57:04.099 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 17 22:57:04 smithi096 podman[72591]: 2024-05-17 22:57:04.096733653 +0000 UTC m=+0.383980946 container init 94f5c3042b00e978655908a995d2d8e835cce62c4696a4f34752c4752e553805 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-4-deactivate, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-17T22:57:04.099 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:03 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:03.799+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:04.099 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:03 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:03.799+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:04.396 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:04 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:04.131+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.731253+0000 front 2024-05-17T22:56:33.731287+0000 (oldest deadline 2024-05-17T22:56:58.431402+0000) 2024-05-17T22:57:04.397 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 17 22:57:04 smithi096 podman[72591]: 2024-05-17 22:57:04.100162889 +0000 UTC m=+0.387410188 container start 94f5c3042b00e978655908a995d2d8e835cce62c4696a4f34752c4752e553805 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-4-deactivate, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T22:57:04.397 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 17 22:57:04 smithi096 podman[72591]: 2024-05-17 22:57:04.188009877 +0000 UTC m=+0.475257191 container attach 94f5c3042b00e978655908a995d2d8e835cce62c4696a4f34752c4752e553805 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-4-deactivate, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-17T22:57:04.490 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:04 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:04.136+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:57:04.490 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:04 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:04.136+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.526326+0000 front 2024-05-17T22:56:33.526441+0000 (oldest deadline 2024-05-17T22:56:58.226157+0000) 2024-05-17T22:57:04.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:04 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:04.342+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:04.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:04 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:04.342+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:04.591 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:04 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:04.479+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:26.427462+0000 front 2024-05-17T22:56:26.427603+0000 (oldest deadline 2024-05-17T22:56:49.327570+0000) 2024-05-17T22:57:04.592 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:04 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:04.479+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:36.329474+0000 front 2024-05-17T22:56:36.329364+0000 (oldest deadline 2024-05-17T22:56:59.829540+0000) 2024-05-17T22:57:04.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:04 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:04.303+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:04.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:04 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:04.303+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:04.760 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 17 22:57:04 smithi096 podman[72591]: 2024-05-17 22:57:04.395088519 +0000 UTC m=+0.682335818 container died 94f5c3042b00e978655908a995d2d8e835cce62c4696a4f34752c4752e553805 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-4-deactivate, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_BRANCH=HEAD) 2024-05-17T22:57:04.994 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:57:04 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5[48151]: 2024-05-17T22:57:04.527+0000 7f96d04e8700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.562993+0000 front 2024-05-17T22:56:24.562934+0000 (oldest deadline 2024-05-17T22:56:49.262999+0000) 2024-05-17T22:57:04.995 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:57:04 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5[48151]: 2024-05-17T22:57:04.527+0000 7f96d04e8700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:34.465624+0000 front 2024-05-17T22:56:34.465500+0000 (oldest deadline 2024-05-17T22:56:59.165495+0000) 2024-05-17T22:57:05.048 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:04 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:04.778+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:05.048 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:04 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:04.778+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:05.078 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@osd.4.service' 2024-05-17T22:57:05.120 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T22:57:05.120 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-17T22:57:05.120 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-17T22:57:05.121 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-4d593780-149c-11ef-bc9a-c7b262605968@osd.5 2024-05-17T22:57:05.455 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:57:05 smithi151 systemd[1]: Stopping Ceph osd.5 for 4d593780-149c-11ef-bc9a-c7b262605968... 2024-05-17T22:57:05.455 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:57:05 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5[48151]: 2024-05-17T22:57:05.451+0000 7f96d7f3a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T22:57:05.456 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:05 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:05.137+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:57:05.456 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:05 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:05.137+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.526326+0000 front 2024-05-17T22:56:33.526441+0000 (oldest deadline 2024-05-17T22:56:58.226157+0000) 2024-05-17T22:57:05.456 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:05 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:05.373+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:05.456 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:05 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:05.373+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:05.548 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:05 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:05.145+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.731253+0000 front 2024-05-17T22:56:33.731287+0000 (oldest deadline 2024-05-17T22:56:58.431402+0000) 2024-05-17T22:57:05.591 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:05 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:05.485+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:26.427462+0000 front 2024-05-17T22:56:26.427603+0000 (oldest deadline 2024-05-17T22:56:49.327570+0000) 2024-05-17T22:57:05.591 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:05 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:05.485+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:36.329474+0000 front 2024-05-17T22:56:36.329364+0000 (oldest deadline 2024-05-17T22:56:59.829540+0000) 2024-05-17T22:57:05.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:05 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:05.327+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:05.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:05 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:05.327+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:05.741 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:57:05 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5[48151]: 2024-05-17T22:57:05.451+0000 7f96d7f3a700 -1 osd.5 93 *** Got signal Terminated *** 2024-05-17T22:57:05.741 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:57:05 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5[48151]: 2024-05-17T22:57:05.451+0000 7f96d7f3a700 -1 osd.5 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T22:57:05.741 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:57:05 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5[48151]: 2024-05-17T22:57:05.542+0000 7f96d04e8700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.562993+0000 front 2024-05-17T22:56:24.562934+0000 (oldest deadline 2024-05-17T22:56:49.262999+0000) 2024-05-17T22:57:05.741 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:57:05 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5[48151]: 2024-05-17T22:57:05.542+0000 7f96d04e8700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:34.465624+0000 front 2024-05-17T22:56:34.465500+0000 (oldest deadline 2024-05-17T22:56:59.165495+0000) 2024-05-17T22:57:06.170 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:05 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:05.803+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:06.170 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:05 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:05.803+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:06.170 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:05 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:05.803+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:06.420 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:06 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:06.091+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:57:06.420 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:06 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:06.091+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.526326+0000 front 2024-05-17T22:56:33.526441+0000 (oldest deadline 2024-05-17T22:56:58.226157+0000) 2024-05-17T22:57:06.548 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:06 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:06.168+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.731253+0000 front 2024-05-17T22:56:33.731287+0000 (oldest deadline 2024-05-17T22:56:58.431402+0000) 2024-05-17T22:57:06.591 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:06 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:06.466+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:26.427462+0000 front 2024-05-17T22:56:26.427603+0000 (oldest deadline 2024-05-17T22:56:49.327570+0000) 2024-05-17T22:57:06.591 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:06 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:06.466+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:36.329474+0000 front 2024-05-17T22:56:36.329364+0000 (oldest deadline 2024-05-17T22:56:59.829540+0000) 2024-05-17T22:57:06.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:06 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:06.335+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:06.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:06 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:06.335+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:06.740 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:06 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:06.418+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:06.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:06 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:06.418+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:06.741 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:57:06 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5[48151]: 2024-05-17T22:57:06.587+0000 7f96d04e8700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.562993+0000 front 2024-05-17T22:56:24.562934+0000 (oldest deadline 2024-05-17T22:56:49.262999+0000) 2024-05-17T22:57:06.741 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:57:06 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5[48151]: 2024-05-17T22:57:06.587+0000 7f96d04e8700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:34.465624+0000 front 2024-05-17T22:56:34.465500+0000 (oldest deadline 2024-05-17T22:56:59.165495+0000) 2024-05-17T22:57:06.741 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:57:06 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5[48151]: 2024-05-17T22:57:06.587+0000 7f96d04e8700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.466564+0000 front 2024-05-17T22:56:41.466585+0000 (oldest deadline 2024-05-17T22:57:06.166743+0000) 2024-05-17T22:57:07.048 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:06 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:06.760+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:07.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:06 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:06.760+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:07.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:06 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:06.760+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:07.412 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:07 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:07.092+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:57:07.412 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:07 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:07.092+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.526326+0000 front 2024-05-17T22:56:33.526441+0000 (oldest deadline 2024-05-17T22:56:58.226157+0000) 2024-05-17T22:57:07.413 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:07 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:07.092+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.527134+0000 front 2024-05-17T22:56:43.527165+0000 (oldest deadline 2024-05-17T22:57:07.027280+0000) 2024-05-17T22:57:07.548 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:07 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:07.126+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.731253+0000 front 2024-05-17T22:56:33.731287+0000 (oldest deadline 2024-05-17T22:56:58.431402+0000) 2024-05-17T22:57:07.741 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:57:07 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5[48151]: 2024-05-17T22:57:07.596+0000 7f96d04e8700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.562993+0000 front 2024-05-17T22:56:24.562934+0000 (oldest deadline 2024-05-17T22:56:49.262999+0000) 2024-05-17T22:57:07.741 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:57:07 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5[48151]: 2024-05-17T22:57:07.596+0000 7f96d04e8700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:34.465624+0000 front 2024-05-17T22:56:34.465500+0000 (oldest deadline 2024-05-17T22:56:59.165495+0000) 2024-05-17T22:57:07.741 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:57:07 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5[48151]: 2024-05-17T22:57:07.596+0000 7f96d04e8700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.466564+0000 front 2024-05-17T22:56:41.466585+0000 (oldest deadline 2024-05-17T22:57:06.166743+0000) 2024-05-17T22:57:07.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:07 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:07.410+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:07.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:07 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:07.410+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:07.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:07 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:07.410+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:07.841 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:07 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:07.479+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:26.427462+0000 front 2024-05-17T22:56:26.427603+0000 (oldest deadline 2024-05-17T22:56:49.327570+0000) 2024-05-17T22:57:07.841 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:07 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:07.479+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:36.329474+0000 front 2024-05-17T22:56:36.329364+0000 (oldest deadline 2024-05-17T22:56:59.829540+0000) 2024-05-17T22:57:07.842 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:07 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:07.374+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:07.842 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:07 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:07.374+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:07.842 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:07 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:07.374+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:44.416729+0000 front 2024-05-17T22:56:44.416947+0000 (oldest deadline 2024-05-17T22:57:07.316815+0000) 2024-05-17T22:57:08.048 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:07 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:07.762+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:08.048 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:07 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:07.762+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:08.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:07 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:07.762+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:08.320 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:08 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:08.133+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.731253+0000 front 2024-05-17T22:56:33.731287+0000 (oldest deadline 2024-05-17T22:56:58.431402+0000) 2024-05-17T22:57:08.432 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:08 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:08.082+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:57:08.433 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:08 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:08.082+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.526326+0000 front 2024-05-17T22:56:33.526441+0000 (oldest deadline 2024-05-17T22:56:58.226157+0000) 2024-05-17T22:57:08.433 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:08 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:08.082+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.527134+0000 front 2024-05-17T22:56:43.527165+0000 (oldest deadline 2024-05-17T22:57:07.027280+0000) 2024-05-17T22:57:08.740 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:08 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:08.430+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:08.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:08 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:08.430+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:08.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:08 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:08.430+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:08.741 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:57:08 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5[48151]: 2024-05-17T22:57:08.617+0000 7f96d04e8700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.562993+0000 front 2024-05-17T22:56:24.562934+0000 (oldest deadline 2024-05-17T22:56:49.262999+0000) 2024-05-17T22:57:08.741 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:57:08 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5[48151]: 2024-05-17T22:57:08.617+0000 7f96d04e8700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:34.465624+0000 front 2024-05-17T22:56:34.465500+0000 (oldest deadline 2024-05-17T22:56:59.165495+0000) 2024-05-17T22:57:08.741 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:57:08 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5[48151]: 2024-05-17T22:57:08.617+0000 7f96d04e8700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.466564+0000 front 2024-05-17T22:56:41.466585+0000 (oldest deadline 2024-05-17T22:57:06.166743+0000) 2024-05-17T22:57:08.841 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:08 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:08.436+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:26.427462+0000 front 2024-05-17T22:56:26.427603+0000 (oldest deadline 2024-05-17T22:56:49.327570+0000) 2024-05-17T22:57:08.841 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:08 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:08.436+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:36.329474+0000 front 2024-05-17T22:56:36.329364+0000 (oldest deadline 2024-05-17T22:56:59.829540+0000) 2024-05-17T22:57:08.842 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:08 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:08.359+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:08.842 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:08 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:08.359+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:08.842 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:08 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:08.359+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:44.416729+0000 front 2024-05-17T22:56:44.416947+0000 (oldest deadline 2024-05-17T22:57:07.316815+0000) 2024-05-17T22:57:09.048 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:08 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:08.745+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:09.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:08 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:08.745+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:09.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:08 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:08.745+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:09.473 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:09 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:09.066+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:57:09.473 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:09 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:09.066+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.526326+0000 front 2024-05-17T22:56:33.526441+0000 (oldest deadline 2024-05-17T22:56:58.226157+0000) 2024-05-17T22:57:09.473 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:09 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:09.066+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.527134+0000 front 2024-05-17T22:56:43.527165+0000 (oldest deadline 2024-05-17T22:57:07.027280+0000) 2024-05-17T22:57:09.548 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:09 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:09.089+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.731253+0000 front 2024-05-17T22:56:33.731287+0000 (oldest deadline 2024-05-17T22:56:58.431402+0000) 2024-05-17T22:57:09.548 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:09 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:09.089+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.132595+0000 front 2024-05-17T22:56:43.132667+0000 (oldest deadline 2024-05-17T22:57:08.432629+0000) 2024-05-17T22:57:09.740 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:57:09 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5[48151]: 2024-05-17T22:57:09.662+0000 7f96d04e8700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.562993+0000 front 2024-05-17T22:56:24.562934+0000 (oldest deadline 2024-05-17T22:56:49.262999+0000) 2024-05-17T22:57:09.741 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:57:09 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5[48151]: 2024-05-17T22:57:09.662+0000 7f96d04e8700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:34.465624+0000 front 2024-05-17T22:56:34.465500+0000 (oldest deadline 2024-05-17T22:56:59.165495+0000) 2024-05-17T22:57:09.741 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:57:09 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5[48151]: 2024-05-17T22:57:09.662+0000 7f96d04e8700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.466564+0000 front 2024-05-17T22:56:41.466585+0000 (oldest deadline 2024-05-17T22:57:06.166743+0000) 2024-05-17T22:57:09.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:09 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:09.471+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:09.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:09 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:09.471+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:09.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:09 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:09.471+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:09.841 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:09 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:09.475+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:26.427462+0000 front 2024-05-17T22:56:26.427603+0000 (oldest deadline 2024-05-17T22:56:49.327570+0000) 2024-05-17T22:57:09.842 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:09 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:09.475+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:36.329474+0000 front 2024-05-17T22:56:36.329364+0000 (oldest deadline 2024-05-17T22:56:59.829540+0000) 2024-05-17T22:57:09.842 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:09 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:09.475+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.930825+0000 front 2024-05-17T22:56:43.930927+0000 (oldest deadline 2024-05-17T22:57:09.230760+0000) 2024-05-17T22:57:09.842 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:09 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:09.340+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:09.842 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:09 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:09.340+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:09.842 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:09 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:09.340+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:44.416729+0000 front 2024-05-17T22:56:44.416947+0000 (oldest deadline 2024-05-17T22:57:07.316815+0000) 2024-05-17T22:57:10.048 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:09 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:09.719+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:10.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:09 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:09.719+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:10.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:09 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:09.719+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:10.451 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:10 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:10.097+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:57:10.451 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:10 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:10.097+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.526326+0000 front 2024-05-17T22:56:33.526441+0000 (oldest deadline 2024-05-17T22:56:58.226157+0000) 2024-05-17T22:57:10.452 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:10 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:10.097+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.527134+0000 front 2024-05-17T22:56:43.527165+0000 (oldest deadline 2024-05-17T22:57:07.027280+0000) 2024-05-17T22:57:10.549 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:10 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:10.105+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.731253+0000 front 2024-05-17T22:56:33.731287+0000 (oldest deadline 2024-05-17T22:56:58.431402+0000) 2024-05-17T22:57:10.549 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:10 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:10.105+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.132595+0000 front 2024-05-17T22:56:43.132667+0000 (oldest deadline 2024-05-17T22:57:08.432629+0000) 2024-05-17T22:57:10.740 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:10 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:10.448+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:10.740 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:10 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:10.448+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:10.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:10 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:10.448+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:10.841 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:10 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:10.429+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:26.427462+0000 front 2024-05-17T22:56:26.427603+0000 (oldest deadline 2024-05-17T22:56:49.327570+0000) 2024-05-17T22:57:10.841 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:10 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:10.429+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:36.329474+0000 front 2024-05-17T22:56:36.329364+0000 (oldest deadline 2024-05-17T22:56:59.829540+0000) 2024-05-17T22:57:10.842 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:10 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:10.429+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.930825+0000 front 2024-05-17T22:56:43.930927+0000 (oldest deadline 2024-05-17T22:57:09.230760+0000) 2024-05-17T22:57:10.842 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:10 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:10.383+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:10.842 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:10 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:10.383+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:10.842 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:10 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:10.383+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:44.416729+0000 front 2024-05-17T22:56:44.416947+0000 (oldest deadline 2024-05-17T22:57:07.316815+0000) 2024-05-17T22:57:11.048 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:10 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:10.747+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:11.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:10 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:10.747+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:11.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:10 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:10.747+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:11.146 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:57:10 smithi151 podman[73638]: 2024-05-17 22:57:10.889312347 +0000 UTC m=+5.573591682 container died 9940a8bbe7b8d5dada5097e21e0d601d2b8e6ba7ae70fc7fcb38bad8879a40d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-17T22:57:11.404 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:57:11 smithi151 podman[73638]: 2024-05-17 22:57:11.237255923 +0000 UTC m=+5.921535255 container cleanup 9940a8bbe7b8d5dada5097e21e0d601d2b8e6ba7ae70fc7fcb38bad8879a40d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-17T22:57:11.404 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:57:11 smithi151 bash[73638]: ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5 2024-05-17T22:57:11.404 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:11 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:11.143+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:57:11.404 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:11 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:11.143+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.526326+0000 front 2024-05-17T22:56:33.526441+0000 (oldest deadline 2024-05-17T22:56:58.226157+0000) 2024-05-17T22:57:11.404 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:11 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:11.143+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.527134+0000 front 2024-05-17T22:56:43.527165+0000 (oldest deadline 2024-05-17T22:57:07.027280+0000) 2024-05-17T22:57:11.548 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:11 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:11.143+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.731253+0000 front 2024-05-17T22:56:33.731287+0000 (oldest deadline 2024-05-17T22:56:58.431402+0000) 2024-05-17T22:57:11.548 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:11 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:11.143+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.132595+0000 front 2024-05-17T22:56:43.132667+0000 (oldest deadline 2024-05-17T22:57:08.432629+0000) 2024-05-17T22:57:11.740 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:57:11 smithi151 podman[73655]: 2024-05-17 22:57:11.456006908 +0000 UTC m=+0.569880305 container remove 9940a8bbe7b8d5dada5097e21e0d601d2b8e6ba7ae70fc7fcb38bad8879a40d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_CLEAN=True) 2024-05-17T22:57:11.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:11 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:11.401+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:11.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:11 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:11.401+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:11.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:11 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:11.401+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:11.842 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:11 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:11.440+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:26.427462+0000 front 2024-05-17T22:56:26.427603+0000 (oldest deadline 2024-05-17T22:56:49.327570+0000) 2024-05-17T22:57:11.842 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:11 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:11.440+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:36.329474+0000 front 2024-05-17T22:56:36.329364+0000 (oldest deadline 2024-05-17T22:56:59.829540+0000) 2024-05-17T22:57:11.842 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:11 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:11.440+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.930825+0000 front 2024-05-17T22:56:43.930927+0000 (oldest deadline 2024-05-17T22:57:09.230760+0000) 2024-05-17T22:57:11.842 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:11 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:11.402+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:11.842 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:11 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:11.402+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:11.842 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:11 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:11.402+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:44.416729+0000 front 2024-05-17T22:56:44.416947+0000 (oldest deadline 2024-05-17T22:57:07.316815+0000) 2024-05-17T22:57:12.048 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:11 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:11.710+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:12.048 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:11 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:11.710+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:12.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:11 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:11.710+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:12.138 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:57:11 smithi151 podman[73747]: 2024-05-17 22:57:11.765840685 +0000 UTC m=+0.017252944 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T22:57:12.138 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:57:11 smithi151 podman[73747]: 2024-05-17 22:57:11.899259726 +0000 UTC m=+0.150671973 container create 3f960ede5797e35b66f028b88c4a68ad2784b5060549d34c03d3e4c253aa4660 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514) 2024-05-17T22:57:12.431 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:12 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:12.385+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:12.431 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:12 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:12.385+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:12.431 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:12 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:12.385+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:12.431 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:57:12 smithi151 podman[73747]: 2024-05-17 22:57:12.136472034 +0000 UTC m=+0.387884295 container init 3f960ede5797e35b66f028b88c4a68ad2784b5060549d34c03d3e4c253aa4660 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-17T22:57:12.431 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:57:12 smithi151 podman[73747]: 2024-05-17 22:57:12.139808134 +0000 UTC m=+0.391220388 container start 3f960ede5797e35b66f028b88c4a68ad2784b5060549d34c03d3e4c253aa4660 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T22:57:12.432 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:57:12 smithi151 podman[73747]: 2024-05-17 22:57:12.236080858 +0000 UTC m=+0.487493117 container attach 3f960ede5797e35b66f028b88c4a68ad2784b5060549d34c03d3e4c253aa4660 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T22:57:12.432 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:12 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:12.177+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:57:12.432 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:12 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:12.177+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.526326+0000 front 2024-05-17T22:56:33.526441+0000 (oldest deadline 2024-05-17T22:56:58.226157+0000) 2024-05-17T22:57:12.432 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:12 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:12.177+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.527134+0000 front 2024-05-17T22:56:43.527165+0000 (oldest deadline 2024-05-17T22:57:07.027280+0000) 2024-05-17T22:57:12.548 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:12 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:12.151+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.731253+0000 front 2024-05-17T22:56:33.731287+0000 (oldest deadline 2024-05-17T22:56:58.431402+0000) 2024-05-17T22:57:12.548 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:12 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:12.151+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.132595+0000 front 2024-05-17T22:56:43.132667+0000 (oldest deadline 2024-05-17T22:57:08.432629+0000) 2024-05-17T22:57:12.740 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 17 22:57:12 smithi151 podman[73747]: 2024-05-17 22:57:12.429810159 +0000 UTC m=+0.681222413 container died 3f960ede5797e35b66f028b88c4a68ad2784b5060549d34c03d3e4c253aa4660 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-5-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-17T22:57:12.841 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:12 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:12.455+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:26.427462+0000 front 2024-05-17T22:56:26.427603+0000 (oldest deadline 2024-05-17T22:56:49.327570+0000) 2024-05-17T22:57:12.842 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:12 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:12.455+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:36.329474+0000 front 2024-05-17T22:56:36.329364+0000 (oldest deadline 2024-05-17T22:56:59.829540+0000) 2024-05-17T22:57:12.842 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:12 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:12.455+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.930825+0000 front 2024-05-17T22:56:43.930927+0000 (oldest deadline 2024-05-17T22:57:09.230760+0000) 2024-05-17T22:57:12.842 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:12 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:12.362+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:12.842 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:12 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:12.362+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:12.842 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:12 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:12.362+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:44.416729+0000 front 2024-05-17T22:56:44.416947+0000 (oldest deadline 2024-05-17T22:57:07.316815+0000) 2024-05-17T22:57:13.048 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:12 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:12.739+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:13.048 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:12 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:12.739+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:13.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:12 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:12.739+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:13.138 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@osd.5.service' 2024-05-17T22:57:13.179 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T22:57:13.179 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-17T22:57:13.179 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-17T22:57:13.179 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-4d593780-149c-11ef-bc9a-c7b262605968@osd.6 2024-05-17T22:57:13.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:13 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:13.337+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:13.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:13 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:13.337+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:13.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:13 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:13.337+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:13.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:13 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:13.219+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:57:13.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:13 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:13.219+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.526326+0000 front 2024-05-17T22:56:33.526441+0000 (oldest deadline 2024-05-17T22:56:58.226157+0000) 2024-05-17T22:57:13.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:13 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:13.219+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.527134+0000 front 2024-05-17T22:56:43.527165+0000 (oldest deadline 2024-05-17T22:57:07.027280+0000) 2024-05-17T22:57:13.496 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:13 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:13.346+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:13.496 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:13 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:13.346+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:13.496 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:13 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:13.346+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:44.416729+0000 front 2024-05-17T22:56:44.416947+0000 (oldest deadline 2024-05-17T22:57:07.316815+0000) 2024-05-17T22:57:13.497 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:13 smithi012 systemd[1]: Stopping Ceph osd.6 for 4d593780-149c-11ef-bc9a-c7b262605968... 2024-05-17T22:57:13.497 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:13 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:13.431+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:26.427462+0000 front 2024-05-17T22:56:26.427603+0000 (oldest deadline 2024-05-17T22:56:49.327570+0000) 2024-05-17T22:57:13.497 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:13 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:13.431+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:36.329474+0000 front 2024-05-17T22:56:36.329364+0000 (oldest deadline 2024-05-17T22:56:59.829540+0000) 2024-05-17T22:57:13.497 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:13 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:13.431+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.930825+0000 front 2024-05-17T22:56:43.930927+0000 (oldest deadline 2024-05-17T22:57:09.230760+0000) 2024-05-17T22:57:13.548 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:13 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:13.114+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.731253+0000 front 2024-05-17T22:56:33.731287+0000 (oldest deadline 2024-05-17T22:56:58.431402+0000) 2024-05-17T22:57:13.548 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:13 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:13.114+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.132595+0000 front 2024-05-17T22:56:43.132667+0000 (oldest deadline 2024-05-17T22:57:08.432629+0000) 2024-05-17T22:57:13.841 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:13 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:13.665+0000 7f460da60700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T22:57:13.841 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:13 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:13.665+0000 7f460da60700 -1 osd.6 93 *** Got signal Terminated *** 2024-05-17T22:57:13.842 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:13 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:13.665+0000 7f460da60700 -1 osd.6 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T22:57:14.048 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:13 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:13.717+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:14.048 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:13 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:13.717+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:14.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:13 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:13.717+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:14.490 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:14 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:14.199+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:57:14.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:14 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:14.199+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.526326+0000 front 2024-05-17T22:56:33.526441+0000 (oldest deadline 2024-05-17T22:56:58.226157+0000) 2024-05-17T22:57:14.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:14 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:14.199+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.527134+0000 front 2024-05-17T22:56:43.527165+0000 (oldest deadline 2024-05-17T22:57:07.027280+0000) 2024-05-17T22:57:14.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:14 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:14.354+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:14.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:14 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:14.354+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:14.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:14 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:14.354+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:14.548 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:14 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:14.147+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.731253+0000 front 2024-05-17T22:56:33.731287+0000 (oldest deadline 2024-05-17T22:56:58.431402+0000) 2024-05-17T22:57:14.548 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:14 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:14.147+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.132595+0000 front 2024-05-17T22:56:43.132667+0000 (oldest deadline 2024-05-17T22:57:08.432629+0000) 2024-05-17T22:57:14.841 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:14 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:14.350+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:14.841 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:14 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:14.350+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:14.841 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:14 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:14.350+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:44.416729+0000 front 2024-05-17T22:56:44.416947+0000 (oldest deadline 2024-05-17T22:57:07.316815+0000) 2024-05-17T22:57:14.842 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:14 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:14.393+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:26.427462+0000 front 2024-05-17T22:56:26.427603+0000 (oldest deadline 2024-05-17T22:56:49.327570+0000) 2024-05-17T22:57:14.842 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:14 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:14.393+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:36.329474+0000 front 2024-05-17T22:56:36.329364+0000 (oldest deadline 2024-05-17T22:56:59.829540+0000) 2024-05-17T22:57:14.842 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:14 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:14.393+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.930825+0000 front 2024-05-17T22:56:43.930927+0000 (oldest deadline 2024-05-17T22:57:09.230760+0000) 2024-05-17T22:57:15.048 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:14 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:14.708+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:15.048 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:14 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:14.708+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:15.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:14 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:14.708+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:15.490 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:15 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:15.393+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:15.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:15 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:15.393+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:15.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:15 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:15.393+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:15.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:15 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:15.172+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:57:15.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:15 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:15.172+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.526326+0000 front 2024-05-17T22:56:33.526441+0000 (oldest deadline 2024-05-17T22:56:58.226157+0000) 2024-05-17T22:57:15.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:15 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:15.172+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.527134+0000 front 2024-05-17T22:56:43.527165+0000 (oldest deadline 2024-05-17T22:57:07.027280+0000) 2024-05-17T22:57:15.548 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:15 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:15.106+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.731253+0000 front 2024-05-17T22:56:33.731287+0000 (oldest deadline 2024-05-17T22:56:58.431402+0000) 2024-05-17T22:57:15.548 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:15 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:15.106+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.132595+0000 front 2024-05-17T22:56:43.132667+0000 (oldest deadline 2024-05-17T22:57:08.432629+0000) 2024-05-17T22:57:15.548 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:15 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:15.106+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.335236+0000 front 2024-05-17T22:56:52.335206+0000 (oldest deadline 2024-05-17T22:57:14.635269+0000) 2024-05-17T22:57:15.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:15 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:15.324+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:15.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:15 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:15.324+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:15.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:15 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:15.324+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:44.416729+0000 front 2024-05-17T22:56:44.416947+0000 (oldest deadline 2024-05-17T22:57:07.316815+0000) 2024-05-17T22:57:15.592 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:15 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:15.359+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:26.427462+0000 front 2024-05-17T22:56:26.427603+0000 (oldest deadline 2024-05-17T22:56:49.327570+0000) 2024-05-17T22:57:15.592 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:15 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:15.359+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:36.329474+0000 front 2024-05-17T22:56:36.329364+0000 (oldest deadline 2024-05-17T22:56:59.829540+0000) 2024-05-17T22:57:15.592 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:15 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:15.359+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.930825+0000 front 2024-05-17T22:56:43.930927+0000 (oldest deadline 2024-05-17T22:57:09.230760+0000) 2024-05-17T22:57:16.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:15 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:15.730+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:16.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:15 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:15.730+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:16.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:15 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:15.730+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:16.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:15 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:15.730+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.174543+0000 front 2024-05-17T22:56:52.174682+0000 (oldest deadline 2024-05-17T22:57:15.074590+0000) 2024-05-17T22:57:16.423 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:16 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:16.125+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:57:16.423 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:16 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:16.125+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.526326+0000 front 2024-05-17T22:56:33.526441+0000 (oldest deadline 2024-05-17T22:56:58.226157+0000) 2024-05-17T22:57:16.423 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:16 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:16.125+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.527134+0000 front 2024-05-17T22:56:43.527165+0000 (oldest deadline 2024-05-17T22:57:07.027280+0000) 2024-05-17T22:57:16.423 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:16 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:16.125+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:49.928478+0000 front 2024-05-17T22:56:49.928448+0000 (oldest deadline 2024-05-17T22:57:15.828486+0000) 2024-05-17T22:57:16.548 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:16 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:16.148+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.731253+0000 front 2024-05-17T22:56:33.731287+0000 (oldest deadline 2024-05-17T22:56:58.431402+0000) 2024-05-17T22:57:16.548 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:16 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:16.148+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.132595+0000 front 2024-05-17T22:56:43.132667+0000 (oldest deadline 2024-05-17T22:57:08.432629+0000) 2024-05-17T22:57:16.548 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:16 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:16.148+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.335236+0000 front 2024-05-17T22:56:52.335206+0000 (oldest deadline 2024-05-17T22:57:14.635269+0000) 2024-05-17T22:57:16.591 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:16 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:16.322+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:26.427462+0000 front 2024-05-17T22:56:26.427603+0000 (oldest deadline 2024-05-17T22:56:49.327570+0000) 2024-05-17T22:57:16.591 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:16 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:16.322+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:36.329474+0000 front 2024-05-17T22:56:36.329364+0000 (oldest deadline 2024-05-17T22:56:59.829540+0000) 2024-05-17T22:57:16.591 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:16 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:16.322+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.930825+0000 front 2024-05-17T22:56:43.930927+0000 (oldest deadline 2024-05-17T22:57:09.230760+0000) 2024-05-17T22:57:16.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:16 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:16.289+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:16.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:16 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:16.289+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:16.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:16 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:16.289+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:44.416729+0000 front 2024-05-17T22:56:44.416947+0000 (oldest deadline 2024-05-17T22:57:07.316815+0000) 2024-05-17T22:57:16.740 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:16 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:16.420+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:16.740 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:16 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:16.420+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:16.740 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:16 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:16.420+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:17.048 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:16 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:16.775+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:17.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:16 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:16.775+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:17.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:16 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:16.775+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:17.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:16 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:16.775+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.174543+0000 front 2024-05-17T22:56:52.174682+0000 (oldest deadline 2024-05-17T22:57:15.074590+0000) 2024-05-17T22:57:17.394 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:17 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:17.092+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:57:17.394 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:17 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:17.092+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.526326+0000 front 2024-05-17T22:56:33.526441+0000 (oldest deadline 2024-05-17T22:56:58.226157+0000) 2024-05-17T22:57:17.394 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:17 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:17.092+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.527134+0000 front 2024-05-17T22:56:43.527165+0000 (oldest deadline 2024-05-17T22:57:07.027280+0000) 2024-05-17T22:57:17.394 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:17 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:17.092+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:49.928478+0000 front 2024-05-17T22:56:49.928448+0000 (oldest deadline 2024-05-17T22:57:15.828486+0000) 2024-05-17T22:57:17.548 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:17 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:17.106+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.731253+0000 front 2024-05-17T22:56:33.731287+0000 (oldest deadline 2024-05-17T22:56:58.431402+0000) 2024-05-17T22:57:17.548 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:17 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:17.106+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.132595+0000 front 2024-05-17T22:56:43.132667+0000 (oldest deadline 2024-05-17T22:57:08.432629+0000) 2024-05-17T22:57:17.549 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:17 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:17.106+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.335236+0000 front 2024-05-17T22:56:52.335206+0000 (oldest deadline 2024-05-17T22:57:14.635269+0000) 2024-05-17T22:57:17.591 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:17 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:17.313+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:26.427462+0000 front 2024-05-17T22:56:26.427603+0000 (oldest deadline 2024-05-17T22:56:49.327570+0000) 2024-05-17T22:57:17.592 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:17 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:17.313+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:36.329474+0000 front 2024-05-17T22:56:36.329364+0000 (oldest deadline 2024-05-17T22:56:59.829540+0000) 2024-05-17T22:57:17.592 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:17 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:17.313+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.930825+0000 front 2024-05-17T22:56:43.930927+0000 (oldest deadline 2024-05-17T22:57:09.230760+0000) 2024-05-17T22:57:17.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:17 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:17.302+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:17.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:17 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:17.302+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:17.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:17 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:17.302+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:44.416729+0000 front 2024-05-17T22:56:44.416947+0000 (oldest deadline 2024-05-17T22:57:07.316815+0000) 2024-05-17T22:57:17.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:17 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:17.302+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:51.318499+0000 front 2024-05-17T22:56:51.318471+0000 (oldest deadline 2024-05-17T22:57:16.618506+0000) 2024-05-17T22:57:17.740 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:17 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:17.391+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:17.740 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:17 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:17.391+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:17.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:17 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:17.391+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:18.048 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:17 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:17.788+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:18.048 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:17 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:17.788+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:18.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:17 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:17.788+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:18.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:17 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:17.788+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.174543+0000 front 2024-05-17T22:56:52.174682+0000 (oldest deadline 2024-05-17T22:57:15.074590+0000) 2024-05-17T22:57:18.490 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:18 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:18.362+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:18.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:18 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:18.362+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:18.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:18 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:18.362+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:18.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:18 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:18.137+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:57:18.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:18 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:18.137+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.526326+0000 front 2024-05-17T22:56:33.526441+0000 (oldest deadline 2024-05-17T22:56:58.226157+0000) 2024-05-17T22:57:18.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:18 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:18.137+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.527134+0000 front 2024-05-17T22:56:43.527165+0000 (oldest deadline 2024-05-17T22:57:07.027280+0000) 2024-05-17T22:57:18.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:18 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:18.137+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:49.928478+0000 front 2024-05-17T22:56:49.928448+0000 (oldest deadline 2024-05-17T22:57:15.828486+0000) 2024-05-17T22:57:18.549 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:18 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:18.059+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.731253+0000 front 2024-05-17T22:56:33.731287+0000 (oldest deadline 2024-05-17T22:56:58.431402+0000) 2024-05-17T22:57:18.549 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:18 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:18.059+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.132595+0000 front 2024-05-17T22:56:43.132667+0000 (oldest deadline 2024-05-17T22:57:08.432629+0000) 2024-05-17T22:57:18.549 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:18 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:18.059+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.335236+0000 front 2024-05-17T22:56:52.335206+0000 (oldest deadline 2024-05-17T22:57:14.635269+0000) 2024-05-17T22:57:18.591 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:18 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:18.318+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:26.427462+0000 front 2024-05-17T22:56:26.427603+0000 (oldest deadline 2024-05-17T22:56:49.327570+0000) 2024-05-17T22:57:18.592 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:18 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:18.318+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:36.329474+0000 front 2024-05-17T22:56:36.329364+0000 (oldest deadline 2024-05-17T22:56:59.829540+0000) 2024-05-17T22:57:18.592 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:18 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6[59695]: 2024-05-17T22:57:18.318+0000 7f460a016700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.930825+0000 front 2024-05-17T22:56:43.930927+0000 (oldest deadline 2024-05-17T22:57:09.230760+0000) 2024-05-17T22:57:18.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:18 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:18.301+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:18.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:18 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:18.301+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:18.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:18 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:18.301+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:44.416729+0000 front 2024-05-17T22:56:44.416947+0000 (oldest deadline 2024-05-17T22:57:07.316815+0000) 2024-05-17T22:57:18.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:18 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:18.301+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:51.318499+0000 front 2024-05-17T22:56:51.318471+0000 (oldest deadline 2024-05-17T22:57:16.618506+0000) 2024-05-17T22:57:19.048 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:18 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:18.796+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:19.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:18 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:18.796+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:19.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:18 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:18.796+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:19.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:18 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:18.796+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.174543+0000 front 2024-05-17T22:56:52.174682+0000 (oldest deadline 2024-05-17T22:57:15.074590+0000) 2024-05-17T22:57:19.451 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:19 smithi012 podman[115507]: 2024-05-17 22:57:19.164543572 +0000 UTC m=+5.775558426 container died 4292790a652470a813c8954d413f2f4a4b9b96fd2ef84edf4a9691a4b7507857 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-17T22:57:19.451 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:19 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:19.263+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:19.451 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:19 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:19.263+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:19.452 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:19 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:19.263+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:44.416729+0000 front 2024-05-17T22:56:44.416947+0000 (oldest deadline 2024-05-17T22:57:07.316815+0000) 2024-05-17T22:57:19.452 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:19 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:19.263+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:51.318499+0000 front 2024-05-17T22:56:51.318471+0000 (oldest deadline 2024-05-17T22:57:16.618506+0000) 2024-05-17T22:57:19.490 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:19 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:19.350+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:19.490 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:19 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:19.350+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:19.490 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:19 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:19.350+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:19.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:19 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:19.136+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:57:19.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:19 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:19.136+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.526326+0000 front 2024-05-17T22:56:33.526441+0000 (oldest deadline 2024-05-17T22:56:58.226157+0000) 2024-05-17T22:57:19.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:19 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:19.136+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.527134+0000 front 2024-05-17T22:56:43.527165+0000 (oldest deadline 2024-05-17T22:57:07.027280+0000) 2024-05-17T22:57:19.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:19 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:19.137+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:49.928478+0000 front 2024-05-17T22:56:49.928448+0000 (oldest deadline 2024-05-17T22:57:15.828486+0000) 2024-05-17T22:57:19.548 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:19 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:19.075+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.731253+0000 front 2024-05-17T22:56:33.731287+0000 (oldest deadline 2024-05-17T22:56:58.431402+0000) 2024-05-17T22:57:19.548 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:19 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:19.075+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.132595+0000 front 2024-05-17T22:56:43.132667+0000 (oldest deadline 2024-05-17T22:57:08.432629+0000) 2024-05-17T22:57:19.549 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:19 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:19.075+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.335236+0000 front 2024-05-17T22:56:52.335206+0000 (oldest deadline 2024-05-17T22:57:14.635269+0000) 2024-05-17T22:57:19.841 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:19 smithi012 podman[115507]: 2024-05-17 22:57:19.6183617 +0000 UTC m=+6.229376572 container cleanup 4292790a652470a813c8954d413f2f4a4b9b96fd2ef84edf4a9691a4b7507857 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T22:57:19.841 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:19 smithi012 bash[115507]: ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6 2024-05-17T22:57:20.261 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:19 smithi012 podman[115525]: 2024-05-17 22:57:19.951209538 +0000 UTC m=+0.779148719 container remove 4292790a652470a813c8954d413f2f4a4b9b96fd2ef84edf4a9691a4b7507857 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-17T22:57:20.298 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:20 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:20.058+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.731253+0000 front 2024-05-17T22:56:33.731287+0000 (oldest deadline 2024-05-17T22:56:58.431402+0000) 2024-05-17T22:57:20.298 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:20 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:20.058+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.132595+0000 front 2024-05-17T22:56:43.132667+0000 (oldest deadline 2024-05-17T22:57:08.432629+0000) 2024-05-17T22:57:20.298 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:20 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:20.058+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.335236+0000 front 2024-05-17T22:56:52.335206+0000 (oldest deadline 2024-05-17T22:57:14.635269+0000) 2024-05-17T22:57:20.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:19 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:19.840+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:20.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:19 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:19.840+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:20.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:19 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:19.840+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:20.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:19 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:19.840+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.174543+0000 front 2024-05-17T22:56:52.174682+0000 (oldest deadline 2024-05-17T22:57:15.074590+0000) 2024-05-17T22:57:20.490 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:20 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:20.314+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:20.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:20 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:20.314+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:20.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:20 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:20.314+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:20.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:20 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:20.137+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:57:20.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:20 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:20.137+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.526326+0000 front 2024-05-17T22:56:33.526441+0000 (oldest deadline 2024-05-17T22:56:58.226157+0000) 2024-05-17T22:57:20.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:20 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:20.137+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.527134+0000 front 2024-05-17T22:56:43.527165+0000 (oldest deadline 2024-05-17T22:57:07.027280+0000) 2024-05-17T22:57:20.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:20 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:20.137+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:49.928478+0000 front 2024-05-17T22:56:49.928448+0000 (oldest deadline 2024-05-17T22:57:15.828486+0000) 2024-05-17T22:57:20.592 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:20 smithi012 podman[115610]: 2024-05-17 22:57:20.178777261 +0000 UTC m=+0.017834788 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T22:57:20.592 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:20 smithi012 podman[115610]: 2024-05-17 22:57:20.285125141 +0000 UTC m=+0.124182659 container create 60a13afa5065e5de8ff31681c65dc71817a10d30fca4a68154890d93446d08c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, ceph=True) 2024-05-17T22:57:20.592 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:20 smithi012 podman[115610]: 2024-05-17 22:57:20.504560562 +0000 UTC m=+0.343618097 container init 60a13afa5065e5de8ff31681c65dc71817a10d30fca4a68154890d93446d08c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a) 2024-05-17T22:57:20.592 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:20 smithi012 podman[115610]: 2024-05-17 22:57:20.508374516 +0000 UTC m=+0.347432051 container start 60a13afa5065e5de8ff31681c65dc71817a10d30fca4a68154890d93446d08c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6-deactivate, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-17T22:57:20.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:20 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:20.259+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:20.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:20 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:20.259+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:20.593 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:20 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:20.259+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:44.416729+0000 front 2024-05-17T22:56:44.416947+0000 (oldest deadline 2024-05-17T22:57:07.316815+0000) 2024-05-17T22:57:20.593 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:20 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:20.259+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:51.318499+0000 front 2024-05-17T22:56:51.318471+0000 (oldest deadline 2024-05-17T22:57:16.618506+0000) 2024-05-17T22:57:21.068 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:20 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:20.797+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:21.068 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:20 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:20.797+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:21.068 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:20 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:20.797+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:21.068 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:20 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:20.797+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.174543+0000 front 2024-05-17T22:56:52.174682+0000 (oldest deadline 2024-05-17T22:57:15.074590+0000) 2024-05-17T22:57:21.091 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:20 smithi012 podman[115610]: 2024-05-17 22:57:20.604092537 +0000 UTC m=+0.443150077 container attach 60a13afa5065e5de8ff31681c65dc71817a10d30fca4a68154890d93446d08c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T22:57:21.092 INFO:journalctl@ceph.osd.6.smithi012.stdout:May 17 22:57:20 smithi012 podman[115610]: 2024-05-17 22:57:20.816893257 +0000 UTC m=+0.655950784 container died 60a13afa5065e5de8ff31681c65dc71817a10d30fca4a68154890d93446d08c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-17T22:57:21.459 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:21 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:21.225+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:21.460 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:21 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:21.225+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:21.460 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:21 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:21.225+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:44.416729+0000 front 2024-05-17T22:56:44.416947+0000 (oldest deadline 2024-05-17T22:57:07.316815+0000) 2024-05-17T22:57:21.460 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:21 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:21.225+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:51.318499+0000 front 2024-05-17T22:56:51.318471+0000 (oldest deadline 2024-05-17T22:57:16.618506+0000) 2024-05-17T22:57:21.487 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@osd.6.service' 2024-05-17T22:57:21.490 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:21 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:21.337+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:21.490 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:21 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:21.337+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:21.490 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:21 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:21.337+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:21.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:21 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:21.131+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:57:21.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:21 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:21.131+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.526326+0000 front 2024-05-17T22:56:33.526441+0000 (oldest deadline 2024-05-17T22:56:58.226157+0000) 2024-05-17T22:57:21.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:21 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:21.131+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.527134+0000 front 2024-05-17T22:56:43.527165+0000 (oldest deadline 2024-05-17T22:57:07.027280+0000) 2024-05-17T22:57:21.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:21 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:21.131+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:49.928478+0000 front 2024-05-17T22:56:49.928448+0000 (oldest deadline 2024-05-17T22:57:15.828486+0000) 2024-05-17T22:57:21.529 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T22:57:21.529 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-17T22:57:21.529 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-17T22:57:21.529 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-4d593780-149c-11ef-bc9a-c7b262605968@osd.7 2024-05-17T22:57:21.548 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:21 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:21.065+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.731253+0000 front 2024-05-17T22:56:33.731287+0000 (oldest deadline 2024-05-17T22:56:58.431402+0000) 2024-05-17T22:57:21.548 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:21 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:21.065+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.132595+0000 front 2024-05-17T22:56:43.132667+0000 (oldest deadline 2024-05-17T22:57:08.432629+0000) 2024-05-17T22:57:21.548 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:21 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:21.065+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.335236+0000 front 2024-05-17T22:56:52.335206+0000 (oldest deadline 2024-05-17T22:57:14.635269+0000) 2024-05-17T22:57:21.817 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:21 smithi096 systemd[1]: Stopping Ceph osd.7 for 4d593780-149c-11ef-bc9a-c7b262605968... 2024-05-17T22:57:22.299 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:21 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:21.852+0000 7fb6ef19f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T22:57:22.299 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:21 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:21.852+0000 7fb6ef19f700 -1 osd.7 93 *** Got signal Terminated *** 2024-05-17T22:57:22.299 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:21 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:21.852+0000 7fb6ef19f700 -1 osd.7 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T22:57:22.299 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:22 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:22.046+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.731253+0000 front 2024-05-17T22:56:33.731287+0000 (oldest deadline 2024-05-17T22:56:58.431402+0000) 2024-05-17T22:57:22.299 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:22 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:22.046+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.132595+0000 front 2024-05-17T22:56:43.132667+0000 (oldest deadline 2024-05-17T22:57:08.432629+0000) 2024-05-17T22:57:22.299 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:22 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:22.046+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.335236+0000 front 2024-05-17T22:56:52.335206+0000 (oldest deadline 2024-05-17T22:57:14.635269+0000) 2024-05-17T22:57:22.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:21 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:21.815+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:22.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:21 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:21.815+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:22.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:21 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:21.815+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:22.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:21 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:21.815+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.174543+0000 front 2024-05-17T22:56:52.174682+0000 (oldest deadline 2024-05-17T22:57:15.074590+0000) 2024-05-17T22:57:22.490 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:22 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:22.312+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:22.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:22 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:22.312+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:22.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:22 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:22.312+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:22.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:22 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:22.163+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:57:22.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:22 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:22.163+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.526326+0000 front 2024-05-17T22:56:33.526441+0000 (oldest deadline 2024-05-17T22:56:58.226157+0000) 2024-05-17T22:57:22.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:22 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:22.163+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.527134+0000 front 2024-05-17T22:56:43.527165+0000 (oldest deadline 2024-05-17T22:57:07.027280+0000) 2024-05-17T22:57:22.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:22 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:22.163+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:49.928478+0000 front 2024-05-17T22:56:49.928448+0000 (oldest deadline 2024-05-17T22:57:15.828486+0000) 2024-05-17T22:57:22.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:22 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:22.258+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:22.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:22 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:22.258+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:22.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:22 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:22.258+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:44.416729+0000 front 2024-05-17T22:56:44.416947+0000 (oldest deadline 2024-05-17T22:57:07.316815+0000) 2024-05-17T22:57:22.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:22 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:22.258+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:51.318499+0000 front 2024-05-17T22:56:51.318471+0000 (oldest deadline 2024-05-17T22:57:16.618506+0000) 2024-05-17T22:57:23.298 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:23 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:23.071+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.731253+0000 front 2024-05-17T22:56:33.731287+0000 (oldest deadline 2024-05-17T22:56:58.431402+0000) 2024-05-17T22:57:23.299 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:23 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:23.071+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.132595+0000 front 2024-05-17T22:56:43.132667+0000 (oldest deadline 2024-05-17T22:57:08.432629+0000) 2024-05-17T22:57:23.299 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:23 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:23.071+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.335236+0000 front 2024-05-17T22:56:52.335206+0000 (oldest deadline 2024-05-17T22:57:14.635269+0000) 2024-05-17T22:57:23.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:22 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:22.825+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:23.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:22 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:22.825+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:23.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:22 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:22.825+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:23.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:22 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:22.825+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.174543+0000 front 2024-05-17T22:56:52.174682+0000 (oldest deadline 2024-05-17T22:57:15.074590+0000) 2024-05-17T22:57:23.490 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:23 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:23.277+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:23.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:23 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:23.277+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:23.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:23 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:23.277+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:23.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:23 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:23.212+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:57:23.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:23 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:23.212+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.526326+0000 front 2024-05-17T22:56:33.526441+0000 (oldest deadline 2024-05-17T22:56:58.226157+0000) 2024-05-17T22:57:23.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:23 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:23.212+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.527134+0000 front 2024-05-17T22:56:43.527165+0000 (oldest deadline 2024-05-17T22:57:07.027280+0000) 2024-05-17T22:57:23.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:23 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:23.212+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:49.928478+0000 front 2024-05-17T22:56:49.928448+0000 (oldest deadline 2024-05-17T22:57:15.828486+0000) 2024-05-17T22:57:23.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:23 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:23.266+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:23.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:23 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:23.266+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:23.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:23 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:23.266+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:44.416729+0000 front 2024-05-17T22:56:44.416947+0000 (oldest deadline 2024-05-17T22:57:07.316815+0000) 2024-05-17T22:57:23.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:23 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:23.266+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:51.318499+0000 front 2024-05-17T22:56:51.318471+0000 (oldest deadline 2024-05-17T22:57:16.618506+0000) 2024-05-17T22:57:24.298 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:24 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:24.046+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.731253+0000 front 2024-05-17T22:56:33.731287+0000 (oldest deadline 2024-05-17T22:56:58.431402+0000) 2024-05-17T22:57:24.299 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:24 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:24.046+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.132595+0000 front 2024-05-17T22:56:43.132667+0000 (oldest deadline 2024-05-17T22:57:08.432629+0000) 2024-05-17T22:57:24.299 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:24 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:24.046+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.335236+0000 front 2024-05-17T22:56:52.335206+0000 (oldest deadline 2024-05-17T22:57:14.635269+0000) 2024-05-17T22:57:24.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:23 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:23.798+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:24.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:23 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:23.798+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:24.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:23 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:23.798+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:24.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:23 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:23.798+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.174543+0000 front 2024-05-17T22:56:52.174682+0000 (oldest deadline 2024-05-17T22:57:15.074590+0000) 2024-05-17T22:57:24.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:23 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:23.798+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.375428+0000 front 2024-05-17T22:57:00.375447+0000 (oldest deadline 2024-05-17T22:57:23.275565+0000) 2024-05-17T22:57:24.490 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:24 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:24.262+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:24.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:24 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:24.262+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:24.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:24 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:24.262+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:24.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:24 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:24.182+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:57:24.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:24 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:24.182+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.526326+0000 front 2024-05-17T22:56:33.526441+0000 (oldest deadline 2024-05-17T22:56:58.226157+0000) 2024-05-17T22:57:24.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:24 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:24.182+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.527134+0000 front 2024-05-17T22:56:43.527165+0000 (oldest deadline 2024-05-17T22:57:07.027280+0000) 2024-05-17T22:57:24.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:24 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:24.182+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:49.928478+0000 front 2024-05-17T22:56:49.928448+0000 (oldest deadline 2024-05-17T22:57:15.828486+0000) 2024-05-17T22:57:24.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:24 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:24.226+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:24.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:24 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:24.226+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:24.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:24 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:24.226+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:44.416729+0000 front 2024-05-17T22:56:44.416947+0000 (oldest deadline 2024-05-17T22:57:07.316815+0000) 2024-05-17T22:57:24.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:24 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:24.226+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:51.318499+0000 front 2024-05-17T22:56:51.318471+0000 (oldest deadline 2024-05-17T22:57:16.618506+0000) 2024-05-17T22:57:25.299 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:25 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:25.029+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.731253+0000 front 2024-05-17T22:56:33.731287+0000 (oldest deadline 2024-05-17T22:56:58.431402+0000) 2024-05-17T22:57:25.299 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:25 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:25.029+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.132595+0000 front 2024-05-17T22:56:43.132667+0000 (oldest deadline 2024-05-17T22:57:08.432629+0000) 2024-05-17T22:57:25.299 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:25 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:25.029+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.335236+0000 front 2024-05-17T22:56:52.335206+0000 (oldest deadline 2024-05-17T22:57:14.635269+0000) 2024-05-17T22:57:25.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:24 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:24.797+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:25.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:24 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:24.797+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:25.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:24 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:24.797+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:25.300 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:24 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:24.797+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.174543+0000 front 2024-05-17T22:56:52.174682+0000 (oldest deadline 2024-05-17T22:57:15.074590+0000) 2024-05-17T22:57:25.300 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:24 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:24.797+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.375428+0000 front 2024-05-17T22:57:00.375447+0000 (oldest deadline 2024-05-17T22:57:23.275565+0000) 2024-05-17T22:57:25.490 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:25 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:25.223+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:57:25.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:25 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:25.223+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.526326+0000 front 2024-05-17T22:56:33.526441+0000 (oldest deadline 2024-05-17T22:56:58.226157+0000) 2024-05-17T22:57:25.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:25 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:25.223+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.527134+0000 front 2024-05-17T22:56:43.527165+0000 (oldest deadline 2024-05-17T22:57:07.027280+0000) 2024-05-17T22:57:25.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:25 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:25.223+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:49.928478+0000 front 2024-05-17T22:56:49.928448+0000 (oldest deadline 2024-05-17T22:57:15.828486+0000) 2024-05-17T22:57:25.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:25 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:25.235+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:25.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:25 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:25.235+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:25.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:25 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:25.235+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:25.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:25 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:25.235+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.932214+0000 front 2024-05-17T22:57:00.932196+0000 (oldest deadline 2024-05-17T22:57:25.032107+0000) 2024-05-17T22:57:25.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:25 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:25.225+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:25.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:25 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:25.225+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:25.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:25 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:25.225+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:44.416729+0000 front 2024-05-17T22:56:44.416947+0000 (oldest deadline 2024-05-17T22:57:07.316815+0000) 2024-05-17T22:57:25.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:25 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:25.225+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:51.318499+0000 front 2024-05-17T22:56:51.318471+0000 (oldest deadline 2024-05-17T22:57:16.618506+0000) 2024-05-17T22:57:26.298 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:26 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:26.023+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.731253+0000 front 2024-05-17T22:56:33.731287+0000 (oldest deadline 2024-05-17T22:56:58.431402+0000) 2024-05-17T22:57:26.299 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:26 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:26.023+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.132595+0000 front 2024-05-17T22:56:43.132667+0000 (oldest deadline 2024-05-17T22:57:08.432629+0000) 2024-05-17T22:57:26.299 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:26 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7[54412]: 2024-05-17T22:57:26.023+0000 7fb6eb755700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.335236+0000 front 2024-05-17T22:56:52.335206+0000 (oldest deadline 2024-05-17T22:57:14.635269+0000) 2024-05-17T22:57:26.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:25 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:25.831+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:26.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:25 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:25.831+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:26.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:25 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:25.831+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:26.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:25 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:25.831+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.174543+0000 front 2024-05-17T22:56:52.174682+0000 (oldest deadline 2024-05-17T22:57:15.074590+0000) 2024-05-17T22:57:26.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:25 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:25.831+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.375428+0000 front 2024-05-17T22:57:00.375447+0000 (oldest deadline 2024-05-17T22:57:23.275565+0000) 2024-05-17T22:57:26.490 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:26 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:26.194+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:26.490 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:26 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:26.194+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:26.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:26 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:26.194+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:26.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:26 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:26.194+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.932214+0000 front 2024-05-17T22:57:00.932196+0000 (oldest deadline 2024-05-17T22:57:25.032107+0000) 2024-05-17T22:57:26.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:26 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:26.228+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:57:26.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:26 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:26.228+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.526326+0000 front 2024-05-17T22:56:33.526441+0000 (oldest deadline 2024-05-17T22:56:58.226157+0000) 2024-05-17T22:57:26.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:26 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:26.228+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.527134+0000 front 2024-05-17T22:56:43.527165+0000 (oldest deadline 2024-05-17T22:57:07.027280+0000) 2024-05-17T22:57:26.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:26 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:26.228+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:49.928478+0000 front 2024-05-17T22:56:49.928448+0000 (oldest deadline 2024-05-17T22:57:15.828486+0000) 2024-05-17T22:57:26.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:26 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:26.199+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:26.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:26 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:26.199+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:26.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:26 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:26.199+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:44.416729+0000 front 2024-05-17T22:56:44.416947+0000 (oldest deadline 2024-05-17T22:57:07.316815+0000) 2024-05-17T22:57:26.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:26 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:26.199+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:51.318499+0000 front 2024-05-17T22:56:51.318471+0000 (oldest deadline 2024-05-17T22:57:16.618506+0000) 2024-05-17T22:57:26.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:26 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:26.199+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.719735+0000 front 2024-05-17T22:57:00.719715+0000 (oldest deadline 2024-05-17T22:57:26.019681+0000) 2024-05-17T22:57:27.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:26 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:26.867+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:27.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:26 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:26.867+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:27.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:26 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:26.867+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:27.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:26 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:26.867+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.174543+0000 front 2024-05-17T22:56:52.174682+0000 (oldest deadline 2024-05-17T22:57:15.074590+0000) 2024-05-17T22:57:27.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:26 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:26.867+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.375428+0000 front 2024-05-17T22:57:00.375447+0000 (oldest deadline 2024-05-17T22:57:23.275565+0000) 2024-05-17T22:57:27.490 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:27 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:27.181+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:57:27.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:27 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:27.181+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.526326+0000 front 2024-05-17T22:56:33.526441+0000 (oldest deadline 2024-05-17T22:56:58.226157+0000) 2024-05-17T22:57:27.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:27 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:27.181+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.527134+0000 front 2024-05-17T22:56:43.527165+0000 (oldest deadline 2024-05-17T22:57:07.027280+0000) 2024-05-17T22:57:27.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:27 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:27.181+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:49.928478+0000 front 2024-05-17T22:56:49.928448+0000 (oldest deadline 2024-05-17T22:57:15.828486+0000) 2024-05-17T22:57:27.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:27 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:27.240+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:27.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:27 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:27.240+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:27.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:27 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:27.240+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:27.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:27 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:27.240+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.932214+0000 front 2024-05-17T22:57:00.932196+0000 (oldest deadline 2024-05-17T22:57:25.032107+0000) 2024-05-17T22:57:27.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:27 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:27.201+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:27.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:27 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:27.201+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:27.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:27 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:27.201+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:44.416729+0000 front 2024-05-17T22:56:44.416947+0000 (oldest deadline 2024-05-17T22:57:07.316815+0000) 2024-05-17T22:57:27.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:27 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:27.201+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:51.318499+0000 front 2024-05-17T22:56:51.318471+0000 (oldest deadline 2024-05-17T22:57:16.618506+0000) 2024-05-17T22:57:27.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:27 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:27.201+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.719735+0000 front 2024-05-17T22:57:00.719715+0000 (oldest deadline 2024-05-17T22:57:26.019681+0000) 2024-05-17T22:57:27.823 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:27 smithi096 podman[72708]: 2024-05-17 22:57:27.561109799 +0000 UTC m=+5.839143754 container died bd9d07228d66330921beb5aff9d422f6fdd340a95037f14230bcdc51e18b53b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-17T22:57:28.238 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:27 smithi096 podman[72708]: 2024-05-17 22:57:27.960714796 +0000 UTC m=+6.238748750 container cleanup bd9d07228d66330921beb5aff9d422f6fdd340a95037f14230bcdc51e18b53b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-17T22:57:28.239 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:27 smithi096 bash[72708]: ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7 2024-05-17T22:57:28.239 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:27 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:27.821+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:28.239 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:27 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:27.821+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:28.239 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:27 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:27.821+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:28.239 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:27 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:27.821+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.174543+0000 front 2024-05-17T22:56:52.174682+0000 (oldest deadline 2024-05-17T22:57:15.074590+0000) 2024-05-17T22:57:28.239 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:27 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:27.821+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.375428+0000 front 2024-05-17T22:57:00.375447+0000 (oldest deadline 2024-05-17T22:57:23.275565+0000) 2024-05-17T22:57:28.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:28 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:28.158+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:57:28.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:28 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:28.158+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.526326+0000 front 2024-05-17T22:56:33.526441+0000 (oldest deadline 2024-05-17T22:56:58.226157+0000) 2024-05-17T22:57:28.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:28 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:28.158+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.527134+0000 front 2024-05-17T22:56:43.527165+0000 (oldest deadline 2024-05-17T22:57:07.027280+0000) 2024-05-17T22:57:28.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:28 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:28.158+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:49.928478+0000 front 2024-05-17T22:56:49.928448+0000 (oldest deadline 2024-05-17T22:57:15.828486+0000) 2024-05-17T22:57:28.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:28 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:28.251+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:28.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:28 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:28.251+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:28.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:28 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:28.251+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:28.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:28 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:28.251+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.932214+0000 front 2024-05-17T22:57:00.932196+0000 (oldest deadline 2024-05-17T22:57:25.032107+0000) 2024-05-17T22:57:28.548 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:28 smithi096 podman[72722]: 2024-05-17 22:57:28.237074679 +0000 UTC m=+0.674480891 container remove bd9d07228d66330921beb5aff9d422f6fdd340a95037f14230bcdc51e18b53b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS) 2024-05-17T22:57:28.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:28 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:28.241+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:28.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:28 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:28.241+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:28.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:28 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:28.241+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:44.416729+0000 front 2024-05-17T22:56:44.416947+0000 (oldest deadline 2024-05-17T22:57:07.316815+0000) 2024-05-17T22:57:28.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:28 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:28.241+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:51.318499+0000 front 2024-05-17T22:56:51.318471+0000 (oldest deadline 2024-05-17T22:57:16.618506+0000) 2024-05-17T22:57:28.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:28 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:28.241+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.719735+0000 front 2024-05-17T22:57:00.719715+0000 (oldest deadline 2024-05-17T22:57:26.019681+0000) 2024-05-17T22:57:28.834 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:28 smithi096 podman[72815]: 2024-05-17 22:57:28.57135356 +0000 UTC m=+0.108420629 container create dc855586b6a70b3eaf857556d7c8ff43d3e6c4b79e79ebacced2e3fb49ea9aeb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, CEPH_POINT_RELEASE=) 2024-05-17T22:57:28.834 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:28 smithi096 podman[72815]: 2024-05-17 22:57:28.480318868 +0000 UTC m=+0.017385899 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T22:57:29.133 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:28 smithi096 podman[72815]: 2024-05-17 22:57:28.832525703 +0000 UTC m=+0.369592730 container init dc855586b6a70b3eaf857556d7c8ff43d3e6c4b79e79ebacced2e3fb49ea9aeb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T22:57:29.133 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:28 smithi096 podman[72815]: 2024-05-17 22:57:28.836113848 +0000 UTC m=+0.373180875 container start dc855586b6a70b3eaf857556d7c8ff43d3e6c4b79e79ebacced2e3fb49ea9aeb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-17T22:57:29.133 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:28 smithi096 podman[72815]: 2024-05-17 22:57:28.940621926 +0000 UTC m=+0.477688965 container attach dc855586b6a70b3eaf857556d7c8ff43d3e6c4b79e79ebacced2e3fb49ea9aeb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, io.buildah.version=1.33.5, ceph=True) 2024-05-17T22:57:29.134 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:28 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:28.861+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:29.134 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:28 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:28.861+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:29.134 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:28 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:28.861+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:29.134 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:28 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:28.861+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.174543+0000 front 2024-05-17T22:56:52.174682+0000 (oldest deadline 2024-05-17T22:57:15.074590+0000) 2024-05-17T22:57:29.134 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:28 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:28.861+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.375428+0000 front 2024-05-17T22:57:00.375447+0000 (oldest deadline 2024-05-17T22:57:23.275565+0000) 2024-05-17T22:57:29.408 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 17 22:57:29 smithi096 podman[72815]: 2024-05-17 22:57:29.132134276 +0000 UTC m=+0.669201319 container died dc855586b6a70b3eaf857556d7c8ff43d3e6c4b79e79ebacced2e3fb49ea9aeb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514) 2024-05-17T22:57:29.490 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:29 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:29.195+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:57:29.490 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:29 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:29.195+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.526326+0000 front 2024-05-17T22:56:33.526441+0000 (oldest deadline 2024-05-17T22:56:58.226157+0000) 2024-05-17T22:57:29.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:29 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:29.195+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.527134+0000 front 2024-05-17T22:56:43.527165+0000 (oldest deadline 2024-05-17T22:57:07.027280+0000) 2024-05-17T22:57:29.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:29 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:29.195+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:49.928478+0000 front 2024-05-17T22:56:49.928448+0000 (oldest deadline 2024-05-17T22:57:15.828486+0000) 2024-05-17T22:57:29.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:29 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:29.211+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:29.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:29 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:29.211+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:29.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:29 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:29.211+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:29.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:29 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:29.211+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.932214+0000 front 2024-05-17T22:57:00.932196+0000 (oldest deadline 2024-05-17T22:57:25.032107+0000) 2024-05-17T22:57:29.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:29 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:29.213+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:29.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:29 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:29.213+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:29.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:29 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:29.213+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:44.416729+0000 front 2024-05-17T22:56:44.416947+0000 (oldest deadline 2024-05-17T22:57:07.316815+0000) 2024-05-17T22:57:29.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:29 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:29.213+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:51.318499+0000 front 2024-05-17T22:56:51.318471+0000 (oldest deadline 2024-05-17T22:57:16.618506+0000) 2024-05-17T22:57:29.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:29 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:29.213+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.719735+0000 front 2024-05-17T22:57:00.719715+0000 (oldest deadline 2024-05-17T22:57:26.019681+0000) 2024-05-17T22:57:29.789 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@osd.7.service' 2024-05-17T22:57:29.865 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T22:57:29.865 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-17T22:57:29.866 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-17T22:57:29.866 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-4d593780-149c-11ef-bc9a-c7b262605968@osd.8 2024-05-17T22:57:30.048 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:29 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:29.839+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:30.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:29 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:29.839+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:30.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:29 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:29.839+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:30.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:29 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:29.839+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.174543+0000 front 2024-05-17T22:56:52.174682+0000 (oldest deadline 2024-05-17T22:57:15.074590+0000) 2024-05-17T22:57:30.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:29 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:29.839+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.375428+0000 front 2024-05-17T22:57:00.375447+0000 (oldest deadline 2024-05-17T22:57:23.275565+0000) 2024-05-17T22:57:30.154 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:29 smithi151 systemd[1]: Stopping Ceph osd.8 for 4d593780-149c-11ef-bc9a-c7b262605968... 2024-05-17T22:57:30.154 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:30 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:30.151+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:57:30.154 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:30 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:30.151+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.526326+0000 front 2024-05-17T22:56:33.526441+0000 (oldest deadline 2024-05-17T22:56:58.226157+0000) 2024-05-17T22:57:30.155 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:30 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:30.151+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.527134+0000 front 2024-05-17T22:56:43.527165+0000 (oldest deadline 2024-05-17T22:57:07.027280+0000) 2024-05-17T22:57:30.155 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:30 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:30.151+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:49.928478+0000 front 2024-05-17T22:56:49.928448+0000 (oldest deadline 2024-05-17T22:57:15.828486+0000) 2024-05-17T22:57:30.490 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:30 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:30.188+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:30.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:30 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:30.188+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:30.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:30 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:30.188+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:30.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:30 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:30.188+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.932214+0000 front 2024-05-17T22:57:00.932196+0000 (oldest deadline 2024-05-17T22:57:25.032107+0000) 2024-05-17T22:57:30.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:30 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:30.169+0000 7f5b7832b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T22:57:30.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:30 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:30.169+0000 7f5b7832b700 -1 osd.8 93 *** Got signal Terminated *** 2024-05-17T22:57:30.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:30 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:30.169+0000 7f5b7832b700 -1 osd.8 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T22:57:30.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:30 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:30.179+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:30.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:30 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:30.179+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:30.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:30 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:30.179+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:44.416729+0000 front 2024-05-17T22:56:44.416947+0000 (oldest deadline 2024-05-17T22:57:07.316815+0000) 2024-05-17T22:57:30.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:30 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:30.179+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:51.318499+0000 front 2024-05-17T22:56:51.318471+0000 (oldest deadline 2024-05-17T22:57:16.618506+0000) 2024-05-17T22:57:30.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:30 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:30.179+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.719735+0000 front 2024-05-17T22:57:00.719715+0000 (oldest deadline 2024-05-17T22:57:26.019681+0000) 2024-05-17T22:57:31.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:30 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:30.832+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:31.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:30 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:30.832+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:31.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:30 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:30.832+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:31.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:30 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:30.832+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.174543+0000 front 2024-05-17T22:56:52.174682+0000 (oldest deadline 2024-05-17T22:57:15.074590+0000) 2024-05-17T22:57:31.300 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:30 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:30.832+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.375428+0000 front 2024-05-17T22:57:00.375447+0000 (oldest deadline 2024-05-17T22:57:23.275565+0000) 2024-05-17T22:57:31.490 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:31 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:31.107+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:57:31.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:31 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:31.107+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.526326+0000 front 2024-05-17T22:56:33.526441+0000 (oldest deadline 2024-05-17T22:56:58.226157+0000) 2024-05-17T22:57:31.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:31 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:31.107+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.527134+0000 front 2024-05-17T22:56:43.527165+0000 (oldest deadline 2024-05-17T22:57:07.027280+0000) 2024-05-17T22:57:31.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:31 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:31.107+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:49.928478+0000 front 2024-05-17T22:56:49.928448+0000 (oldest deadline 2024-05-17T22:57:15.828486+0000) 2024-05-17T22:57:31.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:31 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:31.224+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:31.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:31 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:31.224+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:31.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:31 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:31.224+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:31.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:31 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:31.224+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.932214+0000 front 2024-05-17T22:57:00.932196+0000 (oldest deadline 2024-05-17T22:57:25.032107+0000) 2024-05-17T22:57:31.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:31 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:31.140+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:31.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:31 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:31.140+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:31.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:31 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:31.140+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:44.416729+0000 front 2024-05-17T22:56:44.416947+0000 (oldest deadline 2024-05-17T22:57:07.316815+0000) 2024-05-17T22:57:31.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:31 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:31.140+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:51.318499+0000 front 2024-05-17T22:56:51.318471+0000 (oldest deadline 2024-05-17T22:57:16.618506+0000) 2024-05-17T22:57:31.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:31 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:31.140+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.719735+0000 front 2024-05-17T22:57:00.719715+0000 (oldest deadline 2024-05-17T22:57:26.019681+0000) 2024-05-17T22:57:32.298 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:31 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:31.828+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:32.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:31 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:31.828+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:32.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:31 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:31.828+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:32.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:31 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:31.828+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.174543+0000 front 2024-05-17T22:56:52.174682+0000 (oldest deadline 2024-05-17T22:57:15.074590+0000) 2024-05-17T22:57:32.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:31 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:31.828+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.375428+0000 front 2024-05-17T22:57:00.375447+0000 (oldest deadline 2024-05-17T22:57:23.275565+0000) 2024-05-17T22:57:32.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:32 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:32.067+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:57:32.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:32 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:32.067+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.526326+0000 front 2024-05-17T22:56:33.526441+0000 (oldest deadline 2024-05-17T22:56:58.226157+0000) 2024-05-17T22:57:32.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:32 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:32.067+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.527134+0000 front 2024-05-17T22:56:43.527165+0000 (oldest deadline 2024-05-17T22:57:07.027280+0000) 2024-05-17T22:57:32.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:32 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:32.067+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:49.928478+0000 front 2024-05-17T22:56:49.928448+0000 (oldest deadline 2024-05-17T22:57:15.828486+0000) 2024-05-17T22:57:32.492 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:32 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:32.242+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:32.492 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:32 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:32.242+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:32.492 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:32 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:32.242+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:32.492 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:32 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:32.242+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.932214+0000 front 2024-05-17T22:57:00.932196+0000 (oldest deadline 2024-05-17T22:57:25.032107+0000) 2024-05-17T22:57:32.493 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:32 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:32.165+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:32.493 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:32 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:32.165+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:32.493 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:32 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:32.165+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:44.416729+0000 front 2024-05-17T22:56:44.416947+0000 (oldest deadline 2024-05-17T22:57:07.316815+0000) 2024-05-17T22:57:32.494 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:32 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:32.165+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:51.318499+0000 front 2024-05-17T22:56:51.318471+0000 (oldest deadline 2024-05-17T22:57:16.618506+0000) 2024-05-17T22:57:32.494 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:32 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:32.165+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.719735+0000 front 2024-05-17T22:57:00.719715+0000 (oldest deadline 2024-05-17T22:57:26.019681+0000) 2024-05-17T22:57:32.494 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:32 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:32.165+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:08.321061+0000 front 2024-05-17T22:57:08.321042+0000 (oldest deadline 2024-05-17T22:57:31.820907+0000) 2024-05-17T22:57:33.048 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:32 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:32.783+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:33.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:32 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:32.783+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:33.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:32 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:32.783+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:33.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:32 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:32.783+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.174543+0000 front 2024-05-17T22:56:52.174682+0000 (oldest deadline 2024-05-17T22:57:15.074590+0000) 2024-05-17T22:57:33.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:32 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:32.783+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.375428+0000 front 2024-05-17T22:57:00.375447+0000 (oldest deadline 2024-05-17T22:57:23.275565+0000) 2024-05-17T22:57:33.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:32 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:32.783+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:08.377691+0000 front 2024-05-17T22:57:08.377608+0000 (oldest deadline 2024-05-17T22:57:31.877496+0000) 2024-05-17T22:57:33.490 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:33 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:33.103+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:57:33.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:33 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:33.103+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.526326+0000 front 2024-05-17T22:56:33.526441+0000 (oldest deadline 2024-05-17T22:56:58.226157+0000) 2024-05-17T22:57:33.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:33 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:33.103+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.527134+0000 front 2024-05-17T22:56:43.527165+0000 (oldest deadline 2024-05-17T22:57:07.027280+0000) 2024-05-17T22:57:33.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:33 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:33.103+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:49.928478+0000 front 2024-05-17T22:56:49.928448+0000 (oldest deadline 2024-05-17T22:57:15.828486+0000) 2024-05-17T22:57:33.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:33 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:33.257+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:33.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:33 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:33.257+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:33.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:33 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:33.257+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:33.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:33 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:33.257+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.932214+0000 front 2024-05-17T22:57:00.932196+0000 (oldest deadline 2024-05-17T22:57:25.032107+0000) 2024-05-17T22:57:33.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:33 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:33.198+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:33.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:33 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:33.198+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:33.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:33 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:33.198+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:44.416729+0000 front 2024-05-17T22:56:44.416947+0000 (oldest deadline 2024-05-17T22:57:07.316815+0000) 2024-05-17T22:57:33.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:33 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:33.198+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:51.318499+0000 front 2024-05-17T22:56:51.318471+0000 (oldest deadline 2024-05-17T22:57:16.618506+0000) 2024-05-17T22:57:33.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:33 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:33.198+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.719735+0000 front 2024-05-17T22:57:00.719715+0000 (oldest deadline 2024-05-17T22:57:26.019681+0000) 2024-05-17T22:57:33.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:33 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:33.198+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:08.321061+0000 front 2024-05-17T22:57:08.321042+0000 (oldest deadline 2024-05-17T22:57:31.820907+0000) 2024-05-17T22:57:34.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:33 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:33.806+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:34.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:33 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:33.806+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:34.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:33 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:33.806+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:34.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:33 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:33.806+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.174543+0000 front 2024-05-17T22:56:52.174682+0000 (oldest deadline 2024-05-17T22:57:15.074590+0000) 2024-05-17T22:57:34.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:33 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:33.806+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.375428+0000 front 2024-05-17T22:57:00.375447+0000 (oldest deadline 2024-05-17T22:57:23.275565+0000) 2024-05-17T22:57:34.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:33 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:33.806+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:08.377691+0000 front 2024-05-17T22:57:08.377608+0000 (oldest deadline 2024-05-17T22:57:31.877496+0000) 2024-05-17T22:57:34.490 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:34 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:34.266+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:34.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:34 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:34.266+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:34.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:34 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:34.266+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:34.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:34 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:34.266+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.932214+0000 front 2024-05-17T22:57:00.932196+0000 (oldest deadline 2024-05-17T22:57:25.032107+0000) 2024-05-17T22:57:34.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:34 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:34.140+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:57:34.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:34 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:34.140+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.526326+0000 front 2024-05-17T22:56:33.526441+0000 (oldest deadline 2024-05-17T22:56:58.226157+0000) 2024-05-17T22:57:34.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:34 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:34.140+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.527134+0000 front 2024-05-17T22:56:43.527165+0000 (oldest deadline 2024-05-17T22:57:07.027280+0000) 2024-05-17T22:57:34.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:34 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:34.140+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:49.928478+0000 front 2024-05-17T22:56:49.928448+0000 (oldest deadline 2024-05-17T22:57:15.828486+0000) 2024-05-17T22:57:34.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:34 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:34.140+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:08.631929+0000 front 2024-05-17T22:57:08.631956+0000 (oldest deadline 2024-05-17T22:57:33.932094+0000) 2024-05-17T22:57:34.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:34 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:34.218+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:34.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:34 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:34.218+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:34.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:34 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:34.218+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:44.416729+0000 front 2024-05-17T22:56:44.416947+0000 (oldest deadline 2024-05-17T22:57:07.316815+0000) 2024-05-17T22:57:34.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:34 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:34.218+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:51.318499+0000 front 2024-05-17T22:56:51.318471+0000 (oldest deadline 2024-05-17T22:57:16.618506+0000) 2024-05-17T22:57:34.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:34 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:34.218+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.719735+0000 front 2024-05-17T22:57:00.719715+0000 (oldest deadline 2024-05-17T22:57:26.019681+0000) 2024-05-17T22:57:34.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:34 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:34.218+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:08.321061+0000 front 2024-05-17T22:57:08.321042+0000 (oldest deadline 2024-05-17T22:57:31.820907+0000) 2024-05-17T22:57:35.298 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:34 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:34.842+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:35.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:34 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:34.842+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:35.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:34 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:34.842+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:35.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:34 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:34.842+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.174543+0000 front 2024-05-17T22:56:52.174682+0000 (oldest deadline 2024-05-17T22:57:15.074590+0000) 2024-05-17T22:57:35.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:34 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:34.842+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.375428+0000 front 2024-05-17T22:57:00.375447+0000 (oldest deadline 2024-05-17T22:57:23.275565+0000) 2024-05-17T22:57:35.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:34 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:34.842+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:08.377691+0000 front 2024-05-17T22:57:08.377608+0000 (oldest deadline 2024-05-17T22:57:31.877496+0000) 2024-05-17T22:57:35.490 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:35 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:35.096+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.725285+0000 front 2024-05-17T22:56:27.725348+0000 (oldest deadline 2024-05-17T22:56:51.225038+0000) 2024-05-17T22:57:35.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:35 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:35.096+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:33.526326+0000 front 2024-05-17T22:56:33.526441+0000 (oldest deadline 2024-05-17T22:56:58.226157+0000) 2024-05-17T22:57:35.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:35 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:35.096+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.527134+0000 front 2024-05-17T22:56:43.527165+0000 (oldest deadline 2024-05-17T22:57:07.027280+0000) 2024-05-17T22:57:35.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:35 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:35.096+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:49.928478+0000 front 2024-05-17T22:56:49.928448+0000 (oldest deadline 2024-05-17T22:57:15.828486+0000) 2024-05-17T22:57:35.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:35 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8[55112]: 2024-05-17T22:57:35.096+0000 7f5b708d9700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:08.631929+0000 front 2024-05-17T22:57:08.631956+0000 (oldest deadline 2024-05-17T22:57:33.932094+0000) 2024-05-17T22:57:35.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:35 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:35.230+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:35.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:35 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:35.230+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:35.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:35 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:35.230+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:35.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:35 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:35.230+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.932214+0000 front 2024-05-17T22:57:00.932196+0000 (oldest deadline 2024-05-17T22:57:25.032107+0000) 2024-05-17T22:57:35.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:35 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:35.230+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:10.333578+0000 front 2024-05-17T22:57:10.333491+0000 (oldest deadline 2024-05-17T22:57:34.433626+0000) 2024-05-17T22:57:35.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:35 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:35.215+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:35.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:35 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:35.215+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:35.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:35 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:35.215+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:44.416729+0000 front 2024-05-17T22:56:44.416947+0000 (oldest deadline 2024-05-17T22:57:07.316815+0000) 2024-05-17T22:57:35.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:35 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:35.215+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:51.318499+0000 front 2024-05-17T22:56:51.318471+0000 (oldest deadline 2024-05-17T22:57:16.618506+0000) 2024-05-17T22:57:35.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:35 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:35.215+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.719735+0000 front 2024-05-17T22:57:00.719715+0000 (oldest deadline 2024-05-17T22:57:26.019681+0000) 2024-05-17T22:57:35.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:35 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:35.215+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:08.321061+0000 front 2024-05-17T22:57:08.321042+0000 (oldest deadline 2024-05-17T22:57:31.820907+0000) 2024-05-17T22:57:36.173 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:35 smithi151 podman[73853]: 2024-05-17 22:57:35.86819449 +0000 UTC m=+5.816111419 container died d7128d3bc674fc9bed4c3177ecbf5d4147916c6853bf40b4653dec81e135a724 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-17T22:57:36.298 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:35 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:35.813+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:36.298 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:35 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:35.813+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:36.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:35 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:35.813+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:36.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:35 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:35.813+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.174543+0000 front 2024-05-17T22:56:52.174682+0000 (oldest deadline 2024-05-17T22:57:15.074590+0000) 2024-05-17T22:57:36.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:35 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:35.813+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.375428+0000 front 2024-05-17T22:57:00.375447+0000 (oldest deadline 2024-05-17T22:57:23.275565+0000) 2024-05-17T22:57:36.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:35 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:35.813+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:08.377691+0000 front 2024-05-17T22:57:08.377608+0000 (oldest deadline 2024-05-17T22:57:31.877496+0000) 2024-05-17T22:57:36.492 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:36 smithi151 podman[73853]: 2024-05-17 22:57:36.270794531 +0000 UTC m=+6.218711461 container cleanup d7128d3bc674fc9bed4c3177ecbf5d4147916c6853bf40b4653dec81e135a724 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-17T22:57:36.492 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:36 smithi151 bash[73853]: ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8 2024-05-17T22:57:36.493 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:36 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:36.217+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:36.493 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:36 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:36.217+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:36.493 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:36 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:36.217+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:36.493 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:36 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:36.217+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.932214+0000 front 2024-05-17T22:57:00.932196+0000 (oldest deadline 2024-05-17T22:57:25.032107+0000) 2024-05-17T22:57:36.493 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:36 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:36.217+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:10.333578+0000 front 2024-05-17T22:57:10.333491+0000 (oldest deadline 2024-05-17T22:57:34.433626+0000) 2024-05-17T22:57:36.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:36 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:36.257+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:36.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:36 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:36.257+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:36.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:36 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:36.257+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:44.416729+0000 front 2024-05-17T22:56:44.416947+0000 (oldest deadline 2024-05-17T22:57:07.316815+0000) 2024-05-17T22:57:36.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:36 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:36.257+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:51.318499+0000 front 2024-05-17T22:56:51.318471+0000 (oldest deadline 2024-05-17T22:57:16.618506+0000) 2024-05-17T22:57:36.593 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:36 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:36.257+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.719735+0000 front 2024-05-17T22:57:00.719715+0000 (oldest deadline 2024-05-17T22:57:26.019681+0000) 2024-05-17T22:57:36.593 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:36 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:36.257+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:08.321061+0000 front 2024-05-17T22:57:08.321042+0000 (oldest deadline 2024-05-17T22:57:31.820907+0000) 2024-05-17T22:57:36.842 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:36 smithi151 podman[73871]: 2024-05-17 22:57:36.531203787 +0000 UTC m=+0.663997538 container remove d7128d3bc674fc9bed4c3177ecbf5d4147916c6853bf40b4653dec81e135a724 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-17T22:57:37.128 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:36 smithi151 podman[73960]: 2024-05-17 22:57:36.840628163 +0000 UTC m=+0.110123945 container create fe6755dd75d2ef66001f601cc39a2cb2456d80425754945e89623326d5d52fb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514) 2024-05-17T22:57:37.128 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:36 smithi151 podman[73960]: 2024-05-17 22:57:36.747927443 +0000 UTC m=+0.017423230 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T22:57:37.129 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:37 smithi151 podman[73960]: 2024-05-17 22:57:37.043877704 +0000 UTC m=+0.313373500 container init fe6755dd75d2ef66001f601cc39a2cb2456d80425754945e89623326d5d52fb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T22:57:37.129 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:37 smithi151 podman[73960]: 2024-05-17 22:57:37.047123719 +0000 UTC m=+0.316619513 container start fe6755dd75d2ef66001f601cc39a2cb2456d80425754945e89623326d5d52fb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=) 2024-05-17T22:57:37.129 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:37 smithi151 podman[73960]: 2024-05-17 22:57:37.126947327 +0000 UTC m=+0.396443114 container attach fe6755dd75d2ef66001f601cc39a2cb2456d80425754945e89623326d5d52fb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True) 2024-05-17T22:57:37.298 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:36 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:36.800+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:37.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:36 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:36.800+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:37.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:36 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:36.800+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:37.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:36 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:36.800+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.174543+0000 front 2024-05-17T22:56:52.174682+0000 (oldest deadline 2024-05-17T22:57:15.074590+0000) 2024-05-17T22:57:37.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:36 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:36.800+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.375428+0000 front 2024-05-17T22:57:00.375447+0000 (oldest deadline 2024-05-17T22:57:23.275565+0000) 2024-05-17T22:57:37.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:36 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:36.800+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:08.377691+0000 front 2024-05-17T22:57:08.377608+0000 (oldest deadline 2024-05-17T22:57:31.877496+0000) 2024-05-17T22:57:37.491 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 17 22:57:37 smithi151 podman[73960]: 2024-05-17 22:57:37.339982417 +0000 UTC m=+0.609478204 container died fe6755dd75d2ef66001f601cc39a2cb2456d80425754945e89623326d5d52fb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0) 2024-05-17T22:57:37.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:37 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:37.236+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:37.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:37 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:37.236+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:37.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:37 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:37.236+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:37.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:37 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:37.236+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.932214+0000 front 2024-05-17T22:57:00.932196+0000 (oldest deadline 2024-05-17T22:57:25.032107+0000) 2024-05-17T22:57:37.492 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:37 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:37.236+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:10.333578+0000 front 2024-05-17T22:57:10.333491+0000 (oldest deadline 2024-05-17T22:57:34.433626+0000) 2024-05-17T22:57:37.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:37 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:37.307+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:37.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:37 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:37.307+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:37.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:37 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:37.307+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:44.416729+0000 front 2024-05-17T22:56:44.416947+0000 (oldest deadline 2024-05-17T22:57:07.316815+0000) 2024-05-17T22:57:37.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:37 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:37.307+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:51.318499+0000 front 2024-05-17T22:56:51.318471+0000 (oldest deadline 2024-05-17T22:57:16.618506+0000) 2024-05-17T22:57:37.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:37 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:37.307+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.719735+0000 front 2024-05-17T22:57:00.719715+0000 (oldest deadline 2024-05-17T22:57:26.019681+0000) 2024-05-17T22:57:37.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:37 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:37.307+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:08.321061+0000 front 2024-05-17T22:57:08.321042+0000 (oldest deadline 2024-05-17T22:57:31.820907+0000) 2024-05-17T22:57:38.010 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@osd.8.service' 2024-05-17T22:57:38.048 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T22:57:38.048 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-17T22:57:38.048 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-17T22:57:38.049 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-4d593780-149c-11ef-bc9a-c7b262605968@osd.9 2024-05-17T22:57:38.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:37 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:37.781+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:38.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:37 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:37.781+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:38.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:37 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:37.781+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:38.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:37 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:37.781+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.174543+0000 front 2024-05-17T22:56:52.174682+0000 (oldest deadline 2024-05-17T22:57:15.074590+0000) 2024-05-17T22:57:38.050 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:37 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:37.781+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.375428+0000 front 2024-05-17T22:57:00.375447+0000 (oldest deadline 2024-05-17T22:57:23.275565+0000) 2024-05-17T22:57:38.050 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:37 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:37.781+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:08.377691+0000 front 2024-05-17T22:57:08.377608+0000 (oldest deadline 2024-05-17T22:57:31.877496+0000) 2024-05-17T22:57:38.264 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:38 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:38.222+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:38.265 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:38 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:38.222+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:38.265 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:38 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:38.222+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:38.265 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:38 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:38.222+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.932214+0000 front 2024-05-17T22:57:00.932196+0000 (oldest deadline 2024-05-17T22:57:25.032107+0000) 2024-05-17T22:57:38.265 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:38 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:38.222+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:10.333578+0000 front 2024-05-17T22:57:10.333491+0000 (oldest deadline 2024-05-17T22:57:34.433626+0000) 2024-05-17T22:57:38.341 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:38 smithi012 systemd[1]: Stopping Ceph osd.9 for 4d593780-149c-11ef-bc9a-c7b262605968... 2024-05-17T22:57:38.342 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:38 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:38.316+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:38.342 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:38 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:38.316+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:38.342 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:38 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:38.316+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:44.416729+0000 front 2024-05-17T22:56:44.416947+0000 (oldest deadline 2024-05-17T22:57:07.316815+0000) 2024-05-17T22:57:38.342 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:38 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:38.316+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:51.318499+0000 front 2024-05-17T22:56:51.318471+0000 (oldest deadline 2024-05-17T22:57:16.618506+0000) 2024-05-17T22:57:38.342 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:38 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:38.316+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.719735+0000 front 2024-05-17T22:57:00.719715+0000 (oldest deadline 2024-05-17T22:57:26.019681+0000) 2024-05-17T22:57:38.342 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:38 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:38.316+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:08.321061+0000 front 2024-05-17T22:57:08.321042+0000 (oldest deadline 2024-05-17T22:57:31.820907+0000) 2024-05-17T22:57:38.841 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:38 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:38.386+0000 7f11043bc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T22:57:38.841 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:38 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:38.386+0000 7f11043bc700 -1 osd.9 93 *** Got signal Terminated *** 2024-05-17T22:57:38.842 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:38 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:38.386+0000 7f11043bc700 -1 osd.9 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T22:57:39.073 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:38 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:38.812+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:39.073 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:38 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:38.812+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:39.073 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:38 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:38.812+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:39.073 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:38 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:38.812+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.174543+0000 front 2024-05-17T22:56:52.174682+0000 (oldest deadline 2024-05-17T22:57:15.074590+0000) 2024-05-17T22:57:39.073 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:38 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:38.812+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.375428+0000 front 2024-05-17T22:57:00.375447+0000 (oldest deadline 2024-05-17T22:57:23.275565+0000) 2024-05-17T22:57:39.073 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:38 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:38.812+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:08.377691+0000 front 2024-05-17T22:57:08.377608+0000 (oldest deadline 2024-05-17T22:57:31.877496+0000) 2024-05-17T22:57:39.490 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:39 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:39.258+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:39.490 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:39 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:39.258+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:39.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:39 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:39.258+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:39.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:39 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:39.258+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.932214+0000 front 2024-05-17T22:57:00.932196+0000 (oldest deadline 2024-05-17T22:57:25.032107+0000) 2024-05-17T22:57:39.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:39 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:39.258+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:10.333578+0000 front 2024-05-17T22:57:10.333491+0000 (oldest deadline 2024-05-17T22:57:34.433626+0000) 2024-05-17T22:57:39.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:39 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:39.272+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:39.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:39 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:39.272+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:39.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:39 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:39.272+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:44.416729+0000 front 2024-05-17T22:56:44.416947+0000 (oldest deadline 2024-05-17T22:57:07.316815+0000) 2024-05-17T22:57:39.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:39 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:39.272+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:51.318499+0000 front 2024-05-17T22:56:51.318471+0000 (oldest deadline 2024-05-17T22:57:16.618506+0000) 2024-05-17T22:57:39.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:39 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:39.272+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.719735+0000 front 2024-05-17T22:57:00.719715+0000 (oldest deadline 2024-05-17T22:57:26.019681+0000) 2024-05-17T22:57:39.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:39 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:39.272+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:08.321061+0000 front 2024-05-17T22:57:08.321042+0000 (oldest deadline 2024-05-17T22:57:31.820907+0000) 2024-05-17T22:57:40.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:39 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:39.791+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:40.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:39 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:39.791+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:40.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:39 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:39.791+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:40.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:39 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:39.791+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.174543+0000 front 2024-05-17T22:56:52.174682+0000 (oldest deadline 2024-05-17T22:57:15.074590+0000) 2024-05-17T22:57:40.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:39 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:39.791+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.375428+0000 front 2024-05-17T22:57:00.375447+0000 (oldest deadline 2024-05-17T22:57:23.275565+0000) 2024-05-17T22:57:40.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:39 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:39.791+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:08.377691+0000 front 2024-05-17T22:57:08.377608+0000 (oldest deadline 2024-05-17T22:57:31.877496+0000) 2024-05-17T22:57:40.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:40 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:40.288+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:40.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:40 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:40.288+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:40.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:40 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:40.288+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:44.416729+0000 front 2024-05-17T22:56:44.416947+0000 (oldest deadline 2024-05-17T22:57:07.316815+0000) 2024-05-17T22:57:40.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:40 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:40.288+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:51.318499+0000 front 2024-05-17T22:56:51.318471+0000 (oldest deadline 2024-05-17T22:57:16.618506+0000) 2024-05-17T22:57:40.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:40 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:40.288+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.719735+0000 front 2024-05-17T22:57:00.719715+0000 (oldest deadline 2024-05-17T22:57:26.019681+0000) 2024-05-17T22:57:40.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:40 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:40.288+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:08.321061+0000 front 2024-05-17T22:57:08.321042+0000 (oldest deadline 2024-05-17T22:57:31.820907+0000) 2024-05-17T22:57:40.740 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:40 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:40.258+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:40.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:40 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:40.258+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:40.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:40 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:40.258+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:40.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:40 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:40.258+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.932214+0000 front 2024-05-17T22:57:00.932196+0000 (oldest deadline 2024-05-17T22:57:25.032107+0000) 2024-05-17T22:57:40.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:40 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:40.258+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:10.333578+0000 front 2024-05-17T22:57:10.333491+0000 (oldest deadline 2024-05-17T22:57:34.433626+0000) 2024-05-17T22:57:41.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:40 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:40.755+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:41.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:40 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:40.755+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:41.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:40 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:40.755+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:41.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:40 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:40.755+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.174543+0000 front 2024-05-17T22:56:52.174682+0000 (oldest deadline 2024-05-17T22:57:15.074590+0000) 2024-05-17T22:57:41.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:40 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:40.755+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.375428+0000 front 2024-05-17T22:57:00.375447+0000 (oldest deadline 2024-05-17T22:57:23.275565+0000) 2024-05-17T22:57:41.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:40 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:40.755+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:08.377691+0000 front 2024-05-17T22:57:08.377608+0000 (oldest deadline 2024-05-17T22:57:31.877496+0000) 2024-05-17T22:57:41.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:40 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:40.755+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-17T22:57:14.679555+0000 front 2024-05-17T22:57:14.679578+0000 (oldest deadline 2024-05-17T22:57:39.979308+0000) 2024-05-17T22:57:41.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:41 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:41.292+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:41.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:41 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:41.292+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:41.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:41 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:41.292+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:44.416729+0000 front 2024-05-17T22:56:44.416947+0000 (oldest deadline 2024-05-17T22:57:07.316815+0000) 2024-05-17T22:57:41.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:41 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:41.292+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:51.318499+0000 front 2024-05-17T22:56:51.318471+0000 (oldest deadline 2024-05-17T22:57:16.618506+0000) 2024-05-17T22:57:41.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:41 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:41.292+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.719735+0000 front 2024-05-17T22:57:00.719715+0000 (oldest deadline 2024-05-17T22:57:26.019681+0000) 2024-05-17T22:57:41.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:41 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:41.292+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:08.321061+0000 front 2024-05-17T22:57:08.321042+0000 (oldest deadline 2024-05-17T22:57:31.820907+0000) 2024-05-17T22:57:41.740 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:41 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:41.246+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:41.740 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:41 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:41.246+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:41.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:41 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:41.246+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:41.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:41 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:41.246+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.932214+0000 front 2024-05-17T22:57:00.932196+0000 (oldest deadline 2024-05-17T22:57:25.032107+0000) 2024-05-17T22:57:41.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:41 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:41.246+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:10.333578+0000 front 2024-05-17T22:57:10.333491+0000 (oldest deadline 2024-05-17T22:57:34.433626+0000) 2024-05-17T22:57:42.048 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:41 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:41.739+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:42.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:41 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:41.739+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:42.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:41 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:41.739+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:42.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:41 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:41.739+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.174543+0000 front 2024-05-17T22:56:52.174682+0000 (oldest deadline 2024-05-17T22:57:15.074590+0000) 2024-05-17T22:57:42.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:41 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:41.739+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.375428+0000 front 2024-05-17T22:57:00.375447+0000 (oldest deadline 2024-05-17T22:57:23.275565+0000) 2024-05-17T22:57:42.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:41 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:41.739+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:08.377691+0000 front 2024-05-17T22:57:08.377608+0000 (oldest deadline 2024-05-17T22:57:31.877496+0000) 2024-05-17T22:57:42.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:41 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:41.739+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-17T22:57:14.679555+0000 front 2024-05-17T22:57:14.679578+0000 (oldest deadline 2024-05-17T22:57:39.979308+0000) 2024-05-17T22:57:42.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:42 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:42.264+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:42.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:42 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:42.264+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:42.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:42 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:42.264+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:44.416729+0000 front 2024-05-17T22:56:44.416947+0000 (oldest deadline 2024-05-17T22:57:07.316815+0000) 2024-05-17T22:57:42.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:42 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:42.264+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:51.318499+0000 front 2024-05-17T22:56:51.318471+0000 (oldest deadline 2024-05-17T22:57:16.618506+0000) 2024-05-17T22:57:42.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:42 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:42.264+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.719735+0000 front 2024-05-17T22:57:00.719715+0000 (oldest deadline 2024-05-17T22:57:26.019681+0000) 2024-05-17T22:57:42.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:42 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:42.264+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:08.321061+0000 front 2024-05-17T22:57:08.321042+0000 (oldest deadline 2024-05-17T22:57:31.820907+0000) 2024-05-17T22:57:42.740 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:42 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:42.278+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:42.740 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:42 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:42.278+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:42.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:42 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:42.278+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:42.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:42 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:42.278+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.932214+0000 front 2024-05-17T22:57:00.932196+0000 (oldest deadline 2024-05-17T22:57:25.032107+0000) 2024-05-17T22:57:42.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:42 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:42.278+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:10.333578+0000 front 2024-05-17T22:57:10.333491+0000 (oldest deadline 2024-05-17T22:57:34.433626+0000) 2024-05-17T22:57:43.048 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:42 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:42.703+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:43.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:42 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:42.703+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:43.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:42 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:42.703+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:43.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:42 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:42.703+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.174543+0000 front 2024-05-17T22:56:52.174682+0000 (oldest deadline 2024-05-17T22:57:15.074590+0000) 2024-05-17T22:57:43.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:42 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:42.703+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.375428+0000 front 2024-05-17T22:57:00.375447+0000 (oldest deadline 2024-05-17T22:57:23.275565+0000) 2024-05-17T22:57:43.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:42 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:42.703+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:08.377691+0000 front 2024-05-17T22:57:08.377608+0000 (oldest deadline 2024-05-17T22:57:31.877496+0000) 2024-05-17T22:57:43.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:42 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:42.703+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-17T22:57:14.679555+0000 front 2024-05-17T22:57:14.679578+0000 (oldest deadline 2024-05-17T22:57:39.979308+0000) 2024-05-17T22:57:43.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:43 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:43.276+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:23.913841+0000 front 2024-05-17T22:56:23.913817+0000 (oldest deadline 2024-05-17T22:56:49.814016+0000) 2024-05-17T22:57:43.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:43 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:43.276+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:32.615730+0000 front 2024-05-17T22:56:32.615751+0000 (oldest deadline 2024-05-17T22:56:58.515748+0000) 2024-05-17T22:57:43.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:43 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:43.276+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:44.416729+0000 front 2024-05-17T22:56:44.416947+0000 (oldest deadline 2024-05-17T22:57:07.316815+0000) 2024-05-17T22:57:43.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:43 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:43.276+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:51.318499+0000 front 2024-05-17T22:56:51.318471+0000 (oldest deadline 2024-05-17T22:57:16.618506+0000) 2024-05-17T22:57:43.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:43 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:43.276+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.719735+0000 front 2024-05-17T22:57:00.719715+0000 (oldest deadline 2024-05-17T22:57:26.019681+0000) 2024-05-17T22:57:43.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:43 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9[66260]: 2024-05-17T22:57:43.276+0000 7f1100972700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:08.321061+0000 front 2024-05-17T22:57:08.321042+0000 (oldest deadline 2024-05-17T22:57:31.820907+0000) 2024-05-17T22:57:43.740 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:43 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:43.267+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:43.740 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:43 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:43.267+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:43.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:43 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:43.267+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:43.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:43 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:43.267+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.932214+0000 front 2024-05-17T22:57:00.932196+0000 (oldest deadline 2024-05-17T22:57:25.032107+0000) 2024-05-17T22:57:43.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:43 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:43.267+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:10.333578+0000 front 2024-05-17T22:57:10.333491+0000 (oldest deadline 2024-05-17T22:57:34.433626+0000) 2024-05-17T22:57:44.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:43 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:43.731+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:44.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:43 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:43.731+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:44.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:43 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:43.731+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:44.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:43 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:43.731+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.174543+0000 front 2024-05-17T22:56:52.174682+0000 (oldest deadline 2024-05-17T22:57:15.074590+0000) 2024-05-17T22:57:44.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:43 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:43.731+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.375428+0000 front 2024-05-17T22:57:00.375447+0000 (oldest deadline 2024-05-17T22:57:23.275565+0000) 2024-05-17T22:57:44.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:43 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:43.731+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:08.377691+0000 front 2024-05-17T22:57:08.377608+0000 (oldest deadline 2024-05-17T22:57:31.877496+0000) 2024-05-17T22:57:44.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:43 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:43.731+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-17T22:57:14.679555+0000 front 2024-05-17T22:57:14.679578+0000 (oldest deadline 2024-05-17T22:57:39.979308+0000) 2024-05-17T22:57:44.313 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:44 smithi012 podman[115732]: 2024-05-17 22:57:44.012987267 +0000 UTC m=+5.752952249 container died 0926f45ab2b08b4e024b41ee69b70f768af3dee639000be328ce71263aa5053b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T22:57:44.490 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:44 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:44.231+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:44.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:44 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:44.231+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:44.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:44 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:44.231+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:44.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:44 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:44.231+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.932214+0000 front 2024-05-17T22:57:00.932196+0000 (oldest deadline 2024-05-17T22:57:25.032107+0000) 2024-05-17T22:57:44.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:44 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:44.231+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:10.333578+0000 front 2024-05-17T22:57:10.333491+0000 (oldest deadline 2024-05-17T22:57:34.433626+0000) 2024-05-17T22:57:44.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:44 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:44.231+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-17T22:57:14.434307+0000 front 2024-05-17T22:57:14.434341+0000 (oldest deadline 2024-05-17T22:57:43.834549+0000) 2024-05-17T22:57:44.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:44 smithi012 podman[115732]: 2024-05-17 22:57:44.492261876 +0000 UTC m=+6.232226889 container cleanup 0926f45ab2b08b4e024b41ee69b70f768af3dee639000be328ce71263aa5053b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-17T22:57:44.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:44 smithi012 bash[115732]: ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9 2024-05-17T22:57:45.048 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:44 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:44.767+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:45.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:44 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:44.767+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:45.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:44 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:44.767+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:45.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:44 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:44.767+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.174543+0000 front 2024-05-17T22:56:52.174682+0000 (oldest deadline 2024-05-17T22:57:15.074590+0000) 2024-05-17T22:57:45.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:44 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:44.767+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.375428+0000 front 2024-05-17T22:57:00.375447+0000 (oldest deadline 2024-05-17T22:57:23.275565+0000) 2024-05-17T22:57:45.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:44 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:44.767+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:08.377691+0000 front 2024-05-17T22:57:08.377608+0000 (oldest deadline 2024-05-17T22:57:31.877496+0000) 2024-05-17T22:57:45.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:44 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:44.767+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-17T22:57:14.679555+0000 front 2024-05-17T22:57:14.679578+0000 (oldest deadline 2024-05-17T22:57:39.979308+0000) 2024-05-17T22:57:45.277 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:44 smithi012 podman[115750]: 2024-05-17 22:57:44.927168434 +0000 UTC m=+0.904571128 container remove 0926f45ab2b08b4e024b41ee69b70f768af3dee639000be328ce71263aa5053b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514) 2024-05-17T22:57:45.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:45 smithi012 podman[115836]: 2024-05-17 22:57:45.17870399 +0000 UTC m=+0.018141694 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T22:57:45.591 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:45 smithi012 podman[115836]: 2024-05-17 22:57:45.311251918 +0000 UTC m=+0.150689630 container create e4523ffd0cc1ff4627e5b49fdcf938109d928c82c8cac50295d32e7805431a68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-17T22:57:45.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:45 smithi012 podman[115836]: 2024-05-17 22:57:45.522452844 +0000 UTC m=+0.361890556 container init e4523ffd0cc1ff4627e5b49fdcf938109d928c82c8cac50295d32e7805431a68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-17T22:57:45.592 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:45 smithi012 podman[115836]: 2024-05-17 22:57:45.525966434 +0000 UTC m=+0.365404137 container start e4523ffd0cc1ff4627e5b49fdcf938109d928c82c8cac50295d32e7805431a68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-17T22:57:45.740 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:45 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:45.247+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:45.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:45 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:45.247+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:45.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:45 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:45.247+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:45.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:45 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:45.247+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.932214+0000 front 2024-05-17T22:57:00.932196+0000 (oldest deadline 2024-05-17T22:57:25.032107+0000) 2024-05-17T22:57:45.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:45 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:45.247+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:10.333578+0000 front 2024-05-17T22:57:10.333491+0000 (oldest deadline 2024-05-17T22:57:34.433626+0000) 2024-05-17T22:57:45.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:45 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:45.247+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-17T22:57:14.434307+0000 front 2024-05-17T22:57:14.434341+0000 (oldest deadline 2024-05-17T22:57:43.834549+0000) 2024-05-17T22:57:46.091 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:45 smithi012 podman[115836]: 2024-05-17 22:57:45.613751589 +0000 UTC m=+0.453189306 container attach e4523ffd0cc1ff4627e5b49fdcf938109d928c82c8cac50295d32e7805431a68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T22:57:46.092 INFO:journalctl@ceph.osd.9.smithi012.stdout:May 17 22:57:45 smithi012 podman[115836]: 2024-05-17 22:57:45.831287316 +0000 UTC m=+0.670725025 container died e4523ffd0cc1ff4627e5b49fdcf938109d928c82c8cac50295d32e7805431a68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD) 2024-05-17T22:57:46.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:45 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:45.801+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:46.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:45 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:45.801+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:46.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:45 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:45.801+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:46.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:45 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:45.801+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.174543+0000 front 2024-05-17T22:56:52.174682+0000 (oldest deadline 2024-05-17T22:57:15.074590+0000) 2024-05-17T22:57:46.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:45 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:45.801+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.375428+0000 front 2024-05-17T22:57:00.375447+0000 (oldest deadline 2024-05-17T22:57:23.275565+0000) 2024-05-17T22:57:46.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:45 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:45.801+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:08.377691+0000 front 2024-05-17T22:57:08.377608+0000 (oldest deadline 2024-05-17T22:57:31.877496+0000) 2024-05-17T22:57:46.299 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:45 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:45.801+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-17T22:57:14.679555+0000 front 2024-05-17T22:57:14.679578+0000 (oldest deadline 2024-05-17T22:57:39.979308+0000) 2024-05-17T22:57:46.418 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@osd.9.service' 2024-05-17T22:57:46.461 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T22:57:46.461 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-17T22:57:46.461 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-17T22:57:46.461 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-4d593780-149c-11ef-bc9a-c7b262605968@osd.10 2024-05-17T22:57:46.740 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:46 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:46.247+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:46.740 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:46 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:46.247+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:46.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:46 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:46.248+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:46.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:46 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:46.248+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.932214+0000 front 2024-05-17T22:57:00.932196+0000 (oldest deadline 2024-05-17T22:57:25.032107+0000) 2024-05-17T22:57:46.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:46 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:46.248+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:10.333578+0000 front 2024-05-17T22:57:10.333491+0000 (oldest deadline 2024-05-17T22:57:34.433626+0000) 2024-05-17T22:57:46.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:46 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:46.248+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-17T22:57:14.434307+0000 front 2024-05-17T22:57:14.434341+0000 (oldest deadline 2024-05-17T22:57:43.834549+0000) 2024-05-17T22:57:46.760 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:46 smithi096 systemd[1]: Stopping Ceph osd.10 for 4d593780-149c-11ef-bc9a-c7b262605968... 2024-05-17T22:57:47.048 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:46 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:46.757+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:47.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:46 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:46.757+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:47.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:46 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:46.757+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:47.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:46 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:46.757+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.174543+0000 front 2024-05-17T22:56:52.174682+0000 (oldest deadline 2024-05-17T22:57:15.074590+0000) 2024-05-17T22:57:47.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:46 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:46.757+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.375428+0000 front 2024-05-17T22:57:00.375447+0000 (oldest deadline 2024-05-17T22:57:23.275565+0000) 2024-05-17T22:57:47.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:46 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:46.757+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:08.377691+0000 front 2024-05-17T22:57:08.377608+0000 (oldest deadline 2024-05-17T22:57:31.877496+0000) 2024-05-17T22:57:47.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:46 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:46.757+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-17T22:57:14.679555+0000 front 2024-05-17T22:57:14.679578+0000 (oldest deadline 2024-05-17T22:57:39.979308+0000) 2024-05-17T22:57:47.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:46 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:46.761+0000 7f16d3113700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T22:57:47.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:46 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:46.761+0000 7f16d3113700 -1 osd.10 93 *** Got signal Terminated *** 2024-05-17T22:57:47.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:46 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:46.761+0000 7f16d3113700 -1 osd.10 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T22:57:47.740 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:47 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:47.257+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:47.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:47 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:47.257+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:47.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:47 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:47.257+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:47.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:47 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:47.257+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.932214+0000 front 2024-05-17T22:57:00.932196+0000 (oldest deadline 2024-05-17T22:57:25.032107+0000) 2024-05-17T22:57:47.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:47 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:47.257+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:10.333578+0000 front 2024-05-17T22:57:10.333491+0000 (oldest deadline 2024-05-17T22:57:34.433626+0000) 2024-05-17T22:57:47.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:47 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:47.257+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-17T22:57:14.434307+0000 front 2024-05-17T22:57:14.434341+0000 (oldest deadline 2024-05-17T22:57:43.834549+0000) 2024-05-17T22:57:48.048 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:47 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:47.718+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:48.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:47 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:47.718+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:48.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:47 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:47.718+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:48.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:47 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:47.718+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.174543+0000 front 2024-05-17T22:56:52.174682+0000 (oldest deadline 2024-05-17T22:57:15.074590+0000) 2024-05-17T22:57:48.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:47 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:47.718+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.375428+0000 front 2024-05-17T22:57:00.375447+0000 (oldest deadline 2024-05-17T22:57:23.275565+0000) 2024-05-17T22:57:48.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:47 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:47.718+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:08.377691+0000 front 2024-05-17T22:57:08.377608+0000 (oldest deadline 2024-05-17T22:57:31.877496+0000) 2024-05-17T22:57:48.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:47 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:47.718+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-17T22:57:14.679555+0000 front 2024-05-17T22:57:14.679578+0000 (oldest deadline 2024-05-17T22:57:39.979308+0000) 2024-05-17T22:57:48.490 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:48 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:48.226+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:48.490 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:48 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:48.226+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:48.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:48 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:48.226+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:48.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:48 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:48.226+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.932214+0000 front 2024-05-17T22:57:00.932196+0000 (oldest deadline 2024-05-17T22:57:25.032107+0000) 2024-05-17T22:57:48.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:48 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:48.226+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:10.333578+0000 front 2024-05-17T22:57:10.333491+0000 (oldest deadline 2024-05-17T22:57:34.433626+0000) 2024-05-17T22:57:48.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:48 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:48.226+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-17T22:57:14.434307+0000 front 2024-05-17T22:57:14.434341+0000 (oldest deadline 2024-05-17T22:57:43.834549+0000) 2024-05-17T22:57:48.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:48 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:48.226+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-05-17T22:57:24.335567+0000 front 2024-05-17T22:57:24.335536+0000 (oldest deadline 2024-05-17T22:57:47.835581+0000) 2024-05-17T22:57:49.048 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:48 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:48.730+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:49.048 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:48 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:48.730+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:49.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:48 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:48.730+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:49.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:48 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:48.730+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.174543+0000 front 2024-05-17T22:56:52.174682+0000 (oldest deadline 2024-05-17T22:57:15.074590+0000) 2024-05-17T22:57:49.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:48 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:48.730+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.375428+0000 front 2024-05-17T22:57:00.375447+0000 (oldest deadline 2024-05-17T22:57:23.275565+0000) 2024-05-17T22:57:49.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:48 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:48.730+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:08.377691+0000 front 2024-05-17T22:57:08.377608+0000 (oldest deadline 2024-05-17T22:57:31.877496+0000) 2024-05-17T22:57:49.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:48 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:48.730+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-17T22:57:14.679555+0000 front 2024-05-17T22:57:14.679578+0000 (oldest deadline 2024-05-17T22:57:39.979308+0000) 2024-05-17T22:57:49.740 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:49 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:49.250+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:49.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:49 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:49.250+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:49.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:49 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:49.250+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:49.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:49 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:49.250+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.932214+0000 front 2024-05-17T22:57:00.932196+0000 (oldest deadline 2024-05-17T22:57:25.032107+0000) 2024-05-17T22:57:49.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:49 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:49.250+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:10.333578+0000 front 2024-05-17T22:57:10.333491+0000 (oldest deadline 2024-05-17T22:57:34.433626+0000) 2024-05-17T22:57:49.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:49 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:49.250+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-17T22:57:14.434307+0000 front 2024-05-17T22:57:14.434341+0000 (oldest deadline 2024-05-17T22:57:43.834549+0000) 2024-05-17T22:57:49.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:49 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:49.250+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-05-17T22:57:24.335567+0000 front 2024-05-17T22:57:24.335536+0000 (oldest deadline 2024-05-17T22:57:47.835581+0000) 2024-05-17T22:57:50.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:49 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:49.738+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:50.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:49 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:49.738+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:50.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:49 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:49.738+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:50.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:49 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:49.738+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.174543+0000 front 2024-05-17T22:56:52.174682+0000 (oldest deadline 2024-05-17T22:57:15.074590+0000) 2024-05-17T22:57:50.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:49 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:49.738+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.375428+0000 front 2024-05-17T22:57:00.375447+0000 (oldest deadline 2024-05-17T22:57:23.275565+0000) 2024-05-17T22:57:50.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:49 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:49.738+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:08.377691+0000 front 2024-05-17T22:57:08.377608+0000 (oldest deadline 2024-05-17T22:57:31.877496+0000) 2024-05-17T22:57:50.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:49 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:49.738+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-17T22:57:14.679555+0000 front 2024-05-17T22:57:14.679578+0000 (oldest deadline 2024-05-17T22:57:39.979308+0000) 2024-05-17T22:57:50.740 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:50 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:50.250+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:50.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:50 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:50.250+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:50.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:50 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:50.250+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:50.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:50 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:50.250+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.932214+0000 front 2024-05-17T22:57:00.932196+0000 (oldest deadline 2024-05-17T22:57:25.032107+0000) 2024-05-17T22:57:50.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:50 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:50.250+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:10.333578+0000 front 2024-05-17T22:57:10.333491+0000 (oldest deadline 2024-05-17T22:57:34.433626+0000) 2024-05-17T22:57:50.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:50 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:50.250+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-17T22:57:14.434307+0000 front 2024-05-17T22:57:14.434341+0000 (oldest deadline 2024-05-17T22:57:43.834549+0000) 2024-05-17T22:57:50.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:50 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:50.250+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-05-17T22:57:24.335567+0000 front 2024-05-17T22:57:24.335536+0000 (oldest deadline 2024-05-17T22:57:47.835581+0000) 2024-05-17T22:57:51.048 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:50 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:50.770+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:24.670788+0000 front 2024-05-17T22:56:24.670763+0000 (oldest deadline 2024-05-17T22:56:49.970749+0000) 2024-05-17T22:57:51.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:50 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:50.770+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.271792+0000 front 2024-05-17T22:56:35.271813+0000 (oldest deadline 2024-05-17T22:56:59.971964+0000) 2024-05-17T22:57:51.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:50 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:50.770+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:41.672988+0000 front 2024-05-17T22:56:41.673013+0000 (oldest deadline 2024-05-17T22:57:05.772900+0000) 2024-05-17T22:57:51.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:50 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:50.770+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-05-17T22:56:52.174543+0000 front 2024-05-17T22:56:52.174682+0000 (oldest deadline 2024-05-17T22:57:15.074590+0000) 2024-05-17T22:57:51.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:50 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:50.770+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.375428+0000 front 2024-05-17T22:57:00.375447+0000 (oldest deadline 2024-05-17T22:57:23.275565+0000) 2024-05-17T22:57:51.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:50 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:50.770+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:08.377691+0000 front 2024-05-17T22:57:08.377608+0000 (oldest deadline 2024-05-17T22:57:31.877496+0000) 2024-05-17T22:57:51.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:50 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10[60999]: 2024-05-17T22:57:50.770+0000 7f16cf6c9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-17T22:57:14.679555+0000 front 2024-05-17T22:57:14.679578+0000 (oldest deadline 2024-05-17T22:57:39.979308+0000) 2024-05-17T22:57:51.490 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:51 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:51.237+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:51.490 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:51 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:51.237+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:51.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:51 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:51.237+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:51.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:51 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:51.237+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.932214+0000 front 2024-05-17T22:57:00.932196+0000 (oldest deadline 2024-05-17T22:57:25.032107+0000) 2024-05-17T22:57:51.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:51 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:51.237+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:10.333578+0000 front 2024-05-17T22:57:10.333491+0000 (oldest deadline 2024-05-17T22:57:34.433626+0000) 2024-05-17T22:57:51.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:51 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:51.237+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-17T22:57:14.434307+0000 front 2024-05-17T22:57:14.434341+0000 (oldest deadline 2024-05-17T22:57:43.834549+0000) 2024-05-17T22:57:51.491 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:51 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:51.237+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-05-17T22:57:24.335567+0000 front 2024-05-17T22:57:24.335536+0000 (oldest deadline 2024-05-17T22:57:47.835581+0000) 2024-05-17T22:57:52.680 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:52 smithi096 podman[72933]: 2024-05-17 22:57:52.341744145 +0000 UTC m=+5.694163779 container died 6943c096ae4713596fcd8a0c80feddf0116444a71e3b60234aa9234304bcbb8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-17T22:57:52.740 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:52 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:52.272+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:52.740 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:52 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:52.272+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:52.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:52 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:52.272+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:52.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:52 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:52.272+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.932214+0000 front 2024-05-17T22:57:00.932196+0000 (oldest deadline 2024-05-17T22:57:25.032107+0000) 2024-05-17T22:57:52.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:52 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:52.272+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:10.333578+0000 front 2024-05-17T22:57:10.333491+0000 (oldest deadline 2024-05-17T22:57:34.433626+0000) 2024-05-17T22:57:52.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:52 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:52.272+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-17T22:57:14.434307+0000 front 2024-05-17T22:57:14.434341+0000 (oldest deadline 2024-05-17T22:57:43.834549+0000) 2024-05-17T22:57:52.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:52 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:52.272+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-05-17T22:57:24.335567+0000 front 2024-05-17T22:57:24.335536+0000 (oldest deadline 2024-05-17T22:57:47.835581+0000) 2024-05-17T22:57:53.048 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:52 smithi096 podman[72933]: 2024-05-17 22:57:52.678399564 +0000 UTC m=+6.030819198 container cleanup 6943c096ae4713596fcd8a0c80feddf0116444a71e3b60234aa9234304bcbb8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-17T22:57:53.048 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:52 smithi096 bash[72933]: ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10 2024-05-17T22:57:53.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:52 smithi096 podman[72950]: 2024-05-17 22:57:52.855039678 +0000 UTC m=+0.511622033 container remove 6943c096ae4713596fcd8a0c80feddf0116444a71e3b60234aa9234304bcbb8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-17T22:57:53.548 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:53 smithi096 podman[73041]: 2024-05-17 22:57:53.195337401 +0000 UTC m=+0.017470992 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T22:57:53.549 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:53 smithi096 podman[73041]: 2024-05-17 22:57:53.297483791 +0000 UTC m=+0.119617437 container create 850920801de8adcd6f06c246a2eb70943815675fc245beb7378673f100693673 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-17T22:57:53.549 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:53 smithi096 podman[73041]: 2024-05-17 22:57:53.491322579 +0000 UTC m=+0.313456163 container init 850920801de8adcd6f06c246a2eb70943815675fc245beb7378673f100693673 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10-deactivate, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-05-17T22:57:53.549 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:53 smithi096 podman[73041]: 2024-05-17 22:57:53.494289387 +0000 UTC m=+0.316422978 container start 850920801de8adcd6f06c246a2eb70943815675fc245beb7378673f100693673 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True) 2024-05-17T22:57:53.740 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:53 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:53.309+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:53.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:53 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:53.309+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:53.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:53 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:53.309+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:53.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:53 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:53.309+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.932214+0000 front 2024-05-17T22:57:00.932196+0000 (oldest deadline 2024-05-17T22:57:25.032107+0000) 2024-05-17T22:57:53.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:53 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:53.309+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:10.333578+0000 front 2024-05-17T22:57:10.333491+0000 (oldest deadline 2024-05-17T22:57:34.433626+0000) 2024-05-17T22:57:53.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:53 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:53.309+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-17T22:57:14.434307+0000 front 2024-05-17T22:57:14.434341+0000 (oldest deadline 2024-05-17T22:57:43.834549+0000) 2024-05-17T22:57:53.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:53 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:53.309+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-05-17T22:57:24.335567+0000 front 2024-05-17T22:57:24.335536+0000 (oldest deadline 2024-05-17T22:57:47.835581+0000) 2024-05-17T22:57:54.048 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:53 smithi096 podman[73041]: 2024-05-17 22:57:53.577430968 +0000 UTC m=+0.399564580 container attach 850920801de8adcd6f06c246a2eb70943815675fc245beb7378673f100693673 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux ) 2024-05-17T22:57:54.049 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 17 22:57:53 smithi096 podman[73041]: 2024-05-17 22:57:53.784056164 +0000 UTC m=+0.606189759 container died 850920801de8adcd6f06c246a2eb70943815675fc245beb7378673f100693673 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240514, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T22:57:54.389 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@osd.10.service' 2024-05-17T22:57:54.426 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T22:57:54.426 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-17T22:57:54.426 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-17T22:57:54.426 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-4d593780-149c-11ef-bc9a-c7b262605968@osd.11 2024-05-17T22:57:54.740 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:54 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:54.308+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:54.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:54 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:54.308+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:54.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:54 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:54.308+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:54.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:54 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:54.308+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.932214+0000 front 2024-05-17T22:57:00.932196+0000 (oldest deadline 2024-05-17T22:57:25.032107+0000) 2024-05-17T22:57:54.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:54 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:54.308+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:10.333578+0000 front 2024-05-17T22:57:10.333491+0000 (oldest deadline 2024-05-17T22:57:34.433626+0000) 2024-05-17T22:57:54.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:54 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:54.308+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-17T22:57:14.434307+0000 front 2024-05-17T22:57:14.434341+0000 (oldest deadline 2024-05-17T22:57:43.834549+0000) 2024-05-17T22:57:54.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:54 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:54.308+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-05-17T22:57:24.335567+0000 front 2024-05-17T22:57:24.335536+0000 (oldest deadline 2024-05-17T22:57:47.835581+0000) 2024-05-17T22:57:54.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:54 smithi151 systemd[1]: Stopping Ceph osd.11 for 4d593780-149c-11ef-bc9a-c7b262605968... 2024-05-17T22:57:55.240 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:54 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:54.752+0000 7fab25eab700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T22:57:55.240 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:54 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:54.752+0000 7fab25eab700 -1 osd.11 93 *** Got signal Terminated *** 2024-05-17T22:57:55.240 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:54 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:54.752+0000 7fab25eab700 -1 osd.11 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T22:57:55.740 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:55 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:55.262+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:55.740 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:55 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:55.262+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:55.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:55 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:55.262+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:55.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:55 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:55.262+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.932214+0000 front 2024-05-17T22:57:00.932196+0000 (oldest deadline 2024-05-17T22:57:25.032107+0000) 2024-05-17T22:57:55.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:55 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:55.262+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:10.333578+0000 front 2024-05-17T22:57:10.333491+0000 (oldest deadline 2024-05-17T22:57:34.433626+0000) 2024-05-17T22:57:55.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:55 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:55.262+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-17T22:57:14.434307+0000 front 2024-05-17T22:57:14.434341+0000 (oldest deadline 2024-05-17T22:57:43.834549+0000) 2024-05-17T22:57:55.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:55 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:55.262+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-05-17T22:57:24.335567+0000 front 2024-05-17T22:57:24.335536+0000 (oldest deadline 2024-05-17T22:57:47.835581+0000) 2024-05-17T22:57:56.740 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:56 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:56.276+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:56.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:56 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:56.276+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:56.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:56 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:56.276+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:56.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:56 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:56.276+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.932214+0000 front 2024-05-17T22:57:00.932196+0000 (oldest deadline 2024-05-17T22:57:25.032107+0000) 2024-05-17T22:57:56.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:56 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:56.276+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:10.333578+0000 front 2024-05-17T22:57:10.333491+0000 (oldest deadline 2024-05-17T22:57:34.433626+0000) 2024-05-17T22:57:56.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:56 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:56.276+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-17T22:57:14.434307+0000 front 2024-05-17T22:57:14.434341+0000 (oldest deadline 2024-05-17T22:57:43.834549+0000) 2024-05-17T22:57:56.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:56 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:56.276+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-05-17T22:57:24.335567+0000 front 2024-05-17T22:57:24.335536+0000 (oldest deadline 2024-05-17T22:57:47.835581+0000) 2024-05-17T22:57:57.740 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:57 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:57.247+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:57.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:57 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:57.247+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:57.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:57 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:57.247+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:57.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:57 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:57.247+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.932214+0000 front 2024-05-17T22:57:00.932196+0000 (oldest deadline 2024-05-17T22:57:25.032107+0000) 2024-05-17T22:57:57.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:57 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:57.247+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:10.333578+0000 front 2024-05-17T22:57:10.333491+0000 (oldest deadline 2024-05-17T22:57:34.433626+0000) 2024-05-17T22:57:57.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:57 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:57.247+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-17T22:57:14.434307+0000 front 2024-05-17T22:57:14.434341+0000 (oldest deadline 2024-05-17T22:57:43.834549+0000) 2024-05-17T22:57:57.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:57 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:57.247+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-05-17T22:57:24.335567+0000 front 2024-05-17T22:57:24.335536+0000 (oldest deadline 2024-05-17T22:57:47.835581+0000) 2024-05-17T22:57:57.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:57 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:57.247+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6820 osd.8 since back 2024-05-17T22:57:32.437323+0000 front 2024-05-17T22:57:32.437281+0000 (oldest deadline 2024-05-17T22:57:57.137263+0000) 2024-05-17T22:57:58.740 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:58 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:58.258+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:58.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:58 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:58.258+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:58.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:58 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:58.258+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:58.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:58 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:58.258+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.932214+0000 front 2024-05-17T22:57:00.932196+0000 (oldest deadline 2024-05-17T22:57:25.032107+0000) 2024-05-17T22:57:58.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:58 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:58.258+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:10.333578+0000 front 2024-05-17T22:57:10.333491+0000 (oldest deadline 2024-05-17T22:57:34.433626+0000) 2024-05-17T22:57:58.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:58 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:58.258+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-17T22:57:14.434307+0000 front 2024-05-17T22:57:14.434341+0000 (oldest deadline 2024-05-17T22:57:43.834549+0000) 2024-05-17T22:57:58.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:58 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:58.258+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-05-17T22:57:24.335567+0000 front 2024-05-17T22:57:24.335536+0000 (oldest deadline 2024-05-17T22:57:47.835581+0000) 2024-05-17T22:57:58.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:58 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:58.258+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6820 osd.8 since back 2024-05-17T22:57:32.437323+0000 front 2024-05-17T22:57:32.437281+0000 (oldest deadline 2024-05-17T22:57:57.137263+0000) 2024-05-17T22:57:59.740 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:59 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:59.239+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-05-17T22:56:27.924155+0000 front 2024-05-17T22:56:27.924275+0000 (oldest deadline 2024-05-17T22:56:52.024171+0000) 2024-05-17T22:57:59.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:59 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:59.239+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-17T22:56:35.525566+0000 front 2024-05-17T22:56:35.525515+0000 (oldest deadline 2024-05-17T22:56:59.025552+0000) 2024-05-17T22:57:59.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:59 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:59.239+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-17T22:56:43.126979+0000 front 2024-05-17T22:56:43.126803+0000 (oldest deadline 2024-05-17T22:57:07.226968+0000) 2024-05-17T22:57:59.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:59 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:59.239+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-17T22:57:00.932214+0000 front 2024-05-17T22:57:00.932196+0000 (oldest deadline 2024-05-17T22:57:25.032107+0000) 2024-05-17T22:57:59.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:59 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:59.239+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-17T22:57:10.333578+0000 front 2024-05-17T22:57:10.333491+0000 (oldest deadline 2024-05-17T22:57:34.433626+0000) 2024-05-17T22:57:59.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:59 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:59.239+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-05-17T22:57:14.434307+0000 front 2024-05-17T22:57:14.434341+0000 (oldest deadline 2024-05-17T22:57:43.834549+0000) 2024-05-17T22:57:59.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:59 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:59.239+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-05-17T22:57:24.335567+0000 front 2024-05-17T22:57:24.335536+0000 (oldest deadline 2024-05-17T22:57:47.835581+0000) 2024-05-17T22:57:59.741 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:57:59 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11[61727]: 2024-05-17T22:57:59.239+0000 7fab1e459700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.151:6820 osd.8 since back 2024-05-17T22:57:32.437323+0000 front 2024-05-17T22:57:32.437281+0000 (oldest deadline 2024-05-17T22:57:57.137263+0000) 2024-05-17T22:58:00.538 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:58:00 smithi151 podman[74076]: 2024-05-17 22:58:00.262399412 +0000 UTC m=+5.641458243 container died 8c0fa222680be54cf84c0636b6fad2bd9f31711381f67edb98254bc2e9b56e4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=) 2024-05-17T22:58:00.889 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:58:00 smithi151 podman[74076]: 2024-05-17 22:58:00.644862194 +0000 UTC m=+6.023921024 container cleanup 8c0fa222680be54cf84c0636b6fad2bd9f31711381f67edb98254bc2e9b56e4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-05-17T22:58:00.890 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:58:00 smithi151 bash[74076]: ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11 2024-05-17T22:58:01.240 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:58:00 smithi151 podman[74096]: 2024-05-17 22:58:00.888088146 +0000 UTC m=+0.625898185 container remove 8c0fa222680be54cf84c0636b6fad2bd9f31711381f67edb98254bc2e9b56e4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-17T22:58:01.569 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:58:01 smithi151 podman[74184]: 2024-05-17 22:58:01.154347977 +0000 UTC m=+0.016803514 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T22:58:01.569 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:58:01 smithi151 podman[74184]: 2024-05-17 22:58:01.272735642 +0000 UTC m=+0.135191218 container create 16e474efdcdb20be7358a85a7ed01515cf19f352138f62f11182b59523aea158 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514) 2024-05-17T22:58:01.569 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:58:01 smithi151 podman[74184]: 2024-05-17 22:58:01.492261125 +0000 UTC m=+0.354716662 container init 16e474efdcdb20be7358a85a7ed01515cf19f352138f62f11182b59523aea158 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-17T22:58:01.569 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:58:01 smithi151 podman[74184]: 2024-05-17 22:58:01.49538995 +0000 UTC m=+0.357845487 container start 16e474efdcdb20be7358a85a7ed01515cf19f352138f62f11182b59523aea158 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-17T22:58:01.570 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:58:01 smithi151 podman[74184]: 2024-05-17 22:58:01.567463804 +0000 UTC m=+0.429919374 container attach 16e474efdcdb20be7358a85a7ed01515cf19f352138f62f11182b59523aea158 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T22:58:01.990 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 17 22:58:01 smithi151 podman[74184]: 2024-05-17 22:58:01.79128128 +0000 UTC m=+0.653736821 container died 16e474efdcdb20be7358a85a7ed01515cf19f352138f62f11182b59523aea158 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-17T22:58:02.377 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@osd.11.service' 2024-05-17T22:58:02.418 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T22:58:02.418 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-17T22:58:02.418 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-17T22:58:02.418 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-4d593780-149c-11ef-bc9a-c7b262605968@mds.a 2024-05-17T22:58:02.711 INFO:journalctl@ceph.mds.a.smithi012.stdout:May 17 22:58:02 smithi012 systemd[1]: Stopping Ceph mds.a for 4d593780-149c-11ef-bc9a-c7b262605968... 2024-05-17T22:58:02.986 INFO:journalctl@ceph.mds.a.smithi012.stdout:May 17 22:58:02 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-a[73216]: 2024-05-17T22:58:02.708+0000 7f6c72eff700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T22:58:02.986 INFO:journalctl@ceph.mds.a.smithi012.stdout:May 17 22:58:02 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-a[73216]: 2024-05-17T22:58:02.708+0000 7f6c72eff700 -1 mds.a *** got signal Terminated *** 2024-05-17T22:58:05.679 INFO:journalctl@ceph.mds.a.smithi012.stdout:May 17 22:58:05 smithi012 podman[115941]: 2024-05-17 22:58:05.412563816 +0000 UTC m=+2.794874640 container died c6d5b4f6405ff6f2be10ac0a477116c0a434d3a64eac717f577f6ae1e7f8538e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a) 2024-05-17T22:58:06.036 INFO:journalctl@ceph.mds.a.smithi012.stdout:May 17 22:58:05 smithi012 podman[115941]: 2024-05-17 22:58:05.8418052 +0000 UTC m=+3.224116018 container cleanup c6d5b4f6405ff6f2be10ac0a477116c0a434d3a64eac717f577f6ae1e7f8538e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-05-17T22:58:06.036 INFO:journalctl@ceph.mds.a.smithi012.stdout:May 17 22:58:05 smithi012 bash[115941]: ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-a 2024-05-17T22:58:06.265 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@mds.a.service' 2024-05-17T22:58:06.289 INFO:journalctl@ceph.mds.a.smithi012.stdout:May 17 22:58:06 smithi012 podman[115965]: 2024-05-17 22:58:06.034798634 +0000 UTC m=+0.615296947 container remove c6d5b4f6405ff6f2be10ac0a477116c0a434d3a64eac717f577f6ae1e7f8538e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-a, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True) 2024-05-17T22:58:06.289 INFO:journalctl@ceph.mds.a.smithi012.stdout:May 17 22:58:06 smithi012 systemd[1]: ceph-4d593780-149c-11ef-bc9a-c7b262605968@mds.a.service: Deactivated successfully. 2024-05-17T22:58:06.289 INFO:journalctl@ceph.mds.a.smithi012.stdout:May 17 22:58:06 smithi012 systemd[1]: Stopped Ceph mds.a for 4d593780-149c-11ef-bc9a-c7b262605968. 2024-05-17T22:58:06.289 INFO:journalctl@ceph.mds.a.smithi012.stdout:May 17 22:58:06 smithi012 systemd[1]: ceph-4d593780-149c-11ef-bc9a-c7b262605968@mds.a.service: Consumed 1.199s CPU time. 2024-05-17T22:58:06.302 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T22:58:06.302 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-17T22:58:06.302 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-17T22:58:06.303 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-4d593780-149c-11ef-bc9a-c7b262605968@mds.d 2024-05-17T22:58:06.591 INFO:journalctl@ceph.mds.d.smithi012.stdout:May 17 22:58:06 smithi012 systemd[1]: Stopping Ceph mds.d for 4d593780-149c-11ef-bc9a-c7b262605968... 2024-05-17T22:58:07.091 INFO:journalctl@ceph.mds.d.smithi012.stdout:May 17 22:58:06 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-d[71944]: 2024-05-17T22:58:06.629+0000 7f5a78a80700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T22:58:07.092 INFO:journalctl@ceph.mds.d.smithi012.stdout:May 17 22:58:06 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-d[71944]: 2024-05-17T22:58:06.629+0000 7f5a78a80700 -1 mds.d *** got signal Terminated *** 2024-05-17T22:58:10.923 INFO:journalctl@ceph.mds.d.smithi012.stdout:May 17 22:58:10 smithi012 podman[116074]: 2024-05-17 22:58:10.640641049 +0000 UTC m=+4.110544074 container died 16550dd7efce5dc440eb13283a78176d8498b7ea81fba66b668142a2242c5ff3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_CLEAN=True) 2024-05-17T22:58:11.265 INFO:journalctl@ceph.mds.d.smithi012.stdout:May 17 22:58:11 smithi012 podman[116074]: 2024-05-17 22:58:11.087211102 +0000 UTC m=+4.557114145 container cleanup 16550dd7efce5dc440eb13283a78176d8498b7ea81fba66b668142a2242c5ff3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.build-date=20240514) 2024-05-17T22:58:11.265 INFO:journalctl@ceph.mds.d.smithi012.stdout:May 17 22:58:11 smithi012 bash[116074]: ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-d 2024-05-17T22:58:11.527 INFO:journalctl@ceph.mds.d.smithi012.stdout:May 17 22:58:11 smithi012 podman[116086]: 2024-05-17 22:58:11.263497303 +0000 UTC m=+0.620883518 container remove 16550dd7efce5dc440eb13283a78176d8498b7ea81fba66b668142a2242c5ff3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-17T22:58:11.531 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@mds.d.service' 2024-05-17T22:58:11.565 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T22:58:11.566 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-17T22:58:11.566 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-17T22:58:11.566 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-4d593780-149c-11ef-bc9a-c7b262605968@mds.g 2024-05-17T22:58:11.841 INFO:journalctl@ceph.mds.g.smithi012.stdout:May 17 22:58:11 smithi012 systemd[1]: Stopping Ceph mds.g for 4d593780-149c-11ef-bc9a-c7b262605968... 2024-05-17T22:58:12.341 INFO:journalctl@ceph.mds.g.smithi012.stdout:May 17 22:58:11 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-g[72774]: 2024-05-17T22:58:11.899+0000 7f9304c61700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T22:58:12.341 INFO:journalctl@ceph.mds.g.smithi012.stdout:May 17 22:58:11 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-g[72774]: 2024-05-17T22:58:11.899+0000 7f9304c61700 -1 mds.g *** got signal Terminated *** 2024-05-17T22:58:13.519 INFO:journalctl@ceph.mds.g.smithi012.stdout:May 17 22:58:13 smithi012 podman[116195]: 2024-05-17 22:58:13.22426543 +0000 UTC m=+1.431482235 container died 54adbc3cf4cf59a67f5dffac4cf3083d805a69b47922568aa1d0fd59d3ed7455 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.build-date=20240514) 2024-05-17T22:58:13.841 INFO:journalctl@ceph.mds.g.smithi012.stdout:May 17 22:58:13 smithi012 podman[116195]: 2024-05-17 22:58:13.683192268 +0000 UTC m=+1.890409093 container cleanup 54adbc3cf4cf59a67f5dffac4cf3083d805a69b47922568aa1d0fd59d3ed7455 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-g, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS) 2024-05-17T22:58:13.842 INFO:journalctl@ceph.mds.g.smithi012.stdout:May 17 22:58:13 smithi012 bash[116195]: ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-g 2024-05-17T22:58:14.128 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@mds.g.service' 2024-05-17T22:58:14.161 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T22:58:14.161 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-17T22:58:14.161 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-17T22:58:14.161 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-4d593780-149c-11ef-bc9a-c7b262605968@mds.j 2024-05-17T22:58:14.591 INFO:journalctl@ceph.mds.j.smithi012.stdout:May 17 22:58:14 smithi012 systemd[1]: Stopping Ceph mds.j for 4d593780-149c-11ef-bc9a-c7b262605968... 2024-05-17T22:58:14.591 INFO:journalctl@ceph.mds.j.smithi012.stdout:May 17 22:58:14 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-j[71511]: 2024-05-17T22:58:14.487+0000 7fa076ab3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T22:58:14.592 INFO:journalctl@ceph.mds.j.smithi012.stdout:May 17 22:58:14 smithi012 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-j[71511]: 2024-05-17T22:58:14.487+0000 7fa076ab3700 -1 mds.j *** got signal Terminated *** 2024-05-17T22:58:18.841 INFO:journalctl@ceph.mds.j.smithi012.stdout:May 17 22:58:18 smithi012 podman[116316]: 2024-05-17 22:58:18.500822591 +0000 UTC m=+4.116639429 container died 8b1e97fd39c3d79a05eb026eba662e73ac327575932ae0c108136d8d73646247 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-17T22:58:19.277 INFO:journalctl@ceph.mds.j.smithi012.stdout:May 17 22:58:18 smithi012 podman[116316]: 2024-05-17 22:58:18.853358548 +0000 UTC m=+4.469175385 container cleanup 8b1e97fd39c3d79a05eb026eba662e73ac327575932ae0c108136d8d73646247 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-17T22:58:19.277 INFO:journalctl@ceph.mds.j.smithi012.stdout:May 17 22:58:18 smithi012 bash[116316]: ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-j 2024-05-17T22:58:19.277 INFO:journalctl@ceph.mds.j.smithi012.stdout:May 17 22:58:19 smithi012 podman[116329]: 2024-05-17 22:58:19.054636592 +0000 UTC m=+0.540947132 container remove 8b1e97fd39c3d79a05eb026eba662e73ac327575932ae0c108136d8d73646247 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-j, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_CLEAN=True) 2024-05-17T22:58:19.301 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@mds.j.service' 2024-05-17T22:58:19.334 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T22:58:19.334 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-17T22:58:19.335 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-17T22:58:19.335 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-4d593780-149c-11ef-bc9a-c7b262605968@mds.b 2024-05-17T22:58:19.637 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 17 22:58:19 smithi096 systemd[1]: Stopping Ceph mds.b for 4d593780-149c-11ef-bc9a-c7b262605968... 2024-05-17T22:58:19.897 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 17 22:58:19 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-b[66490]: 2024-05-17T22:58:19.635+0000 7ff2eaf0b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T22:58:19.897 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 17 22:58:19 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-b[66490]: 2024-05-17T22:58:19.635+0000 7ff2eaf0b700 -1 mds.b *** got signal Terminated *** 2024-05-17T22:58:21.462 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 17 22:58:21 smithi096 podman[73149]: 2024-05-17 22:58:21.201291211 +0000 UTC m=+1.671632234 container died 78c4d4ccf405f61b52db9c52150f79308ae0ba7eeedda418197bf292292f2190 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-b, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T22:58:21.798 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 17 22:58:21 smithi096 podman[73149]: 2024-05-17 22:58:21.551681394 +0000 UTC m=+2.022022472 container cleanup 78c4d4ccf405f61b52db9c52150f79308ae0ba7eeedda418197bf292292f2190 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-b, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T22:58:21.799 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 17 22:58:21 smithi096 bash[73149]: ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-b 2024-05-17T22:58:21.966 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@mds.b.service' 2024-05-17T22:58:22.001 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T22:58:22.001 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-17T22:58:22.001 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-17T22:58:22.001 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-4d593780-149c-11ef-bc9a-c7b262605968@mds.e 2024-05-17T22:58:22.298 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 17 22:58:22 smithi096 systemd[1]: Stopping Ceph mds.e for 4d593780-149c-11ef-bc9a-c7b262605968... 2024-05-17T22:58:22.799 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 17 22:58:22 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-e[65880]: 2024-05-17T22:58:22.296+0000 7fd154162700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T22:58:22.799 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 17 22:58:22 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-e[65880]: 2024-05-17T22:58:22.296+0000 7fd154162700 -1 mds.e *** got signal Terminated *** 2024-05-17T22:58:25.451 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 17 22:58:25 smithi096 podman[73283]: 2024-05-17 22:58:25.195527495 +0000 UTC m=+2.982675174 container died 49d01cd75e8ba1dd1eee88367671ec0328ffbdffcded67cc723d035f4fd0749b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-e, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-17T22:58:25.751 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 17 22:58:25 smithi096 podman[73283]: 2024-05-17 22:58:25.547853168 +0000 UTC m=+3.335000789 container cleanup 49d01cd75e8ba1dd1eee88367671ec0328ffbdffcded67cc723d035f4fd0749b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T22:58:25.751 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 17 22:58:25 smithi096 bash[73283]: ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-e 2024-05-17T22:58:25.979 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@mds.e.service' 2024-05-17T22:58:26.001 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 17 22:58:25 smithi096 podman[73298]: 2024-05-17 22:58:25.749257165 +0000 UTC m=+0.543631349 container remove 49d01cd75e8ba1dd1eee88367671ec0328ffbdffcded67cc723d035f4fd0749b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-e, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514) 2024-05-17T22:58:26.001 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 17 22:58:25 smithi096 systemd[1]: ceph-4d593780-149c-11ef-bc9a-c7b262605968@mds.e.service: Deactivated successfully. 2024-05-17T22:58:26.001 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 17 22:58:25 smithi096 systemd[1]: Stopped Ceph mds.e for 4d593780-149c-11ef-bc9a-c7b262605968. 2024-05-17T22:58:26.001 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 17 22:58:25 smithi096 systemd[1]: ceph-4d593780-149c-11ef-bc9a-c7b262605968@mds.e.service: Consumed 2.974s CPU time. 2024-05-17T22:58:26.011 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T22:58:26.011 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-17T22:58:26.011 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-17T22:58:26.012 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-4d593780-149c-11ef-bc9a-c7b262605968@mds.h 2024-05-17T22:58:26.298 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 17 22:58:26 smithi096 systemd[1]: Stopping Ceph mds.h for 4d593780-149c-11ef-bc9a-c7b262605968... 2024-05-17T22:58:26.798 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 17 22:58:26 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-h[66183]: 2024-05-17T22:58:26.360+0000 7f7ba504d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T22:58:26.799 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 17 22:58:26 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-h[66183]: 2024-05-17T22:58:26.360+0000 7f7ba504d700 -1 mds.h *** got signal Terminated *** 2024-05-17T22:58:29.462 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 17 22:58:29 smithi096 podman[73405]: 2024-05-17 22:58:29.196594519 +0000 UTC m=+2.961208770 container died fe0a8da3710df63b0f880e8a03ec06759fe61b95326e63757e8aa5dda6681b81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-h, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-17T22:58:29.798 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 17 22:58:29 smithi096 podman[73405]: 2024-05-17 22:58:29.56829953 +0000 UTC m=+3.332913823 container cleanup fe0a8da3710df63b0f880e8a03ec06759fe61b95326e63757e8aa5dda6681b81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-17T22:58:29.798 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 17 22:58:29 smithi096 bash[73405]: ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-h 2024-05-17T22:58:29.995 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@mds.h.service' 2024-05-17T22:58:30.025 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T22:58:30.026 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-17T22:58:30.026 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-17T22:58:30.026 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-4d593780-149c-11ef-bc9a-c7b262605968@mds.k 2024-05-17T22:58:30.340 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 17 22:58:30 smithi096 systemd[1]: Stopping Ceph mds.k for 4d593780-149c-11ef-bc9a-c7b262605968... 2024-05-17T22:58:30.798 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 17 22:58:30 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-k[65564]: 2024-05-17T22:58:30.337+0000 7f3d9b4b5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T22:58:30.799 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 17 22:58:30 smithi096 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-k[65564]: 2024-05-17T22:58:30.337+0000 7f3d9b4b5700 -1 mds.k *** got signal Terminated *** 2024-05-17T22:58:34.608 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 17 22:58:34 smithi096 podman[73525]: 2024-05-17 22:58:34.354537909 +0000 UTC m=+4.115903250 container died 74fca3e3537123d1ec0aaea202e9f8613086527d41ea35542a52cb7a1eae735a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-k, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.license=GPLv2) 2024-05-17T22:58:35.020 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 17 22:58:34 smithi096 podman[73525]: 2024-05-17 22:58:34.671990503 +0000 UTC m=+4.433355838 container cleanup 74fca3e3537123d1ec0aaea202e9f8613086527d41ea35542a52cb7a1eae735a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-k, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-17T22:58:35.021 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 17 22:58:34 smithi096 bash[73525]: ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-k 2024-05-17T22:58:35.021 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 17 22:58:34 smithi096 podman[73537]: 2024-05-17 22:58:34.831459227 +0000 UTC m=+0.467999329 container remove 74fca3e3537123d1ec0aaea202e9f8613086527d41ea35542a52cb7a1eae735a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-k, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-17T22:58:35.042 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@mds.k.service' 2024-05-17T22:58:35.072 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T22:58:35.072 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-17T22:58:35.072 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-17T22:58:35.072 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-4d593780-149c-11ef-bc9a-c7b262605968@mds.c 2024-05-17T22:58:35.380 INFO:journalctl@ceph.mds.c.smithi151.stdout:May 17 22:58:35 smithi151 systemd[1]: Stopping Ceph mds.c for 4d593780-149c-11ef-bc9a-c7b262605968... 2024-05-17T22:58:35.634 INFO:journalctl@ceph.mds.c.smithi151.stdout:May 17 22:58:35 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-c[67008]: 2024-05-17T22:58:35.377+0000 7fd3c7dc1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T22:58:35.634 INFO:journalctl@ceph.mds.c.smithi151.stdout:May 17 22:58:35 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-c[67008]: 2024-05-17T22:58:35.377+0000 7fd3c7dc1700 -1 mds.c *** got signal Terminated *** 2024-05-17T22:58:37.491 INFO:journalctl@ceph.mds.c.smithi151.stdout:May 17 22:58:37 smithi151 podman[74290]: 2024-05-17 22:58:37.201521608 +0000 UTC m=+1.929024017 container died 25c9781f4db4dbebe6e976afa70f11bd2d279ca79220528b484570a41dfc5d33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514) 2024-05-17T22:58:37.922 INFO:journalctl@ceph.mds.c.smithi151.stdout:May 17 22:58:37 smithi151 podman[74290]: 2024-05-17 22:58:37.677888977 +0000 UTC m=+2.405391378 container cleanup 25c9781f4db4dbebe6e976afa70f11bd2d279ca79220528b484570a41dfc5d33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-c, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=) 2024-05-17T22:58:37.922 INFO:journalctl@ceph.mds.c.smithi151.stdout:May 17 22:58:37 smithi151 bash[74290]: ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-c 2024-05-17T22:58:38.066 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@mds.c.service' 2024-05-17T22:58:38.100 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T22:58:38.101 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-17T22:58:38.101 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-17T22:58:38.101 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-4d593780-149c-11ef-bc9a-c7b262605968@mds.f 2024-05-17T22:58:38.434 INFO:journalctl@ceph.mds.f.smithi151.stdout:May 17 22:58:38 smithi151 systemd[1]: Stopping Ceph mds.f for 4d593780-149c-11ef-bc9a-c7b262605968... 2024-05-17T22:58:38.740 INFO:journalctl@ceph.mds.f.smithi151.stdout:May 17 22:58:38 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-f[66703]: 2024-05-17T22:58:38.430+0000 7f0c8cef4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T22:58:38.741 INFO:journalctl@ceph.mds.f.smithi151.stdout:May 17 22:58:38 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-f[66703]: 2024-05-17T22:58:38.430+0000 7f0c8cef4700 -1 mds.f *** got signal Terminated *** 2024-05-17T22:58:41.471 INFO:journalctl@ceph.mds.f.smithi151.stdout:May 17 22:58:41 smithi151 podman[74424]: 2024-05-17 22:58:41.195452827 +0000 UTC m=+2.877817883 container died 3033908f42192f9b0ec7e43f52f203545201f0c9a294c57a50bb6dec92d0450f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-f, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-05-17T22:58:41.740 INFO:journalctl@ceph.mds.f.smithi151.stdout:May 17 22:58:41 smithi151 podman[74424]: 2024-05-17 22:58:41.548370405 +0000 UTC m=+3.230735460 container cleanup 3033908f42192f9b0ec7e43f52f203545201f0c9a294c57a50bb6dec92d0450f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, ceph=True, org.label-schema.license=GPLv2) 2024-05-17T22:58:41.740 INFO:journalctl@ceph.mds.f.smithi151.stdout:May 17 22:58:41 smithi151 bash[74424]: ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-f 2024-05-17T22:58:42.036 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@mds.f.service' 2024-05-17T22:58:42.068 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T22:58:42.069 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-17T22:58:42.069 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-17T22:58:42.069 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-4d593780-149c-11ef-bc9a-c7b262605968@mds.i 2024-05-17T22:58:42.393 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 17 22:58:42 smithi151 systemd[1]: Stopping Ceph mds.i for 4d593780-149c-11ef-bc9a-c7b262605968... 2024-05-17T22:58:42.740 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 17 22:58:42 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-i[67309]: 2024-05-17T22:58:42.390+0000 7f10aaa01700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T22:58:42.740 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 17 22:58:42 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-i[67309]: 2024-05-17T22:58:42.390+0000 7f10aaa01700 -1 mds.i *** got signal Terminated *** 2024-05-17T22:58:45.477 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 17 22:58:45 smithi151 podman[74545]: 2024-05-17 22:58:45.198359606 +0000 UTC m=+2.913260688 container died b5eb2658cde1e034703198d8fbb5ec0cb6aff966ca3010b30eebbde55f689108 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-i, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-17T22:58:45.740 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 17 22:58:45 smithi151 podman[74545]: 2024-05-17 22:58:45.592006911 +0000 UTC m=+3.306907984 container cleanup b5eb2658cde1e034703198d8fbb5ec0cb6aff966ca3010b30eebbde55f689108 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-i, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-17T22:58:45.740 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 17 22:58:45 smithi151 bash[74545]: ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-i 2024-05-17T22:58:45.977 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@mds.i.service' 2024-05-17T22:58:46.008 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T22:58:46.008 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-17T22:58:46.009 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-17T22:58:46.009 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-4d593780-149c-11ef-bc9a-c7b262605968@mds.l 2024-05-17T22:58:46.240 INFO:journalctl@ceph.mds.l.smithi151.stdout:May 17 22:58:46 smithi151 systemd[1]: Stopping Ceph mds.l for 4d593780-149c-11ef-bc9a-c7b262605968... 2024-05-17T22:58:46.740 INFO:journalctl@ceph.mds.l.smithi151.stdout:May 17 22:58:46 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-l[66397]: 2024-05-17T22:58:46.541+0000 7f90f2325700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T22:58:46.740 INFO:journalctl@ceph.mds.l.smithi151.stdout:May 17 22:58:46 smithi151 ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-l[66397]: 2024-05-17T22:58:46.541+0000 7f90f2325700 -1 mds.l *** got signal Terminated *** 2024-05-17T22:58:48.160 INFO:journalctl@ceph.mds.l.smithi151.stdout:May 17 22:58:47 smithi151 podman[74666]: 2024-05-17 22:58:47.872997112 +0000 UTC m=+1.654475431 container died c7816c3d6909cba88c1b0032a98345f37bc66e4b23614eeba179a693322fb3a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-l, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-17T22:58:48.490 INFO:journalctl@ceph.mds.l.smithi151.stdout:May 17 22:58:48 smithi151 podman[74666]: 2024-05-17 22:58:48.25436117 +0000 UTC m=+2.035839479 container cleanup c7816c3d6909cba88c1b0032a98345f37bc66e4b23614eeba179a693322fb3a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-l, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-17T22:58:48.490 INFO:journalctl@ceph.mds.l.smithi151.stdout:May 17 22:58:48 smithi151 bash[74666]: ceph-4d593780-149c-11ef-bc9a-c7b262605968-mds-l 2024-05-17T22:58:48.633 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4d593780-149c-11ef-bc9a-c7b262605968@mds.l.service' 2024-05-17T22:58:48.664 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T22:58:48.664 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-17T22:58:48.665 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4d593780-149c-11ef-bc9a-c7b262605968 --force --keep-logs 2024-05-17T22:58:48.868 INFO:teuthology.orchestra.run.smithi012.stdout:Deleting cluster with fsid: 4d593780-149c-11ef-bc9a-c7b262605968 2024-05-17T22:58:51.643 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4d593780-149c-11ef-bc9a-c7b262605968 --force --keep-logs 2024-05-17T22:58:51.834 INFO:teuthology.orchestra.run.smithi096.stdout:Deleting cluster with fsid: 4d593780-149c-11ef-bc9a-c7b262605968 2024-05-17T22:58:54.467 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4d593780-149c-11ef-bc9a-c7b262605968 --force --keep-logs 2024-05-17T22:58:54.657 INFO:teuthology.orchestra.run.smithi151.stdout:Deleting cluster with fsid: 4d593780-149c-11ef-bc9a-c7b262605968 2024-05-17T22:58:57.043 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-17T22:58:57.078 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-17T22:58:57.109 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-17T22:58:57.134 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-17T22:58:57.136 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7711773/remote/smithi012/crash 2024-05-17T22:58:57.137 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/crash -- . 2024-05-17T22:58:57.181 INFO:teuthology.orchestra.run.smithi012.stderr:tar: /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/crash: Cannot open: No such file or directory 2024-05-17T22:58:57.181 INFO:teuthology.orchestra.run.smithi012.stderr:tar: Error is not recoverable: exiting now 2024-05-17T22:58:57.183 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7711773/remote/smithi096/crash 2024-05-17T22:58:57.184 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/crash -- . 2024-05-17T22:58:57.222 INFO:teuthology.orchestra.run.smithi096.stderr:tar: /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/crash: Cannot open: No such file or directory 2024-05-17T22:58:57.222 INFO:teuthology.orchestra.run.smithi096.stderr:tar: Error is not recoverable: exiting now 2024-05-17T22:58:57.224 DEBUG:teuthology.misc:Transferring archived files from smithi151:/var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7711773/remote/smithi151/crash 2024-05-17T22:58:57.225 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/crash -- . 2024-05-17T22:58:57.265 INFO:teuthology.orchestra.run.smithi151.stderr:tar: /var/lib/ceph/4d593780-149c-11ef-bc9a-c7b262605968/crash: Cannot open: No such file or directory 2024-05-17T22:58:57.265 INFO:teuthology.orchestra.run.smithi151.stderr:tar: Error is not recoverable: exiting now 2024-05-17T22:58:57.266 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-17T22:58:57.266 DEBUG:teuthology.orchestra.run.smithi012:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-17T22:58:57.369 INFO:tasks.cephadm:Compressing logs... 2024-05-17T22:58:57.369 DEBUG:teuthology.orchestra.run.smithi012:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-17T22:58:57.371 DEBUG:teuthology.orchestra.run.smithi096:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-17T22:58:57.374 DEBUG:teuthology.orchestra.run.smithi151:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-17T22:58:57.397 INFO:teuthology.orchestra.run.smithi012.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-17T22:58:57.397 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-17T22:58:57.397 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph.log 2024-05-17T22:58:57.398 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph.audit.log 2024-05-17T22:58:57.398 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mds.g.log 2024-05-17T22:58:57.398 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph.log: /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mds.j.log 2024-05-17T22:58:57.398 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-osd.3.log 2024-05-17T22:58:57.399 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-osd.0.log 2024-05-17T22:58:57.400 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mgr.x.log 2024-05-17T22:58:57.400 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mon.a.log 2024-05-17T22:58:57.400 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-osd.0.log: /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-osd.6.log 2024-05-17T22:58:57.400 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph.cephadm.log 2024-05-17T22:58:57.401 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mon.a.log: /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mds.a.log 2024-05-17T22:58:57.402 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph.cephadm.log: 87.3% -- replaced with /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph.log.gz 2024-05-17T22:58:57.402 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-volume.log 2024-05-17T22:58:57.402 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mds.d.log 2024-05-17T22:58:57.402 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-osd.9.log 2024-05-17T22:58:57.402 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-05-17T22:58:57.402 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-osd.9.log: /var/log/ceph/ceph-client.admin.log: 94.5% -- replaced with /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mds.g.log.gz 2024-05-17T22:58:57.403 INFO:teuthology.orchestra.run.smithi012.stderr: 94.7% -- replaced with /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mds.j.log.gz 2024-05-17T22:58:57.410 INFO:teuthology.orchestra.run.smithi151.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-17T22:58:57.410 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-17T22:58:57.410 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph.log 2024-05-17T22:58:57.411 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-osd.2.log 2024-05-17T22:58:57.411 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mds.f.log 2024-05-17T22:58:57.411 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph.log: /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph.audit.log 2024-05-17T22:58:57.413 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-osd.5.log 2024-05-17T22:58:57.413 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mds.c.log 2024-05-17T22:58:57.413 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mon.c.log 2024-05-17T22:58:57.413 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mgr.z.log 2024-05-17T22:58:57.414 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mds.i.log 2024-05-17T22:58:57.414 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph.cephadm.log 2024-05-17T22:58:57.414 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-osd.11.log 2024-05-17T22:58:57.414 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-osd.8.log 2024-05-17T22:58:57.414 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-volume.log 2024-05-17T22:58:57.414 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mds.l.log 2024-05-17T22:58:57.417 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-volume.log: /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mds.l.log: 94.5% -- replaced with /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mds.l.log.gz 2024-05-17T22:58:57.423 INFO:teuthology.orchestra.run.smithi096.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-17T22:58:57.423 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-17T22:58:57.423 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph.log 2024-05-17T22:58:57.424 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph.audit.log 2024-05-17T22:58:57.424 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-osd.7.log 2024-05-17T22:58:57.425 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph.log: /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mds.b.log 2024-05-17T22:58:57.425 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-osd.4.log 2024-05-17T22:58:57.425 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-osd.1.log 2024-05-17T22:58:57.425 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mgr.y.log 2024-05-17T22:58:57.425 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mds.h.log 2024-05-17T22:58:57.427 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-osd.10.log 2024-05-17T22:58:57.427 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph.cephadm.log 2024-05-17T22:58:57.427 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mds.e.log 2024-05-17T22:58:57.427 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-volume.log 2024-05-17T22:58:57.427 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mon.b.log 2024-05-17T22:58:57.427 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mds.k.log 2024-05-17T22:58:57.468 INFO:teuthology.orchestra.run.smithi012.stderr: 90.3% -- replaced with /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph.audit.log.gz 2024-05-17T22:58:57.498 INFO:teuthology.orchestra.run.smithi151.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-17T22:58:57.505 INFO:teuthology.orchestra.run.smithi012.stderr: 83.4% -- replaced with /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph.cephadm.log.gz 2024-05-17T22:58:57.512 INFO:teuthology.orchestra.run.smithi012.stderr: 94.6% -- replaced with /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mds.d.log.gz 2024-05-17T22:58:57.515 INFO:teuthology.orchestra.run.smithi151.stderr: 90.5% -- replaced with /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph.audit.log.gz 2024-05-17T22:58:57.523 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mon.b.log: /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mds.k.log: 93.2% -- replaced with /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mgr.y.log.gz 2024-05-17T22:58:57.525 INFO:teuthology.orchestra.run.smithi012.stderr: 94.6% -- replaced with /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mds.a.log.gz 2024-05-17T22:58:57.539 INFO:teuthology.orchestra.run.smithi151.stderr: 93.7% -- replaced with /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mds.f.log.gz 2024-05-17T22:58:57.565 INFO:teuthology.orchestra.run.smithi096.stderr: 83.3% -- replaced with /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph.cephadm.log.gz 2024-05-17T22:58:57.567 INFO:teuthology.orchestra.run.smithi096.stderr: 93.7% -- replaced with /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mds.e.log.gz 2024-05-17T22:58:57.588 INFO:teuthology.orchestra.run.smithi012.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-05-17T22:58:57.597 INFO:teuthology.orchestra.run.smithi151.stderr: 93.4% -- replaced with /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mgr.z.log.gz 2024-05-17T22:58:57.679 INFO:teuthology.orchestra.run.smithi096.stderr: 91.0% -- replaced with /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-volume.log.gz 2024-05-17T22:58:57.682 INFO:teuthology.orchestra.run.smithi096.stderr: 87.3% -- replaced with /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph.log.gz 2024-05-17T22:58:57.731 INFO:teuthology.orchestra.run.smithi151.stderr: 91.0% -- replaced with /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-volume.log.gz 2024-05-17T22:58:57.735 INFO:teuthology.orchestra.run.smithi151.stderr: 83.3% -- replaced with /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph.cephadm.log.gz 2024-05-17T22:58:57.790 INFO:teuthology.orchestra.run.smithi096.stderr: 94.5% -- replaced with /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mds.k.log.gz 2024-05-17T22:58:57.882 INFO:teuthology.orchestra.run.smithi012.stderr: 91.1% -- replaced with /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-volume.log.gz 2024-05-17T22:58:58.043 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph.log.gz 2024-05-17T22:58:58.068 INFO:teuthology.orchestra.run.smithi096.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-17T22:58:58.078 INFO:teuthology.orchestra.run.smithi096.stderr: 90.4% -- replaced with /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph.audit.log.gz 2024-05-17T22:58:58.257 INFO:teuthology.orchestra.run.smithi012.stderr: 94.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-17T22:58:59.410 INFO:teuthology.orchestra.run.smithi151.stderr: 94.5% -- replaced with /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mds.c.log.gz 2024-05-17T22:59:01.007 INFO:teuthology.orchestra.run.smithi012.stderr: 90.4% -- replaced with /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mgr.x.log.gz 2024-05-17T22:59:02.791 INFO:teuthology.orchestra.run.smithi151.stderr: 91.7% -- replaced with /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mon.c.log.gz 2024-05-17T22:59:03.431 INFO:teuthology.orchestra.run.smithi151.stderr: 94.5% -- replaced with /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mds.i.log.gz 2024-05-17T22:59:06.588 INFO:teuthology.orchestra.run.smithi096.stderr: 91.7% -- replaced with /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mon.b.log.gz 2024-05-17T22:59:11.210 INFO:teuthology.orchestra.run.smithi096.stderr: 93.9% -- replaced with /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mds.b.log.gz 2024-05-17T22:59:14.130 INFO:teuthology.orchestra.run.smithi012.stderr: 90.2% -- replaced with /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mon.a.log.gz 2024-05-17T22:59:16.618 INFO:teuthology.orchestra.run.smithi096.stderr: 93.9% -- replaced with /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-mds.h.log.gz 2024-05-17T23:00:12.090 INFO:teuthology.orchestra.run.smithi151.stderr: 90.6% -- replaced with /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-osd.8.log.gz 2024-05-17T23:00:27.748 INFO:teuthology.orchestra.run.smithi096.stderr: 90.6% -- replaced with /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-osd.7.log.gz 2024-05-17T23:01:27.097 INFO:teuthology.orchestra.run.smithi096.stderr: 90.5% -- replaced with /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-osd.10.log.gz 2024-05-17T23:03:53.245 INFO:teuthology.orchestra.run.smithi012.stderr: 90.6% -- replaced with /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-osd.9.log.gz 2024-05-17T23:03:54.020 INFO:teuthology.orchestra.run.smithi096.stderr: 90.6% -- replaced with /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-osd.1.log.gz 2024-05-17T23:04:29.087 INFO:teuthology.orchestra.run.smithi151.stderr: 90.6% -- replaced with /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-osd.11.log.gz 2024-05-17T23:04:47.145 INFO:teuthology.orchestra.run.smithi096.stderr: 90.8% -- replaced with /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-osd.4.log.gz 2024-05-17T23:04:47.147 INFO:teuthology.orchestra.run.smithi096.stderr: 2024-05-17T23:04:47.147 INFO:teuthology.orchestra.run.smithi096.stderr:real 5m49.758s 2024-05-17T23:04:47.147 INFO:teuthology.orchestra.run.smithi096.stderr:user 5m24.366s 2024-05-17T23:04:47.147 INFO:teuthology.orchestra.run.smithi096.stderr:sys 0m11.483s 2024-05-17T23:06:10.612 INFO:teuthology.orchestra.run.smithi012.stderr: 90.6% -- replaced with /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-osd.6.log.gz 2024-05-17T23:06:38.572 INFO:teuthology.orchestra.run.smithi151.stderr: 90.9% -- replaced with /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-osd.2.log.gz 2024-05-17T23:06:58.151 INFO:teuthology.orchestra.run.smithi151.stderr: 90.6% -- replaced with /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-osd.5.log.gz 2024-05-17T23:06:58.153 INFO:teuthology.orchestra.run.smithi151.stderr: 2024-05-17T23:06:58.153 INFO:teuthology.orchestra.run.smithi151.stderr:real 8m0.765s 2024-05-17T23:06:58.154 INFO:teuthology.orchestra.run.smithi151.stderr:user 6m14.542s 2024-05-17T23:06:58.154 INFO:teuthology.orchestra.run.smithi151.stderr:sys 0m14.178s 2024-05-17T23:07:01.071 INFO:teuthology.orchestra.run.smithi012.stderr: 90.6% -- replaced with /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-osd.0.log.gz 2024-05-17T23:07:40.846 INFO:teuthology.orchestra.run.smithi012.stderr: 90.5% -- replaced with /var/log/ceph/4d593780-149c-11ef-bc9a-c7b262605968/ceph-osd.3.log.gz 2024-05-17T23:07:40.848 INFO:teuthology.orchestra.run.smithi012.stderr: 2024-05-17T23:07:40.848 INFO:teuthology.orchestra.run.smithi012.stderr:real 8m43.462s 2024-05-17T23:07:40.848 INFO:teuthology.orchestra.run.smithi012.stderr:user 7m34.671s 2024-05-17T23:07:40.848 INFO:teuthology.orchestra.run.smithi012.stderr:sys 0m15.449s 2024-05-17T23:07:40.848 INFO:tasks.cephadm:Archiving logs... 2024-05-17T23:07:40.849 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7711773/remote/smithi012/log 2024-05-17T23:07:40.850 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-17T23:08:56.979 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7711773/remote/smithi096/log 2024-05-17T23:08:56.980 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-17T23:09:53.824 DEBUG:teuthology.misc:Transferring archived files from smithi151:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7711773/remote/smithi151/log 2024-05-17T23:09:53.839 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-17T23:10:58.793 INFO:tasks.cephadm:Removing cluster... 2024-05-17T23:10:58.793 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4d593780-149c-11ef-bc9a-c7b262605968 --force 2024-05-17T23:10:58.997 INFO:teuthology.orchestra.run.smithi012.stdout:Deleting cluster with fsid: 4d593780-149c-11ef-bc9a-c7b262605968 2024-05-17T23:10:59.860 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4d593780-149c-11ef-bc9a-c7b262605968 --force 2024-05-17T23:11:00.049 INFO:teuthology.orchestra.run.smithi096.stdout:Deleting cluster with fsid: 4d593780-149c-11ef-bc9a-c7b262605968 2024-05-17T23:11:00.746 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4d593780-149c-11ef-bc9a-c7b262605968 --force 2024-05-17T23:11:00.977 INFO:teuthology.orchestra.run.smithi151.stdout:Deleting cluster with fsid: 4d593780-149c-11ef-bc9a-c7b262605968 2024-05-17T23:11:01.712 INFO:tasks.cephadm:Removing cephadm ... 2024-05-17T23:11:01.712 DEBUG:teuthology.orchestra.run.smithi012:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-17T23:11:01.733 DEBUG:teuthology.orchestra.run.smithi096:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-17T23:11:01.750 DEBUG:teuthology.orchestra.run.smithi151:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-17T23:11:01.765 INFO:tasks.cephadm:Teardown complete 2024-05-17T23:11:01.766 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-17T23:11:01.787 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-17T23:11:01.787 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-17T23:11:01.790 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-17T23:11:01.793 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-17T23:11:01.897 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-17T23:11:01.907 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-17T23:11:01.907 DEBUG:teuthology.orchestra.run.smithi012:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-17T23:11:01.910 DEBUG:teuthology.orchestra.run.smithi096:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-17T23:11:01.913 DEBUG:teuthology.orchestra.run.smithi151:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-17T23:11:01.926 INFO:teuthology.orchestra.run.smithi096.stderr:bash: line 1: ntpq: command not found 2024-05-17T23:11:01.926 INFO:teuthology.orchestra.run.smithi151.stderr:bash: line 1: ntpq: command not found 2024-05-17T23:11:01.926 INFO:teuthology.orchestra.run.smithi012.stderr:bash: line 1: ntpq: command not found 2024-05-17T23:11:01.931 INFO:teuthology.orchestra.run.smithi012.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-17T23:11:01.931 INFO:teuthology.orchestra.run.smithi012.stdout:=============================================================================== 2024-05-17T23:11:01.931 INFO:teuthology.orchestra.run.smithi012.stdout:^* hv01.front.sepia.ceph.com 3 6 377 25 +2548us[+2517us] +/- 50ms 2024-05-17T23:11:01.931 INFO:teuthology.orchestra.run.smithi012.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 27 -1474us[-1505us] +/- 61ms 2024-05-17T23:11:01.931 INFO:teuthology.orchestra.run.smithi012.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 27 -2048us[-2079us] +/- 80ms 2024-05-17T23:11:01.931 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T23:11:01.944 INFO:teuthology.orchestra.run.smithi096.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-17T23:11:01.944 INFO:teuthology.orchestra.run.smithi096.stdout:=============================================================================== 2024-05-17T23:11:01.944 INFO:teuthology.orchestra.run.smithi096.stdout:^* hv01.front.sepia.ceph.com 3 6 377 28 +2613us[+2562us] +/- 50ms 2024-05-17T23:11:01.944 INFO:teuthology.orchestra.run.smithi096.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 94 -1369us[-1480us] +/- 60ms 2024-05-17T23:11:01.945 INFO:teuthology.orchestra.run.smithi096.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 92 -1943us[-1995us] +/- 79ms 2024-05-17T23:11:01.945 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T23:11:01.949 INFO:teuthology.orchestra.run.smithi151.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-17T23:11:01.949 INFO:teuthology.orchestra.run.smithi151.stdout:=============================================================================== 2024-05-17T23:11:01.950 INFO:teuthology.orchestra.run.smithi151.stdout:^* hv01.front.sepia.ceph.com 3 6 377 25 +2456us[+2471us] +/- 50ms 2024-05-17T23:11:01.950 INFO:teuthology.orchestra.run.smithi151.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 25 -1542us[-1527us] +/- 61ms 2024-05-17T23:11:01.950 INFO:teuthology.orchestra.run.smithi151.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 25 -2137us[-2122us] +/- 80ms 2024-05-17T23:11:01.950 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T23:11:01.951 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-17T23:11:01.960 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-17T23:11:01.961 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-17T23:11:01.985 DEBUG:teuthology.orchestra.run.smithi012:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-17T23:11:02.136 DEBUG:teuthology.orchestra.run.smithi096:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-17T23:11:02.217 DEBUG:teuthology.orchestra.run.smithi151:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-17T23:11:02.304 DEBUG:teuthology.orchestra.run.smithi012:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-17T23:11:02.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T23:11:02.338 DEBUG:teuthology.orchestra.run.smithi096:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-17T23:11:02.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T23:11:02.368 DEBUG:teuthology.orchestra.run.smithi151:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-17T23:11:02.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T23:11:02.395 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-17T23:11:02.405 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-17T23:11:02.444 INFO:teuthology.task.internal:Duration was 3152.352154 seconds 2024-05-17T23:11:02.444 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-17T23:11:02.477 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-17T23:11:02.478 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-17T23:11:02.481 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-17T23:11:02.483 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-17T23:11:02.546 INFO:teuthology.orchestra.run.smithi096.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-17T23:11:02.570 INFO:teuthology.orchestra.run.smithi151.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-17T23:11:02.583 INFO:teuthology.orchestra.run.smithi012.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-17T23:11:03.039 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-17T23:11:03.039 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi012.front.sepia.ceph.com 2024-05-17T23:11:03.040 DEBUG:teuthology.orchestra.run.smithi012:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-17T23:11:03.066 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi096.front.sepia.ceph.com 2024-05-17T23:11:03.066 DEBUG:teuthology.orchestra.run.smithi096:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-17T23:11:03.089 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi151.front.sepia.ceph.com 2024-05-17T23:11:03.089 DEBUG:teuthology.orchestra.run.smithi151:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-17T23:11:03.119 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-17T23:11:03.120 DEBUG:teuthology.orchestra.run.smithi012:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-17T23:11:03.122 DEBUG:teuthology.orchestra.run.smithi096:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-17T23:11:03.132 DEBUG:teuthology.orchestra.run.smithi151:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-17T23:11:03.185 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-17T23:11:03.186 DEBUG:teuthology.orchestra.run.smithi012:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-17T23:11:03.195 DEBUG:teuthology.orchestra.run.smithi096:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-17T23:11:03.199 DEBUG:teuthology.orchestra.run.smithi151:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-17T23:11:04.309 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-17T23:11:04.324 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-17T23:11:04.324 DEBUG:teuthology.orchestra.run.smithi012:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-17T23:11:04.356 DEBUG:teuthology.orchestra.run.smithi096:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-17T23:11:04.398 DEBUG:teuthology.orchestra.run.smithi151:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-17T23:11:04.432 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-17T23:11:04.448 DEBUG:teuthology.orchestra.run.smithi012:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-17T23:11:04.452 DEBUG:teuthology.orchestra.run.smithi096:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-17T23:11:04.455 DEBUG:teuthology.orchestra.run.smithi151:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-17T23:11:04.477 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern = core 2024-05-17T23:11:04.486 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern = core 2024-05-17T23:11:04.498 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern = core 2024-05-17T23:11:04.545 DEBUG:teuthology.orchestra.run.smithi012:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-17T23:11:04.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T23:11:04.596 DEBUG:teuthology.orchestra.run.smithi096:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-17T23:11:04.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T23:11:04.612 DEBUG:teuthology.orchestra.run.smithi151:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-17T23:11:04.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T23:11:04.627 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-17T23:11:04.638 INFO:teuthology.task.internal:Transferring archived files... 2024-05-17T23:11:04.640 DEBUG:teuthology.misc:Transferring archived files from smithi012:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7711773/remote/smithi012 2024-05-17T23:11:04.640 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-17T23:11:04.702 DEBUG:teuthology.misc:Transferring archived files from smithi096:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7711773/remote/smithi096 2024-05-17T23:11:04.702 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-17T23:11:04.739 DEBUG:teuthology.misc:Transferring archived files from smithi151:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7711773/remote/smithi151 2024-05-17T23:11:04.739 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-17T23:11:04.776 INFO:teuthology.task.internal:Removing archive directory... 2024-05-17T23:11:04.776 DEBUG:teuthology.orchestra.run.smithi012:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-17T23:11:04.779 DEBUG:teuthology.orchestra.run.smithi096:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-17T23:11:04.782 DEBUG:teuthology.orchestra.run.smithi151:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-17T23:11:04.827 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-17T23:11:04.842 INFO:teuthology.task.internal:Not uploading archives. 2024-05-17T23:11:04.842 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-17T23:11:04.862 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-17T23:11:04.862 DEBUG:teuthology.orchestra.run.smithi012:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-17T23:11:04.865 DEBUG:teuthology.orchestra.run.smithi096:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-17T23:11:04.867 DEBUG:teuthology.orchestra.run.smithi151:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-17T23:11:04.878 INFO:teuthology.orchestra.run.smithi012.stdout: 265585 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 17 23:11 /home/ubuntu/cephtest 2024-05-17T23:11:04.880 INFO:teuthology.orchestra.run.smithi096.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 17 23:11 /home/ubuntu/cephtest 2024-05-17T23:11:04.882 INFO:teuthology.orchestra.run.smithi151.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 17 23:11 /home/ubuntu/cephtest 2024-05-17T23:11:04.884 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-17T23:11:04.898 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-17T23:11:04.966 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/postgres}} duration: 3152.352153778076 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-05-17T23:11:04.966 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-17T23:11:05.057 INFO:teuthology.run:pass